diff --git a/src/destinations/azureservicebus.mdx b/src/destinations/azureservicebus.mdx new file mode 100644 index 00000000..d09a0d0e --- /dev/null +++ b/src/destinations/azureservicebus.mdx @@ -0,0 +1,47 @@ +--- +title: Azure Service Bus destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as messages to your Azure Service Bus queue or topic. + +## Prerequisites + +Before setting up an Azure Service Bus destination, ensure that you have: +- An Azure Service Bus namespace with a queue or topic +- A connection string with `Send` permission on that queue or topic + +## Data format + +Ampersand publishes each read or subscribe message to your queue or topic as JSON. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + +## Create an Azure Service Bus destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Azure Service Bus destination. + +You'll need to provide: + +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandServiceBus` | Alias to reference in your `amp.yaml` file | +| **Name** | string | Yes | `ampersand-events` | Name of the queue or topic to publish to | +| **Connection string** | string | Yes | | Connection string with `Send` permission | + +## Refer to the destination in your integration + +After creating your Azure Service Bus destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToServiceBus + displayName: Salesforce to Azure Service Bus + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandServiceBus + - objectName: contact + destination: ampersandServiceBus +``` diff --git a/src/destinations/clickhouse.mdx b/src/destinations/clickhouse.mdx index 954b3bca..0112049a 100644 --- a/src/destinations/clickhouse.mdx +++ b/src/destinations/clickhouse.mdx @@ -46,7 +46,7 @@ You'll need to provide: | **Username** | string | Yes | `default` | ClickHouse username | | **Password** | string | Yes | | ClickHouse password | | **Table name** | string | Yes | `ampersand_events` | Table to insert rows into | -| **Database** | string | No | `default` | Database containing the table | +| **Database** | string | Yes | `analytics` | Database containing the table | | **Batch size** | integer | No | `1000` | Rows to buffer before flushing a batch (defaults to 1000) | | **Max wait (seconds)** | integer | No | `30` | Max seconds to wait before flushing a batch (defaults to 30) | diff --git a/src/destinations/overview.mdx b/src/destinations/overview.mdx index ba756041..6f22ca5e 100644 --- a/src/destinations/overview.mdx +++ b/src/destinations/overview.mdx @@ -2,7 +2,7 @@ title: Overview --- -Ampersand supports webhook, Slack, Kinesis, and S3 destinations, along with a range of data warehouse destinations (ClickHouse, Snowflake, BigQuery, and Redshift). Destinations allow you to route data synced from SaaS instances via [Read Actions](/read-actions) or [Subscribe Actions](/subscribe-actions), and to receive real-time [Notifications](/notifications) about important lifecycle events in your projects. +Ampersand supports webhook and Slack destinations, S3, a range of data warehouse destinations (ClickHouse, Snowflake, BigQuery, and Redshift), and a range of stream and queue destinations (Kinesis, Amazon SQS, Google Cloud Pub/Sub, RabbitMQ, and Azure Service Bus). Destinations allow you to route data synced from SaaS instances via [Read Actions](/read-actions) or [Subscribe Actions](/subscribe-actions), and to receive real-time [Notifications](/notifications) about important lifecycle events in your projects. ## Add a destination to the Ampersand Dashboard @@ -48,9 +48,13 @@ Destinations can also be used to receive [notifications](/notifications) about i * [BigQuery destinations](/destinations/bigquery) * [Amazon Redshift destinations](/destinations/redshift) -### Streams +### Streams and queues +* [Azure Service Bus destinations](/destinations/azureservicebus) * [Amazon Kinesis destinations](/destinations/kinesis) +* [Google Cloud Pub/Sub destinations](/destinations/pubsub) +* [RabbitMQ destinations](/destinations/rabbitmq) +* [Amazon SQS destinations](/destinations/sqs) ## Other Destinations @@ -58,4 +62,3 @@ We have more destination types on the roadmap, including: * Postgres * Ampersand-hosted Postgres -* Azure Service Bus diff --git a/src/destinations/pubsub.mdx b/src/destinations/pubsub.mdx new file mode 100644 index 00000000..b79003dc --- /dev/null +++ b/src/destinations/pubsub.mdx @@ -0,0 +1,48 @@ +--- +title: Google Cloud Pub/Sub destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as messages to your Google Cloud Pub/Sub topic. + +## Prerequisites + +Before setting up a Pub/Sub destination, ensure that you have: +- A Google Cloud project with a Pub/Sub topic +- A service account with the `Pub/Sub Publisher` role (or `pubsub.topics.publish` permission) on that topic, and a JSON key for it + +## Data format + +Ampersand publishes each read or subscribe message to your topic as JSON. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + +## Create a Pub/Sub destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Pub/Sub destination. + +You'll need to provide: + +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandPubSub` | Alias to reference in your `amp.yaml` file | +| **Project ID** | string | Yes | `my-gcp-project` | Google Cloud project ID | +| **Topic ID** | string | Yes | `ampersand-events` | Pub/Sub topic ID | +| **Service account credentials** | string | Yes | | Service account key JSON with publish access to the topic | + +## Refer to the destination in your integration + +After creating your Pub/Sub destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToPubSub + displayName: Salesforce to Pub/Sub + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandPubSub + - objectName: contact + destination: ampersandPubSub +``` diff --git a/src/destinations/rabbitmq.mdx b/src/destinations/rabbitmq.mdx new file mode 100644 index 00000000..c3d7e21e --- /dev/null +++ b/src/destinations/rabbitmq.mdx @@ -0,0 +1,51 @@ +--- +title: RabbitMQ destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as messages to your RabbitMQ instance. + +## Prerequisites + +Before setting up a RabbitMQ destination, ensure that you have: +- A RabbitMQ instance reachable from the internet +- An exchange that already exists on that instance +- A username and password with permission to publish to that exchange + +## Data format + +Ampersand publishes each read or subscribe message to your RabbitMQ exchange as JSON. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + +## Create a RabbitMQ destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new RabbitMQ destination. + +You'll need to provide: + +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandRabbitMQ` | Alias to reference in your `amp.yaml` file | +| **Server URL** | string | Yes | `rabbitmq.example.com:5672` | Host and port of your RabbitMQ instance | +| **Exchange** | string | Yes | `ampersand-events` | Name of an exchange that already exists | +| **TLS** | boolean | No | `false` | Whether to connect over TLS. Defaults to false (plain AMQP); set to true for managed brokers that require TLS, e.g. CloudAMQP | +| **Username** | string | Yes | `ampersand` | Username with permission to publish | +| **Password** | string | Yes | | Password for the username | + +## Refer to the destination in your integration + +After creating your RabbitMQ destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToRabbitMQ + displayName: Salesforce to RabbitMQ + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandRabbitMQ + - objectName: contact + destination: ampersandRabbitMQ +``` diff --git a/src/destinations/redshift.mdx b/src/destinations/redshift.mdx index b4acc378..f748a558 100644 --- a/src/destinations/redshift.mdx +++ b/src/destinations/redshift.mdx @@ -4,12 +4,12 @@ title: Amazon Redshift destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand inserts the payload as rows into your Amazon Redshift table. Both Redshift Serverless (via a workgroup) and provisioned clusters (via a cluster identifier) are supported. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand inserts the payload as rows into your Amazon Redshift table, delivered to a Redshift Serverless workgroup. ## Prerequisites Before setting up a Redshift destination, ensure that you have: -- An Amazon Redshift Serverless workgroup or a provisioned cluster +- An Amazon Redshift Serverless workgroup - A target database, and a table to insert into (see [Data format and table setup](#data-format-and-table-setup)) - AWS credentials with permission to run the Redshift Data API and insert into the target table @@ -39,18 +39,12 @@ You'll need to provide: | **AWS Access Key ID** | string | Yes | `AKIAIOSFODNN7EXAMPLE` | AWS access key with Redshift Data API permissions | | **AWS Secret Access Key** | string | Yes | | AWS secret access key | | **Database** | string | Yes | `dev` | Target database name | -| **Schema** | string | No | `public` | Target schema | +| **Schema** | string | Yes | `public` | Target schema | | **Table name** | string | Yes | `events` | Target table | -| **Workgroup name** | string | No* | `my-workgroup` | Redshift Serverless workgroup name | -| **Cluster identifier** | string | No* | `my-cluster` | Provisioned cluster identifier | -| **DB user** | string | No | `ampersand` | Database user (for provisioned clusters) | +| **Workgroup name** | string | Yes | `my-workgroup` | Redshift Serverless workgroup name | | **Batch size** | integer | No | `1000` | Rows to buffer before flushing a batch (defaults to 1000) | | **Max wait (seconds)** | integer | No | `30` | Max seconds to wait before flushing a batch (defaults to 30) | - -*Either **Workgroup name** or **Cluster identifier** is required. Provide a **workgroup name** for Redshift Serverless, or a **cluster identifier** (and **DB user**) for a provisioned cluster. - - ## Refer to the destination in your integration After creating your Redshift destination, reference it in your `amp.yaml` file: diff --git a/src/destinations/snowflake.mdx b/src/destinations/snowflake.mdx index 4210f7a5..218926c3 100644 --- a/src/destinations/snowflake.mdx +++ b/src/destinations/snowflake.mdx @@ -38,7 +38,7 @@ You'll need to provide: | **Destination name** | string | Yes | `ampersandSnowflake` | Alias to reference in your `amp.yaml` file | | **Account identifier** | string | Yes | `myorg-myaccount` | Your Snowflake account identifier | | **User ID** | string | Yes | `SVC_AMPERSAND` | Snowflake user configured for key-pair auth | -| **Private key** | string | Yes | | PEM-encoded RSA private key for that user | +| **Private key** | string | Yes | | Unencrypted **PKCS#8** PEM-encoded RSA private key for that user (headers `-----BEGIN PRIVATE KEY-----`). Convert a PKCS#1 key with `openssl pkcs8 -topk8 -nocrypt` | | **Database** | string | Yes | `ANALYTICS` | Target database | | **Schema** | string | Yes | `PUBLIC` | Target schema | | **Table name** | string | Yes | `ampersand_events` | Target table | diff --git a/src/destinations/sqs.mdx b/src/destinations/sqs.mdx new file mode 100644 index 00000000..28f3e999 --- /dev/null +++ b/src/destinations/sqs.mdx @@ -0,0 +1,48 @@ +--- +title: Amazon SQS destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as messages to your Amazon SQS queue. + +## Prerequisites + +Before setting up an SQS destination, ensure that you have: +- An AWS account with an SQS queue +- AWS credentials with `sqs:SendMessage` permission on that queue + +## Data format + +Ampersand publishes each read or subscribe message to your queue as JSON. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + +## Create an SQS destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new SQS destination. + +You'll need to provide: + +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandSQS` | Alias to reference in your `amp.yaml` file | +| **Queue URL** | string | Yes | `https://sqs.us-east-1.amazonaws.com/123456789012/my-queue` | URL of your SQS queue | +| **AWS Access Key ID** | string | Yes | `AKIAIOSFODNN7EXAMPLE` | AWS access key with `sqs:SendMessage` permission | +| **AWS Secret Access Key** | string | Yes | | AWS secret access key | + +## Refer to the destination in your integration + +After creating your SQS destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToSQS + displayName: Salesforce to SQS + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandSQS + - objectName: contact + destination: ampersandSQS +``` diff --git a/src/docs.json b/src/docs.json index c92314fc..1eb8c6e5 100644 --- a/src/docs.json +++ b/src/docs.json @@ -75,9 +75,13 @@ ] }, { - "group": "Queues", + "group": "Streams and queues", "pages": [ - "destinations/kinesis" + "destinations/azureservicebus", + "destinations/kinesis", + "destinations/pubsub", + "destinations/rabbitmq", + "destinations/sqs" ] } ] diff --git a/src/generate-docs.ts b/src/generate-docs.ts index 47ab4f42..1c66f0eb 100644 --- a/src/generate-docs.ts +++ b/src/generate-docs.ts @@ -306,9 +306,13 @@ const baseConfig = { ] }, { - group: "Queues", + group: "Streams and queues", pages: [ + "destinations/azureservicebus", "destinations/kinesis", + "destinations/pubsub", + "destinations/rabbitmq", + "destinations/sqs", ] } ]