From 89b5747b5bf0580bef67739b3c69aec8d8f4e86c Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Mon, 17 Aug 2026 12:14:46 -0700 Subject: [PATCH 01/11] docs(destinations): add 11 warehouse/storage/queue destination pages (ENG-4130) Add pages for ClickHouse, Snowflake, BigQuery, Redshift, Azure Blob Storage, Google Cloud Storage, Amazon SQS, Amazon SNS, Amazon EventBridge, Google Cloud Pub/Sub, and RabbitMQ, modeled after the existing Kinesis/S3 destination pages. Register them in generate-docs.ts (regenerated docs.json) and update the destinations overview. Co-Authored-By: Claude Opus 4.8 --- src/destinations/azure-blob.mdx | 57 +++++++++++++++++++++++++++++ src/destinations/bigquery.mdx | 58 +++++++++++++++++++++++++++++ src/destinations/clickhouse.mdx | 59 ++++++++++++++++++++++++++++++ src/destinations/eventbridge.mdx | 58 +++++++++++++++++++++++++++++ src/destinations/gcs.mdx | 55 ++++++++++++++++++++++++++++ src/destinations/overview.mdx | 28 +++++++++++--- src/destinations/pubsub.mdx | 56 ++++++++++++++++++++++++++++ src/destinations/rabbitmq.mdx | 56 ++++++++++++++++++++++++++++ src/destinations/redshift.mdx | 63 ++++++++++++++++++++++++++++++++ src/destinations/snowflake.mdx | 60 ++++++++++++++++++++++++++++++ src/destinations/sns.mdx | 58 +++++++++++++++++++++++++++++ src/destinations/sqs.mdx | 58 +++++++++++++++++++++++++++++ src/docs.json | 13 ++++++- src/generate-docs.ts | 11 ++++++ 14 files changed, 683 insertions(+), 7 deletions(-) create mode 100644 src/destinations/azure-blob.mdx create mode 100644 src/destinations/bigquery.mdx create mode 100644 src/destinations/clickhouse.mdx create mode 100644 src/destinations/eventbridge.mdx create mode 100644 src/destinations/gcs.mdx create mode 100644 src/destinations/pubsub.mdx create mode 100644 src/destinations/rabbitmq.mdx create mode 100644 src/destinations/redshift.mdx create mode 100644 src/destinations/snowflake.mdx create mode 100644 src/destinations/sns.mdx create mode 100644 src/destinations/sqs.mdx diff --git a/src/destinations/azure-blob.mdx b/src/destinations/azure-blob.mdx new file mode 100644 index 00000000..aaf31fa1 --- /dev/null +++ b/src/destinations/azure-blob.mdx @@ -0,0 +1,57 @@ +--- +title: Azure Blob Storage destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can write the payload as blobs to your Azure Blob Storage container. + +## Prerequisites + +Before setting up an Azure Blob Storage destination, ensure that you have: +- An Azure Storage account +- A blob container to write to +- The storage account's access key + +## Create an Azure Blob Storage destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Azure Blob Storage destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Account** | string | Yes | Azure Storage account name | +| **Container** | string | Yes | Blob container to write objects to | +| **Access key** | string | Yes | Storage account access key | + + +Ampersand encrypts and stores your access key securely. + + +## Refer to the destination in your integration + +After creating your Azure Blob Storage destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToAzureBlob + displayName: Salesforce to Azure Blob Storage + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandAzureBlob + - objectName: contact + destination: ampersandAzureBlob +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your Azure Blob Storage container. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about scaling, contact `support@withampersand.com`. + diff --git a/src/destinations/bigquery.mdx b/src/destinations/bigquery.mdx new file mode 100644 index 00000000..2f577294 --- /dev/null +++ b/src/destinations/bigquery.mdx @@ -0,0 +1,58 @@ +--- +title: BigQuery destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can insert the payload as rows into your Google BigQuery table. + +## Prerequisites + +Before setting up a BigQuery destination, ensure that you have: +- A Google Cloud project with BigQuery enabled +- A dataset and table to insert into +- A service account with the `BigQuery Data Editor` role (or `bigquery.tables.updateData` permission) on that dataset, and a JSON key for it + +## Create a BigQuery destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new BigQuery destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Project ID** | string | Yes | Google Cloud project ID | +| **Dataset ID** | string | Yes | BigQuery dataset ID | +| **Table ID** | string | Yes | BigQuery table ID | +| **Service account credentials** | string | Yes | Service account key JSON with write access to the table | + + +Ampersand encrypts and stores your service account credentials securely. + + +## Refer to the destination in your integration + +After creating your BigQuery destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToBigQuery + displayName: Salesforce to BigQuery + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandBigQuery + - objectName: contact + destination: ampersandBigQuery +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your BigQuery table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about mapping payloads to your table schema, contact `support@withampersand.com`. + diff --git a/src/destinations/clickhouse.mdx b/src/destinations/clickhouse.mdx new file mode 100644 index 00000000..1084ee79 --- /dev/null +++ b/src/destinations/clickhouse.mdx @@ -0,0 +1,59 @@ +--- +title: ClickHouse destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can insert the payload as rows into your ClickHouse table. + +## Prerequisites + +Before setting up a ClickHouse destination, ensure that you have: +- A ClickHouse instance reachable over HTTPS (ClickHouse Cloud or self-hosted) +- A target table to insert into +- A username and password with `INSERT` privileges on that table + +## Create a ClickHouse destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new ClickHouse destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **URL** | string | Yes | HTTPS URL of your ClickHouse instance (e.g. `https://abc.clickhouse.cloud:8443`) | +| **Username** | string | Yes | ClickHouse username | +| **Password** | string | Yes | ClickHouse password | +| **Table name** | string | Yes | Table to insert rows into | +| **Database** | string | No | Database containing the table (defaults to the connection's database) | + + +Ampersand encrypts and stores your password securely. + + +## Refer to the destination in your integration + +After creating your ClickHouse destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToClickHouse + displayName: Salesforce to ClickHouse + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandClickHouse + - objectName: contact + destination: ampersandClickHouse +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your ClickHouse table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about mapping payloads to your table schema, contact `support@withampersand.com`. + diff --git a/src/destinations/eventbridge.mdx b/src/destinations/eventbridge.mdx new file mode 100644 index 00000000..46c4199f --- /dev/null +++ b/src/destinations/eventbridge.mdx @@ -0,0 +1,58 @@ +--- +title: Amazon EventBridge destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as events to your Amazon EventBridge event bus. + +## Prerequisites + +Before setting up an EventBridge destination, ensure that you have: +- An AWS account with an EventBridge event bus +- AWS credentials with `events:PutEvents` permission on that event bus + +## Create an EventBridge destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new EventBridge destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Event bus name** | string | Yes | Name of your EventBridge event bus (e.g. `default`) | +| **Region** | string | Yes | AWS region where your event bus is located (e.g. `us-west-2`) | +| **AWS Access Key ID** | string | Yes | AWS access key with `events:PutEvents` permission | +| **AWS Secret Access Key** | string | Yes | AWS secret access key | +| **Detail type** | string | No | Value to set as the EventBridge `detail-type` on published events | + + +Ampersand encrypts and stores your AWS credentials securely. + + +## Refer to the destination in your integration + +After creating your EventBridge destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToEventBridge + displayName: Salesforce to EventBridge + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandEventBridge + - objectName: contact + destination: ampersandEventBridge +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your EventBridge event bus. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about scaling, contact `support@withampersand.com`. + diff --git a/src/destinations/gcs.mdx b/src/destinations/gcs.mdx new file mode 100644 index 00000000..4d90d704 --- /dev/null +++ b/src/destinations/gcs.mdx @@ -0,0 +1,55 @@ +--- +title: Google Cloud Storage destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can write the payload as objects to your Google Cloud Storage bucket. + +## Prerequisites + +Before setting up a Google Cloud Storage destination, ensure that you have: +- A Google Cloud project with a Cloud Storage bucket +- A service account with the `Storage Object Creator` role (or `storage.objects.create` permission) on that bucket, and a JSON key for it + +## Create a Google Cloud Storage destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Google Cloud Storage destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Bucket** | string | Yes | Name of your Cloud Storage bucket | +| **Service account credentials** | string | Yes | Service account key JSON with write access to the bucket | + + +Ampersand encrypts and stores your service account credentials securely. + + +## Refer to the destination in your integration + +After creating your Google Cloud Storage destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToGCS + displayName: Salesforce to Google Cloud Storage + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandGCS + - objectName: contact + destination: ampersandGCS +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your Google Cloud Storage bucket. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about scaling, contact `support@withampersand.com`. + diff --git a/src/destinations/overview.mdx b/src/destinations/overview.mdx index ff42f2ae..4265e6a4 100644 --- a/src/destinations/overview.mdx +++ b/src/destinations/overview.mdx @@ -2,7 +2,7 @@ title: Overview --- -Ampersand currently supports webhook, Kinesis, and S3 destinations. Destinations allow you to route data synced from SaaS instances via [Read Actions](/read-actions) or [Subscribe Actions](/subscribe-actions), and to receive real-time [Notifications](/notifications) about important lifecycle events in your projects. +Ampersand supports webhook, Kinesis, and S3 destinations, along with a range of data warehouse, object storage, and message queue destinations (ClickHouse, Snowflake, BigQuery, Redshift, Azure Blob Storage, Google Cloud Storage, Amazon SQS, Amazon SNS, Amazon EventBridge, Google Cloud Pub/Sub, and RabbitMQ). Destinations allow you to route data synced from SaaS instances via [Read Actions](/read-actions) or [Subscribe Actions](/subscribe-actions), and to receive real-time [Notifications](/notifications) about important lifecycle events in your projects. ## Add a destination to the Ampersand Dashboard @@ -38,14 +38,30 @@ Destinations can also be used to receive [notifications](/notifications) about i * [Amazon Kinesis destinations](/destinations/kinesis) * [Amazon S3 destinations](/destinations/s3) +### Data warehouses + +* [ClickHouse destinations](/destinations/clickhouse) +* [Snowflake destinations](/destinations/snowflake) +* [BigQuery destinations](/destinations/bigquery) +* [Amazon Redshift destinations](/destinations/redshift) + +### Object storage + +* [Azure Blob Storage destinations](/destinations/azure-blob) +* [Google Cloud Storage destinations](/destinations/gcs) + +### Message queues and event buses + +* [Amazon SQS destinations](/destinations/sqs) +* [Amazon SNS destinations](/destinations/sns) +* [Amazon EventBridge destinations](/destinations/eventbridge) +* [Google Cloud Pub/Sub destinations](/destinations/pubsub) +* [RabbitMQ destinations](/destinations/rabbitmq) + ## Other Destinations -We have many other destination types on the roadmap, including: +We have more destination types on the roadmap, including: * Postgres * Ampersand-hosted Postgres -* Amazon SQS -* Google Cloud Storage -* Google PubSub -* RabbitMQ * Azure Service Bus diff --git a/src/destinations/pubsub.mdx b/src/destinations/pubsub.mdx new file mode 100644 index 00000000..0d2f0884 --- /dev/null +++ b/src/destinations/pubsub.mdx @@ -0,0 +1,56 @@ +--- +title: Google Cloud Pub/Sub destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as messages to your Google Cloud Pub/Sub topic. + +## Prerequisites + +Before setting up a Pub/Sub destination, ensure that you have: +- A Google Cloud project with a Pub/Sub topic +- A service account with the `Pub/Sub Publisher` role (or `pubsub.topics.publish` permission) on that topic, and a JSON key for it + +## Create a Pub/Sub destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Pub/Sub destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Project ID** | string | Yes | Google Cloud project ID | +| **Topic ID** | string | Yes | Pub/Sub topic ID | +| **Service account credentials** | string | Yes | Service account key JSON with publish access to the topic | + + +Ampersand encrypts and stores your service account credentials securely. + + +## Refer to the destination in your integration + +After creating your Pub/Sub destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToPubSub + displayName: Salesforce to Pub/Sub + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandPubSub + - objectName: contact + destination: ampersandPubSub +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your Pub/Sub topic. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about scaling, contact `support@withampersand.com`. + diff --git a/src/destinations/rabbitmq.mdx b/src/destinations/rabbitmq.mdx new file mode 100644 index 00000000..5173cef7 --- /dev/null +++ b/src/destinations/rabbitmq.mdx @@ -0,0 +1,56 @@ +--- +title: RabbitMQ destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as messages to your RabbitMQ exchange. + +## Prerequisites + +Before setting up a RabbitMQ destination, ensure that you have: +- A RabbitMQ instance reachable from the internet +- A connection URI with permission to publish +- A routing key for your exchange + +## Create a RabbitMQ destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new RabbitMQ destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Connection URI** | string | Yes | AMQP connection URI, e.g. `amqp://user:pass@host:5672` (it embeds your credentials) | +| **Routing key** | string | Yes | Routing key to publish messages with | + + +The connection URI embeds your username and password. Ampersand encrypts and stores it securely. + + +## Refer to the destination in your integration + +After creating your RabbitMQ destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToRabbitMQ + displayName: Salesforce to RabbitMQ + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandRabbitMQ + - objectName: contact + destination: ampersandRabbitMQ +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your RabbitMQ exchange. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about scaling, contact `support@withampersand.com`. + diff --git a/src/destinations/redshift.mdx b/src/destinations/redshift.mdx new file mode 100644 index 00000000..30a2f1e7 --- /dev/null +++ b/src/destinations/redshift.mdx @@ -0,0 +1,63 @@ +--- +title: Amazon Redshift destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can insert the payload as rows into your Amazon Redshift table. Both Redshift Serverless (via a workgroup) and provisioned clusters (via a cluster identifier) are supported. + +## Prerequisites + +Before setting up a Redshift destination, ensure that you have: +- An Amazon Redshift Serverless workgroup or a provisioned cluster +- A target database, and optionally a schema and table +- AWS credentials with permission to run the Redshift Data API and insert into the target table + +## Create a Redshift destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Redshift destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Region** | string | Yes | AWS region where your Redshift resource lives (e.g. `us-west-2`) | +| **AWS Access Key ID** | string | Yes | AWS access key with Redshift Data API permissions | +| **AWS Secret Access Key** | string | Yes | AWS secret access key | +| **Database** | string | No | Target database name | +| **Schema** | string | No | Target schema | +| **Table name** | string | No | Target table | +| **Workgroup name** | string | No | Redshift Serverless workgroup name | +| **Cluster identifier** | string | No | Provisioned cluster identifier | +| **DB user** | string | No | Database user (for provisioned clusters) | + + +Provide a **workgroup name** for Redshift Serverless, or a **cluster identifier** (and **DB user**) for a provisioned cluster. Ampersand encrypts and stores your AWS credentials securely. + + +## Refer to the destination in your integration + +After creating your Redshift destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToRedshift + displayName: Salesforce to Redshift + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandRedshift + - objectName: contact + destination: ampersandRedshift +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your Redshift table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about mapping payloads to your table schema, contact `support@withampersand.com`. + diff --git a/src/destinations/snowflake.mdx b/src/destinations/snowflake.mdx new file mode 100644 index 00000000..1a9582a3 --- /dev/null +++ b/src/destinations/snowflake.mdx @@ -0,0 +1,60 @@ +--- +title: Snowflake destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can insert the payload as rows into your Snowflake table. + +## Prerequisites + +Before setting up a Snowflake destination, ensure that you have: +- A Snowflake account +- A user configured for [key-pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth) (Ampersand authenticates with an RSA private key, not a password) +- A target database, schema, and table, with `INSERT` privileges granted to that user + +## Create a Snowflake destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Snowflake destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Account identifier** | string | Yes | Your Snowflake account identifier (e.g. `myorg-myaccount`) | +| **User ID** | string | Yes | Snowflake user configured for key-pair auth | +| **Private key** | string | Yes | PEM-encoded RSA private key for that user | +| **Database** | string | No | Target database | +| **Schema** | string | No | Target schema | +| **Table name** | string | No | Target table | + + +Ampersand encrypts and stores your private key securely. + + +## Refer to the destination in your integration + +After creating your Snowflake destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToSnowflake + displayName: Salesforce to Snowflake + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandSnowflake + - objectName: contact + destination: ampersandSnowflake +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your Snowflake table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about mapping payloads to your table schema, contact `support@withampersand.com`. + diff --git a/src/destinations/sns.mdx b/src/destinations/sns.mdx new file mode 100644 index 00000000..4299f83f --- /dev/null +++ b/src/destinations/sns.mdx @@ -0,0 +1,58 @@ +--- +title: Amazon SNS destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as messages to your Amazon SNS topic. + +## Prerequisites + +Before setting up an SNS destination, ensure that you have: +- An AWS account with an SNS topic +- AWS credentials with `sns:Publish` permission on that topic + +## Create an SNS destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new SNS destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Topic ARN** | string | Yes | ARN of your SNS topic | +| **Region** | string | Yes | AWS region where your topic is located (e.g. `us-west-2`) | +| **AWS Access Key ID** | string | Yes | AWS access key with `sns:Publish` permission | +| **AWS Secret Access Key** | string | Yes | AWS secret access key | +| **Endpoint URL** | string | No | Custom SNS endpoint if applicable (e.g. when using [LocalStack](https://www.localstack.cloud)) | + + +Ampersand encrypts and stores your AWS credentials securely. + + +## Refer to the destination in your integration + +After creating your SNS destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToSNS + displayName: Salesforce to SNS + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandSNS + - objectName: contact + destination: ampersandSNS +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your SNS topic. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about scaling, contact `support@withampersand.com`. + diff --git a/src/destinations/sqs.mdx b/src/destinations/sqs.mdx new file mode 100644 index 00000000..ef958b03 --- /dev/null +++ b/src/destinations/sqs.mdx @@ -0,0 +1,58 @@ +--- +title: Amazon SQS destinations +--- + +For more information on destinations, see the [Destinations](/destinations) page. + +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as messages to your Amazon SQS queue. + +## Prerequisites + +Before setting up an SQS destination, ensure that you have: +- An AWS account with an SQS queue +- AWS credentials with `sqs:SendMessage` permission on that queue + +## Create an SQS destination + +Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new SQS destination. + +You'll need to provide: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | +| **Queue URL** | string | Yes | URL of your SQS queue | +| **Region** | string | Yes | AWS region where your queue is located (e.g. `us-west-2`) | +| **AWS Access Key ID** | string | Yes | AWS access key with `sqs:SendMessage` permission | +| **AWS Secret Access Key** | string | Yes | AWS secret access key | +| **Endpoint URL** | string | No | Custom SQS endpoint if applicable (e.g. when using [LocalStack](https://www.localstack.cloud)) | + + +Ampersand encrypts and stores your AWS credentials securely. + + +## Refer to the destination in your integration + +After creating your SQS destination, reference it in your `amp.yaml` file: + +```yaml +specVersion: 1.0.0 +integrations: + - name: salesforceToSQS + displayName: Salesforce to SQS + provider: salesforce + read: + objects: + - objectName: account + destination: ampersandSQS + - objectName: contact + destination: ampersandSQS +``` + +## Message format + +Ampersand delivers the standard read and subscribe action payloads to your SQS queue. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. + + +If you have questions about scaling, contact `support@withampersand.com`. + diff --git a/src/docs.json b/src/docs.json index ab0120ee..b3cd7624 100644 --- a/src/docs.json +++ b/src/docs.json @@ -59,7 +59,18 @@ "destinations/overview", "destinations/webhooks", "destinations/kinesis", - "destinations/s3" + "destinations/s3", + "destinations/clickhouse", + "destinations/snowflake", + "destinations/bigquery", + "destinations/redshift", + "destinations/azure-blob", + "destinations/gcs", + "destinations/sqs", + "destinations/sns", + "destinations/eventbridge", + "destinations/pubsub", + "destinations/rabbitmq" ] }, { diff --git a/src/generate-docs.ts b/src/generate-docs.ts index 8f39d6f9..a18a1f22 100644 --- a/src/generate-docs.ts +++ b/src/generate-docs.ts @@ -291,6 +291,17 @@ const baseConfig = { "destinations/webhooks", "destinations/kinesis", "destinations/s3", + "destinations/clickhouse", + "destinations/snowflake", + "destinations/bigquery", + "destinations/redshift", + "destinations/azure-blob", + "destinations/gcs", + "destinations/sqs", + "destinations/sns", + "destinations/eventbridge", + "destinations/pubsub", + "destinations/rabbitmq", ] }, { From 92996cb4050651364da729142276a384f9ddfe33 Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 10:05:24 -0700 Subject: [PATCH 02/11] docs(destinations): add target-table schemas, batching, and considerations (ENG-4130) Warehouse pages now document the exact table the sink expects (BigQuery id/payload; Snowflake id/created_at/payload; Redshift created_at/payload; ClickHouse one column per top-level field) plus the optional batchSize/maxWaitSecs. Overview adds a "Considerations" section: pre-created warehouse tables, batching, the 10-per-project sink limit, and the GCS/Azure object-naming caveat. Co-Authored-By: Claude Opus 4.8 --- src/destinations/bigquery.mdx | 22 ++++++++++++++++++++++ src/destinations/clickhouse.mdx | 33 +++++++++++++++++++++++++++++++++ src/destinations/overview.mdx | 9 +++++++++ src/destinations/redshift.mdx | 24 +++++++++++++++++++++++- src/destinations/snowflake.mdx | 22 ++++++++++++++++++++++ 5 files changed, 109 insertions(+), 1 deletion(-) diff --git a/src/destinations/bigquery.mdx b/src/destinations/bigquery.mdx index 2f577294..bdfba1cc 100644 --- a/src/destinations/bigquery.mdx +++ b/src/destinations/bigquery.mdx @@ -13,6 +13,22 @@ Before setting up a BigQuery destination, ensure that you have: - A dataset and table to insert into - A service account with the `BigQuery Data Editor` role (or `bigquery.tables.updateData` permission) on that dataset, and a JSON key for it +## Set up your table + +The table must exist before you create the destination. The BigQuery sink writes each +message into two columns: `id` (a generated identifier) and `payload` (the message JSON as a +string). Create the table with exactly these columns: + +```sql +CREATE TABLE `my-project.my_dataset.my_table` ( + id STRING, + payload STRING +); +``` + +To split the payload into typed columns instead, use a +[Svix transformation](https://docs.svix.com/stream/sinks/bigquery). + ## Create a BigQuery destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new BigQuery destination. @@ -26,6 +42,8 @@ You'll need to provide: | **Dataset ID** | string | Yes | BigQuery dataset ID | | **Table ID** | string | Yes | BigQuery table ID | | **Service account credentials** | string | Yes | Service account key JSON with write access to the table | +| **Batch size** | integer | No | Rows to buffer before flushing a batch (defaults to 1000) | +| **Max wait (seconds)** | integer | No | Max seconds to wait before flushing a batch (defaults to 30) | Ampersand encrypts and stores your service account credentials securely. @@ -49,6 +67,10 @@ integrations: destination: ampersandBigQuery ``` +## Batching + +Warehouse destinations buffer rows and flush a batch when it reaches **Batch size** rows or after **Max wait (seconds)**, whichever comes first (defaults: 1000 rows / 30 seconds). Lower the max wait for lower-latency delivery when volume is low. + ## Message format Ampersand delivers the standard read and subscribe action payloads to your BigQuery table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. diff --git a/src/destinations/clickhouse.mdx b/src/destinations/clickhouse.mdx index 1084ee79..0884c8dd 100644 --- a/src/destinations/clickhouse.mdx +++ b/src/destinations/clickhouse.mdx @@ -13,6 +13,33 @@ Before setting up a ClickHouse destination, ensure that you have: - A target table to insert into - A username and password with `INSERT` privileges on that table +## Set up your table + +The table must exist before you create the destination. Unlike the other warehouse +destinations, the ClickHouse sink maps the **top-level fields of each message to table +columns by name** — it does not add its own `id`/`payload` columns. Create a table whose +columns match the delivered fields (fields without a matching column are ignored): + +```sql +CREATE TABLE ampersand_events ( + action String, + provider String, + objectName String, + projectId String, + installationId String, + operationId String, + operationTime String, + result String +) +ENGINE = MergeTree() +ORDER BY tuple(); +``` + +`result` holds the array of synced records as a JSON string; the other columns are the +message envelope (see [Message format](#message-format)). If you'd rather shape the data +differently (for example, one row per record), use a +[Svix transformation](https://docs.svix.com/stream/sinks/clickhouse). + ## Create a ClickHouse destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new ClickHouse destination. @@ -27,6 +54,8 @@ You'll need to provide: | **Password** | string | Yes | ClickHouse password | | **Table name** | string | Yes | Table to insert rows into | | **Database** | string | No | Database containing the table (defaults to the connection's database) | +| **Batch size** | integer | No | Rows to buffer before flushing a batch (defaults to 1000) | +| **Max wait (seconds)** | integer | No | Max seconds to wait before flushing a batch (defaults to 30) | Ampersand encrypts and stores your password securely. @@ -50,6 +79,10 @@ integrations: destination: ampersandClickHouse ``` +## Batching + +Warehouse destinations buffer rows and flush a batch when it reaches **Batch size** rows or after **Max wait (seconds)**, whichever comes first (defaults: 1000 rows / 30 seconds). Lower the max wait for lower-latency delivery when volume is low. + ## Message format Ampersand delivers the standard read and subscribe action payloads to your ClickHouse table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. diff --git a/src/destinations/overview.mdx b/src/destinations/overview.mdx index 4265e6a4..fb9ad8a5 100644 --- a/src/destinations/overview.mdx +++ b/src/destinations/overview.mdx @@ -58,6 +58,15 @@ Destinations can also be used to receive [notifications](/notifications) about i * [Google Cloud Pub/Sub destinations](/destinations/pubsub) * [RabbitMQ destinations](/destinations/rabbitmq) +## Considerations for warehouse, storage, and queue destinations + +These destinations (everything except webhooks) are delivered through managed sink infrastructure. A few things to know when setting them up: + +* **Warehouse destinations need a pre-created table.** ClickHouse, Snowflake, BigQuery, and Redshift each require a table that already exists and whose schema matches how the sink inserts rows — see the destination's page for the exact `CREATE TABLE`. Object-storage and message-queue destinations need no schema; each message is written/published as-is. +* **Warehouse delivery is batched.** Rows are buffered and flushed when a batch fills or a max-wait interval elapses (defaults: 1000 rows / 30 seconds), both configurable per destination. Object-storage and queue destinations deliver per message. +* **Per-project limit.** A project can have up to **10** of these destinations in total (webhook destinations are separate and not counted). Contact `support@withampersand.com` if you need more. +* **Object naming.** Objects written to Google Cloud Storage and Azure Blob Storage are named `object-generated-by-svix` and the name is not currently customizable. + ## Other Destinations We have more destination types on the roadmap, including: diff --git a/src/destinations/redshift.mdx b/src/destinations/redshift.mdx index 30a2f1e7..d6825287 100644 --- a/src/destinations/redshift.mdx +++ b/src/destinations/redshift.mdx @@ -13,6 +13,22 @@ Before setting up a Redshift destination, ensure that you have: - A target database, and optionally a schema and table - AWS credentials with permission to run the Redshift Data API and insert into the target table +## Set up your table + +The table must exist before you create the destination. The Redshift sink writes each message +into two columns: `created_at` and `payload` (the message JSON as a string): + +```sql +CREATE TABLE events ( + created_at TIMESTAMP, + payload VARCHAR(65535) +); +``` + +`VARCHAR(65535)` is Redshift's maximum column size; a message larger than that will disable the +sink. To split the payload into typed columns instead, use a +[Svix transformation](https://docs.svix.com/stream/sinks/redshift). + ## Create a Redshift destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Redshift destination. @@ -31,9 +47,11 @@ You'll need to provide: | **Workgroup name** | string | No | Redshift Serverless workgroup name | | **Cluster identifier** | string | No | Provisioned cluster identifier | | **DB user** | string | No | Database user (for provisioned clusters) | +| **Batch size** | integer | No | Rows to buffer before flushing a batch (defaults to 1000) | +| **Max wait (seconds)** | integer | No | Max seconds to wait before flushing a batch (defaults to 30) | -Provide a **workgroup name** for Redshift Serverless, or a **cluster identifier** (and **DB user**) for a provisioned cluster. Ampersand encrypts and stores your AWS credentials securely. +Provide a **workgroup name** for Redshift Serverless, or a **cluster identifier** (and **DB user**) for a provisioned cluster — one of the two is required. Ampersand encrypts and stores your AWS credentials securely. ## Refer to the destination in your integration @@ -54,6 +72,10 @@ integrations: destination: ampersandRedshift ``` +## Batching + +Warehouse destinations buffer rows and flush a batch when it reaches **Batch size** rows or after **Max wait (seconds)**, whichever comes first (defaults: 1000 rows / 30 seconds). Lower the max wait for lower-latency delivery when volume is low. + ## Message format Ampersand delivers the standard read and subscribe action payloads to your Redshift table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. diff --git a/src/destinations/snowflake.mdx b/src/destinations/snowflake.mdx index 1a9582a3..efecde60 100644 --- a/src/destinations/snowflake.mdx +++ b/src/destinations/snowflake.mdx @@ -13,6 +13,22 @@ Before setting up a Snowflake destination, ensure that you have: - A user configured for [key-pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth) (Ampersand authenticates with an RSA private key, not a password) - A target database, schema, and table, with `INSERT` privileges granted to that user +## Set up your table + +The table must exist before you create the destination. The Snowflake sink writes each +message into three columns: `id`, `created_at`, and `payload` (the message JSON as text): + +```sql +CREATE TABLE my_db.my_schema.my_table ( + id TEXT, + created_at TIMESTAMP, + payload TEXT +); +``` + +To split the payload into typed columns instead, use a +[Svix transformation](https://docs.svix.com/stream/sinks/snowflake). + ## Create a Snowflake destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Snowflake destination. @@ -28,6 +44,8 @@ You'll need to provide: | **Database** | string | No | Target database | | **Schema** | string | No | Target schema | | **Table name** | string | No | Target table | +| **Batch size** | integer | No | Rows to buffer before flushing a batch (defaults to 1000) | +| **Max wait (seconds)** | integer | No | Max seconds to wait before flushing a batch (defaults to 30) | Ampersand encrypts and stores your private key securely. @@ -51,6 +69,10 @@ integrations: destination: ampersandSnowflake ``` +## Batching + +Warehouse destinations buffer rows and flush a batch when it reaches **Batch size** rows or after **Max wait (seconds)**, whichever comes first (defaults: 1000 rows / 30 seconds). Lower the max wait for lower-latency delivery when volume is low. + ## Message format Ampersand delivers the standard read and subscribe action payloads to your Snowflake table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. From f64d0e2fc278620b8031fb5772b7d5f0836b3a62 Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 13:40:03 -0700 Subject: [PATCH 03/11] docs(destinations): rework sink pages per review (ENG-4130) - Merge table-setup + message-format into a "Data format" section placed before "Refer to the destination", linking to webhook.yaml for the payload schema. - Warehouses show the table they need: ClickHouse maps top-level keys to columns (example with ...); BigQuery/Snowflake/Redshift load the payload into a single column (id/created_at/payload as applicable). - Add an Example column to every "Create a destination" table. - Remove em dashes, remove "we store credentials securely" notes, and remove all vendor (Svix) references. - Alphabetize the Destinations sidebar; drop the overview "Considerations" section. Co-Authored-By: Claude Opus 4.8 --- src/destinations/azure-blob.mdx | 30 ++++++--------- src/destinations/bigquery.mdx | 49 ++++++++----------------- src/destinations/clickhouse.mdx | 61 ++++++++++--------------------- src/destinations/eventbridge.mdx | 34 +++++++---------- src/destinations/gcs.mdx | 28 +++++--------- src/destinations/overview.mdx | 9 ----- src/destinations/pubsub.mdx | 30 ++++++--------- src/destinations/rabbitmq.mdx | 30 ++++++--------- src/destinations/redshift.mdx | 63 ++++++++++++-------------------- src/destinations/snowflake.mdx | 52 +++++++++----------------- src/destinations/sns.mdx | 34 +++++++---------- src/destinations/sqs.mdx | 34 +++++++---------- src/docs.json | 22 +++++------ src/generate-docs.ts | 20 +++++----- 14 files changed, 178 insertions(+), 318 deletions(-) diff --git a/src/destinations/azure-blob.mdx b/src/destinations/azure-blob.mdx index aaf31fa1..fb251e6a 100644 --- a/src/destinations/azure-blob.mdx +++ b/src/destinations/azure-blob.mdx @@ -4,7 +4,7 @@ title: Azure Blob Storage destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can write the payload as blobs to your Azure Blob Storage container. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand writes the payload as objects to your Azure Blob Storage container. ## Prerequisites @@ -13,22 +13,22 @@ Before setting up an Azure Blob Storage destination, ensure that you have: - A blob container to write to - The storage account's access key +## Data format + +Ampersand writes each read or subscribe message as a JSON object to your container. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + ## Create an Azure Blob Storage destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Azure Blob Storage destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Account** | string | Yes | Azure Storage account name | -| **Container** | string | Yes | Blob container to write objects to | -| **Access key** | string | Yes | Storage account access key | - - -Ampersand encrypts and stores your access key securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandAzureBlob` | Alias to reference in your `amp.yaml` file | +| **Account** | string | Yes | `mystorageaccount` | Azure Storage account name | +| **Container** | string | Yes | `ampersand-data` | Blob container to write objects to | +| **Access key** | string | Yes | | Storage account access key | ## Refer to the destination in your integration @@ -47,11 +47,3 @@ integrations: - objectName: contact destination: ampersandAzureBlob ``` - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your Azure Blob Storage container. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about scaling, contact `support@withampersand.com`. - diff --git a/src/destinations/bigquery.mdx b/src/destinations/bigquery.mdx index bdfba1cc..7a204f2e 100644 --- a/src/destinations/bigquery.mdx +++ b/src/destinations/bigquery.mdx @@ -4,20 +4,20 @@ title: BigQuery destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can insert the payload as rows into your Google BigQuery table. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand inserts the payload as rows into your Google BigQuery table. ## Prerequisites Before setting up a BigQuery destination, ensure that you have: - A Google Cloud project with BigQuery enabled -- A dataset and table to insert into +- A dataset and table to insert into (see [Data format and table setup](#data-format-and-table-setup)) - A service account with the `BigQuery Data Editor` role (or `bigquery.tables.updateData` permission) on that dataset, and a JSON key for it -## Set up your table +## Data format and table setup -The table must exist before you create the destination. The BigQuery sink writes each -message into two columns: `id` (a generated identifier) and `payload` (the message JSON as a -string). Create the table with exactly these columns: +Ampersand delivers each read or subscribe message as a JSON object. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + +The BigQuery destination writes each message into two columns: `id` (a generated identifier) and `payload` (the full message JSON as a string). Create the table with exactly these columns, then read individual fields from `payload` with BigQuery's [JSON functions](https://cloud.google.com/bigquery/docs/reference/standard-sql/json_functions). ```sql CREATE TABLE `my-project.my_dataset.my_table` ( @@ -26,28 +26,21 @@ CREATE TABLE `my-project.my_dataset.my_table` ( ); ``` -To split the payload into typed columns instead, use a -[Svix transformation](https://docs.svix.com/stream/sinks/bigquery). - ## Create a BigQuery destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new BigQuery destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Project ID** | string | Yes | Google Cloud project ID | -| **Dataset ID** | string | Yes | BigQuery dataset ID | -| **Table ID** | string | Yes | BigQuery table ID | -| **Service account credentials** | string | Yes | Service account key JSON with write access to the table | -| **Batch size** | integer | No | Rows to buffer before flushing a batch (defaults to 1000) | -| **Max wait (seconds)** | integer | No | Max seconds to wait before flushing a batch (defaults to 30) | - - -Ampersand encrypts and stores your service account credentials securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandBigQuery` | Alias to reference in your `amp.yaml` file | +| **Project ID** | string | Yes | `my-gcp-project` | Google Cloud project ID | +| **Dataset ID** | string | Yes | `analytics` | BigQuery dataset ID | +| **Table ID** | string | Yes | `ampersand_events` | BigQuery table ID | +| **Service account credentials** | string | Yes | | Service account key JSON with write access to the table | +| **Batch size** | integer | No | `1000` | Rows to buffer before flushing a batch (defaults to 1000) | +| **Max wait (seconds)** | integer | No | `30` | Max seconds to wait before flushing a batch (defaults to 30) | ## Refer to the destination in your integration @@ -66,15 +59,3 @@ integrations: - objectName: contact destination: ampersandBigQuery ``` - -## Batching - -Warehouse destinations buffer rows and flush a batch when it reaches **Batch size** rows or after **Max wait (seconds)**, whichever comes first (defaults: 1000 rows / 30 seconds). Lower the max wait for lower-latency delivery when volume is low. - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your BigQuery table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about mapping payloads to your table schema, contact `support@withampersand.com`. - diff --git a/src/destinations/clickhouse.mdx b/src/destinations/clickhouse.mdx index 0884c8dd..954b3bca 100644 --- a/src/destinations/clickhouse.mdx +++ b/src/destinations/clickhouse.mdx @@ -4,62 +4,51 @@ title: ClickHouse destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can insert the payload as rows into your ClickHouse table. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand inserts the payload as rows into your ClickHouse table. ## Prerequisites Before setting up a ClickHouse destination, ensure that you have: - A ClickHouse instance reachable over HTTPS (ClickHouse Cloud or self-hosted) -- A target table to insert into +- A table to insert into (see [Data format and table setup](#data-format-and-table-setup)) - A username and password with `INSERT` privileges on that table -## Set up your table +## Data format and table setup -The table must exist before you create the destination. Unlike the other warehouse -destinations, the ClickHouse sink maps the **top-level fields of each message to table -columns by name** — it does not add its own `id`/`payload` columns. Create a table whose -columns match the delivered fields (fields without a matching column are ignored): +Ampersand delivers each read or subscribe message as a JSON object. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + +ClickHouse matches the payload's top-level keys to your table's columns by name. Create a column for each top-level key you want to store. Keys with no matching column are ignored. `result` is the array of synced records; store it as a JSON string. ```sql CREATE TABLE ampersand_events ( action String, - provider String, objectName String, - projectId String, - installationId String, operationId String, + installationId String, operationTime String, - result String + result String, + ... ) ENGINE = MergeTree() ORDER BY tuple(); ``` -`result` holds the array of synced records as a JSON string; the other columns are the -message envelope (see [Message format](#message-format)). If you'd rather shape the data -differently (for example, one row per record), use a -[Svix transformation](https://docs.svix.com/stream/sinks/clickhouse). - ## Create a ClickHouse destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new ClickHouse destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **URL** | string | Yes | HTTPS URL of your ClickHouse instance (e.g. `https://abc.clickhouse.cloud:8443`) | -| **Username** | string | Yes | ClickHouse username | -| **Password** | string | Yes | ClickHouse password | -| **Table name** | string | Yes | Table to insert rows into | -| **Database** | string | No | Database containing the table (defaults to the connection's database) | -| **Batch size** | integer | No | Rows to buffer before flushing a batch (defaults to 1000) | -| **Max wait (seconds)** | integer | No | Max seconds to wait before flushing a batch (defaults to 30) | - - -Ampersand encrypts and stores your password securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandClickHouse` | Alias to reference in your `amp.yaml` file | +| **URL** | string | Yes | `https://abc.clickhouse.cloud:8443` | HTTPS URL of your ClickHouse instance | +| **Username** | string | Yes | `default` | ClickHouse username | +| **Password** | string | Yes | | ClickHouse password | +| **Table name** | string | Yes | `ampersand_events` | Table to insert rows into | +| **Database** | string | No | `default` | Database containing the table | +| **Batch size** | integer | No | `1000` | Rows to buffer before flushing a batch (defaults to 1000) | +| **Max wait (seconds)** | integer | No | `30` | Max seconds to wait before flushing a batch (defaults to 30) | ## Refer to the destination in your integration @@ -78,15 +67,3 @@ integrations: - objectName: contact destination: ampersandClickHouse ``` - -## Batching - -Warehouse destinations buffer rows and flush a batch when it reaches **Batch size** rows or after **Max wait (seconds)**, whichever comes first (defaults: 1000 rows / 30 seconds). Lower the max wait for lower-latency delivery when volume is low. - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your ClickHouse table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about mapping payloads to your table schema, contact `support@withampersand.com`. - diff --git a/src/destinations/eventbridge.mdx b/src/destinations/eventbridge.mdx index 46c4199f..29212d2f 100644 --- a/src/destinations/eventbridge.mdx +++ b/src/destinations/eventbridge.mdx @@ -4,7 +4,7 @@ title: Amazon EventBridge destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as events to your Amazon EventBridge event bus. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as events to your Amazon EventBridge event bus. ## Prerequisites @@ -12,24 +12,24 @@ Before setting up an EventBridge destination, ensure that you have: - An AWS account with an EventBridge event bus - AWS credentials with `events:PutEvents` permission on that event bus +## Data format + +Ampersand publishes each read or subscribe message to your event bus. The message payload is carried in the event's `detail` field, and its schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + ## Create an EventBridge destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new EventBridge destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Event bus name** | string | Yes | Name of your EventBridge event bus (e.g. `default`) | -| **Region** | string | Yes | AWS region where your event bus is located (e.g. `us-west-2`) | -| **AWS Access Key ID** | string | Yes | AWS access key with `events:PutEvents` permission | -| **AWS Secret Access Key** | string | Yes | AWS secret access key | -| **Detail type** | string | No | Value to set as the EventBridge `detail-type` on published events | - - -Ampersand encrypts and stores your AWS credentials securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandEventBridge` | Alias to reference in your `amp.yaml` file | +| **Event bus name** | string | Yes | `default` | Name of your EventBridge event bus | +| **Region** | string | Yes | `us-west-2` | AWS region where your event bus is located | +| **AWS Access Key ID** | string | Yes | `AKIAIOSFODNN7EXAMPLE` | AWS access key with `events:PutEvents` permission | +| **AWS Secret Access Key** | string | Yes | | AWS secret access key | +| **Detail type** | string | No | `ampersand.record` | Value to set as the EventBridge `detail-type` on published events | ## Refer to the destination in your integration @@ -48,11 +48,3 @@ integrations: - objectName: contact destination: ampersandEventBridge ``` - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your EventBridge event bus. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about scaling, contact `support@withampersand.com`. - diff --git a/src/destinations/gcs.mdx b/src/destinations/gcs.mdx index 4d90d704..9c4566c5 100644 --- a/src/destinations/gcs.mdx +++ b/src/destinations/gcs.mdx @@ -4,7 +4,7 @@ title: Google Cloud Storage destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can write the payload as objects to your Google Cloud Storage bucket. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand writes the payload as objects to your Google Cloud Storage bucket. ## Prerequisites @@ -12,21 +12,21 @@ Before setting up a Google Cloud Storage destination, ensure that you have: - A Google Cloud project with a Cloud Storage bucket - A service account with the `Storage Object Creator` role (or `storage.objects.create` permission) on that bucket, and a JSON key for it +## Data format + +Ampersand writes each read or subscribe message as a JSON object to your bucket. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + ## Create a Google Cloud Storage destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Google Cloud Storage destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Bucket** | string | Yes | Name of your Cloud Storage bucket | -| **Service account credentials** | string | Yes | Service account key JSON with write access to the bucket | - - -Ampersand encrypts and stores your service account credentials securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandGCS` | Alias to reference in your `amp.yaml` file | +| **Bucket** | string | Yes | `ampersand-integration-bucket` | Name of your Cloud Storage bucket | +| **Service account credentials** | string | Yes | | Service account key JSON with write access to the bucket | ## Refer to the destination in your integration @@ -45,11 +45,3 @@ integrations: - objectName: contact destination: ampersandGCS ``` - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your Google Cloud Storage bucket. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about scaling, contact `support@withampersand.com`. - diff --git a/src/destinations/overview.mdx b/src/destinations/overview.mdx index fb9ad8a5..4265e6a4 100644 --- a/src/destinations/overview.mdx +++ b/src/destinations/overview.mdx @@ -58,15 +58,6 @@ Destinations can also be used to receive [notifications](/notifications) about i * [Google Cloud Pub/Sub destinations](/destinations/pubsub) * [RabbitMQ destinations](/destinations/rabbitmq) -## Considerations for warehouse, storage, and queue destinations - -These destinations (everything except webhooks) are delivered through managed sink infrastructure. A few things to know when setting them up: - -* **Warehouse destinations need a pre-created table.** ClickHouse, Snowflake, BigQuery, and Redshift each require a table that already exists and whose schema matches how the sink inserts rows — see the destination's page for the exact `CREATE TABLE`. Object-storage and message-queue destinations need no schema; each message is written/published as-is. -* **Warehouse delivery is batched.** Rows are buffered and flushed when a batch fills or a max-wait interval elapses (defaults: 1000 rows / 30 seconds), both configurable per destination. Object-storage and queue destinations deliver per message. -* **Per-project limit.** A project can have up to **10** of these destinations in total (webhook destinations are separate and not counted). Contact `support@withampersand.com` if you need more. -* **Object naming.** Objects written to Google Cloud Storage and Azure Blob Storage are named `object-generated-by-svix` and the name is not currently customizable. - ## Other Destinations We have more destination types on the roadmap, including: diff --git a/src/destinations/pubsub.mdx b/src/destinations/pubsub.mdx index 0d2f0884..b79003dc 100644 --- a/src/destinations/pubsub.mdx +++ b/src/destinations/pubsub.mdx @@ -4,7 +4,7 @@ title: Google Cloud Pub/Sub destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as messages to your Google Cloud Pub/Sub topic. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as messages to your Google Cloud Pub/Sub topic. ## Prerequisites @@ -12,22 +12,22 @@ Before setting up a Pub/Sub destination, ensure that you have: - A Google Cloud project with a Pub/Sub topic - A service account with the `Pub/Sub Publisher` role (or `pubsub.topics.publish` permission) on that topic, and a JSON key for it +## Data format + +Ampersand publishes each read or subscribe message to your topic as JSON. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + ## Create a Pub/Sub destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Pub/Sub destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Project ID** | string | Yes | Google Cloud project ID | -| **Topic ID** | string | Yes | Pub/Sub topic ID | -| **Service account credentials** | string | Yes | Service account key JSON with publish access to the topic | - - -Ampersand encrypts and stores your service account credentials securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandPubSub` | Alias to reference in your `amp.yaml` file | +| **Project ID** | string | Yes | `my-gcp-project` | Google Cloud project ID | +| **Topic ID** | string | Yes | `ampersand-events` | Pub/Sub topic ID | +| **Service account credentials** | string | Yes | | Service account key JSON with publish access to the topic | ## Refer to the destination in your integration @@ -46,11 +46,3 @@ integrations: - objectName: contact destination: ampersandPubSub ``` - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your Pub/Sub topic. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about scaling, contact `support@withampersand.com`. - diff --git a/src/destinations/rabbitmq.mdx b/src/destinations/rabbitmq.mdx index 5173cef7..9c873bb4 100644 --- a/src/destinations/rabbitmq.mdx +++ b/src/destinations/rabbitmq.mdx @@ -4,14 +4,18 @@ title: RabbitMQ destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as messages to your RabbitMQ exchange. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as messages to your RabbitMQ instance. ## Prerequisites Before setting up a RabbitMQ destination, ensure that you have: - A RabbitMQ instance reachable from the internet - A connection URI with permission to publish -- A routing key for your exchange +- A routing key that reaches a queue you can consume from + +## Data format + +Ampersand publishes each read or subscribe message to your RabbitMQ instance as JSON, using the routing key you configure. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). ## Create a RabbitMQ destination @@ -19,15 +23,11 @@ Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/des You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Connection URI** | string | Yes | AMQP connection URI, e.g. `amqp://user:pass@host:5672` (it embeds your credentials) | -| **Routing key** | string | Yes | Routing key to publish messages with | - - -The connection URI embeds your username and password. Ampersand encrypts and stores it securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandRabbitMQ` | Alias to reference in your `amp.yaml` file | +| **Connection URI** | string | Yes | `amqp://user:pass@host:5672` | AMQP connection URI (it embeds your credentials) | +| **Routing key** | string | Yes | `ampersand-events` | Routing key to publish messages with | ## Refer to the destination in your integration @@ -46,11 +46,3 @@ integrations: - objectName: contact destination: ampersandRabbitMQ ``` - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your RabbitMQ exchange. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about scaling, contact `support@withampersand.com`. - diff --git a/src/destinations/redshift.mdx b/src/destinations/redshift.mdx index d6825287..e3d25d42 100644 --- a/src/destinations/redshift.mdx +++ b/src/destinations/redshift.mdx @@ -4,19 +4,20 @@ title: Amazon Redshift destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can insert the payload as rows into your Amazon Redshift table. Both Redshift Serverless (via a workgroup) and provisioned clusters (via a cluster identifier) are supported. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand inserts the payload as rows into your Amazon Redshift table. Both Redshift Serverless (via a workgroup) and provisioned clusters (via a cluster identifier) are supported. ## Prerequisites Before setting up a Redshift destination, ensure that you have: - An Amazon Redshift Serverless workgroup or a provisioned cluster -- A target database, and optionally a schema and table +- A target database, and a table to insert into (see [Data format and table setup](#data-format-and-table-setup)) - AWS credentials with permission to run the Redshift Data API and insert into the target table -## Set up your table +## Data format and table setup -The table must exist before you create the destination. The Redshift sink writes each message -into two columns: `created_at` and `payload` (the message JSON as a string): +Ampersand delivers each read or subscribe message as a JSON object. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + +The Redshift destination writes each message into two columns: `created_at` (the insert time) and `payload` (the full message JSON). The payload is written as a string, so `payload` must be a character column large enough to hold it. Create the table like this, then read individual fields from `payload` with Redshift's [JSON functions](https://docs.aws.amazon.com/redshift/latest/dg/json-functions.html): ```sql CREATE TABLE events ( @@ -25,34 +26,30 @@ CREATE TABLE events ( ); ``` -`VARCHAR(65535)` is Redshift's maximum column size; a message larger than that will disable the -sink. To split the payload into typed columns instead, use a -[Svix transformation](https://docs.svix.com/stream/sinks/redshift). - ## Create a Redshift destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Redshift destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Region** | string | Yes | AWS region where your Redshift resource lives (e.g. `us-west-2`) | -| **AWS Access Key ID** | string | Yes | AWS access key with Redshift Data API permissions | -| **AWS Secret Access Key** | string | Yes | AWS secret access key | -| **Database** | string | No | Target database name | -| **Schema** | string | No | Target schema | -| **Table name** | string | No | Target table | -| **Workgroup name** | string | No | Redshift Serverless workgroup name | -| **Cluster identifier** | string | No | Provisioned cluster identifier | -| **DB user** | string | No | Database user (for provisioned clusters) | -| **Batch size** | integer | No | Rows to buffer before flushing a batch (defaults to 1000) | -| **Max wait (seconds)** | integer | No | Max seconds to wait before flushing a batch (defaults to 30) | - - -Provide a **workgroup name** for Redshift Serverless, or a **cluster identifier** (and **DB user**) for a provisioned cluster — one of the two is required. Ampersand encrypts and stores your AWS credentials securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandRedshift` | Alias to reference in your `amp.yaml` file | +| **Region** | string | Yes | `us-west-2` | AWS region where your Redshift resource lives | +| **AWS Access Key ID** | string | Yes | `AKIAIOSFODNN7EXAMPLE` | AWS access key with Redshift Data API permissions | +| **AWS Secret Access Key** | string | Yes | | AWS secret access key | +| **Database** | string | No | `dev` | Target database name | +| **Schema** | string | No | `public` | Target schema | +| **Table name** | string | No | `events` | Target table | +| **Workgroup name** | string | No | `my-workgroup` | Redshift Serverless workgroup name | +| **Cluster identifier** | string | No | `my-cluster` | Provisioned cluster identifier | +| **DB user** | string | No | `ampersand` | Database user (for provisioned clusters) | +| **Batch size** | integer | No | `1000` | Rows to buffer before flushing a batch (defaults to 1000) | +| **Max wait (seconds)** | integer | No | `30` | Max seconds to wait before flushing a batch (defaults to 30) | + + +Provide a **workgroup name** for Redshift Serverless, or a **cluster identifier** (and **DB user**) for a provisioned cluster. One of the two is required. + ## Refer to the destination in your integration @@ -71,15 +68,3 @@ integrations: - objectName: contact destination: ampersandRedshift ``` - -## Batching - -Warehouse destinations buffer rows and flush a batch when it reaches **Batch size** rows or after **Max wait (seconds)**, whichever comes first (defaults: 1000 rows / 30 seconds). Lower the max wait for lower-latency delivery when volume is low. - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your Redshift table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about mapping payloads to your table schema, contact `support@withampersand.com`. - diff --git a/src/destinations/snowflake.mdx b/src/destinations/snowflake.mdx index efecde60..b93e48ce 100644 --- a/src/destinations/snowflake.mdx +++ b/src/destinations/snowflake.mdx @@ -4,19 +4,20 @@ title: Snowflake destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can insert the payload as rows into your Snowflake table. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand inserts the payload as rows into your Snowflake table. ## Prerequisites Before setting up a Snowflake destination, ensure that you have: - A Snowflake account - A user configured for [key-pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth) (Ampersand authenticates with an RSA private key, not a password) -- A target database, schema, and table, with `INSERT` privileges granted to that user +- A target database, schema, and table (see [Data format and table setup](#data-format-and-table-setup)), with `INSERT` privileges granted to that user -## Set up your table +## Data format and table setup -The table must exist before you create the destination. The Snowflake sink writes each -message into three columns: `id`, `created_at`, and `payload` (the message JSON as text): +Ampersand delivers each read or subscribe message as a JSON object. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + +The Snowflake destination writes each message into three columns: `id` (a generated identifier), `created_at` (the insert time), and `payload` (the full message JSON as text). Create the table with exactly these columns, then read individual fields from `payload` with Snowflake's [semi-structured functions](https://docs.snowflake.com/en/sql-reference/functions-semistructured) (for example `PARSE_JSON(payload):objectName`). ```sql CREATE TABLE my_db.my_schema.my_table ( @@ -26,30 +27,23 @@ CREATE TABLE my_db.my_schema.my_table ( ); ``` -To split the payload into typed columns instead, use a -[Svix transformation](https://docs.svix.com/stream/sinks/snowflake). - ## Create a Snowflake destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Snowflake destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Account identifier** | string | Yes | Your Snowflake account identifier (e.g. `myorg-myaccount`) | -| **User ID** | string | Yes | Snowflake user configured for key-pair auth | -| **Private key** | string | Yes | PEM-encoded RSA private key for that user | -| **Database** | string | No | Target database | -| **Schema** | string | No | Target schema | -| **Table name** | string | No | Target table | -| **Batch size** | integer | No | Rows to buffer before flushing a batch (defaults to 1000) | -| **Max wait (seconds)** | integer | No | Max seconds to wait before flushing a batch (defaults to 30) | - - -Ampersand encrypts and stores your private key securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandSnowflake` | Alias to reference in your `amp.yaml` file | +| **Account identifier** | string | Yes | `myorg-myaccount` | Your Snowflake account identifier | +| **User ID** | string | Yes | `SVC_AMPERSAND` | Snowflake user configured for key-pair auth | +| **Private key** | string | Yes | | PEM-encoded RSA private key for that user | +| **Database** | string | No | `ANALYTICS` | Target database | +| **Schema** | string | No | `PUBLIC` | Target schema | +| **Table name** | string | No | `ampersand_events` | Target table | +| **Batch size** | integer | No | `1000` | Rows to buffer before flushing a batch (defaults to 1000) | +| **Max wait (seconds)** | integer | No | `30` | Max seconds to wait before flushing a batch (defaults to 30) | ## Refer to the destination in your integration @@ -68,15 +62,3 @@ integrations: - objectName: contact destination: ampersandSnowflake ``` - -## Batching - -Warehouse destinations buffer rows and flush a batch when it reaches **Batch size** rows or after **Max wait (seconds)**, whichever comes first (defaults: 1000 rows / 30 seconds). Lower the max wait for lower-latency delivery when volume is low. - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your Snowflake table. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about mapping payloads to your table schema, contact `support@withampersand.com`. - diff --git a/src/destinations/sns.mdx b/src/destinations/sns.mdx index 4299f83f..e1f19851 100644 --- a/src/destinations/sns.mdx +++ b/src/destinations/sns.mdx @@ -4,7 +4,7 @@ title: Amazon SNS destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as messages to your Amazon SNS topic. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as messages to your Amazon SNS topic. ## Prerequisites @@ -12,24 +12,24 @@ Before setting up an SNS destination, ensure that you have: - An AWS account with an SNS topic - AWS credentials with `sns:Publish` permission on that topic +## Data format + +Ampersand publishes each read or subscribe message to your topic as JSON. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + ## Create an SNS destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new SNS destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Topic ARN** | string | Yes | ARN of your SNS topic | -| **Region** | string | Yes | AWS region where your topic is located (e.g. `us-west-2`) | -| **AWS Access Key ID** | string | Yes | AWS access key with `sns:Publish` permission | -| **AWS Secret Access Key** | string | Yes | AWS secret access key | -| **Endpoint URL** | string | No | Custom SNS endpoint if applicable (e.g. when using [LocalStack](https://www.localstack.cloud)) | - - -Ampersand encrypts and stores your AWS credentials securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandSNS` | Alias to reference in your `amp.yaml` file | +| **Topic ARN** | string | Yes | `arn:aws:sns:us-east-1:123456789012:my-topic` | ARN of your SNS topic | +| **Region** | string | Yes | `us-west-2` | AWS region where your topic is located | +| **AWS Access Key ID** | string | Yes | `AKIAIOSFODNN7EXAMPLE` | AWS access key with `sns:Publish` permission | +| **AWS Secret Access Key** | string | Yes | | AWS secret access key | +| **Endpoint URL** | string | No | `https://sns.us-east-1.amazonaws.com` | Custom SNS endpoint if applicable | ## Refer to the destination in your integration @@ -48,11 +48,3 @@ integrations: - objectName: contact destination: ampersandSNS ``` - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your SNS topic. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about scaling, contact `support@withampersand.com`. - diff --git a/src/destinations/sqs.mdx b/src/destinations/sqs.mdx index ef958b03..bf7598d0 100644 --- a/src/destinations/sqs.mdx +++ b/src/destinations/sqs.mdx @@ -4,7 +4,7 @@ title: Amazon SQS destinations For more information on destinations, see the [Destinations](/destinations) page. -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand can publish the payload as messages to your Amazon SQS queue. +When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand publishes the payload as messages to your Amazon SQS queue. ## Prerequisites @@ -12,24 +12,24 @@ Before setting up an SQS destination, ensure that you have: - An AWS account with an SQS queue - AWS credentials with `sqs:SendMessage` permission on that queue +## Data format + +Ampersand publishes each read or subscribe message to your queue as JSON. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). + ## Create an SQS destination Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new SQS destination. You'll need to provide: -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| **Destination name** | string | Yes | Alias to reference in your `amp.yaml` file | -| **Queue URL** | string | Yes | URL of your SQS queue | -| **Region** | string | Yes | AWS region where your queue is located (e.g. `us-west-2`) | -| **AWS Access Key ID** | string | Yes | AWS access key with `sqs:SendMessage` permission | -| **AWS Secret Access Key** | string | Yes | AWS secret access key | -| **Endpoint URL** | string | No | Custom SQS endpoint if applicable (e.g. when using [LocalStack](https://www.localstack.cloud)) | - - -Ampersand encrypts and stores your AWS credentials securely. - +| Field | Type | Required | Example | Description | +|-------|------|----------|---------|-------------| +| **Destination name** | string | Yes | `ampersandSQS` | Alias to reference in your `amp.yaml` file | +| **Queue URL** | string | Yes | `https://sqs.us-east-1.amazonaws.com/123456789012/my-queue` | URL of your SQS queue | +| **Region** | string | Yes | `us-west-2` | AWS region where your queue is located | +| **AWS Access Key ID** | string | Yes | `AKIAIOSFODNN7EXAMPLE` | AWS access key with `sqs:SendMessage` permission | +| **AWS Secret Access Key** | string | Yes | | AWS secret access key | +| **Endpoint URL** | string | No | `https://sqs.us-east-1.amazonaws.com` | Custom SQS endpoint if applicable | ## Refer to the destination in your integration @@ -48,11 +48,3 @@ integrations: - objectName: contact destination: ampersandSQS ``` - -## Message format - -Ampersand delivers the standard read and subscribe action payloads to your SQS queue. See [Message format](/destinations/s3#message-format) for the full schema and examples of read and subscribe messages. - - -If you have questions about scaling, contact `support@withampersand.com`. - diff --git a/src/docs.json b/src/docs.json index b3cd7624..21a81d47 100644 --- a/src/docs.json +++ b/src/docs.json @@ -57,20 +57,20 @@ "group": "Destinations", "pages": [ "destinations/overview", - "destinations/webhooks", - "destinations/kinesis", - "destinations/s3", - "destinations/clickhouse", - "destinations/snowflake", - "destinations/bigquery", + "destinations/eventbridge", "destinations/redshift", - "destinations/azure-blob", - "destinations/gcs", - "destinations/sqs", + "destinations/s3", "destinations/sns", - "destinations/eventbridge", + "destinations/sqs", + "destinations/azure-blob", + "destinations/bigquery", + "destinations/clickhouse", "destinations/pubsub", - "destinations/rabbitmq" + "destinations/gcs", + "destinations/kinesis", + "destinations/rabbitmq", + "destinations/snowflake", + "destinations/webhooks" ] }, { diff --git a/src/generate-docs.ts b/src/generate-docs.ts index a18a1f22..1df7b747 100644 --- a/src/generate-docs.ts +++ b/src/generate-docs.ts @@ -288,20 +288,20 @@ const baseConfig = { group: "Destinations", pages: [ "destinations/overview", - "destinations/webhooks", - "destinations/kinesis", - "destinations/s3", - "destinations/clickhouse", - "destinations/snowflake", - "destinations/bigquery", + "destinations/eventbridge", "destinations/redshift", - "destinations/azure-blob", - "destinations/gcs", - "destinations/sqs", + "destinations/s3", "destinations/sns", - "destinations/eventbridge", + "destinations/sqs", + "destinations/azure-blob", + "destinations/bigquery", + "destinations/clickhouse", "destinations/pubsub", + "destinations/gcs", + "destinations/kinesis", "destinations/rabbitmq", + "destinations/snowflake", + "destinations/webhooks", ] }, { From a9fc17245b8b156131536e8ccc17f85351451d9b Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 14:05:09 -0700 Subject: [PATCH 04/11] docs(destinations): drop usage advice + credential-storage notes (ENG-4130) - Remove "read fields with X's JSON functions" guidance from BigQuery, Snowflake, and Redshift; the pages describe only the table shape Ampersand writes to. - Remove the "Ampersand encrypts and stores your credentials securely" notes from the S3 and Kinesis pages. Co-Authored-By: Claude Opus 4.8 --- src/destinations/bigquery.mdx | 2 +- src/destinations/kinesis.mdx | 4 ---- src/destinations/redshift.mdx | 2 +- src/destinations/s3.mdx | 4 ---- src/destinations/snowflake.mdx | 2 +- 5 files changed, 3 insertions(+), 11 deletions(-) diff --git a/src/destinations/bigquery.mdx b/src/destinations/bigquery.mdx index 7a204f2e..198312e2 100644 --- a/src/destinations/bigquery.mdx +++ b/src/destinations/bigquery.mdx @@ -17,7 +17,7 @@ Before setting up a BigQuery destination, ensure that you have: Ampersand delivers each read or subscribe message as a JSON object. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). -The BigQuery destination writes each message into two columns: `id` (a generated identifier) and `payload` (the full message JSON as a string). Create the table with exactly these columns, then read individual fields from `payload` with BigQuery's [JSON functions](https://cloud.google.com/bigquery/docs/reference/standard-sql/json_functions). +The BigQuery destination writes each message into two columns: `id` (a generated identifier) and `payload` (the full message JSON, written as a string). Create the table with exactly these columns: ```sql CREATE TABLE `my-project.my_dataset.my_table` ( diff --git a/src/destinations/kinesis.mdx b/src/destinations/kinesis.mdx index 8cbcd64d..09280a74 100644 --- a/src/destinations/kinesis.mdx +++ b/src/destinations/kinesis.mdx @@ -83,10 +83,6 @@ You'll need to provide: | **Partition key template** | string | No | [JMESPath](https://jmespath.org) template for partition key | | **Endpoint URL** | string | No | Custom Kinesis endpoint if applicable (e.g. when using [LocalStack](https://www.localstack.cloud)) | - -Ampersand encrypts and stores your AWS credentials securely. - - ## Refer to the destination in your integration After creating your Kinesis destination, reference it in your `amp.yaml` file: diff --git a/src/destinations/redshift.mdx b/src/destinations/redshift.mdx index e3d25d42..0bcfdf51 100644 --- a/src/destinations/redshift.mdx +++ b/src/destinations/redshift.mdx @@ -17,7 +17,7 @@ Before setting up a Redshift destination, ensure that you have: Ampersand delivers each read or subscribe message as a JSON object. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). -The Redshift destination writes each message into two columns: `created_at` (the insert time) and `payload` (the full message JSON). The payload is written as a string, so `payload` must be a character column large enough to hold it. Create the table like this, then read individual fields from `payload` with Redshift's [JSON functions](https://docs.aws.amazon.com/redshift/latest/dg/json-functions.html): +The Redshift destination writes each message into two columns: `created_at` (the insert time) and `payload` (the full message JSON). The payload is written as a string, so `payload` must be a character column large enough to hold it. Create the table like this: ```sql CREATE TABLE events ( diff --git a/src/destinations/s3.mdx b/src/destinations/s3.mdx index 41a9288c..c1d8f40c 100644 --- a/src/destinations/s3.mdx +++ b/src/destinations/s3.mdx @@ -67,10 +67,6 @@ You'll need to provide: | **Object key template** | string | No | [JMESPath](https://jmespath.org) template for object key naming | | **Storage class** | string | No | S3 storage class for written objects (defaults to `STANDARD`) | - -Ampersand encrypts and stores your AWS credentials securely. - - ## Refer to the destination in your integration After creating your S3 destination, reference it in your `amp.yaml` file: diff --git a/src/destinations/snowflake.mdx b/src/destinations/snowflake.mdx index b93e48ce..9f62aef7 100644 --- a/src/destinations/snowflake.mdx +++ b/src/destinations/snowflake.mdx @@ -17,7 +17,7 @@ Before setting up a Snowflake destination, ensure that you have: Ampersand delivers each read or subscribe message as a JSON object. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). -The Snowflake destination writes each message into three columns: `id` (a generated identifier), `created_at` (the insert time), and `payload` (the full message JSON as text). Create the table with exactly these columns, then read individual fields from `payload` with Snowflake's [semi-structured functions](https://docs.snowflake.com/en/sql-reference/functions-semistructured) (for example `PARSE_JSON(payload):objectName`). +The Snowflake destination writes each message into three columns: `id` (a generated identifier), `created_at` (the insert time), and `payload` (the full message JSON, written as text). Create the table with exactly these columns: ```sql CREATE TABLE my_db.my_schema.my_table ( From a85574453cd03721ba1b299e89d6d089859c08fc Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 14:29:42 -0700 Subject: [PATCH 05/11] docs(destinations): group the sidebar by delivery kind (ENG-4130) Nest the Destinations sidebar into Queues (EventBridge, Kinesis, Pub/Sub, RabbitMQ, SNS, SQS), Databases and warehouses (BigQuery, ClickHouse, Redshift, Snowflake), and Blob storage (Azure Blob, GCS, S3). Overview and Webhooks stay top-level. Co-Authored-By: Claude Opus 4.8 --- src/docs.json | 43 +++++++++++++++++++++++++++++-------------- src/generate-docs.ts | 41 ++++++++++++++++++++++++++++------------- 2 files changed, 57 insertions(+), 27 deletions(-) diff --git a/src/docs.json b/src/docs.json index 21a81d47..c40d95e7 100644 --- a/src/docs.json +++ b/src/docs.json @@ -57,20 +57,35 @@ "group": "Destinations", "pages": [ "destinations/overview", - "destinations/eventbridge", - "destinations/redshift", - "destinations/s3", - "destinations/sns", - "destinations/sqs", - "destinations/azure-blob", - "destinations/bigquery", - "destinations/clickhouse", - "destinations/pubsub", - "destinations/gcs", - "destinations/kinesis", - "destinations/rabbitmq", - "destinations/snowflake", - "destinations/webhooks" + { + "group": "Queues", + "pages": [ + "destinations/eventbridge", + "destinations/kinesis", + "destinations/pubsub", + "destinations/rabbitmq", + "destinations/sns", + "destinations/sqs" + ] + }, + { + "group": "Databases and warehouses", + "pages": [ + "destinations/bigquery", + "destinations/clickhouse", + "destinations/redshift", + "destinations/snowflake" + ] + }, + "destinations/webhooks", + { + "group": "Blob storage", + "pages": [ + "destinations/azure-blob", + "destinations/gcs", + "destinations/s3" + ] + } ] }, { diff --git a/src/generate-docs.ts b/src/generate-docs.ts index 1df7b747..deffee77 100644 --- a/src/generate-docs.ts +++ b/src/generate-docs.ts @@ -288,20 +288,35 @@ const baseConfig = { group: "Destinations", pages: [ "destinations/overview", - "destinations/eventbridge", - "destinations/redshift", - "destinations/s3", - "destinations/sns", - "destinations/sqs", - "destinations/azure-blob", - "destinations/bigquery", - "destinations/clickhouse", - "destinations/pubsub", - "destinations/gcs", - "destinations/kinesis", - "destinations/rabbitmq", - "destinations/snowflake", + { + group: "Queues", + pages: [ + "destinations/eventbridge", + "destinations/kinesis", + "destinations/pubsub", + "destinations/rabbitmq", + "destinations/sns", + "destinations/sqs", + ] + }, + { + group: "Databases and warehouses", + pages: [ + "destinations/bigquery", + "destinations/clickhouse", + "destinations/redshift", + "destinations/snowflake", + ] + }, "destinations/webhooks", + { + group: "Blob storage", + pages: [ + "destinations/azure-blob", + "destinations/gcs", + "destinations/s3", + ] + }, ] }, { From 22031bd07d2b68a5c5134464fdd2f84b75e64dfd Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 15:41:58 -0700 Subject: [PATCH 06/11] docs(destinations): withhold Azure Blob + GCS pages (ENG-4200) Blob-storage sinks write objects with an uncontrolled `object-generated-by-svix` prefix, so hold them back until we own the naming. Remove the azure-blob and gcs pages and their sidebar entries; S3 stays under "Blob storage". See Linear ENG-4200. Co-Authored-By: Claude Opus 4.8 --- src/destinations/azure-blob.mdx | 49 --------------------------------- src/destinations/gcs.mdx | 47 ------------------------------- src/docs.json | 2 -- src/generate-docs.ts | 2 -- 4 files changed, 100 deletions(-) delete mode 100644 src/destinations/azure-blob.mdx delete mode 100644 src/destinations/gcs.mdx diff --git a/src/destinations/azure-blob.mdx b/src/destinations/azure-blob.mdx deleted file mode 100644 index fb251e6a..00000000 --- a/src/destinations/azure-blob.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: Azure Blob Storage destinations ---- - -For more information on destinations, see the [Destinations](/destinations) page. - -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand writes the payload as objects to your Azure Blob Storage container. - -## Prerequisites - -Before setting up an Azure Blob Storage destination, ensure that you have: -- An Azure Storage account -- A blob container to write to -- The storage account's access key - -## Data format - -Ampersand writes each read or subscribe message as a JSON object to your container. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). - -## Create an Azure Blob Storage destination - -Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Azure Blob Storage destination. - -You'll need to provide: - -| Field | Type | Required | Example | Description | -|-------|------|----------|---------|-------------| -| **Destination name** | string | Yes | `ampersandAzureBlob` | Alias to reference in your `amp.yaml` file | -| **Account** | string | Yes | `mystorageaccount` | Azure Storage account name | -| **Container** | string | Yes | `ampersand-data` | Blob container to write objects to | -| **Access key** | string | Yes | | Storage account access key | - -## Refer to the destination in your integration - -After creating your Azure Blob Storage destination, reference it in your `amp.yaml` file: - -```yaml -specVersion: 1.0.0 -integrations: - - name: salesforceToAzureBlob - displayName: Salesforce to Azure Blob Storage - provider: salesforce - read: - objects: - - objectName: account - destination: ampersandAzureBlob - - objectName: contact - destination: ampersandAzureBlob -``` diff --git a/src/destinations/gcs.mdx b/src/destinations/gcs.mdx deleted file mode 100644 index 9c4566c5..00000000 --- a/src/destinations/gcs.mdx +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: Google Cloud Storage destinations ---- - -For more information on destinations, see the [Destinations](/destinations) page. - -When new data is read from a SaaS instance via a [Read Action](/read-actions) or [Subscribe Action](/subscribe-actions), Ampersand writes the payload as objects to your Google Cloud Storage bucket. - -## Prerequisites - -Before setting up a Google Cloud Storage destination, ensure that you have: -- A Google Cloud project with a Cloud Storage bucket -- A service account with the `Storage Object Creator` role (or `storage.objects.create` permission) on that bucket, and a JSON key for it - -## Data format - -Ampersand writes each read or subscribe message as a JSON object to your bucket. The full payload schema is defined in [webhook.yaml](https://github.com/amp-labs/openapi/blob/main/webhook/webhook.yaml). - -## Create a Google Cloud Storage destination - -Go to the [Destinations page](https://dashboard.withampersand.com/projects/_/destinations) in the Ampersand Dashboard and create a new Google Cloud Storage destination. - -You'll need to provide: - -| Field | Type | Required | Example | Description | -|-------|------|----------|---------|-------------| -| **Destination name** | string | Yes | `ampersandGCS` | Alias to reference in your `amp.yaml` file | -| **Bucket** | string | Yes | `ampersand-integration-bucket` | Name of your Cloud Storage bucket | -| **Service account credentials** | string | Yes | | Service account key JSON with write access to the bucket | - -## Refer to the destination in your integration - -After creating your Google Cloud Storage destination, reference it in your `amp.yaml` file: - -```yaml -specVersion: 1.0.0 -integrations: - - name: salesforceToGCS - displayName: Salesforce to Google Cloud Storage - provider: salesforce - read: - objects: - - objectName: account - destination: ampersandGCS - - objectName: contact - destination: ampersandGCS -``` diff --git a/src/docs.json b/src/docs.json index c40d95e7..6dd8a89a 100644 --- a/src/docs.json +++ b/src/docs.json @@ -81,8 +81,6 @@ { "group": "Blob storage", "pages": [ - "destinations/azure-blob", - "destinations/gcs", "destinations/s3" ] } diff --git a/src/generate-docs.ts b/src/generate-docs.ts index deffee77..9ff26299 100644 --- a/src/generate-docs.ts +++ b/src/generate-docs.ts @@ -312,8 +312,6 @@ const baseConfig = { { group: "Blob storage", pages: [ - "destinations/azure-blob", - "destinations/gcs", "destinations/s3", ] }, From 829431bf473c0bfb652c8d5c01d1b7b40007920e Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 15:52:41 -0700 Subject: [PATCH 07/11] docs(destinations): drop overview links to withheld blob pages (ENG-4200) Remove the Object storage section (Azure Blob + GCS) and the prose mentions from the destinations overview so the broken-links check passes. See Linear ENG-4200. Co-Authored-By: Claude Opus 4.8 --- src/destinations/overview.mdx | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/src/destinations/overview.mdx b/src/destinations/overview.mdx index 4265e6a4..1124fbcb 100644 --- a/src/destinations/overview.mdx +++ b/src/destinations/overview.mdx @@ -2,7 +2,7 @@ title: Overview --- -Ampersand supports webhook, Kinesis, and S3 destinations, along with a range of data warehouse, object storage, and message queue destinations (ClickHouse, Snowflake, BigQuery, Redshift, Azure Blob Storage, Google Cloud Storage, Amazon SQS, Amazon SNS, Amazon EventBridge, Google Cloud Pub/Sub, and RabbitMQ). Destinations allow you to route data synced from SaaS instances via [Read Actions](/read-actions) or [Subscribe Actions](/subscribe-actions), and to receive real-time [Notifications](/notifications) about important lifecycle events in your projects. +Ampersand supports webhook, Kinesis, and S3 destinations, along with a range of data warehouse and message queue destinations (ClickHouse, Snowflake, BigQuery, Redshift, Amazon SQS, Amazon SNS, Amazon EventBridge, Google Cloud Pub/Sub, and RabbitMQ). Destinations allow you to route data synced from SaaS instances via [Read Actions](/read-actions) or [Subscribe Actions](/subscribe-actions), and to receive real-time [Notifications](/notifications) about important lifecycle events in your projects. ## Add a destination to the Ampersand Dashboard @@ -45,11 +45,6 @@ Destinations can also be used to receive [notifications](/notifications) about i * [BigQuery destinations](/destinations/bigquery) * [Amazon Redshift destinations](/destinations/redshift) -### Object storage - -* [Azure Blob Storage destinations](/destinations/azure-blob) -* [Google Cloud Storage destinations](/destinations/gcs) - ### Message queues and event buses * [Amazon SQS destinations](/destinations/sqs) From fe63ae40da27149d145facb658291091042ff456 Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 16:14:06 -0700 Subject: [PATCH 08/11] docs(destinations): regen sidebar; fix kinesis dupe + redshift one-of note - Remove the duplicate destinations/kinesis from the Queues group (it's a top-level destination, matching the overview); regenerate docs.json. - Redshift: mark Workgroup name / Cluster identifier as No* and clarify the note that exactly one is required (addresses Graphite review comment). Co-Authored-By: Claude Opus 4.8 --- src/destinations/redshift.mdx | 6 +++--- src/docs.json | 1 - src/generate-docs.ts | 1 - 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/src/destinations/redshift.mdx b/src/destinations/redshift.mdx index 0bcfdf51..095a1e02 100644 --- a/src/destinations/redshift.mdx +++ b/src/destinations/redshift.mdx @@ -41,14 +41,14 @@ You'll need to provide: | **Database** | string | No | `dev` | Target database name | | **Schema** | string | No | `public` | Target schema | | **Table name** | string | No | `events` | Target table | -| **Workgroup name** | string | No | `my-workgroup` | Redshift Serverless workgroup name | -| **Cluster identifier** | string | No | `my-cluster` | Provisioned cluster identifier | +| **Workgroup name** | string | No* | `my-workgroup` | Redshift Serverless workgroup name | +| **Cluster identifier** | string | No* | `my-cluster` | Provisioned cluster identifier | | **DB user** | string | No | `ampersand` | Database user (for provisioned clusters) | | **Batch size** | integer | No | `1000` | Rows to buffer before flushing a batch (defaults to 1000) | | **Max wait (seconds)** | integer | No | `30` | Max seconds to wait before flushing a batch (defaults to 30) | -Provide a **workgroup name** for Redshift Serverless, or a **cluster identifier** (and **DB user**) for a provisioned cluster. One of the two is required. +*Either **Workgroup name** or **Cluster identifier** is required. Provide a **workgroup name** for Redshift Serverless, or a **cluster identifier** (and **DB user**) for a provisioned cluster. ## Refer to the destination in your integration diff --git a/src/docs.json b/src/docs.json index 59346f29..0b9b7ca5 100644 --- a/src/docs.json +++ b/src/docs.json @@ -61,7 +61,6 @@ "group": "Queues", "pages": [ "destinations/eventbridge", - "destinations/kinesis", "destinations/pubsub", "destinations/rabbitmq", "destinations/sns", diff --git a/src/generate-docs.ts b/src/generate-docs.ts index 2548679b..3c955f0f 100644 --- a/src/generate-docs.ts +++ b/src/generate-docs.ts @@ -292,7 +292,6 @@ const baseConfig = { group: "Queues", pages: [ "destinations/eventbridge", - "destinations/kinesis", "destinations/pubsub", "destinations/rabbitmq", "destinations/sns", From 641f4d1db5e0b170a9baf3252eef2206ae64a984 Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 16:15:09 -0700 Subject: [PATCH 09/11] docs(destinations): order sidebar overview, webhooks, slack, then categories Top-level overview/webhooks/slack, followed by the categories alphabetically (Blob storage, Databases and warehouses, Queues). Kinesis moves into Queues. Co-Authored-By: Claude Opus 4.8 --- src/docs.json | 22 +++++++++++----------- src/generate-docs.ts | 22 +++++++++++----------- 2 files changed, 22 insertions(+), 22 deletions(-) diff --git a/src/docs.json b/src/docs.json index 0b9b7ca5..6e5af677 100644 --- a/src/docs.json +++ b/src/docs.json @@ -57,14 +57,12 @@ "group": "Destinations", "pages": [ "destinations/overview", + "destinations/webhooks", + "destinations/slack", { - "group": "Queues", + "group": "Blob storage", "pages": [ - "destinations/eventbridge", - "destinations/pubsub", - "destinations/rabbitmq", - "destinations/sns", - "destinations/sqs" + "destinations/s3" ] }, { @@ -76,13 +74,15 @@ "destinations/snowflake" ] }, - "destinations/webhooks", - "destinations/slack", - "destinations/kinesis", { - "group": "Blob storage", + "group": "Queues", "pages": [ - "destinations/s3" + "destinations/eventbridge", + "destinations/kinesis", + "destinations/pubsub", + "destinations/rabbitmq", + "destinations/sns", + "destinations/sqs" ] } ] diff --git a/src/generate-docs.ts b/src/generate-docs.ts index 3c955f0f..40a53080 100644 --- a/src/generate-docs.ts +++ b/src/generate-docs.ts @@ -288,14 +288,12 @@ const baseConfig = { group: "Destinations", pages: [ "destinations/overview", + "destinations/webhooks", + "destinations/slack", { - group: "Queues", + group: "Blob storage", pages: [ - "destinations/eventbridge", - "destinations/pubsub", - "destinations/rabbitmq", - "destinations/sns", - "destinations/sqs", + "destinations/s3", ] }, { @@ -307,13 +305,15 @@ const baseConfig = { "destinations/snowflake", ] }, - "destinations/webhooks", - "destinations/slack", - "destinations/kinesis", { - group: "Blob storage", + group: "Queues", pages: [ - "destinations/s3", + "destinations/eventbridge", + "destinations/kinesis", + "destinations/pubsub", + "destinations/rabbitmq", + "destinations/sns", + "destinations/sqs", ] } ] From b5768a0b0080893fff62269a1c4d9111aec47a29 Mon Sep 17 00:00:00 2001 From: RajatPawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 16:17:29 -0700 Subject: [PATCH 10/11] docs(destinations): align overview with the sidebar Keep only Webhook and Slack at the top level; move Amazon S3 under a Blob storage heading and Amazon Kinesis into Message queues and event buses, matching the sidebar structure. Co-Authored-By: Claude Opus 4.8 --- src/destinations/overview.mdx | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/destinations/overview.mdx b/src/destinations/overview.mdx index 9e0fd0a3..b944640d 100644 --- a/src/destinations/overview.mdx +++ b/src/destinations/overview.mdx @@ -36,7 +36,9 @@ Destinations can also be used to receive [notifications](/notifications) about i * [Webhook destinations](/destinations/webhooks) * [Slack destinations](/destinations/slack) -* [Amazon Kinesis destinations](/destinations/kinesis) + +### Blob storage + * [Amazon S3 destinations](/destinations/s3) ### Data warehouses @@ -48,6 +50,7 @@ Destinations can also be used to receive [notifications](/notifications) about i ### Message queues and event buses +* [Amazon Kinesis destinations](/destinations/kinesis) * [Amazon SQS destinations](/destinations/sqs) * [Amazon SNS destinations](/destinations/sns) * [Amazon EventBridge destinations](/destinations/eventbridge) From 62a8767c2caec1ff6c5cc7f97d3be1dce9dce626 Mon Sep 17 00:00:00 2001 From: Rajat Pawar <18614743+RajatPawar@users.noreply.github.com> Date: Tue, 18 Aug 2026 16:26:52 -0700 Subject: [PATCH 11/11] Update overview.mdx --- src/destinations/overview.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/destinations/overview.mdx b/src/destinations/overview.mdx index b944640d..208f472d 100644 --- a/src/destinations/overview.mdx +++ b/src/destinations/overview.mdx @@ -41,7 +41,7 @@ Destinations can also be used to receive [notifications](/notifications) about i * [Amazon S3 destinations](/destinations/s3) -### Data warehouses +### Databases and warehouses * [ClickHouse destinations](/destinations/clickhouse) * [Snowflake destinations](/destinations/snowflake)