Skip to content

feat(destinations): descope Kafka from Outpost queue destinations (ENG-4130) #400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
RajatPawar merged 1 commit into from
Aug 20, 2026
Merged

feat(destinations): descope Kafka from Outpost queue destinations (ENG-4130) #400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 16 additions & 80 deletions api/api.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3019,10 +3019,10 @@ paths:
For `slack`, set `metadata.url` to a Slack incoming webhook URL.
The warehouse and message-queue types (`clickhouse`,
`snowflake`, `bigquery`, `redshift`, `sqs`,
`pubsub`, `rabbitmq`, `azureservicebus`, `kafka`) are configured via the `metadata`
`pubsub`, `rabbitmq`, `azureservicebus`) are configured via the `metadata`
and `secrets` fields documented below.
example: webhook
enum: [webhook, kinesis, s3, slack, clickhouse, snowflake, bigquery, redshift, sqs, pubsub, rabbitmq, azureservicebus, kafka]
enum: [webhook, kinesis, s3, slack, clickhouse, snowflake, bigquery, redshift, sqs, pubsub, rabbitmq, azureservicebus]
secrets:
type: object
description: Secrets for the destination.
Expand All @@ -3047,8 +3047,8 @@ paths:
x-go-type-skip-optional-pointer: true
password:
type: string
description: The password for the `clickhouse`, `rabbitmq`, and `kafka`
destinations. Required for `rabbitmq` and `kafka`.
description: The password for the `clickhouse` and `rabbitmq`
destinations. Required for `rabbitmq`.
example: my-clickhouse-password
x-go-type-skip-optional-pointer: true
privateKey:
Expand Down Expand Up @@ -3079,8 +3079,8 @@ paths:
x-go-type-skip-optional-pointer: true
username:
type: string
description: The username for the `rabbitmq` and `kafka` destinations.
Required for both.
description: The username for the `rabbitmq` destination.
Required for `rabbitmq`.
example: ampersand
x-go-type-skip-optional-pointer: true
connectionString:
Expand Down Expand Up @@ -3118,7 +3118,7 @@ paths:
partitionKeyTemplate:
type: string
description: The template for the partition key (a JMESPath template).
Used by `kinesis`, and optionally by `kafka`.
Used by `kinesis`.
example: "data.user_id"
x-go-type-skip-optional-pointer: true
bucket:
Expand Down Expand Up @@ -3239,7 +3239,7 @@ paths:
tls:
type: boolean
description: Whether to connect over TLS. Optional for the `rabbitmq`
and `kafka` destinations.
destination.
example: true
x-go-type-skip-optional-pointer: true
name:
Expand All @@ -3248,25 +3248,6 @@ paths:
Required for `azureservicebus`.
example: ampersand-events
x-go-type-skip-optional-pointer: true
brokers:
type: string
description: The comma-separated list of Kafka broker addresses for the
`kafka` destination. Required for `kafka`.
example: broker1.example.com:9092,broker2.example.com:9092
x-go-type-skip-optional-pointer: true
topic:
type: string
description: The Kafka topic to publish to for the `kafka` destination.
Required for `kafka`.
example: ampersand-events
x-go-type-skip-optional-pointer: true
saslMechanism:
type: string
description: The SASL mechanism used to authenticate to Kafka for the
`kafka` destination. Required for `kafka`.
enum: [plain, scram-sha-256, scram-sha-512]
example: scram-sha-256
x-go-type-skip-optional-pointer: true
batchSize:
type: integer
description: For warehouse destinations (`clickhouse`, `snowflake`,
Expand Down Expand Up @@ -3452,9 +3433,6 @@ paths:
- metadata.exchange
- metadata.tls
- metadata.name
- metadata.brokers
- metadata.topic
- metadata.saslMechanism
- metadata.batchSize
- metadata.maxWaitSecs
- secrets.awsKeyId
Expand Down Expand Up @@ -3502,7 +3480,7 @@ paths:
partitionKeyTemplate:
type: string
description: The template for the partition key (a JMESPath template).
Used by `kinesis`, and optionally by `kafka`.
Used by `kinesis`.
example: "data.user_id"
bucket:
type: string
Expand Down Expand Up @@ -3600,29 +3578,13 @@ paths:
tls:
type: boolean
description: Whether to connect over TLS. Optional for the `rabbitmq`
and `kafka` destinations.
destination.
example: true
name:
type: string
description: The queue or topic name for the `azureservicebus` destination.
Required for `azureservicebus`.
example: ampersand-events
brokers:
type: string
description: The comma-separated list of Kafka broker addresses for the
`kafka` destination. Required for `kafka`.
example: broker1.example.com:9092,broker2.example.com:9092
topic:
type: string
description: The Kafka topic to publish to for the `kafka` destination.
Required for `kafka`.
example: ampersand-events
saslMechanism:
type: string
description: The SASL mechanism used to authenticate to Kafka for the
`kafka` destination. Required for `kafka`.
enum: [plain, scram-sha-256, scram-sha-512]
example: scram-sha-256
batchSize:
type: integer
description: For warehouse destinations, rows to buffer before flushing a batch (default 1000).
Expand Down Expand Up @@ -3652,8 +3614,8 @@ paths:
example: AQoEXAMPLEH4aoAH0gNCAPy...
password:
type: string
description: The password for the `clickhouse`, `rabbitmq`, and `kafka`
destinations. Required for `rabbitmq` and `kafka`.
description: The password for the `clickhouse` and `rabbitmq`
destinations. Required for `rabbitmq`.
example: my-clickhouse-password
privateKey:
type: string
Expand All @@ -3677,8 +3639,8 @@ paths:
example: my-azure-access-key
username:
type: string
description: The username for the `rabbitmq` and `kafka` destinations.
Required for both.
description: The username for the `rabbitmq` destination.
Required for `rabbitmq`.
example: ampersand
connectionString:
type: string
Expand Down Expand Up @@ -5504,25 +5466,12 @@ components:
example: ampersand.events
tls:
type: boolean
description: Whether to connect over TLS (rabbitmq, kafka)
description: Whether to connect over TLS (rabbitmq)
example: true
name:
type: string
description: The queue or topic name for the azureservicebus destination
example: ampersand-events
brokers:
type: string
description: The comma-separated list of Kafka broker addresses for the kafka destination
example: broker1.example.com:9092,broker2.example.com:9092
topic:
type: string
description: The Kafka topic to publish to for the kafka destination
example: ampersand-events
saslMechanism:
type: string
description: The SASL mechanism used to authenticate to Kafka for the kafka destination
enum: [plain, scram-sha-256, scram-sha-512]
example: scram-sha-256
batchSize:
type: integer
description: For warehouse destinations, rows to buffer before flushing a batch (default 1000)
Expand Down Expand Up @@ -5673,25 +5622,12 @@ components:
example: ampersand.events
tls:
type: boolean
description: Whether to connect over TLS (rabbitmq, kafka)
description: Whether to connect over TLS (rabbitmq)
example: true
name:
type: string
description: The queue or topic name for the azureservicebus destination
example: ampersand-events
brokers:
type: string
description: The comma-separated list of Kafka broker addresses for the kafka destination
example: broker1.example.com:9092,broker2.example.com:9092
topic:
type: string
description: The Kafka topic to publish to for the kafka destination
example: ampersand-events
saslMechanism:
type: string
description: The SASL mechanism used to authenticate to Kafka for the kafka destination
enum: [plain, scram-sha-256, scram-sha-512]
example: scram-sha-256
batchSize:
type: integer
description: For warehouse destinations, rows to buffer before flushing a batch (default 1000)
Expand Down
Loading
Loading