Skip to content

Migrate from AWS Java SDK v1 to v2 and KPL 1.x#3

Open
a-bates wants to merge 2 commits into
masterfrom
aws-sdk-v2-migration
Open

Migrate from AWS Java SDK v1 to v2 and KPL 1.x#3
a-bates wants to merge 2 commits into
masterfrom
aws-sdk-v2-migration

Conversation

@a-bates

@a-bates a-bates commented Jun 9, 2026

Copy link
Copy Markdown

Summary

  • AWS Java SDK 1.x reached end of support on Dec 31, 2025; AWS Enterprise Support is paging the affected Mavenlink/Kantata accounts every ~16 days until usage stops. This PR removes Maxwell's usage of the com.amazonaws.* namespace.
  • SNS and SQS producers ported to SDK v2 (SnsAsyncClient / SqsAsyncClient) with CompletableFuture.whenComplete() replacing the v1 AsyncHandler callbacks. FIFO messageGroupId, message attributes, and ignoreProducerError semantics preserved.
  • Kinesis producer moves from com.amazonaws:amazon-kinesis-producer:0.15.12 to software.amazon.kinesis:amazon-kinesis-producer:1.0.7 — KPL 1.x is internally on SDK v2 and the public Java API is source-compatible; the change is an import rename.
  • FilterParser's stray com.amazonaws.util.StringInputStream replaced with ByteArrayInputStream (UTF-8).
  • pom.xml: replaced aws-java-sdk-{core,sns,sqs,sts}@1.12.782 with the v2 BOM at 2.34.0 (sns/sqs/sts), removed a duplicate aws-java-sdk-core entry.

Behavior, CLI flags, config properties, and credential chain are unchanged. SDK v2's DefaultCredentialsProvider covers the same sources (env, sys-props, IRSA via STS, profile, IMDS) as v1's chain.

Test plan

  • grep -r com.amazonaws src/ pom.xml returns nothing
  • mvn dependency:tree | grep com.amazonaws returns nothing (zero v1 artifacts remain transitively)
  • mvn -DskipTests package builds cleanly
  • mvn -Dtest=MaxwellSNSProducerTest,KinesisCallbackTest test — 5/5 pass
  • mvn -Dtest=FilterTest test — 16/16 pass (exercises the FilterParser change)
  • End-to-end LocalStack smoke: MySQL 8 (ROW binlog + GTID) → Maxwell --producer=sqs → LocalStack SQS. Insert/update/delete on a test table; all 4 events arrive in the queue with correct payloads (including old values on update). SDK v2 builder + endpoint override + credential chain code path verified end-to-end against a real SQS-compatible endpoint.
  • Verify SNS end-to-end against a real or LocalStack-Pro SNS topic (skipped — SQS smoke exercises the same SDK v2 builder/endpoint/credential code path)
  • Verify Kinesis end-to-end against a real Kinesis stream (skipped — KPL change is an import-only rename and the unit test covers callback behavior)

🤖 Generated with Claude Code

a-bates and others added 2 commits June 9, 2026 14:34
AWS Java SDK 1.x reached end of support on Dec 31, 2025. Port the SNS,
SQS, and Kinesis producers off the com.amazonaws namespace to keep
receiving security and bug fixes. SNS/SQS move to SnsAsyncClient /
SqsAsyncClient with CompletableFuture.whenComplete() in place of the v1
AsyncHandler callbacks; KPL moves to software.amazon.kinesis 1.0.7
(internals already on SDK v2) which is just an import rename. The
FilterParser's stray com.amazonaws.util.StringInputStream is replaced
with ByteArrayInputStream. Behavior, config flags, and credential chain
are preserved.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
KPL 1.x's generated protobuf code requires protobuf-java 4.x's
RuntimeVersion class. Without this pin, BigQuery's transitive
protobuf-java 3.25.5 wins and KinesisProducerConfiguration fails
to load. protobuf-java 4.x is binary-compatible with 3.x generated
code, so the BigQuery / PubSub paths still work (verified via their
unit tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant