feat: gdb failover support#1246
Merged
Merged
Conversation
karenc-bq
force-pushed
the
feat/gdb-failover
branch
from
June 8, 2026 19:53
410a5e4 to
052135f
Compare
karenc-bq
marked this pull request as ready for review
June 8, 2026 19:53
sophia-bq
approved these changes
Jun 9, 2026
karenc-bq
force-pushed
the
feat/gdb-failover
branch
4 times, most recently
from
June 16, 2026 17:29
32dde0e to
69011e9
Compare
karenc-bq
force-pushed
the
feat/gdb-failover
branch
from
June 16, 2026 23:35
69011e9 to
88363d9
Compare
AhmadMasry
added a commit
to AhmadMasry/aws-advanced-python-wrapper
that referenced
this pull request
Jun 17, 2026
Brings in gdb failover support (aws#1246). Conflict resolution: - failover_v2_plugin.py (_failover_writer): took main's RetryUtil-based refactor, which replaces the inline writer-candidate logic our branch had only lightly tweaked. - test_aurora_failover.py (test_fail_from_reader_to_writer matrix): kept our branch's deliberate drop of host_monitoring (v1) entries (libpq teardown SIGSEGV flake fix) while adding main's new gdb_failover variant in its v2-only form, per the same rationale. - retry_util.py (RetryUtil.close_connection): added conn=conn to the CONNECTION_CLOSE execute() call to match our branch's convention that prevents the cross-thread use-after-free race on close-during-failover.
AhmadMasry
added a commit
to AhmadMasry/aws-advanced-python-wrapper
that referenced
this pull request
Jun 17, 2026
The merge of main brought in the sync GDB (Aurora Global Database) failover plugin (GdbFailoverPlugin + RetryUtil) with no async equivalent. This adds that parity: - aio/retry_util.py: AsyncRetryUtil, the async port of the sync RetryUtil deadline-bounded connect-and-verify-role retry loop. Adapted to async: force_connect (not connect) so auth plugins re-apply and the pooled provider is bypassed; abort_connection for close; each connect await bounded by the remaining deadline so a blackholed host can't hang failover. - aio/gdb_failover_plugin.py: AsyncGdbFailoverPlugin(AsyncFailoverPlugin) overriding _do_failover with region-aware, GdbFailoverMode-driven target selection mirroring the sync plugin's _failover_with_mode. - aio/failover_plugin.py: extract _is_strict_writer_failover_mode (behavior-preserving) so the GDB subclass can make the read-only escape hatch region-aware. - aio/plugin_factory.py: register "gdb_failover" (weight 420). Reuses GdbFailoverMode, RdsUtils, properties and message keys as-is. 24 new unit tests (test_aio_retry_util.py, test_aio_gdb_failover.py); full unit suite green (2011 passed), flake8 + isort clean.
AhmadMasry
added a commit
to AhmadMasry/aws-advanced-python-wrapper
that referenced
this pull request
Jun 17, 2026
… async Extends the async GDB failover parity to match what aws#1246 did on the sync side (integration-test matrix + docs): - test_aurora_failover_async.py: add the distinct `gdb_failover` plugin to all 8 failover test matrices alongside `failover_v2` (and `gdb_failover,host_monitoring_v2` / `aurora_connection_tracker,gdb_failover` for the compound ones), mirroring sync test_aurora_failover.py. Async ships one failover plugin so the failover/failover_v2 aliases collapse to one entry, but gdb_failover is a separate region-aware plugin and is exercised on its own. - UsingTheGdbFailoverPlugin.md: add an "Async wrapper." note (matching the one in UsingTheFailover2Plugin.md) documenting the `gdb_failover` async code and that the home-region parameters/modes behave identically in async. The aws#1246 integration_tests.yml change was only the removal of a temporary feat/gdb-rw branch trigger (not gdb test config), so nothing to mirror there; async tests run under the same workflow.
AhmadMasry
added a commit
to AhmadMasry/aws-advanced-python-wrapper
that referenced
this pull request
Jun 18, 2026
… phase-1) This sync-side regression test (the aws#1246 GDB live-run writer-selection fix) was added in phase-1 but missing from this full-feature branch. Add it so feat/async-parity is the complete superset and the phase-2 branch (rebuilt from phase-1 + this branch) is binary-equivalent to it.
AhmadMasry
added a commit
to AhmadMasry/aws-advanced-python-wrapper
that referenced
this pull request
Jul 10, 2026
…ettles Comparing against sync exposed the real invariant my monitor port broke: sync failover can only succeed THROUGH the topology cache (it connects to the cache's writer entry and role-verifies it), so by the time FailoverSuccessError propagates, the cache provably names the new writer and the CONVERTED_TO_* notifications have fired. The async port's candidate-probing fallback (deliberate robustness, aws#1246 lineage) lets failover succeed while aurora_replica_status -- and therefore every publication built from it -- still names the demoted writer for several seconds after promotion. Consumers then act on stale roles: the connection tracker misses the writer change (idle connections survive), and RWS can switch to a stale-epoch writer. Fix at the source: the panic-mode winner was verified via a live is_reader probe (sync's own verification standard, HostMonitor :550-560), so while the post-panic settling window is open the monitor publishes a role-corrected view -- the verified writer's row is WRITER, any other row claiming WRITER is demoted, a missing row is appended. The window is armed at verification (sync arms it at WriterPickedUpFromHostMonitors, :272-274 -- previously missing from the port) and expires after HIGH_REFRESH_PERIOD_SEC so a genuinely newer failover is never masked for long. Publications outside the window are raw, as before. Complements (does not replace) the tracker's departed-host invalidation: that guards the tracker even when the monitor path is bypassed entirely. Unit: stale-fetch winner publication overlay + window-expiry regression tests; 2198 passed, flake8/isort/mypy clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Added GDB Failover support.
Added retry writer connection logic to GDB Failover and Failover 2 plugin.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.