feat: backups taken from a standby via pgBackRest multi-host TLS (draft PoC for #103)#104
Draft
melancholictheory wants to merge 1 commit into
Draft
Conversation
Backups that CloudNativePG schedules on a standby currently fail at stanza-create with pgBackRest exit 56 (unable to find primary cluster), because the plugin only ever configures a single local pgBackRest host. This forces multi-instance clusters back to target: primary and puts all backup I/O on the primary. This adds an opt-in BackupStandby field (y/prefer/n, disabled by default) that, when a backup runs on a standby, configures the current primary as a second pgBackRest host over TLS (pg2-host-type=tls) and passes --backup-standby, so pgBackRest coordinates pg_backup_start/stop on the primary while copying data from the standby. The primary is discovered from the CloudNativePG cluster status (currentPrimary) and resolved to the primary pod's IP via an uncached read; a pods get permission is declared for it. stanza-create is also given the primary peer so it can initialise from a standby, without --backup-standby. This is a draft proof of concept. Config generation, primary discovery and unit tests are included; the long-running pgbackrest TLS server process lifecycle and its certificate provisioning are deferred pending maintainer direction. Refs: operasoftware#103 Signed-off-by: melancholictheory <61789920+melancholictheory@users.noreply.github.com>
melancholictheory
marked this pull request as ready for review
July 22, 2026 19:46
melancholictheory
marked this pull request as draft
July 23, 2026 19:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a draft proof of concept for #103 (support backups taken from a standby). It is meant to make the design concrete and get direction on the operational pieces before investing further. It is not proposed as merge-ready.
Problem
CloudNativePG's default backup target is
prefer-standby, so backups are dispatched to a replica to keep I/O off the primary. Today the plugin configures pgBackRest for a single local instance, so a backup that lands on a standby fails atstanza-createwith pgBackRest exit 56 (unable to find primary cluster). Multi-instance clusters are forced back totarget: primary.As discussed on #103, simply skipping
stanza-createon a standby is not enough:pgbackrest backupitself needs a primary among its configured hosts to runpg_backup_start/pg_backup_stop, so the real fix is to give pgBackRest a connection to the current primary (multi-host).What this PR does
PgbackrestConfiguration.BackupStandbyfield (y/prefer/n, default disabled) that gates the feature.--pg2-host ... --pg2-host-type tls ... --pg2-host-cert-file/key-file/ca-file ... --pg2-path/user/socket-path) and passes--backup-standby=<mode>.stanza-createthe same primary peer (so it can initialise from a standby) without--backup-standby.cluster.Status.CurrentPrimary(already used in the WAL path) and resolves it to the primary pod's IP. The pod is read uncached (added to the manager cacheDisableFor, so it needs onlyget pods, not a cluster-wide Pod informer), and apods getRBAC rule is declared. Falls back to a normal local backup when this instance is the primary or no primary is known.PgbackrestServerOptions(thepgbackrest serverTLS-server invocation) as the client-visible half of the design.yandprefermodes), the server options, the primary-peer decision, andresolveStandbyTopology(via a fake client).Deliberately out of scope (needs maintainer direction)
The operational half from #103 is not wired here, because it depends on decisions I did not want to pre-empt:
pgbackrest serverprocess in each sidecar (lifecycle, port exposure).tls-server-authidentity.Because of that, this cannot yet run a real standby backup end to end without those certificates and the server in place, and there is no e2e test flipping
test/e2e/.../backup/fixtures.goofftarget: primaryyet. The cert and server paths are constants and a command builder, documented as the contract the plugin expects.Testing
go build ./...,go vet,gofmt, and the unit tests (go test ./internal/..., with envtest for the controller package) all pass. New unit tests assert the exact generated pgBackRest invocations, including that the disabled path is byte-for-byte unchanged.Open questions
Same as #103. I would appreciate direction on the TLS server lifecycle, certificate provisioning, and primary discovery (pod IP vs a dedicated headless service) before extending this.