Deploy: same-schema patch upgrade fails the privileges convergence gate on an already-migrated silo

[jrosseel]

Reproduced on dev/testv4 while upgrading 0.9.1 → 0.9.2 at develop head 6d4fbf708d517d9bba4c1e9469922f1d5f02a10e. This is the same gap the deploy ledger already recorded on 2026-08-18 ("a same-schema patch must retain enough reviewed lineage for privilege reconciliation"); that run's workaround was never turned into a structural fix, so it reproduces identically.

Symptom

deploy.sh fails at the PostgreSQL post-upgrade hook Job opencrane-testv4-postgres-database-privileges, container opencrane-privileges (exit 1):

Database 'opencrane' is not an exact fresh or migrated 0.9.0 schema

The application release is never reached — opencrane-testv4 stays at its prior revision — while opencrane-testv4-postgres is left in Helm status failed, and the app pods roll once. No data or privilege mutation occurs: the container exits before its GRANT/REVOKE block, and the obot/litellm sibling containers (which skip the schema gate) complete normally.

Why it happens

• The live database is legitimately mid-lineage: opencrane_migrations.schema_history holds exactly one row (schema_version=0.9.0, source_schema_version=0.8.0, migration_id=0.8.0-to-0.9.0), and opencrane_bootstrap.target_baseline.baseline_sha256 is the frozen original-bootstrap origin, which can never equal a current release's plain baseline digest once a real migration has run.
• apps/postgres/helm/templates/database-privileges-job.yaml therefore only accepts convergence=current through a branch that requires PREVIOUS_MIGRATION_AVAILABLE=true plus an exact schema_history match.
• apps/_infra/deploy-k8s/platform/k8s-deploy.sh (~lines 306–325) correctly resolves 0.9.1 → 0.9.2 to kind=current / migration=null, since the schema versions are identical. Its only fallback for a null migration re-resolves database-transition.mjs against 0.9.2's own previousRepositoryVersion (0.9.1) — another same-schema hop that again yields migration=null. It never walks the ancestor chain (0.9.2 → 0.9.1 → 0.9.1's carriedForwardFromRepositoryVersion 0.8.1 → …) to rediscover the 0.8.0-to-0.9.0 evidence the already-migrated database still carries, so DATABASE_PREVIOUS_MIGRATION_* stays empty and the gate fails closed.

Net effect: every same-schema patch train is undeployable to an already-migrated silo without an operator manually rooting the deploy at an older release.

Proposed fix

In the DATABASE_CONVERGENCE_MIGRATION == null fallback in apps/_infra/deploy-k8s/platform/k8s-deploy.sh, or in resolveDatabaseTransition in scripts/release-versioning/database-validation.mjs: when the target's previous-release lookup also resolves to kind=current, recurse through each ancestor manifest's previousRepositoryVersion / carriedForwardFromRepositoryVersion until reaching the manifest that owns the last real schema-changing migration, and populate DATABASE_PREVIOUS_MIGRATION_* from that owner. DATABASE_TRANSITION_KIND stays current — no migration is performed — while the privileges Job gets the lineage evidence it requires.

Add a contract test that covers exactly this shape: an already-migrated database plus a same-schema patch transition, asserting the resolver emits non-empty previous-migration evidence.

Operational notes

• Interim workaround (ledger precedent): root the deploy at the release that owns the real migration, e.g. --from-release-version 0.8.1.
• A failed -postgres release is left behind on each attempt; the recovery expectation should be documented.
• The privileges Job's rejection message should name what it compared and which evidence was missing, rather than only asserting the database "is not an exact fresh or migrated" schema — the current wording sends operators looking at the database instead of at the resolver.

Pointers

• apps/_infra/deploy-k8s/platform/k8s-deploy.sh (null-migration fallback, ~306–325)
• scripts/release-versioning/database-validation.mjs (resolveDatabaseTransition)
• apps/postgres/helm/templates/database-privileges-job.yaml (convergence gate)
• apps/postgres/tests/migration-release-contract.sh
• docs/agents/deploy-ledger.md (2026-08-18 and 2026-08-19 testv4 entries)

[jrosseel]

The proposed fix in this issue was wrong, and so was the workaround. Verified rather than assumed:

• resolve(0.9.2, 0.8.1) errors with automatic database migration requires exact previous release '0.9.1', and the resolver additionally requires the release version to equal the root package.json version. So "walk back and re-resolve the transition against an older root" is not implementable, and --from-release-version 0.8.1 cannot run at all — it aborts before any cluster contact.
• The claimed 2026-08-18 ledger precedent does not exist; docs/agents/deploy-ledger.md ends at 2026-08-10 (testv3). A prior run clearly happened (helm rev 17 app / rev 23 postgres-failed), but its ledger write-back never landed. I repeated that claim without checking the file — apologies.

The actual fix is in #686: a dedicated resolveSchemaLineage lookup that walks the release chain to the release owning the migration which produced the current schema, and returns its recorded evidence. It leaves the transition at current, does not relax the adjacency policy, and does not loosen the convergence gate.

Also worth doing separately: whatever ran the earlier attempt must append its ledger entry, since a brief that cites the ledger as precedent is only as good as the file.