Skip to content

Guard bronze→silver schema drift with typed dbt models (enforced contracts + parity gate) #1762

Description

@mitasovr

Sub-task of #1744.

Context

Even with placeholder generation fixed, bronze→silver schema drift remains possible: several connectors feed one silver class through union_by_tag's positional SELECT * ... UNION ALL, so a column added / reordered / retyped in one connector's staging model surfaces only at runtime — as Code: 386 NO_COMMON_TYPE, a column-count mismatch, or silently cross-wired data. The pr_id String vs Int64 failure in #1744 is exactly this class of bug.

Task

Adopt typed dbt models as the guard, continuing the approach prototyped in #1344:

  1. Enforced dbt contractscontract: enforced: true + full typed column list on each silver class model, with matching contracts on every connector staging model that feeds it. dbt then validates at build time that the SQL output matches the declared schema.
  2. Offline parity gate — the manifest-only checker from feat(ingestion): cross-connector silver-contract parity gate (git commits) #1344 (check_silver_contract_parity.py): for each contracted silver class, every connector must emit the same column names, order (UNION ALL is positional), and types. Runs on every PR without a warehouse.

#1344 demonstrates both layers on silver:class_git_commits and is opt-in per class. This task = land that PR (rebase/refresh as needed) and roll contracts out to the classes behind the #1744 KPIs: remaining class_git_*, class_task_*, class_collab_*.

Acceptance criteria

Metadata

Metadata

Assignees

Type

Fields

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions