You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Even with placeholder generation fixed, bronze→silver schema drift remains possible: several connectors feed one silver class through union_by_tag's positional SELECT * ... UNION ALL, so a column added / reordered / retyped in one connector's staging model surfaces only at runtime — as Code: 386 NO_COMMON_TYPE, a column-count mismatch, or silently cross-wired data. The pr_id String vs Int64 failure in #1744 is exactly this class of bug.
Task
Adopt typed dbt models as the guard, continuing the approach prototyped in #1344:
Enforced dbt contracts — contract: enforced: true + full typed column list on each silver class model, with matching contracts on every connector staging model that feeds it. dbt then validates at build time that the SQL output matches the declared schema.
Offline parity gate — the manifest-only checker from feat(ingestion): cross-connector silver-contract parity gate (git commits) #1344 (check_silver_contract_parity.py): for each contracted silver class, every connector must emit the same column names, order (UNION ALL is positional), and types. Runs on every PR without a warehouse.
#1344 demonstrates both layers on silver:class_git_commits and is opt-in per class. This task = land that PR (rebase/refresh as needed) and roll contracts out to the classes behind the #1744 KPIs: remaining class_git_*, class_task_*, class_collab_*.
Acceptance criteria
Parity gate green in CI on every PR; a deliberately retyped/reordered column in one connector's staging model fails the gate offline.
Enforced contracts on all class_git_*, class_task_*, class_collab_* silver models and their connector staging models.
Sub-task of #1744.
Context
Even with placeholder generation fixed, bronze→silver schema drift remains possible: several connectors feed one silver class through
union_by_tag's positionalSELECT * ... UNION ALL, so a column added / reordered / retyped in one connector's staging model surfaces only at runtime — asCode: 386 NO_COMMON_TYPE, a column-count mismatch, or silently cross-wired data. Thepr_id StringvsInt64failure in #1744 is exactly this class of bug.Task
Adopt typed dbt models as the guard, continuing the approach prototyped in #1344:
contract: enforced: true+ full typed column list on each silver class model, with matching contracts on every connector staging model that feeds it. dbt then validates at build time that the SQL output matches the declared schema.check_silver_contract_parity.py): for each contracted silver class, every connector must emit the same column names, order (UNION ALL is positional), and types. Runs on every PR without a warehouse.#1344 demonstrates both layers on
silver:class_git_commitsand is opt-in per class. This task = land that PR (rebase/refresh as needed) and roll contracts out to the classes behind the #1744 KPIs: remainingclass_git_*,class_task_*,class_collab_*.Acceptance criteria
class_git_*,class_task_*,class_collab_*silver models and their connector staging models.pr_idtype-drift scenario from New environment: Git + Task Delivery KPIs (Commits, PRs Merged, Tasks Completed, Cycle Time, …) never load until silver tables are manually reset #1744 is impossible to reintroduce without CI failing.