feat(dbt): add column-level meta to AGENTS.DBT_COLUMN

[kevinskim93]

Why

Customer request: bring column meta into the dbt_column spec.

Column-level meta is where dbt projects record governance and semantic context — PII flags, data owners, units, business-friendly labels, masking policies. AGENTS.DBT_MODEL already exposes model-level meta, so an agent can see that context for a table but not for the individual columns it is about to query.

What changed

AGENTS.DBT_COLUMN gains one meta TEXT column holding the column's dbt meta dict serialized as a JSON string, mirroring AGENTS.DBT_MODEL.meta exactly.

CREATE OR REPLACE TABLE AGENTS.DBT_COLUMN (
  model_id    VARCHAR NOT NULL,
  column_name VARCHAR NOT NULL,
  data_type   VARCHAR,
  description TEXT,
  meta        TEXT,          -- new
  PRIMARY KEY (model_id, column_name)
);

Precedence is the same rule DBT_MODEL.meta uses — column.config.meta, else top-level column.meta, else {}.

• src/agents_schema/dbt.py — new schema column plus one row value in _ingest
• SPEC.md — DDL, source-field table, and the summary-table description
• the three built-in analyst skills — meta added to the dbt_model / dbt_column schema-reference rows, so agents know the field is queryable (the dbt_model row was already missing it)
• tests/test_dbt.py — new; covers meta from config.meta, from top-level meta, and {} when absent

Why read both config.meta and top-level meta

Verified empirically rather than assumed, by parsing a project with dbt 1.12 that declares column meta both ways, and by diffing the published manifest JSON schemas:

	manifest v11 (dbt ≤ 1.9)	manifest v12 (dbt 1.10+)
ColumnInfo.config	absent	present, meta mirrored
ColumnInfo.meta	present — the only source	present, mirrored

On current dbt the two locations are always populated identically, whichever YAML form the project author uses (config: meta:, top-level meta:, or dbt_project.yml +meta:) — the same is true at model level. So on v12 either read alone would do. The top-level fallback is what keeps pre-1.10 manifests working, where column config does not exist at all; reading config first is forward-compatible with dbt's deprecation of the legacy top-level field. Neither branch is dead across the supported range.

Compatibility

Tables are CREATE OR REPLACEd on every run, so the column appears on the next sync with no migration. Queries that name columns explicitly are unaffected; SELECT * consumers get one extra field. Type mapping verified across all three writers: Snowflake TEXT, BigQuery STRING, Databricks STRING.

Scoped to meta only. Column-level tags also exist in the manifest and would be a natural follow-up, but were left out of this change.

Drive-by

tests/test_connector_root.py called omni.run without importing omni, so that test errored with NameError on every run — the only failure in the suite. Fixed in a separate commit.

Verification

Full suite:

$ uv run python -m unittest discover -s tests
Ran 133 tests in 0.829s
OK

End-to-end against a real dbt 1.12 manifest (not just the fixture), with order_id declaring meta under config: and customer_email at the top level:

('model.metatest.orders', 'order_id',       '', 'Primary key.',  '{"pii": false}')
('model.metatest.orders', 'customer_email', '', 'Billing email.', '{"pii": true}')

🤖 Generated with Claude Code

[smogili2]

Since meta is always a JSON object, this can be marked as variant data type instead to make it easier for agents to know.

[smogili2]

Should also include dbt model meta column ingest test case.