Skip to content

Commit the agent tooling: workflow hooks, three project skills, and a fully ignored reports/ - #53

Merged
andre-salvati merged 9 commits into
mainfrom
docs/data-divergence-skill
Aug 6, 2026
Merged

Commit the agent tooling: workflow hooks, three project skills, and a fully ignored reports/#53
andre-salvati merged 9 commits into
mainfrom
docs/data-divergence-skill

Conversation

@andre-salvati

@andre-salvati andre-salvati commented Jul 25, 2026

Copy link
Copy Markdown
Owner

What?

Makes this repo's agent tooling part of the repo instead of one developer's local config:

  • .claude/hooks/ + .claude/settings.json — four workflow hooks (branch protection, PR description as merge body, CHANGELOG gate, and a new PR-description freshness gate) and the settings file that registers them.
  • data-divergence — a new skill for investigating why two datasets that should agree don't, shipping a worked investigation of this repo's own prod catalog.
  • project-costs / sql-diagram — converted from slash commands to skills, each shipping the artifact it explains under examples/.
  • reports/ is now entirely gitignored, and the artifacts that were tracked there moved into the skills that use them.

Why?

CLAUDE.md and specs/workflow.md state the hook-enforced rules as project rules, but the hooks lived only in an ignored config — so the rules were a description of one machine, not a property of the repo. A fresh clone got none of them.

The two slash commands only loaded when someone typed them. As skills their descriptions sit in context and match on relevance, so "why is our Databricks spend up?" reaches project-costs without anyone remembering it exists. Typing /project-costs still works.

data-divergence came out of a live investigation this week — batch vs SDP silver in prod, a 13-day date shift. The procedure is written generically (no table or column names); its committed example is the one place it names real tables.

The fourth hook was added late, for a reason visible in this PR's own history: pr-merge-description.sh copies the PR body into the merge commit message, and this description had gone stale twice — advertising an example.md that later commits deleted, and warning that job1_prod was failing daily when the fix had been on main since 2026-07-25. On GitHub that is editable; in a merge commit it is permanent.

How?

.gitignore un-ignores by name, never by wildcard. The AI Dev Kit installs ~30 user-level skills into .claude/skills/; a wildcard there would commit all of them, the exact drift specs/tooling.md#install-layout documents as having shadowed the user-level set for two months. Git also won't descend into an excluded directory, so .claude/skills/ is re-included before its children are re-excluded. git add -An .claude/ is the check — it must list only our paths.

settings.json uses $CLAUDE_PROJECT_DIR. Absolute paths would break every other clone. Machine-specific entries (an update check pointing into ~/.ai-dev-kit/) moved to the still-ignored settings.local.json; Claude Code merges the two.

Every hook self-gates on the command text. pr-merge-description.sh originally relied on the settings-level if: filter alone, and that filter leaks: it rewrote unrelated Bash commands mid-session, appending --subject/--body-file to non-merges and firing a gh pr edit network write each time. Fixed here; the gate belongs in the script.

Freshness is enforced by a sentinel, not by judgement. A hook cannot tell whether prose is accurate. require-fresh-pr-description.sh proves the weaker thing that catches the real failure — that someone looked at the description at the exact commit being merged — via <!-- description-verified: <sha> --> checked against HEAD. It runs before pr-merge-description.sh, since a stale body caught after that point is already history.

Examples live with the skill, not in reports/. .gitignore only governs untracked files, so ignoring reports/ would have left the force-added artifacts in the index and made the rule a lie. They moved to .claude/skills/<skill>/examples/, a committed path. Both example.md commentary files were deleted along the way: an example is the artifact, and two files narrating one artifact drift — the prose is the copy that goes stale.

Validation?

  • git add -An .claude/ lists only our paths — no kit skills leak in. Re-run after touching that block.
  • Hooks commit at mode 100755; verified via git ls-files -s.
  • require-fresh-pr-description.sh has a test suite — 11/11 passing against a stubbed gh (no network): pass-through on non-merge and on unrelated gh commands; blocks on a missing stamp, a stamp from an older commit, and a stamp naming no real commit; allows a full-sha stamp, a short-sha stamp, and a merge with no PR ref. Includes a regression test that pr-merge-description.sh no longer rewrites non-merge commands or fires gh pr edit on them, and still rewrites a real merge.
  • protect-main-branch.sh ran on every commit in this branch and correctly allowed them.
  • require-changelog-entry.sh blocked work twice this session on commands merely containing its trigger phrase — a known false positive, not fixed here.
  • sql-diagram was exercised end to end on a deliberately complex query. That run found real blind spots in the tool — window functions are never drawn, WHERE is only drawn above a join, OUTPUT shows aliases not expressions — now documented in the skill.
  • Secret scan across all tracked files and full git history: no tokens, hosts, org IDs, ARNs, AWS account IDs or keys. README config samples are placeholders; CI uses ${{ secrets.* }} with the client secret scoped to steps rather than workflow env.
  • No Python changed; ruff pre-commit hooks skipped on every commit.

Impact in prod

  • No table schema/data change — no production impact.
  • Schema change — classify and declare the remediation:

Tooling and docs only. No commonSchemas.py, task module, cluster key, job definition or bundle resource is touched, so nothing here reaches a medallion table or a running job.

Chosen strategy: leave as-is — nothing to migrate.


⚠️ The committed data-divergence example documents four live divergences in prod, none fixed here and none introduced by this PR:

  1. curated.order_enriched_sdp carries order_date 13 days early on 6,000,000 rows, root-caused to seed_sources.py:135 anchoring the backfill on run date rather than the existing _EPOCH constant.
  2. country diverges on 5,364,000 rows / 447 of 500 customers — the same append-only freeze, on a column nobody intended to freeze.
  3. report.order_agg is missing 2026-07-24 entirely: the run failed, bronze self-healed via full overwrite, and the silver incremental MERGE is scoped to date == seed_date, so no later run backfills it. Any failed daily run leaves a permanent hole.
  4. 5,000 orders of 2026-06-24 are tripled in the SDP silver, since only the batch path runs DQX.

Recommended order once someone picks these up: anchor _seed_initial on _EPOCH, backfill 07-24 (or widen that filter to a lookback window), deploy, then make drop env=prod. A full refresh alone would restate product_name across all history.

andre-salvati and others added 9 commits July 25, 2026 06:47
… wiring

Adds a repo-owned `data-divergence` skill for investigating why two datasets
that should agree don't, and commits the three workflow hooks plus the
settings file that registers them.

The hooks enforce rules CLAUDE.md and specs/workflow.md already state as
project rules (branch protection, PR description as merge body, CHANGELOG
gate), but they lived only in one developer's ignored config — so the rules
were documentation of one machine rather than a property of the repo.
settings.json now refers to them via $CLAUDE_PROJECT_DIR; machine-specific
entries move to the still-ignored settings.local.json.

.gitignore keeps `.claude/*` blanket-ignored and un-ignores only these paths
by name, so the AI Dev Kit's user-level skills in .claude/skills/ stay out.
`git add -An .claude/` is the check that this holds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…xamples

/project-costs and /sql-diagram were slash commands, so their instructions
loaded only when the user typed them. As skills their descriptions sit in
context and match on relevance — "why is our Databricks spend up?" now reaches
project-costs without anyone remembering the command exists. Typing the slash
name still works.

Each ships an example.md alongside the SKILL.md, walking a committed artifact
end to end: reports/sql-diagram/job_spend_plan.* for the plan-mode reading
(CTE sub-pipelines, LEFT joins that must stay LEFT, the range predicates that
stop a slowly-changing dimension fanning out), and reports/cost/2026-07-22.md
for the cost analysis (per-active-day normalisation, reconciling the
attributed total, treating SQL-warehouse silence as a finding). The examples
reference those artifacts rather than duplicating them, so there is one copy
to keep current.

.claude/commands/ is gone; the .gitignore block is restructured to un-ignore
the three repo skills by name, with the no-wildcard rule stated inline —
a wildcard there would commit the AI Dev Kit's user-level skills.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
reports/ is generated output — coverage, spend reports carrying account cost
figures, query diagrams — and none of it should be in git. The three
directory-level rules are replaced by a single `reports/`.

Four artifacts were tracked there, force-added as worked examples for the
project-costs and sql-diagram skills. .gitignore only governs untracked files,
so ignoring the directory would have left them in the index and made the rule
a lie; they are moved to .claude/skills/<skill>/examples/ instead, which is a
committed path. The examples and the two README links keep working, and
reports/ becomes purely the place the tools write.

The "git add -f one to keep it as an example" advice is removed from both the
gitignore comment and the sql-diagram skill, and replaced with the rule that a
new example is a copy into the skill's examples/ directory. Also drops the now
redundant reports/coverage/ entry.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
workflow.md said the schema-drift guard had a single exception (ops._health).
There are two: raw.order_quarantine also writes with overwriteSchema=true,
added in #52 and already documented in CLAUDE.md — the spec was not updated in
the same commit, which is what its own "keep docs in sync" section exists to
prevent. Also corrects the Co-Authored-By trailer, which named Opus 4.8.

require-changelog-entry.sh printed guidance the spec had superseded: "at most
3 sentences" (dropped at #49 because it constrained nothing) and a header
template built from the branch name. Both now match specs/workflow.md — a
~1000-character single paragraph and the PR-URL header — and the message links
to the section so the two cannot drift silently again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EOF --subject 'Commit the agent tooling: workflow hooks, three project skills, and a fully ignored reports/' --body-file /tmp/pr-body-c1_otrlp.md
The plan diagram is authoritative on scans, joins and join predicates and
silent on windows, CTE-level filters and projection expressions — which is
where the defects that produce wrong numbers live. The skill now traces the
query through a handful of hand-built rows after the chart and ships both as
one self-contained HTML page.

Two rules the trace depends on: never hand-trace (rewrite each source table as
a literal VALUES CTE, run it, paste the real result — NULL propagation,
COUNT(DISTINCT) and window ordering are all easy to get wrong on paper), and
embed the rendered .svg as a data URI rather than re-rendering the .mmd, which
also keeps the SVG's own generic class names out of the page cascade.

examples/job_spend_plan.html is the worked instance: five usage rows through
the repo's own per-job spend query, showing a date-scoped price join picking
one price per row — with the verified counterfactual that dropping the range
predicates doubles quantity and inflates spend from $10.00 to $19.50 — a LEFT
join keeping an unpriced SKU alive, and a filter that discards unattributable
usage, which is why the attributed total never equals the Databricks total.

example.md is deleted. It asserted in prose what the page now demonstrates
with data; its unique content (the regenerate step, the CTE two-node shape)
moved into SKILL.md. project-costs keeps its example.md, because a page of
cost tables cannot explain why its analysis is written the way it is — the
rule is now stated in specs/tooling.md: add a commentary file only when the
artifact cannot speak for itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ample

The data-divergence skill was the only one of the three shipping no example.
This adds one: a full investigation of prod's batch vs SDP silver tables,
re-run against live prod rather than transcribed, and structured the way the
skill prescribes — layer counts, key-level diff, row_commit_version to pin the
blast radius, a Delta-history timeline, proven-vs-inferred, costed fixes, and
an appendix of the queries.

It is kept for its findings, not its format. The reported complaint was one
shifted date column; three further divergences turned up, including a live gap
in prod.report.order_agg (2026-07-24 missing after a failed run, never
backfilled because the incremental MERGE is scoped to seed_date).

Also folds in the README rephrasing of the skill bullets.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every principle in it was already stated generically in SKILL.md — leading with
the cloud split, dollarizing spikes before judging them, using the daily
<details> block to attribute a spike to a date, normalizing partial edge weeks,
reading SQL Serverless silence as a finding, reconciling attributed spend before
trusting the per-job table, and comparing per active day. The file re-narrated
those against one window's numbers.

Two points were not already generic and moved into SKILL.md: that a batch-vs-SDP
gap becomes a finding through durability (holding across the window and
repeating in a second environment) rather than size, and that the example is to
be read for shape, not for figures.

sql-diagram lost its example.md for the same reason last commit, so tooling.md
now states the rule in its general form instead of justifying one exception.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pr-merge-description.sh copies the PR body into the merge commit message, so a
description written five commits ago does not just mislead on GitHub — it becomes
permanent history. This PR's own description was the worked example: it still
advertised an example.md that two commits had deleted, and warned that job1_prod
was failing daily with the fix unmerged, which had been false since 2026-07-25.

A hook cannot judge whether prose is true. require-fresh-pr-description.sh proves
the weaker thing that catches the real failure: that someone looked at the
description at the exact commit being merged. The contract is a sentinel in the
body, <!-- description-verified: <sha> -->, checked against HEAD. When it does not
match, the block lists what landed since it was last verified. It runs before
pr-merge-description.sh, because a stale body caught after that hook is already
in history.

Also fixes a confirmed defect in pr-merge-description.sh: it never checked that
the command was a merge, relying entirely on the settings.json `if:` filter. That
filter leaks — the hook rewrote unrelated Bash commands mid-session and fired a
gh pr edit network write on each one. It now self-gates on the command text like
its siblings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@andre-salvati
andre-salvati merged commit 84cb516 into main Aug 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant