docs(adr-0006): make the audit report optional; give the auditor branch-commit history - #122
Merged
Merged
Conversation
ADR-0006 required the caller to commit every structural audit to docs/audits/<date>-<scope>.md. That requirement is reversed: the directory stays, keeping a report there is now the caller's call. The report is the second copy of a record the closure rule already demands (issue, landed guard, or `unguarded:` invariant), and it is the weaker copy — prose about a tree that changes under it, with no guard and no way to write one, which is the restate-over-derive shape ADR-0005 rejects. A mandatory reviewable document per run also prices the audit above what an average one returns, and the failure mode is a skipped audit, not a missing file. The cost is real and recorded, not papered over: Option B's objection comes back. Without a guaranteed prior report the auditor's "rerun and compare" promise has no guaranteed baseline, measurements do not form a time series, and a checked-and-dropped list can be re-litigated. The amendment accepts that in ADR-0005's `unguarded:` form, with a stated reason and a revisit trigger. The original requirement, its Consequences bullets, and Option C are left in place with pointers to the amendment, so the reversal reads as a decision rather than an absence. The two existing 2026-08-02 reports are KEPT as historical record — dropping the requirement is not a reason to destroy what it produced — and each carries a line saying the rule that produced it is no longer in force. Also updated: the structural-auditor agent definition (a prior report is now "if one is there", and an absent baseline must be stated rather than assumed), the CLAUDE.md audit-closure convention, and the ADR list in CLAUDE.md, which had never listed ADR-0006. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…the ledger Three changes to the audit agents, all downstream of squash-merge hiding evidence and of #100 retyping the invariant ledger. structural-auditor: - Add "a set of merged PRs" to the scope list, so a retrospective over a run of recent reviews is a first-class scope rather than a re-framing each time. - History pass now reads the commit sequence INSIDE merged PRs. main is squash-merged, so a hunk fixed three times on a branch lands as one clean commit and the fix-the-fix chain is unreachable from `git log`. Recovered with `gh pr view <n> --json commits`, which still returns the pre-squash sequence after merge. Scoped to PRs the other probes already implicated, so it stays one API call per candidate rather than per PR in the window. The squash fact is written as a check, not an assertion, so a change of merge strategy is noticed instead of assumed. - Stop naming `(—)`. #100 typed the ledger into the ADR-0005 vocabulary and touched CLAUDE.md only, leaving this file hunting a marker with zero instances — silent under-coverage in the agent whose job is finding unguarded couplings. Fixed by removing the copy rather than guarding it: the cross-section bullet names the categories and defers to the ledger's own header for the markers. An untyped legacy citation is now itself a finding, matching "type it when you touch it". pr-reviewer: - Record that the origin/main anchor also puts the branch's own commit sequence out of scope, and that this is a handoff to the auditor rather than a blind spot. Without it the exclusion reads as an oversight and the next editor removes the anchor, restoring the self-trigger #90 fixed. The `gh pr view` command was run as written before being embedded; the two commands this file has shipped before were both wrong on first landing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011KsnnuPRJW5sUPppm3RSRo
The amendment listed the two reports in docs/audits/ by filename as "the existing reports". #112 commits a third under the now-optional rule, so the list goes stale on merge — a second copy of a directory listing, which is the restate-over-derive shape this very amendment argues against. De-enumerated in all three places (§Consequences, the thinner-than-it-looked bullet, and the reports-stay paragraph) with the reason stated inline so it is not re-enumerated later. No decision changes: the reports are still kept, closure is still filed / landed / recorded, and the commit is still optional. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011KsnnuPRJW5sUPppm3RSRo
§Consequences still said a shape is closed when "recorded as an accepted `(—)`" — the pre-ADR-0005 marker, in the ADR that governs the agent whose identical drift 934e999 just fixed. This one is prescriptive, not historical: a reader following it looks for a token with zero instances in CLAUDE.md, which already states the same rule as `unguarded:`. Left alone deliberately: ADR-0005's own `(—)` uses (it is the record that replaces the marker, and says legacy entries stay valid) and the dated audit reports (snapshots of the tree as it was). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011KsnnuPRJW5sUPppm3RSRo
…y pass Both changes come from running the pass added in 934e999 against #82–#103 and reading what it actually returned. Scope now accepts findings from earlier reviews, pasted by the caller. They carry what no durable record does — the reasoning, and the candidates raised and dropped — because `pr-reviewer` writes no files and its output dies in the invoking conversation. Framed as evidence to cluster rather than a tree to sweep, with dropped findings called out as signal: a candidate the reviews keep raising and dismissing is a shape whether or not any instance was real. They are one source among several, not the record. The branch-history bullet described the wrong target. "Look for fix-the-fix chains" points at the right place, but most chains are the repo working as designed — review finds a fail-open guard, review fixes it — so hunting them rediscovers ADR-0005 at length. The productive query is whether the follow-up's lesson ESCAPED the branch: diff what the follow-up says it learned against the squash body and against the siblings that share the shape. Both shapes the test run surfaced are that query, and neither is reachable from main. Also added, all measured on that run: - Gate the pass on the squash body being a hand-written description rather than concatenated commit messages. That is the entire reason follow-ups are unrecoverable; where it does not hold, the calls buy nothing. - Check whether main still asserts a claim its own branch retracted. #84's squash body says `--check` "always exits 0" — wording `57c2b348` corrected inside that same branch — so the permanent record preserves the error. - Order the cost controls: counts first, drop single-commit PRs unread, then MESSAGES (which stated the finding outright in every case here), diffs last. - State the real yield. 8 of 14 PRs were single-commit and produced nothing; of 6 follow-ups read, 2 were live. Budgeting for ~a third beats implying the pass is dense. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011KsnnuPRJW5sUPppm3RSRo
dlovell
force-pushed
the
docs/adr-0006-drop-audits-requirement
branch
from
August 6, 2026 14:54
8ba0bf9 to
ac7aca6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three commits, one story: what an audit is required to leave behind, and what the auditor is able to read.
1.
810dbad— ADR-0006 amendment: committing an audit report is optionalReverses the "audits get a baseline" requirement. Closure is untouched — every surviving shape is still filed, landed, or recorded as an accepted
unguarded:invariant — but the report itself is no longer mandatory.Why: the report is the second copy and the weaker one. A finding's durable form is an issue, a landed guard, or an
unguarded:invariant, and all three live where the next reader already looks; a committed report restates that in prose nothing keeps true. It is the restate-over-derive shape ADR-0005 rejects, applied to the auditor's own output. A mandatory artifact also taxes the run that was already easiest to skip — the failure mode is not a missing file, it is a skipped audit.The cost is accepted explicitly rather than waved off: with no guaranteed prior report there is no guaranteed baseline, so measurements may not form a time series and a checked-and-dropped list may be re-litigated. Recorded in
unguarded:form with a revisit trigger. Existing reports and the directory stay.2.
934e999— the auditor reads branch commits; the vocabulary is derived, not namedBranch history as an input.
mainis squash-merged, so a hunk fixed three times on a branch lands as one clean commit and the fix-the-fix chain is unreachable fromgit log. Both audit agents excluded that history —pr-reviewerreviews the netbase...headdiff and deliberately anchors its history probe toorigin/main, and the auditor is repo-scoped — so nobody read it.This PR's own subject demonstrates the loss. #90, which created these agents, contains:
Two corrections to the same three lines of prose. On
mainthat is one commit,ba15520.gh pr view <n> --json commitsstill returns the pre-squash sequence after merge, so this needs no pre-merge window. Scoped to PRs the other probes already implicated, so it stays one API call per candidate rather than per PR in the window. The squash fact is written as a check, not an assertion, so a change of merge strategy is noticed instead of assumed.Also adds "a set of merged PRs" to the scope list, making a retrospective over recent reviews a first-class scope.
Vocabulary drift. #100 (
04e9eee) typed the invariant ledger into the ADR-0005 vocabulary and touchedCLAUDE.mdonly.structural-auditor.mdreads that ledger and still named(—)in two places — a marker with zero remaining instances — so an agent following the instruction searched for a token that no longer exists and would have skipped every accepted-risk invariant in the repo. Silent under-coverage, in the agent whose job is finding unguarded couplings.Fixed by removing the copy rather than guarding it: the cross-section bullet names the categories and defers to the ledger's own header for the markers. A legacy untyped citation is now itself a finding, matching "type it when you touch it".
pr-reviewergains a note that theorigin/mainanchor also puts the branch's own commits out of scope, and that this is a handoff to the auditor. Without it the exclusion reads as an oversight and the next editor removes the anchor, restoring the self-triggerab9e77defixed.3.
0d35e4c— stop enumerating the audit reports by nameThe amendment named the two files in
docs/audits/as "the existing reports". #112 commits a third under the now-optional rule, so that list goes stale on merge — a second copy of a directory listing, which is the shape the amendment itself argues against. De-enumerated in all three places with the reason stated inline.Verification
pre-commit run --fileson all three files: clean. Thecheck-gitignore-agentsprobe passes, confirming the agent files can actually be staged (ADR-0003).gh pr viewcommand was run as written before being embedded. The two commands this file has shipped previously were both wrong on first landing; embedding a third unverified one would have repeated the exact failure this PR documents..claude/agents/, which is the point of the issue below.Related
.claude/agents/*.mdis executable content with no guard: shell commands and ledger vocabulary both drift silently #121 — the finding this work produced, filed rather than fixed here:.claude/agents/*.mdships shell commands and ledger vocabulary with no guard, and both halves have now broken once. This PR fixes the two symptoms; the mechanism is open. Deliberately notCloses— and.claude/agents/*.mdis executable content with no guard: shell commands and ledger vocabulary both drift silently #121 argues accept-with-a-reason is a legitimate disposal.main, docs: record two audit shapes as accepted, with reasons #112, fix(devcontainer): repair bind mount points' daemon-created parents #113, test: guard the wiring, not just the unit #110, ci: make each build workflow a trigger path of itself #109, test(nix-user-sync): guard the upstream half of the NIX_USER coupling #108.main): it rewrites the "Guards follow ADR-0005" bullet while810dbadextends the "A structural audit…" bullet directly beneath it. Adjacent lines, independent content — whoever merges second keeps both.mainindependently of this branch, but note it edits the same sentence810dbaddoes: it replaces "ADR-0004 is reserved by fix(claude): bind memory from the host, ending the MEMORY.md index clobber (ADR-0004) #81" with the real entry while this adds ADR-0006. Both additions must survive the resolution; taking either side wholesale drops an ADR from the index.🤖 Generated with Claude Code
https://claude.ai/code/session_011KsnnuPRJW5sUPppm3RSRo