test: make the safe assertion the easy one, and measure guard coverage

[dlovell]

Meta-work from three review rounds this week (#113 ×2, #120 ×1) that produced nine findings. Two had a structural cause with a mechanical fix — this is those two. The other seven needed a reader, and nothing here pretends otherwise.

1. Fail-open assertions → make the safe form ergonomic

assert_contains matches a needle that is a prefix of a longer line, so an assertion about <owner> /a/b is satisfied by an unrelated -R <owner> /a/b/c — and passes while the behaviour it names has stopped happening.

That shipped in tests/test-volume-chown-guard.sh (caught in review on #113) in a file that already documented the trap in a comment. The knowledge was present; the affordance was not. Suites needing line anchoring were wrapping grep -qxF in assert_true/assert_false — workable, but it loses the expected/got diagnostic and leaves each author to rediscover the problem.

assert_line / assert_no_line make the safe form the ergonomic one. tests/test-harness-assertions.sh pins the difference between the two families, since that difference is the only reason the safe pair exists — if assert_line ever degraded to substring matching, every migrated call site would silently go fail-open again.

2. Mutation records prove the suite has teeth, not the assertions

ADR-0005 §2 requires showing a guard fail once. Measured on tests/test-volume-chown-guard.sh with its five recorded mutations: 9 of 46 assertions ever went red — and the one that turned out to be fail-open sat in the other 37 with nothing pointing at it.

tests/mutation-coverage <suite> '<cmd>'... runs mutations in throwaway copies and reports the never-red set as a worklist, not a verdict — most never-red assertions are fine, they pin things the mutations don't touch. It also reports no-op mutations, which matters more than it sounds: a silently non-matching sed otherwise reads as a passing mutation run that tested nothing. It caught one of mine while I was writing this.

3. pr-reviewer gains the rubrics that actually worked

Three instructions produced the highest-value findings across those rounds, and every time I typed them ad hoc into the invoking prompt:

• treat the PR's own claims (body, comments, invariants, mutation records) as unverified assertions — over-claims and under-claims are findings
• sweep new assertions for vacuity, naming the shapes that have actually shipped here
• check whether a new guard reads all channels and fields of the substrate it claims to cover — reporting only when it claims coverage it lacks, so this doesn't become a licence to demand scope

Leaving those in prompts made review quality depend on the invoker remembering — the "convention only its author knows" failure this repo already documents, applied to review itself.

Why CLAUDE.md gets only two lines

The session's central finding is that prose without a reader drifts. The response to that should not be mostly prose. Agent definitions are read at the moment they're relevant; a harness function cannot be forgotten. CLAUDE.md competes with ~330 lines for attention, so it gets the two facts that are genuinely authoring-time rules and nothing else.

Scope notes

• tests/test-nix-user-sync.sh migrates its one hand-rolled site. The remaining grep -qxF uses in test-dockerignore-lib-allowlist and test-image-fingerprint are plain loop conditions, not assertions, and want no migration. (My first draft of the harness comment claimed "four suites worked around it" — that was an over-claim, corrected before commit.)
• tests/test-volume-chown-guard.sh is not migrated here: it is in flight on fix(devcontainer): repair bind mount points' daemon-created parents #113. It should move once that lands.
• tests/mutation-coverage is extensionless so tests/*.sh does not pick it up as a suite, matching the dev/ convention.

Verification

• bash tests/run-all → all suites green, exit 0
• pre-commit run --all-files → clean
• Two mutations recorded in the new suite's header, both run through tests/mutation-coverage itself

🤖 Generated with Claude Code

https://claude.ai/code/session_01MMRkcVZwXpwBqjMY4bSehH