ADR 0003/0004: enforced verification ledger + measured 3-way benchmark

[whchi]

Summary

Lands ADR 0003 (enforced verification & maker/checker integrity) and ADR 0004 (orchestration-value evidence), then closes the loop with a live measured benchmark run and an evidence-first repositioning.

Verification (ADR 0003)

• Capture the maker's Verification status: block as a new verification-report trace event (non-loop path).
• trace-check now fails when a maker reported file changes with status unverified and no verifier delegation followed in the same session; verification-skipped with a reason stays an operator opt-out.
• Custom-agent domain-neutrality guard also fires at agent-definition load time, not only in doctor.

Benchmark (ADR 0004)

• First committed measured run of the execution-quality suite: 6 live opencode run executions (native / same-provider / mixed-provider × 2 exec tasks), fixture tests/fixtures/orchestration-benchmark/2026-07-04-exec-quality-3way.jsonl, reproduced by a regression test.
• Honest results: same-provider orchestration was +85–146% tokens with no quality gain; mixed-provider matched native cost once (cheaper-same-quality). No verifier pass was ever triggered because the maker self-verified — follow-up task for deceptively-green suite design recorded in ROADMAP.
• Fixed extract-benchmark-metrics.ts taskID regex (exec-coding-001 no longer truncates to coding-001).

DX / hygiene

• GitHub Actions CI (bun test, bun run build, git diff --check).
• README now leads with the maker/checker ledger and cites the measured numbers; doc-positioning test flipped to protect the new order.
• CLI help grouped by workflow (Setup / Domains / Loops / Evidence / Benchmark).
• Track bun.lock instead of package-lock.json; collected essays replaced by docs/design-influences.md (repo mappings + claim boundaries); package.json cleanup.

Verification

• bun test: 162 pass, 0 fail
• bun run build, git diff --check: pass
• bun install --frozen-lockfile: pass (CI parity)
• Benchmark rubric scored from run transcripts; all six runs passed acceptance.

🤖 Generated with Claude Code

[whchi]

Code Review — PR #5 (main...HEAD)

Direct review (the superpowers:code-reviewer agent isn't available in this environment, so this is the skill's direct/fallback path). Scope: 4 commits, ~1547 insertions. The genuinely new logic is the non-loop verification ledger in domain-usage-contract.ts (~146 lines, well-tested); the rest is agent-prompt/domain files, benchmark fixtures, tests, and docs.

Findings

MEDIUM — scripts/extract-benchmark-metrics.ts: marker substring collision. The three run markers are yl-exec-quality-20260704 (native), …20260704sp, …20260704mp. The base marker is a prefix of the other two, and extraction uses title LIKE %marker%, so --marker yl-exec-quality-20260704 also pulls the sp/mp sessions. The committed fixture is correct only because it was built with a manual grep '"variant":"native-builder"' filter. A future benchmark run that reuses the script naively will silently double-count. Fix: use a delimiter-anchored marker match (e.g. require a trailing space/word-boundary), or make the native marker not a prefix of the others. Dev-tool only (not in the node build), so non-blocking.

MEDIUM — findUnverifiedFileChangeLedgers session match is fail-open. The predicate (!event.sessionID || !candidate.sessionID || candidate.sessionID === event.sessionID) means that when either event lacks a sessionID, any later verifier delegation anywhere in the trace vouches for the unverified ledger. sessionID is optional, so an unrelated verifier pass could clear a real unverified-change ledger. It fails-open (misses a violation) rather than false-positive, which matches the lenient-evidence philosophy, but it's worth a comment noting the soundness gap.

LOW — no session/time windowing on the ledger check. The trace file is append-only across sessions, so a historical verification-report with unverified changes and no following verifier delegation will fail trace-check permanently until the trace is cleared. This is arguably ADR-0003-intended ("ledger integrity is blocking wherever a ledger exists"), but the permanent-red behavior is surprising operationally — worth one sentence in DOMAIN_OBSERVABILITY.md.

LOW — parseVerificationStatusReport uses lastIndexOf('verification status:'). Prose containing that phrase before the real block would shift the parse start. Low risk given the fixed bullet-block format the builder emits, but it's a heuristic, not a structural anchor.

Strengths

• The verification core is tightly scoped and has real negative-path coverage: capture-with-ledger, skip-without-ledger, verification-skipped opt-out, and — notably — the ordering case proving a verifier delegation that precedes the maker report does not vouch for it.
• The benchmark change commits an unfavorable measured result (same-provider +85–146% tokens, no quality gain) and locks it with a reproducing test — the honest-evidence intent of ADR 0004 in practice.
• Parser is deliberately kept separate from the envelope parser so generic field names (Status:) don't leak into global envelope fields — good boundary call, and the comment explains why.
• Clean commit separation, docs (warn-vs-block policy table) updated in lockstep with the code, CI added and green (11s).

Note on size

At ~1.5k lines the PR is above the comfortable single-review threshold, but the bulk is fixtures/prompts/tests rather than logic, and the commits are individually coherent. Not a blocker.

Assessment

Ready to merge: Yes. No CRITICAL/HIGH issues; bun test 162 pass, build + git diff --check green, CI green. The two MEDIUM items are follow-ups (the extract-script collision is the one most worth fixing before the next benchmark run).

🤖 Reviewed with Claude Code