Audit C1b: a ratio that measured nothing does not vote, and a local variable is not a symbol - #4
Merged
Merged
Conversation
…variable is not a symbol Closes audit C1b and §3.2. Two changes that only work together — landing either alone is measurably worse than landing both. The audit's proposed fix is inert. §3.2 proposed computing `R_struct` over `extractContentMarkers`, excluding the `filepath:` marker that no content transform can destroy, and stated it "would fix both cases at once". Measured, it fixes neither: removing the only marker an item had leaves the before-set empty, and an empty set defaults `R_struct` back to 1.0 — the identical free 0.40 by another route. That change alone was byte-identical across all 586 rows of a frozen 293-file corpus. The free 0.40 comes from the empty-set default, not from `filepath:`. So an unmeasured ratio is now excluded from the score rather than defaulted, with its weight redistributed to the ratio that did measure something. For code `S_k = 1 - R_AST`, and the maximum symbol loss that can pass the 0.40 gate falls from 66.7% to 40%. When neither ratio measured, retention returns 1.0 and stays silent — that case belongs to the measurement gate (§37), and having both refuse would attribute the refusal to the wrong question. This is §33's argument applied to the score rather than to the gate. Applied on its own that cost 14 TypeScript files and 11.75pp, and the loss it was guarding against turned out to be almost entirely fictitious. `extractSymbols` matched `const|let|var` anywhere, so every `const i`, `const result`, `const msg` inside a function body counted as a semantic symbol on par with an exported function — and body elision is precisely the transform that removes them. Measured at `targetReductionRatio: 0.5`: src/core/engine/index.ts 42 of 63 symbols "lost", 41 function-local src/core/hashing/tokenizer.ts 9 of 17 symbols "lost", all 9 function-local Not one exported function, type or interface was lost in either case, because `selectElisionRegions` retains signatures by construction. The audit's "you can destroy two-thirds of every symbol in a file and pass" was measuring temporaries inside bodies the caller asked to have elided. Python is the control, and the asymmetry is what identified the defect: its extractor never had a locals rule, its measured symbol loss under the same elision is 0.0%, and it is untouched by either half. A safety metric should not depend on which language's extractor happens to harvest block-scoped bindings. Measured over the frozen corpus, 586 rows, both routes: arm TS files TS saved Python before (§37 only) 22 14.00% 14.98% C1b alone 8 2.25% 14.98% symbol fix alone 30 25.59% 14.98% both (shipped) 29 23.38% 14.98% The gate is stricter and reduction is higher, because it now measures semantic loss instead of noise. Python, prose and every uncovered-language bucket are unchanged. One file regresses: `src/cli/html-reporter.ts`, whose sole content marker is `TD_PRESERVE:` harvested from a regex literal in the file that implements syntax highlighting for that directive. Its own pattern source is its only structural evidence. Left in deliberately — it fails conservatively (byte-identical fallback, no data loss), it is 1 file in 57, and special-casing it would over-fit the metric to one file in this repository. Two hazard-pinning tests asserted properties this abolishes, and were updated with their findings preserved rather than overwritten: the 0.60 "ceiling for code" (which was the symptom, not a safety property) and C1a's note that the retention gate could never fire for markdown (it now does, and the measurement gate remains the guard for the no-markers case). Known residue, recorded not fixed: `extractSymbols` still harvests from comments, because it is regex over raw content with no lexer — `fn:of` from "pure function of its input", `type:of` from "that class of bug". After the locals fix that is the entire remaining symbol loss on `src/core/engine/index.ts`, 1 of 22. See DECISIONS §40. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes audit C1b and §3.2 — the second half of the
R_structdefect, deferred from Wave 1 because it re-baselines published reduction figures and needed its own measurement pass.It went differently than the audit specified, and the difference is the interesting part.
The audit's proposed fix is inert
max_audit.md §3.2 proposed computing
R_structoverextractContentMarkers, excluding thefilepath:marker that no content transform can destroy, and stated this "would fix both cases at once."It fixes neither. Removing the only marker an item had leaves the before-set empty, and an empty set defaults
R_structback to 1.0 — the identical free 0.40, arriving by a different route. Measured: that change alone was byte-identical across all 586 rows of a frozen 293-file corpus.The free 0.40 comes from the empty-set default, not from
filepath:. That is DECISIONS §33's own argument — "0.0000means 'retained everything' and 'found nothing to look at' indistinguishably" — applied to the score rather than to the gate.Part 1: an unmeasured ratio is excluded, not defaulted
R_structis computed over content markers, and the weight of a ratio whose before-set is empty is redistributed to the ratio that did measure something.For code this makes
S_k = 1 − R_AST, so the maximum symbol loss that can pass the 0.40 gate falls from 66.7% → 40%.When neither ratio measured, retention returns 1.0 and stays silent — that case belongs to the measurement gate (§37, in #3). Having both refuse would attribute the refusal to the wrong question.
Part 2:
extractSymbolscounts semantic surface, not localsPart 1 applied on its own costs 14 TypeScript files and 11.75pp (14.00% → 2.25%). The loss it was guarding against turned out to be almost entirely fictitious:
src/core/engine/index.tssrc/core/hashing/tokenizer.tsNot one exported function, type or interface was lost in either case —
selectElisionRegionsretains signatures by construction. The rule matchedconst|let|varanywhere, so everyconst i,const result,const msginside a function body counted as a semantic symbol on par with an exported function, and body elision is precisely the transform that removes them.So the audit's "you can destroy two-thirds of every symbol in a file and pass" was measuring temporaries inside bodies the caller asked to have elided.
Python is the control, and the asymmetry is what identified the defect. Its extractor never had a locals rule, its measured symbol loss under the same elision is 0.0%, and it is untouched by either half of this change. A safety metric should not depend on which language's extractor happens to harvest block-scoped bindings.
Anchored with
^…/gm: in TS and JS a top-level declaration is a column-0 declaration; indented ones are body content.Measured
Frozen 293-file corpus, 586 rows, both routes, via
tools/corpus-harness:The gate is stricter and reduction is higher (+9.38pp), because it now measures semantic loss instead of noise. Python, prose and every uncovered-language bucket are unchanged.
The one file it costs, and why it stays
src/cli/html-reporter.tsgoes from reducing to falling back. Its sole content marker isdirective:TD_PRESERVE:[^\s&]+)/g,— harvested from a regex literal in the file that implements syntax highlighting for that directive. Its own pattern source is its only structural evidence; eliding it drivesR_structto 0.Left in deliberately: it fails conservatively (fallback, byte-identical output, no data loss), it is 1 file in 57, and special-casing it would over-fit the metric to one file in this repository. Recorded rather than patched.
Two hazard-pinning tests were updated
Both asserted properties this change deliberately abolishes. Findings preserved and supersession stated, not overwritten:
drift-tracker.test.tspinnedS_k === 0.60as "the ceiling for code". That ceiling was the symptom, not a safety property — it existed precisely becauseR_structvoted 1.0 on evidence it never examined. Now 1.0, and the property is stronger:structMeasuredis false, so no number of comments can move the score at all.drift-unwitnessed-elision.test.ts(added in Audit Wave 1: silent data loss, Gateway byte corruption, and a trace that explains #3 for C1a) recorded that the retention gate could never fire for markdown. C1b makes that false — it now fires too. The measurement gate remains the load-bearing guard for the case retention cannot reach: a document with no content markers at all, which is now asserted separately.Verification
npm run typecheck,npm run lint— cleannpx vitest run— 499 passing, 56 files (up from 493)Known residue, recorded not fixed
extractSymbolsstill harvests from comments — it is regex over raw content with no lexer. Measured artifacts:fn:offrom the prose "pure function of its input",type:offrom "that class of bug". After the locals fix this is the entire remaining symbol loss onsrc/core/engine/index.ts(1 of 22, 4.5%). Making extraction comment-aware is a per-language lexing problem and the measured cost does not yet justify it.🤖 Generated with Claude Code