fix(phi-scan): the all-mode sweep reads the bytes git carries, as a union with the walk - #110
Merged
Conversation
…nion with the walk
The repository-wide sweep enumerated the working tree and nothing else. Where
the working tree and the index disagree the walk had the only voice, so the gate
could print `OK, no hits` at exit 0 over tracked content it never opened.
Four such states were reproduced on `dbdf84d`, each over a TRACKED file holding a
whole synthetic patient identity (name, DOB, SSN by OID, bare-numeric MRN,
street, city, postal code, non-555 telecom):
1. the path occupied by a DIRECTORY. `git ls-files` still names it, so a
path-set reconciliation cannot see it either; only reading the object can;
2. the path under a `WALK_SKIP_DIRS` NAME, matched at any depth;
3. 137 of 140 tracked files absent from the working tree. The floor-of-one
only asks that SOMETHING was observed;
4. a GITLINK whose working tree is absent.
`ccda` HAD NO REAL UNSCANNED CORPUS, and that is the honest finding rather than a
sibling's. Measured on `dbdf84d`: 140 tracked, 140 enumerated by the walk, the
only tracked path no sweeping route opens is the one declared literal exclusion,
and all 140 index blobs are byte-identical to the working tree. The four states
were reproduced, not found.
`all` mode now also scans the stage-0 blob of every tracked path whose bytes the
walk did not already read. Deduplication is by CONTENT (git's own `blob <len>\0`
framing, under the repository's object format), so a clean checkout reads nothing
twice and never invokes `git cat-file`; its one fixed cost is a single
`git rev-parse --show-object-format` per run, two git calls before this change
and three after. A path whose two copies differ has BOTH scanned rather than one
standing in for the other. That is the EOL axis: this repo has no
`.gitattributes` and no `core.autocrlf`, measured, so the union is free today.
Three refusals, all exit 2. A tracked path recorded as a symbolic link or a
gitlink carries no content there. An UNMERGED path has no single merged blob and
is refused under its own message. And a sweep cannot run when git will not name
the index or names it empty. None of them ever prints a link target.
`scripts/check-agent-notes-contract.mjs` refuses the last two states; the shape
is taken from that in-repo precedent, and goes one step further by naming the
OBJECT rather than re-reading the path.
THE UNMERGED AXIS DID NOT PORT, and the refuter caught it. `--staged` spots that
state from `--raw`'s status `U` and a destination mode of `000000`, but
`git ls-files -s` reports the path only at stages 1, 2 and/or 3 and never at
stage 0, its records normally at ordinary blob modes, so the mode rule cannot see
it. A draft took the first record per path and never read the stage digit it was
already capturing: it scanned stage 1, the MERGE BASE, and reported it as the
bytes git carries, and it printed `OK, no hits` at exit 0 over a marker living
only in stage 3. The union now keys on the ABSENCE OF STAGE 0, which is
shape-independent. The stage-3 silence is PRE-EXISTING and closing it fell out of
making the docblock's own authoritative sentence true; it is not credited here.
THIS SLICE WROTE A COUNT INTO PROSE AND HAD IT FALSIFIED THREE TIMES: the routes
`EXCLUDED_PATHS` is consulted on, the functions that consult it, and the number
of records an unmerged path carries (add/add and modify/delete carry two, not
three, and a symlink-versus-file conflict carries a `120000` one). Every one was
a claim defect and none was a code defect. The file's own WRITE NO COUNT rule is
the whole remedy; `refuseUnscannable`'s "two groups is the ceiling" line went the
same way when the union added a third group under it.
The order of the `git` calls in `buildTargetsForAll` is load-bearing: they run
after the walk and before the first read, which is what makes the
enumerate-then-read window a deterministic hook for the TOCTOU cases. A first
draft read the index in `main` and silently closed that window.
Three pinned cases changed their message and one was rebuilt. A missing or empty
index now refuses the SWEEP before any byte is read instead of refusing the
vanish TOLERANCE at the first bad read; the observed-nothing floor was re-pinned
against an index holding only the literally excluded path, so the branch is kept
rather than deleted as unreachable.
A positive control lands with it: the suite copies every tracked file into a
throwaway repo, reproduces the clean result, then proves the same sweep fires on
that corpus with one synthetic marker, once on disk and once reachable only
through git. A clean report is a decision, not an absence.
`CLAUDE.md` was at 6 bytes of headroom, so three passages were RELOCATED, not
deleted. Two were already verbatim in `documentation/agent-notes.md`, under the
exact anchor their own bullet points at. The third, a `44 of 140` count its own
neighbouring imperative forbids and which this slice made stale, is present there
as the census and the repo-rooted argument in different words, not verbatim.
PRE-EXISTING, filed not fixed: in a submodule checkout `.git` is a FILE, so the
walk enumerates and scans it. Harmless, and more scanning than claimed. Stages 2
and 3 of an unmerged entry are read by no route; base is identically silent.
NSchatz
force-pushed
the
phi-scan-git-union
branch
from
August 11, 2026 09:45
a9918cc to
0ab229b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pnpm phi-scanwith no arguments (the sweep CI runs) enumerated the working tree and nothing else. Where the working tree and the index disagree the walk had the only voice, so the gate could printOK, no hitsat exit 0 over tracked content it never opened.Four states reproduced on
dbdf84dEach at
exit 0withOK, no hits, over a TRACKED file holding a whole synthetic patient identity (name, DOB, SSN by OID, bare-numeric MRN, street, city, postal code, non-555 telecom):git ls-filesstill names the path, so a path-set reconciliation cannot see it either. Only reading the object can.WALK_SKIP_DIRSNAME, matched at any depth (dist,coverage,.cache, ...).ccdahad NO real unscanned corpusThat is the honest finding rather than a sibling's. Measured on
dbdf84d: 140 tracked, 140 enumerated by the walk, the only tracked path no sweeping route opens is the one declared literal exclusion, and all 140 index blobs are byte-identical to the working tree. The four states were reproduced, not found. The sibling "tracked files outside every walk root" shape cannot exist here: this walk has been rooted at the repo root throughout.What changed
allmode now also scans the stage-0 blob of every tracked path whose bytes the walk did not already read. Deduplication is by CONTENT (git's ownblob <len>\0framing, under the repository's object format), so a clean checkout reads nothing twice and never invokesgit cat-file; its one fixed cost is a singlegit rev-parse --show-object-formatper run, two git calls before and three after. Where the two copies of a path differ, BOTH are scanned. That is the EOL axis: with atextattribute orcore.autocrlfthe index carries LF and the working tree CRLF, and neither form stands in for the other. This repo has no.gitattributesand nocore.autocrlf, measured, so the union is free today.Three refusals, all exit 2, none of which ever prints a link target: a tracked path recorded as a symbolic link or a gitlink, an unmerged path (no single merged blob), and a sweep git cannot give an index for.
scripts/check-agent-notes-contract.mjsalready refuses the last two states; the shape is taken from that in-repo precedent and goes one step further by naming the OBJECT rather than re-reading the path.The positive control
The suite copies every tracked file into a throwaway repository, reproduces the clean result over it, then proves the same sweep fires on that corpus with one synthetic marker planted, once on disk (the walk half) and once committed and removed from disk so only the index can reach it (the union half). Without it,
OK, no hitsis indistinguishable from a scanner that stopped reading.Review gate
conformance-refuter, two passes,REFUTEDthenNOT REFUTED.Pass 1 found one
INTRODUCEDmajor inside acceptance criterion 1's own gitlink/non-blob axis: the unmerged axis did not port.--stagedspots that state from--raw's statusUand a destination mode of000000, butgit ls-files -sreports the path only at stages 1, 2 and/or 3, its records normally at ordinary blob modes. A draft took the first record per path and never read the stage digit it was already capturing, so it scanned stage 1, the merge base, and labelled it as the bytes git carries, and printedOK, no hitsat exit 0 over a marker living only in stage 3. The union now keys on the absence of stage 0, which is shape-independent.This slice wrote a count into prose and had it falsified three times (the routes
EXCLUDED_PATHSis consulted on, the functions that consult it, and the number of records an unmerged path carries). Every one was a claim defect and none was a code defect, which is this item class's whole history.refuseUnscannable's "two groups is the ceiling" line went the same way once the union added a third group under it.CLAUDE.mdwas at 6 bytes of headroom, so three passages were relocated, not deleted: two were already verbatim under the exact anchor their own bullet points at, and the third was a stale count its own neighbouring imperative forbids.scripts/verify.sh ccdagreen, pluspnpm check:agent-notes(which that ladder does not know about) run by hand.PRE-EXISTING, filed not fixed: in a submodule checkout
.gitis a FILE, so the walk enumerates and scans it (harmless, and more scanning than claimed); and stages 2 and 3 of an unmerged entry are read by no route, with base identically silent.