Skip to content

fix(phi-scan): the all-mode sweep reads the bytes git carries, as a union with the walk - #110

Merged
NSchatz merged 1 commit into
mainfrom
phi-scan-git-union
Aug 11, 2026
Merged

fix(phi-scan): the all-mode sweep reads the bytes git carries, as a union with the walk#110
NSchatz merged 1 commit into
mainfrom
phi-scan-git-union

Conversation

@NSchatz

@NSchatz NSchatz commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

pnpm phi-scan with no arguments (the sweep CI runs) enumerated the working tree and nothing else. Where the working tree and the index disagree the walk had the only voice, so the gate could print OK, no hits at exit 0 over tracked content it never opened.

Four states reproduced on dbdf84d

Each at exit 0 with OK, no hits, over a TRACKED file holding a whole synthetic patient identity (name, DOB, SSN by OID, bare-numeric MRN, street, city, postal code, non-555 telecom):

  1. The path is occupied by a DIRECTORY. git ls-files still names the path, so a path-set reconciliation cannot see it either. Only reading the object can.
  2. The path is under a WALK_SKIP_DIRS NAME, matched at any depth (dist, coverage, .cache, ...).
  3. 137 of 140 tracked files absent from the working tree. The floor-of-one only asks that SOMETHING was observed, and a count counts the files that DID exist.
  4. A GITLINK whose working tree is absent.

ccda had NO real unscanned corpus

That is the honest finding rather than a sibling's. Measured on dbdf84d: 140 tracked, 140 enumerated by the walk, the only tracked path no sweeping route opens is the one declared literal exclusion, and all 140 index blobs are byte-identical to the working tree. The four states were reproduced, not found. The sibling "tracked files outside every walk root" shape cannot exist here: this walk has been rooted at the repo root throughout.

What changed

all mode now also scans the stage-0 blob of every tracked path whose bytes the walk did not already read. Deduplication is by CONTENT (git's own blob <len>\0 framing, under the repository's object format), so a clean checkout reads nothing twice and never invokes git cat-file; its one fixed cost is a single git rev-parse --show-object-format per run, two git calls before and three after. Where the two copies of a path differ, BOTH are scanned. That is the EOL axis: with a text attribute or core.autocrlf the index carries LF and the working tree CRLF, and neither form stands in for the other. This repo has no .gitattributes and no core.autocrlf, measured, so the union is free today.

Three refusals, all exit 2, none of which ever prints a link target: a tracked path recorded as a symbolic link or a gitlink, an unmerged path (no single merged blob), and a sweep git cannot give an index for. scripts/check-agent-notes-contract.mjs already refuses the last two states; the shape is taken from that in-repo precedent and goes one step further by naming the OBJECT rather than re-reading the path.

The positive control

The suite copies every tracked file into a throwaway repository, reproduces the clean result over it, then proves the same sweep fires on that corpus with one synthetic marker planted, once on disk (the walk half) and once committed and removed from disk so only the index can reach it (the union half). Without it, OK, no hits is indistinguishable from a scanner that stopped reading.

Review gate

conformance-refuter, two passes, REFUTED then NOT REFUTED.

Pass 1 found one INTRODUCED major inside acceptance criterion 1's own gitlink/non-blob axis: the unmerged axis did not port. --staged spots that state from --raw's status U and a destination mode of 000000, but git ls-files -s reports the path only at stages 1, 2 and/or 3, its records normally at ordinary blob modes. A draft took the first record per path and never read the stage digit it was already capturing, so it scanned stage 1, the merge base, and labelled it as the bytes git carries, and printed OK, no hits at exit 0 over a marker living only in stage 3. The union now keys on the absence of stage 0, which is shape-independent.

This slice wrote a count into prose and had it falsified three times (the routes EXCLUDED_PATHS is consulted on, the functions that consult it, and the number of records an unmerged path carries). Every one was a claim defect and none was a code defect, which is this item class's whole history. refuseUnscannable's "two groups is the ceiling" line went the same way once the union added a third group under it.

CLAUDE.md was at 6 bytes of headroom, so three passages were relocated, not deleted: two were already verbatim under the exact anchor their own bullet points at, and the third was a stale count its own neighbouring imperative forbids.

scripts/verify.sh ccda green, plus pnpm check:agent-notes (which that ladder does not know about) run by hand.

PRE-EXISTING, filed not fixed: in a submodule checkout .git is a FILE, so the walk enumerates and scans it (harmless, and more scanning than claimed); and stages 2 and 3 of an unmerged entry are read by no route, with base identically silent.

…nion with the walk

The repository-wide sweep enumerated the working tree and nothing else. Where
the working tree and the index disagree the walk had the only voice, so the gate
could print `OK, no hits` at exit 0 over tracked content it never opened.

Four such states were reproduced on `dbdf84d`, each over a TRACKED file holding a
whole synthetic patient identity (name, DOB, SSN by OID, bare-numeric MRN,
street, city, postal code, non-555 telecom):

  1. the path occupied by a DIRECTORY. `git ls-files` still names it, so a
     path-set reconciliation cannot see it either; only reading the object can;
  2. the path under a `WALK_SKIP_DIRS` NAME, matched at any depth;
  3. 137 of 140 tracked files absent from the working tree. The floor-of-one
     only asks that SOMETHING was observed;
  4. a GITLINK whose working tree is absent.

`ccda` HAD NO REAL UNSCANNED CORPUS, and that is the honest finding rather than a
sibling's. Measured on `dbdf84d`: 140 tracked, 140 enumerated by the walk, the
only tracked path no sweeping route opens is the one declared literal exclusion,
and all 140 index blobs are byte-identical to the working tree. The four states
were reproduced, not found.

`all` mode now also scans the stage-0 blob of every tracked path whose bytes the
walk did not already read. Deduplication is by CONTENT (git's own `blob <len>\0`
framing, under the repository's object format), so a clean checkout reads nothing
twice and never invokes `git cat-file`; its one fixed cost is a single
`git rev-parse --show-object-format` per run, two git calls before this change
and three after. A path whose two copies differ has BOTH scanned rather than one
standing in for the other. That is the EOL axis: this repo has no
`.gitattributes` and no `core.autocrlf`, measured, so the union is free today.

Three refusals, all exit 2. A tracked path recorded as a symbolic link or a
gitlink carries no content there. An UNMERGED path has no single merged blob and
is refused under its own message. And a sweep cannot run when git will not name
the index or names it empty. None of them ever prints a link target.
`scripts/check-agent-notes-contract.mjs` refuses the last two states; the shape
is taken from that in-repo precedent, and goes one step further by naming the
OBJECT rather than re-reading the path.

THE UNMERGED AXIS DID NOT PORT, and the refuter caught it. `--staged` spots that
state from `--raw`'s status `U` and a destination mode of `000000`, but
`git ls-files -s` reports the path only at stages 1, 2 and/or 3 and never at
stage 0, its records normally at ordinary blob modes, so the mode rule cannot see
it. A draft took the first record per path and never read the stage digit it was
already capturing: it scanned stage 1, the MERGE BASE, and reported it as the
bytes git carries, and it printed `OK, no hits` at exit 0 over a marker living
only in stage 3. The union now keys on the ABSENCE OF STAGE 0, which is
shape-independent. The stage-3 silence is PRE-EXISTING and closing it fell out of
making the docblock's own authoritative sentence true; it is not credited here.

THIS SLICE WROTE A COUNT INTO PROSE AND HAD IT FALSIFIED THREE TIMES: the routes
`EXCLUDED_PATHS` is consulted on, the functions that consult it, and the number
of records an unmerged path carries (add/add and modify/delete carry two, not
three, and a symlink-versus-file conflict carries a `120000` one). Every one was
a claim defect and none was a code defect. The file's own WRITE NO COUNT rule is
the whole remedy; `refuseUnscannable`'s "two groups is the ceiling" line went the
same way when the union added a third group under it.

The order of the `git` calls in `buildTargetsForAll` is load-bearing: they run
after the walk and before the first read, which is what makes the
enumerate-then-read window a deterministic hook for the TOCTOU cases. A first
draft read the index in `main` and silently closed that window.

Three pinned cases changed their message and one was rebuilt. A missing or empty
index now refuses the SWEEP before any byte is read instead of refusing the
vanish TOLERANCE at the first bad read; the observed-nothing floor was re-pinned
against an index holding only the literally excluded path, so the branch is kept
rather than deleted as unreachable.

A positive control lands with it: the suite copies every tracked file into a
throwaway repo, reproduces the clean result, then proves the same sweep fires on
that corpus with one synthetic marker, once on disk and once reachable only
through git. A clean report is a decision, not an absence.

`CLAUDE.md` was at 6 bytes of headroom, so three passages were RELOCATED, not
deleted. Two were already verbatim in `documentation/agent-notes.md`, under the
exact anchor their own bullet points at. The third, a `44 of 140` count its own
neighbouring imperative forbids and which this slice made stale, is present there
as the census and the repo-rooted argument in different words, not verbatim.

PRE-EXISTING, filed not fixed: in a submodule checkout `.git` is a FILE, so the
walk enumerates and scans it. Harmless, and more scanning than claimed. Stages 2
and 3 of an unmerged entry are read by no route; base is identically silent.
@NSchatz
NSchatz force-pushed the phi-scan-git-union branch from a9918cc to 0ab229b Compare August 11, 2026 09:45
@NSchatz
NSchatz merged commit d3807af into main Aug 11, 2026
8 checks passed
@NSchatz
NSchatz deleted the phi-scan-git-union branch August 11, 2026 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant