Skip to content

Lane A (A.17-A.18): the deterministic comparator, and a corpus that is provably what it claims - #13

Merged
tom-snyder merged 1 commit into
mainfrom
feat/phase3-lane-a-comparator
Aug 22, 2026
Merged

Lane A (A.17-A.18): the deterministic comparator, and a corpus that is provably what it claims#13
tom-snyder merged 1 commit into
mainfrom
feat/phase3-lane-a-comparator

Conversation

@tom-snyder

Copy link
Copy Markdown
Member

Summary

A.17 implements the version comparator the whole lane exists to feed; A.18 reviewed it and returned FAIL — 3 blockers, 6 majors. Two more rounds followed. This is the result.

What it refuses matters more than what it covers

internal/match implements dpkg, rpmvercmp and apk, and refuses everything else by name: npm, PyPI, Go, Maven, NuGet, Cargo — and semver as a scheme. No fallback anywhere, and a test asserts the refusal also returns 0, so a caller that ignores the error gets nothing that looks like an answer.

That's the important property. A silently wrong CVE match either tells someone they're safe when they aren't, or floods them until they stop reading any of it. Refusing is the only honest third option.

The three scheme implementations were compared statement-by-statement against dpkg's verrevcmp/order and rpm's rpmvercmp and found faithful — every defect was in the layers above the comparison.

Three blockers, and one fix that over-corrected

  • An empty advisory cache reported a clean host. AssertNotSilentlyClean never read PackagesWithNoAdvisoryData — the field whose own doc says it exists for that. The silent-clean failure mode, inside the guard named for preventing it.
  • An epoch spelled on one side only silently cleared a live vulnerability, on RHEL's routine shape. Now a typed refusal, with ordering left untouched and a test pinning that the two rules can't later be collapsed into one.
  • The identity check accepted a name it then failed to look up — and the fix over-corrected into a second silent clean, because the purl name won unconditionally, so a purl naming a different package was accepted. The rule is now the one already applied to ecosystem and version: identical-after-ASCII-fold is a spelling and canonicalises; a different name is a conflict.

The test couldn't have caught that: it varied only the reported name across spellings of one package, so it exercised the fold and never the disagreement. Its doc now says what it doesn't cover, and the new test varies the purl name — the axis that never moved.

A refused range also still decided by absence, re-arming the exact backport false positive the lane exists to defeat, while the doc asserted the opposite invariant.

The corpus is now true by construction, not by wording

Twice the header overstated what had been transcribed — the second time in the same section. The first instance was the sharpest form of it: the rpm corpus stopped exactly where the implementation would fail, so the provenance claim made a truncated set read as exhaustive.

That's the third time on this project a provenance or compliance claim has been corrected and stayed false. Patching the wording a third time wasn't the fix.

So provenance is now structural: every vector is TRANSCRIBED (carrying file and line) or AUTHORED (carrying the rule it encodes), with disjoint required fields and no third state. Completeness claims are data carrying their count, and a test fails in either direction, fails on duplicate loci so a short corpus can't be padded, and fails if a transcribed vector's source carries no counted claim.

The corpus is the replay now, not a summary of it:

Source Rows Result
rpm tests/rpmvercmp.at all 91 active RPMVERCMP lines 86 pass, 5 refused, 0 wrong
dpkg scripts/t/Dpkg_Version.t all 43 __DATA__ rows 43 pass
apk-tools test/unit/version.data all 738 ordering rows + 31 validity 674 pass, 64 refused, 0 wrong

Three vectors falsely cited to Dpkg_Version.t are re-tagged AUTHORED. rpm_compare.go claimed four separator-only vectors; there are five, and it no longer makes a completeness claim at all.

apk R7a justified a refusal by asserting a tokeniser mechanism that is false against the file it cited. src/version.c actually falls back to string sort on a leading zero, and the published fixture line 735 has 8.2.0015 < 8.2.002 — which numeric comparison orders the other way. Justified from the real text now.

AssertNotSilentlyClean couldn't be made the sufficient statistic its doc promised, so the doc was narrowed and the over-promising sentence deleted rather than softened — and every sentence of the new contract, the negative ones included, is asserted by a test.

Testing

gofmt / vet / build   clean
go test -count=1      green across 21 packages, no t.Skip added

Determinism proven across six separate OS processes — and it earned that: one mutation was caught by child 2 and not child 1, which is precisely why a single-process repeat wouldn't have done. Every fix verified RED against the pre-fix behaviour before being trusted.

…s now

provably what it claims

A.17 implements the version comparator the whole lane exists to feed, and A.18
reviewed it. A.18 returned FAIL with three blockers and six majors; two more
rounds followed. This is the result.

What the comparator is, and what it refuses

S1 corrected the original spec to exactly this: a VERSION COMPARATOR answers
"is this installed package inside a vulnerable range", deterministically and
with no model. internal/match implements three schemes -- dpkg, rpmvercmp and
apk -- and REFUSES everything else by name: npm, PyPI, Go, Maven, NuGet, Cargo,
and semver AS A SCHEME. There is no fallback anywhere, and a test asserts the
refusal also returns 0 so a caller that ignores the error gets nothing that
looks like an answer.

That matters more than the coverage does. A silently wrong CVE match either
tells someone they are safe when they are not, or floods them until they stop
reading any of it. Refusing is the only honest third option.

The scheme implementations were compared statement-by-statement against dpkg
verrevcmp/order and rpm rpmvercmp and found faithful. The defects were all in
the layers ABOVE the comparison.

Three blockers, and one fix that over-corrected

- An empty advisory cache reported a CLEAN HOST. AssertNotSilentlyClean never
  read PackagesWithNoAdvisoryData -- the field whose own doc says it exists for
  that. The silent-clean failure mode, inside the guard named for preventing it.
- An epoch spelled on one side only silently cleared a live vulnerability, on
  RHEL's routine shape. Now a typed refusal, in the direction that would push
  the installed version out of range, with ordering left untouched and a test
  pinning that the two rules cannot later be collapsed into one.
- The identity check accepted a name spelling it then failed to look up. The
  fix for that OVER-CORRECTED into a second silent clean -- the purl name won
  unconditionally, so a purl naming a different package was accepted. The rule
  is now the one already applied to ecosystem and version: identical after
  ASCII fold is a spelling and canonicalises; a different name is a conflict.
  And the surviving name is the CANONICAL form rather than either producer's.

The test could not have caught it: it varied only the REPORTED name across
spellings of one package, so it exercised the fold and never the disagreement.
Its doc now states what it does not cover, so it cannot be counted as covering
it. The new test varies the PURL name against a fixed reported name -- the axis
that never moved.

A refused range also still decided BY ABSENCE, re-arming the exact backport
false positive this lane exists to defeat, while the doc asserted the opposite
invariant. A refusal now blocks its whole group.

The corpus is now true by construction, not by wording

Twice the corpus header overstated what had been transcribed -- the second time
in the same section -- and the first instance was the sharpest form of it: the
rpm corpus stopped exactly where the implementation would fail, so the
provenance claim made a truncated set read as exhaustive. That is the third
time on this project a provenance or compliance claim has been corrected and
stayed false, and patching the wording a third time was not the fix.

So provenance is now structural. Every vector is TRANSCRIBED (carrying its file
and line) or AUTHORED (carrying the rule it encodes); the two sets have
disjoint required fields and there is no third state. Completeness claims are
DATA carrying their count, and a test fails in EITHER direction, fails on
duplicate loci so a short corpus cannot be padded, and fails if a transcribed
vector's source carries no counted claim.

The corpus IS the replay now, not a summary of it: all 91 active RPMVERCMP
lines from tests/rpmvercmp.at (86 pass, 5 refused, 0 wrong), all 43 rows of
scripts/t/Dpkg_Version.t, and -- beyond what was asked -- all 738 ordering rows
of apk-tools test/unit/version.data plus its 31-row validity block. Three
vectors falsely cited to Dpkg_Version.t are re-tagged AUTHORED with the
deb-version(7) rule they encode. rpm_compare.go's header claimed four
separator-only vectors; there are five, and it no longer makes a completeness
claim at all -- it defers to the counted one.

apk R7a justified a refusal by asserting a tokeniser mechanism that is FALSE
against the file it cited. src/version.c actually falls back to string sort on
a leading zero, and the published fixture line 735 has 8.2.0015 < 8.2.002 --
which numeric comparison orders the other way. Justified from the real text
now; R8's rationale, which asserted a token kind the file does not declare,
takes the honest form instead: the ordering is unmodelled, so it is refused.

AssertNotSilentlyClean could not be made the sufficient statistic its doc
promised, so the doc was NARROWED and the over-promising sentence on
CoverageReport.Complete was deleted rather than softened -- and every sentence
of the new contract, the negative ones included, is asserted by a test.

Evidence: gofmt, vet, build clean; go test -count=1 ./... green across 21
packages; no t.Skip added anywhere in internal/match. Determinism proven across
SIX separate OS processes over a 53-package corpus -- and it earned that: one
mutation was caught by child 2 and not child 1, which is why a single-process
repeat would not have done. Every fix verified RED against the pre-fix
behaviour before being trusted.
@tom-snyder
tom-snyder merged commit 74dee08 into main Aug 22, 2026
5 checks passed
@tom-snyder
tom-snyder deleted the feat/phase3-lane-a-comparator branch August 22, 2026 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant