Found by the [gate-potency] audit run before tagging v0.55.0. Not a v0.55 blocker — that release's four surfaces were verified by hand — but the gate does less than its name says.
Measured
check_version_pins.py (the whole of the Version Pin Sweep job) checks:
- intra-workspace path-dep
version = pins
MODULE.bazel module(version = ...)
A release bump actually touches four surfaces:
| surface |
gated? |
path-dep version = pins |
✅ |
MODULE.bazel |
✅ |
Cargo.lock |
❌ no gate |
npm/package.json |
❌ no gate |
artifacts/status.json |
indirectly — regenerated + claim-checked |
Why Cargo.lock specifically has NO backstop
The only --locked in ci.yml is cargo install --locked kani-verifier — installing a tool, not verifying our lockfile. No job runs cargo build/test --locked, so a lock left at the previous version builds and tests green.
This is not hypothetical: during v0.55 assembly the lock had ZERO 0.55.0 entries after the version bump, and it is correct now only because I checked a count by hand. The same defect was caught in the v0.52 cold review, i.e. twice by a human and never by a gate.
Why it matters
Version Pin Sweep is one of the 9 REQUIRED contexts. A reviewer reading "Version Pin Sweep ✅" reasonably concludes the release's versions are consistent. For two of the four surfaces that conclusion is unwarranted — the classic gate-potency shape: the check is potent for what it covers, and its NAME overstates the coverage.
Fix
- Extend
check_version_pins.py to assert Cargo.lock contains the workspace version for every workspace member, and that npm/package.json matches.
- Negative-control it: revert the lock to the previous version, confirm the job goes RED, restore. A gate added without that step is only presumed potent.
- Consider renaming the job to state its scope, or keep the name once the scope matches it.
Related: #923 (codecov, same audit), #910 (what a wired oracle attests).
Found by the [
gate-potency] audit run before tagging v0.55.0. Not a v0.55 blocker — that release's four surfaces were verified by hand — but the gate does less than its name says.Measured
check_version_pins.py(the whole of theVersion Pin Sweepjob) checks:version =pinsMODULE.bazelmodule(version = ...)A release bump actually touches four surfaces:
version =pinsMODULE.bazelCargo.locknpm/package.jsonartifacts/status.jsonWhy Cargo.lock specifically has NO backstop
The only
--lockedinci.ymliscargo install --locked kani-verifier— installing a tool, not verifying our lockfile. No job runscargo build/test --locked, so a lock left at the previous version builds and tests green.This is not hypothetical: during v0.55 assembly the lock had ZERO
0.55.0entries after the version bump, and it is correct now only because I checked a count by hand. The same defect was caught in the v0.52 cold review, i.e. twice by a human and never by a gate.Why it matters
Version Pin Sweepis one of the 9 REQUIRED contexts. A reviewer reading "Version Pin Sweep ✅" reasonably concludes the release's versions are consistent. For two of the four surfaces that conclusion is unwarranted — the classic gate-potency shape: the check is potent for what it covers, and its NAME overstates the coverage.Fix
check_version_pins.pyto assertCargo.lockcontains the workspace version for every workspace member, and thatnpm/package.jsonmatches.Related: #923 (codecov, same audit), #910 (what a wired oracle attests).