Deferred to v0.56 by decision during the v0.55 assembly.
The problem
codecov/patch fails on essentially every PR and has been treated as advisory for dozens of merges. That habit is not free: it is exactly the reflex that once let a red Clippy through as "just codecov" (#683, main went red).
Three facts, measured 2026-08-07:
- There is no
codecov.yml anywhere in the repo. The check runs on codecov's defaults — patch target auto, i.e. changed lines must meet the project average. Nobody chose that threshold.
- The coverage JOB passes.
Rust-test Line Coverage (unit + integration only) → success. Only codecov's external patch status is red.
- VG-009 (shipped this release) explains much of it.
cargo llvm-cov instruments the Rust test suite IN-PROCESS; the execution differentials spawn synth as a separate, uninstrumented process, from other CI jobs, so they emit no profile data at all. synth-backend-aarch64/src/backend.rs reads 41.6 % while being exercised end-to-end constantly.
So an unknown fraction of "untested" lines are executed against wasmtime hundreds of times per run — just not where the instrument can see them. Writing unit tests for those raises a number without adding evidence.
Why it still matters
The rest of the flagged lines genuinely ARE untested, and today they are indistinguishable from the differential-covered ones. That is the same defect shape as #921's unmodeled-op: a signal that says something is wrong but not which, so the only response is to ignore it — and an ignored red is a gate that cannot do its job ([[gate-potency]]: a check everyone is trained to ignore is worse than no check).
Plan
- Measure the split. Run
llvm-cov on a failing lane and classify every flagged line: genuinely-uncovered vs covered-only-by-a-differential. Data, not opinion — this is the step that decides the rest.
- Write the tests the split says are real. Worth having on their own merits.
- Add
codecov.yml stating a threshold someone actually chose, and mark the patch status informational: true EXPLICITLY rather than de-facto. If it is advisory, say so in config so a future reader is not left inferring it from merge habits.
Related: #910 (what a wired oracle attests), VG-009/VG-010 in artifacts/verification-gaps.yaml, #683 (the merge this noise contributed to).
Deferred to v0.56 by decision during the v0.55 assembly.
The problem
codecov/patchfails on essentially every PR and has been treated as advisory for dozens of merges. That habit is not free: it is exactly the reflex that once let a red Clippy through as "just codecov" (#683, main went red).Three facts, measured 2026-08-07:
codecov.ymlanywhere in the repo. The check runs on codecov's defaults — patch targetauto, i.e. changed lines must meet the project average. Nobody chose that threshold.Rust-test Line Coverage (unit + integration only)→success. Only codecov's external patch status is red.cargo llvm-covinstruments the Rust test suite IN-PROCESS; the execution differentials spawnsynthas a separate, uninstrumented process, from other CI jobs, so they emit no profile data at all.synth-backend-aarch64/src/backend.rsreads 41.6 % while being exercised end-to-end constantly.So an unknown fraction of "untested" lines are executed against wasmtime hundreds of times per run — just not where the instrument can see them. Writing unit tests for those raises a number without adding evidence.
Why it still matters
The rest of the flagged lines genuinely ARE untested, and today they are indistinguishable from the differential-covered ones. That is the same defect shape as #921's
unmodeled-op: a signal that says something is wrong but not which, so the only response is to ignore it — and an ignored red is a gate that cannot do its job ([[gate-potency]]: a check everyone is trained to ignore is worse than no check).Plan
llvm-covon a failing lane and classify every flagged line: genuinely-uncovered vs covered-only-by-a-differential. Data, not opinion — this is the step that decides the rest.codecov.ymlstating a threshold someone actually chose, and mark the patch statusinformational: trueEXPLICITLY rather than de-facto. If it is advisory, say so in config so a future reader is not left inferring it from merge habits.Related: #910 (what a wired oracle attests), VG-009/VG-010 in
artifacts/verification-gaps.yaml, #683 (the merge this noise contributed to).