MC/DC gate is a loose floor, not a ratchet — it passed a 13→9 full-MC/DC regression

[avrabe]

The finding

PR #115's MC/DC job reported pass. Comparing it to main's last successful run tells a different story:

metric	main (run 29480398404)	#115	Δ
decisions_total	1331	1367	+36
conditions_proved	292	343	+51
decisions_full_mcdc	13	9	−4

The gate's floors are conditions_proved >= 155 and decisions_full_mcdc >= 3, against actuals of 343 and 9. The slack is ~2× and 3×, so the gate cannot detect a regression — it passed a 30% drop in the strongest metric without comment.

This is the anti-pattern the feature-loop skill names explicitly: "Trust the truth table, not the percentage." A floor that far below the actual is a green light that carries almost no information.

Two separate problems

1. The regression itself. FEAT-070/FEAT-064 (PR #115) converted several single-condition decisions into conjunctions — e.g. .filter(|a| a.func_index == f.abs_index && !is_module_scoped(&a.code)), and the guard peephole's shape match went from 2 arms to 3. A 1-condition decision reaches full MC/DC with 2 vectors; a 2-condition conjunction needs 3. Plausible that previously-full decisions became partial. Needs the truth-table gap rows read, not inferred — the scry-mcdc-viz artifact from the #115 run has them.

2. The gate design. The floors are constants set once and never raised, so every improvement silently widens the slack. They should be a ratchet: assert against the last known-good value (or last-good minus a small tolerance), so a drop fails and an improvement raises the bar. conditions_proved is the easy one — 343 is a genuine improvement and should be locked in. decisions_full_mcdc should be restored to ≥13 before being ratcheted, not pinned at the regressed 9.

Suggested work

1. Read the gap rows in the v3.3 plan + FEAT-070/FEAT-064: agent-verifiable groundwork #115 scry-mcdc-viz artifact; identify which 4 decisions lost full MC/DC.
2. Add the missing vectors (or justify each loss in the truth table).
3. Convert crates/scry-mcdc/mcdc-gate.sh from fixed floors to a ratchet with a checked-in baseline file, so the numbers cannot silently drift apart from the gate — the same discipline claims.yaml applies to the README.

Found while running the feature loop on #115; reported rather than merged past.

[avrabe]

Investigation: the 13→9 attribution was wrong — the metric is measuring the wrong population

I downloaded the scry-mcdc-viz artifact from the #115 run and attributed all 1,508 decisions to their origin. (main's artifact had already expired, so a direct diff wasn't possible — that's a separate retention gap.)

Where the decisions actually live:

origin	decisions	of which full MC/DC
Rust stdlib (library/core, library/alloc)	733	6
lib.rs (mixed; only ~72 are scry-core-sized line numbers)	258	1
binary_reader.rs (wasmparser)	173	0
malloc.c (dlmalloc)	55	1
everything else (parser.rs, types.rs, imports.rs, …)	~289	3
total	1508	11

Statuses across the population: unreached 842, no 489, partial 166, full 11.

The full-MC/DC decisions are almost entirely NOT ours — library/core/src/ptr/mod.rs (×3), library/alloc/src/raw_vec/mod.rs, library/core/src/iter/adapters/map.rs, library/core/src/slice/iter/macros.rs, malloc.c:4925 (realloc). Exactly one is in scry's own source: lib.rs:7036, which is Operator::Select => "select" — a string-mapping match arm in op_report_name, not analysis logic.

Two corrections to my original report

1. I retract the attribution. A ±4 swing in decisions_full_mcdc is dominated by stdlib/allocator/dependency code whose exercised paths shift whenever scry's own code changes (different Vec/BTreeMap usage ⇒ different raw_vec/ptr decisions reached). There is no evidence the drop came from FEAT-070/FEAT-064's branches, and the 13→9 should not block PR v3.3 plan + FEAT-070/FEAT-064: agent-verifiable groundwork #115.

2. The real finding is worse than the one I filed. decisions_full_mcdc counts the entire compiled wasm, so it cannot say anything about scry's own MC/DC — and by this measure scry has essentially zero full MC/DC on its analysis logic. The conditions_proved figure (292 → 343 on v3.3 plan + FEAT-070/FEAT-064: agent-verifiable groundwork #115) is the more meaningful of the two, since the new conditions being proved are attributable to the new code being exercised.

Re-scoped fix (supersedes step 3 of the original)

• Do not ratchet decisions_full_mcdc as-is — a ratchet on a stdlib-dominated number will fail spuriously on unrelated changes.
• Attribute first, then gate. The gate should filter decisions to scry's own crates before computing either metric. Until it does, ratcheting is premature.
• Then ratchet the attributed numbers against a checked-in baseline.
• Fix the CI comment's claim while there: it says the pipeline runs "over the REAL analyzer core" and that "a code change that drops a proved condition turns the build red". The first is true of the instrumentation but misleading about the metric; the second does not hold at a floor of 3 vs an actual of 9. Same discipline as claims.yaml — the claim should match what the gate enforces.
• Consider extending artifact retention for the MC/DC viz, so a baseline diff is possible at all (main's was already gone after ~3 weeks).