Skip to content

Recurring N/A: witness MC/DC (feature-loop step 5) skipped four releases running — record a standing exemption or retire it #912

Description

@avrabe

What this is

The pulseengine-feature-loop skill's own rule, verbatim:

Recurring N/A is a backlog item, not an exemption. Steps 5 and 6 are
conditional ("if the feature lands…"), which makes them easy to wave off. If you
mark either one N/A for the same reason three features running, file it (an
issue / report-tool-friction) — that pattern is exactly how a real-flight
MC/DC gap and a missing attestation chain stayed hidden across ~20 features.

Step 5 is witness — MC/DC truth table on the Wasm artifact. It has been
marked N/A on synth for four consecutive releases (v0.51, v0.52, v0.53,
v0.54), for the same reason each time. The rule's threshold is three. This
issue is the "file it", opened by the v0.55 traceability lane.

I am deliberately not deciding the answer here. The rule says file it, not
resolve it, and the resolution is a project-level call about what synth's
structural-coverage evidence is supposed to be.

Why it keeps coming back

The stated reason is structural rather than circumstantial, which is exactly
what makes it worth recording rather than re-skipping: synth CONSUMES Wasm,
it does not EMIT it.
witness measures MC/DC by instrumenting the branches of
a Wasm component. synth's output is an ARM / RISC-V / AArch64 ELF. There is no
Wasm artifact produced by a synth feature for witness to instrument, so "run
witness on the relevant Wasm component" has no referent for most synth work.

That is a plausible permanent exemption. It is not obviously a complete
one, which is why it should be written down instead of re-derived from scratch
every release by whoever runs the loop.

The part that is already analysed, and points the other way

synth already has a rivet artifact for precisely the gap this exemption would
paper over: VCR-COV-001 (artifacts/verified-codegen-roadmap.yaml,
status: proposed), which tracks #396 — "MC/DC source-to-object
traceability: synth's branch folding/splitting changes the decision set witness
measures on wasm" (closed as an issue; the rivet artifact carries the work).

Its content is the argument that witness is not simply irrelevant to synth:

  • synth's lowering changes the decision set in three ways that are all
    present today — fold-to-predication (selectCSEL/cmov, the cmp→br_if
    fusion), eliminate-provably-constant-branch, and split-into-N-object-branches
    (i64 compare, br_table ladders).
  • For DO-178C Level A / ISO 26262 ASIL D that is the source-to-object
    structural-coverage obligation
    (DO-178C 6.4.4.2). Wasm-level MC/DC measured
    by witness does not, on its own, represent the object code synth emits.
  • VCR-DEC-003 (status: implemented) already settles the contract: the
    synth-provenance-v1 branch-transformation map, keyed on the op_offsets
    side-table synth's decoder already records, with witness's reconciler
    (witness#130, witness object-disposition) mapping each kind to an
    object-verdict.

So the honest position is somewhere between "witness has no artifact here" and
"witness is a gate here", and nobody has written down which.

Candidate resolutions (not a recommendation — pick one)

  1. Standing exemption, recorded. Declare step 5 permanently N/A for synth
    with the "synth compiles Wasm, it does not emit components" rationale
    written into a durable place (rivet artifact and/or the release checklist),
    so future releases cite the exemption instead of silently re-deciding it.
    Cheapest. Its cost is that the DO-178C source-to-object obligation above
    stops being visible at loop time.
  2. Conditional exemption with a named trigger. Same as (1), but the
    exemption is scoped — "N/A until the synth-provenance-v1 emitter lands",
    with VCR-COV-001 as the thing that retires it. Step 5 then becomes "emit
    the provenance map and reconcile", not "run witness on a component we do not
    produce".
  3. Reframe step 5 for consumer-shaped projects. synth's real structural
    evidence is object-level: the frozen differentials, the execution oracles,
    the no-wildcard selector enumeration. If step 5 means "structural coverage
    evidence for what this feature added", synth has an answer that is not
    witness — and the skill, not synth, is where that belongs
    (report-tool-friction against the plugin).

Acceptance

Whatever is chosen: the outcome is recorded somewhere a release reads, so
the fifth consecutive N/A cites a decision rather than repeating a judgement
call. Closing this issue with "it is genuinely N/A" is a perfectly good
outcome — but only once that sentence exists in the repo.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions