repo-standard-selftest proves the ceilings, not that the gates bite — a mutation lane for the org CI

[bdelanghe]

What "the org CI is good" means today, and what it does not

repo-standard-selftest.yml calls repo-standard.yml, repo-scorecard.yml and repo-release.yml at their declared permission ceilings on every push to main. Green means: the reusable workflows load and their permission union has not crept (#55). It is a real guard and it is the only one.

It does not say that any gate refuses what it claims to refuse. Nothing in this repo runs the standard against a known-bad input and asserts red. docs/agentic-code-hygiene.md rule 3 — a gate's own claim about itself is not evidence — applies to the standard exactly as it applies to the repos that call it. Today the evidence that osv reds on a vulnerable lockfile, that test reds on exit 1, that quality refuses pull_request_target, is the source text.

The obstacle, to verify before designing

A reusable-workflow call job (jobs.<id>.uses) does not accept continue-on-error (claim from the workflow syntax reference; not measured here — verify on a throwaway workflow first). If that holds, a "call the standard and expect red" job reds the selftest run itself, so the negative cases cannot live as sibling jobs behind a verdict.

Options, in order of how much they change:

1. Extract the deciding logic into scripts and test those. The osv triage (SARIF → pass/fail), the corpus builder, the lockfile discovery are inline run: bodies today. Moved to scripts/ under node --test, the mutation cases are ordinary unit tests: an empty SARIF from a failed scanner must red in every mode; a folded docs file must not crash the corpus. This is the .github-private "extract it" pattern (#399) and needs no new GitHub primitive.
2. A fixture caller on a branch whose red is expected, read by a verdict job through the Actions API. Works, but it is a second lane whose red is normal, which trains readers to ignore red.
3. A mutant: input on repo-standard.yml that inverts the exit of the named job. Changes the callee's contract to serve a test; refused unless 1 turns out insufficient.

Recommendation: 1, then re-measure what 1 cannot reach (the if: guards, the continue-on-error on dependency-review), and only then decide on 2.

Also in scope

• The pins: every uses: in the three reusable workflows is a 40-hex SHA (a test in workflows.test.mjs shape).
• conformance-kit-ref is a SHA and resolves.

Consumer

The conformance projection (#381) carries a standard block with the selftest's latest conclusion; when this lane exists, that block carries its conclusion too, and "org CI: good" on the desk means both.