strict:true with no merge queue makes every concurrent PR's heavy CI dead work

[avrabe]

The mechanism

main enforces required_status_checks.strict = true ("require branches to be up to date before merging") and there is no merge queue (gh api repos/pulseengine/spar/rulesets returns empty).

Consequence: only the first PR to merge can use the CI results it already has. The moment it lands, every other open PR is BEHIND. Updating a branch creates a new commit, a new commit fires synchronize, and synchronize re-runs all 18 required contexts. So for N concurrent PRs, N−1 full CI cycles are discarded — every round.

This is not hypothetical; it is the current state of the repo.

Measured cost (2026-07-30)

Runner topology — 12 runner processes on one host, org-shared:

label	runners
lean-mem	1, 2, 3, 4
rust-cpu	5, 6, 7, 9, 10, 11, 12
light	8
netns	12
podman	9 only

Mutation Testing requires lean-mem, so it draws on a 4-runner pool, and it has no timeout-minutes (GitHub's 360-minute default applies). Successful durations over the last 40 CI runs: n=15, min 35.2 min, median 68.9 min, max 118.4 min.

With 7 PRs open, 4 concurrent cargo mutants -p spar-analysis --jobs 4 runs were executing on all 4 lean-mem runners at once — on a single physical host, which is precisely the contention the job's own comment warns about (# lean-mem — many parallel cargo invocations, RAM pressure under -j 4). Under that contention the run on main reached 170 minutes, 44% past the historical maximum.

Six of those seven runs are dead work by construction.

Cancelling doomed runs does not help: the freed runner is claimed by the next queued — equally doomed — mutation job within seconds. Verified by doing it.

Secondary finding: the podman runner is not reserved

Trace-Topology Fixture Generation needs runs-on: [self-hosted, linux, x64, podman], and exactly one runner carries podman (runner-9). But runner-9 is also in the general rust-cpu pool, so any ordinary Clippy/Test/Proptest job can occupy the only host the fixture workflow can ever use. A workflow_dispatch of it sat queued 25+ minutes behind unrelated Rust work. Not a deadlock — the label set does exist — but the scarce uniquely-labelled runner is unprotected.

Options

1. Enable a GitHub merge queue on main. This is the mechanism designed for exactly this: PRs are tested speculatively against the projected merge result, batched, and merged when green — so N PRs cost ~1 CI cycle instead of N. Requires adding a merge_group trigger to the workflows.
2. Drop strict and rely on the checks plus Verification Gate to catch semantic conflicts. Cheaper, but reintroduces the "both PRs green, merged result broken" risk that strict exists to prevent.
3. Keep strict, but stop paying for it: don't run the heavy tier on a PR until it is next in line to merge (e.g. gate the heavy jobs behind a label such as ready-to-merge). Keeps the guarantee, removes the dead work.
4. Give podman its own runner (or remove rust-cpu from runner-9) so the fixture workflow cannot be starved by ordinary Rust jobs.

Option 1 addresses the root cause; option 4 is independent and cheap.

Not doing anything unilaterally

Each of these changes branch-protection or runner policy, which is a maintainer decision — filing with the measurements rather than picking one.

Related: #379 (path filter misses rivet.yaml, so pure-manifest PRs pay the full heavy tier — compounds this).

[avrabe]

A second cost, measured today: the pool is org-shared, so spar is not the only queue

The body above analyses contention within spar. That undercounts it. Measured at
2026-07-30T20:07:29Z:

runners busy	11 of 12 (only pulseengine-ci-01-8, the light runner, was free)
spar's own self-hosted jobs running	3 (Mutation Testing on runners 1, 3, 4)
active runs, org-wide	19 — loom 5, synth 4, spar 8, gale 1, rivet 1

So eight busy runners — including runner-9, the only host carrying the podman
label — were executing work belonging to other repositories. spar's queue depth
is a function of loom's and synth's schedules, not only its own.

The concrete consequence, same window: the trace-fixtures dispatch
(run 30575158798)
was queued from 19:31:02 to 20:07:16 — 36m14s — before runner-9 freed. Nothing
spar did caused that wait and nothing spar could do would shorten it.

That matters for this issue in two directions:

• It strengthens the case. Every discarded CI cycle is not just spar's own
  wall-clock, it is capacity taken from four other repos on the same 12-runner host.
  N−1 wasted rounds × ~70 min of 4-way-parallel cargo mutants is an org-level cost.
• It bounds what a merge queue fixes. A merge queue removes the self-inflicted
  N−1 re-runs. It does not help the podman label having cardinality 1, which is a
  separate single-point bottleneck worth its own issue if trace-fixtures becomes a
  required context.

Interim mitigation (no gate change)

Until a merge queue is a maintainer decision, I am landing the remaining PRs as one
integration branch rather than six sequential merges: merge the branches together,
push once, let all 18 required contexts evaluate the exact union that will land, merge
that. One round instead of six.

This deliberately does not touch branch protection, does not use --admin, and does
not reduce what is checked — the bytes that land are the bytes that were gated. What it
costs is bisectability: a red combined run does not say which branch caused it. That is
an acceptable trade for six PRs that git merge-tree shows to be pairwise
conflict-free, and it is strictly better coverage for the three that interact
semantically through the rivet artifact graph (#368, #373, #374 all rewrite
artifacts/*.yaml / rivet.yaml) — sequential merges test that interaction one edge at
a time; the union tests it at once.