test: coverage.enforce runs at 83-86% of its 180s budget and times out under load

[AlexJeffcott]

Summary

coverage.enforce runs at 83–86% of its 180s tier budget. It crossed the line once during #171–#173 verification and failed the whole --all run. The budget has no headroom, so the tier fails on machine load rather than on a coverage regression.

Context

Background

Found while verifying #171/#172/#168. A bun run test:tiers --all run reported:

⏱ coverage › coverage.enforce  179991ms — timed out after 180000ms

It is not a coverage regression. Run in isolation immediately afterwards, the same case passed in 154235ms with no code change.

Current State

packages/polly/scripts/test/registry.ts:117 sets timeoutMs: 180_000 for the coverage tier. Measured on one machine, one day, same commit range:

context	duration	% of budget
--all run 1	150005ms	83.3%
--all run 2	154774ms	86.0%
--all run 3	179991ms — timed out	100%
isolation, after the timeout	154235ms	85.7%

The whole --all wall clock grew across those runs — 389.7s, 407.5s, 468.8s — while Docker containers from the verify work were running. coverage.enforce is the longest case in the plan, so it absorbs load first.

The case is expensive by design: it self-spawns bun test --coverage, re-running the unit suite. The comment at registry.ts:120-123 records that the ~15s overlap with the unit tier is intentional. What is not recorded is that the case then costs ~10x that and sits just under a fixed cap.

Desired State

The tier fails when coverage policy is violated, and only then. A slow machine produces a slow run, not a red one.

Requirements

Functional Requirements

• Raise coverage tier timeoutMs so a normal run sits well under it — 300000ms puts the observed 154235ms at 51%
• Record the measured duty cycle in a comment beside the value, so the next person changing it knows what headroom exists
• Report the case's duration against its budget when it exceeds a threshold, so the margin erodes visibly rather than silently

Edge Cases

• A genuine hang must still be caught — the timeout is a real safety net, not to be removed
• --strict-needs and CI-style invocations must keep the same budget

Acceptance Criteria

• bun run test:tiers coverage passes with the duration reported
• bun run test:tiers --all passes on a loaded machine (reproduce by running it while the verify tier's Docker cases run)
• The new budget is at least 2x the slowest observed clean run

Implementation Notes

Files to Modify

• packages/polly/scripts/test/registry.ts:117 — timeoutMs: 180_000 on the coverage tier
• packages/polly/tools/test/src/tiers/engine.ts — where a case's durationMs is compared against timeoutMs, if a margin warning is added

Key Code Locations

• Tier definition: packages/polly/scripts/test/registry.ts:115-137
• Case argv (tools/test/src/coverage-policy/cli.ts): registry.ts:130-136
• Timeout resolution: packages/polly/tools/test/src/tiers/engine.ts — spec.timeoutMs ?? tier.timeoutMs ?? DEFAULT_TIMEOUT_MS

Suggested Approach

Raising the number is the one-line fix and is worth doing first. The durable fix is making margin visible: a case that finishes above ~70% of its budget should say so, because a fixed cap silently tightens every time the suite it wraps grows. coverage.enforce re-runs the whole unit suite, which grew to 1863 tests, so its cost tracks that suite's growth.

Reducing the cost itself is a separate question — the case deliberately re-runs the unit suite under --coverage and that overlap is intentional per registry.ts:120-123.

Dependencies

• Related: test: random listen ports make the integration tier fail unreproducibly (3.45% per run) #174 — the other pre-existing --all instability found in the same verification

Out of Scope

• Making coverage.enforce faster
• Changing the coverage policy or its 80/80 floor
• Reworking the --coverage re-run overlap with the unit tier

Technical Notes

Engine behaviour on timeout: the case is reported with outcome ⏱ … — timed out after 180000ms and the run fails. The reported 179991ms is the harness's own measurement up to the kill, not the work's true duration, so the real overrun is unknown — the run was killed, not measured.