fix/split relay decline classification

[Peyton-Spencer]

Summary

failRelayUnavailable stamped validator_infrastructure / model_relay_unavailable / retryable: true on every finalize failure. That is the platform's no-fault class: it mints a retry grant, raises the attempt cap, and re-leases. At least nine unrelated conditions funnelled into it, several of them entirely the harness's doing — so an agent that failed the same way every run re-leased itself without bound, holding validator slots and never scoring.

The broker already parsed the platform's precise decline code (platformDeclineCode) and then wrote it only to a log line before collapsing everything into one code. errors.failure_detail() prefers code over message, so the discriminating text was destroyed too. This is the same disease #282 fixed one layer up, recurring one layer down.

Companion: ditto-subnet #288. Builds on #117 and #118.

Direction and safety

Every reclassification moves conditions the agent controls OUT of no-fault. Nothing the platform can cause unilaterally — grant revocation, lease expiry, relay unreachable, provider 5xx, lane saturation — loses its grant. No genuine platform failure starts billing a miner.

Three properties enforce that:

1. The default is unchanged and stays no-fault. relayFinalizeFailure returns validator_infrastructure/model_relay_unavailable/retryable for everything it does not affirmatively recognise. Only a typed sentinel leaves it.
2. Infrastructure wins ties. A run is charged to the agent only when every denial it saw was agent-attributable. One revoked lease, one expired grant, one decline this build could not attribute — mixed in with a hundred budget declines — and the whole run keeps its grant.
3. Which runs FAIL is unchanged. grant_denials and usage_unavailable keep their exact previous values and predicates; the new counters are subsets that sit alongside them. Only who is charged moves. The full existing suite passes untouched.

There is in-tree precedent for exactly this move: SandboxOomError and LeaseDeadlineError were both pulled out of infrastructure for the same reason (ditto/validator/errors.py).

Guarding the other direction, since helios_v2 is already being charged for platform failures because the default bin over in the sandbox path is agent-fault: this change does not widen that bin. It adds one narrowly-bounded agent code reachable only from the table below.

What moved

Condition	Platform code	Before	After
Request-count allowance spent	4102 BUDGET_EXHAUSTED	no-fault	sandbox_failure / inference_allowance_exhausted / not retryable
Token allowance spent	4104 TOKEN_BUDGET_EXHAUSTED	no-fault	same
One request too large to reserve	4109 RESERVATION_TOO_LARGE	no-fault	same
Pre-reservation 4xx (400 schema, 403 model, 413 size)	—	no-fault	same

What deliberately did NOT move

Condition	Why it keeps its grant
Capacity-wait exhaustion	Distinguished in diagnostics (relay_cause: inference_lane_saturated), keeping the infrastructure class and the already-deployed model_relay_unavailable code — see "Rollout safety" below for why it must not be a new code. Genuinely ambiguous — an under-provisioned platform, or one embed-heavy ticket crowding out its neighbours — and a miner can neither provision the lane nor see its contention. Blaming a miner for a saturated platform rail is the mirror-image of the bug being fixed.
4101 GRANT_REVOKED, 4105 LEASE_EXPIRED, 4108 GRANT_NOT_EXCHANGED	Grant lifecycle, owned entirely by the platform and this broker.
4106 NONCE_REPLAYED, 4107 MODEL_NOT_PERMITTED	Deviates from the obvious reading. A harness cannot reach either: this broker mints a fresh uuid nonce per upstream attempt on both lanes, and overwrites the caller's model before forwarding (rewriteRequestModel on chat, a hard-coded hostedEmbeddingModel on embeddings). If the platform still says the nonce repeated or the model is not permitted, the disagreement is between the grant and the ticket — platform and broker state the miner never touches. Billing these would be the exact mistake this PR fixes, pointed the other way.
4100 unattributed	The platform holds an unknown grant id and a failed bearer comparison deliberately indistinguishable. If it refuses to say, this must not guess.
Any unrecognised code, and "no code at all"	Cannot be classified ⇒ stays infrastructure.
Provider returned success but omitted usage	Provider-side; nothing the harness does can make usage appear.

Since the exact failing branch could not be confirmed (container stderr from #117 is pinned in ditto-subnet #285 / v0.37.0, but the fleet is still on v0.36.0/v0.34.1), this is designed to be correct whichever branch fired — it fixes the collapse of nine conditions into one, not one specific condition. Every branch that genuinely cannot be attributed without logs was left as infrastructure, and is listed above.

Rollout safety — the two directions are not symmetric

This was nearly a second instance of the same bug, arriving through the rollout instead of through the classifier, and it is worth stating explicitly because it is not obvious.

ditto-subnet's _sandbox_infrastructure_failure_code returns None for any code outside its allowlist, and None becomes fail_job("scoring_error"). So the no-fault class is only no-fault on validators that recognise the specific code. Shipping a brand-new infrastructure code would have charged miners for a saturated platform rail on every validator predating the matching release — and the fleet lags scorer releases by design (it is on v0.36.0 / v0.34.1 right now, while the #117 container-stderr repin is v0.37.0).

• New agent codes are skew-safe. An old validator sees sandbox_failure / retryable: false and its terminal default is already the intended outcome. It needs to know nothing about inference_allowance_exhausted.
• New infrastructure codes are not. So saturation keeps model_relay_unavailable — in every deployed validator's allowlist since the code existed — and carries the distinction in Diagnostics, a field the subnet's classifier never consults and which therefore cannot change the class anywhere.

TestNoFaultFailuresOnlyUseCodesTheDeployedFleetAllowlists pins this: every no-fault failure this package can produce must use a code the deployed fleet already allowlists. A future genuinely-new no-fault code requires the validator side to ship and the fleet to roll first, and that test is where the conversation starts.

Consequence: #120 and #288 have no ordering hazard in either direction. Neither repo depends on the other having shipped.

Forged-classification resistance

The agent class is the only bin that costs a miner one of a small number of finite attempts, and harness stderr and harness-authored error strings flow through this process. So it cannot be reached by text:

• Typed sentinels + errors.Is, following fix: stop charging screened-image acquisition failures to the miner #118's ErrScreenedImageUnavailable precedent — never substring matching. Only code in this package can wrap errAgentInferenceDeclined.
• The only route from miner-influenced bytes to the attribution table is platformDeclineCode, which parses one integer from a JSON envelope, allowlists it, and returns 0 ("say nothing") for anything else. 0 is never agent-fault.
• declineFaultFor is a table with a safe default, deliberately not a ladder of ifs — the same shape and rationale as the platform's own _TERMINAL_DECLINE_RESPONSES. A decline nobody wires up falls into the safe bin.
• A subset counter that outruns its total is refused rather than concluded from: corrupt arithmetic cannot produce "all denials were the agent's".

Also fixed

The broker recognised only 4100–4103 while the platform has been emitting 4100–4109. 4104 TOKEN_BUDGET_EXHAUSTED — the decline that actually ends heavy v7 runs — was arriving, being parsed, and then discarded onto the "the platform did not say why" branch. TestEveryPlatformDeclineCodeIsRecognised pins the full range so the two repos cannot drift again.

Tests

go test -race ./... green across the repo. New: cmd/dittobench-api/relay_decline_attribution_test.go.

• TestUnrecognisedFinalizeFailuresCannotReachTheAgentClass — the guard, mirroring TestUnrecognisedFailuresCannotReachTheNoFaultClass from fix: stop charging screened-image acquisition failures to the miner #118. Note the arrow is reversed here by construction: fix: stop charging screened-image acquisition failures to the miner #118's default was terminal and it moved conditions into no-fault, so its guard protected the no-fault class. Here the default is no-fault and conditions move out, so the property that needs guarding is the mirror image — nothing reaches the agent class by falling through, and in particular nothing reaches it via text. Both sentinels' own words, verbatim, and an echoed {"error_code": 4102} envelope are in the table and must not classify.
• TestFinalizeFailureAttributionContract — the whole class table in one place; moving a row requires editing the test.
• TestDeclineAttributionCannotBeForgedFromABody — 18 bodies including string-not-number, nested-out-of-contract, prose naming the code, and unknown future codes.
• TestNoFaultFailuresOnlyUseCodesTheDeployedFleetAllowlists — the rollout invariant above.
• TestAMixedRunKeepsItsGrant, TestAnImpossibleSubsetIsRefusedRatherThanBlamed, TestLaneSaturationStaysNoFault, TestACleanRunIsUnaffected.
• End-to-end through the real broker on both lanes — TestBudgetDeclineIsAttributedToTheAgentEndToEnd, TestPreReservationRejectionIsAttributedToTheAgentEndToEnd, TestCapacityExhaustionIsNamedButKeepsItsGrantEndToEnd — so the counters are proven to be incremented at the right sites, not just classified correctly in the abstract. Each asserts the harness-visible status is byte-identical to before: a run being discarded must not observe a changed gateway contract mid-benchmark just because blame moved.

Concurrency note

ditto-subnet #288 (the companion) edits _SANDBOX_INFRASTRUCTURE_CODES, which ditto-subnet #286 also edits to add screened_image_unavailable. #288 is stacked on #286's branch rather than conflicting with it, and #286 itself was left completely untouched — its base is still main, it was not adopted into a GitHub Stack object, and no git operation was run against its branch (it is checked out in another live worktree). See #288 for details.

🤖 Generated with Claude Code

[Peyton-Spencer]

Companion: ditto-assistant/ditto-subnet#288 (stacked on ditto-assistant/ditto-subnet#286). Merge dittobench-api #120 and ditto-subnet #288 together — #120 emits inference_allowance_exhausted and inference_lane_saturated; #288 is what routes them. Landing #120 alone leaves inference_lane_saturated unrecognised on the validator side, which would make a saturated lane terminal instead of no-fault.

[Peyton-Spencer]

Correction to the ordering note above. The earlier comment said #120 must not land ahead of ditto-subnet#288. That is no longer true, and the reason it was true is itself now fixed.

Saturation originally shipped a new no-fault code (inference_lane_saturated). Because ditto-subnet maps an unrecognised validator_infrastructure code to scoring_error, that would have charged miners for a saturated platform rail on every validator predating the matching release — and the fleet is on v0.36.0/v0.34.1 while the relevant repin is v0.37.0. Same bug as this PR fixes, arriving through the rollout.

Saturation now keeps model_relay_unavailable and carries the cause in Diagnostics, so it stays no-fault on every fleet version including ones that never upgrade. #120 and #288 can now land in either order.