fix(validator): stop charging screened-image acquisition to the miner - #286
Merged
Peyton-Spencer merged 1 commit intoJul 28, 2026
Merged
Conversation
This was referenced Jul 28, 2026
Add `screened_image_unavailable` to `_SANDBOX_INFRASTRUCTURE_CODES`, which activates the classification dittobench-api emits in #118. Until now that code fell through to the terminal `scoring_error` path and spent one of the miner's finite attempts on a fetch failure for the platform's own image. The bound is asserted here too: a verification failure carrying the terminal `sandbox_failure` envelope must stay the agent's, so the no-fault class cannot re-lease a permanently broken image without bound. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Peyton-Spencer
force-pushed
the
peyton/screened-image-acquisition-not-agent-fault
branch
from
July 28, 2026 04:35
6036c93 to
85d10aa
Compare
Peyton-Spencer
added a commit
that referenced
this pull request
Jul 28, 2026
…fault
dittobench-api split the relay's single finalize failure code apart. Its
grant-denial counter collapsed nine unrelated conditions -- several entirely
the harness's doing -- into validator_infrastructure/model_relay_unavailable/
retryable:true, which mints a retry grant, RAISES the attempt cap, and
re-leases. An agent that reliably spent its own allowance re-leased itself
forever.
Two new codes reach this side, and they go in opposite directions:
- inference_allowance_exhausted (sandbox_failure/retryable:false) is the
agent's: it spent the request-count or token allowance its own ticket
granted, or sent one request too large to reserve. Deliberately NOT added
to _SANDBOX_INFRASTRUCTURE_CODES, so it maps to scoring_error.
- inference_lane_saturated IS added. The scorer's call spent its whole
backpressure budget on a full platform lane. Split out only so a saturated
rail is legible without container logs; a miner can neither provision the
lane nor see its contention, so it keeps its grant.
Guards: a module-level assert, a set-property test, and a poll-path test that
pins the agent code to scoring_error under BOTH the envelope the scorer sends
and the no-fault envelope it must never be smuggled under.
Stacked on #286 (screened_image_unavailable), which edits the same set.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Peyton-Spencer
marked this pull request as ready for review
July 28, 2026 04:35
Peyton-Spencer
deleted the
peyton/screened-image-acquisition-not-agent-fault
branch
July 28, 2026 04:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Today a validator that fails to download the screened image the platform itself
produced charges the miner an attempt.
_sandbox_infrastructure_failure_code()in dittobench.py:222 honours only a closed
5-member set of codes;
screened_image_unavailableis not in it, so the workertakes
DittobenchError→_report_ticket_failed(job, "scoring_error")(worker.py:615) and spends one of the miner's
finite attempts on an object-store outage.
This adds
screened_image_unavailableto the set. It is the consumer half ofditto-assistant/dittobench-api#118, which emits the code but deliberately does
not touch this repo — that PR is inert on retry policy until this one lands.
⛔ Do not merge before ditto-assistant/dittobench-api#118 merges. Landing
this first is harmless (no scorer emits the code yet) but leaves a half-wired
contract; merging it never is the actual regression.
Why this cannot recreate the mnemox loop
infrastructureis the platform's no-fault class: it mints a retry grant,raises the attempt cap, and re-leases. On 2026-07-27 the
mnemoxfamily hadevery failure classified
infrastructure;mnemox-v55reachedattempts_used: 10against a base budget of 2 with zero scores, and 9submissions had to be manually evicted at 20:56Z to free the fleet. Adding a
member to this set is exactly the kind of change that caused it, so the argument
has to be load-bearing rather than asserted. I verified both halves against
#118's diff at review time rather than taking its description on trust:
1. The artifact is not running, and cannot be. A self-sustaining no-fault
loop needs the artifact's own behaviour in the loop — mnemox minted its grants
by hanging for ~60 minutes. Every failure carrying this code arises inside
sandbox.loadScreenedImage, and the classifierscreenedImageInfraFailureisattached at exactly two call sites, both the error return of
s.sandbox.Build(…)(
runSandboxJob,runSizeJob). A Build error means the container never startedand the miner's harness has not executed an instruction. Whether a TCP
connection to the object store succeeds is not something a harness can steer.
The feedback edge that made mnemox self-sustaining does not exist on this path.
2. Only transient-by-construction conditions carry the code. In #118 the
unavailable()wrapper appears at fivereturnsites and nowhere else:transport error, object-store status where
status >= 500 || status == 429,temp-file create, mid-stream copy error, and file close. Everything
deterministic in the bytes the platform stored is left on the terminal default —
sha256 / size / image-id mismatch, malformed archive, every 4xx other than
429 (so an expired 403 grant or a 404 missing blob stays terminal), and
docker image load. A permanently broken image therefore cannot re-leasewithout bound, because a permanently broken image never produces this code.
3. It is a typed sentinel, not a substring match. The code is minted only by
errors.Is(err, sandbox.ErrScreenedImageUnavailable). A harness printingscreened image unavailableon stderr cannot forge itself a grant — #118 teststhat, and so does this PR from the consuming side.
4. This side degrades the slot, it does not just retry. The
ValidatorInfrastructureErrorarm discards the slot from_healthy_slotsandbreaks out of it for the sweep (worker.py:596),
so a host whose disk or network is the problem stops claiming work rather than
chewing through tickets.
Changes
_SANDBOX_INFRASTRUCTURE_CODESgainsscreened_image_unavailable, with theacquisition-vs-verification boundary recorded in the comment.
test_sandbox_resource_failure_is_retryable_infrastructurecovers the newcode (raises
ValidatorInfrastructureError, private scorererrortext stillredacted from the message).
test_screened_image_verification_failure_stays_the_agentspins thebound: seven realistic deterministic failures — sha256 / size / image-id
mismatch, 404, malformed archive,
docker image load, and the sentinel's ownwords as harness prose — each carrying the terminal
sandbox_failure/retryable=falseenvelope, must still raiseDittobenchError. This is a cross-repo invariant with no shared package, soit is asserted independently on both sides.
Test plan
make lint typecheck test— 996 passed, 9 deselectedditto/tests/validator/test_dittobench.py— 142 passed (+8 new)failure reports
infrastructure(notscoring_error) and that theagent's
attempts_usedis unchanged by itattempts_usedvs. base budget across the fleet for one epoch afterrollout — the mnemox signature is attempts climbing past the cap with zero
scores
Related