Skip to content

fix(validator): stop charging screened-image acquisition to the miner - #286

Merged
Peyton-Spencer merged 1 commit into
mainfrom
peyton/screened-image-acquisition-not-agent-fault
Jul 28, 2026
Merged

fix(validator): stop charging screened-image acquisition to the miner#286
Peyton-Spencer merged 1 commit into
mainfrom
peyton/screened-image-acquisition-not-agent-fault

Conversation

@Peyton-Spencer

@Peyton-Spencer Peyton-Spencer commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Today a validator that fails to download the screened image the platform itself
produced
charges the miner an attempt. _sandbox_infrastructure_failure_code()
in dittobench.py:222 honours only a closed
5-member set of codes; screened_image_unavailable is not in it, so the worker
takes DittobenchError_report_ticket_failed(job, "scoring_error")
(worker.py:615) and spends one of the miner's
finite attempts on an object-store outage.

This adds screened_image_unavailable to the set. It is the consumer half of
ditto-assistant/dittobench-api#118, which emits the code but deliberately does
not touch this repo — that PR is inert on retry policy until this one lands.

⛔ Do not merge before ditto-assistant/dittobench-api#118 merges. Landing
this first is harmless (no scorer emits the code yet) but leaves a half-wired
contract; merging it never is the actual regression.

Why this cannot recreate the mnemox loop

infrastructure is the platform's no-fault class: it mints a retry grant,
raises the attempt cap, and re-leases. On 2026-07-27 the mnemox family had
every failure classified infrastructure; mnemox-v55 reached
attempts_used: 10 against a base budget of 2 with zero scores, and 9
submissions had to be manually evicted at 20:56Z to free the fleet. Adding a
member to this set is exactly the kind of change that caused it, so the argument
has to be load-bearing rather than asserted. I verified both halves against
#118's diff at review time rather than taking its description on trust:

1. The artifact is not running, and cannot be. A self-sustaining no-fault
loop needs the artifact's own behaviour in the loop — mnemox minted its grants
by hanging for ~60 minutes. Every failure carrying this code arises inside
sandbox.loadScreenedImage, and the classifier screenedImageInfraFailure is
attached at exactly two call sites, both the error return of s.sandbox.Build(…)
(runSandboxJob, runSizeJob). A Build error means the container never started
and the miner's harness has not executed an instruction. Whether a TCP
connection to the object store succeeds is not something a harness can steer.
The feedback edge that made mnemox self-sustaining does not exist on this path.

2. Only transient-by-construction conditions carry the code. In #118 the
unavailable() wrapper appears at five return sites and nowhere else:
transport error, object-store status where status >= 500 || status == 429,
temp-file create, mid-stream copy error, and file close. Everything
deterministic in the bytes the platform stored is left on the terminal default —
sha256 / size / image-id mismatch, malformed archive, every 4xx other than
429
(so an expired 403 grant or a 404 missing blob stays terminal), and
docker image load. A permanently broken image therefore cannot re-lease
without bound, because a permanently broken image never produces this code.

3. It is a typed sentinel, not a substring match. The code is minted only by
errors.Is(err, sandbox.ErrScreenedImageUnavailable). A harness printing
screened image unavailable on stderr cannot forge itself a grant — #118 tests
that, and so does this PR from the consuming side.

4. This side degrades the slot, it does not just retry. The
ValidatorInfrastructureError arm discards the slot from _healthy_slots and
breaks out of it for the sweep (worker.py:596),
so a host whose disk or network is the problem stops claiming work rather than
chewing through tickets.

Changes

  • _SANDBOX_INFRASTRUCTURE_CODES gains screened_image_unavailable, with the
    acquisition-vs-verification boundary recorded in the comment.
  • test_sandbox_resource_failure_is_retryable_infrastructure covers the new
    code (raises ValidatorInfrastructureError, private scorer error text still
    redacted from the message).
  • New test_screened_image_verification_failure_stays_the_agents pins the
    bound
    : seven realistic deterministic failures — sha256 / size / image-id
    mismatch, 404, malformed archive, docker image load, and the sentinel's own
    words as harness prose — each carrying the terminal
    sandbox_failure/retryable=false envelope, must still raise
    DittobenchError. This is a cross-repo invariant with no shared package, so
    it is asserted independently on both sides.

Test plan

  • make lint typecheck test — 996 passed, 9 deselected
  • ditto/tests/validator/test_dittobench.py — 142 passed (+8 new)
  • After chore: remove extracted screener runtime #118 merges and the scorer is repinned: confirm a real acquisition
    failure reports infrastructure (not scoring_error) and that the
    agent's attempts_used is unchanged by it
  • Watch attempts_used vs. base budget across the fleet for one epoch after
    rollout — the mnemox signature is attempts climbing past the cap with zero
    scores

Related

Add `screened_image_unavailable` to `_SANDBOX_INFRASTRUCTURE_CODES`, which
activates the classification dittobench-api emits in #118. Until now that code
fell through to the terminal `scoring_error` path and spent one of the miner's
finite attempts on a fetch failure for the platform's own image.

The bound is asserted here too: a verification failure carrying the terminal
`sandbox_failure` envelope must stay the agent's, so the no-fault class cannot
re-lease a permanently broken image without bound.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Peyton-Spencer
Peyton-Spencer force-pushed the peyton/screened-image-acquisition-not-agent-fault branch from 6036c93 to 85d10aa Compare July 28, 2026 04:35
Peyton-Spencer added a commit that referenced this pull request Jul 28, 2026
…fault

dittobench-api split the relay's single finalize failure code apart. Its
grant-denial counter collapsed nine unrelated conditions -- several entirely
the harness's doing -- into validator_infrastructure/model_relay_unavailable/
retryable:true, which mints a retry grant, RAISES the attempt cap, and
re-leases. An agent that reliably spent its own allowance re-leased itself
forever.

Two new codes reach this side, and they go in opposite directions:

  - inference_allowance_exhausted (sandbox_failure/retryable:false) is the
    agent's: it spent the request-count or token allowance its own ticket
    granted, or sent one request too large to reserve. Deliberately NOT added
    to _SANDBOX_INFRASTRUCTURE_CODES, so it maps to scoring_error.
  - inference_lane_saturated IS added. The scorer's call spent its whole
    backpressure budget on a full platform lane. Split out only so a saturated
    rail is legible without container logs; a miner can neither provision the
    lane nor see its contention, so it keeps its grant.

Guards: a module-level assert, a set-property test, and a poll-path test that
pins the agent code to scoring_error under BOTH the envelope the scorer sends
and the no-fault envelope it must never be smuggled under.

Stacked on #286 (screened_image_unavailable), which edits the same set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Peyton-Spencer
Peyton-Spencer marked this pull request as ready for review July 28, 2026 04:35
@Peyton-Spencer
Peyton-Spencer merged commit b7a0e0c into main Jul 28, 2026
4 checks passed
@Peyton-Spencer
Peyton-Spencer deleted the peyton/screened-image-acquisition-not-agent-fault branch July 28, 2026 04:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant