Skip to content

fix(validator): widen failure_detail from 200 to 4096 chars - #291

Merged
Peyton-Spencer merged 1 commit into
mainfrom
validator/widen-failure-detail
Jul 28, 2026
Merged

fix(validator): widen failure_detail from 200 to 4096 chars#291
Peyton-Spencer merged 1 commit into
mainfrom
validator/widen-failure-detail

Conversation

@Peyton-Spencer

@Peyton-Spencer Peyton-Spencer commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

failure_detail (#282) was bounded to 200 characters. The first time it carried a real diagnostic message rather than a failure code, it cut it mid-word — and that message was the one that finally identified a root cause three separate investigations had failed to reach:

DittobenchError: run 2b7c6b6c-ae45-493d-b8f5-b1a4a6ff8b3a failed: harness exhausted
its inference allowance: agent-attributable inference decline: the platform rejected
81 of the harness's inference r[equest(s) outright, before reserving any capacity]

The bracketed remainder never left this repo. The count (81) and the verb (rejected) survived by luck of word order. And the fragment that did arrive ends on a word boundary and reads as a finished sentence — so there was no signal that a second clause had ever existed.

This raises the sender-side cap to 4096, makes truncation visible when it still happens, and handles the one version-skew direction that widening creates.

Pairs with ditto-assistant/ditto-platform#553 (the enforcing side).

The bound: why 4096 and not unlimited

The ask was "remove the limit or make it very long". I chose a large explicit cap, reasoning on the record:

  • The platform stores this per ticket on a hot table (validator_tickets) — a ledger row, not a log line.
  • Its content is derived from strings a miner's harness can influence. Unbounded plus attacker-influenceable plus a write path into the ledger is a combination worth not having; the only thing protecting an uncapped field is that nobody has tried.
  • 4096 is ~16x the longest message observed (the one above is ~250 chars) and fits a scorer message with a run id, a full account of an exhaustion, and its counts.

Generous enough that no realistic diagnostic hits it; bounded enough that the field cannot become a log-shipping channel.

Truncation now announces itself

errors.truncate_failure_detail appends ...[truncated, N chars], sized so the result still fits the limit including the marker — a marker that pushed the value back over the cap would 422 the hand-back it exists to preserve. It carries the pre-truncation length because "how much is missing" is the next question and is unrecoverable afterward. A detail that fits gets no marker: a whole message must not acquire one it hasn't earned.

failure_detail()'s code-over-message precedence is untouched. A coded error still reports its code alone. That precedence is what the classifier reads, and it is deliberately independent of the cap — a code was never long enough for the bound to apply. What the wider bound buys is the fallback branch, which is where real diagnostic messages live and where 200 characters was demonstrably not enough. Pinned by test_a_structured_code_still_wins_over_the_message.

Version skew

Validators run 0.34.1 through 0.37.3 concurrently, and the platform is independently versioned relative to all of them. Both directions:

old validator (≤200) new validator (up to 4096)
new platform Fine. Widening a max_length only ever admits more. Fine, and the point of the change.
old platform (still 200) Fine, unchanged. 422 → retry once at the legacy 200 bound.

That retry is the new code in PlatformClient.report_ticket_failed, and it exists because a 422 here does not lose a field, it loses the entire hand-back: the lease stays live to its deadline and the slot idles — precisely the silent expiry failure_detail was introduced to eliminate. Trading the tail of one message for the whole report is unambiguously the right side of that trade.

It is safe to replay with the same nonce and signature:

  • A 422 is request validation, raised before the endpoint body runs, so the nonce was never consumed.
  • failure_detail is unsigned by design (as reason is), so shortening it leaves the signed payload byte-identical. Nothing is re-signed.

Deliberately narrow: it fires once, only on 422, and only when the detail exceeds 200 — so length is a plausible cause. Any other 422 falls through to the existing typed PlatformError after one wasted round trip (test_a_short_detail_is_not_retried_on_422).

Deploy order is therefore not load-bearing in either direction, though platform-first means full messages from the first widened validator onward.

Changes

  • api_models/validator.py: FAILURE_DETAIL_MAX_LENGTH 200 → 4096; added LEGACY_FAILURE_DETAIL_MAX_LENGTH = 200.
  • validator/errors.py: new truncate_failure_detail + TRUNCATION_MARKER_TEMPLATE; failure_detail() routes both branches through it.
  • validator/platform.py: extracted _post_job_fail so the retry sends through identical serialization (exclude_none in particular, which is what keeps a detail-free report byte-identical to the pre-failure_detail wire format); added the 422 legacy-bound retry.
  • tests/contract/validator_contract.json: regenerated from ditto-platform's models via scripts/gen_validator_contract.py, per the documented flow. Only the maxLength line changed, which confirms the two repos' models are otherwise structurally identical. Note ditto-platform#553 also adds FailJobRequest/FailJobResponse to its SHARED_MODELS — they were only ever in this repo's copy, so the golden's entry for this field had been self-generated rather than taken from the source of truth.

Test plan

  • test_the_real_inference_decline_message_survives_the_round_trip — the actual production message, asserted whole through failure_detail() and the real FailJobRequest, with an explicit assertion on the clause the old cap ate.
  • test_a_long_detail_reaches_an_upgraded_platform_whole — same message over the real client; exactly one request, arriving intact.
  • test_a_long_detail_is_retried_at_the_legacy_bound_on_422 — reverse skew: hand-back lands, retry re-truncates visibly, signed fields byte-identical, signature re-verified against the replayed payload.
  • test_a_short_detail_is_not_retried_on_422, test_a_detail_at_the_widened_cap_is_sent_unmodified, test_truncation_announces_itself_instead_of_cutting_silently, test_a_detail_that_fits_gets_no_truncation_marker, test_a_structured_code_still_wins_over_the_message.
  • make lint typecheck clean; full suite 1034 passed.

Out of scope

Failure classification (#288 / dittobench-api#120), validator dispatch and slot claiming (#287), lease code, heartbeat schema, validator_lease_audit, and the scores read path are all untouched.

@Peyton-Spencer Peyton-Spencer changed the title validator/widen failure detail fix(validator): widen failure_detail from 200 to 4096 chars Jul 28, 2026
@Peyton-Spencer
Peyton-Spencer marked this pull request as ready for review July 28, 2026 15:52
@Peyton-Spencer
Peyton-Spencer merged commit 1b4f031 into main Jul 28, 2026
4 of 5 checks passed
@Peyton-Spencer
Peyton-Spencer deleted the validator/widen-failure-detail branch July 28, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant