fix(validator): widen failure_detail from 200 to 4096 chars - #291
Merged
Conversation
Merged
5 tasks
Peyton-Spencer
marked this pull request as ready for review
July 28, 2026 15:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
failure_detail(#282) was bounded to 200 characters. The first time it carried a real diagnostic message rather than a failure code, it cut it mid-word — and that message was the one that finally identified a root cause three separate investigations had failed to reach:The bracketed remainder never left this repo. The count (
81) and the verb (rejected) survived by luck of word order. And the fragment that did arrive ends on a word boundary and reads as a finished sentence — so there was no signal that a second clause had ever existed.This raises the sender-side cap to 4096, makes truncation visible when it still happens, and handles the one version-skew direction that widening creates.
Pairs with ditto-assistant/ditto-platform#553 (the enforcing side).
The bound: why 4096 and not unlimited
The ask was "remove the limit or make it very long". I chose a large explicit cap, reasoning on the record:
validator_tickets) — a ledger row, not a log line.Generous enough that no realistic diagnostic hits it; bounded enough that the field cannot become a log-shipping channel.
Truncation now announces itself
errors.truncate_failure_detailappends...[truncated, N chars], sized so the result still fits the limit including the marker — a marker that pushed the value back over the cap would 422 the hand-back it exists to preserve. It carries the pre-truncation length because "how much is missing" is the next question and is unrecoverable afterward. A detail that fits gets no marker: a whole message must not acquire one it hasn't earned.failure_detail()'scode-over-messageprecedence is untouched. A coded error still reports its code alone. That precedence is what the classifier reads, and it is deliberately independent of the cap — a code was never long enough for the bound to apply. What the wider bound buys is the fallback branch, which is where real diagnostic messages live and where 200 characters was demonstrably not enough. Pinned bytest_a_structured_code_still_wins_over_the_message.Version skew
Validators run 0.34.1 through 0.37.3 concurrently, and the platform is independently versioned relative to all of them. Both directions:
max_lengthonly ever admits more.That retry is the new code in
PlatformClient.report_ticket_failed, and it exists because a 422 here does not lose a field, it loses the entire hand-back: the lease stays live to its deadline and the slot idles — precisely the silent expiryfailure_detailwas introduced to eliminate. Trading the tail of one message for the whole report is unambiguously the right side of that trade.It is safe to replay with the same nonce and signature:
failure_detailis unsigned by design (asreasonis), so shortening it leaves the signed payload byte-identical. Nothing is re-signed.Deliberately narrow: it fires once, only on 422, and only when the detail exceeds 200 — so length is a plausible cause. Any other 422 falls through to the existing typed
PlatformErrorafter one wasted round trip (test_a_short_detail_is_not_retried_on_422).Deploy order is therefore not load-bearing in either direction, though platform-first means full messages from the first widened validator onward.
Changes
api_models/validator.py:FAILURE_DETAIL_MAX_LENGTH200 → 4096; addedLEGACY_FAILURE_DETAIL_MAX_LENGTH = 200.validator/errors.py: newtruncate_failure_detail+TRUNCATION_MARKER_TEMPLATE;failure_detail()routes both branches through it.validator/platform.py: extracted_post_job_failso the retry sends through identical serialization (exclude_nonein particular, which is what keeps a detail-free report byte-identical to the pre-failure_detailwire format); added the 422 legacy-bound retry.tests/contract/validator_contract.json: regenerated from ditto-platform's models viascripts/gen_validator_contract.py, per the documented flow. Only themaxLengthline changed, which confirms the two repos' models are otherwise structurally identical. Note ditto-platform#553 also addsFailJobRequest/FailJobResponseto itsSHARED_MODELS— they were only ever in this repo's copy, so the golden's entry for this field had been self-generated rather than taken from the source of truth.Test plan
test_the_real_inference_decline_message_survives_the_round_trip— the actual production message, asserted whole throughfailure_detail()and the realFailJobRequest, with an explicit assertion on the clause the old cap ate.test_a_long_detail_reaches_an_upgraded_platform_whole— same message over the real client; exactly one request, arriving intact.test_a_long_detail_is_retried_at_the_legacy_bound_on_422— reverse skew: hand-back lands, retry re-truncates visibly, signed fields byte-identical, signature re-verified against the replayed payload.test_a_short_detail_is_not_retried_on_422,test_a_detail_at_the_widened_cap_is_sent_unmodified,test_truncation_announces_itself_instead_of_cutting_silently,test_a_detail_that_fits_gets_no_truncation_marker,test_a_structured_code_still_wins_over_the_message.make lint typecheckclean; full suite 1034 passed.Out of scope
Failure classification (#288 / dittobench-api#120), validator dispatch and slot claiming (#287), lease code, heartbeat schema,
validator_lease_audit, and the scores read path are all untouched.