perf: project only the KOTH detail keys the retest path reads

[Peyton-Spencer]

Summary

Third in the live hotfix stack (on top of #674, which is on #673). This is the one the profiler actually pointed at.

_current_koth_entries selected every eligible agent's entire score telemetry blob and read exactly three fields out of it: composite_stderr (via _ledger_stderr) plus the legacy confirmation_seeds/confirmation_composites pair. That blob averages ~22KB a row on bench v8, across every eligible agent.

Only the retest path pays for it. _current_koth_entries is reached solely from _current_emission_set and _current_retest_cohort; the canonical /validator/job lane never calls it. That is precisely the asymmetry seen in production — new submissions were being scored normally while the continual-retest lane crawled.

Measured on the live prod worker

py-spy record, 2383 samples over 30s, --nonblocking:

• 41% of CPU inside asyncpg's JSONB decoder — raw_decode the top leaf frame by a wide margin
• /validator/top5-confirmation-job slowest request: 151,884ms — 152 seconds. Validators are timing out before a ticket returns, which is why the lane looks dead rather than slow.
• /validator/job slowest over the same period: 20-40s
• the decode is charged to asyncpg's event-loop callback, not the awaiting coroutine — which is why no request-level metric ever attributed it, and why an earlier guess at the wrong caller (perf: stop hydrating score telemetry in the allocator floor read #674) produced no measurable change

The change

list_eligible_ledger(details_keys=(...)) has Postgres build a details object from only the named keys, so the remainder never detoasts, never crosses the wire, and never reaches the JSON decoder. Absent keys arrive as None, which all three readers already treat as absent.

Same class of fix as the existing include_details=False, but for a consumer that needs part of the blob rather than none of it. _KOTH_DETAIL_KEYS sits directly above the function that reads them, with a comment tying the tuple to its readers.

Validation

• uv run pytest ditto/tests/api_server/endpoints/ -q → 1150 passed
• uv run pytest ditto/tests/db/queries/ ditto/tests/api_server/endpoints/test_validator.py -q → 850 passed, 4 skipped
• ruff check / ruff format --check / mypy → clean

The validator endpoint suite covers the KOTH and retest paths, so stderr and legacy-confirmation behaviour are exercised against the narrowed projection. New tests pin that only requested keys are returned and that an absent key arrives as None.

Honest scope

This attacks per-request cost on the retest path. It does not reduce the number of allocator recomputations — subnet#388's batched claim contract is still unimplemented and is still the structural fix. Whether 152s becomes acceptable or merely better is an empirical question; I'll post before/after from the same profiler after deploy.

[Peyton-Spencer]

Closing unmerged: superseded by the monorepo cutover.

Production is now served from ditto-assistant/ditto-subnet (/opt/ditto-subnet/apps/platform), so this repository no longer deploys anywhere. This PR's content already landed in the monorepo and is live:

• fix: keep emission catch-up running between waves #673 → ditto-subnet#554 (emission catch-up between waves)
• perf: stop hydrating score telemetry in the allocator floor read #674 → ditto-subnet#559 (allocator floor read stops hydrating score telemetry)
• perf: project only the KOTH detail keys the retest path reads #675 → ditto-subnet#563 (KOTH detail-key projection — took /validator/job from 20-40s to 438-1247ms)

Merging here would land the fix on a branch nothing ships, which is worse than closing: it makes this repo look maintained and invites a deploy from it. See ditto-subnet#566 for the cutover follow-ups.