G1.3 — Wall-clock <= 2x API + per-swap timing breakdown

[mattdani21]

Goal

G1.3 — Mean wall-clock per task <= 2x the frontier API's time, with per-phase and per-swap timing breakdowns published.

Context

API baseline mean = 19.3 s/task. Bar: pipeline mean <= ~38.6 s/task on comparable hardware; T4-tier local numbers are the honest floor.

Acceptance criteria

• Mean wall-clock per task in the results JSON
• Per-phase load_s / evict_s / ttft_s breakdown per task (already recorded in phases[])
• Breakdown published in the parity report

Gate

jq on pipeline results (mean_wall_clock_s, mean_load_s, mean_evict_s)

[mattdani21]

Near-miss, open: mean 40.0s/task vs 2× bar (38.6s) = 2.07×. Breakdown published in docs/parity-report-phase1.md (loads 1.96s, evicts 0.17s — swap overhead negligible; generation dominates at 27B Q4 on L4). Levers identified: A100 rerun (~15-20s/task, comfortably under bar), resident-mode for repeated models, fewer retries on easy tasks. Recommendation: A100 confirmation run also doubles as the G1.2 confirmation.

[mattdani21]

Blackwell confirmation timing (temp 0.2): mean wall 44.4s = 2.30x API (bar 2x, 38.6s). The bigger GPU did not move the mean — wall time is phase-serialization + retry dominated, not generation bandwidth. Near-miss stands on both GPUs (2.07x / 2.30x). Closing lever: resident-mode / fewer retries (Phase 2, G2.1-G2.2). Keeping this issue OPEN pending Phase 2 measurement.

[mattdani21]

G1.3 lever measured: the overlap engine (G2.2) at 27B scale gives mean wall 34.66s/task on the 8-task subset vs the Phase-1 sequential 50.05s -> projection 50 x 34.66 = 1733s vs API 965s = 1.80x, UNDER the 2x bar. A 50-task overlap run (colab-phase2-eval.ipynb) closes this.

[mattdani21]

Full-suite 8192 data: overlap mean wall 40.83s = 2.12x vs the 2x bar (sequential 41.99s = 2.18x). Still a near-miss; the overlap engine removed the load tax (1.995 -> 0.806 s/phase) so the remaining gap is generation throughput. Primary hypothesis: the multi-arch build runs Blackwell via PTX JIT (sm_89 -> sm_100) — a native-arch cache (100;120, LLAMA_CACHE_VERSION v4) is the obvious lever, expected 5-20% on generation. Retry storms add tail variance. Keeping open.

[mattdani21]

Lever measurement landed (2026-08-25, main @ b762749): cross-turn KV prefix cache (issue #16) cut retry-phase prefill 63.7s -> 5.2s (-91.8%) on the local 4B stack, reviewer-approved. Honest projection at 27B scale: retry prefill is only 0.51% of wall, so the lever moves 2.12x -> ~2.11x (upper bound 2.109x) — it CANNOT close the <2x bar alone. G1.3 stays open; next levers: retry-storm reduction and native-arch llama builds (docs/g1.3-kv-prefix-cache.md, docs/parity-report-phase1.md Addendum 3).