Research prompt: per-request CPU-time instrumentation

[Augustas11]

Status

DRAFT — RESEARCH PROMPT ONLY, NO IMPLEMENTATION. This PR contains only a Codex session prompt (specs/RESEARCH_CPU_TIME_INSTRUMENTATION_PROMPT.md) that stages the research + SPEC-drafting work. No code changes.

Why this exists

grep -rEln "cpu.time|cpu_time|cpu.util|gpu.util|stage.timing|per.stage.timing" --include='*.go' --include='*.py' returns one spec-doc hit and zero code hits. macprovider measures throughput and TTFT; it does not measure CPU-vs-GPU share of stage time.

Critical timing: SPEC-028 (chain speculative decoding) is landing now, adding CPU work per verify round (drafter forward, acceptance sampling). Without CPU-time instrumentation, the next sweep re-run under SPEC-028 will produce winner knobs that are uninterpretable — you cannot tell whether a knob's win came from better compute utilization or from adjacent unrelated variance. Ship this before the class-aware sweep so future perf decisions stay honest.

Second-order value: unlocks a defensible "Apple Silicon is CPU-favored on consumer inference" positioning claim, receipt-backable once measured. Shard's 2026-07-03 report argues consumer fleets are CPU-bound; if that generalizes to M-series unified memory + strong CPU cores, it's a first-principle marketing claim macprovider cannot make today without measuring.

What has to achieve to be considered implemented

1. Research session runs the prompt in this PR and produces:

   • docs/research/cpu-time-instrumentation-2026-07.md — memo answering §A–F with citations (measurement mechanism on Apple Silicon, schema change, harness integration, report surface, cross-run reproducibility, publishing surface).
   • specs/SPEC-XXX-cpu-time-instrumentation.md v0.1-draft — normative FRs (schema change, measurement protocol, overhead budget), non-goals, open questions, acceptance criteria.
   • .omc/logs/cpu-time-instrumentation-open-questions-2026-07.md — ≤5 maintainer-input items.

2. SPEC audit — v0.1-draft passes CODE + SECURITY + ARCHITECT audit lanes at 0-CRITICAL / 0-HIGH.

3. Implementation (separate PR, gated on locked SPEC):

   • New column(s) in beta/runs.sqlite for CPU-time measurement (additive schema; existing rows treat NULL as "not measured").
   • beta/harness.py probes per-request CPU time with <1% wall-time overhead, measured.
   • Measurement brackets the model-forward-pass work, not the SSE decode loop or network wait.
   • beta/report.py::summarize_per_workload surfaces cpu_share_median and cpu_share_p95 per class per hardware target.
   • Reproducibility protocol implemented — e.g., 3-run median with thermal state annotation.

4. Measurable acceptance (in the impl PR's tests):

   • A completed sweep report answers "on air5 for code_completion, what percent of stage time was CPU-bound?" with a concrete number.
   • Overhead measurement shows <1% wall-time addition.
   • CPU-share metrics reproduce within reasonable variance across 3 identical sweep runs on the same hardware.

Do NOT (until gates 1+2 close)

• Modify beta/runs.sqlite schema or beta/harness.py.
• Assume the outcome — do not pre-write "CPU-bound on Apple Silicon confirmed" documentation.
• Introduce dependencies on Xcode/Metal instrumentation frameworks that break Air-only providers.

Related

• Companion to SPEC-028 (speculative decoding) and the sweep-workload-class research PR.
• Depends on: none.
• Blocks: interpretation of any post-SPEC-028 sweep re-run.