feat(server): lossless CPU-isolated tool-call prediction

[davide221]

Summary

• add automatic, model-agnostic tool-call prediction to the server
• support a native Qwen3 predictor lane and any OpenAI-compatible HTTP predictor
• prelaunch allowlisted read-only tools on an isolated CPU lane
• expose a private result only after an exact canonical name-and-arguments match
• preserve DS4/DSpark decoding with no token injection or autoregressive fallback
• include a no-training trace compiler for recurring multi-tool workflows

Why

Agent latency is often dominated by the repeated sequence “decode a tool call,
wait for the tool, decode the next call.” This change overlaps the first tool
with target decoding and, for a previously validated recurring workflow,
collapses redundant model/tool synchronization barriers into one typed macro.

Design

1. The request's normalized dialogue and tool schemas go to a semantic
   predictor. The core interface is model-agnostic; Qwen3 is one native adapter.
2. The predictor proposes one concrete function and JSON argument object.
3. The engine canonicalizes the call and starts it privately only when its tool
   is explicitly allowlisted and its measured execution lane passes admission.
4. DeepSeek-V4 continues through DS4/DSpark and remains authoritative.
5. An exact canonical match commits the private result. A miss, timeout,
   disconnect, malformed prediction, or different target call cancels/discards
   it; the caller runs the authoritative call normally.

The Lucebox default is before-model: Qwen runs briefly on Strix, leaves the
compute window, and the CPU tool overlaps DeepSeek on R9700 + Strix. This avoids
shared-GPU predictor contention. An explicit overlap mode remains available
for deployments with independently qualified accelerator capacity.

Production result

Lucebox5, DeepSeek-V4-0731 + DS4/DSpark, Qwen3-0.6B Q8_0 on Strix, CPU tools on
14-15,30-31, paired randomized tasks with 10, 15, and 20 leaf calls:

Metric	Result
Normal stage-batched workflow, p50	81.030 s
Trace-compiled + speculative workflow, p50	14.597 s
End-to-end paired speedup, p50	5.5961x
Bootstrap 95% CI	5.4577x–5.6599x
Trace compilation alone	3.2806x
Early launch on top of compilation	1.6954x
Early-launch 95% CI	1.6760x–1.7210x
Exposed tool wait, compiled / speculative p50	10.143 s / 0.027 ms
Target model-compute change, p50 / p95	-0.458% / -0.332%
Target decode change, p50 / p95	-0.101% / +0.219%
Exact predictor hits	6 / 6

The 5.60x number applies to recognized, side-effect-free recurring
workflows. The isolated contribution from early tool launch is 1.70x; this
PR does not claim that arbitrary single tool calls become 5.60x faster.

Safety and compatibility

• disabled unless predictor, executor, empirical lane profile, and allowlist exist
• only read-only/idempotent tools are eligible
• child executor is launched directly without a shell, pinned to verified CPUs,
  bounded by timeout and output-size limits
• results remain private until the target call matches exactly
• predictor failures do not alter target output
• active tool prediction explicitly disables AR retry; DS4/DSpark remains active
• a per-request opt-out restores the ordinary decoder behavior
• single-GPU deployments use the same API and run prediction before target compute
• HTTP predictors allow other model families without changing the execution gate

Verification

• clean HIP build on Lucebox5 (gfx1151;gfx1201)
• server suite on current main: 399 / 399 passed
• benchmark/compiler suite: 27 / 27 passed
• native Qwen smoke: 12 / 12 valid names, 9 / 12 exact arguments
• current-main production smoke: exact Qwen hit, 10 / 10 leaf calls/results, correct final output, DS4 active (90.6% acceptance)
• production workflow gate: 20 / 20 checks passed
• identical leaf-call digests, result digests, macro calls, and final outputs
• positive DS4 acceptance on every measured tool-call turn
• no production retrying with AR or tool-speculation errors
• git diff --check

Reproduction commands and the compact evidence artifact are in
optimizations/ooo_spec_lucebox5_cpu/README.md.

Review map

• semantic predictor contract/parsing: server/src/server/semantic_tool_hint.*
• native Qwen adapter: server/src/server/native_semantic_tool_predictor.*
• isolated predictor IPC: server/src/common/qwen3_tool_predictor_ipc*
• request scheduling and exact commit: server/src/server/http_server.*
• canonical execution gate: server/src/server/tool_speculation.*
• trace compiler and benchmark: optimizations/ooo_spec_lucebox5_cpu/

[davide221]

Consolidated into #614 to preserve the original tool-speculation PR and its history. PR #614 now contains the exact tested final tree, benchmark results, and current-main integration from this draft; no implementation was dropped.