bench(ane): the Neural Engine as a third execution engine, ANE-0..ANE-4

[chrishayuk]

What the Neural Engine is worth to LARQL decode, on this machine, established by falsification. Harnesses, banked measurements with provenance, and eight adjudications that each state what they do not license.

No production code paths change — this is bench/ane/, bench/baselines/ane/, three larql-compute-metal examples, and a roadmap.

The ladder

rung	result
ANE-0	Apple's 2026 Core AI stack is unavailable here (active SDK 15.5). The ANE is reachable only through macOS 15-era Core ML — and computeUnits is a preference, so "ran on ANE" without an MLComputePlan is not a measurement.
ANE-0b	Frozen same-shape GPU control at Qwen3.8's real gate/up shape, 5120 → 17408: 288.7 / 289.1 GB/s across two sessions, 0.15% apart.
ANE-1	A real decode-shaped op places on the ANE — 111.9 GB/s equivalent, and its output agrees with Metal's to 1.3e-3 on the same projection.
ANE-2	Placement is governed by reduction depth, not size. Admissible k ≤ 16384; preferred from ~4992 (f16) / ~4096 (int8). Batching to N=8 costs 4%. int8 halves footprint, buys ~1.15×.
ANE-3	Concurrency is bubble recovery: GPU+ANE deliver 1.20–1.24× the GPU alone, costing the GPU 17–18%.
ANE-4	A real Qwen FFN block stays 8/12 on the ANE, seam cost ~2.4%. The down_proj k-split achieves full residency and is rejected — 5–8% slower and 6.2× less accurate.

The result that changes how placement should be chosen

Production FFN plan:  ANE gate/up  ->  CPU tail

Maximum accelerator residency is not the optimal physical plan. Forcing the tail back onto the ANE buys more residency, worse latency and substantially worse numerics simultaneously. That argues for VINDEX3 owning placement by measured cost and quality rather than treating accelerator percentage as an objective.

A second distinction fell out of the k-split: k ≤ 16384 behaves like a hard per-operator capability, while the ~4992 lower edge is a soft, graph-contextual preference — 4352-wide pieces stayed on the ANE when their neighbours were already there. A backend model should separate can_run from cost from place.

Two corrections, kept beside the results they correct

• "GPU + ANE ≈ 400 GB/s, the fabric ceiling" was a peak-vs-sustained artifact — those were min-latency rates. Sustained is 233–257 + 103–107, concurrent aggregate 311–323, all under ~400. The fabric was never tested.
• The ANE accuracy gap is a flat ~1.5e-2 precision floor, not fp16 accumulation. Error is constant across a 3.2× range of reduction depth (0.97×) while the CPU's grows 2.31×. The first explanation was wrong in a way that would have made the k-split look promising.

Both are recorded in place rather than rewritten, because the reasoning is the part worth reading.

Method

Every rung is banked with provenance — git SHA, hardware, OS, thermal state before and after, process snapshot — and every runner enforces its own protocol rather than relying on memory: refusing on battery unless overridden with a label that says so, refusing to overwrite a banked session. Measurements are synthetic-tensor and battery-regime throughout, and each adjudication says so.

Three hazards were caught by controls rather than luck:

• a silent CPU-BLAS fallback below flop_threshold that would have reported a device switch as a batching curve;
• a SIGPIPE that killed a runner between writing provenance and producing a result;
• a 9.8% power-state drift that had flipped ANE-3's verdict, caught by order-reversal and cut to 0.82% by a pre-load ramp.

Follow-on

The wide-and-shallow drafter thesis this produced is being pursued separately in #311, which builds the reduced-depth execution slice it needs.