src/model.c:2137 gates the vectorized VQ4P apply:
#if (defined(__ARM_NEON) || defined(__aarch64__)) && !defined(WASTE_P6_SCALAR)
The vectorized side is four vqtbl4q_s8 per 16-row tile — a 64-byte table
lookup — over a vld3q_u8 deinterleave, with widening adds into int16. An x86
build takes the else branch: one row at a time, four scalar table loads per
row. It is not a narrower kernel, it is the absence of one — and on the same
build VQ3R's table gets simd_avx512.c.
Reported by @fab2s in #11, while trying to run the 64-entry crossover
docs/LEARNED.md §50 left open. Confirmed here by reading rather than by
measurement: this repo has no x86 machine that is not emulated, which is also
why docs/BACKENDS.md records AVX-512 as "compiled and dispatched, never
executed".
What is being assumed because of it
- The 64-entry crossover cannot be measured on an x86 CPU at all. §50 left
open whether VQ4P's 72 KB int8 table inverts the LUT-versus-reconstruct
result it measured at 256 entries. On x86 that comparison puts a scalar apply
against a vectorized one, so the number would be answering an ISA question
and not a table-size one.
- §47's tuning table is an ARM result.
WASTE_XPAR, WASTE_XPAR_BATCH,
WASTE_P6_CHUNK and the thread counts were measured against the vectorized
apply, and CLAUDE.md carries them as engine-wide guidance. On x86 the kernel
underneath them is a different kernel, and the inversion between Kimi-Linear
and K3 has never been retested there.
- VQ4P is the format
convert.py produces by default, so this is the path
on every x86 build, not a corner case.
What might close it
vqtbl4q_s8 looks up 64 bytes across four registers. The AVX-512 analogue is
VBMI's vpermi2b, which indexes 128 bytes across two zmm registers — the
same primitive with room to spare for a 64-entry table. Zen 4 and later, and
Ice Lake and later, have VBMI. src/simd_avx512.c already exists as the place
it would go, and dispatch resolves once at init, so this is a slot to fill
rather than an #ifdef in model.c.
Unmeasured in both directions, and worth stating before anyone spends a
weekend on it: the path may still be bound by the deinterleave rather than by
the lookup, in which case the table primitive buys much less than it looks
like it should. Cross-lane byte permutes are also not uniformly cheap across
x86 microarchitectures. This is a hypothesis with an obvious first experiment,
not a plan.
The doc sentence to qualify either way
docs/BACKENDS.md says:
The third hot path, the VQ gather, gets nothing — no x86 SIMD helps it
either, for the same reason NEON does not.
That was written about VQ3R's 256-entry table, where it holds: 256 entries fit
no byte-permute primitive. It is now being read as covering VQ4P, where the
premise it rests on is false. The sentence needs qualifying whether or not
anyone writes the kernel.
Acceptance, if someone does write it
- Bit-identity, not "close".
-DWASTE_P6_SCALAR already makes the ARM path
self-checking against the portable one, and §43 says why the bar is that
high: an int8 table makes the engine discontinuous, so an approximate match
is not a match. An x86 path has to clear the same bar against the same
reference.
- The check that decides it is
tests/run.sh's SIMD backend matches the
CPU baseline, which runs WASTE_BACKEND=cpu against the dispatched path and
compares logits. It has to run on an --index-bits 6 container: the suite's
default WASTE_REF_MODEL is a VQ3R conversion, and per CLAUDE.md running
the suite on the wrong container shape is exactly how a load path once stayed
broken through green runs.
- No machine here can execute it. Whoever writes it is also the only person
who can test it, so the measurement matters more than the patch.
src/model.c:2137gates the vectorized VQ4P apply:The vectorized side is four
vqtbl4q_s8per 16-row tile — a 64-byte tablelookup — over a
vld3q_u8deinterleave, with widening adds into int16. An x86build takes the
elsebranch: one row at a time, four scalar table loads perrow. It is not a narrower kernel, it is the absence of one — and on the same
build VQ3R's table gets
simd_avx512.c.Reported by @fab2s in #11, while trying to run the 64-entry crossover
docs/LEARNED.md§50 left open. Confirmed here by reading rather than bymeasurement: this repo has no x86 machine that is not emulated, which is also
why
docs/BACKENDS.mdrecords AVX-512 as "compiled and dispatched, neverexecuted".
What is being assumed because of it
open whether VQ4P's 72 KB int8 table inverts the LUT-versus-reconstruct
result it measured at 256 entries. On x86 that comparison puts a scalar apply
against a vectorized one, so the number would be answering an ISA question
and not a table-size one.
WASTE_XPAR,WASTE_XPAR_BATCH,WASTE_P6_CHUNKand the thread counts were measured against the vectorizedapply, and
CLAUDE.mdcarries them as engine-wide guidance. On x86 the kernelunderneath them is a different kernel, and the inversion between Kimi-Linear
and K3 has never been retested there.
convert.pyproduces by default, so this is the pathon every x86 build, not a corner case.
What might close it
vqtbl4q_s8looks up 64 bytes across four registers. The AVX-512 analogue isVBMI's
vpermi2b, which indexes 128 bytes across two zmm registers — thesame primitive with room to spare for a 64-entry table. Zen 4 and later, and
Ice Lake and later, have VBMI.
src/simd_avx512.calready exists as the placeit would go, and dispatch resolves once at init, so this is a slot to fill
rather than an
#ifdefinmodel.c.Unmeasured in both directions, and worth stating before anyone spends a
weekend on it: the path may still be bound by the deinterleave rather than by
the lookup, in which case the table primitive buys much less than it looks
like it should. Cross-lane byte permutes are also not uniformly cheap across
x86 microarchitectures. This is a hypothesis with an obvious first experiment,
not a plan.
The doc sentence to qualify either way
docs/BACKENDS.mdsays:That was written about VQ3R's 256-entry table, where it holds: 256 entries fit
no byte-permute primitive. It is now being read as covering VQ4P, where the
premise it rests on is false. The sentence needs qualifying whether or not
anyone writes the kernel.
Acceptance, if someone does write it
-DWASTE_P6_SCALARalready makes the ARM pathself-checking against the portable one, and §43 says why the bar is that
high: an int8 table makes the engine discontinuous, so an approximate match
is not a match. An x86 path has to clear the same bar against the same
reference.
tests/run.sh's SIMD backend matches theCPU baseline, which runs
WASTE_BACKEND=cpuagainst the dispatched path andcompares logits. It has to run on an
--index-bits 6container: the suite'sdefault
WASTE_REF_MODELis a VQ3R conversion, and perCLAUDE.mdrunningthe suite on the wrong container shape is exactly how a load path once stayed
broken through green runs.
who can test it, so the measurement matters more than the patch.