perf(speed): dispatch the SpEED matrix product through bit-exact AVX2/AVX-512 kernels

[lusoris]

Summary

A perf record profile of the CPU pipeline under the default model
(vmaf_v1.0.16_3d0h) put matrix_mul in core/src/feature/speed.c at
20.78 % of total samples — the dense product behind SpEED's Householder QR
and its QᵀB solve, running on 128-bit mulps/addps because the generic C
library compiles at the x86-64 baseline while the host has AVX-512. This PR
adds bit-exact AVX2 and AVX-512 twins, runtime-dispatched exactly like the
existing compute_cov_kernel_* family, and publishes the full profile as a
research digest including the wins that were found and deliberately not
taken because they would move scores.

Result: 1.20x on the default-model CPU run, scores byte-identical on the
scalar, AVX2 and AVX-512 dispatch paths, Netflix golden gate 271 passed /
12 skipped / 0 failed.

Type

• perf — performance improvement
• sycl / cuda / simd — backend-specific

Checklist

• Commits follow Conventional Commits (the commit-msg hook enforces this).
• make format && make lint is green locally. clang-format clean on every touched file; clang-tidy -p <no-LTO CPU build> reports 0 findings on both new kernels and on speed.c; core/test/test_speed_simd.c stays at its ADR-1142 baseline of 5 (4 × modernize-use-nullptr + 1 × readability-function-size) — the extra return NULL the new helper would have added is offset by turning run_tests's early return NULL guard into an if (have) block. No baseline file is touched. scripts/ci/assertion-density.sh, check-copyright.sh, check-adr-numbering.sh, check-conflict-markers.sh, check-no-tracked-venv.sh, check-state-md-rows.sh and both fragment --checks pass.
• Unit tests pass: meson test -C core/build → 129 Ok, 0 Fail.
• If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. — stronger than that here: 0 ULP, byte-identical --precision=max JSON. See "Cross-backend numerical results". No GPU path is touched by this PR.
• If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below.
• If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (Copyright 2026 Lusoris, verified by scripts/ci/check-copyright.sh).
• If this is a breaking change, the commit message uses ! or BREAKING CHANGE: — not a breaking change; no public header, CLI flag or meson_options.txt entry changes.
• If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/1196-speed-matmul-simd-dispatch.md and the slug is appended to _order.txt; docs/adr/README.md is regenerated by the script.

Bug-status hygiene (ADR-0165)

• docs/state.md updated — no state delta: performance work with no user-visible behaviour change; no bug is opened, closed or ruled not-affecting by this PR. The two open GPU rows the digest cites (T-GPU-MOTION-FLUSH-DOUBLE-EMIT-2026-09-06, T-GPU-CLI-THREADS-CTX-SYNC-2026-09-06) already exist and are unchanged.

Netflix golden-data gate (ADR-0024)

• I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests. The gate is untouched and green:

271 passed, 12 skipped, 5979 warnings in 79.10s

• If I believe a golden value must change — I do not. No score moves anywhere in this PR.

Cross-backend numerical results

This PR changes only CPU dispatch, so the meaningful comparison is
scalar vs AVX2 vs AVX-512 against the pre-change binary. --precision=max
(%.17g) JSON, unmodified origin/master build vs this branch, same fixture
(src01_hrc00_576x324.yuv / src01_hrc01_576x324.yuv), --cpumask bits name
the ISAs to disable:

                       default v1.0.16_3d0h       vmaf_v0.6.1.json
cpumask=56  (scalar)   IDENTICAL  82.81605876127689   IDENTICAL  76.66783086300072
cpumask=48  (AVX2)     IDENTICAL  82.81605876127689   IDENTICAL  76.66783086300072
cpumask=0   (AVX-512)  IDENTICAL  82.81605876127689   IDENTICAL  76.66783086300072

"IDENTICAL" = diff of the two JSON files with only the "fps" and
"version" lines excluded is empty. Worst ULP: 0.

core/test/test_speed_simd additionally gates the twins with memcmp
equality against the production scalar reference (not a copy of it) across the
25x25x25 QR shape, the 25x25x448 rectangular solve, a cols=41 shape hitting
every tail branch, and a cols=3, inner=1 degenerate shape:

16 tests run, 16 passed

Performance (if perf or feat)

Host ryzen-4090-arc, base = origin/master build, new = this branch, both
--buildtype=release -Db_ndebug=true, CUDA/SYCL/HIP off. Timing is
ru_utime + ru_stime of the child process via os.wait4, base and new
interleaved run-for-run so machine drift hits both sides equally. Median of N
repetitions after a discarded warm-up; the 1-minute load average during each
round is quoted because this workstation was not idle for most of the session.

Fixture	Model	Reps	Load	Base median	New median	Speedup
src01 pair, 48 f	default v1.0.16_3d0h	11	4.10	0.0829 s (spread 4.7 %)	0.0689 s (spread 1.8 %)	1.202x
src01 pair ×30, 1440 f	default v1.0.16_3d0h	9	2.81	2.4820 s (spread 4.3 %)	2.0674 s (spread 5.0 %)	1.201x
src01 pair ×30, 1440 f	vmaf_v0.6.1.json	9	2.25	2.1293 s (spread 8.3 %)	2.1212 s (spread 8.1 %)	1.004x — no change

The v0.6.1 row is the control: that model has no SpEED feature, so these
kernels are never reached, and 1.004x is inside the noise of no change (four
rounds were run in total; the two that returned 0.977x and 0.938x had 24–50 %
spread under load 14–51, with the patched side marginally faster at its
minimum). Earlier default-model rounds under load 15–16 gave 1.188x and
1.134x — the direction is stable across every round and the magnitude
compresses under contention, which is why the near-idle round is the one
quoted.

Profile before → after, perf report --no-children -g none, same command,
13 289 / 10 522 samples:

                                       before    after
preprocess_and_extract_cambi           23.52%   28.17%
calculate_c_values_avx2                 9.09%   11.12%
convolution_f32_avx_s                   5.88%    6.66%
matrix_mul  ->  speed_matmul_avx512    20.78%    5.67%   <-- this PR
est_params                              4.97%    5.48%

A 15.1-point drop in total samples predicts 1/(1−0.151) = 1.18x; the measured
1.20x agrees.

GPU profilers are not installed on this host — ncu, nsys, vtune and
rocprof are all absent (/opt/cuda/bin has nvcc, cuda-gdb and
compute-sanitizer only), so no CUDA / SYCL / HIP kernel profile was
collected. A GPU profile would in any case have been blocked by
T-GPU-MOTION-FLUSH-DOUBLE-EMIT-2026-09-06: no GPU backend currently
completes a scored run longer than one motion batch.

Deep-dive deliverables (ADR-0108)

• Research digest — docs/research/2030-speed-matmul-and-cambi-cpu-hot-path.md.
• Decision matrix — docs/adr/1196-speed-matmul-simd-dispatch.md ## Alternatives considered (six options, including the two larger wins that were rejected for moving scores).
• AGENTS.md invariant note — core/src/feature/AGENTS.md, new section "matrix_mul dispatches; si_mat_mul deliberately does not (ADR-1196)".
• Reproducer / smoke-test command — pasted below under "Reproducer".
• CHANGELOG fragment — changelog.d/changed/speed-matmul-simd-dispatch.md; CHANGELOG.md regenerated by scripts/release/concat-changelog-fragments.sh.
• Rebase note — docs/rebase-notes.md, "perf/hot-path-1245 — SpEED matrix_mul gains a kernel-pointer parameter (2026-09-06)".

Reproducer

meson setup build-prof core -Denable_cuda=false -Denable_sycl=false -Denable_hip=false \
      --buildtype=release -Db_ndebug=true \
      -Dc_args='-g -fno-omit-frame-pointer' -Dcpp_args='-g -fno-omit-frame-pointer'
ninja -C build-prof

# 1. bit-exactness: all three dispatch paths must agree, byte for byte
for mask in 0 48 56; do
  ./build-prof/tools/vmaf \
      -r python/test/resource/yuv/src01_hrc00_576x324.yuv \
      -d python/test/resource/yuv/src01_hrc01_576x324.yuv \
      -w 576 -h 324 -p 420 -b 8 --no_cuda --no_sycl --no_hip \
      --cpumask $mask --json --precision=max -o out-$mask.json
done
diff <(grep -vE '"fps"|"version"' out-0.json) <(grep -vE '"fps"|"version"' out-56.json)   # empty

# 2. SIMD parity unit tests (memcmp-exact)
./build-prof/test/test_speed_simd            # 16 tests run, 16 passed

# 3. the profile that motivated the change
perf record -F 4999 -g --call-graph=fp -o perf.data -- \
  ./build-prof/tools/vmaf -r ref1440.yuv -d dis1440.yuv -w 576 -h 324 -p 420 -b 8 \
     --no_cuda --no_sycl --no_hip --json -o /dev/null
perf report -i perf.data --no-children -g none --percent-limit 0.3 --stdio

Known follow-ups

Ranked in the digest with the evidence for each:

1. Householder rank-1 update (Q ← Q − 2v(vᵀQ)) instead of the explicit
   25x25x25 product — O(n²) per QR iteration instead of O(n³), roughly a
   further 25x on the same code, and still the largest single SpEED CPU item
   after this PR. Not taken: it changes the summation order, so scores
   move. Needs its own PR with a snapshot-regeneration justification and a
   cross-backend re-parity pass.
2. anti_dithering_filter in cambi.c (4.98 % of the baseline run) — a
   2x2 uint16_t box filter with >>2, done in place, and the in-place
   update provably has no read-after-write hazard within a row. Widening it to
   AVX2 via 32-bit unpack/add/shift/pack is integer-exact by construction.
   Left out only to keep this PR to one subsystem; it is the obvious next
   bit-exact increment.
3. CAMBI's quick_select_partition (10.22 %) and average_topk_elements
   (3.70 %) — now the largest remaining items, but both are order-sensitive
   (the top-k sum accumulates in array order into a double), so neither
   admits a bit-exact rewrite.
4. si_mat_mul in speed_internal.c — the ADR-0964 duplicate of the same
   loop used by the host side of the CUDA / SYCL / HIP SpEED twins. Routing it
   through the same dispatch is bit-exact by the identical argument, but the
   benefit cannot be measured on this host while
   T-GPU-MOTION-FLUSH-DOUBLE-EMIT-2026-09-06 is open, so it is deliberately
   left scalar rather than landed on faith. Documented in
   core/src/feature/AGENTS.md so a future reader does not "resync" it as
   drift.

No GPU twin is out of date as a result of this PR: speed_chroma /
speed_temporal on CUDA, SYCL and HIP go through speed_internal.c, which is
byte-for-byte unchanged.