Skip to content

fix(gpu): normalise float_moment by the bit-depth scaler on CUDA/SYCL/HIP, and size HIP ciede chroma with ceil - #1371

Draft
lusoris wants to merge 5 commits into
masterfrom
fix/moment-bit-depth-and-ciede-hip-chroma
Draft

fix(gpu): normalise float_moment by the bit-depth scaler on CUDA/SYCL/HIP, and size HIP ciede chroma with ceil#1371
lusoris wants to merge 5 commits into
masterfrom
fix/moment-bit-depth-and-ciede-hip-chroma

Conversation

@lusoris

@lusoris lusoris commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Two twin-drift defects found by the sweep and upheld by three independent adversarial verifiers before any code was touched. Both hardware-verified on all three GPUs here.

ADR-1212 — float_moment on CUDA/SYCL/HIP was 4×/16×/256× too large above 8 bpc. The CPU reference runs picture_copy() first, which divides every 10/12/16-bit sample by 4/16/256 before moment.c accumulates; the three GPU twins accumulated the raw codeword and only divided by the pixel count (the HIP kernel's own comment said "no normalisation"). Metal was the only conforming twin. Nothing could see it: every parity fixture in the tree is 8-bit, where the scaler is 1.

Netflix 576x324 pair, yuv420p10le, frame 0:

ref1st ref2nd
CPU 61.928749785665296 4935.612488211591
CUDA before 247.71499914266118 78969.79981138546
CUDA / SYCL / HIP after 61.9287498 4935.61249

Fix is on the host: divide the exact integer device sums by the scaler (and scaler² for the 2nd moment). Not an approximation — every x/scaler the CPU sums is an exact multiple of 1/scaler, so this is bit-identical at 10 and 12 bpc; 16 bpc agrees to float precision (the CPU rounds each float square). 8-bit results don't change. Verified identical at 8/10/12 bpc on RTX 4090 (CUDA), Arc A380 (SYCL) and gfx1030 (HIP).

ADR-1213 — ciede_hip read past its chroma staging on odd dimensions. It sized staging with w >> 1 where picture.c allocates chroma as (w + 1) >> 1 (ceil; its comment names this exact hazard) and every other backend consumes the picture's real w[1]. At 577 wide the last chroma column was never uploaded and cx = x >> 1 for the last luma column read the first sample of the next row — past the allocation on the last row. Every ciede fixture was even-sized.

Gates added, because both bugs lived in the same blind spot (one fixture, one bit depth, even sizes):

  • float_moment parity TUs are bit-depth generic (FIXTURE_BPC > 8 writes uint16 with an independent low-bit pattern) and registered again at 10 bpc on CUDA and SYCL.
  • HIP had no float_moment parity test at all — added, at 8 and 10 bpc.
  • test_hip_ciede_parity_oddw at 577×325.

All new tests pass on their hardware: CUDA 1/1 + 1/1, SYCL 2/2 + 2/2, HIP 2/2 + 2/2, ciede even 2/2 + odd 2/2.

Type

  • fix — bug fix
  • test — test-only (new gates)
  • sycl / cuda / simd — backend-specific (CUDA, SYCL, HIP)

Checklist

  • Commits follow Conventional Commits.
  • make format && make lint is green locally (pre-commit run --files clean; check-state-md-rows OK, ADR index in sync).
  • Unit tests pass: meson test -C build — the touched targets on all three backends, listed above.
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2 — equivalent done directly: CPU vs each GPU twin prints identical values to 9 significant digits at 8/10/12 bpc.
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap — CUDA, SYCL and HIP fixed; Metal was already correct on normalisation, but has a separate float32-tile-reduction drift at ≥10 bpc listed under follow-ups.
  • If I added a new .c / .h, it has the appropriate license header — test_hip_float_moment_parity.c carries the Copyright 2026 Lusoris header.
  • If this is a breaking change — not breaking; the previous GPU values were wrong.
  • If this PR adds an ADR, the row lives in docs/adr/_index_fragments/ — in sync.

Bug-status hygiene (ADR-0165)

  • docs/state.md updated — T-GPU-FLOAT-MOMENT-RAW-CODEWORD-2026-09-06 and T-HIP-CIEDE-CHROMA-FLOOR-OOB-2026-09-06 closed; T-METAL-FLOAT-MOMENT-FLOAT32-TILE-REDUCE-2026-09-06 opened.

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change — none does; the golden gate is CPU-only and 8-bit, neither of which changes.

Cross-backend numerical results

Per-backend float_moment vs CPU on src01_hrc00/hrc01_576x324, frame 0, all four features (ref1st dis1st ref2nd dis2nd):

bpc CPU CUDA SYCL HIP
8 61.9287498 / 62.1152049 / 4935.61249 / 4909.31265 same same same
10 same same (was 247.715 / 248.461 / 78969.8 / 78549.0) same same
12 same same same same

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: the method is docs/research/2032-gpu-parity-resolution-blind-spot.md applied to bit depth instead of resolution, and ADR-1212's Context carries the specific evidence.
  • Decision matrix## Alternatives considered in ADR-1212 (host division vs in-kernel normalisation vs staging through picture_copy vs rejecting bpc != 8) and ADR-1213.
  • AGENTS.md invariant note — added to core/src/feature/AGENTS.md.
  • Reproducer / smoke-test command — below.
  • CHANGELOG fragmentchangelog.d/fixed/gpu-float-moment-bit-depth-and-hip-ciede-chroma.md.
  • Rebase notedocs/rebase-notes.md, "ADR-1212 / ADR-1213 — bit-depth normalisation in GPU moment twins, HIP chroma geometry".

Reproducer

Y=python/test/resource/yuv
for f in float_moment float_moment_cuda; do
  vmaf -r $Y/src01_hrc00_576x324.yuv420p10le.yuv -d $Y/src01_hrc01_576x324.yuv420p10le.yuv \
       --width 576 --height 324 --pixel_format 420 --bitdepth 10 --frame_cnt 1 --no_prediction \
       --feature $f --output /dev/stdout --json --precision=max | grep -oE '"float_moment_ref1st": *[0-9.]+'
done
meson test -C build test_cuda_float_moment_parity_10bit test_hip_float_moment_parity_10bit test_hip_ciede_parity_oddw

On master the CUDA line prints ~247.7 against the CPU's ~61.9; on this branch both print 61.928749785665296 and the tests pass.

Known follow-ups

  • T-METAL-FLOAT-MOMENT-FLOAT32-TILE-REDUCE-2026-09-06: the Metal twin reduces per-tile partials in float32 where the CPU uses double; exact at 8 bpc, drifts at ≥10 bpc on large frames. Needs Apple hardware.
  • 16 bpc agrees to float precision rather than bit-for-bit because the CPU rounds each float square before accumulating; within the places=4 gate.

🤖 Generated with Claude Code

lusoris pushed a commit that referenced this pull request Sep 6, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
lusoris pushed a commit that referenced this pull request Sep 7, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/moment-bit-depth-and-ciede-hip-chroma branch from f090b1a to 9a2baeb Compare September 7, 2026 06:54
lusoris pushed a commit that referenced this pull request Sep 7, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/moment-bit-depth-and-ciede-hip-chroma branch from 9a2baeb to 5b4ca99 Compare September 7, 2026 09:14
lusoris pushed a commit that referenced this pull request Sep 7, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/moment-bit-depth-and-ciede-hip-chroma branch from 5b4ca99 to c0513e2 Compare September 7, 2026 09:17
lusoris pushed a commit that referenced this pull request Sep 7, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/moment-bit-depth-and-ciede-hip-chroma branch from c0513e2 to bda9cae Compare September 7, 2026 10:18
lusoris pushed a commit that referenced this pull request Sep 7, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/moment-bit-depth-and-ciede-hip-chroma branch from bda9cae to 6ae49c9 Compare September 7, 2026 11:14
lusoris pushed a commit that referenced this pull request Sep 7, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/moment-bit-depth-and-ciede-hip-chroma branch from 6ae49c9 to d7f56a7 Compare September 7, 2026 16:08
Lusoris and others added 5 commits September 7, 2026 18:38
…/HIP, and size HIP ciede chroma with ceil

Two twin-drift defects, both hardware-reachable with ordinary inputs.

ADR-1212 — float_moment on CUDA, SYCL and HIP accumulated the RAW 10/12/16-bit
codeword. The CPU reference runs picture_copy() first, which divides every
sample by 4 (10 bpc), 16 (12 bpc) or 256 (16 bpc) before moment.c accumulates
it. Measured on src01_hrc00/hrc01 576x324 yuv420p10le, frame 0:

    CPU  float_moment_ref1st = 61.928749785665296   ref2nd = 4935.612488211591
    CUDA float_moment_ref1st = 247.71499914266118   ref2nd = 78969.79981138546

i.e. exactly 4x and 16x too large. Metal was the only conforming twin. The
device sums are exact integers, so dividing them by the scaler (and scaler^2
for the second moment) on the host reproduces the CPU's sum(x / scaler)
bit-for-bit at 10 and 12 bpc; at 16 bpc the CPU rounds each float square, so
agreement is to float precision. No 8-bit result changes.

ADR-1213 — ciede_hip sized its chroma staging buffers with floor(w/2) /
floor(h/2), while core/src/picture.c allocates chroma planes as
(w + ss) >> ss (ceil) and every other implementation consumes the picture's
real w[1]/h[1]. On odd luma dimensions the last chroma column/row was never
uploaded and the kernel's cx = x >> 1 for the last luma column read one element
past the staged row: into the next row, and past the allocation on the last
row. Fixed by using the same ceil formula as picture.c.

Both found by the twin-drift sweep and independently upheld by adversarial
verification (reachability, arithmetic, not-already-fixed) before any code was
touched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d add 10-bit variants

Every parity fixture in the tree was 8-bit, so a twin that accumulates the raw
codeword instead of the bit-depth-normalised sample could never be caught.
The CUDA and SYCL float_moment parity TUs now write uint16 samples when
FIXTURE_BPC > 8 — 8-bit pattern in the high bits, a second pattern in the low
bits so a twin that only looks at the high bits cannot match by accident —
and meson registers each once more at 10 bpc (ADR-1212).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… and an odd-width ciede variant

float_moment_hip had no cross-backend gate at all, which is how it shipped
accumulating the raw high-bit-depth codeword (ADR-1212). The new TU mirrors
test_hip_float_psnr_parity.c's skip contract and is registered at 8 bpc and,
via -DFIXTURE_BPC=10u, at 10 bpc. test_hip_ciede_parity is additionally
registered at 577x325 (ADR-1213): every ciede fixture was even-sized, so the
floor(w/2) chroma staging could never be caught.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s, add the bit-depth fixture invariant

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/moment-bit-depth-and-ciede-hip-chroma branch from d7f56a7 to f3a4a9f Compare September 7, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant