Skip to content

fix(gpu): collapse the four divergent ADM angle_flag predicates onto one - #1342

Draft
lusoris wants to merge 5 commits into
masterfrom
fix/t-upstream-930-adm-angle-flag-predicate-
Draft

fix(gpu): collapse the four divergent ADM angle_flag predicates onto one#1342
lusoris wants to merge 5 commits into
masterfrom
fix/t-upstream-930-adm-angle-flag-predicate-

Conversation

@lusoris

@lusoris lusoris commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

What

decouple()'s integer-ADM 1-degree angle_flag test had four different
spellings in the tree (T-UPSTREAM-930, Netflix/vmaf#930):

Site Evaluation
integer_adm.c, adm_avx2.c, adm_avx512.c narrow each int64 operand to float, compare in doublethe golden-frozen form
CUDA / HIP decouple_angle_flag_s0 compare the exact int64 products in double
CUDA / HIP decouple_angle_flag_s123 the golden form — so s0 and s123 disagreed inside one backend
SYCL integer_adm_sycl.cpp (both scales) the whole comparison in float
Metal iadm_angle_flag_s0 exact int64 products narrowed to float

angle_flag selects the enhancement-gain-limited branch of decouple(), so a
flipped flag moves adm scores.

The predicate now lives once, in core/src/feature/adm_angle_flag.h:

  • adm_angle_flag_fp64() — the golden expression, verbatim. Scalar CPU, CUDA
    and HIP call it, at both scales.
  • adm_angle_flag_i64() — a bit-identical evaluation in 64-bit integer
    arithmetic using no floating point of any width. SYCL calls it (Intel Arc
    A-series and most iGPUs expose no fp64, and one fp64 instruction anywhere in
    that TU makes the runtime reject the whole SPIR-V module — the binary
    confirms it: SYCL: device lacks native fp64); metal/integer_adm.metal
    mirrors it in MSL, which has no double type at all.

The CPU expression is untouched — core/src/feature/integer_adm.c compiles to
byte-identical machine code against origin/master.

Why it is reachable

Instrumenting the scalar CPU adm_decouple() to evaluate all four historical
forms per scale-0 pixel:

Clip scale-0 pixels CUDA/HIP flips SYCL flips Metal flips
src01_hrc00/hrc01_576x324 (the Netflix golden pair) 1 540 608 0 2 3
sparks_ref/dis_480x270 10-bit 111 870 0 1 0
synthetic full-contrast 1080p noise, ±1 LSB distortion 1 017 036 11 11 18

Reproducer / smoke-test command

# 1. the regression test (host-only, no GPU)
meson setup core/build-cpu -Denable_cuda=false -Denable_sycl=false
ninja -C core/build-cpu -j 4 test/test_adm_angle_flag
core/build-cpu/test/test_adm_angle_flag        # 7 tests run, 7 passed

# 2. prove the CPU/golden lane did not move
git show origin/master:core/src/feature/integer_adm.c > /tmp/old.c
cc -Icore/build-cpu/src -Icore/src -Icore/src/feature -Icore/src/feature/common \
   -Icore/include -Icore/build-cpu/include -O3 -std=c23 -D_GNU_SOURCE -fPIC \
   -DVMAF_BUILDING_LIBVMAF -DVMAF_HAVE_DNN=1 -c core/src/feature/integer_adm.c -o /tmp/new.o
cc <same flags> -c /tmp/old.c -o /tmp/old.o
objdump -d --no-show-raw-insn /tmp/old.o | tail -n +3 > /tmp/a
objdump -d --no-show-raw-insn /tmp/new.o | tail -n +3 > /tmp/b
diff /tmp/a /tmp/b && echo "CPU code unchanged"

# 3. SYCL end-to-end on the golden pair (Arc A380), adm_enhn_gain_limit=1.2
source /opt/intel/oneapi/setvars.sh
CXX=icpx CC=icx meson setup core/build-sycl -Denable_sycl=true -Denable_cuda=false
ninja -C core/build-sycl -j 4
core/build-sycl/tools/vmaf --reference python/test/resource/yuv/src01_hrc00_576x324.yuv \
  --distorted python/test/resource/yuv/src01_hrc01_576x324.yuv --width 576 --height 324 \
  --pixel_format 420 --bitdepth 8 --backend sycl --model path=<v0.6.1 + adm_enhn_gain_limit=1.2> \
  --json --precision max -o /tmp/sycl.json

Evidence

Regression test, both directions. With adm_angle_flag_i64()'s body
temporarily replaced by the pre-fix SYCL all-float predicate:

test_cos_constant_is_one_float_everywhere: pass
test_golden_expression_on_divergent_quads: pass
test_legacy_gpu_forms_are_the_divergent_ones: pass
test_i64_helper_matches_golden_on_divergent_quads: fail
adm_angle_flag_i64 must match the golden angle_flag
4 tests run, 1 failed

With the shipped helper:

test_cos_constant_is_one_float_everywhere: pass
test_golden_expression_on_divergent_quads: pass
test_legacy_gpu_forms_are_the_divergent_ones: pass
test_i64_helper_matches_golden_on_divergent_quads: pass
test_i64_helper_handles_degenerate_operands: pass
test_i64_helper_matches_golden_on_scale0_sweep: pass
test_i64_helper_matches_golden_on_boundary_walk: pass
7 tests run, 7 passed

SYCL on the Arc A380, golden 576x324 pair, 48 frames, adm_enhn_gain_limit=1.2:

max |CPU - SYCL| over adm metrics:
   legacy float predicate : 1.174971e-05
   shared int64 predicate : 8.219720e-07
legacy vs fixed SYCL: 6 of 240 adm metrics changed (frames 3, 23, 30)

CUDA on the RTX 4090, synthetic noise clip, adm_enhn_gain_limit=1.2:

integer_adm2_egl_1.2        0.9993743386943251 -> 0.9993743655248627
integer_adm3_egl_1.2        0.9996871693471625 -> 0.9996871827624314
integer_adm_scale0_egl_1.2  0.999429676185588  -> 0.9994297761196714

Only scale 0 moves — exactly the site the CUDA/HIP change touches. At the
shipped default adm_enhn_gain_limit = 1.0 the flag gates only the
MIN(rst, t) clamp, which is frequently a no-op, and the CUDA adm output on
that clip is byte-identical before and after.

Fuzz. adm_angle_flag_i64() vs the golden expression: 200 000 000 scale-0
band quadruples and 103 813 779 boundary-walk triples spanning all 62 magnitude
classes plus a 20x20x20 edge-operand cube — 0 mismatches, repeated under
gcc and clang at -O0, -O2 -ffp-contract=off and -O3 -march=native.

Suite. meson test -C core/build-cpu --suite=fastOk: 115, Fail: 0.

Lint. clang-tidy -p core/build-tidy reports 0 diagnostics on both
core/src/feature/integer_adm.c and core/test/test_adm_angle_flag.c
(-Db_lto=false, per the tidy-lane note). pre-commit run --files <all touched>
is clean.

Not verified here

  • HIP was changed identically to CUDA (the two files are byte-for-byte twins
    for this helper) but not executed — the AMD iGPU lane was not exercised.
  • Metal cannot be built or run on Linux; its MSL mirror is verified only by
    review against the C header that the unit test pins. core/src/feature/AGENTS.md
    now records that the mirror is manual and must be edited in lockstep.
  • The nightly cuda / hip / sycl clang-tidy ratchet lanes (they do not
    run on PRs) may report a stale-high baseline for the three touched backend
    files, since the change removes inline expressions from them. The cpu lane —
    the one that gates PRs — is unaffected: neither touched cpu-lane file appears
    in scripts/ci/tidy-baseline-cpu.json and both measure 0.

Deep-dive deliverables

  • Research digestdocs/research/2030-adm-angle-flag-fp64-free.md

  • Decision matrix — ADR-1194 ## Alternatives considered (six rows)

  • AGENTS.md invariant notecore/src/feature/AGENTS.md, "angle_flag has exactly one definition"

  • Reproducer / smoke-test command — above

  • CHANGELOG fragmentchangelog.d/fixed/adm-angle-flag-predicate-divergence.md

  • Rebase notedocs/rebase-notes.md, "fix/t-upstream-930-adm-angle-flag"

  • ADR: docs/adr/1194-adm-angle-flag-single-source.md + index row

  • docs/state.md updated (row moved to Recently closed)

  • no docs needed: no user-discoverable surface changes — no new CLI flag, C API,
    build option, feature name or output key; the only user-visible effect is that
    GPU adm values move toward the CPU reference, which is what the ADR and the
    changelog fragment record.

  • no ffmpeg patch impact: no public header, CLI flag or meson_options.txt
    entry changes.

Closes T-UPSTREAM-930-ADM-ANGLE-FLAG-PREDICATE-DIVERGENCE-2026-09-03.
Refs Netflix/vmaf#930.

🤖 Generated with Claude Code

@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 6, 2026
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…osed

Fixed by PR #1342 (a3155b2, ADR-1194). ADR-0165 requires the ledger
row to move in the same PR as the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…osed

Fixed by PR #1342 (a3155b2, ADR-1194). ADR-0165 requires the ledger
row to move in the same PR as the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-930-adm-angle-flag-predicate- branch from 5719c79 to a6a10ff Compare September 6, 2026 03:04
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…osed

Fixed by PR #1342 (a3155b2, ADR-1194). ADR-0165 requires the ledger
row to move in the same PR as the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-930-adm-angle-flag-predicate- branch from a6a10ff to 7a961d7 Compare September 6, 2026 06:40
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…osed

Fixed by PR #1342 (a3155b2, ADR-1194). ADR-0165 requires the ledger
row to move in the same PR as the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…md move

Two loose ends on top of the ADR-1194 change:

1. `adm_angle_flag_i64()` tripped `readability-function-size` (71 lines
   against the 60-line threshold) once clang-tidy was pointed at the new
   header with `-header-filter='adm_angle_flag\.h'` — the default filter is
   anchored at `^core/` and so misses it when the build dir is a sibling.
   ADR-0141 wants the file clean, not suppressed, so the right-hand-side
   rounding is now `adm_angle_flag_round53_v()`. Pure extraction: no
   expression changes, and `core/test/test_adm_angle_flag.c` still passes
   7/7. `metal/integer_adm.metal` gets the same split so the manual MSL
   mirror stays line-for-line (the AGENTS.md lockstep invariant).

2. `docs/state.md` still carried T-UPSTREAM-930 under **Open bugs** with a
   comment claiming it had moved: an earlier duplicate-row cleanup dropped
   the Recently-closed copy instead of the stale open one. The open row is
   gone and the closed row is back at the head of Recently closed, citing
   PR #1342 and ADR-1194 (the previous text cited a commit sha that the
   rebase invalidated).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-930-adm-angle-flag-predicate- branch from 7a961d7 to 21ac8d2 Compare September 6, 2026 09:51
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…osed

Fixed by PR #1342 (a3155b2, ADR-1194). ADR-0165 requires the ledger
row to move in the same PR as the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…md move

Two loose ends on top of the ADR-1194 change:

1. `adm_angle_flag_i64()` tripped `readability-function-size` (71 lines
   against the 60-line threshold) once clang-tidy was pointed at the new
   header with `-header-filter='adm_angle_flag\.h'` — the default filter is
   anchored at `^core/` and so misses it when the build dir is a sibling.
   ADR-0141 wants the file clean, not suppressed, so the right-hand-side
   rounding is now `adm_angle_flag_round53_v()`. Pure extraction: no
   expression changes, and `core/test/test_adm_angle_flag.c` still passes
   7/7. `metal/integer_adm.metal` gets the same split so the manual MSL
   mirror stays line-for-line (the AGENTS.md lockstep invariant).

2. `docs/state.md` still carried T-UPSTREAM-930 under **Open bugs** with a
   comment claiming it had moved: an earlier duplicate-row cleanup dropped
   the Recently-closed copy instead of the stale open one. The open row is
   gone and the closed row is back at the head of Recently closed, citing
   PR #1342 and ADR-1194 (the previous text cited a commit sha that the
   rebase invalidated).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-930-adm-angle-flag-predicate- branch from 21ac8d2 to d308cfe Compare September 6, 2026 11:09
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…osed

Fixed by PR #1342 (a3155b2, ADR-1194). ADR-0165 requires the ledger
row to move in the same PR as the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris pushed a commit that referenced this pull request Sep 6, 2026
…md move

Two loose ends on top of the ADR-1194 change:

1. `adm_angle_flag_i64()` tripped `readability-function-size` (71 lines
   against the 60-line threshold) once clang-tidy was pointed at the new
   header with `-header-filter='adm_angle_flag\.h'` — the default filter is
   anchored at `^core/` and so misses it when the build dir is a sibling.
   ADR-0141 wants the file clean, not suppressed, so the right-hand-side
   rounding is now `adm_angle_flag_round53_v()`. Pure extraction: no
   expression changes, and `core/test/test_adm_angle_flag.c` still passes
   7/7. `metal/integer_adm.metal` gets the same split so the manual MSL
   mirror stays line-for-line (the AGENTS.md lockstep invariant).

2. `docs/state.md` still carried T-UPSTREAM-930 under **Open bugs** with a
   comment claiming it had moved: an earlier duplicate-row cleanup dropped
   the Recently-closed copy instead of the stale open one. The open row is
   gone and the closed row is back at the head of Recently closed, citing
   PR #1342 and ADR-1194 (the previous text cited a commit sha that the
   rebase invalidated).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-930-adm-angle-flag-predicate- branch from d308cfe to 4543cab Compare September 6, 2026 18:26
lusoris pushed a commit that referenced this pull request Sep 7, 2026
…osed

Fixed by PR #1342 (a3155b2, ADR-1194). ADR-0165 requires the ledger
row to move in the same PR as the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris pushed a commit that referenced this pull request Sep 7, 2026
…md move

Two loose ends on top of the ADR-1194 change:

1. `adm_angle_flag_i64()` tripped `readability-function-size` (71 lines
   against the 60-line threshold) once clang-tidy was pointed at the new
   header with `-header-filter='adm_angle_flag\.h'` — the default filter is
   anchored at `^core/` and so misses it when the build dir is a sibling.
   ADR-0141 wants the file clean, not suppressed, so the right-hand-side
   rounding is now `adm_angle_flag_round53_v()`. Pure extraction: no
   expression changes, and `core/test/test_adm_angle_flag.c` still passes
   7/7. `metal/integer_adm.metal` gets the same split so the manual MSL
   mirror stays line-for-line (the AGENTS.md lockstep invariant).

2. `docs/state.md` still carried T-UPSTREAM-930 under **Open bugs** with a
   comment claiming it had moved: an earlier duplicate-row cleanup dropped
   the Recently-closed copy instead of the stale open one. The open row is
   gone and the closed row is back at the head of Recently closed, citing
   PR #1342 and ADR-1194 (the previous text cited a commit sha that the
   rebase invalidated).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-930-adm-angle-flag-predicate- branch from 4543cab to 63440da Compare September 7, 2026 17:03
Lusoris and others added 4 commits September 7, 2026 20:32
`decouple()`'s 1-degree `angle_flag` test had four different spellings in
the tree. The CPU, AVX2 and AVX-512 paths narrow the exact int64 operands
to `float` and then compare in `double` — the form the Netflix golden gate
freezes. CUDA and HIP scale 0 compared the *exact* int64 products in
`double` (a strictly more accurate angle test, and therefore a different
one); SYCL did the whole comparison in `float`; Metal narrowed the exact
products to `float`. CUDA/HIP scales 1-3 already used the golden form, so
`s0` and `s123` disagreed inside a single backend.

`angle_flag` selects the enhancement-gain-limited branch of `decouple()`,
so a flipped flag moves the adm scores. Instrumenting the scalar CPU
decouple shows the divergence is reachable on the Netflix golden pair
itself: of 1 540 608 scale-0 pixels, the legacy SYCL form flips 2 and the
legacy Metal form flips 3; on a full-contrast 1080p noise clip the counts
are 11 / 11 / 18 (CUDA/HIP, SYCL, Metal).

The predicate now lives once, in `core/src/feature/adm_angle_flag.h`:

  * `adm_angle_flag_fp64()` — the golden expression, verbatim. The scalar
    CPU path, CUDA and HIP call it at both scales.
  * `adm_angle_flag_i64()` — a bit-identical evaluation in 64-bit integer
    arithmetic that uses no floating point of any width. SYCL calls it
    (Intel Arc A-series and most iGPUs expose no fp64, and one fp64
    instruction anywhere in that translation unit makes the runtime reject
    the whole SPIR-V module); `metal/integer_adm.metal` mirrors it in MSL
    (MSL has no `double` type).

Verified: `integer_adm.c` compiles to byte-identical machine code against
origin/master, so the CPU/golden lane cannot have moved. On the golden
576x324 pair with `adm_enhn_gain_limit=1.2`, the SYCL-vs-CPU adm gap drops
from 1.175e-05 to 8.220e-07 on the Arc A380. `adm_angle_flag_i64()` was
fuzzed against the golden expression over 200 M scale-0 quadruples and
104 M boundary-walk triples spanning all 62 magnitude classes, under gcc
and clang at -O0/-O2/-O3: zero mismatches.

Closes T-UPSTREAM-930-ADM-ANGLE-FLAG-PREDICATE-DIVERGENCE-2026-09-03.
Refs Netflix/vmaf#930.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…osed

Fixed by PR #1342 (a3155b2, ADR-1194). ADR-0165 requires the ledger
row to move in the same PR as the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each dropped row restates one origin/master already carries; master is the
authoritative record. Verified with scripts/ci/check-state-md-rows.sh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…md move

Two loose ends on top of the ADR-1194 change:

1. `adm_angle_flag_i64()` tripped `readability-function-size` (71 lines
   against the 60-line threshold) once clang-tidy was pointed at the new
   header with `-header-filter='adm_angle_flag\.h'` — the default filter is
   anchored at `^core/` and so misses it when the build dir is a sibling.
   ADR-0141 wants the file clean, not suppressed, so the right-hand-side
   rounding is now `adm_angle_flag_round53_v()`. Pure extraction: no
   expression changes, and `core/test/test_adm_angle_flag.c` still passes
   7/7. `metal/integer_adm.metal` gets the same split so the manual MSL
   mirror stays line-for-line (the AGENTS.md lockstep invariant).

2. `docs/state.md` still carried T-UPSTREAM-930 under **Open bugs** with a
   comment claiming it had moved: an earlier duplicate-row cleanup dropped
   the Recently-closed copy instead of the stale open one. The open row is
   gone and the closed row is back at the head of Recently closed, citing
   PR #1342 and ADR-1194 (the previous text cited a commit sha that the
   rebase invalidated).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-930-adm-angle-flag-predicate- branch from 63440da to a0b06a6 Compare September 7, 2026 18:35
…TS.md

A previous rebase committed unresolved markers: integer_adm_sycl.cpp kept both
sides of an #include hunk and core/src/feature/AGENTS.md kept both sides of an
appended section. Both are additive — the includes are distinct headers and the
sections document different invariants — so both sides are kept.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant