Skip to content

feat(cuda)!: require compute capability 8.0 (Ampere), and put every CI lane on CUDA 13.3.1 - #1384

Draft
lusoris wants to merge 1 commit into
masterfrom
feat/cuda-ampere-floor-and-133
Draft

feat(cuda)!: require compute capability 8.0 (Ampere), and put every CI lane on CUDA 13.3.1#1384
lusoris wants to merge 1 commit into
masterfrom
feat/cuda-ampere-floor-and-133

Conversation

@lusoris

@lusoris lusoris commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Breaking: the CUDA backend now requires compute capability 8.0 (Ampere).
Turing (sm_75 — RTX 20xx, GTX 16xx, Tesla T4) and older are dropped. Every CI
lane moves to CUDA 13.3.1.

CUDA 13.x had already dropped Maxwell, Pascal and Volta, so sm_75 was the last
pre-Ampere architecture still receiving a cubin.

Architecture floor

  • drop -gencode=arch=compute_75,code=sm_75
  • drop the CUDA-12-only -gencode=arch=compute_50,code=compute_50
  • move the clang-CUDA fallback from --cuda-gpu-arch=sm_75 to sm_80

Dropping the gencode entry alone would have been sloppy: the failure would
surface as CUDA_ERROR_NO_BINARY_FOR_GPU (222) out of cuModuleLoadData,
inside whichever feature extractor happened to load first, with nothing pointing
at the cause. vmaf_cuda_state_init() now checks the device capability and
returns -ENOTSUP:

CUDA: device "NVIDIA GeForce RTX 2080 Ti" has compute capability 7.5,
      below the minimum 8.0 (Ampere).
      libvmaf ships no cubin or PTX below sm_80, so no kernel can be loaded
      on this GPU.
      Turing (sm_75) and older were dropped in ADR-1223. Use an Ampere or
      newer GPU, or build with -Denable_cuda=false and run on the CPU backend.

The check runs on both init paths — before retaining a primary context (so
the unwind is free), and on the caller-supplied-context path, whose device still
has to clear the floor.

The predicate is split out as vmaf_cuda_arch_supported(major, minor) so it can
be pinned without a GPU — which matters because the architectures it most
needs to cover, Turing and older, are exactly the ones no runner in the fleet
has. It compares lexicographically, not major * 10 + minor, and
core/test/test_cuda_arch_floor.c pins both
edges (7.9 rejected, 8.0 accepted).

CI on 13.3.1

build.yml's Linux leg already installed 13.3.1. Its Windows leg and both
legs of libvmaf-build-matrix.yml were still on 13.2.0. The matrix pinned itself
there citing T-CI-JIMVER-CUDA-133-NOT-AVAILABLE — "Jimver/cuda-toolkit does not
publish a 13.3.0 build". That was true of action v0.2.35 and is stale at
v0.2.36, the version the Linux leg already uses to install 13.3.1. The
Windows legs never used Jimver at all; they fetch NVIDIA's network installer
directly and were never blocked by it.

Verified before changing: cuda_13.3.0_windows_network.exe and
cuda_13.3.1_windows_network.exe both return HTTP 200; 13.3.2 does not exist.
The docker images were already on 13.3.1, so the "Docker-13.3-vs-CI-13.2
split" that comment described is now genuinely closed rather than restated.

Reproducer / smoke test

meson setup build-cuda core -Denable_cuda=true -Denable_sycl=false -Db_lto=false
ninja -C build-cuda

# 1. the fatbin has no sub-Ampere code
cuobjdump --list-elf build-cuda/src/adm_csf_den.fatbin | grep -oE 'sm_[0-9]+' | sort -u
cuobjdump --list-ptx build-cuda/src/adm_csf_den.fatbin

# 2. the floor predicate
meson test -C build-cuda test_cuda_arch_floor --print-errorlogs -v

# 3. behaviour is unchanged on supported hardware
R=python/test/resource/yuv
./build-cuda/tools/vmaf -r $R/checkerboard_1920_1080_10_3_0_0.yuv \
  -d $R/checkerboard_1920_1080_10_3_1_0.yuv -w 1920 -h 1080 -p 420 -b 8 \
  --backend cuda --output /dev/stdout --json

Measured on this workstation (RTX 4090, sm_89, nvcc 13.3.73):

ELF:  sm_100 sm_120 sm_80 sm_86 sm_89 sm_90      (no sm_75)
PTX:  adm_csf_den.1.sm_80.ptx, adm_csf_den.2.sm_120.ptx   (no compute_50)

Netflix checkerboard golden pair, 1920x1080 8-bit:
  CUDA vmaf = 45.315104
  CPU  vmaf = 45.315104      identical

meson test --suite=fast: 165/165.

Note on CUDA Tile

Compute capability 8.0 is also what CUDA Tile C++ requires, so this removes one
precondition for a possible later adoption. That is a separate decision,
still under audit, and is not what motivated this change — the ADR says so
explicitly so nobody later reads this as tacit approval.

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: the decision is a maintainer scope call, and the two external facts it rests on (CUDA Tile's CC 8.0 requirement; GitHub/Jimver installer availability) are cited inline in the ADR with their sources.
  • Decision matrixdocs/adr/1223-cuda-ampere-architecture-floor.md ## Alternatives considered (five options; the runner-up "drop sm_75 without the runtime guard" turns a support decision into a debugging exercise for the user).
  • AGENTS.md invariant notecore/src/cuda/AGENTS.md gains a rebase-sensitive invariant: the floor lives in exactly two places that must move together, and upstream Netflix still ships sm_75, so a rebase that takes their gencode block wholesale reintroduces it.
  • Reproducer / smoke-test command — see above.
  • CHANGELOG fragmentchangelog.d/removed/1223-cuda-turing-support.md (the breaking hardware drop) and changelog.d/changed/1223-ci-cuda-1331.md (the toolkit bump).
  • Rebase note in docs/rebase-notes.md — entry ADR-1223 — CUDA floor at compute capability 8.0, CI on 13.3.1 (2026-09-07).

Docs (rule 10)

docs/backends/cuda/overview.md: the
architecture table drops the Turing row, states the 8.0 minimum up front, and
shows the exact diagnostic an unsupported GPU now gets.
docs/development/sycl-toolchains.md and
docs/development/build-flags.md: the
AdaptiveCpp omp;cuda:sm_75 example becomes sm_80.
scripts/ci/gpu_ulp_calibration.yaml: the Turing row is marked retired rather
than deleted, so the row shape and id stay a stable reference — verified the
loader still parses it and tolerance_for("cuda:7.5", ...) still resolves.

Bug status (rule 13 / ADR-0165)

  • docs/state.md — no state delta: this is a maintainer scope decision and a CI version bump, not a bug opened, closed, or ruled not-affecting.

🤖 Generated with Claude Code

@lusoris
lusoris force-pushed the feat/cuda-ampere-floor-and-133 branch from f0067f1 to 9d4c791 Compare September 7, 2026 10:15
…I lane on CUDA 13.3.1

BREAKING CHANGE: the CUDA backend no longer supports Turing (sm_75) or older.

The gencode list shipped cubins from Turing through Blackwell plus a compute_80
PTX and, on CUDA 12.x only, a compute_50 PTX. CUDA 13.x had already dropped
Maxwell, Pascal and Volta, so sm_75 was the last pre-Ampere architecture still
receiving a cubin. Per maintainer direction this fork does not carry old
hardware, so the floor moves to compute capability 8.0:

  - drop `-gencode=arch=compute_75,code=sm_75`
  - drop the CUDA-12-only `-gencode=arch=compute_50,code=compute_50`
  - move the clang-CUDA fallback from `--cuda-gpu-arch=sm_75` to `sm_80`

Dropping the gencode entry alone would be sloppy: the failure would surface as
CUDA_ERROR_NO_BINARY_FOR_GPU (222) out of cuModuleLoadData, inside whichever
feature extractor happened to load first, with nothing pointing at the cause.
`vmaf_cuda_state_init()` now queries the device compute capability and returns
-ENOTSUP with a message naming the device, its capability, the required floor
and the CPU-backend escape hatch. The check runs on both paths -- before
retaining a primary context (so the unwind is free), and on the
caller-supplied-context path, whose device still has to clear the floor.

The predicate is split out as `vmaf_cuda_arch_supported(major, minor)` so it can
be pinned without a GPU, which matters because the architectures it most needs
to cover -- Turing and older -- are exactly the ones no runner in the fleet has.
It compares lexicographically, not `major * 10 + minor`;
core/test/test_cuda_arch_floor.c pins both edges (7.9 rejected, 8.0 accepted).

CI: every lane now installs CUDA 13.3.1, matching the dev container and
build.yml's Linux leg. build.yml's Windows leg and both legs of
libvmaf-build-matrix.yml were still on 13.2.0. The matrix pinned itself there
citing T-CI-JIMVER-CUDA-133-NOT-AVAILABLE ("Jimver/cuda-toolkit does not publish
a 13.3.0 build") -- true of action v0.2.35, stale at v0.2.36, which the Linux leg
already uses to install 13.3.1. The Windows legs never used Jimver at all; they
fetch NVIDIA's network installer directly and were never blocked by it. Verified
that cuda_13.3.0_ and cuda_13.3.1_windows_network.exe both resolve (HTTP 200) and
13.3.2 does not exist. The docker images were already on 13.3.1, so the
"Docker-13.3-vs-CI-13.2 split" that comment described is now genuinely closed.

Verification:
  - cuobjdump --list-elf on a built fatbin reports exactly
    sm_80 sm_86 sm_89 sm_90 sm_100 sm_120; --list-ptx reports sm_80 and sm_120.
    No sm_75, no compute_50.
  - End-to-end on the RTX 4090 (sm_89), Netflix checkerboard golden pair at
    1920x1080 8-bit: CUDA 45.315104, CPU 45.315104 -- identical, so the change
    is behaviour-neutral on supported hardware.
  - meson test --suite=fast: 165/165.

Compute capability 8.0 also happens to be what CUDA Tile C++ requires, so this
removes one precondition for a possible later adoption -- but that is a separate
decision under audit and is not what motivated this change.

ADR-1223.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the feat/cuda-ampere-floor-and-133 branch from 9d4c791 to 5a28d02 Compare September 7, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant