feat(hip): migrate every ROCm consumer to 10.0.0 from digest-pinned images (ADR-1225) - #1386
Draft
lusoris wants to merge 2 commits into
Draft
feat(hip): migrate every ROCm consumer to 10.0.0 from digest-pinned images (ADR-1225)#1386lusoris wants to merge 2 commits into
lusoris wants to merge 2 commits into
Conversation
lusoris
force-pushed
the
feat/rocm-10-migration
branch
from
September 7, 2026 10:14
3f11190 to
20bdf88
Compare
…mages
AMD froze the repo.radeon.com apt channel at 7.2.4 when ROCm moved to the
"TheRock" build/release system at 7.14: apt/7.14 and apt/10.0.0 both 404,
the manylinux channel stops at rocm-rel-7.2.4, and the TheRock wheel index
carries only 7.14.0 alphas. The fork was therefore stuck on a ROCm-legacy
toolchain heading into 1.0.0, with no version-string edit available to get
off it.
Take ROCm from the digest-pinned rocm/dev-ubuntu-24.04:10.0.0-full image
instead. dev/Containerfile, docker/Dockerfile.production-gpu and
docker/Dockerfile.node copy a pruned /opt/rocm out of it; the two CI HIP
lanes use the new scripts/ci/install-rocm-from-image.sh, which streams the
image's /opt/rocm out of the registry layer by layer. A docker pull is not
viable on a runner that already carries CUDA and oneAPI -- 8.2 GB
compressed, 29 GB extracted -- so nothing is ever stored whole: peak disk
is the 5.5 GB result.
Three things ROCm 10 changed that this had to handle:
- /opt/rocm/{bin,lib,include,llvm,...} are now /etc/alternatives symlinks
into /opt/rocm/core-10.0/. Copying or extracting /opt/rocm alone lands
dangling links, so both the rocm-src stage and the extractor repoint
them at core-10.0/<name> relative to /opt/rocm.
- libamdhip64.so's link closure widened to 19 libraries (234 MB) wired by
$ORIGIN-relative RPATHs. The node-rocm runtime stage's flat copy of
libamdhip64.so* + libhsa-runtime64.so* no longer loads; it now ships
the whole closure with its layout intact, verified at 0 unresolved
libraries in a scratch stage carrying only the copied files.
- gfx1036 is natively supported, so HSA_OVERRIDE_GFX_VERSION=10.3.0 is
dropped from dev/docker-compose.yml. Keeping it would alias the agent
to gfx1030 while meson compiles gfx1036 code objects.
The prune list (19 GB -> 5.5 GB) is gated by a hipcc smoke compile in the
same stage: an earlier pass removed librocprofiler-register with a tidy
librocprof* glob and every HIP binary then failed at load.
Renovate follows the image digest now rather than scraping the frozen apt
directory listing.
Verified on AMD gfx1036 under Linux 7.2.3: HIP suite 19 Ok / 0 Fail, and an
end-to-end HIP score of 45.315104, bit-identical to the CPU reference.
Refs: ADR-1225
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The badge scraped `ARG ROCM_VER=` out of dev/Containerfile, which this branch removes along with the apt install. Read the version out of the digest-pinned rocm/dev-ubuntu-24.04:<version>-full reference instead, so it keeps reporting a live value rather than going blank. Verified by running the badge's own regex against the new Containerfile: matches `rocm/dev-ubuntu-24.04:10.0.0-full`, renders `10.0.0`. Refs: ADR-1225 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
force-pushed
the
feat/rocm-10-migration
branch
from
September 7, 2026 12:16
20bdf88 to
a334256
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AMD froze the
repo.radeon.com/rocm/apt/channel at 7.2.4 when ROCm moved tothe "TheRock" build/release system at 7.14 —
apt/7.14andapt/10.0.0both 404,the manylinux channel stops at
rocm-rel-7.2.4, and the TheRock wheel index carriesonly 7.14.0 alphas. The fork was heading into 1.0.0 on a ROCm-legacy toolchain with
no version-string edit available to get off it.
This takes ROCm from the digest-pinned
rocm/dev-ubuntu-24.04:10.0.0-fullimageinstead, across every consumer at once: the dev container, both published GPU
images, and both CI HIP lanes.
Three things ROCm 10 changed that this had to handle — each found by a check
that failed, not by reading release notes:
/opt/rocm/{bin,lib,include,llvm,…}are/etc/alternativessymlinks into/opt/rocm/core-10.0/. Copying or extracting/opt/rocmalone lands a directoryof dangling links; both the
rocm-srcstage and the extractor repoint them atcore-10.0/<name>relative to/opt/rocm.libamdhip64.so's link closure widened to 19 libraries / 234 MB(
libLLVM,libclang-cpp,libamd_comgr,librocm_kpack,librocprofiler-register, therocm_sysdepsbundle), wired by$ORIGIN-relative RPATHs. Thenode-rocmstage's flat two-library copy nolonger loads; it now ships the whole closure with its layout intact.
gfx1036is natively supported, soHSA_OVERRIDE_GFX_VERSION=10.3.0is droppedfrom
dev/docker-compose.yml— keeping it would alias the agent togfx1030while meson compiles
gfx1036code objects.docker pullis not viable in CI (8.2 GB compressed / 29 GB extracted, on a runnerthat already carries CUDA and oneAPI), so
scripts/ci/install-rocm-from-image.shstreams each layer blob from the registry straight into
tar, extracting only/opt/rocmminus the math libraries libvmaf never links. Peak disk is the 5.5 GBresult, not the 29 GB image.
The prune list (19 GB → 5.5 GB) is gated by a hipcc smoke compile in the same
stage. That gate earned its keep: an earlier pass removed
librocprofiler-registerwith a tidy-looking
librocprof*glob, and every HIP binary then failed at loadwith
cannot open shared object file.Verification
Measured on the dev host (Linux 7.2.3-1-cachyos, AMD
gfx1036), not inferred:Container stage, built and run:
Reproducer / smoke-test command
Deep-dive deliverables (ADR-0108)
docs/research/1225-rocm-10-distribution-and-layout.md: the channel survey, the alternatives layout, the link closure, the prune measurements, and why CI streams instead of pulling.## Alternatives considered(six options, including the deferred GHCR mirror).AGENTS.mdinvariant note —dev/AGENTS.md(never restore the apt path; never prunelibrocprofiler-register;HSA_OVERRIDE_GFX_VERSIONstays out) and the rootAGENTS.mdcontainer-pin section.changelog.d/changed/1225-rocm-10-therock-migration.md,CHANGELOG.mdregenerated.docs/rebase-notes.md§feat/rocm-10-migration (ADR-1225), five invariants.Docs (rule 10)
docs/backends/hip/overview.md— required ROCm version, how to get it, and thegfx1036story.docs/development/dev-mcp.md— the container pin table and the GPU env-var table.docs/research/1225-rocm-10-distribution-and-layout.md— new digest.State (rule 13)
no state delta: dependency migration — no bug opened, closed, or ruled not-affecting.
ffmpeg-patches (rule 14)
no patch impact: no C-API surface, CLI flag,
meson_options.txtentry or publicheader changes; the meson edit is comment-only.
🤖 Generated with Claude Code