ci(gpu): native self-hosted GPU build + test workflow#1044
Conversation
Problem: there is still no GPU CI (the origin issue #538). ci-cmake_tests.yml is CPU-only, and the existing gpu-smoke-test (main.yml) targeted a non-existent `v100` self-hosted runner and shelled out to Docker (not installed on the runner), so it never ran a single job. Separately, gpu_test_main could not even compile after #1004: its host-compiled .cpp TUs include Type.hpp, which pulls <cuda/std/complex>, but cytnx adds the CUDA toolkit + CCCL include dirs PRIVATE, so they never propagated to the test target. Fix: - tests/gpu/CMakeLists.txt: give gpu_test_main the CUDA toolkit + CCCL include dirs (fixes "fatal error: cuda/std/complex: No such file or directory"), tag its cases with the `gpu` ctest label, and enumerate them DISCOVERY_MODE PRE_TEST so a CUDA-enabled build still links on a driver-less machine. - CytnxBKNDCMakeLists.cmake: expose the detected CCCL dir as CYTNX_CCCL_INCLUDE_DIR so the test target reuses cytnx's single-source detection instead of duplicating it. - CMakePresets.json: add a `gpu-only` test preset (filters on the `gpu` label). - .github/workflows/ci-gpu_tests.yml: native build on the self-hosted GPU runner using the box's CUDA + cuTENSOR + cuQuantum (roots via repo vars), building gpu_test_main and running `ctest -L gpu`. Manual dispatch plus same-repo PRs only; fork PRs never execute on self-hosted hardware. - Remove the dead gpu-smoke-test workflow (main.yml). Testing: on the runner box (2x RTX 4070 Ti SUPER, sm_89, CUDA 13.0, cuTENSOR 2.0, cuQuantum) configure + `cmake --build --target gpu_test_main` succeed, `ctest -L gpu` discovers 788 tests, and AccessorTest passes 15/15 on-GPU. Toward #538. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request adds a "gpu-only" test preset and updates the CMake configuration to ensure host-compiled consumers of Cytnx public headers (like gpu_test_main) can correctly locate CUDA and CCCL headers. It also tags discovered GPU tests with the gpu label and sets the discovery mode to PRE_TEST. The review feedback recommends wrapping the CUDAToolkit_INCLUDE_DIRS include directory command in an if condition to prevent a CMake syntax error if the variable is empty.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| # and CCCL include dirs PRIVATEly, so they do not propagate here through | ||
| # linking; add them explicitly, otherwise the host compile fails with | ||
| # "fatal error: cuda/std/complex: No such file or directory". | ||
| target_include_directories(gpu_test_main PRIVATE ${CUDAToolkit_INCLUDE_DIRS}) |
There was a problem hiding this comment.
If CUDAToolkit_INCLUDE_DIRS is empty or undefined (which can happen in some environments where CUDA is installed in standard system paths), the command expands to target_include_directories(gpu_test_main PRIVATE), which is a CMake syntax error and will cause the configuration step to fail. Wrapping it in an if check ensures that the directory is only added if the variable is non-empty, preventing potential configuration failures.
if(CUDAToolkit_INCLUDE_DIRS)
target_include_directories(gpu_test_main PRIVATE ${CUDAToolkit_INCLUDE_DIRS})
endif()
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1044 +/- ##
=======================================
Coverage 57.71% 57.71%
=======================================
Files 229 229
Lines 33468 33468
Branches 71 71
=======================================
Hits 19315 19315
Misses 14132 14132
Partials 21 21
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
BlockUniTensorTest.gpu_Trace read the rank-0 scalar from UT_diag.Trace(0, 1)
via at({0}), which #1026 (rank-0 UniTensor semantics) now rejects with
"rank-0 scalar expects no locator indices". The CPU Trace test was updated to
at({}) in #1026; the GPU test was missed because GPU tests don't run in CI.
Mirror the CPU fix. Surfaced by this PR's new GPU CI on its first run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| run: >- | ||
| cmake -S . -B "$BUILD_DIR" | ||
| -DCMAKE_BUILD_TYPE=Release | ||
| -DUSE_CUDA=ON -DUSE_CUTENSOR=ON -DUSE_CUQUANTUM=ON | ||
| -DRUN_TESTS=ON -DBUILD_PYTHON=OFF | ||
| -DCMAKE_CUDA_ARCHITECTURES="$GPU_CUDA_ARCH" | ||
|
|
There was a problem hiding this comment.
Consider to use preset debug-openblas-cuda or add a new test preset like #1023
| CUTENSOR_ROOT: ${{ vars.CUTENSOR_ROOT }} | ||
| CUQUANTUM_ROOT: ${{ vars.CUQUANTUM_ROOT }} |
There was a problem hiding this comment.
If we can control the dependency versions like #1023 will be better. Because that will be the versions we will release.
Problem
No GPU CI exists (origin issue #538).
ci-cmake_tests.ymlis CPU-only; the oldgpu-smoke-test(main.yml) targeted a non-existentv100runner and shelled out to Docker (not installed), so it never ran a single job. And after #1004gpu_test_maindidn't even compile: its host-compiled.cppTUs includeType.hpp→<cuda/std/complex>, butcytnxadds the CUDA toolkit + CCCL includes PRIVATE, so they never reached the test target (fatal error: cuda/std/complex: No such file or directory).Fix
tests/gpu/CMakeLists.txt— givegpu_test_mainthe CUDA toolkit + CCCL include dirs (the compile fix); tag its cases with thegpuctest label; enumerateDISCOVERY_MODE PRE_TESTso a CUDA build links even on a driver-less machine.CytnxBKNDCMakeLists.cmake— expose the detected CCCL dir asCYTNX_CCCL_INCLUDE_DIR(single source of truth, reused by the test target rather than re-detected).CMakePresets.json— add agpu-onlytest preset (filters on thegpulabel)..github/workflows/ci-gpu_tests.yml— native build on the self-hosted GPU runner using the box's CUDA / cuTENSOR / cuQuantum (roots come from repovars.CUTENSOR_ROOT/vars.CUQUANTUM_ROOT, already configured); buildsgpu_test_mainand runsctest -L gpu. Manual dispatch + same-repo PRs only — fork PRs never execute on self-hosted hardware.gpu-smoke-testworkflow (main.yml).The
gpulabel +PRE_TESTgroundwork is adapted from the draft #1023 (which targeted a GitHub-hosted runner and, being unrun, still carried the CCCL compile blocker). This supersedes #1023's direction — suggest closing it once this lands.Testing
On the runner box (2× RTX 4070 Ti SUPER, sm_89, CUDA 13.0, cuTENSOR 2.0, cuQuantum):
cmakeconfigure +cmake --build --target gpu_test_mainsucceed — CUDA lib + test binary, nocuda/std/complexerror.ctest -L gpudiscovers 788 GPU-labeled tests;AccessorTestpasses 15/15 on-GPU.pull_requestrun exercises the workflow on the self-hosted runner end-to-end.Known runner-env caveat (not a blocker)
The runner's
cuquantumconda env is CUDA-11-built, so linking warnslibcublas.so.11 … may conflict with libcublas.so.13. Core tensor tests are unaffected; updating that env to a CUDA-13 cuQuantum build clears the warning.Toward #538.
🤖 Generated with Claude Code