-
Notifications
You must be signed in to change notification settings - Fork 22
build: require GCC>=13, CUDA>=12 (12 & 13), cuTENSOR>=2.0 with early actionable errors; harden CUDA finders #950
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
12a01a9
5436e8b
de94f3f
fe7b181
10b1536
6e2a7c3
6bfc3d1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -247,17 +247,70 @@ endif() | |
|
|
||
| set(CMAKE_EXPORT_COMPILE_COMMANDS ON) | ||
| if(USE_CUDA) | ||
| # Resolve the CUDA toolkit up front (before enable_language) so we can both | ||
| # enforce the version requirement early and choose an architecture list the | ||
| # toolkit can actually compile. find_package(CUDAToolkit) does not require the | ||
| # CUDA language to be enabled; the later find_package in CytnxBKNDCMakeLists | ||
| # re-uses this cached result. | ||
| find_package(CUDAToolkit QUIET) | ||
|
|
||
| # enable_language(CUDA) finds the compiler ONLY via CMAKE_CUDA_COMPILER, the | ||
| # CUDACXX environment variable, or nvcc on PATH -- it does not reuse the | ||
| # toolkit located above. Catch the "no usable nvcc" case here with an | ||
| # actionable message instead of CMake's terse "No CMAKE_CUDA_COMPILER could be | ||
| # found". Skip the check when the user pinned a compiler explicitly. | ||
| if(NOT CUDAToolkit_FOUND OR (NOT CUDAToolkit_NVCC_EXECUTABLE | ||
| AND NOT CMAKE_CUDA_COMPILER AND NOT DEFINED ENV{CUDACXX})) | ||
| message(FATAL_ERROR | ||
| "USE_CUDA is ON but no usable CUDA compiler (nvcc) could be found.\n" | ||
| "enable_language(CUDA) searches only CMAKE_CUDA_COMPILER, the CUDACXX " | ||
| "environment variable, and nvcc on PATH; it does not reuse the libraries " | ||
| "found by find_package(CUDAToolkit), so installed CUDA runtime/dev " | ||
| "packages alone are not enough.\n" | ||
| "Do one of the following, then delete the build directory (a stale " | ||
| "CMakeCache.txt caches the failed detection) and re-configure:\n" | ||
| " * pass -DCMAKE_CUDA_COMPILER=/path/to/nvcc " | ||
| "(e.g. /usr/local/cuda-12/bin/nvcc), or\n" | ||
| " * export CUDACXX=/path/to/nvcc, or\n" | ||
| " * put the CUDA bin directory first on PATH " | ||
| "(export PATH=/usr/local/cuda-12/bin:$PATH).") | ||
| endif() | ||
|
|
||
| # CUDA requirement: >= 12.0, with 12.x and 13.x explicitly supported. The | ||
| # 12.0 floor comes from device-side C++20 (nvcc gained -std=c++20 in CUDA | ||
| # 12.0; 11.x tops out at C++17), matching CMAKE_CUDA_STANDARD 20. Fail early | ||
| # with an actionable message instead of a cryptic later C++20/arch error. | ||
| if(CUDAToolkit_VERSION VERSION_LESS 12.0) | ||
| message(FATAL_ERROR | ||
| "CUDA Toolkit >= 12.0 is required (device-side C++20 support), but found " | ||
| "${CUDAToolkit_VERSION} at ${CUDAToolkit_BIN_DIR}.\n" | ||
| "Install CUDA 12 or 13 and select it with " | ||
| "-DCMAKE_CUDA_COMPILER=/path/to/cuda-12+/bin/nvcc (or set CUDACXX / put " | ||
| "the desired nvcc first on PATH).") | ||
| elseif(CUDAToolkit_VERSION VERSION_GREATER_EQUAL 14.0) | ||
| message(WARNING | ||
| "CUDA Toolkit ${CUDAToolkit_VERSION} is newer than the versions Cytnx is " | ||
| "tested against (12.x and 13.x). The build will proceed; please report " | ||
| "any issues.") | ||
| endif() | ||
|
Comment on lines
+290
to
+295
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For packages supporting cuda, there are many configurations. It's hard to test on all configurations as the discussion in #583 and #584. Some packages don't explicitly set the range of supported system dependencies. They just list the tested configuration in the document. This reduces the responsibility of package, makes maintenance easier and let the users who want to build from source take the responsibility to solve the configuration problem. We may support and build with CUDA 14 in the future, so guarding with the maximum version is not needed. |
||
| message(STATUS " CUDA Toolkit: ${CUDAToolkit_VERSION} (supported: 12.x, 13.x)") | ||
|
|
||
| # Default to a portable fat binary unless the caller picked architectures via | ||
| # -D, the cache, a preset's cacheVariables, or the CUDAARCHS environment | ||
| # variable. This must run before enable_language(CUDA): afterwards | ||
| # CMAKE_CUDA_ARCHITECTURES is never empty (CMake fills in its own default), so | ||
| # the "not specified" case can no longer be detected. enable_language(CUDA) | ||
| # reads CUDAARCHS on its own, so we only avoid shadowing it here, not copy it. | ||
| # The default embeds SASS for each supported real architecture (Volta sm_70 is | ||
| # the floor required by cuTENSOR/cuQuantum, up through Hopper sm_90) plus PTX | ||
| # of the newest (90-virtual) so the driver can JIT for newer/unknown GPUs. | ||
| # The default embeds SASS for each supported real architecture plus PTX of the | ||
| # newest (90-virtual) so the driver can JIT for newer/unknown GPUs. CUDA 13.0 | ||
| # removed offline support for compute capability < 7.5, so Volta sm_70 is only | ||
| # included for CUDA 12.x. | ||
| if(NOT CMAKE_CUDA_ARCHITECTURES AND NOT DEFINED ENV{CUDAARCHS}) | ||
| set(CMAKE_CUDA_ARCHITECTURES 70-real 75-real 80-real 86-real 89-real 90-real 90-virtual) | ||
| if(CUDAToolkit_VERSION VERSION_GREATER_EQUAL 13.0) | ||
| set(CMAKE_CUDA_ARCHITECTURES 75-real 80-real 86-real 89-real 90-real 90-virtual) | ||
| else() | ||
| set(CMAKE_CUDA_ARCHITECTURES 70-real 75-real 80-real 86-real 89-real 90-real 90-virtual) | ||
| endif() | ||
| endif() | ||
| enable_language(CUDA) | ||
| # Disable generation of "--option-file" flag in compile_commands.json. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -49,6 +49,18 @@ find_library( | |
| ) | ||
| message(STATUS "CUQUANTUM_TENSORNET_LIB: ${CUQUANTUM_TENSORNET_LIB}") | ||
| message(STATUS "CUQUANTUM_CUSTATEVEC_LIB: ${CUQUANTUM_CUSTATEVEC_LIB}") | ||
| set(CUQUANTUM_LIBRARIES "${CUQUANTUM_TENSORNET_LIB};${CUQUANTUM_CUSTATEVEC_LIB}") | ||
| message(STATUS "ok") | ||
| set(CUQUANTUM_FOUND TRUE) | ||
| set(CUQUANTUM_LIBRARIES "") | ||
| if(CUQUANTUM_TENSORNET_LIB) | ||
| list(APPEND CUQUANTUM_LIBRARIES "${CUQUANTUM_TENSORNET_LIB}") | ||
| endif() | ||
| if(CUQUANTUM_CUSTATEVEC_LIB) | ||
| list(APPEND CUQUANTUM_LIBRARIES "${CUQUANTUM_CUSTATEVEC_LIB}") | ||
| endif() | ||
|
|
||
| # CUQUANTUM_FOUND must reflect whether the libraries were actually located; | ||
| # both cuTensorNet and cuStateVec are linked, so both are required. Setting it | ||
| # unconditionally would let a NOTFOUND (e.g. an "...-NOTFOUND" entry on the link | ||
| # line) silently pass the caller's REQUIRED check. | ||
| include(FindPackageHandleStandardArgs) | ||
| find_package_handle_standard_args(CUQUANTUM | ||
| REQUIRED_VARS CUQUANTUM_TENSORNET_LIB CUQUANTUM_CUSTATEVEC_LIB CUQUANTUM_INCLUDE_DIRS) | ||
|
Comment on lines
+65
to
+66
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When Useful? React with 👍 / 👎. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -27,22 +27,64 @@ else() | |
| endif() | ||
|
|
||
| message(STATUS " cudaver: ${CUDAToolkit_VERSION_MAJOR}" ) | ||
| if(EXISTS "${CUTENSOR_ROOT}/lib") | ||
| set(CUTNLIB_DIR "lib/") | ||
| endif() | ||
| if((${CUDAToolkit_VERSION_MAJOR} LESS_EQUAL 10)) | ||
| set(CUTNLIB_DIR "${CUTNLIB_DIR}10.2") | ||
| elseif((${CUDAToolkit_VERSION_MAJOR} GREATER_EQUAL 11) AND (${CUDAToolkit_VERSION_MAJOR} LESS 12) AND (${CUDAToolkit_VERSION_MINOR} LESS_EQUAL 0)) | ||
| set(CUTNLIB_DIR "${CUTNLIB_DIR}11.0") | ||
| elseif((${CUDAToolkit_VERSION_MAJOR} GREATER_EQUAL 11) AND (${CUDAToolkit_VERSION_MAJOR} LESS 12) AND (${CUDAToolkit_VERSION_MINOR} GREATER_EQUAL 1)) | ||
| set(CUTNLIB_DIR "${CUTNLIB_DIR}11") | ||
| elseif((${CUDAToolkit_VERSION_MAJOR} GREATER_EQUAL 12)) | ||
| set(CUTNLIB_DIR "${CUTNLIB_DIR}12") | ||
| # Cytnx requires CUDA >= 12 (enforced in CMakeLists.txt) and cuTENSOR >= 2.0. | ||
| # Search both library layouts: cuTENSOR 2.x tarballs place the libraries | ||
| # directly under lib/, while 1.x tarballs and apt use a per-CUDA subdir | ||
| # (lib/<cuda-major>, e.g. lib/12, lib/13). Listing both as find_library | ||
| # PATH_SUFFIXES lets the supported 2.x flat layout and the legacy versioned | ||
| # layout resolve for both CUDA 12 and 13. The older lib/10.2 and lib/11 | ||
| # branches were removed as dead code; apt multiarch paths remain (issue #946). | ||
| if(CUDAToolkit_VERSION_MAJOR GREATER_EQUAL 12) | ||
| set(CUTNLIB_DIR lib lib/${CUDAToolkit_VERSION_MAJOR}) | ||
| else() | ||
| message(FATAL_ERROR | ||
| "cuTENSOR support requires CUDA >= 12, but CUDAToolkit_VERSION_MAJOR is " | ||
| "'${CUDAToolkit_VERSION_MAJOR}'.") | ||
| endif() | ||
|
|
||
| set(CUTENSOR_LIBRARY_DIRS ${CUTENSOR_ROOT}/${CUTNLIB_DIR}) | ||
| set(CUTENSOR_INCLUDE_DIRS ${CUTENSOR_ROOT}/include) | ||
|
|
||
| # Require cuTENSOR >= 2.0. The version macros (CUTENSOR_MAJOR/MINOR/PATCH) live | ||
| # in cutensor.h (older releases) or cutensor/types.h (newer ones); read whichever | ||
| # defines them and fail early on the 1.x API, which Cytnx no longer supports. | ||
| set(_cutensor_version_header "") | ||
| foreach(_hdr "${CUTENSOR_INCLUDE_DIRS}/cutensor.h" "${CUTENSOR_INCLUDE_DIRS}/cutensor/types.h") | ||
| if(EXISTS "${_hdr}") | ||
| file(STRINGS "${_hdr}" _cutensor_major_line REGEX "^#define[ \t]+CUTENSOR_MAJOR[ \t]+[0-9]+") | ||
| if(_cutensor_major_line) | ||
| set(_cutensor_version_header "${_hdr}") | ||
| break() | ||
| endif() | ||
| endif() | ||
| endforeach() | ||
|
|
||
| if(_cutensor_version_header) | ||
| file(STRINGS "${_cutensor_version_header}" _cutensor_minor_line REGEX "^#define[ \t]+CUTENSOR_MINOR[ \t]+[0-9]+") | ||
| string(REGEX REPLACE ".*CUTENSOR_MAJOR[ \t]+([0-9]+).*" "\\1" CUTENSOR_VERSION_MAJOR "${_cutensor_major_line}") | ||
| string(REGEX REPLACE ".*CUTENSOR_MINOR[ \t]+([0-9]+).*" "\\1" CUTENSOR_VERSION_MINOR "${_cutensor_minor_line}") | ||
| set(CUTENSOR_VERSION "${CUTENSOR_VERSION_MAJOR}.${CUTENSOR_VERSION_MINOR}") | ||
| message(STATUS "cuTENSOR version: ${CUTENSOR_VERSION} (from ${_cutensor_version_header})") | ||
| if(CUTENSOR_VERSION_MAJOR LESS 2) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
With CUDA 13 selected, Useful? React with 👍 / 👎.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in 6bfc3d1. Added a CUDA-13 gate right after the if(CUDAToolkit_VERSION_MAJOR GREATER_EQUAL 13 AND CUTENSOR_VERSION VERSION_LESS 2.3)
message(FATAL_ERROR "CUDA ${CUDAToolkit_VERSION_MAJOR} requires cuTENSOR >= 2.3 ...")
endif()
|
||
| message(FATAL_ERROR | ||
| "cuTENSOR >= 2.0 is required, but found ${CUTENSOR_VERSION} in " | ||
| "${CUTENSOR_ROOT}. Install cuTENSOR 2.x and point CUTENSOR_ROOT at it.") | ||
| endif() | ||
| # CUDA 13 support was added in cuTENSOR 2.3.0 (NVIDIA cuTENSOR release notes). | ||
| # cuTENSOR 2.0-2.2 are CUDA-12 builds: their major version passes the >= 2.0 | ||
| # check above, but they can fail at link/load time against a CUDA 13 toolkit, | ||
| # so require >= 2.3 in that case. | ||
| if(CUDAToolkit_VERSION_MAJOR GREATER_EQUAL 13 AND CUTENSOR_VERSION VERSION_LESS 2.3) | ||
| message(FATAL_ERROR | ||
| "CUDA ${CUDAToolkit_VERSION_MAJOR} requires cuTENSOR >= 2.3 (CUDA 13 support " | ||
| "was added in cuTENSOR 2.3.0), but found ${CUTENSOR_VERSION} in " | ||
| "${CUTENSOR_ROOT}. Install cuTENSOR >= 2.3 and point CUTENSOR_ROOT at it.") | ||
| endif() | ||
| else() | ||
| message(WARNING | ||
| "Could not determine the cuTENSOR version from headers under " | ||
| "${CUTENSOR_INCLUDE_DIRS}; Cytnx requires cuTENSOR >= 2.0.") | ||
| endif() | ||
|
|
||
| # set libs: | ||
| find_library( | ||
| CUTENSOR_LIB | ||
|
|
@@ -60,6 +102,24 @@ find_library( | |
| ) | ||
| message(STATUS "CUTENSOR_LIB: ${CUTENSOR_LIB}") | ||
| message(STATUS "CUTENSORMg_LIB: ${CUTENSORMg_LIB}") | ||
| set(CUTENSOR_LIBRARIES "${CUTENSOR_LIB};${CUTENSORMg_LIB}") | ||
| message(STATUS "ok") | ||
| set(CUTENSOR_FOUND TRUE) | ||
| # Report the directory the library was actually found in (flat lib/ or the | ||
| # versioned lib/<major>) rather than guessing a subdir, so callers and runtime | ||
| # guidance reference the real location. | ||
| if(CUTENSOR_LIB) | ||
| get_filename_component(CUTENSOR_LIBRARY_DIRS "${CUTENSOR_LIB}" DIRECTORY) | ||
| endif() | ||
| set(CUTENSOR_LIBRARIES "") | ||
| if(CUTENSOR_LIB) | ||
| list(APPEND CUTENSOR_LIBRARIES "${CUTENSOR_LIB}") | ||
| endif() | ||
| if(CUTENSORMg_LIB) | ||
| list(APPEND CUTENSOR_LIBRARIES "${CUTENSORMg_LIB}") | ||
| endif() | ||
|
|
||
| # CUTENSOR_FOUND must reflect whether the core library was actually located: | ||
| # the main cutensor lib is mandatory, cutensorMg is optional. Setting it | ||
| # unconditionally would let a NOTFOUND silently pass the caller's REQUIRED check. | ||
| include(FindPackageHandleStandardArgs) | ||
| find_package_handle_standard_args(CUTENSOR | ||
| REQUIRED_VARS CUTENSOR_LIB CUTENSOR_INCLUDE_DIRS | ||
| VERSION_VAR CUTENSOR_VERSION) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -29,9 +29,9 @@ In addition, you might want to install the following optional dependencies if yo | |
|
|
||
| [CUDA] | ||
|
|
||
| * Nvidia cuda library v10+ | ||
| * Nvidia CUDA toolkit >= 12.0 (12.x and 13.x supported) | ||
| * Nvidia cuDNN library | ||
| * Nvidia cuTensor library | ||
| * Nvidia cuTENSOR library >= 2.0 (>= 2.3 with CUDA 13, which cuTENSOR 2.3.0 added support for) | ||
| * Nvidia cuQuantum library | ||
|
|
||
|
|
||
|
|
@@ -154,6 +154,43 @@ Similarly, cuqauantum (compile option -DUSE_CUTENSOR=ON), requires: | |
|
|
||
| $conda install -c nvidia cuquantum | ||
|
|
||
| .. important:: | ||
|
|
||
| **If you install cuTENSOR and/or cuQuantum from NVIDIA tarballs** | ||
| (instead of conda), you must configure two separate paths -- one for the | ||
| build and one for runtime: | ||
|
|
||
| 1. **Build time** -- point CMake at the extracted directories so the | ||
| libraries are found during configuration: | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| $export CUTENSOR_ROOT=/path/to/libcutensor-<version> | ||
| $export CUQUANTUM_ROOT=/path/to/cuquantum-<version> | ||
|
|
||
| 2. **Runtime** -- the dynamic loader must also find the shared libraries | ||
| when you ``import cytnx`` (or run a C++ binary). Tarball libraries live | ||
| outside the system loader's default search path, and unlike conda they | ||
| are **not** registered with ``ldconfig``, so add their library | ||
| directories to ``LD_LIBRARY_PATH``: | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| $export LD_LIBRARY_PATH=$CUTENSOR_ROOT/lib:$CUQUANTUM_ROOT/lib:$LD_LIBRARY_PATH | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The documented runtime workaround does not match the library directory that this commit's finder accepts: Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Addressed in Posted by Claude Code on behalf of @pcchen |
||
|
|
||
| For cuTENSOR 2.x tarballs the libraries sit directly under ``lib/``, so | ||
| the path above is sufficient at both build and runtime. Older cuTENSOR | ||
| 1.x tarballs instead use a per-CUDA subdirectory (``lib/<cuda-major>``, | ||
| e.g. ``lib/12``); on that legacy layout point ``LD_LIBRARY_PATH`` at the | ||
| subdirectory. Cytnx's ``FindCUTENSOR`` searches both layouts at build | ||
| time, so the runtime path matches whichever one was found. If | ||
| ``LD_LIBRARY_PATH`` is not set | ||
| up, importing/running Cytnx fails with ``error while loading shared | ||
| libraries: libcutensor.so... cannot open shared object file``, or | ||
| silently binds to a mismatched system copy of the library if one is | ||
| present. Add these ``export`` lines to your shell profile (e.g. | ||
| ``~/.bashrc``) to make them persistent. | ||
|
Comment on lines
+181
to
+192
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not for all 2.x. cuTENSOR start to separate the package for different CUDA version since 2.3. And we don't support cuTENSOR < 2.0 and cuQuantum < 24.0. See #447. I will not recommend users export LD_LIBRARY_PATH to ~/.bashrc. This will pollute the environment system-wide just for running cytnx built for local. Actually, having multiple version of the same libraries without isolated environment is always easy to get trouble. We can tell users how |
||
|
|
||
| **Option B. Install dependencies via system package manager** | ||
|
|
||
| You can also choose to install dependencies directly from the system package manager, but one needs to carefully resolve the dependency path for cmake to capture them successfully. | ||
|
|
@@ -344,6 +381,61 @@ where ``CYTNX_ROOT`` is the ``-DCMAKE_INSTALL_PREFIX`` used for the :ref:`Option | |
| Build troubleshooting | ||
| ************************************* | ||
|
|
||
| Choosing between CUDA 12 and CUDA 13 when both are installed | ||
| ------------------------------------------------------------------------------------- | ||
|
|
||
| Cytnx supports both CUDA 12.x and 13.x. If more than one CUDA toolkit is present | ||
| (for example CUDA 12 installed via ``apt`` and CUDA 13 unpacked under | ||
| ``/usr/local/cuda-13``), you must select the one to build against; otherwise | ||
| CMake picks up whichever ``nvcc`` it finds first on ``PATH``, which may not be | ||
| the one you intended. There are two independent things to pin -- the **build** | ||
| toolkit and the **runtime** libraries. | ||
|
|
||
| **1. Build time -- select the compiler.** ``enable_language(CUDA)`` finds the | ||
| compiler only from ``CMAKE_CUDA_COMPILER``, the ``CUDACXX`` environment | ||
| variable, or ``nvcc`` on ``PATH`` (it does *not* reuse the libraries found by | ||
| ``find_package(CUDAToolkit)``). Point it at the toolkit you want -- the rest of | ||
| the toolkit (libraries, headers, version) is then derived from that ``nvcc``: | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| # Use CUDA 13: | ||
| $cmake -S . -B build -DUSE_CUDA=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13/bin/nvcc | ||
|
|
||
| # ...or use CUDA 12: | ||
| $cmake -S . -B build -DUSE_CUDA=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12/bin/nvcc | ||
|
|
||
| Equivalent alternatives are ``export CUDACXX=/usr/local/cuda-13/bin/nvcc`` or | ||
| putting the desired ``bin`` directory first on ``PATH`` | ||
| (``export PATH=/usr/local/cuda-13/bin:$PATH``). For the ``pip`` build, pass it | ||
| through scikit-build-core: | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| $pip install . --config-settings=cmake.args="-DUSE_CUDA=ON;-DCMAKE_CUDA_COMPILER=/usr/local/cuda-13/bin/nvcc" | ||
|
Comment on lines
+400
to
+415
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
|
|
||
| The configure output reports the resolved toolkit (``CUDA Version``, | ||
| ``CUDA Toolkit bin dir``, ``CUDA compiler (nvcc)``) -- check these match the | ||
| version you intended. CMake caches the detected compiler, so if you previously | ||
| configured with the wrong one, delete the build directory before reconfiguring; | ||
| a stale ``CMakeCache.txt`` keeps the old choice. | ||
|
|
||
| **2. Runtime -- match the shared libraries.** Selecting the compiler does not | ||
| control which CUDA runtime libraries are loaded at run time. The dynamic loader | ||
| resolves ``libcudart.so.<major>`` (and cuTENSOR/cuQuantum, etc.) via | ||
| ``LD_LIBRARY_PATH``, then the ``ldconfig`` cache. If a different major version is | ||
| registered with ``ldconfig`` (commonly the ``apt`` copy under | ||
| ``/usr/lib/x86_64-linux-gnu``), it can be loaded instead of the toolkit you | ||
| built against, causing version-skew crashes. To force the matching runtime, put | ||
| its library directory first: | ||
|
Comment on lines
+423
to
+430
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This content seems duplicated with the section of cuTENSOR and cuQuantum. |
||
|
|
||
| .. code-block:: shell | ||
|
|
||
| $export LD_LIBRARY_PATH=/usr/local/cuda-13/lib64:$LD_LIBRARY_PATH | ||
|
|
||
| (Adjust the path to the toolkit you built against; append the cuTENSOR/cuQuantum | ||
| ``lib`` directories too if you enabled them -- see the tarball note above.) | ||
|
|
||
| CUDA device link fails with ``elfLink linker library load error`` | ||
| ------------------------------------------------------------------------------------- | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This probes
FindCUDAToolkitbefore the CUDA language is enabled, butcmake --help-module FindCUDAToolkitsays the compiler directory is searched only after CUDA has been enabled; at this point it searchesCUDAToolkit_ROOT/CUDA_PATH/PATH/defaults instead ofCMAKE_CUDA_COMPILERorCUDACXX. A user with nvcc only under a non-default path who follows the new error text (-DCMAKE_CUDA_COMPILER=/opt/cuda-13/bin/nvccorCUDACXX=...) will still hit this fatal path, and a user with another CUDA onPATHcan have the version check/link libraries come from the wrong toolkit beforeenable_language(CUDA)honors the pinned compiler.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Confirmed —
find_package(CUDAToolkit)runs beforeenable_language(CUDA), so it consultsCUDAToolkit_ROOT/CUDA_PATH/PATHrather thanCMAKE_CUDA_COMPILER/CUDACXX, and a pinned-but-not-on-PATH nvcc (or a stray CUDA on PATH) can drive the early check from the wrong toolkit. The intended fix is to deriveCUDAToolkit_ROOTfrom a pinnedCMAKE_CUDA_COMPILER/CUDACXXbefore the probe. Deferring this to a follow-up PR since it needs a Linux box with multiple CUDA toolkits to verify; tracked there.Posted by Claude Code on behalf of @pcchen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we defer this issue to following PR, this PR fixed the problem in the wrong way. In this change, setting
CMAKE_CUDA_COMPILERto a path other than the path found infind_package(CUDAToolkit)may cause the version of nvcc and cuda toolkit different, and the build may fail. Suggest pattern is not to find cuda toolkit before enabling cudaThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CMAKE_CUDA_ARCHITECTURES all-majorcannot be used here because that will include 50 and 60 for cuda 12.x. 50 and 60 are not supported by our required cuTENSOR and cuQuantum.