-
Notifications
You must be signed in to change notification settings - Fork 191
[fix] Let wheels resolve CUDA math libraries from pip packages #76
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
behnamasadi
wants to merge
5
commits into
nvidia-isaac:main
Choose a base branch
from
behnamasadi:behnamasadi/wheel-cuda-math-libs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
077f30b
[fix] Let wheels resolve CUDA math libraries from pip packages
behnamasadi d919069
[fix] Address review on the wheel CUDA math library fix
behnamasadi d6e7ad6
[fix] Correct the rationale for the x86_64-only CUDA extra check
behnamasadi 6e8011d
[fix] Address the second review round on the wheel CUDA library fix
behnamasadi 087b1b1
[fix] Derive the README install extra from the wheel tag
behnamasadi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| # Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. | ||
| # | ||
| # NVIDIA software released under the NVIDIA Community License is intended to be used to enable | ||
| # the further development of AI and robotics technologies. Such software has been designed, tested, | ||
| # and optimized for use with NVIDIA hardware, and this License grants permission to use the software | ||
| # solely with such hardware. | ||
| # Subject to the terms of this License, NVIDIA confirms that you are free to commercially use, | ||
| # modify, and distribute the software with NVIDIA hardware. NVIDIA does not claim ownership of any | ||
| # outputs generated using the software or derivative works thereof. Any code contributions that you | ||
| # share with NVIDIA are licensed to NVIDIA as feedback under this License and may be incorporated | ||
| # in future releases without notice or attribution. | ||
| # By using, reproducing, modifying, distributing, performing, or displaying any portion or element | ||
| # of the software or derivative works thereof, you agree to be bound by this License. | ||
|
|
||
| """Resolution of the CUDA math libraries that PyCuVSLAM links against but does not bundle. | ||
|
|
||
| ``libcuvslam.so`` links against cuBLAS, cuSOLVER, cuSPARSE and their dependencies. Wheels deliberately exclude them | ||
| during ``auditwheel repair`` (see ``scripts/build_pycuvslam_in_docker.sh``) because they are large, so they have to be | ||
| provided by the environment: either by a CUDA Toolkit installation, or by the ``nvidia-*`` pip packages that the | ||
| ``cu12``/``cu13`` extras of this package declare. | ||
|
|
||
| Pip installs those packages into ``<site-packages>/nvidia/<component>/lib``, a location the dynamic loader does not | ||
| search. Loading them here by absolute path, before the extension module is imported, registers their sonames with the | ||
| loader so ``libcuvslam.so`` resolves against them without the caller having to set ``LD_LIBRARY_PATH``. | ||
| """ | ||
|
|
||
| import ctypes | ||
| import os | ||
| from glob import glob | ||
|
|
||
| # Components of the pip CUDA layout that PyCuVSLAM needs, ordered so that a library is loaded after the libraries it | ||
| # depends on (cuSOLVER needs cuBLAS and cuSPARSE, which in turn need nvJitLink). preload() does not rely on the order | ||
| # being exactly right, it retries, but a good order keeps the common case to a single pass. | ||
| CUDA_COMPONENTS = ('nvjitlink', 'cublas', 'cusparse', 'cusolver') | ||
|
|
||
| # Appended to the ImportError raised when the extension module cannot be loaded because of a missing library. | ||
| MISSING_LIBRARY_HINT = ( | ||
| "PyCuVSLAM links against the CUDA math libraries (cuBLAS, cuSOLVER, cuSPARSE) but does not bundle them.\n" | ||
| "Provide them either by installing a CUDA Toolkit whose major version matches this wheel's cu12/cu13 tag, or by\n" | ||
| "installing the matching pip packages:\n" | ||
| " pip install 'cuvslam[cu12]' # CUDA 12 wheels\n" | ||
| " pip install 'cuvslam[cu13]' # CUDA 13 wheels") | ||
|
|
||
|
|
||
| def nvidia_root(package_root=None): | ||
| """Return the directory the nvidia-* pip packages install into, next to this package.""" | ||
| if package_root is None: | ||
| package_root = os.path.dirname(os.path.abspath(__file__)) | ||
| return os.path.join(os.path.dirname(package_root), 'nvidia') | ||
|
|
||
|
|
||
| def candidate_libraries(package_root=None): | ||
| """Return the pip-provided CUDA libraries to preload, in dependency order.""" | ||
| root = nvidia_root(package_root) | ||
| if not os.path.isdir(root): | ||
| return [] | ||
| candidates = [] | ||
| for component in CUDA_COMPONENTS: | ||
| candidates.extend(sorted(glob(os.path.join(root, component, 'lib', 'lib*.so.*')))) | ||
| return candidates | ||
|
|
||
|
|
||
| def preload(package_root=None): | ||
| """Load the pip-provided CUDA math libraries, if any, and return the paths that were loaded. | ||
|
|
||
| Does nothing when the nvidia-* packages are not installed: the libraries are then expected to come from a system | ||
| CUDA Toolkit, which the loader finds on its own. | ||
| """ | ||
| pending = candidate_libraries(package_root) | ||
| loaded = [] | ||
| # A library whose own dependencies are not loaded yet fails to load, so sweep the list until a full pass makes no | ||
| # progress. Whatever is left unloaded is not reported here; the extension import below produces the actionable | ||
| # error message. | ||
| while pending: | ||
| remaining = [] | ||
| for path in pending: | ||
| try: | ||
| ctypes.CDLL(path, mode=ctypes.RTLD_GLOBAL) | ||
| except OSError: | ||
| remaining.append(path) | ||
| else: | ||
| loaded.append(path) | ||
| if len(remaining) == len(pending): | ||
| break | ||
| pending = remaining | ||
| return loaded |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| # Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. | ||
| # | ||
| # NVIDIA software released under the NVIDIA Community License is intended to be used to enable | ||
| # the further development of AI and robotics technologies. Such software has been designed, tested, | ||
| # and optimized for use with NVIDIA hardware, and this License grants permission to use the software | ||
| # solely with such hardware. | ||
| # Subject to the terms of this License, NVIDIA confirms that you are free to commercially use, | ||
| # modify, and distribute the software with NVIDIA hardware. NVIDIA does not claim ownership of any | ||
| # outputs generated using the software or derivative works thereof. Any code contributions that you | ||
| # share with NVIDIA are licensed to NVIDIA as feedback under this License and may be incorporated | ||
| # in future releases without notice or attribution. | ||
| # By using, reproducing, modifying, distributing, performing, or displaying any portion or element | ||
| # of the software or derivative works thereof, you agree to be bound by this License. | ||
|
|
||
| import importlib.util | ||
| import os | ||
| import subprocess | ||
| import sys | ||
| import tempfile | ||
| import unittest | ||
|
|
||
|
|
||
| def load_cuda_libs(): | ||
| """Load ``cuvslam/_cuda_libs.py`` as a standalone module, without importing ``cuvslam``. | ||
|
|
||
| Importing the package runs ``preload()`` and loads the extension module, which is exactly the side effect | ||
| test_import_resolved_the_cuda_math_libraries has to observe in an interpreter that has not done it yet. | ||
| """ | ||
| package_root = importlib.util.find_spec('cuvslam').submodule_search_locations[0] | ||
| spec = importlib.util.spec_from_file_location('_cuda_libs_under_test', | ||
| os.path.join(package_root, '_cuda_libs.py')) | ||
| module = importlib.util.module_from_spec(spec) | ||
| spec.loader.exec_module(module) | ||
| return module | ||
|
|
||
|
|
||
| _cuda_libs = load_cuda_libs() | ||
|
|
||
| # Run in a fresh interpreter by test_import_resolved_the_cuda_math_libraries. | ||
| IMPORT_SIDE_EFFECT_PROBE = """ | ||
| import cuvslam # noqa: F401 (imported for its side effect on the process' loaded libraries) | ||
|
|
||
| with open('/proc/self/maps') as maps_file: | ||
| maps = maps_file.read() | ||
|
|
||
| missing = [soname for soname in ('libcublas.so.', 'libcusolver.so.', 'libcusparse.so.') if soname not in maps] | ||
| if missing: | ||
| raise SystemExit('not loaded after "import cuvslam": ' + ', '.join(missing)) | ||
| """ | ||
|
|
||
|
|
||
| def make_pip_cuda_layout(root, components): | ||
| """Create a fake <site-packages>/nvidia layout next to a fake cuvslam package directory.""" | ||
| package_root = os.path.join(root, 'cuvslam') | ||
| os.makedirs(package_root) | ||
| for component, libraries in components.items(): | ||
| lib_dir = os.path.join(root, 'nvidia', component, 'lib') | ||
| os.makedirs(lib_dir) | ||
| for library in libraries: | ||
| # Not valid ELF: enough to be discovered, never loadable. | ||
| open(os.path.join(lib_dir, library), 'wb').close() | ||
| return package_root | ||
|
|
||
|
|
||
| class TestCudaLibs(unittest.TestCase): | ||
| def test_no_pip_cuda_packages_installed(self): | ||
| # The libraries then come from the system CUDA Toolkit and there is nothing to preload. | ||
| with tempfile.TemporaryDirectory() as root: | ||
| package_root = os.path.join(root, 'cuvslam') | ||
| os.makedirs(package_root) | ||
| self.assertEqual(_cuda_libs.candidate_libraries(package_root), []) | ||
| self.assertEqual(_cuda_libs.preload(package_root), []) | ||
|
|
||
| def test_candidates_are_discovered_in_dependency_order(self): | ||
| with tempfile.TemporaryDirectory() as root: | ||
| package_root = make_pip_cuda_layout(root, { | ||
| 'cusolver': ['libcusolver.so.11'], | ||
| 'cublas': ['libcublas.so.12', 'libcublasLt.so.12'], | ||
| 'cusparse': ['libcusparse.so.12'], | ||
| 'nvjitlink': ['libnvJitLink.so.12']}) | ||
| found = [os.path.basename(path) for path in _cuda_libs.candidate_libraries(package_root)] | ||
| self.assertEqual(found, [ | ||
| 'libnvJitLink.so.12', 'libcublas.so.12', 'libcublasLt.so.12', 'libcusparse.so.12', | ||
| 'libcusolver.so.11']) | ||
|
|
||
| def test_unrelated_components_are_ignored(self): | ||
| with tempfile.TemporaryDirectory() as root: | ||
| package_root = make_pip_cuda_layout(root, { | ||
| 'cudnn': ['libcudnn.so.9'], | ||
| 'cusparse': ['libcusparse.so.12']}) | ||
| found = [os.path.basename(path) for path in _cuda_libs.candidate_libraries(package_root)] | ||
| self.assertEqual(found, ['libcusparse.so.12']) | ||
|
|
||
| def test_unloadable_libraries_do_not_raise(self): | ||
| # A library that cannot be loaded is left to the extension import to report, and the retry loop that | ||
| # tolerates an unknown load order must still terminate. | ||
| with tempfile.TemporaryDirectory() as root: | ||
| package_root = make_pip_cuda_layout(root, { | ||
| 'cublas': ['libcublas.so.12'], | ||
| 'cusolver': ['libcusolver.so.11']}) | ||
| self.assertEqual(_cuda_libs.preload(package_root), []) | ||
|
|
||
| def test_import_resolved_the_cuda_math_libraries(self): | ||
| # Whatever provided them, the CUDA math libraries libcuvslam.so links against are loaded once cuvslam | ||
| # imports: this is the regression guard for wheels that neither bundle nor declare them. It runs in a | ||
| # fresh interpreter, because a cuvslam already imported by another test would load them regardless of | ||
| # whether importing it still does. | ||
| probe = subprocess.run([sys.executable, '-c', IMPORT_SIDE_EFFECT_PROBE], | ||
| capture_output=True, text=True, check=False) | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| self.assertEqual(probe.returncode, 0, probe.stderr) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: nvidia-isaac/cuVSLAM
Length of output: 1787
🏁 Script executed:
Repository: nvidia-isaac/cuVSLAM
Length of output: 946
Require a single wheel match before deriving the CUDA extra.
wheel=$(echo cuvslam-*.whl)can expand to the literal glob when nothing matches, or to multiple filenames when more than one wheel is present; in the latter case thegrepoutput is concatenated into an invalidpip installargument. Guard for exactly one wheel before building the extra.🤖 Prompt for AI Agents