From 2523040b0850cce748e02b1fa611f7f43932b1e7 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 20 May 2026 21:16:15 +1200 Subject: [PATCH 01/48] nanovdb python: Phase 0 foundation for C++ API mirror (#2209) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: Phase 0 foundation for C++ API mirror First slice of the Python bindings restructure tracked in #2208 and laid out in nanovdb-python-plan.md. Phase 0 is the mechanical groundwork the rest of the plan builds on: - BuildTypes.def: single X-macro list of currently-bound BuildT types (scalar / vector / point / sampleable). NanoVDBModule.cc, PyMath.cc, PySampleFromVoxels.cc, PyCreateNanoGrid.cc, PyTools.cc, PyGridHandle.h and cuda/PyDeviceGridHandle.cu now drive their per-type instantiations from this one file. Adding a BuildT in Phase 2 becomes one line. - CMakeLists.txt: under SKBUILD, ship the nanovdb/ headers inside the Python wheel at nanovdb/include/nanovdb/ so downstream extension authors can compile against the same headers the wheel was built with. Also wire nanobind_add_stub so a nanovdb.pyi (and py.typed marker) are emitted into the wheel for IDE / type-checker support; gated behind NANOVDB_BUILD_PYTHON_STUBS and silently skipped on older nanobind. - __init__.py: add nanovdb.get_include() returning the bundled include dir. Also fix the Windows DLL shim, which referenced an undefined `directory` variable instead of the local `openvdb_dll_directory`. - Relocate the batched sampleFromVoxels CUDA kernel binding from the phantom nanovdb.math.cuda submodule (which has no C++ counterpart) to nanovdb.tools.cuda, alongside the existing signedFloodFill and pointsToRGBA8Grid kernels. The math.cuda submodule is no longer registered. TestNanoVDB.py updated to match. - Pre-existing observable bug fixes: * GridHandle.__bool__ returned None (lambda was missing return); it now returns !handle.empty() as intended. * Enable __repr__ on GridType, GridClass and io::Codec via the nanovdb::toStr / strlen<> helpers — previously commented out. Build + pytest verified locally (CPU-only): 28 tests pass; the single pre-existing test_read_write_grid BLOSC failure is unrelated (test doesn't wrap the optional codec call in try/except like its sibling does). Part of #2208. Signed-off-by: Jonathan Swartz * nanovdb python: align Phase 0 X-macros with codingstyle.txt Audit of the new BuildTypes.def + consumers against nanovdb/nanovdb/docs/codingstyle.txt: - Rename the X-macro family NVDB_PY_FOR_EACH_*_BUILDT to NANOVDB_PY_FOR_EACH_*_BUILDT and the helper sentinels NVDB_PY_LOCAL_DEFINED_* to NANOVDB_PY_LOCAL_DEFINED_* to match the established NANOVDB_ macro prefix used everywhere else in the codebase (NANOVDB_USE_CUDA, NANOVDB_BUILD_PYTHON_MODULE, NANOVDB_HOSTDEV, ...). - Bring lines under the 100-column limit: * Wrap the long VECTOR_BUILDT lines in BuildTypes.def across two lines each (cleaner alignment, no behavior change). * Rename the consumer macro parameter DeviceHandleMethod to DeviceMethod so the #define line itself fits (was 102 cols). * Reformat the GridHandle __bool__ lambda onto three lines instead of one 119-col line (made worse by the `return` fix in the previous commit). - Add a top-of-file justification block in BuildTypes.def explaining why this file is a deliberate exception to the codingstyle "avoid macro functions" rule (templates can't emit top-level declarations and explicit instantiations across translation units from a single canonical list). No functional change. Rebuild + pytest_nanovdb is identical to the previous commit: 28 tests pass, 8 CUDA skips, 1 pre-existing BLOSC failure. Signed-off-by: Jonathan Swartz * nanovdb python: only generate .pyi stubs under SKBUILD by default Fixes the macOS CI failure on #2209. The previous commit defaulted NANOVDB_BUILD_PYTHON_STUBS to ON, which made the nanovdb_python_stub target fire on every in-tree CI build. The macOS GitHub Actions runner wraps the stubgen invocation with cmake -E env DYLD_INSERT_LIBRARIES=.../libclang_rt.tsan_osx_dynamic.dylib: .../libclang_rt.asan_osx_dynamic.dylib: .../libclang_rt.ubsan_osx_dynamic.dylib ASAN_OPTIONS=detect_leaks=0 python stubgen.py -m nanovdb ... (this preamble is added by the runner / CMake env wrapper — nothing in our nanobind_add_stub call sets it, and the .so itself is not built with -fsanitize). ThreadSanitizer can only install its interceptors at process start; Python loads first and then dlopen's the compiled .so, which TSan considers "too late" and aborts: ==49907==ERROR: Interceptors are not working. This may be because ThreadSanitizer is loaded too late (e.g. via dlopen). So the macOS build target failed with exit 2 after the .so itself built fine. The other matrix legs (linux-nanovdb Debug/Release for clang/gcc) all passed. Stubs are only useful to wheel consumers — they ship next to the .so in the installed package layout under SKBUILD. The in-source OpenVDB CI build never consumes them, so making stub generation default ON only when SKBUILD is set keeps the wheel build path unchanged and stops the macOS CI from invoking stubgen under the sanitizer wrapper. The user can still force generation with -DNANOVDB_BUILD_PYTHON_STUBS=ON for local dev builds where it's useful. Verified locally: - in-tree config (SKBUILD unset): no nanovdb_python_stub target defined; `make nanovdb_python_stub` errors with "no rule". Matches desired CI behavior. - SKBUILD=ON config: stub target exists, builds, emits nanovdb.pyi and py.typed into the install layout (28 kB stub file with all the expected GridHandle/GridType/GridClass symbols). Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/BuildTypes.def | 92 +++++++++++++++++++ nanovdb/nanovdb/python/CMakeLists.txt | 47 ++++++++++ nanovdb/nanovdb/python/NanoVDBModule.cc | 55 +++++------ nanovdb/nanovdb/python/PyCreateNanoGrid.cc | 7 +- nanovdb/nanovdb/python/PyGridHandle.h | 37 ++++---- nanovdb/nanovdb/python/PyIO.cc | 12 +-- nanovdb/nanovdb/python/PyMath.cc | 34 ++----- nanovdb/nanovdb/python/PySampleFromVoxels.cc | 25 ++--- nanovdb/nanovdb/python/PyTools.cc | 11 ++- nanovdb/nanovdb/python/__init__.py | 25 ++++- .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 27 +++--- nanovdb/nanovdb/python/test/TestNanoVDB.py | 4 +- 12 files changed, 247 insertions(+), 129 deletions(-) create mode 100644 nanovdb/nanovdb/python/BuildTypes.def diff --git a/nanovdb/nanovdb/python/BuildTypes.def b/nanovdb/nanovdb/python/BuildTypes.def new file mode 100644 index 0000000000..cac3f89c21 --- /dev/null +++ b/nanovdb/nanovdb/python/BuildTypes.def @@ -0,0 +1,92 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +// +// X-macro list of NanoVDB BuildT types currently exposed to Python. +// +// NanoVDB coding style says "avoid macro functions; use inline and templates +// instead" (docs/codingstyle.txt). This file is a deliberate exception: it +// must emit top-level template instantiations and binding registrations into +// several translation units from one canonical list, which a C++ template +// cannot do. The pattern is contained to this file plus the matching +// per-consumer macro definitions; no runtime behavior is hidden behind a +// macro function. +// +// Consumers define one or more of the kind-specific macros below before +// including this file. Each macro is invoked once per matching type; any +// macro the consumer does not define is treated as a no-op and reset after +// the file is processed. This is the single point that must be edited when +// adding (or removing) a Python-visible BuildT. +// +// Macros: +// NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) +// Scalar value types — exposed with full NodeInfo accessors. Suffix +// forms Python class names (e.g. "Float" -> "FloatGrid"). HandleMethod +// and DeviceMethod are the legacy lower-camel-case method names +// on GridHandle / DeviceGridHandle (e.g. "floatGrid"). +// +// NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) +// Vector value types — exposed with a setVoxel accessor but no +// NodeInfo. AccessorName is passed explicitly because the legacy +// Python class names for these are inconsistent. +// +// NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix) +// The nanovdb::Point build type — exposed with a bare accessor and no +// GridHandle method (Point is not currently surfaced through +// handle.*Grid()). +// +// NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) +// Subset that has C++ sampler specializations (used by PyMath samplers +// and PyCreateNanoGrid create*Grid factories). + +#ifndef NANOVDB_PY_FOR_EACH_SCALAR_BUILDT +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) +#define NANOVDB_PY_LOCAL_DEFINED_SCALAR +#endif +#ifndef NANOVDB_PY_FOR_EACH_VECTOR_BUILDT +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) +#define NANOVDB_PY_LOCAL_DEFINED_VECTOR +#endif +#ifndef NANOVDB_PY_FOR_EACH_POINT_BUILDT +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix) +#define NANOVDB_PY_LOCAL_DEFINED_POINT +#endif +#ifndef NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT +#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) +#define NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE +#endif + +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(float, Float, "floatGrid", "deviceFloatGrid") +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(double, Double, "doubleGrid", "deviceDoubleGrid") +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int32_t, Int32, "int32Grid", "deviceInt32Grid") + +NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3f, Vec3f, + "Vec3fReadVectorAccessor", + "vec3fGrid", "deviceVec3fGrid") +NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::math::Rgba8, RGBA8, + "RGBA8ReadAccessor", + "rgba8Grid", "deviceRGBA8Grid") + +NANOVDB_PY_FOR_EACH_POINT_BUILDT(::nanovdb::Point, Point) + +NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(float, Float) +NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(double, Double) +NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(int32_t, Int32) +NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(::nanovdb::Vec3f, Vec3f) + +#ifdef NANOVDB_PY_LOCAL_DEFINED_SCALAR +#undef NANOVDB_PY_LOCAL_DEFINED_SCALAR +#endif +#ifdef NANOVDB_PY_LOCAL_DEFINED_VECTOR +#undef NANOVDB_PY_LOCAL_DEFINED_VECTOR +#endif +#ifdef NANOVDB_PY_LOCAL_DEFINED_POINT +#undef NANOVDB_PY_LOCAL_DEFINED_POINT +#endif +#ifdef NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE +#undef NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE +#endif + +#undef NANOVDB_PY_FOR_EACH_SCALAR_BUILDT +#undef NANOVDB_PY_FOR_EACH_VECTOR_BUILDT +#undef NANOVDB_PY_FOR_EACH_POINT_BUILDT +#undef NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 7350ce84da..1b5fc4166e 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -4,6 +4,16 @@ option(NANOVDB_BUILD_PYTHON_UNITTESTS [=[ "Include the NanoVDB Python unit test. Requires a python interpreter]=] ${NANOVDB_BUILD_UNITTESTS}) +# Stubs are only consumed from an installed wheel (they ship next to the .so +# in nanovdb/), so default ON only when SKBUILD is driving the build. In an +# in-source dev / OpenVDB-CI build the .pyi is not used and stub generation +# can be skipped — which also dodges the macOS CI's sanitizer-runtime +# DYLD_INSERT_LIBRARIES wrapper, under which Python dlopen'ing the .so +# triggers TSan "Interceptors are not working" and aborts stubgen. +option(NANOVDB_BUILD_PYTHON_STUBS + "Generate .pyi type stubs for the nanovdb Python module (requires nanobind 2.x or newer)." + ${SKBUILD}) + nanobind_add_module(nanovdb_python NB_STATIC NanoVDBModule.cc PyCreateNanoGrid.cc @@ -33,10 +43,47 @@ if(SKBUILD) set_target_properties(nanovdb_python PROPERTIES INSTALL_RPATH "$ORIGIN/../../openvdb/lib") install(TARGETS nanovdb_python DESTINATION ${NANOVDB_INSTALL_LIBDIR}) install(FILES __init__.py DESTINATION nanovdb) + # Ship the nanovdb C/C++ headers inside the Python wheel so downstream + # extension authors can compile against the same NanoVDB the wheel was + # built with. nanovdb.get_include() resolves to this directory at runtime. + install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/../ + DESTINATION nanovdb/include/nanovdb + FILES_MATCHING + PATTERN "*.h" + PATTERN "*.cuh" + PATTERN "python" EXCLUDE + PATTERN "examples" EXCLUDE + PATTERN "unittest" EXCLUDE + PATTERN "cmd" EXCLUDE + PATTERN "docs" EXCLUDE) else() install(TARGETS nanovdb_python DESTINATION ${VDB_PYTHON_INSTALL_DIRECTORY}) endif() +# .pyi type stubs for IDE / type-checker support. Driven by the CMake helper +# that ships with nanobind 2.x; silently skipped on older nanobind where the +# helper does not exist. +if(NANOVDB_BUILD_PYTHON_STUBS AND COMMAND nanobind_add_stub) + set(NANOVDB_STUB_OUTPUT_DIR "${CMAKE_CURRENT_BINARY_DIR}") + nanobind_add_stub(nanovdb_python_stub + MODULE nanovdb + OUTPUT "${NANOVDB_STUB_OUTPUT_DIR}/nanovdb.pyi" + PYTHON_PATH $ + DEPENDS nanovdb_python + MARKER_FILE "${NANOVDB_STUB_OUTPUT_DIR}/py.typed" + INCLUDE_PRIVATE) + if(SKBUILD) + install(FILES + "${NANOVDB_STUB_OUTPUT_DIR}/nanovdb.pyi" + "${NANOVDB_STUB_OUTPUT_DIR}/py.typed" + DESTINATION nanovdb) + endif() +elseif(NANOVDB_BUILD_PYTHON_STUBS) + message(STATUS + "NANOVDB_BUILD_PYTHON_STUBS is ON but nanobind_add_stub is unavailable " + "(nanobind >= 2.0 required). Skipping stub generation.") +endif() + # pytest if(NANOVDB_BUILD_PYTHON_UNITTESTS) diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index c812f2f431..c0d3566a37 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -343,12 +343,12 @@ NB_MODULE(nanovdb, m) .value("Vec3u8", GridType::Vec3u8) .value("Vec3u16", GridType::Vec3u16) .value("End", GridType::End) - .export_values(); - // .def("__repr__", [](const GridType& gridType) { - // char str[strlen()]; - // toStr(str, gridType); - // return std::string(str); - // }); + .export_values() + .def("__repr__", [](const GridType& gridType) { + char str[strlen()]; + toStr(str, gridType); + return std::string(str); + }); nb::enum_(m, "GridClass") .value("Unknown", GridClass::Unknown) @@ -362,12 +362,12 @@ NB_MODULE(nanovdb, m) .value("IndexGrid", GridClass::IndexGrid) .value("TensorGrid", GridClass::TensorGrid) .value("End", GridClass::End) - .export_values(); - // .def("__repr__", [](const GridClass& gridClass) { - // char str[strlen()]; - // toStr(str, gridClass); - // return std::string(str); - // }); + .export_values() + .def("__repr__", [](const GridClass& gridClass) { + char str[strlen()]; + toStr(str, gridClass); + return std::string(str); + }); defineVersion(m); @@ -385,26 +385,17 @@ NB_MODULE(nanovdb, m) defineGridData(m); - defineGrid(m, "FloatGrid"); - defineScalarAccessor(m, "FloatReadAccessor"); - defineNodeInfo(m, "FloatNodeInfo"); - - defineGrid(m, "DoubleGrid"); - defineScalarAccessor(m, "DoubleReadAccessor"); - defineNodeInfo(m, "DoubleNodeInfo"); - - defineGrid(m, "Int32Grid"); - defineScalarAccessor(m, "Int32ReadAccessor"); - defineNodeInfo(m, "Int32NodeInfo"); - - defineGrid(m, "Vec3fGrid"); - defineVectorAccessor(m, "Vec3fReadVectorAccessor"); - - defineGrid(m, "RGBA8Grid"); - defineVectorAccessor(m, "RGBA8ReadAccessor"); - - defineGrid(m, "PointGrid"); - defineAccessor(m, "PointReadAccessor"); +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) \ + defineGrid(m, #Suffix "Grid"); \ + defineScalarAccessor(m, #Suffix "ReadAccessor"); \ + defineNodeInfo(m, #Suffix "NodeInfo"); +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) \ + defineGrid(m, #Suffix "Grid"); \ + defineVectorAccessor(m, AccessorName); +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix) \ + defineGrid(m, #Suffix "Grid"); \ + defineAccessor(m, #Suffix "ReadAccessor"); +#include "BuildTypes.def" defineHostBuffer(m); defineHostGridHandle(m); diff --git a/nanovdb/nanovdb/python/PyCreateNanoGrid.cc b/nanovdb/nanovdb/python/PyCreateNanoGrid.cc index eed931766c..afe70287b8 100644 --- a/nanovdb/nanovdb/python/PyCreateNanoGrid.cc +++ b/nanovdb/nanovdb/python/PyCreateNanoGrid.cc @@ -45,10 +45,9 @@ template void defineOpenToNanoVDB(nb::module_& m) #endif } -template void defineCreateNanoGrid(nb::module_&, const char*); -template void defineCreateNanoGrid(nb::module_&, const char*); -template void defineCreateNanoGrid(nb::module_&, const char*); -template void defineCreateNanoGrid(nb::module_&, const char*); +#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \ + template void defineCreateNanoGrid(nb::module_&, const char*); +#include "BuildTypes.def" template void defineOpenToNanoVDB(nb::module_&); diff --git a/nanovdb/nanovdb/python/PyGridHandle.h b/nanovdb/nanovdb/python/PyGridHandle.h index b102899365..f23269d392 100644 --- a/nanovdb/nanovdb/python/PyGridHandle.h +++ b/nanovdb/nanovdb/python/PyGridHandle.h @@ -14,31 +14,30 @@ namespace pynanovdb { template nb::class_> defineGridHandle(nb::module_& m, const char* name) { - return nb::class_>(m, name) + auto cls = nb::class_>(m, name) .def(nb::init<>()) .def("reset", &nanovdb::GridHandle::reset) .def("size", &nanovdb::GridHandle::bufferSize) .def("isEmpty", &nanovdb::GridHandle::isEmpty) .def("empty", &nanovdb::GridHandle::empty) .def( - "__bool__", [](const nanovdb::GridHandle& handle) { handle.empty(); }, nb::is_operator()) - .def("floatGrid", nb::overload_cast(&nanovdb::GridHandle::template grid), nb::arg("n") = 0, nb::rv_policy::reference_internal) - .def("doubleGrid", - nb::overload_cast(&nanovdb::GridHandle::template grid), - nb::arg("n") = 0, - nb::rv_policy::reference_internal) - .def("int32Grid", - nb::overload_cast(&nanovdb::GridHandle::template grid), - nb::arg("n") = 0, - nb::rv_policy::reference_internal) - .def("vec3fGrid", - nb::overload_cast(&nanovdb::GridHandle::template grid), - nb::arg("n") = 0, - nb::rv_policy::reference_internal) - .def("rgba8Grid", - nb::overload_cast(&nanovdb::GridHandle::template grid), - nb::arg("n") = 0, - nb::rv_policy::reference_internal) + "__bool__", + [](const nanovdb::GridHandle& handle) { return !handle.empty(); }, + nb::is_operator()); + +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) \ + cls.def(HandleMethod, \ + nb::overload_cast(&nanovdb::GridHandle::template grid), \ + nb::arg("n") = 0, \ + nb::rv_policy::reference_internal); +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) \ + cls.def(HandleMethod, \ + nb::overload_cast(&nanovdb::GridHandle::template grid), \ + nb::arg("n") = 0, \ + nb::rv_policy::reference_internal); +#include "BuildTypes.def" + + return cls .def("isPadded", &nanovdb::GridHandle::isPadded) .def("gridCount", &nanovdb::GridHandle::gridCount) .def("gridSize", &nanovdb::GridHandle::gridSize, nb::arg("n") = 0) diff --git a/nanovdb/nanovdb/python/PyIO.cc b/nanovdb/nanovdb/python/PyIO.cc index 83f5089483..fb50e3d08a 100644 --- a/nanovdb/nanovdb/python/PyIO.cc +++ b/nanovdb/nanovdb/python/PyIO.cc @@ -138,12 +138,12 @@ void defineIOModule(nb::module_& m) .value("NONE", io::Codec::NONE) .value("ZIP", io::Codec::ZIP) .value("BLOSC", io::Codec::BLOSC) - .export_values(); - // .def("__repr__", [](const io::Codec& codec) { - // char str[strlen()]; - // toStr(str, codec); - // return std::string(str); - // }); + .export_values() + .def("__repr__", [](const io::Codec& codec) { + char str[strlen()]; + toStr(str, codec); + return std::string(str); + }); defineFileGridMetaData(m); defineHostReadWriteGrid(m); diff --git a/nanovdb/nanovdb/python/PyMath.cc b/nanovdb/nanovdb/python/PyMath.cc index 337865ed8c..4ae8c852fe 100644 --- a/nanovdb/nanovdb/python/PyMath.cc +++ b/nanovdb/nanovdb/python/PyMath.cc @@ -11,7 +11,6 @@ #include #include "PySampleFromVoxels.h" -#include "cuda/PySampleFromVoxels.h" namespace nb = nanobind; using namespace nb::literals; @@ -377,33 +376,12 @@ void defineMathModule(nb::module_& m) defineBaseBBox(m, "CoordBaseBBox"); defineBBoxInteger(m, "CoordBBox", "Bounding box for Coord minimum and maximum"); - defineNearestNeighborSampler(m, "FloatNearestNeighborSampler"); - defineTrilinearSampler(m, "FloatTrilinearSampler"); - defineTriquadraticSampler(m, "FloatTriquadraticSampler"); - defineTricubicSampler(m, "FloatTricubicSampler"); - - defineNearestNeighborSampler(m, "DoubleNearestNeighborSampler"); - defineTrilinearSampler(m, "DoubleTrilinearSampler"); - defineTriquadraticSampler(m, "DoubleTriquadraticSampler"); - defineTricubicSampler(m, "DoubleTricubicSampler"); - - defineNearestNeighborSampler(m, "Int32NearestNeighborSampler"); - defineTrilinearSampler(m, "Int32TrilinearSampler"); - defineTriquadraticSampler(m, "Int32TriquadraticSampler"); - defineTricubicSampler(m, "Int32TricubicSampler"); - - defineNearestNeighborSampler(m, "Vec3fNearestNeighborSampler"); - defineTrilinearSampler(m, "Vec3fTrilinearSampler"); - defineTriquadraticSampler(m, "Vec3fTriquadraticSampler"); - defineTricubicSampler(m, "Vec3fTricubicSampler"); - -#ifdef NANOVDB_USE_CUDA - nb::module_ cudaModule = m.def_submodule("cuda"); - cudaModule.doc() = "A submodule that implements CUDA-accelerated math functions"; - - defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); - defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); -#endif +#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \ + defineNearestNeighborSampler(m, #Suffix "NearestNeighborSampler"); \ + defineTrilinearSampler(m, #Suffix "TrilinearSampler"); \ + defineTriquadraticSampler(m, #Suffix "TriquadraticSampler"); \ + defineTricubicSampler(m, #Suffix "TricubicSampler"); +#include "BuildTypes.def" } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PySampleFromVoxels.cc b/nanovdb/nanovdb/python/PySampleFromVoxels.cc index 82dc9a26d1..f7f114c15d 100644 --- a/nanovdb/nanovdb/python/PySampleFromVoxels.cc +++ b/nanovdb/nanovdb/python/PySampleFromVoxels.cc @@ -65,24 +65,11 @@ template void defineTricubicSampler(nb::module_& m, const char* defineCreateSampler(m, "createTricubicSampler"); } -template void defineNearestNeighborSampler(nb::module_&, const char*); -template void defineTrilinearSampler(nb::module_&, const char*); -template void defineTriquadraticSampler(nb::module_&, const char*); -template void defineTricubicSampler(nb::module_&, const char*); - -template void defineNearestNeighborSampler(nb::module_&, const char*); -template void defineTrilinearSampler(nb::module_&, const char*); -template void defineTriquadraticSampler(nb::module_&, const char*); -template void defineTricubicSampler(nb::module_&, const char*); - -template void defineNearestNeighborSampler(nb::module_&, const char*); -template void defineTrilinearSampler(nb::module_&, const char*); -template void defineTriquadraticSampler(nb::module_&, const char*); -template void defineTricubicSampler(nb::module_&, const char*); - -template void defineNearestNeighborSampler(nb::module_&, const char*); -template void defineTrilinearSampler(nb::module_&, const char*); -template void defineTriquadraticSampler(nb::module_&, const char*); -template void defineTricubicSampler(nb::module_&, const char*); +#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \ + template void defineNearestNeighborSampler(nb::module_&, const char*); \ + template void defineTrilinearSampler(nb::module_&, const char*); \ + template void defineTriquadraticSampler(nb::module_&, const char*); \ + template void defineTricubicSampler(nb::module_&, const char*); +#include "BuildTypes.def" } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 4df996d557..a5c29652c3 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -16,6 +16,7 @@ #include "PyNanoToOpenVDB.h" #ifdef NANOVDB_USE_CUDA #include "cuda/PyPointsToGrid.h" +#include "cuda/PySampleFromVoxels.h" #include "cuda/PySignedFloodFill.h" #endif @@ -33,10 +34,9 @@ void defineToolsModule(nb::module_& m) definePrimitives(m); - defineCreateNanoGrid(m, "createFloatGrid"); - defineCreateNanoGrid(m, "createDoubleGrid"); - defineCreateNanoGrid(m, "createInt32Grid"); - defineCreateNanoGrid(m, "createVec3fGrid"); +#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \ + defineCreateNanoGrid(m, "create" #Suffix "Grid"); +#include "BuildTypes.def" #ifdef NANOVDB_USE_OPENVDB defineOpenToNanoVDB(m); @@ -55,6 +55,9 @@ void defineToolsModule(nb::module_& m) defineSignedFloodFill(cudaModule, "signedFloodFill"); definePointsToGrid(cudaModule, "pointsToRGBA8Grid"); + + defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); + defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); #endif } diff --git a/nanovdb/nanovdb/python/__init__.py b/nanovdb/nanovdb/python/__init__.py index 4f955e4a02..9ff10b6c28 100644 --- a/nanovdb/nanovdb/python/__init__.py +++ b/nanovdb/nanovdb/python/__init__.py @@ -1,9 +1,26 @@ # Copyright Contributors to the OpenVDB Project # SPDX-License-Identifier: Apache-2.0 +import os import sys + if sys.platform == "win32": - import os - openvdb_dll_directory = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir, 'openvdb', 'lib') - os.add_dll_directory(directory) + _openvdb_dll_directory = os.path.join( + os.path.dirname(os.path.abspath(__file__)), os.pardir, "openvdb", "lib" + ) + if os.path.isdir(_openvdb_dll_directory): + os.add_dll_directory(_openvdb_dll_directory) + + +def get_include(): + """Return the absolute path to the bundled NanoVDB C/C++ include directory. + + Use this from a downstream Python extension's build system so the extension + compiles against the same NanoVDB headers the installed wheel was built with:: + + import nanovdb + ext_kwargs = dict(include_dirs=[nanovdb.get_include()]) + """ + return os.path.join(os.path.dirname(os.path.abspath(__file__)), "include") + -from .lib.nanovdb import * +from .lib.nanovdb import * # noqa: E402,F401,F403 diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index 983caebd05..390614358b 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -16,7 +16,7 @@ namespace pynanovdb { void defineDeviceGridHandle(nb::module_& m) { using BufferT = nanovdb::cuda::DeviceBuffer; - defineGridHandle(m, "DeviceGridHandle") + auto cls = defineGridHandle(m, "DeviceGridHandle") .def( "__init__", [](GridHandle& handle, @@ -27,16 +27,21 @@ void defineDeviceGridHandle(nb::module_& m) new (&handle) GridHandle(std::move(buffer)); }, "cpu_t"_a.noconvert(), - "cuda_t"_a.noconvert()) - .def("deviceFloatGrid", nb::overload_cast(&GridHandle::template deviceGrid), "n"_a = 0, nb::rv_policy::reference_internal) - .def("deviceDoubleGrid", nb::overload_cast(&GridHandle::template deviceGrid), "n"_a = 0, nb::rv_policy::reference_internal) - .def("deviceInt32Grid", nb::overload_cast(&GridHandle::template deviceGrid), "n"_a = 0, nb::rv_policy::reference_internal) - .def("deviceVec3fGrid", nb::overload_cast(&GridHandle::template deviceGrid), "n"_a = 0, nb::rv_policy::reference_internal) - .def("deviceRGBA8Grid", - nb::overload_cast(&GridHandle::template deviceGrid), - "n"_a = 0, - nb::rv_policy::reference_internal) - .def( + "cuda_t"_a.noconvert()); + +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) \ + cls.def(DeviceMethod, \ + nb::overload_cast(&GridHandle::template deviceGrid), \ + "n"_a = 0, \ + nb::rv_policy::reference_internal); +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) \ + cls.def(DeviceMethod, \ + nb::overload_cast(&GridHandle::template deviceGrid), \ + "n"_a = 0, \ + nb::rv_policy::reference_internal); +#include "../BuildTypes.def" + + cls.def( "deviceUpload", [](GridHandle& handle, bool sync) { handle.deviceUpload(nullptr, sync); }, "sync"_a = true) .def( "deviceDownload", [](GridHandle& handle, bool sync) { handle.deviceDownload(nullptr, sync); }, "sync"_a = true); diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index bb86eea752..75936a49eb 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -663,7 +663,7 @@ def test_sample_from_points_float(self): dtype=torch.float32, device=torch.device("cuda", 0), ) - nanovdb.math.cuda.sampleFromVoxels(points, grid, values, gradients) + nanovdb.tools.cuda.sampleFromVoxels(points, grid, values, gradients) for i in range(5): self.assertEqual(values[i], expected_values[i]) for i in range(5): @@ -710,7 +710,7 @@ def test_sample_from_points_double(self): expected_values = torch.tensor( [-value, 0.0, -1.0, 1.0, value], dtype=torch.float64 ) - nanovdb.math.cuda.sampleFromVoxels(points, grid, values) + nanovdb.tools.cuda.sampleFromVoxels(points, grid, values) for i in range(5): self.assertEqual(values[i], expected_values[i]) From dd67710848bb60c60c7f88727853f119ad933f82 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 21 May 2026 09:14:41 +1200 Subject: [PATCH 02/48] =?UTF-8?q?nanovdb=20python:=20Phase=201=20=E2=80=94?= =?UTF-8?q?=20polymorphic=20Grid,=20GridMetaData,=20blind=20data,=20PointA?= =?UTF-8?q?ccessor=20(#2210)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: Phase 1 — polymorphic Grid, GridMetaData, blind data, PointAccessor Second slice of the Python bindings restructure tracked in #2208 and laid out in nanovdb-python-plan.md. Phase 1 bundles the three sub-todos (1a polymorphic API, 1b type-erased introspection + blind data, 1c PointAccessor + handle utilities) into a single change on top of the Phase 0 X-macro foundation. API changes (pre-1.0 breaks called out in the plan): - Polymorphic accessors. handle.grid(n=0) and handle.deviceGrid(n=0) replace the typed handle.floatGrid()/doubleGrid()/int32Grid()/ vec3fGrid()/rgba8Grid() and their device equivalents. Dispatch is driven by gridType(n) through a switch generated from BuildTypes.def; unbound BuildTs route to None rather than throwing. - Grid base class rename. The Python class previously bound as nanovdb.GridData is now nanovdb.Grid (matching the C++ user-facing class name Grid). All typed grid classes (FloatGrid, ...) inherit from Grid. - Base-class method lift. version/gridSize/gridIndex/gridCount/voxelSize/ map/gridType/gridClass/checksum/isLevelSet/isFogVolume/.../hasMinMax/ hasBBox/.../isBreadthFirst/shortGridName move from per-BuildT defineNanoGrid up to defineGrid via lambdas that read GridData data members directly. defineNanoGrid now only binds getAccessor, activeVoxelCount, and isSequential — the BuildT-dependent slice. Additive surface: - GridMetaData. Bound as nanovdb.GridMetaData with constructor from a Grid and the full read-only accessor surface (gridType, gridClass, shortGridName, gridSize/Index/Count, map, worldBBox, indexBBox, voxelSize, blindDataCount, activeVoxelCount, activeTileCount(level), nodeCount(level), checksum, version, isValid, isLevelSet/..., hasMinMax/..., isBreadthFirst, rootTableSize, isEmpty). Type-erased introspector — answer "what's in this buffer?" without knowing BuildT. - Blind data API on Grid. blindDataCount, blindMetaData(n), findBlindData(name), findBlindDataForSemantic(sem), getBlindData(n). getBlindData returns a zero-copy NumPy view typed by mDataType (Float -> 1D float32, Vec3f -> (N, 3) float32, RGBA8 -> (N, 4) uint8, etc.). Out-of-range and unknown-type paths return None / fall back to a flat uint8 byte view. - Enums: GridBlindDataClass (Unknown/IndexArray/AttributeArray/GridName/ ChannelArray/End) and GridBlindDataSemantic (Unknown/PointPosition/ PointColor/PointNormal/PointRadius/PointVelocity/PointId/WorldCoords/ GridCoords/VoxelCoords/LevelSet/FogVolume/Staggered/End). Bound as nb::enum_ with .export_values() so the names are also top-level attributes of the module. - GridBlindMetaData struct. Read-only fields valueCount/valueSize/ semantic/dataClass/dataType, name() accessor, isValid(), blindDataSize(). - PointAccessor variants. nanovdb.PointIndexAccessor (uint32 indices, used by PointIndex grids) and nanovdb.PointDataAccessor (Vec3f positions, used by PointData grids). Methods gridPoints(), leafPoints(ijk), voxelPoints(ijk) each return a zero-copy NumPy view onto the underlying blind-data buffer, anchored to the accessor lifetime via keep_alive. - GridHandle utilities. handle.copy() does a deep copy into a freshly allocated buffer of the same buffer type. Module-scope splitGrids(h) -> list[GridHandle] and mergeGrids(handles) -> GridHandle are registered for both host and device handles via the existing defineGridHandleUtilities template (nanobind merges them as an overload set). Mechanical X-macro changes: - BuildTypes.def gains a GridTypeEnum column on each row so the polymorphic dispatch in pyHostGrid/pyDeviceGrid can `case nanovdb::GridType:::` on it. The Point row maps to GridType::PointIndex (there is no GridType::Point). - HandleMethod/DeviceMethod columns dropped — the typed handle.fooGrid() accessors no longer exist. NB_MODULE bind order: - defineCheckMode + defineChecksum now bind BEFORE defineGrid because Grid.checksum() returns Checksum by value (registration must precede use). - defineGridBlindData binds BEFORE defineGrid for the same reason (Grid.findBlindDataForSemantic / blindMetaData reference the new enum and class in their signatures). Tests (TestNanoVDB.py): all typed-accessor call sites rewritten to handle.grid(i)/handle.deviceGrid(i). New test classes cover the new surface — TestPolymorphicGridAccess, TestGridBase, TestGridMetaData, TestBlindDataEmpty, TestSplitMergeCopy — 11 new tests; the full suite is now 48 tests, 39 pass on a minimal CPU build (8 CUDA skip, 1 pre-existing test_read_write_grid BLOSC failure unrelated to this PR). Signed-off-by: Jonathan Swartz * nanovdb python: don't expose splitGrids/mergeGrids for DeviceGridHandle Caught during a real-CUDA verification pass of #2210: calling nanovdb.mergeGrids([device_h1, device_h2]) raised std::bad_cast on sm_120 (Blackwell, CUDA 13.2). Both the host and device overloads of splitGrids/mergeGrids take nb::list, so nanobind's overload resolution can't disambiguate by element type — it picks the first match and the inner nb::cast(device_h) fails. The host-only variant is what the Phase 1 plan calls for. A properly typed device variant (with its own name, or strongly-typed std::vector args via nanobind/stl/vector.h) can land later if it's actually needed. handle.copy() on a DeviceGridHandle continues to work because copy() is a regular method, no overload resolution involved. Full CUDA pytest now reports 46/48 (the 2 failures are the pre-existing test_read_write_grid BLOSC bug, host + device variants — both call writeGrid(..., Codec.BLOSC) without try/except, identical to master). Signed-off-by: Jonathan Swartz * nanovdb python: address Copilot review on #2210 Three concrete fixes from Copilot's review of the Phase 1 PR (https://github.com/AcademySoftwareFoundation/openvdb/pull/2210): 1) mergeGrids no longer consumes its input handles. The previous implementation built a std::vector via nb::cast(h), which move-constructs the C++ GridHandle out of the Python wrapper — leaving caller's h1/h2 silently emptied (gridCount went 1 -> 0, size went non-zero -> 0). Reproduced before fix; locked into a regression test (TestSplitMergeCopy.test_merge_does_not_consume_inputs). Rewrote the binding to read each handle by const reference and inline the merge concat directly. The nanovdb::mergeGrids C++ helper's signature requires a std::vector (a move-only type), so reusing it from Python without moving from the inputs would have meant deep-copying each handle twice; the inlined version is ~15 lines and does one memcpy per source grid with tools::updateGridCount fixing up the per-grid header. 2) getBlindData validates mValueSize against the implied dtype/shape before building a typed NumPy view. A blind-data channel with mDataType=Float but mValueSize != 4 (corruption, version mismatch, or an unknown variant of a known tag) would previously be exposed as `count` float32 elements — i.e. count*4 bytes — even though the underlying region is only count*mValueSize bytes. That overruns the channel and returns a view onto unrelated bytes. Added a `valueSize == sizeof(...)` (or `dim*sizeof(scalar)` for vector cases) check on every handled GridType. On mismatch the binding falls back to a raw uint8 byte view of mValueCount * mValueSize, which is by definition the actual byte extent and therefore always safe. 3) GridMetaData ctor + safeCast guard against invalid grids before calling into NanoVDB, where NANOVDB_ASSERT(gridData->isValid()) would abort debug builds and undefined-behave in release. nanobind's type system already rejects Python None at the bind- site (None can't bind to const GridData*), so the literal "GridMetaData(None)" case Copilot called out is a TypeError today — but the broader concern (an otherwise-valid Grid object wrapping a corrupted buffer) is real. __init__ now does an explicit `gd == nullptr || !gd->isValid()` check and raises nb::value_error with a descriptive message before calling into nanovdb::GridMetaData. safeCast does the same and returns False on bad input, matching the spirit of "is this safe to cast?". New tests: TestGridMetaDataGuards covers the rejection paths and the still-works happy path. Build + test verified locally on both CPU (52 tests, 43 pass, 8 CUDA skip, 1 pre-existing BLOSC) and full CUDA (52 tests, 50 pass, 2 pre- existing BLOSC host+device). Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/BuildTypes.def | 45 +- nanovdb/nanovdb/python/NanoVDBModule.cc | 422 ++++++++++++++++-- nanovdb/nanovdb/python/PyGridHandle.cc | 1 + nanovdb/nanovdb/python/PyGridHandle.h | 128 +++++- .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 69 ++- nanovdb/nanovdb/python/test/TestNanoVDB.py | 198 ++++++-- 6 files changed, 736 insertions(+), 127 deletions(-) diff --git a/nanovdb/nanovdb/python/BuildTypes.def b/nanovdb/nanovdb/python/BuildTypes.def index cac3f89c21..5edae74ff7 100644 --- a/nanovdb/nanovdb/python/BuildTypes.def +++ b/nanovdb/nanovdb/python/BuildTypes.def @@ -18,36 +18,39 @@ // adding (or removing) a Python-visible BuildT. // // Macros: -// NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) +// NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) // Scalar value types — exposed with full NodeInfo accessors. Suffix -// forms Python class names (e.g. "Float" -> "FloatGrid"). HandleMethod -// and DeviceMethod are the legacy lower-camel-case method names -// on GridHandle / DeviceGridHandle (e.g. "floatGrid"). +// forms Python class names (e.g. "Float" -> "FloatGrid"). +// GridTypeEnum names the nanovdb::GridType:: enumerator a grid of +// this BuildT carries (used by the polymorphic handle.grid(n) +// dispatch). Usually identical to Suffix. // -// NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) +// NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) // Vector value types — exposed with a setVoxel accessor but no // NodeInfo. AccessorName is passed explicitly because the legacy // Python class names for these are inconsistent. // -// NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix) -// The nanovdb::Point build type — exposed with a bare accessor and no -// GridHandle method (Point is not currently surfaced through -// handle.*Grid()). +// NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) +// The nanovdb::Point build type — exposed with a bare accessor. +// Point grids carry GridType::PointIndex (NOT a GridType::Point; +// that enumerator doesn't exist), so GridTypeEnum is given +// explicitly. Polymorphic dispatch routes PointIndex grids to +// NanoGrid. // // NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) -// Subset that has C++ sampler specializations (used by PyMath samplers -// and PyCreateNanoGrid create*Grid factories). +// Subset that has C++ sampler specializations (used by PyMath +// samplers and PyCreateNanoGrid create*Grid factories). #ifndef NANOVDB_PY_FOR_EACH_SCALAR_BUILDT -#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) #define NANOVDB_PY_LOCAL_DEFINED_SCALAR #endif #ifndef NANOVDB_PY_FOR_EACH_VECTOR_BUILDT -#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) #define NANOVDB_PY_LOCAL_DEFINED_VECTOR #endif #ifndef NANOVDB_PY_FOR_EACH_POINT_BUILDT -#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix) +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) #define NANOVDB_PY_LOCAL_DEFINED_POINT #endif #ifndef NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT @@ -55,18 +58,16 @@ #define NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE #endif -NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(float, Float, "floatGrid", "deviceFloatGrid") -NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(double, Double, "doubleGrid", "deviceDoubleGrid") -NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int32_t, Int32, "int32Grid", "deviceInt32Grid") +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(float, Float, Float) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(double, Double, Double) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int32_t, Int32, Int32) NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3f, Vec3f, - "Vec3fReadVectorAccessor", - "vec3fGrid", "deviceVec3fGrid") + "Vec3fReadVectorAccessor", Vec3f) NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::math::Rgba8, RGBA8, - "RGBA8ReadAccessor", - "rgba8Grid", "deviceRGBA8Grid") + "RGBA8ReadAccessor", RGBA8) -NANOVDB_PY_FOR_EACH_POINT_BUILDT(::nanovdb::Point, Point) +NANOVDB_PY_FOR_EACH_POINT_BUILDT(::nanovdb::Point, Point, PointIndex) NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(float, Float) NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(double, Double) diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index c0d3566a37..80bf4586dc 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -1,6 +1,7 @@ // Copyright Contributors to the OpenVDB Project // SPDX-License-Identifier: Apache-2.0 #include +#include #include #include #include @@ -137,9 +138,25 @@ void defineMap(nb::module_& m) .def("getVoxelSize", &Map::getVoxelSize); } -void defineGridData(nb::module_& m) +// Forward declaration — body lives below defineGridBlindData() so it can +// reference the enum / class bindings registered there. +static nb::object pyGetBlindData(nb::handle py_grid, uint32_t n); + +// Type-erased Grid base class. nb::class_ is bound to nanovdb::GridData +// (the 672 B POD prefix present at the start of every NanoGrid), +// but the Python name is "Grid" to match the C++ class hierarchy where +// Grid (a.k.a. NanoGrid) is the user-facing type and +// GridData is the implementation-detail POD. +// +// Every BuildT-independent method lives here. The lambdas below read public +// data members on GridData directly because the accessor methods named +// version()/gridType()/isLevelSet()/etc. are defined on Grid, not on +// GridData itself — but they all just forward to a GridData data member, +// so the same value is reachable from the base. +void defineGrid(nb::module_& m) { - nb::class_(m, "GridData") + nb::class_(m, "Grid") + // Validation and flag mutators (already member functions on GridData). .def("isValid", &GridData::isValid) .def("setMinMaxOn", &GridData::setMinMaxOn, "on"_a = true) .def("setBBoxOn", &GridData::setBBoxOn, "on"_a = true) @@ -147,6 +164,7 @@ void defineGridData(nb::module_& m) .def("setAverageOn", &GridData::setAverageOn, "on"_a = true) .def("setStdDeviationOn", &GridData::setStdDeviationOn, "on"_a = true) .def("setGridName", &GridData::setGridName, "src"_a) + // Affine transforms (already member functions on GridData). .def("applyMap", nb::overload_cast(&GridData::template applyMap, nb::const_), "xyz"_a) .def("applyMap", nb::overload_cast(&GridData::template applyMap, nb::const_), "xyz"_a) .def("applyMapF", nb::overload_cast(&GridData::template applyMapF, nb::const_), "xyz"_a) @@ -167,50 +185,350 @@ void defineGridData(nb::module_& m) .def("applyIJT", nb::overload_cast(&GridData::template applyIJT, nb::const_), "xyz"_a) .def("applyIJTF", nb::overload_cast(&GridData::template applyIJTF, nb::const_), "xyz"_a) .def("applyIJTF", nb::overload_cast(&GridData::template applyIJTF, nb::const_), "xyz"_a) + // Strings, geometry, layout (already member functions on GridData). .def("gridName", &GridData::gridName) .def("memUsage", &GridData::memUsage) .def("worldBBox", &GridData::worldBBox) .def("indexBBox", &GridData::indexBBox) - .def("isEmpty", &GridData::isEmpty); + .def("isEmpty", &GridData::isEmpty) + // Lifted from Grid via direct data-member access. + .def("version", [](const GridData& g) { return g.mVersion; }) + .def("gridSize", [](const GridData& g) { return g.mGridSize; }) + .def("gridIndex", [](const GridData& g) { return g.mGridIndex; }) + .def("gridCount", [](const GridData& g) { return g.mGridCount; }) + .def("voxelSize", [](const GridData& g) -> const Vec3d& { return g.mVoxelSize; }, + nb::rv_policy::reference_internal) + .def("map", [](const GridData& g) -> const Map& { return g.mMap; }, + nb::rv_policy::reference_internal) + .def("gridType", [](const GridData& g) { return g.mGridType; }) + .def("gridClass", [](const GridData& g) { return g.mGridClass; }) + .def("checksum", [](const GridData& g) { return g.mChecksum; }) + .def("isLevelSet", [](const GridData& g) { return g.mGridClass == GridClass::LevelSet; }) + .def("isFogVolume", [](const GridData& g) { return g.mGridClass == GridClass::FogVolume; }) + .def("isStaggered", [](const GridData& g) { return g.mGridClass == GridClass::Staggered; }) + .def("isPointIndex", + [](const GridData& g) { return g.mGridClass == GridClass::PointIndex; }) + .def("isGridIndex", [](const GridData& g) { return g.mGridClass == GridClass::IndexGrid; }) + .def("isPointData", [](const GridData& g) { return g.mGridClass == GridClass::PointData; }) + .def("isMask", [](const GridData& g) { return g.mGridClass == GridClass::Topology; }) + .def("isUnknown", [](const GridData& g) { return g.mGridClass == GridClass::Unknown; }) + .def("hasMinMax", [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasMinMax); }) + .def("hasBBox", [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasBBox); }) + .def("hasLongGridName", + [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasLongGridName); }) + .def("hasAverage", + [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasAverage); }) + .def("hasStdDeviation", + [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasStdDeviation); }) + .def("isBreadthFirst", + [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::IsBreadthFirst); }) + .def("shortGridName", [](const GridData& g) { return std::string(g.mGridName); }) + // Blind data — exposes the sidecar channels that PointGrid and + // OnIndexGrid use to carry their actual values, colors, normals, IDs, + // etc. blindMetaData(n) returns the descriptor; getBlindData(n) + // returns a zero-copy NumPy view onto the underlying bytes typed by + // mDataType (Float -> float32 ndarray, Vec3f -> (N, 3) float32, etc.; + // unrecognized types fall back to a flat uint8 byte view). + .def("blindDataCount", [](const GridData& g) { return g.mBlindMetadataCount; }) + .def("blindMetaData", + [](const GridData& g, uint32_t n) -> const GridBlindMetaData* { + return n < g.mBlindMetadataCount ? g.blindMetaData(n) : nullptr; + }, + nb::rv_policy::reference_internal, "n"_a) + .def("findBlindData", [](const GridData& g, const std::string& name) -> int { + for (uint32_t i = 0; i < g.mBlindMetadataCount; ++i) { + const auto* meta = g.blindMetaData(i); + if (std::strncmp(meta->mName, name.c_str(), GridBlindMetaData::MaxNameSize) == 0) + return static_cast(i); + } + return -1; + }, "name"_a) + .def("findBlindDataForSemantic", [](const GridData& g, GridBlindDataSemantic sem) -> int { + for (uint32_t i = 0; i < g.mBlindMetadataCount; ++i) { + if (g.blindMetaData(i)->mSemantic == sem) + return static_cast(i); + } + return -1; + }, "semantic"_a) + .def("getBlindData", &pyGetBlindData, "n"_a, + "Return a zero-copy NumPy view of the n-th blind data channel, " + "or None if n is out of range. dtype and shape are derived from " + "the channel's mDataType / mValueCount."); } -template void defineGrid(nb::module_& m, const char* name) +// BuildT-dependent slice of the typed grid Python class. Inherits the +// type-erased Grid base bound by defineGrid() above — anything that doesn't +// need to know BuildT lives there, not here. +template void defineNanoGrid(nb::module_& m, const char* name) { nb::class_, GridData>(m, name) - .def("version", &NanoGrid::version) - .def("memUsage", &NanoGrid::memUsage) - .def("gridSize", &NanoGrid::gridSize) - .def("gridIndex", &NanoGrid::gridIndex) - .def("gridCount", &NanoGrid::gridCount) .def("getAccessor", &NanoGrid::getAccessor) - .def("voxelSize", &NanoGrid::voxelSize) - .def("map", &NanoGrid::map) - .def("worldBBox", &NanoGrid::worldBBox) - .def("indexBBox", &NanoGrid::indexBBox) .def("activeVoxelCount", &NanoGrid::activeVoxelCount) - .def("isValid", &NanoGrid::isValid) - .def("gridType", &NanoGrid::gridType) - .def("gridClass", &NanoGrid::gridClass) - .def("isLevelSet", &NanoGrid::isLevelSet) - .def("isFogVolume", &NanoGrid::isFogVolume) - .def("isStaggered", &NanoGrid::isStaggered) - .def("isPointIndex", &NanoGrid::isPointIndex) - .def("isGridIndex", &NanoGrid::isGridIndex) - .def("isPointData", &NanoGrid::isPointData) - .def("isMask", &NanoGrid::isMask) - .def("isUnknown", &NanoGrid::isUnknown) - .def("hasMinMax", &NanoGrid::hasMinMax) - .def("hasBBox", &NanoGrid::hasBBox) - .def("hasLongGridName", &NanoGrid::hasLongGridName) - .def("hasAverage", &NanoGrid::hasAverage) - .def("hasStdDeviation", &NanoGrid::hasStdDeviation) - .def("isBreadthFirst", &NanoGrid::isBreadthFirst) - // .def("isLexicographic", &NanoGrid::isLexicographic) - .def("isSequential", [](const NanoGrid& grid) { return grid.isSequential(); }) - .def("gridName", &NanoGrid::gridName) - .def("shortGridName", &NanoGrid::shortGridName) - .def("checksum", &NanoGrid::checksum) - .def("isEmpty", &NanoGrid::isEmpty); + .def("isSequential", [](const NanoGrid& grid) { return grid.isSequential(); }); +} + +void defineGridBlindData(nb::module_& m) +{ + nb::enum_(m, "GridBlindDataClass") + .value("Unknown", GridBlindDataClass::Unknown) + .value("IndexArray", GridBlindDataClass::IndexArray) + .value("AttributeArray", GridBlindDataClass::AttributeArray) + .value("GridName", GridBlindDataClass::GridName) + .value("ChannelArray", GridBlindDataClass::ChannelArray) + .value("End", GridBlindDataClass::End) + .export_values(); + + nb::enum_(m, "GridBlindDataSemantic") + .value("Unknown", GridBlindDataSemantic::Unknown) + .value("PointPosition", GridBlindDataSemantic::PointPosition) + .value("PointColor", GridBlindDataSemantic::PointColor) + .value("PointNormal", GridBlindDataSemantic::PointNormal) + .value("PointRadius", GridBlindDataSemantic::PointRadius) + .value("PointVelocity", GridBlindDataSemantic::PointVelocity) + .value("PointId", GridBlindDataSemantic::PointId) + .value("WorldCoords", GridBlindDataSemantic::WorldCoords) + .value("GridCoords", GridBlindDataSemantic::GridCoords) + .value("VoxelCoords", GridBlindDataSemantic::VoxelCoords) + .value("LevelSet", GridBlindDataSemantic::LevelSet) + .value("FogVolume", GridBlindDataSemantic::FogVolume) + .value("Staggered", GridBlindDataSemantic::Staggered) + .value("End", GridBlindDataSemantic::End) + .export_values(); + + nb::class_(m, "GridBlindMetaData", + "Sidecar metadata for one blind-data channel attached to a Grid.") + .def_ro("valueCount", &GridBlindMetaData::mValueCount) + .def_ro("valueSize", &GridBlindMetaData::mValueSize) + .def_ro("semantic", &GridBlindMetaData::mSemantic) + .def_ro("dataClass", &GridBlindMetaData::mDataClass) + .def_ro("dataType", &GridBlindMetaData::mDataType) + .def("name", [](const GridBlindMetaData& m) { return std::string(m.mName); }) + .def("isValid", &GridBlindMetaData::isValid) + .def("blindDataSize", &GridBlindMetaData::blindDataSize); +} + +// Resolve a blind-data channel into a zero-copy NumPy view. The dtype and +// shape are derived from the GridBlindMetaData's mDataType / mValueSize. +// For unrecognized types we fall back to a flat uint8 byte view so callers +// can still copy out the raw bytes. Falls back to None on a count mismatch +// between mValueSize and the GridType-implied stride. +static nb::object pyGetBlindData(nb::handle py_grid, uint32_t n) +{ + const auto& grid = nb::cast(py_grid); + if (n >= grid.mBlindMetadataCount) return nb::none(); + const auto* meta = grid.blindMetaData(n); + void* data = const_cast(static_cast( + util::PtrAdd(meta, meta->mDataOffset))); + const size_t count = static_cast(meta->mValueCount); + const uint32_t valueSize = meta->mValueSize; + + auto make1D = [&](void* p, size_t n_elems, auto sentinel) -> nb::object { + using T = decltype(sentinel); + size_t shape[1] = {n_elems}; + return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( + static_cast(p), 1, shape, py_grid), + nb::rv_policy::reference); + }; + auto make2D = [&](void* p, size_t n_outer, size_t n_inner, auto sentinel) -> nb::object { + using T = decltype(sentinel); + size_t shape[2] = {n_outer, n_inner}; + return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( + static_cast(p), 2, shape, py_grid), + nb::rv_policy::reference); + }; + // Raw byte view fallback. Used either when the data type is unknown OR + // when the recorded mValueSize doesn't match the stride implied by + // mDataType — in that case constructing a typed ndarray would overrun the + // underlying blind-data region. mValueCount * mValueSize is by definition + // the actual byte extent of the channel, so this is always safe. + auto raw = [&]() -> nb::object { + return make1D(data, count * valueSize, uint8_t{}); + }; + + switch (meta->mDataType) { + case GridType::Float: + return valueSize == sizeof(float) ? make1D(data, count, float{}) : raw(); + case GridType::Double: + return valueSize == sizeof(double) ? make1D(data, count, double{}) : raw(); + case GridType::Int16: + return valueSize == sizeof(int16_t) ? make1D(data, count, int16_t{}) : raw(); + case GridType::Int32: + return valueSize == sizeof(int32_t) ? make1D(data, count, int32_t{}) : raw(); + case GridType::Int64: + return valueSize == sizeof(int64_t) ? make1D(data, count, int64_t{}) : raw(); + case GridType::UInt8: + return valueSize == sizeof(uint8_t) ? make1D(data, count, uint8_t{}) : raw(); + case GridType::UInt32: + return valueSize == sizeof(uint32_t) ? make1D(data, count, uint32_t{}) : raw(); + case GridType::Vec3f: + return valueSize == 3 * sizeof(float) ? make2D(data, count, 3, float{}) : raw(); + case GridType::Vec3d: + return valueSize == 3 * sizeof(double) ? make2D(data, count, 3, double{}) : raw(); + case GridType::Vec4f: + return valueSize == 4 * sizeof(float) ? make2D(data, count, 4, float{}) : raw(); + case GridType::Vec4d: + return valueSize == 4 * sizeof(double) ? make2D(data, count, 4, double{}) : raw(); + case GridType::Vec3u8: + return valueSize == 3 * sizeof(uint8_t) ? make2D(data, count, 3, uint8_t{}) : raw(); + case GridType::Vec3u16: + return valueSize == 3 * sizeof(uint16_t) ? make2D(data, count, 3, uint16_t{}) : raw(); + case GridType::RGBA8: + return valueSize == 4 * sizeof(uint8_t) ? make2D(data, count, 4, uint8_t{}) : raw(); + default: + return raw(); + } +} + +// PointAccessor — exposes the per-voxel point attributes that PointGrid +// carries as blind data. PointIndex grids store uint32 voxel indices; +// PointData grids store Vec3f positions. Constructor asserts the grid is +// the right shape; in Python an exception is the result of a mismatch. +// +// gridPoints() / leafPoints(ijk) / voxelPoints(ijk) all return a zero-copy +// NumPy view onto the underlying blind-data buffer, sliced to just the +// range associated with the call. Lifetime is anchored to the accessor. +template +struct PyPointAccessorTraits; +template<> struct PyPointAccessorTraits { using Scalar = uint32_t; }; +template<> struct PyPointAccessorTraits { using Scalar = float; }; + +template +static nb::object pyPointsToNdarray(nb::handle py_self, + const AttT* begin, + uint64_t count); + +template<> +nb::object pyPointsToNdarray(nb::handle py_self, + const uint32_t* begin, + uint64_t count) +{ + size_t shape[1] = {static_cast(count)}; + return nb::cast( + nb::ndarray, nb::c_contig, nb::device::cpu>( + const_cast(begin), 1, shape, py_self), + nb::rv_policy::reference); +} + +template<> +nb::object pyPointsToNdarray(nb::handle py_self, + const Vec3f* begin, + uint64_t count) +{ + size_t shape[2] = {static_cast(count), 3}; + return nb::cast( + nb::ndarray, nb::c_contig, nb::device::cpu>( + reinterpret_cast(const_cast(begin)), 2, shape, py_self), + nb::rv_policy::reference); +} + +template void definePointAccessor(nb::module_& m, const char* name) +{ + using PA = PointAccessor; + nb::class_(m, name, + "Per-voxel access to the point attributes carried as blind " + "data on a PointGrid. gridPoints / leafPoints / voxelPoints " + "return zero-copy NumPy views.") + .def(nb::init&>(), "grid"_a, nb::keep_alive<1, 2>()) + .def("__bool__", [](const PA& a) { return bool(a); }) + .def("grid", &PA::grid, nb::rv_policy::reference_internal) + .def("gridPoints", [](nb::handle py_self) -> nb::object { + auto& acc = nb::cast(py_self); + const AttT* begin = nullptr; + const AttT* end = nullptr; + uint64_t count = acc.gridPoints(begin, end); + if (begin == nullptr || count == 0) return nb::none(); + return pyPointsToNdarray(py_self, begin, count); + }, "Return all point attributes in the grid as a single NumPy view.") + .def("leafPoints", [](nb::handle py_self, const Coord& ijk) -> nb::object { + auto& acc = nb::cast(py_self); + const AttT* begin = nullptr; + const AttT* end = nullptr; + uint64_t count = acc.leafPoints(ijk, begin, end); + if (begin == nullptr || count == 0) return nb::none(); + return pyPointsToNdarray(py_self, begin, count); + }, "ijk"_a, + "Return the point attributes contained within the leaf node " + "covering ijk, or None if no leaf is present.") + .def("voxelPoints", [](nb::handle py_self, const Coord& ijk) -> nb::object { + auto& acc = nb::cast(py_self); + const AttT* begin = nullptr; + const AttT* end = nullptr; + uint64_t count = acc.voxelPoints(ijk, begin, end); + if (begin == nullptr || count == 0) return nb::none(); + return pyPointsToNdarray(py_self, begin, count); + }, "ijk"_a, + "Return the point attributes at the specific voxel ijk, or None " + "if the voxel is inactive / empty."); +} + +// Type-erased grid introspector. Mirrors nanovdb::GridMetaData (768B) and +// answers "what's in this buffer?" questions without needing to know +// BuildT. Construct from a Grid (which is the Python-side GridData); all +// queries below are flat data-member reads with no tree traversal. +void defineGridMetaData(nb::module_& m) +{ + // Constructing GridMetaData calls into nanovdb::GridMetaData::safeCast + // which has a NANOVDB_ASSERT(gridData && gridData->isValid()). nanobind + // already rejects Python None at the type-check level (None can't bind to + // const GridData*), but a Grid wrapping a corrupted / partially-formed + // buffer would still abort debug builds and undefined-behave in release. + // Guard explicitly: validate first, raise nb::value_error on bad input. + nb::class_(m, "GridMetaData", + "Type-erased introspector. Mirrors FileMetaData " + "but reads from an in-memory grid header.") + .def("__init__", + [](GridMetaData* self, const GridData* gd) { + if (gd == nullptr) { + throw nb::value_error("GridMetaData: grid must not be None"); + } + if (!gd->isValid()) { + throw nb::value_error("GridMetaData: grid header is invalid " + "(bad magic, version, or class/type tags)"); + } + new (self) GridMetaData(gd); + }, "grid"_a) + .def_static("safeCast", + [](const GridData* gd) { + // Mirror the spirit of NanoVDB's static safeCast: "is + // it safe to cast this gridData to a GridMetaData?". + // null and invalid grids are by definition not safe; + // return False rather than dereference. + if (gd == nullptr || !gd->isValid()) return false; + return GridMetaData::safeCast(gd); + }, "grid"_a) + .def("isValid", &GridMetaData::isValid) + .def("gridType", &GridMetaData::gridType) + .def("gridClass", &GridMetaData::gridClass) + .def("isLevelSet", &GridMetaData::isLevelSet) + .def("isFogVolume", &GridMetaData::isFogVolume) + .def("isStaggered", &GridMetaData::isStaggered) + .def("isPointIndex", &GridMetaData::isPointIndex) + .def("isGridIndex", &GridMetaData::isGridIndex) + .def("isPointData", &GridMetaData::isPointData) + .def("isMask", &GridMetaData::isMask) + .def("isUnknown", &GridMetaData::isUnknown) + .def("hasMinMax", &GridMetaData::hasMinMax) + .def("hasBBox", &GridMetaData::hasBBox) + .def("hasLongGridName", &GridMetaData::hasLongGridName) + .def("hasAverage", &GridMetaData::hasAverage) + .def("hasStdDeviation", &GridMetaData::hasStdDeviation) + .def("isBreadthFirst", &GridMetaData::isBreadthFirst) + .def("gridSize", &GridMetaData::gridSize) + .def("gridIndex", &GridMetaData::gridIndex) + .def("gridCount", &GridMetaData::gridCount) + .def("shortGridName", [](const GridMetaData& m) { return std::string(m.shortGridName()); }) + .def("map", &GridMetaData::map, nb::rv_policy::reference_internal) + .def("worldBBox", &GridMetaData::worldBBox, nb::rv_policy::reference_internal) + .def("indexBBox", &GridMetaData::indexBBox, nb::rv_policy::reference_internal) + .def("voxelSize", &GridMetaData::voxelSize) + .def("blindDataCount", &GridMetaData::blindDataCount) + .def("activeVoxelCount", &GridMetaData::activeVoxelCount) + .def("activeTileCount", &GridMetaData::activeTileCount, "level"_a) + .def("nodeCount", &GridMetaData::nodeCount, "level"_a) + .def("checksum", &GridMetaData::checksum, nb::rv_policy::reference_internal) + .def("rootTableSize", &GridMetaData::rootTableSize) + .def("isEmpty", &GridMetaData::isEmpty) + .def("version", &GridMetaData::version); } template nb::class_> defineAccessor(nb::module_& m, const char* name) @@ -383,26 +701,38 @@ NB_MODULE(nanovdb, m) defineMap(m); - defineGridData(m); + // CheckMode + Checksum must be bound before defineGrid() because + // Grid.checksum() returns Checksum by value. + defineCheckMode(m); + defineChecksum(m); + + // GridBlindData enums + GridBlindMetaData class — must be bound before + // defineGrid() because Grid.blindMetaData()/findBlindDataForSemantic() + // use them in their signatures. + defineGridBlindData(m); + defineGrid(m); + defineGridMetaData(m); -#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) \ - defineGrid(m, #Suffix "Grid"); \ +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + defineNanoGrid(m, #Suffix "Grid"); \ defineScalarAccessor(m, #Suffix "ReadAccessor"); \ defineNodeInfo(m, #Suffix "NodeInfo"); -#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) \ - defineGrid(m, #Suffix "Grid"); \ +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + defineNanoGrid(m, #Suffix "Grid"); \ defineVectorAccessor(m, AccessorName); -#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix) \ - defineGrid(m, #Suffix "Grid"); \ +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + defineNanoGrid(m, #Suffix "Grid"); \ defineAccessor(m, #Suffix "ReadAccessor"); #include "BuildTypes.def" + // PointAccessor variants — PointIndex grids carry uint32 indices, + // PointData grids carry Vec3f positions. + definePointAccessor(m, "PointIndexAccessor"); + definePointAccessor(m, "PointDataAccessor"); + defineHostBuffer(m); defineHostGridHandle(m); - defineCheckMode(m); - defineChecksum(m); - #ifdef NANOVDB_USE_CUDA defineDeviceBuffer(m); defineDeviceGridHandle(m); diff --git a/nanovdb/nanovdb/python/PyGridHandle.cc b/nanovdb/nanovdb/python/PyGridHandle.cc index efd4253337..fdcdc8228d 100644 --- a/nanovdb/nanovdb/python/PyGridHandle.cc +++ b/nanovdb/nanovdb/python/PyGridHandle.cc @@ -22,6 +22,7 @@ void defineHostGridHandle(nb::module_& m) new (&handle) GridHandle(std::move(buffer)); }, "t"_a.noconvert()); + defineGridHandleUtilities(m); } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyGridHandle.h b/nanovdb/nanovdb/python/PyGridHandle.h index f23269d392..38af851f66 100644 --- a/nanovdb/nanovdb/python/PyGridHandle.h +++ b/nanovdb/nanovdb/python/PyGridHandle.h @@ -7,14 +7,115 @@ #include #include +#include +#include // for tools::updateGridCount + +#include +#include namespace nb = nanobind; namespace pynanovdb { +// Polymorphic host-side `handle.grid(n)`: dispatch on gridType(n) to the +// matching NanoGrid subclass currently bound in Python. Returns +// None when the underlying BuildT is not yet Python-visible (e.g. Boolean, +// Half, Fp16 — they land in Phase 2). The returned object is parented to +// the handle so the handle is kept alive at least as long as the grid. +template +inline nb::object pyHostGrid(nb::handle py_handle, uint32_t n) +{ + auto& handle = nb::cast&>(py_handle); + if (n >= handle.gridCount()) return nb::none(); + switch (handle.gridType(n)) { +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template grid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template grid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template grid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#include "BuildTypes.def" + default: + return nb::none(); + } +} + +// Free functions splitGrids / mergeGrids exposed at module scope. Templated +// on BufferT so the same definitions work for both the host GridHandle and +// the device GridHandle bindings. +template void defineGridHandleUtilities(nb::module_& m) +{ + using HandleT = nanovdb::GridHandle; + m.def("splitGrids", [](const HandleT& handle) { + auto handles = nanovdb::splitGrids(handle); + nb::list out; + for (auto& h : handles) { + out.append(std::move(h)); + } + return out; + }, nb::arg("handle"), + "Split a multi-grid handle into a list of single-grid handles, " + "each owning a freshly-allocated buffer."); + // mergeGrids: walk the Python sequence by CONST ref to each handle and + // concatenate buffer bytes into a freshly-allocated output. The original + // nanovdb::mergeGrids takes a `const std::vector&`, but + // building such a vector from a Python list requires moving from each + // wrapper (GridHandle is move-only) — which would silently empty the + // caller's `h1`/`h2` Python objects. Instead we inline the merge logic + // here so we never need to move from the inputs. + m.def("mergeGrids", [](nb::sequence handles) { + // Collect const refs so we touch each Python wrapper exactly once. + std::vector sources; + sources.reserve(nb::len(handles)); + for (nb::handle item : handles) { + sources.push_back(&nb::cast(item)); + } + + uint64_t totalSize = 0; + uint32_t totalGrids = 0; + for (const HandleT* h : sources) { + totalGrids += h->gridCount(); + for (uint32_t n = 0; n < h->gridCount(); ++n) { + totalSize += h->gridSize(n); + } + } + + auto buffer = BufferT::create(totalSize); + uint8_t* dst = static_cast(buffer.data()); + uint32_t writeIndex = 0; + for (const HandleT* h : sources) { + const uint8_t* src = static_cast(h->data()); + for (uint32_t n = 0; n < h->gridCount(); ++n) { + const uint64_t gs = h->gridSize(n); + std::memcpy(dst, src, gs); + auto* gd = reinterpret_cast(dst); + nanovdb::tools::updateGridCount(gd, writeIndex++, totalGrids); + dst += gs; + src += gs; + } + } + return HandleT(std::move(buffer)); + }, nb::arg("handles"), + "Combine a list of GridHandles into a single multi-grid GridHandle. " + "Input handles are read by const reference; the new handle owns a " + "freshly-allocated buffer and the inputs are left untouched."); +} + template nb::class_> defineGridHandle(nb::module_& m, const char* name) { - auto cls = nb::class_>(m, name) + return nb::class_>(m, name) .def(nb::init<>()) .def("reset", &nanovdb::GridHandle::reset) .def("size", &nanovdb::GridHandle::bufferSize) @@ -23,21 +124,15 @@ template nb::class_> defineGridHa .def( "__bool__", [](const nanovdb::GridHandle& handle) { return !handle.empty(); }, - nb::is_operator()); - -#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) \ - cls.def(HandleMethod, \ - nb::overload_cast(&nanovdb::GridHandle::template grid), \ - nb::arg("n") = 0, \ - nb::rv_policy::reference_internal); -#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) \ - cls.def(HandleMethod, \ - nb::overload_cast(&nanovdb::GridHandle::template grid), \ - nb::arg("n") = 0, \ - nb::rv_policy::reference_internal); -#include "BuildTypes.def" - - return cls + nb::is_operator()) + .def("copy", + [](const nanovdb::GridHandle& handle) { + return handle.template copy(); + }, + "Return a deep copy of this GridHandle backed by a freshly-allocated buffer.") + .def("grid", &pyHostGrid, nb::arg("n") = 0, + "Return the n-th grid as a typed Grid subclass selected by " + "gridType(n), or None if the BuildT is not bound in Python.") .def("isPadded", &nanovdb::GridHandle::isPadded) .def("gridCount", &nanovdb::GridHandle::gridCount) .def("gridSize", &nanovdb::GridHandle::gridSize, nb::arg("n") = 0) @@ -61,7 +156,6 @@ template nb::class_> defineGridHa [](nanovdb::GridHandle& handle, const std::string& fileName, const std::string& gridName) { handle.read(fileName, gridName); }, nb::arg("fileName"), nb::arg("gridName")); - } void defineHostGridHandle(nb::module_& m); diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index 390614358b..f6e4e87e2f 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -6,6 +6,7 @@ #include #include +#include namespace nb = nanobind; using namespace nb::literals; @@ -13,10 +14,46 @@ using namespace nanovdb; namespace pynanovdb { +// Device-side polymorphic deviceGrid(n) — same dispatch shape as +// pyHostGrid in PyGridHandle.h, but returns the device pointer. +// gridType(n) is read from the host-side GridData header (the handle keeps +// a host mirror), so this works whether or not the grid has been uploaded. +// Returns None if the device-side grid is null (i.e. no deviceUpload yet) +// or the BuildT is not Python-visible. +static nb::object pyDeviceGrid(nb::handle py_handle, uint32_t n) +{ + using BufferT = nanovdb::cuda::DeviceBuffer; + auto& handle = nb::cast&>(py_handle); + if (n >= handle.gridCount()) return nb::none(); + switch (handle.gridType(n)) { +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template deviceGrid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template deviceGrid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template deviceGrid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#include "../BuildTypes.def" + default: + return nb::none(); + } +} + void defineDeviceGridHandle(nb::module_& m) { using BufferT = nanovdb::cuda::DeviceBuffer; - auto cls = defineGridHandle(m, "DeviceGridHandle") + defineGridHandle(m, "DeviceGridHandle") .def( "__init__", [](GridHandle& handle, @@ -27,24 +64,24 @@ void defineDeviceGridHandle(nb::module_& m) new (&handle) GridHandle(std::move(buffer)); }, "cpu_t"_a.noconvert(), - "cuda_t"_a.noconvert()); - -#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, HandleMethod, DeviceMethod) \ - cls.def(DeviceMethod, \ - nb::overload_cast(&GridHandle::template deviceGrid), \ - "n"_a = 0, \ - nb::rv_policy::reference_internal); -#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, HandleMethod, DeviceMethod) \ - cls.def(DeviceMethod, \ - nb::overload_cast(&GridHandle::template deviceGrid), \ - "n"_a = 0, \ - nb::rv_policy::reference_internal); -#include "../BuildTypes.def" - - cls.def( + "cuda_t"_a.noconvert()) + .def("deviceGrid", &pyDeviceGrid, "n"_a = 0, + "Return the n-th device-resident grid as a typed Grid subclass " + "selected by gridType(n), or None if the BuildT is not bound in " + "Python or the device copy has not been uploaded yet.") + .def( "deviceUpload", [](GridHandle& handle, bool sync) { handle.deviceUpload(nullptr, sync); }, "sync"_a = true) .def( "deviceDownload", [](GridHandle& handle, bool sync) { handle.deviceDownload(nullptr, sync); }, "sync"_a = true); + // NOTE: defineGridHandleUtilities intentionally NOT called for + // DeviceBuffer. Registering nanovdb.splitGrids / nanovdb.mergeGrids as a + // second overload taking a DeviceGridHandle list conflicts with the host + // overload because both signatures take nb::list, and nanobind's + // overload resolution can't disambiguate by element type — it picks one + // and the inner cast fails with std::bad_cast. The host-only utilities + // are what the Phase 1 plan calls for; a properly typed device variant + // (with its own name, or with strongly-typed std::vector args + // and nanobind/stl/vector.h support) can land later if it's needed. } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index 75936a49eb..05ec516bf1 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -307,7 +307,7 @@ def test_float_grid(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Float) - grid = handle.floatGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) accessor = nanovdb.FloatReadAccessor(grid) coord = nanovdb.math.Coord(0) @@ -325,7 +325,7 @@ def test_checksum(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Float) - grid = handle.floatGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) checksum = grid.checksum() nanovdb.tools.updateChecksum(grid, nanovdb.CheckMode.Default) @@ -337,7 +337,7 @@ class TestGridHandleExchange(unittest.TestCase): def test_list_to_vector(self): handle = nanovdb.tools.createLevelSetTorus(nanovdb.GridType.Double) self.assertEqual(handle.gridCount(), 1) - self.assertIsNotNone(handle.doubleGrid()) + self.assertIsNotNone(handle.grid()) handles = [handle, handle] dstFile = tempfile.NamedTemporaryFile(delete=False) dstFile.close() @@ -347,6 +347,152 @@ def test_list_to_vector(self): os.unlink(dstFile.name) +class TestPolymorphicGridAccess(unittest.TestCase): + """Phase 1a: handle.grid(n) returns the correct typed Grid subclass.""" + + def test_float_grid(self): + h = nanovdb.tools.createFogVolumeSphere() + self.assertIsInstance(h.grid(), nanovdb.FloatGrid) + self.assertEqual(h.grid().gridType(), nanovdb.GridType.Float) + + def test_double_grid(self): + h = nanovdb.tools.createLevelSetTorus(nanovdb.GridType.Double) + self.assertIsInstance(h.grid(), nanovdb.DoubleGrid) + self.assertEqual(h.grid().gridType(), nanovdb.GridType.Double) + + def test_out_of_range_returns_none(self): + h = nanovdb.tools.createFogVolumeSphere() + self.assertIsNone(h.grid(99)) + + def test_empty_handle_returns_none(self): + self.assertIsNone(nanovdb.GridHandle().grid()) + + def test_typed_accessors_removed(self): + h = nanovdb.tools.createFogVolumeSphere() + # Phase 1a.3 removed these in favour of handle.grid(n). + self.assertFalse(hasattr(h, "floatGrid")) + self.assertFalse(hasattr(h, "doubleGrid")) + self.assertFalse(hasattr(h, "int32Grid")) + self.assertFalse(hasattr(h, "vec3fGrid")) + self.assertFalse(hasattr(h, "rgba8Grid")) + + +class TestGridBase(unittest.TestCase): + """Phase 1a.1: BuildT-independent methods resolve via the Grid base class.""" + + def test_grid_base_class_name(self): + # Typed grids inherit from a base class named "Grid" (no more "GridData"). + self.assertTrue(any(b.__name__ == "Grid" for b in nanovdb.FloatGrid.__bases__)) + self.assertFalse(hasattr(nanovdb, "GridData")) + + def test_lifted_methods_accessible_via_inheritance(self): + h = nanovdb.tools.createFogVolumeSphere(name="probe") + g = h.grid() + self.assertEqual(g.gridType(), nanovdb.GridType.Float) + self.assertEqual(g.gridClass(), nanovdb.GridClass.FogVolume) + self.assertTrue(g.isFogVolume()) + self.assertFalse(g.isLevelSet()) + self.assertEqual(g.gridName(), "probe") + self.assertEqual(g.shortGridName(), "probe") + self.assertGreater(g.gridSize(), 0) + self.assertEqual(g.gridCount(), 1) + + +class TestGridMetaData(unittest.TestCase): + """Phase 1b.1: type-erased GridMetaData introspector.""" + + def test_constructed_from_grid(self): + h = nanovdb.tools.createFogVolumeSphere(name="probe") + m = nanovdb.GridMetaData(h.grid()) + self.assertEqual(m.gridType(), nanovdb.GridType.Float) + self.assertEqual(m.gridClass(), nanovdb.GridClass.FogVolume) + self.assertEqual(m.shortGridName(), "probe") + self.assertTrue(m.isValid()) + self.assertTrue(m.isFogVolume()) + self.assertGreater(m.activeVoxelCount(), 0) + self.assertEqual(m.blindDataCount(), 0) + self.assertTrue(nanovdb.GridMetaData.safeCast(h.grid())) + + +class TestBlindDataEmpty(unittest.TestCase): + """Phase 1b.2: blind data API works on grids that have none.""" + + def test_no_blind_data(self): + h = nanovdb.tools.createFogVolumeSphere() + g = h.grid() + self.assertEqual(g.blindDataCount(), 0) + self.assertIsNone(g.blindMetaData(0)) + self.assertEqual(g.findBlindData("anything"), -1) + self.assertEqual( + g.findBlindDataForSemantic(nanovdb.GridBlindDataSemantic.PointPosition), + -1, + ) + self.assertIsNone(g.getBlindData(0)) + + +class TestSplitMergeCopy(unittest.TestCase): + """Phase 1c.2: splitGrids / mergeGrids / handle.copy().""" + + def test_split_and_merge_roundtrip(self): + h1 = nanovdb.tools.createFogVolumeSphere(name="a") + h2 = nanovdb.tools.createLevelSetTorus(nanovdb.GridType.Float, name="b") + merged = nanovdb.mergeGrids([h1, h2]) + self.assertEqual(merged.gridCount(), 2) + split = nanovdb.splitGrids(merged) + self.assertEqual(len(split), 2) + for s in split: + self.assertEqual(s.gridCount(), 1) + + def test_merge_does_not_consume_inputs(self): + # Regression: original Phase 1 mergeGrids used nb::cast + # which moved the underlying C++ handle out of the Python wrapper, + # silently emptying h1/h2. The fixed version reads each handle by + # const reference. + h1 = nanovdb.tools.createFogVolumeSphere(name="a") + h2 = nanovdb.tools.createLevelSetTorus(nanovdb.GridType.Float, name="b") + sz1, sz2 = h1.size(), h2.size() + gc1, gc2 = h1.gridCount(), h2.gridCount() + + merged = nanovdb.mergeGrids([h1, h2]) + + self.assertEqual(h1.gridCount(), gc1) + self.assertEqual(h2.gridCount(), gc2) + self.assertEqual(h1.size(), sz1) + self.assertEqual(h2.size(), sz2) + # merged still works + self.assertEqual(merged.gridCount(), 2) + self.assertEqual(merged.grid(0).gridName(), "a") + self.assertEqual(merged.grid(1).gridName(), "b") + + def test_copy_is_deep(self): + src = nanovdb.tools.createFogVolumeSphere(name="orig") + cp = src.copy() + self.assertIsNot(src, cp) + self.assertEqual(cp.gridCount(), src.gridCount()) + self.assertEqual(cp.grid().gridName(), "orig") + + +class TestGridMetaDataGuards(unittest.TestCase): + """Copilot review #3: GridMetaData ctor + safeCast guard against bad input.""" + + def test_init_rejects_none(self): + # nanobind's type system rejects None for const GridData* before our + # validity check runs. Either is acceptable as long as we don't + # crash / abort. + with self.assertRaises((TypeError, ValueError)): + nanovdb.GridMetaData(None) + + def test_safeCast_rejects_none(self): + with self.assertRaises((TypeError, ValueError)): + nanovdb.GridMetaData.safeCast(None) + + def test_valid_grid_still_works(self): + h = nanovdb.tools.createFogVolumeSphere() + m = nanovdb.GridMetaData(h.grid()) + self.assertTrue(m.isValid()) + self.assertTrue(nanovdb.GridMetaData.safeCast(h.grid())) + + class TestReadWriteGrids(unittest.TestCase): def setUp(self): self.gridName = "sphere_ls" @@ -391,7 +537,7 @@ def test_read_write_grid(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Float) - grid = handle.floatGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) self.assertTrue(grid.activeVoxelCount() > 0) self.assertTrue(grid.isSequential()) @@ -470,14 +616,14 @@ def test_read_write_grid(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Float) - grid = handle.floatGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) - deviceGrid = handle.deviceFloatGrid(i) + deviceGrid = handle.deviceGrid(i) self.assertIsNone(deviceGrid) handle.deviceUpload() - deviceGrid = handle.deviceFloatGrid(i) + deviceGrid = handle.deviceGrid(i) handle.deviceDownload() - grid = handle.floatGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) self.assertIsNotNone(deviceGrid) self.assertTrue(grid.activeVoxelCount() > 0) @@ -526,12 +672,12 @@ def test_points_to_grid(self): [[1, 2, 3]], dtype=torch.int32, device=torch.device("cuda", 0) ) handle = nanovdb.tools.cuda.pointsToRGBA8Grid(tensor) - deviceGrid = handle.deviceRGBA8Grid() + deviceGrid = handle.deviceGrid() self.assertTrue(deviceGrid) - grid = handle.rgba8Grid() + grid = handle.grid() self.assertFalse(grid) handle.deviceDownload() - grid = handle.rgba8Grid() + grid = handle.grid() self.assertTrue(grid) except ImportError: print("PyTorch not found. Skipping...") @@ -547,7 +693,7 @@ def test_points_to_grid(self): class TestSignedFloodFill(unittest.TestCase): def test_signed_flood_fill_float(self): handle = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Float, 100) - grid = handle.floatGrid() + grid = handle.grid() self.assertIsNotNone(grid) accessor = grid.getAccessor() self.assertFalse(accessor.isActive(nanovdb.math.Coord(103, 0, 0))) @@ -562,11 +708,11 @@ def test_signed_flood_fill_float(self): self.assertEqual(0.0, accessor(100, 0, 0)) self.assertEqual(1.0, accessor(97, 0, 0)) handle.deviceUpload() - deviceGrid = handle.deviceFloatGrid(0) + deviceGrid = handle.deviceGrid(0) self.assertIsNotNone(deviceGrid) nanovdb.tools.cuda.signedFloodFill(deviceGrid) handle.deviceDownload() - grid = handle.floatGrid() + grid = handle.grid() self.assertIsNotNone(grid) accessor = grid.getAccessor() self.assertEqual(3.0, accessor(103, 0, 0)) @@ -577,7 +723,7 @@ def test_signed_flood_fill_float(self): def test_signed_flood_fill_double(self): handle = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Double, 100) - grid = handle.doubleGrid() + grid = handle.grid() self.assertIsNotNone(grid) accessor = grid.getAccessor() self.assertFalse(accessor.isActive(nanovdb.math.Coord(103, 0, 0))) @@ -592,11 +738,11 @@ def test_signed_flood_fill_double(self): self.assertEqual(0.0, accessor(100, 0, 0)) self.assertEqual(1.0, accessor(97, 0, 0)) handle.deviceUpload() - deviceGrid = handle.deviceDoubleGrid(0) + deviceGrid = handle.deviceGrid(0) self.assertIsNotNone(deviceGrid) nanovdb.tools.cuda.signedFloodFill(deviceGrid) handle.deviceDownload() - grid = handle.doubleGrid() + grid = handle.grid() self.assertIsNotNone(grid) accessor = grid.getAccessor() self.assertEqual(3.0, accessor(103, 0, 0)) @@ -629,7 +775,7 @@ def test_sample_from_points_float(self): voxelSize=voxelSize, ) handle.deviceUpload() - grid = handle.deviceFloatGrid() + grid = handle.deviceGrid() self.assertIsNotNone(grid) points = torch.tensor( @@ -690,7 +836,7 @@ def test_sample_from_points_double(self): voxelSize=voxelSize, ) handle.deviceUpload() - grid = handle.deviceDoubleGrid() + grid = handle.deviceGrid() self.assertIsNotNone(grid) points = torch.tensor( @@ -733,7 +879,7 @@ def test_float_sampler(self): halfWidth=halfWidth, voxelSize=voxelSize, ) - grid = handle.floatGrid() + grid = handle.grid() xform = grid.map() index_space_pos = xform.applyInverseMap(world_space_pos) sampler = nanovdb.math.createNearestNeighborSampler(grid) @@ -761,7 +907,7 @@ def test_double_sampler(self): halfWidth=halfWidth, voxelSize=voxelSize, ) - grid = handle.doubleGrid() + grid = handle.grid() xform = grid.map() index_space_pos = xform.applyInverseMap(world_space_pos) sampler = nanovdb.math.createNearestNeighborSampler(grid) @@ -789,7 +935,7 @@ def test_create_float_nano_grid(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Float) - grid = handle.floatGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) self.assertTrue(grid.activeVoxelCount() > 0) self.assertTrue(grid.isSequential()) @@ -807,7 +953,7 @@ def test_create_double_nano_grid(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Double) - grid = handle.doubleGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) self.assertTrue(grid.activeVoxelCount() > 0) self.assertTrue(grid.isSequential()) @@ -825,7 +971,7 @@ def test_create_int_nano_grid(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Int32) - grid = handle.int32Grid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) self.assertTrue(grid.activeVoxelCount() > 0) self.assertTrue(grid.isSequential()) @@ -847,7 +993,7 @@ def test_create_vec3f_nano_grid(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Vec3f) - grid = handle.vec3fGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) self.assertTrue(grid.activeVoxelCount() > 0) self.assertTrue(grid.isSequential()) @@ -881,7 +1027,7 @@ def test_function(self): for i in range(handle.gridCount()): self.assertTrue(handle.gridSize(i) > 0) self.assertEqual(handle.gridType(i), nanovdb.GridType.Float) - grid = handle.floatGrid(i) + grid = handle.grid(i) self.assertIsNotNone(grid) self.assertTrue(grid.activeVoxelCount() > 0) self.assertTrue(grid.isSequential()) From efa513a56943ee4efe06548878036741f27a120c Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 21 May 2026 10:07:46 +1200 Subject: [PATCH 03/48] =?UTF-8?q?nanovdb=20python:=20Phase=202=20=E2=80=94?= =?UTF-8?q?=20broaden=20BuildT=20coverage=20to=2024=20grid=20types=20(#221?= =?UTF-8?q?1)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: Phase 2 — broaden BuildT coverage to 24 grid types Third slice of the Python bindings restructure tracked in #2208 and laid out in nanovdb-python-plan.md. Phase 2 expands the bound BuildT list from the six surfaced in Phase 0/1 (float, double, int32_t, Vec3f, Rgba8, Point) to seventeen new types, riding the Phase 0 X-macro so each addition is a single row in BuildTypes.def plus a polymorphic dispatch arm. New BuildTs by category: - SCALAR (+4): int16_t, int64_t, uint8_t, uint32_t. Bound exactly like the existing float/double/int32_t — full defineScalarAccessor (with setVoxel) plus defineNodeInfo. Class names follow the GridType enum: Int16Grid, Int64Grid, UInt8Grid, UInt32Grid. - VECTOR (+5): Vec3d, Vec4f, Vec4d, Vec3u8, Vec3u16. Same shape as the existing Vec3f / Rgba8 — defineVectorAccessor with setVoxel, no NodeInfo. Accessor names use the consistent "ReadAccessor" form (Vec3dReadAccessor, ...); only the legacy Vec3fReadVectorAccessor / RGBA8ReadAccessor names from Phase 0 stay as-is. - READONLY (new category, +8): bool, Fp4, Fp8, Fp16, FpN, ValueIndex, ValueOnIndex, ValueMask. These all have nanovdb::BuildTraits::is_special == true, which means the C++ SetVoxel specialization static_asserts and won't compile. They get a bare defineAccessor binding — getValue() only, no setVoxel, no NodeInfo. Class names: BooleanGrid, Fp4Grid / Fp8Grid / Fp16Grid / FpNGrid (quantized — getValue returns float), IndexGrid / OnIndexGrid (getValue returns uint64), and MaskGrid (getValue returns bool). The accessor's value type now resolves through nanovdb::BuildToValueMap::Type rather than DefaultReadAccessor::ValueType. For ordinary types they're identical, but for Half / Fp* the accessor decodes to float on read, for ValueIndex / OnIndex it returns uint64, and for ValueMask / bool it returns bool — the bound probeValue() out-parameter and the Python-side return type both want the decoded form. (Without this change, probeValue's instantiation chain mismatches its own ProbeValue::ValueT = float.) nanovdb::Half is intentionally NOT bound. The source declares it as `class Half{};` (an empty placeholder for IEEE 754 half-precision) and the C++ ProbeValue chain is inconsistent — leaf storage carries Half but ProbeValue expects float, so the template doesn't instantiate. When the upstream Half implementation lands we can add it via the same X-macro path. Polymorphic dispatch in pyHostGrid / pyDeviceGrid gains a fourth arm (NANOVDB_PY_FOR_EACH_READONLY_BUILDT) covering all eight new GridType enumerators (Boolean, Fp4/Fp8/Fp16/FpN, Index, OnIndex, Mask). handle.grid(n) / handle.deviceGrid(n) now return the right typed subclass for these too. Tests: a single new TestPhase2BuildTCoverage class with 5 methods verifies every new BuildT registered, every accessor surface matches its category (scalars have setVoxel + getNodeInfo; vectors have setVoxel; read-only have neither), and all the new typed grids inherit from the polymorphic Grid base. We can't host-construct Int16Grid / Fp4Grid / IndexGrid / etc. yet because the C++ create*Grid factories for those types land in Phase 5 — but the registration and the dispatch surface are locked in. Build + test verified locally: - CPU (no CUDA, no OpenVDB, no BLOSC): 57 tests, 48 pass, 8 CUDA skip, 1 pre-existing test_read_write_grid BLOSC failure unrelated to this PR. - CUDA sm_120 (Blackwell, CUDA 13.2): 57 tests, 55 pass, 2 pre- existing BLOSC failures (host + device variants). No new lines over 100 cols. Signed-off-by: Jonathan Swartz * nanovdb python: address Copilot review on #2211 Three concrete fixes from Copilot's review of the Phase 2 PR (https://github.com/AcademySoftwareFoundation/openvdb/pull/2211): 1) Bind GridType.UInt8 in the Python enum. Real bug — a Phase 0 oversight that was harmless until Phase 2 surfaced UInt8Grid. The C++ enumerator nanovdb::GridType::UInt8 existed (value 26), and the polymorphic dispatch routes it correctly internally, but the enum binding was missing the `.value("UInt8", GridType::UInt8)` line. Python users therefore couldn't write `handle.gridType(n) == nanovdb.GridType.UInt8` to discriminate UInt8 grids. New regression test TestPhase2BuildTCoverage. test_all_grid_type_enums_reachable walks every BuildT we bind and confirms its GridType enumerator is reachable from Python, so the next oversight gets caught at test time. 2) Update pyHostGrid() docstring in PyGridHandle.h. Was claiming "Boolean, Half, Fp16 land in Phase 2" as examples of types that weren't yet Python-visible — those examples are now stale (Phase 2 binds Boolean and Fp16; Half stays unbound but for a different reason). Reworded to point at BuildTypes.def as the source of truth so the docstring can't go stale again as Phase 5+ adds more types. 3) Disambiguate "bool for ValueMask and bool" wording in the READONLY macro doc comment in BuildTypes.def. The second "bool" referred to the literal BuildT=bool grid; the phrasing read as redundant. Now: "bool for ValueMask and for the BuildT=bool (Boolean) grid". Same fix also enumerates the quantized types explicitly (Fp4/Fp8/Fp16/FpN) instead of writing "Fp*". Build + test on CUDA sm_120 with BLOSC + ZLIB: 58/58 pass. Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/BuildTypes.def | 66 ++++++++++++++++--- nanovdb/nanovdb/python/NanoVDBModule.cc | 12 +++- nanovdb/nanovdb/python/PyGridHandle.h | 12 +++- .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 6 ++ nanovdb/nanovdb/python/test/TestNanoVDB.py | 64 ++++++++++++++++++ 5 files changed, 147 insertions(+), 13 deletions(-) diff --git a/nanovdb/nanovdb/python/BuildTypes.def b/nanovdb/nanovdb/python/BuildTypes.def index 5edae74ff7..bed2f199b3 100644 --- a/nanovdb/nanovdb/python/BuildTypes.def +++ b/nanovdb/nanovdb/python/BuildTypes.def @@ -19,16 +19,17 @@ // // Macros: // NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) -// Scalar value types — exposed with full NodeInfo accessors. Suffix -// forms Python class names (e.g. "Float" -> "FloatGrid"). -// GridTypeEnum names the nanovdb::GridType:: enumerator a grid of -// this BuildT carries (used by the polymorphic handle.grid(n) -// dispatch). Usually identical to Suffix. +// Scalar value types with arithmetic semantics — exposed with full +// NodeInfo + setVoxel accessors. Suffix forms Python class names +// (e.g. "Float" -> "FloatGrid"). GridTypeEnum names the +// nanovdb::GridType:: enumerator a grid of this BuildT carries +// (used by the polymorphic handle.grid(n) dispatch). // // NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) // Vector value types — exposed with a setVoxel accessor but no // NodeInfo. AccessorName is passed explicitly because the legacy -// Python class names for these are inconsistent. +// Python class names for the original two (Vec3f, RGBA8) are +// inconsistent — kept as-is for backwards compatibility. // // NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) // The nanovdb::Point build type — exposed with a bare accessor. @@ -37,6 +38,16 @@ // explicitly. Polymorphic dispatch routes PointIndex grids to // NanoGrid. // +// NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) +// BuildTs whose nanovdb::SetVoxel is unavailable (bool, the +// quantized Fp4/Fp8/Fp16/FpN types, the index types ValueIndex / +// ValueOnIndex, and ValueMask). nanovdb::BuildTraits::is_special +// is true for all of these. Exposed with a bare accessor only — no +// setVoxel, no NodeInfo. The Python accessor's getValue() returns +// the type given by nanovdb::BuildToValueMap::Type — float for +// Fp4 / Fp8 / Fp16 / FpN, uint64 for ValueIndex / ValueOnIndex, +// bool for ValueMask and for the BuildT=bool (Boolean) grid. +// // NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) // Subset that has C++ sampler specializations (used by PyMath // samplers and PyCreateNanoGrid create*Grid factories). @@ -53,22 +64,55 @@ #define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) #define NANOVDB_PY_LOCAL_DEFINED_POINT #endif +#ifndef NANOVDB_PY_FOR_EACH_READONLY_BUILDT +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) +#define NANOVDB_PY_LOCAL_DEFINED_READONLY +#endif #ifndef NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT #define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) #define NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE #endif -NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(float, Float, Float) -NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(double, Double, Double) -NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int32_t, Int32, Int32) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(float, Float, Float) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(double, Double, Double) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int16_t, Int16, Int16) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int32_t, Int32, Int32) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int64_t, Int64, Int64) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(uint8_t, UInt8, UInt8) +NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(uint32_t, UInt32, UInt32) +// nanovdb::Half is intentionally not bound. The source declares it as +// `class Half{};` (an empty placeholder for IEEE 754 half-precision, see +// NanoVDB.h around line 180) and the C++ ProbeValue chain is +// inconsistent — leaf storage carries Half but the ProbeValue specialization +// expects float, causing a type mismatch during instantiation. When the +// upstream Half implementation lands this can be added in a follow-up. NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3f, Vec3f, "Vec3fReadVectorAccessor", Vec3f) +NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3d, Vec3d, + "Vec3dReadAccessor", Vec3d) +NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec4f, Vec4f, + "Vec4fReadAccessor", Vec4f) +NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec4d, Vec4d, + "Vec4dReadAccessor", Vec4d) +NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3u8, Vec3u8, + "Vec3u8ReadAccessor", Vec3u8) +NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3u16, Vec3u16, + "Vec3u16ReadAccessor", Vec3u16) NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::math::Rgba8, RGBA8, "RGBA8ReadAccessor", RGBA8) NANOVDB_PY_FOR_EACH_POINT_BUILDT(::nanovdb::Point, Point, PointIndex) +NANOVDB_PY_FOR_EACH_READONLY_BUILDT(bool, Boolean, Boolean) +NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::Fp4, Fp4, Fp4) +NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::Fp8, Fp8, Fp8) +NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::Fp16, Fp16, Fp16) +NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::FpN, FpN, FpN) +NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::ValueIndex, Index, Index) +NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::ValueOnIndex, OnIndex, OnIndex) +NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::ValueMask, Mask, Mask) + NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(float, Float) NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(double, Double) NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(int32_t, Int32) @@ -83,6 +127,9 @@ NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(::nanovdb::Vec3f, Vec3f) #ifdef NANOVDB_PY_LOCAL_DEFINED_POINT #undef NANOVDB_PY_LOCAL_DEFINED_POINT #endif +#ifdef NANOVDB_PY_LOCAL_DEFINED_READONLY +#undef NANOVDB_PY_LOCAL_DEFINED_READONLY +#endif #ifdef NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE #undef NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE #endif @@ -90,4 +137,5 @@ NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(::nanovdb::Vec3f, Vec3f) #undef NANOVDB_PY_FOR_EACH_SCALAR_BUILDT #undef NANOVDB_PY_FOR_EACH_VECTOR_BUILDT #undef NANOVDB_PY_FOR_EACH_POINT_BUILDT +#undef NANOVDB_PY_FOR_EACH_READONLY_BUILDT #undef NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 80bf4586dc..a90e4b83bb 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -533,7 +533,13 @@ void defineGridMetaData(nb::module_& m) template nb::class_> defineAccessor(nb::module_& m, const char* name) { - using ValueType = typename DefaultReadAccessor::ValueType; + // Use the decoded value type (nanovdb::BuildToValueMap::Type) + // rather than DefaultReadAccessor::ValueType. For ordinary types + // (float/double/Int*/Vec*) the two are identical, but for Half / Fp* the + // accessor decodes to float on read, for ValueIndex/OnIndex it returns + // uint64, and for ValueMask / bool it returns bool. The probeValue out- + // parameter and the Python return type both want the decoded form. + using ValueType = typename nanovdb::BuildToValueMap::Type; using CoordType = typename DefaultReadAccessor::CoordType; nb::class_> accessor(m, name); @@ -660,6 +666,7 @@ NB_MODULE(nanovdb, m) .value("PointIndex", GridType::PointIndex) .value("Vec3u8", GridType::Vec3u8) .value("Vec3u16", GridType::Vec3u16) + .value("UInt8", GridType::UInt8) .value("End", GridType::End) .export_values() .def("__repr__", [](const GridType& gridType) { @@ -723,6 +730,9 @@ NB_MODULE(nanovdb, m) #define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ defineNanoGrid(m, #Suffix "Grid"); \ defineAccessor(m, #Suffix "ReadAccessor"); +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + defineNanoGrid(m, #Suffix "Grid"); \ + defineAccessor(m, #Suffix "ReadAccessor"); #include "BuildTypes.def" // PointAccessor variants — PointIndex grids carry uint32 indices, diff --git a/nanovdb/nanovdb/python/PyGridHandle.h b/nanovdb/nanovdb/python/PyGridHandle.h index 38af851f66..8f3f464469 100644 --- a/nanovdb/nanovdb/python/PyGridHandle.h +++ b/nanovdb/nanovdb/python/PyGridHandle.h @@ -19,9 +19,9 @@ namespace pynanovdb { // Polymorphic host-side `handle.grid(n)`: dispatch on gridType(n) to the // matching NanoGrid subclass currently bound in Python. Returns -// None when the underlying BuildT is not yet Python-visible (e.g. Boolean, -// Half, Fp16 — they land in Phase 2). The returned object is parented to -// the handle so the handle is kept alive at least as long as the grid. +// None when the underlying BuildT is not bound in this build — see the +// list of bound types in BuildTypes.def. The returned object is parented +// to the handle so the handle is kept alive at least as long as the grid. template inline nb::object pyHostGrid(nb::handle py_handle, uint32_t n) { @@ -46,6 +46,12 @@ inline nb::object pyHostGrid(nb::handle py_handle, uint32_t n) return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ : nb::none(); \ } +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template grid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } #include "BuildTypes.def" default: return nb::none(); diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index f6e4e87e2f..cbec582c91 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -44,6 +44,12 @@ static nb::object pyDeviceGrid(nb::handle py_handle, uint32_t n) return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ : nb::none(); \ } +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template deviceGrid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } #include "../BuildTypes.def" default: return nb::none(); diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index 05ec516bf1..a87bae3ba2 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -472,6 +472,70 @@ def test_copy_is_deep(self): self.assertEqual(cp.grid().gridName(), "orig") +class TestPhase2BuildTCoverage(unittest.TestCase): + """Phase 2: every additional BuildT registers a Grid + ReadAccessor (and + NodeInfo where applicable). We can't host-construct most of these (the + primitives that produce them land in Phase 5), but we can confirm + registration completed and the accessor surfaces match the type kind. + """ + + SCALARS = ["Int16", "Int64", "UInt8", "UInt32"] + VECTORS = ["Vec3d", "Vec4f", "Vec4d", "Vec3u8", "Vec3u16"] + READONLY = ["Boolean", "Fp4", "Fp8", "Fp16", "FpN", "Index", "OnIndex", "Mask"] + + def test_all_grid_classes_registered(self): + for suffix in self.SCALARS + self.VECTORS + self.READONLY: + cls = getattr(nanovdb, suffix + "Grid", None) + self.assertIsNotNone(cls, f"{suffix}Grid missing") + # All inherit from the type-erased Grid base. + self.assertIn(nanovdb.Grid, cls.__mro__) + + def test_all_accessors_registered(self): + for suffix in self.SCALARS + self.VECTORS + self.READONLY: + acc = getattr(nanovdb, suffix + "ReadAccessor", None) + self.assertIsNotNone(acc, f"{suffix}ReadAccessor missing") + + def test_scalar_accessors_have_setvoxel_and_nodeinfo(self): + for suffix in self.SCALARS: + acc = getattr(nanovdb, suffix + "ReadAccessor") + self.assertTrue(hasattr(acc, "setVoxel"), + f"{suffix}ReadAccessor missing setVoxel") + self.assertTrue(hasattr(acc, "getNodeInfo"), + f"{suffix}ReadAccessor missing getNodeInfo") + self.assertIsNotNone(getattr(nanovdb, suffix + "NodeInfo", None), + f"{suffix}NodeInfo missing") + + def test_vector_accessors_have_setvoxel_no_nodeinfo(self): + # Vector accessor names are aligned in Phase 2 (Vec3dReadAccessor, + # ...). The legacy Vec3f one is still Vec3fReadVectorAccessor. + for suffix in self.VECTORS: + acc = getattr(nanovdb, suffix + "ReadAccessor") + self.assertTrue(hasattr(acc, "setVoxel")) + self.assertFalse(hasattr(acc, "getNodeInfo")) + + def test_all_grid_type_enums_reachable(self): + # GridType enum binding must cover every BuildT we register — + # otherwise Python users can't compare against handle.gridType(n). + # UInt8 was missed in Phase 0; this test locks the fix in. + for name in ["Float", "Double", "Int16", "Int32", "Int64", "UInt8", + "UInt32", "Boolean", "Half", "RGBA8", "Vec3f", "Vec3d", + "Vec4f", "Vec4d", "Vec3u8", "Vec3u16", "Mask", "Fp4", + "Fp8", "Fp16", "FpN", "Index", "OnIndex", "PointIndex"]: + self.assertTrue(hasattr(nanovdb.GridType, name), + f"nanovdb.GridType.{name} not bound") + + def test_readonly_accessors_have_neither_setvoxel_nor_nodeinfo(self): + # The plan calls out: quantized types decode to float on read but + # do not bind setVoxel; index types return uint64 and ValueMask + # exposes only active-state queries. + for suffix in self.READONLY: + acc = getattr(nanovdb, suffix + "ReadAccessor") + self.assertFalse(hasattr(acc, "setVoxel"), + f"{suffix}ReadAccessor should not have setVoxel") + self.assertFalse(hasattr(acc, "getNodeInfo"), + f"{suffix}ReadAccessor should not have getNodeInfo") + + class TestGridMetaDataGuards(unittest.TestCase): """Copilot review #3: GridMetaData ctor + safeCast guard against bad input.""" From 2dcabf97dd6b5160342a42c5df46fdc1d8ab4ea7 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 21 May 2026 11:48:54 +1200 Subject: [PATCH 04/48] =?UTF-8?q?nanovdb=20python:=20Phase=203=20=E2=80=94?= =?UTF-8?q?=20tree=20/=20nodes=20/=20NodeManager=20/=20leaf=5Fvalues=20(#2?= =?UTF-8?q?212)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: Phase 3 — tree / nodes / NodeManager / leaf_values Fourth slice of the Python bindings restructure tracked in #2208 and laid out in nanovdb-python-plan.md. Phase 3 makes the tree itself walkable from Python: every BuildT now has a bound NanoTree, Root, Upper / Lower internal node, Leaf, and a host-side NodeManager. Where the leaf actually carries a contiguous T mValues[512] (regular scalar BuildTs) we also expose zero-copy NumPy views, including a high-level grid.leaf_values() bulk extractor. Per BuildT we now register six new classes: - Leaf — origin, bbox, dim, voxelCount, memUsage, flags, isActive(ijk|n), getValue(offset|ijk), getFirstValue, getLastValue, minimum / maximum / average / stdDeviation, valueMask, probeValue, and (for arithmetic non-special ValueTs) values() returning a zero-copy (512,) NumPy view of the leaf's mValues array. - Upper, Lower — origin, bbox, dim, memUsage, minimum / maximum / average / stdDeviation, valueMask, childMask, getValue, getFirstValue, getLastValue, isActive, probeValue. The two internal node levels share a single defineInternalNodeBase helper since their C++ APIs are identical. - Root — background, tileCount, getTableSize, isEmpty, bbox, minimum / maximum / average / stdDeviation, memUsage, getValue, isActive, probeValue. - Tree — root, background, activeVoxelCount, activeTileCount(level), nodeCount(level), totalNodeCount, memUsage, getValue, isActive, probeValue, extrema() (returns (min, max) tuple), getFirstLeaf / getFirstLower / getFirstUpper. Grid.tree() is bound on NanoGrid and returns the typed tree as reference_internal. - NodeManager — isLinear, memUsage, nodeCount(level), leafCount, lowerCount, upperCount, leaf(i), lower(i), upper(i) returning typed node refs. Constructed via the module-scope nanovdb.createNodeManager(grid) which polymorphically picks the right BuildT and returns a NodeManagerHandle. Handle exposes size(), __bool__(), and mgr() — which itself dispatches by stored gridType to return the right typed NodeManager. - grid.leaf_values() bulk extractor — for arithmetic non-special BuildTs (float, double, Int16/32/64, UInt8/UInt32), walks the breadth-first leaf array and returns a strided zero-copy (N_leaves, 512) NumPy view. Stride between leaves is sizeof(LeafT) / sizeof(ValueT), reflecting the leaf header between value blocks. Throws ValueError on non-breadth-first grids (createNanoGrid produces breadth-first by default). Mechanical bits: - All six bindings driven from BuildTypes.def via the existing X-macro. New file PyTree.h holds the templated definitions; PyTree.cc holds the non-templated NodeManagerHandle + createNodeManager bindings (the latter dispatches over every BuildT via the same X-macro). - Tree / node classes are registered BEFORE defineNanoGrid because NanoGrid.tree() returns NanoTree& and nanobind needs the return type registered first. Skipped / deferred: - LeafT::variance() is NOT bound. NanoVDB.h line 4388 reads `Pow2(DataType::getDev())` unqualified, which fails ADL for non-float ValueTs (ValueIndex / ValueMask). Same for InternalNode. Users can compute variance from stdDeviation() in Python. - VoxelBlockManager (OnIndexGrid-specific) deferred to a follow-up; surface is sizeable enough to merit its own PR. - Vector leaf values() (Vec3f / Vec3d / Vec4f / Vec4d / Vec3u8 / Vec3u16 / Rgba8) deferred — these need a flattened (count, dim) component view since nanobind ndarray isn't well-formed. A future PR can add float[N, 512, 3] views. - Tree iterators (beginValueOn etc.) deferred — the bulk leaf_values view + per-leaf valueMask covers the most common use case (mask the bulk array and you have your active values). Verified locally (BLOSC + ZLIB on so the optional-codec tests pass): - CPU build: 58 tests + 8 new TestPhase3TreeNodes — **all 66 pass**. - CUDA sm_120 build (RTX PRO 6000 Blackwell, CUDA 13.2): **all 66 pass**. - numpy-backed shape/dtype assertions in the new tests confirm the per-leaf values() (512,) float32 and bulk leaf_values() (N_leaves, 512) float32 views. NodeManager.leaf(i).values() is byte-identical to tree.getFirstLeaf().values() for i==0. No new lines over 100 cols. Signed-off-by: Jonathan Swartz * nanovdb python: drop phase numbers + reviewer markers from tests and comments User-facing strings in tests/comments that reference in-flight project history (phase numbers, reviewer names) lose their meaning once the project merges. Drop them and reword the surrounding text so each name and comment self-describes the feature being tested. - Renamed TestPhase2BuildTCoverage -> TestBuildTRegistrations. - Renamed TestPhase3TreeNodes -> TestTreeNodeWalking. - Reworded docstrings on TestPolymorphicGridAccess, TestGridBase, TestGridMetaData, TestBlindDataEmpty, TestSplitMergeCopy, TestGridMetaDataGuards to describe the feature instead of the phase. - Inline comments referencing "Phase 0", "Phase 1a.3", "Copilot review", etc. reworded into prose about the actual behavior being asserted or the underlying bug being regression-tested. - Same cleanup on the cuda/PyDeviceGridHandle.cu note about why we don't register splitGrids/mergeGrids on DeviceBuffer. No behavior change. 66/66 tests still pass with BLOSC+ZLIB on. Signed-off-by: Jonathan Swartz * nanovdb python: fix Clang Tree.nodeCount overload + drop exception-driven createNodeManager dispatch Two real issues caught in CI / review on #2212: 1) Tree.nodeCount(int) binding doesn't compile on Clang. PyTree.h used `nb::overload_cast(&TreeT::nodeCount, nb::const_)` to select the non-templated `uint32_t nodeCount(int) const` overload on nanovdb::Tree. Tree also has a templated `template uint32_t nodeCount() const` overload. GCC accepted the overload_cast under SFINAE rules; Clang (used by the linux-nanovdb:cxx:clang++-Debug CI leg) rejects it with `no matching function for call to object of type 'const detail::overload_cast_impl'`, repeated once per BuildT instantiation. Replaced with a direct static_cast to the function pointer type, which is unambiguous to both compilers: static_cast(&TreeT::nodeCount) 2) createNodeManager dispatch was exception-driven. The previous binding tried nb::cast&>(py_grid) for every BuildT and caught nb::cast_error on each mismatch — so a single call to nanovdb.createNodeManager(grid) threw and caught 22 cast_error exceptions before landing on the matching BuildT. Replaced with an nb::isinstance(py_grid) pre-check so the cast is only ever attempted on the matching BuildT. Both linux-nanovdb:cxx:clang++-Debug should now build, and createNodeManager() no longer pays per-call exception overhead. Verified locally on CPU and CUDA sm_120 builds: 66/66 tests pass with BLOSC + ZLIB on. Signed-off-by: Jonathan Swartz * nanovdb python: bounds-check + lifetime fixes on #2212 Addresses Copilot review notes on PR #2212. Two real classes of bug. (1) Out-of-range arguments fell through into raw memory access. Several entry points on Leaf, Tree, and NodeManager rely on NANOVDB_ASSERT in the underlying C++ to catch invalid indices. That assertion is a no-op in release builds, so passing an OOB index from Python would read off the end of mValueMask / mValues / mNodeOffset[] arrays. Wrapped each with an explicit range check that raises a Python IndexError or ValueError: - Leaf.isActive(n) n must be < voxelCount() (512) - Leaf.getValue(offset) offset must be < voxelCount() - Tree.activeTileCount(level) level must be 1, 2, or 3 - Tree.nodeCount(level) level must be 0, 1, or 2 - NodeManager.nodeCount(L) same as Tree - NodeManager.leaf(i) i must be < leafCount() - NodeManager.lower(i) i must be < lowerCount() - NodeManager.upper(i) i must be < upperCount() (2) Returned pointers / NumPy views did not actually keep their backing buffers alive. The pattern `nb::cast(value, nb::rv_policy::reference, parent)` was used at multiple sites under the assumption that the third argument established a Python-level keep_alive linkage between the returned object and `parent`. It doesn't — rv_policy::reference is "no ownership, no keep_alive" by definition; the parent argument is only a hint to the cast machinery, not a lifetime guarantee. As a result, expressions that drop the intermediate handle would silently free the underlying buffer: g = nanovdb.tools.createFogVolumeSphere(name='probe').grid() g.gridName() # SEGFAULT — handle was GC'd vals = (nanovdb.tools.createFogVolumeSphere() .grid().tree().getFirstLeaf().values()) vals.sum() # SEGFAULT nm = nanovdb.createNodeManager( nanovdb.tools.createFogVolumeSphere().grid()).mgr() nm.leaf(0) # SEGFAULT — NodeManager holds raw ptr to grid Added explicit `nb::keep_alive<0, 1>()` to the .def for every affected site so the returned value keeps its parent alive: - GridHandle.grid(n) (PyGridHandle.h) - DeviceGridHandle.deviceGrid(n) (cuda/PyDeviceGridHandle.cu) - NodeManagerHandle.mgr() (PyTree.cc) - Grid.getBlindData(n) (NanoVDBModule.cc) - Leaf.values() (PyTree.h) - Grid.leaf_values() (PyTree.h) - PointAccessor.gridPoints / leafPoints / voxelPoints (NanoVDBModule.cc) - nanovdb.createNodeManager(grid) keep arg 1 (grid) alive as long as the returned NodeManagerHandle lives — because the underlying NodeManager stores a raw pointer to the grid. These were pre-existing bugs introduced in Phase 1 (pyHostGrid) and inherited by every subsequent zero-copy view; Phase 3 surfaced more of them via the new Tree/Leaf/NodeManager bindings. Two new test classes lock the fixes in: - TestBoundsChecks asserts each guarded entry point raises the correct exception type at every boundary and still accepts in- range inputs. - TestZeroCopyViewLifetimes invokes each fixed binding in the chained-temporary form (handle.grid().tree().getFirstLeaf() .values(), createNodeManager(temp).mgr().leaf(0).values(), ...), runs gc.collect(), then touches the returned value. Pre-fix these segfaulted; now they pass cleanly. CPU + CUDA sm_120 (BLOSC + ZLIB on): 75/75 pass. Signed-off-by: Jonathan Swartz * nanovdb python: address Copilot follow-up review on #2212 Three further Copilot notes on the bounds-check/lifetime commit (f6735717), all valid. (1) PyTree.h relied on transitive includes for std::is_arithmetic_v / std::enable_if. Works on GCC + libstdc++ because is pulled in by -> standard headers, but MSVC / libc++ may break the chain. Added an explicit #include near the top. (2) grid.leaf_values() returned None for empty grids (nLeaves == 0 or getFirstLeaf() == nullptr) while its docstring promised an (N_leaves, 512) NumPy view. Callers had to special-case the None sentinel before iterating. Now returns an empty (0, 512) ndarray of the right dtype, so the contract reads cleanly: for row in grid.leaf_values(): ... works on every grid, empty or not. When nLeaves == 0 we pass a dummy non-null aligned pointer (the grid itself) to nb::ndarray so nanobind has a valid base for the empty array — no data is read since the leading shape is 0. Docstring updated to call out the empty-grid behavior explicitly. (3) The ValueError raised on a non-breadth-first grid said "rebuild via tools::createNanoGrid(...)" which reads like a C++ symbol. Reworded to the Python-API form "rebuild via nanovdb.tools.createNanoGrid(...)" so Python users see a Python entry point. New test method TestTreeNodeWalking.test_bulk_leaf_values_empty_grid_returns_empty_array constructs a grid with an empty bbox (nLeaves == 0) and asserts leaf_values() is an (0, 512) float32 NumPy array (not None). CPU + CUDA sm_120 (BLOSC + ZLIB on): 76/76 pass. Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 1 + nanovdb/nanovdb/python/NanoVDBModule.cc | 71 ++- nanovdb/nanovdb/python/PyGridHandle.h | 4 +- nanovdb/nanovdb/python/PyTree.cc | 119 +++++ nanovdb/nanovdb/python/PyTree.h | 418 ++++++++++++++++++ .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 11 +- nanovdb/nanovdb/python/test/TestNanoVDB.py | 284 +++++++++++- 7 files changed, 871 insertions(+), 37 deletions(-) create mode 100644 nanovdb/nanovdb/python/PyTree.cc create mode 100644 nanovdb/nanovdb/python/PyTree.h diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 1b5fc4166e..714e53360f 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -28,6 +28,7 @@ nanobind_add_module(nanovdb_python NB_STATIC PyPrimitives.cc PySampleFromVoxels.cc PyTools.cc + PyTree.cc cuda/PyDeviceBuffer.cc cuda/PyDeviceGridHandle.cu cuda/PyPointsToGrid.cu diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index a90e4b83bb..57d0a1b3f6 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -20,6 +20,7 @@ #include "PyIO.h" #include "PyMath.h" #include "PyTools.h" +#include "PyTree.h" #include "PyGridChecksum.h" namespace nb = nanobind; @@ -251,9 +252,11 @@ void defineGrid(nb::module_& m) return -1; }, "semantic"_a) .def("getBlindData", &pyGetBlindData, "n"_a, + nb::keep_alive<0, 1>(), "Return a zero-copy NumPy view of the n-th blind data channel, " "or None if n is out of range. dtype and shape are derived from " - "the channel's mDataType / mValueCount."); + "the channel's mDataType / mValueCount. The view keeps the grid " + "alive (and therefore the GridHandle that owns the buffer)."); } // BuildT-dependent slice of the typed grid Python class. Inherits the @@ -261,10 +264,17 @@ void defineGrid(nb::module_& m) // need to know BuildT lives there, not here. template void defineNanoGrid(nb::module_& m, const char* name) { - nb::class_, GridData>(m, name) + auto cls = nb::class_, GridData>(m, name) .def("getAccessor", &NanoGrid::getAccessor) .def("activeVoxelCount", &NanoGrid::activeVoxelCount) - .def("isSequential", [](const NanoGrid& grid) { return grid.isSequential(); }); + .def("isSequential", [](const NanoGrid& grid) { return grid.isSequential(); }) + .def("tree", + nb::overload_cast<>(&NanoGrid::tree, nb::const_), + nb::rv_policy::reference_internal, + "Return the tree associated with this grid. Lifetime is " + "anchored to the grid (and therefore to the GridHandle)."); + // Add leaf_values() only for BuildTs whose LeafData carries T mValues[512]. + PyLeafValuesBinder::apply(cls); } void defineGridBlindData(nb::module_& m) @@ -438,7 +448,9 @@ template void definePointAccessor(nb::module_& m, const char* nam uint64_t count = acc.gridPoints(begin, end); if (begin == nullptr || count == 0) return nb::none(); return pyPointsToNdarray(py_self, begin, count); - }, "Return all point attributes in the grid as a single NumPy view.") + }, nb::keep_alive<0, 1>(), + "Return all point attributes in the grid as a single NumPy view. " + "The view keeps this accessor alive.") .def("leafPoints", [](nb::handle py_self, const Coord& ijk) -> nb::object { auto& acc = nb::cast(py_self); const AttT* begin = nullptr; @@ -446,9 +458,10 @@ template void definePointAccessor(nb::module_& m, const char* nam uint64_t count = acc.leafPoints(ijk, begin, end); if (begin == nullptr || count == 0) return nb::none(); return pyPointsToNdarray(py_self, begin, count); - }, "ijk"_a, + }, "ijk"_a, nb::keep_alive<0, 1>(), "Return the point attributes contained within the leaf node " - "covering ijk, or None if no leaf is present.") + "covering ijk, or None if no leaf is present. The view keeps " + "this accessor alive.") .def("voxelPoints", [](nb::handle py_self, const Coord& ijk) -> nb::object { auto& acc = nb::cast(py_self); const AttT* begin = nullptr; @@ -456,9 +469,10 @@ template void definePointAccessor(nb::module_& m, const char* nam uint64_t count = acc.voxelPoints(ijk, begin, end); if (begin == nullptr || count == 0) return nb::none(); return pyPointsToNdarray(py_self, begin, count); - }, "ijk"_a, + }, "ijk"_a, nb::keep_alive<0, 1>(), "Return the point attributes at the specific voxel ijk, or None " - "if the voxel is inactive / empty."); + "if the voxel is inactive / empty. The view keeps this accessor " + "alive."); } // Type-erased grid introspector. Mirrors nanovdb::GridMetaData (768B) and @@ -720,6 +734,43 @@ NB_MODULE(nanovdb, m) defineGrid(m); defineGridMetaData(m); + // Tree / node bindings must come BEFORE defineNanoGrid because + // NanoGrid.tree() returns NanoTree (registered here) by const + // reference. Per-BuildT, register Leaf, Lower, Upper, Root, Tree in + // child->parent order so each return type is registered before the + // method binding that returns it. +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + defineNanoLeaf(m, #Suffix "Leaf"); \ + defineNanoLower(m, #Suffix "Lower"); \ + defineNanoUpper(m, #Suffix "Upper"); \ + defineNanoRoot(m, #Suffix "Root"); \ + defineNanoTree(m, #Suffix "Tree"); \ + defineNodeManager(m, #Suffix "NodeManager"); +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + defineNanoLeaf(m, #Suffix "Leaf"); \ + defineNanoLower(m, #Suffix "Lower"); \ + defineNanoUpper(m, #Suffix "Upper"); \ + defineNanoRoot(m, #Suffix "Root"); \ + defineNanoTree(m, #Suffix "Tree"); \ + defineNodeManager(m, #Suffix "NodeManager"); +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + defineNanoLeaf(m, #Suffix "Leaf"); \ + defineNanoLower(m, #Suffix "Lower"); \ + defineNanoUpper(m, #Suffix "Upper"); \ + defineNanoRoot(m, #Suffix "Root"); \ + defineNanoTree(m, #Suffix "Tree"); \ + defineNodeManager(m, #Suffix "NodeManager"); +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + defineNanoLeaf(m, #Suffix "Leaf"); \ + defineNanoLower(m, #Suffix "Lower"); \ + defineNanoUpper(m, #Suffix "Upper"); \ + defineNanoRoot(m, #Suffix "Root"); \ + defineNanoTree(m, #Suffix "Tree"); \ + defineNodeManager(m, #Suffix "NodeManager"); +#include "BuildTypes.def" + + // Now bind the per-BuildT NanoGrid + accessors (tree() return type now + // registered above). #define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ defineNanoGrid(m, #Suffix "Grid"); \ defineScalarAccessor(m, #Suffix "ReadAccessor"); \ @@ -735,6 +786,10 @@ NB_MODULE(nanovdb, m) defineAccessor(m, #Suffix "ReadAccessor"); #include "BuildTypes.def" + // Host-side NodeManagerHandle + module-scope createNodeManager. + defineNodeManagerHandle(m); + defineCreateNodeManager(m); + // PointAccessor variants — PointIndex grids carry uint32 indices, // PointData grids carry Vec3f positions. definePointAccessor(m, "PointIndexAccessor"); diff --git a/nanovdb/nanovdb/python/PyGridHandle.h b/nanovdb/nanovdb/python/PyGridHandle.h index 8f3f464469..4774bd3cec 100644 --- a/nanovdb/nanovdb/python/PyGridHandle.h +++ b/nanovdb/nanovdb/python/PyGridHandle.h @@ -137,8 +137,10 @@ template nb::class_> defineGridHa }, "Return a deep copy of this GridHandle backed by a freshly-allocated buffer.") .def("grid", &pyHostGrid, nb::arg("n") = 0, + nb::keep_alive<0, 1>(), "Return the n-th grid as a typed Grid subclass selected by " - "gridType(n), or None if the BuildT is not bound in Python.") + "gridType(n), or None if the BuildT is not bound in Python. " + "The returned grid keeps this handle alive.") .def("isPadded", &nanovdb::GridHandle::isPadded) .def("gridCount", &nanovdb::GridHandle::gridCount) .def("gridSize", &nanovdb::GridHandle::gridSize, nb::arg("n") = 0) diff --git a/nanovdb/nanovdb/python/PyTree.cc b/nanovdb/nanovdb/python/PyTree.cc new file mode 100644 index 0000000000..e2fc6c515d --- /dev/null +++ b/nanovdb/nanovdb/python/PyTree.cc @@ -0,0 +1,119 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyTree.h" + +#include + +namespace nb = nanobind; +using namespace nb::literals; +using namespace nanovdb; + +namespace pynanovdb { + +// Polymorphic mgr() that returns the right typed NodeManager based on the +// handle's stored gridType. Dispatch follows the same X-macro pattern as +// pyHostGrid / pyDeviceGrid; unbound BuildTs return None rather than the +// generic getMgr() ptr that would be reinterpreted. +template +static nb::object pyNodeMgr(nb::handle py_self) +{ + using HandleT = NodeManagerHandle; + auto& handle = nb::cast(py_self); + if (!handle.data()) return nb::none(); + // We need to read the stored gridType, but it's private. The public + // mgr() returns NULL for type mismatch, so iterate by BuildT. + // The X-macro produces one case per bound BuildT; first non-null wins. +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto* m = handle.template mgr()) { \ + return nb::cast(m, nb::rv_policy::reference, py_self); \ + } +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + if (auto* m = handle.template mgr()) { \ + return nb::cast(m, nb::rv_policy::reference, py_self); \ + } +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto* m = handle.template mgr()) { \ + return nb::cast(m, nb::rv_policy::reference, py_self); \ + } +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto* m = handle.template mgr()) { \ + return nb::cast(m, nb::rv_policy::reference, py_self); \ + } +#include "BuildTypes.def" + return nb::none(); +} + +void defineNodeManagerHandle(nb::module_& m) +{ + using HandleT = NodeManagerHandle; + nb::class_(m, "NodeManagerHandle", + "Owns the memory backing a NodeManager. Move-only. " + "Obtain via nanovdb.createNodeManager(grid).") + .def("size", + [](const HandleT& h) { return h.size(); }) + .def( + "__bool__", + [](const HandleT& h) { return h.data() != nullptr; }, + nb::is_operator()) + .def("mgr", &pyNodeMgr, + nb::keep_alive<0, 1>(), + "Return the typed NodeManager for the grid this handle was " + "built from, or None if the BuildT is not Python-visible. The " + "returned NodeManager keeps this handle alive."); +} + +// createNodeManager has one template instantiation per BuildT. We expose a +// single polymorphic `createNodeManager(grid)` that picks the right one +// based on the runtime type of `grid` (any nb::class_-bound NanoGrid). +// nb::isinstance is a fast type check that avoids the exception-on-mismatch +// overhead that would come from trying nb::cast and catching cast_error for +// every non-matching BuildT. +template +static nb::object tryCreateNodeManager(nb::handle py_grid) +{ + using GridT = NanoGrid; + if (!nb::isinstance(py_grid)) { + return nb::object(); // sentinel: "not this BuildT, try next" + } + auto& grid = nb::cast(py_grid); + return nb::cast(createNodeManager(grid)); +} + +void defineCreateNodeManager(nb::module_& m) +{ + m.def("createNodeManager", + [](nb::handle py_grid) -> nb::object { + // Try every bound BuildT; first successful cast wins. +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto obj = tryCreateNodeManager(py_grid); obj.is_valid()) { \ + return obj; \ + } +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + if (auto obj = tryCreateNodeManager(py_grid); obj.is_valid()) { \ + return obj; \ + } +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto obj = tryCreateNodeManager(py_grid); obj.is_valid()) { \ + return obj; \ + } +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto obj = tryCreateNodeManager(py_grid); obj.is_valid()) { \ + return obj; \ + } +#include "BuildTypes.def" + throw nb::type_error( + "createNodeManager: argument is not a NanoVDB grid of any " + "bound BuildT"); + }, + "grid"_a, + // The constructed NodeManager stores a raw pointer back to the + // grid; the handle must therefore keep the grid (and transitively + // the GridHandle that owns the grid's buffer) alive. + nb::keep_alive<0, 1>(), + "Build a NodeManager for the given grid, returning a " + "NodeManagerHandle that owns the underlying buffer. The handle's " + "mgr() method returns the typed NodeManager. The handle keeps the " + "source grid alive for as long as it lives."); +} + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyTree.h b/nanovdb/nanovdb/python/PyTree.h new file mode 100644 index 0000000000..e8fbad1014 --- /dev/null +++ b/nanovdb/nanovdb/python/PyTree.h @@ -0,0 +1,418 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_PYTREE_HAS_BEEN_INCLUDED +#define NANOVDB_PYTREE_HAS_BEEN_INCLUDED + +#include +#include +#include + +#include +#include +#include + +#include // std::is_arithmetic_v, std::enable_if + +namespace nb = nanobind; + +namespace pynanovdb { + +// -------------------- NanoLeaf -------------------- +// +// Binds the 8^3 leaf node. Methods that don't depend on whether the leaf +// stores a contiguous T[512] array are bound unconditionally; the +// zero-copy 512-element NumPy values() view is only bound when the leaf +// layout actually carries T mValues[512] (the BuildTraits::is_special +// types use packed / index / mask layouts where a 512-element T view is +// either impossible or misleading). +template void defineNanoLeaf(nb::module_& m, const char* name) +{ + using LeafT = nanovdb::NanoLeaf; + using ValueT = typename LeafT::ValueType; + using CoordT = typename LeafT::CoordType; + + auto cls = nb::class_(m, name, + "Leaf node — 8x8x8 voxels. Inherits stats and bbox from the same " + "leaf-data block bound across BuildTs."); + + cls.def("origin", &LeafT::origin) + .def("bbox", &LeafT::bbox) + .def("hasBBox", &LeafT::hasBBox) + .def_static("dim", &LeafT::dim) + .def_static("voxelCount", &LeafT::voxelCount) + .def("memUsage", &LeafT::memUsage) + .def("isActive", + nb::overload_cast(&LeafT::isActive, nb::const_), + nb::arg("ijk")) + .def("isActive", + [](const LeafT& leaf, uint32_t n) { + // Underlying mValueMask.isOn(n) is unchecked; release builds + // skip the C++ NANOVDB_ASSERT and would silently read OOB. + if (n >= LeafT::voxelCount()) { + throw nb::index_error( + "Leaf.isActive(n): n out of range [0, voxelCount)"); + } + return leaf.isActive(n); + }, + nb::arg("n")) + .def("getValue", + [](const LeafT& leaf, uint32_t offset) { + // mValues[offset] is unchecked in C++; guard the Python side. + if (offset >= LeafT::voxelCount()) { + throw nb::index_error( + "Leaf.getValue(offset): offset out of range [0, voxelCount)"); + } + return leaf.getValue(offset); + }, + nb::arg("offset")) + .def("getValue", + nb::overload_cast(&LeafT::getValue, nb::const_), + nb::arg("ijk")) + .def("getFirstValue", &LeafT::getFirstValue) + .def("getLastValue", &LeafT::getLastValue) + .def("minimum", &LeafT::minimum) + .def("maximum", &LeafT::maximum) + .def("average", &LeafT::average) + .def("stdDeviation", &LeafT::stdDeviation) + // NOTE: variance() omitted — NanoVDB.h line 4388 uses unqualified + // Pow2() which fails ADL for non-float ValueTs (ValueIndex / + // ValueMask / etc.). Users can compute it as stdDeviation() ** 2. + .def("flags", &LeafT::flags) + .def("valueMask", &LeafT::valueMask, nb::rv_policy::reference_internal) + .def("probeValue", + [](const LeafT& leaf, const CoordT& ijk) { + ValueT v; + bool on = leaf.probeValue(ijk, v); + return std::make_tuple(v, on); + }, + nb::arg("ijk")); + + // Zero-copy 512-element NumPy view of mValues. Only enabled for + // BuildTs whose ValueType is a primitive arithmetic type (float, + // double, int*). For Fp* / Index / Mask / bool / Point the leaf uses + // packed / mask / void layouts. For Vec3f / Vec3d / Vec4f / Vec4d / + // Vec3u8 / Vec3u16 / Rgba8 the leaf carries a struct array that + // nanobind's ndarray can't represent directly — those would need a + // flattened (count, dim) component-typed view which a follow-up can + // add. Users can still walk every leaf via the bound getValue(). + if constexpr (std::is_arithmetic_v + && !nanovdb::BuildTraits::is_special) { + cls.def("values", + [](nb::handle py_self) { + auto& leaf = nb::cast(py_self); + size_t shape[1] = {LeafT::voxelCount()}; + return nb::cast( + nb::ndarray, nb::c_contig, nb::device::cpu>( + static_cast(leaf.data()->mValues), + size_t(1), shape, py_self), + nb::rv_policy::reference); + }, + nb::keep_alive<0, 1>(), + "Return a zero-copy NumPy view of the 512 leaf values. The view " + "keeps the leaf (and transitively the GridHandle that owns the " + "underlying buffer) alive."); + } +} + +// -------------------- NanoUpper / NanoLower -------------------- +// +// Both internal node levels share the same C++ API surface (just different +// LOG2DIM). One helper templated on the concrete InternalNode type. +template +void defineInternalNodeBase(nb::class_& cls) +{ + using ValueT = typename InternalT::ValueType; + using CoordT = typename InternalT::CoordType; + cls.def("origin", &InternalT::origin) + .def("bbox", &InternalT::bbox) + .def_static("dim", &InternalT::dim) + .def_static("memUsage", []() { return InternalT::memUsage(); }) + .def("minimum", &InternalT::minimum) + .def("maximum", &InternalT::maximum) + .def("average", &InternalT::average) + .def("stdDeviation", &InternalT::stdDeviation) + // variance() omitted for parity with the leaf binding; compute as + // stdDeviation() ** 2 in Python. + .def("valueMask", &InternalT::valueMask, nb::rv_policy::reference_internal) + .def("childMask", &InternalT::childMask, nb::rv_policy::reference_internal) + .def("getValue", + nb::overload_cast(&InternalT::getValue, nb::const_), + nb::arg("ijk")) + .def("getFirstValue", &InternalT::getFirstValue) + .def("getLastValue", &InternalT::getLastValue) + .def("isActive", + nb::overload_cast(&InternalT::isActive, nb::const_), + nb::arg("ijk")) + .def("probeValue", + [](const InternalT& node, const CoordT& ijk) { + ValueT v; + bool on = node.probeValue(ijk, v); + return std::make_tuple(v, on); + }, + nb::arg("ijk")); +} + +template void defineNanoUpper(nb::module_& m, const char* name) +{ + using UpperT = nanovdb::NanoUpper; + nb::class_ cls(m, name, + "Upper internal node — 32x32x32 (covers a 4096^3 region in index space)."); + defineInternalNodeBase(cls); +} + +template void defineNanoLower(nb::module_& m, const char* name) +{ + using LowerT = nanovdb::NanoLower; + nb::class_ cls(m, name, + "Lower internal node — 16x16x16 (covers a 128^3 region in index space)."); + defineInternalNodeBase(cls); +} + +// -------------------- NanoRoot -------------------- +template void defineNanoRoot(nb::module_& m, const char* name) +{ + using RootT = nanovdb::NanoRoot; + using ValueT = typename RootT::ValueType; + using CoordT = typename RootT::CoordType; + + nb::class_(m, name, "Root node — top of the tree, holds the tile table.") + .def("background", &RootT::background, nb::rv_policy::reference_internal) + .def("tileCount", &RootT::tileCount) + .def("getTableSize", &RootT::getTableSize) + .def("isEmpty", &RootT::isEmpty) + .def("bbox", &RootT::bbox, nb::rv_policy::reference_internal) + .def("minimum", &RootT::minimum, nb::rv_policy::reference_internal) + .def("maximum", &RootT::maximum, nb::rv_policy::reference_internal) + .def("average", &RootT::average, nb::rv_policy::reference_internal) + .def("stdDeviation", &RootT::stdDeviation, nb::rv_policy::reference_internal) + .def("memUsage", + nb::overload_cast<>(&RootT::memUsage, nb::const_)) + .def("getValue", + nb::overload_cast(&RootT::getValue, nb::const_), + nb::arg("ijk")) + .def("isActive", + nb::overload_cast(&RootT::isActive, nb::const_), + nb::arg("ijk")) + .def("probeValue", + [](const RootT& root, const CoordT& ijk) { + ValueT v; + bool on = root.probeValue(ijk, v); + return std::make_tuple(v, on); + }, + nb::arg("ijk")); +} + +// -------------------- NanoTree -------------------- +template void defineNanoTree(nb::module_& m, const char* name) +{ + using TreeT = nanovdb::NanoTree; + using ValueT = typename TreeT::ValueType; + using CoordT = typename TreeT::CoordType; + using RootT = typename TreeT::RootType; + using UpperT = typename TreeT::UpperNodeType; + using LowerT = typename TreeT::LowerNodeType; + using LeafT = typename TreeT::LeafNodeType; + + nb::class_(m, name, + "Tree — owns the root and provides bulk metadata queries " + "(node counts, active voxel count, extrema).") + .def("root", + nb::overload_cast<>(&TreeT::root, nb::const_), + nb::rv_policy::reference_internal) + .def("background", &TreeT::background, nb::rv_policy::reference_internal) + .def("activeVoxelCount", &TreeT::activeVoxelCount) + // activeTileCount(level): valid range is 1..3 (lower / upper / root + // tile counts). C++ uses NANOVDB_ASSERT(level > 0 && level <= 3) + // which is a no-op in release builds — so guard explicitly. + .def("activeTileCount", + [](const TreeT& tree, uint32_t level) -> uint32_t { + if (level < 1 || level > 3) { + throw nb::value_error( + "Tree.activeTileCount(level): level must be 1, 2, or 3"); + } + return tree.activeTileCount(level); + }, + nb::arg("level")) + // nodeCount(level): valid range is 0..2 (leaf / lower / upper). + // C++ uses NANOVDB_ASSERT(level < 3), again no-op in release. + // The lambda's `int level` argument disambiguates the call against + // Tree's templated nodeCount() overload at the C++ level, + // so we don't need an overload_cast / static_cast wrapper here. + .def("nodeCount", + [](const TreeT& tree, int level) -> uint32_t { + if (level < 0 || level >= 3) { + throw nb::value_error( + "Tree.nodeCount(level): level must be 0, 1, or 2"); + } + return tree.nodeCount(level); + }, + nb::arg("level")) + .def("totalNodeCount", &TreeT::totalNodeCount) + .def_static("memUsage", &TreeT::memUsage) + .def("getValue", + nb::overload_cast(&TreeT::getValue, nb::const_), + nb::arg("ijk")) + .def("isActive", &TreeT::isActive, nb::arg("ijk")) + .def("probeValue", + [](const TreeT& tree, const CoordT& ijk) { + ValueT v; + bool on = tree.probeValue(ijk, v); + return std::make_tuple(v, on); + }, + nb::arg("ijk")) + .def("extrema", + [](const TreeT& tree) { + ValueT mn, mx; + tree.extrema(mn, mx); + return std::make_tuple(mn, mx); + }, + "Return (min, max) of the active values over the whole tree.") + .def("getFirstLeaf", + nb::overload_cast<>(&TreeT::getFirstLeaf, nb::const_), + nb::rv_policy::reference_internal, + "First leaf node in breadth-first order, or None if the tree is empty.") + .def("getFirstLower", + nb::overload_cast<>(&TreeT::getFirstLower, nb::const_), + nb::rv_policy::reference_internal) + .def("getFirstUpper", + nb::overload_cast<>(&TreeT::getFirstUpper, nb::const_), + nb::rv_policy::reference_internal); +} + +// -------------------- NodeManager -------------------- +// +// NodeManager is heap-managed by a NodeManagerHandle (move-only, owns the +// underlying memory). We bind one NodeManager class per BuildT and one +// host-side NodeManagerHandle class. Users get a handle from +// nanovdb.createNodeManager(grid); they then call handle.mgr() to obtain a +// borrowed pointer to the typed NodeManager — its lifetime is anchored to +// the handle via reference_internal. +template void defineNodeManager(nb::module_& m, const char* name) +{ + using NMT = nanovdb::NodeManager; + nb::class_(m, name, + "Sequential breadth-first accessor for the leaf / lower / upper " + "internal nodes of a NanoGrid. Construct via " + "nanovdb.createNodeManager(grid).") + .def("isLinear", + nb::overload_cast<>(&NMT::isLinear, nb::const_)) + .def("memUsage", + nb::overload_cast<>(&NMT::memUsage, nb::const_)) + .def("nodeCount", + [](const NMT& nm, int level) -> uint64_t { + // Mirror Tree.nodeCount bounds (NodeManager forwards to Tree). + if (level < 0 || level >= 3) { + throw nb::value_error( + "NodeManager.nodeCount(level): level must be 0, 1, or 2"); + } + return nm.nodeCount(level); + }, + nb::arg("level")) + .def("leafCount", &NMT::leafCount) + .def("lowerCount", &NMT::lowerCount) + .def("upperCount", &NMT::upperCount) + // leaf / lower / upper: NANOVDB_ASSERT(i < nodeCount(LEVEL)) in C++ is + // no-op in release, so guard explicitly to convert OOB access into a + // Python IndexError instead of memory corruption. + .def("leaf", + [](const NMT& nm, uint32_t i) -> const nanovdb::NanoLeaf& { + if (i >= nm.leafCount()) { + throw nb::index_error( + "NodeManager.leaf(i): i out of range [0, leafCount)"); + } + return nm.leaf(i); + }, + nb::rv_policy::reference_internal, nb::arg("i")) + .def("lower", + [](const NMT& nm, uint32_t i) -> const nanovdb::NanoLower& { + if (i >= nm.lowerCount()) { + throw nb::index_error( + "NodeManager.lower(i): i out of range [0, lowerCount)"); + } + return nm.lower(i); + }, + nb::rv_policy::reference_internal, nb::arg("i")) + .def("upper", + [](const NMT& nm, uint32_t i) -> const nanovdb::NanoUpper& { + if (i >= nm.upperCount()) { + throw nb::index_error( + "NodeManager.upper(i): i out of range [0, upperCount)"); + } + return nm.upper(i); + }, + nb::rv_policy::reference_internal, nb::arg("i")); +} + +void defineNodeManagerHandle(nb::module_& m); +void defineCreateNodeManager(nb::module_& m); + +// -------------------- grid.leaf_values() bulk extractor -------------------- +// +// For non-special BuildTs with breadth-first, fixed-size leaves, the leaf +// values can be reached as a contiguous (N_leaves, 512) array — every leaf +// occupies sizeof(NanoLeaf) bytes and mValues starts at a known offset +// inside each leaf. We bind this on NanoGrid as leaf_values() for +// efficient bulk analytics from Python. +template +struct PyLeafValuesBinder +{ + template static void apply(ClsT&) {} +}; + +template +struct PyLeafValuesBinder::ValueType> + && !nanovdb::BuildTraits::is_special>::type> +{ + template + static void apply(ClsT& cls) + { + using GridT = nanovdb::NanoGrid; + using LeafT = nanovdb::NanoLeaf; + using ValueT = typename LeafT::ValueType; + cls.def("leaf_values", + [](nb::handle py_self) -> nb::object { + auto& grid = nb::cast(py_self); + const auto& tree = grid.tree(); + const uint32_t nLeaves = tree.template nodeCount(); + if (!grid.isBreadthFirst()) { + throw nb::value_error( + "leaf_values() requires a breadth-first grid " + "layout; rebuild via " + "nanovdb.tools.createNanoGrid(...)."); + } + // For an empty grid (no leaves) we still return an ndarray + // — shape (0, 512) — so callers can iterate / np.asarray() + // / shape-test without branching on a None sentinel. + LeafT* first = const_cast(tree.getFirstLeaf()); + size_t shape[2] = {nLeaves, LeafT::voxelCount()}; + int64_t strides[2] = { + static_cast(sizeof(LeafT) / sizeof(ValueT)), + 1 + }; + // first is non-null whenever nLeaves > 0; when nLeaves == 0 + // we pass a dummy non-null aligned pointer (the grid itself) + // so nanobind has something to base the empty array on. + // Nothing will be read since the leading shape is 0. + void* data = (first != nullptr) + ? static_cast(first->data()->mValues) + : static_cast(&grid); + return nb::cast( + nb::ndarray, nb::device::cpu>( + data, size_t(2), shape, py_self, strides), + nb::rv_policy::reference); + }, + nb::keep_alive<0, 1>(), + "Return a zero-copy (N_leaves, 512) NumPy view of every leaf's " + "values, in breadth-first leaf order. Available only for " + "BuildTs whose leaf layout carries T mValues[512] (i.e. not " + "Fp*, Index, Mask, bool, or Point) and only on breadth-first " + "grids. Returns an empty (0, 512) array for grids with no " + "leaves. The view keeps the grid alive."); + } +}; + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index cbec582c91..9647e9739d 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -72,9 +72,11 @@ void defineDeviceGridHandle(nb::module_& m) "cpu_t"_a.noconvert(), "cuda_t"_a.noconvert()) .def("deviceGrid", &pyDeviceGrid, "n"_a = 0, + nb::keep_alive<0, 1>(), "Return the n-th device-resident grid as a typed Grid subclass " "selected by gridType(n), or None if the BuildT is not bound in " - "Python or the device copy has not been uploaded yet.") + "Python or the device copy has not been uploaded yet. The " + "returned grid keeps this handle alive.") .def( "deviceUpload", [](GridHandle& handle, bool sync) { handle.deviceUpload(nullptr, sync); }, "sync"_a = true) .def( @@ -84,10 +86,9 @@ void defineDeviceGridHandle(nb::module_& m) // second overload taking a DeviceGridHandle list conflicts with the host // overload because both signatures take nb::list, and nanobind's // overload resolution can't disambiguate by element type — it picks one - // and the inner cast fails with std::bad_cast. The host-only utilities - // are what the Phase 1 plan calls for; a properly typed device variant - // (with its own name, or with strongly-typed std::vector args - // and nanobind/stl/vector.h support) can land later if it's needed. + // and the inner cast fails with std::bad_cast. A properly typed device + // variant (with its own name, or strongly-typed std::vector + // args via nanobind/stl/vector.h) can land later if it's needed. } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index a87bae3ba2..f0fc2dba60 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -348,7 +348,9 @@ def test_list_to_vector(self): class TestPolymorphicGridAccess(unittest.TestCase): - """Phase 1a: handle.grid(n) returns the correct typed Grid subclass.""" + """handle.grid(n) returns the correct typed Grid subclass selected by + gridType(n); the legacy per-type accessors (floatGrid(), etc.) are not + bound.""" def test_float_grid(self): h = nanovdb.tools.createFogVolumeSphere() @@ -368,8 +370,8 @@ def test_empty_handle_returns_none(self): self.assertIsNone(nanovdb.GridHandle().grid()) def test_typed_accessors_removed(self): + # The legacy per-type accessors were replaced by handle.grid(n). h = nanovdb.tools.createFogVolumeSphere() - # Phase 1a.3 removed these in favour of handle.grid(n). self.assertFalse(hasattr(h, "floatGrid")) self.assertFalse(hasattr(h, "doubleGrid")) self.assertFalse(hasattr(h, "int32Grid")) @@ -378,7 +380,9 @@ def test_typed_accessors_removed(self): class TestGridBase(unittest.TestCase): - """Phase 1a.1: BuildT-independent methods resolve via the Grid base class.""" + """Methods that don't depend on BuildT (gridType, gridClass, voxelSize, + isLevelSet/...) resolve via the Grid base class shared by every typed + grid subclass.""" def test_grid_base_class_name(self): # Typed grids inherit from a base class named "Grid" (no more "GridData"). @@ -399,7 +403,9 @@ def test_lifted_methods_accessible_via_inheritance(self): class TestGridMetaData(unittest.TestCase): - """Phase 1b.1: type-erased GridMetaData introspector.""" + """nanovdb.GridMetaData is a type-erased introspector — construct from a + Grid and query gridType/gridClass/voxelSize/etc. without knowing + BuildT.""" def test_constructed_from_grid(self): h = nanovdb.tools.createFogVolumeSphere(name="probe") @@ -415,7 +421,9 @@ def test_constructed_from_grid(self): class TestBlindDataEmpty(unittest.TestCase): - """Phase 1b.2: blind data API works on grids that have none.""" + """Blind data API (blindDataCount, blindMetaData, findBlindData, + findBlindDataForSemantic, getBlindData) returns sensible None/-1 + sentinels on grids that have no blind data channels.""" def test_no_blind_data(self): h = nanovdb.tools.createFogVolumeSphere() @@ -431,7 +439,9 @@ def test_no_blind_data(self): class TestSplitMergeCopy(unittest.TestCase): - """Phase 1c.2: splitGrids / mergeGrids / handle.copy().""" + """splitGrids(h) -> list of single-grid handles, mergeGrids([h1, h2]) + -> combined handle, h.copy() -> deep buffer copy. mergeGrids must not + consume its input handles.""" def test_split_and_merge_roundtrip(self): h1 = nanovdb.tools.createFogVolumeSphere(name="a") @@ -444,10 +454,10 @@ def test_split_and_merge_roundtrip(self): self.assertEqual(s.gridCount(), 1) def test_merge_does_not_consume_inputs(self): - # Regression: original Phase 1 mergeGrids used nb::cast - # which moved the underlying C++ handle out of the Python wrapper, - # silently emptying h1/h2. The fixed version reads each handle by - # const reference. + # Regression: a previous mergeGrids implementation used + # nb::cast which moved the underlying C++ handle out of + # the Python wrapper, silently emptying h1/h2 after the call. The + # binding now reads each handle by const reference. h1 = nanovdb.tools.createFogVolumeSphere(name="a") h2 = nanovdb.tools.createLevelSetTorus(nanovdb.GridType.Float, name="b") sz1, sz2 = h1.size(), h2.size() @@ -472,12 +482,12 @@ def test_copy_is_deep(self): self.assertEqual(cp.grid().gridName(), "orig") -class TestPhase2BuildTCoverage(unittest.TestCase): - """Phase 2: every additional BuildT registers a Grid + ReadAccessor (and - NodeInfo where applicable). We can't host-construct most of these (the - primitives that produce them land in Phase 5), but we can confirm - registration completed and the accessor surfaces match the type kind. - """ +class TestBuildTRegistrations(unittest.TestCase): + """Every BuildT we bind exposes the right shape — a Grid class, a + ReadAccessor, and (for arithmetic-valued scalars) a NodeInfo. Accessor + surface depends on the type kind: scalar accessors have setVoxel + + getNodeInfo; vector accessors have setVoxel only; read-only accessors + (Boolean, Fp*, Index, OnIndex, Mask) have neither.""" SCALARS = ["Int16", "Int64", "UInt8", "UInt32"] VECTORS = ["Vec3d", "Vec4f", "Vec4d", "Vec3u8", "Vec3u16"] @@ -506,8 +516,10 @@ def test_scalar_accessors_have_setvoxel_and_nodeinfo(self): f"{suffix}NodeInfo missing") def test_vector_accessors_have_setvoxel_no_nodeinfo(self): - # Vector accessor names are aligned in Phase 2 (Vec3dReadAccessor, - # ...). The legacy Vec3f one is still Vec3fReadVectorAccessor. + # The newer vector accessor names follow the consistent + # ReadAccessor pattern (Vec3dReadAccessor, ...); the + # original Vec3f one is named Vec3fReadVectorAccessor for backwards + # compatibility. for suffix in self.VECTORS: acc = getattr(nanovdb, suffix + "ReadAccessor") self.assertTrue(hasattr(acc, "setVoxel")) @@ -516,7 +528,8 @@ def test_vector_accessors_have_setvoxel_no_nodeinfo(self): def test_all_grid_type_enums_reachable(self): # GridType enum binding must cover every BuildT we register — # otherwise Python users can't compare against handle.gridType(n). - # UInt8 was missed in Phase 0; this test locks the fix in. + # Locks in the full set; missing entries (e.g. an unbound enumerator + # for a freshly-added BuildT) get caught here. for name in ["Float", "Double", "Int16", "Int32", "Int64", "UInt8", "UInt32", "Boolean", "Half", "RGBA8", "Vec3f", "Vec3d", "Vec4f", "Vec4d", "Vec3u8", "Vec3u16", "Mask", "Fp4", @@ -525,9 +538,9 @@ def test_all_grid_type_enums_reachable(self): f"nanovdb.GridType.{name} not bound") def test_readonly_accessors_have_neither_setvoxel_nor_nodeinfo(self): - # The plan calls out: quantized types decode to float on read but - # do not bind setVoxel; index types return uint64 and ValueMask - # exposes only active-state queries. + # Quantized types decode to float on read but don't bind setVoxel; + # index types return uint64; ValueMask exposes only active-state + # queries. All share a bare ReadAccessor. for suffix in self.READONLY: acc = getattr(nanovdb, suffix + "ReadAccessor") self.assertFalse(hasattr(acc, "setVoxel"), @@ -536,8 +549,233 @@ def test_readonly_accessors_have_neither_setvoxel_nor_nodeinfo(self): f"{suffix}ReadAccessor should not have getNodeInfo") +class TestTreeNodeWalking(unittest.TestCase): + """Walk a grid's tree from Python: Grid.tree(), Root/Upper/Lower/Leaf + node access and metadata, per-leaf zero-copy values() and bulk + grid.leaf_values() NumPy views, NodeManager + createNodeManager.""" + + @classmethod + def setUpClass(cls): + cls.h = nanovdb.tools.createFogVolumeSphere(name="probe") + cls.g = cls.h.grid() + cls.tree = cls.g.tree() + + def test_grid_tree_basic(self): + self.assertIsInstance(self.tree, nanovdb.FloatTree) + self.assertEqual(self.tree.background(), 3.0) # halfwidth*voxelsize default + self.assertGreater(self.tree.activeVoxelCount(), 0) + self.assertGreaterEqual(self.tree.totalNodeCount(), self.tree.nodeCount(0)) + + def test_extrema(self): + mn, mx = self.tree.extrema() + # FogVolumeSphere produces values in [0, 1]. + self.assertGreaterEqual(mn, 0.0) + self.assertLessEqual(mx, 1.0) + self.assertLessEqual(mn, mx) + + def test_first_leaf_and_node_metadata(self): + leaf = self.tree.getFirstLeaf() + self.assertIsInstance(leaf, nanovdb.FloatLeaf) + self.assertEqual(nanovdb.FloatLeaf.dim(), 8) + self.assertEqual(nanovdb.FloatLeaf.voxelCount(), 512) + # Origin should be aligned to LeafNode dim=8. + for c in (leaf.origin().x, leaf.origin().y, leaf.origin().z): + self.assertEqual(c % 8, 0) + + def test_root_metadata(self): + root = self.tree.root() + self.assertIsInstance(root, nanovdb.FloatRoot) + self.assertGreater(root.tileCount(), 0) + # Root bbox covers ALL active voxels — non-empty for a fog sphere. + bb = root.bbox() + self.assertFalse(bb.empty()) + self.assertEqual(root.background(), self.tree.background()) + + def test_leaf_values_zero_copy(self): + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + leaf = self.tree.getFirstLeaf() + vals = leaf.values() + self.assertEqual(vals.shape, (512,)) + self.assertEqual(vals.dtype, np.float32) + # Mutation through the view writes back into the grid buffer. + original = float(vals[0]) + vals[0] = original + 1.0 + self.assertAlmostEqual(float(leaf.getValue(0)), original + 1.0) + vals[0] = original # restore + + def test_leaf_values_unavailable_for_special_buildts(self): + # ValueIndex / ValueMask / bool / Fp* leaves don't carry T mValues[512], + # so the `values` accessor is not bound for them. + for cls_name in ("BooleanLeaf", "Fp4Leaf", "IndexLeaf", "MaskLeaf"): + leaf_cls = getattr(nanovdb, cls_name) + self.assertFalse(hasattr(leaf_cls, "values"), + f"{cls_name}.values should not be bound") + + def test_bulk_leaf_values(self): + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + bulk = self.g.leaf_values() + self.assertEqual(bulk.shape, (self.tree.nodeCount(0), 512)) + self.assertEqual(bulk.dtype, np.float32) + # First row should match per-leaf values(). + first_via_bulk = np.asarray(bulk[0]) + first_via_leaf = np.asarray(self.tree.getFirstLeaf().values()) + self.assertTrue(np.array_equal(first_via_bulk, first_via_leaf)) + + def test_bulk_leaf_values_empty_grid_returns_empty_array(self): + # A grid with no leaves returns an empty (0, 512) NumPy view rather + # than None, so callers can iterate / shape-test without a sentinel. + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + empty_bbox = nanovdb.math.CoordBBox() # default-constructed = empty + empty_h = nanovdb.tools.createFloatGrid( + 0.0, "empty", nanovdb.GridClass.Unknown, + lambda ijk: 0.0, empty_bbox) + bulk = empty_h.grid().leaf_values() + self.assertEqual(bulk.shape, (0, 512)) + self.assertEqual(bulk.dtype, np.float32) + + def test_node_manager_round_trip(self): + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + handle = nanovdb.createNodeManager(self.g) + self.assertGreater(handle.size(), 0) + self.assertTrue(bool(handle)) + nm = handle.mgr() + self.assertIsInstance(nm, nanovdb.FloatNodeManager) + self.assertTrue(nm.isLinear()) # createNanoGrid produces breadth-first + self.assertEqual(nm.leafCount(), self.tree.nodeCount(0)) + self.assertEqual(nm.lowerCount(), self.tree.nodeCount(1)) + self.assertEqual(nm.upperCount(), self.tree.nodeCount(2)) + # NodeManager.leaf(0) should be the same leaf as tree.getFirstLeaf() + # (breadth-first order). + self.assertEqual(nm.leaf(0).origin(), self.tree.getFirstLeaf().origin()) + self.assertTrue(np.array_equal( + nm.leaf(0).values(), self.tree.getFirstLeaf().values())) + + +class TestBoundsChecks(unittest.TestCase): + """Out-of-range indices on Leaf / Tree / NodeManager raise Python + exceptions rather than falling through into raw memory access. The + underlying C++ uses NANOVDB_ASSERT which is a no-op in release builds, + so the Python layer guards every entry point that takes a level or + index argument. + """ + + @classmethod + def setUpClass(cls): + cls.h = nanovdb.tools.createFogVolumeSphere() + cls.g = cls.h.grid() + cls.tree = cls.g.tree() + cls.leaf = cls.tree.getFirstLeaf() + cls.nm_handle = nanovdb.createNodeManager(cls.g) + cls.nm = cls.nm_handle.mgr() + + def test_leaf_offset_bounds(self): + n = nanovdb.FloatLeaf.voxelCount() + with self.assertRaises(IndexError): + self.leaf.isActive(n) + with self.assertRaises(IndexError): + self.leaf.isActive(n + 1000) + with self.assertRaises(IndexError): + self.leaf.getValue(n) + # In-range still works. + self.assertIsNotNone(self.leaf.getValue(0)) + self.assertIsNotNone(self.leaf.getValue(n - 1)) + + def test_tree_active_tile_count_level(self): + # activeTileCount levels are 1..3 (level 0 is leaves, not tiles). + with self.assertRaises(ValueError): + self.tree.activeTileCount(0) + with self.assertRaises(ValueError): + self.tree.activeTileCount(4) + # In-range still works. + self.assertGreaterEqual(self.tree.activeTileCount(3), 0) + + def test_tree_node_count_level(self): + # nodeCount levels are 0..2 (leaf / lower / upper). + with self.assertRaises(ValueError): + self.tree.nodeCount(-1) + with self.assertRaises(ValueError): + self.tree.nodeCount(3) + self.assertGreater(self.tree.nodeCount(0), 0) + + def test_node_manager_indexed_access(self): + with self.assertRaises(IndexError): + self.nm.leaf(self.nm.leafCount()) + with self.assertRaises(IndexError): + self.nm.lower(self.nm.lowerCount()) + with self.assertRaises(IndexError): + self.nm.upper(self.nm.upperCount()) + with self.assertRaises(ValueError): + self.nm.nodeCount(3) + # In-range still works. + self.assertEqual(self.nm.leaf(0).origin(), self.leaf.origin()) + + +class TestZeroCopyViewLifetimes(unittest.TestCase): + """Returned typed grids, trees, leaves, NodeManagers, and zero-copy + NumPy views must keep their backing buffers alive across the chained + temporary expressions used at the call site (e.g. + `nanovdb.tools.createFogVolumeSphere().grid().tree().getFirstLeaf().values()`). + Without explicit nb::keep_alive linkages the intermediate handle gets + GC'd between expressions and the returned object reads freed memory. + """ + + def _force_gc(self): + import gc + for _ in range(3): + gc.collect() + + def test_handle_grid_temporary(self): + g = nanovdb.tools.createFogVolumeSphere(name="probe").grid() + self._force_gc() + self.assertEqual(g.gridName(), "probe") + + def test_handle_grid_tree_leaf_values_chain(self): + vals = (nanovdb.tools.createFogVolumeSphere() + .grid().tree().getFirstLeaf().values()) + self._force_gc() + # Touching the view shouldn't crash. + self.assertEqual(vals.shape, (512,)) + _ = float(vals[0]) + + def test_grid_leaf_values_temporary(self): + bulk = nanovdb.tools.createFogVolumeSphere().grid().leaf_values() + self._force_gc() + self.assertEqual(bulk.shape[1], 512) + _ = float(bulk[0, 0]) + + def test_node_manager_temporary_grid(self): + nm = nanovdb.createNodeManager( + nanovdb.tools.createFogVolumeSphere().grid()).mgr() + self._force_gc() + self.assertGreater(nm.leafCount(), 0) + leaf0_vals = nm.leaf(0).values() + self._force_gc() + self.assertEqual(leaf0_vals.shape, (512,)) + + def test_blind_data_temporary(self): + # The grid has no blind data so getBlindData returns None — what we + # care about here is that the temporary chain doesn't segfault. + result = nanovdb.tools.createFogVolumeSphere().grid().getBlindData(0) + self._force_gc() + self.assertIsNone(result) + + class TestGridMetaDataGuards(unittest.TestCase): - """Copilot review #3: GridMetaData ctor + safeCast guard against bad input.""" + """GridMetaData() constructor and safeCast() reject bad input (None, a + Grid wrapping an invalid buffer) with a Python exception or False + rather than asserting / null-dereferencing inside NanoVDB.""" def test_init_rejects_none(self): # nanobind's type system rejects None for const GridData* before our From 31838423750d16da05dde999cb91c3e9a26251e5 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 21 May 2026 16:20:55 +1200 Subject: [PATCH 05/48] =?UTF-8?q?nanovdb=20python:=20VoxelBlockManager=20(?= =?UTF-8?q?host)=20=E2=80=94=20Phase=203=20follow-up=20(#2213)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: VoxelBlockManager (host) — Phase 3 follow-up Phase 3 deferred the VoxelBlockManager surface; this picks it up before moving on to Phase 4. Adds host-side bindings for everything under nanovdb::tools::VoxelBlockManager*, plus a minimal createOnIndexGrid scaffold needed to build OnIndex grids from Python (the broader createNanoGrid binding lands in Phase 5). New module surface, all under nanovdb.tools: - VoxelBlockManagerHandle (host) — owns the firstLeafID + jumpMap metadata buffers; exposes blockCount(), firstOffset(), lastOffset(), reset(), __bool__. Buffers exposed as zero-copy NumPy views: firstLeafID() -> (blockCount,) uint32 jumpMap(jump_map_length=1) -> (blockCount, jump_map_length) uint64 jumpMap takes a jump_map_length argument because the value depends on log2_block_width (= 1 << (log2_block_width - 6)) and isn't stored on the handle. handle.decodeBlock(grid, i, log2_block_width=6) is a convenience method that slices firstLeafID / jumpMap for block i and calls decodeInverseMaps internally. - buildVoxelBlockManager(grid, log2_block_width=6, first_offset=0, last_offset=0, n_blocks=0) — runtime switch over Log2BlockWidth ∈ {6, 7, 8, 9} (BlockWidth 64/128/256/512) dispatching to the right template instantiation. Rejects non-OnIndex grids with TypeError; rejects out-of-range log2_block_width with ValueError. - decodeInverseMaps(grid, first_leaf_id, jump_map, block_first_offset, log2_block_width=6) — free function. jump_map is a uint64 NumPy array of length JumpMapLength (= 1 << (log2_block_width - 6)); returns (leaf_index, voxel_offset) freshly-allocated NumPy arrays (uint32, uint16) of length BlockWidth. - createOnIndexGrid(src_grid, channels=0, include_stats=True, include_tiles=True, verbose=0) — minimal test-scaffold factory that binds tools::createNanoGrid. Accepts FloatGrid, DoubleGrid, Int32Grid, Vec3fGrid sources. Required for end-to-end VBM testing since no other path produces an OnIndex grid from Python today. The full createNanoGrid surface remains scoped to Phase 5. Defensive checks: - decodeBlock validates firstLeafID[block_index] is in [0, leafCount) before passing it into the C++ decodeInverseMaps. The underlying NanoVDB algorithm doesn't always initialize firstLeafID — blocks that no leaf's iteration sweep reaches (e.g. on tile-compressed OnIndex grids where some sequential offsets correspond to tile values rather than leaf voxels) are left with uninitialized memory. Without the guard, decodeInverseMaps would read tree.getFirstNode<0>()[garbage] and crash; the guard converts that into a Python ValueError with a clear message pointing at the workaround (build the source grid voxel-by-voxel via build::Grid so it stays uncompressed). - All entry points reject out-of-range indices, levels, and grid build types up front (TypeError / ValueError / IndexError). Tests under TestVoxelBlockManager exercise: - createOnIndexGrid produces an OnIndex / IndexGrid / sequential grid - Buffer shapes and dtypes (firstLeafID, jumpMap default + reshape) - decodeBlock(0) returns (uint32, uint16) arrays of length BlockWidth - decodeInverseMaps free function agrees with handle.decodeBlock - Out-of-range block_index raises IndexError - Out-of-range log2_block_width (5, 10) raises ValueError - Non-OnIndex grid argument raises TypeError - createOnIndexGrid(None) raises TypeError End-to-end decode verification across every block is intentionally deferred until Phase 4's build::Grid bindings land — that's the only host-side path to construct a tile-free OnIndex grid where every block's firstLeafID is reliably initialized by the current upstream algorithm. Verified locally: - CPU build: 84 tests, 75 pass + 8 new VBM tests, 1 pre-existing test_read_write_grid BLOSC env-dependent failure unrelated to this PR. - CUDA sm_120 + BLOSC + ZLIB: 84/84 pass. No new lines over 100 cols. Signed-off-by: Jonathan Swartz * nanovdb python: skip ndarray tests cleanly when numpy is missing Windows CI surfaced three test errors on #2213 from TestZeroCopyViewLifetimes — the test methods invoked .values() / .leaf_values() directly without an `import numpy` guard, so on environments without numpy installed (e.g. the Windows runner) the bindings raised: TypeError: could not export nanobind::ndarray: ModuleNotFoundError: No module named 'numpy' The other ndarray-touching tests (test_leaf_values_zero_copy, test_bulk_leaf_values, test_node_manager_round_trip, the VBM test_decode_block_zero, etc.) already guard with `import numpy` and call self.skipTest. The three lifetime tests just skipped that guard. Added the same try/except ImportError + skipTest pattern to: - TestZeroCopyViewLifetimes.test_handle_grid_tree_leaf_values_chain - TestZeroCopyViewLifetimes.test_grid_leaf_values_temporary - TestZeroCopyViewLifetimes.test_node_manager_temporary_grid While I was here, also dropped an unnecessary numpy guard on TestVoxelBlockManager.test_create_on_index_grid_rejects_unsupported_source — that test only checks TypeError on a non-grid input and doesn't touch numpy, so it can run unconditionally. Verified locally with both a numpy-enabled venv (84/84 pass minus the pre-existing BLOSC env failure) and a numpy-less venv (84 ran, 17 skipped, no new errors). Signed-off-by: Jonathan Swartz * nanovdb python: install numpy on Windows CI so ndarray tests run The Windows job for the NanoVDB workflow was missing the install_numpy step that the main openvdb build.yml and weekly.yml workflows already run after install_windows.ps1. As a result the vcpkg-supplied Python on the Windows runner had no numpy, so every test that touches nb::ndarray either errored (before #2213's skip guards) or now skips silently. Add the install_numpy step using the existing ci/install_windows_numpy.ps1 helper, matching the pattern already in build.yml. The Windows runner will then actually execute the ndarray-touching VoxelBlockManager / Tree / GridHandle tests instead of skipping them. Signed-off-by: Jonathan Swartz * nanovdb python: address Copilot review on #2213 Four issues raised by Copilot on the open VBM follow-up PR, all real: 1. VoxelBlockManagerHandle.jumpMap(jump_map_length) accepted any caller-supplied length and used it to shape a zero-copy NumPy view over hostJumpMap(). Since the underlying buffer was sized as blockCount * JumpMapLength (where JumpMapLength is derived from the log2_block_width the handle was BUILT with), a caller passing a larger value produced an ndarray whose elements lived past the end of the allocated buffer — an OOB read on access. Fix: wrap VoxelBlockManagerHandle in a small PyVBMHandle struct that also stores the log2_block_width. jumpMap() takes no arguments and derives JumpMapLength from the stored width, so the returned view always matches the buffer exactly. decodeBlock() no longer accepts a log2_block_width either, removing the equivalent mismatch hazard there. The new PyVBMHandle exposes log2_block_width / block_width / jump_map_length as read-only properties for introspection. 2. decodeInverseMaps() did not validate first_leaf_id against grid.tree().nodeCount(0). VoxelBlockManager::decodeInverseMaps indexes tree.getFirstNode<0>()[first_leaf_id] unconditionally, so a stray ID produced an OOB read of the leaf array. Validate up front and raise IndexError. 3. buildVoxelBlockManager() did not enforce grid.isSequential() (a precondition guarded only by NANOVDB_ASSERT, which is a no-op in release) nor that first_offset == 1 (mod BlockWidth). Validate both at the Python boundary and raise ValueError; the zero first_offset default still flows through unchanged because the C++ helper normalizes it to 1 itself. 4. The zero-copy shape/dtype test exercised vbm.jumpMap(jump_map_length=2) on a handle built with log2_block_width=6 — exactly the OOB case (1) was about. Rewrite the test to build a second handle with log2_block_width=7 and verify its jumpMap shape is (blockCount, 2), confirming the shape now follows the handle. Drop the obsolete log2_block_width=6 kwarg from decodeBlock. Add two new tests: misaligned first_offset is rejected, and an out-of-range first_leaf_id on decodeInverseMaps raises IndexError. Signed-off-by: Jonathan Swartz * nanovdb python: empty-handle guards + accurate ownership comment Two follow-up items from Copilot on #2213: 1. firstLeafID() and jumpMap() called hostFirstLeafID() / hostJumpMap() on the underlying VoxelBlockManagerHandle and fed the result straight into nb::ndarray, but both accessors legally return nullptr on a default-constructed or reset() handle (blockCount() == 0). Passing nullptr into nb::ndarray is unsafe even with a zero leading shape. Mirror the pattern PyTree.h uses for the empty-grid leaf_values() case: when the buffer is null, substitute a non-null dummy pointer (the handle itself) so nanobind has something to base the empty ndarray on; nothing is read because the leading shape is 0. Add tests for both default-constructed and reset() handles. 2. The pyDecodeInverseMapsImpl comment described nanobind as allocating "fresh memory through numpy", which doesn't match the implementation (which uses new[] + a capsule with delete[] as deleter). Rewrite the comment to describe the actual ownership model so future maintainers aren't misled. Signed-off-by: Jonathan Swartz * nanovdb python: VBM exception-safety + firstLeafID sentinel prefill Two more items from Copilot on #2213: 1. pyDecodeInverseMapsImpl allocated leafIndex / voxelOffset with raw new[] and only wrapped them in nb::capsule after several intervening operations (a second new[], the decodeInverseMaps call itself, and the first capsule's own construction). If anything in that window threw, the half-built state leaked. Hold the raw arrays in std::unique_ptr until each capsule has been constructed, then release() so ownership transfers cleanly; any throw during that sequence now unwinds without leaking. 2. The C++ allocating overload of buildVoxelBlockManager uses HostBuffer::create (uninitialized malloc) for firstLeafID, then touches only the slots its iteration sweep reaches. Blocks the algorithm doesn't visit retain arbitrary values; the existing decodeBlock guard (firstLeafID >= nLeaves) catches values past the leaf array but cannot tell garbage that happens to be < nLeaves from a real leaf id, so a low garbage byte would silently decode against the wrong leaf. Switch the Python binding to allocate the metadata buffers itself, prefill every firstLeafID slot with the sentinel value `nLeaves`, then call the in-place buildVoxelBlockManager(grid, handle) overload. The in-place builder zeros the jumpMap and only writes firstLeafID slots it visits, so every untouched slot keeps the sentinel and deterministically trips the decodeBlock guard. Add test_untouched_blocks_trip_sentinel_guard, which sweeps every block of the cube VBM and asserts each firstLeafID slot is either a real leaf id (< nLeaves) or exactly the sentinel — never any other value. Signed-off-by: Jonathan Swartz * nanovdb python: VBM popcount upper-bound + n_blocks validation Two more items from Copilot on #2213: 1. The decodeBlock guard only checked firstLeafID < nLeaves, but the C++ decodeInverseMaps loops leafID = firstLeafID .. firstLeafID + nExtraLeaves where nExtraLeaves is the popcount of this block's jumpMap (each set bit marks an additional leaf boundary crossed within the block). With a corrupt jumpMap or a handle paired with a different grid, the loop could read past tree.getFirstNode<0>() even with a valid firstLeafID. Hoist a popcount-based upper-bound check into pyDecodeInverseMapsImpl: compute nExtraLeaves locally and raise ValueError if firstLeafID + nExtraLeaves >= grid.tree().nodeCount(0). Because both the handle.decodeBlock() and the free-function decodeInverseMaps() funnel through this impl, both paths are covered without per-caller duplication. 2. buildVoxelBlockManager accepted an explicit n_blocks but didn't validate it against the documented precondition n_blocks >= ceil((last_offset - first_offset + 1) / BlockWidth). A smaller value produced a handle whose blockCount() < the coverage implied by lastOffset, silently truncating later decodeBlock sweeps. Validate when nonzero and raise ValueError with the minimum required value in the message. New test_build_voxel_block_manager_rejects_undersized_n_blocks covers the n_blocks=1 case on the cube VBM. Signed-off-by: Jonathan Swartz * nanovdb python: explicit std headers in PyVoxelBlockManager.cc Copilot noted PyVoxelBlockManager.cc uses std::integral_constant, std::string/std::to_string, and std::move but relied on transitive includes from nanobind / NanoVDB headers to bring those in. That works on the toolchains we currently test but is fragile against stricter ones. Add explicit , , and includes; drop and , neither of which the translation unit actually uses now. Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- .github/workflows/nanovdb.yml | 3 + nanovdb/nanovdb/python/CMakeLists.txt | 1 + nanovdb/nanovdb/python/NanoVDBModule.cc | 2 + nanovdb/nanovdb/python/PyVoxelBlockManager.cc | 522 ++++++++++++++++++ nanovdb/nanovdb/python/PyVoxelBlockManager.h | 20 + nanovdb/nanovdb/python/test/TestNanoVDB.py | 240 ++++++++ 6 files changed, 788 insertions(+) create mode 100644 nanovdb/nanovdb/python/PyVoxelBlockManager.cc create mode 100644 nanovdb/nanovdb/python/PyVoxelBlockManager.h diff --git a/.github/workflows/nanovdb.yml b/.github/workflows/nanovdb.yml index 5b5b5d59ae..442b88e9fb 100644 --- a/.github/workflows/nanovdb.yml +++ b/.github/workflows/nanovdb.yml @@ -115,6 +115,9 @@ jobs: - name: install shell: powershell run: .\ci\install_windows.ps1 + - name: install_numpy + shell: powershell + run: .\ci\install_windows_numpy.ps1 - name: build # nvcc doesn't set _WIN32 when run in bash so we need to set it manually shell: bash diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 714e53360f..2dba4f661d 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -29,6 +29,7 @@ nanobind_add_module(nanovdb_python NB_STATIC PySampleFromVoxels.cc PyTools.cc PyTree.cc + PyVoxelBlockManager.cc cuda/PyDeviceBuffer.cc cuda/PyDeviceGridHandle.cu cuda/PyPointsToGrid.cu diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 57d0a1b3f6..334c101b96 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -21,6 +21,7 @@ #include "PyMath.h" #include "PyTools.h" #include "PyTree.h" +#include "PyVoxelBlockManager.h" #include "PyGridChecksum.h" namespace nb = nanobind; @@ -806,6 +807,7 @@ NB_MODULE(nanovdb, m) nb::module_ toolsModule = m.def_submodule("tools"); toolsModule.doc() = "A submodule that implements tools for NanoVDB grids"; defineToolsModule(toolsModule); + defineVoxelBlockManagerModule(toolsModule); nb::module_ ioModule = m.def_submodule("io"); ioModule.doc() = "A submodule that implements I/O functionality for NanoVDB grids"; diff --git a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc new file mode 100644 index 0000000000..ba1546d05b --- /dev/null +++ b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc @@ -0,0 +1,522 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyVoxelBlockManager.h" + +#include +#include + +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; +using namespace nanovdb; +using nanovdb::tools::VoxelBlockManager; +using nanovdb::tools::VoxelBlockManagerBase; +using nanovdb::tools::VoxelBlockManagerHandle; +using nanovdb::tools::buildVoxelBlockManager; + +namespace pynanovdb { + +// ----------------------- Log2BlockWidth dispatch -------------------------- +// +// Log2BlockWidth is a compile-time template parameter on every VBM helper. +// We expose it to Python as a runtime int and dispatch via a switch that +// instantiates the four useful widths (BlockWidth = 64, 128, 256, 512). +// Larger widths are not bound by default; callers who need them can add a +// new case below. + +template +static auto dispatchLog2BlockWidth(int log2BlockWidth, F&& fn) +{ + switch (log2BlockWidth) { + case 6: return fn(std::integral_constant{}); + case 7: return fn(std::integral_constant{}); + case 8: return fn(std::integral_constant{}); + case 9: return fn(std::integral_constant{}); + default: + throw nb::value_error( + "VoxelBlockManager: log2_block_width must be 6, 7, 8, or 9 " + "(BlockWidth = 64, 128, 256, or 512). Larger widths are not " + "bound in Python by default."); + } +} + +// PyVBMHandle wraps the C++ VoxelBlockManagerHandle and carries the +// log2_block_width the handle was built with. The C++ handle does NOT store +// log2_block_width itself, so without this wrapper the Python binding would +// have to ask the caller every time — which the user can lie about and +// trigger out-of-bounds reads of the metadata buffers. Storing it once at +// build time and consulting it in every accessor closes that hole. +struct PyVBMHandle +{ + VoxelBlockManagerHandle handle; + int log2BlockWidth = 6; + + PyVBMHandle() = default; + PyVBMHandle(VoxelBlockManagerHandle&& h, int lbw) noexcept + : handle(std::move(h)), log2BlockWidth(lbw) {} + + PyVBMHandle(const PyVBMHandle&) = delete; + PyVBMHandle& operator=(const PyVBMHandle&) = delete; + PyVBMHandle(PyVBMHandle&&) = default; + PyVBMHandle& operator=(PyVBMHandle&&) = default; + + uint64_t blockCount() const { return handle.blockCount(); } + uint64_t firstOffset() const { return handle.firstOffset(); } + uint64_t lastOffset() const { return handle.lastOffset(); } + void reset() { handle.reset(); } + int blockWidth() const { return 1 << log2BlockWidth; } + int jumpMapLength() const { return 1 << (log2BlockWidth - 6); } +}; + +// ----------------------- decodeInverseMaps helper ------------------------- +// +// Common implementation used by both the free function and the +// handle.decodeBlock(i) method. Allocates fresh leafIndex (uint32) and +// voxelOffset (uint16) NumPy arrays of length BlockWidth and fills them. +template +static nb::object pyDecodeInverseMapsImpl(const NanoGrid& grid, + uint32_t firstLeafID, + const uint64_t* jumpMap, + uint64_t blockFirstOffset) +{ + constexpr int BlockWidth = 1 << Log2BlockWidth; + constexpr int JumpMapLength = + VoxelBlockManagerBase::JumpMapLength; + + // The C++ decodeInverseMaps iterates leafID = firstLeafID .. + // firstLeafID + nExtraLeaves, where nExtraLeaves is the popcount of + // this block's jumpMap (each set bit marks an additional leaf + // boundary crossed within the block). If the jumpMap is corrupt or + // was built against a different grid, the loop could read past + // tree.getFirstNode<0>(). Pre-compute the upper bound and validate + // it against grid.tree().nodeCount(0) before any allocation. + uint32_t nExtraLeaves = 0; + for (int i = 0; i < JumpMapLength; ++i) + nExtraLeaves += util::countOn(jumpMap[i]); + const uint32_t nLeaves = grid.tree().nodeCount(0); + if (uint64_t(firstLeafID) + uint64_t(nExtraLeaves) >= nLeaves) { + throw nb::value_error( + "decodeInverseMaps: firstLeafID + popcount(jumpMap) would " + "index past grid.tree().nodeCount(0) — the jumpMap is " + "either corrupt or was paired with a different grid."); + } + + // Each call allocates fresh BlockWidth-sized output arrays for the + // leaf-index and voxel-offset results. We use plain new[] (rather than + // a numpy-allocated buffer) because the produced ndarrays are returned + // by reference and Python owns them via the capsule deleters below — + // when the ndarray is destroyed, the capsule's deleter runs delete[]. + // + // The raw pointers live in std::unique_ptr until the matching capsule + // has been constructed; that way if the second allocation, the + // decodeInverseMaps call, or either capsule construction throws, the + // unique_ptr unwinds the half-built state cleanly. After a capsule + // takes ownership we release() so the unique_ptr no longer double-frees. + std::unique_ptr leafIndex(new uint32_t[BlockWidth]); + std::unique_ptr voxelOffset(new uint16_t[BlockWidth]); + + using VBM = VoxelBlockManager; + VBM::template decodeInverseMaps( + &grid, firstLeafID, jumpMap, blockFirstOffset, + leafIndex.get(), voxelOffset.get()); + + // nb::capsule wraps the raw pointer + matching delete[] so it can serve + // as the ndarray's owner — the capsule lives as long as the ndarray and + // its destruction runs the deleter. + nb::capsule leafOwner(leafIndex.get(), + [](void* p) noexcept { delete[] static_cast(p); }); + auto* leafRaw = leafIndex.release(); + nb::capsule offsetOwner(voxelOffset.get(), + [](void* p) noexcept { delete[] static_cast(p); }); + auto* offsetRaw = voxelOffset.release(); + + size_t shape[1] = {static_cast(BlockWidth)}; + nb::ndarray, nb::c_contig, nb::device::cpu> + leafArr(leafRaw, size_t(1), shape, leafOwner); + nb::ndarray, nb::c_contig, nb::device::cpu> + offsetArr(offsetRaw, size_t(1), shape, offsetOwner); + return nb::make_tuple( + nb::cast(leafArr, nb::rv_policy::reference), + nb::cast(offsetArr, nb::rv_policy::reference)); +} + +// ------------------- VoxelBlockManagerHandle binding ---------------------- + +static const NanoGrid* castOnIndexGrid(nb::handle py_grid, + const char* fn_name) +{ + if (!nb::isinstance>(py_grid)) { + std::string msg(fn_name); + msg += ": grid must be a NanoVDB grid of build type ValueOnIndex (OnIndexGrid)"; + throw nb::type_error(msg.c_str()); + } + return &nb::cast&>(py_grid); +} + +static void defineHandle(nb::module_& toolsModule) +{ + nb::class_(toolsModule, "VoxelBlockManagerHandle", + "Owns the firstLeafID / jumpMap metadata buffers backing a " + "VoxelBlockManager. Constructed by nanovdb.tools.buildVoxelBlockManager.") + .def(nb::init<>()) + .def("blockCount", &PyVBMHandle::blockCount) + .def("firstOffset", &PyVBMHandle::firstOffset) + .def("lastOffset", &PyVBMHandle::lastOffset) + .def("reset", &PyVBMHandle::reset) + .def_prop_ro("log2_block_width", [](const PyVBMHandle& h) { return h.log2BlockWidth; }, + "The log2_block_width this handle was built with. The jumpMap " + "and decodeBlock outputs derive their shapes from this value.") + .def_prop_ro("block_width", &PyVBMHandle::blockWidth, + "BlockWidth = 1 << log2_block_width (64, 128, 256, or 512).") + .def_prop_ro("jump_map_length", &PyVBMHandle::jumpMapLength, + "JumpMapLength = BlockWidth / 64 (1, 2, 4, or 8).") + .def( + "__bool__", + [](const PyVBMHandle& h) { return h.blockCount() > 0; }, + nb::is_operator()) + // Zero-copy view of the (blockCount,) firstLeafID array. + .def("firstLeafID", + [](nb::handle py_self) -> nb::object { + auto& h = nb::cast(py_self); + size_t shape[1] = {static_cast(h.blockCount())}; + // A default-constructed or reset() handle has a null + // hostFirstLeafID(); we still return an empty (0,) ndarray + // so callers don't have to branch on a None sentinel. The + // dummy non-null pointer (the handle itself) keeps nanobind + // happy; nothing is read since the leading shape is 0. + uint32_t* raw = h.handle.hostFirstLeafID(); + void* data = (raw != nullptr) ? static_cast(raw) + : static_cast(&h); + return nb::cast( + nb::ndarray, + nb::c_contig, nb::device::cpu>( + data, size_t(1), shape, py_self), + nb::rv_policy::reference); + }, + nb::keep_alive<0, 1>(), + "Return a zero-copy (blockCount,) uint32 NumPy view of the " + "firstLeafID array. Returns an empty (0,) array on a " + "default-constructed or reset() handle. The view keeps this " + "handle alive.") + // jumpMap is uint64_t[blockCount * JumpMapLength]. JumpMapLength is + // determined by the log2_block_width recorded on the handle, not by + // the caller — that way the returned view always covers exactly the + // allocated buffer, with no risk of OOB reads. + .def("jumpMap", + [](nb::handle py_self) -> nb::object { + auto& h = nb::cast(py_self); + size_t shape[2] = {static_cast(h.blockCount()), + static_cast(h.jumpMapLength())}; + // Same null-buffer guard as firstLeafID(): a + // default-constructed / reset() handle has a null + // hostJumpMap(); return an empty (0, jump_map_length) + // ndarray rather than passing nullptr to nanobind. + uint64_t* raw = h.handle.hostJumpMap(); + void* data = (raw != nullptr) ? static_cast(raw) + : static_cast(&h); + return nb::cast( + nb::ndarray, + nb::c_contig, nb::device::cpu>( + data, size_t(2), shape, py_self), + nb::rv_policy::reference); + }, + nb::keep_alive<0, 1>(), + "Return a zero-copy (blockCount, jump_map_length) uint64 NumPy " + "view of the jumpMap. The shape is determined by the " + "log2_block_width the handle was built with. Returns an empty " + "(0, jump_map_length) array on a default-constructed or reset() " + "handle. The view keeps this handle alive.") + // Decode the inverse maps for a single block of this VBM. The + // log2_block_width is taken from the handle, so the caller cannot + // request a width that doesn't match what was built. + .def("decodeBlock", + [](PyVBMHandle& self, + nb::handle py_grid, + uint64_t block_index) -> nb::object { + const auto* grid = castOnIndexGrid(py_grid, + "VoxelBlockManagerHandle.decodeBlock"); + if (block_index >= self.blockCount()) { + throw nb::index_error( + "VoxelBlockManagerHandle.decodeBlock(block_index): " + "block_index out of range [0, blockCount)."); + } + // Defensive: NanoVDB's buildVoxelBlockManager doesn't always + // initialize firstLeafID for blocks where no leaf starts at + // a block boundary AND no leaf's iteration sweep reaches + // them (e.g. when the source grid is tile-compressed, so + // some sequential offsets correspond to tile values rather + // than leaf voxels). The slot is then uninitialized memory; + // passing it into decodeInverseMaps would lead to an OOB + // read of tree.getFirstNode<0>()[garbage]. Catch the case + // and raise rather than segfault. + const uint32_t firstLeafID = + self.handle.hostFirstLeafID()[block_index]; + const uint32_t nLeaves = grid->tree().nodeCount(0); + if (firstLeafID >= nLeaves) { + throw nb::value_error( + "VoxelBlockManagerHandle.decodeBlock: the VBM's " + "firstLeafID for this block was not initialized by " + "buildVoxelBlockManager (the underlying algorithm " + "doesn't cover blocks that no leaf reaches via its " + "iteration). This typically happens on OnIndex " + "grids built from tile-compressed source grids; " + "until the issue is fixed upstream the workaround " + "is to build the source grid voxel-by-voxel with " + "build::Grid so it stays uncompressed."); + } + return dispatchLog2BlockWidth(self.log2BlockWidth, [&](auto W) { + constexpr int LBW = decltype(W)::value; + constexpr int BlockWidth = 1 << LBW; + constexpr int JumpMapLength = + VoxelBlockManagerBase::JumpMapLength; + const uint64_t blockFirstOffset = + self.firstOffset() + block_index * BlockWidth; + return pyDecodeInverseMapsImpl( + *grid, + firstLeafID, + self.handle.hostJumpMap() + block_index * JumpMapLength, + blockFirstOffset); + }); + }, + "grid"_a, "block_index"_a, + "Decode the inverse maps for the block_index-th block of this " + "VBM. Returns (leaf_index, voxel_offset) uint32 / uint16 NumPy " + "arrays of length BlockWidth = 1< PyVBMHandle { + const auto* grid = castOnIndexGrid(py_grid, "buildVoxelBlockManager"); + // The C++ implementation only NANOVDB_ASSERTs these preconditions, + // which makes them no-ops in release builds. Validate them here + // so Python callers get a clear error instead of UB / abort. + if (!grid->isSequential()) { + throw nb::value_error( + "buildVoxelBlockManager: grid must satisfy " + "grid.isSequential() (fixed-size, breadth-first node " + "layout). NanoVDB grids constructed via " + "tools.createOnIndexGrid satisfy this by default."); + } + return dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + constexpr int LBW = decltype(W)::value; + using Base = VoxelBlockManagerBase; + constexpr uint64_t BlockWidth = Base::BlockWidth; + constexpr uint64_t JumpMapLength = Base::JumpMapLength; + // first_offset must be 1 (mod BlockWidth). The single-arg + // C++ helper would normalize a zero input to 1; we do the + // same here so the in-place builder below sees a valid + // value. Validate the nonzero case ourselves. + if (first_offset != 0 && + ((first_offset - 1) & (BlockWidth - 1)) != 0) { + throw nb::value_error( + "buildVoxelBlockManager: first_offset must satisfy " + "first_offset == 1 (mod BlockWidth). Pass 0 (the " + "default) to let the implementation use 1."); + } + if (first_offset == 0) first_offset = 1; + if (last_offset == 0) last_offset = grid->activeVoxelCount(); + if (last_offset < first_offset) return PyVBMHandle(); + // Capacity must hold at least ceil((last - first + 1) / + // BlockWidth) blocks; otherwise the handle's lastOffset + // would advertise more coverage than blockCount allows + // and decodeBlock would silently truncate. The formula + // below equals the ceil() above when BlockWidth is a + // power of two. + const uint64_t minBlocks = + (last_offset - first_offset + BlockWidth) >> LBW; + if (n_blocks != 0 && n_blocks < minBlocks) { + std::string msg( + "buildVoxelBlockManager: n_blocks must be at " + "least ceil((last_offset - first_offset + 1) / " + "BlockWidth) = "); + msg += std::to_string(minBlocks); + msg += ". Pass 0 (the default) to use the minimum " + "required capacity."; + throw nb::value_error(msg.c_str()); + } + if (n_blocks == 0) n_blocks = minBlocks; + // Allocate the metadata buffers ourselves so we can + // pre-initialize firstLeafID with a sentinel value before + // the in-place builder runs. The C++ allocating overload + // calls HostBuffer::create() which returns uninitialized + // memory; blocks that the algorithm doesn't touch would + // then retain arbitrary values, and our decodeBlock guard + // (firstLeafID >= nLeaves) might miss any garbage value + // that happens to be < nLeaves. By prefilling with nLeaves + // up front, every untouched slot deterministically trips + // the guard. + auto firstLeafIDBuf = HostBuffer::create( + n_blocks * sizeof(uint32_t)); + auto jumpMapBuf = HostBuffer::create( + n_blocks * JumpMapLength * sizeof(uint64_t)); + const uint32_t nLeaves = grid->tree().nodeCount(0); + { + uint32_t* slots = static_cast( + firstLeafIDBuf.data()); + for (uint64_t i = 0; i < n_blocks; ++i) { + slots[i] = nLeaves; + } + } + VoxelBlockManagerHandle handle( + std::move(firstLeafIDBuf), std::move(jumpMapBuf), + n_blocks, first_offset, last_offset); + // In-place builder zeros the jumpMap itself and only + // touches firstLeafID slots it actually visits. + buildVoxelBlockManager(grid, handle); + return PyVBMHandle(std::move(handle), LBW); + }); + }, + "grid"_a, + "log2_block_width"_a = 6, + "first_offset"_a = 0, + "last_offset"_a = 0, + "n_blocks"_a = 0, + "Build a host-side VoxelBlockManager from an OnIndexGrid. " + "log2_block_width selects the per-block active-voxel count " + "(6=64, 7=128, 8=256, 9=512). Pass 0 for first_offset / " + "last_offset / n_blocks to use the full grid (first active " + "voxel through grid.activeVoxelCount(), minimum block count). " + "first_offset, if nonzero, must satisfy first_offset == 1 " + "(mod BlockWidth)."); +} + +// --------------------- decodeInverseMaps binding -------------------------- + +static void defineDecode(nb::module_& toolsModule) +{ + toolsModule.def("decodeInverseMaps", + [](nb::handle py_grid, + uint32_t first_leaf_id, + nb::ndarray, + nb::c_contig, nb::device::cpu> jump_map, + uint64_t block_first_offset, + int log2_block_width) -> nb::object { + const auto* grid = castOnIndexGrid(py_grid, "decodeInverseMaps"); + // The C++ helper indexes tree.getFirstNode<0>()[first_leaf_id] + // without a bounds check, so a stray first_leaf_id leads to an + // OOB read. Validate up front. (We also require isSequential(); + // getFirstNode only makes sense on a sequential tree.) + if (!grid->isSequential()) { + throw nb::value_error( + "decodeInverseMaps: grid must satisfy " + "grid.isSequential()."); + } + const uint32_t nLeaves = grid->tree().nodeCount(0); + if (first_leaf_id >= nLeaves) { + throw nb::index_error( + "decodeInverseMaps: first_leaf_id out of range " + "[0, grid.tree().nodeCount(0))."); + } + return dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + constexpr int LBW = decltype(W)::value; + constexpr int JumpMapLength = + VoxelBlockManagerBase::JumpMapLength; + if (jump_map.shape(0) != JumpMapLength) { + std::string msg("decodeInverseMaps: jump_map must have " + "length JumpMapLength = "); + msg += std::to_string(JumpMapLength); + msg += " for log2_block_width="; + msg += std::to_string(LBW); + throw nb::value_error(msg.c_str()); + } + return pyDecodeInverseMapsImpl( + *grid, first_leaf_id, jump_map.data(), + block_first_offset); + }); + }, + "grid"_a, + "first_leaf_id"_a, + "jump_map"_a, + "block_first_offset"_a, + "log2_block_width"_a = 6, + "Decode the inverse maps for a single voxel block of an OnIndexGrid. " + "Returns a (leaf_index, voxel_offset) tuple of fresh NumPy arrays of " + "length BlockWidth = 1< +static nb::object tryCreateOnIndexGrid(nb::handle py_grid, + uint32_t channels, + bool include_stats, + bool include_tiles, + int verbose) +{ + using SrcGridT = NanoGrid; + if (!nb::isinstance(py_grid)) { + return nb::object(); + } + const SrcGridT& src = nb::cast(py_grid); + return nb::cast( + tools::createNanoGrid( + src, channels, include_stats, include_tiles, verbose)); +} + +static void defineCreateOnIndexGrid(nb::module_& toolsModule) +{ + toolsModule.def("createOnIndexGrid", + [](nb::handle py_grid, + uint32_t channels, + bool include_stats, + bool include_tiles, + int verbose) -> nb::object { + // Try every source BuildT we accept. + if (auto r = tryCreateOnIndexGrid( + py_grid, channels, include_stats, include_tiles, verbose); + r.is_valid()) return r; + if (auto r = tryCreateOnIndexGrid( + py_grid, channels, include_stats, include_tiles, verbose); + r.is_valid()) return r; + if (auto r = tryCreateOnIndexGrid( + py_grid, channels, include_stats, include_tiles, verbose); + r.is_valid()) return r; + if (auto r = tryCreateOnIndexGrid( + py_grid, channels, include_stats, include_tiles, verbose); + r.is_valid()) return r; + throw nb::type_error( + "createOnIndexGrid: source grid must be a FloatGrid, " + "DoubleGrid, Int32Grid, or Vec3fGrid (other source BuildTs " + "are not yet bound)."); + }, + "src_grid"_a, + "channels"_a = 0u, + "include_stats"_a = true, + "include_tiles"_a = true, + "verbose"_a = 0, + "Convert a source grid into a NanoGrid " + "(OnIndexGrid). Accepts FloatGrid / DoubleGrid / Int32Grid / " + "Vec3fGrid. Required for constructing inputs to " + "buildVoxelBlockManager. The broader createNanoGrid surface lands in a later phase."); +} + +void defineVoxelBlockManagerModule(nb::module_& toolsModule) +{ + defineHandle(toolsModule); + defineBuild(toolsModule); + defineDecode(toolsModule); + defineCreateOnIndexGrid(toolsModule); +} + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyVoxelBlockManager.h b/nanovdb/nanovdb/python/PyVoxelBlockManager.h new file mode 100644 index 0000000000..6fe29570c6 --- /dev/null +++ b/nanovdb/nanovdb/python/PyVoxelBlockManager.h @@ -0,0 +1,20 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_PYVOXELBLOCKMANAGER_HAS_BEEN_INCLUDED +#define NANOVDB_PYVOXELBLOCKMANAGER_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +/// @brief Bind VoxelBlockManagerHandle, +/// tools.buildVoxelBlockManager, tools.decodeInverseMaps, and the +/// test-scaffold tools.createOnIndexGrid factory under the given +/// Python submodule (expected to be the existing nanovdb.tools). +void defineVoxelBlockManagerModule(nb::module_& toolsModule); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index f0fc2dba60..0f0807633d 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -742,6 +742,12 @@ def test_handle_grid_temporary(self): self.assertEqual(g.gridName(), "probe") def test_handle_grid_tree_leaf_values_chain(self): + # nb::ndarray requires numpy at runtime, so the + # binding raises TypeError if numpy isn't installed. Skip then. + try: + import numpy # noqa: F401 + except ImportError: + self.skipTest("numpy not installed") vals = (nanovdb.tools.createFogVolumeSphere() .grid().tree().getFirstLeaf().values()) self._force_gc() @@ -750,12 +756,20 @@ def test_handle_grid_tree_leaf_values_chain(self): _ = float(vals[0]) def test_grid_leaf_values_temporary(self): + try: + import numpy # noqa: F401 + except ImportError: + self.skipTest("numpy not installed") bulk = nanovdb.tools.createFogVolumeSphere().grid().leaf_values() self._force_gc() self.assertEqual(bulk.shape[1], 512) _ = float(bulk[0, 0]) def test_node_manager_temporary_grid(self): + try: + import numpy # noqa: F401 + except ImportError: + self.skipTest("numpy not installed") nm = nanovdb.createNodeManager( nanovdb.tools.createFogVolumeSphere().grid()).mgr() self._force_gc() @@ -772,6 +786,232 @@ def test_blind_data_temporary(self): self.assertIsNone(result) +class TestVoxelBlockManager(unittest.TestCase): + """nanovdb.tools.buildVoxelBlockManager + VoxelBlockManagerHandle + + decodeInverseMaps and the createOnIndexGrid test-scaffold factory. + + NOTE: end-to-end decode verification across every block is intentionally + deferred until the Phase 4 build::Grid bindings land. The C++ + buildVoxelBlockManager has an algorithmic gap when the source OnIndex + grid is tile-compressed (blocks not reached by any leaf's iteration + sweep are left with uninitialized firstLeafID). The current host-side + createFloatGrid + createOnIndexGrid path triggers tile compression on + uniform regions, so we only exercise decodeBlock(0) here — that block + is guaranteed to be covered when the grid's firstOffset is 1. The + bindings include a defensive check that raises ValueError if a + user hits the uninitialized-block case rather than crashing. + """ + + def _make_cube_on_index_grid(self): + # 21^3 fully-active cube — matches the C++ unit test's input. + bbox = nanovdb.math.CoordBBox( + nanovdb.math.Coord(125), nanovdb.math.Coord(145)) + float_h = nanovdb.tools.createFloatGrid( + 0.0, "cube", nanovdb.GridClass.Unknown, + lambda ijk: 1.0, bbox) + return nanovdb.tools.createOnIndexGrid( + float_h.grid(), include_stats=False, include_tiles=False) + + def test_create_on_index_grid(self): + h = self._make_cube_on_index_grid() + g = h.grid() + self.assertEqual(g.gridType(), nanovdb.GridType.OnIndex) + self.assertEqual(g.gridClass(), nanovdb.GridClass.IndexGrid) + self.assertGreater(g.activeVoxelCount(), 0) + self.assertTrue(g.isSequential()) + + def test_create_on_index_grid_rejects_unsupported_source(self): + # createOnIndexGrid only accepts {float, double, int32, Vec3f} + # source grids; passing None (or any non-grid object) should raise + # TypeError at the first BuildT-isinstance check. + with self.assertRaises(TypeError): + nanovdb.tools.createOnIndexGrid(None) + + def test_build_voxel_block_manager_handle(self): + h = self._make_cube_on_index_grid() + g = h.grid() + vbm = nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=6) + self.assertGreater(vbm.blockCount(), 0) + self.assertEqual(vbm.firstOffset(), 1) + self.assertEqual(vbm.lastOffset(), g.activeVoxelCount()) + self.assertTrue(bool(vbm)) + + def test_buffers_zero_copy_shape_and_dtype(self): + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + h = self._make_cube_on_index_grid() + # log2_block_width=6 -> JumpMapLength=1. + vbm6 = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2_block_width=6) + self.assertEqual(vbm6.log2_block_width, 6) + self.assertEqual(vbm6.block_width, 64) + self.assertEqual(vbm6.jump_map_length, 1) + fl = vbm6.firstLeafID() + self.assertEqual(fl.shape, (vbm6.blockCount(),)) + self.assertEqual(fl.dtype, np.uint32) + jm = vbm6.jumpMap() + self.assertEqual(jm.shape, (vbm6.blockCount(), 1)) + self.assertEqual(jm.dtype, np.uint64) + # log2_block_width=7 -> JumpMapLength=2 (independent build, separate + # allocation; the jumpMap shape comes from the handle, not the caller). + vbm7 = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2_block_width=7) + self.assertEqual(vbm7.jump_map_length, 2) + self.assertEqual(vbm7.jumpMap().shape, (vbm7.blockCount(), 2)) + + def test_decode_block_zero(self): + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + h = self._make_cube_on_index_grid() + g = h.grid() + vbm = nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=6) + leaf_index, voxel_offset = vbm.decodeBlock(g, 0) + self.assertEqual(leaf_index.shape, (64,)) + self.assertEqual(leaf_index.dtype, np.uint32) + self.assertEqual(voxel_offset.shape, (64,)) + self.assertEqual(voxel_offset.dtype, np.uint16) + # Free function should produce the same result for the same input. + fl = np.asarray(vbm.firstLeafID()) + jm = np.asarray(vbm.jumpMap()) + li_free, vo_free = nanovdb.tools.decodeInverseMaps( + g, int(fl[0]), jm[0], vbm.firstOffset(), log2_block_width=6) + self.assertTrue(np.array_equal(leaf_index, li_free)) + self.assertTrue(np.array_equal(voxel_offset, vo_free)) + + def test_decode_block_out_of_range(self): + h = self._make_cube_on_index_grid() + g = h.grid() + vbm = nanovdb.tools.buildVoxelBlockManager(g) + with self.assertRaises(IndexError): + vbm.decodeBlock(g, vbm.blockCount()) + + def test_log2_block_width_out_of_range(self): + h = self._make_cube_on_index_grid() + g = h.grid() + with self.assertRaises(ValueError): + nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=5) + with self.assertRaises(ValueError): + nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=10) + + def test_build_voxel_block_manager_rejects_misaligned_first_offset(self): + # first_offset must satisfy first_offset == 1 (mod BlockWidth). + # For log2_block_width=6, BlockWidth=64, so 1, 65, 129, ... are valid + # but 2 is not. + h = self._make_cube_on_index_grid() + g = h.grid() + with self.assertRaises(ValueError): + nanovdb.tools.buildVoxelBlockManager( + g, log2_block_width=6, first_offset=2) + # And the wider-block case: log2_block_width=7 -> BlockWidth=128, + # first_offset=65 is valid for width=6 but misaligned for width=7. + with self.assertRaises(ValueError): + nanovdb.tools.buildVoxelBlockManager( + g, log2_block_width=7, first_offset=65) + + def test_decode_inverse_maps_rejects_bad_first_leaf_id(self): + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + h = self._make_cube_on_index_grid() + g = h.grid() + vbm = nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=6) + jm0 = np.asarray(vbm.jumpMap())[0] + n_leaves = g.tree().nodeCount(0) + with self.assertRaises(IndexError): + nanovdb.tools.decodeInverseMaps( + g, n_leaves, jm0, vbm.firstOffset(), log2_block_width=6) + + def test_build_voxel_block_manager_rejects_undersized_n_blocks(self): + # Caller-supplied n_blocks must hold at least + # ceil((last_offset - first_offset + 1) / BlockWidth) blocks; + # smaller values would silently truncate coverage. + h = self._make_cube_on_index_grid() + g = h.grid() + # The cube grid has ~9261 active voxels, so at log2_block_width=6 + # (BlockWidth=64) the minimum is roughly 145 blocks. Passing 1 + # must be rejected. + with self.assertRaises(ValueError): + nanovdb.tools.buildVoxelBlockManager( + g, log2_block_width=6, n_blocks=1) + + def test_build_voxel_block_manager_rejects_non_on_index_grid(self): + # FloatGrid is not an OnIndexGrid. + h_float = nanovdb.tools.createFogVolumeSphere() + with self.assertRaises(TypeError): + nanovdb.tools.buildVoxelBlockManager(h_float.grid()) + + def test_untouched_blocks_trip_sentinel_guard(self): + # Build the cube VBM and probe every block. The Python binding + # prefills firstLeafID with a sentinel (== nLeaves) before calling + # the in-place builder, so any block the upstream algorithm doesn't + # touch deterministically trips the firstLeafID >= nLeaves guard in + # decodeBlock (raising ValueError). The sweep must therefore see + # only two outcomes per block: a successful decode or a sentinel + # ValueError — no segfaults, no IndexError (those would indicate + # block_index out of range, not sentinel), and no silent wrong + # decode. + try: + import numpy as np # noqa: F401 + except ImportError: + self.skipTest("numpy not installed") + h = self._make_cube_on_index_grid() + g = h.grid() + vbm = nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=6) + n_leaves = g.tree().nodeCount(0) + fl = vbm.firstLeafID() + for b in range(vbm.blockCount()): + slot = int(fl[b]) + # Slot must be a real leaf id or the sentinel — never garbage. + self.assertTrue(slot < n_leaves or slot == n_leaves, + f"block {b}: firstLeafID={slot} is neither a real leaf id " + f"(< {n_leaves}) nor the sentinel (== {n_leaves}); " + "uninitialized memory leaked through.") + if slot >= n_leaves: + with self.assertRaises(ValueError): + vbm.decodeBlock(g, b) + else: + # Successful decode path — just confirm the shapes. + li, vo = vbm.decodeBlock(g, b) + self.assertEqual(li.shape, (64,)) + self.assertEqual(vo.shape, (64,)) + + def test_default_constructed_handle_returns_empty_arrays(self): + # A default-constructed VoxelBlockManagerHandle has null backing + # buffers; firstLeafID() and jumpMap() should still return empty + # ndarrays rather than crash on the null pointer. + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + vbm = nanovdb.tools.VoxelBlockManagerHandle() + self.assertEqual(vbm.blockCount(), 0) + self.assertFalse(bool(vbm)) + fl = np.asarray(vbm.firstLeafID()) + self.assertEqual(fl.shape, (0,)) + self.assertEqual(fl.dtype, np.uint32) + jm = np.asarray(vbm.jumpMap()) + # Default-constructed handle uses log2_block_width=6 -> JumpMapLength=1. + self.assertEqual(jm.shape, (0, 1)) + self.assertEqual(jm.dtype, np.uint64) + + def test_reset_handle_returns_empty_arrays(self): + # Same guard but exercising reset() after a real build. + try: + import numpy as np + except ImportError: + self.skipTest("numpy not installed") + h = self._make_cube_on_index_grid() + vbm = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2_block_width=6) + self.assertGreater(vbm.blockCount(), 0) + vbm.reset() + self.assertEqual(vbm.blockCount(), 0) + self.assertEqual(np.asarray(vbm.firstLeafID()).shape, (0,)) + self.assertEqual(np.asarray(vbm.jumpMap()).shape, (0, 1)) + + class TestGridMetaDataGuards(unittest.TestCase): """GridMetaData() constructor and safeCast() reject bad input (None, a Grid wrapping an invalid buffer) with a Python exception or False From 68b26c4579915c8faac860a2be4402f74fcc7899 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 21 May 2026 17:04:01 +1200 Subject: [PATCH 06/48] =?UTF-8?q?nanovdb=20python:=20Phase=204a=20?= =?UTF-8?q?=E2=80=94=20tools.build.Grid=20mutable=20CPU=20builder=20(#2?= =?UTF-8?q?214)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: Phase 4a — tools.build.Grid mutable CPU builder Bind nanovdb::tools::build::Grid, its ValueAccessor, and Tree::WriteAccessor under a new nanovdb.tools.build submodule. One set of classes per writable BuildT in BuildTypes.def — every scalar (float, double, int16, int32, int64, uint8, uint32) and every vector (Vec3f, Vec3d, Vec4f, Vec4d, Vec3u8, Vec3u16, Rgba8). Naming mirrors the C++ namespace: nanovdb.tools.build.FloatGrid is the mutable counterpart of the read-only nanovdb.FloatGrid. The binding exposes: - Grid(background, name='', gridClass=Unknown) — constructor - getValue / setValue / setValueOn / isActive (the last two convenience- wrap a fresh ValueAccessor under the hood, since C++ has no setValueOn/isActive on Grid itself) - nodeCount, gridType, gridClass, getName/setName, setTransform, .background property - getAccessor() / getWriteAccessor() returning the typed Value / WriteAccessor proxies - .to_nanovdb(sMode, cMode, verbose) — bakes the build grid into a host NanoGrid handle by calling tools::createNanoGrid ValueAccessor exposes getValue/setValue/setValueOn/isActive/isValueOn. WriteAccessor exposes setValue/setValueOn/merge. Both are wired up so the parent grid is kept alive by Python while the accessor lives. Implementation notes: - WriteAccessor's defaulted move constructor leaves its internal ValueAccessor::mRoot reference dangling (the reference points into the moved-from WriteAccessor's own mRoot field, which is per-object state — not the parent Tree's mRoot). The Python binding bypasses the move path by heap-allocating via nb::rv_policy::take_ownership so the C++ object's address is stable for its entire lifetime. - ValueAccessor doesn't have this hazard because its mRoot reference points at the parent Tree's mRoot (a stable address external to the accessor), so move construction is safe. - Read-only special BuildTs (Boolean, Fp4/8/16/N, ValueIndex, ValueOnIndex, ValueMask) and Point are deliberately excluded — they have no SetValue specialization and can't be built voxel-by-voxel. Test coverage in new TestBuildGrid: constructor defaults, setValue marks active, setValueOn preserves background, accessor parity with grid, WriteAccessor merge-on-destruction, .to_nanovdb() round-trip with metadata preserved, .to_nanovdb() doesn't consume the source, Int32Grid + Vec3fGrid spot-checks, and setTransform propagation to the baked grid's voxelSize / map. Signed-off-by: Jonathan Swartz * nanovdb python: address Copilot review on #2214 Four items from the Phase 4a review, all valid: 1. Grid.background property read self.mRoot.mBackground directly, touching an internal field even though RootNode exposes a background() accessor. Switch to self.mRoot.background() so the binding doesn't depend on the underlying field name staying put. 2. .to_nanovdb() can be an expensive bake for large grids but held the GIL the whole time. Add nb::call_guard() so other Python threads can run during the conversion (the lambda only touches C++ state, no Python object handling). 3. nodeCount()'s docstring said the tuple was "internal node counts", but the first element is the leaf (level-0) count — leaves are not internal nodes. Reword to just "(leaf_count, lower_count, upper_count)" and drop the "internal" mislabel. 4. The WriteAccessor merge test was named "merges_on_destruction" but actually called wa.merge() explicitly and never forced the destructor to run. Split into two cases: - test_write_accessor_explicit_merge — calls .merge() explicitly, matching what the original test actually checked - test_write_accessor_merges_on_destruction — drops the only reference (del wa) and runs gc.collect() so the C++ destructor fires, then asserts the change is visible Signed-off-by: Jonathan Swartz * nanovdb python: use Vec3f.__eq__ in build::Vec3fGrid test Copilot noted the component-by-component comparison in test_vec3f_build_grid was justified by an out-of-date claim that Vec3f equality isn't bound — it is (PyMath.cc defineVec3 wires nb::self == nb::self). Use self.assertEqual(got, v) directly. Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 1 + nanovdb/nanovdb/python/NanoVDBModule.cc | 2 + nanovdb/nanovdb/python/PyBuildGrid.cc | 231 +++++++++++++++++++++ nanovdb/nanovdb/python/PyBuildGrid.h | 20 ++ nanovdb/nanovdb/python/test/TestNanoVDB.py | 129 ++++++++++++ 5 files changed, 383 insertions(+) create mode 100644 nanovdb/nanovdb/python/PyBuildGrid.cc create mode 100644 nanovdb/nanovdb/python/PyBuildGrid.h diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 2dba4f661d..9be44ce852 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -16,6 +16,7 @@ option(NANOVDB_BUILD_PYTHON_STUBS nanobind_add_module(nanovdb_python NB_STATIC NanoVDBModule.cc + PyBuildGrid.cc PyCreateNanoGrid.cc PyGridChecksum.cc PyGridHandle.cc diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 334c101b96..f847e9eee3 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -15,6 +15,7 @@ #include #include "cuda/PyDeviceBuffer.h" +#include "PyBuildGrid.h" #include "PyGridHandle.h" #include "PyHostBuffer.h" #include "PyIO.h" @@ -808,6 +809,7 @@ NB_MODULE(nanovdb, m) toolsModule.doc() = "A submodule that implements tools for NanoVDB grids"; defineToolsModule(toolsModule); defineVoxelBlockManagerModule(toolsModule); + defineBuildGridModule(toolsModule); nb::module_ ioModule = m.def_submodule("io"); ioModule.doc() = "A submodule that implements I/O functionality for NanoVDB grids"; diff --git a/nanovdb/nanovdb/python/PyBuildGrid.cc b/nanovdb/nanovdb/python/PyBuildGrid.cc new file mode 100644 index 0000000000..5ef1e09134 --- /dev/null +++ b/nanovdb/nanovdb/python/PyBuildGrid.cc @@ -0,0 +1,231 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyBuildGrid.h" + +#include +#include + +#include +#include +#include +#include + +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; +using namespace nanovdb; + +namespace pynanovdb { + +// Bind nanovdb::tools::build::Grid together with its +// ValueAccessor and Tree::WriteAccessor proxies, plus a +// .to_nanovdb() shortcut that bakes the build grid into a host NanoGrid +// via tools::createNanoGrid. +// +// One instantiation per writable BuildT in BuildTypes.def (scalars + +// vectors). All three classes live under the nanovdb.tools.build submodule; +// the Python class names mirror the existing typed-grid naming so +// nanovdb.tools.build.FloatGrid is the mutable counterpart of the read-only +// nanovdb.FloatGrid. +template +static void defineBuildGrid(nb::module_& m, + const char* gridName, + const char* valueAccName, + const char* writeAccName) +{ + using GridT = tools::build::Grid; + using TreeT = tools::build::Tree; + using AccT = tools::build::ValueAccessor; + using WriteAccT = typename TreeT::WriteAccessor; + using ValueT = typename GridT::ValueType; + + // ----- build::Grid ----- + nb::class_(m, gridName) + .def(nb::init(), + "background"_a, + "name"_a = std::string(""), + "gridClass"_a = GridClass::Unknown, + "Construct an empty mutable build grid. Voxels read as " + "background until written.") + .def("getValue", + [](const GridT& self, const Coord& ijk) -> ValueT { + return self.getValue(ijk); + }, + "ijk"_a, + "Return the value at ijk (background if no leaf covers it).") + .def("setValue", + [](GridT& self, const Coord& ijk, const ValueT& value) { + self.setValue(ijk, value); + }, + "ijk"_a, "value"_a, + "Set the voxel value at ijk and mark the voxel active.") + // build::Grid has no top-level isActive(ijk); the read path is via + // ValueAccessor. Spin up a fresh accessor for the single query so + // Python callers don't have to. + .def("isActive", + [](GridT& self, const Coord& ijk) { + AccT acc = self.getAccessor(); + return acc.isActive(ijk); + }, + "ijk"_a, + "Return True iff ijk is in an active voxel. Equivalent to " + "self.getAccessor().isActive(ijk), but allocates a fresh " + "accessor for each call — for repeated queries use " + "self.getAccessor() and reuse it.") + .def("setValueOn", + [](GridT& self, const Coord& ijk) { + AccT acc = self.getAccessor(); + acc.setValueOn(ijk); + }, + "ijk"_a, + "Mark ijk active without changing the stored value. Equivalent " + "to self.getAccessor().setValueOn(ijk).") + .def("nodeCount", + [](const GridT& self) -> std::array { + return self.nodeCount(); + }, + "Return a 3-tuple (leaf_count, lower_count, upper_count).") + .def("gridType", &GridT::gridType, + "Return the GridType enumerator this BuildT carries.") + .def("gridClass", &GridT::gridClass, + "Return the GridClass assigned at construction time.") + .def("getName", &GridT::getName, + "Return the grid name (as passed at construction).") + .def("setName", &GridT::setName, "name"_a, + "Replace the grid name.") + .def("setTransform", &GridT::setTransform, + "scale"_a = 1.0, + "translation"_a = Vec3d(0.0), + "Set an affine index-to-world map from a uniform scale and " + "translation. Replaces any prior transform.") + .def_prop_ro("background", + [](const GridT& self) -> ValueT { + return self.mRoot.background(); + }, + "The background value supplied at construction.") + .def("getAccessor", &GridT::getAccessor, + nb::keep_alive<0, 1>(), + "Return a ValueAccessor wired to this grid's root. Thread-safe " + "for reads, NOT thread-safe for writes. The accessor borrows " + "from this grid — the grid must outlive it.") + // WriteAccessor's defaulted move constructor would leave its + // internal ValueAccessor's `mRoot&` reference dangling (it points + // into the moved-from WriteAccessor's own mRoot field). Bypass + // the move path entirely by heap-allocating and handing nanobind + // ownership — the C++ object stays at a stable address for its + // whole lifetime. + .def("getWriteAccessor", + [](GridT& self) -> WriteAccT* { + return new WriteAccT(self.mRoot, self.mMutex); + }, + nb::rv_policy::take_ownership, + nb::keep_alive<0, 1>(), + "Return a WriteAccessor for thread-safe writes; the accessor " + "buffers changes into a private root and merges them into the " + "parent grid on destruction (or on an explicit merge() call). " + "Held by nanobind on the heap so the accessor's internal " + "references stay valid.") + // Baking a large grid is the most expensive operation on this + // class; release the GIL so other Python threads can run during + // the conversion (the lambda only touches C++ state). + .def("to_nanovdb", + [](const GridT& self, + tools::StatsMode sMode, + CheckMode cMode, + int verbose) { + return tools::createNanoGrid( + self, sMode, cMode, verbose); + }, + nb::call_guard(), + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, + "verbose"_a = 0, + "Bake this mutable grid into a host NanoGrid and " + "return its GridHandle. The build grid is unchanged. " + "Releases the GIL during conversion."); + + // ----- build::ValueAccessor ----- + // + // Move-only (copy is deleted) — returned by getAccessor(). Caches the + // last leaf / lower / upper node it touched, so repeated access to + // neighboring coordinates is fast. + nb::class_(m, valueAccName) + .def("getValue", + [](const AccT& self, const Coord& ijk) -> ValueT { + return self.getValue(ijk); + }, + "ijk"_a, "Return the value at ijk (uses the cache).") + .def("setValue", + [](AccT& self, const Coord& ijk, const ValueT& value) { + self.setValue(ijk, value); + }, + "ijk"_a, "value"_a, + "Set the value at ijk and mark it active (uses the cache).") + .def("setValueOn", + [](AccT& self, const Coord& ijk) { self.setValueOn(ijk); }, + "ijk"_a, + "Mark ijk active without changing the stored value.") + .def("isActive", + [](const AccT& self, const Coord& ijk) { + return self.isActive(ijk); + }, + "ijk"_a, + "Return True iff ijk is in an active voxel.") + .def("isValueOn", + [](const AccT& self, const Coord& ijk) { + return self.isValueOn(ijk); + }, + "ijk"_a, + "Alias for isActive(ijk)."); + + // ----- build::Tree::WriteAccessor ----- + // + // Move-only. Holds its own root node + a reference to the parent root's + // mutex; on destruction (or explicit merge()) it locks the mutex and + // splices its buffered nodes into the parent. Designed for multi-thread + // writes — one WriteAccessor per thread, no shared mutable state. + nb::class_(m, writeAccName) + .def("setValue", + [](WriteAccT& self, const Coord& ijk, const ValueT& value) { + self.setValue(ijk, value); + }, + "ijk"_a, "value"_a, + "Buffer a set into this accessor's private root.") + .def("setValueOn", + [](WriteAccT& self, const Coord& ijk) { self.setValueOn(ijk); }, + "ijk"_a, + "Buffer an active-state set into this accessor's private root.") + .def("merge", &WriteAccT::merge, + "Lock the parent mutex and splice this accessor's buffered " + "nodes into the parent grid. Called automatically when the " + "accessor is destroyed; calling it explicitly is only " + "necessary if you want the changes visible to the parent " + "before the accessor goes out of scope."); +} + +void defineBuildGridModule(nb::module_& toolsModule) +{ + nb::module_ buildModule = toolsModule.def_submodule("build"); + buildModule.doc() = + "Mutable, voxel-by-voxel CPU grid builder mirroring " + "nanovdb::tools::build::*. Construct a typed Grid (e.g. " + "FloatGrid(0.0, 'mygrid')), populate it with setValue / " + "ValueAccessor / WriteAccessor, then call .to_nanovdb() to bake " + "a host NanoGrid handle."; + + // X-macro instantiation over every writable BuildT: scalars (full + // arithmetic) and vectors. Read-only special BuildTs (Boolean, Fp*, + // Index, Mask) and Point are deliberately excluded — they have no + // SetValue specialization and can't be built voxel-by-voxel. +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + defineBuildGrid(buildModule, #Suffix "Grid", \ + #Suffix "ValueAccessor", #Suffix "WriteAccessor"); +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + defineBuildGrid(buildModule, #Suffix "Grid", \ + #Suffix "ValueAccessor", #Suffix "WriteAccessor"); +#include "BuildTypes.def" +} + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyBuildGrid.h b/nanovdb/nanovdb/python/PyBuildGrid.h new file mode 100644 index 0000000000..71547887ab --- /dev/null +++ b/nanovdb/nanovdb/python/PyBuildGrid.h @@ -0,0 +1,20 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_PYBUILDGRID_HAS_BEEN_INCLUDED +#define NANOVDB_PYBUILDGRID_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +/// @brief Register the nanovdb.tools.build submodule and its per-BuildT +/// Grid / ValueAccessor / WriteAccessor classes (one set per writable +/// scalar and vector BuildT in BuildTypes.def). Constructs the submodule +/// as `toolsModule.def_submodule("build")`. +void defineBuildGridModule(nb::module_& toolsModule); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index 0f0807633d..34ce09c274 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -1543,6 +1543,135 @@ def test_create_vec3f_nano_grid(self): self.assertEqual(grid.gridClass(), nanovdb.GridClass.Unknown) +class TestBuildGrid(unittest.TestCase): + """nanovdb.tools.build.* — mutable voxel-by-voxel CPU grid builder.""" + + def test_constructor_defaults_and_metadata(self): + g = nanovdb.tools.build.FloatGrid(0.0) + self.assertEqual(g.getName(), "") + self.assertEqual(g.gridClass(), nanovdb.GridClass.Unknown) + self.assertEqual(g.gridType(), nanovdb.GridType.Float) + self.assertEqual(g.background, 0.0) + self.assertEqual(g.nodeCount(), [0, 0, 0]) + g.setName("renamed") + self.assertEqual(g.getName(), "renamed") + + def test_set_get_value_marks_active(self): + g = nanovdb.tools.build.FloatGrid(0.0, "demo") + ijk = nanovdb.math.Coord(1, 2, 3) + self.assertFalse(g.isActive(ijk)) + self.assertEqual(g.getValue(ijk), 0.0) + g.setValue(ijk, 4.5) + self.assertTrue(g.isActive(ijk)) + self.assertEqual(g.getValue(ijk), 4.5) + # An untouched voxel is still background-valued and inactive. + self.assertEqual(g.getValue(nanovdb.math.Coord(10, 0, 0)), 0.0) + self.assertFalse(g.isActive(nanovdb.math.Coord(10, 0, 0))) + + def test_set_value_on_keeps_background_value(self): + g = nanovdb.tools.build.FloatGrid(-1.0, "demo") + ijk = nanovdb.math.Coord(5, 6, 7) + g.setValueOn(ijk) + self.assertTrue(g.isActive(ijk)) + # setValueOn does not change the stored value — still background. + self.assertEqual(g.getValue(ijk), -1.0) + + def test_value_accessor_parity_with_grid(self): + g = nanovdb.tools.build.FloatGrid(0.0) + acc = g.getAccessor() + ijk = nanovdb.math.Coord(100, 200, 300) + acc.setValue(ijk, 7.5) + self.assertEqual(g.getValue(ijk), 7.5) + self.assertEqual(acc.getValue(ijk), 7.5) + self.assertTrue(acc.isActive(ijk)) + # isValueOn is an alias for isActive. + self.assertEqual(acc.isValueOn(ijk), acc.isActive(ijk)) + + def test_write_accessor_explicit_merge(self): + g = nanovdb.tools.build.FloatGrid(0.0) + ijk = nanovdb.math.Coord(50, 50, 50) + wa = g.getWriteAccessor() + wa.setValue(ijk, 9.0) + # Before merge, the parent grid hasn't seen the change yet. + self.assertEqual(g.getValue(ijk), 0.0) + wa.merge() + self.assertEqual(g.getValue(ijk), 9.0) + self.assertTrue(g.isActive(ijk)) + + def test_write_accessor_merges_on_destruction(self): + # When the Python wrapper for a WriteAccessor is collected, the + # C++ destructor runs merge() automatically. Force collection by + # dropping the only reference and running the GC. + import gc + g = nanovdb.tools.build.FloatGrid(0.0) + ijk = nanovdb.math.Coord(60, 60, 60) + wa = g.getWriteAccessor() + wa.setValue(ijk, 3.5) + self.assertEqual(g.getValue(ijk), 0.0) + del wa + gc.collect() + self.assertEqual(g.getValue(ijk), 3.5) + self.assertTrue(g.isActive(ijk)) + + def test_to_nanovdb_roundtrip(self): + g = nanovdb.tools.build.FloatGrid(0.0, "trip", nanovdb.GridClass.FogVolume) + g.setValue(nanovdb.math.Coord(0, 0, 0), 1.0) + g.setValue(nanovdb.math.Coord(1, 0, 0), 2.0) + g.setValue(nanovdb.math.Coord(2, 0, 0), 3.0) + h = g.to_nanovdb() + self.assertEqual(h.gridCount(), 1) + ng = h.grid() + self.assertEqual(ng.gridType(), nanovdb.GridType.Float) + self.assertEqual(ng.gridClass(), nanovdb.GridClass.FogVolume) + self.assertEqual(ng.gridName(), "trip") + self.assertEqual(ng.activeVoxelCount(), 3) + + def test_to_nanovdb_does_not_consume_source(self): + # Source build::Grid must remain usable after .to_nanovdb(). + g = nanovdb.tools.build.FloatGrid(0.0) + g.setValue(nanovdb.math.Coord(0, 0, 0), 1.0) + _ = g.to_nanovdb() + g.setValue(nanovdb.math.Coord(1, 0, 0), 2.0) + h2 = g.to_nanovdb() + self.assertEqual(h2.grid().activeVoxelCount(), 2) + + def test_int32_build_grid(self): + g = nanovdb.tools.build.Int32Grid(0, "ints", nanovdb.GridClass.Unknown) + g.setValue(nanovdb.math.Coord(0, 0, 0), 42) + g.setValue(nanovdb.math.Coord(1, 1, 1), -7) + self.assertEqual(g.getValue(nanovdb.math.Coord(0, 0, 0)), 42) + self.assertEqual(g.getValue(nanovdb.math.Coord(1, 1, 1)), -7) + h = g.to_nanovdb() + self.assertEqual(h.grid().gridType(), nanovdb.GridType.Int32) + self.assertEqual(h.grid().activeVoxelCount(), 2) + + def test_vec3f_build_grid(self): + g = nanovdb.tools.build.Vec3fGrid( + nanovdb.math.Vec3f(0.0), "v", nanovdb.GridClass.Unknown) + v = nanovdb.math.Vec3f(1.0, 2.0, 3.0) + g.setValue(nanovdb.math.Coord(0, 0, 0), v) + self.assertEqual(g.getValue(nanovdb.math.Coord(0, 0, 0)), v) + h = g.to_nanovdb() + self.assertEqual(h.grid().gridType(), nanovdb.GridType.Vec3f) + + def test_set_transform(self): + g = nanovdb.tools.build.FloatGrid(0.0) + g.setTransform(scale=0.5, translation=nanovdb.math.Vec3d(1.0, 2.0, 3.0)) + g.setValue(nanovdb.math.Coord(0, 0, 0), 1.0) + h = g.to_nanovdb() + ng = h.grid() + vs = ng.voxelSize() + self.assertAlmostEqual(vs[0], 0.5) + self.assertAlmostEqual(vs[1], 0.5) + self.assertAlmostEqual(vs[2], 0.5) + # Index (0,0,0) mapped through (scale=0.5, translation=(1,2,3)) + # lands at world-space (1, 2, 3). + w = ng.map().applyMap(nanovdb.math.Vec3d(0.0, 0.0, 0.0)) + self.assertAlmostEqual(w[0], 1.0) + self.assertAlmostEqual(w[1], 2.0) + self.assertAlmostEqual(w[2], 3.0) + + class TestNanoToOpenVDB(unittest.TestCase): def test_function(self): handle = nanovdb.tools.createLevelSetSphere() From 3c6f12149314258fac1547fcfdc00698de05ee4c Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 21 May 2026 17:47:28 +1200 Subject: [PATCH 07/48] =?UTF-8?q?nanovdb=20python:=20Phase=204b=20+=204c?= =?UTF-8?q?=20=E2=80=94=20stats,=20validation,=20checksum=20(#2215)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: Phase 4b + 4c — stats, validation, checksum Round out Phase 4 by binding the GridStats, GridValidator, and GridChecksum surfaces that didn't ship with Phase 4a. Phase 4b — Stats ================ * Per-BuildT `tools.Extrema` and `tools.Stats` classes for every scalar and vector BuildT in `BuildTypes.def`. Extrema exposes `min` / `max` / `add(value)` and a truthy `bool()` for "has at least one sample"; Stats inherits from Extrema and adds `size`, `avg` / `mean`, `var` / `variance`, `std` / `stdDev`. Static predicates `hasMinMax` / `hasAverage` / `hasStdDeviation` / `hasStats` mirror the C++ trait queries. * `tools.updateGridStats(grid, mode=Default)` — polymorphic dispatch via `callNanoGrid` over an `UpdateGridStatsOp`. Scalar and vector BuildTs route to `tools::updateGridStats`; the special / quantized / index / mask types (`Fp4/8/16/N`, `ValueIndex`, `ValueOnIndex`, `ValueMask`, `Point`) raise `ValueError` because `Stats` isn't meaningful for them. `bool` falls through to the C++ `NoopStats` arm. * Per-BuildT `tools.getExtrema(grid, bbox)` returning the matching `Extrema`. Restricted to scalar + vector BuildTs (the only ones with an arithmetic ValueType). Both `updateGridStats` and `getExtrema` release the GIL during the traversal. Phase 4c — Validation & checksum ================================ * `tools.validateGrid(handle, gridID, mode=Default, verbose=False)` for `GridHandle` — single-grid complement of the existing `validateGrids`. Returns `False` (without raising) when the gridID is out of range. * `tools.checkGrid(grid, mode=Full)` — polymorphic via `callNanoGrid`. Returns `(ok, error_message)`. The 256-byte char buffer the C++ helper writes into is hidden inside the binding so Python callers see a `(bool, str)` tuple. * `tools.isValid(grid, mode=Default, verbose=False)` — polymorphic via `callNanoGrid`, equivalent to `checkGrid` + checksum verification rolled into a single bool. * `tools.evalChecksum(grid, mode=Default)` and `tools.validateChecksum(grid, mode=Default)` — accept any bound NanoGrid via the existing `GridData` Python upcast (every `NanoGrid` Python class is registered with `GridData` as its base, so nanobind handles the dispatch transparently). All three release the GIL. Test coverage in new `TestGridStats`, `TestGridValidate`, and `TestGridChecksum` (11 cases total): Extrema/Stats default-and-add, polymorphic `updateGridStats` on a float grid, `updateGridStats` rejection on an OnIndex grid, `getExtrema` over a sub-bbox, `checkGrid` / `isValid` / `validateGrid` happy paths plus out-of-range `gridID` and `CheckMode.Disable` short-circuit, and `evalChecksum` → `updateChecksum` → `validateChecksum` round-trip. Signed-off-by: Jonathan Swartz * nanovdb python: address Copilot review on #2215 Four items from the Phase 4b/4c review, all valid: 1. UpdateGridStatsOp rejected every BuildTraits::is_special type unconditionally (except bool), but tools::updateGridStats actually supports StatsMode::BBox on any ValueT via the NoopStats path. Drop directly into NoopStats for special BuildTs when mode is Disable or BBox; only MinMax / All raise (because those would instantiate Stats / Extrema over a non-arithmetic ValueT and the semantics are ill-defined). Update the binding docstring to describe the actual matrix. 2. The test_update_grid_stats_rejects_index_grid case was renamed to test_update_grid_stats_on_index_grid and extended to cover all four StatsMode arms on an OnIndexGrid: MinMax and All raise, BBox and Disable now succeed. 3. validateGrid's docstring didn't mention the CheckMode.Disable short-circuit (which returns True without inspecting the handle or gridID). The Python tests already exercised the short-circuit via test_validateGrid_disable_mode_always_true, so the behavior was correct — just the docstring was missing it. 4. IsValidOp::unknown ignored verbose=True and silently returned false, but the C++ callNanoGrid::unknown arm writes an "Unsupported GridType" message to std::cerr when verbose is set. Mirror that — pull in in the binding TU and emit the same diagnostic from C++. 5. test_get_extrema_over_active_bbox used a bbox that exactly equals the root's active bbox, which triggers C++ getExtrema's "bbox contains root.bbox()" short-circuit that unconditionally folds the grid background into the extrema — so the min came back as 0.0 (background), not 1.0 (the smallest active voxel). The test passed but the semantics were muddy: it was really asserting "background gets added in this branch" rather than anything about getExtrema's bbox restriction. Replace with test_get_extrema_strictly_inside_active_region, which uses a bbox strictly inside the active region (so the recursive branch runs) and asserts min/max are exactly the smallest/largest sampled active values. Signed-off-by: Jonathan Swartz * nanovdb python: bind validateGrid for device handles too Copilot noted tools.validateGrids was registered for both GridHandle and GridHandle (the latter behind NANOVDB_USE_CUDA), but my Phase 4c addition of the single-grid tools.validateGrid only covered HostBuffer — leaving device handles able to validate the whole bundle but not an individual grid. Add the matching #ifdef NANOVDB_USE_CUDA overload binding tools::validateGrid>. The C++ helper does host-side dispatch via callNanoGrid on the host-resident gridData() pointer that DeviceGridHandle exposes, so the same Python overload pair is appropriate. New test_validateGrid_on_device_handle exercises the device path end-to-end (build a CUDA level-set sphere, validate it, confirm the same out-of-range and Disable-mode short-circuits as the host overload). The test is gated on cuda module availability so it's skipped cleanly on CPU-only builds. Signed-off-by: Jonathan Swartz * nanovdb python: address more Copilot review on #2215 Two more items from the review, both valid: 1. CheckGridOp wrote tools::checkGrid's error message into a 256-byte stack buffer. tools::checkGrid (and its util::sprint / util::strcpy helpers) trust the caller's buffer is large enough — there is no bounds-checked variant. The current error messages are at most ~80 characters once GridType and GridClass enumerator names are stringified, so 256 wasn't actually overflowing today, but the margin is uncomfortable and a future error string addition could push past it. Bump to a 4096-byte buffer (named via a constexpr kErrorBufSize) and zero-init the first byte before the call so the ok-detection still works if the helper bails out before writing anything. 2. updateGridStats's docstring claimed special (quantized / index / mask) BuildTs only accept Disable / BBox, but Boolean grids are special yet fall through the if-constexpr filter to the regular tools::updateGridStats path (which routes any mode to NoopStats for ValueT=bool inside the C++ helper). Reword the docstring to call out Boolean as the exception that accepts every StatsMode via the NoopStats internal path. Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyGridChecksum.cc | 36 ++++ nanovdb/nanovdb/python/PyGridChecksum.h | 5 + nanovdb/nanovdb/python/PyGridStats.cc | 200 +++++++++++++++++++++ nanovdb/nanovdb/python/PyGridStats.h | 6 + nanovdb/nanovdb/python/PyGridValidator.cc | 135 +++++++++++++- nanovdb/nanovdb/python/PyGridValidator.h | 6 + nanovdb/nanovdb/python/PyTools.cc | 4 + nanovdb/nanovdb/python/test/TestNanoVDB.py | 165 +++++++++++++++++ 8 files changed, 555 insertions(+), 2 deletions(-) diff --git a/nanovdb/nanovdb/python/PyGridChecksum.cc b/nanovdb/nanovdb/python/PyGridChecksum.cc index 281fd05c0b..b0cc350653 100644 --- a/nanovdb/nanovdb/python/PyGridChecksum.cc +++ b/nanovdb/nanovdb/python/PyGridChecksum.cc @@ -33,4 +33,40 @@ void defineUpdateChecksum(nb::module_& m) "updateChecksum", [](GridData* gridData, CheckMode mode) { tools::updateChecksum(gridData, mode); }, "gridData"_a, "mode"_a); } +void defineEvalChecksumModule(nb::module_& toolsModule) +{ + // tools.evalChecksum(grid, mode) — compute a fresh checksum for the + // given grid without writing it back. Mirrors the GridData* overload + // in tools/GridChecksum.h; the polymorphism over BuildT is implicit + // because every NanoGrid is-a GridData in the C++ hierarchy (and + // the Python class binding declares NanoGrid as derived from + // GridData). + toolsModule.def("evalChecksum", + [](const GridData* gridData, CheckMode mode) -> Checksum { + if (gridData == nullptr) { + throw nb::value_error("evalChecksum: grid is None."); + } + return tools::evalChecksum(gridData, mode); + }, + "grid"_a, "mode"_a = CheckMode::Default, + nb::call_guard(), + "Compute and return the Checksum for the given grid using the " + "specified CheckMode. Does not modify the grid."); + + // tools.validateChecksum(grid, mode) — compare the stored checksum + // against a freshly computed one and return a bool. + toolsModule.def("validateChecksum", + [](const GridData* gridData, CheckMode mode) -> bool { + if (gridData == nullptr) { + throw nb::value_error("validateChecksum: grid is None."); + } + return tools::validateChecksum(gridData, mode); + }, + "grid"_a, "mode"_a = CheckMode::Default, + nb::call_guard(), + "Return True iff the grid's stored checksum matches a freshly " + "computed one for the given CheckMode. A grid with no stored " + "checksum (Checksum.isEmpty()) is considered valid."); +} + } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyGridChecksum.h b/nanovdb/nanovdb/python/PyGridChecksum.h index dd988f871c..d3f0f2af39 100644 --- a/nanovdb/nanovdb/python/PyGridChecksum.h +++ b/nanovdb/nanovdb/python/PyGridChecksum.h @@ -13,6 +13,11 @@ void defineCheckMode(nb::module_& m); void defineChecksum(nb::module_& m); void defineUpdateChecksum(nb::module_& m); +/// @brief Bind tools.evalChecksum and tools.validateChecksum on the +/// nanovdb.tools submodule. Both accept any bound NanoGrid via +/// GridData* upcast (handled by nanobind's class hierarchy). +void defineEvalChecksumModule(nb::module_& toolsModule); + } // namespace pynanovdb #endif diff --git a/nanovdb/nanovdb/python/PyGridStats.cc b/nanovdb/nanovdb/python/PyGridStats.cc index fbd8caec15..844360e773 100644 --- a/nanovdb/nanovdb/python/PyGridStats.cc +++ b/nanovdb/nanovdb/python/PyGridStats.cc @@ -2,9 +2,15 @@ // SPDX-License-Identifier: Apache-2.0 #include "PyGridStats.h" +#include + +#include #include +#include + namespace nb = nanobind; +using namespace nb::literals; using namespace nanovdb; namespace pynanovdb { @@ -20,4 +26,198 @@ void defineStatsMode(nb::module_& m) .value("End", tools::StatsMode::End); } +namespace { + +// ----- Extrema binding (rank 0 and rank 1 share the same surface) ----- +template +static void defineExtrema(nb::module_& m, const char* name) +{ + using ValueT = typename NanoGrid::ValueType; + using ExtremaT = tools::Extrema; + + nb::class_(m, name) + .def(nb::init<>(), + "Default-construct an Extrema with min = numeric_limits::max and " + "max = numeric_limits::lowest, so any subsequent .add(v) gives " + "exact min/max.") + .def("min", + [](const ExtremaT& self) -> ValueT { return self.min(); }, + "Return the minimum value observed so far.") + .def("max", + [](const ExtremaT& self) -> ValueT { return self.max(); }, + "Return the maximum value observed so far.") + .def("add", + [](ExtremaT& self, const ValueT& v) { self.add(v); }, + "value"_a, + "Update min/max with a single sample.") + .def("__bool__", + [](const ExtremaT& self) { return bool(self); }, + "True iff the Extrema has accumulated at least one sample " + "(i.e. min <= max).") + .def_static("hasMinMax", &ExtremaT::hasMinMax, + "True for value types where min/max is meaningful " + "(everything except bool).") + .def_static("hasAverage", &ExtremaT::hasAverage, + "Always False — Extrema does not compute averages; " + "use Stats for that.") + .def_static("hasStdDeviation", &ExtremaT::hasStdDeviation, + "Always False — Extrema does not compute standard " + "deviation; use Stats for that.") + .def_static("hasStats", &ExtremaT::hasStats, + "True iff the value type supports the min/max bookkeeping " + "(everything except bool)."); +} + +// ----- Stats binding (inherits Extrema) ----- +template +static void defineStats(nb::module_& m, const char* name, const char* baseName) +{ + using ValueT = typename NanoGrid::ValueType; + using BaseT = tools::Extrema; + using StatsT = tools::Stats; + (void)baseName; // kept in signature for parity with extrema name lookup + + nb::class_(m, name) + .def(nb::init<>(), + "Default-construct a Stats accumulator with zero samples.") + .def("add", + [](StatsT& self, const ValueT& v) { self.add(v); }, + "value"_a, + "Add a single sample.") + .def("size", + [](const StatsT& self) -> size_t { return self.size(); }, + "Number of samples accumulated so far.") + .def("avg", + [](const StatsT& self) -> double { return self.avg(); }, + "Arithmetic mean of all samples.") + .def("mean", + [](const StatsT& self) -> double { return self.mean(); }, + "Alias for avg().") + .def("var", + [](const StatsT& self) -> double { return self.var(); }, + "Population variance (Sum(x-mean)^2 / N). Returns 0 if " + "fewer than two samples have been added.") + .def("variance", + [](const StatsT& self) -> double { return self.variance(); }, + "Alias for var().") + .def("std", + [](const StatsT& self) -> double { return self.std(); }, + "Standard deviation = sqrt(var()).") + .def("stdDev", + [](const StatsT& self) -> double { return self.stdDev(); }, + "Alias for std().") + .def_static("hasMinMax", &StatsT::hasMinMax, + "True for value types where min/max is meaningful.") + .def_static("hasAverage", &StatsT::hasAverage, + "True for value types that support mean/variance.") + .def_static("hasStdDeviation", &StatsT::hasStdDeviation, + "True for value types that support standard deviation.") + .def_static("hasStats", &StatsT::hasStats, + "True for value types where full statistics is meaningful."); +} + +// ----- updateGridStats polymorphic dispatch ---------------------------------- +// +// Match the IsNanoGridValid/callNanoGrid pattern: an Op struct with `known` +// (called for every BuildT that's in scope) and `unknown` (fallback). Only +// scalar + vector BuildTs have a meaningful Stats specialization, so the +// other arms raise. +struct UpdateGridStatsOp +{ + template + static void known(GridData* gridData, tools::StatsMode mode) + { + using GridT = NanoGrid; + using ValueT = typename GridT::ValueType; + if constexpr (BuildTraits::is_special && + !util::is_same::value) { + // Special / quantized / index / mask BuildTs don't have an + // arithmetic ValueT, so tools::updateGridStats's MinMax / All + // branches would instantiate Stats / Extrema + // with no meaningful semantics (and may not even compile). + // The Disable and BBox branches use NoopStats, which works + // for any ValueT — drive that directly here so the BBox path + // remains available on special grids (it just recomputes + // node bounding boxes without touching min/max/avg). + if (mode == tools::StatsMode::Disable) { + return; + } else if (mode == tools::StatsMode::BBox) { + tools::GridStats> stats; + stats.update(*static_cast(gridData)); + } else { + throw nb::value_error( + "updateGridStats: this grid's BuildT (special / " + "quantized / index / mask) has no arithmetic value " + "type — only StatsMode.Disable and StatsMode.BBox " + "are supported."); + } + } else { + tools::updateGridStats(static_cast(gridData), mode); + } + } + static void unknown(GridData*, tools::StatsMode) { + throw nb::value_error( + "updateGridStats: unsupported GridType / BuildT combination."); + } +}; + +// ----- getExtrema (per-BuildT factory) --------------------------------------- +template +static tools::Extrema::ValueType> +pyGetExtrema(const NanoGrid& grid, const CoordBBox& bbox) +{ + return tools::getExtrema(grid, bbox); +} + +} // namespace + +void defineGridStatsModule(nb::module_& toolsModule) +{ + // Per-BuildT Extrema + Stats. One pair per scalar/vector BuildT — the + // value types are all distinct so we get N pairs of new Python classes. +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + defineExtrema(toolsModule, #Suffix "Extrema"); \ + defineStats(toolsModule, #Suffix "Stats", #Suffix "Extrema"); +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + defineExtrema(toolsModule, #Suffix "Extrema"); \ + defineStats(toolsModule, #Suffix "Stats", #Suffix "Extrema"); +#include "BuildTypes.def" + + // Polymorphic updateGridStats. Accepts any bound NanoGrid (via + // upcast to GridData*) and dispatches on its mGridType. + toolsModule.def("updateGridStats", + [](GridData* gridData, tools::StatsMode mode) { + if (gridData == nullptr) { + throw nb::value_error("updateGridStats: grid is None."); + } + callNanoGrid(gridData, mode); + }, + "grid"_a, "mode"_a = tools::StatsMode::Default, + nb::call_guard(), + "Recompute and write per-node statistics into the given grid in " + "place. Polymorphic over BuildT. Scalar, vector, and Boolean " + "grids accept every StatsMode (Disable / BBox / MinMax / All); " + "Boolean grids use the C++ NoopStats path internally regardless " + "of mode because there's no arithmetic min/max/avg/dev on bool. " + "Other special (quantized / index / mask) grids accept Disable " + "and BBox (the latter recomputes node bounding boxes only); " + "MinMax and All raise ValueError because their value type has " + "no arithmetic semantics."); + + // Per-BuildT getExtrema. We expose one overload per scalar/vector + // BuildT — they each return a Python-side Extrema of the + // matching name. +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + toolsModule.def("getExtrema", &pyGetExtrema, \ + "grid"_a, "bbox"_a, nb::call_guard(), \ + "Return the Extrema of all values in the grid that intersect " \ + "the given bbox."); +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + toolsModule.def("getExtrema", &pyGetExtrema, \ + "grid"_a, "bbox"_a, nb::call_guard(), \ + "Return the Extrema of all values in the grid that intersect " \ + "the given bbox."); +#include "BuildTypes.def" +} + } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyGridStats.h b/nanovdb/nanovdb/python/PyGridStats.h index 90254bc23b..07b0c3a0dd 100644 --- a/nanovdb/nanovdb/python/PyGridStats.h +++ b/nanovdb/nanovdb/python/PyGridStats.h @@ -11,6 +11,12 @@ namespace pynanovdb { void defineStatsMode(nb::module_& m); +/// @brief Register per-BuildT Extrema and Stats classes (one set per +/// scalar / vector BuildT in BuildTypes.def) and the polymorphic +/// tools.updateGridStats / tools.getExtrema helpers under the +/// nanovdb.tools submodule. +void defineGridStatsModule(nb::module_& toolsModule); + } #endif diff --git a/nanovdb/nanovdb/python/PyGridValidator.cc b/nanovdb/nanovdb/python/PyGridValidator.cc index 8e4f20df64..db9a46fc8f 100644 --- a/nanovdb/nanovdb/python/PyGridValidator.cc +++ b/nanovdb/nanovdb/python/PyGridValidator.cc @@ -2,13 +2,21 @@ // SPDX-License-Identifier: Apache-2.0 #include "PyGridValidator.h" +#include +#include +#include + #include +#include #include #ifdef NANOVDB_USE_CUDA #include #endif -#include +#include +#include +#include +#include namespace nb = nanobind; using namespace nb::literals; @@ -18,7 +26,8 @@ namespace pynanovdb { template void defineValidateGrids(nb::module_& m) { - m.def("validateGrids", &tools::validateGrids>, "handle"_a, "mode"_a, "verbose"_a); + m.def("validateGrids", &tools::validateGrids>, + "handle"_a, "mode"_a, "verbose"_a); } template void defineValidateGrids(nb::module_&); @@ -26,4 +35,126 @@ template void defineValidateGrids(nb::module_&); template void defineValidateGrids(nb::module_&); #endif +namespace { + +// callNanoGrid op for checkGrid: writes into a fixed-size buffer and returns +// it as a std::pair for nb::make_tuple consumption. +// +// tools::checkGrid writes error messages with util::sprint / util::strcpy, +// neither of which is bounded — they trust the caller's buffer is big +// enough. Size the buffer well above what any current error message +// produces (the longest formatted message in GridValidator.h is on the +// order of ~80 characters once both GridType and GridClass enumerator +// names are stringified) and leave generous headroom for future +// additions. +struct CheckGridOp +{ + static constexpr size_t kErrorBufSize = 4096; + + template + static std::pair known(const GridData* gridData, + CheckMode mode) + { + char buf[kErrorBufSize]; + buf[0] = '\0'; + tools::checkGrid( + static_cast*>(gridData), buf, mode); + const bool ok = (buf[0] == '\0'); + return {ok, std::string(buf)}; + } + static std::pair unknown(const GridData* gridData, + CheckMode /*mode*/) + { + (void)gridData; + return {false, "Unsupported GridType for checkGrid"}; + } +}; + +// callNanoGrid op for isValid — wraps tools::isValid for every +// switched-over BuildT. +struct IsValidOp +{ + template + static bool known(const GridData* gridData, CheckMode mode, bool verbose) + { + return tools::isValid( + static_cast*>(gridData), mode, verbose); + } + static bool unknown(const GridData* gridData, CheckMode /*mode*/, bool verbose) + { + if (verbose && gridData != nullptr) { + char str[16]; + std::cerr << "Validation failed: Unsupported GridType: \"" + << toStr(str, gridData->mGridType) << "\"" + << std::endl; + } + return false; + } +}; + +} // namespace + +void defineGridValidatorModule(nb::module_& toolsModule) +{ + // Single-grid validate. Takes a handle, a grid index, and the + // usual mode + verbose flags. Returns true iff the grid passes all + // tests for the given mode. Bound for both host and device handles + // (when CUDA is enabled), matching validateGrids' coverage — + // tools::validateGrid does host-side dispatch via callNanoGrid on + // the host-resident gridData() pointer that DeviceGridHandle also + // exposes, so the same overload pair is appropriate. + toolsModule.def("validateGrid", + &tools::validateGrid>, + "handle"_a, "gridID"_a, + "mode"_a = CheckMode::Default, "verbose"_a = false, + nb::call_guard(), + "Validate the gridID'th grid in the handle against the given " + "CheckMode. Returns False (without raising) if gridID is out " + "of range or the grid fails any check. CheckMode.Disable is a " + "short-circuit that always returns True without inspecting " + "the grid (even when gridID is out of range), matching the " + "C++ behavior. Complements validateGrids() which checks the " + "whole handle."); +#ifdef NANOVDB_USE_CUDA + toolsModule.def("validateGrid", + &tools::validateGrid>, + "handle"_a, "gridID"_a, + "mode"_a = CheckMode::Default, "verbose"_a = false, + nb::call_guard(), + "Validate the gridID'th grid in the device handle (uses the " + "host-resident copy of the grid metadata for the actual " + "checks). Same semantics as the host-handle overload."); +#endif + + // Polymorphic checkGrid — returns (ok, error_message). Mirrors the C++ + // char-buffer-out signature, but the buffer is hidden inside the + // binding so Python callers get a Python str. + toolsModule.def("checkGrid", + [](const GridData* gridData, CheckMode mode) + -> std::pair { + if (gridData == nullptr) { + return {false, "Grid is None"}; + } + return callNanoGrid(gridData, mode); + }, + "grid"_a, "mode"_a = CheckMode::Full, + nb::call_guard(), + "Run structural validation checks on the grid for the given " + "CheckMode. Returns a (ok, error_message) tuple — error_message " + "is empty when ok is True."); + + // Polymorphic isValid — convenience wrapper. Same as checkGrid + a + // checksum check, returning just the bool. + toolsModule.def("isValid", + [](const GridData* gridData, CheckMode mode, bool verbose) { + if (gridData == nullptr) return false; + return callNanoGrid(gridData, mode, verbose); + }, + "grid"_a, "mode"_a = CheckMode::Default, "verbose"_a = false, + nb::call_guard(), + "Return True iff the grid passes structural validation AND its " + "stored checksum matches a freshly computed one for the given " + "CheckMode."); +} + } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyGridValidator.h b/nanovdb/nanovdb/python/PyGridValidator.h index 659dede241..6ce2f2db2b 100644 --- a/nanovdb/nanovdb/python/PyGridValidator.h +++ b/nanovdb/nanovdb/python/PyGridValidator.h @@ -11,6 +11,12 @@ namespace pynanovdb { template void defineValidateGrids(nb::module_& m); +/// @brief Register tools.validateGrid (single grid in a handle), +/// tools.checkGrid (polymorphic, returns (bool, error_str)) and +/// tools.isValid (polymorphic shortcut) under the nanovdb.tools +/// submodule. Bound once; polymorphic dispatch uses callNanoGrid. +void defineGridValidatorModule(nb::module_& toolsModule); + } // namespace pynanovdb #endif diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index a5c29652c3..01378f952e 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -32,6 +32,10 @@ void defineToolsModule(nb::module_& m) defineStatsMode(m); + defineGridStatsModule(m); + defineGridValidatorModule(m); + defineEvalChecksumModule(m); + definePrimitives(m); #define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \ diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index 34ce09c274..427369e769 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -1543,6 +1543,171 @@ def test_create_vec3f_nano_grid(self): self.assertEqual(grid.gridClass(), nanovdb.GridClass.Unknown) +class TestGridStats(unittest.TestCase): + """nanovdb.tools.Extrema*, Stats*, updateGridStats, getExtrema.""" + + def _five_voxel_float_grid(self): + g = nanovdb.tools.build.FloatGrid(0.0, "stats", nanovdb.GridClass.FogVolume) + for i in range(5): + g.setValue(nanovdb.math.Coord(i, 0, 0), float(i + 1)) + return g.to_nanovdb(sMode=nanovdb.tools.StatsMode.All) + + def test_extrema_default_and_add(self): + ex = nanovdb.tools.FloatExtrema() + self.assertFalse(bool(ex)) + ex.add(2.5) + ex.add(1.0) + ex.add(7.0) + self.assertTrue(bool(ex)) + self.assertEqual(ex.min(), 1.0) + self.assertEqual(ex.max(), 7.0) + # Extrema doesn't compute averages or std deviation. + self.assertTrue(nanovdb.tools.FloatExtrema.hasMinMax()) + self.assertFalse(nanovdb.tools.FloatExtrema.hasAverage()) + self.assertFalse(nanovdb.tools.FloatExtrema.hasStdDeviation()) + + def test_stats_default_and_accumulate(self): + st = nanovdb.tools.FloatStats() + for v in (1.0, 2.0, 3.0, 4.0, 5.0): + st.add(v) + self.assertEqual(st.size(), 5) + self.assertEqual(st.min(), 1.0) + self.assertEqual(st.max(), 5.0) + self.assertAlmostEqual(st.avg(), 3.0) + self.assertAlmostEqual(st.mean(), 3.0) + # Population variance of 1..5 = (((-2)^2 + (-1)^2 + 0 + 1 + 4) / 5) = 2 + self.assertAlmostEqual(st.var(), 2.0) + self.assertAlmostEqual(st.std() ** 2, 2.0) + self.assertTrue(nanovdb.tools.FloatStats.hasAverage()) + self.assertTrue(nanovdb.tools.FloatStats.hasStdDeviation()) + + def test_get_extrema_strictly_inside_active_region(self): + # Pick a bbox strictly inside the root's active bbox so the C++ + # implementation takes the recursive-traversal branch (the + # "bbox contains root.bbox()" branch unconditionally adds the + # background value, which would muddy this assertion). With + # only the three active voxels at (1,0,0)..(3,0,0) sampled, the + # extrema should be exactly their min and max. + h = self._five_voxel_float_grid() + ng = h.grid() + ex = nanovdb.tools.getExtrema( + ng, nanovdb.math.CoordBBox( + nanovdb.math.Coord(1, 0, 0), nanovdb.math.Coord(3, 0, 0))) + self.assertTrue(bool(ex)) + self.assertEqual(ex.min(), 2.0) + self.assertEqual(ex.max(), 4.0) + + def test_update_grid_stats_polymorphic(self): + # Building with StatsMode.Disable leaves stats uncomputed; calling + # tools.updateGridStats on the resulting handle should populate + # them in-place. Asserting "no exception" is the round-trip we + # care about — the actual stats live inside the grid's nodes. + g = nanovdb.tools.build.FloatGrid(0.0) + for i in range(3): + g.setValue(nanovdb.math.Coord(i, 0, 0), float(i + 10)) + h = g.to_nanovdb(sMode=nanovdb.tools.StatsMode.Disable) + ng = h.grid() + nanovdb.tools.updateGridStats(ng, nanovdb.tools.StatsMode.All) + # checkGrid still passes after writing stats. + ok, msg = nanovdb.tools.checkGrid(ng, nanovdb.CheckMode.Full) + self.assertTrue(ok, msg) + + def test_update_grid_stats_on_index_grid(self): + # OnIndexGrid is a special BuildT — MinMax and All raise because + # Stats isn't meaningful, but BBox (NoopStats) is still + # accepted because it only touches node bounding boxes. + bbox = nanovdb.math.CoordBBox( + nanovdb.math.Coord(0), nanovdb.math.Coord(4)) + h_float = nanovdb.tools.createFloatGrid( + 0.0, "src", nanovdb.GridClass.Unknown, lambda ijk: 1.0, bbox) + h_index = nanovdb.tools.createOnIndexGrid(h_float.grid()) + ng = h_index.grid() + with self.assertRaises(ValueError): + nanovdb.tools.updateGridStats(ng, nanovdb.tools.StatsMode.MinMax) + with self.assertRaises(ValueError): + nanovdb.tools.updateGridStats(ng, nanovdb.tools.StatsMode.All) + # BBox mode is a NoopStats path — must succeed. + nanovdb.tools.updateGridStats(ng, nanovdb.tools.StatsMode.BBox) + # And Disable is a true no-op. + nanovdb.tools.updateGridStats(ng, nanovdb.tools.StatsMode.Disable) + + +class TestGridValidate(unittest.TestCase): + """nanovdb.tools.validateGrid, checkGrid, isValid.""" + + def _good_handle(self): + bbox = nanovdb.math.CoordBBox( + nanovdb.math.Coord(0), nanovdb.math.Coord(3)) + return nanovdb.tools.createFloatGrid( + 0.0, "v", nanovdb.GridClass.Unknown, lambda ijk: 1.0, bbox) + + def test_checkGrid_on_valid_grid(self): + h = self._good_handle() + ok, msg = nanovdb.tools.checkGrid(h.grid(), nanovdb.CheckMode.Full) + self.assertTrue(ok) + self.assertEqual(msg, "") + + def test_isValid_on_valid_grid(self): + h = self._good_handle() + self.assertTrue(nanovdb.tools.isValid(h.grid(), nanovdb.CheckMode.Default)) + + def test_validateGrid_on_valid_handle(self): + h = self._good_handle() + self.assertTrue(nanovdb.tools.validateGrid(h, 0)) + # validateGrid with out-of-range gridID returns False, never raises. + self.assertFalse(nanovdb.tools.validateGrid(h, 99)) + + def test_validateGrid_disable_mode_always_true(self): + h = self._good_handle() + self.assertTrue( + nanovdb.tools.validateGrid(h, 99, nanovdb.CheckMode.Disable)) + + @unittest.skipUnless( + hasattr(nanovdb.tools, "cuda") and + hasattr(nanovdb.tools.cuda, "createLevelSetSphere"), + "device handles require a CUDA-enabled build", + ) + def test_validateGrid_on_device_handle(self): + # validateGrid is bound for both host and device handles. The + # device overload routes through the same callNanoGrid dispatch + # against the host-resident copy of the grid metadata. + h = nanovdb.tools.cuda.createLevelSetSphere() + self.assertTrue(nanovdb.tools.validateGrid(h, 0)) + # Out-of-range gridID returns False (without raising); Disable + # mode short-circuits to True even on an out-of-range gridID. + self.assertFalse(nanovdb.tools.validateGrid(h, 99)) + self.assertTrue( + nanovdb.tools.validateGrid(h, 99, nanovdb.CheckMode.Disable)) + + +class TestGridChecksum(unittest.TestCase): + """nanovdb.tools.evalChecksum and validateChecksum.""" + + def _handle(self): + bbox = nanovdb.math.CoordBBox( + nanovdb.math.Coord(0), nanovdb.math.Coord(3)) + return nanovdb.tools.createFloatGrid( + 0.0, "cs", nanovdb.GridClass.Unknown, lambda ijk: 1.0, bbox) + + def test_eval_then_update_then_validate(self): + h = self._handle() + ng = h.grid() + cs1 = nanovdb.tools.evalChecksum(ng, nanovdb.CheckMode.Full) + nanovdb.tools.updateChecksum(ng, nanovdb.CheckMode.Full) + cs2 = nanovdb.tools.evalChecksum(ng, nanovdb.CheckMode.Full) + # Recomputing on an unchanged grid gives the same checksum. + self.assertEqual(cs1, cs2) + self.assertTrue( + nanovdb.tools.validateChecksum(ng, nanovdb.CheckMode.Full)) + + def test_validate_empty_stored_returns_true(self): + # A grid with no stored checksum is considered valid by the C++ + # rule (Checksum.isEmpty() short-circuit). + h = self._handle() + self.assertTrue( + nanovdb.tools.validateChecksum(h.grid(), nanovdb.CheckMode.Default)) + + class TestBuildGrid(unittest.TestCase): """nanovdb.tools.build.* — mutable voxel-by-voxel CPU grid builder.""" From f22bf6a8f77c19f11cc496ee2a9eec0d565b4839 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 21 May 2026 23:28:42 +1200 Subject: [PATCH 08/48] =?UTF-8?q?nanovdb=20python:=20Phase=205=20=E2=80=94?= =?UTF-8?q?=20primitives=20+=20quantized/index=20createNanoGrid=20(#2216)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: Phase 5 — primitives + quantized/index createNanoGrid Closes out Phase 5 of the NanoVDB Python bindings restructure (#2208). Lands all three sub-phases (5a primitives, 5b quantized createNanoGrid overloads, 5c generic createNanoGrid from build::Grid) in one PR. Phase 5a — Host primitives ========================== Bind the nine primitives that didn't ship with Phase 0: * tools.createLevelSetBox(gridType, width, height, depth, ...) — narrow- band level set of a solid axis-aligned box. * tools.createLevelSetBBox(gridType, width, height, depth, thickness, ...) — narrow-band level set of a hollow box wireframe. * tools.createLevelSetOctahedron(gridType, scale, ...) — narrow-band level set of an octahedron. * tools.createFogVolumeBox(...) and tools.createFogVolumeOctahedron(...) — the fog-volume counterparts of the above. * tools.createPointSphere / createPointTorus / createPointBox(gridType, pointsPerVoxel, ...) — PointDataGrids scattered on the surface of each primitive shape. * tools.createPointScatter(srcGrid, pointsPerVoxel, ...) — scatter a PointDataGrid into the active voxels of a NanoGrid level set or fog volume. The C++ template also accepts double sources; the binding is float-only for simplicity (the runtime grid pointer carries the source BuildT and adding a per-type dispatch is mechanical follow-up). Each non-point primitive is instantiated for float and double via the same runtime-GridType switch the existing createLevelSetSphere et al. already use. The FpN overloads of these primitives are deliberately not bound here — quantization is reachable via Phase 5b's generic createNanoGrid* path with explicit oracle and dither parameters. Phase 5b — Quantized createNanoGrid overloads ============================================= * tools.AbsDiff(tolerance=-1.0) and tools.RelDiff(tolerance=-1.0) — compression-oracle classes used by FpN. Both expose getTolerance / setTolerance and a truthy __bool__ that returns True iff the tolerance has been initialized (>= 0). The default tolerance of -1 matches the C++ "uninitialized — fill in via init()" sentinel. * tools.createNanoGridFp4 / Fp8 / Fp16(src, sMode, cMode, ditherOn, verbose) — quantize a float source into a fixed-bit-width grid. ditherOn adds sub-quantum noise to break up banding. * tools.createNanoGridFpN(src, oracle, sMode, cMode, ditherOn, verbose) — variable-bit-width quantization. Two overloads: one accepting an AbsDiff oracle (the default), one accepting a RelDiff oracle. Python picks the right overload from the oracle argument's type. The C++ Fp{4,8,16,N} preProcess templates static_assert SrcValueT == float, so the binding rejects double sources with a TypeError instead of letting the compile-time assertion trip a hard abort. Phase 5c — Generic createNanoGrid from build::Grid ================================================== Each conversion entry accepts both NanoGrid and tools::build::Grid as its source. Internally a small template helper tries each source-side BuildT in turn (matching the existing tryCreateOnIndexGrid pattern from the Phase 3 follow-up) and dispatches via nb::isinstance. * tools.createNanoGridIndex(src, channels=0, includeStats=True, includeTiles=True, verbose=0) — NEW. Bake any supported source into a NanoGrid with all voxels (active and inactive) given a uint64 sequential index. Original values can be carried as blind data when channels > 0. * tools.createNanoGridOnIndex(src, ...) — same as Index but only the active voxels get a sequential index. Supersedes the Phase 3 follow-up's tools.createOnIndexGrid test scaffold (which is kept alive in PyVoxelBlockManager.cc for backwards compatibility with the VBM tests; new code should prefer createNanoGridOnIndex). Source types accepted by the index path are NanoGrid and tools::build::Grid. The same-type bake of build::Grid -> NanoGrid remains served by the Phase 4a .to_nanovdb() method. Test plan ========= Three new TestCase classes covering: every primitive (TestNewPrimitives, 11 cases), every quantization path including double-source rejection and oracle defaults (TestCreateNanoGridQuantized, 8 cases), and every index/onindex source type including the build::Grid path (TestCreateNanoGridIndex, 6 cases). All 25 new cases pass locally; the full pytest_nanovdb suite stays green except for the two pre-existing BLOSC-disabled errors in my local build. Signed-off-by: Jonathan Swartz * nanovdb python: address Copilot review on #2216 Four items from the Phase 5 review, plus a follow-on segfault fix caught while addressing the third item: 1. openToNanoVDB binding accidentally dropped the sMode argument and gave `base` a default of tools::StatsMode::Default — nonsense both semantically (base is an OpenVDB GridBase::Ptr) and structurally (the C++ template expects 4 args, the binding declared 3 keyword args). The CI build broke for openvdb-enabled configs with a "number of nb::arg annotations must match the argument count" static_assert from nanobind. Restore the correct signature: base has no default, sMode/cMode/verbose carry their defaults. 2. Octahedron primitive default name strings carried the upstream C++ typo "octadedron" (sic). The C++ side keeps the misspelling for binary compatibility with old grids; correct it on the Python side since the default propagates into help() output and grid metadata where end-users see it. 3. The point primitives took a `gridType` argument that was misleading — it controls the intermediate level-set's precision the scatter starts from, not the returned PointDataGrid's type (which is always UInt32). While investigating the rename, I found that the createPointSphere / createPointTorus / createPointBox code paths actually segfault during scatter with the current C++ implementation (only the float path is exercised by the C++ unit tests). Take Copilot's "remove it and always use float" alternative: drop the argument entirely. The binding now always uses the float intermediate level-set. createPointSphere / Torus / Box no longer have the parameter; their docstrings explicitly note "always returns a UInt32 PointDataGrid". 4. The defineCreateNanoGridConversions doc-comment claimed sources were accepted for "every scalar/vector BuildT in BuildTypes.def that tools::createNanoGrid supports", which was wishful — the implementation actually tries a narrower set per destination kind. Spell out the actual matrix: quantized Fp{4,8,16,N} paths accept float only (the C++ preProcess static-asserts SrcValueT == float); the createNanoGridIndex / OnIndex paths accept float / double / int32 / Vec3f. Adding more source types is a one-line extension of the explicit try-each-SrcBuildT chains. Signed-off-by: Jonathan Swartz * nanovdb python: address more Copilot review on #2216 + CI GPU gating Four items in this round, three from Copilot plus a CI failure caught on the linux-clang++ runner. 1. test_validateGrid_on_device_handle was gated only on the cuda submodule being importable, but the linux-clang++ CI runner has a CUDA toolkit installed (so the submodule loads) but no GPU driver. The test then crashed inside the C++ helper with "CUDA error 35: CUDA driver version is insufficient for CUDA runtime version". Switch the gate to the project's existing pair of helpers — nanovdb.isCudaAvailable() (build-time CUDA support) AND nanovdb.isGpuAvailable() (runtime device probe) — matching how TestPointsToGrid / TestSignedFloodFill / TestSampleFromPoints already gate. The test skips cleanly on CPU-only runners and driverless CUDA runners alike. 2. The Phase 5b/5c conversion helpers (tryQuantizeFpX, tryQuantizeFpN, tryIndexify) ran the heavy tools::createNanoGrid traversal while holding the Python GIL. Restructure each helper so the GIL is held only for the isinstance / cast dispatch, then released around the conversion itself (the source data lives in stable C++ storage anchored by the Python wrapper passed in via py_src, so it's safe to read without the GIL). Other Python threads can now make progress during large-grid quantization / indexing. 3. AbsDiff's class docstring said "pass an explicit positive value" for the tolerance, but the C++ operator bool() actually treats any non-negative value (including 0.0) as initialized. Reword to match what the API actually does. 4. The createNanoGridIndex/OnIndex source-rejection test was named test_index_rejects_unsupported_source with a comment about "bool / Boolean grids" but really only exercised the None case. Split into two cases with single, accurate purposes: - test_index_rejects_none — passes None; matches neither the NanoGrid nor the build::Grid isinstance arms. - test_index_rejects_unsupported_buildt — builds a tools.build.Vec3dGrid (structurally valid but a BuildT outside the float/double/int32/Vec3f source set) and confirms the try-each-SrcBuildT chain falls through to a TypeError. Signed-off-by: Jonathan Swartz * nanovdb python: correct createPointScatter docs + add fog-rejection test Copilot noted the createPointScatter binding docstring claimed the source could be a "level set or fog volume", but the C++ implementation explicitly checks srcGrid.isLevelSet() at line 1679 of tools/CreatePrimitives.h and throws std::runtime_error("Expected a level set grid") otherwise. Match the actual behavior: the docstring now says the source must satisfy srcGrid.isLevelSet() and that non-level-set sources (e.g. fog volumes) raise RuntimeError. Add test_create_point_scatter_rejects_fog_volume to lock in the behavior (build a fog volume sphere, confirm createPointScatter raises RuntimeError). Signed-off-by: Jonathan Swartz * nanovdb python: fully-qualify nanovdb.tools.build.* in 5b/5c docs Copilot noted the Phase 5b/5c TypeError messages and per-function docstrings referred to the mutable build grids as "tools.build.FloatGrid", but the actual Python import path is "nanovdb.tools.build.FloatGrid". Replace every occurrence so help() / autocomplete / error messages point at a path users can actually import. Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyCreateNanoGrid.cc | 320 ++++++++++++++++++- nanovdb/nanovdb/python/PyCreateNanoGrid.h | 12 + nanovdb/nanovdb/python/PyPrimitives.cc | 337 +++++++++++++++++++++ nanovdb/nanovdb/python/PyTools.cc | 2 + nanovdb/nanovdb/python/test/TestNanoVDB.py | 208 ++++++++++++- 5 files changed, 874 insertions(+), 5 deletions(-) diff --git a/nanovdb/nanovdb/python/PyCreateNanoGrid.cc b/nanovdb/nanovdb/python/PyCreateNanoGrid.cc index afe70287b8..c132b84e4e 100644 --- a/nanovdb/nanovdb/python/PyCreateNanoGrid.cc +++ b/nanovdb/nanovdb/python/PyCreateNanoGrid.cc @@ -11,6 +11,8 @@ #include #include +#include + namespace nb = nanobind; using namespace nb::literals; using namespace nanovdb; @@ -41,10 +43,326 @@ template void defineCreateNanoGrid(nb::module_& m, const char* template void defineOpenToNanoVDB(nb::module_& m) { #ifdef NANOVDB_USE_OPENVDB - m.def("openToNanoVDB", &tools::openToNanoVDB, "base"_a, "sMode"_a = tools::StatsMode::Default, "cMode"_a = CheckMode::Default, "verbose"_a = 0); + m.def("openToNanoVDB", &tools::openToNanoVDB, + "base"_a, + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, + "verbose"_a = 0); #endif } +// ============================================================================ +// Phase 5b/5c conversion bindings: AbsDiff/RelDiff oracle classes, and the +// polymorphic createNanoGrid free functions for quantized + index destination +// BuildTs. Each accepts source = NanoGrid OR build::Grid. +// ============================================================================ + +namespace { + +// ----- Quantized (Fp4/Fp8/Fp16) ----- +// +// C++ signature: createNanoGrid(srcGrid, +// sMode, cMode, ditherOn, verbose, buffer). +// +// Try SrcBuildT against both NanoGrid and build::Grid +// and return an empty nb::object on no match so the caller can fall through +// to the next SrcBuildT. +// +// GIL is held for the isinstance / cast dispatch (which touches the Python +// object's type and reference graph) but released around the underlying +// tools::createNanoGrid traversal — the source data lives in stable C++ +// storage whose lifetime is anchored by the Python wrapper passed in via +// py_src, so it's safe to read without holding the GIL. +template +nb::object tryQuantizeFpX(nb::handle py_src, + tools::StatsMode sMode, + CheckMode cMode, + bool ditherOn, + int verbose) +{ + using NanoSrcT = NanoGrid; + using BuildSrcT = tools::build::Grid; + if (nb::isinstance(py_src)) { + const auto& src = nb::cast(py_src); + GridHandle handle; + { + nb::gil_scoped_release release; + handle = tools::createNanoGrid( + src, sMode, cMode, ditherOn, verbose); + } + return nb::cast(std::move(handle)); + } + if (nb::isinstance(py_src)) { + const auto& src = nb::cast(py_src); + GridHandle handle; + { + nb::gil_scoped_release release; + handle = tools::createNanoGrid( + src, sMode, cMode, ditherOn, verbose); + } + return nb::cast(std::move(handle)); + } + return nb::object(); +} + +template +nb::object createNanoGridFpX(nb::handle py_src, + tools::StatsMode sMode, + CheckMode cMode, + bool ditherOn, + int verbose, + const char* pyFnName) +{ + // The C++ Fp{4,8,16,N} preProcess static_asserts SrcValueT == float; + // double sources hit a compile-time error, so we accept float only. + if (auto r = tryQuantizeFpX( + py_src, sMode, cMode, ditherOn, verbose); r.is_valid()) return r; + std::string msg(pyFnName); + msg += ": source must be a FloatGrid or nanovdb.tools.build.FloatGrid " + "(Fp4/Fp8/Fp16/FpN require a float source value type)."; + throw nb::type_error(msg.c_str()); +} + +// ----- FpN (variable bit-width) ----- +// +// C++ signature: createNanoGrid(srcGrid, +// sMode, cMode, ditherOn, verbose, oracle, buffer). OracleT is AbsDiff or +// RelDiff; the binding exposes both as separate Python overloads. +template +nb::object tryQuantizeFpN(nb::handle py_src, + tools::StatsMode sMode, + CheckMode cMode, + bool ditherOn, + int verbose, + const OracleT& oracle) +{ + using NanoSrcT = NanoGrid; + using BuildSrcT = tools::build::Grid; + // Same GIL pattern as tryQuantizeFpX: hold the GIL through the + // isinstance / cast dispatch, release it for the conversion. + if (nb::isinstance(py_src)) { + const auto& src = nb::cast(py_src); + GridHandle handle; + { + nb::gil_scoped_release release; + handle = tools::createNanoGrid( + src, sMode, cMode, ditherOn, verbose, oracle); + } + return nb::cast(std::move(handle)); + } + if (nb::isinstance(py_src)) { + const auto& src = nb::cast(py_src); + GridHandle handle; + { + nb::gil_scoped_release release; + handle = tools::createNanoGrid( + src, sMode, cMode, ditherOn, verbose, oracle); + } + return nb::cast(std::move(handle)); + } + return nb::object(); +} + +template +nb::object createNanoGridFpNImpl(nb::handle py_src, + const OracleT& oracle, + tools::StatsMode sMode, + CheckMode cMode, + bool ditherOn, + int verbose) +{ + if (auto r = tryQuantizeFpN( + py_src, sMode, cMode, ditherOn, verbose, oracle); r.is_valid()) return r; + throw nb::type_error( + "createNanoGridFpN: source must be a FloatGrid or " + "nanovdb.tools.build.FloatGrid (FpN requires a float source value type)."); +} + +// ----- Index / OnIndex ----- +// +// C++ signature: createNanoGrid(srcGrid, +// channels, includeStats, includeTiles, verbose, buffer). DstBuildT is +// ValueIndex or ValueOnIndex; the binding exposes both as separate +// named functions. Source set is wider than the quantized variants — +// any arithmetic or vector source can be re-cast as an index grid. +template +nb::object tryIndexify(nb::handle py_src, + uint32_t channels, + bool includeStats, + bool includeTiles, + int verbose) +{ + using NanoSrcT = NanoGrid; + using BuildSrcT = tools::build::Grid; + // Same GIL pattern as tryQuantizeFpX. + if (nb::isinstance(py_src)) { + const auto& src = nb::cast(py_src); + GridHandle handle; + { + nb::gil_scoped_release release; + handle = tools::createNanoGrid( + src, channels, includeStats, includeTiles, verbose); + } + return nb::cast(std::move(handle)); + } + if (nb::isinstance(py_src)) { + const auto& src = nb::cast(py_src); + GridHandle handle; + { + nb::gil_scoped_release release; + handle = tools::createNanoGrid( + src, channels, includeStats, includeTiles, verbose); + } + return nb::cast(std::move(handle)); + } + return nb::object(); +} + +template +nb::object createIndexImpl(nb::handle py_src, + uint32_t channels, + bool includeStats, + bool includeTiles, + int verbose, + const char* pyFnName) +{ + if (auto r = tryIndexify(py_src, channels, includeStats, includeTiles, verbose); r.is_valid()) return r; + if (auto r = tryIndexify(py_src, channels, includeStats, includeTiles, verbose); r.is_valid()) return r; + if (auto r = tryIndexify(py_src, channels, includeStats, includeTiles, verbose); r.is_valid()) return r; + if (auto r = tryIndexify(py_src, channels, includeStats, includeTiles, verbose); r.is_valid()) return r; + std::string msg(pyFnName); + msg += ": source must be a FloatGrid, DoubleGrid, Int32Grid, " + "Vec3fGrid, or the matching nanovdb.tools.build.* mutable grid."; + throw nb::type_error(msg.c_str()); +} + +} // namespace + +void defineCreateNanoGridConversions(nb::module_& toolsModule) +{ + // ------ Oracle classes ------ + nb::class_(toolsModule, "AbsDiff", + "Compression oracle for FpN: accept the approximation when " + "|exact - approx| <= tolerance. A tolerance of -1.0 (the " + "default) means uninitialized; any non-negative value (including " + "0.0) is treated as initialized by the operator bool() check, " + "or the C++ create function can fill it in via init().") + .def(nb::init(), "tolerance"_a = -1.0f) + .def("getTolerance", &tools::AbsDiff::getTolerance) + .def("setTolerance", &tools::AbsDiff::setTolerance, "tolerance"_a) + .def("__bool__", + [](const tools::AbsDiff& self) { return bool(self); }, + "True iff the tolerance has been initialized (>= 0)."); + + nb::class_(toolsModule, "RelDiff", + "Compression oracle for FpN: accept the approximation when " + "|exact - approx| / max(|exact|, |approx|) <= tolerance.") + .def(nb::init(), "tolerance"_a = -1.0f) + .def("getTolerance", &tools::RelDiff::getTolerance) + .def("setTolerance", &tools::RelDiff::setTolerance, "tolerance"_a) + .def("__bool__", + [](const tools::RelDiff& self) { return bool(self); }, + "True iff the tolerance has been initialized (>= 0)."); + + // ------ Quantized fixed-width: Fp4 / Fp8 / Fp16 ------ + toolsModule.def("createNanoGridFp4", + [](nb::handle src, tools::StatsMode sMode, CheckMode cMode, + bool ditherOn, int verbose) { + return createNanoGridFpX(src, sMode, cMode, ditherOn, verbose, + "createNanoGridFp4"); + }, + "src"_a, "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0, + "Quantize a NanoGrid or nanovdb.tools.build.FloatGrid into a " + "NanoGrid (4 bits per voxel). ditherOn adds sub-quantum " + "noise to break up banding."); + + toolsModule.def("createNanoGridFp8", + [](nb::handle src, tools::StatsMode sMode, CheckMode cMode, + bool ditherOn, int verbose) { + return createNanoGridFpX(src, sMode, cMode, ditherOn, verbose, + "createNanoGridFp8"); + }, + "src"_a, "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0, + "Quantize a NanoGrid or nanovdb.tools.build.FloatGrid into a NanoGrid (8 bits " + "per voxel). ditherOn adds sub-quantum noise."); + + toolsModule.def("createNanoGridFp16", + [](nb::handle src, tools::StatsMode sMode, CheckMode cMode, + bool ditherOn, int verbose) { + return createNanoGridFpX(src, sMode, cMode, ditherOn, verbose, + "createNanoGridFp16"); + }, + "src"_a, "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0, + "Quantize a NanoGrid or nanovdb.tools.build.FloatGrid into a NanoGrid (16 bits " + "per voxel). ditherOn adds sub-quantum noise."); + + // ------ Variable bit-width: FpN with AbsDiff or RelDiff oracle ------ + // + // Two overloads — one per oracle type. Python dispatch picks the + // right one from the oracle argument's type. The createNanoGrid C++ + // template's parameter order is (src, sMode, cMode, ditherOn, verbose, + // oracle, buffer); we reorder for Python so oracle comes second + // (most callers want to specify it explicitly), then mode/dither + // parameters as kwargs with defaults. + toolsModule.def("createNanoGridFpN", + [](nb::handle src, const tools::AbsDiff& oracle, + tools::StatsMode sMode, CheckMode cMode, bool ditherOn, int verbose) { + return createNanoGridFpNImpl(src, oracle, sMode, cMode, ditherOn, verbose); + }, + "src"_a, "oracle"_a = tools::AbsDiff(), + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0, + "Quantize a NanoGrid or nanovdb.tools.build.FloatGrid into a NanoGrid (variable " + "bits per voxel; each leaf picks the smallest N that satisfies " + "the oracle's tolerance). Pass an AbsDiff oracle for absolute " + "error bound, or use the RelDiff overload for relative error."); + + toolsModule.def("createNanoGridFpN", + [](nb::handle src, const tools::RelDiff& oracle, + tools::StatsMode sMode, CheckMode cMode, bool ditherOn, int verbose) { + return createNanoGridFpNImpl(src, oracle, sMode, cMode, ditherOn, verbose); + }, + "src"_a, "oracle"_a, + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0, + "FpN overload accepting a RelDiff oracle for relative error."); + + // ------ Index / OnIndex ------ + // + // createOnIndexGrid (the test-scaffold factory from Phase 3 follow-up) + // is now superseded by createNanoGridOnIndex. The legacy name keeps + // working through PyVoxelBlockManager.cc; the official Phase 5 name + // lives here. + toolsModule.def("createNanoGridIndex", + [](nb::handle src, uint32_t channels, bool includeStats, + bool includeTiles, int verbose) { + return createIndexImpl( + src, channels, includeStats, includeTiles, verbose, + "createNanoGridIndex"); + }, + "src"_a, "channels"_a = 0u, "includeStats"_a = true, + "includeTiles"_a = true, "verbose"_a = 0, + "Convert a source grid into a NanoGrid. Every voxel " + "(active or inactive) gets a unique uint64 sequential index, with " + "the original values stored as blind data when channels > 0."); + + toolsModule.def("createNanoGridOnIndex", + [](nb::handle src, uint32_t channels, bool includeStats, + bool includeTiles, int verbose) { + return createIndexImpl( + src, channels, includeStats, includeTiles, verbose, + "createNanoGridOnIndex"); + }, + "src"_a, "channels"_a = 0u, "includeStats"_a = true, + "includeTiles"_a = true, "verbose"_a = 0, + "Convert a source grid into a NanoGrid. Only the " + "active voxels get a sequential index — the canonical input to " + "buildVoxelBlockManager."); +} + #define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \ template void defineCreateNanoGrid(nb::module_&, const char*); #include "BuildTypes.def" diff --git a/nanovdb/nanovdb/python/PyCreateNanoGrid.h b/nanovdb/nanovdb/python/PyCreateNanoGrid.h index 3402ca67b8..9e5f188e4c 100644 --- a/nanovdb/nanovdb/python/PyCreateNanoGrid.h +++ b/nanovdb/nanovdb/python/PyCreateNanoGrid.h @@ -15,6 +15,18 @@ template void defineCreateNanoGrid(nb::module_& m, const char* template void defineOpenToNanoVDB(nb::module_& m); #endif +/// @brief Bind the AbsDiff / RelDiff quantization oracle classes and the +/// polymorphic createNanoGridFp4 / Fp8 / Fp16 / FpN / Index / OnIndex +/// free functions on the nanovdb.tools submodule. Sources accepted +/// include both NanoGrid and tools::build::Grid: +/// the quantized createNanoGridFp* / FpN paths accept float only +/// (C++ Fp{4,8,16,N}::preProcess static-asserts SrcValueT == float); +/// the createNanoGridIndex / OnIndex paths accept float, double, +/// int32_t, and Vec3f sources. Additional source BuildTs can be +/// added by extending the explicit try-each-SrcBuildT chains in +/// createNanoGridFpX / FpNImpl / createIndexImpl. +void defineCreateNanoGridConversions(nb::module_& toolsModule); + } // namespace pynanovdb #endif diff --git a/nanovdb/nanovdb/python/PyPrimitives.cc b/nanovdb/nanovdb/python/PyPrimitives.cc index 29053d4e68..e2a8f9175f 100644 --- a/nanovdb/nanovdb/python/PyPrimitives.cc +++ b/nanovdb/nanovdb/python/PyPrimitives.cc @@ -115,6 +115,212 @@ GridHandle createFogVolumeTorus(GridType gridType, } } +// ---------- New primitives ---------- +// Same float/double switch pattern as the four existing primitives above. +// The C++ templates also accept Fp4/Fp8/Fp16/FpN; those flavors live on +// the createNanoGrid path with explicit oracle and dither parameters +// rather than being expressed as primitive overloads here. + +template +GridHandle createLevelSetBox(GridType gridType, + double width, + double height, + double depth, + const Vec3d& center, + double voxelSize, + double halfWidth, + const Vec3d& origin, + const std::string& name, + tools::StatsMode sMode, + CheckMode cMode, + const BufferT& buffer) +{ + switch (gridType) { + case GridType::Float: + return tools::createLevelSetBox( + width, height, depth, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + case GridType::Double: + return tools::createLevelSetBox( + width, height, depth, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + default: + throw std::runtime_error( + "createLevelSetBox: only float and double grid types are supported"); + } +} + +template +GridHandle createLevelSetBBox(GridType gridType, + double width, + double height, + double depth, + double thickness, + const Vec3d& center, + double voxelSize, + double halfWidth, + const Vec3d& origin, + const std::string& name, + tools::StatsMode sMode, + CheckMode cMode, + const BufferT& buffer) +{ + switch (gridType) { + case GridType::Float: + return tools::createLevelSetBBox( + width, height, depth, thickness, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + case GridType::Double: + return tools::createLevelSetBBox( + width, height, depth, thickness, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + default: + throw std::runtime_error( + "createLevelSetBBox: only float and double grid types are supported"); + } +} + +template +GridHandle createLevelSetOctahedron(GridType gridType, + double scale, + const Vec3d& center, + double voxelSize, + double halfWidth, + const Vec3d& origin, + const std::string& name, + tools::StatsMode sMode, + CheckMode cMode, + const BufferT& buffer) +{ + switch (gridType) { + case GridType::Float: + return tools::createLevelSetOctahedron( + scale, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + case GridType::Double: + return tools::createLevelSetOctahedron( + scale, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + default: + throw std::runtime_error( + "createLevelSetOctahedron: only float and double grid types are supported"); + } +} + +template +GridHandle createFogVolumeBox(GridType gridType, + double width, + double height, + double depth, + const Vec3d& center, + double voxelSize, + double halfWidth, + const Vec3d& origin, + const std::string& name, + tools::StatsMode sMode, + CheckMode cMode, + const BufferT& buffer) +{ + switch (gridType) { + case GridType::Float: + return tools::createFogVolumeBox( + width, height, depth, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + case GridType::Double: + return tools::createFogVolumeBox( + width, height, depth, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + default: + throw std::runtime_error( + "createFogVolumeBox: only float and double grid types are supported"); + } +} + +template +GridHandle createFogVolumeOctahedron(GridType gridType, + double scale, + const Vec3d& center, + double voxelSize, + double halfWidth, + const Vec3d& origin, + const std::string& name, + tools::StatsMode sMode, + CheckMode cMode, + const BufferT& buffer) +{ + switch (gridType) { + case GridType::Float: + return tools::createFogVolumeOctahedron( + scale, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + case GridType::Double: + return tools::createFogVolumeOctahedron( + scale, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); + default: + throw std::runtime_error( + "createFogVolumeOctahedron: only float and double grid types are supported"); + } +} + +// Point primitives. The result is always a PointDataGrid (uint32 storage), +// so unlike the level-set / fog-volume primitives there's no value-type +// dispatch worth exposing — the intermediate level-set's precision is +// not user-controllable in this binding. The C++ template also accepts +// BuildT=double, but that path segfaults during scatter at least with the +// current C++ implementation, so the binding stays on the float-only +// instantiation that's exercised by the C++ unit tests. +template +GridHandle createPointSphere(int pointsPerVoxel, + double radius, + const Vec3d& center, + double voxelSize, + const Vec3d& origin, + const std::string& name, + CheckMode mode, + const BufferT& buffer) +{ + return tools::createPointSphere( + pointsPerVoxel, radius, center, voxelSize, origin, name, mode, buffer); +} + +template +GridHandle createPointTorus(int pointsPerVoxel, + double majorRadius, + double minorRadius, + const Vec3d& center, + double voxelSize, + const Vec3d& origin, + const std::string& name, + CheckMode cMode, + const BufferT& buffer) +{ + return tools::createPointTorus( + pointsPerVoxel, majorRadius, minorRadius, center, voxelSize, origin, name, cMode, buffer); +} + +template +GridHandle createPointBox(int pointsPerVoxel, + double width, + double height, + double depth, + const Vec3d& center, + double voxelSize, + const Vec3d& origin, + const std::string& name, + CheckMode mode, + const BufferT& buffer) +{ + return tools::createPointBox( + pointsPerVoxel, width, height, depth, center, voxelSize, origin, name, mode, buffer); +} + +// createPointScatter takes an existing level set as its source. We bind +// the float source variant — the C++ template also accepts double, but +// the existing primitives and tests use float, and the source grid is +// the runtime-typed nanovdb::NanoGrid, so a single overload keeps +// the Python surface simple. +template +GridHandle createPointScatter(const NanoGrid& srcGrid, + int pointsPerVoxel, + const std::string& name, + CheckMode mode, + const BufferT& buffer) +{ + return tools::createPointScatter( + srcGrid, pointsPerVoxel, name, mode, buffer); +} + } // namespace template void definePrimitives(nb::module_& m) @@ -194,6 +400,137 @@ template void definePrimitives(nb::module_& m) "sMode"_a = tools::StatsMode::Default, "cMode"_a = CheckMode::Default, "buffer"_a = BufferT()); + + // ---------- Level-set / fog-volume primitives added in Phase 5a ---------- + m.def("createLevelSetBox", &createLevelSetBox, + "gridType"_a = GridType::Float, + "width"_a = 40.0, + "height"_a = 60.0, + "depth"_a = 100.0, + "center"_a = Vec3d(0.0), + "voxelSize"_a = 1.0, + "halfWidth"_a = 3.0, + "origin"_a = Vec3d(0.0), + "name"_a = "box_ls", + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "Narrow-band level set of an axis-aligned box."); + + m.def("createLevelSetBBox", &createLevelSetBBox, + "gridType"_a = GridType::Float, + "width"_a = 40.0, + "height"_a = 60.0, + "depth"_a = 100.0, + "thickness"_a = 10.0, + "center"_a = Vec3d(0.0), + "voxelSize"_a = 1.0, + "halfWidth"_a = 3.0, + "origin"_a = Vec3d(0.0), + "name"_a = "bbox_ls", + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "Narrow-band level set of a hollow box wireframe (BBox = bounding " + "box edges with the given thickness)."); + + m.def("createLevelSetOctahedron", &createLevelSetOctahedron, + "gridType"_a = GridType::Float, + "scale"_a = 100.0, + "center"_a = Vec3d(0.0), + "voxelSize"_a = 1.0, + "halfWidth"_a = 3.0, + "origin"_a = Vec3d(0.0), + // Default name spells the shape correctly even though the + // upstream C++ default still carries the historical + // "octadedron_ls" typo. Callers can override either way. + "name"_a = "octahedron_ls", + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "Narrow-band level set of an octahedron."); + + m.def("createFogVolumeBox", &createFogVolumeBox, + "gridType"_a = GridType::Float, + "width"_a = 40.0, + "height"_a = 60.0, + "depth"_a = 100.0, + "center"_a = Vec3d(0.0), + "voxelSize"_a = 1.0, + "halfWidth"_a = 3.0, + "origin"_a = Vec3d(0.0), + "name"_a = "box_fog", + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "Sparse fog volume of a box (exterior 0/inactive, interior active " + "with values smoothly varying from 0 at the surface to 1 inside)."); + + m.def("createFogVolumeOctahedron", &createFogVolumeOctahedron, + "gridType"_a = GridType::Float, + "scale"_a = 100.0, + "center"_a = Vec3d(0.0), + "voxelSize"_a = 1.0, + "halfWidth"_a = 3.0, + "origin"_a = Vec3d(0.0), + "name"_a = "octahedron_fog", + "sMode"_a = tools::StatsMode::Default, + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "Sparse fog volume of an octahedron."); + + // ---------- Point primitives added in Phase 5a ---------- + m.def("createPointSphere", &createPointSphere, + "pointsPerVoxel"_a = 1, + "radius"_a = 100.0, + "center"_a = Vec3d(0.0), + "voxelSize"_a = 1.0, + "origin"_a = Vec3d(0.0), + "name"_a = "sphere_points", + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "PointDataGrid of points scattered on the surface of a sphere. " + "The output grid is always a UInt32 PointDataGrid; the " + "intermediate level-set's value type is hard-coded to float."); + + m.def("createPointTorus", &createPointTorus, + "pointsPerVoxel"_a = 1, + "majorRadius"_a = 100.0, + "minorRadius"_a = 50.0, + "center"_a = Vec3d(0.0), + "voxelSize"_a = 1.0, + "origin"_a = Vec3d(0.0), + "name"_a = "torus_points", + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "PointDataGrid of points scattered on the surface of a torus. " + "Always returns a UInt32 PointDataGrid."); + + m.def("createPointBox", &createPointBox, + "pointsPerVoxel"_a = 1, + "width"_a = 40.0, + "height"_a = 60.0, + "depth"_a = 100.0, + "center"_a = Vec3d(0.0), + "voxelSize"_a = 1.0, + "origin"_a = Vec3d(0.0), + "name"_a = "box_points", + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "PointDataGrid of points scattered on the surface of a box. " + "Always returns a UInt32 PointDataGrid."); + + m.def("createPointScatter", &createPointScatter, + "srcGrid"_a, + "pointsPerVoxel"_a = 1, + "name"_a = "point_scatter", + "cMode"_a = CheckMode::Default, + "buffer"_a = BufferT(), + "Scatter a PointDataGrid into the active voxels of a " + "NanoGrid level set. The source grid must satisfy " + "srcGrid.isLevelSet() and have an active bounding box; " + "non-level-set sources (e.g. fog volumes) raise RuntimeError. " + "Point coordinates are stored as blind data in world space."); } template void definePrimitives(nb::module_&); diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 01378f952e..9aa8f93452 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -36,6 +36,8 @@ void defineToolsModule(nb::module_& m) defineGridValidatorModule(m); defineEvalChecksumModule(m); + defineCreateNanoGridConversions(m); + definePrimitives(m); #define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \ diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index 427369e769..59215960aa 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -1543,6 +1543,203 @@ def test_create_vec3f_nano_grid(self): self.assertEqual(grid.gridClass(), nanovdb.GridClass.Unknown) +class TestNewPrimitives(unittest.TestCase): + """Phase 5a: the 9 host primitives that didn't ship in Phase 0.""" + + def test_create_level_set_box(self): + h = nanovdb.tools.createLevelSetBox(width=10.0, height=15.0, depth=20.0) + self.assertEqual(h.gridCount(), 1) + self.assertEqual(h.gridType(0), nanovdb.GridType.Float) + self.assertGreater(h.grid().activeVoxelCount(), 0) + self.assertEqual(h.grid().gridClass(), nanovdb.GridClass.LevelSet) + + def test_create_level_set_box_double(self): + h = nanovdb.tools.createLevelSetBox( + gridType=nanovdb.GridType.Double, width=10.0) + self.assertEqual(h.gridType(0), nanovdb.GridType.Double) + + def test_create_level_set_bbox(self): + h = nanovdb.tools.createLevelSetBBox( + width=40.0, height=40.0, depth=40.0, thickness=5.0) + self.assertEqual(h.gridType(0), nanovdb.GridType.Float) + self.assertGreater(h.grid().activeVoxelCount(), 0) + + def test_create_level_set_octahedron(self): + h = nanovdb.tools.createLevelSetOctahedron(scale=20.0) + self.assertEqual(h.gridType(0), nanovdb.GridType.Float) + self.assertGreater(h.grid().activeVoxelCount(), 0) + + def test_create_fog_volume_box(self): + h = nanovdb.tools.createFogVolumeBox(width=10.0) + self.assertEqual(h.gridType(0), nanovdb.GridType.Float) + self.assertEqual(h.grid().gridClass(), nanovdb.GridClass.FogVolume) + self.assertGreater(h.grid().activeVoxelCount(), 0) + + def test_create_fog_volume_octahedron(self): + h = nanovdb.tools.createFogVolumeOctahedron(scale=20.0) + self.assertEqual(h.grid().gridClass(), nanovdb.GridClass.FogVolume) + + def test_create_point_sphere(self): + h = nanovdb.tools.createPointSphere(pointsPerVoxel=2, radius=10.0) + self.assertEqual(h.gridCount(), 1) + # PointGrid stores point counts as UInt32 sequential indices. + self.assertEqual(h.gridType(0), nanovdb.GridType.UInt32) + self.assertEqual(h.grid().gridClass(), nanovdb.GridClass.PointData) + self.assertGreater(h.grid().activeVoxelCount(), 0) + + def test_create_point_torus(self): + h = nanovdb.tools.createPointTorus( + pointsPerVoxel=1, majorRadius=10.0, minorRadius=3.0) + self.assertEqual(h.gridType(0), nanovdb.GridType.UInt32) + self.assertGreater(h.grid().activeVoxelCount(), 0) + + def test_create_point_box(self): + # Box must be large enough to enclose at least one active voxel + # for createPointScatter's internal "ActiveVoxelCount is required" + # precondition to pass. + h = nanovdb.tools.createPointBox( + pointsPerVoxel=1, width=40.0, height=40.0, depth=40.0) + self.assertEqual(h.gridType(0), nanovdb.GridType.UInt32) + self.assertGreater(h.grid().activeVoxelCount(), 0) + + def test_create_point_scatter(self): + # Source level set, then scatter points into it. + sphere = nanovdb.tools.createLevelSetSphere(radius=10.0).grid() + h = nanovdb.tools.createPointScatter(sphere, pointsPerVoxel=2) + self.assertEqual(h.gridType(0), nanovdb.GridType.UInt32) + self.assertGreater(h.grid().activeVoxelCount(), 0) + + def test_create_point_scatter_rejects_non_float_source(self): + # The binding accepts NanoGrid only — other source types + # should raise TypeError at the conversion boundary. + h_double = nanovdb.tools.createLevelSetSphere( + gridType=nanovdb.GridType.Double, radius=10.0) + with self.assertRaises(TypeError): + nanovdb.tools.createPointScatter(h_double.grid()) + + def test_create_point_scatter_rejects_fog_volume(self): + # createPointScatter's C++ implementation requires the source to + # pass srcGrid.isLevelSet(); fog volumes raise RuntimeError. + h_fog = nanovdb.tools.createFogVolumeSphere(radius=10.0) + with self.assertRaises(RuntimeError): + nanovdb.tools.createPointScatter(h_fog.grid()) + + +class TestCreateNanoGridQuantized(unittest.TestCase): + """Phase 5b: tools.createNanoGridFp4 / Fp8 / Fp16 / FpN with AbsDiff/RelDiff.""" + + def _float_sphere(self): + return nanovdb.tools.createLevelSetSphere(radius=10.0).grid() + + def test_quantize_fp4(self): + h = nanovdb.tools.createNanoGridFp4(self._float_sphere()) + self.assertEqual(h.gridType(0), nanovdb.GridType.Fp4) + self.assertGreater(h.grid().activeVoxelCount(), 0) + + def test_quantize_fp8(self): + h = nanovdb.tools.createNanoGridFp8(self._float_sphere()) + self.assertEqual(h.gridType(0), nanovdb.GridType.Fp8) + + def test_quantize_fp16(self): + h = nanovdb.tools.createNanoGridFp16(self._float_sphere()) + self.assertEqual(h.gridType(0), nanovdb.GridType.Fp16) + + def test_quantize_fpn_absdiff(self): + oracle = nanovdb.tools.AbsDiff(0.05) + self.assertAlmostEqual(oracle.getTolerance(), 0.05, places=5) + self.assertTrue(bool(oracle)) + h = nanovdb.tools.createNanoGridFpN(self._float_sphere(), oracle) + self.assertEqual(h.gridType(0), nanovdb.GridType.FpN) + + def test_quantize_fpn_reldiff(self): + oracle = nanovdb.tools.RelDiff(0.1) + self.assertAlmostEqual(oracle.getTolerance(), 0.1, places=5) + h = nanovdb.tools.createNanoGridFpN(self._float_sphere(), oracle) + self.assertEqual(h.gridType(0), nanovdb.GridType.FpN) + + def test_oracle_default_tolerance(self): + # Default-constructed oracle has tolerance == -1, which means + # "uninitialized"; the operator bool() detects that. + a = nanovdb.tools.AbsDiff() + self.assertEqual(a.getTolerance(), -1.0) + self.assertFalse(bool(a)) + a.setTolerance(0.5) + self.assertEqual(a.getTolerance(), 0.5) + self.assertTrue(bool(a)) + + def test_quantize_rejects_double_source(self): + # The C++ Fp{4,8,16,N} preProcess static-asserts SrcValueT == float; + # Python must surface this as a TypeError at the conversion boundary. + h_double = nanovdb.tools.createLevelSetSphere( + gridType=nanovdb.GridType.Double, radius=5.0) + with self.assertRaises(TypeError): + nanovdb.tools.createNanoGridFp16(h_double.grid()) + with self.assertRaises(TypeError): + nanovdb.tools.createNanoGridFpN( + h_double.grid(), nanovdb.tools.AbsDiff(0.05)) + + def test_quantize_from_build_grid(self): + # Phase 5c: build::FloatGrid accepted as quantization source. + bg = nanovdb.tools.build.FloatGrid(0.0) + for i in range(5): + bg.setValue(nanovdb.math.Coord(i, 0, 0), float(i + 1)) + h = nanovdb.tools.createNanoGridFp16(bg) + self.assertEqual(h.gridType(0), nanovdb.GridType.Fp16) + self.assertEqual(h.grid().activeVoxelCount(), 5) + + +class TestCreateNanoGridIndex(unittest.TestCase): + """Phase 5b/5c: tools.createNanoGridIndex / OnIndex with broad source set.""" + + def test_index_from_float_nanogrid(self): + sphere = nanovdb.tools.createLevelSetSphere(radius=10.0).grid() + h = nanovdb.tools.createNanoGridIndex(sphere) + self.assertEqual(h.gridType(0), nanovdb.GridType.Index) + + def test_on_index_from_float_nanogrid(self): + sphere = nanovdb.tools.createLevelSetSphere(radius=10.0).grid() + h = nanovdb.tools.createNanoGridOnIndex(sphere) + self.assertEqual(h.gridType(0), nanovdb.GridType.OnIndex) + + def test_index_from_double_nanogrid(self): + sphere = nanovdb.tools.createLevelSetSphere( + gridType=nanovdb.GridType.Double, radius=10.0).grid() + h = nanovdb.tools.createNanoGridIndex(sphere) + self.assertEqual(h.gridType(0), nanovdb.GridType.Index) + + def test_index_from_int32_build(self): + # Phase 5c source: build::Int32Grid is accepted by the index path. + bg = nanovdb.tools.build.Int32Grid(0) + bg.setValue(nanovdb.math.Coord(0, 0, 0), 42) + bg.setValue(nanovdb.math.Coord(1, 0, 0), -7) + h = nanovdb.tools.createNanoGridOnIndex(bg) + self.assertEqual(h.gridType(0), nanovdb.GridType.OnIndex) + self.assertEqual(h.grid().activeVoxelCount(), 2) + + def test_index_from_vec3f_build(self): + bv = nanovdb.tools.build.Vec3fGrid(nanovdb.math.Vec3f(0.0)) + bv.setValue(nanovdb.math.Coord(0, 0, 0), nanovdb.math.Vec3f(1, 2, 3)) + h = nanovdb.tools.createNanoGridOnIndex(bv) + self.assertEqual(h.gridType(0), nanovdb.GridType.OnIndex) + + def test_index_rejects_none(self): + # The conversion functions accept either a NanoGrid or a + # build::Grid; None matches neither and is rejected at the + # isinstance dispatch. + with self.assertRaises(TypeError): + nanovdb.tools.createNanoGridOnIndex(None) + + def test_index_rejects_unsupported_buildt(self): + # The Phase 5 index conversion accepts float / double / int32 / + # Vec3f sources (NanoGrid or build::Grid). A Vec3d build::Grid + # is a structurally valid grid but a BuildT outside that set — + # the try-each-SrcBuildT chain falls through and raises. + bv = nanovdb.tools.build.Vec3dGrid(nanovdb.math.Vec3d(0.0)) + bv.setValue(nanovdb.math.Coord(0, 0, 0), nanovdb.math.Vec3d(1, 2, 3)) + with self.assertRaises(TypeError): + nanovdb.tools.createNanoGridOnIndex(bv) + + class TestGridStats(unittest.TestCase): """nanovdb.tools.Extrema*, Stats*, updateGridStats, getExtrema.""" @@ -1662,10 +1859,13 @@ def test_validateGrid_disable_mode_always_true(self): self.assertTrue( nanovdb.tools.validateGrid(h, 99, nanovdb.CheckMode.Disable)) - @unittest.skipUnless( - hasattr(nanovdb.tools, "cuda") and - hasattr(nanovdb.tools.cuda, "createLevelSetSphere"), - "device handles require a CUDA-enabled build", + @unittest.skipIf( + not nanovdb.isCudaAvailable(), + "nanovdb module was compiled without CUDA support", + ) + @unittest.skipIf( + not nanovdb.isGpuAvailable(), + "No CUDA-capable GPU available at runtime", ) def test_validateGrid_on_device_handle(self): # validateGrid is bound for both host and device handles. The From 03ed5c8f00475afc2988161a944250c6f4c3f90d Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Fri, 22 May 2026 01:00:46 +1200 Subject: [PATCH 09/48] nanovdb python: examples + docstring coverage sweep (#2218) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nanovdb python: Phase 6 — docs, migration guide, examples Closes Phase 6 of the NanoVDB Python bindings restructure (#2208). Ships: * doc/nanovdb/PythonMigration.md — porting guide for users on the pre-Phase-1 typed-accessor API. Covers the polymorphic handle.grid(n) replacement of floatGrid() / doubleGrid() / etc., the removal of the nanovdb.math.cuda submodule (sampleFromVoxels is now in nanovdb.tools.cuda), the GridHandle.__bool__ + enum __repr__ behavior changes, and a survey of every new surface that used to require dropping to C++ (GridMetaData, blind data, PointAccessor, tree/node walking, NodeManager, leaf_values, VoxelBlockManager, tools.build.Grid, stats, validation, primitives, quantized createNanoGrid). Closes with a mechanical before/after replacement table. * doc/nanovdb/PythonAPI.md — narrative API reference for the final surface. One section per submodule (nanovdb root, math, io, tools, tools.build, tools.cuda, cuda) listing the bound classes and functions with one-line descriptions. Points readers at help(x) for full signatures. * nanovdb/nanovdb/python/examples/ — five runnable .py scripts: - load_inspect.py: polymorphic handle.grid(n), GridMetaData introspection, mixed-type handle via mergeGrids. - build_grid.py: tools.build.FloatGrid setValue / ValueAccessor / WriteAccessor / .to_nanovdb() round-trip. - bulk_leaf_numpy.py: zero-copy (N_leaves, 512) NumPy view via grid.leaf_values(), with global-stats reduction and in-place mutation demonstrating the no-copy semantics. - quantize.py: createNanoGridFp{4,8,16} fixed-width + createNanoGridFpN with AbsDiff and RelDiff oracles. Prints the per-format size so users see the compression tradeoff at a glance. - validate.py: validateGrid / validateGrids / checkGrid / isValid plus evalChecksum / updateChecksum / validateChecksum. * nanovdb/nanovdb/python/examples/README.md — index of the above with one-line summaries. * nanovdb/nanovdb/Readme.md — added a "Python bindings" section linking the API reference, migration guide, and examples. Each example is self-contained (builds its own input data, prints a small stdout summary) and was smoke-tested locally against the post-Phase-5 binding. The bulk_leaf_numpy example is the only one requiring NumPy; it prints a friendly skip message otherwise. API reference docs via Sphinx / mkdocs are deferred to a follow-up; the existing docstrings already make help() useful and the .pyi type stubs (from Phase 0) cover IDE / type-checker integration. Signed-off-by: Jonathan Swartz * nanovdb python: drop end-user docs, scrub refactor framing from examples Two changes in response to review feedback: * Remove doc/nanovdb/PythonAPI.md and doc/nanovdb/PythonMigration.md. End users don't need a separate narrative reference (help() and the .pyi stubs cover the same ground) and don't need a migration guide framed around the multi-phase refactor history. * Rewrite the docstring blurb at the top of each example (load_inspect.py, build_grid.py, bulk_leaf_numpy.py, quantize.py, validate.py) so the prose describes what the example demonstrates about the current API, with no mention of which phase of the refactor added the feature. Same scrub on python/examples/README.md (drop the "bindings restructure" framing and the now-dead links to the deleted docs) and on nanovdb/nanovdb/Readme.md (drop the new "Python bindings" section header — just leave the single examples link alongside the existing Examples link). Examples themselves remain unchanged in behavior; all five still run cleanly against the current binding. Signed-off-by: Jonathan Swartz * nanovdb python: docstring coverage sweep across the binding surface Add a short prose docstring to every nanobind .def() and nb::class_ call that didn't already carry one. The audit dropped from 374 distinct undocumented call sites (3278 raw, counting per-BuildT template duplicates) to 0. Style: * One short imperative sentence per method, focused on what it does in Python terms. * For methods that exactly mirror a C++ member of the same name, a "See nanovdb:::: in NanoVDB.h." tail so users can find the canonical reference without us re-stating it. * Class-level docstrings on every bound class (Grid, GridHandle, GridMetaData, Tree, Root, Upper, Lower, Leaf, NodeManager, the per-BuildT accessor subclasses, Map, Coord, BBox*, samplers, build::Grid + ValueAccessor + WriteAccessor, Extrema / Stats, oracle classes, Checksum, etc.). * Two-to-three sentence blurbs on the entry-point classes that genuinely need orientation. Files touched: every Py*.cc / Py*.h binding file under nanovdb/nanovdb/python/ plus the three cuda/ binding files. No code changes — only the trailing const char* docstring argument on .def calls and the third arg of nb::class_. The auto-generated .pyi stub output picks the new prose up automatically; help() shows the same text at runtime. Also drop the few remaining "Phase N" / "from Phase N follow-up" references that survived in internal C++ comments in PyCreateNanoGrid.cc, PyPrimitives.cc, and PyVoxelBlockManager.cc, keeping the source consistent with the no-refactor-history policy already applied to the public docstrings, tests, and examples. Test plan: full pytest_nanovdb green locally (138/140; the two errors are pre-existing BLOSC-disabled errors). Module compiles cleanly with CUDA + nanobind stub generation enabled. Signed-off-by: Jonathan Swartz * nanovdb python: address Copilot review on #2218 Two items from the review: 1. The Readme link to the Python examples directory used the wrong relative path. From nanovdb/nanovdb/Readme.md, the examples live at python/examples/ (sibling), not nanovdb/python/examples/ (which would resolve to nanovdb/nanovdb/nanovdb/python/examples/). 2. The Map.applyMap / applyJacobian / applyInverseMap / applyInverseJacobian docstrings used the words "double precision" and "single precision" to describe the variant suffix. That wording is misleading: the F-suffixed variants use 32-bit math internally, the unsuffixed variants use 64-bit math, but BOTH return a vector whose dtype matches the input (Vec3f -> Vec3f, Vec3d -> Vec3d). Switching to "uses 64-bit math" / "uses 32-bit math" with an explicit "returns a vector of the same dtype as the input" tail makes the semantics unambiguous. Same fix on the Jacobian variants (which previously didn't mention precision at all on the 64-bit overloads, only on the 32-bit ones — also confusing). Signed-off-by: Jonathan Swartz * nanovdb python: clarify Grid.gridName docstring Copilot caught that the Grid.gridName docstring described it as "the in-header buffer; truncated if very long" — which is actually the semantics of shortGridName(). gridName() reads the LONG-form name from blind data when the HasLongGridName flag is set, falling back to the in-header buffer otherwise. Reword the docstring to match and explicitly point at shortGridName() for the truncated form. (The other comment in the same review pass flagged the Examples table in python/examples/README.md as having "double leading pipes" but the actual file uses single-pipe GitHub-flavored markdown syntax — verified via per-character inspection. The table renders correctly on GitHub; no change needed there.) Signed-off-by: Jonathan Swartz * nanovdb python: clarify VBM offset and Root.bbox docstrings Two more Copilot review items: * VoxelBlockManagerHandle.firstOffset / lastOffset docstrings said "Linear leaf offset of the first/last block..." but these are sequential VOXEL indices, not leaf IDs. Switch the wording to "Sequential voxel index of the first/last active voxel covered by this handle" so users aren't confused into thinking the value is a leaf number. * RootT.bbox docstring said "Index-space bounding box of every active tile" but RootNode::bbox() returns the bounding box of every active VALUE in the tree, not just the tile entries in the root table. Drop the misleading "tile" wording. Signed-off-by: Jonathan Swartz * nanovdb python: remove dead code in bulk_leaf_numpy example Copilot caught that bulk_leaf_numpy.py computed n_active_after via a per-voxel sum over leaf.isActive() but never used the result — dead code that linters would flag. Replace with leaf.getFirstValue(), which is actually informative for what the surrounding comment is trying to show (the values changed in place via the zero-copy NumPy write) and prints in the same line. Signed-off-by: Jonathan Swartz --------- Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/Readme.md | 1 + nanovdb/nanovdb/python/NanoVDBModule.cc | 633 ++++++++++++------ nanovdb/nanovdb/python/PyBuildGrid.cc | 14 +- nanovdb/nanovdb/python/PyCreateNanoGrid.cc | 39 +- nanovdb/nanovdb/python/PyGridChecksum.cc | 17 +- nanovdb/nanovdb/python/PyGridHandle.h | 65 +- nanovdb/nanovdb/python/PyGridStats.cc | 14 +- nanovdb/nanovdb/python/PyHostBuffer.cc | 5 +- nanovdb/nanovdb/python/PyIO.cc | 114 +++- nanovdb/nanovdb/python/PyMath.cc | 478 ++++++++----- nanovdb/nanovdb/python/PyPrimitives.cc | 16 +- nanovdb/nanovdb/python/PySampleFromVoxels.cc | 17 +- nanovdb/nanovdb/python/PyTree.cc | 6 +- nanovdb/nanovdb/python/PyTree.h | 204 ++++-- nanovdb/nanovdb/python/PyVoxelBlockManager.cc | 24 +- nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc | 5 +- .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 12 +- nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu | 3 +- nanovdb/nanovdb/python/examples/README.md | 33 + nanovdb/nanovdb/python/examples/build_grid.py | 80 +++ .../python/examples/bulk_leaf_numpy.py | 63 ++ .../nanovdb/python/examples/load_inspect.py | 67 ++ nanovdb/nanovdb/python/examples/quantize.py | 59 ++ nanovdb/nanovdb/python/examples/validate.py | 61 ++ 24 files changed, 1495 insertions(+), 535 deletions(-) create mode 100644 nanovdb/nanovdb/python/examples/README.md create mode 100644 nanovdb/nanovdb/python/examples/build_grid.py create mode 100644 nanovdb/nanovdb/python/examples/bulk_leaf_numpy.py create mode 100644 nanovdb/nanovdb/python/examples/load_inspect.py create mode 100644 nanovdb/nanovdb/python/examples/quantize.py create mode 100644 nanovdb/nanovdb/python/examples/validate.py diff --git a/nanovdb/nanovdb/Readme.md b/nanovdb/nanovdb/Readme.md index 21f247a5e5..ebf6616d16 100644 --- a/nanovdb/nanovdb/Readme.md +++ b/nanovdb/nanovdb/Readme.md @@ -7,6 +7,7 @@ A lightweight GPU friendly version of VDB initially targeting rendering applicat * [Frequently asked questions](../../doc/nanovdb/FAQ.md) * [Source tree](../../doc/nanovdb/SourceTree.md) * [Examples](../../doc/nanovdb/HelloWorld.md) +* [Python examples](python/examples/) ### Copyright Contributors to the OpenVDB Project ### SPDX-License-Identifier: Apache-2.0 diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index f847e9eee3..65fc861b26 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -34,19 +34,32 @@ namespace pynanovdb { void defineVersion(nb::module_& m) { nb::class_(m, "Version", "Bit-compacted representation of all three version numbers") - .def(nb::init<>()) - .def(nb::init(), "data"_a) - .def(nb::init(), "major"_a, "minor"_a, "patch"_a) - .def(nb::self == nb::self, "rhs"_a) - .def(nb::self < nb::self, "rhs"_a) - .def(nb::self <= nb::self, "rhs"_a) - .def(nb::self > nb::self, "rhs"_a) - .def(nb::self >= nb::self, "rhs"_a) - .def("id", &Version::id) - .def("getMajor", &Version::getMajor) - .def("getMinor", &Version::getMinor) - .def("getPatch", &Version::getPatch) - .def("age", &Version::age) + .def(nb::init<>(), + "Construct a default-initialized Version matching the running NanoVDB build.") + .def(nb::init(), "data"_a, + "Construct a Version from a raw bit-packed uint32 value.") + .def(nb::init(), "major"_a, "minor"_a, "patch"_a, + "Construct a Version from explicit major, minor, and patch numbers.") + .def(nb::self == nb::self, "rhs"_a, + "Equality of major, minor and patch numbers.") + .def(nb::self < nb::self, "rhs"_a, + "Lexicographic less-than comparison over (major, minor, patch).") + .def(nb::self <= nb::self, "rhs"_a, + "Lexicographic less-than-or-equal comparison.") + .def(nb::self > nb::self, "rhs"_a, + "Lexicographic greater-than comparison.") + .def(nb::self >= nb::self, "rhs"_a, + "Lexicographic greater-than-or-equal comparison.") + .def("id", &Version::id, + "Return the bit-packed uint32 representation of this version.") + .def("getMajor", &Version::getMajor, + "Major version number.") + .def("getMinor", &Version::getMinor, + "Minor version number.") + .def("getPatch", &Version::getPatch, + "Patch version number.") + .def("age", &Version::age, + "Numeric age of this version relative to the running NanoVDB build.") .def("__repr__", [](const Version& version) { char str[strlen()]; toStr(str, version); @@ -56,7 +69,10 @@ void defineVersion(nb::module_& m) void definePointTypes(nb::module_& m) { - nb::enum_(m, "PointType") + nb::enum_(m, "PointType", + "Encoding selector for point attributes attached to a PointGrid. " + "Controls the bit width and frame (world / grid / voxel) used when " + "rasterising points to blind data.") .value("Disable", PointType::Disable) .value("PointID", PointType::PointID) .value("World64", PointType::World64) @@ -73,72 +89,129 @@ void definePointTypes(nb::module_& m) template void defineMask(nb::module_& m, const char* name, const char* doc) { nb::class_>(m, name, doc) - .def_static("memUsage", &Mask::memUsage) - .def_static("bitCount", &Mask::bitCount) - .def_static("wordCount", &Mask::wordCount) - .def("countOn", nb::overload_cast<>(&Mask::countOn, nb::const_)) - .def("countOn", nb::overload_cast(&Mask::countOn, nb::const_), "i"_a) - .def(nb::init<>()) - .def(nb::init(), "on"_a) - .def(nb::init>(), "other"_a) - .def(nb::self == nb::self, "other"_a) - .def(nb::self != nb::self, "other"_a) - .def("isOn", nb::overload_cast(&Mask::isOn, nb::const_), "n"_a) - .def("isOff", nb::overload_cast(&Mask::isOff, nb::const_), "n"_a) - .def("isOn", nb::overload_cast<>(&Mask::isOn, nb::const_)) - .def("isOff", nb::overload_cast<>(&Mask::isOff, nb::const_)) - .def("setOn", nb::overload_cast(&Mask::setOn), "n"_a) - .def("setOff", nb::overload_cast(&Mask::setOff), "n"_a) - .def("set", nb::overload_cast(&Mask::set), "n"_a, "on"_a) - .def("setOn", nb::overload_cast<>(&Mask::setOn)) - .def("setOff", nb::overload_cast<>(&Mask::setOff)) - .def("set", nb::overload_cast(&Mask::set), "on"_a) - .def("toggle", nb::overload_cast<>(&Mask::toggle)) - .def("toggle", nb::overload_cast(&Mask::toggle), "n"_a) + .def_static("memUsage", &Mask::memUsage, + "Byte size of a Mask instance.") + .def_static("bitCount", &Mask::bitCount, + "Total number of bits this mask can store.") + .def_static("wordCount", &Mask::wordCount, + "Number of 64-bit words used to back this mask.") + .def("countOn", nb::overload_cast<>(&Mask::countOn, nb::const_), + "Number of bits currently set in the mask.") + .def("countOn", nb::overload_cast(&Mask::countOn, nb::const_), "i"_a, + "Number of bits currently set in the prefix [0, i).") + .def(nb::init<>(), + "Construct an all-off mask.") + .def(nb::init(), "on"_a, + "Construct a mask with every bit set to on.") + .def(nb::init>(), "other"_a, + "Copy-construct from another Mask.") + .def(nb::self == nb::self, "other"_a, + "Bit-for-bit equality with another Mask.") + .def(nb::self != nb::self, "other"_a, + "Bit-for-bit inequality with another Mask.") + .def("isOn", nb::overload_cast(&Mask::isOn, nb::const_), "n"_a, + "True iff bit n is set.") + .def("isOff", nb::overload_cast(&Mask::isOff, nb::const_), "n"_a, + "True iff bit n is cleared.") + .def("isOn", nb::overload_cast<>(&Mask::isOn, nb::const_), + "True iff every bit is set.") + .def("isOff", nb::overload_cast<>(&Mask::isOff, nb::const_), + "True iff every bit is cleared.") + .def("setOn", nb::overload_cast(&Mask::setOn), "n"_a, + "Set bit n.") + .def("setOff", nb::overload_cast(&Mask::setOff), "n"_a, + "Clear bit n.") + .def("set", nb::overload_cast(&Mask::set), "n"_a, "on"_a, + "Assign bit n to the given on/off state.") + .def("setOn", nb::overload_cast<>(&Mask::setOn), + "Set every bit.") + .def("setOff", nb::overload_cast<>(&Mask::setOff), + "Clear every bit.") + .def("set", nb::overload_cast(&Mask::set), "on"_a, + "Set every bit to the same on/off state.") + .def("toggle", nb::overload_cast<>(&Mask::toggle), + "Flip every bit.") + .def("toggle", nb::overload_cast(&Mask::toggle), "n"_a, + "Flip bit n.") .def( - "__iand__", [](Mask& a, const Mask& b) { return a &= b; }, nb::is_operator(), "other"_a) + "__iand__", [](Mask& a, const Mask& b) { return a &= b; }, nb::is_operator(), "other"_a, + "In-place bitwise AND with another Mask.") .def( - "__ior__", [](Mask& a, const Mask& b) { return a |= b; }, nb::is_operator(), "other"_a) + "__ior__", [](Mask& a, const Mask& b) { return a |= b; }, nb::is_operator(), "other"_a, + "In-place bitwise OR with another Mask.") .def( - "__isub__", [](Mask& a, const Mask& b) { return a -= b; }, nb::is_operator(), "other"_a) + "__isub__", [](Mask& a, const Mask& b) { return a -= b; }, nb::is_operator(), "other"_a, + "In-place bitwise difference (clear every bit that is set in other).") .def( - "__ixor__", [](Mask& a, const Mask& b) { return a ^= b; }, nb::is_operator(), "other"_a) - .def("findFirstOn", &Mask::template findFirst) - .def("findFirstOff", &Mask::template findFirst) - .def("findNextOn", &Mask::template findNext, "start"_a) - .def("findNextOff", &Mask::template findNext, "start"_a) - .def("findPrevOn", &Mask::template findPrev, "start"_a) - .def("findPrevOff", &Mask::template findPrev, "start"_a); + "__ixor__", [](Mask& a, const Mask& b) { return a ^= b; }, nb::is_operator(), "other"_a, + "In-place bitwise XOR with another Mask.") + .def("findFirstOn", &Mask::template findFirst, + "Index of the first set bit, or bitCount() if every bit is clear.") + .def("findFirstOff", &Mask::template findFirst, + "Index of the first clear bit, or bitCount() if every bit is set.") + .def("findNextOn", &Mask::template findNext, "start"_a, + "Index of the first set bit at or after start.") + .def("findNextOff", &Mask::template findNext, "start"_a, + "Index of the first clear bit at or after start.") + .def("findPrevOn", &Mask::template findPrev, "start"_a, + "Index of the first set bit at or before start.") + .def("findPrevOff", &Mask::template findPrev, "start"_a, + "Index of the first clear bit at or before start."); } void defineMap(nb::module_& m) { nb::class_(m, "Map", "Defines an affine transform and its inverse represented as a 3x3 matrix and a vec3 translation") - .def(nb::init<>()) - .def(nb::init(), "s"_a, "t"_a = Vec3d(0.)) - .def("set", nb::overload_cast(&Map::template set), "scale"_a, "translation"_a, "taper"_a = 1.) - .def("set", nb::overload_cast(&Map::template set), "scale"_a, "translation"_a, "taper"_a = 1.) - .def("applyMap", nb::overload_cast(&Map::template applyMap, nb::const_), "ijk"_a) - .def("applyMap", nb::overload_cast(&Map::template applyMap, nb::const_), "ijk"_a) - .def("applyMapF", nb::overload_cast(&Map::template applyMapF, nb::const_), "ijk"_a) - .def("applyMapF", nb::overload_cast(&Map::template applyMapF, nb::const_), "ijk"_a) - .def("applyJacobian", nb::overload_cast(&Map::template applyJacobian, nb::const_), "ijk"_a) - .def("applyJacobian", nb::overload_cast(&Map::template applyJacobian, nb::const_), "ijk"_a) - .def("applyJacobianF", nb::overload_cast(&Map::template applyJacobianF, nb::const_), "ijk"_a) - .def("applyJacobianF", nb::overload_cast(&Map::template applyJacobianF, nb::const_), "ijk"_a) - .def("applyInverseMap", nb::overload_cast(&Map::template applyInverseMap, nb::const_), "xyz"_a) - .def("applyInverseMap", nb::overload_cast(&Map::template applyInverseMap, nb::const_), "xyz"_a) - .def("applyInverseMapF", nb::overload_cast(&Map::template applyInverseMapF, nb::const_), "xyz"_a) - .def("applyInverseMapF", nb::overload_cast(&Map::template applyInverseMapF, nb::const_), "xyz"_a) - .def("applyInverseJacobian", nb::overload_cast(&Map::template applyInverseJacobian, nb::const_), "xyz"_a) - .def("applyInverseJacobian", nb::overload_cast(&Map::template applyInverseJacobian, nb::const_), "xyz"_a) - .def("applyInverseJacobianF", nb::overload_cast(&Map::template applyInverseJacobianF, nb::const_), "xyz"_a) - .def("applyInverseJacobianF", nb::overload_cast(&Map::template applyInverseJacobianF, nb::const_), "xyz"_a) - .def("applyIJT", nb::overload_cast(&Map::template applyIJT, nb::const_), "xyz"_a) - .def("applyIJT", nb::overload_cast(&Map::template applyIJT, nb::const_), "xyz"_a) - .def("applyIJTF", nb::overload_cast(&Map::template applyIJTF, nb::const_), "xyz"_a) - .def("applyIJTF", nb::overload_cast(&Map::template applyIJTF, nb::const_), "xyz"_a) - .def("getVoxelSize", &Map::getVoxelSize); + .def(nb::init<>(), + "Construct an identity Map (uniform unit scale, zero translation).") + .def(nb::init(), "s"_a, "t"_a = Vec3d(0.), + "Construct a Map with uniform scale s and translation t.") + .def("set", nb::overload_cast(&Map::template set), "scale"_a, "translation"_a, "taper"_a = 1., + "Rebuild this Map from a uniform scale, translation and optional frustum taper.") + .def("set", nb::overload_cast(&Map::template set), "scale"_a, "translation"_a, "taper"_a = 1., + "Rebuild this Map from a uniform scale, translation and optional frustum taper.") + .def("applyMap", nb::overload_cast(&Map::template applyMap, nb::const_), "ijk"_a, + "Transform an index-space point to world space using 64-bit math; returns a vector of the same dtype as the input.") + .def("applyMap", nb::overload_cast(&Map::template applyMap, nb::const_), "ijk"_a, + "Transform an index-space point to world space using 64-bit math; returns a vector of the same dtype as the input.") + .def("applyMapF", nb::overload_cast(&Map::template applyMapF, nb::const_), "ijk"_a, + "Transform an index-space point to world space using 32-bit math; returns a vector of the same dtype as the input.") + .def("applyMapF", nb::overload_cast(&Map::template applyMapF, nb::const_), "ijk"_a, + "Transform an index-space point to world space using 32-bit math; returns a vector of the same dtype as the input.") + .def("applyJacobian", nb::overload_cast(&Map::template applyJacobian, nb::const_), "ijk"_a, + "Apply the linear (Jacobian) part of the transform using 64-bit math, ignoring translation.") + .def("applyJacobian", nb::overload_cast(&Map::template applyJacobian, nb::const_), "ijk"_a, + "Apply the linear (Jacobian) part of the transform using 64-bit math, ignoring translation.") + .def("applyJacobianF", nb::overload_cast(&Map::template applyJacobianF, nb::const_), "ijk"_a, + "Apply the linear (Jacobian) part of the transform using 32-bit math, ignoring translation.") + .def("applyJacobianF", nb::overload_cast(&Map::template applyJacobianF, nb::const_), "ijk"_a, + "Apply the linear (Jacobian) part of the transform using 32-bit math, ignoring translation.") + .def("applyInverseMap", nb::overload_cast(&Map::template applyInverseMap, nb::const_), "xyz"_a, + "Transform a world-space point back to index space using 64-bit math; returns a vector of the same dtype as the input.") + .def("applyInverseMap", nb::overload_cast(&Map::template applyInverseMap, nb::const_), "xyz"_a, + "Transform a world-space point back to index space using 64-bit math; returns a vector of the same dtype as the input.") + .def("applyInverseMapF", nb::overload_cast(&Map::template applyInverseMapF, nb::const_), "xyz"_a, + "Transform a world-space point back to index space using 32-bit math; returns a vector of the same dtype as the input.") + .def("applyInverseMapF", nb::overload_cast(&Map::template applyInverseMapF, nb::const_), "xyz"_a, + "Transform a world-space point back to index space using 32-bit math; returns a vector of the same dtype as the input.") + .def("applyInverseJacobian", nb::overload_cast(&Map::template applyInverseJacobian, nb::const_), "xyz"_a, + "Apply the inverse linear (Jacobian) part of the transform using 64-bit math.") + .def("applyInverseJacobian", nb::overload_cast(&Map::template applyInverseJacobian, nb::const_), "xyz"_a, + "Apply the inverse linear (Jacobian) part of the transform using 64-bit math.") + .def("applyInverseJacobianF", nb::overload_cast(&Map::template applyInverseJacobianF, nb::const_), "xyz"_a, + "Apply the inverse linear (Jacobian) part using 32-bit math.") + .def("applyInverseJacobianF", nb::overload_cast(&Map::template applyInverseJacobianF, nb::const_), "xyz"_a, + "Apply the inverse linear (Jacobian) part using 32-bit math.") + .def("applyIJT", nb::overload_cast(&Map::template applyIJT, nb::const_), "xyz"_a, + "Apply the inverse-Jacobian-transpose used for transforming normals.") + .def("applyIJT", nb::overload_cast(&Map::template applyIJT, nb::const_), "xyz"_a, + "Apply the inverse-Jacobian-transpose used for transforming normals.") + .def("applyIJTF", nb::overload_cast(&Map::template applyIJTF, nb::const_), "xyz"_a, + "Apply the inverse-Jacobian-transpose in single precision.") + .def("applyIJTF", nb::overload_cast(&Map::template applyIJTF, nb::const_), "xyz"_a, + "Apply the inverse-Jacobian-transpose in single precision.") + .def("getVoxelSize", &Map::getVoxelSize, + "World-space size of a single voxel implied by this Map."); } // Forward declaration — body lives below defineGridBlindData() so it can @@ -158,86 +231,151 @@ static nb::object pyGetBlindData(nb::handle py_grid, uint32_t n); // so the same value is reachable from the base. void defineGrid(nb::module_& m) { - nb::class_(m, "Grid") + nb::class_(m, "Grid", + "Type-erased base for every NanoVDB grid. Carries header fields, " + "the affine transform, grid flags and blind-data accessors. Concrete " + "BuildT-typed subclasses (FloatGrid, Vec3fGrid, ...) add tree access " + "and per-voxel queries.") // Validation and flag mutators (already member functions on GridData). - .def("isValid", &GridData::isValid) - .def("setMinMaxOn", &GridData::setMinMaxOn, "on"_a = true) - .def("setBBoxOn", &GridData::setBBoxOn, "on"_a = true) - .def("setLongGridNameOn", &GridData::setLongGridNameOn, "on"_a = true) - .def("setAverageOn", &GridData::setAverageOn, "on"_a = true) - .def("setStdDeviationOn", &GridData::setStdDeviationOn, "on"_a = true) - .def("setGridName", &GridData::setGridName, "src"_a) + .def("isValid", &GridData::isValid, + "True iff the grid header looks consistent (magic / version / class tags).") + .def("setMinMaxOn", &GridData::setMinMaxOn, "on"_a = true, + "Toggle the HasMinMax grid flag.") + .def("setBBoxOn", &GridData::setBBoxOn, "on"_a = true, + "Toggle the HasBBox grid flag.") + .def("setLongGridNameOn", &GridData::setLongGridNameOn, "on"_a = true, + "Toggle the HasLongGridName grid flag.") + .def("setAverageOn", &GridData::setAverageOn, "on"_a = true, + "Toggle the HasAverage grid flag.") + .def("setStdDeviationOn", &GridData::setStdDeviationOn, "on"_a = true, + "Toggle the HasStdDeviation grid flag.") + .def("setGridName", &GridData::setGridName, "src"_a, + "Overwrite the grid's short name (truncated to the in-header buffer).") // Affine transforms (already member functions on GridData). - .def("applyMap", nb::overload_cast(&GridData::template applyMap, nb::const_), "xyz"_a) - .def("applyMap", nb::overload_cast(&GridData::template applyMap, nb::const_), "xyz"_a) - .def("applyMapF", nb::overload_cast(&GridData::template applyMapF, nb::const_), "xyz"_a) - .def("applyMapF", nb::overload_cast(&GridData::template applyMapF, nb::const_), "xyz"_a) - .def("applyJacobian", nb::overload_cast(&GridData::template applyJacobian, nb::const_), "xyz"_a) - .def("applyJacobian", nb::overload_cast(&GridData::template applyJacobian, nb::const_), "xyz"_a) - .def("applyJacobianF", nb::overload_cast(&GridData::template applyJacobianF, nb::const_), "xyz"_a) - .def("applyJacobianF", nb::overload_cast(&GridData::template applyJacobianF, nb::const_), "xyz"_a) - .def("applyInverseMap", nb::overload_cast(&GridData::template applyInverseMap, nb::const_), "xyz"_a) - .def("applyInverseMap", nb::overload_cast(&GridData::template applyInverseMap, nb::const_), "xyz"_a) - .def("applyInverseMapF", nb::overload_cast(&GridData::template applyInverseMapF, nb::const_), "xyz"_a) - .def("applyInverseMapF", nb::overload_cast(&GridData::template applyInverseMapF, nb::const_), "xyz"_a) - .def("applyInverseJacobian", nb::overload_cast(&GridData::template applyInverseJacobian, nb::const_), "xyz"_a) - .def("applyInverseJacobian", nb::overload_cast(&GridData::template applyInverseJacobian, nb::const_), "xyz"_a) - .def("applyInverseJacobianF", nb::overload_cast(&GridData::template applyInverseJacobianF, nb::const_), "xyz"_a) - .def("applyInverseJacobianF", nb::overload_cast(&GridData::template applyInverseJacobianF, nb::const_), "xyz"_a) - .def("applyIJT", nb::overload_cast(&GridData::template applyIJT, nb::const_), "xyz"_a) - .def("applyIJT", nb::overload_cast(&GridData::template applyIJT, nb::const_), "xyz"_a) - .def("applyIJTF", nb::overload_cast(&GridData::template applyIJTF, nb::const_), "xyz"_a) - .def("applyIJTF", nb::overload_cast(&GridData::template applyIJTF, nb::const_), "xyz"_a) + .def("applyMap", nb::overload_cast(&GridData::template applyMap, nb::const_), "xyz"_a, + "Transform an index-space point to world space using this grid's Map.") + .def("applyMap", nb::overload_cast(&GridData::template applyMap, nb::const_), "xyz"_a, + "Transform an index-space point to world space using this grid's Map.") + .def("applyMapF", nb::overload_cast(&GridData::template applyMapF, nb::const_), "xyz"_a, + "Transform an index-space point to world space in single precision.") + .def("applyMapF", nb::overload_cast(&GridData::template applyMapF, nb::const_), "xyz"_a, + "Transform an index-space point to world space in single precision.") + .def("applyJacobian", nb::overload_cast(&GridData::template applyJacobian, nb::const_), "xyz"_a, + "Apply the linear (Jacobian) part of the transform using 64-bit math, ignoring translation.") + .def("applyJacobian", nb::overload_cast(&GridData::template applyJacobian, nb::const_), "xyz"_a, + "Apply the linear (Jacobian) part of the transform using 64-bit math, ignoring translation.") + .def("applyJacobianF", nb::overload_cast(&GridData::template applyJacobianF, nb::const_), "xyz"_a, + "Apply the linear part in single precision.") + .def("applyJacobianF", nb::overload_cast(&GridData::template applyJacobianF, nb::const_), "xyz"_a, + "Apply the linear part in single precision.") + .def("applyInverseMap", nb::overload_cast(&GridData::template applyInverseMap, nb::const_), "xyz"_a, + "Transform a world-space point back to index space.") + .def("applyInverseMap", nb::overload_cast(&GridData::template applyInverseMap, nb::const_), "xyz"_a, + "Transform a world-space point back to index space.") + .def("applyInverseMapF", nb::overload_cast(&GridData::template applyInverseMapF, nb::const_), "xyz"_a, + "Transform a world-space point back to index space in single precision.") + .def("applyInverseMapF", nb::overload_cast(&GridData::template applyInverseMapF, nb::const_), "xyz"_a, + "Transform a world-space point back to index space in single precision.") + .def("applyInverseJacobian", nb::overload_cast(&GridData::template applyInverseJacobian, nb::const_), "xyz"_a, + "Apply the inverse linear (Jacobian) part of the transform using 64-bit math.") + .def("applyInverseJacobian", nb::overload_cast(&GridData::template applyInverseJacobian, nb::const_), "xyz"_a, + "Apply the inverse linear (Jacobian) part of the transform using 64-bit math.") + .def("applyInverseJacobianF", nb::overload_cast(&GridData::template applyInverseJacobianF, nb::const_), "xyz"_a, + "Apply the inverse linear (Jacobian) part using 32-bit math.") + .def("applyInverseJacobianF", nb::overload_cast(&GridData::template applyInverseJacobianF, nb::const_), "xyz"_a, + "Apply the inverse linear (Jacobian) part using 32-bit math.") + .def("applyIJT", nb::overload_cast(&GridData::template applyIJT, nb::const_), "xyz"_a, + "Apply the inverse-Jacobian-transpose used for transforming normals.") + .def("applyIJT", nb::overload_cast(&GridData::template applyIJT, nb::const_), "xyz"_a, + "Apply the inverse-Jacobian-transpose used for transforming normals.") + .def("applyIJTF", nb::overload_cast(&GridData::template applyIJTF, nb::const_), "xyz"_a, + "Apply the inverse-Jacobian-transpose in single precision.") + .def("applyIJTF", nb::overload_cast(&GridData::template applyIJTF, nb::const_), "xyz"_a, + "Apply the inverse-Jacobian-transpose in single precision.") // Strings, geometry, layout (already member functions on GridData). - .def("gridName", &GridData::gridName) - .def("memUsage", &GridData::memUsage) - .def("worldBBox", &GridData::worldBBox) - .def("indexBBox", &GridData::indexBBox) - .def("isEmpty", &GridData::isEmpty) + .def("gridName", &GridData::gridName, + "Full grid name as a C string. Reads the long-form name from " + "blind data when the HasLongGridName flag is set, falling " + "back to the in-header buffer otherwise. Use shortGridName() " + "if you specifically want the truncated 255-byte header copy.") + .def("memUsage", &GridData::memUsage, + "Byte size of this grid header.") + .def("worldBBox", &GridData::worldBBox, + "World-space bounding box of active voxels.") + .def("indexBBox", &GridData::indexBBox, + "Index-space bounding box of active voxels.") + .def("isEmpty", &GridData::isEmpty, + "True iff this grid has no active voxels.") // Lifted from Grid via direct data-member access. - .def("version", [](const GridData& g) { return g.mVersion; }) - .def("gridSize", [](const GridData& g) { return g.mGridSize; }) - .def("gridIndex", [](const GridData& g) { return g.mGridIndex; }) - .def("gridCount", [](const GridData& g) { return g.mGridCount; }) + .def("version", [](const GridData& g) { return g.mVersion; }, + "NanoVDB Version stored in this grid's header.") + .def("gridSize", [](const GridData& g) { return g.mGridSize; }, + "Total byte size of this grid (header + tree + blind data).") + .def("gridIndex", [](const GridData& g) { return g.mGridIndex; }, + "Zero-based index of this grid within its parent GridHandle.") + .def("gridCount", [](const GridData& g) { return g.mGridCount; }, + "Number of grids stored alongside this one in the parent GridHandle.") .def("voxelSize", [](const GridData& g) -> const Vec3d& { return g.mVoxelSize; }, - nb::rv_policy::reference_internal) + nb::rv_policy::reference_internal, + "World-space voxel size as a Vec3d.") .def("map", [](const GridData& g) -> const Map& { return g.mMap; }, - nb::rv_policy::reference_internal) - .def("gridType", [](const GridData& g) { return g.mGridType; }) - .def("gridClass", [](const GridData& g) { return g.mGridClass; }) - .def("checksum", [](const GridData& g) { return g.mChecksum; }) - .def("isLevelSet", [](const GridData& g) { return g.mGridClass == GridClass::LevelSet; }) - .def("isFogVolume", [](const GridData& g) { return g.mGridClass == GridClass::FogVolume; }) - .def("isStaggered", [](const GridData& g) { return g.mGridClass == GridClass::Staggered; }) + nb::rv_policy::reference_internal, + "Affine index-to-world Map associated with this grid.") + .def("gridType", [](const GridData& g) { return g.mGridType; }, + "GridType enumerator naming the BuildT carried by this grid.") + .def("gridClass", [](const GridData& g) { return g.mGridClass; }, + "GridClass enumerator (LevelSet, FogVolume, ...).") + .def("checksum", [](const GridData& g) { return g.mChecksum; }, + "Checksum stored in the grid header; compare against tools.evalChecksum.") + .def("isLevelSet", [](const GridData& g) { return g.mGridClass == GridClass::LevelSet; }, + "True iff this grid's class is LevelSet.") + .def("isFogVolume", [](const GridData& g) { return g.mGridClass == GridClass::FogVolume; }, + "True iff this grid's class is FogVolume.") + .def("isStaggered", [](const GridData& g) { return g.mGridClass == GridClass::Staggered; }, + "True iff this grid's class is Staggered.") .def("isPointIndex", - [](const GridData& g) { return g.mGridClass == GridClass::PointIndex; }) - .def("isGridIndex", [](const GridData& g) { return g.mGridClass == GridClass::IndexGrid; }) - .def("isPointData", [](const GridData& g) { return g.mGridClass == GridClass::PointData; }) - .def("isMask", [](const GridData& g) { return g.mGridClass == GridClass::Topology; }) - .def("isUnknown", [](const GridData& g) { return g.mGridClass == GridClass::Unknown; }) - .def("hasMinMax", [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasMinMax); }) - .def("hasBBox", [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasBBox); }) + [](const GridData& g) { return g.mGridClass == GridClass::PointIndex; }, + "True iff this grid's class is PointIndex.") + .def("isGridIndex", [](const GridData& g) { return g.mGridClass == GridClass::IndexGrid; }, + "True iff this grid's class is IndexGrid.") + .def("isPointData", [](const GridData& g) { return g.mGridClass == GridClass::PointData; }, + "True iff this grid's class is PointData.") + .def("isMask", [](const GridData& g) { return g.mGridClass == GridClass::Topology; }, + "True iff this grid's class is Topology.") + .def("isUnknown", [](const GridData& g) { return g.mGridClass == GridClass::Unknown; }, + "True iff this grid's class is Unknown.") + .def("hasMinMax", [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasMinMax); }, + "True iff per-node min/max stats are stored in this grid.") + .def("hasBBox", [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasBBox); }, + "True iff per-node bbox stats are stored in this grid.") .def("hasLongGridName", - [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasLongGridName); }) + [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasLongGridName); }, + "True iff the grid's name was too long to fit in the header buffer.") .def("hasAverage", - [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasAverage); }) + [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasAverage); }, + "True iff per-node average stats are stored in this grid.") .def("hasStdDeviation", - [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasStdDeviation); }) + [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::HasStdDeviation); }, + "True iff per-node standard-deviation stats are stored in this grid.") .def("isBreadthFirst", - [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::IsBreadthFirst); }) - .def("shortGridName", [](const GridData& g) { return std::string(g.mGridName); }) + [](const GridData& g) { return g.mFlags.isMaskOn(GridFlags::IsBreadthFirst); }, + "True iff this grid's nodes are laid out in breadth-first order.") + .def("shortGridName", [](const GridData& g) { return std::string(g.mGridName); }, + "Short in-header copy of the grid name as a Python string.") // Blind data — exposes the sidecar channels that PointGrid and // OnIndexGrid use to carry their actual values, colors, normals, IDs, // etc. blindMetaData(n) returns the descriptor; getBlindData(n) // returns a zero-copy NumPy view onto the underlying bytes typed by // mDataType (Float -> float32 ndarray, Vec3f -> (N, 3) float32, etc.; // unrecognized types fall back to a flat uint8 byte view). - .def("blindDataCount", [](const GridData& g) { return g.mBlindMetadataCount; }) + .def("blindDataCount", [](const GridData& g) { return g.mBlindMetadataCount; }, + "Number of blind-data channels attached to this grid.") .def("blindMetaData", [](const GridData& g, uint32_t n) -> const GridBlindMetaData* { return n < g.mBlindMetadataCount ? g.blindMetaData(n) : nullptr; }, - nb::rv_policy::reference_internal, "n"_a) + nb::rv_policy::reference_internal, "n"_a, + "GridBlindMetaData descriptor for the n-th blind-data channel, or None if n is out of range.") .def("findBlindData", [](const GridData& g, const std::string& name) -> int { for (uint32_t i = 0; i < g.mBlindMetadataCount; ++i) { const auto* meta = g.blindMetaData(i); @@ -245,14 +383,16 @@ void defineGrid(nb::module_& m) return static_cast(i); } return -1; - }, "name"_a) + }, "name"_a, + "Index of the blind-data channel whose name matches name, or -1 if none.") .def("findBlindDataForSemantic", [](const GridData& g, GridBlindDataSemantic sem) -> int { for (uint32_t i = 0; i < g.mBlindMetadataCount; ++i) { if (g.blindMetaData(i)->mSemantic == sem) return static_cast(i); } return -1; - }, "semantic"_a) + }, "semantic"_a, + "Index of the blind-data channel whose semantic matches the given enum, or -1 if none.") .def("getBlindData", &pyGetBlindData, "n"_a, nb::keep_alive<0, 1>(), "Return a zero-copy NumPy view of the n-th blind data channel, " @@ -266,10 +406,16 @@ void defineGrid(nb::module_& m) // need to know BuildT lives there, not here. template void defineNanoGrid(nb::module_& m, const char* name) { - auto cls = nb::class_, GridData>(m, name) - .def("getAccessor", &NanoGrid::getAccessor) - .def("activeVoxelCount", &NanoGrid::activeVoxelCount) - .def("isSequential", [](const NanoGrid& grid) { return grid.isSequential(); }) + auto cls = nb::class_, GridData>(m, name, + "BuildT-typed NanoVDB grid. Inherits the type-erased Grid base for " + "header / transform / blind data, and adds the typed tree and " + "accessor API for the underlying value type.") + .def("getAccessor", &NanoGrid::getAccessor, + "Return a DefaultReadAccessor caching the most recently visited path.") + .def("activeVoxelCount", &NanoGrid::activeVoxelCount, + "Total number of active voxels in this grid. See nanovdb::Grid::activeVoxelCount in NanoVDB.h.") + .def("isSequential", [](const NanoGrid& grid) { return grid.isSequential(); }, + "True iff this grid's nodes are laid out sequentially (per-level contiguous).") .def("tree", nb::overload_cast<>(&NanoGrid::tree, nb::const_), nb::rv_policy::reference_internal, @@ -281,7 +427,10 @@ template void defineNanoGrid(nb::module_& m, const char* name) void defineGridBlindData(nb::module_& m) { - nb::enum_(m, "GridBlindDataClass") + nb::enum_(m, "GridBlindDataClass", + "Coarse classifier for a blind-data channel (index, attribute, " + "channel, ...). Pairs with GridBlindDataSemantic to describe what " + "the channel actually carries.") .value("Unknown", GridBlindDataClass::Unknown) .value("IndexArray", GridBlindDataClass::IndexArray) .value("AttributeArray", GridBlindDataClass::AttributeArray) @@ -290,7 +439,9 @@ void defineGridBlindData(nb::module_& m) .value("End", GridBlindDataClass::End) .export_values(); - nb::enum_(m, "GridBlindDataSemantic") + nb::enum_(m, "GridBlindDataSemantic", + "Fine-grained role of a blind-data channel: PointPosition, " + "PointColor, PointNormal, ... Used by find* helpers on Grid.") .value("Unknown", GridBlindDataSemantic::Unknown) .value("PointPosition", GridBlindDataSemantic::PointPosition) .value("PointColor", GridBlindDataSemantic::PointColor) @@ -309,14 +460,22 @@ void defineGridBlindData(nb::module_& m) nb::class_(m, "GridBlindMetaData", "Sidecar metadata for one blind-data channel attached to a Grid.") - .def_ro("valueCount", &GridBlindMetaData::mValueCount) - .def_ro("valueSize", &GridBlindMetaData::mValueSize) - .def_ro("semantic", &GridBlindMetaData::mSemantic) - .def_ro("dataClass", &GridBlindMetaData::mDataClass) - .def_ro("dataType", &GridBlindMetaData::mDataType) - .def("name", [](const GridBlindMetaData& m) { return std::string(m.mName); }) - .def("isValid", &GridBlindMetaData::isValid) - .def("blindDataSize", &GridBlindMetaData::blindDataSize); + .def_ro("valueCount", &GridBlindMetaData::mValueCount, + "Number of values stored in this channel.") + .def_ro("valueSize", &GridBlindMetaData::mValueSize, + "Byte size of a single value in this channel.") + .def_ro("semantic", &GridBlindMetaData::mSemantic, + "GridBlindDataSemantic describing what this channel carries.") + .def_ro("dataClass", &GridBlindMetaData::mDataClass, + "GridBlindDataClass coarsely classifying this channel.") + .def_ro("dataType", &GridBlindMetaData::mDataType, + "GridType enumerator giving the dtype of a single value.") + .def("name", [](const GridBlindMetaData& m) { return std::string(m.mName); }, + "Name of this blind-data channel as a Python string.") + .def("isValid", &GridBlindMetaData::isValid, + "True iff this descriptor's class/semantic/type combination looks consistent.") + .def("blindDataSize", &GridBlindMetaData::blindDataSize, + "Total byte size of this channel (valueCount * valueSize, padded)."); } // Resolve a blind-data channel into a zero-copy NumPy view. The dtype and @@ -440,9 +599,12 @@ template void definePointAccessor(nb::module_& m, const char* nam "Per-voxel access to the point attributes carried as blind " "data on a PointGrid. gridPoints / leafPoints / voxelPoints " "return zero-copy NumPy views.") - .def(nb::init&>(), "grid"_a, nb::keep_alive<1, 2>()) - .def("__bool__", [](const PA& a) { return bool(a); }) - .def("grid", &PA::grid, nb::rv_policy::reference_internal) + .def(nb::init&>(), "grid"_a, nb::keep_alive<1, 2>(), + "Construct a PointAccessor bound to the given PointGrid.") + .def("__bool__", [](const PA& a) { return bool(a); }, + "True iff this accessor is bound to a valid PointGrid.") + .def("grid", &PA::grid, nb::rv_policy::reference_internal, + "Return the PointGrid this accessor is bound to.") .def("gridPoints", [](nb::handle py_self) -> nb::object { auto& acc = nb::cast(py_self); const AttT* begin = nullptr; @@ -502,7 +664,8 @@ void defineGridMetaData(nb::module_& m) "(bad magic, version, or class/type tags)"); } new (self) GridMetaData(gd); - }, "grid"_a) + }, "grid"_a, + "Construct from a Grid. Raises ValueError if the grid is None or has an invalid header.") .def_static("safeCast", [](const GridData* gd) { // Mirror the spirit of NanoVDB's static safeCast: "is @@ -511,40 +674,74 @@ void defineGridMetaData(nb::module_& m) // return False rather than dereference. if (gd == nullptr || !gd->isValid()) return false; return GridMetaData::safeCast(gd); - }, "grid"_a) - .def("isValid", &GridMetaData::isValid) - .def("gridType", &GridMetaData::gridType) - .def("gridClass", &GridMetaData::gridClass) - .def("isLevelSet", &GridMetaData::isLevelSet) - .def("isFogVolume", &GridMetaData::isFogVolume) - .def("isStaggered", &GridMetaData::isStaggered) - .def("isPointIndex", &GridMetaData::isPointIndex) - .def("isGridIndex", &GridMetaData::isGridIndex) - .def("isPointData", &GridMetaData::isPointData) - .def("isMask", &GridMetaData::isMask) - .def("isUnknown", &GridMetaData::isUnknown) - .def("hasMinMax", &GridMetaData::hasMinMax) - .def("hasBBox", &GridMetaData::hasBBox) - .def("hasLongGridName", &GridMetaData::hasLongGridName) - .def("hasAverage", &GridMetaData::hasAverage) - .def("hasStdDeviation", &GridMetaData::hasStdDeviation) - .def("isBreadthFirst", &GridMetaData::isBreadthFirst) - .def("gridSize", &GridMetaData::gridSize) - .def("gridIndex", &GridMetaData::gridIndex) - .def("gridCount", &GridMetaData::gridCount) - .def("shortGridName", [](const GridMetaData& m) { return std::string(m.shortGridName()); }) - .def("map", &GridMetaData::map, nb::rv_policy::reference_internal) - .def("worldBBox", &GridMetaData::worldBBox, nb::rv_policy::reference_internal) - .def("indexBBox", &GridMetaData::indexBBox, nb::rv_policy::reference_internal) - .def("voxelSize", &GridMetaData::voxelSize) - .def("blindDataCount", &GridMetaData::blindDataCount) - .def("activeVoxelCount", &GridMetaData::activeVoxelCount) - .def("activeTileCount", &GridMetaData::activeTileCount, "level"_a) - .def("nodeCount", &GridMetaData::nodeCount, "level"_a) - .def("checksum", &GridMetaData::checksum, nb::rv_policy::reference_internal) - .def("rootTableSize", &GridMetaData::rootTableSize) - .def("isEmpty", &GridMetaData::isEmpty) - .def("version", &GridMetaData::version); + }, "grid"_a, + "True iff the given grid header is well-formed enough to wrap in a GridMetaData.") + .def("isValid", &GridMetaData::isValid, + "True iff the wrapped header looks consistent.") + .def("gridType", &GridMetaData::gridType, + "GridType enumerator naming the BuildT of the wrapped grid.") + .def("gridClass", &GridMetaData::gridClass, + "GridClass enumerator (LevelSet, FogVolume, ...).") + .def("isLevelSet", &GridMetaData::isLevelSet, + "True iff the wrapped grid's class is LevelSet.") + .def("isFogVolume", &GridMetaData::isFogVolume, + "True iff the wrapped grid's class is FogVolume.") + .def("isStaggered", &GridMetaData::isStaggered, + "True iff the wrapped grid's class is Staggered.") + .def("isPointIndex", &GridMetaData::isPointIndex, + "True iff the wrapped grid's class is PointIndex.") + .def("isGridIndex", &GridMetaData::isGridIndex, + "True iff the wrapped grid's class is IndexGrid.") + .def("isPointData", &GridMetaData::isPointData, + "True iff the wrapped grid's class is PointData.") + .def("isMask", &GridMetaData::isMask, + "True iff the wrapped grid's class is Topology.") + .def("isUnknown", &GridMetaData::isUnknown, + "True iff the wrapped grid's class is Unknown.") + .def("hasMinMax", &GridMetaData::hasMinMax, + "True iff per-node min/max stats are stored in the wrapped grid.") + .def("hasBBox", &GridMetaData::hasBBox, + "True iff per-node bbox stats are stored in the wrapped grid.") + .def("hasLongGridName", &GridMetaData::hasLongGridName, + "True iff the wrapped grid's name was too long for the header buffer.") + .def("hasAverage", &GridMetaData::hasAverage, + "True iff per-node average stats are stored in the wrapped grid.") + .def("hasStdDeviation", &GridMetaData::hasStdDeviation, + "True iff per-node standard-deviation stats are stored in the wrapped grid.") + .def("isBreadthFirst", &GridMetaData::isBreadthFirst, + "True iff the wrapped grid is laid out breadth-first.") + .def("gridSize", &GridMetaData::gridSize, + "Total byte size of the wrapped grid.") + .def("gridIndex", &GridMetaData::gridIndex, + "Index of this grid within its parent GridHandle.") + .def("gridCount", &GridMetaData::gridCount, + "Number of grids in the parent GridHandle.") + .def("shortGridName", [](const GridMetaData& m) { return std::string(m.shortGridName()); }, + "Short in-header copy of the grid name as a Python string.") + .def("map", &GridMetaData::map, nb::rv_policy::reference_internal, + "Affine index-to-world Map of the wrapped grid.") + .def("worldBBox", &GridMetaData::worldBBox, nb::rv_policy::reference_internal, + "World-space bounding box of the wrapped grid's active voxels.") + .def("indexBBox", &GridMetaData::indexBBox, nb::rv_policy::reference_internal, + "Index-space bounding box of the wrapped grid's active voxels.") + .def("voxelSize", &GridMetaData::voxelSize, + "World-space voxel size of the wrapped grid.") + .def("blindDataCount", &GridMetaData::blindDataCount, + "Number of blind-data channels attached to the wrapped grid.") + .def("activeVoxelCount", &GridMetaData::activeVoxelCount, + "Total active voxel count of the wrapped grid.") + .def("activeTileCount", &GridMetaData::activeTileCount, "level"_a, + "Number of active tiles at the given tree level (1=lower, 2=upper, 3=root).") + .def("nodeCount", &GridMetaData::nodeCount, "level"_a, + "Number of nodes at the given tree level (0=leaf, 1=lower, 2=upper).") + .def("checksum", &GridMetaData::checksum, nb::rv_policy::reference_internal, + "Checksum stored in the wrapped grid's header.") + .def("rootTableSize", &GridMetaData::rootTableSize, + "Number of entries in the wrapped grid's root tile table.") + .def("isEmpty", &GridMetaData::isEmpty, + "True iff the wrapped grid has no active voxels.") + .def("version", &GridMetaData::version, + "NanoVDB Version stored in the wrapped grid's header."); } template nb::class_> defineAccessor(nb::module_& m, const char* name) @@ -558,20 +755,28 @@ template nb::class_> defineAccessor using ValueType = typename nanovdb::BuildToValueMap::Type; using CoordType = typename DefaultReadAccessor::CoordType; - nb::class_> accessor(m, name); - accessor.def(nb::init&>(), "grid"_a) - .def("getValue", nb::overload_cast(&DefaultReadAccessor::getValue, nb::const_), "ijk"_a) - .def("getValue", nb::overload_cast(&DefaultReadAccessor::getValue, nb::const_), "i"_a, "j"_a, "k"_a) + nb::class_> accessor(m, name, + "Read accessor that caches the most recently visited tree path for " + "fast neighbor lookups. Construct from a typed grid."); + accessor.def(nb::init&>(), "grid"_a, + "Construct an accessor bound to the given grid.") + .def("getValue", nb::overload_cast(&DefaultReadAccessor::getValue, nb::const_), "ijk"_a, + "Return the grid's value at the integer Coord ijk.") + .def("getValue", nb::overload_cast(&DefaultReadAccessor::getValue, nb::const_), "i"_a, "j"_a, "k"_a, + "Return the grid's value at the integer voxel (i, j, k).") .def( - "__call__", [](const DefaultReadAccessor& accessor, const CoordType& ijk) { return accessor.getValue(ijk); }, nb::is_operator(), "ijk"_a) + "__call__", [](const DefaultReadAccessor& accessor, const CoordType& ijk) { return accessor.getValue(ijk); }, nb::is_operator(), "ijk"_a, + "Operator form of getValue(ijk).") .def( "__call__", [](const DefaultReadAccessor& accessor, int i, int j, int k) { return accessor.getValue(i, j, k); }, nb::is_operator(), "i"_a, "j"_a, - "k"_a) - .def("isActive", &DefaultReadAccessor::isActive, "ijk"_a) + "k"_a, + "Operator form of getValue(i, j, k).") + .def("isActive", &DefaultReadAccessor::isActive, "ijk"_a, + "True iff the voxel at ijk is active.") .def( "probeValue", [](const DefaultReadAccessor& accessor, const CoordType& ijk) { @@ -579,7 +784,8 @@ template nb::class_> defineAccessor bool isOn = accessor.probeValue(ijk, v); return std::make_tuple(v, isOn); }, - "ijk"_a); + "ijk"_a, + "Return (value, isActive) for the voxel at ijk in a single tree traversal."); return accessor; } @@ -589,7 +795,8 @@ template void defineScalarAccessor(nb::module_& m, const char* using CoordType = typename DefaultReadAccessor::CoordType; defineAccessor(m, name) - .def("getNodeInfo", &DefaultReadAccessor::getNodeInfo, "ijk"_a) + .def("getNodeInfo", &DefaultReadAccessor::getNodeInfo, "ijk"_a, + "Return a NodeInfo describing the deepest tree node covering ijk.") .def( "setVoxel", [](DefaultReadAccessor& accessor, const CoordType& ijk, const ValueType& v) { @@ -597,7 +804,8 @@ template void defineScalarAccessor(nb::module_& m, const char* accessor.template set(ijk, v); }, "ijk"_a, - "v"_a); + "v"_a, + "Set the value at ijk to v and mark the voxel active."); } template void defineVectorAccessor(nb::module_& m, const char* name) @@ -612,19 +820,29 @@ template void defineVectorAccessor(nb::module_& m, const char* accessor.template set(ijk, v); }, "ijk"_a, - "v"_a); + "v"_a, + "Set the vector value at ijk to v and mark the voxel active."); } template void defineNodeInfo(nb::module_& m, const char* name) { - nb::class_::NodeInfo>(m, name) - .def_ro("level", &GetNodeInfo::NodeInfo::level) - .def_ro("dim", &GetNodeInfo::NodeInfo::dim) - .def_ro("minimum", &GetNodeInfo::NodeInfo::minimum) - .def_ro("maximum", &GetNodeInfo::NodeInfo::maximum) - .def_ro("average", &GetNodeInfo::NodeInfo::average) - .def_ro("stdDevi", &GetNodeInfo::NodeInfo::stdDevi) - .def_ro("bbox", &GetNodeInfo::NodeInfo::bbox); + nb::class_::NodeInfo>(m, name, + "Descriptor of the deepest tree node covering a queried voxel. " + "Returned by ReadAccessor.getNodeInfo().") + .def_ro("level", &GetNodeInfo::NodeInfo::level, + "Tree level of the node (0=leaf, 1=lower, 2=upper, 3=root).") + .def_ro("dim", &GetNodeInfo::NodeInfo::dim, + "Side length of the node's covered region in voxels.") + .def_ro("minimum", &GetNodeInfo::NodeInfo::minimum, + "Minimum active value within this node.") + .def_ro("maximum", &GetNodeInfo::NodeInfo::maximum, + "Maximum active value within this node.") + .def_ro("average", &GetNodeInfo::NodeInfo::average, + "Average of active values within this node.") + .def_ro("stdDevi", &GetNodeInfo::NodeInfo::stdDevi, + "Standard deviation of active values within this node.") + .def_ro("bbox", &GetNodeInfo::NodeInfo::bbox, + "Index-space bounding box of this node's covered region."); } bool isCudaAvailable() @@ -657,7 +875,9 @@ NB_MODULE(nanovdb, m) m.def("isCudaAvailable", &isCudaAvailable, "Returns whether or not the module was compiled with CUDA support"); m.def("isGpuAvailable", &isGpuAvailable, "Returns whether a CUDA-capable GPU is available at runtime"); - nb::enum_(m, "GridType") + nb::enum_(m, "GridType", + "Enumerator naming every BuildT a NanoVDB grid can carry. Used by " + "handle.gridType(n) and as the dtype field on GridBlindMetaData.") .value("Unknown", GridType::Unknown) .value("Float", GridType::Float) .value("Double", GridType::Double) @@ -691,7 +911,10 @@ NB_MODULE(nanovdb, m) return std::string(str); }); - nb::enum_(m, "GridClass") + nb::enum_(m, "GridClass", + "Semantic class of a grid (LevelSet, FogVolume, PointIndex, ...). " + "Independent of GridType / BuildT; consumed by tools that special-" + "case sign / point / index grids.") .value("Unknown", GridClass::Unknown) .value("LevelSet", GridClass::LevelSet) .value("FogVolume", GridClass::FogVolume) diff --git a/nanovdb/nanovdb/python/PyBuildGrid.cc b/nanovdb/nanovdb/python/PyBuildGrid.cc index 5ef1e09134..5eb5cafcd7 100644 --- a/nanovdb/nanovdb/python/PyBuildGrid.cc +++ b/nanovdb/nanovdb/python/PyBuildGrid.cc @@ -42,7 +42,10 @@ static void defineBuildGrid(nb::module_& m, using ValueT = typename GridT::ValueType; // ----- build::Grid ----- - nb::class_(m, gridName) + nb::class_(m, gridName, + "Mutable host-side grid builder for this BuildT. Write voxels via " + "setValue / getAccessor / getWriteAccessor, then call to_nanovdb() " + "to bake a frozen NanoGrid GridHandle.") .def(nb::init(), "background"_a, "name"_a = std::string(""), @@ -151,7 +154,9 @@ static void defineBuildGrid(nb::module_& m, // Move-only (copy is deleted) — returned by getAccessor(). Caches the // last leaf / lower / upper node it touched, so repeated access to // neighboring coordinates is fast. - nb::class_(m, valueAccName) + nb::class_(m, valueAccName, + "Move-only read/write accessor returned by build.Grid.getAccessor(). " + "Caches the most recently visited tree path for fast neighbor access.") .def("getValue", [](const AccT& self, const Coord& ijk) -> ValueT { return self.getValue(ijk); @@ -186,7 +191,10 @@ static void defineBuildGrid(nb::module_& m, // mutex; on destruction (or explicit merge()) it locks the mutex and // splices its buffered nodes into the parent. Designed for multi-thread // writes — one WriteAccessor per thread, no shared mutable state. - nb::class_(m, writeAccName) + nb::class_(m, writeAccName, + "Thread-safe write accessor returned by build.Grid.getWriteAccessor(). " + "Buffers writes into a private root and merges them into the parent " + "grid on destruction or explicit merge().") .def("setValue", [](WriteAccT& self, const Coord& ijk, const ValueT& value) { self.setValue(ijk, value); diff --git a/nanovdb/nanovdb/python/PyCreateNanoGrid.cc b/nanovdb/nanovdb/python/PyCreateNanoGrid.cc index c132b84e4e..37183a8c34 100644 --- a/nanovdb/nanovdb/python/PyCreateNanoGrid.cc +++ b/nanovdb/nanovdb/python/PyCreateNanoGrid.cc @@ -37,7 +37,9 @@ GridHandle createNanoGridFromFunc(const BuildT& template void defineCreateNanoGrid(nb::module_& m, const char* name) { - m.def(name, &createNanoGridFromFunc, nb::call_guard(), "background"_a, "name"_a, "gridClass"_a, "func"_a, "bbox"_a); + m.def(name, &createNanoGridFromFunc, nb::call_guard(), "background"_a, "name"_a, "gridClass"_a, "func"_a, "bbox"_a, + "Construct a NanoGrid by evaluating func(Coord) over every voxel in bbox. " + "Returns a GridHandle owning the freshly-built grid."); } template void defineOpenToNanoVDB(nb::module_& m) @@ -47,14 +49,15 @@ template void defineOpenToNanoVDB(nb::module_& m) "base"_a, "sMode"_a = tools::StatsMode::Default, "cMode"_a = CheckMode::Default, - "verbose"_a = 0); + "verbose"_a = 0, + "Convert an OpenVDB base grid to a NanoVDB GridHandle."); #endif } // ============================================================================ -// Phase 5b/5c conversion bindings: AbsDiff/RelDiff oracle classes, and the -// polymorphic createNanoGrid free functions for quantized + index destination -// BuildTs. Each accepts source = NanoGrid OR build::Grid. +// Conversion bindings: AbsDiff/RelDiff oracle classes, and the polymorphic +// createNanoGrid free functions for quantized + index destination BuildTs. +// Each accepts source = NanoGrid OR build::Grid. // ============================================================================ namespace { @@ -247,9 +250,12 @@ void defineCreateNanoGridConversions(nb::module_& toolsModule) "default) means uninitialized; any non-negative value (including " "0.0) is treated as initialized by the operator bool() check, " "or the C++ create function can fill it in via init().") - .def(nb::init(), "tolerance"_a = -1.0f) - .def("getTolerance", &tools::AbsDiff::getTolerance) - .def("setTolerance", &tools::AbsDiff::setTolerance, "tolerance"_a) + .def(nb::init(), "tolerance"_a = -1.0f, + "Construct an AbsDiff oracle with the given absolute tolerance.") + .def("getTolerance", &tools::AbsDiff::getTolerance, + "Return the current absolute tolerance.") + .def("setTolerance", &tools::AbsDiff::setTolerance, "tolerance"_a, + "Replace the current absolute tolerance.") .def("__bool__", [](const tools::AbsDiff& self) { return bool(self); }, "True iff the tolerance has been initialized (>= 0)."); @@ -257,9 +263,12 @@ void defineCreateNanoGridConversions(nb::module_& toolsModule) nb::class_(toolsModule, "RelDiff", "Compression oracle for FpN: accept the approximation when " "|exact - approx| / max(|exact|, |approx|) <= tolerance.") - .def(nb::init(), "tolerance"_a = -1.0f) - .def("getTolerance", &tools::RelDiff::getTolerance) - .def("setTolerance", &tools::RelDiff::setTolerance, "tolerance"_a) + .def(nb::init(), "tolerance"_a = -1.0f, + "Construct a RelDiff oracle with the given relative tolerance.") + .def("getTolerance", &tools::RelDiff::getTolerance, + "Return the current relative tolerance.") + .def("setTolerance", &tools::RelDiff::setTolerance, "tolerance"_a, + "Replace the current relative tolerance.") .def("__bool__", [](const tools::RelDiff& self) { return bool(self); }, "True iff the tolerance has been initialized (>= 0)."); @@ -332,10 +341,10 @@ void defineCreateNanoGridConversions(nb::module_& toolsModule) // ------ Index / OnIndex ------ // - // createOnIndexGrid (the test-scaffold factory from Phase 3 follow-up) - // is now superseded by createNanoGridOnIndex. The legacy name keeps - // working through PyVoxelBlockManager.cc; the official Phase 5 name - // lives here. + // createNanoGridIndex / createNanoGridOnIndex are the canonical names + // for the broad-source-coverage index conversion bindings. A narrower + // createOnIndexGrid factory still lives in PyVoxelBlockManager.cc as + // the test scaffolding entry point used by the VBM unit tests. toolsModule.def("createNanoGridIndex", [](nb::handle src, uint32_t channels, bool includeStats, bool includeTiles, int verbose) { diff --git a/nanovdb/nanovdb/python/PyGridChecksum.cc b/nanovdb/nanovdb/python/PyGridChecksum.cc index b0cc350653..b0f42744b7 100644 --- a/nanovdb/nanovdb/python/PyGridChecksum.cc +++ b/nanovdb/nanovdb/python/PyGridChecksum.cc @@ -14,7 +14,10 @@ namespace pynanovdb { void defineCheckMode(nb::module_& m) { - nb::enum_(m, "CheckMode") + nb::enum_(m, "CheckMode", + "Selector controlling how aggressively a grid checksum is computed: " + "Disable skips checksumming, Partial covers only the header, Full " + "covers the whole grid, and Default picks the recommended mode.") .value("Disable", CheckMode::Disable) .value("Partial", CheckMode::Partial) .value("Full", CheckMode::Full) @@ -24,13 +27,21 @@ void defineCheckMode(nb::module_& m) void defineChecksum(nb::module_& m) { - nb::class_(m, "Checksum").def(nb::self == nb::self, "rhs"_a).def(nb::self != nb::self, "rhs"_a); + nb::class_(m, "Checksum", + "64-bit checksum value stored in a grid header. Produced by " + "tools.evalChecksum and compared against the stored one by " + "tools.validateChecksum.") + .def(nb::self == nb::self, "rhs"_a, + "Equality of two Checksum values.") + .def(nb::self != nb::self, "rhs"_a, + "Inequality of two Checksum values."); } void defineUpdateChecksum(nb::module_& m) { m.def( - "updateChecksum", [](GridData* gridData, CheckMode mode) { tools::updateChecksum(gridData, mode); }, "gridData"_a, "mode"_a); + "updateChecksum", [](GridData* gridData, CheckMode mode) { tools::updateChecksum(gridData, mode); }, "gridData"_a, "mode"_a, + "Recompute and store the checksum of gridData using the given CheckMode."); } void defineEvalChecksumModule(nb::module_& toolsModule) diff --git a/nanovdb/nanovdb/python/PyGridHandle.h b/nanovdb/nanovdb/python/PyGridHandle.h index 4774bd3cec..5ffd2ad497 100644 --- a/nanovdb/nanovdb/python/PyGridHandle.h +++ b/nanovdb/nanovdb/python/PyGridHandle.h @@ -121,16 +121,28 @@ template void defineGridHandleUtilities(nb::module_& m) template nb::class_> defineGridHandle(nb::module_& m, const char* name) { - return nb::class_>(m, name) - .def(nb::init<>()) - .def("reset", &nanovdb::GridHandle::reset) - .def("size", &nanovdb::GridHandle::bufferSize) - .def("isEmpty", &nanovdb::GridHandle::isEmpty) - .def("empty", &nanovdb::GridHandle::empty) + return nb::class_>(m, name, + "Owns a buffer holding one or more serialized NanoVDB grids. " + "Construct via nanovdb.tools.create* factories or nanovdb.io.readGrid(s); " + "access individual grids via handle.grid(n).") + .def(nb::init<>(), + "Construct an empty handle. Use the nanovdb.tools.create* " + "factories or nanovdb.io.readGrid(s) instead in normal use.") + .def("reset", &nanovdb::GridHandle::reset, + "Drop the underlying buffer; the handle becomes empty.") + .def("size", &nanovdb::GridHandle::bufferSize, + "Total byte size of the buffer backing this handle (sum of " + "every grid plus any internal padding).") + .def("isEmpty", &nanovdb::GridHandle::isEmpty, + "True iff the handle owns no buffer.") + .def("empty", &nanovdb::GridHandle::empty, + "Same as isEmpty(). Retained for parity with the C++ " + "GridHandle::empty() member.") .def( "__bool__", [](const nanovdb::GridHandle& handle) { return !handle.empty(); }, - nb::is_operator()) + nb::is_operator(), + "True iff the handle owns a non-empty buffer (`not isEmpty()`).") .def("copy", [](const nanovdb::GridHandle& handle) { return handle.template copy(); @@ -141,29 +153,48 @@ template nb::class_> defineGridHa "Return the n-th grid as a typed Grid subclass selected by " "gridType(n), or None if the BuildT is not bound in Python. " "The returned grid keeps this handle alive.") - .def("isPadded", &nanovdb::GridHandle::isPadded) - .def("gridCount", &nanovdb::GridHandle::gridCount) - .def("gridSize", &nanovdb::GridHandle::gridSize, nb::arg("n") = 0) - .def("gridType", &nanovdb::GridHandle::gridType, nb::arg("n") = 0) + .def("isPadded", &nanovdb::GridHandle::isPadded, + "True iff this handle's buffer is aligned past the natural " + "GridData alignment (used by the I/O code path).") + .def("gridCount", &nanovdb::GridHandle::gridCount, + "Number of grids stored in this handle.") + .def("gridSize", &nanovdb::GridHandle::gridSize, nb::arg("n") = 0, + "Byte size of the n-th grid (without padding).") + .def("gridType", &nanovdb::GridHandle::gridType, nb::arg("n") = 0, + "GridType enumerator of the n-th grid (e.g. GridType.Float). " + "Cheap to query — does not require materializing the grid.") .def( "gridData", [](nanovdb::GridHandle& handle, uint32_t n) { return nb::bytes(handle.gridData(n), handle.gridSize(n)); }, nb::arg("n") = 0, - nb::rv_policy::reference_internal) - .def("write", nb::overload_cast(&nanovdb::GridHandle::write, nb::const_), nb::arg("fileName")) - .def("write", nb::overload_cast(&nanovdb::GridHandle::write, nb::const_), nb::arg("fileName"), nb::arg("n")) + nb::rv_policy::reference_internal, + "Raw byte contents of the n-th grid as a Python bytes object. " + "Useful for hashing or for handing off to non-NanoVDB tooling.") + .def("write", nb::overload_cast(&nanovdb::GridHandle::write, nb::const_), + nb::arg("fileName"), + "Write every grid in this handle to the given .nvdb file.") + .def("write", nb::overload_cast(&nanovdb::GridHandle::write, nb::const_), + nb::arg("fileName"), nb::arg("n"), + "Write just the n-th grid in this handle to the given .nvdb file.") .def( - "read", [](nanovdb::GridHandle& handle, const std::string& fileName) { handle.read(fileName); }, nb::arg("fileName")) + "read", [](nanovdb::GridHandle& handle, const std::string& fileName) { handle.read(fileName); }, + nb::arg("fileName"), + "Replace this handle's contents with every grid read from the " + "given .nvdb file.") .def( "read", [](nanovdb::GridHandle& handle, const std::string& fileName, uint32_t n) { handle.read(fileName, n); }, nb::arg("fileName"), - nb::arg("n")) + nb::arg("n"), + "Replace this handle's contents with the n-th grid read from " + "the given .nvdb file.") .def( "read", [](nanovdb::GridHandle& handle, const std::string& fileName, const std::string& gridName) { handle.read(fileName, gridName); }, nb::arg("fileName"), - nb::arg("gridName")); + nb::arg("gridName"), + "Replace this handle's contents with the grid of the given " + "name read from the .nvdb file."); } void defineHostGridHandle(nb::module_& m); diff --git a/nanovdb/nanovdb/python/PyGridStats.cc b/nanovdb/nanovdb/python/PyGridStats.cc index 844360e773..908af0b0b6 100644 --- a/nanovdb/nanovdb/python/PyGridStats.cc +++ b/nanovdb/nanovdb/python/PyGridStats.cc @@ -17,7 +17,11 @@ namespace pynanovdb { void defineStatsMode(nb::module_& m) { - nb::enum_(m, "StatsMode") + nb::enum_(m, "StatsMode", + "Selector controlling which per-node statistics are computed by " + "tools.updateGridStats: Disable skips stats, BBox refreshes only " + "bounding boxes, MinMax adds min/max, and All adds average and " + "standard deviation as well.") .value("Disable", tools::StatsMode::Disable) .value("BBox", tools::StatsMode::BBox) .value("MinMax", tools::StatsMode::MinMax) @@ -35,7 +39,9 @@ static void defineExtrema(nb::module_& m, const char* name) using ValueT = typename NanoGrid::ValueType; using ExtremaT = tools::Extrema; - nb::class_(m, name) + nb::class_(m, name, + "Running minimum / maximum accumulator over a stream of values. " + "Build via repeated add(v) calls or via tools.getExtrema(grid, bbox).") .def(nb::init<>(), "Default-construct an Extrema with min = numeric_limits::max and " "max = numeric_limits::lowest, so any subsequent .add(v) gives " @@ -77,7 +83,9 @@ static void defineStats(nb::module_& m, const char* name, const char* baseName) using StatsT = tools::Stats; (void)baseName; // kept in signature for parity with extrema name lookup - nb::class_(m, name) + nb::class_(m, name, + "Running min/max/mean/variance/std accumulator over a stream of " + "values. Extends Extrema with sample-count-weighted moments.") .def(nb::init<>(), "Default-construct a Stats accumulator with zero samples.") .def("add", diff --git a/nanovdb/nanovdb/python/PyHostBuffer.cc b/nanovdb/nanovdb/python/PyHostBuffer.cc index e8fdfb3946..7a929aacf4 100644 --- a/nanovdb/nanovdb/python/PyHostBuffer.cc +++ b/nanovdb/nanovdb/python/PyHostBuffer.cc @@ -11,7 +11,10 @@ namespace pynanovdb { void defineHostBuffer(nb::module_& m) { - nb::class_(m, "HostBuffer"); + nb::class_(m, "HostBuffer", + "Default host-side buffer used to back a GridHandle. Memory is " + "owned by this buffer and freed when the handle (and therefore " + "the buffer) is destroyed."); } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyIO.cc b/nanovdb/nanovdb/python/PyIO.cc index fb50e3d08a..87d8435011 100644 --- a/nanovdb/nanovdb/python/PyIO.cc +++ b/nanovdb/nanovdb/python/PyIO.cc @@ -21,40 +21,67 @@ namespace { void defineFileGridMetaData(nb::module_& m) { - nb::class_(m, "FileMetaData") - .def_ro("gridSize", &io::FileMetaData::gridSize) - .def_ro("fileSize", &io::FileMetaData::fileSize) - .def_ro("nameKey", &io::FileMetaData::nameKey) - .def_ro("voxelCount", &io::FileMetaData::voxelCount) - .def_ro("gridType", &io::FileMetaData::gridType) - .def_ro("gridClass", &io::FileMetaData::gridClass) - .def_ro("indexBBox", &io::FileMetaData::indexBBox) - .def_ro("worldBBox", &io::FileMetaData::worldBBox) - .def_ro("voxelSize", &io::FileMetaData::voxelSize) - .def_ro("nameSize", &io::FileMetaData::nameSize) + nb::class_(m, "FileMetaData", + "Per-grid header read from the .nvdb file index. Mirrors the C++ " + "io::FileMetaData layout; subclassed by FileGridMetaData which adds " + "the grid name string.") + .def_ro("gridSize", &io::FileMetaData::gridSize, + "Uncompressed grid size in bytes.") + .def_ro("fileSize", &io::FileMetaData::fileSize, + "On-disk byte size of this grid (post-codec).") + .def_ro("nameKey", &io::FileMetaData::nameKey, + "Hash of the grid name used as a fast lookup key.") + .def_ro("voxelCount", &io::FileMetaData::voxelCount, + "Number of active voxels in this grid.") + .def_ro("gridType", &io::FileMetaData::gridType, + "GridType enumerator naming the BuildT of this grid.") + .def_ro("gridClass", &io::FileMetaData::gridClass, + "GridClass enumerator (LevelSet, FogVolume, ...).") + .def_ro("indexBBox", &io::FileMetaData::indexBBox, + "Axis-aligned bounding box of active voxels in index space.") + .def_ro("worldBBox", &io::FileMetaData::worldBBox, + "Axis-aligned bounding box of active voxels in world space.") + .def_ro("voxelSize", &io::FileMetaData::voxelSize, + "World-space size of a single voxel.") + .def_ro("nameSize", &io::FileMetaData::nameSize, + "Length of the grid name string including the null terminator.") .def_prop_ro("nodeCount", [](io::FileMetaData& metaData) { return std::make_tuple(metaData.nodeCount[0], metaData.nodeCount[1], metaData.nodeCount[2], metaData.nodeCount[3]); - }) + }, + "Tuple (leaf, lower, upper, root) of node counts in this grid.") .def_prop_ro("tileCount", - [](io::FileMetaData& metaData) { return std::make_tuple(metaData.tileCount[0], metaData.tileCount[1], metaData.tileCount[2]); }) - .def_ro("codec", &io::FileMetaData::codec) - .def_ro("blindDataCount", &io::FileMetaData::blindDataCount) - .def_ro("version", &io::FileMetaData::version); - - nb::bind_vector>(m, "FileMetaDataVector"); - - nb::class_(m, "FileGridMetaData") - .def_ro("gridName", &io::FileGridMetaData::gridName) - .def("memUsage", &io::FileGridMetaData::memUsage); - - nb::bind_vector>(m, "FileGridMetaDataVector"); + [](io::FileMetaData& metaData) { return std::make_tuple(metaData.tileCount[0], metaData.tileCount[1], metaData.tileCount[2]); }, + "Tuple (lower-tile, upper-tile, root-tile) of active-tile counts.") + .def_ro("codec", &io::FileMetaData::codec, + "Codec used to compress this grid on disk.") + .def_ro("blindDataCount", &io::FileMetaData::blindDataCount, + "Number of blind-data channels attached to this grid.") + .def_ro("version", &io::FileMetaData::version, + "NanoVDB version stored in the file when this grid was written."); + + nb::bind_vector>(m, "FileMetaDataVector", + "List of FileMetaData entries, one per grid in a .nvdb file."); + + nb::class_(m, "FileGridMetaData", + "FileMetaData extended with the grid name. Returned by " + "readGridMetaData() so callers can identify grids by name without " + "materializing them.") + .def_ro("gridName", &io::FileGridMetaData::gridName, + "Grid name as a Python string.") + .def("memUsage", &io::FileGridMetaData::memUsage, + "Byte size of this metadata record in memory."); + + nb::bind_vector>(m, "FileGridMetaDataVector", + "List of FileGridMetaData entries, one per grid in a .nvdb file."); } template void defineReadWriteGrid(nb::module_& m) { - m.def("hasGrid", nb::overload_cast(&io::hasGrid), "fileName"_a, "gridName"_a); - m.def("readGridMetaData", nb::overload_cast(&io::readGridMetaData), "fileName"_a); + m.def("hasGrid", nb::overload_cast(&io::hasGrid), "fileName"_a, "gridName"_a, + "Return True iff the .nvdb file at fileName contains a grid named gridName."); + m.def("readGridMetaData", nb::overload_cast(&io::readGridMetaData), "fileName"_a, + "Return a FileGridMetaDataVector describing every grid stored in the .nvdb file."); } template nb::list readGrids(const std::string& fileName, int verbose, const BufferT& buffer) @@ -84,21 +111,26 @@ void defineHostReadWriteGrid(nb::module_& m) "fileName"_a, "handle"_a, "codec"_a = io::Codec::NONE, - "verbose"_a = 0); - m.def("writeGrids", &writeGrids, "fileName"_a, "handles"_a, "codec"_a = io::Codec::NONE, "verbose"_a = 0); + "verbose"_a = 0, + "Write the grids in handle to the .nvdb file at fileName using the given codec."); + m.def("writeGrids", &writeGrids, "fileName"_a, "handles"_a, "codec"_a = io::Codec::NONE, "verbose"_a = 0, + "Write every GridHandle in the handles list to the .nvdb file at fileName."); m.def("readGrid", nb::overload_cast(&io::template readGrid), "fileName"_a, "n"_a = 0, "verbose"_a = 0, - "buffer"_a = BufferT()); + "buffer"_a = BufferT(), + "Read the n-th grid from the .nvdb file at fileName into a fresh GridHandle."); m.def("readGrid", nb::overload_cast(&io::template readGrid), "fileName"_a, "gridName"_a, "verbose"_a = 0, - "buffer"_a = BufferT()); - m.def("readGrids", &readGrids, "fileName"_a, "verbose"_a = 0, "buffer"_a = BufferT()); + "buffer"_a = BufferT(), + "Read the grid named gridName from the .nvdb file at fileName into a fresh GridHandle."); + m.def("readGrids", &readGrids, "fileName"_a, "verbose"_a = 0, "buffer"_a = BufferT(), + "Read every grid from the .nvdb file at fileName, returning a list of GridHandles."); } #ifdef NANOVDB_USE_CUDA @@ -112,21 +144,26 @@ void defineDeviceReadWriteGrid(nb::module_& m) "fileName"_a, "handle"_a, "codec"_a = io::Codec::NONE, - "verbose"_a = 0); - m.def("deviceWriteGrids", &writeGrids, "fileName"_a, "handles"_a, "codec"_a = io::Codec::NONE, "verbose"_a = 0); + "verbose"_a = 0, + "Write the grids in a device-backed handle to the .nvdb file at fileName."); + m.def("deviceWriteGrids", &writeGrids, "fileName"_a, "handles"_a, "codec"_a = io::Codec::NONE, "verbose"_a = 0, + "Write every device-backed GridHandle in handles to the .nvdb file at fileName."); m.def("deviceReadGrid", nb::overload_cast(&io::template readGrid), "fileName"_a, "n"_a = 0, "verbose"_a = 0, - "buffer"_a = BufferT()); + "buffer"_a = BufferT(), + "Read the n-th grid from the .nvdb file at fileName into a fresh DeviceGridHandle."); m.def("deviceReadGrid", nb::overload_cast(&io::template readGrid), "fileName"_a, "gridName"_a, "verbose"_a = 0, - "buffer"_a = BufferT()); - m.def("deviceReadGrids", &readGrids, "fileName"_a, "verbose"_a = 0, "buffer"_a = BufferT()); + "buffer"_a = BufferT(), + "Read the grid named gridName from the .nvdb file at fileName into a fresh DeviceGridHandle."); + m.def("deviceReadGrids", &readGrids, "fileName"_a, "verbose"_a = 0, "buffer"_a = BufferT(), + "Read every grid from the .nvdb file at fileName into device-backed handles."); } #endif @@ -134,7 +171,10 @@ void defineDeviceReadWriteGrid(nb::module_& m) void defineIOModule(nb::module_& m) { - nb::enum_(m, "Codec") + nb::enum_(m, "Codec", + "Compression codec selector used when writing a .nvdb file. NONE " + "writes raw bytes; ZIP uses zlib; BLOSC uses the blosc codec when " + "compiled in.") .value("NONE", io::Codec::NONE) .value("ZIP", io::Codec::ZIP) .value("BLOSC", io::Codec::BLOSC) diff --git a/nanovdb/nanovdb/python/PyMath.cc b/nanovdb/nanovdb/python/PyMath.cc index 4ae8c852fe..2e212f09b9 100644 --- a/nanovdb/nanovdb/python/PyMath.cc +++ b/nanovdb/nanovdb/python/PyMath.cc @@ -25,18 +25,27 @@ void defineCoord(nb::module_& m) using ValueType = math::Coord::ValueType; nb::class_(m, "Coord", "Signed (i, j, k) 32-bit integer coordinate class, similar to openvdb::math::Coord") - .def(nb::init<>()) - .def(nb::init(), "n"_a) - .def(nb::init(), "i"_a, "j"_a, "k"_a) + .def(nb::init<>(), + "Construct (0, 0, 0).") + .def(nb::init(), "n"_a, + "Construct (n, n, n).") + .def(nb::init(), "i"_a, "j"_a, "k"_a, + "Construct (i, j, k).") .def_prop_rw( - "x", [](const math::Coord& ijk) { return ijk.x(); }, [](math::Coord& ijk, int32_t i) { ijk.x() = i; }) + "x", [](const math::Coord& ijk) { return ijk.x(); }, [](math::Coord& ijk, int32_t i) { ijk.x() = i; }, + "First component of the (i, j, k) triple.") .def_prop_rw( - "y", [](const math::Coord& ijk) { return ijk.y(); }, [](math::Coord& ijk, int32_t j) { ijk.y() = j; }) + "y", [](const math::Coord& ijk) { return ijk.y(); }, [](math::Coord& ijk, int32_t j) { ijk.y() = j; }, + "Second component of the (i, j, k) triple.") .def_prop_rw( - "z", [](const math::Coord& ijk) { return ijk.z(); }, [](math::Coord& ijk, int32_t k) { ijk.z() = k; }) - .def_static("max", &math::Coord::max) - .def_static("min", &math::Coord::min) - .def_static("memUsage", &math::Coord::memUsage) + "z", [](const math::Coord& ijk) { return ijk.z(); }, [](math::Coord& ijk, int32_t k) { ijk.z() = k; }, + "Third component of the (i, j, k) triple.") + .def_static("max", &math::Coord::max, + "Largest representable Coord (INT32_MAX in every component).") + .def_static("min", &math::Coord::min, + "Smallest representable Coord (INT32_MIN in every component).") + .def_static("memUsage", &math::Coord::memUsage, + "Byte size of a Coord instance.") .def( "__getitem__", [](const math::Coord& ijk, size_t i) { @@ -45,7 +54,8 @@ void defineCoord(nb::module_& m) } return ijk[static_cast(i)]; }, - "i"_a) + "i"_a, + "Read the i-th component (0=x, 1=y, 2=z).") .def( "__setitem__", [](math::Coord& ijk, size_t i, ValueType value) { @@ -55,41 +65,69 @@ void defineCoord(nb::module_& m) ijk[static_cast(i)] = value; }, "i"_a, - "value"_a) + "value"_a, + "Write the i-th component (0=x, 1=y, 2=z).") .def( - "__and__", [](const math::Coord& a, math::Coord::IndexType b) { return a & b; }, nb::is_operator(), "n"_a) + "__and__", [](const math::Coord& a, math::Coord::IndexType b) { return a & b; }, nb::is_operator(), "n"_a, + "Component-wise bitwise AND with the scalar n.") .def( - "__lshift__", [](const math::Coord& a, math::Coord::IndexType b) { return a << b; }, nb::is_operator(), "n"_a) + "__lshift__", [](const math::Coord& a, math::Coord::IndexType b) { return a << b; }, nb::is_operator(), "n"_a, + "Component-wise left shift by n bits.") .def( - "__rshift__", [](const math::Coord& a, math::Coord::IndexType b) { return a >> b; }, nb::is_operator(), "n"_a) - .def(nb::self < nb::self, "rhs"_a) - .def(nb::self == nb::self, "rhs"_a) - .def(nb::self != nb::self, "rhs"_a) + "__rshift__", [](const math::Coord& a, math::Coord::IndexType b) { return a >> b; }, nb::is_operator(), "n"_a, + "Component-wise right shift by n bits.") + .def(nb::self < nb::self, "rhs"_a, + "Lexicographic less-than comparison.") + .def(nb::self == nb::self, "rhs"_a, + "Equality of all three components.") + .def(nb::self != nb::self, "rhs"_a, + "Inequality of any one component.") .def( - "__iand__", [](math::Coord& a, int b) { return a &= b; }, nb::is_operator(), "n"_a) + "__iand__", [](math::Coord& a, int b) { return a &= b; }, nb::is_operator(), "n"_a, + "In-place component-wise bitwise AND with the scalar n.") .def( - "__ilshift__", [](math::Coord& a, uint32_t b) { return a <<= b; }, nb::is_operator(), "n"_a) + "__ilshift__", [](math::Coord& a, uint32_t b) { return a <<= b; }, nb::is_operator(), "n"_a, + "In-place component-wise left shift by n bits.") .def( - "__irshift__", [](math::Coord& a, uint32_t b) { return a >>= b; }, nb::is_operator(), "n"_a) + "__irshift__", [](math::Coord& a, uint32_t b) { return a >>= b; }, nb::is_operator(), "n"_a, + "In-place component-wise right shift by n bits.") .def( - "__iadd__", [](math::Coord& a, int b) { return a += b; }, nb::is_operator(), "n"_a) - .def(nb::self + nb::self, "rhs"_a) - .def(nb::self - nb::self, "rhs"_a) - .def(-nb::self) - .def(nb::self += nb::self, "rhs"_a) - .def(nb::self -= nb::self, "rhs"_a) - .def("minComponent", &math::Coord::minComponent, "other"_a) - .def("maxComponent", &math::Coord::maxComponent, "other"_a) - .def("offsetBy", nb::overload_cast(&math::Coord::offsetBy, nb::const_), "dx"_a, "dy"_a, "dz"_a) - .def("offsetBy", nb::overload_cast(&math::Coord::offsetBy, nb::const_), "n"_a) - .def_static("lessThan", &math::Coord::lessThan, "a"_a, "b"_a) - .def_static("Floor", &math::Coord::template Floor>, "xyz"_a) - .def_static("Floor", &math::Coord::template Floor>, "xyz"_a) - .def("hash", &math::Coord::template hash<12>) - .def("octant", &math::Coord::octant) - .def("asVec3s", &math::Coord::asVec3s) - .def("asVec3d", &math::Coord::asVec3d) - .def("round", &math::Coord::round) + "__iadd__", [](math::Coord& a, int b) { return a += b; }, nb::is_operator(), "n"_a, + "In-place add the scalar n to every component.") + .def(nb::self + nb::self, "rhs"_a, + "Component-wise addition.") + .def(nb::self - nb::self, "rhs"_a, + "Component-wise subtraction.") + .def(-nb::self, + "Negate every component.") + .def(nb::self += nb::self, "rhs"_a, + "In-place component-wise addition.") + .def(nb::self -= nb::self, "rhs"_a, + "In-place component-wise subtraction.") + .def("minComponent", &math::Coord::minComponent, "other"_a, + "Component-wise minimum with other. See nanovdb::math::Coord::minComponent in NanoVDB.h.") + .def("maxComponent", &math::Coord::maxComponent, "other"_a, + "Component-wise maximum with other. See nanovdb::math::Coord::maxComponent in NanoVDB.h.") + .def("offsetBy", nb::overload_cast(&math::Coord::offsetBy, nb::const_), "dx"_a, "dy"_a, "dz"_a, + "Return a new Coord offset by (dx, dy, dz). See nanovdb::math::Coord::offsetBy in NanoVDB.h.") + .def("offsetBy", nb::overload_cast(&math::Coord::offsetBy, nb::const_), "n"_a, + "Return a new Coord offset by n in every component.") + .def_static("lessThan", &math::Coord::lessThan, "a"_a, "b"_a, + "Component-wise a < b returning a Coord of 0 / 1 flags.") + .def_static("Floor", &math::Coord::template Floor>, "xyz"_a, + "Floor each component of a Vec3f to produce an integer Coord.") + .def_static("Floor", &math::Coord::template Floor>, "xyz"_a, + "Floor each component of a Vec3d to produce an integer Coord.") + .def("hash", &math::Coord::template hash<12>, + "Spatial hash of this coordinate suited for hashed root-table lookups.") + .def("octant", &math::Coord::octant, + "Return the 0..7 octant index of this coordinate's sign bits.") + .def("asVec3s", &math::Coord::asVec3s, + "Convert to a Vec3f (float) with no scaling.") + .def("asVec3d", &math::Coord::asVec3d, + "Convert to a Vec3d (double) with no scaling.") + .def("round", &math::Coord::round, + "Component-wise round; for an integer Coord this is the identity.") .def("__repr__", [](const math::Coord& ijk) { std::stringstream ostr; ostr << ijk; @@ -100,13 +138,20 @@ void defineCoord(nb::module_& m) template void defineVec3(nb::module_& m, const char* name, const char* doc) { nb::class_>(m, name, doc) - .def(nb::init<>()) - .def(nb::init(), "x"_a) - .def(nb::init(), "x"_a, "y"_a, "z"_a) - .def(nb::init>(), "v"_a) - .def(nb::init(), "ijk"_a) - .def(nb::self == nb::self, "rhs"_a) - .def(nb::self != nb::self, "rhs"_a) + .def(nb::init<>(), + "Construct a zero-initialized vector.") + .def(nb::init(), "x"_a, + "Construct (x, x, x).") + .def(nb::init(), "x"_a, "y"_a, "z"_a, + "Construct (x, y, z).") + .def(nb::init>(), "v"_a, + "Copy-construct from another Vec3.") + .def(nb::init(), "ijk"_a, + "Construct from an integer Coord, casting each component.") + .def(nb::self == nb::self, "rhs"_a, + "Component-wise equality.") + .def(nb::self != nb::self, "rhs"_a, + "Component-wise inequality.") .def( "__getitem__", [](const math::Vec3& v, size_t i) { @@ -115,7 +160,8 @@ template void defineVec3(nb::module_& m, const char* name, const cha } return v[static_cast(i)]; }, - "i"_a) + "i"_a, + "Read the i-th component (0=x, 1=y, 2=z).") .def( "__setitem__", [](math::Vec3& v, size_t i, T value) { @@ -125,38 +171,68 @@ template void defineVec3(nb::module_& m, const char* name, const cha v[static_cast(i)] = value; }, "i"_a, - "value"_a) - .def("dot", &math::Vec3::template dot>, "v"_a) - .def("cross", &math::Vec3::template cross>, "v"_a) - .def("lengthSqr", &math::Vec3::lengthSqr) - .def("length", &math::Vec3::length) - .def(-nb::self) - .def(nb::self * nb::self, "v"_a) - .def(nb::self / nb::self, "v"_a) - .def(nb::self + nb::self, "v"_a) - .def(nb::self - nb::self, "v"_a) - .def(nb::self + math::Coord(), "ijk"_a) - .def(nb::self - math::Coord(), "ijk"_a) - .def(nb::self * T(), "s"_a) - .def(nb::self / T(), "s"_a) - .def(nb::self += nb::self, "v"_a) - .def(nb::self += math::Coord(), "ijk"_a) - .def(nb::self -= nb::self, "v"_a) - .def(nb::self -= math::Coord(), "ijk"_a) - .def(nb::self *= T(), "s"_a) - .def(nb::self /= T(), "s"_a) - .def("normalize", &math::Vec3::normalize) - .def("minComponent", &math::Vec3::minComponent, "other"_a) - .def("maxComponent", &math::Vec3::maxComponent, "other"_a) - .def("min", &math::Vec3::min) - .def("max", &math::Vec3::max) - .def("floor", &math::Vec3::floor) - .def("ceil", &math::Vec3::ceil) - .def("round", &math::Vec3::round) + "value"_a, + "Write the i-th component (0=x, 1=y, 2=z).") + .def("dot", &math::Vec3::template dot>, "v"_a, + "Dot product with another vector.") + .def("cross", &math::Vec3::template cross>, "v"_a, + "Cross product with another vector.") + .def("lengthSqr", &math::Vec3::lengthSqr, + "Squared Euclidean length (cheaper than length()).") + .def("length", &math::Vec3::length, + "Euclidean length of this vector.") + .def(-nb::self, + "Negate every component.") + .def(nb::self * nb::self, "v"_a, + "Component-wise multiplication.") + .def(nb::self / nb::self, "v"_a, + "Component-wise division.") + .def(nb::self + nb::self, "v"_a, + "Component-wise addition.") + .def(nb::self - nb::self, "v"_a, + "Component-wise subtraction.") + .def(nb::self + math::Coord(), "ijk"_a, + "Add an integer Coord component-wise.") + .def(nb::self - math::Coord(), "ijk"_a, + "Subtract an integer Coord component-wise.") + .def(nb::self * T(), "s"_a, + "Multiply every component by the scalar s.") + .def(nb::self / T(), "s"_a, + "Divide every component by the scalar s.") + .def(nb::self += nb::self, "v"_a, + "In-place component-wise addition.") + .def(nb::self += math::Coord(), "ijk"_a, + "In-place add an integer Coord.") + .def(nb::self -= nb::self, "v"_a, + "In-place component-wise subtraction.") + .def(nb::self -= math::Coord(), "ijk"_a, + "In-place subtract an integer Coord.") + .def(nb::self *= T(), "s"_a, + "In-place scalar multiply.") + .def(nb::self /= T(), "s"_a, + "In-place scalar divide.") + .def("normalize", &math::Vec3::normalize, + "Scale this vector to unit length in place.") + .def("minComponent", &math::Vec3::minComponent, "other"_a, + "Component-wise minimum with other.") + .def("maxComponent", &math::Vec3::maxComponent, "other"_a, + "Component-wise maximum with other.") + .def("min", &math::Vec3::min, + "Smallest single component of this vector.") + .def("max", &math::Vec3::max, + "Largest single component of this vector.") + .def("floor", &math::Vec3::floor, + "Component-wise floor.") + .def("ceil", &math::Vec3::ceil, + "Component-wise ceiling.") + .def("round", &math::Vec3::round, + "Component-wise round.") .def( - "__mul__", [](const T& a, math::Vec3 b) { return a * b; }, nb::is_operator(), "b"_a) + "__mul__", [](const T& a, math::Vec3 b) { return a * b; }, nb::is_operator(), "b"_a, + "Right-multiply: scalar * Vec3.") .def( - "__truediv__", [](const T& a, math::Vec3 b) { return a / b; }, nb::is_operator(), "b"_a) + "__truediv__", [](const T& a, math::Vec3 b) { return a / b; }, nb::is_operator(), "b"_a, + "Right-divide: scalar / Vec3, component-wise.") .def("__repr__", [](const math::Vec3& v) { std::stringstream ostr; ostr << v; @@ -167,12 +243,18 @@ template void defineVec3(nb::module_& m, const char* name, const cha template void defineVec4(nb::module_& m, const char* name, const char* doc) { nb::class_>(m, name, doc) - .def(nb::init<>()) - .def(nb::init(), "x"_a) - .def(nb::init(), "x"_a, "y"_a, "z"_a, "w"_a) - .def(nb::init>(), "v"_a) - .def(nb::self == nb::self, "rhs"_a) - .def(nb::self != nb::self, "rhs"_a) + .def(nb::init<>(), + "Construct a zero-initialized vector.") + .def(nb::init(), "x"_a, + "Construct (x, x, x, x).") + .def(nb::init(), "x"_a, "y"_a, "z"_a, "w"_a, + "Construct (x, y, z, w).") + .def(nb::init>(), "v"_a, + "Copy-construct from another Vec4.") + .def(nb::self == nb::self, "rhs"_a, + "Component-wise equality.") + .def(nb::self != nb::self, "rhs"_a, + "Component-wise inequality.") .def( "__getitem__", [](const math::Vec4& v, size_t i) { @@ -181,7 +263,8 @@ template void defineVec4(nb::module_& m, const char* name, const cha } return v[static_cast(i)]; }, - "i"_a) + "i"_a, + "Read the i-th component (0=x, 1=y, 2=z, 3=w).") .def( "__setitem__", [](math::Vec4& v, size_t i, T value) { @@ -191,28 +274,48 @@ template void defineVec4(nb::module_& m, const char* name, const cha v[static_cast(i)] = value; }, "i"_a, - "value"_a) - .def("dot", &math::Vec4::template dot>, "v"_a) - .def("lengthSqr", &math::Vec4::lengthSqr) - .def("length", &math::Vec4::length) - .def(-nb::self) - .def(nb::self * nb::self, "v"_a) - .def(nb::self / nb::self, "v"_a) - .def(nb::self + nb::self, "v"_a) - .def(nb::self - nb::self, "v"_a) - .def(nb::self * T(), "s"_a) - .def(nb::self / T(), "s"_a) - .def(nb::self += nb::self, "v"_a) - .def(nb::self -= nb::self, "v"_a) - .def(nb::self *= T(), "s"_a) - .def(nb::self /= T(), "s"_a) - .def("normalize", &math::Vec4::normalize) - .def("minComponent", &math::Vec4::minComponent, "other"_a) - .def("maxComponent", &math::Vec4::maxComponent, "other"_a) + "value"_a, + "Write the i-th component (0=x, 1=y, 2=z, 3=w).") + .def("dot", &math::Vec4::template dot>, "v"_a, + "Dot product with another vector.") + .def("lengthSqr", &math::Vec4::lengthSqr, + "Squared Euclidean length (cheaper than length()).") + .def("length", &math::Vec4::length, + "Euclidean length of this vector.") + .def(-nb::self, + "Negate every component.") + .def(nb::self * nb::self, "v"_a, + "Component-wise multiplication.") + .def(nb::self / nb::self, "v"_a, + "Component-wise division.") + .def(nb::self + nb::self, "v"_a, + "Component-wise addition.") + .def(nb::self - nb::self, "v"_a, + "Component-wise subtraction.") + .def(nb::self * T(), "s"_a, + "Multiply every component by the scalar s.") + .def(nb::self / T(), "s"_a, + "Divide every component by the scalar s.") + .def(nb::self += nb::self, "v"_a, + "In-place component-wise addition.") + .def(nb::self -= nb::self, "v"_a, + "In-place component-wise subtraction.") + .def(nb::self *= T(), "s"_a, + "In-place scalar multiply.") + .def(nb::self /= T(), "s"_a, + "In-place scalar divide.") + .def("normalize", &math::Vec4::normalize, + "Scale this vector to unit length in place.") + .def("minComponent", &math::Vec4::minComponent, "other"_a, + "Component-wise minimum with other.") + .def("maxComponent", &math::Vec4::maxComponent, "other"_a, + "Component-wise maximum with other.") .def( - "__mul__", [](const T& a, math::Vec4 b) { return a * b; }, nb::is_operator(), "b"_a) + "__mul__", [](const T& a, math::Vec4 b) { return a * b; }, nb::is_operator(), "b"_a, + "Right-multiply: scalar * Vec4.") .def( - "__truediv__", [](const T& a, math::Vec4 b) { return a / b; }, nb::is_operator(), "b"_a) + "__truediv__", [](const T& a, math::Vec4 b) { return a / b; }, nb::is_operator(), "b"_a, + "Right-divide: scalar / Vec4, component-wise.") .def("__repr__", [](const math::Vec4& v) { std::stringstream ostr; ostr << v; @@ -225,18 +328,30 @@ void defineRgba8(nb::module_& m) using ValueType = math::Rgba8::ValueType; nb::class_(m, "Rgba8", "8-bit red, green, blue, alpha packed into 32 bit unsigned int") - .def(nb::init<>()) - .def(nb::init(), "other"_a) - .def(nb::init(), "r"_a, "g"_a, "b"_a, "a"_a = 255) - .def(nb::init(), "v"_a) - .def(nb::init(), "r"_a, "g"_a, "b"_a, "a"_a = 1.0) - .def(nb::init(), "rgb"_a) - .def(nb::init(), "rgba"_a) - .def(nb::self < nb::self, "rhs"_a) - .def(nb::self == nb::self, "rhs"_a) - .def("lengthSqr", &math::Rgba8::lengthSqr) - .def("length", &math::Rgba8::length) - .def("asFloat", &math::Rgba8::asFloat, "n"_a) + .def(nb::init<>(), + "Construct a fully transparent black Rgba8 (all components 0).") + .def(nb::init(), "other"_a, + "Copy-construct from another Rgba8.") + .def(nb::init(), "r"_a, "g"_a, "b"_a, "a"_a = 255, + "Construct from four 0..255 uint8 channels; a defaults to fully opaque.") + .def(nb::init(), "v"_a, + "Construct a gray Rgba8 with every channel set to v.") + .def(nb::init(), "r"_a, "g"_a, "b"_a, "a"_a = 1.0, + "Construct from four 0..1 floats, clamped and quantized to uint8.") + .def(nb::init(), "rgb"_a, + "Construct from an RGB float triple; alpha defaults to opaque.") + .def(nb::init(), "rgba"_a, + "Construct from an RGBA float quadruple.") + .def(nb::self < nb::self, "rhs"_a, + "Less-than comparison on the packed uint32 representation.") + .def(nb::self == nb::self, "rhs"_a, + "Equality on the packed uint32 representation.") + .def("lengthSqr", &math::Rgba8::lengthSqr, + "Squared length over (r, g, b, a) as integers.") + .def("length", &math::Rgba8::length, + "Euclidean length over (r, g, b, a) as floats.") + .def("asFloat", &math::Rgba8::asFloat, "n"_a, + "Return the n-th channel as a 0..1 float.") .def( "__getitem__", [](const math::Rgba8& rgba, size_t i) { @@ -245,7 +360,8 @@ void defineRgba8(nb::module_& m) } return rgba[static_cast(i)]; }, - "i"_a) + "i"_a, + "Read the i-th channel as a uint8 (0=r, 1=g, 2=b, 3=a).") .def( "__setitem__", [](math::Rgba8& rgba, size_t i, ValueType value) { @@ -255,26 +371,38 @@ void defineRgba8(nb::module_& m) rgba[static_cast(i)] = value; }, "i"_a, - "value"_a) + "value"_a, + "Write the i-th channel as a uint8 (0=r, 1=g, 2=b, 3=a).") .def_prop_rw( - "packed", [](const math::Rgba8& rgba) { return rgba.packed(); }, [](math::Rgba8& rgba, uint32_t packed) { rgba.packed() = packed; }) + "packed", [](const math::Rgba8& rgba) { return rgba.packed(); }, [](math::Rgba8& rgba, uint32_t packed) { rgba.packed() = packed; }, + "The raw 32-bit packed RGBA representation.") .def_prop_rw( - "r", [](const math::Rgba8& rgba) { return rgba.r(); }, [](math::Rgba8& rgba, uint8_t r) { rgba.r() = r; }) + "r", [](const math::Rgba8& rgba) { return rgba.r(); }, [](math::Rgba8& rgba, uint8_t r) { rgba.r() = r; }, + "Red channel as a uint8 0..255.") .def_prop_rw( - "g", [](const math::Rgba8& rgba) { return rgba.g(); }, [](math::Rgba8& rgba, uint8_t g) { rgba.g() = g; }) + "g", [](const math::Rgba8& rgba) { return rgba.g(); }, [](math::Rgba8& rgba, uint8_t g) { rgba.g() = g; }, + "Green channel as a uint8 0..255.") .def_prop_rw( - "b", [](const math::Rgba8& rgba) { return rgba.b(); }, [](math::Rgba8& rgba, uint8_t b) { rgba.b() = b; }) + "b", [](const math::Rgba8& rgba) { return rgba.b(); }, [](math::Rgba8& rgba, uint8_t b) { rgba.b() = b; }, + "Blue channel as a uint8 0..255.") .def_prop_rw( - "a", [](const math::Rgba8& rgba) { return rgba.a(); }, [](math::Rgba8& rgba, uint8_t a) { rgba.a() = a; }) - .def("asVec3f", [](const math::Rgba8& rgba) { return Vec3f(rgba); }) - .def("asVec4f", [](const math::Rgba8& rgba) { return Vec4f(rgba); }); + "a", [](const math::Rgba8& rgba) { return rgba.a(); }, [](math::Rgba8& rgba, uint8_t a) { rgba.a() = a; }, + "Alpha channel as a uint8 0..255.") + .def("asVec3f", [](const math::Rgba8& rgba) { return Vec3f(rgba); }, + "Convert RGB channels to a Vec3f of 0..1 floats (alpha dropped).") + .def("asVec4f", [](const math::Rgba8& rgba) { return Vec4f(rgba); }, + "Convert RGBA channels to a Vec4f of 0..1 floats."); } template void defineBaseBBox(nb::module_& m, const char* name) { - nb::class_>(m, name) - .def(nb::self == nb::self, "rhs"_a) - .def(nb::self != nb::self, "rhs"_a) + nb::class_>(m, name, + "Axis-aligned bounding-box base class. Stores a min / max corner; " + "concrete subclasses add the open-interval vs closed-interval semantics.") + .def(nb::self == nb::self, "rhs"_a, + "Equality of both min and max corners.") + .def(nb::self != nb::self, "rhs"_a, + "Inequality of either min or max corner.") .def( "__getitem__", [](const math::BaseBBox& bbox, size_t i) { @@ -283,7 +411,8 @@ template void defineBaseBBox(nb::module_& m, const char* name) } return bbox[static_cast(i)]; }, - "i"_a) + "i"_a, + "Read corner 0 (min) or corner 1 (max).") .def( "__setitem__", [](math::BaseBBox& bbox, size_t i, const Vec3T& value) { @@ -293,29 +422,45 @@ template void defineBaseBBox(nb::module_& m, const char* name) bbox[static_cast(i)] = value; }, "i"_a, - "value"_a) + "value"_a, + "Write corner 0 (min) or corner 1 (max).") .def_prop_rw( - "min", [](const math::BaseBBox& bbox) { return bbox.min(); }, [](math::BaseBBox& bbox, const Vec3T& min) { bbox.min() = min; }) + "min", [](const math::BaseBBox& bbox) { return bbox.min(); }, [](math::BaseBBox& bbox, const Vec3T& min) { bbox.min() = min; }, + "Minimum corner of the bounding box.") .def_prop_rw( - "max", [](const math::BaseBBox& bbox) { return bbox.max(); }, [](math::BaseBBox& bbox, const Vec3T& max) { bbox.max() = max; }) - .def("translate", &math::BaseBBox::translate, "xyz"_a) - .def("expand", nb::overload_cast(&math::BaseBBox::expand), "xyz"_a) - .def("expand", nb::overload_cast&>(&math::BaseBBox::expand), "bbox"_a) - .def("intersect", &math::BaseBBox::intersect, "bbox"_a) - .def("isInside", &math::BaseBBox::isInside, "xyz"_a); + "max", [](const math::BaseBBox& bbox) { return bbox.max(); }, [](math::BaseBBox& bbox, const Vec3T& max) { bbox.max() = max; }, + "Maximum corner of the bounding box.") + .def("translate", &math::BaseBBox::translate, "xyz"_a, + "Translate this bounding box by xyz in place.") + .def("expand", nb::overload_cast(&math::BaseBBox::expand), "xyz"_a, + "Grow this bounding box to include the point xyz.") + .def("expand", nb::overload_cast&>(&math::BaseBBox::expand), "bbox"_a, + "Grow this bounding box to include another bbox in its entirety.") + .def("intersect", &math::BaseBBox::intersect, "bbox"_a, + "Shrink this bounding box to the intersection with bbox.") + .def("isInside", &math::BaseBBox::isInside, "xyz"_a, + "True iff xyz lies inside this bounding box."); } template void defineBBoxFloatingPoint(nb::module_& m, const char* name, const char* doc) { nb::class_, math::BaseBBox>(m, name, doc) - .def(nb::init<>()) - .def(nb::init(), "min"_a, "max"_a) - .def(nb::init(), "min"_a, "max"_a) - .def_static("createCube", &math::BBox::createCube, "min"_a, "dim"_a) - .def(nb::init&>(), "bbox"_a) - .def("empty", &math::BBox::empty) - .def("dim", &math::BBox::dim) - .def("isInside", &math::BBox::isInside, "p"_a) + .def(nb::init<>(), + "Construct an empty bounding box (min > max sentinel).") + .def(nb::init(), "min"_a, "max"_a, + "Construct from explicit min and max corners.") + .def(nb::init(), "min"_a, "max"_a, + "Construct from integer Coord corners, cast to floating-point.") + .def_static("createCube", &math::BBox::createCube, "min"_a, "dim"_a, + "Construct an axis-aligned cube of side dim anchored at min.") + .def(nb::init&>(), "bbox"_a, + "Construct from an integer CoordBBox, cast to floating-point.") + .def("empty", &math::BBox::empty, + "True iff this bbox is empty (any min component > the matching max).") + .def("dim", &math::BBox::dim, + "Return max - min as a Vec3 of side lengths.") + .def("isInside", &math::BBox::isInside, "p"_a, + "True iff p lies inside this bounding box (half-open interval).") .def("__repr__", [](const math::BBox& b) { std::stringstream ostr; ostr << b; @@ -328,24 +473,39 @@ template void defineBBoxInteger(nb::module_& m, const char* nam using ValueType = typename CoordT::ValueType; nb::class_, math::BaseBBox>(m, name, doc) - .def(nb::init<>()) - .def(nb::init(), "min"_a, "max"_a) + .def(nb::init<>(), + "Construct an empty CoordBBox (min > max sentinel).") + .def(nb::init(), "min"_a, "max"_a, + "Construct from explicit min and max Coord corners (inclusive).") .def( "__iter__", [](const math::BBox& b) { return nb::make_iterator(nb::type>(), "CoordBBoxIterator", b.begin(), b.end()); }, - nb::keep_alive<0, 1>()) - .def_static("createCube", nb::overload_cast(&math::BBox::createCube), "min"_a, "dim"_a) - .def_static("createCube", nb::overload_cast(&math::BBox::createCube), "min"_a, "max"_a) - .def("is_divisible", &math::BBox::is_divisible) - .def("empty", &math::BBox::empty) - .def("dim", &math::BBox::dim) - .def("volume", &math::BBox::volume) - .def("isInside", nb::overload_cast(&math::BBox::isInside, nb::const_), "p"_a) - .def("isInside", nb::overload_cast&>(&math::BBox::isInside, nb::const_), "b"_a) - .def("asFloat", &math::BBox::template asReal) - .def("asDouble", &math::BBox::template asReal) - .def("hasOverlap", &math::BBox::hasOverlap, "b"_a) - .def("expandBy", &math::BBox::expandBy, "padding"_a) + nb::keep_alive<0, 1>(), + "Iterate over every Coord in this CoordBBox in row-major order.") + .def_static("createCube", nb::overload_cast(&math::BBox::createCube), "min"_a, "dim"_a, + "Construct a cube of side dim voxels anchored at the min Coord.") + .def_static("createCube", nb::overload_cast(&math::BBox::createCube), "min"_a, "max"_a, + "Construct a cube spanning [min, max] in every axis.") + .def("is_divisible", &math::BBox::is_divisible, + "True iff this CoordBBox has more than one voxel in every axis.") + .def("empty", &math::BBox::empty, + "True iff this CoordBBox is empty (any min component > the matching max).") + .def("dim", &math::BBox::dim, + "Return max - min + 1 as a Coord of side lengths (inclusive).") + .def("volume", &math::BBox::volume, + "Total number of voxels enclosed by this CoordBBox.") + .def("isInside", nb::overload_cast(&math::BBox::isInside, nb::const_), "p"_a, + "True iff the integer point p lies inside this CoordBBox.") + .def("isInside", nb::overload_cast&>(&math::BBox::isInside, nb::const_), "b"_a, + "True iff b lies entirely inside this CoordBBox.") + .def("asFloat", &math::BBox::template asReal, + "Convert this CoordBBox to a floating-point BBox of Vec3f.") + .def("asDouble", &math::BBox::template asReal, + "Convert this CoordBBox to a floating-point BBox of Vec3d.") + .def("hasOverlap", &math::BBox::hasOverlap, "b"_a, + "True iff this CoordBBox shares any voxel with b.") + .def("expandBy", &math::BBox::expandBy, "padding"_a, + "Grow this CoordBBox by the given padding in every direction.") .def("__repr__", [](const CoordBBox& b) { std::stringstream ostr; ostr << b; diff --git a/nanovdb/nanovdb/python/PyPrimitives.cc b/nanovdb/nanovdb/python/PyPrimitives.cc index e2a8f9175f..495df3d515 100644 --- a/nanovdb/nanovdb/python/PyPrimitives.cc +++ b/nanovdb/nanovdb/python/PyPrimitives.cc @@ -337,7 +337,8 @@ template void definePrimitives(nb::module_& m) "name"_a = "sphere_ls", "sMode"_a = tools::StatsMode::Default, "cMode"_a = CheckMode::Default, - "buffer"_a = BufferT()); + "buffer"_a = BufferT(), + "Narrow-band level set of a sphere of the given radius and center."); m.def("createLevelSetTorus", nb::overload_cast void definePrimitives(nb::module_& m) "name"_a = "torus_ls", "sMode"_a = tools::StatsMode::Default, "cMode"_a = CheckMode::Default, - "buffer"_a = BufferT()); + "buffer"_a = BufferT(), + "Narrow-band level set of a torus with the given major and minor radii."); m.def("createFogVolumeSphere", nb::overload_cast( @@ -375,7 +377,8 @@ template void definePrimitives(nb::module_& m) "name"_a = "sphere_fog", "sMode"_a = tools::StatsMode::Default, "cMode"_a = CheckMode::Default, - "buffer"_a = BufferT()); + "buffer"_a = BufferT(), + "Sparse fog volume of a sphere of the given radius and center."); m.def("createFogVolumeTorus", nb::overload_cast void definePrimitives(nb::module_& m) "name"_a = "torus_fog", "sMode"_a = tools::StatsMode::Default, "cMode"_a = CheckMode::Default, - "buffer"_a = BufferT()); + "buffer"_a = BufferT(), + "Sparse fog volume of a torus with the given major and minor radii."); - // ---------- Level-set / fog-volume primitives added in Phase 5a ---------- + // ---------- Level-set / fog-volume box / bbox / octahedron primitives ---- m.def("createLevelSetBox", &createLevelSetBox, "gridType"_a = GridType::Float, "width"_a = 40.0, @@ -479,7 +483,7 @@ template void definePrimitives(nb::module_& m) "buffer"_a = BufferT(), "Sparse fog volume of an octahedron."); - // ---------- Point primitives added in Phase 5a ---------- + // ---------- Point primitives ---------- m.def("createPointSphere", &createPointSphere, "pointsPerVoxel"_a = 1, "radius"_a = 100.0, diff --git a/nanovdb/nanovdb/python/PySampleFromVoxels.cc b/nanovdb/nanovdb/python/PySampleFromVoxels.cc index f7f114c15d..297c5f28c2 100644 --- a/nanovdb/nanovdb/python/PySampleFromVoxels.cc +++ b/nanovdb/nanovdb/python/PySampleFromVoxels.cc @@ -16,19 +16,26 @@ namespace { template void defineSampleFromVoxels(nb::module_& m, const char* name) { using CoordT = typename TreeT::CoordType; - nb::class_>(m, name) + nb::class_>(m, name, + "Callable sampler that reconstructs a grid value at an arbitrary " + "index-space position. Build via the matching create*Sampler() factory.") .def( - "__call__", [](const math::SampleFromVoxels& sampler, const CoordT& ijk) { return sampler(ijk); }, nb::is_operator(), "ijk"_a) + "__call__", [](const math::SampleFromVoxels& sampler, const CoordT& ijk) { return sampler(ijk); }, nb::is_operator(), "ijk"_a, + "Sample the grid at integer voxel coordinate ijk.") .def( - "__call__", [](const math::SampleFromVoxels& sampler, const Vec3f& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a) + "__call__", [](const math::SampleFromVoxels& sampler, const Vec3f& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a, + "Sample the grid at fractional index-space position xyz.") .def( - "__call__", [](const math::SampleFromVoxels& sampler, const Vec3d& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a); + "__call__", [](const math::SampleFromVoxels& sampler, const Vec3d& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a, + "Sample the grid at fractional index-space position xyz (double)."); } template void defineCreateSampler(nb::module_& m, const char* name) { m.def( - name, [](const Grid& grid) { return math::createSampler(grid.tree()); }, "grid"_a); + name, [](const Grid& grid) { return math::createSampler(grid.tree()); }, "grid"_a, + "Build a sampler of the matching order that reads values from the " + "given grid's tree."); } } // namespace diff --git a/nanovdb/nanovdb/python/PyTree.cc b/nanovdb/nanovdb/python/PyTree.cc index e2fc6c515d..64eb7533b7 100644 --- a/nanovdb/nanovdb/python/PyTree.cc +++ b/nanovdb/nanovdb/python/PyTree.cc @@ -50,11 +50,13 @@ void defineNodeManagerHandle(nb::module_& m) "Owns the memory backing a NodeManager. Move-only. " "Obtain via nanovdb.createNodeManager(grid).") .def("size", - [](const HandleT& h) { return h.size(); }) + [](const HandleT& h) { return h.size(); }, + "Byte size of the buffer backing this NodeManagerHandle.") .def( "__bool__", [](const HandleT& h) { return h.data() != nullptr; }, - nb::is_operator()) + nb::is_operator(), + "True iff this handle owns a non-empty buffer.") .def("mgr", &pyNodeMgr, nb::keep_alive<0, 1>(), "Return the typed NodeManager for the grid this handle was " diff --git a/nanovdb/nanovdb/python/PyTree.h b/nanovdb/nanovdb/python/PyTree.h index e8fbad1014..f6029317e2 100644 --- a/nanovdb/nanovdb/python/PyTree.h +++ b/nanovdb/nanovdb/python/PyTree.h @@ -35,15 +35,22 @@ template void defineNanoLeaf(nb::module_& m, const char* name) "Leaf node — 8x8x8 voxels. Inherits stats and bbox from the same " "leaf-data block bound across BuildTs."); - cls.def("origin", &LeafT::origin) - .def("bbox", &LeafT::bbox) - .def("hasBBox", &LeafT::hasBBox) - .def_static("dim", &LeafT::dim) - .def_static("voxelCount", &LeafT::voxelCount) - .def("memUsage", &LeafT::memUsage) + cls.def("origin", &LeafT::origin, + "Index-space origin (minimum corner) of this leaf node.") + .def("bbox", &LeafT::bbox, + "Index-space bounding box of this leaf's active voxels.") + .def("hasBBox", &LeafT::hasBBox, + "True iff this leaf carries a cached active bounding box.") + .def_static("dim", &LeafT::dim, + "Side length of a leaf node in voxels (always 8).") + .def_static("voxelCount", &LeafT::voxelCount, + "Total voxel count in a leaf (always 512).") + .def("memUsage", &LeafT::memUsage, + "Byte size of this leaf node.") .def("isActive", nb::overload_cast(&LeafT::isActive, nb::const_), - nb::arg("ijk")) + nb::arg("ijk"), + "True iff the voxel at index-space ijk is active.") .def("isActive", [](const LeafT& leaf, uint32_t n) { // Underlying mValueMask.isOn(n) is unchecked; release builds @@ -54,7 +61,8 @@ template void defineNanoLeaf(nb::module_& m, const char* name) } return leaf.isActive(n); }, - nb::arg("n")) + nb::arg("n"), + "True iff the n-th voxel (linear index into the 512-element leaf) is active.") .def("getValue", [](const LeafT& leaf, uint32_t offset) { // mValues[offset] is unchecked in C++; guard the Python side. @@ -64,28 +72,39 @@ template void defineNanoLeaf(nb::module_& m, const char* name) } return leaf.getValue(offset); }, - nb::arg("offset")) + nb::arg("offset"), + "Return the value at linear offset (0..511) within this leaf.") .def("getValue", nb::overload_cast(&LeafT::getValue, nb::const_), - nb::arg("ijk")) - .def("getFirstValue", &LeafT::getFirstValue) - .def("getLastValue", &LeafT::getLastValue) - .def("minimum", &LeafT::minimum) - .def("maximum", &LeafT::maximum) - .def("average", &LeafT::average) - .def("stdDeviation", &LeafT::stdDeviation) + nb::arg("ijk"), + "Return the value at index-space ijk; ijk must lie inside this leaf.") + .def("getFirstValue", &LeafT::getFirstValue, + "Value at the leaf's first voxel (linear offset 0).") + .def("getLastValue", &LeafT::getLastValue, + "Value at the leaf's last voxel (linear offset 511).") + .def("minimum", &LeafT::minimum, + "Minimum active value within this leaf.") + .def("maximum", &LeafT::maximum, + "Maximum active value within this leaf.") + .def("average", &LeafT::average, + "Average of active values within this leaf.") + .def("stdDeviation", &LeafT::stdDeviation, + "Standard deviation of active values within this leaf.") // NOTE: variance() omitted — NanoVDB.h line 4388 uses unqualified // Pow2() which fails ADL for non-float ValueTs (ValueIndex / // ValueMask / etc.). Users can compute it as stdDeviation() ** 2. - .def("flags", &LeafT::flags) - .def("valueMask", &LeafT::valueMask, nb::rv_policy::reference_internal) + .def("flags", &LeafT::flags, + "Raw leaf flag bits.") + .def("valueMask", &LeafT::valueMask, nb::rv_policy::reference_internal, + "Reference to the leaf's 512-bit active-value mask.") .def("probeValue", [](const LeafT& leaf, const CoordT& ijk) { ValueT v; bool on = leaf.probeValue(ijk, v); return std::make_tuple(v, on); }, - nb::arg("ijk")); + nb::arg("ijk"), + "Return (value, isActive) for the voxel at index-space ijk."); // Zero-copy 512-element NumPy view of mValues. Only enabled for // BuildTs whose ValueType is a primitive arithmetic type (float, @@ -123,33 +142,48 @@ void defineInternalNodeBase(nb::class_& cls) { using ValueT = typename InternalT::ValueType; using CoordT = typename InternalT::CoordType; - cls.def("origin", &InternalT::origin) - .def("bbox", &InternalT::bbox) - .def_static("dim", &InternalT::dim) - .def_static("memUsage", []() { return InternalT::memUsage(); }) - .def("minimum", &InternalT::minimum) - .def("maximum", &InternalT::maximum) - .def("average", &InternalT::average) - .def("stdDeviation", &InternalT::stdDeviation) + cls.def("origin", &InternalT::origin, + "Index-space origin (minimum corner) of this internal node.") + .def("bbox", &InternalT::bbox, + "Index-space bounding box of this internal node's active voxels.") + .def_static("dim", &InternalT::dim, + "Side length in voxels covered by this internal node.") + .def_static("memUsage", []() { return InternalT::memUsage(); }, + "Byte size of an internal node.") + .def("minimum", &InternalT::minimum, + "Minimum active value within this node's subtree.") + .def("maximum", &InternalT::maximum, + "Maximum active value within this node's subtree.") + .def("average", &InternalT::average, + "Average of active values within this node's subtree.") + .def("stdDeviation", &InternalT::stdDeviation, + "Standard deviation of active values within this node's subtree.") // variance() omitted for parity with the leaf binding; compute as // stdDeviation() ** 2 in Python. - .def("valueMask", &InternalT::valueMask, nb::rv_policy::reference_internal) - .def("childMask", &InternalT::childMask, nb::rv_policy::reference_internal) + .def("valueMask", &InternalT::valueMask, nb::rv_policy::reference_internal, + "Reference to the node's active-tile mask.") + .def("childMask", &InternalT::childMask, nb::rv_policy::reference_internal, + "Reference to the node's child-pointer mask (1 where a child node exists).") .def("getValue", nb::overload_cast(&InternalT::getValue, nb::const_), - nb::arg("ijk")) - .def("getFirstValue", &InternalT::getFirstValue) - .def("getLastValue", &InternalT::getLastValue) + nb::arg("ijk"), + "Return the value at index-space ijk by descending into the subtree.") + .def("getFirstValue", &InternalT::getFirstValue, + "Value at the first (lowest-indexed) tile in this node.") + .def("getLastValue", &InternalT::getLastValue, + "Value at the last (highest-indexed) tile in this node.") .def("isActive", nb::overload_cast(&InternalT::isActive, nb::const_), - nb::arg("ijk")) + nb::arg("ijk"), + "True iff the voxel at index-space ijk is active.") .def("probeValue", [](const InternalT& node, const CoordT& ijk) { ValueT v; bool on = node.probeValue(ijk, v); return std::make_tuple(v, on); }, - nb::arg("ijk")); + nb::arg("ijk"), + "Return (value, isActive) for the voxel at index-space ijk."); } template void defineNanoUpper(nb::module_& m, const char* name) @@ -176,30 +210,43 @@ template void defineNanoRoot(nb::module_& m, const char* name) using CoordT = typename RootT::CoordType; nb::class_(m, name, "Root node — top of the tree, holds the tile table.") - .def("background", &RootT::background, nb::rv_policy::reference_internal) - .def("tileCount", &RootT::tileCount) - .def("getTableSize", &RootT::getTableSize) - .def("isEmpty", &RootT::isEmpty) - .def("bbox", &RootT::bbox, nb::rv_policy::reference_internal) - .def("minimum", &RootT::minimum, nb::rv_policy::reference_internal) - .def("maximum", &RootT::maximum, nb::rv_policy::reference_internal) - .def("average", &RootT::average, nb::rv_policy::reference_internal) - .def("stdDeviation", &RootT::stdDeviation, nb::rv_policy::reference_internal) + .def("background", &RootT::background, nb::rv_policy::reference_internal, + "Background value returned for inactive voxels.") + .def("tileCount", &RootT::tileCount, + "Number of active tiles in the root tile table.") + .def("getTableSize", &RootT::getTableSize, + "Total number of entries in the root tile table.") + .def("isEmpty", &RootT::isEmpty, + "True iff the root has no active tiles.") + .def("bbox", &RootT::bbox, nb::rv_policy::reference_internal, + "Index-space bounding box of every active value in this tree.") + .def("minimum", &RootT::minimum, nb::rv_policy::reference_internal, + "Minimum active value in the tree.") + .def("maximum", &RootT::maximum, nb::rv_policy::reference_internal, + "Maximum active value in the tree.") + .def("average", &RootT::average, nb::rv_policy::reference_internal, + "Average of active values in the tree.") + .def("stdDeviation", &RootT::stdDeviation, nb::rv_policy::reference_internal, + "Standard deviation of active values in the tree.") .def("memUsage", - nb::overload_cast<>(&RootT::memUsage, nb::const_)) + nb::overload_cast<>(&RootT::memUsage, nb::const_), + "Byte size of the root node and its tile table.") .def("getValue", nb::overload_cast(&RootT::getValue, nb::const_), - nb::arg("ijk")) + nb::arg("ijk"), + "Return the value at index-space ijk by walking from the root.") .def("isActive", nb::overload_cast(&RootT::isActive, nb::const_), - nb::arg("ijk")) + nb::arg("ijk"), + "True iff the voxel at index-space ijk is active.") .def("probeValue", [](const RootT& root, const CoordT& ijk) { ValueT v; bool on = root.probeValue(ijk, v); return std::make_tuple(v, on); }, - nb::arg("ijk")); + nb::arg("ijk"), + "Return (value, isActive) for the voxel at index-space ijk."); } // -------------------- NanoTree -------------------- @@ -218,9 +265,12 @@ template void defineNanoTree(nb::module_& m, const char* name) "(node counts, active voxel count, extrema).") .def("root", nb::overload_cast<>(&TreeT::root, nb::const_), - nb::rv_policy::reference_internal) - .def("background", &TreeT::background, nb::rv_policy::reference_internal) - .def("activeVoxelCount", &TreeT::activeVoxelCount) + nb::rv_policy::reference_internal, + "Root node of this tree. Lifetime is anchored to the tree.") + .def("background", &TreeT::background, nb::rv_policy::reference_internal, + "Background value returned for inactive voxels.") + .def("activeVoxelCount", &TreeT::activeVoxelCount, + "Total number of active voxels in this tree.") // activeTileCount(level): valid range is 1..3 (lower / upper / root // tile counts). C++ uses NANOVDB_ASSERT(level > 0 && level <= 3) // which is a no-op in release builds — so guard explicitly. @@ -232,7 +282,8 @@ template void defineNanoTree(nb::module_& m, const char* name) } return tree.activeTileCount(level); }, - nb::arg("level")) + nb::arg("level"), + "Number of active tiles at the given tree level (1=lower, 2=upper, 3=root).") // nodeCount(level): valid range is 0..2 (leaf / lower / upper). // C++ uses NANOVDB_ASSERT(level < 3), again no-op in release. // The lambda's `int level` argument disambiguates the call against @@ -246,20 +297,26 @@ template void defineNanoTree(nb::module_& m, const char* name) } return tree.nodeCount(level); }, - nb::arg("level")) - .def("totalNodeCount", &TreeT::totalNodeCount) - .def_static("memUsage", &TreeT::memUsage) + nb::arg("level"), + "Number of nodes at the given tree level (0=leaf, 1=lower, 2=upper).") + .def("totalNodeCount", &TreeT::totalNodeCount, + "Sum of node counts across every tree level.") + .def_static("memUsage", &TreeT::memUsage, + "Byte size of a single tree structure (header only).") .def("getValue", nb::overload_cast(&TreeT::getValue, nb::const_), - nb::arg("ijk")) - .def("isActive", &TreeT::isActive, nb::arg("ijk")) + nb::arg("ijk"), + "Return the value at index-space ijk by descending from the root.") + .def("isActive", &TreeT::isActive, nb::arg("ijk"), + "True iff the voxel at index-space ijk is active.") .def("probeValue", [](const TreeT& tree, const CoordT& ijk) { ValueT v; bool on = tree.probeValue(ijk, v); return std::make_tuple(v, on); }, - nb::arg("ijk")) + nb::arg("ijk"), + "Return (value, isActive) for the voxel at index-space ijk.") .def("extrema", [](const TreeT& tree) { ValueT mn, mx; @@ -273,10 +330,12 @@ template void defineNanoTree(nb::module_& m, const char* name) "First leaf node in breadth-first order, or None if the tree is empty.") .def("getFirstLower", nb::overload_cast<>(&TreeT::getFirstLower, nb::const_), - nb::rv_policy::reference_internal) + nb::rv_policy::reference_internal, + "First lower internal node in breadth-first order, or None if none exists.") .def("getFirstUpper", nb::overload_cast<>(&TreeT::getFirstUpper, nb::const_), - nb::rv_policy::reference_internal); + nb::rv_policy::reference_internal, + "First upper internal node in breadth-first order, or None if none exists."); } // -------------------- NodeManager -------------------- @@ -295,9 +354,11 @@ template void defineNodeManager(nb::module_& m, const char* nam "internal nodes of a NanoGrid. Construct via " "nanovdb.createNodeManager(grid).") .def("isLinear", - nb::overload_cast<>(&NMT::isLinear, nb::const_)) + nb::overload_cast<>(&NMT::isLinear, nb::const_), + "True iff this NodeManager is laid out as a linear offset table over a breadth-first grid.") .def("memUsage", - nb::overload_cast<>(&NMT::memUsage, nb::const_)) + nb::overload_cast<>(&NMT::memUsage, nb::const_), + "Byte size of this NodeManager.") .def("nodeCount", [](const NMT& nm, int level) -> uint64_t { // Mirror Tree.nodeCount bounds (NodeManager forwards to Tree). @@ -307,10 +368,14 @@ template void defineNodeManager(nb::module_& m, const char* nam } return nm.nodeCount(level); }, - nb::arg("level")) - .def("leafCount", &NMT::leafCount) - .def("lowerCount", &NMT::lowerCount) - .def("upperCount", &NMT::upperCount) + nb::arg("level"), + "Number of nodes at the given tree level (0=leaf, 1=lower, 2=upper).") + .def("leafCount", &NMT::leafCount, + "Number of leaf nodes managed by this NodeManager.") + .def("lowerCount", &NMT::lowerCount, + "Number of lower internal nodes managed by this NodeManager.") + .def("upperCount", &NMT::upperCount, + "Number of upper internal nodes managed by this NodeManager.") // leaf / lower / upper: NANOVDB_ASSERT(i < nodeCount(LEVEL)) in C++ is // no-op in release, so guard explicitly to convert OOB access into a // Python IndexError instead of memory corruption. @@ -322,7 +387,8 @@ template void defineNodeManager(nb::module_& m, const char* nam } return nm.leaf(i); }, - nb::rv_policy::reference_internal, nb::arg("i")) + nb::rv_policy::reference_internal, nb::arg("i"), + "Return the i-th leaf node in breadth-first order.") .def("lower", [](const NMT& nm, uint32_t i) -> const nanovdb::NanoLower& { if (i >= nm.lowerCount()) { @@ -331,7 +397,8 @@ template void defineNodeManager(nb::module_& m, const char* nam } return nm.lower(i); }, - nb::rv_policy::reference_internal, nb::arg("i")) + nb::rv_policy::reference_internal, nb::arg("i"), + "Return the i-th lower internal node in breadth-first order.") .def("upper", [](const NMT& nm, uint32_t i) -> const nanovdb::NanoUpper& { if (i >= nm.upperCount()) { @@ -340,7 +407,8 @@ template void defineNodeManager(nb::module_& m, const char* nam } return nm.upper(i); }, - nb::rv_policy::reference_internal, nb::arg("i")); + nb::rv_policy::reference_internal, nb::arg("i"), + "Return the i-th upper internal node in breadth-first order."); } void defineNodeManagerHandle(nb::module_& m); diff --git a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc index ba1546d05b..1755baf493 100644 --- a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc +++ b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc @@ -168,11 +168,19 @@ static void defineHandle(nb::module_& toolsModule) nb::class_(toolsModule, "VoxelBlockManagerHandle", "Owns the firstLeafID / jumpMap metadata buffers backing a " "VoxelBlockManager. Constructed by nanovdb.tools.buildVoxelBlockManager.") - .def(nb::init<>()) - .def("blockCount", &PyVBMHandle::blockCount) - .def("firstOffset", &PyVBMHandle::firstOffset) - .def("lastOffset", &PyVBMHandle::lastOffset) - .def("reset", &PyVBMHandle::reset) + .def(nb::init<>(), + "Construct an empty VoxelBlockManagerHandle with no buffers.") + .def("blockCount", &PyVBMHandle::blockCount, + "Number of voxel blocks managed by this handle.") + .def("firstOffset", &PyVBMHandle::firstOffset, + "Sequential voxel index of the first active voxel covered " + "by this handle (1 by default when the handle covers the " + "full grid).") + .def("lastOffset", &PyVBMHandle::lastOffset, + "Sequential voxel index of the last active voxel covered " + "by this handle.") + .def("reset", &PyVBMHandle::reset, + "Release this handle's buffers and reset it to the empty state.") .def_prop_ro("log2_block_width", [](const PyVBMHandle& h) { return h.log2BlockWidth; }, "The log2_block_width this handle was built with. The jumpMap " "and decodeBlock outputs derive their shapes from this value.") @@ -454,7 +462,11 @@ static void defineDecode(nb::module_& toolsModule) "BlockWidth/64. first_leaf_id must be in [0, grid.tree().nodeCount(0))."); } -// ----- createOnIndexGrid test-scaffold factory (subset of Phase 5) -------- +// ----- createOnIndexGrid test-scaffold factory ---------------------------- +// +// Narrow source-coverage factory used by the VoxelBlockManager unit tests. +// New code should prefer tools.createNanoGridOnIndex (in PyCreateNanoGrid.cc) +// which accepts a wider source set. template static nb::object tryCreateOnIndexGrid(nb::handle py_grid, diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc index 523e4b5834..e024d63939 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc @@ -13,7 +13,10 @@ namespace pynanovdb { void defineDeviceBuffer(nb::module_& m) { - nb::class_(m, "DeviceBuffer"); + nb::class_(m, "DeviceBuffer", + "CUDA device-side buffer used to back a DeviceGridHandle. Holds a " + "host mirror and a device pointer; deviceUpload / deviceDownload on " + "the handle move bytes between the two."); } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index 9647e9739d..0214474420 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -70,7 +70,9 @@ void defineDeviceGridHandle(nb::module_& m) new (&handle) GridHandle(std::move(buffer)); }, "cpu_t"_a.noconvert(), - "cuda_t"_a.noconvert()) + "cuda_t"_a.noconvert(), + "Construct a DeviceGridHandle that wraps an existing pair of " + "host and device uint32 arrays of equal length.") .def("deviceGrid", &pyDeviceGrid, "n"_a = 0, nb::keep_alive<0, 1>(), "Return the n-th device-resident grid as a typed Grid subclass " @@ -78,9 +80,13 @@ void defineDeviceGridHandle(nb::module_& m) "Python or the device copy has not been uploaded yet. The " "returned grid keeps this handle alive.") .def( - "deviceUpload", [](GridHandle& handle, bool sync) { handle.deviceUpload(nullptr, sync); }, "sync"_a = true) + "deviceUpload", [](GridHandle& handle, bool sync) { handle.deviceUpload(nullptr, sync); }, "sync"_a = true, + "Copy the host-side buffer to the device. If sync is True the " + "call blocks until the transfer completes.") .def( - "deviceDownload", [](GridHandle& handle, bool sync) { handle.deviceDownload(nullptr, sync); }, "sync"_a = true); + "deviceDownload", [](GridHandle& handle, bool sync) { handle.deviceDownload(nullptr, sync); }, "sync"_a = true, + "Copy the device-side buffer back to the host. If sync is True " + "the call blocks until the transfer completes."); // NOTE: defineGridHandleUtilities intentionally NOT called for // DeviceBuffer. Registering nanovdb.splitGrids / nanovdb.mergeGrids as a // second overload taking a DeviceGridHandle list conflicts with the host diff --git a/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu b/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu index 933f5570b6..7ec30e4df3 100644 --- a/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu @@ -50,7 +50,8 @@ template void definePointsToGrid(nb::module_& m, const char* na auto handle = converter.getHandle(points, tensor.shape(0)); return handle; }, - "tensor"_a); + "tensor"_a, + "Rasterize the given (N, 3) int32 device tensor of points into a fresh GridHandle."); } template void definePointsToGrid(nb::module_&, const char*); diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md new file mode 100644 index 0000000000..a635a359c0 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/README.md @@ -0,0 +1,33 @@ +# NanoVDB Python Examples + +Runnable `.py` scripts demonstrating the NanoVDB Python bindings. +Each script is self-contained, builds its own input data, and prints +a small summary to stdout. + +Run any example with: + +```bash +python load_inspect.py +``` + +When working from the source tree, the `nanovdb` module needs to be +on `PYTHONPATH` (e.g. via the build output directory): + +```bash +cd /nanovdb/nanovdb/python +PYTHONPATH=. python /path/to/.py +``` + +## Examples + +| Script | What it shows | +| --- | --- | +| [`load_inspect.py`](load_inspect.py) | Polymorphic `handle.grid(n)` access, `GridMetaData` type-erased introspection, mixed-type handles via `mergeGrids`. | +| [`build_grid.py`](build_grid.py) | Voxel-by-voxel construction with `nanovdb.tools.build.FloatGrid`, the cached `ValueAccessor`, the thread-safe `WriteAccessor`, and `.to_nanovdb()` to bake into a host `GridHandle`. | +| [`bulk_leaf_numpy.py`](bulk_leaf_numpy.py) | Zero-copy `(N_leaves, 512)` NumPy view of every leaf's mValues via `grid.leaf_values()`. Includes a global-stats reduction and an in-place mutation that propagates back into the grid. Requires NumPy. | +| [`quantize.py`](quantize.py) | Quantize a `NanoGrid` through `nanovdb.tools.createNanoGridFp{4,8,16,N}`. Shows fixed-width quantization with dithering and variable-width `FpN` with both `AbsDiff` and `RelDiff` oracles. | +| [`validate.py`](validate.py) | `nanovdb.tools.validateGrid` / `validateGrids`, `checkGrid`, `isValid`, and the `evalChecksum` / `validateChecksum` / `updateChecksum` round-trip. | + +For full API signatures and per-argument docstrings, use Python's +`help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`. +The bindings ship `.pyi` type stubs for IDE / type-checker support. diff --git a/nanovdb/nanovdb/python/examples/build_grid.py b/nanovdb/nanovdb/python/examples/build_grid.py new file mode 100644 index 0000000000..572c49b51e --- /dev/null +++ b/nanovdb/nanovdb/python/examples/build_grid.py @@ -0,0 +1,80 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Construct a NanoVDB grid voxel-by-voxel in pure Python. + +nanovdb.tools.build.Grid is the mutable CPU grid builder +that mirrors nanovdb::tools::build::Grid. This example shows the +three construction loops you'll typically reach for (setValue +directly, the cached ValueAccessor, and the thread-safe +WriteAccessor), then bakes each build grid into a host NanoGrid via +.to_nanovdb() and reads it back through the regular polymorphic +handle.grid() API. + +Run with: python build_grid.py +""" +import nanovdb + + +def fill_with_setValue(): + """Simplest path: setValue() directly on the build grid.""" + g = nanovdb.tools.build.FloatGrid( + background=0.0, name="setValue_demo", + gridClass=nanovdb.GridClass.FogVolume) + # Plant five active voxels along the x axis. + for i in range(5): + g.setValue(nanovdb.math.Coord(i, 0, 0), float(i + 1)) + return g + + +def fill_with_accessor(): + """Cached path: getAccessor() avoids re-walking the tree for + each write when consecutive coordinates share a leaf.""" + g = nanovdb.tools.build.FloatGrid(background=0.0, name="accessor_demo") + acc = g.getAccessor() + # The accessor caches the last leaf / lower / upper node, so a + # burst of writes in a 16^3 neighborhood only walks the tree once. + for x in range(8): + for y in range(8): + for z in range(8): + if x + y + z == 7: + acc.setValue(nanovdb.math.Coord(x, y, z), + float(x * 64 + y * 8 + z)) + return g + + +def fill_with_write_accessor(): + """Thread-safe path: getWriteAccessor() buffers writes into a + private root and merges them into the parent on destruction or + on an explicit .merge() call. Useful when fanning out to multiple + threads — one WriteAccessor per thread, no shared mutable state.""" + g = nanovdb.tools.build.FloatGrid(background=0.0, name="write_accessor_demo") + wa = g.getWriteAccessor() + wa.setValue(nanovdb.math.Coord(50, 50, 50), 9.0) + # Before merge, the parent doesn't see the change yet. + assert g.getValue(nanovdb.math.Coord(50, 50, 50)) == 0.0 + wa.merge() + assert g.getValue(nanovdb.math.Coord(50, 50, 50)) == 9.0 + return g + + +def main(): + for builder in (fill_with_setValue, fill_with_accessor, + fill_with_write_accessor): + g = builder() + print(f"=== {g.getName()} ===") + print(f" nodeCount (leaf, lower, upper) = {g.nodeCount()}") + + # Bake into a host NanoGrid. + handle = g.to_nanovdb(sMode=nanovdb.tools.StatsMode.All) + ng = handle.grid() + print(f" baked NanoGrid: type={ng.gridType()}, " + f"active={ng.activeVoxelCount()}, " + f"worldBBox={ng.worldBBox()}") + + # The build grid is left untouched — we can bake again. + handle2 = g.to_nanovdb() + assert handle2.grid().activeVoxelCount() == ng.activeVoxelCount() + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/bulk_leaf_numpy.py b/nanovdb/nanovdb/python/examples/bulk_leaf_numpy.py new file mode 100644 index 0000000000..f967a2ae4a --- /dev/null +++ b/nanovdb/nanovdb/python/examples/bulk_leaf_numpy.py @@ -0,0 +1,63 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Bulk per-leaf value access as a zero-copy NumPy array. + +grid.leaf_values() is the highest-bandwidth path from NanoVDB into +NumPy. It returns an (N_leaves, 512) view of every leaf's mValues +without copying — modify it, slice it, feed it into a PyTorch tensor, +hash it for cache lookup, whatever you need. + +Run with: python bulk_leaf_numpy.py +""" +import nanovdb + + +def main(): + try: + import numpy as np + except ImportError: + print("This example requires numpy. Install it with: pip install numpy") + return + + # Build a fog volume sphere with stats so the leaves have meaningful + # min/max attached (just for the printing below — not required by + # leaf_values itself). + handle = nanovdb.tools.createFogVolumeSphere( + radius=20.0, name="bulk_demo") + grid = handle.grid() + print(f"Grid: {grid.gridType()}, active voxels = {grid.activeVoxelCount()}, " + f"leaves = {grid.tree().nodeCount(0)}") + + # leaf_values() is the zero-copy view. Modifying it modifies the grid. + bulk = grid.leaf_values() + # np.asarray adds a NumPy wrapper but doesn't copy. + arr = np.asarray(bulk) + print(f"leaf_values: shape={arr.shape}, dtype={arr.dtype}, " + f"backed by grid memory (no copy).") + + # Global statistics across every voxel in every leaf, computed in C. + # 0.0 voxels (background) are excluded by using a mask. + nonzero = arr[arr != 0.0] + print(f" non-background voxels = {nonzero.size}") + print(f" min = {nonzero.min()}, max = {nonzero.max()}, " + f"mean = {nonzero.mean()}") + + # Per-leaf reductions: each row of `arr` is one leaf's 512 voxels. + per_leaf_max = arr.max(axis=1) + print(f" per-leaf max (first 5): {per_leaf_max[:5]}") + + # Zero-copy means writes propagate. Zero out the first leaf's values + # and read one back through the regular accessor to confirm the grid + # actually changed. (The active *mask* is unchanged — we wrote into + # mValues only — so activeVoxelCount() stays the same.) + arr[0] = 0.0 + leaf = grid.tree().getFirstLeaf() + if leaf is not None: + first_value_after = leaf.getFirstValue() + print(f" zeroed first leaf's values in place: " + f"leaf.getFirstValue() = {first_value_after}, " + f"activeVoxelCount unchanged: {grid.activeVoxelCount()}") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/load_inspect.py b/nanovdb/nanovdb/python/examples/load_inspect.py new file mode 100644 index 0000000000..632f78ec33 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/load_inspect.py @@ -0,0 +1,67 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Load a NanoVDB grid and inspect it polymorphically. + +handle.grid(i) returns the correct typed Python subclass for whatever +GridType the grid carries, so a single call site can handle a mixed +bundle of grid types. This example builds two grids of different +value types into one handle, then walks the handle inspecting each +grid via the polymorphic accessor plus GridMetaData (the type-erased +introspector that answers "what's in this buffer?" without knowing +BuildT at compile time). + +Run with: python load_inspect.py +""" +import nanovdb + + +def describe_handle(handle): + print(f"Handle contains {handle.gridCount()} grid(s).") + for i in range(handle.gridCount()): + # GridType / gridSize are cheap to query on the handle itself. + gtype = handle.gridType(i) + gsize = handle.gridSize(i) + print(f" [{i}] type={gtype}, size={gsize} bytes") + + # handle.grid(i) returns the matching Grid subclass + # at runtime — no isinstance dispatch needed at the call site. + # The grid name lives on the grid itself, not the handle. + grid = handle.grid(i) + print(f" name={grid.gridName()!r}, " + f"gridClass={grid.gridClass()}") + print(f" activeVoxelCount={grid.activeVoxelCount()}") + print(f" worldBBox={grid.worldBBox()}") + + # GridMetaData is the type-erased introspector — answers + # "what's in this buffer?" without knowing BuildT. + meta = nanovdb.GridMetaData(grid) + print(f" gridSize={meta.gridSize()}, " + f"isLevelSet={meta.isLevelSet()}, " + f"isFogVolume={meta.isFogVolume()}") + + +def main(): + # Build two grids of different types into one handle so we can + # exercise the polymorphic accessor. + h_float = nanovdb.tools.createLevelSetSphere( + radius=10.0, name="sphere_float") + h_double = nanovdb.tools.createLevelSetSphere( + gridType=nanovdb.GridType.Double, radius=10.0, name="sphere_double") + handle = nanovdb.mergeGrids([h_float, h_double]) + + describe_handle(handle) + + print() + print("Polymorphic dispatch from a runtime GridType:") + for i in range(handle.gridCount()): + grid = handle.grid(i) + # Each typed grid carries a getAccessor() that returns the + # appropriate ReadAccessor — float for FloatGrid, + # double for DoubleGrid, etc. + acc = grid.getAccessor() + v = acc.getValue(nanovdb.math.Coord(0, 0, 0)) + print(f" grid[{i}] accessor.getValue(0,0,0) = {v}") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/quantize.py b/nanovdb/nanovdb/python/examples/quantize.py new file mode 100644 index 0000000000..83935f3b68 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/quantize.py @@ -0,0 +1,59 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Quantize a NanoGrid down to a quantized BuildT. + +nanovdb.tools.createNanoGridFp4 / Fp8 / Fp16 / FpN are the quantized +counterparts of the regular grid types. Fp4 / Fp8 / Fp16 use a fixed +bit-width per voxel. FpN picks the bit-width per leaf so each leaf +hits a user-supplied tolerance (the "oracle"). Smaller bit widths +give smaller files at the cost of approximation error. + +Run with: python quantize.py +""" +import nanovdb + + +def gridSize_in_kb(handle): + return handle.gridSize(0) / 1024 + + +def main(): + # Source: a 50-radius sphere fog volume at full float precision. + src_handle = nanovdb.tools.createFogVolumeSphere(radius=50.0) + src_grid = src_handle.grid() + print(f"Source FloatGrid: {gridSize_in_kb(src_handle):.1f} KB, " + f"active voxels = {src_grid.activeVoxelCount()}") + + # Fixed-width quantization. Each subsequent format roughly halves + # the per-voxel storage cost; dithering optional. + for fn_name, label in [ + ("createNanoGridFp16", "Fp16 (16-bit fixed)"), + ("createNanoGridFp8", "Fp8 (8-bit fixed)"), + ("createNanoGridFp4", "Fp4 (4-bit fixed)"), + ]: + h = getattr(nanovdb.tools, fn_name)(src_grid, ditherOn=True) + print(f" {label}: {gridSize_in_kb(h):.1f} KB, " + f"active voxels = {h.grid().activeVoxelCount()}") + + # Variable-width FpN. The oracle picks the per-leaf bit-width to + # meet a tolerance — AbsDiff for absolute error, RelDiff for + # relative error. -1 tolerance means "uninitialized" so we pass + # an explicit value. + abs_oracle = nanovdb.tools.AbsDiff(0.05) # ±0.05 per voxel + h_fpn_abs = nanovdb.tools.createNanoGridFpN(src_grid, abs_oracle) + print(f" FpN (AbsDiff 0.05): {gridSize_in_kb(h_fpn_abs):.1f} KB") + + rel_oracle = nanovdb.tools.RelDiff(0.1) # 10% relative error + h_fpn_rel = nanovdb.tools.createNanoGridFpN(src_grid, rel_oracle) + print(f" FpN (RelDiff 0.10): {gridSize_in_kb(h_fpn_rel):.1f} KB") + + # The output is a regular NanoGrid — read-only, but exposes + # the standard surface (gridType, activeVoxelCount, accessor.getValue + # returning a decoded float, etc). + fpn = h_fpn_abs.grid() + print(f"\nFpN grid type: {fpn.gridType()}, " + f"voxel at (0,0,0) = {fpn.getAccessor().getValue(nanovdb.math.Coord(0, 0, 0))}") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/validate.py b/nanovdb/nanovdb/python/examples/validate.py new file mode 100644 index 0000000000..92469c5f4e --- /dev/null +++ b/nanovdb/nanovdb/python/examples/validate.py @@ -0,0 +1,61 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Validate a NanoVDB grid and round-trip its checksum. + +nanovdb.tools exposes the grid-quality surface: validateGrid (single +grid), checkGrid -> (ok, error), isValid, plus the checksum helpers +evalChecksum / validateChecksum / updateChecksum. This example walks +through a typical "load and verify" workflow. + +Run with: python validate.py +""" +import nanovdb + + +def main(): + # A well-formed grid: createLevelSetSphere returns a NanoGrid + # with stats and checksum populated to the defaults. + handle = nanovdb.tools.createLevelSetSphere(radius=10.0) + grid = handle.grid() + + # Whole-handle validation. validateGrids returns a single bool. + # The verbose=True flag prints failure details to std::cerr + # (visible from Python via std::cerr -> sys.stderr on most stdlibs). + all_ok = nanovdb.tools.validateGrids( + handle, nanovdb.CheckMode.Default, verbose=False) + print(f"validateGrids(handle, Default) = {all_ok}") + + # Per-grid validation, with a helpful message on failure. + one_ok = nanovdb.tools.validateGrid(handle, 0, nanovdb.CheckMode.Full) + print(f"validateGrid(handle, 0, Full) = {one_ok}") + + # validateGrid returns False (no raise) on out-of-range gridID; + # CheckMode.Disable short-circuits and always returns True. + print(f"validateGrid(handle, 99) = " + f"{nanovdb.tools.validateGrid(handle, 99)} (out of range)") + print(f"validateGrid(handle, 99, Disable) = " + f"{nanovdb.tools.validateGrid(handle, 99, nanovdb.CheckMode.Disable)}" + f" (Disable short-circuit)") + + # tools.checkGrid returns the structural check result plus a + # human-readable error message (empty on success). + ok, msg = nanovdb.tools.checkGrid(grid, nanovdb.CheckMode.Full) + print(f"checkGrid(grid, Full) = ok={ok}, msg={msg!r}") + + # tools.isValid is checkGrid + checksum verification, returning + # one bool. + print(f"isValid(grid, Default) = " + f"{nanovdb.tools.isValid(grid, nanovdb.CheckMode.Default)}") + + # Checksum round-trip. evalChecksum is non-mutating; updateChecksum + # writes back into the grid header. + cs1 = nanovdb.tools.evalChecksum(grid, nanovdb.CheckMode.Full) + nanovdb.tools.updateChecksum(grid, nanovdb.CheckMode.Full) + cs2 = nanovdb.tools.evalChecksum(grid, nanovdb.CheckMode.Full) + print(f"evalChecksum equal after no-op update: {cs1 == cs2}") + print(f"validateChecksum(grid, Full) = " + f"{nanovdb.tools.validateChecksum(grid, nanovdb.CheckMode.Full)}") + + +if __name__ == "__main__": + main() From 88a4f9056971de658e48ee7bec2e71d771bdde45 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Fri, 29 May 2026 04:27:11 +0000 Subject: [PATCH 10/48] nanovdb python: bounds-check GridHandle grid accessors; fix stale docstring MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses review feedback on PR #2219. GridHandle::gridSize(n) and gridType(n) index mMetaData[n] without a bounds check (the C++ API documents "assumed to be less than gridCount()"), so calling handle.gridSize()/gridType()/gridData() with an out-of-range n — including any n on an empty handle — was undefined behaviour reachable straight from Python. Wrap all three in bounds-checked lambdas that raise IndexError. gridData() is guarded too: gridData(n) itself returns nullptr for bad n, but the binding passes gridSize(n) alongside it, and that call is the one that reads mMetaData[n] out of bounds. Also refresh the createOnIndexGrid docstring: it claimed the broader createNanoGrid surface "lands in a later phase", but tools.createNanoGridOnIndex already ships in this PR (PyCreateNanoGrid.cc). Point users at the canonical binding instead. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyGridHandle.h | 30 +++++++++++++++++-- nanovdb/nanovdb/python/PyVoxelBlockManager.cc | 7 +++-- 2 files changed, 31 insertions(+), 6 deletions(-) diff --git a/nanovdb/nanovdb/python/PyGridHandle.h b/nanovdb/nanovdb/python/PyGridHandle.h index 5ffd2ad497..068ecf4551 100644 --- a/nanovdb/nanovdb/python/PyGridHandle.h +++ b/nanovdb/nanovdb/python/PyGridHandle.h @@ -158,14 +158,38 @@ template nb::class_> defineGridHa "GridData alignment (used by the I/O code path).") .def("gridCount", &nanovdb::GridHandle::gridCount, "Number of grids stored in this handle.") - .def("gridSize", &nanovdb::GridHandle::gridSize, nb::arg("n") = 0, + .def( + "gridSize", + [](const nanovdb::GridHandle& handle, uint32_t n) { + // GridHandle::gridSize(n) indexes mMetaData[n] unchecked, so an + // out-of-range n (including any n on an empty handle) is UB. + if (n >= handle.gridCount()) + throw nb::index_error("gridSize: grid index out of range [0, gridCount())."); + return handle.gridSize(n); + }, + nb::arg("n") = 0, "Byte size of the n-th grid (without padding).") - .def("gridType", &nanovdb::GridHandle::gridType, nb::arg("n") = 0, + .def( + "gridType", + [](const nanovdb::GridHandle& handle, uint32_t n) { + // GridHandle::gridType(n) indexes mMetaData[n] unchecked, so an + // out-of-range n (including any n on an empty handle) is UB. + if (n >= handle.gridCount()) + throw nb::index_error("gridType: grid index out of range [0, gridCount())."); + return handle.gridType(n); + }, + nb::arg("n") = 0, "GridType enumerator of the n-th grid (e.g. GridType.Float). " "Cheap to query — does not require materializing the grid.") .def( "gridData", - [](nanovdb::GridHandle& handle, uint32_t n) { return nb::bytes(handle.gridData(n), handle.gridSize(n)); }, + [](nanovdb::GridHandle& handle, uint32_t n) { + // gridData(n) returns nullptr for out-of-range n, but gridSize(n) + // below indexes mMetaData[n] unchecked — guard both here. + if (n >= handle.gridCount()) + throw nb::index_error("gridData: grid index out of range [0, gridCount())."); + return nb::bytes(handle.gridData(n), handle.gridSize(n)); + }, nb::arg("n") = 0, nb::rv_policy::reference_internal, "Raw byte contents of the n-th grid as a Python bytes object. " diff --git a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc index 1755baf493..ac8e433bc9 100644 --- a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc +++ b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc @@ -518,9 +518,10 @@ static void defineCreateOnIndexGrid(nb::module_& toolsModule) "verbose"_a = 0, "Convert a source grid into a NanoGrid " "(OnIndexGrid). Accepts FloatGrid / DoubleGrid / Int32Grid / " - "Vec3fGrid. Required for constructing inputs to " - "buildVoxelBlockManager. The broader createNanoGrid surface lands in a later phase."); + "Vec3fGrid. This is a narrow helper kept alongside " + "buildVoxelBlockManager; for general index conversion (broader " + "source coverage, blind-data channels) prefer " + "nanovdb.tools.createNanoGridOnIndex."); } void defineVoxelBlockManagerModule(nb::module_& toolsModule) From 7a1e66396d89b896eeb1ffa115d4ad50e2f10208 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Fri, 29 May 2026 11:32:37 +0000 Subject: [PATCH 11/48] nanovdb python: harden mergeGrids for empty input and use gridData(n) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses review feedback on PR #2219. Empty input: mergeGrids([]) — or a sequence of only-empty handles — produced totalGrids == 0 and called BufferT::create(0). For HostBuffer that yields a buffer whose data() is non-null over a zero-byte region, so the GridHandle(buffer&&) ctor then reads a full GridData header out of it (heap-overflow read; in practice an opaque "invalid host buffer" throw). Return an empty handle up front when there's nothing to merge. Per-grid source pointer: copy from h->gridData(n) — the authoritative start pointer that applies mMetaData[n].offset — instead of walking a raw data() pointer advanced by gridSize(n). The two are equivalent for the current tightly-packed layout (offsets are a running sum of grid sizes), but the accessor form doesn't bake in that assumption and drops the manual pointer arithmetic. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyGridHandle.h | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/nanovdb/nanovdb/python/PyGridHandle.h b/nanovdb/nanovdb/python/PyGridHandle.h index 068ecf4551..625e8031e4 100644 --- a/nanovdb/nanovdb/python/PyGridHandle.h +++ b/nanovdb/nanovdb/python/PyGridHandle.h @@ -98,18 +98,26 @@ template void defineGridHandleUtilities(nb::module_& m) } } + // Nothing to merge (empty sequence, or only empty handles): return an + // empty handle. BufferT::create(0) is ill-defined — for HostBuffer it + // yields a non-null data() over a zero-byte region, and the + // GridHandle(buffer) ctor would then read a full GridData header out of + // it (heap overflow / "invalid host buffer" throw). + if (totalGrids == 0) return HandleT(); + auto buffer = BufferT::create(totalSize); uint8_t* dst = static_cast(buffer.data()); uint32_t writeIndex = 0; for (const HandleT* h : sources) { - const uint8_t* src = static_cast(h->data()); for (uint32_t n = 0; n < h->gridCount(); ++n) { + // gridData(n) is the authoritative per-grid start pointer (it + // applies mMetaData[n].offset), so we don't assume the source + // grids are laid out contiguously in the buffer. const uint64_t gs = h->gridSize(n); - std::memcpy(dst, src, gs); + std::memcpy(dst, h->gridData(n), gs); auto* gd = reinterpret_cast(dst); nanovdb::tools::updateGridCount(gd, writeIndex++, totalGrids); dst += gs; - src += gs; } } return HandleT(std::move(buffer)); From 4d923768fa85abcff5137fae4908695173acb42e Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Fri, 29 May 2026 11:52:07 +0000 Subject: [PATCH 12/48] nanovdb python: release the GIL around VoxelBlockManager compute kernels Addresses review feedback on PR #2219. buildVoxelBlockManager, decodeBlock, and decodeInverseMaps all run non-trivial C++ (the build may parallelize internally via util::forEach; the decode sweeps multiple leaves) while holding the GIL, blocking unrelated Python threads. Release the GIL around the pure-C++ kernels only, matching the established pattern in PyCreateNanoGrid.cc (tryIndexify / tryQuantizeFpX): - pyDecodeInverseMapsImpl (shared by decodeBlock and the free decodeInverseMaps): scope a gil_scoped_release around the VBM::decodeInverseMaps call. The surrounding grid cast, bounds checks (which throw Python exceptions), and the NumPy array / capsule / tuple construction stay under the GIL. - buildVoxelBlockManager: scope a gil_scoped_release around the buildVoxelBlockManager call only. Deliberately not a blanket nb::call_guard() as suggested: these entry points take an nb::handle and call castOnIndexGrid (isinstance/cast) plus allocate NumPy arrays inside the function body, so dropping the GIL across the whole call would touch the Python C API without the GIL. The scoped release covers exactly the GIL-free compute. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyVoxelBlockManager.cc | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc index ac8e433bc9..5e61913050 100644 --- a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc +++ b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc @@ -126,9 +126,16 @@ static nb::object pyDecodeInverseMapsImpl(const NanoGrid& grid, std::unique_ptr voxelOffset(new uint16_t[BlockWidth]); using VBM = VoxelBlockManager; - VBM::template decodeInverseMaps( - &grid, firstLeafID, jumpMap, blockFirstOffset, - leafIndex.get(), voxelOffset.get()); + { + // Release the GIL around the pure-C++ decode kernel — the heavy part, + // and the only part of this helper that touches no Python objects. The + // GIL is re-acquired on scope exit (including during exception unwind) + // before the capsules / ndarrays below are constructed. + nb::gil_scoped_release release; + VBM::template decodeInverseMaps( + &grid, firstLeafID, jumpMap, blockFirstOffset, + leafIndex.get(), voxelOffset.get()); + } // nb::capsule wraps the raw pointer + matching delete[] so it can serve // as the ndarray's owner — the capsule lives as long as the ndarray and @@ -388,8 +395,13 @@ static void defineBuild(nb::module_& toolsModule) std::move(firstLeafIDBuf), std::move(jumpMapBuf), n_blocks, first_offset, last_offset); // In-place builder zeros the jumpMap itself and only - // touches firstLeafID slots it actually visits. - buildVoxelBlockManager(grid, handle); + // touches firstLeafID slots it actually visits. Release the + // GIL around it — it's pure C++ (touches no Python objects) + // and may parallelize internally via util::forEach. + { + nb::gil_scoped_release release; + buildVoxelBlockManager(grid, handle); + } return PyVBMHandle(std::move(handle), LBW); }); }, From 80aa15966e1f18c8987acd7f294f511429939d7f Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 02:31:51 +0000 Subject: [PATCH 13/48] =?UTF-8?q?nanovdb=20python:=20GPU=20epoch=20Phase?= =?UTF-8?q?=20A=20=E2=80=94=20nanovdb.cuda=20namespace=20+=20buffer/handle?= =?UTF-8?q?=20seam?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First step of the GPU/device bindings epoch (issue #2208 follow-on). - Create a real root `nanovdb.cuda` submodule (mirrors C++ nanovdb::cuda) and MOVE DeviceBuffer + DeviceGridHandle into it. Breaking, pre-1.0 relocation: `nanovdb.DeviceBuffer` -> `nanovdb.cuda.DeviceBuffer`, `nanovdb.DeviceGridHandle` -> `nanovdb.cuda.DeviceGridHandle`. `nanovdb.tools.cuda` is untouched. No in-repo call site named the moved classes (handles come from io.deviceReadGrid / tools.cuda.create*), so only a migration note in python/examples/README.md was needed. - Add a BufferT-templated seam in PyDeviceBuffer.h: defineDeviceBufferLike () + addDeviceInterop() keyed only on the duck-typed buffer surface (size()/data()/deviceData()), so a future resource-backed buffer or UnifiedBuffer registers via one call. Phase A binds only the additive size() accessor; CAI/DLPack/pointers/streams land in Phase B. - Expose nanovdb.cuda.compile_options(*extra) via __init__.py, hasattr- guarded so non-CUDA builds still import. Compatible with nanobind >= 2.5.0 (no newer-only APIs). Verified by per-TU g++/nvcc compile checks of the changed TUs; runtime import is exercised by the shared build.yml pytest target. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/NanoVDBModule.cc | 6 ++-- nanovdb/nanovdb/python/__init__.py | 23 ++++++++++++++ nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc | 5 +--- nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h | 30 +++++++++++++++++++ nanovdb/nanovdb/python/examples/README.md | 13 ++++++++ 5 files changed, 71 insertions(+), 6 deletions(-) diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 65fc861b26..b1216b17f8 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -1024,8 +1024,10 @@ NB_MODULE(nanovdb, m) defineHostGridHandle(m); #ifdef NANOVDB_USE_CUDA - defineDeviceBuffer(m); - defineDeviceGridHandle(m); + nb::module_ cudaModule = m.def_submodule("cuda"); + cudaModule.doc() = "CUDA device buffers, the device GridHandle, and device infrastructure (mirrors nanovdb::cuda)."; + defineDeviceBuffer(cudaModule); + defineDeviceGridHandle(cudaModule); #endif nb::module_ toolsModule = m.def_submodule("tools"); diff --git a/nanovdb/nanovdb/python/__init__.py b/nanovdb/nanovdb/python/__init__.py index 9ff10b6c28..2ce52cc294 100644 --- a/nanovdb/nanovdb/python/__init__.py +++ b/nanovdb/nanovdb/python/__init__.py @@ -24,3 +24,26 @@ def get_include(): from .lib.nanovdb import * # noqa: E402,F401,F403 + +# The compiled extension only exposes a `cuda` submodule when built with CUDA +# support. Attach `nanovdb.cuda.compile_options(*extra)` there so downstream +# code can obtain the NanoVDB include flag (plus any extra flags) for runtime +# CUDA compilation. Guarded so `import nanovdb` still succeeds in non-CUDA +# builds where the `cuda` submodule does not exist. +from .lib import nanovdb as _ext # noqa: E402 + +if hasattr(_ext, "cuda"): + + def _cuda_compile_options(*extra): + """Return the NanoVDB include flag followed by any extra flags. + + Suitable for passing to a runtime CUDA compiler (e.g. NVRTC) so kernels + compile against the same NanoVDB headers as the installed wheel:: + + opts = nanovdb.cuda.compile_options("-std=c++17") + """ + return (f"-I{get_include()}",) + tuple(extra) + + _ext.cuda.compile_options = _cuda_compile_options + +del _ext diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc index e024d63939..2fcc51092c 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc @@ -13,10 +13,7 @@ namespace pynanovdb { void defineDeviceBuffer(nb::module_& m) { - nb::class_(m, "DeviceBuffer", - "CUDA device-side buffer used to back a DeviceGridHandle. Holds a " - "host mirror and a device pointer; deviceUpload / deviceDownload on " - "the handle move bytes between the two."); + defineDeviceBufferLike(m, "DeviceBuffer"); } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h index 87a081f638..5dd0d8fada 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h @@ -10,6 +10,36 @@ namespace nb = nanobind; namespace pynanovdb { #ifdef NANOVDB_USE_CUDA + +/// @brief Bind the device-interop surface (CUDA Array Interface / DLPack, raw +/// device/host pointers, streams) onto a device-buffer-like class. +/// +/// @details This is the Phase-B interop hook. It is templated only on the +/// duck-typed buffer surface (size()/data()/deviceData()) so it can +/// be reused for any DeviceBuffer-like type; it must NOT reference +/// DeviceBuffer-specific members. For Phase A this is a minimal seam: +/// it binds only the trivial @c size() accessor. The CUDA Array +/// Interface, DLPack, raw pointer, and stream bindings land in Phase B. +template +void addDeviceInterop(nb::class_& cls) +{ + cls.def("size", &BufferT::size, "Total number of bytes managed by this buffer."); +} + +/// @brief Create an @c nb::class_ for a device-buffer-like type, attach the +/// shared device-interop surface via addDeviceInterop, and return it so +/// callers may chain additional @c .def() bindings. +template +nb::class_ defineDeviceBufferLike(nb::module_& m, const char* name) +{ + nb::class_ cls(m, name, + "CUDA device-side buffer used to back a DeviceGridHandle. Holds a " + "host mirror and a device pointer; deviceUpload / deviceDownload on " + "the handle move bytes between the two."); + addDeviceInterop(cls); + return cls; +} + void defineDeviceBuffer(nb::module_& m); #endif diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index a635a359c0..0e70b784e4 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -31,3 +31,16 @@ PYTHONPATH=. python /path/to/.py For full API signatures and per-argument docstrings, use Python's `help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`. The bindings ship `.pyi` type stubs for IDE / type-checker support. + +## Migration notes + +The CUDA device buffer and device grid handle classes now live under the +`nanovdb.cuda` submodule (mirroring the C++ `nanovdb::cuda` namespace): + +- `nanovdb.DeviceBuffer` → `nanovdb.cuda.DeviceBuffer` +- `nanovdb.DeviceGridHandle` → `nanovdb.cuda.DeviceGridHandle` + +Factory functions that return a device handle are unaffected — they never name +the class — so `nanovdb.io.deviceReadGrid(s)`, `nanovdb.tools.cuda.create*`, +`nanovdb.tools.cuda.pointsToRGBA8Grid`, and the `deviceUpload` / +`deviceDownload` / `deviceGrid` handle methods continue to work unchanged. From ebbd1a7db56b772ac3f523d6c2f56a9e9fc0e625 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 02:54:11 +0000 Subject: [PATCH 14/48] =?UTF-8?q?nanovdb=20python:=20GPU=20epoch=20Phase?= =?UTF-8?q?=20B=20=E2=80=94=20device=20array=20interop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fills the Phase-A addDeviceInterop seam and the device GridHandle with the GPU interop surface: - __cuda_array_interface__ v3 (read-only dict, whole device buffer as 1-D uint8) on DeviceBuffer and DeviceGridHandle. - __dlpack__ / __dlpack_device__ on both: __dlpack__ builds an nb::ndarray view parented to the owner (keep_alive) and returns nanobind's "dltensor" capsule directly — nanobind owns the capsule/deleter/alignment; no hand-built DLManagedTensor. - Raw pointers: DeviceBuffer.device_ptr()/host_ptr(), DeviceGridHandle .device_ptr(), and a generic grid.data_ptr() (host or device per provenance — grid(n) vs deviceGrid(n)). - stream:int=0 threaded through deviceUpload/deviceDownload (was hardcoded nullptr), via the current-device (void*,bool) overload so the targeted device matches deviceData(). - DeviceBuffer.from_external(size, gpu_ptr, cpu_ptr) (non-owning wrap of external host+device memory; mManaged==0 so neither ptr is freed) and DeviceGridHandle.from_buffer(buffer) (moves the buffer; ctor validates the grid header). Device-only array protocols live ONLY on DeviceBuffer/DeviceGridHandle, never the shared NanoGrid class (which also wraps host grids). Two correctness fixes from adversarial review folded in: __dlpack__ now returns the cast capsule directly (calling .attr("__dlpack__") on it would AttributeError, since nb::cast of a no-framework device ndarray already IS the capsule); deviceUpload/deviceDownload use the current-device overload instead of hardcoding device 0. Known limitation: from_external requires BOTH a host and device pointer because nanovdb::cuda::DeviceBuffer's external ctor asserts both non-null; wrapping a device-only CuPy pointer would need a device-only external ctor in the core (follow-up / resource-backed buffer). Compatible with nanobind >= 2.5.0. Verified by per-TU g++/nvcc compile checks. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/NanoVDBModule.cc | 12 ++- nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc | 35 ++++++- nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h | 84 +++++++++++++++- .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 96 +++++++++++++++++-- 4 files changed, 216 insertions(+), 11 deletions(-) diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index b1216b17f8..2bdae5bb81 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -12,6 +12,7 @@ #include #endif +#include #include #include "cuda/PyDeviceBuffer.h" @@ -420,7 +421,16 @@ template void defineNanoGrid(nb::module_& m, const char* name) nb::overload_cast<>(&NanoGrid::tree, nb::const_), nb::rv_policy::reference_internal, "Return the tree associated with this grid. Lifetime is " - "anchored to the grid (and therefore to the GridHandle)."); + "anchored to the grid (and therefore to the GridHandle).") + .def("data_ptr", + [](const NanoGrid& grid) { + return reinterpret_cast(&grid); + }, + "Raw pointer to this grid as a Python int. The address is a HOST " + "pointer when the grid came from handle.grid(n) and a DEVICE " + "pointer when it came from handle.deviceGrid(n) — provenance is " + "the caller's responsibility, the grid object itself cannot tell " + "host from device."); // Add leaf_values() only for BuildTs whose LeafData carries T mValues[512]. PyLeafValuesBinder::apply(cls); } diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc index 2fcc51092c..d5533cb436 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc @@ -4,16 +4,49 @@ #include "PyDeviceBuffer.h" +#include + #include namespace nb = nanobind; +using namespace nb::literals; using namespace nanovdb; namespace pynanovdb { void defineDeviceBuffer(nb::module_& m) { - defineDeviceBufferLike(m, "DeviceBuffer"); + using BufferT = nanovdb::cuda::DeviceBuffer; + defineDeviceBufferLike(m, "DeviceBuffer") + .def_static( + "from_external", + [](uint64_t size, uintptr_t gpu_ptr, uintptr_t cpu_ptr) { + // Wrap externally-managed host + device memory in a NON-OWNING + // DeviceBuffer (mManaged == 0). The buffer will NOT free either + // pointer on destruction, upload, or download — the caller + // retains ownership of both allocations. + if (gpu_ptr == 0) + throw nb::value_error( + "from_external: gpu_ptr must be a non-null device pointer."); + if (cpu_ptr == 0) + throw nb::value_error( + "from_external: cpu_ptr must be a non-null host pointer; the " + "externally-managed DeviceBuffer constructor requires both a " + "host and a device pointer."); + return BufferT::create(size, + reinterpret_cast(cpu_ptr), + reinterpret_cast(gpu_ptr)); + }, + "size"_a, + "gpu_ptr"_a, + "cpu_ptr"_a, + "Wrap externally-managed host and device memory in a NON-OWNING " + "DeviceBuffer. size is the byte size of both allocations; gpu_ptr " + "and cpu_ptr are raw pointers (Python ints). The returned buffer " + "does NOT take ownership: it will never free either pointer, so " + "the caller must keep both allocations alive for the buffer's " + "lifetime. The device pointer is associated with the current CUDA " + "device."); } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h index 5dd0d8fada..d8cec8de14 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h @@ -5,6 +5,14 @@ #include +#ifdef NANOVDB_USE_CUDA +#include + +#include + +#include +#endif + namespace nb = nanobind; namespace pynanovdb { @@ -17,13 +25,83 @@ namespace pynanovdb { /// @details This is the Phase-B interop hook. It is templated only on the /// duck-typed buffer surface (size()/data()/deviceData()) so it can /// be reused for any DeviceBuffer-like type; it must NOT reference -/// DeviceBuffer-specific members. For Phase A this is a minimal seam: -/// it binds only the trivial @c size() accessor. The CUDA Array -/// Interface, DLPack, raw pointer, and stream bindings land in Phase B. +/// DeviceBuffer-specific members. It exposes the whole device buffer +/// as 1-D bytes through the CUDA Array Interface and DLPack so it can +/// be consumed zero-copy by CuPy / PyTorch / Numba, plus the raw +/// device/host pointers as Python ints. template void addDeviceInterop(nb::class_& cls) { cls.def("size", &BufferT::size, "Total number of bytes managed by this buffer."); + + cls.def( + "device_ptr", + [](const BufferT& buf) { + return reinterpret_cast(buf.deviceData()); + }, + "Raw device pointer to the current device's buffer as a Python int " + "(0 if no device allocation exists yet)."); + + cls.def( + "host_ptr", + [](const BufferT& buf) { + return reinterpret_cast(buf.data()); + }, + "Raw host pointer to the buffer's host mirror as a Python int " + "(0 if no host allocation exists)."); + + cls.def_prop_ro( + "__cuda_array_interface__", + [](const BufferT& buf) { + // Hand-rolled CUDA Array Interface (version 3) describing the whole + // device buffer as a 1-D contiguous uint8 array. stream=1 selects + // the legacy default stream per the CAI v3 spec. + nb::dict iface; + iface["shape"] = nb::make_tuple(buf.size()); + iface["typestr"] = "|u1"; + iface["data"] = + nb::make_tuple(reinterpret_cast(buf.deviceData()), false); + iface["version"] = 3; + iface["strides"] = nb::none(); + iface["stream"] = 1; + return iface; + }, + "CUDA Array Interface (v3) view of the whole device buffer as 1-D " + "uint8. Lets CuPy / Numba / PyTorch consume the device bytes " + "zero-copy. Returns a null data pointer until the buffer is populated " + "on the device."); + + cls.def( + "__dlpack_device__", + [](const BufferT&) { + int device = 0; + cudaGetDevice(&device); + // 2 == kDLCUDA. + return nb::make_tuple(2, device); + }, + "DLPack device tuple (kDLCUDA, device_id) for the device buffer."); + + cls.def( + "__dlpack__", + [](nb::handle self, nb::handle /*stream*/) { + const BufferT& buf = nb::cast(self); + size_t shape[1] = {static_cast(buf.size())}; + // Delegate the capsule construction to nanobind: build a device + // ndarray view parented to this buffer (keep_alive via owner) and + // forward to its own __dlpack__ producer. + nb::ndarray> arr( + buf.deviceData(), 1, shape, self); + // nb::cast of a no-framework device ndarray IS the "dltensor" + // PyCapsule (nanobind ndarray_export), which is exactly what + // __dlpack__ must return — so return it directly (do NOT call + // .attr("__dlpack__") on it; a capsule has no such attribute). + // ndarray_inc_ref + owner=self keep the device memory alive. + return nb::cast(arr, nb::rv_policy::reference); + }, + nb::arg("stream") = nb::none(), + "DLPack capsule exporting the whole device buffer as 1-D uint8. " + "Delegates to a nanobind device ndarray view parented to this " + "buffer."); } /// @brief Create an @c nb::class_ for a device-buffer-like type, attach the diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index 0214474420..515aaa8d9c 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -5,6 +5,10 @@ #include "../PyGridHandle.h" #include +#include + +#include + #include #include @@ -80,13 +84,93 @@ void defineDeviceGridHandle(nb::module_& m) "Python or the device copy has not been uploaded yet. The " "returned grid keeps this handle alive.") .def( - "deviceUpload", [](GridHandle& handle, bool sync) { handle.deviceUpload(nullptr, sync); }, "sync"_a = true, - "Copy the host-side buffer to the device. If sync is True the " - "call blocks until the transfer completes.") + "deviceUpload", + [](GridHandle& handle, uintptr_t stream, bool sync) { + cudaStream_t s = reinterpret_cast(stream); + // Use the current-device overload (void*, bool) — NOT the + // (int device, void*, bool) form — so the targeted device + // matches deviceData()/CAI/__dlpack__ (which use cudaGetDevice). + handle.deviceUpload(reinterpret_cast(s), sync); + }, + "stream"_a = 0, "sync"_a = true, + "Copy the host-side buffer to the device. stream is a raw CUDA " + "stream handle (Python int; 0 = default stream). If sync is True " + "the call blocks until the transfer completes.") + .def( + "deviceDownload", + [](GridHandle& handle, uintptr_t stream, bool sync) { + cudaStream_t s = reinterpret_cast(stream); + // Current-device overload, matching deviceData() (see deviceUpload). + handle.deviceDownload(reinterpret_cast(s), sync); + }, + "stream"_a = 0, "sync"_a = true, + "Copy the device-side buffer back to the host. stream is a raw " + "CUDA stream handle (Python int; 0 = default stream). If sync is " + "True the call blocks until the transfer completes.") + .def( + "device_ptr", + [](GridHandle& handle) { + return reinterpret_cast(handle.buffer().deviceData()); + }, + "Raw device pointer to the base of the whole device buffer as a " + "Python int (0 if the handle has not been uploaded to the device " + "yet).") + .def_prop_ro( + "__cuda_array_interface__", + [](GridHandle& handle) { + // CUDA Array Interface (v3) over the whole device buffer as a + // 1-D contiguous uint8 array. stream=1 selects the legacy + // default stream per the CAI v3 spec. + nb::dict iface; + iface["shape"] = nb::make_tuple(handle.buffer().size()); + iface["typestr"] = "|u1"; + iface["data"] = nb::make_tuple( + reinterpret_cast(handle.buffer().deviceData()), false); + iface["version"] = 3; + iface["strides"] = nb::none(); + iface["stream"] = 1; + return iface; + }, + "CUDA Array Interface (v3) view of the whole device buffer as 1-D " + "uint8 — lets CuPy / Numba / PyTorch consume the serialized grid " + "bytes zero-copy. Returns a null data pointer until deviceUpload.") + .def( + "__dlpack_device__", + [](GridHandle&) { + int device = 0; + cudaGetDevice(&device); + return nb::make_tuple(2, device); // 2 == kDLCUDA + }, + "DLPack device tuple (kDLCUDA, device_id) for the device buffer.") .def( - "deviceDownload", [](GridHandle& handle, bool sync) { handle.deviceDownload(nullptr, sync); }, "sync"_a = true, - "Copy the device-side buffer back to the host. If sync is True " - "the call blocks until the transfer completes."); + "__dlpack__", + [](nb::handle self, nb::handle /*stream*/) { + auto& handle = nb::cast&>(self); + size_t shape[1] = {static_cast(handle.buffer().size())}; + nb::ndarray> arr( + handle.buffer().deviceData(), 1, shape, self); + // nb::cast of a no-framework device ndarray IS the "dltensor" + // capsule (what __dlpack__ must return); return it directly. + return nb::cast(arr, nb::rv_policy::reference); + }, + "stream"_a = nb::none(), + "DLPack capsule exporting the whole device buffer as 1-D uint8, " + "parented to this handle.") + .def_static( + "from_buffer", + [](BufferT& buffer) { + // Consumes (moves from) the buffer; the GridHandle ctor peeks + // the GridData header (host side if present, else a D2H copy of + // the device side) and throws std::runtime_error if it is not a + // valid grid. + return GridHandle(std::move(buffer)); + }, + "buffer"_a, + "Build a DeviceGridHandle that takes ownership of a DeviceBuffer. " + "The buffer is MOVED FROM (left empty), and its first GridData " + "header is validated — a RuntimeError is raised if it does not hold " + "a valid NanoVDB grid. Pair with DeviceBuffer.from_external to wrap " + "externally-managed device/host memory zero-copy."); // NOTE: defineGridHandleUtilities intentionally NOT called for // DeviceBuffer. Registering nanovdb.splitGrids / nanovdb.mergeGrids as a // second overload taking a DeviceGridHandle list conflicts with the host From a6bde5c9a7a1e75637feecb6eba5247fd8c7fcf3 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 03:39:16 +0000 Subject: [PATCH 15/48] =?UTF-8?q?nanovdb=20python:=20GPU=20epoch=20Phase?= =?UTF-8?q?=20C=20=E2=80=94=20device=20NodeManager=20+=20device=20VoxelBlo?= =?UTF-8?q?ckManager?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - nanovdb.cuda.createDeviceNodeManager(device_grid, stream=0) + a DeviceNodeManagerHandle (over NodeManagerHandle): polymorphic over BuildTypes.def; takes a device grid (from DeviceGridHandle.deviceGrid(n)); mgr() returns the typed device NodeManager (kernel-only `this`). New cuda/PyDeviceNodeManager.cu. - nanovdb.tools.cuda.buildVoxelBlockManager(device_onindex_grid, log2_block_width=6, ..., stream=0) + a DeviceVoxelBlockManagerHandle whose firstLeafID() (uint32) / jumpMap() (uint64) are zero-copy DEVICE arrays exposing the Phase-B CAI/DLPack interop. New cuda/PyDeviceVoxelBlockManager.cu. The __device__ decodeInverseMaps is intentionally not bound (kernel-only; use the shipped headers). Four adversarial-review/runtime fixes folded in (all validated on the GPU): 1. Dropped nb::keep_alive<0,1> on the device firstLeafID()/jumpMap() views — the returned no-framework device ndarray is a DLPack capsule (not weak-referenceable), so keep_alive threw "could not create a weak reference"; the ndarray's py_self owner already anchors lifetime. 2. DeviceNodeManagerHandle mgr()/__bool__ now gate on size(), not the host data() pointer (which is null for a device-only NodeManager). 3. Dropped DeviceNodeManagerHandle.deviceUpload/deviceDownload — they null-deref on a device-built NodeManager (no host mirror); the handle is created device-resident, nothing to upload. 4. Device VoxelBlockManager registered on the existing nanovdb.tools.cuda (mirrors nanovdb::tools::cuda), not a bogus new nanovdb.cuda.tools. Compatible with nanobind >= 2.5.0. Verified by per-TU compile + a real incremental build + GPU runtime smoke (createDeviceNodeManager, device VBM build, cupy CAI/DLPack round-trips). Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 2 + nanovdb/nanovdb/python/NanoVDBModule.cc | 6 + nanovdb/nanovdb/python/PyTools.cc | 4 + nanovdb/nanovdb/python/PyTree.h | 5 + nanovdb/nanovdb/python/PyVoxelBlockManager.h | 7 + .../python/cuda/PyDeviceNodeManager.cu | 172 +++++++++ .../python/cuda/PyDeviceVoxelBlockManager.cu | 327 ++++++++++++++++++ 7 files changed, 523 insertions(+) create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 9be44ce852..0650cce2ca 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -33,6 +33,8 @@ nanobind_add_module(nanovdb_python NB_STATIC PyVoxelBlockManager.cc cuda/PyDeviceBuffer.cc cuda/PyDeviceGridHandle.cu + cuda/PyDeviceNodeManager.cu + cuda/PyDeviceVoxelBlockManager.cu cuda/PyPointsToGrid.cu cuda/PySampleFromVoxels.cu cuda/PySignedFloodFill.cu diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 2bdae5bb81..2a10699420 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -1038,6 +1038,12 @@ NB_MODULE(nanovdb, m) cudaModule.doc() = "CUDA device buffers, the device GridHandle, and device infrastructure (mirrors nanovdb::cuda)."; defineDeviceBuffer(cudaModule); defineDeviceGridHandle(cudaModule); + // Device NodeManager (DeviceNodeManagerHandle + createDeviceNodeManager) + // on nanovdb.cuda, alongside the device GridHandle. + defineDeviceNodeManager(cudaModule); + // Device VoxelBlockManager (mirrors nanovdb::tools::cuda) is registered on + // the existing nanovdb.tools.cuda submodule in PyTools.cc — NOT here — so + // the Python layout matches the C++ namespaces. #endif nb::module_ toolsModule = m.def_submodule("tools"); diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 9aa8f93452..64d0672bf3 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -14,6 +14,7 @@ #include "PyGridChecksum.h" #include "PyGridValidator.h" #include "PyNanoToOpenVDB.h" +#include "PyVoxelBlockManager.h" // for defineDeviceVoxelBlockManager (CUDA) #ifdef NANOVDB_USE_CUDA #include "cuda/PyPointsToGrid.h" #include "cuda/PySampleFromVoxels.h" @@ -64,6 +65,9 @@ void defineToolsModule(nb::module_& m) defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); + + // Device VoxelBlockManager (nanovdb::tools::cuda) on nanovdb.tools.cuda. + defineDeviceVoxelBlockManager(cudaModule); #endif } diff --git a/nanovdb/nanovdb/python/PyTree.h b/nanovdb/nanovdb/python/PyTree.h index f6029317e2..d695ac85a2 100644 --- a/nanovdb/nanovdb/python/PyTree.h +++ b/nanovdb/nanovdb/python/PyTree.h @@ -413,6 +413,11 @@ template void defineNodeManager(nb::module_& m, const char* nam void defineNodeManagerHandle(nb::module_& m); void defineCreateNodeManager(nb::module_& m); +#ifdef NANOVDB_USE_CUDA +// Device-side NodeManagerHandle + createDeviceNodeManager, registered on the +// nanovdb.cuda submodule (defined in cuda/PyDeviceNodeManager.cu). +void defineDeviceNodeManager(nb::module_& m); +#endif // -------------------- grid.leaf_values() bulk extractor -------------------- // diff --git a/nanovdb/nanovdb/python/PyVoxelBlockManager.h b/nanovdb/nanovdb/python/PyVoxelBlockManager.h index 6fe29570c6..fa79909370 100644 --- a/nanovdb/nanovdb/python/PyVoxelBlockManager.h +++ b/nanovdb/nanovdb/python/PyVoxelBlockManager.h @@ -15,6 +15,13 @@ namespace pynanovdb { /// Python submodule (expected to be the existing nanovdb.tools). void defineVoxelBlockManagerModule(nb::module_& toolsModule); +#ifdef NANOVDB_USE_CUDA +/// @brief Bind the device VoxelBlockManagerHandle wrapper and the device +/// buildVoxelBlockManager onto the given submodule (expected to be the +/// existing nanovdb.tools.cuda). Defined in cuda/PyDeviceVoxelBlockManager.cu. +void defineDeviceVoxelBlockManager(nb::module_& cudaToolsModule); +#endif + } // namespace pynanovdb #endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu new file mode 100644 index 0000000000..e4e9aeb2a4 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu @@ -0,0 +1,172 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include "../PyTree.h" + +#include + +#include + +#include +#include +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; +using namespace nanovdb; + +namespace pynanovdb { + +// Device-side polymorphic mgr() — same dispatch shape as pyNodeMgr in +// PyTree.cc, but probes the DEVICE-resident NodeManager pointer +// (handle.deviceMgr()) instead of the host one. The per-BuildT +// NanoVDB NodeManager classes registered on the root module are reused +// as-is; nanobind does not distinguish a host vs device `this` pointer — the +// returned object is a NodeManager whose underlying address lives in device +// memory, so its accessors (leaf(i)/lower(i)/upper(i)/...) must only be used +// from CUDA kernels, never dereferenced on the host. +static nb::object pyDeviceNodeMgr(nb::handle py_self) +{ + using BufferT = nanovdb::cuda::DeviceBuffer; + using HandleT = NodeManagerHandle; + auto& handle = nb::cast(py_self); + // cuda::createNodeManager builds a DEVICE-only NodeManager: the buffer's + // host mirror (handle.data()) stays null while the device side is + // populated. Gate on size() (the allocated NodeManagerData byte count), + // NOT data(), so the device handle is observable. + if (handle.size() == 0) return nb::none(); + // The stored gridType is private; deviceMgr() returns NULL on a + // type mismatch, so iterate by BuildT (first non-null wins). The X-macro + // produces one case per bound BuildT. +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto* m = handle.template deviceMgr()) { \ + return nb::cast(m, nb::rv_policy::reference, py_self); \ + } +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + if (auto* m = handle.template deviceMgr()) { \ + return nb::cast(m, nb::rv_policy::reference, py_self); \ + } +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto* m = handle.template deviceMgr()) { \ + return nb::cast(m, nb::rv_policy::reference, py_self); \ + } +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto* m = handle.template deviceMgr()) { \ + return nb::cast(m, nb::rv_policy::reference, py_self); \ + } +#include "../BuildTypes.def" + return nb::none(); +} + +static void defineDeviceNodeManagerHandle(nb::module_& m) +{ + using BufferT = nanovdb::cuda::DeviceBuffer; + using HandleT = NodeManagerHandle; + // Distinct name from the host "NodeManagerHandle" because both live in the + // same nanobind type registry; a clashing name would collide. The device + // variant lives on the nanovdb.cuda submodule. + // + // No deviceUpload/deviceDownload here: cuda::createNodeManager builds the + // NodeManager directly on the device (no host mirror), so there is nothing + // to upload; NodeManagerHandle::deviceUpload would null-deref its host + // NodeManagerData. The handle is created device-resident and ready for + // kernel use. + nb::class_(m, "DeviceNodeManagerHandle", + "Owns the device memory backing a device-resident NodeManager. " + "Move-only. Obtain via nanovdb.cuda.createDeviceNodeManager(device_grid). " + "The NodeManager returned by mgr() is a device pointer: its node " + "accessors must only be used from CUDA kernels, never dereferenced on " + "the host.") + .def("size", + [](const HandleT& h) { return h.size(); }, + "Byte size of the device buffer backing this handle.") + .def( + "__bool__", + [](const HandleT& h) { return h.size() != 0; }, + nb::is_operator(), + "True iff this handle owns a non-empty (device-resident) buffer.") + .def("mgr", &pyDeviceNodeMgr, + nb::keep_alive<0, 1>(), + "Return the typed device NodeManager for the grid this handle was " + "built from, or None if the BuildT is not Python-visible. The " + "returned NodeManager's `this` is a DEVICE pointer — use it only " + "from CUDA kernels. It keeps this handle alive."); +} + +// cuda::createNodeManager has one template instantiation per BuildT. We expose +// a single polymorphic createDeviceNodeManager(device_grid, stream) that picks +// the right one based on the runtime type of `device_grid` (any bound +// NanoGrid whose underlying pointer is a device pointer, e.g. from +// DeviceGridHandle.deviceGrid(n)). The created handle stores a raw pointer back +// to the device grid, so the handle must keep the grid alive. +template +static nb::object tryCreateDeviceNodeManager(nb::handle py_grid, cudaStream_t stream) +{ + using GridT = NanoGrid; + if (!nb::isinstance(py_grid)) { + return nb::object(); // sentinel: "not this BuildT, try next" + } + // &grid is the device pointer (the NanoGrid object wraps a device this). + auto* d_grid = &nb::cast(py_grid); + NodeManagerHandle handle; + { + nb::gil_scoped_release release; + handle = nanovdb::cuda::createNodeManager( + d_grid, nanovdb::cuda::DeviceBuffer(), stream); + } + return nb::cast(std::move(handle)); +} + +static void defineCreateDeviceNodeManager(nb::module_& m) +{ + m.def("createDeviceNodeManager", + [](nb::handle py_grid, uintptr_t stream) -> nb::object { + cudaStream_t s = reinterpret_cast(stream); + // Try every bound BuildT; first matching runtime type wins. +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto obj = tryCreateDeviceNodeManager(py_grid, s); obj.is_valid()) { \ + return obj; \ + } +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + if (auto obj = tryCreateDeviceNodeManager(py_grid, s); obj.is_valid()) { \ + return obj; \ + } +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto obj = tryCreateDeviceNodeManager(py_grid, s); obj.is_valid()) { \ + return obj; \ + } +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + if (auto obj = tryCreateDeviceNodeManager(py_grid, s); obj.is_valid()) { \ + return obj; \ + } +#include "../BuildTypes.def" + throw nb::type_error( + "createDeviceNodeManager: argument is not a NanoVDB device " + "grid of any bound BuildT. Pass a device grid obtained from " + "DeviceGridHandle.deviceGrid(n)."); + }, + "device_grid"_a, "stream"_a = 0, + // The constructed NodeManager stores a raw pointer back to the device + // grid; the handle must therefore keep the grid (and transitively the + // DeviceGridHandle that owns the grid's device buffer) alive. + nb::keep_alive<0, 1>(), + "Build a device-resident NodeManager for the given DEVICE grid, " + "returning a DeviceNodeManagerHandle that owns the underlying device " + "buffer. device_grid MUST be a device grid (from " + "DeviceGridHandle.deviceGrid(n)); passing a host grid is a usage " + "error. stream is a raw CUDA stream handle (Python int; 0 = default " + "stream). The handle's mgr() returns the typed device NodeManager and " + "keeps the source grid alive for as long as it lives."); +} + +void defineDeviceNodeManager(nb::module_& m) +{ + defineDeviceNodeManagerHandle(m); + defineCreateDeviceNodeManager(m); +} + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu new file mode 100644 index 0000000000..0b052e0de7 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu @@ -0,0 +1,327 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include +#include + +#include +#include +#include + +#include + +#include +#include +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; +using namespace nanovdb; +using nanovdb::tools::VoxelBlockManagerBase; +using nanovdb::tools::VoxelBlockManagerHandle; + +namespace pynanovdb { + +// ----------------------- Log2BlockWidth dispatch -------------------------- +// +// Mirrors the host dispatchLog2BlockWidth (PyVoxelBlockManager.cc): turn the +// runtime log2_block_width into one of the four compile-time widths the +// device builder is instantiated for (BlockWidth = 64, 128, 256, 512). +template +static auto dispatchLog2BlockWidth(int log2BlockWidth, F&& fn) +{ + switch (log2BlockWidth) { + case 6: return fn(std::integral_constant{}); + case 7: return fn(std::integral_constant{}); + case 8: return fn(std::integral_constant{}); + case 9: return fn(std::integral_constant{}); + default: + throw nb::value_error( + "VoxelBlockManager: log2_block_width must be 6, 7, 8, or 9 " + "(BlockWidth = 64, 128, 256, or 512). Larger widths are not " + "bound in Python by default."); + } +} + +// PyDeviceVBMHandle wraps the device VoxelBlockManagerHandle and carries the +// log2_block_width it was built with (parallel to the host PyVBMHandle). The +// C++ handle does NOT store log2_block_width itself, so recording it once at +// build time keeps the jumpMap view shape derivable from the handle rather +// than from a caller who could spoof it. +struct PyDeviceVBMHandle +{ + VoxelBlockManagerHandle handle; + int log2BlockWidth = 6; + + PyDeviceVBMHandle() = default; + PyDeviceVBMHandle(VoxelBlockManagerHandle&& h, + int lbw) noexcept + : handle(std::move(h)), log2BlockWidth(lbw) {} + + PyDeviceVBMHandle(const PyDeviceVBMHandle&) = delete; + PyDeviceVBMHandle& operator=(const PyDeviceVBMHandle&) = delete; + PyDeviceVBMHandle(PyDeviceVBMHandle&&) = default; + PyDeviceVBMHandle& operator=(PyDeviceVBMHandle&&) = default; + + uint64_t blockCount() const { return handle.blockCount(); } + uint64_t firstOffset() const { return handle.firstOffset(); } + uint64_t lastOffset() const { return handle.lastOffset(); } + void reset() { handle.reset(); } + int blockWidth() const { return 1 << log2BlockWidth; } + int jumpMapLength() const { return 1 << (log2BlockWidth - 6); } +}; + +// ------------------- OnIndex device grid cast helper ---------------------- + +static NanoGrid* castOnIndexDeviceGrid(nb::handle py_grid, + const char* fn_name) +{ + if (!nb::isinstance>(py_grid)) { + std::string msg(fn_name); + msg += ": device_grid must be a NanoVDB device grid of build type " + "ValueOnIndex (OnIndexGrid), obtained from " + "DeviceGridHandle.deviceGrid(n)"; + throw nb::type_error(msg.c_str()); + } + // The device builder takes a non-const NanoGrid*; the + // returned object's underlying address IS the device pointer. It must NOT + // be dereferenced on the host — it is only passed to device kernels. + return &nb::cast&>(py_grid); +} + +// ------------------- DeviceVoxelBlockManagerHandle binding ----------------- + +static void defineHandle(nb::module_& m) +{ + nb::class_(m, "DeviceVoxelBlockManagerHandle", + "Owns the device-resident firstLeafID / jumpMap metadata buffers " + "backing a device VoxelBlockManager. Constructed by " + "nanovdb.tools.cuda.buildVoxelBlockManager. The firstLeafID / jumpMap " + "buffers are exposed zero-copy to CuPy / PyTorch / Numba via the CUDA " + "Array Interface and DLPack.") + .def("blockCount", &PyDeviceVBMHandle::blockCount, + "Number of voxel blocks managed by this handle.") + .def("firstOffset", &PyDeviceVBMHandle::firstOffset, + "Sequential voxel index of the first active voxel covered " + "by this handle (1 by default when the handle covers the " + "full grid).") + .def("lastOffset", &PyDeviceVBMHandle::lastOffset, + "Sequential voxel index of the last active voxel covered " + "by this handle.") + .def("reset", &PyDeviceVBMHandle::reset, + "Release this handle's device buffers and reset it to the empty state.") + .def_prop_ro("log2_block_width", + [](const PyDeviceVBMHandle& h) { return h.log2BlockWidth; }, + "The log2_block_width this handle was built with. The jumpMap " + "view derives its shape from this value.") + .def_prop_ro("block_width", &PyDeviceVBMHandle::blockWidth, + "BlockWidth = 1 << log2_block_width (64, 128, 256, or 512).") + .def_prop_ro("jump_map_length", &PyDeviceVBMHandle::jumpMapLength, + "JumpMapLength = BlockWidth / 64 (1, 2, 4, or 8).") + .def( + "__bool__", + [](const PyDeviceVBMHandle& h) { return h.blockCount() > 0; }, + nb::is_operator()) + // ------------------- raw device pointers ------------------- + .def( + "first_leaf_id_ptr", + [](PyDeviceVBMHandle& h) { + return reinterpret_cast(h.handle.deviceFirstLeafID()); + }, + "Raw device pointer to the firstLeafID array (uint32 x blockCount) " + "as a Python int (0 if the handle is empty).") + .def( + "jump_map_ptr", + [](PyDeviceVBMHandle& h) { + return reinterpret_cast(h.handle.deviceJumpMap()); + }, + "Raw device pointer to the jumpMap array " + "(uint64 x blockCount x jump_map_length) as a Python int (0 if the " + "handle is empty).") + // ------------------- firstLeafID device view ------------------- + // Zero-copy DEVICE view of the (blockCount,) uint32 firstLeafID array, + // mirroring the host firstLeafID() but using the Phase-B device + // interop (nb::device::cuda ndarray) instead of a host numpy view. + .def( + "firstLeafID", + [](nb::handle py_self) -> nb::object { + auto& h = nb::cast(py_self); + size_t shape[1] = {static_cast(h.blockCount())}; + // A default-constructed / reset() handle has a null + // deviceFirstLeafID(); still return an empty (0,) array so + // callers don't branch on a None sentinel. The dummy non-null + // pointer (the handle itself) keeps nanobind happy; nothing is + // read since the leading shape is 0. + uint32_t* raw = h.handle.deviceFirstLeafID(); + void* data = (raw != nullptr) ? static_cast(raw) + : static_cast(&h); + nb::ndarray> arr( + data, size_t(1), shape, py_self); + return nb::cast(arr, nb::rv_policy::reference); + }, + // No nb::keep_alive<0,1> here: the returned no-framework device + // ndarray is exported as a DLPack capsule (not weak-referenceable), + // so keep_alive would throw "could not create a weak reference". + // Lifetime is already anchored by the ndarray's py_self owner arg. + "Return a zero-copy (blockCount,) uint32 DEVICE array view of the " + "firstLeafID array, consumable by CuPy / PyTorch / Numba. Returns " + "an empty (0,) array on a default-constructed or reset() handle. " + "The view keeps this handle alive.") + // ------------------- jumpMap device view ------------------- + // Zero-copy DEVICE view of the (blockCount, jumpMapLength) uint64 + // jumpMap. jumpMapLength derives from the handle's recorded + // log2_block_width, never the caller, so the view exactly covers the + // allocated buffer. + .def( + "jumpMap", + [](nb::handle py_self) -> nb::object { + auto& h = nb::cast(py_self); + size_t shape[2] = {static_cast(h.blockCount()), + static_cast(h.jumpMapLength())}; + uint64_t* raw = h.handle.deviceJumpMap(); + void* data = (raw != nullptr) ? static_cast(raw) + : static_cast(&h); + nb::ndarray> arr( + data, size_t(2), shape, py_self); + return nb::cast(arr, nb::rv_policy::reference); + }, + // No keep_alive (see firstLeafID): the device-ndarray capsule is + // not weak-referenceable; the py_self owner arg anchors lifetime. + "Return a zero-copy (blockCount, jump_map_length) uint64 DEVICE " + "array view of the jumpMap, consumable by CuPy / PyTorch / Numba. " + "The shape is determined by the log2_block_width the handle was " + "built with. Returns an empty (0, jump_map_length) array on a " + "default-constructed or reset() handle. The view keeps this handle " + "alive.") + // ------------------- CUDA Array Interface (v3) ------------------- + // CAI describing the firstLeafID array as 1-D uint32. (CuPy / Numba + // consume the __cuda_array_interface__ attribute directly; the jumpMap + // is reachable as a 2-D device ndarray via jumpMap() above.) + .def_prop_ro( + "__cuda_array_interface__", + [](PyDeviceVBMHandle& h) { + nb::dict iface; + iface["shape"] = nb::make_tuple(h.blockCount()); + iface["typestr"] = "(h.handle.deviceFirstLeafID()), + false); + iface["version"] = 3; + iface["strides"] = nb::none(); + iface["stream"] = 1; + return iface; + }, + "CUDA Array Interface (v3) view of the firstLeafID array as 1-D " + "uint32 — lets CuPy / Numba / PyTorch consume it zero-copy. The " + "jumpMap is available as a 2-D device array via jumpMap().") + .def( + "__dlpack_device__", + [](const PyDeviceVBMHandle&) { + int device = 0; + cudaGetDevice(&device); + return nb::make_tuple(2, device); // 2 == kDLCUDA + }, + "DLPack device tuple (kDLCUDA, device_id) for the firstLeafID " + "device buffer.") + .def( + "__dlpack__", + [](nb::handle self, nb::handle /*stream*/) { + auto& h = nb::cast(self); + size_t shape[1] = {static_cast(h.blockCount())}; + uint32_t* raw = h.handle.deviceFirstLeafID(); + void* data = (raw != nullptr) ? static_cast(raw) + : static_cast(&h); + // nb::cast of a no-framework device ndarray IS the "dltensor" + // capsule (what __dlpack__ must return); return it directly. + nb::ndarray> arr( + data, size_t(1), shape, self); + return nb::cast(arr, nb::rv_policy::reference); + }, + "stream"_a = nb::none(), + "DLPack capsule exporting the firstLeafID array as 1-D uint32, " + "parented to this handle. The jumpMap is available as a 2-D device " + "array via jumpMap()."); +} + +// ------------------- buildVoxelBlockManager (device) binding --------------- + +static void defineBuild(nb::module_& m) +{ + m.def("buildVoxelBlockManager", + [](nb::handle py_grid, + int log2_block_width, + uint64_t first_offset, + uint64_t last_offset, + uint64_t n_blocks, + uintptr_t stream) -> PyDeviceVBMHandle { + auto* d_grid = castOnIndexDeviceGrid(py_grid, "buildVoxelBlockManager"); + cudaStream_t s = reinterpret_cast(stream); + return dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + constexpr int LBW = decltype(W)::value; + using Base = VoxelBlockManagerBase; + constexpr uint64_t BlockWidth = Base::BlockWidth; + // first_offset, if nonzero, must satisfy first_offset == 1 + // (mod BlockWidth); the C++ builder only NANOVDB_ASSERTs this + // (a no-op in release), so validate it here for a clear error. + if (first_offset != 0 && + ((first_offset - 1) & (BlockWidth - 1)) != 0) { + throw nb::value_error( + "buildVoxelBlockManager: first_offset must satisfy " + "first_offset == 1 (mod BlockWidth). Pass 0 (the " + "default) to let the implementation use 1."); + } + VoxelBlockManagerHandle handle; + { + // The device builder reads activeVoxelCount / lowerCount + // from device memory and launches kernels; pure C++/CUDA + // touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + handle = nanovdb::tools::cuda::buildVoxelBlockManager< + LBW, nanovdb::cuda::DeviceBuffer>( + d_grid, first_offset, last_offset, n_blocks, s); + } + return PyDeviceVBMHandle(std::move(handle), LBW); + }); + }, + "device_grid"_a, + "log2_block_width"_a = 6, + "first_offset"_a = 0, + "last_offset"_a = 0, + "n_blocks"_a = 0, + "stream"_a = 0, + // The device builder reads the grid from device memory; keep the + // device grid (and its owning DeviceGridHandle) alive for the duration + // of the build. The returned handle owns its own device metadata + // buffers, so it does NOT need to keep the grid alive afterwards. + "Build a device-side VoxelBlockManager from an OnIndex DEVICE grid. " + "device_grid MUST be a device grid (from " + "DeviceGridHandle.deviceGrid(n)); passing a host grid is a usage " + "error. log2_block_width selects the per-block active-voxel count " + "(6=64, 7=128, 8=256, 9=512). Pass 0 for first_offset / last_offset / " + "n_blocks to use the full grid (first active voxel through " + "activeVoxelCount, minimum block count); these are read from device " + "memory. first_offset, if nonzero, must satisfy first_offset == 1 " + "(mod BlockWidth). stream is a raw CUDA stream handle (Python int; 0 = " + "default stream)."); +} + +// NOTE: decodeInverseMaps is intentionally NOT bound on the device. The device +// decode (VoxelBlockManager::decodeInverseMaps) is a __device__ +// function: it uses threadIdx / __syncthreads / shared-memory output arrays and +// is callable only from within a CUDA kernel, never from host code. Users who +// want device-side decode should call it from their own kernels via the shipped +// header , feeding it the firstLeafID +// / jumpMap device pointers (first_leaf_id_ptr / jump_map_ptr above) and the +// device grid pointer. + +void defineDeviceVoxelBlockManager(nb::module_& m) +{ + defineHandle(m); + defineBuild(m); +} + +} // namespace pynanovdb + +#endif From e479a6bd798bfd37dfc9bb15da93c3c4f807f9da Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 04:01:08 +0000 Subject: [PATCH 16/48] =?UTF-8?q?nanovdb=20python:=20GPU=20epoch=20Phase?= =?UTF-8?q?=20D=20=E2=80=94=20generalize=20pointsToGrid=20+=20stream=20par?= =?UTF-8?q?ams?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Generalize the CUDA points->grid surface beyond Rgba8 (nanovdb.tools.cuda): - Coordinate input (int32 (N,3) voxel coords) -> grid via voxelsToGrid for Rgba8 / ValueOnIndex / ValueIndex (voxelsToRGBA8Grid / voxelsToOnIndexGrid / voxelsToIndexGrid). Legacy pointsToRGBA8Grid kept (routes to voxelsToGrid), so existing callers/tests are unchanged. - World-position input ((N,3) float OR double) -> NanoGrid via a new pointsToGrid (both precisions under one name; nanobind dtype dispatch). - Point is intentionally NOT offered on the coord path: PointsToGrid static_asserts Vec3f/Vec3d coords, so Point grids build from world positions only (documented). - Add stream:int=0 (reinterpret_cast) to every CUDA-launching binding — pointsToGrid/voxelsToGrid, signedFloodFill, both sampleFromVoxels overloads — with nb::gil_scoped_release around each kernel launch. Stream is a trailing default kwarg, so existing call sites are unaffected. - World path constructs PointsToGrid(voxelSize, ...) directly rather than the fixed-voxelSize free function, which is declared-but-undefined in the header (would link-fail). Returns DeviceGridHandle (Phase-A/B device interop). Compatible with nanobind >= 2.5.0. Verified by per-TU compile + real incremental build + GPU runtime smoke (world float32/float64 -> Point grids, coord -> OnIndex/Index/ RGBA8, non-default streams) and the full Python suite (140/140 pass). Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyTools.cc | 15 ++- nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu | 105 ++++++++++++++++-- nanovdb/nanovdb/python/cuda/PyPointsToGrid.h | 9 ++ .../nanovdb/python/cuda/PySampleFromVoxels.cu | 26 ++++- .../nanovdb/python/cuda/PySignedFloodFill.cu | 16 ++- 5 files changed, 156 insertions(+), 15 deletions(-) diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 64d0672bf3..92dd6841ce 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -61,7 +61,20 @@ void defineToolsModule(nb::module_& m) defineSignedFloodFill(cudaModule, "signedFloodFill"); defineSignedFloodFill(cudaModule, "signedFloodFill"); - definePointsToGrid(cudaModule, "pointsToRGBA8Grid"); + // Coordinate-input (index-space int32 (N,3)) -> grid. The legacy Rgba8 + // entry keeps its original Python name; OnIndex/Index get descriptive + // names. (Point is excluded here -- see PyPointsToGrid.cu -- and is built + // from world positions via pointsToGrid below.) + defineVoxelsToGrid(cudaModule, "pointsToRGBA8Grid"); + defineVoxelsToGrid(cudaModule, "voxelsToRGBA8Grid"); + defineVoxelsToGrid(cudaModule, "voxelsToOnIndexGrid"); + defineVoxelsToGrid(cudaModule, "voxelsToIndexGrid"); + + // World-position-input ((N,3) float OR double) -> NanoGrid. Both + // scalar precisions are bound under the same Python name; nanobind picks + // the overload that matches the input tensor dtype. + definePointsToGrid(cudaModule, "pointsToGrid"); + definePointsToGrid(cudaModule, "pointsToGrid"); defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); diff --git a/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu b/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu index 7ec30e4df3..7129822251 100644 --- a/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu @@ -4,6 +4,8 @@ #include +#include + #include namespace nb = nanobind; @@ -11,6 +13,9 @@ using namespace nb::literals; namespace pynanovdb { +// Fancy pointer adapter over an (N, 3) int32 device tensor that dereferences to +// a nanovdb::Coord, i.e. INDEX-space voxel coordinates. Used by the +// voxelsToGrid path (BuildT != Point, plus the Point/index build types). class NdArrayCoordPtr { const int32_t* data; @@ -40,20 +45,106 @@ public: } }; +// Fancy pointer adapter over an (N, 3) float/double device tensor that +// dereferences to a nanovdb::math::Vec3, i.e. WORLD-space point positions. +// Used by the pointsToGrid path which builds NanoGrid. The world +// pointsToGrid requires the dereferenced element to be Vec3f or Vec3d. +template class NdArrayVec3Ptr +{ + const ScalarT* data; + int64_t stride0, stride1; + +public: + using element_type = nanovdb::math::Vec3; + + __hostdev__ NdArrayVec3Ptr(const ScalarT* data, int64_t stride0, int64_t stride1) + : data(data) + , stride0(stride0) + , stride1(stride1) + { + } + __hostdev__ inline element_type operator[](size_t i) const + { + return element_type(data[i * stride0 + 0 * stride1], + data[i * stride0 + 1 * stride1], + data[i * stride0 + 2 * stride1]); + } + __hostdev__ inline element_type operator*() const + { + return element_type(data[0 * stride1], data[1 * stride1], data[2 * stride1]); + } +}; + +template void defineVoxelsToGrid(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::ndarray, nb::c_contig, nb::device::cuda> tensor, + double voxelSize, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + NdArrayCoordPtr points(tensor.data(), tensor.stride(0), tensor.stride(1)); + const size_t count = tensor.shape(0); + // voxelsToGrid only builds the grid topology (no blind data); pure + // CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + return nanovdb::tools::cuda::voxelsToGrid( + points, count, voxelSize, nanovdb::cuda::DeviceBuffer(), s); + }, + "tensor"_a, + "voxelSize"_a = 1.0, + "stream"_a = 0, + "Rasterize the given (N, 3) int32 device tensor of index-space voxel " + "coordinates into a fresh device GridHandle. voxelSize is the world " + "size of a voxel; stream is a raw CUDA stream handle (Python int; " + "0 = default stream)."); +} + template void definePointsToGrid(nb::module_& m, const char* name) { m.def( name, - [](nb::ndarray, nb::c_contig, nb::device::cuda> tensor) { - NdArrayCoordPtr points(tensor.data(), tensor.stride(0), tensor.stride(1)); - nanovdb::tools::cuda::PointsToGrid converter; - auto handle = converter.getHandle(points, tensor.shape(0)); - return handle; + [](nb::ndarray, nb::c_contig, nb::device::cuda> tensor, + double voxelSize, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + NdArrayVec3Ptr points(tensor.data(), tensor.stride(0), tensor.stride(1)); + const size_t count = tensor.shape(0); + // Build a NanoGrid at the requested fixed voxel size, + // encoding the world points as blind data. Mirror the converter + // usage of the legacy Rgba8 path (the fixed-voxelSize free function + // has no out-of-line definition in the header), but set the Point + // build type and the world->index map scale. + nb::gil_scoped_release release; + nanovdb::tools::cuda::PointsToGrid converter( + voxelSize, nanovdb::Vec3d(0.0), s); + converter.setPointType(nanovdb::PointType::Default); + return converter.getHandle(points, count); }, "tensor"_a, - "Rasterize the given (N, 3) int32 device tensor of points into a fresh GridHandle."); + "voxelSize"_a = 1.0, + "stream"_a = 0, + "Rasterize the given (N, 3) float or double device tensor of " + "WORLD-space point positions into a fresh device GridHandle of type " + "NanoGrid, encoding the point coordinates as blind data. " + "voxelSize is the world size of a voxel; stream is a raw CUDA stream " + "handle (Python int; 0 = default stream)."); } -template void definePointsToGrid(nb::module_&, const char*); +// voxelsToGrid (index-space int32 Coord input). The legacy Rgba8 binding maps +// onto defineVoxelsToGrid; additional build types build matching grids. +// +// NOTE: voxelsToGrid is intentionally NOT instantiated. The C++ +// PointsToGrid::countNodes static_asserts that Point coordinates be +// Vec3f or Vec3d (PointsToGrid.cuh:590), so the int32-Coord voxelsToGrid input +// is rejected at compile time for BuildT == Point. Build Point grids from +// world-space float/double positions via definePointsToGrid. +template void defineVoxelsToGrid(nb::module_&, const char*); +template void defineVoxelsToGrid(nb::module_&, const char*); +template void defineVoxelsToGrid(nb::module_&, const char*); + +// pointsToGrid (world-space float/double input -> NanoGrid). +template void definePointsToGrid(nb::module_&, const char*); +template void definePointsToGrid(nb::module_&, const char*); } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyPointsToGrid.h b/nanovdb/nanovdb/python/cuda/PyPointsToGrid.h index 4af07f5657..166ede420f 100644 --- a/nanovdb/nanovdb/python/cuda/PyPointsToGrid.h +++ b/nanovdb/nanovdb/python/cuda/PyPointsToGrid.h @@ -9,6 +9,15 @@ namespace nb = nanobind; namespace pynanovdb { +// Rasterize an (N, 3) int32 device tensor of voxel (index-space) coordinates +// into a fresh device GridHandle of type NanoGrid. Used for the +// legacy pointsToRGBA8Grid binding and the new voxelsTo*Grid bindings. +template void defineVoxelsToGrid(nb::module_& m, const char* name); + +// Rasterize an (N, 3) float OR double device tensor of WORLD-space point +// positions into a fresh device GridHandle of type NanoGrid, encoding +// the point coordinates as blind data. BuildT selects the world coordinate +// scalar type (float or double). template void definePointsToGrid(nb::module_& m, const char* name); } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PySampleFromVoxels.cu b/nanovdb/nanovdb/python/cuda/PySampleFromVoxels.cu index aaebf66772..4805a93d6e 100644 --- a/nanovdb/nanovdb/python/cuda/PySampleFromVoxels.cu +++ b/nanovdb/nanovdb/python/cuda/PySampleFromVoxels.cu @@ -4,6 +4,8 @@ #include +#include + #include #include @@ -61,28 +63,40 @@ template void defineSampleFromVoxels(nb::module_& m, const char name, [](nb::ndarray, nb::c_contig, nb::device::cuda> points, NanoGrid* d_grid, - nb::ndarray, nb::device::cuda> values) { + nb::ndarray, nb::device::cuda> values, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); constexpr unsigned int numThreads = 128; unsigned int numBlocks = (points.shape(0) + numThreads - 1) / numThreads; - sampleFromVoxels<<>>(points.shape(0), points.data(), d_grid, values.data()); + // Raw kernel launch on the supplied stream; touches no Python + // objects, so release the GIL. + nb::gil_scoped_release release; + sampleFromVoxels<<>>(points.shape(0), points.data(), d_grid, values.data()); }, "points"_a, "d_grid"_a, - "values"_a); + "values"_a, + "stream"_a = 0); m.def( name, [](nb::ndarray, nb::c_contig, nb::device::cuda> points, NanoGrid* d_grid, nb::ndarray, nb::device::cuda> values, - nb::ndarray, nb::device::cuda> gradients) { + nb::ndarray, nb::device::cuda> gradients, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); constexpr unsigned int numThreads = 128; unsigned int numBlocks = (points.shape(0) + numThreads - 1) / numThreads; - sampleFromVoxels<<>>(points.shape(0), points.data(), d_grid, values.data(), gradients.data()); + // Raw kernel launch on the supplied stream; touches no Python + // objects, so release the GIL. + nb::gil_scoped_release release; + sampleFromVoxels<<>>(points.shape(0), points.data(), d_grid, values.data(), gradients.data()); }, "points"_a, "d_grid"_a, "values"_a, - "gradients"_a); + "gradients"_a, + "stream"_a = 0); } template void defineSampleFromVoxels(nb::module_&, const char*); diff --git a/nanovdb/nanovdb/python/cuda/PySignedFloodFill.cu b/nanovdb/nanovdb/python/cuda/PySignedFloodFill.cu index ddbd5c67cb..eb6fe7ea8b 100644 --- a/nanovdb/nanovdb/python/cuda/PySignedFloodFill.cu +++ b/nanovdb/nanovdb/python/cuda/PySignedFloodFill.cu @@ -2,6 +2,8 @@ // SPDX-License-Identifier: Apache-2.0 #include "PySignedFloodFill.h" +#include + #include namespace nb = nanobind; @@ -13,7 +15,19 @@ namespace pynanovdb { template void defineSignedFloodFill(nb::module_& m, const char* name) { m.def( - name, [](NanoGrid* d_grid, bool verbose) { return tools::cuda::signedFloodFill(d_grid, verbose); }, "d_grid"_a, "verbose"_a = false); + name, + [](NanoGrid* d_grid, bool verbose, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // signedFloodFill launches kernels and synchronizes the stream; + // pure CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + tools::cuda::signedFloodFill(d_grid, verbose, s); + }, + "d_grid"_a, + "verbose"_a = false, + "stream"_a = 0, + "Perform a signed flood fill on a device float/double grid in place. " + "stream is a raw CUDA stream handle (Python int; 0 = default stream)."); } template void defineSignedFloodFill(nb::module_&, const char*); From bbfaf9c493bd055643ba03f7620acdbb80054374 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 04:30:03 +0000 Subject: [PATCH 17/48] =?UTF-8?q?nanovdb=20python:=20GPU=20epoch=20Phase?= =?UTF-8?q?=20E1=20=E2=80=94=20tools.cuda=20morphology/topology=20ops?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bind 5 of the 6 tools::cuda morphology/topology grid operators into nanovdb.tools.cuda, one Py*.cu + .h per header: - dilateGrid(d_grid, op=26, stream=0) (NN_FACE=6 / NN_FACE_EDGE_VERTEX=26) - coarsenGrid(d_grid, stream=0) (2x topological downsample) - refineGrid(d_grid, stream=0) (2x topological upsample) - pruneGrid(d_grid, leafMask, stream=0) (leafMask: device uint64 array reinterpreted as Mask<3>* per leaf; word count validated as a multiple of 8) - mergeGrids(d_grid1, d_grid2, stream=0) (binary active-mask union; chain for >2) All take a device grid (NanoGrid* from deviceGrid(n)) + a stream:int=0, run getHandle() under nb::gil_scoped_release, and return a DeviceGridHandle (Phase-A/B CAI/DLPack). Instantiated for ValueOnIndex ONLY: every op embeds TopologyBuilder, which static_asserts is_onindex. TopologyBuilder itself is intentionally NOT bound — it is the low-level multi-stage engine (ordered allocate/count/process* calls, no one-shot getHandle), not a user-facing op. Each .cu drops `using namespace nanovdb;` and fully-qualifies nanovdb:: to avoid a CUB DeviceScan vs nanovdb::cuda 'reference to cuda is ambiguous' error (same fix PyPointsToGrid.cu already uses). Compatible with nanobind >= 2.5.0. Verified by per-TU compile + real incremental build + GPU runtime smoke: all 5 ops run on a device OnIndex grid with correct active-voxel deltas (dilate grows, coarsen ~8x shrink, refine ~8x grow; prune + binary merge succeed). Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 5 ++ nanovdb/nanovdb/python/PyTools.cc | 14 +++++ nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu | 39 +++++++++++++ nanovdb/nanovdb/python/cuda/PyCoarsenGrid.h | 16 +++++ nanovdb/nanovdb/python/cuda/PyDilateGrid.cu | 44 ++++++++++++++ nanovdb/nanovdb/python/cuda/PyDilateGrid.h | 16 +++++ nanovdb/nanovdb/python/cuda/PyMergeGrids.cu | 44 ++++++++++++++ nanovdb/nanovdb/python/cuda/PyMergeGrids.h | 16 +++++ nanovdb/nanovdb/python/cuda/PyPruneGrid.cu | 61 ++++++++++++++++++++ nanovdb/nanovdb/python/cuda/PyPruneGrid.h | 16 +++++ nanovdb/nanovdb/python/cuda/PyRefineGrid.cu | 39 +++++++++++++ nanovdb/nanovdb/python/cuda/PyRefineGrid.h | 16 +++++ 12 files changed, 326 insertions(+) create mode 100644 nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyCoarsenGrid.h create mode 100644 nanovdb/nanovdb/python/cuda/PyDilateGrid.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyDilateGrid.h create mode 100644 nanovdb/nanovdb/python/cuda/PyMergeGrids.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyMergeGrids.h create mode 100644 nanovdb/nanovdb/python/cuda/PyPruneGrid.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyPruneGrid.h create mode 100644 nanovdb/nanovdb/python/cuda/PyRefineGrid.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyRefineGrid.h diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 0650cce2ca..45137d7dc4 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -38,6 +38,11 @@ nanobind_add_module(nanovdb_python NB_STATIC cuda/PyPointsToGrid.cu cuda/PySampleFromVoxels.cu cuda/PySignedFloodFill.cu + cuda/PyDilateGrid.cu + cuda/PyCoarsenGrid.cu + cuda/PyRefineGrid.cu + cuda/PyPruneGrid.cu + cuda/PyMergeGrids.cu ) target_include_directories(nanovdb_python PRIVATE ${CUDA_INCLUDE_DIRECTORY}) diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 92dd6841ce..1ccd550fe5 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -19,6 +19,11 @@ #include "cuda/PyPointsToGrid.h" #include "cuda/PySampleFromVoxels.h" #include "cuda/PySignedFloodFill.h" +#include "cuda/PyDilateGrid.h" +#include "cuda/PyCoarsenGrid.h" +#include "cuda/PyRefineGrid.h" +#include "cuda/PyPruneGrid.h" +#include "cuda/PyMergeGrids.h" #endif namespace nb = nanobind; @@ -79,6 +84,15 @@ void defineToolsModule(nb::module_& m) defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); + // Topological/morphological ops on OnIndex grids (nanovdb::tools::cuda). + // Each is constrained to OnIndex build types by TopologyBuilder's + // is_onindex static_assert, so only ValueOnIndex is instantiated. + defineDilateGrid(cudaModule, "dilateGrid"); + defineCoarsenGrid(cudaModule, "coarsenGrid"); + defineRefineGrid(cudaModule, "refineGrid"); + definePruneGrid(cudaModule, "pruneGrid"); + defineMergeGrids(cudaModule, "mergeGrids"); + // Device VoxelBlockManager (nanovdb::tools::cuda) on nanovdb.tools.cuda. defineDeviceVoxelBlockManager(cudaModule); #endif diff --git a/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu b/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu new file mode 100644 index 0000000000..e76302358a --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu @@ -0,0 +1,39 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyCoarsenGrid.h" + +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +template void defineCoarsenGrid(nb::module_& m, const char* name) +{ + m.def( + name, + [](nanovdb::NanoGrid* d_grid, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // CoarsenGrid::getHandle launches kernels and synchronizes the + // stream; pure CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + nanovdb::tools::cuda::CoarsenGrid coarsener(d_grid, s); + return coarsener.getHandle(); + }, + "d_grid"_a, + "stream"_a = 0, + "Topologically coarsen (2x downsample) a device OnIndex grid and return " + "a fresh device GridHandle of the coarsened grid. stream is a raw CUDA " + "stream handle (Python int; 0 = default stream)."); +} + +template void defineCoarsenGrid(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.h b/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.h new file mode 100644 index 0000000000..419e264120 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.h @@ -0,0 +1,16 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYCOARSENGRID_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYCOARSENGRID_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +template void defineCoarsenGrid(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDilateGrid.cu b/nanovdb/nanovdb/python/cuda/PyDilateGrid.cu new file mode 100644 index 0000000000..b7e110bff2 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDilateGrid.cu @@ -0,0 +1,44 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyDilateGrid.h" + +#include + +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +template void defineDilateGrid(nb::module_& m, const char* name) +{ + m.def( + name, + [](nanovdb::NanoGrid* d_grid, int op, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // DilateGrid::getHandle launches kernels and synchronizes the + // stream; pure CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + nanovdb::tools::cuda::DilateGrid dilator(d_grid, s); + dilator.setOperation(static_cast(op)); + return dilator.getHandle(); + }, + "d_grid"_a, + "op"_a = static_cast(nanovdb::tools::morphology::NN_FACE_EDGE_VERTEX), + "stream"_a = 0, + "Morphologically dilate a device OnIndex grid and return a fresh device " + "GridHandle of the dilated grid. op is a nearest-neighbor stencil: 6 " + "(NN_FACE) or 26 (NN_FACE_EDGE_VERTEX). NN_FACE_EDGE (18) is accepted by " + "the C++ setter but is not implemented and raises at getHandle time. " + "stream is a raw CUDA stream handle (Python int; 0 = default stream)."); +} + +template void defineDilateGrid(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDilateGrid.h b/nanovdb/nanovdb/python/cuda/PyDilateGrid.h new file mode 100644 index 0000000000..6e7eff9887 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDilateGrid.h @@ -0,0 +1,16 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYDILATEGRID_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYDILATEGRID_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +template void defineDilateGrid(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu b/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu new file mode 100644 index 0000000000..25a98a12b4 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu @@ -0,0 +1,44 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyMergeGrids.h" + +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +template void defineMergeGrids(nb::module_& m, const char* name) +{ + m.def( + name, + [](nanovdb::NanoGrid* d_grid1, + nanovdb::NanoGrid* d_grid2, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // MergeGrids::getHandle launches kernels and synchronizes the + // stream; pure CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + nanovdb::tools::cuda::MergeGrids merger(d_grid1, d_grid2, s); + return merger.getHandle(); + }, + "d_grid1"_a, + "d_grid2"_a, + "stream"_a = 0, + "Topologically merge (active-mask union) two device OnIndex grids and " + "return a fresh device GridHandle of the union. The operation is " + "strictly binary; chain calls to union more than two grids. Output " + "metadata is taken from d_grid1. stream is a raw CUDA stream handle " + "(Python int; 0 = default stream)."); +} + +template void defineMergeGrids(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyMergeGrids.h b/nanovdb/nanovdb/python/cuda/PyMergeGrids.h new file mode 100644 index 0000000000..9ddb8958c3 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyMergeGrids.h @@ -0,0 +1,16 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYMERGEGRIDS_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYMERGEGRIDS_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +template void defineMergeGrids(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu b/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu new file mode 100644 index 0000000000..d857e86dbc --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu @@ -0,0 +1,61 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyPruneGrid.h" + +#include + +#include +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +template void definePruneGrid(nb::module_& m, const char* name) +{ + m.def( + name, + [](nanovdb::NanoGrid* d_grid, + nb::ndarray leafMask, + uintptr_t stream) { + // The sidecar is a device array of nanovdb::Mask<3> (one 512-bit / + // 8^3 leaf mask per leaf node, voxels to RETAIN), passed as raw + // uint64 words. Mask<3> is exactly WORD_COUNT 64-bit words, so the + // total word count must be a whole multiple of that, and should + // equal (leaf count) * Mask<3>::WORD_COUNT for a well-formed call. + constexpr size_t wordsPerMask = nanovdb::Mask<3>::WORD_COUNT; + const size_t totalWords = leafMask.size(); + if (totalWords % wordsPerMask != 0) + throw std::invalid_argument( + "pruneGrid: leafMask uint64 word count must be a multiple of " + "Mask<3>::WORD_COUNT (8); supply one 512-bit mask per leaf node"); + cudaStream_t s = reinterpret_cast(stream); + const nanovdb::Mask<3>* d_mask = + reinterpret_cast*>(leafMask.data()); + // PruneGrid::getHandle launches kernels and synchronizes the + // stream; pure CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + nanovdb::tools::cuda::PruneGrid pruner(d_grid, d_mask, s); + return pruner.getHandle(); + }, + "d_grid"_a, + "leafMask"_a, + "stream"_a = 0, + "Morphologically prune a device OnIndex grid against a per-leaf retain " + "mask and return a fresh device GridHandle of the pruned grid. leafMask " + "is a device uint64 array holding one nanovdb::Mask<3> (8 x uint64 = " + "512 bits) per leaf node, in leaf order, marking voxels to RETAIN; its " + "length must be (leaf count) * 8. stream is a raw CUDA stream handle " + "(Python int; 0 = default stream)."); +} + +template void definePruneGrid(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyPruneGrid.h b/nanovdb/nanovdb/python/cuda/PyPruneGrid.h new file mode 100644 index 0000000000..c015958951 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyPruneGrid.h @@ -0,0 +1,16 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYPRUNEGRID_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYPRUNEGRID_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +template void definePruneGrid(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyRefineGrid.cu b/nanovdb/nanovdb/python/cuda/PyRefineGrid.cu new file mode 100644 index 0000000000..316e05ff6c --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyRefineGrid.cu @@ -0,0 +1,39 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyRefineGrid.h" + +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +template void defineRefineGrid(nb::module_& m, const char* name) +{ + m.def( + name, + [](nanovdb::NanoGrid* d_grid, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // RefineGrid::getHandle launches kernels and synchronizes the + // stream; pure CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + nanovdb::tools::cuda::RefineGrid refiner(d_grid, s); + return refiner.getHandle(); + }, + "d_grid"_a, + "stream"_a = 0, + "Topologically refine (2x upsample) a device OnIndex grid and return a " + "fresh device GridHandle of the refined grid. stream is a raw CUDA " + "stream handle (Python int; 0 = default stream)."); +} + +template void defineRefineGrid(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyRefineGrid.h b/nanovdb/nanovdb/python/cuda/PyRefineGrid.h new file mode 100644 index 0000000000..9af34dcc1c --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyRefineGrid.h @@ -0,0 +1,16 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYREFINEGRID_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYREFINEGRID_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +template void defineRefineGrid(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif From 32307718fc3cdefe8ffb39beecdb9288be789733 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 05:11:09 +0000 Subject: [PATCH 18/48] nanovdb cuda: fix tools::cuda::evalChecksum device->host copy size MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both evalChecksum overloads (GridData* and NanoGrid*) copied the computed head/tail CRC from device to host with size `headSize` (sizeof(GridData)+sizeof(TreeData), ~672 bytes) into the 4-byte Checksum::head()/tail() uint32_t destinations, reading ~672 bytes from `d_lut.get()+256` — one uint32_t past the 1024-byte CRC LUT. On a real device this trips "CUDA error 1: invalid argument" and cudaCheck aborts. The sibling validateChecksum copies the CRC correctly with sizeof(uint32_t); match it. The full-GridData header copies (which legitimately use headSize) are unchanged. Surfaced by the new nanovdb.tools.cuda.evalChecksum Python binding; verified fixed on a Blackwell GPU (evalChecksum returns a Checksum for CheckMode.Partial and Full instead of aborting). Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/tools/cuda/GridChecksum.cuh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/nanovdb/nanovdb/tools/cuda/GridChecksum.cuh b/nanovdb/nanovdb/tools/cuda/GridChecksum.cuh index eef49d5ebc..2d9bc02af5 100644 --- a/nanovdb/nanovdb/tools/cuda/GridChecksum.cuh +++ b/nanovdb/nanovdb/tools/cuda/GridChecksum.cuh @@ -234,7 +234,7 @@ inline Checksum evalChecksum(const GridData *d_gridData, CheckMode mode, cudaStr if (mode != CheckMode::Empty) { auto d_lut = util::cuda::createCrc32Lut(1, stream); crc32Head(d_gridData, d_lut.get(), d_lut.get() + 256, stream); - cudaCheck(cudaMemcpyAsync(&(cs.head()), d_lut.get() + 256, headSize, cudaMemcpyDeviceToHost, stream)); + cudaCheck(cudaMemcpyAsync(&(cs.head()), d_lut.get() + 256, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream)); if (mode == CheckMode::Full) { std::unique_ptr buffer(new char[headSize]); auto *gridData = (GridData*)(buffer.get()); @@ -244,7 +244,7 @@ inline Checksum evalChecksum(const GridData *d_gridData, CheckMode mode, cudaStr } else { callNanoGrid(d_gridData, gridData, d_lut.get(), d_lut.get() + 256, stream); } - cudaCheck(cudaMemcpyAsync(&(cs.tail()), d_lut.get() + 256, headSize, cudaMemcpyDeviceToHost, stream)); + cudaCheck(cudaMemcpyAsync(&(cs.tail()), d_lut.get() + 256, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream)); } } return cs; @@ -265,7 +265,7 @@ Checksum evalChecksum(const NanoGrid *d_grid, CheckMode mode, cudaStream if (mode != CheckMode::Empty) { auto d_lut = util::cuda::createCrc32Lut(1, stream); crc32Head(d_grid, d_lut.get(), d_lut.get() + 256, stream); - cudaCheck(cudaMemcpyAsync(&(cs.head()), d_lut.get() + 256, headSize, cudaMemcpyDeviceToHost, stream)); + cudaCheck(cudaMemcpyAsync(&(cs.head()), d_lut.get() + 256, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream)); if (mode == CheckMode::Full) { std::unique_ptr buffer(new char[headSize]); auto *gridData = (GridData*)(buffer.get()); @@ -275,7 +275,7 @@ Checksum evalChecksum(const NanoGrid *d_grid, CheckMode mode, cudaStream } else { crc32TailOld(d_grid, gridData, d_lut.get(), d_lut.get() + 256, stream); } - cudaCheck(cudaMemcpyAsync(&(cs.tail()), d_lut.get() + 256, headSize, cudaMemcpyDeviceToHost, stream)); + cudaCheck(cudaMemcpyAsync(&(cs.tail()), d_lut.get() + 256, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream)); } } return cs; From 1c1fa6d5dfc67943dcf5cb3ff6d11925e8006ff7 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 05:11:09 +0000 Subject: [PATCH 19/48] =?UTF-8?q?nanovdb=20python:=20GPU=20epoch=20Phase?= =?UTF-8?q?=20E2=20=E2=80=94=20tools.cuda=20index=20utils=20+=20device=20Q?= =?UTF-8?q?C?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bind into nanovdb.tools.cuda (one Py*.cu + .h per header; stream:int=0 + nb::gil_scoped_release; fully-qualified nanovdb:: per the CUB ambiguity fix): - indexToGrid(index_grid, values, stream=0) -> DeviceGridHandle: device ValueIndex/ValueOnIndex grid + a device values ndarray -> typed grid. Scalar (float/double, flat 1-D) and Vec3 (Vec3f/Vec3d, (N,3) c_contig) destinations. Dst restricted to non-special types (processLeafsKernel static_assert !is_special); Src restricted to index types (SFINAE). - addBlindData(grid, blind_data, class, semantic, name, stream=0) -> DeviceGridHandle: appends a device blind-data channel (valueCount from the array length). Reuses existing GridBlindDataClass/Semantic enums. - Device QC mirroring the host names on tools.cuda: - updateGridStats(grid, mode=StatsMode.Default, stream=0) -> None - isValid(grid, mode=CheckMode.Default, verbose=False, stream=0) -> bool - evalChecksum/validateChecksum/updateChecksum(grid, mode, stream=0) (device grid reinterpreted as GridData*; the GridData* overloads only copy the header host-side). updateGridStats bound for scalar/vector/bool BuildTs (special/index/mask trip Extrema/Stats static_asserts); isValid + checksum bound across the full 20-type set. Include ordering: the QC .cu files include before the tool .cuh (the tool headers aren't self-contained for the device GridHandle's updateChecksum/NodeManager uses), matching unittest/TestNanoVDB.cu. Compatible with nanobind >= 2.5.0. Verified by per-TU compile + real incremental build + GPU runtime smoke (updateGridStats/isValid/evalChecksum/ validateChecksum/updateChecksum, indexToGrid float+OnIndex, addBlindData). Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 5 + nanovdb/nanovdb/python/PyTools.cc | 94 +++++++++++++++ nanovdb/nanovdb/python/cuda/PyAddBlindData.cu | 78 +++++++++++++ nanovdb/nanovdb/python/cuda/PyAddBlindData.h | 21 ++++ .../python/cuda/PyDeviceGridChecksum.cu | 108 ++++++++++++++++++ .../python/cuda/PyDeviceGridChecksum.h | 21 ++++ .../nanovdb/python/cuda/PyDeviceGridStats.cu | 66 +++++++++++ .../nanovdb/python/cuda/PyDeviceGridStats.h | 20 ++++ .../python/cuda/PyDeviceGridValidator.cu | 72 ++++++++++++ .../python/cuda/PyDeviceGridValidator.h | 20 ++++ nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu | 99 ++++++++++++++++ nanovdb/nanovdb/python/cuda/PyIndexToGrid.h | 25 ++++ 12 files changed, 629 insertions(+) create mode 100644 nanovdb/nanovdb/python/cuda/PyAddBlindData.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyAddBlindData.h create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceGridStats.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceGridStats.h create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.h create mode 100644 nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyIndexToGrid.h diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 45137d7dc4..358a480e14 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -43,6 +43,11 @@ nanobind_add_module(nanovdb_python NB_STATIC cuda/PyRefineGrid.cu cuda/PyPruneGrid.cu cuda/PyMergeGrids.cu + cuda/PyIndexToGrid.cu + cuda/PyAddBlindData.cu + cuda/PyDeviceGridStats.cu + cuda/PyDeviceGridValidator.cu + cuda/PyDeviceGridChecksum.cu ) target_include_directories(nanovdb_python PRIVATE ${CUDA_INCLUDE_DIRECTORY}) diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 1ccd550fe5..196e0badf2 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -24,6 +24,11 @@ #include "cuda/PyRefineGrid.h" #include "cuda/PyPruneGrid.h" #include "cuda/PyMergeGrids.h" +#include "cuda/PyIndexToGrid.h" +#include "cuda/PyAddBlindData.h" +#include "cuda/PyDeviceGridStats.h" +#include "cuda/PyDeviceGridValidator.h" +#include "cuda/PyDeviceGridChecksum.h" #endif namespace nb = nanobind; @@ -95,6 +100,95 @@ void defineToolsModule(nb::module_& m) // Device VoxelBlockManager (nanovdb::tools::cuda) on nanovdb.tools.cuda. defineDeviceVoxelBlockManager(cudaModule); + + // IndexGrid -> regular Grid (nanovdb::tools::cuda::indexToGrid). Source is + // an index grid (ValueIndex / ValueOnIndex); destination value type is a + // non-special type (float / double scalar, or Vec3f / Vec3d). All register + // under "indexToGrid"; nanobind disambiguates on the source grid class and + // the values ndarray dtype/shape. + defineIndexToGridScalar(cudaModule, "indexToGrid"); + defineIndexToGridScalar(cudaModule, "indexToGrid"); + defineIndexToGridScalar(cudaModule, "indexToGrid"); + defineIndexToGridScalar(cudaModule, "indexToGrid"); + defineIndexToGridVec3(cudaModule, "indexToGrid"); + defineIndexToGridVec3(cudaModule, "indexToGrid"); + defineIndexToGridVec3(cudaModule, "indexToGrid"); + defineIndexToGridVec3(cudaModule, "indexToGrid"); + + // Append blind data to a device grid (nanovdb::tools::cuda::addBlindData). + // Registered under "addBlindData" for a set of (grid BuildT, blind-data + // element type) combinations; nanobind disambiguates on the grid class and + // the blindData ndarray dtype. + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + + // Device quality-control tools (mirror the host tools.* names on + // tools.cuda). updateGridStats covers scalar/vector/bool grids; isValid + // and the checksum entries cover the full callNanoGrid BuildT set. + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + defineDeviceUpdateGridStats(cudaModule, "updateGridStats"); + + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); + + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); + defineDeviceGridChecksum(cudaModule); #endif } diff --git a/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu b/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu new file mode 100644 index 0000000000..68c468eaa8 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu @@ -0,0 +1,78 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyAddBlindData.h" + +#include +#include + +#include +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +template +void defineAddBlindData(nb::module_& m, const char* name) +{ + m.def( + name, + [](nanovdb::NanoGrid* d_grid, + nb::ndarray, nb::c_contig, nb::device::cuda> blindData, + nanovdb::GridBlindDataClass blindClass, + nanovdb::GridBlindDataSemantic semantics, + const std::string& dataName, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + const BlindDataT* d_blindData = blindData.data(); + const uint64_t valueCount = static_cast(blindData.size()); + // addBlindData copies the grid into a fresh device buffer with the + // blind data appended and launches kernels on the stream; pure + // CUDA touching no Python objects, so release the GIL. dataName is + // already a C++-owned std::string (nanobind's stl/string caster + // materialized it from the Python str), so its c_str() stays valid + // across the GIL release for the duration of this call frame. + nb::gil_scoped_release release; + return nanovdb::tools::cuda::addBlindData( + d_grid, d_blindData, valueCount, blindClass, semantics, + dataName.c_str(), nanovdb::cuda::DeviceBuffer(), s); + }, + "d_grid"_a, + "blindData"_a, + "blindClass"_a = nanovdb::GridBlindDataClass::Unknown, + "semantics"_a = nanovdb::GridBlindDataSemantic::Unknown, + "name"_a = "", + "stream"_a = 0, + "Append a flat 1-D device array of blind data to a copy of a device " + "grid and return a fresh device GridHandle with the blind data " + "attached. valueCount is taken from the array length; blindClass and " + "semantics tag the new GridBlindMetaData entry (a RuntimeError is " + "raised on an invalid combination). stream is a raw CUDA stream " + "handle (Python int; 0 = default stream)."); +} + +// Grid BuildTs x blind-data element types. addBlindData itself has no BuildT +// restriction beyond BufferTraits::hasDeviceDual (satisfied by +// DeviceBuffer); we expose the common combinations. The same overload name is +// reused so nanobind dispatches on the grid class plus the array dtype. +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyAddBlindData.h b/nanovdb/nanovdb/python/cuda/PyAddBlindData.h new file mode 100644 index 0000000000..91ec75bbc0 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyAddBlindData.h @@ -0,0 +1,21 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYADDBLINDDATA_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYADDBLINDDATA_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +// Bind nanovdb::tools::cuda::addBlindData for a (grid BuildT, blind-data +// element type) pair. The blind data is a flat 1-D device array; all +// instantiations register under the same Python name and are disambiguated by +// nanobind on the grid class and the blind-data ndarray dtype. +template +void defineAddBlindData(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu new file mode 100644 index 0000000000..836f7c5ef3 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu @@ -0,0 +1,108 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyDeviceGridChecksum.h" + +#include + +// Pull in cuda/GridHandle.cuh first: it includes GridChecksum.cuh and then +// uses nanovdb::tools::cuda::updateChecksum, so it must be parsed before any +// translation unit sets GridChecksum.cuh's include guard (otherwise the guard +// would skip GridChecksum.cuh's body and leave updateChecksum undeclared when +// GridHandle.cuh is parsed). Matches the unittest's include ordering. +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +// All three entries take the device NanoGrid* and use the GridData* +// device overloads in GridChecksum.cuh, which copy just the header host-side +// and run CRC32 on device — they never dereference the device pointer on the +// host, so reinterpreting the typed device grid pointer to GridData* is safe. +template +void defineDeviceGridChecksum(nb::module_& m) +{ + m.def( + "evalChecksum", + [](const nanovdb::NanoGrid* d_grid, nanovdb::CheckMode mode, + uintptr_t stream) -> nanovdb::Checksum { + cudaStream_t s = reinterpret_cast(stream); + const nanovdb::GridData* d_gridData = + reinterpret_cast(d_grid); + // Pure CUDA (CRC32 on device, header copied D2H); release the GIL. + nb::gil_scoped_release release; + return nanovdb::tools::cuda::evalChecksum(d_gridData, mode, s); + }, + "d_grid"_a, + "mode"_a = nanovdb::CheckMode::Default, + "stream"_a = 0, + "Compute and return the Checksum of the device grid for the given " + "CheckMode without modifying it. stream is a raw CUDA stream handle " + "(Python int; 0 = default stream)."); + + m.def( + "validateChecksum", + [](const nanovdb::NanoGrid* d_grid, nanovdb::CheckMode mode, + uintptr_t stream) -> bool { + cudaStream_t s = reinterpret_cast(stream); + const nanovdb::GridData* d_gridData = + reinterpret_cast(d_grid); + nb::gil_scoped_release release; + return nanovdb::tools::cuda::validateChecksum(d_gridData, mode, s); + }, + "d_grid"_a, + "mode"_a = nanovdb::CheckMode::Default, + "stream"_a = 0, + "Return True iff the device grid's stored checksum matches a freshly " + "computed one for the given CheckMode. A grid with an empty stored " + "checksum is considered valid. stream is a raw CUDA stream handle " + "(Python int; 0 = default stream)."); + + m.def( + "updateChecksum", + [](nanovdb::NanoGrid* d_grid, nanovdb::CheckMode mode, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + nanovdb::GridData* d_gridData = + reinterpret_cast(d_grid); + nb::gil_scoped_release release; + nanovdb::tools::cuda::updateChecksum(d_gridData, mode, s); + }, + "d_grid"_a, + "mode"_a = nanovdb::CheckMode::Default, + "stream"_a = 0, + "Recompute and write the checksum of the device grid in place using " + "the given CheckMode (returns None). stream is a raw CUDA stream " + "handle (Python int; 0 = default stream)."); +} + +// No BuildT restriction on the GridChecksum device entries; instantiate for +// the same set the host checksum dispatch covers via callNanoGrid. +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); +template void defineDeviceGridChecksum(nb::module_&); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h new file mode 100644 index 0000000000..3838df1542 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h @@ -0,0 +1,21 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYDEVICEGRIDCHECKSUM_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYDEVICEGRIDCHECKSUM_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +// Bind the device checksum entry points for one grid BuildT. Each registers +// nanovdb.tools.cuda.evalChecksum / validateChecksum / updateChecksum as an +// overload taking a (device) NanoGrid* reinterpreted as GridData*. +// nanobind disambiguates the overloads on the device grid class. +template +void defineDeviceGridChecksum(nb::module_& m); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.cu new file mode 100644 index 0000000000..718fa90a12 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.cu @@ -0,0 +1,66 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyDeviceGridStats.h" + +#include + +// GridStats.cuh uses NodeManager and (via GridHandle) the device +// checksum path but is not self-contained for either. Pull in +// cuda/GridHandle.cuh first: it includes cuda/NodeManager.cuh (for +// NodeManager) and GridChecksum.cuh, and must be parsed before any TU sets +// GridChecksum.cuh's include guard so tools::cuda::updateChecksum is declared +// when GridHandle.cuh is parsed. Matches the unittest's include ordering. +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +template +void defineDeviceUpdateGridStats(nb::module_& m, const char* name) +{ + m.def( + name, + [](nanovdb::NanoGrid* d_grid, nanovdb::tools::StatsMode mode, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // updateGridStats launches kernels and synchronizes the stream; + // pure CUDA touching no Python objects, so release the GIL. The + // operation mutates the device grid in place. + nb::gil_scoped_release release; + nanovdb::tools::cuda::updateGridStats(d_grid, mode, s); + }, + "d_grid"_a, + "mode"_a = nanovdb::tools::StatsMode::Default, + "stream"_a = 0, + "Recompute and write per-node statistics into the given device grid " + "in place (returns None). Does NOT recompute the grid checksum — call " + "updateChecksum afterward if the checksum must stay valid. stream is a " + "raw CUDA stream handle (Python int; 0 = default stream)."); +} + +// Scalar + vector + bool BuildTs. updateGridStats's MinMax / All branches +// instantiate Extrema / Stats, which are only meaningful for +// arithmetic value types, so the quantized / index / mask special BuildTs are +// intentionally NOT instantiated here (they would fail GridStats's ValueT +// static_assert / lack a usable Stats specialization). bool routes to the +// NoopStats path internally regardless of mode. +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); +template void defineDeviceUpdateGridStats(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.h b/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.h new file mode 100644 index 0000000000..bf26314a14 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.h @@ -0,0 +1,20 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYDEVICEGRIDSTATS_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYDEVICEGRIDSTATS_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +// Bind nanovdb::tools::cuda::updateGridStats for one grid BuildT. All +// instantiations register under the same Python name ("updateGridStats") and +// are disambiguated by nanobind on the (device) grid class. +template +void defineDeviceUpdateGridStats(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu new file mode 100644 index 0000000000..84524fd50a --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu @@ -0,0 +1,72 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyDeviceGridValidator.h" + +#include + +// Pull in cuda/GridHandle.cuh first: it includes GridChecksum.cuh (which +// GridValidator.cuh also pulls in) and then uses +// nanovdb::tools::cuda::updateChecksum, so it must be parsed before any +// translation unit sets GridChecksum.cuh's include guard. Matches the +// unittest's include ordering. +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +template +void defineDeviceIsValid(nb::module_& m, const char* name) +{ + m.def( + name, + [](const nanovdb::NanoGrid* d_grid, nanovdb::CheckMode mode, + bool verbose, uintptr_t stream) -> bool { + cudaStream_t s = reinterpret_cast(stream); + // isValid runs structural checks in a device kernel plus a device + // checksum validation; pure CUDA touching no Python objects (the + // optional verbose diagnostic goes to std::cerr), so release the + // GIL. + nb::gil_scoped_release release; + return nanovdb::tools::cuda::isValid(d_grid, mode, verbose, s); + }, + "d_grid"_a, + "mode"_a = nanovdb::CheckMode::Default, + "verbose"_a = false, + "stream"_a = 0, + "Return True iff the device grid passes structural validation for the " + "given CheckMode AND its stored checksum matches a freshly computed " + "one. If verbose, the first failure is printed to stderr. stream is a " + "raw CUDA stream handle (Python int; 0 = default stream)."); +} + +// Instantiate for the same BuildT set the host isValid covers via +// callNanoGrid (NanoVDB.h). tools::checkGrid compiles for all of these. +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.h b/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.h new file mode 100644 index 0000000000..28d01cf8b1 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.h @@ -0,0 +1,20 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYDEVICEGRIDVALIDATOR_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYDEVICEGRIDVALIDATOR_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +// Bind nanovdb::tools::cuda::isValid for one grid BuildT. All instantiations +// register under the same Python name ("isValid") and are disambiguated by +// nanobind on the (device) grid class. +template +void defineDeviceIsValid(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu b/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu new file mode 100644 index 0000000000..439d466465 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu @@ -0,0 +1,99 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyIndexToGrid.h" + +#include + +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +// Scalar destination value type (e.g. float, double): d_srcValues is a flat, +// 1-D device array indexed by the IndexGrid's per-element uint64 indices. +template +void defineIndexToGridScalar(nb::module_& m, const char* name) +{ + m.def( + name, + [](nanovdb::NanoGrid* d_srcGrid, + nb::ndarray, nb::c_contig, nb::device::cuda> values, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + const DstBuildT* d_values = values.data(); + // indexToGrid launches kernels and synchronizes the stream; pure + // CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + return nanovdb::tools::cuda::indexToGrid( + d_srcGrid, d_values, nanovdb::cuda::DeviceBuffer(), s); + }, + "d_srcGrid"_a, + "values"_a, + "stream"_a = 0, + "Combine a device IndexGrid (ValueIndex / ValueOnIndex) with a flat " + "1-D device array of destination values into a fresh device " + "GridHandle of the destination value type. values is indexed by the " + "IndexGrid's per-voxel / per-node uint64 indices, so it must be sized " + "to cover the grid's value count. stream is a raw CUDA stream handle " + "(Python int; 0 = default stream)."); +} + +// Vec3 destination value type (Vec3f / Vec3d): d_srcValues is an (N, 3) device +// array of the matching scalar; Vec3 is exactly three contiguous scalars so +// the c_contig tensor reinterprets element-for-element as Vec3*. +template +void defineIndexToGridVec3(nb::module_& m, const char* name) +{ + using ScalarT = typename DstBuildT::ValueType; + m.def( + name, + [](nanovdb::NanoGrid* d_srcGrid, + nb::ndarray, nb::c_contig, nb::device::cuda> values, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + const DstBuildT* d_values = reinterpret_cast(values.data()); + // indexToGrid launches kernels and synchronizes the stream; pure + // CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + return nanovdb::tools::cuda::indexToGrid( + d_srcGrid, d_values, nanovdb::cuda::DeviceBuffer(), s); + }, + "d_srcGrid"_a, + "values"_a, + "stream"_a = 0, + "Combine a device IndexGrid (ValueIndex / ValueOnIndex) with an (N, 3) " + "device array of destination vector values into a fresh device " + "GridHandle of the destination Vec3 value type. The array is indexed " + "by the IndexGrid's per-voxel / per-node uint64 indices, so N must " + "cover the grid's value count. stream is a raw CUDA stream handle " + "(Python int; 0 = default stream)."); +} + +// Destination types: float / double (scalar) and Vec3f / Vec3d (vector), each +// for both index source types (ValueIndex and ValueOnIndex). DstBuildT is +// restricted by IndexToGrid's processLeafsKernel static_assert to non-special +// types, so the quantized / index / mask BuildTs are intentionally NOT +// instantiated. +template void defineIndexToGridScalar(nb::module_&, const char*); +template void defineIndexToGridScalar(nb::module_&, const char*); +template void defineIndexToGridScalar(nb::module_&, const char*); +template void defineIndexToGridScalar(nb::module_&, const char*); + +template void +defineIndexToGridVec3(nb::module_&, const char*); +template void +defineIndexToGridVec3(nb::module_&, const char*); +template void +defineIndexToGridVec3(nb::module_&, const char*); +template void +defineIndexToGridVec3(nb::module_&, const char*); + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyIndexToGrid.h b/nanovdb/nanovdb/python/cuda/PyIndexToGrid.h new file mode 100644 index 0000000000..6a71216aa9 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyIndexToGrid.h @@ -0,0 +1,25 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYINDEXTOGRID_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYINDEXTOGRID_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +// Bind nanovdb::tools::cuda::indexToGrid for a (DstBuildT, SrcBuildT) pair. +// SrcBuildT must be an index build type (ValueIndex / ValueOnIndex); DstBuildT +// must be a non-special value type (float / double / Vec3f / ...). All +// instantiations are registered under the same Python name and disambiguated +// by nanobind on the source grid class and the value ndarray dtype/shape. +template +void defineIndexToGridScalar(nb::module_& m, const char* name); + +template +void defineIndexToGridVec3(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif From ca176a9eef9fd97b04d132f86284d65a36da7595 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 05:42:58 +0000 Subject: [PATCH 20/48] =?UTF-8?q?nanovdb=20python:=20GPU=20epoch=20Phase?= =?UTF-8?q?=20F=20=E2=80=94=20cuda=20infrastructure=20+=20DistributedPoint?= =?UTF-8?q?sToGrid?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On nanovdb.cuda (NanoVDBModule.cc): - UnifiedBuffer: registered via the Phase-A defineDeviceBufferLike seam, so it gets CAI/DLPack/device_ptr/host_ptr/size for free (managed memory: host_ptr == device_ptr). Plus ctors, capacity/resize/clear, advise, prefetch, deviceUpload/deviceDownload. - UnifiedGridHandle (GridHandle): needed because DistributedPointsToGrid.getHandle returns one; full GridHandle surface + deviceGrid/deviceUpload/deviceDownload/device_ptr. - DeviceMesh (+ DeviceNode), DeviceStreamMap (+ DeviceType enum): host-query surface for multi-device/stream coordination. - DeviceResource (static allocateAsync/deallocateAsync + DEFAULT_ALIGNMENT) and TempDevicePool (TempPool) bound in their CURRENT shape only — not reshaped into the future resource concept. On nanovdb.tools.cuda (PyTools.cc): - DistributedPointsToGrid / DistributedIndexPointsToGrid / DistributedRGBA8PointsToGrid (ValueOnIndex / ValueIndex / Rgba8): ctor (DeviceMesh + scale/translation, mesh kept alive via keep_alive) + getHandle((N,3) int32 coords) -> UnifiedGridHandle. getHandle requires nb::device::cuda_managed input: the C++ pipeline issues cudaMemAdvise/memPrefetchAsync directly on the pointer, so the array MUST be CUDA managed/unified memory. (Adversarial review caught that the initial nb::device::cuda tag made it uncallable — managed input rejected, plain device input crashed; fixed to cuda_managed.) Forward-compat guardrail honored: none of the reserved future names (Resource/MemoryResource/AsyncResource/PinnedResource/ResourceDeviceBuffer/ DeviceBuffer2/default_resource/set_default_resource) are bound. NCCL-gated and __device__-only internals left unbound. Compatible with nanobind >= 2.5.0. Verified by concurrent per-TU compile + real incremental build + GPU runtime smoke: UnifiedBuffer zero-copy CAI/DLPack round-trip, DeviceMesh/DeviceStreamMap construct, and all three Distributed*PointsToGrid build OnIndex/Index/RGBA8 grids from managed coords on a single GPU (plain-device input cleanly rejected). Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 6 + nanovdb/nanovdb/python/NanoVDBModule.cc | 16 ++ nanovdb/nanovdb/python/PyTools.cc | 10 ++ nanovdb/nanovdb/python/cuda/PyDeviceMesh.cu | 79 +++++++++ nanovdb/nanovdb/python/cuda/PyDeviceMesh.h | 23 +++ .../nanovdb/python/cuda/PyDeviceStreamMap.cu | 128 +++++++++++++++ .../nanovdb/python/cuda/PyDeviceStreamMap.h | 25 +++ .../python/cuda/PyDistributedPointsToGrid.cu | 82 ++++++++++ .../python/cuda/PyDistributedPointsToGrid.h | 24 +++ nanovdb/nanovdb/python/cuda/PyTempPool.cu | 102 ++++++++++++ nanovdb/nanovdb/python/cuda/PyTempPool.h | 23 +++ .../nanovdb/python/cuda/PyUnifiedBuffer.cu | 152 ++++++++++++++++++ nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.h | 24 +++ .../python/cuda/PyUnifiedGridHandle.cu | 103 ++++++++++++ .../nanovdb/python/cuda/PyUnifiedGridHandle.h | 23 +++ 15 files changed, 820 insertions(+) create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceMesh.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceMesh.h create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceStreamMap.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyDeviceStreamMap.h create mode 100644 nanovdb/nanovdb/python/cuda/PyDistributedPointsToGrid.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyDistributedPointsToGrid.h create mode 100644 nanovdb/nanovdb/python/cuda/PyTempPool.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyTempPool.h create mode 100644 nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.h create mode 100644 nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.h diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 358a480e14..9e99e8025e 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -32,6 +32,12 @@ nanobind_add_module(nanovdb_python NB_STATIC PyTree.cc PyVoxelBlockManager.cc cuda/PyDeviceBuffer.cc + cuda/PyUnifiedBuffer.cu + cuda/PyUnifiedGridHandle.cu + cuda/PyDeviceMesh.cu + cuda/PyDeviceStreamMap.cu + cuda/PyTempPool.cu + cuda/PyDistributedPointsToGrid.cu cuda/PyDeviceGridHandle.cu cuda/PyDeviceNodeManager.cu cuda/PyDeviceVoxelBlockManager.cu diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 2a10699420..8d5f56eb5a 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -16,6 +16,13 @@ #include #include "cuda/PyDeviceBuffer.h" +#ifdef NANOVDB_USE_CUDA +#include "cuda/PyUnifiedBuffer.h" +#include "cuda/PyUnifiedGridHandle.h" +#include "cuda/PyDeviceMesh.h" +#include "cuda/PyDeviceStreamMap.h" +#include "cuda/PyTempPool.h" +#endif #include "PyBuildGrid.h" #include "PyGridHandle.h" #include "PyHostBuffer.h" @@ -1037,10 +1044,19 @@ NB_MODULE(nanovdb, m) nb::module_ cudaModule = m.def_submodule("cuda"); cudaModule.doc() = "CUDA device buffers, the device GridHandle, and device infrastructure (mirrors nanovdb::cuda)."; defineDeviceBuffer(cudaModule); + // UnifiedBuffer (CUDA managed memory) + its GridHandle. The unified handle + // is the type returned by nanovdb.tools.cuda.DistributedPointsToGrid. + defineUnifiedBuffer(cudaModule); + defineUnifiedGridHandle(cudaModule); defineDeviceGridHandle(cudaModule); // Device NodeManager (DeviceNodeManagerHandle + createDeviceNodeManager) // on nanovdb.cuda, alongside the device GridHandle. defineDeviceNodeManager(cudaModule); + // Multi-GPU / device infrastructure (mirrors nanovdb::cuda): the device + // mesh, the device->stream map, and the CUB temp-storage pool primitives. + defineDeviceMesh(cudaModule); + defineDeviceStreamMap(cudaModule); + defineTempPool(cudaModule); // Device VoxelBlockManager (mirrors nanovdb::tools::cuda) is registered on // the existing nanovdb.tools.cuda submodule in PyTools.cc — NOT here — so // the Python layout matches the C++ namespaces. diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 196e0badf2..82b88b21ae 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -17,6 +17,7 @@ #include "PyVoxelBlockManager.h" // for defineDeviceVoxelBlockManager (CUDA) #ifdef NANOVDB_USE_CUDA #include "cuda/PyPointsToGrid.h" +#include "cuda/PyDistributedPointsToGrid.h" #include "cuda/PySampleFromVoxels.h" #include "cuda/PySignedFloodFill.h" #include "cuda/PyDilateGrid.h" @@ -86,6 +87,15 @@ void defineToolsModule(nb::module_& m) definePointsToGrid(cudaModule, "pointsToGrid"); definePointsToGrid(cudaModule, "pointsToGrid"); + // Multi-GPU voxel-coordinate -> grid builder (nanovdb::tools::cuda:: + // DistributedPointsToGrid). Distributes an (N, 3) int32 unified-memory + // array of index-space voxel coordinates over a DeviceMesh. Bound for the + // index / Rgba8 build types (matching the voxelsTo*Grid set); Point is + // excluded (its coords must be world-space Vec3, not int32 Coord). + defineDistributedPointsToGrid(cudaModule, "DistributedPointsToGrid"); + defineDistributedPointsToGrid(cudaModule, "DistributedIndexPointsToGrid"); + defineDistributedPointsToGrid(cudaModule, "DistributedRGBA8PointsToGrid"); + defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); defineSampleFromVoxels(cudaModule, "sampleFromVoxels"); diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceMesh.cu b/nanovdb/nanovdb/python/cuda/PyDeviceMesh.cu new file mode 100644 index 0000000000..8aa9d8c59d --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceMesh.cu @@ -0,0 +1,79 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include "PyDeviceMesh.h" + +#include + +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; + +namespace pynanovdb { + +void defineDeviceMesh(nb::module_& m) +{ + using DeviceNode = nanovdb::cuda::DeviceNode; + using DeviceMesh = nanovdb::cuda::DeviceMesh; + + nb::class_(m, "DeviceNode", + "A device id paired with a CUDA stream created on that device.") + .def_ro("id", &DeviceNode::id, + "CUDA device id this node refers to (-1 if unset).") + .def_prop_ro( + "stream", + [](const DeviceNode& node) { + return reinterpret_cast(node.stream); + }, + "Raw CUDA stream handle for this device as a Python int " + "(0 = default stream)."); + + nb::class_(m, "DeviceMesh", + "Multi-GPU context: enumerates every CUDA device on the host, creates " + "a stream per device, and caches peer-to-peer connectivity. Pass it to " + "nanovdb.tools.cuda.DistributedPointsToGrid. Move-only / not copyable.") + .def( + "__init__", + [](DeviceMesh* self) { + // The ctor touches every device (stream creation, P2P probe), + // so release the GIL while it runs. + nb::gil_scoped_release release; + new (self) DeviceMesh(); + }, + "Construct a DeviceMesh spanning every CUDA device on the host. " + "Each device must support managed memory.") + .def( + "deviceCount", + [](const DeviceMesh& mesh) { return static_cast(mesh.deviceCount()); }, + "Number of devices in this mesh.") + .def( + "__len__", + [](const DeviceMesh& mesh) { return static_cast(mesh.deviceCount()); }, + "Number of devices in this mesh (same as deviceCount()).") + .def( + "__getitem__", + [](const DeviceMesh& mesh, int deviceId) -> const DeviceNode& { + if (deviceId < 0 || static_cast(deviceId) >= mesh.deviceCount()) + throw nb::index_error("DeviceMesh index out of range [0, deviceCount())."); + return mesh[deviceId]; + }, + "deviceId"_a, + nb::rv_policy::reference_internal, + "Return the DeviceNode (id + stream) for the given device index.") + .def( + "canAccessPeer", + [](const DeviceMesh& mesh, int deviceId, int peerId) { + return mesh.canAccessPeer(deviceId, peerId); + }, + "deviceId"_a, "peerId"_a, + "True iff `deviceId` can directly access memory on `peerId` " + "(peer-to-peer support)."); +} + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceMesh.h b/nanovdb/nanovdb/python/cuda/PyDeviceMesh.h new file mode 100644 index 0000000000..05d0e6928a --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceMesh.h @@ -0,0 +1,23 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYDEVICEMESH_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYDEVICEMESH_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +#ifdef NANOVDB_USE_CUDA +/// @brief Register nanovdb::cuda::DeviceNode and nanovdb::cuda::DeviceMesh on +/// the nanovdb.cuda submodule. DeviceMesh enumerates every CUDA device +/// on the host, creates a per-device stream, and caches P2P +/// connectivity; it is the multi-GPU context object consumed by +/// nanovdb.tools.cuda.DistributedPointsToGrid. +void defineDeviceMesh(nb::module_& m); +#endif + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceStreamMap.cu b/nanovdb/nanovdb/python/cuda/PyDeviceStreamMap.cu new file mode 100644 index 0000000000..c61a6188b3 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceStreamMap.cu @@ -0,0 +1,128 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include "PyDeviceStreamMap.h" + +#include + +#include +#include + +#include +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; + +namespace pynanovdb { + +void defineDeviceStreamMap(nb::module_& m) +{ + using DeviceStreamMap = nanovdb::cuda::DeviceStreamMap; + + nb::class_ cls(m, "DeviceStreamMap", + "Maps each suitable CUDA device id to a freshly-created stream. The " + "constructor filters devices by a DeviceType policy. Iterates as " + "(device_id -> raw stream handle) pairs."); + + nb::enum_(cls, "DeviceType", + "Device-inclusion policy for DeviceStreamMap.") + .value("Any", DeviceStreamMap::DeviceType::Any, + "Include every available device (no filtering).") + .value("PeerToPeer", DeviceStreamMap::DeviceType::PeerToPeer, + "Include only devices that can peer-access every already-added " + "device.") + .value("Unified", DeviceStreamMap::DeviceType::Unified, + "Include only devices supporting unified addressing + concurrent " + "managed access AND peer access to the already-added devices " + "(the default)."); + + cls + .def( + "__init__", + [](DeviceStreamMap* self, DeviceStreamMap::DeviceType type, + std::vector exclude, int verbose) { + // Touches every device (attribute queries, stream creation), + // so release the GIL. + nb::gil_scoped_release release; + new (self) DeviceStreamMap(type, std::move(exclude), verbose); + }, + "type"_a = DeviceStreamMap::DeviceType::Unified, + "exclude"_a = std::vector{}, + "verbose"_a = 0, + "Build a map of device id -> CUDA stream over the devices that " + "satisfy `type`. `exclude` is a list of device ids to skip; " + "`verbose` is 0 (quiet), 1 (print ignored devices) or 2 (print " + "included devices).") + .def("deviceCount", &DeviceStreamMap::deviceCount, + "Number of devices in this map.") + .def("__len__", &DeviceStreamMap::deviceCount, + "Number of devices in this map (same as deviceCount()).") + .def( + "getMinPageSize", + [](const DeviceStreamMap& map) { + nb::gil_scoped_release release; + return map.getMinPageSize(); + }, + "Minimum CUDA allocation granularity (in bytes) across all devices " + "in this map.") + .def( + "printDevInfo", + [](const DeviceStreamMap& map) { map.printDevInfo(); }, + "Print device information for every device in this map to stdout.") + .def( + "items", + [](const DeviceStreamMap& map) { + // Expose the underlying std::map as a Python + // dict {device_id -> raw stream handle (int)}. + nb::dict out; + for (const auto& kv : map) + out[nb::int_(kv.first)] = + nb::int_(reinterpret_cast(kv.second)); + return out; + }, + "Return a dict mapping each device id to its raw CUDA stream handle " + "(Python int).") + .def( + "stream", + [](const DeviceStreamMap& map, int deviceId) { + auto it = map.find(deviceId); + if (it == map.end()) + throw nb::key_error("DeviceStreamMap has no stream for that device id."); + return reinterpret_cast(it->second); + }, + "deviceId"_a, + "Raw CUDA stream handle (Python int) for the given device id.") + .def( + "__contains__", + [](const DeviceStreamMap& map, int deviceId) { + return map.find(deviceId) != map.end(); + }, + "deviceId"_a, + "True iff this map holds a stream for the given device id.") + .def( + "__getitem__", + [](const DeviceStreamMap& map, int deviceId) { + auto it = map.find(deviceId); + if (it == map.end()) + throw nb::key_error("DeviceStreamMap has no stream for that device id."); + return reinterpret_cast(it->second); + }, + "deviceId"_a, + "Raw CUDA stream handle (Python int) for the given device id.") + .def( + "__iter__", + [](const DeviceStreamMap& map) { + nb::list keys; + for (const auto& kv : map) keys.append(kv.first); + return nb::iter(keys); + }, + "Iterate over the device ids in this map."); +} + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceStreamMap.h b/nanovdb/nanovdb/python/cuda/PyDeviceStreamMap.h new file mode 100644 index 0000000000..d8a79a643d --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDeviceStreamMap.h @@ -0,0 +1,25 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYDEVICESTREAMMAP_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYDEVICESTREAMMAP_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +#ifdef NANOVDB_USE_CUDA +/// @brief Register nanovdb::cuda::DeviceStreamMap and its DeviceType enum on +/// the nanovdb.cuda submodule. DeviceStreamMap is a std::map that, on construction, filters the available devices by +/// a DeviceType policy and creates one stream per surviving device. +/// @note DeviceStreamMap.h defines its (non-template) ctor/dtor in the header +/// without `inline`, so it must be included in exactly ONE translation +/// unit — this one. +void defineDeviceStreamMap(nb::module_& m); +#endif + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDistributedPointsToGrid.cu b/nanovdb/nanovdb/python/cuda/PyDistributedPointsToGrid.cu new file mode 100644 index 0000000000..1a8b507fd7 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDistributedPointsToGrid.cu @@ -0,0 +1,82 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include "PyDistributedPointsToGrid.h" + +#include + +#include + +#include +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; + +namespace pynanovdb { + +template void defineDistributedPointsToGrid(nb::module_& m, const char* name) +{ + using ConverterT = nanovdb::tools::cuda::DistributedPointsToGrid; + using BufferT = nanovdb::cuda::UnifiedBuffer; + + nb::class_(m, name, + "Multi-GPU builder of a NanoVDB grid from an array of index-space voxel " + "coordinates, distributed over a DeviceMesh. Construct with a DeviceMesh " + "(which must outlive this object) plus a voxel scale and translation, " + "then call getHandle(voxels, count). On a single-GPU mesh this still " + "runs the trivial single-device path.") + .def( + "__init__", + [](ConverterT* self, const nanovdb::cuda::DeviceMesh& mesh, + double scale, nb::tuple trans) { + nanovdb::Vec3d t(0.0); + if (trans.size() == 3) + t = nanovdb::Vec3d(nb::cast(trans[0]), + nb::cast(trans[1]), + nb::cast(trans[2])); + else if (trans.size() != 0) + throw nb::value_error("translation must be a 3-tuple or empty."); + nb::gil_scoped_release release; + new (self) ConverterT(mesh, scale, t); + }, + "mesh"_a, "scale"_a = 1.0, "translation"_a = nb::make_tuple(0.0, 0.0, 0.0), + nb::keep_alive<1, 2>(), // keep the DeviceMesh alive for our lifetime + "Construct a converter over the given DeviceMesh. scale is the " + "uniform voxel size and translation is a 3-tuple world offset used " + "to build the output grid's index-to-world map. The DeviceMesh is " + "held by reference and MUST outlive this converter.") + .def( + "getHandle", + [](ConverterT& self, + nb::ndarray, nb::c_contig, nb::device::cuda_managed> voxels) { + // The (N, 3) c-contiguous int32 array is bit-compatible with a + // dense nanovdb::Coord[N] (Coord == 3 x int32). The pipeline + // issues cudaMemAdvise / cudaMemPrefetchAsync on this pointer, + // so the memory MUST be CUDA managed (unified) memory — hence the + // nb::device::cuda_managed constraint (DLPack kDLCUDAManaged=13). + // Plain device memory (cuda=2) is rejected: prefetch to a device + // ordinal is invalid on non-managed memory. + auto* coords = reinterpret_cast(voxels.data()); + const size_t count = voxels.shape(0); + nb::gil_scoped_release release; + return self.template getHandle( + coords, count, BufferT()); + }, + "voxels"_a, + "Rasterize the given (N, 3) int32 array of index-space voxel " + "coordinates into a fresh UnifiedGridHandle of type " + "NanoGrid. The array MUST be backed by CUDA managed " + "(unified) memory — the multi-GPU pipeline applies memory advise " + "and prefetch directly to its pointer."); +} + +template void defineDistributedPointsToGrid(nb::module_&, const char*); +template void defineDistributedPointsToGrid(nb::module_&, const char*); +template void defineDistributedPointsToGrid(nb::module_&, const char*); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDistributedPointsToGrid.h b/nanovdb/nanovdb/python/cuda/PyDistributedPointsToGrid.h new file mode 100644 index 0000000000..a77161c9a3 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyDistributedPointsToGrid.h @@ -0,0 +1,24 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYDISTRIBUTEDPOINTSTOGRID_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYDISTRIBUTEDPOINTSTOGRID_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +#ifdef NANOVDB_USE_CUDA +/// @brief Register the multi-GPU nanovdb::tools::cuda::DistributedPointsToGrid< +/// BuildT> on the nanovdb.tools.cuda submodule. The Python class wraps a +/// nanovdb::cuda::DeviceMesh and a scale/translation (or unit map), and +/// getHandle(voxels, count) rasterizes an (N, 3) int32 unified-memory +/// array of index-space voxel coordinates into a UnifiedGridHandle. +/// @note The DeviceMesh passed to the constructor must outlive the converter. +template void defineDistributedPointsToGrid(nb::module_& m, const char* name); +#endif + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyTempPool.cu b/nanovdb/nanovdb/python/cuda/PyTempPool.cu new file mode 100644 index 0000000000..8a575dff1e --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyTempPool.cu @@ -0,0 +1,102 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include "PyTempPool.h" + +#include + +#include + +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; + +namespace pynanovdb { + +void defineTempPool(nb::module_& m) +{ + using DeviceResource = nanovdb::cuda::DeviceResource; + using TempDevicePool = nanovdb::cuda::TempDevicePool; + + // DeviceResource: a stateless static async allocator over the current CUDA + // device. Exposed in its current shape only — raw pointers are Python ints. + nb::class_(m, "DeviceResource", + "Stateless CUDA async allocator: allocateAsync / deallocateAsync issue " + "cudaMallocAsync / cudaFreeAsync on a stream. Pointers are raw Python " + "ints. Backs TempDevicePool.") + .def_ro_static("DEFAULT_ALIGNMENT", &DeviceResource::DEFAULT_ALIGNMENT, + "Default allocation alignment in bytes (256).") + .def_static( + "allocateAsync", + [](size_t bytes, size_t alignment, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + void* p = DeviceResource::allocateAsync(bytes, alignment, s); + return reinterpret_cast(p); + }, + "bytes"_a, "alignment"_a = DeviceResource::DEFAULT_ALIGNMENT, "stream"_a = 0, + "Asynchronously allocate `bytes` of device memory on `stream` and " + "return the raw device pointer as a Python int. The alignment " + "argument is accepted for API parity but ignored by cudaMallocAsync.") + .def_static( + "deallocateAsync", + [](uintptr_t ptr, size_t bytes, size_t alignment, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + DeviceResource::deallocateAsync(reinterpret_cast(ptr), + bytes, alignment, s); + }, + "ptr"_a, "bytes"_a = 0, "alignment"_a = DeviceResource::DEFAULT_ALIGNMENT, + "stream"_a = 0, + "Asynchronously free a device pointer (Python int) on `stream`. The " + "bytes / alignment arguments are accepted for API parity but ignored " + "by cudaFreeAsync."); + + // TempDevicePool: a thin pool for CUB temporary storage. It owns a raw + // device pointer, so it is intentionally non-copyable (the default copy + // would double-free). Exposed in its current shape only. + nb::class_(m, "TempDevicePool", + "Thin pool of CUB temporary device storage backed by DeviceResource. " + "Owns a raw device pointer (non-copyable). reallocate(stream) grows it " + "to requestedSize when needed.") + .def(nb::init<>(), + "Construct an empty pool (no device allocation yet).") + .def( + "data", + [](TempDevicePool& pool) { + return reinterpret_cast(pool.data()); + }, + "Raw device pointer to the pooled storage as a Python int " + "(0 if nothing has been allocated yet).") + .def( + "size", + [](TempDevicePool& pool) { return pool.size(); }, + "Currently allocated size of the pool in bytes.") + .def( + "requestedSize", + [](TempDevicePool& pool) { return pool.requestedSize(); }, + "Size in bytes most recently requested for the pool. reallocate() " + "grows the allocation to this value when it exceeds size().") + .def( + "setRequestedSize", + [](TempDevicePool& pool, size_t value) { pool.requestedSize() = value; }, + "value"_a, + "Set the requested size (in bytes) used by the next reallocate().") + .def( + "reallocate", + [](TempDevicePool& pool, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + pool.reallocate(s); + }, + "stream"_a = 0, + "Reallocate the pool on `stream` (a raw CUDA stream handle, Python " + "int) if it is empty or requestedSize() exceeds the current size()."); +} + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyTempPool.h b/nanovdb/nanovdb/python/cuda/PyTempPool.h new file mode 100644 index 0000000000..6522a8c34e --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyTempPool.h @@ -0,0 +1,23 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYTEMPPOOL_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYTEMPPOOL_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +#ifdef NANOVDB_USE_CUDA +/// @brief Register nanovdb::cuda::DeviceResource (static async allocator) and +/// nanovdb::cuda::TempDevicePool (= TempPool) on the +/// nanovdb.cuda submodule, in their current shape only. These are the +/// low-level temp-storage primitives used by the CUDA tools; they are +/// NOT reshaped into any resource-concept abstraction. +void defineTempPool(nb::module_& m); +#endif + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.cu b/nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.cu new file mode 100644 index 0000000000..a6bb8ab8a7 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.cu @@ -0,0 +1,152 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include "PyUnifiedBuffer.h" +#include "PyDeviceBuffer.h" // for defineDeviceBufferLike (shared interop) + +#include + +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; + +namespace pynanovdb { + +void defineUnifiedBuffer(nb::module_& m) +{ + using BufferT = nanovdb::cuda::UnifiedBuffer; + + // defineDeviceBufferLike gives size() / device_ptr / host_ptr / + // __cuda_array_interface__ / __dlpack_device__ / __dlpack__ for free. For + // UnifiedBuffer the managed pointer is valid on host AND device, so + // host_ptr and device_ptr report the SAME address. + defineDeviceBufferLike(m, "UnifiedBuffer") + .def( + "__init__", + [](BufferT* self, size_t size) { + // Allocate a managed page table of `size` bytes (size == capacity). + nb::gil_scoped_release release; + new (self) BufferT(size); + }, + "size"_a, + "Construct a UnifiedBuffer backed by `size` bytes of CUDA managed " + "(unified) memory. The same pointer is valid on the host and on " + "every device.") + .def( + "__init__", + [](BufferT* self, size_t size, int device, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // The (size, device, stream) ctor allocates and applies a + // preferred-location advise + prefetch to `device`. + nb::gil_scoped_release release; + new (self) BufferT(static_cast(size), device, s); + }, + "size"_a, + "device"_a, + "stream"_a = 0, + "Construct a UnifiedBuffer of `size` bytes and set the preferred " + "location advise plus an async prefetch to `device`. stream is a " + "raw CUDA stream handle (Python int; 0 = default stream).") + .def("capacity", &BufferT::capacity, + "Number of bytes reserved in the managed page table (room for " + "growth; may exceed size()).") + .def("empty", &BufferT::empty, + "True iff this buffer manages no memory.") + .def("isEmpty", &BufferT::isEmpty, + "Same as empty(). Retained for parity with the C++ " + "UnifiedBuffer::isEmpty() member.") + .def( + "clear", + [](BufferT& buf) { + nb::gil_scoped_release release; + buf.clear(); + }, + "Free all managed memory and reset this buffer to empty.") + .def( + "resize", + [](BufferT& buf, size_t size, int device) { + nb::gil_scoped_release release; + buf.resize(size, device); + }, + "size"_a, "device"_a = cudaCpuDeviceId, + "Resize the managed memory block. If the new size fits inside the " + "current capacity this only redefines size(); otherwise a new page " + "table is allocated (with a preferred-location advise on `device`) " + "and the old contents are copied over. device defaults to the host " + "(cudaCpuDeviceId == -1).") + .def( + "advise", + [](const BufferT& buf, ptrdiff_t byteOffset, size_t size, int device, + int adv) { + nb::gil_scoped_release release; + buf.advise(byteOffset, size, device, + static_cast(adv)); + }, + "byteOffset"_a, "size"_a, "device"_a, "advise"_a, + "Apply a single cudaMemoryAdvise (passed as its integer enumerator, " + "e.g. cudaMemAdviseSetPreferredLocation == 3) to the [byteOffset, " + "byteOffset + size) range for `device` (cudaCpuDeviceId == -1 " + "selects the host).") + .def( + "prefetch", + [](const BufferT& buf, ptrdiff_t byteOffset, size_t size, int device, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + buf.prefetch(byteOffset, size, device, s); + }, + "byteOffset"_a = 0, "size"_a = 0, "device"_a = cudaCpuDeviceId, + "stream"_a = 0, + "Prefetch the [byteOffset, byteOffset + size) range to `device` " + "(cudaCpuDeviceId == -1 selects the host). size == 0 prefetches all " + "size() bytes. stream is a raw CUDA stream handle (Python int).") + .def( + "deviceUpload", + [](const BufferT& buf, int device, uintptr_t stream, bool sync) { + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + buf.deviceUpload(device, s, sync); + }, + "device"_a = 0, "stream"_a = 0, "sync"_a = false, + "Prefetch all managed bytes to `device` (legacy DeviceBuffer-compat; " + "internally a memPrefetchAsync). stream is a raw CUDA stream handle " + "(Python int); if sync is True the call blocks until the prefetch " + "completes.") + .def( + "deviceDownload", + [](const BufferT& buf, uintptr_t stream, bool sync) { + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + buf.deviceDownload(s, sync); + }, + "stream"_a = 0, "sync"_a = false, + "Prefetch all managed bytes back to the host (legacy " + "DeviceBuffer-compat). stream is a raw CUDA stream handle (Python " + "int); if sync is True the call blocks until the prefetch " + "completes.") + .def_static( + "create", + [](size_t size) { + nb::gil_scoped_release release; + return BufferT::create(size); + }, + "size"_a, + "Create a UnifiedBuffer of `size` bytes (size == capacity).") + .def_static( + "create", + [](size_t size, size_t capacity) { + nb::gil_scoped_release release; + return BufferT::create(size, capacity); + }, + "size"_a, "capacity"_a, + "Create a UnifiedBuffer with `size` used bytes and a managed page " + "table of `capacity` bytes for future growth."); +} + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.h b/nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.h new file mode 100644 index 0000000000..2e0eafec4f --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyUnifiedBuffer.h @@ -0,0 +1,24 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYUNIFIEDBUFFER_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYUNIFIEDBUFFER_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +#ifdef NANOVDB_USE_CUDA +/// @brief Register nanovdb::cuda::UnifiedBuffer as "UnifiedBuffer" on the +/// nanovdb.cuda submodule. UnifiedBuffer is CUDA Unified (managed) +/// memory: a single pointer valid on the host AND every device, so it +/// gets the shared device-interop surface (CAI / DLPack / device_ptr / +/// host_ptr / size) for free — note host_ptr and device_ptr are the +/// same managed pointer. +void defineUnifiedBuffer(nb::module_& m); +#endif + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.cu new file mode 100644 index 0000000000..ac0657a5b8 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.cu @@ -0,0 +1,103 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include "PyUnifiedGridHandle.h" +#include "../PyGridHandle.h" + +#include + +#include + +#include +#include + +namespace nb = nanobind; +using namespace nb::literals; + +namespace pynanovdb { + +// Polymorphic deviceGrid(n) for a unified GridHandle — same dispatch shape as +// pyHostGrid / the DeviceBuffer pyDeviceGrid, returning the grid via +// the unified (managed) pointer. With UnifiedBuffer host and device pointers +// coincide, so this is valid as soon as the handle holds a grid. +static nb::object pyUnifiedDeviceGrid(nb::handle py_handle, uint32_t n) +{ + using BufferT = nanovdb::cuda::UnifiedBuffer; + auto& handle = nb::cast&>(py_handle); + if (n >= handle.gridCount()) return nb::none(); + switch (handle.gridType(n)) { +#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template deviceGrid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template deviceGrid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template deviceGrid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \ + case nanovdb::GridType::GridTypeEnum: { \ + auto* grid = handle.template deviceGrid(n); \ + return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ + : nb::none(); \ + } +#include "../BuildTypes.def" + default: + return nb::none(); + } +} + +void defineUnifiedGridHandle(nb::module_& m) +{ + using BufferT = nanovdb::cuda::UnifiedBuffer; + defineGridHandle(m, "UnifiedGridHandle") + .def("deviceGrid", &pyUnifiedDeviceGrid, "n"_a = 0, + nb::keep_alive<0, 1>(), + "Return the n-th grid as a typed Grid subclass selected by " + "gridType(n), accessed through the unified (managed) pointer, or " + "None if the BuildT is not bound in Python. The returned grid " + "keeps this handle alive.") + .def( + "deviceUpload", + [](nanovdb::GridHandle& handle, uintptr_t stream, bool sync) { + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + handle.deviceUpload(reinterpret_cast(s), sync); + }, + "stream"_a = 0, "sync"_a = true, + "Prefetch the unified buffer to the current device. stream is a raw " + "CUDA stream handle (Python int; 0 = default stream). If sync is " + "True the call blocks until the prefetch completes.") + .def( + "deviceDownload", + [](nanovdb::GridHandle& handle, uintptr_t stream, bool sync) { + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + handle.deviceDownload(reinterpret_cast(s), sync); + }, + "stream"_a = 0, "sync"_a = true, + "Prefetch the unified buffer back to the host. stream is a raw CUDA " + "stream handle (Python int; 0 = default stream). If sync is True " + "the call blocks until the prefetch completes.") + .def( + "device_ptr", + [](nanovdb::GridHandle& handle) { + return reinterpret_cast(handle.buffer().deviceData()); + }, + "Raw device (managed) pointer to the base of the whole buffer as a " + "Python int. For unified memory this equals the host pointer."); +} + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.h b/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.h new file mode 100644 index 0000000000..ec5a669616 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.h @@ -0,0 +1,23 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYUNIFIEDGRIDHANDLE_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYUNIFIEDGRIDHANDLE_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +#ifdef NANOVDB_USE_CUDA +/// @brief Register GridHandle as +/// "UnifiedGridHandle" on the nanovdb.cuda submodule. This is the handle +/// type returned by nanovdb.tools.cuda.DistributedPointsToGrid.getHandle +/// (its default BufferT is UnifiedBuffer), so the class must be +/// registered for nanobind to cast the result. +void defineUnifiedGridHandle(nb::module_& m); +#endif + +} // namespace pynanovdb + +#endif From e3b4f24643c31b453b18d14eb263f6cc139785af Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 06:08:32 +0000 Subject: [PATCH 21/48] =?UTF-8?q?nanovdb=20python:=20GPU=20epoch=20Phase?= =?UTF-8?q?=20G=20=E2=80=94=20GPU=20interop=20tests,=20custom-kernel=20exa?= =?UTF-8?q?mples,=20and=20docs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the test/example/documentation surface for the NanoVDB Python GPU bindings landed in the preceding phases. No C++ binding changes. - test/TestGpuInterop.py: stdlib-unittest suite (37 tests) mirroring TestNanoVDB.py gating — every case is class-guarded on isCudaAvailable() and isGpuAvailable(), CuPy is guarded per-test, and Torch via try-import + skipTest, so the suite self-skips cleanly on a non-CUDA/no-GPU build. Covers CAI v3 dict + DLPack round-trips aliasing device_ptr (nbytes==size), from_external (managed wrap + null rejection), from_buffer move/reject, non-default streams, the host-accessor-on-device-grid SIGSEGV contract (asserted in a subprocess so it cannot abort the runner), UnifiedBuffer, the generalized point/voxel rasterizers, device morphology/QC/flood-fill/ sampling, device NodeManager and VoxelBlockManager (firstLeafID/jumpMap device views), the cuda infrastructure types, and DistributedPointsToGrid over managed memory. - examples/{gpu_load_inspect,cupy_rawkernel,numba_cuda,triton_kernel}.py: runnable, import-guarded examples. cupy_rawkernel.py demonstrates the device-pointer ABI — a cp.RawKernel that #includes NanoVDB.h via compile_options() and reads a const NanoGrid* from grid.data_ptr(). numba/triton examples self-skip with an install hint when absent. - examples/README.md: a GPU/CUDA section covering the nanovdb.cuda (infra) vs nanovdb.tools.cuda (algorithms) split, attribute-not-submodule access, the device-grid build recipe, zero-copy interop + device-pointer ABI, the host-accessor caveat, streams, from_external/from_buffer, the managed-memory distributed pipeline, compile_options for NVRTC, the relocation migration note, and a forward-looking pluggable-memory-resources roadmap. - python/CMakeLists.txt: registers TestGpuInterop.py as its own ctest test (pytest_nanovdb_gpu_interop) mirroring pytest_nanovdb; self-skips without a GPU so it is harmless in non-CUDA CI. Validated locally on a Blackwell GPU with CuPy: full suite green (1 torch skip), both required examples run end-to-end, TestNanoVDB.py shows no regression, and a CMake reconfigure confirms both ctest entries register. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 14 +- nanovdb/nanovdb/python/examples/README.md | 172 +++++ .../nanovdb/python/examples/cupy_rawkernel.py | 103 +++ .../python/examples/gpu_load_inspect.py | 101 +++ nanovdb/nanovdb/python/examples/numba_cuda.py | 86 +++ .../nanovdb/python/examples/triton_kernel.py | 88 +++ nanovdb/nanovdb/python/test/TestGpuInterop.py | 592 ++++++++++++++++++ 7 files changed, 1154 insertions(+), 2 deletions(-) create mode 100644 nanovdb/nanovdb/python/examples/cupy_rawkernel.py create mode 100644 nanovdb/nanovdb/python/examples/gpu_load_inspect.py create mode 100644 nanovdb/nanovdb/python/examples/numba_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/triton_kernel.py create mode 100644 nanovdb/nanovdb/python/test/TestGpuInterop.py diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 9e99e8025e..d9133e896c 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -117,12 +117,22 @@ if(NANOVDB_BUILD_PYTHON_UNITTESTS) COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/test/TestNanoVDB.py -v WORKING_DIRECTORY "${NANOVDB_PYTHON_WORKING_DIR}") + # GPU interop suite. Self-skips when the build has no CUDA or no GPU is + # present (so it is harmless to register unconditionally), but is a + # separate test file that the pytest_nanovdb command does not discover, + # so register it as its own ctest test mirroring the block above. + add_test(NAME pytest_nanovdb_gpu_interop + COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/test/TestGpuInterop.py -v + WORKING_DIRECTORY "${NANOVDB_PYTHON_WORKING_DIR}") + if(WIN32) set(PYTHONPATH "$ENV{PYTHONPATH};${NANOVDB_PYTHON_WORKING_DIR}") string(REPLACE "\\;" ";" PYTHONPATH "${PYTHONPATH}") string(REPLACE ";" "\\;" PYTHONPATH "${PYTHONPATH}") - set_tests_properties(pytest_nanovdb PROPERTIES ENVIRONMENT "PYTHONPATH=${PYTHONPATH}") + set_tests_properties(pytest_nanovdb pytest_nanovdb_gpu_interop + PROPERTIES ENVIRONMENT "PYTHONPATH=${PYTHONPATH}") else() - set_tests_properties(pytest_nanovdb PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${NANOVDB_PYTHON_WORKING_DIR}") + set_tests_properties(pytest_nanovdb pytest_nanovdb_gpu_interop + PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${NANOVDB_PYTHON_WORKING_DIR}") endif() endif() diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index 0e70b784e4..0cc9ec5934 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -28,6 +28,19 @@ PYTHONPATH=. python /path/to/.py | [`quantize.py`](quantize.py) | Quantize a `NanoGrid` through `nanovdb.tools.createNanoGridFp{4,8,16,N}`. Shows fixed-width quantization with dithering and variable-width `FpN` with both `AbsDiff` and `RelDiff` oracles. | | [`validate.py`](validate.py) | `nanovdb.tools.validateGrid` / `validateGrids`, `checkGrid`, `isValid`, and the `evalChecksum` / `validateChecksum` / `updateChecksum` round-trip. | +### GPU examples + +These require a CUDA build of `nanovdb` and a CUDA-capable GPU; each +self-skips with a printed message when the build, GPU, or the optional +GPU-array framework it uses is unavailable. + +| Script | What it shows | +| --- | --- | +| [`gpu_load_inspect.py`](gpu_load_inspect.py) | `nanovdb.io.deviceReadGrid` → `handle.deviceUpload()` → zero-copy `cupy.asarray(handle)` view of the device buffer; the host `grid(n)` vs device `deviceGrid(n)` `data_ptr()` split. Requires CuPy. | +| [`cupy_rawkernel.py`](cupy_rawkernel.py) | A `cupy.RawKernel` that `#include ` (compiled with `nanovdb.cuda.compile_options()`) and reads a `const nanovdb::NanoGrid*` straight from `grid.data_ptr()`. Documents the device-pointer ABI. Requires CuPy. | +| [`numba_cuda.py`](numba_cuda.py) | Adopting a NanoVDB device buffer as a zero-copy `numba.cuda` array (Numba can't parse the C++ ABI, so it operates on the raw buffer). Requires Numba. | +| [`triton_kernel.py`](triton_kernel.py) | Handing a NanoVDB device buffer to a Triton kernel via a zero-copy `torch.from_dlpack` tensor. Requires Triton + PyTorch. | + For full API signatures and per-argument docstrings, use Python's `help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`. The bindings ship `.pyi` type stubs for IDE / type-checker support. @@ -44,3 +57,162 @@ Factory functions that return a device handle are unaffected — they never name the class — so `nanovdb.io.deviceReadGrid(s)`, `nanovdb.tools.cuda.create*`, `nanovdb.tools.cuda.pointsToRGBA8Grid`, and the `deviceUpload` / `deviceDownload` / `deviceGrid` handle methods continue to work unchanged. + +## GPU / CUDA + +The CUDA surface is only present when `nanovdb` is built with CUDA. Gate any +GPU code on both checks (compiled-with-CUDA and a GPU actually present): + +```python +import nanovdb +if nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable(): + ... # nanovdb.cuda / nanovdb.tools.cuda are safe to use +``` + +In a non-CUDA build the `nanovdb.cuda` submodule does not exist, so guard with +`hasattr(nanovdb, "cuda")` (or the checks above) before touching it. + +`nanovdb.cuda` and `nanovdb.tools.cuda` are **attributes** of the package, not +importable submodules. Use them via attribute access after `import nanovdb` +(`import nanovdb.tools.cuda` raises `ModuleNotFoundError`): + +```python +import nanovdb +TC = nanovdb.tools.cuda # ok +# import nanovdb.tools.cuda # NOT ok +``` + +### Namespace layout: `nanovdb.cuda` vs `nanovdb.tools.cuda` + +- **`nanovdb.cuda`** — CUDA *infrastructure* (mirrors the C++ `nanovdb::cuda` + namespace): the buffer / handle types and device plumbing. + `DeviceBuffer`, `DeviceGridHandle`, `UnifiedBuffer`, `UnifiedGridHandle`, + `DeviceMesh`, `DeviceStreamMap`, `DeviceResource`, `TempDevicePool`, + `DeviceNodeManagerHandle`, `createDeviceNodeManager`, and + `compile_options`. +- **`nanovdb.tools.cuda`** — device *algorithms* (mirrors + `nanovdb::tools::cuda`): point / voxel rasterizers (`pointsToGrid`, + `voxelsTo{OnIndex,Index,RGBA8}Grid`, `pointsToRGBA8Grid`), morphology / + topology (`dilateGrid`, `coarsenGrid`, `refineGrid`, `pruneGrid`, + `mergeGrids`), index utilities (`indexToGrid`, `addBlindData`), in-place + device QC (`updateGridStats`, `updateChecksum`, `evalChecksum`, + `validateChecksum`, `isValid`), `signedFloodFill`, `sampleFromVoxels`, + `buildVoxelBlockManager`, and the multi-GPU `Distributed*PointsToGrid` + converters. + +### Building a device grid + +```python +h = nanovdb.tools.createLevelSetSphere(radius=100.0, voxelSize=1.0) # host +fg = h.grid(0) # host FloatGrid +onh = nanovdb.tools.createOnIndexGrid(fg) # host OnIndexGrid handle +nanovdb.io.writeGrid("sphere.nvdb", onh) +dh = nanovdb.io.deviceReadGrid("sphere.nvdb") # nanovdb.cuda.DeviceGridHandle +dh.deviceUpload(0, True) # stream=0, sync=True +dg = dh.deviceGrid(0) # DEVICE grid; feed to tools.cuda.* +``` + +### Zero-copy interop: CAI, DLPack, and `data_ptr()` + +`DeviceGridHandle`, `DeviceBuffer`, and `UnifiedBuffer` expose the whole +device buffer as a 1-D `uint8` array through both the **CUDA Array Interface +(v3)** (`__cuda_array_interface__`) and **DLPack** (`__dlpack__` / +`__dlpack_device__`). So they bridge to CuPy / PyTorch / Numba with no copy: + +```python +import cupy as cp +dh.deviceUpload(0, True) +buf = cp.asarray(dh) # zero-copy; buf.data.ptr == dh.device_ptr() +buf = cp.from_dlpack(dh) # same, via DLPack +assert buf.nbytes == dh.size() +``` + +The data pointer is null and the array empty until `deviceUpload` runs. +`UnifiedGridHandle` intentionally does **not** expose the CAI / DLPack bridges +(it is the managed-memory handle returned by `Distributed*PointsToGrid`); read +its grid through `deviceGrid(n)` and the `tools.cuda.*` ops instead. + +Every typed grid exposes `grid.data_ptr() -> int`. **It is a HOST pointer when +the grid came from `handle.grid(n)` and a DEVICE pointer when it came from +`handle.deviceGrid(n)`** — the grid object itself cannot tell host from device, +so provenance is the caller's responsibility. The device pointer is the base +of a `nanovdb::NanoGrid` in GPU memory (the same C++ ABI), which is +exactly what you pass to a custom CUDA kernel (see `cupy_rawkernel.py`). + +> **Caveat — host accessors segfault on device grids.** A grid from +> `deviceGrid(n)` holds a DEVICE pointer; calling a host-side accessor on it +> (e.g. `dg.getAccessor().getValue(...)`) dereferences GPU memory on the CPU +> and **crashes the process** (SIGSEGV). Host reads are only legal on +> `grid(n)` / host grids. Feed `deviceGrid(n)` **only** to `tools.cuda.*` +> device entry points, or read its bytes via `data_ptr()` / the CAI / DLPack +> buffer from a device kernel. + +### Streams + +Stream arguments everywhere are **raw CUDA stream handles passed as Python +ints** (`0` == the default stream). `deviceUpload` / `deviceDownload` and the +`tools.cuda.*` ops all take a trailing `stream` int. From CuPy, pass +`stream.ptr`: + +```python +s = cp.cuda.Stream(non_blocking=True) +h = nanovdb.tools.cuda.pointsToGrid(points, 1.0, s.ptr) +``` + +### Wrapping external memory (`from_external`) + +`nanovdb.cuda.DeviceBuffer.from_external(size, gpu_ptr, cpu_ptr)` builds a +**non-owning** wrapper around memory you already allocated (e.g. a CuPy / +PyTorch buffer); it never frees, and the caller must keep the source alive. A +null `gpu_ptr` is rejected. Pair it with +`DeviceGridHandle.from_buffer(buffer)` (which **moves** the buffer, leaving it +empty, and validates the NanoVDB header) for a fully zero-copy adopt of grid +bytes you produced elsewhere. + +### Managed memory for the distributed pipeline + +`Distributed{Points,IndexPoints,RGBA8Points}ToGrid.getHandle(voxels)` requires +the `(N, 3)` int32 coordinate array to live in **CUDA managed (unified)** +memory and returns a `nanovdb.cuda.UnifiedGridHandle`. With CuPy, route the +allocation through the managed allocator: + +```python +prev = cp.cuda.get_allocator() +cp.cuda.set_allocator(cp.cuda.malloc_managed) +try: + voxels = cp.asarray(coords_int32) # managed (N,3) int32 + mesh = nanovdb.cuda.DeviceMesh() # must outlive the converter + conv = nanovdb.tools.cuda.DistributedPointsToGrid(mesh, 1.0, (0, 0, 0)) + uh = conv.getHandle(voxels) # UnifiedGridHandle (OnIndex) +finally: + cp.cuda.set_allocator(prev) +``` + +A plain (device-pool) array does not satisfy the `cuda_managed` constraint. + +### Compiling custom kernels against the bundled headers + +`nanovdb.cuda.compile_options(*extra)` returns the NanoVDB include flag +(`-I`) followed by any extra flags, for feeding NVRTC / a runtime CUDA +compiler so your kernel compiles against the same headers as the wheel: + +```python +opts = nanovdb.cuda.compile_options("-std=c++17") +kernel = cp.RawKernel(src, "my_kernel", options=opts, backend="nvrtc") +``` + +The include dir is only physically present in an installed wheel; in an +in-source dev build tree the path resolves but does not exist (see +`cupy_rawkernel.py` for a `NANOVDB_INCLUDE` fallback). + +### Future: pluggable memory resources (roadmap) + +The device buffers above own their allocations directly. A planned evolution is +a **pluggable memory-resource** model (à la a polymorphic allocator), so device +buffers can draw from a caller-supplied pool / async / pinned allocator — +e.g. RAPIDS Memory Manager–style resources. See Mark Harris's design sketch: +. The +following names are **reserved** for that work and should not be used today: +`Resource`, `MemoryResource`, `AsyncResource`, `PinnedResource`, +`ResourceDeviceBuffer`, `DeviceBuffer2`, `default_resource`, +`set_default_resource`. diff --git a/nanovdb/nanovdb/python/examples/cupy_rawkernel.py b/nanovdb/nanovdb/python/examples/cupy_rawkernel.py new file mode 100644 index 0000000000..09bef2008c --- /dev/null +++ b/nanovdb/nanovdb/python/examples/cupy_rawkernel.py @@ -0,0 +1,103 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Run a custom CUDA kernel over a NanoVDB device grid with CuPy. + +This shows the full "bring your own kernel" path: compile a CUDA kernel +that ``#include `` against the SAME headers the nanovdb +wheel was built with (via ``nanovdb.cuda.compile_options()``), then launch +it on the device grid pointer obtained from ``grid.data_ptr()``. + +Device-pointer ABI (read this before writing a kernel): + +* ``handle.deviceGrid(n).data_ptr()`` is a raw DEVICE pointer (a Python + int) to the base of a ``nanovdb::NanoGrid`` in GPU memory. For a + float grid that is ``const nanovdb::NanoGrid*``. +* Pass it to a kernel as a plain pointer argument. With CuPy RawKernel the + argument tuple takes the int directly (CuPy forwards it as a pointer). +* The grid layout in device memory is exactly the C++ ``NanoGrid`` + ABI, so the same accessor / sampler code you would write in C++ works + inside the kernel. +* ``grid.data_ptr()`` cannot tell host from device — only pass a pointer + from ``deviceGrid(n)`` to a device kernel. A host pointer from + ``grid(n)`` would dereference host memory on the GPU (garbage / fault). + +``nanovdb.cuda.compile_options(*extra)`` returns ``-I`` followed +by any extra flags. The header dir is only physically present in an +installed wheel; in an in-source dev build tree it resolves but does not +exist, so this example falls back to a ``NANOVDB_INCLUDE`` env var hint. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python cupy_rawkernel.py +""" +import os + +import nanovdb + +KERNEL_SRC = r""" +#include + +// d_grid is the raw device pointer from FloatGrid.data_ptr(); out is a +// 2-float device buffer that receives [value@origin, activeVoxelCount]. +extern "C" __global__ +void inspect_float_grid(const nanovdb::NanoGrid* d_grid, float* out) +{ + auto acc = d_grid->getAccessor(); + out[0] = acc.getValue(nanovdb::Coord(0, 0, 0)); + out[1] = static_cast(d_grid->activeVoxelCount()); +} +""" + + +def _include_options(): + """compile_options(), falling back to $NANOVDB_INCLUDE in a dev tree.""" + opts = list(nanovdb.cuda.compile_options("-std=c++17")) + inc_dir = opts[0][2:] # strip the leading -I + if not os.path.isdir(inc_dir): + env_inc = os.environ.get("NANOVDB_INCLUDE") + if env_inc and os.path.isdir(env_inc): + opts[0] = f"-I{env_inc}" + else: + print(f"NanoVDB headers not found at {inc_dir!r} (expected in an " + "installed wheel). Set NANOVDB_INCLUDE to the dir that " + "contains nanovdb/NanoVDB.h to run from a source tree.") + return None + return tuple(opts) + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + options = _include_options() + if options is None: + return + + # Build a float level-set sphere directly on the device. + handle = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Float, 20) + handle.deviceUpload(0, True) + device_grid = handle.deviceGrid(0) + print(f"Device FloatGrid at {hex(device_grid.data_ptr())}") + + kernel = cp.RawKernel( + KERNEL_SRC, "inspect_float_grid", options=options, backend="nvrtc") + + out = cp.zeros(2, dtype=cp.float32) + # Launch with the raw device-grid pointer as the first argument. + kernel((1,), (1,), (device_grid.data_ptr(), out.data.ptr)) + cp.cuda.runtime.deviceSynchronize() + + value, active = cp.asnumpy(out) + print(f" kernel read value@origin = {value}") + print(f" kernel read activeVoxelCount = {int(active)}") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/gpu_load_inspect.py b/nanovdb/nanovdb/python/examples/gpu_load_inspect.py new file mode 100644 index 0000000000..98611da35d --- /dev/null +++ b/nanovdb/nanovdb/python/examples/gpu_load_inspect.py @@ -0,0 +1,101 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Load a NanoVDB grid onto the GPU and inspect its device buffer zero-copy. + +This is the GPU counterpart to ``load_inspect.py``. It builds a grid, +writes it to disk, reads it straight onto the device with +``nanovdb.io.deviceReadGrid``, uploads it with ``deviceUpload``, and then +views the whole device buffer as a CuPy array WITHOUT copying via the +CUDA Array Interface (``cupy.asarray(handle)``). + +Key facts demonstrated: + +* ``handle.deviceUpload(stream, sync)`` takes a RAW CUDA stream handle as + a Python int (0 == the default stream). +* ``handle.__cuda_array_interface__`` (and ``__dlpack__``) expose the + whole device buffer as a 1-D ``uint8`` array; the data pointer is null + until ``deviceUpload`` runs. +* ``cupy.asarray(handle)`` aliases ``handle.device_ptr()`` with no copy; + ``arr.nbytes == handle.size()``. +* A single handle exposes BOTH a host grid via ``handle.grid(n)`` and a + device grid via ``handle.deviceGrid(n)``. They have different + ``data_ptr()`` values and the grid object cannot tell host from device + (see the docstring on ``Grid.data_ptr``) — feed ``deviceGrid(n)`` only + to ``nanovdb.tools.cuda.*`` device entry points. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python gpu_load_inspect.py +""" +import os +import tempfile + +import nanovdb + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + # Build a small level-set sphere on the host, then write it so we can + # read it back straight onto the device. + host_handle = nanovdb.tools.createLevelSetSphere( + radius=20.0, voxelSize=1.0, name="gpu_sphere") + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + nanovdb.io.writeGrid(tmp.name, host_handle) + + try: + # deviceReadGrid returns a nanovdb.cuda.DeviceGridHandle. + handle = nanovdb.io.deviceReadGrid(tmp.name) + finally: + os.unlink(tmp.name) + + print(f"Read device handle: gridCount={handle.gridCount()}, " + f"size={handle.size()} bytes") + print(f" device_ptr before upload = {handle.device_ptr()} " + f"(0 == not uploaded yet)") + print(f" deviceGrid(0) before upload = {handle.deviceGrid(0)}") + + # Upload to the device on the default stream, synchronously. + handle.deviceUpload(0, True) + print(f" device_ptr after upload = {hex(handle.device_ptr())}") + + # Zero-copy view of the whole device buffer as a CuPy uint8 array. + buf = cp.asarray(handle) + print(f" cupy.asarray(handle): shape={buf.shape}, dtype={buf.dtype}, " + f"nbytes={buf.nbytes}") + print(f" aliases device_ptr (no copy): " + f"{int(buf.data.ptr) == handle.device_ptr()}") + + # The host grid and the device grid share the handle but differ in + # provenance: the host grid's data_ptr is a host address, the device + # grid's data_ptr is a device address. The grid object itself cannot + # tell them apart. + host_grid = handle.grid(0) + device_grid = handle.deviceGrid(0) + print(f" host grid.data_ptr() = {hex(host_grid.data_ptr())} (CPU)") + print(f" device grid.data_ptr() = {hex(device_grid.data_ptr())} (GPU)") + print(f" device grid.data_ptr() == handle.device_ptr(): " + f"{device_grid.data_ptr() == handle.device_ptr()}") + + # The device grid is the input to nanovdb.tools.cuda.* ops. Validate + # it entirely on the device. + print(f" tools.cuda.isValid(device_grid) = " + f"{nanovdb.tools.cuda.isValid(device_grid)}") + + print("WARNING: calling a host-side accessor (e.g. " + "device_grid.getAccessor().getValue(...)) on a DEVICE grid " + "dereferences GPU memory on the CPU and SEGFAULTS. Use host_grid " + "for host reads.") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/numba_cuda.py b/nanovdb/nanovdb/python/examples/numba_cuda.py new file mode 100644 index 0000000000..4a0d8832ac --- /dev/null +++ b/nanovdb/nanovdb/python/examples/numba_cuda.py @@ -0,0 +1,86 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Consume a NanoVDB device grid from a Numba CUDA kernel. + +Numba does not parse the C++ NanoVDB header, so it cannot use the +``NanoGrid`` accessor types directly the way the CuPy RawKernel example +(``cupy_rawkernel.py``) does. The realistic Numba pattern is therefore to +operate on the zero-copy *device arrays* NanoVDB hands back — the whole +grid buffer (CAI / DLPack on the handle) or the typed per-leaf / blind-data +buffers — rather than to dereference the ``NanoGrid`` ABI from JIT'd code. + +Device-pointer ABI recap (see cupy_rawkernel.py for the C++ ABI route): + +* ``handle.deviceGrid(n).data_ptr()`` is a raw DEVICE pointer (Python int) + to a ``nanovdb::NanoGrid``. Decoding the NanoGrid layout from a + Numba kernel means re-implementing the offset math by hand — out of + scope for this example. +* The supported, zero-copy Numba path: wrap a NanoVDB buffer that already + exposes ``__cuda_array_interface__`` (the DeviceGridHandle, a + UnifiedBuffer, or the VoxelBlockManager firstLeafID / jumpMap DLPack + capsules) with ``numba.cuda.as_cuda_array`` and process the raw bytes / + indices in a kernel. +* Provenance is the caller's responsibility: only feed device pointers / + device CAI buffers to device kernels. A host pointer from ``grid(n)`` + would fault on the GPU. + +This example wraps the whole device grid buffer as a uint8 CUDA array and +runs a trivial reduction over it to show the interop plumbing. Requires +Numba (with CUDA support) and a CUDA-capable GPU; it skips with a message +when Numba is not installed. + +Run with: python numba_cuda.py +""" +import nanovdb + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + from numba import cuda + except ImportError: + print("This example requires Numba (with CUDA support). Install it " + "with: pip install numba") + return + if not cuda.is_available(): + print("Numba reports no CUDA device available. Skipping.") + return + + # Build a float level-set sphere on the device. + handle = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Float, 20) + handle.deviceUpload(0, True) + print(f"Device grid buffer: {handle.size()} bytes at " + f"{hex(handle.device_ptr())}") + + # Zero-copy: Numba adopts the handle's __cuda_array_interface__ (the + # whole device buffer as 1-D uint8) with no copy. + dev_bytes = cuda.as_cuda_array(handle) + print(f" numba CUDA array: shape={dev_bytes.shape}, " + f"dtype={dev_bytes.dtype}") + + # Trivial per-element kernel over the raw buffer (XOR-fold into a + # device scalar) just to demonstrate launching JIT'd Numba code on a + # NanoVDB-owned device buffer. + @cuda.jit + def fold_xor(buf, out): + i = cuda.grid(1) + if i < buf.size: + cuda.atomic.xor(out, 0, buf[i]) + + import numpy as np + out = cuda.to_device(np.zeros(1, dtype=np.uint8)) + threads = 256 + blocks = (dev_bytes.size + threads - 1) // threads + fold_xor[blocks, threads](dev_bytes, out) + cuda.synchronize() + print(f" XOR-fold of device buffer = {int(out.copy_to_host()[0])}") + print(" (Decoding the NanoGrid ABI itself from Numba requires hand-" + "rolled offset math; use the CuPy RawKernel route for C++ " + "accessor access — see cupy_rawkernel.py.)") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/triton_kernel.py b/nanovdb/nanovdb/python/examples/triton_kernel.py new file mode 100644 index 0000000000..bfc519dbe4 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/triton_kernel.py @@ -0,0 +1,88 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Consume a NanoVDB device buffer from a Triton kernel. + +Like Numba (``numba_cuda.py``), Triton does not parse the C++ NanoVDB +header and has no notion of the ``NanoGrid`` ABI. Triton kernels operate +on flat tensors addressed by pointer + offset, so the natural NanoVDB +interop is over the zero-copy *device buffers* NanoVDB exposes, not the +structured grid. + +Device-pointer ABI recap (see cupy_rawkernel.py for the C++ ABI route): + +* ``handle.deviceGrid(n).data_ptr()`` is a raw DEVICE pointer (Python int) + to a ``nanovdb::NanoGrid``. Triton cannot decode that layout, so + this example does not pass the grid pointer to the kernel directly. +* Triton works through a framework tensor: take the handle's zero-copy + ``__cuda_array_interface__`` / ``__dlpack__`` view (e.g. + ``torch.from_dlpack(handle)``) and hand the resulting tensor to the + kernel. Triton then sees a normal contiguous device tensor. +* Provenance is the caller's responsibility: only device buffers go to + device kernels. + +This example wraps the device grid buffer as a Torch CUDA tensor and runs +a trivial Triton element-wise kernel over it. Requires Triton, PyTorch +(with CUDA), and a CUDA-capable GPU; it skips with a message if any are +missing. + +Run with: python triton_kernel.py +""" +import nanovdb + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import triton + import triton.language as tl + except ImportError: + print("This example requires Triton. Install it with: " + "pip install triton") + return + try: + import torch + except ImportError: + print("This example requires PyTorch (Triton kernels are launched " + "over Torch tensors). Install it with: pip install torch") + return + if not torch.cuda.is_available(): + print("PyTorch reports no CUDA device available. Skipping.") + return + + # Build a float level-set sphere on the device. + handle = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Float, 20) + handle.deviceUpload(0, True) + print(f"Device grid buffer: {handle.size()} bytes at " + f"{hex(handle.device_ptr())}") + + # Zero-copy: Torch adopts the handle's DLPack device buffer (1-D uint8). + buf = torch.from_dlpack(handle) + print(f" torch tensor: shape={tuple(buf.shape)}, dtype={buf.dtype}, " + f"is_cuda={buf.is_cuda}") + + @triton.jit + def count_nonzero(x_ptr, out_ptr, n, BLOCK: tl.constexpr): + pid = tl.program_id(0) + offs = pid * BLOCK + tl.arange(0, BLOCK) + mask = offs < n + x = tl.load(x_ptr + offs, mask=mask, other=0) + tl.atomic_add(out_ptr, tl.sum((x != 0).to(tl.int32))) + + n = buf.numel() + out = torch.zeros(1, dtype=torch.int32, device="cuda") + BLOCK = 1024 + grid = (triton.cdiv(n, BLOCK),) + count_nonzero[grid](buf, out, n, BLOCK=BLOCK) + torch.cuda.synchronize() + print(f" Triton counted {int(out.item())} non-zero bytes in the device " + "grid buffer") + print(" (Decoding the NanoGrid ABI itself is out of scope for Triton; " + "use the CuPy RawKernel route for C++ accessor access — see " + "cupy_rawkernel.py.)") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/test/TestGpuInterop.py b/nanovdb/nanovdb/python/test/TestGpuInterop.py new file mode 100644 index 0000000000..38a3308a4e --- /dev/null +++ b/nanovdb/nanovdb/python/test/TestGpuInterop.py @@ -0,0 +1,592 @@ +#!/usr/bin/env python +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 + +"""GPU interop unit tests for the NanoVDB Python bindings. + +These exercise the ``nanovdb.cuda`` and ``nanovdb.tools.cuda`` surface: +the CUDA-Array-Interface (CAI v3) and DLPack zero-copy bridges on +``DeviceBuffer`` / ``DeviceGridHandle`` / ``UnifiedBuffer``, the +``grid.data_ptr()`` host/device pointer ABI, raw-stream arguments, the +device-grid morphology / index / QC ops, the device NodeManager and +VoxelBlockManager, and the multi-GPU ``DistributedPointsToGrid`` +pipeline over managed (unified) memory. + +The whole module self-skips when the extension was built without CUDA +or when no CUDA-capable GPU is present, mirroring the device-test +gating in TestNanoVDB.py. Individual tests additionally skip when CuPy +(the only GPU-array framework assumed present) or PyTorch is missing. + +Run directly with: python TestGpuInterop.py -v +""" + +import os +import subprocess +import sys +import tempfile +import unittest + +# If on Windows, add required dll directories from our binary build tree +# (mirrors the bootstrap in TestNanoVDB.py). +if 'add_dll_directory' in dir(os): + for p in os.environ.get('PATH', '').split(os.pathsep): + if os.path.isdir(p): + try: + os.add_dll_directory(p) + except OSError: + pass + +import nanovdb + + +def _require_cupy(test): + """Skip ``test`` (returns the cupy module) unless CuPy is importable.""" + try: + import cupy as cp + except ImportError: + test.skipTest("CuPy not installed") + return cp + + +def _build_device_onindex_grid(radius=20.0): + """Build, write, read-back, and upload a device OnIndex grid. + + Returns (handle, deviceGrid) where handle is a + nanovdb.cuda.DeviceGridHandle that has been deviceUpload()ed and + deviceGrid is the DEVICE OnIndexGrid (data_ptr is a device pointer). + The handle must be kept alive for as long as deviceGrid is used. + """ + h = nanovdb.tools.createLevelSetSphere(radius=radius, voxelSize=1.0) + fg = h.grid(0) + onh = nanovdb.tools.createOnIndexGrid(fg) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + nanovdb.io.writeGrid(tmp.name, onh) + try: + dh = nanovdb.io.deviceReadGrid(tmp.name) + finally: + os.unlink(tmp.name) + dh.deviceUpload(0, True) + return dh, dh.deviceGrid(0) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestCompileOptions(unittest.TestCase): + """nanovdb.cuda.compile_options for feeding a runtime CUDA compiler.""" + + def test_include_flag_first(self): + opts = nanovdb.cuda.compile_options() + self.assertIsInstance(opts, tuple) + self.assertEqual(len(opts), 1) + self.assertTrue(opts[0].startswith("-I")) + # The include flag points at the bundled NanoVDB header dir. + self.assertTrue(opts[0].endswith(os.path.join("nanovdb", "include"))) + + def test_extra_flags_appended_in_order(self): + opts = nanovdb.cuda.compile_options("-std=c++17", "-O3") + self.assertEqual(opts[0][:2], "-I") + self.assertEqual(opts[1:], ("-std=c++17", "-O3")) + + def test_top_level_alias_matches(self): + # nanovdb._cuda_compile_options is the underlying closure. + self.assertEqual( + nanovdb.cuda.compile_options(), nanovdb._cuda_compile_options()) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceBufferInterop(unittest.TestCase): + """DeviceBuffer CAI v3 / DLPack and from_external wrapping.""" + + def test_from_external_wraps_managed_memory(self): + cp = _require_cupy(self) + prev = cp.cuda.get_allocator() + cp.cuda.set_allocator(cp.cuda.malloc_managed) + try: + buf = cp.zeros(256, dtype=cp.uint8) + gpu_ptr = int(buf.data.ptr) + # Managed memory: a single pointer is valid on both host and + # device, so host_ptr == device_ptr is legal here. + ext = nanovdb.cuda.DeviceBuffer.from_external(256, gpu_ptr, gpu_ptr) + self.assertEqual(ext.size(), 256) + self.assertEqual(ext.device_ptr(), gpu_ptr) + self.assertEqual(ext.host_ptr(), gpu_ptr) + finally: + cp.cuda.set_allocator(prev) + + def test_from_external_rejects_null_device_pointer(self): + # gpu_ptr == 0 is a usage error: there is no device memory to wrap. + with self.assertRaises(ValueError): + nanovdb.cuda.DeviceBuffer.from_external(256, 0, 0) + with self.assertRaises(ValueError): + nanovdb.cuda.DeviceBuffer.from_external(256, 0, 12345) + + def test_cuda_array_interface_v3(self): + cp = _require_cupy(self) + prev = cp.cuda.get_allocator() + cp.cuda.set_allocator(cp.cuda.malloc_managed) + try: + buf = cp.zeros(256, dtype=cp.uint8) + gpu_ptr = int(buf.data.ptr) + ext = nanovdb.cuda.DeviceBuffer.from_external(256, gpu_ptr, gpu_ptr) + cai = ext.__cuda_array_interface__ + self.assertEqual(cai["version"], 3) + self.assertEqual(cai["typestr"], "|u1") + self.assertEqual(cai["shape"], (256,)) + self.assertEqual(cai["data"][0], gpu_ptr) + # cupy zero-copy view aliases the same device pointer. + arr = cp.asarray(ext) + self.assertEqual(arr.nbytes, ext.size()) + self.assertEqual(int(arr.data.ptr), ext.device_ptr()) + finally: + cp.cuda.set_allocator(prev) + + def test_dlpack_round_trip(self): + cp = _require_cupy(self) + prev = cp.cuda.get_allocator() + cp.cuda.set_allocator(cp.cuda.malloc_managed) + try: + buf = cp.zeros(256, dtype=cp.uint8) + gpu_ptr = int(buf.data.ptr) + ext = nanovdb.cuda.DeviceBuffer.from_external(256, gpu_ptr, gpu_ptr) + # kDLCUDA == 2 + self.assertEqual(ext.__dlpack_device__()[0], 2) + arr = cp.from_dlpack(ext) + self.assertEqual(arr.nbytes, ext.size()) + self.assertEqual(int(arr.data.ptr), ext.device_ptr()) + finally: + cp.cuda.set_allocator(prev) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceGridHandleInterop(unittest.TestCase): + """DeviceGridHandle CAI/DLPack, the host/device data_ptr ABI, and + from_buffer zero-copy adoption.""" + + def test_cai_exposes_device_buffer_after_upload(self): + dh, dg = _build_device_onindex_grid(20.0) + # After upload the whole device buffer is exposed as 1-D uint8. + cai = dh.__cuda_array_interface__ + self.assertEqual(cai["version"], 3) + self.assertEqual(cai["typestr"], "|u1") + self.assertEqual(cai["shape"], (dh.size(),)) + self.assertNotEqual(dh.device_ptr(), 0) + self.assertEqual(cai["data"][0], dh.device_ptr()) + + def test_cupy_zero_copy_aliases_device_ptr(self): + cp = _require_cupy(self) + dh, dg = _build_device_onindex_grid(20.0) + arr = cp.asarray(dh) + self.assertEqual(arr.nbytes, dh.size()) + self.assertEqual(int(arr.data.ptr), dh.device_ptr()) + # The typed device grid's data_ptr() points at the same buffer base. + self.assertEqual(dg.data_ptr(), dh.device_ptr()) + + def test_dlpack_zero_copy(self): + cp = _require_cupy(self) + dh, dg = _build_device_onindex_grid(20.0) + arr = cp.from_dlpack(dh) + self.assertEqual(arr.nbytes, dh.size()) + self.assertEqual(int(arr.data.ptr), dh.device_ptr()) + + def test_host_vs_device_data_ptr_ambiguity(self): + # A single handle exposes BOTH a host grid() and a device + # deviceGrid() after upload; the data_ptr() values differ and the + # grid object cannot tell host from device — provenance is the + # caller's responsibility (see the docstring on data_ptr()). + dh, dg = _build_device_onindex_grid(20.0) + hg = dh.grid(0) + self.assertIsNotNone(hg) + self.assertIsNotNone(dg) + self.assertNotEqual(hg.data_ptr(), dg.data_ptr()) + self.assertEqual(dg.data_ptr(), dh.device_ptr()) + + def test_host_accessor_on_device_grid_segfaults_in_subprocess(self): + # Calling a HOST-side accessor on a grid from deviceGrid(n) + # dereferences device memory on the CPU and crashes the process. + # Run it in a subprocess so the SIGSEGV does not abort the test + # session, and assert the documented crash contract. + code = ( + "import nanovdb, tempfile, os\n" + "h = nanovdb.tools.createLevelSetSphere(radius=20.0, voxelSize=1.0)\n" + "fg = h.grid(0)\n" + "onh = nanovdb.tools.createOnIndexGrid(fg)\n" + "t = tempfile.NamedTemporaryFile(suffix='.nvdb', delete=False)\n" + "t.close()\n" + "nanovdb.io.writeGrid(t.name, onh)\n" + "dh = nanovdb.io.deviceReadGrid(t.name)\n" + "os.unlink(t.name)\n" + "dh.deviceUpload(0, True)\n" + "dg = dh.deviceGrid(0)\n" + "acc = dg.getAccessor()\n" + "acc.getValue(nanovdb.math.Coord(0, 0, 0))\n" + ) + r = subprocess.run([sys.executable, "-c", code], capture_output=True) + # Killed by a signal => negative return code; -11 is SIGSEGV. Any + # crash (negative) satisfies the "this is illegal" contract. + self.assertLess( + r.returncode, 0, + "host accessor on a device grid was expected to crash, " + f"got returncode {r.returncode}") + + def test_from_buffer_adopts_managed_buffer(self): + cp = _require_cupy(self) + dh, dg = _build_device_onindex_grid(20.0) + src = cp.asarray(dh) # zero-copy device view of a valid grid + prev = cp.cuda.get_allocator() + cp.cuda.set_allocator(cp.cuda.malloc_managed) + try: + # Managed buffer holding a copy of the valid grid bytes; a + # single pointer serves as both host and device pointer. + mbuf = cp.empty(int(dh.size()), dtype=cp.uint8) + mbuf[:] = src + cp.cuda.runtime.deviceSynchronize() + ptr = int(mbuf.data.ptr) + ext = nanovdb.cuda.DeviceBuffer.from_external( + int(dh.size()), ptr, ptr) + adopted = nanovdb.cuda.DeviceGridHandle.from_buffer(ext) + self.assertEqual(adopted.gridType(0), nanovdb.GridType.OnIndex) + self.assertEqual(adopted.size(), dh.size()) + # from_buffer MOVES the buffer; the source is left empty. + self.assertEqual(ext.size(), 0) + finally: + cp.cuda.set_allocator(prev) + + def test_from_buffer_rejects_garbage(self): + cp = _require_cupy(self) + prev = cp.cuda.get_allocator() + cp.cuda.set_allocator(cp.cuda.malloc_managed) + try: + mbuf = cp.zeros(512, dtype=cp.uint8) # not a NanoVDB grid header + ptr = int(mbuf.data.ptr) + ext = nanovdb.cuda.DeviceBuffer.from_external(512, ptr, ptr) + with self.assertRaises(RuntimeError): + nanovdb.cuda.DeviceGridHandle.from_buffer(ext) + finally: + cp.cuda.set_allocator(prev) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestUnifiedBufferInterop(unittest.TestCase): + """UnifiedBuffer create / capacity / zero-copy.""" + + def test_create_size_equals_capacity(self): + ub = nanovdb.cuda.UnifiedBuffer.create(1024) + self.assertEqual(ub.size(), 1024) + self.assertEqual(ub.capacity(), 1024) + self.assertFalse(ub.isEmpty()) + + def test_create_with_reserved_capacity(self): + ub = nanovdb.cuda.UnifiedBuffer.create(1024, 4096) + self.assertEqual(ub.size(), 1024) + self.assertEqual(ub.capacity(), 4096) + + def test_resize_within_capacity_keeps_page_table(self): + ub = nanovdb.cuda.UnifiedBuffer.create(1024, 4096) + ub.resize(2048) + self.assertEqual(ub.size(), 2048) + self.assertEqual(ub.capacity(), 4096) + + def test_cupy_zero_copy(self): + cp = _require_cupy(self) + ub = nanovdb.cuda.UnifiedBuffer.create(2048) + cai = ub.__cuda_array_interface__ + self.assertEqual(cai["version"], 3) + self.assertEqual(cai["shape"], (2048,)) + arr = cp.asarray(ub) + self.assertEqual(arr.nbytes, ub.size()) + arr2 = cp.from_dlpack(ub) + self.assertEqual(arr2.nbytes, ub.size()) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestPointAndVoxelRasterizers(unittest.TestCase): + """tools.cuda point / voxel rasterizers, including raw-stream args.""" + + def test_points_to_grid_world_float(self): + cp = _require_cupy(self) + import numpy as np + pts = cp.asarray(np.array([[0, 0, 0], [1, 1, 1], [5, 5, 5]], + dtype=np.float32)) + h = nanovdb.tools.cuda.pointsToGrid(pts, 1.0, 0) + self.assertEqual(h.gridType(0), nanovdb.GridType.PointIndex) + + def test_points_to_grid_world_double(self): + cp = _require_cupy(self) + import numpy as np + pts = cp.asarray(np.array([[0, 0, 0], [1, 1, 1]], dtype=np.float64)) + h = nanovdb.tools.cuda.pointsToGrid(pts, 1.0, 0) + self.assertEqual(h.gridType(0), nanovdb.GridType.PointIndex) + + def test_voxels_to_index_family(self): + cp = _require_cupy(self) + import numpy as np + vox = cp.asarray(np.array([[0, 0, 0], [1, 0, 0], [0, 1, 0]], + dtype=np.int32)) + self.assertEqual( + nanovdb.tools.cuda.voxelsToOnIndexGrid(vox, 1.0, 0).gridType(0), + nanovdb.GridType.OnIndex) + self.assertEqual( + nanovdb.tools.cuda.voxelsToIndexGrid(vox, 1.0, 0).gridType(0), + nanovdb.GridType.Index) + self.assertEqual( + nanovdb.tools.cuda.voxelsToRGBA8Grid(vox, 1.0, 0).gridType(0), + nanovdb.GridType.RGBA8) + self.assertEqual( + nanovdb.tools.cuda.pointsToRGBA8Grid(vox, 1.0, 0).gridType(0), + nanovdb.GridType.RGBA8) + + def test_non_default_stream_accepted(self): + cp = _require_cupy(self) + import numpy as np + pts = cp.asarray(np.array([[0, 0, 0], [2, 2, 2]], dtype=np.float32)) + stream = cp.cuda.Stream(non_blocking=True) + # Stream args everywhere are raw CUDA stream handles as Python ints. + h = nanovdb.tools.cuda.pointsToGrid(pts, 1.0, stream.ptr) + self.assertEqual(h.gridType(0), nanovdb.GridType.PointIndex) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceGridOps(unittest.TestCase): + """tools.cuda morphology / QC / sampling on device grids.""" + + def test_morphology_returns_onindex(self): + dh, dg = _build_device_onindex_grid(20.0) + # op 26 == NN_FACE_EDGE_VERTEX (verified). 18 is unimplemented. + self.assertEqual( + nanovdb.tools.cuda.dilateGrid(dg, 26, 0).gridType(0), + nanovdb.GridType.OnIndex) + self.assertEqual( + nanovdb.tools.cuda.coarsenGrid(dg, 0).gridType(0), + nanovdb.GridType.OnIndex) + self.assertEqual( + nanovdb.tools.cuda.refineGrid(dg, 0).gridType(0), + nanovdb.GridType.OnIndex) + + def test_is_valid_on_device_grid(self): + dh, dg = _build_device_onindex_grid(20.0) + self.assertTrue(nanovdb.tools.cuda.isValid(dg)) + + def test_signed_flood_fill_in_place(self): + h = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Float, 20) + h.deviceUpload(0, True) + dg = h.deviceGrid(0) + self.assertIsNotNone(dg) + # In place on the device grid; no return value. + self.assertIsNone(nanovdb.tools.cuda.signedFloodFill(dg)) + + def test_sample_from_voxels_float(self): + cp = _require_cupy(self) + import numpy as np + h = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Float, 20) + h.deviceUpload(0, True) + dg = h.deviceGrid(0) + pts = cp.asarray(np.array([[0, 0, 0], [20, 0, 0], [10, 0, 0]], + dtype=np.float32)) + vals = cp.empty(3, dtype=cp.float32) + nanovdb.tools.cuda.sampleFromVoxels(pts, dg, vals, 0) + out = cp.asnumpy(vals) + self.assertEqual(out.shape, (3,)) + + def test_sample_from_voxels_with_gradients(self): + cp = _require_cupy(self) + import numpy as np + h = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Float, 20) + h.deviceUpload(0, True) + dg = h.deviceGrid(0) + pts = cp.asarray(np.array([[10, 0, 0]], dtype=np.float32)) + vals = cp.empty(1, dtype=cp.float32) + grads = cp.empty((1, 3), dtype=cp.float32) + nanovdb.tools.cuda.sampleFromVoxels(pts, dg, vals, grads, 0) + self.assertEqual(cp.asnumpy(grads).shape, (1, 3)) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceNodeManager(unittest.TestCase): + """createDeviceNodeManager over a device OnIndex grid.""" + + def test_node_manager_handle(self): + dh, dg = _build_device_onindex_grid(20.0) + nmh = nanovdb.cuda.createDeviceNodeManager(dg, 0) + self.assertIsNotNone(nmh) + self.assertGreater(nmh.size(), 0) + mgr = nmh.mgr() + # mgr() returns the TYPED device NodeManager, not a raw int. + self.assertIsNotNone(mgr) + self.assertNotIsInstance(mgr, int) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceVoxelBlockManager(unittest.TestCase): + """buildVoxelBlockManager on a device OnIndex grid + zero-copy buffers.""" + + def test_block_manager_properties(self): + dh, dg = _build_device_onindex_grid(100.0) + vbm = nanovdb.tools.cuda.buildVoxelBlockManager(dg, 6, 0, 0, 0, 0) + self.assertGreater(vbm.blockCount(), 0) + # block_width / log2_block_width / jump_map_length are PROPERTIES. + self.assertEqual(vbm.block_width, 64) + self.assertEqual(vbm.log2_block_width, 6) + # firstOffset == 1 (mod BlockWidth) for a full grid. + self.assertEqual(vbm.firstOffset(), 1) + self.assertGreater(vbm.lastOffset(), 0) + + def test_first_leaf_id_zero_copy(self): + cp = _require_cupy(self) + dh, dg = _build_device_onindex_grid(100.0) + vbm = nanovdb.tools.cuda.buildVoxelBlockManager(dg, 6, 0, 0, 0, 0) + # firstLeafID() / jumpMap() return DLPack capsules directly. + fl = cp.from_dlpack(vbm.firstLeafID()) + self.assertEqual(fl.dtype, cp.uint32) + self.assertEqual(fl.shape[0], vbm.blockCount()) + self.assertEqual(int(fl.data.ptr), vbm.first_leaf_id_ptr()) + + def test_jump_map_zero_copy(self): + cp = _require_cupy(self) + dh, dg = _build_device_onindex_grid(100.0) + vbm = nanovdb.tools.cuda.buildVoxelBlockManager(dg, 6, 0, 0, 0, 0) + jm = cp.from_dlpack(vbm.jumpMap()) + self.assertEqual(jm.dtype, cp.uint64) + self.assertEqual(jm.shape[0], vbm.blockCount()) + self.assertEqual(int(jm.data.ptr), vbm.jump_map_ptr()) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceInfra(unittest.TestCase): + """DeviceMesh / DeviceStreamMap / DeviceResource / TempDevicePool.""" + + def test_device_mesh(self): + mesh = nanovdb.cuda.DeviceMesh() + self.assertGreaterEqual(mesh.deviceCount(), 1) + # canAccessPeer is well-defined (self-peer is typically False). + self.assertIsInstance(mesh.canAccessPeer(0, 0), bool) + + def test_device_stream_map(self): + DSM = nanovdb.cuda.DeviceStreamMap + sm = DSM(DSM.DeviceType.Unified, [], 0) + self.assertGreaterEqual(sm.deviceCount(), 1) + # stream(deviceId) is a raw CUDA stream handle as a Python int. + self.assertIsInstance(sm.stream(0), int) + + def test_device_resource_alloc_dealloc(self): + DR = nanovdb.cuda.DeviceResource + self.assertEqual(DR.DEFAULT_ALIGNMENT, 256) + ptr = DR.allocateAsync(1024, 256, 0) + self.assertIsInstance(ptr, int) + self.assertNotEqual(ptr, 0) + DR.deallocateAsync(ptr, 1024, 256, 0) + + def test_temp_device_pool(self): + tp = nanovdb.cuda.TempDevicePool() + tp.setRequestedSize(2048) + tp.reallocate(0) + self.assertEqual(tp.requestedSize(), 2048) + self.assertGreaterEqual(tp.size(), 2048) + self.assertNotEqual(tp.data(), 0) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDistributedPointsToGrid(unittest.TestCase): + """Multi-GPU DistributedPointsToGrid over MANAGED coordinate arrays. + + With a single device this exercises the trivial single-GPU path, but + still validates the cuda_managed memory constraint and the + UnifiedGridHandle result type. + """ + + def test_managed_coords_produce_unified_handle(self): + cp = _require_cupy(self) + import numpy as np + prev = cp.cuda.get_allocator() + cp.cuda.set_allocator(cp.cuda.malloc_managed) + try: + pts = np.array([[0, 0, 0], [1, 0, 0], [0, 1, 0], [5, 5, 5]], + dtype=np.int32) + voxels = cp.asarray(pts) # MANAGED (unified) memory + mesh = nanovdb.cuda.DeviceMesh() + # The mesh must outlive the converter (held by reference). + conv = nanovdb.tools.cuda.DistributedPointsToGrid( + mesh, 1.0, (0.0, 0.0, 0.0)) + uh = conv.getHandle(voxels) + self.assertEqual(uh.gridType(0), nanovdb.GridType.OnIndex) + # UnifiedGridHandle intentionally does NOT expose the + # CAI / DLPack bridges that DeviceGridHandle does. + self.assertFalse(hasattr(uh, "__cuda_array_interface__")) + finally: + cp.cuda.set_allocator(prev) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestTorchInterop(unittest.TestCase): + """Optional PyTorch interop (skips when torch is not installed).""" + + def test_torch_cuda_array_interface(self): + try: + import torch + except ImportError: + self.skipTest("PyTorch not installed") + if not torch.cuda.is_available(): + self.skipTest("PyTorch built without CUDA / no GPU") + dh, dg = _build_device_onindex_grid(20.0) + # torch can adopt the DeviceGridHandle device buffer via DLPack. + t = torch.from_dlpack(dh) + self.assertEqual(t.numel(), dh.size()) + self.assertTrue(t.is_cuda) + + +if __name__ == "__main__": + unittest.main() From 7c70bec80bc65294ea9c00f1f6ab2ca077d0fe16 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 06:21:15 +0000 Subject: [PATCH 22/48] nanovdb python: bump nanobind to 2.12.0 for the NanoVDB CI job and the wheel The NanoVDB Python GPU bindings (CUDA array interface, DLPack export, device interop) benefit from a newer nanobind than the repo-wide 2.5.0 floor. Scope the bump to the two places that build the published NanoVDB surface: - .github/workflows/nanovdb.yml: install nanobind 2.12.0 (latest 2.x) for the dedicated NanoVDB job. - pyproject.toml: pin the wheel's nanobind to >=2.12.0,<3 (it was previously unpinned and silently floated to latest, diverging from CI). The other seven install sites (build/houdini/docs/ax/weekly) and the global FUTURE_MINIMUM_NANOBIND_VERSION stay at 2.5.0: nanobind is a shared dependency and the co-compiled pyopenvdb module plus the shared build.yml still target 2.5.0, so the binding code remains >=2.5.0 source-compatible. The <3 ceiling keeps the wheel on the 2.x series. Verified locally on a Blackwell GPU: nanobind 2.12.0 (installed to a separate prefix, CMake-resolved version confirmed 2.12.0) compiles both nanovdb_python and openvdb_python with no errors and no deprecation warnings. Signed-off-by: Jonathan Swartz --- .github/workflows/nanovdb.yml | 6 +++++- pyproject.toml | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/.github/workflows/nanovdb.yml b/.github/workflows/nanovdb.yml index 442b88e9fb..9b67049efb 100644 --- a/.github/workflows/nanovdb.yml +++ b/.github/workflows/nanovdb.yml @@ -70,7 +70,11 @@ jobs: echo "/usr/local/cuda-12/bin" >> $GITHUB_PATH echo "LD_LIBRARY_PATH=/usr/local/cuda-12/lib64:$LD_LIBRARY_PATH" >> $GITHUB_ENV - name: nanobind - run: ./ci/install_nanobind.sh 2.5.0 + # The NanoVDB Python bindings (CUDA interop, DLPack) track a newer + # nanobind than the repo-wide 2.5.0 floor. Bump here only; the shared + # build.yml still co-compiles the bindings against 2.5.0, so the + # binding code must stay >=2.5.0 source-compatible. + run: ./ci/install_nanobind.sh 2.12.0 - name: build # NOTE: CMAKE_POSITION_INDEPENDENT_CODE set to fix default behaviour change in clang 14 # https://github.com/AcademySoftwareFoundation/aswf-docker/issues/228 diff --git a/pyproject.toml b/pyproject.toml index b42bcd5c80..6823cd8f66 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,5 +1,5 @@ [build-system] -requires = ["scikit_build_core", "nanobind"] +requires = ["scikit_build_core", "nanobind>=2.12.0,<3"] build-backend = "scikit_build_core.build" [project] From 10405638f5704717dd4deb149f8f8a76667102b4 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 22:54:33 +0000 Subject: [PATCH 23/48] nanovdb python: bind tools.cuda.inject + injectPredicateToMask MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expose NanoVDB's sidecar-injection operators (util/cuda/Injection.cuh) on the nanovdb.tools.cuda submodule. These carry a grid's per-voxel data across a topology change (dilate/prune/merge), which the morphology ops alone do not do — the OnIndex value-index space changes, so values must be re-injected. - inject(src_grid, dst_grid, src_sidecar, dst_sidecar, stream=0): copies sidecar values for every voxel present in BOTH grids (the intersection); destination voxels with no source counterpart are left unchanged. Wraps InjectGridDataFunctor via util::cuda::operatorKernel (one block per source leaf, bit-parallel per warp). Bound for float and double sidecars. - injectPredicateToMask(grid, predicate, leaf_masks, stream=0): turns a boolean predicate over a grid's value indices into the per-leaf Mask<3> retain mask that pruneGrid consumes, so a value-driven trim (e.g. |phi| <= halfWidth) is two device ops with no hand-rolled kernel. Wraps InjectPredicateToMaskFunctor. leaf_masks need only be >= (leaf count)*8 uint64; activeVoxelCount*8 is a safe size since every leaf holds at least one active voxel. This is the GPU realization of the injectData abstraction in the NanoVDB 2.0 paper (section 3.4), and completes the dilate -> inject -> prune narrow-band rebuild loop entirely with bound device ops. Validated locally on a Blackwell GPU: inject copies exactly the src/dst intersection (leaving the rest, including the background slot, untouched); injectPredicateToMask + pruneGrid trims to exactly the predicate-true voxel set; a second inject carries values onto the pruned grid; bad grid arguments raise. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 1 + nanovdb/nanovdb/python/PyTools.cc | 7 + nanovdb/nanovdb/python/cuda/PyInjectData.cu | 138 ++++++++++++++++++++ nanovdb/nanovdb/python/cuda/PyInjectData.h | 17 +++ 4 files changed, 163 insertions(+) create mode 100644 nanovdb/nanovdb/python/cuda/PyInjectData.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyInjectData.h diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index d9133e896c..00e146b79c 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -49,6 +49,7 @@ nanobind_add_module(nanovdb_python NB_STATIC cuda/PyRefineGrid.cu cuda/PyPruneGrid.cu cuda/PyMergeGrids.cu + cuda/PyInjectData.cu cuda/PyIndexToGrid.cu cuda/PyAddBlindData.cu cuda/PyDeviceGridStats.cu diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 82b88b21ae..a818159d5f 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -25,6 +25,7 @@ #include "cuda/PyRefineGrid.h" #include "cuda/PyPruneGrid.h" #include "cuda/PyMergeGrids.h" +#include "cuda/PyInjectData.h" #include "cuda/PyIndexToGrid.h" #include "cuda/PyAddBlindData.h" #include "cuda/PyDeviceGridStats.h" @@ -108,6 +109,12 @@ void defineToolsModule(nb::module_& m) definePruneGrid(cudaModule, "pruneGrid"); defineMergeGrids(cudaModule, "mergeGrids"); + // Sidecar value transfer across a topology change, and the predicate->mask + // helper that feeds pruneGrid (nanovdb::util::cuda::Inject* functors). + defineInject(cudaModule, "inject"); + defineInject(cudaModule, "inject"); + defineInjectPredicateToMask(cudaModule, "injectPredicateToMask"); + // Device VoxelBlockManager (nanovdb::tools::cuda) on nanovdb.tools.cuda. defineDeviceVoxelBlockManager(cudaModule); diff --git a/nanovdb/nanovdb/python/cuda/PyInjectData.cu b/nanovdb/nanovdb/python/cuda/PyInjectData.cu new file mode 100644 index 0000000000..00ed65c8c9 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyInjectData.cu @@ -0,0 +1,138 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifdef NANOVDB_USE_CUDA + +#include "PyInjectData.h" + +#include + +#include +#include + +#include + +#include +#include // cudaCheck, operatorKernel +#include // leaf count of a device grid +#include // Inject*Functor + +namespace nb = nanobind; +using namespace nb::literals; +// Deliberately NOT `using namespace nanovdb;`: keep the device-grid type names +// fully qualified, matching the sibling tools.cuda bindings. + +namespace pynanovdb { + +namespace { + +// Cast a Python device-grid object to NanoGrid*. The returned +// object's underlying address IS the device pointer; it must NOT be +// dereferenced on the host -- it is only passed to device kernels. +nanovdb::NanoGrid* +castOnIndexDeviceGrid(nb::handle py_grid, const char* fn_name) +{ + if (!nb::isinstance>(py_grid)) { + std::string msg(fn_name); + msg += ": expected a NanoVDB device grid of build type ValueOnIndex " + "(OnIndexGrid), obtained from DeviceGridHandle.deviceGrid(n)"; + throw nb::type_error(msg.c_str()); + } + return &nb::cast&>(py_grid); +} + +// Leaf-node count read from device memory (one D2H copy of the tree header). +uint32_t leafCountOf(const nanovdb::NanoGrid* d_grid) +{ + using Traits = nanovdb::util::cuda::DeviceGridTraits; + return Traits::getTreeData(d_grid).mNodeCount[0]; +} + +} // anonymous namespace + +template void defineInject(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::handle src_grid, nb::handle dst_grid, + nb::ndarray, nb::c_contig, nb::device::cuda> src_sidecar, + nb::ndarray, nb::c_contig, nb::device::cuda> dst_sidecar, + uintptr_t stream) { + auto* src = castOnIndexDeviceGrid(src_grid, "inject"); + auto* dst = castOnIndexDeviceGrid(dst_grid, "inject"); + cudaStream_t s = reinterpret_cast(stream); + const T* dSrc = src_sidecar.data(); + T* dDst = dst_sidecar.data(); + const uint32_t srcLeafCount = leafCountOf(src); + using Op = nanovdb::util::cuda::InjectGridDataFunctor; + // operatorKernel launches one block per SOURCE leaf and copies the + // src/dst intersection bit-parallel per warp; pure CUDA, no Python. + nb::gil_scoped_release release; + if (srcLeafCount) + nanovdb::util::cuda::operatorKernel + <<>>(src, dst, dSrc, dDst); + cudaCheck(cudaStreamSynchronize(s)); + }, + "src_grid"_a, "dst_grid"_a, "src_sidecar"_a, "dst_sidecar"_a, "stream"_a = 0, + "Inject sidecar values from a source OnIndex device grid onto a " + "destination OnIndex device grid (injectData; NanoVDB 2.0 paper, " + "section 3.4). For every voxel present in BOTH grids the source sidecar " + "value is copied to that voxel's slot in the destination sidecar; " + "destination voxels with no source counterpart are left unchanged " + "(so the source need not be a subset -- the copy is over the " + "intersection). src_grid / dst_grid are device grids from " + "DeviceGridHandle.deviceGrid(n); src_sidecar / dst_sidecar are 1-D " + "device arrays indexed by each grid's value index (entry 0 is the " + "background slot, untouched). Wraps " + "nanovdb::util::cuda::InjectGridDataFunctor. stream is a raw CUDA " + "stream handle (Python int; 0 = default stream)."); +} + +void defineInjectPredicateToMask(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::handle grid, + nb::ndarray, nb::c_contig, nb::device::cuda> predicate, + nb::ndarray, nb::c_contig, nb::device::cuda> leaf_masks, + uintptr_t stream) { + auto* d_grid = castOnIndexDeviceGrid(grid, "injectPredicateToMask"); + cudaStream_t s = reinterpret_cast(stream); + const uint32_t leafCount = leafCountOf(d_grid); + constexpr size_t W = nanovdb::Mask<3>::WORD_COUNT; // 8 uint64 / leaf + if (leaf_masks.size() < static_cast(leafCount) * W) + throw nb::value_error( + "injectPredicateToMask: leaf_masks length must be at least " + "(leaf count) * 8 uint64 (one Mask<3> per leaf). A safe " + "upper bound is activeVoxelCount * 8, since every leaf " + "holds at least one active voxel."); + const bool* dPred = predicate.data(); + nanovdb::Mask<3>* dMask = + reinterpret_cast*>(leaf_masks.data()); + using Op = nanovdb::util::cuda::InjectPredicateToMaskFunctor; + // One block per leaf; the functor zeroes each leaf mask, then sets + // the bit of every active voxel whose predicate slot is true. + nb::gil_scoped_release release; + if (leafCount) + nanovdb::util::cuda::operatorKernel + <<>>(d_grid, dPred, dMask); + cudaCheck(cudaStreamSynchronize(s)); + }, + "grid"_a, "predicate"_a, "leaf_masks"_a, "stream"_a = 0, + "Build a per-leaf retain mask for pruneGrid from a boolean predicate " + "over an OnIndex device grid's value indices. grid is a device grid " + "from DeviceGridHandle.deviceGrid(n); predicate is a 1-D device bool " + "array indexed by value index (entry n true => keep that voxel); " + "leaf_masks is a 1-D device uint64 output of length at least " + "(leaf count) * 8 (one nanovdb::Mask<3> per leaf, in leaf order), " + "ready to pass straight to pruneGrid; activeVoxelCount * 8 is a safe " + "size since every leaf holds at least one active voxel. Wraps " + "nanovdb::util::cuda::InjectPredicateToMaskFunctor. stream is a raw " + "CUDA stream handle (Python int; 0 = default stream)."); +} + +template void defineInject(nb::module_&, const char*); +template void defineInject(nb::module_&, const char*); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyInjectData.h b/nanovdb/nanovdb/python/cuda/PyInjectData.h new file mode 100644 index 0000000000..4b7be57e97 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyInjectData.h @@ -0,0 +1,17 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYINJECTDATA_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYINJECTDATA_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +template void defineInject(nb::module_& m, const char* name); +void defineInjectPredicateToMask(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif From aa3050f39d3e12a38abb0528af9d41dcd9a37226 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 23:05:30 +0000 Subject: [PATCH 24/48] nanovdb python: bind the remaining inject operators (features + intersection mask) Completes the util/cuda/Injection.cuh surface on nanovdb.tools.cuda: - inject(...) gains a vector-valued overload accepting 2-D (value count, dim) device sidecars, wrapping InjectGridFeatureFunctor. nanobind dispatches on ndim, so the same `inject` name handles both scalar (1-D) and feature (2-D) sidecars; the feature dimension is taken from shape[1]. - injectGridMask(src_grid, dst_grid, leaf_masks, stream=0): builds a per-leaf Mask<3> over the destination grid marking the voxels also active in the source (the src/dst intersection), ready to feed pruneGrid. Wraps InjectGridMaskFunctor via util::cuda::lambdaKernel. Like injectPredicateToMask, leaf_masks need only be >= (dst leaf count)*8 uint64. All four Injection.cuh functors are now exposed (inject scalar + feature, injectPredicateToMask, injectGridMask). Validated locally on a Blackwell GPU: the feature overload copies the intersection rows (leaving the rest, including the background row, untouched); injectGridMask + pruneGrid recovers exactly the source/destination intersection; the 1-D scalar inject overload still dispatches correctly. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyTools.cc | 3 + nanovdb/nanovdb/python/cuda/PyInjectData.cu | 79 +++++++++++++++++++++ nanovdb/nanovdb/python/cuda/PyInjectData.h | 2 + 3 files changed, 84 insertions(+) diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index a818159d5f..db917d02a7 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -113,7 +113,10 @@ void defineToolsModule(nb::module_& m) // helper that feeds pruneGrid (nanovdb::util::cuda::Inject* functors). defineInject(cudaModule, "inject"); defineInject(cudaModule, "inject"); + defineInjectFeatures(cudaModule, "inject"); + defineInjectFeatures(cudaModule, "inject"); defineInjectPredicateToMask(cudaModule, "injectPredicateToMask"); + defineInjectGridMask(cudaModule, "injectGridMask"); // Device VoxelBlockManager (nanovdb::tools::cuda) on nanovdb.tools.cuda. defineDeviceVoxelBlockManager(cudaModule); diff --git a/nanovdb/nanovdb/python/cuda/PyInjectData.cu b/nanovdb/nanovdb/python/cuda/PyInjectData.cu index 00ed65c8c9..997ec97d18 100644 --- a/nanovdb/nanovdb/python/cuda/PyInjectData.cu +++ b/nanovdb/nanovdb/python/cuda/PyInjectData.cu @@ -87,6 +87,43 @@ template void defineInject(nb::module_& m, const char* name) "stream handle (Python int; 0 = default stream)."); } +template void defineInjectFeatures(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::handle src_grid, nb::handle dst_grid, + nb::ndarray, nb::c_contig, nb::device::cuda> src_sidecar, + nb::ndarray, nb::c_contig, nb::device::cuda> dst_sidecar, + uintptr_t stream) { + auto* src = castOnIndexDeviceGrid(src_grid, "inject"); + auto* dst = castOnIndexDeviceGrid(dst_grid, "inject"); + if (src_sidecar.shape(1) != dst_sidecar.shape(1)) + throw nb::value_error( + "inject: src and dst feature sidecars must share the same " + "feature dimension (shape[1])."); + cudaStream_t s = reinterpret_cast(stream); + const T* dSrc = src_sidecar.data(); + T* dDst = dst_sidecar.data(); + const size_t dim = src_sidecar.shape(1); + const uint32_t srcLeafCount = leafCountOf(src); + using Op = nanovdb::util::cuda::InjectGridFeatureFunctor; + nb::gil_scoped_release release; + if (srcLeafCount) + nanovdb::util::cuda::operatorKernel + <<>>(src, dst, dSrc, dDst, dim); + cudaCheck(cudaStreamSynchronize(s)); + }, + "src_grid"_a, "dst_grid"_a, "src_sidecar"_a, "dst_sidecar"_a, "stream"_a = 0, + "Inject vector-valued (feature) sidecar data across OnIndex device " + "grids -- the multi-channel form of inject. src_sidecar / dst_sidecar " + "are 2-D device arrays of shape (value count, dim), row-major per voxel " + "(row 0 is the background slot); the feature dimension dim is taken " + "from shape[1] and must match. Values are copied for the src/dst voxel " + "intersection; the rest of the destination is left unchanged. Wraps " + "nanovdb::util::cuda::InjectGridFeatureFunctor. stream is a raw CUDA " + "stream handle (Python int; 0 = default stream)."); +} + void defineInjectPredicateToMask(nb::module_& m, const char* name) { m.def( @@ -130,8 +167,50 @@ void defineInjectPredicateToMask(nb::module_& m, const char* name) "CUDA stream handle (Python int; 0 = default stream)."); } +void defineInjectGridMask(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::handle src_grid, nb::handle dst_grid, + nb::ndarray, nb::c_contig, nb::device::cuda> leaf_masks, + uintptr_t stream) { + auto* src = castOnIndexDeviceGrid(src_grid, "injectGridMask"); + auto* dst = castOnIndexDeviceGrid(dst_grid, "injectGridMask"); + cudaStream_t s = reinterpret_cast(stream); + const uint32_t dstLeafCount = leafCountOf(dst); + constexpr size_t W = nanovdb::Mask<3>::WORD_COUNT; // 8 uint64 / leaf + if (leaf_masks.size() < static_cast(dstLeafCount) * W) + throw nb::value_error( + "injectGridMask: leaf_masks length must be at least " + "(dst leaf count) * 8 uint64 (one Mask<3> per leaf). A safe " + "upper bound is the destination grid's activeVoxelCount * 8."); + nanovdb::Mask<3>* dMask = + reinterpret_cast*>(leaf_masks.data()); + using Op = nanovdb::util::cuda::InjectGridMaskFunctor; + constexpr unsigned threads = 128; + nb::gil_scoped_release release; + if (dstLeafCount) + nanovdb::util::cuda::lambdaKernel + <<>>(dstLeafCount, Op{}, src, dst, dMask); + cudaCheck(cudaStreamSynchronize(s)); + }, + "src_grid"_a, "dst_grid"_a, "leaf_masks"_a, "stream"_a = 0, + "Build a per-leaf mask over the DESTINATION grid marking the voxels " + "that are ALSO active in the source grid (the src/dst intersection). " + "grid args are device grids from DeviceGridHandle.deviceGrid(n); " + "leaf_masks is a 1-D device uint64 output of length at least " + "(dst leaf count) * 8 (one nanovdb::Mask<3> per leaf, in leaf order; " + "the destination activeVoxelCount * 8 is a safe size). Pass it to " + "pruneGrid to keep only the intersection. Wraps " + "nanovdb::util::cuda::InjectGridMaskFunctor. stream is a raw CUDA " + "stream handle (Python int; 0 = default stream)."); +} + template void defineInject(nb::module_&, const char*); template void defineInject(nb::module_&, const char*); +template void defineInjectFeatures(nb::module_&, const char*); +template void defineInjectFeatures(nb::module_&, const char*); } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyInjectData.h b/nanovdb/nanovdb/python/cuda/PyInjectData.h index 4b7be57e97..b2171b682b 100644 --- a/nanovdb/nanovdb/python/cuda/PyInjectData.h +++ b/nanovdb/nanovdb/python/cuda/PyInjectData.h @@ -10,7 +10,9 @@ namespace nb = nanobind; namespace pynanovdb { template void defineInject(nb::module_& m, const char* name); +template void defineInjectFeatures(nb::module_& m, const char* name); void defineInjectPredicateToMask(nb::module_& m, const char* name); +void defineInjectGridMask(nb::module_& m, const char* name); } // namespace pynanovdb From d04c8d1ccc2d11003ad0a7e5ac0d96eeda398014 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 2 Jun 2026 23:56:00 +0000 Subject: [PATCH 25/48] nanovdb python: fix grid.getBlindData() returning via a weak-referenceable owner MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit getBlindData() constructed its NumPy view with the generic nb::ndarray type (no framework tag) owned by the grid, then returned it with rv_policy::reference. Without the nb::numpy tag nanobind takes a keep-alive path that needs the owner (the grid) to be weak-referenceable; grid classes are not, so every call raised "nb::detail::keep_alive(): could not create a weak reference!" and the function was unusable. Tag the views as nb::ndarray (1-D and 2-D), matching the working zero-copy views elsewhere (LeafNode.values(), the VoxelBlockManager firstLeafID()/jumpMap() host views). nanobind then attaches the owner as the NumPy array's base (a strong reference, no weakref), and the existing keep_alive<0,1> on the def keeps the grid — and the GridHandle that owns the buffer — alive for the view's lifetime. Verified locally: grid.getBlindData(0) on an OnIndex grid with a float SDF channel returns the expected (valueCount,) float32 array, and the view stays valid after the grid handle is dropped and garbage-collected. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/NanoVDBModule.cc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 8d5f56eb5a..6680bc3b2e 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -513,14 +513,14 @@ static nb::object pyGetBlindData(nb::handle py_grid, uint32_t n) auto make1D = [&](void* p, size_t n_elems, auto sentinel) -> nb::object { using T = decltype(sentinel); size_t shape[1] = {n_elems}; - return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( + return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( static_cast(p), 1, shape, py_grid), nb::rv_policy::reference); }; auto make2D = [&](void* p, size_t n_outer, size_t n_inner, auto sentinel) -> nb::object { using T = decltype(sentinel); size_t shape[2] = {n_outer, n_inner}; - return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( + return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( static_cast(p), 2, shape, py_grid), nb::rv_policy::reference); }; From 36e9ae8ec70d4534305d7a3bde5258cb34f10f9b Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 3 Jun 2026 03:21:18 +0000 Subject: [PATCH 26/48] nanovdb python: add the cupy_levelset_filter GPU example A full GPU level-set filter on a .nvdb file, driven by the device VoxelBlockManager, exercising the inject / injectPredicateToMask bindings and grid.getBlindData added in this branch. Each iteration runs a Laplacian-flow deform and a first-order Godunov reinitialisation as VBM stencil kernels (buildVoxelBlockManager + decodeInverseMaps/computeBoxStencil from a cupy.RawModule), then retracks the narrow band natively with dilateGrid -> inject -> injectPredicateToMask -> pruneGrid -> inject. Reads either a FloatGrid or an OnIndexGrid with the SDF in a blind channel (detected via gridType), normalising both to an OnIndex topology + a float SDF sidecar read with grid.getBlindData, and writes the result back in the same style. Run with no arguments it self-tests both forms (sphere shrinks under curvature flow); it self-skips when CUDA / a GPU / CuPy is unavailable. Listed in python/examples/README.md. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/examples/README.md | 1 + .../python/examples/cupy_levelset_filter.py | 463 ++++++++++++++++++ 2 files changed, 464 insertions(+) create mode 100644 nanovdb/nanovdb/python/examples/cupy_levelset_filter.py diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index 0cc9ec5934..99f1731a94 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -40,6 +40,7 @@ GPU-array framework it uses is unavailable. | [`cupy_rawkernel.py`](cupy_rawkernel.py) | A `cupy.RawKernel` that `#include ` (compiled with `nanovdb.cuda.compile_options()`) and reads a `const nanovdb::NanoGrid*` straight from `grid.data_ptr()`. Documents the device-pointer ABI. Requires CuPy. | | [`numba_cuda.py`](numba_cuda.py) | Adopting a NanoVDB device buffer as a zero-copy `numba.cuda` array (Numba can't parse the C++ ABI, so it operates on the raw buffer). Requires Numba. | | [`triton_kernel.py`](triton_kernel.py) | Handing a NanoVDB device buffer to a Triton kernel via a zero-copy `torch.from_dlpack` tensor. Requires Triton + PyTorch. | +| [`cupy_levelset_filter.py`](cupy_levelset_filter.py) | A full GPU level-set filter (`tools::LevelSetFilter`-style diffusion + Godunov renormalisation + narrow-band retrack) on a `.nvdb` file, driven by the device `VoxelBlockManager`: `buildVoxelBlockManager` plus `decodeInverseMaps` / `computeBoxStencil` called from a `cupy.RawModule` (nvcc backend), with `dilateGrid` / `inject` / `injectPredicateToMask` / `pruneGrid` for the retrack. Reads/writes either a `FloatGrid` or an `OnIndexGrid` with the SDF in a blind channel, preserving the input style. No args runs a self-test. Requires CuPy + nvcc. | For full API signatures and per-argument docstrings, use Python's `help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`. diff --git a/nanovdb/nanovdb/python/examples/cupy_levelset_filter.py b/nanovdb/nanovdb/python/examples/cupy_levelset_filter.py new file mode 100644 index 0000000000..1e2545d222 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/cupy_levelset_filter.py @@ -0,0 +1,463 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""GPU LevelSetFilter on NanoVDB .nvdb files, driven by the VoxelBlockManager. + +Reads a .nvdb file, runs N iterations of the GPU LevelSetFilter loop (the +diffusion + renormalisation + narrow-band retrack that OpenVDB's +tools::LevelSetFilter + LevelSetTracker perform), and writes the result to +another .nvdb file: + + python cupy_levelset_filter.py input.nvdb output.nvdb [outer_iterations] + +The input grid may be EITHER form (the script detects which): + * a FloatGrid level set (per-voxel float SDF), or + * an OnIndexGrid whose float SDF is stored in blind-data channel 0. + +Both are normalised to "(OnIndex topology on the device) + (float SDF sidecar)", +which is the representation the VoxelBlockManager operates on: + * FloatGrid -> tools.createOnIndexGrid(fg, channels=1) bakes the SDF into a + blind channel; write to a temp .nvdb; io.deviceReadGrid it. + * OnIndexGrid -> io.deviceReadGrid directly. +The SDF sidecar is read on the host with grid.getBlindData(0) (value-index +order: [0] is the background slot, 1..N the active voxels -- the same order the +VBM decode uses), then uploaded to the device. + +The OUTPUT is written in the SAME style as the input: a FloatGrid input yields a +FloatGrid .nvdb, an OnIndex+SDF input yields an OnIndexGrid .nvdb with the SDF in +blind channel 0. The result is baked on the host (tools.build.FloatGrid -> +to_nanovdb), optionally converted back to OnIndex via createOnIndexGrid, then +io.writeGrid. (Writing the result as a device-built grid via indexToGrid is +avoided: in testing it dropped the high-value-index voxels for these +stats/tiles-free index grids.) + +Each filter iteration runs three stages: + 1. DEFORM Laplacian flow phi += (sum6 - 6 phi)/6 (VBM stencil) + 2. RENORMALIZE Godunov reinit phi -= dt*S(phi)*(|grad|-1) (VBM stencil) + 3. REBUILD dilateGrid -> inject -> extrapolate -> injectPredicateToMask + -> pruneGrid -> inject (native bound ops + one extrapolate kernel) + +Scope / limitations: first-order Godunov reinitialisation (not higher-order +WENO), no advection and no alpha mask. The prune keeps |phi| <= band*voxelSize, +so the active band tracks the surface, but the output's inactive interior +carries +background (there is no signed flood-fill). + +Run without arguments for a self-test: it builds sphere .nvdb files in both +forms (FloatGrid and OnIndex+SDF), filters each, and asserts that the output +style matches the input and the sphere shrinks under curvature flow. + +Requires CuPy, a CUDA-capable GPU, and nvcc (CuPy honours the NVCC env var). In a +dev/source tree, set NANOVDB_INCLUDE to the dir containing nanovdb/NanoVDB.h. +""" +import os +import sys +import tempfile + +import numpy as np + +import nanovdb + + +LOG2_BLOCK_WIDTH = 9 +BLOCK_WIDTH = 1 << LOG2_BLOCK_WIDTH +NN_FACE = 6 # nanovdb::tools::morphology::NN_FACE (6-face dilation) +SENTINEL = 1.0e30 # "value not yet known" marker for freshly-dilated voxels +SDF_BLIND_CHANNEL = 0 # blind-data channel holding the float SDF on OnIndex input + +# computeBoxStencil spoke ids: spoke = (di+1)*9 + (dj+1)*3 + (dk+1). +# centre=13; +x=22 -x=4; +y=16 -y=10; +z=14 -z=12 +KERNEL_SRC = r""" +#include + +using namespace nanovdb; +using VBM = nanovdb::tools::cuda::VoxelBlockManager<9>; // BlockWidth = 512 +static constexpr int BLOCK_WIDTH = 512; +static constexpr int JUMP_MAP_LENGTH = BLOCK_WIDTH / 64; // = 8 +static constexpr float SENTINEL = 1.0e30f; + +__device__ static unsigned long long +decodeBlock(const NanoGrid* grid, + const uint32_t* firstLeafID, const uint64_t* jumpMap, + uint64_t firstOffset, + uint32_t* smem_leafIndex, uint16_t* smem_voxelOffset) +{ + const int blockID = blockIdx.x; + const uint64_t blockFirstOffset = + firstOffset + (uint64_t)blockID * BLOCK_WIDTH; + VBM::decodeInverseMaps( + grid, firstLeafID[blockID], + &jumpMap[(uint64_t)blockID * JUMP_MAP_LENGTH], + blockFirstOffset, smem_leafIndex, smem_voxelOffset); + return blockFirstOffset; // decodeInverseMaps ends in __syncthreads() +} + +__device__ static inline float +readNbr(const float* v, uint64_t spoke, float vc, float background) +{ + return (spoke == 0) ? copysignf(background, vc) : v[spoke]; +} + +extern "C" __global__ +void decode_coords(const NanoGrid* grid, + const uint32_t* firstLeafID, const uint64_t* jumpMap, + uint64_t firstOffset, int* coords) +{ + __shared__ uint32_t smem_leafIndex[BLOCK_WIDTH]; + __shared__ uint16_t smem_voxelOffset[BLOCK_WIDTH]; + decodeBlock(grid, firstLeafID, jumpMap, firstOffset, + smem_leafIndex, smem_voxelOffset); + const int tID = threadIdx.x; + if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + const auto& leaf = grid->tree().getFirstNode<0>()[smem_leafIndex[tID]]; + const uint16_t off = smem_voxelOffset[tID]; + const Coord c = leaf.offsetToGlobalCoord(off); + const uint64_t idx = leaf.getValue(off); + coords[idx * 3 + 0] = c[0]; + coords[idx * 3 + 1] = c[1]; + coords[idx * 3 + 2] = c[2]; +} + +// One Laplacian-flow iteration (OpenVDB LevelSetFilter::laplacianImpl). +extern "C" __global__ +void laplacian_step(const NanoGrid* grid, + const uint32_t* firstLeafID, const uint64_t* jumpMap, + uint64_t firstOffset, float background, + const float* vin, float* vout) +{ + __shared__ uint32_t smem_leafIndex[BLOCK_WIDTH]; + __shared__ uint16_t smem_voxelOffset[BLOCK_WIDTH]; + decodeBlock(grid, firstLeafID, jumpMap, firstOffset, + smem_leafIndex, smem_voxelOffset); + const int tID = threadIdx.x; + if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + + uint64_t st[27]; + VBM::computeBoxStencil(grid, smem_leafIndex, smem_voxelOffset, st); + + const uint64_t c = st[13]; + const float vc = vin[c]; + const float sum6 = readNbr(vin, st[22], vc, background) + + readNbr(vin, st[4], vc, background) + + readNbr(vin, st[16], vc, background) + + readNbr(vin, st[10], vc, background) + + readNbr(vin, st[14], vc, background) + + readNbr(vin, st[12], vc, background); + vout[c] = vc + (sum6 - 6.0f * vc) / 6.0f; +} + +// One Godunov reinitialisation iteration: phi -= dt*S(phi)*(|grad phi| - 1). +extern "C" __global__ +void godunov_step(const NanoGrid* grid, + const uint32_t* firstLeafID, const uint64_t* jumpMap, + uint64_t firstOffset, float dx, float dt, float background, + const float* vin, float* vout) +{ + __shared__ uint32_t smem_leafIndex[BLOCK_WIDTH]; + __shared__ uint16_t smem_voxelOffset[BLOCK_WIDTH]; + decodeBlock(grid, firstLeafID, jumpMap, firstOffset, + smem_leafIndex, smem_voxelOffset); + const int tID = threadIdx.x; + if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + + uint64_t st[27]; + VBM::computeBoxStencil(grid, smem_leafIndex, smem_voxelOffset, st); + + const uint64_t c = st[13]; + const float vc = vin[c]; + const float xm = readNbr(vin, st[4], vc, background); + const float xp = readNbr(vin, st[22], vc, background); + const float ym = readNbr(vin, st[10], vc, background); + const float yp = readNbr(vin, st[16], vc, background); + const float zm = readNbr(vin, st[12], vc, background); + const float zp = readNbr(vin, st[14], vc, background); + + const float Dxm = (vc - xm) / dx, Dxp = (xp - vc) / dx; + const float Dym = (vc - ym) / dx, Dyp = (yp - vc) / dx; + const float Dzm = (vc - zm) / dx, Dzp = (zp - vc) / dx; + + const float s = vc / sqrtf(vc * vc + dx * dx); // smoothed sign + float gx, gy, gz; + if (s > 0.0f) { + gx = fmaxf(powf(fmaxf(Dxm, 0.0f), 2), powf(fminf(Dxp, 0.0f), 2)); + gy = fmaxf(powf(fmaxf(Dym, 0.0f), 2), powf(fminf(Dyp, 0.0f), 2)); + gz = fmaxf(powf(fmaxf(Dzm, 0.0f), 2), powf(fminf(Dzp, 0.0f), 2)); + } else { + gx = fmaxf(powf(fminf(Dxm, 0.0f), 2), powf(fmaxf(Dxp, 0.0f), 2)); + gy = fmaxf(powf(fminf(Dym, 0.0f), 2), powf(fmaxf(Dyp, 0.0f), 2)); + gz = fmaxf(powf(fminf(Dzm, 0.0f), 2), powf(fmaxf(Dzp, 0.0f), 2)); + } + const float grad = sqrtf(gx + gy + gz); + vout[c] = vc - dt * s * (grad - 1.0f); +} + +// Fill freshly-dilated (sentinel) voxels by SDF extrapolation from their +// nearest in-band face neighbour: phi = phi_nbr + sign(phi_nbr)*dx. +extern "C" __global__ +void extrapolate(const NanoGrid* grid, + const uint32_t* firstLeafID, const uint64_t* jumpMap, + uint64_t firstOffset, float dx, const float* vin, float* vout) +{ + __shared__ uint32_t smem_leafIndex[BLOCK_WIDTH]; + __shared__ uint16_t smem_voxelOffset[BLOCK_WIDTH]; + decodeBlock(grid, firstLeafID, jumpMap, firstOffset, + smem_leafIndex, smem_voxelOffset); + const int tID = threadIdx.x; + if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + + uint64_t st[27]; + VBM::computeBoxStencil(grid, smem_leafIndex, smem_voxelOffset, st); + + const uint64_t c = st[13]; + const float vc = vin[c]; + if (vc != SENTINEL) { vout[c] = vc; return; } // already known + const int faces[6] = {22, 4, 16, 10, 14, 12}; + float best = SENTINEL; + for (int f = 0; f < 6; f++) { + const uint64_t ni = st[faces[f]]; + if (ni != 0) { // active neighbour + const float v = vin[ni]; + if (v != SENTINEL && fabsf(v) < fabsf(best)) best = v; + } + } + vout[c] = (best != SENTINEL) ? best + copysignf(dx, best) : vc; +} +""" + + +def _include_options(): + opts = list(nanovdb.cuda.compile_options("-std=c++17")) + inc = opts[0][2:] + if not os.path.isdir(inc): + env = os.environ.get("NANOVDB_INCLUDE") + if env and os.path.isdir(os.path.join(env, "nanovdb")): + opts[0] = f"-I{env}" + else: + print(f"NanoVDB headers not found at {inc!r}. In a source tree set " + "NANOVDB_INCLUDE to the dir containing nanovdb/NanoVDB.h.") + return None + return tuple(opts) + + +class Filter: + """Holds the compiled kernels + bound ops and runs the level-set filter.""" + + def __init__(self, cp, band=3, deform_iters=4, normalize_iters=5): + self.cp = cp + self.tc = nanovdb.tools.cuda + self.band = band + self.deform_iters = deform_iters + self.normalize_iters = normalize_iters + options = _include_options() + if options is None: + raise SystemExit(1) + m = cp.RawModule(code=KERNEL_SRC, backend="nvcc", options=options) + self.k_decode = m.get_function("decode_coords") + self.k_laplacian = m.get_function("laplacian_step") + self.k_godunov = m.get_function("godunov_step") + self.k_extrapolate = m.get_function("extrapolate") + + # ---- device OnIndex grid + VBM bookkeeping ------------------------------- + def setup(self, handle): + cp = self.cp + grid = handle.deviceGrid(0) + if grid is None or grid.data_ptr() == 0: + handle.deviceUpload(0, True) + grid = handle.deviceGrid(0) + vbm = self.tc.buildVoxelBlockManager(grid, log2_block_width=LOG2_BLOCK_WIDTH) + n, bc = int(vbm.lastOffset()), int(vbm.blockCount()) + coords = cp.zeros((n + 1, 3), dtype=cp.int32) + self.k_decode((bc,), (BLOCK_WIDTH,), + (grid.data_ptr(), vbm.first_leaf_id_ptr(), vbm.jump_map_ptr(), + np.uint64(vbm.firstOffset()), coords)) + cp.cuda.runtime.deviceSynchronize() + return dict(handle=handle, grid=grid, vbm=vbm, n=n, bc=bc, + fo=np.uint64(vbm.firstOffset()), + fid=vbm.first_leaf_id_ptr(), jmp=vbm.jump_map_ptr(), + gptr=grid.data_ptr(), coords=coords) + + def _vbm(self, g): + return (g["gptr"], g["fid"], g["jmp"], g["fo"]) + + # ---- one full LevelSetFilter iteration ----------------------------------- + def step(self, g, vals, vx, half_width): + cp = self.cp + bg = np.float32(half_width) + buf = cp.empty_like(vals) + # 1. DEFORM: Laplacian flow. + for _ in range(self.deform_iters): + self.k_laplacian((g["bc"],), (BLOCK_WIDTH,), (*self._vbm(g), bg, vals, buf)) + vals, buf = buf, vals + # 2. RENORMALIZE: Godunov reinitialisation. + for _ in range(self.normalize_iters): + self.k_godunov((g["bc"],), (BLOCK_WIDTH,), + (*self._vbm(g), np.float32(vx), np.float32(0.3 * vx), bg, vals, buf)) + vals, buf = buf, vals + cp.cuda.runtime.deviceSynchronize() + # 3. REBUILD BAND: dilateGrid -> inject -> extrapolate -> prune -> inject. + gd = self.setup(self.tc.dilateGrid(g["grid"], op=NN_FACE)) + vals_d = cp.full(gd["n"] + 1, SENTINEL, dtype=cp.float32) + self.tc.inject(g["grid"], gd["grid"], vals, vals_d) + ebuf = cp.empty_like(vals_d) + self.k_extrapolate((gd["bc"],), (BLOCK_WIDTH,), (*self._vbm(gd), np.float32(vx), vals_d, ebuf)) + vals_d = ebuf + predicate = cp.abs(vals_d) <= half_width + leaf_masks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) # activeVoxelCount*8 + self.tc.injectPredicateToMask(gd["grid"], predicate, leaf_masks) + gp = self.setup(self.tc.pruneGrid(gd["grid"], leaf_masks)) + vals_p = cp.full(gp["n"] + 1, half_width, dtype=cp.float32) + self.tc.inject(gd["grid"], gp["grid"], vals_d, vals_p) + return gp, vals_p + + def surface_radius(self, g, vals, vx): + cp = self.cp + v = vals[1:g["n"] + 1] + near = cp.abs(v) < 0.5 * vx + c = g["coords"][1:g["n"] + 1][near].astype(cp.float64) * vx + return float(cp.mean(cp.linalg.norm(c, axis=1))) if int(near.sum()) else float("nan") + + +def read_to_device(flt, path): + """Read a .nvdb (FloatGrid OR OnIndex+SDF) -> (device-grid dict, sidecar, vx, half_width, style).""" + cp = flt.cp + io, T = nanovdb.io, nanovdb.tools + host = io.readGrid(path) + gtype = host.gridType(0) + vx = float(host.grid(0).voxelSize()[0]) + half_width = flt.band * vx + tmp = None + if gtype == nanovdb.GridType.Float: + # Bake the per-voxel SDF into an OnIndex blind channel; read it on the + # host via grid.getBlindData (value-index order: [0]=background, 1..N). + idx_host = T.createOnIndexGrid(host.grid(0), channels=1, + include_stats=False, include_tiles=False) + sdf = np.array(idx_host.grid(0).getBlindData(SDF_BLIND_CHANNEL), dtype=np.float32) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() + io.writeGrid(tmp.name, idx_host) + dev = io.deviceReadGrid(tmp.name) + elif gtype == nanovdb.GridType.OnIndex: + if host.grid(0).blindDataCount() == 0: + raise SystemExit(f"{path}: OnIndex grid has no blind-data SDF channel.") + sdf = np.array(host.grid(0).getBlindData(SDF_BLIND_CHANNEL), dtype=np.float32) + dev = io.deviceReadGrid(path) + else: + raise SystemExit(f"{path}: unsupported grid type {gtype} " + "(expected Float or OnIndex).") + dev.deviceUpload(0, True) + g = flt.setup(dev) + if sdf.shape[0] != g["n"] + 1: + raise SystemExit(f"{path}: SDF channel length {sdf.shape[0]} != activeVoxelCount+1 " + f"({g['n'] + 1}); the OnIndex grid must use contiguous voxel " + "indexing (built with include_stats=False, include_tiles=False).") + vals = cp.asarray(sdf) # value-indexed sidecar; vals[0] is the background slot + if tmp is not None: + os.unlink(tmp.name) + return g, vals, vx, half_width, gtype + + +def write_output(flt, g, vals, vx, path, style, name="filtered"): + """Bake the final (coords, SDF) into a host FloatGrid; write it in `style`. + + style == GridType.Float -> a FloatGrid .nvdb + style == GridType.OnIndex -> an OnIndexGrid .nvdb with the SDF in blind channel 0 + """ + cp = flt.cp + T, io = nanovdb.tools, nanovdb.io + coords = cp.asnumpy(g["coords"]) + v = cp.asnumpy(vals) + builder = T.build.FloatGrid(float(flt.band * vx)) + builder.setName(name) + builder.setTransform(vx) + acc = builder.getAccessor() + Coord = nanovdb.math.Coord + for k in range(1, g["n"] + 1): + i, j, kk = coords[k] + acc.setValue(Coord(int(i), int(j), int(kk)), float(v[k])) + fh = builder.to_nanovdb() + if style == nanovdb.GridType.OnIndex: + idx_out = T.createOnIndexGrid(fh.grid(0), channels=1, + include_stats=False, include_tiles=False) + io.writeGrid(path, idx_out) + else: + io.writeGrid(path, fh) + + +def _gpu_or_skip(): + """Return the cupy module, or None (with a printed reason) if GPU filtering + is unavailable -- lets the example self-skip cleanly like the others.""" + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. Skipping.") + return None + try: + import cupy as cp + except ImportError: + print("This example requires CuPy (plus nvcc on PATH or $NVCC). Skipping.") + return None + return cp + + +def filter_file(in_path, out_path, outer_iters=6): + cp = _gpu_or_skip() + if cp is None: + return None + flt = Filter(cp) + g, vals, vx, half_width, gtype = read_to_device(flt, in_path) + print(f"read {in_path}: {gtype}, {g['n']} active voxels, voxelSize {vx:g}") + r0 = flt.surface_radius(g, vals, vx) + for it in range(outer_iters): + g, vals = flt.step(g, vals, vx, half_width) + r = flt.surface_radius(g, vals, vx) + print(f" iter {it + 1}: {g['n']:7d} active, surface radius = {r:.4f}") + write_output(flt, g, vals, vx, out_path, gtype) + style = "OnIndex+SDF" if gtype == nanovdb.GridType.OnIndex else "FloatGrid" + print(f"wrote {out_path}: {style} (same style as input), {g['n']} active voxels " + f"(surface radius {r0:.4f} -> {r:.4f})") + return r0, r, gtype + + +def self_test(): + """No-args run: build sphere .nvdb files of both styles, filter, assert invariants.""" + if _gpu_or_skip() is None: + return + io, T, GT = nanovdb.io, nanovdb.tools, nanovdb.GridType + tmps = [tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) for _ in range(4)] + for t in tmps: + t.close() + f_in, f_out, o_in, o_out = (t.name for t in tmps) + + # 1. FloatGrid in -> FloatGrid out. + io.writeGrid(f_in, T.createLevelSetSphere(radius=20.0, voxelSize=1.0, name="sphere")) + print("self-test 1: FloatGrid sphere -> filter -> FloatGrid") + r0, r, style = filter_file(f_in, f_out, outer_iters=6) + rb = io.readGrid(f_out) + assert style == GT.Float and rb.gridType(0) == GT.Float, "output style is not FloatGrid" + assert r < r0 - 0.05, "sphere did not shrink under curvature flow" + + # 2. OnIndex+SDF in -> OnIndex+SDF out (style preserved). + sphere = T.createLevelSetSphere(radius=18.0, voxelSize=1.0, name="sphere") + io.writeGrid(o_in, T.createOnIndexGrid(sphere.grid(0), channels=1, + include_stats=False, include_tiles=False)) + print("self-test 2: OnIndex+SDF sphere -> filter -> OnIndex+SDF") + r0b, rb2, style2 = filter_file(o_in, o_out, outer_iters=4) + ro = io.readGrid(o_out) + assert style2 == GT.OnIndex and ro.gridType(0) == GT.OnIndex, "output style is not OnIndex" + assert ro.grid(0).blindDataCount() >= 1, "OnIndex output has no SDF blind channel" + assert rb2 < r0b - 0.05, "sphere did not shrink under curvature flow" + + print("OK: both styles filter correctly and the output style matches the input.") + for n in (f_in, f_out, o_in, o_out): + os.unlink(n) + + +def main(argv): + if len(argv) >= 3: + outer = int(argv[3]) if len(argv) >= 4 else 6 + filter_file(argv[1], argv[2], outer) + elif len(argv) == 1: + self_test() + else: + print(__doc__) + raise SystemExit("usage: cupy_levelset_filter.py input.nvdb output.nvdb " + "[outer_iterations] (no args = self-test)") + + +if __name__ == "__main__": + main(sys.argv) From a5d1a00dfbefdfd4433fa0289b5d7f0229dc4174 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 3 Jun 2026 03:35:40 +0000 Subject: [PATCH 27/48] nanovdb python: add tools.cuda.gatherBoxStencil (dense neighbour gather) Materialise, for every active voxel of an OnIndex device grid, the values of its 3x3x3 neighbourhood into a dense (valueCount, 27) device array, so tile / array frameworks that cannot pointer-chase the VDB tree (CuPy, cuTile, ...) can run VDB stencils on a plain dense array. A transient VoxelBlockManager is built internally; each block decodes its inverse maps and calls the device computeBoxStencil, then writes the 27 neighbour values per voxel (inactive spokes read the sidecar's background slot, value index 0). Bound for float and double sidecars. Column j is the 3x3x3 spoke (di+1)*9+(dj+1)*3+(dk+1): centre j=13, the six faces j = 4, 10, 12, 14, 16, 22. This is the gather half of the VBM stencil pipeline (decodeInverseMaps + computeBoxStencil are device-only); pairing it with a dense sidecar lets the per-voxel arithmetic run in CuPy/cuTile without a hand-written CUDA kernel. Validated locally on a Blackwell GPU: with values[k]=k the centre column equals each voxel's own value index and all six opposite-face pairs are symmetric (k's +x neighbour's -x neighbour is k); float and double overloads agree. Signed-off-by: Jonathan Swartz --- .../python/cuda/PyDeviceVoxelBlockManager.cu | 80 +++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu index 0b052e0de7..7074b3f272 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu @@ -316,10 +316,90 @@ static void defineBuild(nb::module_& m) // / jumpMap device pointers (first_leaf_id_ptr / jump_map_ptr above) and the // device grid pointer. +// ------------------- gatherBoxStencil (VBM box-stencil gather) ------------- +// +// Materialise, for every active voxel, the values of its 3x3x3 neighbourhood +// into a dense (valueCount, 27) array -- the "dense-ise the sparse stencil" +// bridge that lets tile / array frameworks (CuPy, cuTile, ...) run VDB stencils +// without pointer-chasing the tree. Column j is the 3x3x3 spoke +// (di+1)*9 + (dj+1)*3 + (dk+1): the centre is column 13 and the six faces are +// columns 4, 10, 12, 14, 16, 22. Inactive neighbours read the sidecar's +// background slot (value index 0). +template +__global__ void gatherBoxStencilKernel( + const NanoGrid* grid, + const uint32_t* firstLeafID, const uint64_t* jumpMap, uint64_t firstOffset, + const T* values, T* out) +{ + constexpr int BW = 1 << Log2BlockWidth; + constexpr int JML = BW / 64; + using VBM = nanovdb::tools::cuda::VoxelBlockManager; + __shared__ uint32_t smem_leafIndex[BW]; + __shared__ uint16_t smem_voxelOffset[BW]; + const uint64_t blockFirstOffset = firstOffset + uint64_t(blockIdx.x) * BW; + VBM::template decodeInverseMaps( + grid, firstLeafID[blockIdx.x], &jumpMap[uint64_t(blockIdx.x) * JML], + blockFirstOffset, smem_leafIndex, smem_voxelOffset); + const int tID = threadIdx.x; + if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + uint64_t st[27]; + VBM::template computeBoxStencil( + grid, smem_leafIndex, smem_voxelOffset, st); + const uint64_t c = st[13]; // centre value index + #pragma unroll + for (int j = 0; j < 27; ++j) out[c * 27 + j] = values[st[j]]; // 0 -> background +} + +template void defineGatherBoxStencil(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::handle py_grid, + nb::ndarray, nb::c_contig, nb::device::cuda> values, + nb::ndarray, nb::c_contig, nb::device::cuda> out, + int log2_block_width, uintptr_t stream) { + auto* d_grid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencil"); + cudaStream_t s = reinterpret_cast(stream); + const T* dVals = values.data(); + T* dOut = out.data(); + // Build a transient VBM, then one block per VBM block decodes and + // gathers the 27 neighbour values; pure CUDA, so release the GIL. + nb::gil_scoped_release release; + dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + constexpr int LBW = decltype(W)::value; + auto handle = nanovdb::tools::cuda::buildVoxelBlockManager< + LBW, nanovdb::cuda::DeviceBuffer>(d_grid, 0, 0, 0, s); + const uint32_t bc = static_cast(handle.blockCount()); + if (bc) + gatherBoxStencilKernel<<>>( + d_grid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), + handle.firstOffset(), dVals, dOut); + cudaCheck(cudaStreamSynchronize(s)); + return 0; + }); + }, + "device_grid"_a, "values"_a, "out"_a, "log2_block_width"_a = 9, "stream"_a = 0, + "Gather the 3x3x3 box-stencil neighbour values of every active voxel " + "into a dense (valueCount, 27) array -- the bridge that lets tile / " + "array frameworks (CuPy, cuTile, ...) run VDB stencils without pointer-" + "chasing the tree. device_grid is an OnIndex device grid from " + "DeviceGridHandle.deviceGrid(n); values is a 1-D device array indexed by " + "value index (the per-voxel sidecar; entry 0 is the background slot); out " + "is a 2-D device array of shape (rows, 27) with rows >= valueCount, " + "filled so out[k, j] is voxel k's neighbour value at 3x3x3 spoke " + "j = (di+1)*9+(dj+1)*3+(dk+1) (centre j=13; the six faces are " + "j = 4, 10, 12, 14, 16, 22). Inactive neighbours read values[0]. A " + "transient VoxelBlockManager is built internally at log2_block_width " + "(6/7/8/9). stream is a raw CUDA stream handle (Python int; 0 = default " + "stream)."); +} + void defineDeviceVoxelBlockManager(nb::module_& m) { defineHandle(m); defineBuild(m); + defineGatherBoxStencil(m, "gatherBoxStencil"); + defineGatherBoxStencil(m, "gatherBoxStencil"); } } // namespace pynanovdb From 6de2af509cc09deb442771f5c0759497d9011790 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 3 Jun 2026 03:42:46 +0000 Subject: [PATCH 28/48] nanovdb python: add cupy_gather_stencil, the kernel-free VBM-stencil example The kernel-free counterpart to cupy_levelset_filter.py. Instead of calling the device VoxelBlockManager decode + computeBoxStencil from a hand-written cupy.RawModule kernel, it uses tools.cuda.gatherBoxStencil to materialise every active voxel's 3x3x3 neighbourhood into a dense (valueCount, 27) array, then runs the Laplacian-flow deform and first-order Godunov reinitialisation as plain CuPy array math -- no CUDA kernel, no thread indexing, no coordinates. (The same dense array is what a cuTile @ct.kernel would ct.load as tiles.) The sidecar's background slot is set to a sentinel so inactive neighbours are detectable in the gathered array, which lets the pure-CuPy code apply the same sign-consistent clamped boundary condition (copysign(background, phi_centre)) the kernel version uses. Fixed-topology stencils only; the narrow-band retrack and file I/O stay in cupy_levelset_filter.py. Requires only CuPy (no nvcc / NanoVDB headers, since nothing is compiled at runtime); self-skips otherwise. Verified on a Blackwell GPU: a few Laplacian steps raise mean||grad phi|-1| from ~0 to ~0.057, and the Godunov reinit brings it back to ~0.010 -- all computed kernel-free from gatherBoxStencil. Listed in python/examples/README.md. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/examples/README.md | 1 + .../python/examples/cupy_gather_stencil.py | 153 ++++++++++++++++++ 2 files changed, 154 insertions(+) create mode 100644 nanovdb/nanovdb/python/examples/cupy_gather_stencil.py diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index 99f1731a94..5065b49bef 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -41,6 +41,7 @@ GPU-array framework it uses is unavailable. | [`numba_cuda.py`](numba_cuda.py) | Adopting a NanoVDB device buffer as a zero-copy `numba.cuda` array (Numba can't parse the C++ ABI, so it operates on the raw buffer). Requires Numba. | | [`triton_kernel.py`](triton_kernel.py) | Handing a NanoVDB device buffer to a Triton kernel via a zero-copy `torch.from_dlpack` tensor. Requires Triton + PyTorch. | | [`cupy_levelset_filter.py`](cupy_levelset_filter.py) | A full GPU level-set filter (`tools::LevelSetFilter`-style diffusion + Godunov renormalisation + narrow-band retrack) on a `.nvdb` file, driven by the device `VoxelBlockManager`: `buildVoxelBlockManager` plus `decodeInverseMaps` / `computeBoxStencil` called from a `cupy.RawModule` (nvcc backend), with `dilateGrid` / `inject` / `injectPredicateToMask` / `pruneGrid` for the retrack. Reads/writes either a `FloatGrid` or an `OnIndexGrid` with the SDF in a blind channel, preserving the input style. No args runs a self-test. Requires CuPy + nvcc. | +| [`cupy_gather_stencil.py`](cupy_gather_stencil.py) | The **kernel-free** counterpart: `tools.cuda.gatherBoxStencil` materialises each active voxel's 3×3×3 neighbourhood into a dense `(valueCount, 27)` array, then the Laplacian deform and Godunov reinitialisation run as plain **CuPy** array math — no `cupy.RawModule`, no CUDA kernel, no coordinates (the same dense array a cuTile `@ct.kernel` would `ct.load`). Verified by an invariant: reinitialisation drives \|∇φ\|→1 after a deform degrades it. Requires only CuPy (no nvcc/headers). | For full API signatures and per-argument docstrings, use Python's `help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`. diff --git a/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py b/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py new file mode 100644 index 0000000000..c5036fadf1 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py @@ -0,0 +1,153 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Kernel-free VBM stencils in CuPy via nanovdb.tools.cuda.gatherBoxStencil. + +`cupy_levelset_filter.py` runs the per-voxel level-set stencils (Laplacian +deform, Godunov reinitialisation) as hand-written `cupy.RawModule` kernels that +call the device `VoxelBlockManager` decode + `computeBoxStencil`. This example +shows the *kernel-free* alternative: `gatherBoxStencil` materialises every +active voxel's 3x3x3 neighbourhood into a dense `(valueCount, 27)` array, and +the stencil arithmetic then runs as plain **CuPy** array math — no CUDA kernel, +no thread indexing, no coordinates. The same dense `(N, 27)` array is exactly +what a cuTile `@ct.kernel` would `ct.load` as tiles. + +Data model: an OnIndexGrid plus a value-indexed float sidecar `phi` (entry 0 is +the background slot). `gatherBoxStencil(grid, phi, out)` fills `out[k, j]` with +the value of voxel `k`'s neighbour at 3x3x3 spoke `j` (centre `j=13`; the six +faces `j = 4, 22 (-/+x), 10, 16 (-/+y), 12, 14 (-/+z)`); inactive spokes read +`phi[0]`. We set `phi[0]` to a sentinel so inactive neighbours are detectable in +the gathered array — that lets us apply the same sign-consistent clamped +boundary condition the kernel version uses (`copysign(background, phi_centre)`), +in pure CuPy. + +Scope: fixed-topology stencils only (deform + reinit). The narrow-band retrack +(`dilateGrid`/`inject`/`pruneGrid`) and file I/O live in `cupy_levelset_filter.py`. +Verified by a physical invariant: after a few Laplacian steps degrade the +signed-distance property, the Godunov reinitialisation drives |grad phi| back +toward 1 — all computed kernel-free from the gathered neighbourhood. + +Requires CuPy and a CUDA-capable GPU; self-skips otherwise. +""" +import os +import tempfile + +import nanovdb + + +SENTINEL = 1.0e30 # phi[0]: marks inactive neighbours in the gathered array +# 3x3x3 spoke columns for the six faces, in -/+ x, y, z order. +FACES = [4, 22, 10, 16, 12, 14] + + +def _gpu_or_skip(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. Skipping.") + return None + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Skipping.") + return None + return cp + + +def load_sphere(cp, radius=20.0, vx=1.0): + """Build a sphere level set as an OnIndex device grid + value-indexed phi (CuPy).""" + import numpy as np + io, T = nanovdb.io, nanovdb.tools + fg = T.createLevelSetSphere(radius=radius, voxelSize=vx, name="sphere") + onh = T.createOnIndexGrid(fg.grid(0), channels=1, + include_stats=False, include_tiles=False) + # The SDF is baked into blind channel 0 in value-index order (entry 0 is the + # background slot) -- read it on the host with grid.getBlindData (no kernel). + phi = cp.asarray(np.array(onh.grid(0).getBlindData(0), dtype=np.float32)) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() + io.writeGrid(tmp.name, onh) + dh = io.deviceReadGrid(tmp.name); dh.deviceUpload(0, True) + os.unlink(tmp.name) + phi[0] = SENTINEL # inactive-neighbour marker + return dh, dh.deviceGrid(0), int(phi.shape[0]) - 1, phi + + +def gather_faces(cp, grid, phi, n, background): + """gatherBoxStencil -> the 6 face-neighbour values with the clamped-background BC.""" + nbrs = cp.empty((n + 1, 27), dtype=cp.float32) + nanovdb.tools.cuda.gatherBoxStencil(grid, phi, nbrs) + c = phi[1:n + 1] + f = nbrs[1:n + 1][:, FACES] # (n, 6): -x,+x,-y,+y,-z,+z + # Inactive spokes came back as SENTINEL; substitute the sign-consistent + # clamped background, exactly as the kernel's readNbr does. + f = cp.where(f == SENTINEL, cp.copysign(cp.float32(background), c)[:, None], f) + return c, f + + +def laplacian_step(cp, grid, phi, n, background): + """phi += (sum6 - 6 phi) / 6 -- pure CuPy on the gathered neighbourhood.""" + c, f = gather_faces(cp, grid, phi, n, background) + out = phi.copy() + out[1:n + 1] = c + (f.sum(axis=1) - 6.0 * c) / 6.0 + out[0] = SENTINEL + return out + + +def godunov_step(cp, grid, phi, n, dx, dt, background): + """phi -= dt*S(phi)*(|grad phi| - 1) with a first-order Godunov upwind gradient.""" + c, f = gather_faces(cp, grid, phi, n, background) + xm, xp, ym, yp, zm, zp = (f[:, i] for i in range(6)) + s = c / cp.sqrt(c * c + dx * dx) + + def g(dm, dp): # Rouy-Tourin upwind selection + return cp.where(s > 0, + cp.maximum(cp.maximum(dm, 0.0) ** 2, cp.minimum(dp, 0.0) ** 2), + cp.maximum(cp.minimum(dm, 0.0) ** 2, cp.maximum(dp, 0.0) ** 2)) + + grad = cp.sqrt(g((c - xm) / dx, (xp - c) / dx) + + g((c - ym) / dx, (yp - c) / dx) + + g((c - zm) / dx, (zp - c) / dx)) + out = phi.copy() + out[1:n + 1] = c - dt * s * (grad - 1.0) + out[0] = SENTINEL + return out + + +def grad_error(cp, grid, phi, n, dx): + """mean ||grad phi| - 1| over band-interior voxels (all six faces active).""" + nbrs = cp.empty((n + 1, 27), dtype=cp.float32) + nanovdb.tools.cuda.gatherBoxStencil(grid, phi, nbrs) + f = nbrs[1:n + 1][:, FACES] + interior = (f != SENTINEL).all(axis=1) + xm, xp, ym, yp, zm, zp = (f[:, i] for i in range(6)) + mag = cp.sqrt(((xp - xm) / (2 * dx)) ** 2 + + ((yp - ym) / (2 * dx)) ** 2 + + ((zp - zm) / (2 * dx)) ** 2) + return float(cp.mean(cp.abs(mag - 1.0)[interior])) if int(interior.sum()) else float("nan") + + +def main(): + cp = _gpu_or_skip() + if cp is None: + return + vx, background = 1.0, 3.0 + dh, grid, n, phi = load_sphere(cp, radius=20.0, vx=vx) # keep dh alive (owns the grid) + print(f"sphere OnIndex level set: {n} active voxels") + print(f" initial mean||grad|-1| = {grad_error(cp, grid, phi, n, vx):.4f}") + + # DEFORM: a few kernel-free Laplacian steps degrade the signed-distance property. + for _ in range(4): + phi = laplacian_step(cp, grid, phi, n, background) + err_deformed = grad_error(cp, grid, phi, n, vx) + print(f" after deform mean||grad|-1| = {err_deformed:.4f}") + + # RENORMALISE: kernel-free Godunov reinit drives |grad phi| back toward 1. + for _ in range(8): + phi = godunov_step(cp, grid, phi, n, vx, 0.3 * vx, background) + err_reinit = grad_error(cp, grid, phi, n, vx) + print(f" after reinit mean||grad|-1| = {err_reinit:.4f}") + + assert err_reinit < err_deformed, "kernel-free Godunov reinit did not restore |grad phi|" + print("OK: deform + reinit ran as pure CuPy on gatherBoxStencil (no CUDA kernel); " + "reinitialisation restored |grad phi| -> 1.") + + +if __name__ == "__main__": + main() From c47366ccc290a41c5f1d6fa8030104b42323baa3 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 3 Jun 2026 03:55:20 +0000 Subject: [PATCH 29/48] nanovdb python: add tools.cuda.activeVoxelCoords; make cupy_gather_stencil a full kernel-free filter Binding: activeVoxelCoords(device_grid, out) writes each active voxel's index-space coordinate into a dense (valueCount, 3) int32 array keyed by value index -- the decode companion to gatherBoxStencil. It is the VBM coordinate decode exposed as one bound op (transient VBM built internally), so callers can recover per-voxel positions without a hand-written decode kernel (e.g. to bake a sidecar result back into a grid). Example: with activeVoxelCoords closing the last gap (the write needed per-voxel coordinates), cupy_gather_stencil.py becomes the full kernel-free counterpart to cupy_levelset_filter.py -- the same .nvdb->.nvdb level-set filter (deform + Godunov reinit + narrow-band retrack, style-preserving) with no cupy.RawModule / CUDA kernel anywhere: * read : createOnIndexGrid + grid.getBlindData * deform : gatherBoxStencil -> dense (N,27); Laplacian in CuPy * renorm : same gather; first-order Godunov in CuPy * retrack: dilateGrid -> inject -> extrapolate (CuPy on the gather) -> |phi|<=halfWidth predicate (CuPy) -> injectPredicateToMask -> pruneGrid -> inject * write : activeVoxelCoords -> tools.build.FloatGrid, written in the input's style (FloatGrid, or OnIndex with the SDF in a blind channel) All per-voxel data lives in dense arrays (the shape a cuTile @ct.kernel would ct.load). Requires only CuPy -- no nvcc / NanoVDB headers, since nothing is compiled at runtime. Validated on a Blackwell GPU: activeVoxelCoords reproduces the exact active coordinate set; the kernel-free filter self-test matches the RawModule version (FloatGrid sphere 20.02 -> 19.52, OnIndex sphere 18.05 -> 17.67) for both input styles, output style preserved. Listed in python/examples/README.md. Signed-off-by: Jonathan Swartz --- .../python/cuda/PyDeviceVoxelBlockManager.cu | 70 ++++ nanovdb/nanovdb/python/examples/README.md | 2 +- .../python/examples/cupy_gather_stencil.py | 321 +++++++++++++----- 3 files changed, 299 insertions(+), 94 deletions(-) diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu index 7074b3f272..affe47c1aa 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu @@ -394,12 +394,82 @@ template void defineGatherBoxStencil(nb::module_& m, const char* nam "stream)."); } +// ------------------- activeVoxelCoords (VBM coordinate decode) ------------- +// +// Write each active voxel's index-space coordinate into a dense (valueCount, 3) +// int32 array, keyed by value index -- the decode companion to +// gatherBoxStencil. Lets callers recover "where is value index k" (e.g. to bake +// a result back into a grid, or to scatter to a dense field) without a +// hand-written decode kernel. +template +__global__ void activeVoxelCoordsKernel( + const NanoGrid* grid, + const uint32_t* firstLeafID, const uint64_t* jumpMap, uint64_t firstOffset, + int32_t* out) +{ + constexpr int BW = 1 << Log2BlockWidth; + constexpr int JML = BW / 64; + using VBM = nanovdb::tools::cuda::VoxelBlockManager; + __shared__ uint32_t smem_leafIndex[BW]; + __shared__ uint16_t smem_voxelOffset[BW]; + const uint64_t blockFirstOffset = firstOffset + uint64_t(blockIdx.x) * BW; + VBM::template decodeInverseMaps( + grid, firstLeafID[blockIdx.x], &jumpMap[uint64_t(blockIdx.x) * JML], + blockFirstOffset, smem_leafIndex, smem_voxelOffset); + const int tID = threadIdx.x; + if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + const auto& leaf = grid->tree().getFirstNode<0>()[smem_leafIndex[tID]]; + const Coord c = leaf.offsetToGlobalCoord(smem_voxelOffset[tID]); + const uint64_t idx = leaf.getValue(smem_voxelOffset[tID]); + out[idx * 3 + 0] = c[0]; + out[idx * 3 + 1] = c[1]; + out[idx * 3 + 2] = c[2]; +} + +void defineActiveVoxelCoords(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::handle py_grid, + nb::ndarray, nb::c_contig, nb::device::cuda> out, + int log2_block_width, uintptr_t stream) { + auto* d_grid = castOnIndexDeviceGrid(py_grid, "activeVoxelCoords"); + cudaStream_t s = reinterpret_cast(stream); + int32_t* dOut = out.data(); + nb::gil_scoped_release release; + dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + constexpr int LBW = decltype(W)::value; + auto handle = nanovdb::tools::cuda::buildVoxelBlockManager< + LBW, nanovdb::cuda::DeviceBuffer>(d_grid, 0, 0, 0, s); + const uint32_t bc = static_cast(handle.blockCount()); + if (bc) + activeVoxelCoordsKernel<<>>( + d_grid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), + handle.firstOffset(), dOut); + cudaCheck(cudaStreamSynchronize(s)); + return 0; + }); + }, + "device_grid"_a, "out"_a, "log2_block_width"_a = 9, "stream"_a = 0, + "Write each active voxel's index-space coordinate into a dense " + "(rows, 3) int32 device array keyed by value index (rows >= valueCount); " + "out[k] is the (i, j, k) coordinate of value index k (row 0, the " + "background slot, is left untouched). The decode companion to " + "gatherBoxStencil -- recovers per-voxel coordinates without a " + "hand-written decode kernel (e.g. to bake a sidecar result back into a " + "grid). device_grid is an OnIndex device grid from " + "DeviceGridHandle.deviceGrid(n); a transient VoxelBlockManager is built " + "internally at log2_block_width (6/7/8/9). stream is a raw CUDA stream " + "handle (Python int; 0 = default stream)."); +} + void defineDeviceVoxelBlockManager(nb::module_& m) { defineHandle(m); defineBuild(m); defineGatherBoxStencil(m, "gatherBoxStencil"); defineGatherBoxStencil(m, "gatherBoxStencil"); + defineActiveVoxelCoords(m, "activeVoxelCoords"); } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index 5065b49bef..28c3add954 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -41,7 +41,7 @@ GPU-array framework it uses is unavailable. | [`numba_cuda.py`](numba_cuda.py) | Adopting a NanoVDB device buffer as a zero-copy `numba.cuda` array (Numba can't parse the C++ ABI, so it operates on the raw buffer). Requires Numba. | | [`triton_kernel.py`](triton_kernel.py) | Handing a NanoVDB device buffer to a Triton kernel via a zero-copy `torch.from_dlpack` tensor. Requires Triton + PyTorch. | | [`cupy_levelset_filter.py`](cupy_levelset_filter.py) | A full GPU level-set filter (`tools::LevelSetFilter`-style diffusion + Godunov renormalisation + narrow-band retrack) on a `.nvdb` file, driven by the device `VoxelBlockManager`: `buildVoxelBlockManager` plus `decodeInverseMaps` / `computeBoxStencil` called from a `cupy.RawModule` (nvcc backend), with `dilateGrid` / `inject` / `injectPredicateToMask` / `pruneGrid` for the retrack. Reads/writes either a `FloatGrid` or an `OnIndexGrid` with the SDF in a blind channel, preserving the input style. No args runs a self-test. Requires CuPy + nvcc. | -| [`cupy_gather_stencil.py`](cupy_gather_stencil.py) | The **kernel-free** counterpart: `tools.cuda.gatherBoxStencil` materialises each active voxel's 3×3×3 neighbourhood into a dense `(valueCount, 27)` array, then the Laplacian deform and Godunov reinitialisation run as plain **CuPy** array math — no `cupy.RawModule`, no CUDA kernel, no coordinates (the same dense array a cuTile `@ct.kernel` would `ct.load`). Verified by an invariant: reinitialisation drives \|∇φ\|→1 after a deform degrades it. Requires only CuPy (no nvcc/headers). | +| [`cupy_gather_stencil.py`](cupy_gather_stencil.py) | The **kernel-free** counterpart to `cupy_levelset_filter.py`: the *same* full `.nvdb`→`.nvdb` level-set filter (deform + Godunov reinit + narrow-band retrack, style-preserving) with **no `cupy.RawModule` / CUDA kernel anywhere**. `tools.cuda.gatherBoxStencil` gives each voxel's 3×3×3 neighbourhood as a dense `(N, 27)` array and `tools.cuda.activeVoxelCoords` gives positions as `(N, 3)`, so all per-voxel math is plain **CuPy** (the same dense arrays a cuTile `@ct.kernel` would `ct.load`); the retrack uses the bound `dilateGrid` / `inject` / `injectPredicateToMask` / `pruneGrid`. No args runs a self-test over both input styles. Requires only CuPy (no nvcc / NanoVDB headers). | For full API signatures and per-argument docstrings, use Python's `help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`. diff --git a/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py b/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py index c5036fadf1..75bc3728d6 100644 --- a/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py +++ b/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py @@ -1,41 +1,55 @@ # Copyright Contributors to the OpenVDB Project # SPDX-License-Identifier: Apache-2.0 -"""Kernel-free VBM stencils in CuPy via nanovdb.tools.cuda.gatherBoxStencil. - -`cupy_levelset_filter.py` runs the per-voxel level-set stencils (Laplacian -deform, Godunov reinitialisation) as hand-written `cupy.RawModule` kernels that -call the device `VoxelBlockManager` decode + `computeBoxStencil`. This example -shows the *kernel-free* alternative: `gatherBoxStencil` materialises every -active voxel's 3x3x3 neighbourhood into a dense `(valueCount, 27)` array, and -the stencil arithmetic then runs as plain **CuPy** array math — no CUDA kernel, -no thread indexing, no coordinates. The same dense `(N, 27)` array is exactly -what a cuTile `@ct.kernel` would `ct.load` as tiles. - -Data model: an OnIndexGrid plus a value-indexed float sidecar `phi` (entry 0 is -the background slot). `gatherBoxStencil(grid, phi, out)` fills `out[k, j]` with -the value of voxel `k`'s neighbour at 3x3x3 spoke `j` (centre `j=13`; the six -faces `j = 4, 22 (-/+x), 10, 16 (-/+y), 12, 14 (-/+z)`); inactive spokes read -`phi[0]`. We set `phi[0]` to a sentinel so inactive neighbours are detectable in -the gathered array — that lets us apply the same sign-consistent clamped -boundary condition the kernel version uses (`copysign(background, phi_centre)`), -in pure CuPy. - -Scope: fixed-topology stencils only (deform + reinit). The narrow-band retrack -(`dilateGrid`/`inject`/`pruneGrid`) and file I/O live in `cupy_levelset_filter.py`. -Verified by a physical invariant: after a few Laplacian steps degrade the -signed-distance property, the Godunov reinitialisation drives |grad phi| back -toward 1 — all computed kernel-free from the gathered neighbourhood. - -Requires CuPy and a CUDA-capable GPU; self-skips otherwise. +"""A full GPU LevelSetFilter on .nvdb files with NO hand-written CUDA kernel. + +This is the kernel-free counterpart to `cupy_levelset_filter.py`. That example +runs the per-voxel stencils as `cupy.RawModule` kernels that call the device +VoxelBlockManager decode + `computeBoxStencil`; here every stage runs as plain +**CuPy** array math on top of bound NanoVDB ops -- no `RawModule`, no CUDA C++, +no `nvcc`: + + python cupy_gather_stencil.py input.nvdb output.nvdb [outer_iterations] + +How each stage stays kernel-free: + * READ `createOnIndexGrid` + `grid.getBlindData(0)` -> value-indexed SDF. + * DEFORM `gatherBoxStencil` -> dense (N, 27) neighbour values; the Laplacian + `phi += (sum6 - 6 phi)/6` is then a CuPy expression. + * RENORM same gather; a first-order Godunov reinit in CuPy. + * RETRACK `dilateGrid` -> `inject` -> extrapolate new voxels (CuPy on the + gather) -> `|phi|<=halfWidth` predicate (CuPy) -> `injectPredicate- + ToMask` -> `pruneGrid` -> `inject`. + * WRITE `activeVoxelCoords` -> per-voxel coords, baked into a grid with + `tools.build.FloatGrid`; written back in the input's style + (`FloatGrid`, or `OnIndexGrid` with the SDF in a blind channel). + +The whole per-voxel surface lives in dense arrays (`gatherBoxStencil` for the +neighbourhood, `activeVoxelCoords` for positions), which is exactly the shape a +tile framework such as cuTile consumes -- the CuPy math here would map onto a +`@ct.kernel` operating on the same `(N, 27)` / `(N, 3)` arrays. + +The sidecar's background slot `phi[0]` is set to a sentinel so inactive +neighbours are detectable in the gathered array; that lets the CuPy code apply +the same sign-consistent clamped boundary condition (`copysign(background, +phi_centre)`) the kernel version uses. + +Scope: first-order Godunov reinitialisation, no advection or alpha mask; the +output's inactive interior carries +background (no signed flood-fill). Run with +no arguments for a self-test over both input styles. Requires only CuPy and a +CUDA-capable GPU (no nvcc / NanoVDB headers); self-skips otherwise. """ import os +import sys import tempfile +import numpy as np + import nanovdb -SENTINEL = 1.0e30 # phi[0]: marks inactive neighbours in the gathered array -# 3x3x3 spoke columns for the six faces, in -/+ x, y, z order. +LOG2_BLOCK_WIDTH = 9 +SENTINEL = 1.0e30 # phi[0]: marks inactive neighbours in the gathered array +NN_FACE = 6 # nanovdb::tools::morphology::NN_FACE (6-face dilation) +# 3x3x3 spoke columns for the six faces, in -/+ x, y, z order (centre is col 13). FACES = [4, 22, 10, 16, 12, 14] @@ -51,103 +65,224 @@ def _gpu_or_skip(): return cp -def load_sphere(cp, radius=20.0, vx=1.0): - """Build a sphere level set as an OnIndex device grid + value-indexed phi (CuPy).""" - import numpy as np +def _setup(cp, handle): + """DeviceGridHandle -> {handle, grid, n}; n = active-voxel count (= valueCount-1).""" + grid = handle.deviceGrid(0) + if grid is None or grid.data_ptr() == 0: + handle.deviceUpload(0, True) + grid = handle.deviceGrid(0) + n = int(nanovdb.tools.cuda.buildVoxelBlockManager( + grid, log2_block_width=LOG2_BLOCK_WIDTH).lastOffset()) + return {"handle": handle, "grid": grid, "n": n} + + +def read_to_device(cp, path, band): + """Read a .nvdb (FloatGrid OR OnIndex+SDF) -> (g, phi, vx, half_width, style).""" io, T = nanovdb.io, nanovdb.tools - fg = T.createLevelSetSphere(radius=radius, voxelSize=vx, name="sphere") - onh = T.createOnIndexGrid(fg.grid(0), channels=1, - include_stats=False, include_tiles=False) - # The SDF is baked into blind channel 0 in value-index order (entry 0 is the - # background slot) -- read it on the host with grid.getBlindData (no kernel). - phi = cp.asarray(np.array(onh.grid(0).getBlindData(0), dtype=np.float32)) - tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() - io.writeGrid(tmp.name, onh) - dh = io.deviceReadGrid(tmp.name); dh.deviceUpload(0, True) - os.unlink(tmp.name) - phi[0] = SENTINEL # inactive-neighbour marker - return dh, dh.deviceGrid(0), int(phi.shape[0]) - 1, phi - - -def gather_faces(cp, grid, phi, n, background): - """gatherBoxStencil -> the 6 face-neighbour values with the clamped-background BC.""" + host = io.readGrid(path) + gtype = host.gridType(0) + vx = float(host.grid(0).voxelSize()[0]) + tmp = None + if gtype == nanovdb.GridType.Float: + onh = T.createOnIndexGrid(host.grid(0), channels=1, + include_stats=False, include_tiles=False) + sdf = np.array(onh.grid(0).getBlindData(0), dtype=np.float32) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() + io.writeGrid(tmp.name, onh) + dh = io.deviceReadGrid(tmp.name) + elif gtype == nanovdb.GridType.OnIndex: + if host.grid(0).blindDataCount() == 0: + raise SystemExit(f"{path}: OnIndex grid has no blind-data SDF channel.") + sdf = np.array(host.grid(0).getBlindData(0), dtype=np.float32) + dh = io.deviceReadGrid(path) + else: + raise SystemExit(f"{path}: unsupported grid type {gtype} (expected Float or OnIndex).") + g = _setup(cp, dh) + if sdf.shape[0] != g["n"] + 1: + raise SystemExit(f"{path}: SDF channel length {sdf.shape[0]} != activeVoxelCount+1 " + f"({g['n'] + 1}); the OnIndex grid must use contiguous voxel indexing " + "(built with include_stats=False, include_tiles=False).") + phi = cp.asarray(sdf) + phi[0] = SENTINEL # inactive-neighbour marker + if tmp is not None: + os.unlink(tmp.name) + return g, phi, vx, band * vx, gtype + + +def _gather_faces(cp, g, phi, background): + """gatherBoxStencil -> the 6 face values with the clamped-background BC.""" + n = g["n"] nbrs = cp.empty((n + 1, 27), dtype=cp.float32) - nanovdb.tools.cuda.gatherBoxStencil(grid, phi, nbrs) + nanovdb.tools.cuda.gatherBoxStencil(g["grid"], phi, nbrs) c = phi[1:n + 1] - f = nbrs[1:n + 1][:, FACES] # (n, 6): -x,+x,-y,+y,-z,+z - # Inactive spokes came back as SENTINEL; substitute the sign-consistent - # clamped background, exactly as the kernel's readNbr does. + f = nbrs[1:n + 1][:, FACES] f = cp.where(f == SENTINEL, cp.copysign(cp.float32(background), c)[:, None], f) return c, f -def laplacian_step(cp, grid, phi, n, background): - """phi += (sum6 - 6 phi) / 6 -- pure CuPy on the gathered neighbourhood.""" - c, f = gather_faces(cp, grid, phi, n, background) +def laplacian_step(cp, g, phi, background): + """phi += (sum6 - 6 phi)/6 -- pure CuPy on the gathered neighbourhood.""" + c, f = _gather_faces(cp, g, phi, background) out = phi.copy() - out[1:n + 1] = c + (f.sum(axis=1) - 6.0 * c) / 6.0 + out[1:g["n"] + 1] = c + (f.sum(axis=1) - 6.0 * c) / 6.0 out[0] = SENTINEL return out -def godunov_step(cp, grid, phi, n, dx, dt, background): - """phi -= dt*S(phi)*(|grad phi| - 1) with a first-order Godunov upwind gradient.""" - c, f = gather_faces(cp, grid, phi, n, background) +def godunov_step(cp, g, phi, dx, dt, background): + """phi -= dt*S(phi)*(|grad phi| - 1) -- first-order Godunov, pure CuPy.""" + c, f = _gather_faces(cp, g, phi, background) xm, xp, ym, yp, zm, zp = (f[:, i] for i in range(6)) s = c / cp.sqrt(c * c + dx * dx) - def g(dm, dp): # Rouy-Tourin upwind selection + def gd(dm, dp): return cp.where(s > 0, cp.maximum(cp.maximum(dm, 0.0) ** 2, cp.minimum(dp, 0.0) ** 2), cp.maximum(cp.minimum(dm, 0.0) ** 2, cp.maximum(dp, 0.0) ** 2)) - grad = cp.sqrt(g((c - xm) / dx, (xp - c) / dx) - + g((c - ym) / dx, (yp - c) / dx) - + g((c - zm) / dx, (zp - c) / dx)) + grad = cp.sqrt(gd((c - xm) / dx, (xp - c) / dx) + + gd((c - ym) / dx, (yp - c) / dx) + + gd((c - zm) / dx, (zp - c) / dx)) out = phi.copy() - out[1:n + 1] = c - dt * s * (grad - 1.0) + out[1:g["n"] + 1] = c - dt * s * (grad - 1.0) out[0] = SENTINEL return out -def grad_error(cp, grid, phi, n, dx): - """mean ||grad phi| - 1| over band-interior voxels (all six faces active).""" +def _extrapolate(cp, g, phi, dx): + """Fill freshly-dilated (sentinel) voxels from the nearest in-band face + neighbour: phi = phi_nbr + sign(phi_nbr)*dx -- pure CuPy on the gather.""" + n = g["n"] nbrs = cp.empty((n + 1, 27), dtype=cp.float32) - nanovdb.tools.cuda.gatherBoxStencil(grid, phi, nbrs) - f = nbrs[1:n + 1][:, FACES] - interior = (f != SENTINEL).all(axis=1) - xm, xp, ym, yp, zm, zp = (f[:, i] for i in range(6)) - mag = cp.sqrt(((xp - xm) / (2 * dx)) ** 2 - + ((yp - ym) / (2 * dx)) ** 2 - + ((zp - zm) / (2 * dx)) ** 2) - return float(cp.mean(cp.abs(mag - 1.0)[interior])) if int(interior.sum()) else float("nan") + nanovdb.tools.cuda.gatherBoxStencil(g["grid"], phi, nbrs) + c = phi[1:n + 1] + f = nbrs[1:n + 1][:, FACES] # raw: inactive spokes are SENTINEL + known = f != SENTINEL + best = cp.take_along_axis( + f, cp.argmin(cp.where(known, cp.abs(f), cp.inf), axis=1)[:, None], axis=1)[:, 0] + out = phi.copy() + fill = (c == SENTINEL) & known.any(axis=1) + out[1:n + 1] = cp.where(fill, best + cp.copysign(cp.float32(dx), best), c) + out[0] = SENTINEL + return out + + +def rebuild(cp, g, phi, vx, half_width): + """Narrow-band retrack: dilate -> inject -> extrapolate -> prune -> inject.""" + TC = nanovdb.tools.cuda + gd = _setup(cp, TC.dilateGrid(g["grid"], op=NN_FACE)) + phi_d = cp.full(gd["n"] + 1, SENTINEL, dtype=cp.float32) + TC.inject(g["grid"], gd["grid"], phi, phi_d) # carry old phi (intersection) + phi_d = _extrapolate(cp, gd, phi_d, vx) # fill the new ring (CuPy) + predicate = cp.abs(phi_d) <= half_width # phi_d[0]=SENTINEL -> False + leaf_masks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) + TC.injectPredicateToMask(gd["grid"], predicate, leaf_masks) + gp = _setup(cp, TC.pruneGrid(gd["grid"], leaf_masks)) + phi_p = cp.full(gp["n"] + 1, SENTINEL, dtype=cp.float32) + TC.inject(gd["grid"], gp["grid"], phi_d, phi_p) + return gp, phi_p + + +def surface_radius(cp, g, phi, vx): + """Mean world radius of zero-crossing voxels (|phi| < dx/2), coords via activeVoxelCoords.""" + coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) + nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) + v = phi[1:g["n"] + 1] + near = cp.abs(v) < 0.5 * vx + c = coords[1:g["n"] + 1][near].astype(cp.float64) * vx + return float(cp.mean(cp.linalg.norm(c, axis=1))) if int(near.sum()) else float("nan") + + +def write_output(cp, g, phi, vx, path, style, band, name="filtered"): + """Bake (coords, phi) into a host FloatGrid; write in `style` (Float or OnIndex+SDF).""" + T, io = nanovdb.tools, nanovdb.io + coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) + nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) # kernel-free coord decode + coords = cp.asnumpy(coords) + v = cp.asnumpy(phi) + builder = T.build.FloatGrid(float(band * vx)) + builder.setName(name) + builder.setTransform(vx) + acc = builder.getAccessor() + Coord = nanovdb.math.Coord + for k in range(1, g["n"] + 1): + i, j, kk = coords[k] + acc.setValue(Coord(int(i), int(j), int(kk)), float(v[k])) + fh = builder.to_nanovdb() + if style == nanovdb.GridType.OnIndex: + io.writeGrid(path, T.createOnIndexGrid(fh.grid(0), channels=1, + include_stats=False, include_tiles=False)) + else: + io.writeGrid(path, fh) -def main(): +def filter_file(in_path, out_path, outer_iters=6, band=3, deform_iters=4, normalize_iters=5): + cp = _gpu_or_skip() + if cp is None: + return None + g, phi, vx, half_width, gtype = read_to_device(cp, in_path, band) + print(f"read {in_path}: {gtype}, {g['n']} active voxels, voxelSize {vx:g}") + r0 = surface_radius(cp, g, phi, vx) + for it in range(outer_iters): + for _ in range(deform_iters): + phi = laplacian_step(cp, g, phi, half_width) + for _ in range(normalize_iters): + phi = godunov_step(cp, g, phi, vx, 0.3 * vx, half_width) + g, phi = rebuild(cp, g, phi, vx, half_width) + r = surface_radius(cp, g, phi, vx) + print(f" iter {it + 1}: {g['n']:7d} active, surface radius = {r:.4f}") + write_output(cp, g, phi, vx, out_path, gtype, band) + style = "OnIndex+SDF" if gtype == nanovdb.GridType.OnIndex else "FloatGrid" + print(f"wrote {out_path}: {style} (same style as input), {g['n']} active voxels " + f"(surface radius {r0:.4f} -> {r:.4f}); no CUDA kernel used") + return r0, r, gtype + + +def self_test(): cp = _gpu_or_skip() if cp is None: return - vx, background = 1.0, 3.0 - dh, grid, n, phi = load_sphere(cp, radius=20.0, vx=vx) # keep dh alive (owns the grid) - print(f"sphere OnIndex level set: {n} active voxels") - print(f" initial mean||grad|-1| = {grad_error(cp, grid, phi, n, vx):.4f}") + io, T, GT = nanovdb.io, nanovdb.tools, nanovdb.GridType + tmps = [tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) for _ in range(4)] + for t in tmps: + t.close() + f_in, f_out, o_in, o_out = (t.name for t in tmps) + + io.writeGrid(f_in, T.createLevelSetSphere(radius=20.0, voxelSize=1.0, name="sphere")) + print("self-test 1: FloatGrid sphere -> kernel-free filter -> FloatGrid") + r0, r, style = filter_file(f_in, f_out, outer_iters=6) + rb = io.readGrid(f_out) + assert style == GT.Float and rb.gridType(0) == GT.Float, "output style is not FloatGrid" + assert r < r0 - 0.05, "sphere did not shrink under curvature flow" + + sphere = T.createLevelSetSphere(radius=18.0, voxelSize=1.0, name="sphere") + io.writeGrid(o_in, T.createOnIndexGrid(sphere.grid(0), channels=1, + include_stats=False, include_tiles=False)) + print("self-test 2: OnIndex+SDF sphere -> kernel-free filter -> OnIndex+SDF") + r0b, rb2, style2 = filter_file(o_in, o_out, outer_iters=4) + ro = io.readGrid(o_out) + assert style2 == GT.OnIndex and ro.gridType(0) == GT.OnIndex, "output style is not OnIndex" + assert ro.grid(0).blindDataCount() >= 1, "OnIndex output has no SDF blind channel" + assert rb2 < r0b - 0.05, "sphere did not shrink under curvature flow" - # DEFORM: a few kernel-free Laplacian steps degrade the signed-distance property. - for _ in range(4): - phi = laplacian_step(cp, grid, phi, n, background) - err_deformed = grad_error(cp, grid, phi, n, vx) - print(f" after deform mean||grad|-1| = {err_deformed:.4f}") + print("OK: full file->file LevelSetFilter ran entirely in CuPy on bound ops " + "(gatherBoxStencil / activeVoxelCoords / inject / dilateGrid / pruneGrid) -- " + "no CUDA kernel; output style matches input.") + for n in (f_in, f_out, o_in, o_out): + os.unlink(n) - # RENORMALISE: kernel-free Godunov reinit drives |grad phi| back toward 1. - for _ in range(8): - phi = godunov_step(cp, grid, phi, n, vx, 0.3 * vx, background) - err_reinit = grad_error(cp, grid, phi, n, vx) - print(f" after reinit mean||grad|-1| = {err_reinit:.4f}") - assert err_reinit < err_deformed, "kernel-free Godunov reinit did not restore |grad phi|" - print("OK: deform + reinit ran as pure CuPy on gatherBoxStencil (no CUDA kernel); " - "reinitialisation restored |grad phi| -> 1.") +def main(argv): + if len(argv) >= 3: + outer = int(argv[3]) if len(argv) >= 4 else 6 + filter_file(argv[1], argv[2], outer) + elif len(argv) == 1: + self_test() + else: + print(__doc__) + raise SystemExit("usage: cupy_gather_stencil.py input.nvdb output.nvdb " + "[outer_iterations] (no args = self-test)") if __name__ == "__main__": - main() + main(sys.argv) From 0c582312b9b841d3f8724271772f16d65ead729f Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 3 Jun 2026 04:50:19 +0000 Subject: [PATCH 30/48] nanovdb python: add levelset_filter_cutile example; rename the filter trio to levelset_filter_ The three GPU level-set filter examples are one and the same filter (tools::LevelSetFilter-style Laplacian deform + Godunov reinit + narrow-band retrack, driven by the device VoxelBlockManager, style- preserving over FloatGrid / OnIndexGrid+blind-SDF inputs). They differ only in how the dense per-voxel stencils are computed, so name them by that backend so they sort together and read as a set: cupy_levelset_filter.py -> levelset_filter_rawkernel.py (fused cupy.RawModule CUDA kernel) cupy_gather_stencil.py -> levelset_filter_cupy.py (kernel-free, plain CuPy array ops) (new) levelset_filter_cutile.py (NVIDIA cuTile tile kernels) levelset_filter_cutile.py is levelset_filter_cupy.py with the per-voxel stencil math (deform / renorm / extrapolate) moved from CuPy array ops into cuda.tile (`@ct.kernel`, ct.load/ct.store over (TILE,) tiles); the sparse half (gatherBoxStencil / activeVoxelCoords / inject / dilateGrid / pruneGrid) is shared. Imports of cupy / cuda.tile are guarded so the file imports without a GPU and the self-test skips cleanly. Docstrings cross-reference the trio; README groups them under one table framing them as the same filter in three compute backends. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/examples/README.md | 20 +- ...her_stencil.py => levelset_filter_cupy.py} | 22 +- .../python/examples/levelset_filter_cutile.py | 316 ++++++++++++++++++ ...filter.py => levelset_filter_rawkernel.py} | 15 +- 4 files changed, 360 insertions(+), 13 deletions(-) rename nanovdb/nanovdb/python/examples/{cupy_gather_stencil.py => levelset_filter_cupy.py} (92%) create mode 100644 nanovdb/nanovdb/python/examples/levelset_filter_cutile.py rename nanovdb/nanovdb/python/examples/{cupy_levelset_filter.py => levelset_filter_rawkernel.py} (96%) diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index 28c3add954..a660393cf4 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -40,8 +40,24 @@ GPU-array framework it uses is unavailable. | [`cupy_rawkernel.py`](cupy_rawkernel.py) | A `cupy.RawKernel` that `#include ` (compiled with `nanovdb.cuda.compile_options()`) and reads a `const nanovdb::NanoGrid*` straight from `grid.data_ptr()`. Documents the device-pointer ABI. Requires CuPy. | | [`numba_cuda.py`](numba_cuda.py) | Adopting a NanoVDB device buffer as a zero-copy `numba.cuda` array (Numba can't parse the C++ ABI, so it operates on the raw buffer). Requires Numba. | | [`triton_kernel.py`](triton_kernel.py) | Handing a NanoVDB device buffer to a Triton kernel via a zero-copy `torch.from_dlpack` tensor. Requires Triton + PyTorch. | -| [`cupy_levelset_filter.py`](cupy_levelset_filter.py) | A full GPU level-set filter (`tools::LevelSetFilter`-style diffusion + Godunov renormalisation + narrow-band retrack) on a `.nvdb` file, driven by the device `VoxelBlockManager`: `buildVoxelBlockManager` plus `decodeInverseMaps` / `computeBoxStencil` called from a `cupy.RawModule` (nvcc backend), with `dilateGrid` / `inject` / `injectPredicateToMask` / `pruneGrid` for the retrack. Reads/writes either a `FloatGrid` or an `OnIndexGrid` with the SDF in a blind channel, preserving the input style. No args runs a self-test. Requires CuPy + nvcc. | -| [`cupy_gather_stencil.py`](cupy_gather_stencil.py) | The **kernel-free** counterpart to `cupy_levelset_filter.py`: the *same* full `.nvdb`→`.nvdb` level-set filter (deform + Godunov reinit + narrow-band retrack, style-preserving) with **no `cupy.RawModule` / CUDA kernel anywhere**. `tools.cuda.gatherBoxStencil` gives each voxel's 3×3×3 neighbourhood as a dense `(N, 27)` array and `tools.cuda.activeVoxelCoords` gives positions as `(N, 3)`, so all per-voxel math is plain **CuPy** (the same dense arrays a cuTile `@ct.kernel` would `ct.load`); the retrack uses the bound `dilateGrid` / `inject` / `injectPredicateToMask` / `pruneGrid`. No args runs a self-test over both input styles. Requires only CuPy (no nvcc / NanoVDB headers). | + +#### The same level-set filter, three compute backends + +`levelset_filter_*.py` are **three implementations of one GPU level-set filter** — +`tools::LevelSetFilter`-style Laplacian diffusion + Godunov renormalisation + +narrow-band retrack on a `.nvdb` file, driven by the device `VoxelBlockManager`, +reading/writing either a `FloatGrid` or an `OnIndexGrid` (SDF in a blind channel) +and preserving the input style. They share the *sparse* half (the bound +`gatherBoxStencil` / `activeVoxelCoords` / `dilateGrid` / `inject` / +`injectPredicateToMask` / `pruneGrid` ops) and differ **only in how the dense +per-voxel stencils are computed** — a useful side-by-side of three GPU styles. +Each runs a self-test with no arguments. + +| Script | Per-voxel compute backend | +| --- | --- | +| [`levelset_filter_rawkernel.py`](levelset_filter_rawkernel.py) | A hand-written CUDA kernel: `buildVoxelBlockManager` + `decodeInverseMaps` / `computeBoxStencil` fused with the update in a `cupy.RawModule` (nvcc backend). Fastest, most control. Requires CuPy + nvcc. | +| [`levelset_filter_cupy.py`](levelset_filter_cupy.py) | **Kernel-free**: `gatherBoxStencil` → dense `(N, 27)` neighbourhood + `activeVoxelCoords` → `(N, 3)`, then all per-voxel math as plain **CuPy** array ops. No `RawModule` / CUDA C++ / nvcc. Requires only CuPy. | +| [`levelset_filter_cutile.py`](levelset_filter_cutile.py) | The same dense arrays, with the per-voxel stencils as **NVIDIA cuTile** (`cuda.tile`) tile kernels (`ct.load` / `ct.store` over `(TILE,)` tiles). Requires CuPy + `cuda-tile`. | For full API signatures and per-argument docstrings, use Python's `help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`. diff --git a/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py b/nanovdb/nanovdb/python/examples/levelset_filter_cupy.py similarity index 92% rename from nanovdb/nanovdb/python/examples/cupy_gather_stencil.py rename to nanovdb/nanovdb/python/examples/levelset_filter_cupy.py index 75bc3728d6..90027ef6a3 100644 --- a/nanovdb/nanovdb/python/examples/cupy_gather_stencil.py +++ b/nanovdb/nanovdb/python/examples/levelset_filter_cupy.py @@ -1,14 +1,20 @@ # Copyright Contributors to the OpenVDB Project # SPDX-License-Identifier: Apache-2.0 -"""A full GPU LevelSetFilter on .nvdb files with NO hand-written CUDA kernel. +"""GPU LevelSetFilter on NanoVDB .nvdb files -- pure-CuPy (kernel-free) backend. -This is the kernel-free counterpart to `cupy_levelset_filter.py`. That example -runs the per-voxel stencils as `cupy.RawModule` kernels that call the device -VoxelBlockManager decode + `computeBoxStencil`; here every stage runs as plain -**CuPy** array math on top of bound NanoVDB ops -- no `RawModule`, no CUDA C++, -no `nvcc`: +One of three sibling examples applying the SAME GPU level-set filter (Laplacian +deform + Godunov reinit + narrow-band retrack, driven by the VoxelBlockManager); +they differ only in how the per-voxel stencils are computed: + * levelset_filter_rawkernel.py -- a hand-written CUDA kernel + * levelset_filter_cupy.py -- pure CuPy array ops (no kernel) (this file) + * levelset_filter_cutile.py -- NVIDIA cuTile tile kernels - python cupy_gather_stencil.py input.nvdb output.nvdb [outer_iterations] +Where levelset_filter_rawkernel.py runs the per-voxel stencils as `cupy.RawModule` +kernels calling the device VoxelBlockManager decode + `computeBoxStencil`, here +every stage runs as plain **CuPy** array math on top of bound NanoVDB ops -- no +`RawModule`, no CUDA C++, no `nvcc`: + + python levelset_filter_cupy.py input.nvdb output.nvdb [outer_iterations] How each stage stays kernel-free: * READ `createOnIndexGrid` + `grid.getBlindData(0)` -> value-indexed SDF. @@ -280,7 +286,7 @@ def main(argv): self_test() else: print(__doc__) - raise SystemExit("usage: cupy_gather_stencil.py input.nvdb output.nvdb " + raise SystemExit("usage: levelset_filter_cupy.py input.nvdb output.nvdb " "[outer_iterations] (no args = self-test)") diff --git a/nanovdb/nanovdb/python/examples/levelset_filter_cutile.py b/nanovdb/nanovdb/python/examples/levelset_filter_cutile.py new file mode 100644 index 0000000000..45b411b40d --- /dev/null +++ b/nanovdb/nanovdb/python/examples/levelset_filter_cutile.py @@ -0,0 +1,316 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""GPU LevelSetFilter on NanoVDB .nvdb files -- NVIDIA cuTile backend. + +One of three sibling examples applying the SAME GPU level-set filter (Laplacian +deform + Godunov reinit + narrow-band retrack, driven by the VoxelBlockManager); +they differ only in how the per-voxel stencils are computed: + * levelset_filter_rawkernel.py -- a hand-written CUDA kernel + * levelset_filter_cupy.py -- pure CuPy array ops (no kernel) + * levelset_filter_cutile.py -- NVIDIA cuTile tile kernels (this file) + +This is levelset_filter_cupy.py with the per-voxel stencil math moved from CuPy +array ops into NVIDIA cuTile (`cuda.tile`) kernels. The whole filter runs in one +process: + + read : nanovdb createOnIndexGrid + grid.getBlindData (value-indexed SDF) + deform : gatherBoxStencil -> dense (N,27); a cuTile @_kernel does the + Laplacian phi += (sum6 - 6 phi)/6 over (TILE,) tiles + renorm : same gather; a cuTile kernel does the first-order Godunov reinit + retrack : dilateGrid -> inject -> cuTile extrapolate kernel (fills the new + ring) -> |phi|<=halfWidth predicate -> injectPredicateToMask -> + pruneGrid -> inject + write : activeVoxelCoords -> tools.build.FloatGrid, in the input's style + +So the SPARSE work (neighbour gather, coord decode, topology) stays in bound +NanoVDB ops, and the DENSE per-voxel compute is cuTile. The cuTile kernels load +six 1-D face arrays per tile (cuTile tile dims must be compile-time powers of +two, so a (TILE,6) tile of the gather columns isn't allowed); the sign-clamped +background BC is applied in CuPy before launch (inactive spokes come back as the +sentinel phi[0]); the extrapolation BC is done inside its kernel. + +Validated against levelset_filter_cupy.py's results (same spheres shrink the +same amount). Run with no arguments for a self-test over both input styles. +Requires CuPy + cuda-tile (`cuda.tile`) and a CUDA-capable GPU; self-skips +otherwise. + python levelset_filter_cutile.py [in.nvdb out.nvdb [iters]] +""" +import os +import sys +import tempfile + +import numpy as np + +import nanovdb + +try: + import cupy as cp + import cuda.tile as ct + HAVE_CUTILE = True +except ImportError: + HAVE_CUTILE = False + + +def _kernel(fn): + """Apply @cuda.tile.kernel when available; else a no-op so the module still + imports (the kernels are never called when cuTile / a GPU is absent).""" + return ct.kernel(fn) if HAVE_CUTILE else fn + + +def _gpu_or_skip(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. Skipping.") + return False + if not HAVE_CUTILE: + print("This example requires CuPy + cuda-tile (cuda.tile). Skipping.") + return False + return True + + +TILE = 256 +SENTINEL = 1.0e30 +NN_FACE = 6 +FACES = [4, 22, 10, 16, 12, 14] # 3x3x3 spokes: -x,+x,-y,+y,-z,+z + + +# ----------------------------- cuTile kernels ----------------------------- +@_kernel +def laplacian_kernel(c, xm, xp, ym, yp, zm, zp, out): + b = ct.bid(0) + cc = ct.load(c, index=(b,), shape=(TILE,)) + s6 = (ct.load(xm, index=(b,), shape=(TILE,)) + ct.load(xp, index=(b,), shape=(TILE,)) + + ct.load(ym, index=(b,), shape=(TILE,)) + ct.load(yp, index=(b,), shape=(TILE,)) + + ct.load(zm, index=(b,), shape=(TILE,)) + ct.load(zp, index=(b,), shape=(TILE,))) + ct.store(out, index=(b,), tile=cc + (s6 - 6.0 * cc) / 6.0) + + +@_kernel +def godunov_kernel(c, xm, xp, ym, yp, zm, zp, dx, dt, out): + b = ct.bid(0) + cc = ct.load(c, index=(b,), shape=(TILE,)) + xm_ = ct.load(xm, index=(b,), shape=(TILE,)); xp_ = ct.load(xp, index=(b,), shape=(TILE,)) + ym_ = ct.load(ym, index=(b,), shape=(TILE,)); yp_ = ct.load(yp, index=(b,), shape=(TILE,)) + zm_ = ct.load(zm, index=(b,), shape=(TILE,)); zp_ = ct.load(zp, index=(b,), shape=(TILE,)) + s = cc / ct.sqrt(cc * cc + dx * dx) + pos = s > 0.0 + + def axis(dm, dp): + return ct.where(pos, + ct.maximum(ct.maximum(dm, 0.0) * ct.maximum(dm, 0.0), + ct.minimum(dp, 0.0) * ct.minimum(dp, 0.0)), + ct.maximum(ct.minimum(dm, 0.0) * ct.minimum(dm, 0.0), + ct.maximum(dp, 0.0) * ct.maximum(dp, 0.0))) + + grad = ct.sqrt(axis((cc - xm_) / dx, (xp_ - cc) / dx) + + axis((cc - ym_) / dx, (yp_ - cc) / dx) + + axis((cc - zm_) / dx, (zp_ - cc) / dx)) + ct.store(out, index=(b,), tile=cc - dt * s * (grad - 1.0)) + + +@_kernel +def extrapolate_kernel(c, xm, xp, ym, yp, zm, zp, dx, out): + b = ct.bid(0) + cc = ct.load(c, index=(b,), shape=(TILE,)) + f0 = ct.load(xm, index=(b,), shape=(TILE,)); f1 = ct.load(xp, index=(b,), shape=(TILE,)) + f2 = ct.load(ym, index=(b,), shape=(TILE,)); f3 = ct.load(yp, index=(b,), shape=(TILE,)) + f4 = ct.load(zm, index=(b,), shape=(TILE,)); f5 = ct.load(zp, index=(b,), shape=(TILE,)) + best = ct.full((TILE,), SENTINEL, ct.float32) # min-|value| active neighbour + best = ct.where(ct.abs(f0) < ct.abs(best), f0, best) + best = ct.where(ct.abs(f1) < ct.abs(best), f1, best) + best = ct.where(ct.abs(f2) < ct.abs(best), f2, best) + best = ct.where(ct.abs(f3) < ct.abs(best), f3, best) + best = ct.where(ct.abs(f4) < ct.abs(best), f4, best) + best = ct.where(ct.abs(f5) < ct.abs(best), f5, best) + filled = best + ct.where(best >= 0.0, dx, -dx) # phi_nbr + sign*dx + is_new = cc == SENTINEL + has = ct.abs(best) < SENTINEL + ct.store(out, index=(b,), tile=ct.where(is_new, ct.where(has, filled, cc), cc)) + + +# ----------------------------- helpers ----------------------------- +def _setup(handle): + grid = handle.deviceGrid(0) + if grid is None or grid.data_ptr() == 0: + handle.deviceUpload(0, True) + grid = handle.deviceGrid(0) + n = int(nanovdb.tools.cuda.buildVoxelBlockManager(grid, log2_block_width=9).lastOffset()) + return {"handle": handle, "grid": grid, "n": n} + + +def read_to_device(path, band): + io, T = nanovdb.io, nanovdb.tools + host = io.readGrid(path) + gtype = host.gridType(0) + vx = float(host.grid(0).voxelSize()[0]) + tmp = None + if gtype == nanovdb.GridType.Float: + onh = T.createOnIndexGrid(host.grid(0), channels=1, + include_stats=False, include_tiles=False) + sdf = np.array(onh.grid(0).getBlindData(0), dtype=np.float32) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() + io.writeGrid(tmp.name, onh) + dh = io.deviceReadGrid(tmp.name) + elif gtype == nanovdb.GridType.OnIndex: + if host.grid(0).blindDataCount() == 0: + raise SystemExit(f"{path}: OnIndex grid has no blind-data SDF channel.") + sdf = np.array(host.grid(0).getBlindData(0), dtype=np.float32) + dh = io.deviceReadGrid(path) + else: + raise SystemExit(f"{path}: unsupported grid type {gtype}.") + g = _setup(dh) + if sdf.shape[0] != g["n"] + 1: + raise SystemExit(f"{path}: SDF channel length {sdf.shape[0]} != activeVoxelCount+1 " + f"({g['n'] + 1}).") + phi = cp.asarray(sdf); phi[0] = SENTINEL + if tmp is not None: + os.unlink(tmp.name) + return g, phi, vx, band * vx, gtype + + +def _gather_faces(grid, phi, n, clamp, background): + nbrs = cp.empty((n + 1, 27), dtype=cp.float32) + nanovdb.tools.cuda.gatherBoxStencil(grid, phi, nbrs) + c = phi[1:n + 1] + cols = [] + for col in FACES: + f = nbrs[1:n + 1, col] + if clamp: + f = cp.where(f == SENTINEL, cp.copysign(cp.float32(background), c), f) + cols.append(cp.ascontiguousarray(f)) + return cp.ascontiguousarray(c), cols + + +def _pad(a, m): + out = cp.zeros(m, dtype=a.dtype) + out[:a.shape[0]] = a + return out + + +def _apply(kernel, g, phi, clamp, background, extra): + """gather -> pad -> cuTile launch -> new value-indexed sidecar.""" + n = g["n"] + c, faces = _gather_faces(g["grid"], phi, n, clamp, background) + m = ct.cdiv(n, TILE) * TILE + args = [_pad(a, m) for a in (c, *faces)] + out = cp.zeros(m, dtype=cp.float32) + ct.launch(cp.cuda.get_current_stream(), (ct.cdiv(m, TILE), 1, 1), + kernel, (*args, *extra, out)) + cp.cuda.runtime.deviceSynchronize() + new = cp.full(n + 1, SENTINEL, dtype=cp.float32) + new[1:n + 1] = out[:n] + return new + + +def laplacian(g, phi, half_width): + return _apply(laplacian_kernel, g, phi, True, half_width, ()) + + +def godunov(g, phi, vx, half_width): + return _apply(godunov_kernel, g, phi, True, half_width, (float(vx), float(0.3 * vx))) + + +def extrapolate(g, phi, vx): + return _apply(extrapolate_kernel, g, phi, False, 0.0, (float(vx),)) + + +def rebuild(g, phi, vx, half_width): + TC = nanovdb.tools.cuda + gd = _setup(TC.dilateGrid(g["grid"], op=NN_FACE)) + phi_d = cp.full(gd["n"] + 1, SENTINEL, dtype=cp.float32) + TC.inject(g["grid"], gd["grid"], phi, phi_d) + phi_d = extrapolate(gd, phi_d, vx) # cuTile fills the new ring + predicate = cp.abs(phi_d) <= half_width + leaf_masks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) + TC.injectPredicateToMask(gd["grid"], predicate, leaf_masks) + gp = _setup(TC.pruneGrid(gd["grid"], leaf_masks)) + phi_p = cp.full(gp["n"] + 1, SENTINEL, dtype=cp.float32) + TC.inject(gd["grid"], gp["grid"], phi_d, phi_p) + return gp, phi_p + + +def surface_radius(g, phi, vx): + coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) + nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) + v = phi[1:g["n"] + 1] + near = cp.abs(v) < 0.5 * vx + c = coords[1:g["n"] + 1][near].astype(cp.float64) * vx + return float(cp.mean(cp.linalg.norm(c, axis=1))) if int(near.sum()) else float("nan") + + +def write_output(g, phi, vx, path, style, band, name="filtered"): + T, io = nanovdb.tools, nanovdb.io + coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) + nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) + coords = cp.asnumpy(coords); v = cp.asnumpy(phi) + builder = T.build.FloatGrid(float(band * vx)); builder.setName(name); builder.setTransform(vx) + acc = builder.getAccessor(); Coord = nanovdb.math.Coord + for k in range(1, g["n"] + 1): + i, j, kk = coords[k] + acc.setValue(Coord(int(i), int(j), int(kk)), float(v[k])) + fh = builder.to_nanovdb() + if style == nanovdb.GridType.OnIndex: + io.writeGrid(path, T.createOnIndexGrid(fh.grid(0), channels=1, + include_stats=False, include_tiles=False)) + else: + io.writeGrid(path, fh) + + +def filter_file(in_path, out_path, outer_iters=6, band=3, deform_iters=4, normalize_iters=5): + if not _gpu_or_skip(): + return None + g, phi, vx, half_width, gtype = read_to_device(in_path, band) + print(f"read {in_path}: {gtype}, {g['n']} active voxels, voxelSize {vx:g}") + r0 = surface_radius(g, phi, vx) + for it in range(outer_iters): + for _ in range(deform_iters): + phi = laplacian(g, phi, half_width) + for _ in range(normalize_iters): + phi = godunov(g, phi, vx, half_width) + g, phi = rebuild(g, phi, vx, half_width) + r = surface_radius(g, phi, vx) + print(f" iter {it + 1}: {g['n']:7d} active, surface radius = {r:.4f}") + write_output(g, phi, vx, out_path, gtype, band) + style = "OnIndex+SDF" if gtype == nanovdb.GridType.OnIndex else "FloatGrid" + print(f"wrote {out_path}: {style} (same style as input), {g['n']} active voxels " + f"(surface radius {r0:.4f} -> {r:.4f}); per-voxel stencils ran as cuTile kernels") + return r0, r, gtype + + +def self_test(): + if not _gpu_or_skip(): + return + io, T, GT = nanovdb.io, nanovdb.tools, nanovdb.GridType + tmps = [tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) for _ in range(4)] + for t in tmps: + t.close() + f_in, f_out, o_in, o_out = (t.name for t in tmps) + io.writeGrid(f_in, T.createLevelSetSphere(radius=20.0, voxelSize=1.0, name="sphere")) + print("self-test 1: FloatGrid sphere -> cuTile filter -> FloatGrid") + r0, r, style = filter_file(f_in, f_out, outer_iters=6) + assert style == GT.Float and io.readGrid(f_out).gridType(0) == GT.Float + assert r < r0 - 0.05, "sphere did not shrink" + sphere = T.createLevelSetSphere(radius=18.0, voxelSize=1.0, name="sphere") + io.writeGrid(o_in, T.createOnIndexGrid(sphere.grid(0), channels=1, + include_stats=False, include_tiles=False)) + print("self-test 2: OnIndex+SDF sphere -> cuTile filter -> OnIndex+SDF") + r0b, rb2, style2 = filter_file(o_in, o_out, outer_iters=4) + ro = io.readGrid(o_out) + assert style2 == GT.OnIndex and ro.gridType(0) == GT.OnIndex and ro.grid(0).blindDataCount() >= 1 + assert rb2 < r0b - 0.05, "sphere did not shrink" + print("OK: full file->file LevelSetFilter ran with cuTile per-voxel kernels " + "(deform/renorm/extrapolate) + bound NanoVDB ops; output style preserved.") + for nm in (f_in, f_out, o_in, o_out): + os.unlink(nm) + + +def main(argv): + if len(argv) >= 3: + filter_file(argv[1], argv[2], int(argv[3]) if len(argv) >= 4 else 6) + elif len(argv) == 1: + self_test() + else: + raise SystemExit("usage: levelset_filter_cutile.py [in.nvdb out.nvdb [iters]]") + + +if __name__ == "__main__": + main(sys.argv) diff --git a/nanovdb/nanovdb/python/examples/cupy_levelset_filter.py b/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py similarity index 96% rename from nanovdb/nanovdb/python/examples/cupy_levelset_filter.py rename to nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py index 1e2545d222..895a32572c 100644 --- a/nanovdb/nanovdb/python/examples/cupy_levelset_filter.py +++ b/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py @@ -1,13 +1,22 @@ # Copyright Contributors to the OpenVDB Project # SPDX-License-Identifier: Apache-2.0 -"""GPU LevelSetFilter on NanoVDB .nvdb files, driven by the VoxelBlockManager. +"""GPU LevelSetFilter on NanoVDB .nvdb files -- compiled-CUDA-kernel backend. + +One of three sibling examples applying the SAME GPU level-set filter (Laplacian +deform + Godunov reinit + narrow-band retrack, driven by the VoxelBlockManager); +they differ only in how the per-voxel stencils are computed: + * levelset_filter_rawkernel.py -- a hand-written CUDA kernel (this file) + * levelset_filter_cupy.py -- pure CuPy array ops (no kernel) + * levelset_filter_cutile.py -- NVIDIA cuTile tile kernels +This one fuses the VBM decode + neighbour gather + update into one cupy.RawModule +CUDA kernel per stage. Reads a .nvdb file, runs N iterations of the GPU LevelSetFilter loop (the diffusion + renormalisation + narrow-band retrack that OpenVDB's tools::LevelSetFilter + LevelSetTracker perform), and writes the result to another .nvdb file: - python cupy_levelset_filter.py input.nvdb output.nvdb [outer_iterations] + python levelset_filter_rawkernel.py input.nvdb output.nvdb [outer_iterations] The input grid may be EITHER form (the script detects which): * a FloatGrid level set (per-voxel float SDF), or @@ -455,7 +464,7 @@ def main(argv): self_test() else: print(__doc__) - raise SystemExit("usage: cupy_levelset_filter.py input.nvdb output.nvdb " + raise SystemExit("usage: levelset_filter_rawkernel.py input.nvdb output.nvdb " "[outer_iterations] (no args = self-test)") From 7cae36c7694f0d616c223fa654a5028636bcf67f Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 3 Jun 2026 05:18:52 +0000 Subject: [PATCH 31/48] nanovdb python: refactor the level-set filter trio into a driver + 3 pluggable backends The three GPU level-set filter examples shared ~150 lines of identical code (the .nvdb read, the style-preserving write, the narrow-band retrack, the surface probe, the outer loop, the self-test) and differed only in the three per-voxel stencil functions. Split that common half into a runnable driver and reduce each backend to just its distinctive compute: levelset_filter.py -- driver: all shared I/O / retrack / loop, a small Backend protocol, and a `--backend` selector; runs the full file->file filter. levelset_filter_rawkernel.py -- Backend: the fused cupy.RawModule CUDA kernel levelset_filter_cupy.py -- Backend: kernel-free CuPy array ops levelset_filter_cutile.py -- Backend: NVIDIA cuTile tile kernels A backend implements setup / active_coords / laplacian / godunov / extrapolate plus a module-level make_backend(); the gather-based CuPy and cuTile backends share a GatherBackend base (gatherBoxStencil / activeVoxelCoords), while the rawkernel backend keeps its own VBM bookkeeping in the opaque per-grid context. Each backend file also runs standalone as a stencil-only smoke test (deform + renorm on a sphere, no file I/O and no retrack) of just that backend's math. Net -197 lines. Behaviour is unchanged: all three backends produce identical results to before (FloatGrid + OnIndex spheres shrink 18.0525 -> 17.6678, 23964 active voxels) and to each other. README updated to describe the driver + the three backends. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/examples/README.md | 40 +- .../python/examples/levelset_filter.py | 330 +++++++++++++++++ .../python/examples/levelset_filter_cupy.py | 345 ++++-------------- .../python/examples/levelset_filter_cutile.py | 293 ++++----------- .../examples/levelset_filter_rawkernel.py | 329 ++++------------- 5 files changed, 570 insertions(+), 767 deletions(-) create mode 100644 nanovdb/nanovdb/python/examples/levelset_filter.py diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index a660393cf4..b3a29bb8c1 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -41,22 +41,32 @@ GPU-array framework it uses is unavailable. | [`numba_cuda.py`](numba_cuda.py) | Adopting a NanoVDB device buffer as a zero-copy `numba.cuda` array (Numba can't parse the C++ ABI, so it operates on the raw buffer). Requires Numba. | | [`triton_kernel.py`](triton_kernel.py) | Handing a NanoVDB device buffer to a Triton kernel via a zero-copy `torch.from_dlpack` tensor. Requires Triton + PyTorch. | -#### The same level-set filter, three compute backends - -`levelset_filter_*.py` are **three implementations of one GPU level-set filter** — -`tools::LevelSetFilter`-style Laplacian diffusion + Godunov renormalisation + -narrow-band retrack on a `.nvdb` file, driven by the device `VoxelBlockManager`, -reading/writing either a `FloatGrid` or an `OnIndexGrid` (SDF in a blind channel) -and preserving the input style. They share the *sparse* half (the bound -`gatherBoxStencil` / `activeVoxelCoords` / `dilateGrid` / `inject` / -`injectPredicateToMask` / `pruneGrid` ops) and differ **only in how the dense -per-voxel stencils are computed** — a useful side-by-side of three GPU styles. -Each runs a self-test with no arguments. - -| Script | Per-voxel compute backend | +#### One level-set filter, three compute backends + +[`levelset_filter.py`](levelset_filter.py) is the runnable driver for **one** GPU +level-set filter — `tools::LevelSetFilter`-style Laplacian diffusion + Godunov +renormalisation + narrow-band retrack on a `.nvdb` file, driven by the device +`VoxelBlockManager`, reading/writing either a `FloatGrid` or an `OnIndexGrid` +(SDF in a blind channel) and preserving the input style — with the per-voxel +stencil math supplied by one of **three interchangeable backends**: + +``` +python levelset_filter.py {rawkernel|cupy|cutile} in.nvdb out.nvdb [outer_iters] +python levelset_filter.py {rawkernel|cupy|cutile} # self-test +``` + +The driver owns everything that isn't backend-specific (the style-detecting read, +the style-preserving write, the `dilateGrid` / `inject` / `injectPredicateToMask` +/ `pruneGrid` retrack, the outer loop). Each backend is a small `Backend` object +implementing only the three per-voxel stencils, and differs **only in how the +dense per-voxel math is computed** — a side-by-side of three GPU styles. Each +backend file also runs standalone (`python levelset_filter_.py`) as a +stencil-only smoke test (deform + renorm on a sphere, no I/O, no retrack). + +| Backend | Per-voxel compute | | --- | --- | -| [`levelset_filter_rawkernel.py`](levelset_filter_rawkernel.py) | A hand-written CUDA kernel: `buildVoxelBlockManager` + `decodeInverseMaps` / `computeBoxStencil` fused with the update in a `cupy.RawModule` (nvcc backend). Fastest, most control. Requires CuPy + nvcc. | -| [`levelset_filter_cupy.py`](levelset_filter_cupy.py) | **Kernel-free**: `gatherBoxStencil` → dense `(N, 27)` neighbourhood + `activeVoxelCoords` → `(N, 3)`, then all per-voxel math as plain **CuPy** array ops. No `RawModule` / CUDA C++ / nvcc. Requires only CuPy. | +| [`levelset_filter_rawkernel.py`](levelset_filter_rawkernel.py) | A hand-written CUDA kernel: `decodeInverseMaps` / `computeBoxStencil` fused with the update in a `cupy.RawModule` (nvcc backend), no dense gather. Fastest, most control. Requires CuPy + nvcc. | +| [`levelset_filter_cupy.py`](levelset_filter_cupy.py) | **Kernel-free**: `gatherBoxStencil` → dense `(N, 6)` face values, then all per-voxel math as plain **CuPy** array ops. No `RawModule` / CUDA C++ / nvcc. Requires only CuPy. | | [`levelset_filter_cutile.py`](levelset_filter_cutile.py) | The same dense arrays, with the per-voxel stencils as **NVIDIA cuTile** (`cuda.tile`) tile kernels (`ct.load` / `ct.store` over `(TILE,)` tiles). Requires CuPy + `cuda-tile`. | For full API signatures and per-argument docstrings, use Python's diff --git a/nanovdb/nanovdb/python/examples/levelset_filter.py b/nanovdb/nanovdb/python/examples/levelset_filter.py new file mode 100644 index 0000000000..f5cb205afe --- /dev/null +++ b/nanovdb/nanovdb/python/examples/levelset_filter.py @@ -0,0 +1,330 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""GPU LevelSetFilter on NanoVDB .nvdb files -- driver for three compute backends. + +This is the runnable driver for ONE GPU level-set filter -- a +tools::LevelSetFilter-style Laplacian deform + first-order Godunov reinit + +narrow-band retrack, driven by the device VoxelBlockManager -- with the +per-voxel stencil math supplied by one of three interchangeable backends: + + python levelset_filter.py rawkernel in.nvdb out.nvdb [outer_iters] + python levelset_filter.py cupy in.nvdb out.nvdb [outer_iters] + python levelset_filter.py cutile in.nvdb out.nvdb [outer_iters] + +(no input/output files => a self-test for that backend over both input styles) + +The backends live in sibling files and differ ONLY in how the dense per-voxel +stencils are computed: + * levelset_filter_rawkernel.py -- a hand-written CUDA kernel (cupy.RawModule) + that decodes the VBM + gathers in-kernel + * levelset_filter_cupy.py -- pure CuPy array ops over a dense gather + * levelset_filter_cutile.py -- NVIDIA cuTile tile kernels over the gather +Each also runs standalone: `python levelset_filter_.py` executes a +stencil-only smoke test (deform + renorm on a sphere, no file I/O / no retrack) +of just that backend's math. + +Everything that is NOT backend-specific lives here and is shared by all three: +the .nvdb read (FloatGrid OR OnIndexGrid+blind-SDF, auto-detected), the +style-preserving write, the narrow-band retrack (dilateGrid -> inject -> +extrapolate -> injectPredicateToMask -> pruneGrid -> inject), the surface-radius +probe, and the outer filter loop. A backend is a small object implementing: + + class Backend: + NAME # "rawkernel" | "cupy" | "cutile" + cp # the cupy module + setup(handle) -> g # {"grid", "n", ...}; per topology change + active_coords(g) -> (n+1, 3) int32 # value-indexed voxel coords + laplacian(g, phi, half_width) -> phi' + godunov(g, phi, vx, half_width) -> phi' + extrapolate(g, phi, vx) -> phi' + +and a module-level `make_backend()` that returns a `Backend` (or None, with a +printed reason, if that backend's requirements -- a CUDA build, a GPU, CuPy, +nvcc, or cuda.tile -- are absent). The opaque `g` context is produced by the +backend and handed back to it, so each backend stashes whatever bookkeeping it +needs (the rawkernel backend caches VBM pointers + decoded coords; the gather +backends keep it minimal). `g["grid"]` (the device OnIndex grid) and `g["n"]` +(the active-voxel count, = valueCount - 1) are the only fields the driver reads. + +The SDF is carried as a value-indexed device array `phi` of length n+1: slot 0 +is the background, slots 1..n the active voxels (the order the VBM decode and +`getBlindData` use). `phi[0]` is held at a sentinel so inactive neighbours are +detectable in a dense gather. Scope: first-order Godunov reinit, no advection or +alpha mask; the output's inactive interior carries +background (no signed +flood-fill). +""" +import importlib +import os +import sys +import tempfile + +import numpy as np + +import nanovdb + + +BACKENDS = ("rawkernel", "cupy", "cutile") + +LOG2_BLOCK_WIDTH = 9 +SENTINEL = 1.0e30 # phi[0]: marks inactive neighbours in a dense gather +NN_FACE = 6 # nanovdb::tools::morphology::NN_FACE (6-face dilation) +# 3x3x3 box-stencil spoke columns for the six faces, in -/+ x, y, z order +# (spoke = (di+1)*9 + (dj+1)*3 + (dk+1); centre = 13). +FACES = [4, 22, 10, 16, 12, 14] +BAND = 3 # narrow-band half width, in voxels +DEFORM_ITERS = 4 # Laplacian deform sub-iterations per outer iteration +NORMALIZE_ITERS = 5 # Godunov reinit sub-iterations per outer iteration + + +class GatherBackend: + """Base for backends that read each voxel's 3x3x3 neighbourhood as a dense + (n+1, 27) array via the bound `gatherBoxStencil` -- the CuPy and cuTile + backends. Subclasses implement only the per-voxel stencils + (`laplacian`/`godunov`/`extrapolate`); `setup`/`active_coords` are generic.""" + + def __init__(self, cp): + self.cp = cp + + def setup(self, handle): + cp = self.cp + grid = handle.deviceGrid(0) + if grid is None or grid.data_ptr() == 0: + handle.deviceUpload(0, True) + grid = handle.deviceGrid(0) + n = int(nanovdb.tools.cuda.buildVoxelBlockManager( + grid, log2_block_width=LOG2_BLOCK_WIDTH).lastOffset()) + return {"handle": handle, "grid": grid, "n": n} + + def active_coords(self, g): + cp = self.cp + coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) + nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) + return coords + + +def load_backend(name): + """Import levelset_filter_ and construct its Backend, or return None + (the backend prints why) if its requirements are unavailable.""" + if name not in BACKENDS: + raise SystemExit(f"unknown backend {name!r}; choose one of {', '.join(BACKENDS)}") + return importlib.import_module(f"levelset_filter_{name}").make_backend() + + +def gather_faces(cp, g, phi, background, clamp=True): + """`gatherBoxStencil` -> (centre values c, (n, 6) face values f). + + With clamp=True the sign-clamped background BC (`copysign(background, c)`) + replaces inactive (sentinel) spokes -- the boundary condition the deform and + reinit stencils use. With clamp=False the raw sentinel is preserved, so the + extrapolation stencil can tell which face neighbours are actually active.""" + n = g["n"] + nbrs = cp.empty((n + 1, 27), dtype=cp.float32) + nanovdb.tools.cuda.gatherBoxStencil(g["grid"], phi, nbrs) + c = phi[1:n + 1] + f = nbrs[1:n + 1][:, FACES] + if clamp: + f = cp.where(f == SENTINEL, cp.copysign(cp.float32(background), c)[:, None], f) + return c, f + + +def read_to_device(backend, path, band): + """Read a .nvdb (FloatGrid OR OnIndex+SDF) -> (g, phi, vx, half_width, style). + + A FloatGrid is baked to OnIndex+blind-SDF first (the representation the VBM + operates on); an OnIndexGrid is read directly. `style` (the input grid type) + is returned so the output can be written back in the same form.""" + cp = backend.cp + io, T = nanovdb.io, nanovdb.tools + host = io.readGrid(path) + gtype = host.gridType(0) + vx = float(host.grid(0).voxelSize()[0]) + tmp = None + if gtype == nanovdb.GridType.Float: + onh = T.createOnIndexGrid(host.grid(0), channels=1, + include_stats=False, include_tiles=False) + sdf = np.array(onh.grid(0).getBlindData(0), dtype=np.float32) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() + io.writeGrid(tmp.name, onh) + dh = io.deviceReadGrid(tmp.name) + elif gtype == nanovdb.GridType.OnIndex: + if host.grid(0).blindDataCount() == 0: + raise SystemExit(f"{path}: OnIndex grid has no blind-data SDF channel.") + sdf = np.array(host.grid(0).getBlindData(0), dtype=np.float32) + dh = io.deviceReadGrid(path) + else: + raise SystemExit(f"{path}: unsupported grid type {gtype} (expected Float or OnIndex).") + g = backend.setup(dh) + if sdf.shape[0] != g["n"] + 1: + raise SystemExit(f"{path}: SDF channel length {sdf.shape[0]} != activeVoxelCount+1 " + f"({g['n'] + 1}); the OnIndex grid must use contiguous voxel indexing " + "(built with include_stats=False, include_tiles=False).") + phi = cp.asarray(sdf) + phi[0] = SENTINEL # inactive-neighbour marker + if tmp is not None: + os.unlink(tmp.name) + return g, phi, vx, band * vx, gtype + + +def sphere_on_device(backend, radius, voxel_size=1.0, band=BAND, name="sphere"): + """Build a level-set sphere as OnIndex+SDF on the device -> (g, phi, vx, + half_width). Used by the backends' standalone stencil smoke tests.""" + cp = backend.cp + io, T = nanovdb.io, nanovdb.tools + fg = T.createLevelSetSphere(radius=radius, voxelSize=voxel_size, name=name) + onh = T.createOnIndexGrid(fg.grid(0), channels=1, + include_stats=False, include_tiles=False) + sdf = np.array(onh.grid(0).getBlindData(0), dtype=np.float32) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() + io.writeGrid(tmp.name, onh) + dh = io.deviceReadGrid(tmp.name) + os.unlink(tmp.name) + g = backend.setup(dh) + phi = cp.asarray(sdf) + phi[0] = SENTINEL + return g, phi, voxel_size, band * voxel_size + + +def rebuild(backend, g, phi, vx, half_width): + """Narrow-band retrack: dilate -> inject old phi -> extrapolate the new ring + (backend) -> |phi| <= halfWidth predicate -> injectPredicateToMask -> prune + -> inject. Returns the pruned grid context + its value-indexed sidecar.""" + cp, TC = backend.cp, nanovdb.tools.cuda + gd = backend.setup(TC.dilateGrid(g["grid"], op=NN_FACE)) + phi_d = cp.full(gd["n"] + 1, SENTINEL, dtype=cp.float32) + TC.inject(g["grid"], gd["grid"], phi, phi_d) # carry old phi (intersection) + phi_d = backend.extrapolate(gd, phi_d, vx) # fill the freshly-dilated ring + predicate = cp.abs(phi_d) <= half_width # phi_d[0]=SENTINEL -> False + leaf_masks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) + TC.injectPredicateToMask(gd["grid"], predicate, leaf_masks) + gp = backend.setup(TC.pruneGrid(gd["grid"], leaf_masks)) + phi_p = cp.full(gp["n"] + 1, SENTINEL, dtype=cp.float32) + TC.inject(gd["grid"], gp["grid"], phi_d, phi_p) + return gp, phi_p + + +def surface_radius(backend, g, phi, vx): + """Mean world radius of the zero-crossing voxels (|phi| < dx/2).""" + cp = backend.cp + coords = backend.active_coords(g) + v = phi[1:g["n"] + 1] + near = cp.abs(v) < 0.5 * vx + c = coords[1:g["n"] + 1][near].astype(cp.float64) * vx + return float(cp.mean(cp.linalg.norm(c, axis=1))) if int(near.sum()) else float("nan") + + +def write_output(backend, g, phi, vx, path, style, band, name="filtered"): + """Bake (coords, phi) into a host FloatGrid and write it in `style` + (GridType.Float -> a FloatGrid; GridType.OnIndex -> an OnIndexGrid with the + SDF in blind channel 0).""" + cp = backend.cp + T, io = nanovdb.tools, nanovdb.io + coords = cp.asnumpy(backend.active_coords(g)) + v = cp.asnumpy(phi) + builder = T.build.FloatGrid(float(band * vx)) + builder.setName(name) + builder.setTransform(vx) + acc = builder.getAccessor() + Coord = nanovdb.math.Coord + for k in range(1, g["n"] + 1): + i, j, kk = coords[k] + acc.setValue(Coord(int(i), int(j), int(kk)), float(v[k])) + fh = builder.to_nanovdb() + if style == nanovdb.GridType.OnIndex: + io.writeGrid(path, T.createOnIndexGrid(fh.grid(0), channels=1, + include_stats=False, include_tiles=False)) + else: + io.writeGrid(path, fh) + + +def run_filter(backend, in_path, out_path, outer_iters=6, band=BAND, + deform_iters=DEFORM_ITERS, normalize_iters=NORMALIZE_ITERS): + """Read -> N outer iterations (deform x k, renorm x k, retrack) -> write.""" + g, phi, vx, half_width, gtype = read_to_device(backend, in_path, band) + print(f"read {in_path}: {gtype}, {g['n']} active voxels, voxelSize {vx:g} " + f"[backend: {backend.NAME}]") + r0 = r = surface_radius(backend, g, phi, vx) + for it in range(outer_iters): + for _ in range(deform_iters): + phi = backend.laplacian(g, phi, half_width) + for _ in range(normalize_iters): + phi = backend.godunov(g, phi, vx, half_width) + g, phi = rebuild(backend, g, phi, vx, half_width) + r = surface_radius(backend, g, phi, vx) + print(f" iter {it + 1}: {g['n']:7d} active, surface radius = {r:.4f}") + write_output(backend, g, phi, vx, out_path, gtype, band) + style = "OnIndex+SDF" if gtype == nanovdb.GridType.OnIndex else "FloatGrid" + print(f"wrote {out_path}: {style} (same style as input), {g['n']} active voxels " + f"(surface radius {r0:.4f} -> {r:.4f}) [backend: {backend.NAME}]") + return r0, r, gtype + + +def stencil_demo(backend, radius=20.0, deform_iters=DEFORM_ITERS, + normalize_iters=NORMALIZE_ITERS): + """Run ONLY the per-voxel stencils (deform + renorm) on a sphere -- no file + I/O and no narrow-band retrack -- as a focused check of one backend's dense + math. Invoked by each backend file's `__main__`.""" + g, phi, vx, half_width = sphere_on_device(backend, radius) + r0 = surface_radius(backend, g, phi, vx) + for _ in range(deform_iters): + phi = backend.laplacian(g, phi, half_width) + for _ in range(normalize_iters): + phi = backend.godunov(g, phi, vx, half_width) + r = surface_radius(backend, g, phi, vx) + print(f"[{backend.NAME}] stencil-only smoke test on a sphere (radius {radius:g}, " + f"{g['n']} voxels, no retrack / no I/O): surface radius {r0:.4f} -> {r:.4f}") + assert r < r0, "Laplacian deform should shrink the sphere" + print(f"OK [{backend.NAME}]: deform + renorm stencils run and move the surface.") + + +def self_test(backend): + """Filter sphere .nvdb files of both styles; assert the style round-trips and + the sphere shrinks under curvature flow.""" + io, T, GT = nanovdb.io, nanovdb.tools, nanovdb.GridType + tmps = [tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) for _ in range(4)] + for t in tmps: + t.close() + f_in, f_out, o_in, o_out = (t.name for t in tmps) + + io.writeGrid(f_in, T.createLevelSetSphere(radius=20.0, voxelSize=1.0, name="sphere")) + print(f"self-test 1 [{backend.NAME}]: FloatGrid sphere -> filter -> FloatGrid") + r0, r, style = run_filter(backend, f_in, f_out, outer_iters=6) + assert style == GT.Float and io.readGrid(f_out).gridType(0) == GT.Float, \ + "output style is not FloatGrid" + assert r < r0 - 0.05, "sphere did not shrink under curvature flow" + + sphere = T.createLevelSetSphere(radius=18.0, voxelSize=1.0, name="sphere") + io.writeGrid(o_in, T.createOnIndexGrid(sphere.grid(0), channels=1, + include_stats=False, include_tiles=False)) + print(f"self-test 2 [{backend.NAME}]: OnIndex+SDF sphere -> filter -> OnIndex+SDF") + r0b, rb, style2 = run_filter(backend, o_in, o_out, outer_iters=4) + ro = io.readGrid(o_out) + assert style2 == GT.OnIndex and ro.gridType(0) == GT.OnIndex, "output style is not OnIndex" + assert ro.grid(0).blindDataCount() >= 1, "OnIndex output has no SDF blind channel" + assert rb < r0b - 0.05, "sphere did not shrink under curvature flow" + + print(f"OK [{backend.NAME}]: both input styles filter correctly and the output " + "style matches the input.") + for n in (f_in, f_out, o_in, o_out): + os.unlink(n) + + +def main(argv): + if len(argv) < 2 or argv[1] not in BACKENDS: + print(__doc__) + raise SystemExit(f"usage: levelset_filter.py {{{'|'.join(BACKENDS)}}} " + "[input.nvdb output.nvdb [outer_iterations]] (no files = self-test)") + backend = load_backend(argv[1]) + if backend is None: + return + rest = argv[2:] + if len(rest) >= 2: + outer = int(rest[2]) if len(rest) >= 3 else 6 + run_filter(backend, rest[0], rest[1], outer) + elif len(rest) == 0: + self_test(backend) + else: + raise SystemExit("provide BOTH input.nvdb and output.nvdb, or neither (self-test).") + + +if __name__ == "__main__": + main(sys.argv) diff --git a/nanovdb/nanovdb/python/examples/levelset_filter_cupy.py b/nanovdb/nanovdb/python/examples/levelset_filter_cupy.py index 90027ef6a3..520e0ca24c 100644 --- a/nanovdb/nanovdb/python/examples/levelset_filter_cupy.py +++ b/nanovdb/nanovdb/python/examples/levelset_filter_cupy.py @@ -1,294 +1,93 @@ # Copyright Contributors to the OpenVDB Project # SPDX-License-Identifier: Apache-2.0 -"""GPU LevelSetFilter on NanoVDB .nvdb files -- pure-CuPy (kernel-free) backend. +"""levelset_filter pure-CuPy backend -- per-voxel stencils as plain CuPy arrays. -One of three sibling examples applying the SAME GPU level-set filter (Laplacian -deform + Godunov reinit + narrow-band retrack, driven by the VoxelBlockManager); -they differ only in how the per-voxel stencils are computed: - * levelset_filter_rawkernel.py -- a hand-written CUDA kernel - * levelset_filter_cupy.py -- pure CuPy array ops (no kernel) (this file) - * levelset_filter_cutile.py -- NVIDIA cuTile tile kernels +The kernel-free backend for levelset_filter.py: every stage runs as plain +**CuPy** array math on the dense `(n, 6)` face values returned by the bound +`gatherBoxStencil` -- no `cupy.RawModule`, no CUDA C++, no nvcc. Run the full +file->file filter through the driver: -Where levelset_filter_rawkernel.py runs the per-voxel stencils as `cupy.RawModule` -kernels calling the device VoxelBlockManager decode + `computeBoxStencil`, here -every stage runs as plain **CuPy** array math on top of bound NanoVDB ops -- no -`RawModule`, no CUDA C++, no `nvcc`: + python levelset_filter.py cupy input.nvdb output.nvdb [outer_iterations] - python levelset_filter_cupy.py input.nvdb output.nvdb [outer_iterations] +Running this file directly executes a stencil-only smoke test (the deform + +renorm steps on a sphere, no file I/O and no narrow-band retrack): -How each stage stays kernel-free: - * READ `createOnIndexGrid` + `grid.getBlindData(0)` -> value-indexed SDF. - * DEFORM `gatherBoxStencil` -> dense (N, 27) neighbour values; the Laplacian - `phi += (sum6 - 6 phi)/6` is then a CuPy expression. - * RENORM same gather; a first-order Godunov reinit in CuPy. - * RETRACK `dilateGrid` -> `inject` -> extrapolate new voxels (CuPy on the - gather) -> `|phi|<=halfWidth` predicate (CuPy) -> `injectPredicate- - ToMask` -> `pruneGrid` -> `inject`. - * WRITE `activeVoxelCoords` -> per-voxel coords, baked into a grid with - `tools.build.FloatGrid`; written back in the input's style - (`FloatGrid`, or `OnIndexGrid` with the SDF in a blind channel). + python levelset_filter_cupy.py -The whole per-voxel surface lives in dense arrays (`gatherBoxStencil` for the -neighbourhood, `activeVoxelCoords` for positions), which is exactly the shape a -tile framework such as cuTile consumes -- the CuPy math here would map onto a -`@ct.kernel` operating on the same `(N, 27)` / `(N, 3)` arrays. - -The sidecar's background slot `phi[0]` is set to a sentinel so inactive -neighbours are detectable in the gathered array; that lets the CuPy code apply -the same sign-consistent clamped boundary condition (`copysign(background, -phi_centre)`) the kernel version uses. - -Scope: first-order Godunov reinitialisation, no advection or alpha mask; the -output's inactive interior carries +background (no signed flood-fill). Run with -no arguments for a self-test over both input styles. Requires only CuPy and a -CUDA-capable GPU (no nvcc / NanoVDB headers); self-skips otherwise. +The dense `(n, 6)` arrays this operates on are exactly the shape a tile +framework consumes -- see levelset_filter_cutile.py for the same math as cuTile +kernels, and levelset_filter_rawkernel.py for a fused hand-written CUDA kernel. +Requires only CuPy and a CUDA-capable GPU (no nvcc / NanoVDB headers). """ -import os -import sys -import tempfile - -import numpy as np - import nanovdb - -LOG2_BLOCK_WIDTH = 9 -SENTINEL = 1.0e30 # phi[0]: marks inactive neighbours in the gathered array -NN_FACE = 6 # nanovdb::tools::morphology::NN_FACE (6-face dilation) -# 3x3x3 spoke columns for the six faces, in -/+ x, y, z order (centre is col 13). -FACES = [4, 22, 10, 16, 12, 14] +import levelset_filter as lsf -def _gpu_or_skip(): +def make_backend(): if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): - print("This example requires a CUDA build of nanovdb and a GPU. Skipping.") + print("This backend requires a CUDA build of nanovdb and a GPU. Skipping.") return None try: import cupy as cp except ImportError: - print("This example requires CuPy. Skipping.") + print("This backend requires CuPy. Skipping.") return None - return cp - - -def _setup(cp, handle): - """DeviceGridHandle -> {handle, grid, n}; n = active-voxel count (= valueCount-1).""" - grid = handle.deviceGrid(0) - if grid is None or grid.data_ptr() == 0: - handle.deviceUpload(0, True) - grid = handle.deviceGrid(0) - n = int(nanovdb.tools.cuda.buildVoxelBlockManager( - grid, log2_block_width=LOG2_BLOCK_WIDTH).lastOffset()) - return {"handle": handle, "grid": grid, "n": n} - - -def read_to_device(cp, path, band): - """Read a .nvdb (FloatGrid OR OnIndex+SDF) -> (g, phi, vx, half_width, style).""" - io, T = nanovdb.io, nanovdb.tools - host = io.readGrid(path) - gtype = host.gridType(0) - vx = float(host.grid(0).voxelSize()[0]) - tmp = None - if gtype == nanovdb.GridType.Float: - onh = T.createOnIndexGrid(host.grid(0), channels=1, - include_stats=False, include_tiles=False) - sdf = np.array(onh.grid(0).getBlindData(0), dtype=np.float32) - tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() - io.writeGrid(tmp.name, onh) - dh = io.deviceReadGrid(tmp.name) - elif gtype == nanovdb.GridType.OnIndex: - if host.grid(0).blindDataCount() == 0: - raise SystemExit(f"{path}: OnIndex grid has no blind-data SDF channel.") - sdf = np.array(host.grid(0).getBlindData(0), dtype=np.float32) - dh = io.deviceReadGrid(path) - else: - raise SystemExit(f"{path}: unsupported grid type {gtype} (expected Float or OnIndex).") - g = _setup(cp, dh) - if sdf.shape[0] != g["n"] + 1: - raise SystemExit(f"{path}: SDF channel length {sdf.shape[0]} != activeVoxelCount+1 " - f"({g['n'] + 1}); the OnIndex grid must use contiguous voxel indexing " - "(built with include_stats=False, include_tiles=False).") - phi = cp.asarray(sdf) - phi[0] = SENTINEL # inactive-neighbour marker - if tmp is not None: - os.unlink(tmp.name) - return g, phi, vx, band * vx, gtype - - -def _gather_faces(cp, g, phi, background): - """gatherBoxStencil -> the 6 face values with the clamped-background BC.""" - n = g["n"] - nbrs = cp.empty((n + 1, 27), dtype=cp.float32) - nanovdb.tools.cuda.gatherBoxStencil(g["grid"], phi, nbrs) - c = phi[1:n + 1] - f = nbrs[1:n + 1][:, FACES] - f = cp.where(f == SENTINEL, cp.copysign(cp.float32(background), c)[:, None], f) - return c, f - - -def laplacian_step(cp, g, phi, background): - """phi += (sum6 - 6 phi)/6 -- pure CuPy on the gathered neighbourhood.""" - c, f = _gather_faces(cp, g, phi, background) - out = phi.copy() - out[1:g["n"] + 1] = c + (f.sum(axis=1) - 6.0 * c) / 6.0 - out[0] = SENTINEL - return out - - -def godunov_step(cp, g, phi, dx, dt, background): - """phi -= dt*S(phi)*(|grad phi| - 1) -- first-order Godunov, pure CuPy.""" - c, f = _gather_faces(cp, g, phi, background) - xm, xp, ym, yp, zm, zp = (f[:, i] for i in range(6)) - s = c / cp.sqrt(c * c + dx * dx) - - def gd(dm, dp): - return cp.where(s > 0, - cp.maximum(cp.maximum(dm, 0.0) ** 2, cp.minimum(dp, 0.0) ** 2), - cp.maximum(cp.minimum(dm, 0.0) ** 2, cp.maximum(dp, 0.0) ** 2)) - - grad = cp.sqrt(gd((c - xm) / dx, (xp - c) / dx) - + gd((c - ym) / dx, (yp - c) / dx) - + gd((c - zm) / dx, (zp - c) / dx)) - out = phi.copy() - out[1:g["n"] + 1] = c - dt * s * (grad - 1.0) - out[0] = SENTINEL - return out - - -def _extrapolate(cp, g, phi, dx): - """Fill freshly-dilated (sentinel) voxels from the nearest in-band face - neighbour: phi = phi_nbr + sign(phi_nbr)*dx -- pure CuPy on the gather.""" - n = g["n"] - nbrs = cp.empty((n + 1, 27), dtype=cp.float32) - nanovdb.tools.cuda.gatherBoxStencil(g["grid"], phi, nbrs) - c = phi[1:n + 1] - f = nbrs[1:n + 1][:, FACES] # raw: inactive spokes are SENTINEL - known = f != SENTINEL - best = cp.take_along_axis( - f, cp.argmin(cp.where(known, cp.abs(f), cp.inf), axis=1)[:, None], axis=1)[:, 0] - out = phi.copy() - fill = (c == SENTINEL) & known.any(axis=1) - out[1:n + 1] = cp.where(fill, best + cp.copysign(cp.float32(dx), best), c) - out[0] = SENTINEL - return out - - -def rebuild(cp, g, phi, vx, half_width): - """Narrow-band retrack: dilate -> inject -> extrapolate -> prune -> inject.""" - TC = nanovdb.tools.cuda - gd = _setup(cp, TC.dilateGrid(g["grid"], op=NN_FACE)) - phi_d = cp.full(gd["n"] + 1, SENTINEL, dtype=cp.float32) - TC.inject(g["grid"], gd["grid"], phi, phi_d) # carry old phi (intersection) - phi_d = _extrapolate(cp, gd, phi_d, vx) # fill the new ring (CuPy) - predicate = cp.abs(phi_d) <= half_width # phi_d[0]=SENTINEL -> False - leaf_masks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) - TC.injectPredicateToMask(gd["grid"], predicate, leaf_masks) - gp = _setup(cp, TC.pruneGrid(gd["grid"], leaf_masks)) - phi_p = cp.full(gp["n"] + 1, SENTINEL, dtype=cp.float32) - TC.inject(gd["grid"], gp["grid"], phi_d, phi_p) - return gp, phi_p - - -def surface_radius(cp, g, phi, vx): - """Mean world radius of zero-crossing voxels (|phi| < dx/2), coords via activeVoxelCoords.""" - coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) - nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) - v = phi[1:g["n"] + 1] - near = cp.abs(v) < 0.5 * vx - c = coords[1:g["n"] + 1][near].astype(cp.float64) * vx - return float(cp.mean(cp.linalg.norm(c, axis=1))) if int(near.sum()) else float("nan") - - -def write_output(cp, g, phi, vx, path, style, band, name="filtered"): - """Bake (coords, phi) into a host FloatGrid; write in `style` (Float or OnIndex+SDF).""" - T, io = nanovdb.tools, nanovdb.io - coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) - nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) # kernel-free coord decode - coords = cp.asnumpy(coords) - v = cp.asnumpy(phi) - builder = T.build.FloatGrid(float(band * vx)) - builder.setName(name) - builder.setTransform(vx) - acc = builder.getAccessor() - Coord = nanovdb.math.Coord - for k in range(1, g["n"] + 1): - i, j, kk = coords[k] - acc.setValue(Coord(int(i), int(j), int(kk)), float(v[k])) - fh = builder.to_nanovdb() - if style == nanovdb.GridType.OnIndex: - io.writeGrid(path, T.createOnIndexGrid(fh.grid(0), channels=1, - include_stats=False, include_tiles=False)) - else: - io.writeGrid(path, fh) - - -def filter_file(in_path, out_path, outer_iters=6, band=3, deform_iters=4, normalize_iters=5): - cp = _gpu_or_skip() - if cp is None: - return None - g, phi, vx, half_width, gtype = read_to_device(cp, in_path, band) - print(f"read {in_path}: {gtype}, {g['n']} active voxels, voxelSize {vx:g}") - r0 = surface_radius(cp, g, phi, vx) - for it in range(outer_iters): - for _ in range(deform_iters): - phi = laplacian_step(cp, g, phi, half_width) - for _ in range(normalize_iters): - phi = godunov_step(cp, g, phi, vx, 0.3 * vx, half_width) - g, phi = rebuild(cp, g, phi, vx, half_width) - r = surface_radius(cp, g, phi, vx) - print(f" iter {it + 1}: {g['n']:7d} active, surface radius = {r:.4f}") - write_output(cp, g, phi, vx, out_path, gtype, band) - style = "OnIndex+SDF" if gtype == nanovdb.GridType.OnIndex else "FloatGrid" - print(f"wrote {out_path}: {style} (same style as input), {g['n']} active voxels " - f"(surface radius {r0:.4f} -> {r:.4f}); no CUDA kernel used") - return r0, r, gtype - - -def self_test(): - cp = _gpu_or_skip() - if cp is None: - return - io, T, GT = nanovdb.io, nanovdb.tools, nanovdb.GridType - tmps = [tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) for _ in range(4)] - for t in tmps: - t.close() - f_in, f_out, o_in, o_out = (t.name for t in tmps) - - io.writeGrid(f_in, T.createLevelSetSphere(radius=20.0, voxelSize=1.0, name="sphere")) - print("self-test 1: FloatGrid sphere -> kernel-free filter -> FloatGrid") - r0, r, style = filter_file(f_in, f_out, outer_iters=6) - rb = io.readGrid(f_out) - assert style == GT.Float and rb.gridType(0) == GT.Float, "output style is not FloatGrid" - assert r < r0 - 0.05, "sphere did not shrink under curvature flow" - - sphere = T.createLevelSetSphere(radius=18.0, voxelSize=1.0, name="sphere") - io.writeGrid(o_in, T.createOnIndexGrid(sphere.grid(0), channels=1, - include_stats=False, include_tiles=False)) - print("self-test 2: OnIndex+SDF sphere -> kernel-free filter -> OnIndex+SDF") - r0b, rb2, style2 = filter_file(o_in, o_out, outer_iters=4) - ro = io.readGrid(o_out) - assert style2 == GT.OnIndex and ro.gridType(0) == GT.OnIndex, "output style is not OnIndex" - assert ro.grid(0).blindDataCount() >= 1, "OnIndex output has no SDF blind channel" - assert rb2 < r0b - 0.05, "sphere did not shrink under curvature flow" - - print("OK: full file->file LevelSetFilter ran entirely in CuPy on bound ops " - "(gatherBoxStencil / activeVoxelCoords / inject / dilateGrid / pruneGrid) -- " - "no CUDA kernel; output style matches input.") - for n in (f_in, f_out, o_in, o_out): - os.unlink(n) - - -def main(argv): - if len(argv) >= 3: - outer = int(argv[3]) if len(argv) >= 4 else 6 - filter_file(argv[1], argv[2], outer) - elif len(argv) == 1: - self_test() - else: - print(__doc__) - raise SystemExit("usage: levelset_filter_cupy.py input.nvdb output.nvdb " - "[outer_iterations] (no args = self-test)") + return Backend(cp) + + +class Backend(lsf.GatherBackend): + """Per-voxel stencils as CuPy array ops; `setup`/`active_coords` come from the + shared GatherBackend (gatherBoxStencil / activeVoxelCoords).""" + + NAME = "cupy" + + def laplacian(self, g, phi, half_width): + """phi += (sum6 - 6 phi)/6.""" + cp = self.cp + c, f = lsf.gather_faces(cp, g, phi, half_width, clamp=True) + out = phi.copy() + out[1:g["n"] + 1] = c + (f.sum(axis=1) - 6.0 * c) / 6.0 + out[0] = lsf.SENTINEL + return out + + def godunov(self, g, phi, vx, half_width): + """phi -= dt*S(phi)*(|grad phi| - 1) -- first-order Godunov reinit.""" + cp = self.cp + c, f = lsf.gather_faces(cp, g, phi, half_width, clamp=True) + xm, xp, ym, yp, zm, zp = (f[:, i] for i in range(6)) + dt = 0.3 * vx + s = c / cp.sqrt(c * c + vx * vx) + + def gd(dm, dp): + return cp.where(s > 0, + cp.maximum(cp.maximum(dm, 0.0) ** 2, cp.minimum(dp, 0.0) ** 2), + cp.maximum(cp.minimum(dm, 0.0) ** 2, cp.maximum(dp, 0.0) ** 2)) + + grad = cp.sqrt(gd((c - xm) / vx, (xp - c) / vx) + + gd((c - ym) / vx, (yp - c) / vx) + + gd((c - zm) / vx, (zp - c) / vx)) + out = phi.copy() + out[1:g["n"] + 1] = c - dt * s * (grad - 1.0) + out[0] = lsf.SENTINEL + return out + + def extrapolate(self, g, phi, vx): + """Fill freshly-dilated (sentinel) voxels from the nearest in-band face + neighbour: phi = phi_nbr + sign(phi_nbr)*dx.""" + cp = self.cp + c, f = lsf.gather_faces(cp, g, phi, 0.0, clamp=False) # raw spokes + known = f != lsf.SENTINEL + best = cp.take_along_axis( + f, cp.argmin(cp.where(known, cp.abs(f), cp.inf), axis=1)[:, None], axis=1)[:, 0] + out = phi.copy() + fill = (c == lsf.SENTINEL) & known.any(axis=1) + out[1:g["n"] + 1] = cp.where(fill, best + cp.copysign(cp.float32(vx), best), c) + out[0] = lsf.SENTINEL + return out if __name__ == "__main__": - main(sys.argv) + backend = make_backend() + if backend is not None: + lsf.stencil_demo(backend) diff --git a/nanovdb/nanovdb/python/examples/levelset_filter_cutile.py b/nanovdb/nanovdb/python/examples/levelset_filter_cutile.py index 45b411b40d..cde4172e5e 100644 --- a/nanovdb/nanovdb/python/examples/levelset_filter_cutile.py +++ b/nanovdb/nanovdb/python/examples/levelset_filter_cutile.py @@ -1,48 +1,32 @@ # Copyright Contributors to the OpenVDB Project # SPDX-License-Identifier: Apache-2.0 -"""GPU LevelSetFilter on NanoVDB .nvdb files -- NVIDIA cuTile backend. - -One of three sibling examples applying the SAME GPU level-set filter (Laplacian -deform + Godunov reinit + narrow-band retrack, driven by the VoxelBlockManager); -they differ only in how the per-voxel stencils are computed: - * levelset_filter_rawkernel.py -- a hand-written CUDA kernel - * levelset_filter_cupy.py -- pure CuPy array ops (no kernel) - * levelset_filter_cutile.py -- NVIDIA cuTile tile kernels (this file) - -This is levelset_filter_cupy.py with the per-voxel stencil math moved from CuPy -array ops into NVIDIA cuTile (`cuda.tile`) kernels. The whole filter runs in one -process: - - read : nanovdb createOnIndexGrid + grid.getBlindData (value-indexed SDF) - deform : gatherBoxStencil -> dense (N,27); a cuTile @_kernel does the - Laplacian phi += (sum6 - 6 phi)/6 over (TILE,) tiles - renorm : same gather; a cuTile kernel does the first-order Godunov reinit - retrack : dilateGrid -> inject -> cuTile extrapolate kernel (fills the new - ring) -> |phi|<=halfWidth predicate -> injectPredicateToMask -> - pruneGrid -> inject - write : activeVoxelCoords -> tools.build.FloatGrid, in the input's style - -So the SPARSE work (neighbour gather, coord decode, topology) stays in bound -NanoVDB ops, and the DENSE per-voxel compute is cuTile. The cuTile kernels load -six 1-D face arrays per tile (cuTile tile dims must be compile-time powers of -two, so a (TILE,6) tile of the gather columns isn't allowed); the sign-clamped -background BC is applied in CuPy before launch (inactive spokes come back as the -sentinel phi[0]); the extrapolation BC is done inside its kernel. - -Validated against levelset_filter_cupy.py's results (same spheres shrink the -same amount). Run with no arguments for a self-test over both input styles. -Requires CuPy + cuda-tile (`cuda.tile`) and a CUDA-capable GPU; self-skips -otherwise. - python levelset_filter_cutile.py [in.nvdb out.nvdb [iters]] -""" -import os -import sys -import tempfile +"""levelset_filter NVIDIA cuTile backend -- per-voxel stencils as tile kernels. + +The cuTile backend for levelset_filter.py: this is levelset_filter_cupy.py with +the per-voxel stencil math (deform / renorm / extrapolate) moved from CuPy array +ops into NVIDIA cuTile (`cuda.tile`) kernels that `ct.load`/`ct.store` over +`(TILE,)` tiles. The SPARSE half (neighbour gather, coord decode, topology) +stays in bound NanoVDB ops; only the DENSE per-voxel compute is cuTile. Run the +full file->file filter through the driver: + + python levelset_filter.py cutile input.nvdb output.nvdb [outer_iterations] + +Running this file directly executes a stencil-only smoke test (the deform + +renorm kernels on a sphere, no file I/O and no narrow-band retrack): -import numpy as np + python levelset_filter_cutile.py +The kernels load six 1-D face arrays per tile (cuTile tile dims must be +compile-time powers of two, so a `(TILE, 6)` tile of the gather columns isn't +allowed); the sign-clamped background BC is applied in CuPy before launch +(`lsf.gather_faces`), and the extrapolation BC is done inside its kernel. +Validated against levelset_filter_cupy.py (same spheres shrink the same amount). +Requires CuPy + cuda-tile (`cuda.tile`) and a CUDA-capable GPU. +""" import nanovdb +import levelset_filter as lsf + try: import cupy as cp import cuda.tile as ct @@ -57,20 +41,18 @@ def _kernel(fn): return ct.kernel(fn) if HAVE_CUTILE else fn -def _gpu_or_skip(): +def make_backend(): if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): - print("This example requires a CUDA build of nanovdb and a GPU. Skipping.") - return False + print("This backend requires a CUDA build of nanovdb and a GPU. Skipping.") + return None if not HAVE_CUTILE: - print("This example requires CuPy + cuda-tile (cuda.tile). Skipping.") - return False - return True + print("This backend requires CuPy + cuda-tile (cuda.tile). Skipping.") + return None + return Backend(cp) TILE = 256 -SENTINEL = 1.0e30 -NN_FACE = 6 -FACES = [4, 22, 10, 16, 12, 14] # 3x3x3 spokes: -x,+x,-y,+y,-z,+z +SENTINEL = 1.0e30 # must match levelset_filter.SENTINEL (read inside a kernel) # ----------------------------- cuTile kernels ----------------------------- @@ -127,190 +109,47 @@ def extrapolate_kernel(c, xm, xp, ym, yp, zm, zp, dx, out): ct.store(out, index=(b,), tile=ct.where(is_new, ct.where(has, filled, cc), cc)) -# ----------------------------- helpers ----------------------------- -def _setup(handle): - grid = handle.deviceGrid(0) - if grid is None or grid.data_ptr() == 0: - handle.deviceUpload(0, True) - grid = handle.deviceGrid(0) - n = int(nanovdb.tools.cuda.buildVoxelBlockManager(grid, log2_block_width=9).lastOffset()) - return {"handle": handle, "grid": grid, "n": n} - - -def read_to_device(path, band): - io, T = nanovdb.io, nanovdb.tools - host = io.readGrid(path) - gtype = host.gridType(0) - vx = float(host.grid(0).voxelSize()[0]) - tmp = None - if gtype == nanovdb.GridType.Float: - onh = T.createOnIndexGrid(host.grid(0), channels=1, - include_stats=False, include_tiles=False) - sdf = np.array(onh.grid(0).getBlindData(0), dtype=np.float32) - tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() - io.writeGrid(tmp.name, onh) - dh = io.deviceReadGrid(tmp.name) - elif gtype == nanovdb.GridType.OnIndex: - if host.grid(0).blindDataCount() == 0: - raise SystemExit(f"{path}: OnIndex grid has no blind-data SDF channel.") - sdf = np.array(host.grid(0).getBlindData(0), dtype=np.float32) - dh = io.deviceReadGrid(path) - else: - raise SystemExit(f"{path}: unsupported grid type {gtype}.") - g = _setup(dh) - if sdf.shape[0] != g["n"] + 1: - raise SystemExit(f"{path}: SDF channel length {sdf.shape[0]} != activeVoxelCount+1 " - f"({g['n'] + 1}).") - phi = cp.asarray(sdf); phi[0] = SENTINEL - if tmp is not None: - os.unlink(tmp.name) - return g, phi, vx, band * vx, gtype - - -def _gather_faces(grid, phi, n, clamp, background): - nbrs = cp.empty((n + 1, 27), dtype=cp.float32) - nanovdb.tools.cuda.gatherBoxStencil(grid, phi, nbrs) - c = phi[1:n + 1] - cols = [] - for col in FACES: - f = nbrs[1:n + 1, col] - if clamp: - f = cp.where(f == SENTINEL, cp.copysign(cp.float32(background), c), f) - cols.append(cp.ascontiguousarray(f)) - return cp.ascontiguousarray(c), cols - - -def _pad(a, m): +def _pad(cp, a, m): out = cp.zeros(m, dtype=a.dtype) out[:a.shape[0]] = a return out -def _apply(kernel, g, phi, clamp, background, extra): - """gather -> pad -> cuTile launch -> new value-indexed sidecar.""" - n = g["n"] - c, faces = _gather_faces(g["grid"], phi, n, clamp, background) - m = ct.cdiv(n, TILE) * TILE - args = [_pad(a, m) for a in (c, *faces)] - out = cp.zeros(m, dtype=cp.float32) - ct.launch(cp.cuda.get_current_stream(), (ct.cdiv(m, TILE), 1, 1), - kernel, (*args, *extra, out)) - cp.cuda.runtime.deviceSynchronize() - new = cp.full(n + 1, SENTINEL, dtype=cp.float32) - new[1:n + 1] = out[:n] - return new - - -def laplacian(g, phi, half_width): - return _apply(laplacian_kernel, g, phi, True, half_width, ()) - - -def godunov(g, phi, vx, half_width): - return _apply(godunov_kernel, g, phi, True, half_width, (float(vx), float(0.3 * vx))) - - -def extrapolate(g, phi, vx): - return _apply(extrapolate_kernel, g, phi, False, 0.0, (float(vx),)) - - -def rebuild(g, phi, vx, half_width): - TC = nanovdb.tools.cuda - gd = _setup(TC.dilateGrid(g["grid"], op=NN_FACE)) - phi_d = cp.full(gd["n"] + 1, SENTINEL, dtype=cp.float32) - TC.inject(g["grid"], gd["grid"], phi, phi_d) - phi_d = extrapolate(gd, phi_d, vx) # cuTile fills the new ring - predicate = cp.abs(phi_d) <= half_width - leaf_masks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) - TC.injectPredicateToMask(gd["grid"], predicate, leaf_masks) - gp = _setup(TC.pruneGrid(gd["grid"], leaf_masks)) - phi_p = cp.full(gp["n"] + 1, SENTINEL, dtype=cp.float32) - TC.inject(gd["grid"], gp["grid"], phi_d, phi_p) - return gp, phi_p - - -def surface_radius(g, phi, vx): - coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) - nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) - v = phi[1:g["n"] + 1] - near = cp.abs(v) < 0.5 * vx - c = coords[1:g["n"] + 1][near].astype(cp.float64) * vx - return float(cp.mean(cp.linalg.norm(c, axis=1))) if int(near.sum()) else float("nan") - - -def write_output(g, phi, vx, path, style, band, name="filtered"): - T, io = nanovdb.tools, nanovdb.io - coords = cp.empty((g["n"] + 1, 3), dtype=cp.int32) - nanovdb.tools.cuda.activeVoxelCoords(g["grid"], coords) - coords = cp.asnumpy(coords); v = cp.asnumpy(phi) - builder = T.build.FloatGrid(float(band * vx)); builder.setName(name); builder.setTransform(vx) - acc = builder.getAccessor(); Coord = nanovdb.math.Coord - for k in range(1, g["n"] + 1): - i, j, kk = coords[k] - acc.setValue(Coord(int(i), int(j), int(kk)), float(v[k])) - fh = builder.to_nanovdb() - if style == nanovdb.GridType.OnIndex: - io.writeGrid(path, T.createOnIndexGrid(fh.grid(0), channels=1, - include_stats=False, include_tiles=False)) - else: - io.writeGrid(path, fh) - - -def filter_file(in_path, out_path, outer_iters=6, band=3, deform_iters=4, normalize_iters=5): - if not _gpu_or_skip(): - return None - g, phi, vx, half_width, gtype = read_to_device(in_path, band) - print(f"read {in_path}: {gtype}, {g['n']} active voxels, voxelSize {vx:g}") - r0 = surface_radius(g, phi, vx) - for it in range(outer_iters): - for _ in range(deform_iters): - phi = laplacian(g, phi, half_width) - for _ in range(normalize_iters): - phi = godunov(g, phi, vx, half_width) - g, phi = rebuild(g, phi, vx, half_width) - r = surface_radius(g, phi, vx) - print(f" iter {it + 1}: {g['n']:7d} active, surface radius = {r:.4f}") - write_output(g, phi, vx, out_path, gtype, band) - style = "OnIndex+SDF" if gtype == nanovdb.GridType.OnIndex else "FloatGrid" - print(f"wrote {out_path}: {style} (same style as input), {g['n']} active voxels " - f"(surface radius {r0:.4f} -> {r:.4f}); per-voxel stencils ran as cuTile kernels") - return r0, r, gtype - - -def self_test(): - if not _gpu_or_skip(): - return - io, T, GT = nanovdb.io, nanovdb.tools, nanovdb.GridType - tmps = [tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) for _ in range(4)] - for t in tmps: - t.close() - f_in, f_out, o_in, o_out = (t.name for t in tmps) - io.writeGrid(f_in, T.createLevelSetSphere(radius=20.0, voxelSize=1.0, name="sphere")) - print("self-test 1: FloatGrid sphere -> cuTile filter -> FloatGrid") - r0, r, style = filter_file(f_in, f_out, outer_iters=6) - assert style == GT.Float and io.readGrid(f_out).gridType(0) == GT.Float - assert r < r0 - 0.05, "sphere did not shrink" - sphere = T.createLevelSetSphere(radius=18.0, voxelSize=1.0, name="sphere") - io.writeGrid(o_in, T.createOnIndexGrid(sphere.grid(0), channels=1, - include_stats=False, include_tiles=False)) - print("self-test 2: OnIndex+SDF sphere -> cuTile filter -> OnIndex+SDF") - r0b, rb2, style2 = filter_file(o_in, o_out, outer_iters=4) - ro = io.readGrid(o_out) - assert style2 == GT.OnIndex and ro.gridType(0) == GT.OnIndex and ro.grid(0).blindDataCount() >= 1 - assert rb2 < r0b - 0.05, "sphere did not shrink" - print("OK: full file->file LevelSetFilter ran with cuTile per-voxel kernels " - "(deform/renorm/extrapolate) + bound NanoVDB ops; output style preserved.") - for nm in (f_in, f_out, o_in, o_out): - os.unlink(nm) - - -def main(argv): - if len(argv) >= 3: - filter_file(argv[1], argv[2], int(argv[3]) if len(argv) >= 4 else 6) - elif len(argv) == 1: - self_test() - else: - raise SystemExit("usage: levelset_filter_cutile.py [in.nvdb out.nvdb [iters]]") +class Backend(lsf.GatherBackend): + """Per-voxel stencils as cuTile kernels; `setup`/`active_coords` come from the + shared GatherBackend (gatherBoxStencil / activeVoxelCoords).""" + + NAME = "cutile" + + def _apply(self, kernel, g, phi, clamp, background, extra): + """gather (driver) -> contiguous columns -> pad -> cuTile launch -> new + value-indexed sidecar.""" + cp = self.cp + n = g["n"] + c, f = lsf.gather_faces(cp, g, phi, background, clamp=clamp) + cols = [cp.ascontiguousarray(f[:, i]) for i in range(6)] + m = ct.cdiv(n, TILE) * TILE + args = [_pad(cp, a, m) for a in (cp.ascontiguousarray(c), *cols)] + out = cp.zeros(m, dtype=cp.float32) + ct.launch(cp.cuda.get_current_stream(), (ct.cdiv(m, TILE), 1, 1), + kernel, (*args, *extra, out)) + cp.cuda.runtime.deviceSynchronize() + new = cp.full(n + 1, SENTINEL, dtype=cp.float32) + new[1:n + 1] = out[:n] + return new + + def laplacian(self, g, phi, half_width): + return self._apply(laplacian_kernel, g, phi, True, half_width, ()) + + def godunov(self, g, phi, vx, half_width): + return self._apply(godunov_kernel, g, phi, True, half_width, + (float(vx), float(0.3 * vx))) + + def extrapolate(self, g, phi, vx): + return self._apply(extrapolate_kernel, g, phi, False, 0.0, (float(vx),)) if __name__ == "__main__": - main(sys.argv) + backend = make_backend() + if backend is not None: + lsf.stencil_demo(backend) diff --git a/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py b/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py index 895a32572c..1e2c85e833 100644 --- a/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py +++ b/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py @@ -1,76 +1,39 @@ # Copyright Contributors to the OpenVDB Project # SPDX-License-Identifier: Apache-2.0 -"""GPU LevelSetFilter on NanoVDB .nvdb files -- compiled-CUDA-kernel backend. - -One of three sibling examples applying the SAME GPU level-set filter (Laplacian -deform + Godunov reinit + narrow-band retrack, driven by the VoxelBlockManager); -they differ only in how the per-voxel stencils are computed: - * levelset_filter_rawkernel.py -- a hand-written CUDA kernel (this file) - * levelset_filter_cupy.py -- pure CuPy array ops (no kernel) - * levelset_filter_cutile.py -- NVIDIA cuTile tile kernels -This one fuses the VBM decode + neighbour gather + update into one cupy.RawModule -CUDA kernel per stage. - -Reads a .nvdb file, runs N iterations of the GPU LevelSetFilter loop (the -diffusion + renormalisation + narrow-band retrack that OpenVDB's -tools::LevelSetFilter + LevelSetTracker perform), and writes the result to -another .nvdb file: - - python levelset_filter_rawkernel.py input.nvdb output.nvdb [outer_iterations] - -The input grid may be EITHER form (the script detects which): - * a FloatGrid level set (per-voxel float SDF), or - * an OnIndexGrid whose float SDF is stored in blind-data channel 0. - -Both are normalised to "(OnIndex topology on the device) + (float SDF sidecar)", -which is the representation the VoxelBlockManager operates on: - * FloatGrid -> tools.createOnIndexGrid(fg, channels=1) bakes the SDF into a - blind channel; write to a temp .nvdb; io.deviceReadGrid it. - * OnIndexGrid -> io.deviceReadGrid directly. -The SDF sidecar is read on the host with grid.getBlindData(0) (value-index -order: [0] is the background slot, 1..N the active voxels -- the same order the -VBM decode uses), then uploaded to the device. - -The OUTPUT is written in the SAME style as the input: a FloatGrid input yields a -FloatGrid .nvdb, an OnIndex+SDF input yields an OnIndexGrid .nvdb with the SDF in -blind channel 0. The result is baked on the host (tools.build.FloatGrid -> -to_nanovdb), optionally converted back to OnIndex via createOnIndexGrid, then -io.writeGrid. (Writing the result as a device-built grid via indexToGrid is -avoided: in testing it dropped the high-value-index voxels for these -stats/tiles-free index grids.) - -Each filter iteration runs three stages: - 1. DEFORM Laplacian flow phi += (sum6 - 6 phi)/6 (VBM stencil) - 2. RENORMALIZE Godunov reinit phi -= dt*S(phi)*(|grad|-1) (VBM stencil) - 3. REBUILD dilateGrid -> inject -> extrapolate -> injectPredicateToMask - -> pruneGrid -> inject (native bound ops + one extrapolate kernel) - -Scope / limitations: first-order Godunov reinitialisation (not higher-order -WENO), no advection and no alpha mask. The prune keeps |phi| <= band*voxelSize, -so the active band tracks the surface, but the output's inactive interior -carries +background (there is no signed flood-fill). - -Run without arguments for a self-test: it builds sphere .nvdb files in both -forms (FloatGrid and OnIndex+SDF), filters each, and asserts that the output -style matches the input and the sphere shrinks under curvature flow. - -Requires CuPy, a CUDA-capable GPU, and nvcc (CuPy honours the NVCC env var). In a -dev/source tree, set NANOVDB_INCLUDE to the dir containing nanovdb/NanoVDB.h. +"""levelset_filter compiled-CUDA-kernel backend -- one fused cupy.RawModule. + +The hand-written-CUDA-kernel backend for levelset_filter.py: each stencil is a +`cupy.RawModule` (nvcc) kernel that fuses the VoxelBlockManager decode +(`decodeInverseMaps`) + 3x3x3 neighbour gather (`computeBoxStencil`) + the +per-voxel update into a single launch -- no dense `gatherBoxStencil` array. Run +the full file->file filter through the driver: + + python levelset_filter.py rawkernel input.nvdb output.nvdb [outer_iterations] + +Running this file directly executes a stencil-only smoke test (the deform + +renorm kernels on a sphere, no file I/O and no narrow-band retrack): + + python levelset_filter_rawkernel.py + +This is the fastest of the three backends (the gather is fused into the compute, +no `(n, 27)` array is materialised) and the most code -- compare the kernel-free +levelset_filter_cupy.py / cuTile levelset_filter_cutile.py for the same math. +Requires CuPy, a CUDA-capable GPU, and nvcc (CuPy honours the NVCC env var). In +a dev/source tree, set NANOVDB_INCLUDE to the dir containing nanovdb/NanoVDB.h. """ import os -import sys -import tempfile import numpy as np import nanovdb +import levelset_filter as lsf + -LOG2_BLOCK_WIDTH = 9 -BLOCK_WIDTH = 1 << LOG2_BLOCK_WIDTH -NN_FACE = 6 # nanovdb::tools::morphology::NN_FACE (6-face dilation) -SENTINEL = 1.0e30 # "value not yet known" marker for freshly-dilated voxels -SDF_BLIND_CHANNEL = 0 # blind-data channel holding the float SDF on OnIndex input +LOG2_BLOCK_WIDTH = lsf.LOG2_BLOCK_WIDTH # 9 +BLOCK_WIDTH = 1 << LOG2_BLOCK_WIDTH # 512 +NN_FACE = lsf.NN_FACE # 6-face dilation +SENTINEL = lsf.SENTINEL # "value not yet known" marker # computeBoxStencil spoke ids: spoke = (di+1)*9 + (dj+1)*3 + (dk+1). # centre=13; +x=22 -x=4; +y=16 -y=10; +z=14 -z=12 @@ -246,26 +209,39 @@ def _include_options(): return tuple(opts) -class Filter: - """Holds the compiled kernels + bound ops and runs the level-set filter.""" +def make_backend(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This backend requires a CUDA build of nanovdb and a GPU. Skipping.") + return None + try: + import cupy as cp + except ImportError: + print("This backend requires CuPy (plus nvcc on PATH or $NVCC). Skipping.") + return None + options = _include_options() + if options is None: + return None + return Backend(cp, options) + + +class Backend: + """Per-voxel stencils as fused cupy.RawModule CUDA kernels that decode the VBM + and gather the 3x3x3 neighbourhood in-kernel (no dense gatherBoxStencil), so + this backend keeps its own VBM bookkeeping in the `g` context.""" - def __init__(self, cp, band=3, deform_iters=4, normalize_iters=5): + NAME = "rawkernel" + + def __init__(self, cp, options): self.cp = cp self.tc = nanovdb.tools.cuda - self.band = band - self.deform_iters = deform_iters - self.normalize_iters = normalize_iters - options = _include_options() - if options is None: - raise SystemExit(1) m = cp.RawModule(code=KERNEL_SRC, backend="nvcc", options=options) self.k_decode = m.get_function("decode_coords") self.k_laplacian = m.get_function("laplacian_step") self.k_godunov = m.get_function("godunov_step") self.k_extrapolate = m.get_function("extrapolate") - # ---- device OnIndex grid + VBM bookkeeping ------------------------------- def setup(self, handle): + """Device OnIndex grid + VBM pointers; decode value-indexed coords once.""" cp = self.cp grid = handle.deviceGrid(0) if grid is None or grid.data_ptr() == 0: @@ -283,190 +259,39 @@ def setup(self, handle): fid=vbm.first_leaf_id_ptr(), jmp=vbm.jump_map_ptr(), gptr=grid.data_ptr(), coords=coords) + def active_coords(self, g): + return g["coords"] # decoded in setup() + def _vbm(self, g): return (g["gptr"], g["fid"], g["jmp"], g["fo"]) - # ---- one full LevelSetFilter iteration ----------------------------------- - def step(self, g, vals, vx, half_width): - cp = self.cp - bg = np.float32(half_width) - buf = cp.empty_like(vals) - # 1. DEFORM: Laplacian flow. - for _ in range(self.deform_iters): - self.k_laplacian((g["bc"],), (BLOCK_WIDTH,), (*self._vbm(g), bg, vals, buf)) - vals, buf = buf, vals - # 2. RENORMALIZE: Godunov reinitialisation. - for _ in range(self.normalize_iters): - self.k_godunov((g["bc"],), (BLOCK_WIDTH,), - (*self._vbm(g), np.float32(vx), np.float32(0.3 * vx), bg, vals, buf)) - vals, buf = buf, vals - cp.cuda.runtime.deviceSynchronize() - # 3. REBUILD BAND: dilateGrid -> inject -> extrapolate -> prune -> inject. - gd = self.setup(self.tc.dilateGrid(g["grid"], op=NN_FACE)) - vals_d = cp.full(gd["n"] + 1, SENTINEL, dtype=cp.float32) - self.tc.inject(g["grid"], gd["grid"], vals, vals_d) - ebuf = cp.empty_like(vals_d) - self.k_extrapolate((gd["bc"],), (BLOCK_WIDTH,), (*self._vbm(gd), np.float32(vx), vals_d, ebuf)) - vals_d = ebuf - predicate = cp.abs(vals_d) <= half_width - leaf_masks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) # activeVoxelCount*8 - self.tc.injectPredicateToMask(gd["grid"], predicate, leaf_masks) - gp = self.setup(self.tc.pruneGrid(gd["grid"], leaf_masks)) - vals_p = cp.full(gp["n"] + 1, half_width, dtype=cp.float32) - self.tc.inject(gd["grid"], gp["grid"], vals_d, vals_p) - return gp, vals_p - - def surface_radius(self, g, vals, vx): + def laplacian(self, g, phi, half_width): cp = self.cp - v = vals[1:g["n"] + 1] - near = cp.abs(v) < 0.5 * vx - c = g["coords"][1:g["n"] + 1][near].astype(cp.float64) * vx - return float(cp.mean(cp.linalg.norm(c, axis=1))) if int(near.sum()) else float("nan") - - -def read_to_device(flt, path): - """Read a .nvdb (FloatGrid OR OnIndex+SDF) -> (device-grid dict, sidecar, vx, half_width, style).""" - cp = flt.cp - io, T = nanovdb.io, nanovdb.tools - host = io.readGrid(path) - gtype = host.gridType(0) - vx = float(host.grid(0).voxelSize()[0]) - half_width = flt.band * vx - tmp = None - if gtype == nanovdb.GridType.Float: - # Bake the per-voxel SDF into an OnIndex blind channel; read it on the - # host via grid.getBlindData (value-index order: [0]=background, 1..N). - idx_host = T.createOnIndexGrid(host.grid(0), channels=1, - include_stats=False, include_tiles=False) - sdf = np.array(idx_host.grid(0).getBlindData(SDF_BLIND_CHANNEL), dtype=np.float32) - tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() - io.writeGrid(tmp.name, idx_host) - dev = io.deviceReadGrid(tmp.name) - elif gtype == nanovdb.GridType.OnIndex: - if host.grid(0).blindDataCount() == 0: - raise SystemExit(f"{path}: OnIndex grid has no blind-data SDF channel.") - sdf = np.array(host.grid(0).getBlindData(SDF_BLIND_CHANNEL), dtype=np.float32) - dev = io.deviceReadGrid(path) - else: - raise SystemExit(f"{path}: unsupported grid type {gtype} " - "(expected Float or OnIndex).") - dev.deviceUpload(0, True) - g = flt.setup(dev) - if sdf.shape[0] != g["n"] + 1: - raise SystemExit(f"{path}: SDF channel length {sdf.shape[0]} != activeVoxelCount+1 " - f"({g['n'] + 1}); the OnIndex grid must use contiguous voxel " - "indexing (built with include_stats=False, include_tiles=False).") - vals = cp.asarray(sdf) # value-indexed sidecar; vals[0] is the background slot - if tmp is not None: - os.unlink(tmp.name) - return g, vals, vx, half_width, gtype - - -def write_output(flt, g, vals, vx, path, style, name="filtered"): - """Bake the final (coords, SDF) into a host FloatGrid; write it in `style`. - - style == GridType.Float -> a FloatGrid .nvdb - style == GridType.OnIndex -> an OnIndexGrid .nvdb with the SDF in blind channel 0 - """ - cp = flt.cp - T, io = nanovdb.tools, nanovdb.io - coords = cp.asnumpy(g["coords"]) - v = cp.asnumpy(vals) - builder = T.build.FloatGrid(float(flt.band * vx)) - builder.setName(name) - builder.setTransform(vx) - acc = builder.getAccessor() - Coord = nanovdb.math.Coord - for k in range(1, g["n"] + 1): - i, j, kk = coords[k] - acc.setValue(Coord(int(i), int(j), int(kk)), float(v[k])) - fh = builder.to_nanovdb() - if style == nanovdb.GridType.OnIndex: - idx_out = T.createOnIndexGrid(fh.grid(0), channels=1, - include_stats=False, include_tiles=False) - io.writeGrid(path, idx_out) - else: - io.writeGrid(path, fh) - - -def _gpu_or_skip(): - """Return the cupy module, or None (with a printed reason) if GPU filtering - is unavailable -- lets the example self-skip cleanly like the others.""" - if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): - print("This example requires a CUDA build of nanovdb and a GPU. Skipping.") - return None - try: - import cupy as cp - except ImportError: - print("This example requires CuPy (plus nvcc on PATH or $NVCC). Skipping.") - return None - return cp - + out = cp.empty_like(phi) + self.k_laplacian((g["bc"],), (BLOCK_WIDTH,), + (*self._vbm(g), np.float32(half_width), phi, out)) + out[0] = SENTINEL + return out -def filter_file(in_path, out_path, outer_iters=6): - cp = _gpu_or_skip() - if cp is None: - return None - flt = Filter(cp) - g, vals, vx, half_width, gtype = read_to_device(flt, in_path) - print(f"read {in_path}: {gtype}, {g['n']} active voxels, voxelSize {vx:g}") - r0 = flt.surface_radius(g, vals, vx) - for it in range(outer_iters): - g, vals = flt.step(g, vals, vx, half_width) - r = flt.surface_radius(g, vals, vx) - print(f" iter {it + 1}: {g['n']:7d} active, surface radius = {r:.4f}") - write_output(flt, g, vals, vx, out_path, gtype) - style = "OnIndex+SDF" if gtype == nanovdb.GridType.OnIndex else "FloatGrid" - print(f"wrote {out_path}: {style} (same style as input), {g['n']} active voxels " - f"(surface radius {r0:.4f} -> {r:.4f})") - return r0, r, gtype - - -def self_test(): - """No-args run: build sphere .nvdb files of both styles, filter, assert invariants.""" - if _gpu_or_skip() is None: - return - io, T, GT = nanovdb.io, nanovdb.tools, nanovdb.GridType - tmps = [tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) for _ in range(4)] - for t in tmps: - t.close() - f_in, f_out, o_in, o_out = (t.name for t in tmps) - - # 1. FloatGrid in -> FloatGrid out. - io.writeGrid(f_in, T.createLevelSetSphere(radius=20.0, voxelSize=1.0, name="sphere")) - print("self-test 1: FloatGrid sphere -> filter -> FloatGrid") - r0, r, style = filter_file(f_in, f_out, outer_iters=6) - rb = io.readGrid(f_out) - assert style == GT.Float and rb.gridType(0) == GT.Float, "output style is not FloatGrid" - assert r < r0 - 0.05, "sphere did not shrink under curvature flow" - - # 2. OnIndex+SDF in -> OnIndex+SDF out (style preserved). - sphere = T.createLevelSetSphere(radius=18.0, voxelSize=1.0, name="sphere") - io.writeGrid(o_in, T.createOnIndexGrid(sphere.grid(0), channels=1, - include_stats=False, include_tiles=False)) - print("self-test 2: OnIndex+SDF sphere -> filter -> OnIndex+SDF") - r0b, rb2, style2 = filter_file(o_in, o_out, outer_iters=4) - ro = io.readGrid(o_out) - assert style2 == GT.OnIndex and ro.gridType(0) == GT.OnIndex, "output style is not OnIndex" - assert ro.grid(0).blindDataCount() >= 1, "OnIndex output has no SDF blind channel" - assert rb2 < r0b - 0.05, "sphere did not shrink under curvature flow" - - print("OK: both styles filter correctly and the output style matches the input.") - for n in (f_in, f_out, o_in, o_out): - os.unlink(n) - - -def main(argv): - if len(argv) >= 3: - outer = int(argv[3]) if len(argv) >= 4 else 6 - filter_file(argv[1], argv[2], outer) - elif len(argv) == 1: - self_test() - else: - print(__doc__) - raise SystemExit("usage: levelset_filter_rawkernel.py input.nvdb output.nvdb " - "[outer_iterations] (no args = self-test)") + def godunov(self, g, phi, vx, half_width): + cp = self.cp + out = cp.empty_like(phi) + self.k_godunov((g["bc"],), (BLOCK_WIDTH,), + (*self._vbm(g), np.float32(vx), np.float32(0.3 * vx), + np.float32(half_width), phi, out)) + out[0] = SENTINEL + return out + + def extrapolate(self, g, phi, vx): + cp = self.cp + out = cp.empty_like(phi) + self.k_extrapolate((g["bc"],), (BLOCK_WIDTH,), + (*self._vbm(g), np.float32(vx), phi, out)) + out[0] = SENTINEL + return out if __name__ == "__main__": - main(sys.argv) + backend = make_backend() + if backend is not None: + lsf.stencil_demo(backend) From 13f28b0139626c30a2865372ae31edcbcc38b80c Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 9 Jun 2026 04:32:00 +0000 Subject: [PATCH 32/48] nanovdb python: integer payloads for device index-grid tools, + a guard The device VoxelBlockManager gather and the sidecar-transfer / index conversion bindings were instantiated only for float/double, even though their kernels are type-generic copies. Expose the integer types so an integer label / id / occupancy sidecar can be carried through the GPU pipeline without a float round-trip: * gatherBoxStencil: add int32/uint32 overloads. Lets callers gather a neighbour-VALUE-INDEX table directly in int32 instead of the old float64-gather + int64-cast, halving the (valueCount, 27) table. * inject / injectFeatures: add int32/uint32 (InjectGrid*Functor copies via the assignment operator, so any trivially-copyable T works). * addBlindData: add int32/int64 blind payloads for every grid BuildT. * indexToGrid: add an Int32 scalar destination (Int32 is non-special, so it satisfies processLeafsKernel's static_assert) -> an Int32Grid. Also guard gatherBoxStencil / activeVoxelCoords. They write out[valueIndex] for each active voxel with the caller sizing out to activeVoxelCount + 1, which is valid only when active-voxel indexing is contiguous (the VBM invariant). On a grid carrying per-node stats / tile values (the createOnIndexGrid defaults) value indices run past activeVoxelCount and the writes go out of bounds -- previously an illegal memory access. Read valueCount / activeVoxelCount from the device grid header (DeviceGridTraits) and raise a clear error pointing at the fix. Add TestGpuInterop coverage -- the first tests for these device ops: TestDeviceInject (inject / injectFeatures over float32/int32/uint32), TestDeviceTypedSidecars (addBlindData int32/int64, indexToGrid -> Int32), and gatherBoxStencil dtype + non-contiguous-grid-rejection tests. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyTools.cc | 15 ++ nanovdb/nanovdb/python/cuda/PyAddBlindData.cu | 10 ++ .../python/cuda/PyDeviceVoxelBlockManager.cu | 32 ++++ nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu | 13 +- nanovdb/nanovdb/python/cuda/PyInjectData.cu | 4 + nanovdb/nanovdb/python/test/TestGpuInterop.py | 165 ++++++++++++++++++ 6 files changed, 234 insertions(+), 5 deletions(-) diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index db917d02a7..d95f9dce12 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -113,8 +113,12 @@ void defineToolsModule(nb::module_& m) // helper that feeds pruneGrid (nanovdb::util::cuda::Inject* functors). defineInject(cudaModule, "inject"); defineInject(cudaModule, "inject"); + defineInject(cudaModule, "inject"); + defineInject(cudaModule, "inject"); defineInjectFeatures(cudaModule, "inject"); defineInjectFeatures(cudaModule, "inject"); + defineInjectFeatures(cudaModule, "inject"); + defineInjectFeatures(cudaModule, "inject"); defineInjectPredicateToMask(cudaModule, "injectPredicateToMask"); defineInjectGridMask(cudaModule, "injectGridMask"); @@ -130,6 +134,8 @@ void defineToolsModule(nb::module_& m) defineIndexToGridScalar(cudaModule, "indexToGrid"); defineIndexToGridScalar(cudaModule, "indexToGrid"); defineIndexToGridScalar(cudaModule, "indexToGrid"); + defineIndexToGridScalar(cudaModule, "indexToGrid"); + defineIndexToGridScalar(cudaModule, "indexToGrid"); defineIndexToGridVec3(cudaModule, "indexToGrid"); defineIndexToGridVec3(cudaModule, "indexToGrid"); defineIndexToGridVec3(cudaModule, "indexToGrid"); @@ -151,6 +157,15 @@ void defineToolsModule(nb::module_& m) defineAddBlindData(cudaModule, "addBlindData"); defineAddBlindData(cudaModule, "addBlindData"); defineAddBlindData(cudaModule, "addBlindData"); + // Signed-integer blind payloads (Int32 / Int64), e.g. integer labels / ids. + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); + defineAddBlindData(cudaModule, "addBlindData"); // Device quality-control tools (mirror the host tools.* names on // tools.cuda). updateGridStats covers scalar/vector/bool grids; isValid diff --git a/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu b/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu index 68c468eaa8..259568d2b5 100644 --- a/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu +++ b/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu @@ -74,5 +74,15 @@ template void defineAddBlindData(nb::module_&, template void defineAddBlindData(nb::module_&, const char*); template void defineAddBlindData(nb::module_&, const char*); template void defineAddBlindData(nb::module_&, const char*); +// Signed-integer blind payloads (e.g. integer labels / ids), for the same grid +// BuildTs. +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); +template void defineAddBlindData(nb::module_&, const char*); } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu index affe47c1aa..a68bcf2f4a 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu @@ -15,6 +15,7 @@ #include #include #include +#include namespace nb = nanobind; using namespace nb::literals; @@ -91,6 +92,33 @@ static NanoGrid* castOnIndexDeviceGrid(nb::handle py_grid, return &nb::cast&>(py_grid); } +// gatherBoxStencil / activeVoxelCoords write out[valueIndex] for each active +// voxel, the caller having sized `out` to activeVoxelCount + 1. That is valid +// only when active-voxel indexing is CONTIGUOUS (value index 0 = background, +// 1..N = the N active voxels) -- the VoxelBlockManager invariant. A grid built +// with per-node statistics / tile values (the createOnIndexGrid defaults) has +// value indices beyond activeVoxelCount, so those writes run out of bounds (an +// illegal memory access). Detect it cheaply (two D2H header reads) and raise a +// clear error instead of crashing. +static void requireContiguousIndexing(const NanoGrid* d_grid, + const char* fn_name) +{ + using Traits = nanovdb::util::cuda::DeviceGridTraits; + const uint64_t valueCount = Traits::getValueCount(d_grid); + const uint64_t activeCount = Traits::getActiveVoxelCount(d_grid); + if (valueCount != activeCount + 1) { + std::string msg(fn_name); + msg += ": requires an OnIndex grid with contiguous active-voxel indexing " + "(value index 0 = background, 1..N = the N active voxels), but this " + "grid has " + std::to_string(valueCount) + " values for " + + std::to_string(activeCount) + " active voxels -- it carries " + "per-node statistics and/or tile values. Rebuild it with " + "createOnIndexGrid(grid, include_stats=False, include_tiles=False) " + "or voxelsToOnIndexGrid."; + throw nb::value_error(msg.c_str()); + } +} + // ------------------- DeviceVoxelBlockManagerHandle binding ----------------- static void defineHandle(nb::module_& m) @@ -359,6 +387,7 @@ template void defineGatherBoxStencil(nb::module_& m, const char* nam nb::ndarray, nb::c_contig, nb::device::cuda> out, int log2_block_width, uintptr_t stream) { auto* d_grid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencil"); + requireContiguousIndexing(d_grid, "gatherBoxStencil"); cudaStream_t s = reinterpret_cast(stream); const T* dVals = values.data(); T* dOut = out.data(); @@ -434,6 +463,7 @@ void defineActiveVoxelCoords(nb::module_& m, const char* name) nb::ndarray, nb::c_contig, nb::device::cuda> out, int log2_block_width, uintptr_t stream) { auto* d_grid = castOnIndexDeviceGrid(py_grid, "activeVoxelCoords"); + requireContiguousIndexing(d_grid, "activeVoxelCoords"); cudaStream_t s = reinterpret_cast(stream); int32_t* dOut = out.data(); nb::gil_scoped_release release; @@ -469,6 +499,8 @@ void defineDeviceVoxelBlockManager(nb::module_& m) defineBuild(m); defineGatherBoxStencil(m, "gatherBoxStencil"); defineGatherBoxStencil(m, "gatherBoxStencil"); + defineGatherBoxStencil(m, "gatherBoxStencil"); + defineGatherBoxStencil(m, "gatherBoxStencil"); defineActiveVoxelCoords(m, "activeVoxelCoords"); } diff --git a/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu b/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu index 439d466465..1824832f01 100644 --- a/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu @@ -77,15 +77,18 @@ void defineIndexToGridVec3(nb::module_& m, const char* name) "(Python int; 0 = default stream)."); } -// Destination types: float / double (scalar) and Vec3f / Vec3d (vector), each -// for both index source types (ValueIndex and ValueOnIndex). DstBuildT is -// restricted by IndexToGrid's processLeafsKernel static_assert to non-special -// types, so the quantized / index / mask BuildTs are intentionally NOT -// instantiated. +// Destination types: float / double / int32 (scalar) and Vec3f / Vec3d +// (vector), each for both index source types (ValueIndex and ValueOnIndex). +// DstBuildT is restricted by IndexToGrid's processLeafsKernel static_assert to +// non-special types (is_special = index || Fp || Point/bool/Mask), so the +// quantized / index / mask BuildTs are intentionally NOT instantiated; Int32 is +// a regular type, so it materialises an Int32Grid (e.g. an integer label grid). template void defineIndexToGridScalar(nb::module_&, const char*); template void defineIndexToGridScalar(nb::module_&, const char*); template void defineIndexToGridScalar(nb::module_&, const char*); template void defineIndexToGridScalar(nb::module_&, const char*); +template void defineIndexToGridScalar(nb::module_&, const char*); +template void defineIndexToGridScalar(nb::module_&, const char*); template void defineIndexToGridVec3(nb::module_&, const char*); diff --git a/nanovdb/nanovdb/python/cuda/PyInjectData.cu b/nanovdb/nanovdb/python/cuda/PyInjectData.cu index 997ec97d18..a6f69efb2c 100644 --- a/nanovdb/nanovdb/python/cuda/PyInjectData.cu +++ b/nanovdb/nanovdb/python/cuda/PyInjectData.cu @@ -209,8 +209,12 @@ void defineInjectGridMask(nb::module_& m, const char* name) template void defineInject(nb::module_&, const char*); template void defineInject(nb::module_&, const char*); +template void defineInject(nb::module_&, const char*); +template void defineInject(nb::module_&, const char*); template void defineInjectFeatures(nb::module_&, const char*); template void defineInjectFeatures(nb::module_&, const char*); +template void defineInjectFeatures(nb::module_&, const char*); +template void defineInjectFeatures(nb::module_&, const char*); } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/test/TestGpuInterop.py b/nanovdb/nanovdb/python/test/TestGpuInterop.py index 38a3308a4e..6c23f1c86f 100644 --- a/nanovdb/nanovdb/python/test/TestGpuInterop.py +++ b/nanovdb/nanovdb/python/test/TestGpuInterop.py @@ -70,6 +70,21 @@ def _build_device_onindex_grid(radius=20.0): return dh, dh.deviceGrid(0) +def _device_onindex_from_coords(cp, coords_np): + """Build a device OnIndex grid directly from (N,3) int32 index coords (no + host round-trip; voxelsToOnIndexGrid returns a device grid). Returns + (handle, deviceGrid, activeVoxelCount, coordsByValueIndex) where the last is + a CuPy (count+1, 3) int32 array mapping value index -> coord (row 0 unused).""" + import numpy as np + coords = cp.asarray(np.ascontiguousarray(coords_np, dtype=np.int32)) + dh = nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, 1.0) + dg = dh.deviceGrid(0) + n = int(nanovdb.tools.cuda.buildVoxelBlockManager(dg, 9, 0, 0, 0, 0).lastOffset()) + by_index = cp.empty((n + 1, 3), dtype=cp.int32) + nanovdb.tools.cuda.activeVoxelCoords(dg, by_index) + return dh, dg, n, by_index + + @unittest.skipIf( not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" ) @@ -489,6 +504,156 @@ def test_jump_map_zero_copy(self): self.assertEqual(jm.shape[0], vbm.blockCount()) self.assertEqual(int(jm.data.ptr), vbm.jump_map_ptr()) + def test_gather_box_stencil_dtypes(self): + """gatherBoxStencil works for float32 AND the integer payload dtypes + (int32/uint32). The centre spoke (column 13) is each voxel's own value, + inactive neighbours read values[0] (=0 here), and gathering arange yields + the SAME neighbour-index pattern in every dtype -- the property that lets + callers gather a neighbour-INDEX table directly in int32. + + Built from explicit coords (voxelsToOnIndexGrid) so active-voxel indexing + is contiguous -- the invariant the VoxelBlockManager gather relies on.""" + cp = _require_cupy(self) + import numpy as np + block = np.array([(i, j, k) for i in range(5) for j in range(5) for k in range(5)], + dtype=np.int32) + _h, dg, n, _co = _device_onindex_from_coords(cp, block) + N = n + 1 + ref = None + for dt in (cp.float32, cp.int32, cp.uint32): + idx = cp.arange(N, dtype=dt) + out = cp.empty((N, 27), dtype=dt) + nanovdb.tools.cuda.gatherBoxStencil(dg, idx, out) + self.assertEqual(out.dtype, cp.dtype(dt)) + self.assertTrue(bool((out[1:N, 13] == idx[1:N]).all())) # centre == self + self.assertTrue(bool((out[1:N] == 0).any())) # inactive -> values[0] + # row 0 (background slot) is never written by the kernel, so compare + # only the written rows across dtypes. + as_i64 = out[1:N].astype(cp.int64) + if ref is None: + ref = as_i64 + else: + self.assertTrue(bool((as_i64 == ref).all())) # dtype-independent + + def test_gather_rejects_noncontiguous_grid(self): + """gatherBoxStencil / activeVoxelCoords raise a clear ValueError (rather + than an illegal memory access) on a grid whose active-voxel indexing is + NOT contiguous -- e.g. createOnIndexGrid's default include_stats / + include_tiles, which add value slots past activeVoxelCount.""" + cp = _require_cupy(self) + dh, dg = _build_device_onindex_grid(20.0) # createOnIndexGrid defaults: stats+tiles + N = int(nanovdb.tools.cuda.buildVoxelBlockManager(dg, 9, 0, 0, 0, 0).lastOffset()) + 1 + with self.assertRaises(ValueError): + nanovdb.tools.cuda.gatherBoxStencil(dg, cp.arange(N, dtype=cp.int32), + cp.empty((N, 27), dtype=cp.int32)) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.activeVoxelCoords(dg, cp.empty((N, 3), dtype=cp.int32)) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceInject(unittest.TestCase): + """inject / injectFeatures copy a sidecar over the src/dst voxel + intersection, for float AND the integer payloads (int32/uint32) -- the copy + is via the assignment operator, so it is type-generic.""" + + # src = a 4^3 block; dst = a 6^3 block (a superset), so the intersection is + # exactly src. enc(coord) gives each voxel a distinct in-range value. + SRC = [(i, j, k) for i in range(4) for j in range(4) for k in range(4)] + DST = [(i, j, k) for i in range(6) for j in range(6) for k in range(6)] + SENT = 255 # distinct from every enc value; exact in f32/i32/u32 + + @staticmethod + def _enc(c): # (...,3) int -> (...) int + return (c[..., 0] * 6 + c[..., 1]) * 6 + c[..., 2] + + def test_inject_scalar_dtypes(self): + cp = _require_cupy(self) + import numpy as np + _sh, sdg, ns, sco = _device_onindex_from_coords(cp, np.array(self.SRC, np.int32)) + _dh, ddg, nd, dco = _device_onindex_from_coords(cp, np.array(self.DST, np.int32)) + sco_h = cp.asnumpy(sco[1:]).astype(np.int64) + dco_h = cp.asnumpy(dco[1:]).astype(np.int64) + in_src = (dco_h < 4).all(axis=1) + expect = np.where(in_src, self._enc(dco_h), self.SENT) + for dt in (cp.float32, cp.int32, cp.uint32): + with self.subTest(dtype=np.dtype(dt).name): + src_vals = cp.zeros(ns + 1, dtype=dt) + src_vals[1:] = cp.asarray(self._enc(sco_h), dtype=dt) + dst_vals = cp.full(nd + 1, self.SENT, dtype=dt) + nanovdb.tools.cuda.inject(sdg, ddg, src_vals, dst_vals) + self.assertEqual(dst_vals.dtype, cp.dtype(dt)) + got = cp.asnumpy(dst_vals[1:]).astype(np.int64) + self.assertTrue(bool((got == expect).all())) + + def test_inject_features_dtypes(self): + """injectFeatures: the 2-D (value count, dim) form, integer + float.""" + cp = _require_cupy(self) + import numpy as np + _sh, sdg, ns, sco = _device_onindex_from_coords(cp, np.array(self.SRC, np.int32)) + _dh, ddg, nd, dco = _device_onindex_from_coords(cp, np.array(self.DST, np.int32)) + sco_h = cp.asnumpy(sco[1:]).astype(np.int64) + dco_h = cp.asnumpy(dco[1:]).astype(np.int64) + in_src = (dco_h < 4).all(axis=1) + enc_d = self._enc(dco_h) + for dt in (cp.float32, cp.int32): + with self.subTest(dtype=np.dtype(dt).name): + src_vals = cp.zeros((ns + 1, 2), dtype=dt) + e = self._enc(sco_h) + src_vals[1:, 0] = cp.asarray(e, dtype=dt) + src_vals[1:, 1] = cp.asarray(e + 1000, dtype=dt) # second channel + dst_vals = cp.full((nd + 1, 2), self.SENT, dtype=dt) + nanovdb.tools.cuda.inject(sdg, ddg, src_vals, dst_vals) + got = cp.asnumpy(dst_vals[1:]).astype(np.int64) + exp0 = np.where(in_src, enc_d, self.SENT) + exp1 = np.where(in_src, enc_d + 1000, self.SENT) + self.assertTrue(bool((got[:, 0] == exp0).all())) + self.assertTrue(bool((got[:, 1] == exp1).all())) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceTypedSidecars(unittest.TestCase): + """Integer payloads / build types: addBlindData with int32/int64 blind data, + and indexToGrid materialising an Int32 destination grid.""" + + BLOCK = [(i, j, k) for i in range(4) for j in range(4) for k in range(4)] + + def test_add_blind_data_int_dtypes(self): + cp = _require_cupy(self) + import numpy as np + _h, dg, n, _co = _device_onindex_from_coords(cp, np.array(self.BLOCK, np.int32)) + for dt in (cp.int32, cp.int64, cp.float32): # float32 = regression + with self.subTest(dtype=np.dtype(dt).name): + blind = (cp.arange(n + 1, dtype=dt) * 3) + out_dh = nanovdb.tools.cuda.addBlindData( + dg, blind, nanovdb.GridBlindDataClass.ChannelArray, + nanovdb.GridBlindDataSemantic.Unknown, "labels") + out_dh.deviceDownload(0, True) + g = out_dh.grid(0) + self.assertEqual(g.blindDataCount(), 1) + bd = np.asarray(g.getBlindData(0)) + self.assertEqual(bd.dtype, np.dtype(dt)) + self.assertTrue(bool((bd == cp.asnumpy(blind)).all())) + + def test_index_to_grid_int32(self): + cp = _require_cupy(self) + import numpy as np + _h, dg, n, _co = _device_onindex_from_coords(cp, np.array(self.BLOCK, np.int32)) + vals = cp.arange(n + 1, dtype=cp.int32) * 7 # int32 sidecar -> Int32Grid + out_dh = nanovdb.tools.cuda.indexToGrid(dg, vals) + self.assertEqual(out_dh.gridType(0), nanovdb.GridType.Int32) + out_dh.deviceDownload(0, True) + self.assertEqual(out_dh.grid(0).activeVoxelCount(), n) + @unittest.skipIf( not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" From ea1d2a0d5e19b5c35696bcd34433eac5652f712c Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 9 Jun 2026 04:51:37 +0000 Subject: [PATCH 33/48] nanovdb python: add gatherBoxStencilColumns (subset box-stencil gather) gatherBoxStencil always materialises the full (valueCount, 27) neighbour table, but a caller often needs only a handful of the 27 spokes (e.g. an SDF mesher's 8 corners + 6 faces -- 14 of 27). Add a sibling that writes only a chosen subset into an (valueCount, K) array. The kernel is the gatherBoxStencil kernel with the write narrowed: the VBM decode still computes all 27 neighbour shared), then it writes out[k, col] = values[st[spokes[col]]] for the K requested spokes. The K (<=27) spoke indice struct, so there is no device scratch. spokes is a 1-D host int32 array, validated in [0, 27) (and against out.shape the same contiguous-active-indexing guard as gatherBoxStencil. Instantiated for float / double / int32 / u Cuts the dominant array roughly in proporti contouring mesher this is the (N,14) vs (N,27) table difference, the final step taking its full-resolution peak Add TestDeviceVoxelBlockManager.test_gather subset equals the matching columns of the full gather, and an out-of- range spoke / mismatched out.shape[1] raise Signed-off-by: Jonathan Swartz --- .../python/cuda/PyDeviceVoxelBlockManager.cu | 90 +++++++++++++++++++ nanovdb/nanovdb/python/test/TestGpuInterop.py | 23 +++++ 2 files changed, 113 insertions(+) diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu index a68bcf2f4a..077e9bf14e 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu @@ -423,6 +423,92 @@ template void defineGatherBoxStencil(nb::module_& m, const char* nam "stream)."); } +// ------------------- gatherBoxStencilColumns (subset of the 27 spokes) ----- +// +// Like gatherBoxStencil, but writes only a chosen SUBSET of the 27 box-stencil +// spokes into an (valueCount, K) array. The VBM decode still computes all 27 +// neighbour indices in registers (that work is cheap and shared), so the only +// saving is the output table -- which matters when a kernel needs just a handful +// of the 27 neighbours (e.g. an SDF mesher's 8 corners + 6 faces): a (N, K) table +// instead of (N, 27). The K (<=27) spoke indices pass by value, no device scratch. +struct ColumnSpokes { int s[27]; }; + +template +__global__ void gatherBoxStencilColumnsKernel( + const NanoGrid* grid, + const uint32_t* firstLeafID, const uint64_t* jumpMap, uint64_t firstOffset, + const T* values, T* out, ColumnSpokes spokes, int K) +{ + constexpr int BW = 1 << Log2BlockWidth; + constexpr int JML = BW / 64; + using VBM = nanovdb::tools::cuda::VoxelBlockManager; + __shared__ uint32_t smem_leafIndex[BW]; + __shared__ uint16_t smem_voxelOffset[BW]; + const uint64_t blockFirstOffset = firstOffset + uint64_t(blockIdx.x) * BW; + VBM::template decodeInverseMaps( + grid, firstLeafID[blockIdx.x], &jumpMap[uint64_t(blockIdx.x) * JML], + blockFirstOffset, smem_leafIndex, smem_voxelOffset); + const int tID = threadIdx.x; + if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + uint64_t st[27]; + VBM::template computeBoxStencil( + grid, smem_leafIndex, smem_voxelOffset, st); + const uint64_t c = st[13]; // centre value index + for (int col = 0; col < K; ++col) out[c * K + col] = values[st[spokes.s[col]]]; +} + +template void defineGatherBoxStencilColumns(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::handle py_grid, + nb::ndarray, nb::c_contig, nb::device::cuda> values, + nb::ndarray, nb::c_contig, nb::device::cuda> out, + nb::ndarray, nb::c_contig> spokes, + int log2_block_width, uintptr_t stream) { + auto* d_grid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencilColumns"); + requireContiguousIndexing(d_grid, "gatherBoxStencilColumns"); + const int K = static_cast(spokes.shape(0)); + if (K < 1 || K > 27) + throw nb::value_error("gatherBoxStencilColumns: len(spokes) must be in [1, 27]."); + if (static_cast(out.shape(1)) != K) + throw nb::value_error("gatherBoxStencilColumns: out.shape[1] must equal len(spokes)."); + ColumnSpokes sp{}; // copy + validate spokes host-side + const int32_t* hSpokes = spokes.data(); + for (int i = 0; i < K; ++i) { + if (hSpokes[i] < 0 || hSpokes[i] >= 27) + throw nb::value_error("gatherBoxStencilColumns: each spoke must be in [0, 27)."); + sp.s[i] = static_cast(hSpokes[i]); + } + cudaStream_t s = reinterpret_cast(stream); + const T* dVals = values.data(); + T* dOut = out.data(); + nb::gil_scoped_release release; + dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + constexpr int LBW = decltype(W)::value; + auto handle = nanovdb::tools::cuda::buildVoxelBlockManager< + LBW, nanovdb::cuda::DeviceBuffer>(d_grid, 0, 0, 0, s); + const uint32_t bc = static_cast(handle.blockCount()); + if (bc) + gatherBoxStencilColumnsKernel<<>>( + d_grid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), + handle.firstOffset(), dVals, dOut, sp, K); + cudaCheck(cudaStreamSynchronize(s)); + return 0; + }); + }, + "device_grid"_a, "values"_a, "out"_a, "spokes"_a, "log2_block_width"_a = 9, "stream"_a = 0, + "Like gatherBoxStencil, but gathers only a chosen SUBSET of the 27 box-" + "stencil spokes into a dense (valueCount, K) array: out[k, col] is voxel " + "k's neighbour value at spoke spokes[col], where spoke " + "j = (di+1)*9+(dj+1)*3+(dk+1) (centre 13). `spokes` is a 1-D HOST int32 " + "array of K (1..27) spoke indices in [0,27); out has shape (rows, K) with " + "rows >= valueCount. Halves/shrinks the table when only a handful of the " + "27 neighbours are needed (e.g. an SDF mesher's corners + faces). Inactive " + "neighbours read values[0]; the grid must use contiguous active-voxel " + "indexing. stream is a raw CUDA stream handle (Python int; 0 = default)."); +} + // ------------------- activeVoxelCoords (VBM coordinate decode) ------------- // // Write each active voxel's index-space coordinate into a dense (valueCount, 3) @@ -501,6 +587,10 @@ void defineDeviceVoxelBlockManager(nb::module_& m) defineGatherBoxStencil(m, "gatherBoxStencil"); defineGatherBoxStencil(m, "gatherBoxStencil"); defineGatherBoxStencil(m, "gatherBoxStencil"); + defineGatherBoxStencilColumns(m, "gatherBoxStencilColumns"); + defineGatherBoxStencilColumns(m, "gatherBoxStencilColumns"); + defineGatherBoxStencilColumns(m, "gatherBoxStencilColumns"); + defineGatherBoxStencilColumns(m, "gatherBoxStencilColumns"); defineActiveVoxelCoords(m, "activeVoxelCoords"); } diff --git a/nanovdb/nanovdb/python/test/TestGpuInterop.py b/nanovdb/nanovdb/python/test/TestGpuInterop.py index 6c23f1c86f..5e3c98758f 100644 --- a/nanovdb/nanovdb/python/test/TestGpuInterop.py +++ b/nanovdb/nanovdb/python/test/TestGpuInterop.py @@ -549,6 +549,29 @@ def test_gather_rejects_noncontiguous_grid(self): with self.assertRaises(ValueError): nanovdb.tools.cuda.activeVoxelCoords(dg, cp.empty((N, 3), dtype=cp.int32)) + def test_gather_box_stencil_columns(self): + """gatherBoxStencilColumns gathers a CHOSEN subset of the 27 spokes into + an (N,K) table -- equal to the matching columns of the full gather -- and + validates its (host) spoke list + the out shape.""" + cp = _require_cupy(self) + import numpy as np + block = np.array([(i, j, k) for i in range(5) for j in range(5) for k in range(5)], + dtype=np.int32) + _h, dg, n, _co = _device_onindex_from_coords(cp, block) + N = n + 1 + idx = cp.arange(N, dtype=cp.int32) + full = cp.empty((N, 27), dtype=cp.int32) + nanovdb.tools.cuda.gatherBoxStencil(dg, idx, full) + spokes = np.array([13, 22, 16, 14, 4, 10, 12], dtype=np.int32) # centre + 6 faces + sub = cp.empty((N, spokes.shape[0]), dtype=cp.int32) + nanovdb.tools.cuda.gatherBoxStencilColumns(dg, idx, sub, spokes) + self.assertTrue(bool((sub[1:N] == full[1:N][:, cp.asarray(spokes)]).all())) + with self.assertRaises(ValueError): # spoke out of [0, 27) + nanovdb.tools.cuda.gatherBoxStencilColumns( + dg, idx, cp.empty((N, 2), dtype=cp.int32), np.array([13, 99], np.int32)) + with self.assertRaises(ValueError): # out.shape[1] != len(spokes) + nanovdb.tools.cuda.gatherBoxStencilColumns(dg, idx, cp.empty((N, 3), dtype=cp.int32), spokes) + @unittest.skipIf( not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" From 70925c6bcf96d4a4930c2df9d326df8334165429 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 10 Jun 2026 00:43:49 +0000 Subject: [PATCH 34/48] nanovdb python: add setGridClass / setTransform grid-header setters The grid bindings exposed getters for gridClass and the transform but no way to set them after a grid was built. indexToGrid, for one, copies the source's GridClass onto its output, so an SDF FloatGrid materialised from an IndexGrid came out tagged IndexGrid with no way to retag it LevelSet. Add mutable header setters on both the host and device grid: - tools.cuda.setGridClass(d_grid, gridClass, stream): new defineDeviceGridMetadata in PyDeviceGridChecksum.cu (a one-thread setGridClassKernel writes mGridClass on the device grid, then tools::cuda::updateChecksum refreshes the checksum preserving its mode), registered for the full checksum BuildT set in PyTools.cc. - Grid.setGridClass(gridClass) and Grid.setTransform(voxelSize, translation=Vec3d(0,0,0)) on the host GridData type. setTransform builds a uniform-scale + translation Map, updates mVoxelSize, and recomputes mWorldBBox from the index bbox under the new map (mirroring GridStats). Also fix a latent wart: the existing host header setters (setGridName and the setMinMaxOn/setBBoxOn/setLongGridNameOn/setAverageOn/setStdDeviationOn flag toggles) edited the header without refreshing the checksum, silently desyncing a checksummed grid so a validating reader would reject it. They now call updateChecksum after the edit, which preserves the checksum mode and is a no-op when the checksum is disabled (so it never enables one). Tests: TestNanoVDB.TestGridHeaderSetters (host setGridClass/setTransform, the flag/name setters keeping a Full checksum valid, and all surviving a file round-trip) and TestGpuInterop.TestDeviceGridMetadata (device setGridClass proven to refresh a pre-populated Full checksum, and surviving a round-trip as FogVolume). Full suites: TestNanoVDB 144 OK, TestGpuInterop 46 OK (1 skip). Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/NanoVDBModule.cc | 66 ++++++++++++---- nanovdb/nanovdb/python/PyTools.cc | 21 +++++ .../python/cuda/PyDeviceGridChecksum.cu | 59 ++++++++++++++ .../python/cuda/PyDeviceGridChecksum.h | 6 ++ nanovdb/nanovdb/python/test/TestGpuInterop.py | 58 ++++++++++++++ nanovdb/nanovdb/python/test/TestNanoVDB.py | 78 +++++++++++++++++++ 6 files changed, 275 insertions(+), 13 deletions(-) diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 6680bc3b2e..d52aef4573 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -8,6 +8,7 @@ #include #include // for __repr__ +#include // host updateChecksum for the header setters #ifdef NANOVDB_USE_CUDA #include #endif @@ -244,21 +245,60 @@ void defineGrid(nb::module_& m) "the affine transform, grid flags and blind-data accessors. Concrete " "BuildT-typed subclasses (FloatGrid, Vec3fGrid, ...) add tree access " "and per-voxel queries.") - // Validation and flag mutators (already member functions on GridData). + // Validation and header mutators. Every mutator refreshes the grid's + // checksum afterwards (preserving its mode; a no-op when the checksum is + // disabled), so editing a header field never silently desyncs a + // checksummed grid -- a validating reader would otherwise reject it. .def("isValid", &GridData::isValid, "True iff the grid header looks consistent (magic / version / class tags).") - .def("setMinMaxOn", &GridData::setMinMaxOn, "on"_a = true, - "Toggle the HasMinMax grid flag.") - .def("setBBoxOn", &GridData::setBBoxOn, "on"_a = true, - "Toggle the HasBBox grid flag.") - .def("setLongGridNameOn", &GridData::setLongGridNameOn, "on"_a = true, - "Toggle the HasLongGridName grid flag.") - .def("setAverageOn", &GridData::setAverageOn, "on"_a = true, - "Toggle the HasAverage grid flag.") - .def("setStdDeviationOn", &GridData::setStdDeviationOn, "on"_a = true, - "Toggle the HasStdDeviation grid flag.") - .def("setGridName", &GridData::setGridName, "src"_a, - "Overwrite the grid's short name (truncated to the in-header buffer).") + .def("setMinMaxOn", + [](GridData& g, bool on) { g.setMinMaxOn(on); nanovdb::tools::updateChecksum(&g); }, + "on"_a = true, "Toggle the HasMinMax grid flag (refreshes the checksum).") + .def("setBBoxOn", + [](GridData& g, bool on) { g.setBBoxOn(on); nanovdb::tools::updateChecksum(&g); }, + "on"_a = true, "Toggle the HasBBox grid flag (refreshes the checksum).") + .def("setLongGridNameOn", + [](GridData& g, bool on) { g.setLongGridNameOn(on); nanovdb::tools::updateChecksum(&g); }, + "on"_a = true, "Toggle the HasLongGridName grid flag (refreshes the checksum).") + .def("setAverageOn", + [](GridData& g, bool on) { g.setAverageOn(on); nanovdb::tools::updateChecksum(&g); }, + "on"_a = true, "Toggle the HasAverage grid flag (refreshes the checksum).") + .def("setStdDeviationOn", + [](GridData& g, bool on) { g.setStdDeviationOn(on); nanovdb::tools::updateChecksum(&g); }, + "on"_a = true, "Toggle the HasStdDeviation grid flag (refreshes the checksum).") + .def("setGridName", + [](GridData& g, const char* src) { + const bool ok = g.setGridName(src); + nanovdb::tools::updateChecksum(&g); + return ok; + }, + "src"_a, + "Overwrite the grid's short name (truncated to the in-header buffer; " + "refreshes the checksum). Returns False if the name had to be truncated.") + .def("setGridClass", + [](GridData& g, GridClass gridClass) { + g.mGridClass = gridClass; + nanovdb::tools::updateChecksum(&g); + }, + "gridClass"_a, + "Set the grid's GridClass (e.g. GridClass.LevelSet) in place and refresh " + "the checksum. Host twin of tools.cuda.setGridClass for device grids.") + .def("setTransform", + [](GridData& g, double voxelSize, const Vec3d& translation) { + // Uniform scale + translation index->world map, then recompute the + // world-space AABB from the index bbox under the new map (mirrors + // how GridStats sets mWorldBBox). + g.mMap.set(voxelSize, translation, 1.0); + g.mVoxelSize = g.mMap.getVoxelSize(); + const CoordBBox& ib = g.indexBBox(); + g.mWorldBBox = CoordBBox(ib[0], ib[1].offsetBy(1)).transform(g.mMap); + nanovdb::tools::updateChecksum(&g); + }, + "voxelSize"_a, "translation"_a = Vec3d(0.0, 0.0, 0.0), + "Set a uniform-scale + translation index->world transform (voxelSize is " + "world units per voxel; translation is in world space), recompute the " + "world bounding box, and refresh the checksum. For non-uniform or rotated " + "maps, build the grid with the desired transform instead.") // Affine transforms (already member functions on GridData). .def("applyMap", nb::overload_cast(&GridData::template applyMap, nb::const_), "xyz"_a, "Transform an index-space point to world space using this grid's Map.") diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index d95f9dce12..5a9bbf58ea 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -224,6 +224,27 @@ void defineToolsModule(nb::module_& m) defineDeviceGridChecksum(cudaModule); defineDeviceGridChecksum(cudaModule); defineDeviceGridChecksum(cudaModule); + + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); + defineDeviceGridMetadata(cudaModule); #endif } diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu index 836f7c5ef3..f66badcbf0 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu @@ -82,6 +82,43 @@ void defineDeviceGridChecksum(nb::module_& m) "handle (Python int; 0 = default stream)."); } +// One-thread kernel that overwrites the grid's GridClass field in place. Grid +// publicly derives GridData, so mGridClass is reachable on the device pointer. +template +__global__ void setGridClassKernel(nanovdb::NanoGrid* d_grid, + nanovdb::GridClass gridClass) +{ + if (blockIdx.x == 0 && threadIdx.x == 0) d_grid->mGridClass = gridClass; +} + +// Device-side mutable grid-header metadata (the read-only counterparts already +// exist as getters on the grid objects) +template +void defineDeviceGridMetadata(nb::module_& m) +{ + m.def( + "setGridClass", + [](nanovdb::NanoGrid* d_grid, nanovdb::GridClass gridClass, + uintptr_t stream) { + if (!d_grid) throw nb::value_error("setGridClass: d_grid is None."); + cudaStream_t s = reinterpret_cast(stream); + nb::gil_scoped_release release; + setGridClassKernel<<<1, 1, 0, s>>>(d_grid, gridClass); + // Refresh the checksum (preserving its existing mode) so that + // checksum-validating readers still accept the grid after the + // class field changed; a no-op for grids with checksum disabled. + nanovdb::tools::cuda::updateChecksum( + reinterpret_cast(d_grid), s); + cudaStreamSynchronize(s); + }, + "d_grid"_a, + "gridClass"_a, + "stream"_a = 0, + "Set the device grid's GridClass (e.g. GridClass.LevelSet) in place and " + "refresh its checksum, preserving the checksum mode (returns None). " + "stream is a raw CUDA stream handle (Python int; 0 = default stream)."); +} + // No BuildT restriction on the GridChecksum device entries; instantiate for // the same set the host checksum dispatch covers via callNanoGrid. template void defineDeviceGridChecksum(nb::module_&); @@ -105,4 +142,26 @@ template void defineDeviceGridChecksum(nb::module_&); template void defineDeviceGridChecksum(nb::module_&); template void defineDeviceGridChecksum(nb::module_&); +// Same BuildT set as the checksum entries above. +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); +template void defineDeviceGridMetadata(nb::module_&); + } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h index 3838df1542..58565f8c8d 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h @@ -16,6 +16,12 @@ namespace pynanovdb { template void defineDeviceGridChecksum(nb::module_& m); +// Bind mutable grid-header metadata setters for one grid BuildT. Currently +// registers nanovdb.tools.cuda.setGridClass(d_grid, gridClass, stream), which +// overwrites the device grid's GridClass in place and refreshes its checksum. +template +void defineDeviceGridMetadata(nb::module_& m); + } // namespace pynanovdb #endif diff --git a/nanovdb/nanovdb/python/test/TestGpuInterop.py b/nanovdb/nanovdb/python/test/TestGpuInterop.py index 5e3c98758f..e5d8299fa2 100644 --- a/nanovdb/nanovdb/python/test/TestGpuInterop.py +++ b/nanovdb/nanovdb/python/test/TestGpuInterop.py @@ -678,6 +678,64 @@ def test_index_to_grid_int32(self): self.assertEqual(out_dh.grid(0).activeVoxelCount(), n) +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestDeviceGridMetadata(unittest.TestCase): + """tools.cuda.setGridClass: retag a device grid's GridClass in place, + refreshing its checksum, with the change surviving download and file I/O.""" + + BLOCK = [(i, j, k) for i in range(4) for j in range(4) for k in range(4)] + + def _float_grid_from_block(self, cp): + """OnIndex(BLOCK) + per-voxel SDF -> device FloatGrid handle. The float + grid inherits the source's IndexGrid class (what setGridClass overrides).""" + import numpy as np + _h, dg, n, _co = _device_onindex_from_coords(cp, np.array(self.BLOCK, np.int32)) + sdf = cp.arange(n + 1, dtype=cp.float32) * 0.25 + float_dh = nanovdb.tools.cuda.indexToGrid(dg, sdf) + self.assertEqual(float_dh.gridType(0), nanovdb.GridType.Float) + return float_dh + + def test_set_grid_class_refreshes_checksum(self): + cp = _require_cupy(self) + float_dh = self._float_grid_from_block(cp) + float_dg = float_dh.deviceGrid(0) + # Populate a Full checksum so the retag has something to invalidate: if + # setGridClass failed to refresh it, the post-change Full validation + # below (computed over the new class field) would no longer match. + nanovdb.tools.cuda.updateChecksum(float_dg, nanovdb.CheckMode.Full) + self.assertTrue(nanovdb.tools.cuda.validateChecksum(float_dg, nanovdb.CheckMode.Full)) + + nanovdb.tools.cuda.setGridClass(float_dg, nanovdb.GridClass.LevelSet) + # Class changed AND the Full checksum still validates -> it was refreshed. + self.assertTrue(nanovdb.tools.cuda.validateChecksum(float_dg, nanovdb.CheckMode.Full)) + + float_dh.deviceDownload(0, True) + g = float_dh.grid(0) + self.assertEqual(g.gridClass(), nanovdb.GridClass.LevelSet) + self.assertTrue(g.isLevelSet()) + self.assertTrue(nanovdb.tools.validateChecksum(g, nanovdb.CheckMode.Full)) + + def test_set_grid_class_survives_file_roundtrip(self): + cp = _require_cupy(self) + float_dh = self._float_grid_from_block(cp) + nanovdb.tools.cuda.setGridClass(float_dh.deviceGrid(0), nanovdb.GridClass.FogVolume) + float_dh.deviceDownload(0, True) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + try: + float_dh.write(tmp.name) + g = nanovdb.io.readGrid(tmp.name).grid(0) + self.assertEqual(g.gridClass(), nanovdb.GridClass.FogVolume) + self.assertTrue(g.isFogVolume()) + finally: + os.unlink(tmp.name) + + @unittest.skipIf( not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" ) diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index 59215960aa..5c45e41bec 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -420,6 +420,84 @@ def test_constructed_from_grid(self): self.assertTrue(nanovdb.GridMetaData.safeCast(h.grid())) +class TestGridHeaderSetters(unittest.TestCase): + """Header mutators on the Grid base (setGridClass / setTransform / setGridName + / the flag toggles) edit the grid in place and refresh its checksum, so a + validating reader still accepts the grid after the edit.""" + + def _sphere(self): + # voxelSize 1.0 so setTransform's effect on voxelSize() is unambiguous. + h = nanovdb.tools.createLevelSetSphere(radius=10.0, voxelSize=1.0) + return h, h.grid() + + def test_set_grid_class_refreshes_checksum(self): + h, g = self._sphere() + self.assertTrue(g.isLevelSet()) + # Populate a Full checksum so a missing refresh would leave it stale. + nanovdb.tools.updateChecksum(g, nanovdb.CheckMode.Full) + self.assertTrue(nanovdb.tools.validateChecksum(g, nanovdb.CheckMode.Full)) + + g.setGridClass(nanovdb.GridClass.FogVolume) + self.assertEqual(g.gridClass(), nanovdb.GridClass.FogVolume) + self.assertTrue(g.isFogVolume()) + self.assertFalse(g.isLevelSet()) + # Still validates -> the class change refreshed the checksum. + self.assertTrue(nanovdb.tools.validateChecksum(g, nanovdb.CheckMode.Full)) + self.assertTrue(g.isValid()) + + def test_set_transform(self): + h, g = self._sphere() + index_bbox_before = str(g.indexBBox()) + nanovdb.tools.updateChecksum(g, nanovdb.CheckMode.Full) + + # Uniform scale 2 + world translation (5,0,0). + g.setTransform(2.0, nanovdb.math.Vec3d(5.0, 0.0, 0.0)) + self.assertEqual(g.voxelSize()[0], 2.0) + # The transform relabels world coordinates; it does not move voxels. + self.assertEqual(str(g.indexBBox()), index_bbox_before) + # world = scale * index + translation + self.assertEqual(g.applyMap(nanovdb.math.Vec3d(0, 0, 0)), nanovdb.math.Vec3d(5, 0, 0)) + self.assertEqual(g.applyMap(nanovdb.math.Vec3d(1, 0, 0)), nanovdb.math.Vec3d(7, 0, 0)) + # World bbox was recomputed under the new map (no longer the unit-scale box). + self.assertEqual(g.voxelSize(), nanovdb.math.Vec3d(2.0)) + self.assertTrue(nanovdb.tools.validateChecksum(g, nanovdb.CheckMode.Full)) + + # Default translation is the origin. + g.setTransform(0.5) + self.assertEqual(g.voxelSize()[0], 0.5) + self.assertEqual(g.applyMap(nanovdb.math.Vec3d(0, 0, 0)), nanovdb.math.Vec3d(0, 0, 0)) + + def test_flag_and_name_setters_refresh_checksum(self): + h, g = self._sphere() + nanovdb.tools.updateChecksum(g, nanovdb.CheckMode.Full) + for toggle in (lambda: g.setBBoxOn(True), lambda: g.setMinMaxOn(False), + lambda: g.setAverageOn(True), lambda: g.setStdDeviationOn(False), + lambda: g.setLongGridNameOn(False)): + toggle() + self.assertTrue(nanovdb.tools.validateChecksum(g, nanovdb.CheckMode.Full)) + self.assertTrue(g.setGridName("renamed_grid")) + self.assertEqual(g.shortGridName(), "renamed_grid") + self.assertTrue(nanovdb.tools.validateChecksum(g, nanovdb.CheckMode.Full)) + + def test_setters_survive_file_roundtrip(self): + h, g = self._sphere() + g.setGridClass(nanovdb.GridClass.FogVolume) + g.setTransform(4.0, nanovdb.math.Vec3d(1.0, 2.0, 3.0)) + g.setGridName("retagged") + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + try: + nanovdb.io.writeGrid(tmp.name, h) + g2 = nanovdb.io.readGrid(tmp.name).grid() + self.assertEqual(g2.gridClass(), nanovdb.GridClass.FogVolume) + self.assertEqual(g2.voxelSize()[0], 4.0) + self.assertEqual(g2.shortGridName(), "retagged") + self.assertEqual(g2.applyMap(nanovdb.math.Vec3d(0, 0, 0)), + nanovdb.math.Vec3d(1, 2, 3)) + finally: + os.unlink(tmp.name) + + class TestBlindDataEmpty(unittest.TestCase): """Blind data API (blindDataCount, blindMetaData, findBlindData, findBlindDataForSemantic, getBlindData) returns sensible None/-1 From f5a6f2738b75c906b401798bf8506d20b688bad6 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 10 Jun 2026 03:45:49 +0000 Subject: [PATCH 35/48] nanovdb: fix indexToGrid writing a truncated grid (wrong mGridSize) processGridTreeRootKernel builds the destination grid header by copying the source index grid's header verbatim (*dstGrid.data() = *srcGrid.data()) and then only patches mGridType and mData1. It never updated mGridSize or the blind-metadata fields, so the destination inherited the SOURCE grid's values for them. A destination value grid has a different memory footprint than its source index grid -- e.g. a NanoLeaf stores a full 512-value array where the compact OnIndex leaf stores none -- so a dense float grid is roughly 2x the bytes of the index grid it came from. With mGridSize left at the smaller source value, GridHandle::write emits only mGridSize bytes and truncates the grid: about half the leaves are lost and the surviving node offsets dangle past the end of the file. The device buffer itself was allocated correctly (getBuffer uses nodeAcc.size), so the grid was fine in memory and only the written file was corrupt -- which then crashed downstream readers such as nanoToOpenVDB / nanovdb_convert. The bug only bites when the destination is larger than the source (large/dense grids); small or sparse grids, where the inherited size still covers the data, were unaffected. Set mGridSize to nodeAcc->size (the actual destination layout) and reset the blind-metadata fields, since indexToGrid copies no blind data. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh b/nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh index 81110f31d1..60bcc25d0d 100644 --- a/nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh +++ b/nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh @@ -147,6 +147,10 @@ __global__ void processGridTreeRootKernel(typename IndexToGrid::NodeA *dstGrid.data() = *srcGrid.data(); dstGrid.mGridType = toGridType(); dstGrid.mData1 = 0u; + + dstGrid.mGridSize = nodeAcc->size; + dstGrid.mBlindMetadataOffset = nodeAcc->size; + dstGrid.mBlindMetadataCount = 0u; // we will recompute GridData::mChecksum later // process Tree From 592b4998fec5a077c10770442a23a8b7ea1eb0ef Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 25 Jun 2026 23:27:45 +0000 Subject: [PATCH 36/48] Add binding and tests for n-ary overload of nanovdb::tools::cuda::MergeGrids Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/cuda/PyMergeGrids.cu | 27 ++++++++ nanovdb/nanovdb/python/test/TestGpuInterop.py | 63 +++++++++++++++++++ 2 files changed, 90 insertions(+) diff --git a/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu b/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu index 25a98a12b4..aa3e789b63 100644 --- a/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu +++ b/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu @@ -3,6 +3,8 @@ #include "PyMergeGrids.h" #include +#include +#include #include @@ -37,6 +39,31 @@ template void defineMergeGrids(nb::module_& m, const char* name "strictly binary; chain calls to union more than two grids. Output " "metadata is taken from d_grid1. stream is a raw CUDA stream handle " "(Python int; 0 = default stream)."); + + // List overload: N-ary merge + using GridT = nanovdb::NanoGrid; + m.def( + name, + [](nb::sequence grids_seq, uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // Collect device-grid pointers (touches Python -> GIL held). + std::vector grids; + const size_t n = nb::len(grids_seq); + grids.reserve(n); + for (size_t i = 0; i < n; ++i) + grids.push_back(nb::cast(grids_seq[i])); + if (grids.empty()) + throw nb::value_error("mergeGrids: empty grid list"); + nb::gil_scoped_release release; + return nanovdb::tools::cuda::MergeGrids(grids, s).getHandle(); + }, + "grids"_a, + "stream"_a = 0, + "Topologically merge (active-mask union) a list of device OnIndex grids " + "into one fresh device GridHandle in a single N-ary pass. Pass device " + "grids, e.g. [h.deviceGrid(0) for h in handles]. Output metadata is taken " + "from the first grid. stream is a raw CUDA stream handle (Python int; " + "0 = default stream)."); } template void defineMergeGrids(nb::module_&, const char*); diff --git a/nanovdb/nanovdb/python/test/TestGpuInterop.py b/nanovdb/nanovdb/python/test/TestGpuInterop.py index e5d8299fa2..326eb46f26 100644 --- a/nanovdb/nanovdb/python/test/TestGpuInterop.py +++ b/nanovdb/nanovdb/python/test/TestGpuInterop.py @@ -834,5 +834,68 @@ def test_torch_cuda_array_interface(self): self.assertTrue(t.is_cuda) +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestMergeGridsList(unittest.TestCase): + """nanovdb.tools.cuda.mergeGrids -- topological (active-mask) union of device + OnIndex grids. The binary form merges exactly two; the list overload folds an + arbitrary number in a single native N-ary pass and must agree with chaining + the binary form.""" + + def _grid(self, cp, pts): + import numpy as np + coords = cp.asarray(np.ascontiguousarray(pts, dtype=np.int32)) + return nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, 1.0) + + def _stats(self, h): + """(active voxels, (leaf, lower, upper) node counts, grid bytes).""" + h.deviceDownload(0, True) + t = h.grid(0).tree() + return (int(t.activeVoxelCount()), + (t.nodeCount(0), t.nodeCount(1), t.nodeCount(2)), + int(h.gridSize(0))) + + def test_disjoint_union_is_sum(self): + cp = _require_cupy(self) + sets = [[[0, 0, 0], [1, 0, 0]], [[5, 0, 0]], + [[0, 9, 0], [0, 10, 0]], [[4096, 0, 0]]] # last spans a new upper + hs = [self._grid(cp, s) for s in sets] + res = nanovdb.tools.cuda.mergeGrids([h.deviceGrid(0) for h in hs]) + self.assertEqual(self._stats(res)[0], sum(len(s) for s in sets)) + + def test_overlap_dedups_and_matches_pairwise(self): + cp = _require_cupy(self) + # B shares (2,0,0) with A; C lives in a separate upper region. + A = [[0, 0, 0], [1, 0, 0], [2, 0, 0]] + B = [[2, 0, 0], [3, 0, 0]] + C = [[0, 9, 0], [4096, 0, 0]] + hs = [self._grid(cp, s) for s in (A, B, C)] + truth = len({tuple(p) for p in A + B + C}) + self.assertLess(truth, len(A) + len(B) + len(C)) # overlap really exists + + nary = nanovdb.tools.cuda.mergeGrids([h.deviceGrid(0) for h in hs]) + ab = nanovdb.tools.cuda.mergeGrids(hs[0].deviceGrid(0), hs[1].deviceGrid(0)) + abc = nanovdb.tools.cuda.mergeGrids(ab.deviceGrid(0), hs[2].deviceGrid(0)) + + s_nary, s_chain = self._stats(nary), self._stats(abc) + self.assertEqual(s_nary[0], truth) # union dedups the shared voxel + self.assertEqual(s_nary, s_chain) # node counts + bytes identical to chaining + + def test_single_element_is_copy(self): + cp = _require_cupy(self) + h = self._grid(cp, [[0, 0, 0], [0, 1, 0], [0, 2, 0]]) + res = nanovdb.tools.cuda.mergeGrids([h.deviceGrid(0)]) + self.assertEqual(self._stats(res)[0], 3) + + def test_empty_list_raises(self): + _require_cupy(self) + with self.assertRaises(Exception): + nanovdb.tools.cuda.mergeGrids([]) + + if __name__ == "__main__": unittest.main() From 1a3a7b7b10b82bb477c10809d3c4c26bb7975adc Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Fri, 3 Jul 2026 02:54:47 +0000 Subject: [PATCH 37/48] nanovdb python: index-grid completeness, transform aliases, sampler gradients Close the residual gaps between the Python bindings and the C++ host API: - Bind Grid.valueCount() on Index/OnIndex grids and Grid.pointCount() on PointGrid, gated per BuildT like the C++ SFINAE. - Add tools.CreateNanoGrid mirroring the C++ converter class, exposing addBlindData() so blind-data channels can be authored from Python and filled through the writable getBlindData() NumPy view. Authored channels are zero-filled (the C++ path leaves them uninitialized). - Bind ChannelAccessor for Index/OnIndex x float/double/int32/Vec3f, plus a polymorphic createChannelAccessor() dispatching on the channel's recorded dataType. - Add the worldToIndex/indexToWorld convenience names (Dir/Grad/F variants included) as aliases on the type-erased Grid base. - Bind gradient() and zeroCrossing() on the host samplers for the orders/BuildTs where the C++ templates compile (trilinear gradient, trilinear+triquadratic zeroCrossing, floating-point only). - Fix a pre-existing bug: getBlindData() and the PointAccessor views built ndarrays without the nb::numpy framework tag, so the def-site keep_alive raised 'could not create a weak reference' whenever a non-empty view was returned. - Paper cuts: restore the empty error messages in the sphere/torus primitive dispatchers, drop the unused baseName parameter from defineStats, and bind Checksum.isEmpty/isHalf/isFull/mode. TestNanoVDB.py grows 139 -> 161 cases, covering the new surface plus the previously untested populated blind-data and channel-accessor paths. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/NanoVDBModule.cc | 202 +++++++++++- nanovdb/nanovdb/python/PyCreateNanoGrid.cc | 211 ++++++++++++ nanovdb/nanovdb/python/PyCreateNanoGrid.h | 5 +- nanovdb/nanovdb/python/PyGridChecksum.cc | 10 +- nanovdb/nanovdb/python/PyGridStats.cc | 7 +- nanovdb/nanovdb/python/PyPrimitives.cc | 32 +- nanovdb/nanovdb/python/PySampleFromVoxels.cc | 36 ++- nanovdb/nanovdb/python/test/TestNanoVDB.py | 321 +++++++++++++++++++ 8 files changed, 789 insertions(+), 35 deletions(-) diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 65fc861b26..38b69f2db8 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -292,6 +292,50 @@ void defineGrid(nb::module_& m) "Apply the inverse-Jacobian-transpose in single precision.") .def("applyIJTF", nb::overload_cast(&GridData::template applyIJTF, nb::const_), "xyz"_a, "Apply the inverse-Jacobian-transpose in single precision.") + // The C++ Grid convenience names for the same transforms + // (worldToIndex / indexToWorld and friends, NanoVDB.h). Bound as + // aliases on the type-erased base so the names most users know + // from OpenVDB / NanoVDB are directly discoverable in Python. + .def("worldToIndex", [](const GridData& g, const Vec3f& xyz) { return g.applyInverseMap(xyz); }, "xyz"_a, + "Transform a world-space point to index space. Alias of applyInverseMap.") + .def("worldToIndex", [](const GridData& g, const Vec3d& xyz) { return g.applyInverseMap(xyz); }, "xyz"_a, + "Transform a world-space point to index space. Alias of applyInverseMap.") + .def("indexToWorld", [](const GridData& g, const Vec3f& xyz) { return g.applyMap(xyz); }, "xyz"_a, + "Transform an index-space point to world space. Alias of applyMap.") + .def("indexToWorld", [](const GridData& g, const Vec3d& xyz) { return g.applyMap(xyz); }, "xyz"_a, + "Transform an index-space point to world space. Alias of applyMap.") + .def("worldToIndexDir", [](const GridData& g, const Vec3f& dir) { return g.applyInverseJacobian(dir); }, "dir"_a, + "Transform a world-space direction to index space. Alias of applyInverseJacobian.") + .def("worldToIndexDir", [](const GridData& g, const Vec3d& dir) { return g.applyInverseJacobian(dir); }, "dir"_a, + "Transform a world-space direction to index space. Alias of applyInverseJacobian.") + .def("indexToWorldDir", [](const GridData& g, const Vec3f& dir) { return g.applyJacobian(dir); }, "dir"_a, + "Transform an index-space direction to world space. Alias of applyJacobian.") + .def("indexToWorldDir", [](const GridData& g, const Vec3d& dir) { return g.applyJacobian(dir); }, "dir"_a, + "Transform an index-space direction to world space. Alias of applyJacobian.") + .def("indexToWorldGrad", [](const GridData& g, const Vec3f& grad) { return g.applyIJT(grad); }, "grad"_a, + "Transform an index-space gradient (normal) to world space. Alias of applyIJT.") + .def("indexToWorldGrad", [](const GridData& g, const Vec3d& grad) { return g.applyIJT(grad); }, "grad"_a, + "Transform an index-space gradient (normal) to world space. Alias of applyIJT.") + .def("worldToIndexF", [](const GridData& g, const Vec3f& xyz) { return g.applyInverseMapF(xyz); }, "xyz"_a, + "worldToIndex in single precision. Alias of applyInverseMapF.") + .def("worldToIndexF", [](const GridData& g, const Vec3d& xyz) { return g.applyInverseMapF(xyz); }, "xyz"_a, + "worldToIndex in single precision. Alias of applyInverseMapF.") + .def("indexToWorldF", [](const GridData& g, const Vec3f& xyz) { return g.applyMapF(xyz); }, "xyz"_a, + "indexToWorld in single precision. Alias of applyMapF.") + .def("indexToWorldF", [](const GridData& g, const Vec3d& xyz) { return g.applyMapF(xyz); }, "xyz"_a, + "indexToWorld in single precision. Alias of applyMapF.") + .def("worldToIndexDirF", [](const GridData& g, const Vec3f& dir) { return g.applyInverseJacobianF(dir); }, "dir"_a, + "worldToIndexDir in single precision. Alias of applyInverseJacobianF.") + .def("worldToIndexDirF", [](const GridData& g, const Vec3d& dir) { return g.applyInverseJacobianF(dir); }, "dir"_a, + "worldToIndexDir in single precision. Alias of applyInverseJacobianF.") + .def("indexToWorldDirF", [](const GridData& g, const Vec3f& dir) { return g.applyJacobianF(dir); }, "dir"_a, + "indexToWorldDir in single precision. Alias of applyJacobianF.") + .def("indexToWorldDirF", [](const GridData& g, const Vec3d& dir) { return g.applyJacobianF(dir); }, "dir"_a, + "indexToWorldDir in single precision. Alias of applyJacobianF.") + .def("indexToWorldGradF", [](const GridData& g, const Vec3f& grad) { return g.applyIJTF(grad); }, "grad"_a, + "indexToWorldGrad in single precision. Alias of applyIJTF.") + .def("indexToWorldGradF", [](const GridData& g, const Vec3d& grad) { return g.applyIJTF(grad); }, "grad"_a, + "indexToWorldGrad in single precision. Alias of applyIJTF.") // Strings, geometry, layout (already member functions on GridData). .def("gridName", &GridData::gridName, "Full grid name as a C string. Reads the long-form name from " @@ -421,6 +465,20 @@ template void defineNanoGrid(nb::module_& m, const char* name) nb::rv_policy::reference_internal, "Return the tree associated with this grid. Lifetime is " "anchored to the grid (and therefore to the GridHandle)."); + // Grid::valueCount / pointCount are SFINAE-gated in C++ to the + // index and Point BuildTs respectively — mirror that gating here. + if constexpr (BuildTraits::is_index) { + cls.def("valueCount", + [](const NanoGrid& grid) { return grid.valueCount(); }, + "Total number of values indexed by this grid. Sizes the " + "external or blind-data channel arrays that this index " + "grid's per-voxel indices map into."); + } + if constexpr (util::is_same::value) { + cls.def("pointCount", + [](const NanoGrid& grid) { return grid.pointCount(); }, + "Total number of points indexed by this PointGrid."); + } // Add leaf_values() only for BuildTs whose LeafData carries T mValues[512]. PyLeafValuesBinder::apply(cls); } @@ -493,17 +551,20 @@ static nb::object pyGetBlindData(nb::handle py_grid, uint32_t n) const size_t count = static_cast(meta->mValueCount); const uint32_t valueSize = meta->mValueSize; + // The nb::numpy framework tag matters: the def-site keep_alive<0, 1>() + // needs a weak-referenceable return value, which numpy.ndarray is and + // nanobind's framework-agnostic ndarray wrapper is not. auto make1D = [&](void* p, size_t n_elems, auto sentinel) -> nb::object { using T = decltype(sentinel); size_t shape[1] = {n_elems}; - return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( + return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( static_cast(p), 1, shape, py_grid), nb::rv_policy::reference); }; auto make2D = [&](void* p, size_t n_outer, size_t n_inner, auto sentinel) -> nb::object { using T = decltype(sentinel); size_t shape[2] = {n_outer, n_inner}; - return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( + return nb::cast(nb::ndarray, nb::c_contig, nb::device::cpu>( static_cast(p), 2, shape, py_grid), nb::rv_policy::reference); }; @@ -574,8 +635,9 @@ nb::object pyPointsToNdarray(nb::handle py_self, uint64_t count) { size_t shape[1] = {static_cast(count)}; + // nb::numpy tag required — see the equivalent note in pyGetBlindData. return nb::cast( - nb::ndarray, nb::c_contig, nb::device::cpu>( + nb::ndarray, nb::c_contig, nb::device::cpu>( const_cast(begin), 1, shape, py_self), nb::rv_policy::reference); } @@ -586,8 +648,9 @@ nb::object pyPointsToNdarray(nb::handle py_self, uint64_t count) { size_t shape[2] = {static_cast(count), 3}; + // nb::numpy tag required — see the equivalent note in pyGetBlindData. return nb::cast( - nb::ndarray, nb::c_contig, nb::device::cpu>( + nb::ndarray, nb::c_contig, nb::device::cpu>( reinterpret_cast(const_cast(begin)), 2, shape, py_self), nb::rv_policy::reference); } @@ -639,6 +702,121 @@ template void definePointAccessor(nb::module_& m, const char* nam "alive."); } +// Typed ChannelAccessor over an Index/OnIndex grid: combines the uint64 +// per-voxel index lookup with a read of the matching blind-data channel, so +// channel values can be queried directly by Coord. The C++ ctor only +// NANOVDB_ASSERTs its preconditions (debug builds), so the binding validates +// them explicitly and raises instead of returning an accessor that would +// dereference a null channel pointer. +template +void defineChannelAccessor(nb::module_& m, const char* name) +{ + using CA = ChannelAccessor; + nb::class_(m, name, + "Accessor that reads one blind-data channel of an Index/OnIndex " + "grid at Coord positions. Build via createChannelAccessor() for " + "automatic channel-dtype dispatch.") + .def( + "__init__", + [](CA* self, const NanoGrid& grid, uint32_t channelID) { + if (grid.gridClass() != GridClass::IndexGrid) + throw nb::value_error( + "ChannelAccessor: grid must have GridClass.IndexGrid."); + if (channelID >= grid.blindDataCount()) + throw nb::index_error( + "ChannelAccessor: channelID is out of range."); + new (self) CA(grid, channelID); + if (!*self) + throw nb::type_error( + "ChannelAccessor: the blind-data channel's dataType " + "does not match this accessor's channel type."); + }, + "grid"_a, "channelID"_a = 0u, nb::keep_alive<1, 2>(), + "Construct an accessor over the given index grid's channelID-th " + "blind-data channel. Raises if the grid is not an IndexGrid, the " + "channel is out of range, or its dtype does not match.") + .def("__bool__", [](const CA& acc) { return bool(acc); }, + "True iff this accessor is bound to a valid channel.") + .def("grid", &CA::grid, nb::rv_policy::reference_internal, + "Return the index grid this accessor is bound to.") + .def("valueCount", [](const CA& acc) { return acc.valueCount(); }, + "Total number of values indexed by the underlying index grid.") + .def( + "setChannel", + [](CA& acc, uint32_t channelID) { + if (channelID >= acc.grid().blindDataCount()) + throw nb::index_error( + "setChannel: channelID is out of range."); + if (acc.setChannel(channelID) == nullptr) + throw nb::type_error( + "setChannel: the blind-data channel's dataType does " + "not match this accessor's channel type."); + }, + "channelID"_a, + "Switch this accessor to another blind-data channel of the same " + "grid. Raises if the channel is out of range or its dtype does " + "not match.") + .def("getIndex", [](const CA& acc, const Coord& ijk) { return acc.getIndex(ijk); }, "ijk"_a, + "Linear offset into the channel array for the voxel at ijk.") + .def("idx", [](const CA& acc, int i, int j, int k) { return acc.idx(i, j, k); }, "i"_a, "j"_a, "k"_a, + "Linear offset into the channel array for the voxel at (i, j, k).") + .def("getValue", [](const CA& acc, const Coord& ijk) -> ChannelT { return acc.getValue(ijk); }, "ijk"_a, + "Channel value mapped to the voxel at ijk.") + .def( + "__call__", [](const CA& acc, const Coord& ijk) -> ChannelT { return acc.getValue(ijk); }, nb::is_operator(), "ijk"_a, + "Channel value mapped to the voxel at ijk.") + .def( + "__call__", [](const CA& acc, int i, int j, int k) -> ChannelT { return acc(i, j, k); }, nb::is_operator(), "i"_a, "j"_a, "k"_a, + "Channel value mapped to the voxel at (i, j, k).") + .def("isActive", [](const CA& acc, const Coord& ijk) { return acc.isActive(ijk); }, "ijk"_a, + "True iff the voxel at ijk is active in the index grid.") + .def( + "probeValue", + [](const CA& acc, const Coord& ijk) { + typename util::remove_const::type v; + bool isOn = acc.probeValue(ijk, v); + return std::make_tuple(v, isOn); + }, + "ijk"_a, + "Return (channel value, isActive) for the voxel at ijk in a " + "single tree traversal."); +} + +// Polymorphic factory: dispatch on the grid's index BuildT and the +// channel's recorded dataType, returning the matching typed +// ChannelAccessor. The def-site keep_alive<0, 1>() anchors the grid. +template +nb::object tryCreateChannelAccessor(nb::handle py_grid, uint32_t channelID) +{ + using GridT = NanoGrid; + if (!nb::isinstance(py_grid)) return nb::object(); + const auto& grid = nb::cast(py_grid); + if (grid.gridClass() != GridClass::IndexGrid) + throw nb::value_error( + "createChannelAccessor: grid must have GridClass.IndexGrid."); + if (channelID >= grid.blindDataCount()) + throw nb::index_error( + "createChannelAccessor: channelID is out of range."); + switch (grid.blindMetaData(channelID).mDataType) { + case GridType::Float: return nb::cast(ChannelAccessor(grid, channelID)); + case GridType::Double: return nb::cast(ChannelAccessor(grid, channelID)); + case GridType::Int32: return nb::cast(ChannelAccessor(grid, channelID)); + case GridType::Vec3f: return nb::cast(ChannelAccessor(grid, channelID)); + default: + throw nb::type_error( + "createChannelAccessor: the channel's dataType has no bound " + "ChannelAccessor (supported: Float, Double, Int32, Vec3f)."); + } +} + +nb::object createChannelAccessorImpl(nb::handle py_grid, uint32_t channelID) +{ + if (auto r = tryCreateChannelAccessor(py_grid, channelID); r.is_valid()) return r; + if (auto r = tryCreateChannelAccessor(py_grid, channelID); r.is_valid()) return r; + throw nb::type_error( + "createChannelAccessor: grid must be an IndexGrid or OnIndexGrid."); +} + // Type-erased grid introspector. Mirrors nanovdb::GridMetaData (768B) and // answers "what's in this buffer?" questions without needing to know // BuildT. Construct from a Grid (which is the Python-side GridData); all @@ -1020,6 +1198,22 @@ NB_MODULE(nanovdb, m) definePointAccessor(m, "PointIndexAccessor"); definePointAccessor(m, "PointDataAccessor"); + // ChannelAccessor — the channel dtypes match the + // source BuildTs accepted by tools.createNanoGridIndex / OnIndex. + defineChannelAccessor(m, "IndexFloatChannelAccessor"); + defineChannelAccessor(m, "IndexDoubleChannelAccessor"); + defineChannelAccessor(m, "IndexInt32ChannelAccessor"); + defineChannelAccessor(m, "IndexVec3fChannelAccessor"); + defineChannelAccessor(m, "OnIndexFloatChannelAccessor"); + defineChannelAccessor(m, "OnIndexDoubleChannelAccessor"); + defineChannelAccessor(m, "OnIndexInt32ChannelAccessor"); + defineChannelAccessor(m, "OnIndexVec3fChannelAccessor"); + m.def("createChannelAccessor", &createChannelAccessorImpl, + "grid"_a, "channelID"_a = 0u, nb::keep_alive<0, 1>(), + "Return a typed ChannelAccessor over the channelID-th blind-data " + "channel of an IndexGrid or OnIndexGrid, dispatching on the " + "channel's recorded dataType. The accessor keeps the grid alive."); + defineHostBuffer(m); defineHostGridHandle(m); diff --git a/nanovdb/nanovdb/python/PyCreateNanoGrid.cc b/nanovdb/nanovdb/python/PyCreateNanoGrid.cc index 37183a8c34..a6026ffdc6 100644 --- a/nanovdb/nanovdb/python/PyCreateNanoGrid.cc +++ b/nanovdb/nanovdb/python/PyCreateNanoGrid.cc @@ -11,7 +11,9 @@ #include #include +#include #include +#include namespace nb = nanobind; using namespace nb::literals; @@ -239,6 +241,175 @@ nb::object createIndexImpl(nb::handle py_src, throw nb::type_error(msg.c_str()); } +// ----- tools.CreateNanoGrid: converter class with blind-data authoring ----- +// +// Mirrors nanovdb::tools::CreateNanoGrid. The C++ class is +// templated on the source grid type, so this binding stores the source +// Python object plus the recorded settings and addBlindData() calls, then +// dispatches over the supported SrcBuildTs when getHandle() is called — +// constructing the C++ converter, replaying the recorded state, and baking +// the handle. The destination BuildT is the source BuildT (the C++ +// default); quantized and index destinations remain on the +// createNanoGridFp* / createNanoGridIndex free functions above. Authored +// channels come back zeroed — fill them through the writable NumPy view +// returned by grid.getBlindData(n). + +// Byte size of one element of the given blind-data GridType, or 0 when the +// size cannot be derived (the caller must then pass it explicitly). Matches +// the per-type size table enforced by GridBlindMetaData::isValid(). +uint32_t blindDataTypeSize(GridType dataType) +{ + switch (dataType) { + case GridType::Float: return 4u; + case GridType::Double: return 8u; + case GridType::Int16: return 2u; + case GridType::Int32: return 4u; + case GridType::Int64: return 8u; + case GridType::UInt8: return 1u; + case GridType::UInt32: return 4u; + case GridType::Half: return 2u; + case GridType::RGBA8: return 4u; + case GridType::Fp8: return 1u; + case GridType::Fp16: return 2u; + case GridType::Vec3f: return 12u; + case GridType::Vec3d: return 24u; + case GridType::Vec4f: return 16u; + case GridType::Vec4d: return 32u; + case GridType::Vec3u8: return 3u; + case GridType::Vec3u16: return 6u; + default: return 0u; + } +} + +class PyCreateNanoGrid +{ + struct BlindDataSpec + { + std::string name; + GridBlindDataSemantic semantic; + GridBlindDataClass dataClass; + GridType dataType; + uint64_t count; + uint32_t size; + }; + +public: + explicit PyCreateNanoGrid(nb::object src) + : mSrc(std::move(src)) + { + if (!(matches() || matches() || + matches() || matches())) { + throw nb::type_error( + "CreateNanoGrid: source must be a FloatGrid, DoubleGrid, " + "Int32Grid, Vec3fGrid, or the matching " + "nanovdb.tools.build.* mutable grid."); + } + } + + // Validates eagerly (the C++ ctor only NANOVDB_ASSERTs, which release + // builds skip) so mistakes surface here rather than as an invalid grid. + uint64_t addBlindData(const std::string& name, + uint64_t count, + GridType dataType, + GridBlindDataSemantic semantic, + GridBlindDataClass dataClass, + uint32_t size) + { + if (name.size() >= GridBlindMetaData::MaxNameSize) { + throw nb::value_error( + "addBlindData: name exceeds the 255 character limit."); + } + if (size == 0u) size = blindDataTypeSize(dataType); + if (size == 0u) { + throw nb::value_error( + "addBlindData: the element size cannot be derived from this " + "dataType — pass size explicitly."); + } + const GridBlindMetaData meta(0, count, size, semantic, dataClass, dataType); + if (!meta.isValid()) { + throw nb::value_error( + "addBlindData: invalid combination of dataSemantic, " + "dataClass, dataType, and size."); + } + mBlind.push_back(BlindDataSpec{name, semantic, dataClass, dataType, count, size}); + return static_cast(mBlind.size() - 1); + } + + void setStats(tools::StatsMode mode) { mStats = mode; } + void setChecksum(CheckMode mode) { mChecksum = mode; } + void setVerbose(int mode) { mVerbose = mode; } + void enableDithering(bool on) { mDither = on; } + + nb::object getHandle() const + { + if (auto r = tryGetHandle(); r.is_valid()) return r; + if (auto r = tryGetHandle(); r.is_valid()) return r; + if (auto r = tryGetHandle(); r.is_valid()) return r; + if (auto r = tryGetHandle(); r.is_valid()) return r; + throw nb::type_error("CreateNanoGrid: unsupported source grid type."); + } + +private: + template bool matches() const + { + return nb::isinstance>(mSrc) || + nb::isinstance>(mSrc); + } + + template nb::object tryGetHandle() const + { + using NanoSrcT = NanoGrid; + using BuildSrcT = tools::build::Grid; + if (nb::isinstance(mSrc)) return this->bake(nb::cast(mSrc)); + if (nb::isinstance(mSrc)) return this->bake(nb::cast(mSrc)); + return nb::object(); + } + + // Same GIL pattern as tryQuantizeFpX: the dispatch above runs with the + // GIL held, the traversal runs without it (the source's lifetime is + // anchored by mSrc). + template nb::object bake(const SrcGridT& src) const + { + GridHandle handle; + { + nb::gil_scoped_release release; + tools::CreateNanoGrid converter(src); + converter.setStats(mStats); + converter.setChecksum(mChecksum); + converter.setVerbose(mVerbose); + converter.enableDithering(mDither); + for (const auto& b : mBlind) { + converter.addBlindData(b.name, b.semantic, b.dataClass, b.dataType, + static_cast(b.count), + static_cast(b.size)); + } + handle = converter.getHandle(); + // The C++ converter allocates authored channels without clearing + // them (C++ callers memcpy their payload in). Zero-fill here so + // the NumPy view starts deterministic. The authored channels are + // the first mBlind.size() blind-data entries — any converter- + // added channel (e.g. a long grid name) is appended after them. + if (!mBlind.empty()) { + if (auto* dst = const_cast(handle.gridData())) { + for (size_t i = 0; i < mBlind.size(); ++i) { + const GridBlindMetaData* meta = dst->blindMetaData(uint32_t(i)); + std::memset(const_cast(meta->blindData()), 0, + meta->blindDataSize()); + } + } + } + } + return nb::cast(std::move(handle)); + } + + nb::object mSrc; + std::vector mBlind; + tools::StatsMode mStats = tools::StatsMode::Default; + CheckMode mChecksum = CheckMode::Default; + int mVerbose = 0; + bool mDither = false; +}; + } // namespace void defineCreateNanoGridConversions(nb::module_& toolsModule) @@ -370,6 +541,46 @@ void defineCreateNanoGridConversions(nb::module_& toolsModule) "Convert a source grid into a NanoGrid. Only the " "active voxels get a sequential index — the canonical input to " "buildVoxelBlockManager."); + + // ------ CreateNanoGrid converter class (blind-data authoring) ------ + nb::class_(toolsModule, "CreateNanoGrid", + "Reusable converter mirroring nanovdb::tools::CreateNanoGrid. " + "Construct from a NanoGrid or nanovdb.tools.build.* grid (float, " + "double, int32, or Vec3f), optionally declare blind-data channels " + "with addBlindData(), then bake a fresh NanoGrid of the same BuildT " + "with getHandle(). Authored channels come back zero-filled — write " + "their contents through the writable NumPy view returned by " + "grid.getBlindData(n). For quantized (Fp*) or index destinations " + "use the createNanoGridFp* / createNanoGridIndex functions instead.") + .def(nb::init(), "srcGrid"_a, + "Construct a converter reading from srcGrid (a NanoGrid or " + "nanovdb.tools.build.* grid of BuildT float, double, int32, or " + "Vec3f). The converter keeps srcGrid alive.") + .def("addBlindData", &PyCreateNanoGrid::addBlindData, + "name"_a, "count"_a, "dataType"_a = GridType::Float, + "dataSemantic"_a = GridBlindDataSemantic::Unknown, + "dataClass"_a = GridBlindDataClass::AttributeArray, + "size"_a = 0u, + "Declare a blind-data channel of count elements of dataType to " + "be allocated in the destination grid, and return its channel " + "index. size (bytes per element) is derived from dataType when " + "omitted; pass it explicitly for dataTypes without a fixed " + "element size. The C++ signature orders the parameters (name, " + "dataSemantic, dataClass, dataType, count, size) — reordered " + "here so the common case reads addBlindData(name, count).") + .def("setStats", &PyCreateNanoGrid::setStats, "mode"_a, + "Set the StatsMode used when baking the destination grid.") + .def("setChecksum", &PyCreateNanoGrid::setChecksum, "mode"_a, + "Set the CheckMode used when baking the destination grid.") + .def("setVerbose", &PyCreateNanoGrid::setVerbose, "mode"_a = 1, + "Set the verbosity level used when baking the destination grid.") + .def("enableDithering", &PyCreateNanoGrid::enableDithering, "on"_a = true, + "Toggle dithering of the destination grid (only meaningful for " + "quantized BuildTs; kept for parity with the C++ class).") + .def("getHandle", &PyCreateNanoGrid::getHandle, + "Bake and return a GridHandle owning a NanoGrid of the source's " + "BuildT, including any channels declared via addBlindData(). " + "Each call bakes a fresh grid."); } #define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \ diff --git a/nanovdb/nanovdb/python/PyCreateNanoGrid.h b/nanovdb/nanovdb/python/PyCreateNanoGrid.h index 9e5f188e4c..e7abafbd90 100644 --- a/nanovdb/nanovdb/python/PyCreateNanoGrid.h +++ b/nanovdb/nanovdb/python/PyCreateNanoGrid.h @@ -24,7 +24,10 @@ template void defineOpenToNanoVDB(nb::module_& m); /// the createNanoGridIndex / OnIndex paths accept float, double, /// int32_t, and Vec3f sources. Additional source BuildTs can be /// added by extending the explicit try-each-SrcBuildT chains in -/// createNanoGridFpX / FpNImpl / createIndexImpl. +/// createNanoGridFpX / FpNImpl / createIndexImpl. Also binds the +/// tools.CreateNanoGrid converter class, which mirrors the C++ +/// tools::CreateNanoGrid and adds blind-data authoring via +/// addBlindData() (same source BuildT set as the index paths). void defineCreateNanoGridConversions(nb::module_& toolsModule); } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/PyGridChecksum.cc b/nanovdb/nanovdb/python/PyGridChecksum.cc index b0f42744b7..113c7a3a18 100644 --- a/nanovdb/nanovdb/python/PyGridChecksum.cc +++ b/nanovdb/nanovdb/python/PyGridChecksum.cc @@ -34,7 +34,15 @@ void defineChecksum(nb::module_& m) .def(nb::self == nb::self, "rhs"_a, "Equality of two Checksum values.") .def(nb::self != nb::self, "rhs"_a, - "Inequality of two Checksum values."); + "Inequality of two Checksum values.") + .def("isEmpty", &Checksum::isEmpty, + "True iff no checksum is stored (checksumming was disabled).") + .def("isHalf", &Checksum::isHalf, + "True iff only the header portion (grid + tree + root) is checksummed.") + .def("isFull", &Checksum::isFull, + "True iff both the header portion and all nodes are checksummed.") + .def("mode", &Checksum::mode, + "CheckMode this checksum was computed with (Disable, Partial, or Full)."); } void defineUpdateChecksum(nb::module_& m) diff --git a/nanovdb/nanovdb/python/PyGridStats.cc b/nanovdb/nanovdb/python/PyGridStats.cc index 908af0b0b6..b528bac395 100644 --- a/nanovdb/nanovdb/python/PyGridStats.cc +++ b/nanovdb/nanovdb/python/PyGridStats.cc @@ -76,12 +76,11 @@ static void defineExtrema(nb::module_& m, const char* name) // ----- Stats binding (inherits Extrema) ----- template -static void defineStats(nb::module_& m, const char* name, const char* baseName) +static void defineStats(nb::module_& m, const char* name) { using ValueT = typename NanoGrid::ValueType; using BaseT = tools::Extrema; using StatsT = tools::Stats; - (void)baseName; // kept in signature for parity with extrema name lookup nb::class_(m, name, "Running min/max/mean/variance/std accumulator over a stream of " @@ -185,10 +184,10 @@ void defineGridStatsModule(nb::module_& toolsModule) // value types are all distinct so we get N pairs of new Python classes. #define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \ defineExtrema(toolsModule, #Suffix "Extrema"); \ - defineStats(toolsModule, #Suffix "Stats", #Suffix "Extrema"); + defineStats(toolsModule, #Suffix "Stats"); #define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \ defineExtrema(toolsModule, #Suffix "Extrema"); \ - defineStats(toolsModule, #Suffix "Stats", #Suffix "Extrema"); + defineStats(toolsModule, #Suffix "Stats"); #include "BuildTypes.def" // Polymorphic updateGridStats. Accepts any bound NanoGrid (via diff --git a/nanovdb/nanovdb/python/PyPrimitives.cc b/nanovdb/nanovdb/python/PyPrimitives.cc index 495df3d515..e061435af9 100644 --- a/nanovdb/nanovdb/python/PyPrimitives.cc +++ b/nanovdb/nanovdb/python/PyPrimitives.cc @@ -32,11 +32,9 @@ GridHandle createLevelSetSphere(GridType gridType, switch (gridType) { case GridType::Float: return createLevelSetSphere(radius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); case GridType::Double: return createLevelSetSphere(radius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); - default: { - std::stringstream ss; - // ss << "Cannot createLevelSetSphere for grid of type \"" << toStr(gridType); - throw std::runtime_error(ss.str() + "\""); - } + default: + throw std::runtime_error( + "createLevelSetSphere: only float and double grid types are supported"); } } @@ -58,11 +56,9 @@ GridHandle createLevelSetTorus(GridType gridType, return createLevelSetTorus(majorRadius, minorRadius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); case GridType::Double: return createLevelSetTorus(majorRadius, minorRadius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); - default: { - std::stringstream ss; - // ss << "Cannot createLevelSetTorus for grid of type \"" << toStr(gridType); - throw std::runtime_error(ss.str() + "\""); - } + default: + throw std::runtime_error( + "createLevelSetTorus: only float and double grid types are supported"); } } @@ -81,11 +77,9 @@ GridHandle createFogVolumeSphere(GridType gridType, switch (gridType) { case GridType::Float: return createFogVolumeSphere(radius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); case GridType::Double: return createFogVolumeSphere(radius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); - default: { - std::stringstream ss; - // ss << "Cannot createFogVolumeSphere for grid of type \"" << toStr(gridType); - throw std::runtime_error(ss.str() + "\""); - } + default: + throw std::runtime_error( + "createFogVolumeSphere: only float and double grid types are supported"); } } @@ -107,11 +101,9 @@ GridHandle createFogVolumeTorus(GridType gridType, return createFogVolumeTorus(majorRadius, minorRadius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); case GridType::Double: return createFogVolumeTorus(majorRadius, minorRadius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer); - default: { - std::stringstream ss; - // ss << "Cannot createFogVolumeTorus for grid of type \"" << toStr(gridType); - throw std::runtime_error(ss.str() + "\""); - } + default: + throw std::runtime_error( + "createFogVolumeTorus: only float and double grid types are supported"); } } diff --git a/nanovdb/nanovdb/python/PySampleFromVoxels.cc b/nanovdb/nanovdb/python/PySampleFromVoxels.cc index 297c5f28c2..23816e776c 100644 --- a/nanovdb/nanovdb/python/PySampleFromVoxels.cc +++ b/nanovdb/nanovdb/python/PySampleFromVoxels.cc @@ -15,19 +15,45 @@ namespace { template void defineSampleFromVoxels(nb::module_& m, const char* name) { - using CoordT = typename TreeT::CoordType; - nb::class_>(m, name, + using CoordT = typename TreeT::CoordType; + using ValueT = typename TreeT::ValueType; + using SamplerT = math::SampleFromVoxels; + auto cls = nb::class_(m, name, "Callable sampler that reconstructs a grid value at an arbitrary " "index-space position. Build via the matching create*Sampler() factory.") .def( - "__call__", [](const math::SampleFromVoxels& sampler, const CoordT& ijk) { return sampler(ijk); }, nb::is_operator(), "ijk"_a, + "__call__", [](const SamplerT& sampler, const CoordT& ijk) { return sampler(ijk); }, nb::is_operator(), "ijk"_a, "Sample the grid at integer voxel coordinate ijk.") .def( - "__call__", [](const math::SampleFromVoxels& sampler, const Vec3f& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a, + "__call__", [](const SamplerT& sampler, const Vec3f& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a, "Sample the grid at fractional index-space position xyz.") .def( - "__call__", [](const math::SampleFromVoxels& sampler, const Vec3d& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a, + "__call__", [](const SamplerT& sampler, const Vec3d& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a, "Sample the grid at fractional index-space position xyz (double)."); + // gradient() exists on the trilinear sampler only, zeroCrossing() on the + // trilinear and triquadratic samplers, and both static_assert a + // floating-point ValueT in C++ — mirror that gating here. + if constexpr (Order == 1 && util::is_floating_point::value) { + cls.def( + "gradient", [](const SamplerT& sampler, const Vec3f& xyz) { return sampler.gradient(xyz); }, "xyz"_a, + "Return the index-space gradient of the trilinear reconstruction " + "at fractional index-space position xyz. Use " + "grid.indexToWorldGrad() to move it to world space.") + .def( + "gradient", [](const SamplerT& sampler, const Vec3d& xyz) { return sampler.gradient(xyz); }, "xyz"_a, + "Return the index-space gradient at fractional index-space " + "position xyz (double)."); + } + if constexpr ((Order == 1 || Order == 2) && util::is_floating_point::value) { + cls.def( + "zeroCrossing", [](const SamplerT& sampler, const Vec3f& xyz) { return sampler.zeroCrossing(xyz); }, "xyz"_a, + "True iff the reconstruction stencil at fractional index-space " + "position xyz straddles the zero iso-surface.") + .def( + "zeroCrossing", [](const SamplerT& sampler, const Vec3d& xyz) { return sampler.zeroCrossing(xyz); }, "xyz"_a, + "True iff the reconstruction stencil at fractional index-space " + "position xyz (double) straddles the zero iso-surface."); + } } template void defineCreateSampler(nb::module_& m, const char* name) diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index 59215960aa..5730aa8551 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -2074,5 +2074,326 @@ def test_function(self): pass +class TestGridTransformAliases(unittest.TestCase): + """worldToIndex / indexToWorld and friends are aliases of the apply* + transform family, mirroring the C++ Grid convenience names.""" + + def setUp(self): + self.handle = nanovdb.tools.createLevelSetSphere( + nanovdb.GridType.Float, radius=10.0, voxelSize=0.5 + ) + self.grid = self.handle.grid() + + def test_world_index_point_aliases(self): + p = nanovdb.math.Vec3d(1.5, -2.0, 3.25) + self.assertEqual(self.grid.worldToIndex(p), self.grid.applyInverseMap(p)) + self.assertEqual(self.grid.indexToWorld(p), self.grid.applyMap(p)) + roundtrip = self.grid.indexToWorld(self.grid.worldToIndex(p)) + for i in range(3): + self.assertAlmostEqual(roundtrip[i], p[i], places=12) + + def test_direction_and_gradient_aliases(self): + d = nanovdb.math.Vec3d(0.25, 1.0, -0.5) + self.assertEqual(self.grid.worldToIndexDir(d), self.grid.applyInverseJacobian(d)) + self.assertEqual(self.grid.indexToWorldDir(d), self.grid.applyJacobian(d)) + self.assertEqual(self.grid.indexToWorldGrad(d), self.grid.applyIJT(d)) + + def test_single_precision_aliases(self): + p = nanovdb.math.Vec3f(1.5, -2.0, 3.25) + self.assertEqual(self.grid.worldToIndexF(p), self.grid.applyInverseMapF(p)) + self.assertEqual(self.grid.indexToWorldF(p), self.grid.applyMapF(p)) + self.assertEqual(self.grid.worldToIndexDirF(p), self.grid.applyInverseJacobianF(p)) + self.assertEqual(self.grid.indexToWorldDirF(p), self.grid.applyJacobianF(p)) + self.assertEqual(self.grid.indexToWorldGradF(p), self.grid.applyIJTF(p)) + + +class TestGridValuePointCount(unittest.TestCase): + """valueCount() on Index/OnIndex grids and pointCount() on PointGrid, + mirroring the SFINAE-gated C++ Grid methods.""" + + def setUp(self): + self.src = nanovdb.tools.createLevelSetSphere( + nanovdb.GridType.Float, radius=5.0, voxelSize=1.0 + ) + + def test_on_index_value_count(self): + handle = nanovdb.tools.createNanoGridOnIndex(self.src.grid()) + grid = handle.grid() + self.assertIsInstance(grid, nanovdb.OnIndexGrid) + self.assertGreaterEqual(grid.valueCount(), self.src.grid().activeVoxelCount()) + + def test_index_value_count(self): + handle = nanovdb.tools.createNanoGridIndex(self.src.grid()) + grid = handle.grid() + self.assertIsInstance(grid, nanovdb.IndexGrid) + self.assertGreaterEqual(grid.valueCount(), self.src.grid().activeVoxelCount()) + + def test_point_count_bound_on_point_grid(self): + # pointCount() lives on NanoGrid (GridType.PointIndex). The + # point primitives bake UInt32 PointData grids, so only the binding's + # presence can be verified host-side without an OpenVDB conversion. + self.assertIn("pointCount", dir(nanovdb.PointGrid)) + self.assertNotIn("valueCount", dir(nanovdb.PointGrid)) + point_data_grid = nanovdb.tools.createPointSphere( + pointsPerVoxel=2, radius=5.0, voxelSize=1.0 + ).grid() + self.assertIsInstance(point_data_grid, nanovdb.UInt32Grid) + self.assertFalse(hasattr(point_data_grid, "pointCount")) + + def test_gated_to_matching_buildts(self): + float_grid = self.src.grid() + self.assertFalse(hasattr(float_grid, "valueCount")) + self.assertFalse(hasattr(float_grid, "pointCount")) + on_index_grid = nanovdb.tools.createNanoGridOnIndex(float_grid).grid() + self.assertFalse(hasattr(on_index_grid, "pointCount")) + + +class TestSamplerGradient(unittest.TestCase): + """gradient() on the trilinear sampler and zeroCrossing() on the + trilinear + triquadratic samplers, for floating-point grids only.""" + + def setUp(self): + self.radius = 10.0 + self.voxelSize = 0.5 + self.handle = nanovdb.tools.createLevelSetSphere( + nanovdb.GridType.Float, radius=self.radius, voxelSize=self.voxelSize + ) + self.grid = self.handle.grid() + # Index-space position on the sphere surface, on the +x axis. + self.surface = self.grid.worldToIndex(nanovdb.math.Vec3d(self.radius, 0.0, 0.0)) + + def test_trilinear_gradient_points_outward(self): + sampler = nanovdb.math.createTrilinearSampler(self.grid) + surface_f = nanovdb.math.Vec3f( + self.surface[0], self.surface[1], self.surface[2] + ) + g = sampler.gradient(surface_f) + # An SDF in world units sampled on an index-space lattice changes by + # ~voxelSize per index step along the outward normal (+x here). The + # tangential components pick up the sphere's curvature across the + # stencil cell, so they are small but not zero. + self.assertAlmostEqual(g[0], self.voxelSize, places=3) + self.assertAlmostEqual(g[1], 0.0, delta=0.05) + self.assertAlmostEqual(g[2], 0.0, delta=0.05) + # The Vec3d overload agrees. + gd = sampler.gradient(nanovdb.math.Vec3d(self.surface)) + for i in range(3): + self.assertAlmostEqual(g[i], gd[i], places=5) + + def test_zero_crossing(self): + # Probe just inside the surface: an exactly-zero stencil corner is + # not a strict sign change, so the on-surface lattice point (20,0,0) + # itself does not count as a crossing. + inside = nanovdb.math.Vec3d(self.surface[0] - 0.5, 0.0, 0.0) + for make in ( + nanovdb.math.createTrilinearSampler, + nanovdb.math.createTriquadraticSampler, + ): + sampler = make(self.grid) + self.assertTrue(abs(sampler(inside)) < self.voxelSize) + self.assertTrue(sampler.zeroCrossing(inside)) + # Deep inside the narrow band there is no crossing. + self.assertFalse(sampler.zeroCrossing(nanovdb.math.Vec3d(0.0, 0.0, 0.0))) + + def test_gated_to_matching_orders_and_buildts(self): + nn = nanovdb.math.createNearestNeighborSampler(self.grid) + self.assertFalse(hasattr(nn, "gradient")) + self.assertFalse(hasattr(nn, "zeroCrossing")) + tq = nanovdb.math.createTriquadraticSampler(self.grid) + self.assertFalse(hasattr(tq, "gradient")) + tc = nanovdb.math.createTricubicSampler(self.grid) + self.assertFalse(hasattr(tc, "gradient")) + self.assertFalse(hasattr(tc, "zeroCrossing")) + bbox = nanovdb.math.CoordBBox(nanovdb.math.Coord(0), nanovdb.math.Coord(7)) + int_grid = nanovdb.tools.createInt32Grid( + 0, "ints", nanovdb.GridClass.Unknown, lambda ijk: 1, bbox + ).grid() + int_sampler = nanovdb.math.createTrilinearSampler(int_grid) + self.assertFalse(hasattr(int_sampler, "gradient")) + self.assertFalse(hasattr(int_sampler, "zeroCrossing")) + + +class TestChecksumMethods(unittest.TestCase): + def test_mode_queries(self): + handle = nanovdb.tools.createLevelSetSphere( + nanovdb.GridType.Float, radius=5.0, voxelSize=1.0 + ) + grid = handle.grid() + stored = grid.checksum() + self.assertFalse(stored.isEmpty()) + self.assertNotEqual(stored.mode(), nanovdb.CheckMode.Disable) + self.assertEqual(stored.isFull(), stored.mode() == nanovdb.CheckMode.Full) + self.assertEqual(stored.isHalf(), stored.mode() == nanovdb.CheckMode.Partial) + disabled = nanovdb.tools.evalChecksum(grid, nanovdb.CheckMode.Disable) + self.assertTrue(disabled.isEmpty()) + self.assertEqual(disabled.mode(), nanovdb.CheckMode.Disable) + + +class TestCreateNanoGridClass(unittest.TestCase): + """tools.CreateNanoGrid converter: bake with authored blind-data + channels, filled through the writable getBlindData() NumPy view.""" + + def _build_source(self): + g = nanovdb.tools.build.FloatGrid(0.0, "blind_src", nanovdb.GridClass.Unknown) + for i in range(8): + g.setValue(nanovdb.math.Coord(i, 0, 0), float(i + 1)) + return g + + def test_bake_without_blind_data(self): + src = self._build_source() + handle = nanovdb.tools.CreateNanoGrid(src).getHandle() + grid = handle.grid() + self.assertIsInstance(grid, nanovdb.FloatGrid) + acc = grid.getAccessor() + for i in range(8): + self.assertEqual(acc.getValue(nanovdb.math.Coord(i, 0, 0)), float(i + 1)) + self.assertEqual(grid.blindDataCount(), 0) + + def test_author_float_channel(self): + import numpy as np + + conv = nanovdb.tools.CreateNanoGrid(self._build_source()) + channel = conv.addBlindData("uv", count=100) + self.assertEqual(channel, 0) + handle = conv.getHandle() + grid = handle.grid() + self.assertEqual(grid.blindDataCount(), 1) + n = grid.findBlindData("uv") + self.assertEqual(n, 0) + meta = grid.blindMetaData(n) + self.assertEqual(meta.valueCount, 100) + self.assertEqual(meta.valueSize, 4) + self.assertEqual(meta.dataType, nanovdb.GridType.Float) + self.assertTrue(meta.isValid()) + view = grid.getBlindData(n) + self.assertEqual(view.shape, (100,)) + self.assertTrue(np.all(view == 0.0)) + view[:] = np.arange(100, dtype=np.float32) + again = grid.getBlindData(n) + self.assertTrue(np.array_equal(again, np.arange(100, dtype=np.float32))) + + def test_author_vec3f_channel_with_semantic(self): + conv = nanovdb.tools.CreateNanoGrid(self._build_source()) + conv.addBlindData( + "N", + count=10, + dataType=nanovdb.GridType.Vec3f, + dataSemantic=nanovdb.GridBlindDataSemantic.PointNormal, + ) + grid = conv.getHandle().grid() + n = grid.findBlindDataForSemantic(nanovdb.GridBlindDataSemantic.PointNormal) + self.assertEqual(n, 0) + self.assertEqual(grid.blindMetaData(n).valueSize, 12) + self.assertEqual(grid.getBlindData(n).shape, (10, 3)) + + def test_multiple_channels_from_nanogrid_source(self): + src = nanovdb.tools.createLevelSetSphere( + nanovdb.GridType.Float, radius=5.0, voxelSize=1.0 + ) + conv = nanovdb.tools.CreateNanoGrid(src.grid()) + self.assertEqual(conv.addBlindData("a", count=4), 0) + self.assertEqual( + conv.addBlindData("b", count=4, dataType=nanovdb.GridType.Int32), 1 + ) + grid = conv.getHandle().grid() + self.assertEqual(grid.blindDataCount(), 2) + self.assertEqual(grid.findBlindData("a"), 0) + self.assertEqual(grid.findBlindData("b"), 1) + # The baked grid still carries the source's values. + self.assertEqual(grid.activeVoxelCount(), src.grid().activeVoxelCount()) + + def test_rejects_invalid_specs(self): + conv = nanovdb.tools.CreateNanoGrid(self._build_source()) + with self.assertRaises(ValueError): + conv.addBlindData("x" * 300, count=1) + with self.assertRaises(ValueError): + conv.addBlindData( + "bad", count=1, dataClass=nanovdb.GridBlindDataClass.GridName + ) + with self.assertRaises(ValueError): + conv.addBlindData("opaque", count=1, dataType=nanovdb.GridType.Unknown) + # Unknown dataType is allowed when the element size is explicit. + conv.addBlindData( + "opaque", count=16, dataType=nanovdb.GridType.Unknown, size=1 + ) + self.assertEqual(conv.getHandle().grid().blindDataCount(), 1) + + def test_rejects_unsupported_source(self): + src = nanovdb.tools.createLevelSetSphere( + nanovdb.GridType.Float, radius=5.0, voxelSize=1.0 + ) + on_index = nanovdb.tools.createNanoGridOnIndex(src.grid()).grid() + with self.assertRaises(TypeError): + nanovdb.tools.CreateNanoGrid(on_index) + with self.assertRaises(TypeError): + nanovdb.tools.CreateNanoGrid(None) + + +class TestChannelAccessor(unittest.TestCase): + """ChannelAccessor reads an Index/OnIndex grid's blind-data channel by + Coord; createChannelAccessor dispatches on the channel's dataType.""" + + def setUp(self): + self.src = nanovdb.tools.createLevelSetSphere( + nanovdb.GridType.Float, radius=5.0, voxelSize=1.0 + ) + self.surface = nanovdb.math.Coord(5, 0, 0) + + def test_factory_reads_channel_values(self): + handle = nanovdb.tools.createNanoGridIndex(self.src.grid(), channels=1) + grid = handle.grid() + acc = nanovdb.createChannelAccessor(grid, 0) + self.assertIsInstance(acc, nanovdb.IndexFloatChannelAccessor) + self.assertTrue(bool(acc)) + self.assertEqual(acc.valueCount(), grid.valueCount()) + src_acc = self.src.grid().getAccessor() + for ijk in (self.surface, nanovdb.math.Coord(0, 5, 0), nanovdb.math.Coord(0, 0, 5)): + self.assertEqual(acc.getValue(ijk), src_acc.getValue(ijk)) + self.assertEqual(acc(ijk), src_acc(ijk)) + self.assertEqual( + acc(self.surface.x, self.surface.y, self.surface.z), + src_acc(self.surface), + ) + self.assertTrue(acc.isActive(self.surface)) + value, is_on = acc.probeValue(self.surface) + self.assertEqual(value, src_acc.getValue(self.surface)) + self.assertTrue(is_on) + self.assertGreater(acc.getIndex(self.surface), 0) + self.assertEqual( + acc.getIndex(self.surface), + acc.idx(self.surface.x, self.surface.y, self.surface.z), + ) + + def test_on_index_factory(self): + handle = nanovdb.tools.createNanoGridOnIndex(self.src.grid(), channels=1) + acc = nanovdb.createChannelAccessor(handle.grid()) + self.assertIsInstance(acc, nanovdb.OnIndexFloatChannelAccessor) + src_acc = self.src.grid().getAccessor() + self.assertEqual(acc.getValue(self.surface), src_acc.getValue(self.surface)) + + def test_direct_constructor_and_set_channel(self): + handle = nanovdb.tools.createNanoGridIndex(self.src.grid(), channels=2) + grid = handle.grid() + acc = nanovdb.IndexFloatChannelAccessor(grid, 1) + self.assertTrue(bool(acc)) + acc.setChannel(0) + self.assertTrue(bool(acc)) + with self.assertRaises(IndexError): + acc.setChannel(2) + + def test_errors(self): + handle = nanovdb.tools.createNanoGridIndex(self.src.grid(), channels=1) + grid = handle.grid() + with self.assertRaises(IndexError): + nanovdb.createChannelAccessor(grid, 1) + with self.assertRaises(TypeError): + nanovdb.IndexDoubleChannelAccessor(grid, 0) + with self.assertRaises(TypeError): + nanovdb.createChannelAccessor(self.src.grid(), 0) + bare = nanovdb.tools.createNanoGridIndex(self.src.grid(), channels=0) + with self.assertRaises(IndexError): + nanovdb.createChannelAccessor(bare.grid(), 0) + + if __name__ == "__main__": unittest.main() From 7c172d99a4bca3722cbe62036e7dbbc205382e37 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 7 Jul 2026 04:05:52 +0000 Subject: [PATCH 38/48] nanovdb python: port host C++ examples to Python, smoke-test all examples MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Nine new scripts under python/examples/, each a Python equivalent of a host-capable C++ example (or its host code path): - io_roundtrip.py: ex_write_nanovdb_grids + ex_read_nanovdb_sphere(_accessor) as a self-contained write/read round trip, plus splitGrids and zero-copy point positions via getBlindData. - make_funny_nanovdb.py: ex_make_funny_nanovdb via the functor-based createFloatGrid factory (reduced domain — one Python call per voxel). - make_typed_grids.py: ex_make_typed_grids across seven tools.build grid types with polymorphic re-read. - raytrace_level_set.py / raytrace_fog_volume.py: the host render paths, re-expressed with the bound samplers (sphere tracing + zeroCrossing + gradient shading) and accessor marching respectively; PGM output. - collide_level_set.py: the host particle-collision path using worldToIndexF, narrow-band isActive, and sampler.gradient normals. - index_grid_channels.py: host half of ex_index_grid_cuda extended with ChannelAccessor read-back and CreateNanoGrid blind-data authoring. - node_manager.py: host half of ex_nodemanager_cuda. - openvdb_interop.py: ex_openvdb_to_nanovdb(_accessor); self-skips unless built with NANOVDB_USE_OPENVDB and openvdb imports. Not ported: the pool-buffer examples (HostBuffer pool API unbound), the MagicaVoxel converter (external parser/asset), device-only examples, and a dedicated PointAccessor example (host point primitives bake UInt32 PointData grids; Point{Index,Data}Accessor require GridType.PointIndex). test/TestExamples.py smoke-runs all 14 example scripts in subprocesses; wired as ctest pytest_nanovdb_examples (~5 s) next to pytest_nanovdb, gated on NANOVDB_BUILD_PYTHON_UNITTESTS. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 11 +- nanovdb/nanovdb/python/examples/README.md | 15 +++ .../python/examples/collide_level_set.py | 85 +++++++++++++ .../python/examples/index_grid_channels.py | 86 +++++++++++++ .../nanovdb/python/examples/io_roundtrip.py | 103 +++++++++++++++ .../python/examples/make_funny_nanovdb.py | 61 +++++++++ .../python/examples/make_typed_grids.py | 72 +++++++++++ .../nanovdb/python/examples/node_manager.py | 50 ++++++++ .../python/examples/openvdb_interop.py | 53 ++++++++ .../python/examples/raytrace_fog_volume.py | 95 ++++++++++++++ .../python/examples/raytrace_level_set.py | 120 ++++++++++++++++++ nanovdb/nanovdb/python/test/TestExamples.py | 77 +++++++++++ 12 files changed, 826 insertions(+), 2 deletions(-) create mode 100644 nanovdb/nanovdb/python/examples/collide_level_set.py create mode 100644 nanovdb/nanovdb/python/examples/index_grid_channels.py create mode 100644 nanovdb/nanovdb/python/examples/io_roundtrip.py create mode 100644 nanovdb/nanovdb/python/examples/make_funny_nanovdb.py create mode 100644 nanovdb/nanovdb/python/examples/make_typed_grids.py create mode 100644 nanovdb/nanovdb/python/examples/node_manager.py create mode 100644 nanovdb/nanovdb/python/examples/openvdb_interop.py create mode 100644 nanovdb/nanovdb/python/examples/raytrace_fog_volume.py create mode 100644 nanovdb/nanovdb/python/examples/raytrace_level_set.py create mode 100644 nanovdb/nanovdb/python/test/TestExamples.py diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 9be44ce852..e7b0bab3ae 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -99,12 +99,19 @@ if(NANOVDB_BUILD_PYTHON_UNITTESTS) COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/test/TestNanoVDB.py -v WORKING_DIRECTORY "${NANOVDB_PYTHON_WORKING_DIR}") + # Smoke-run every example script (they self-skip when optional + # dependencies such as NumPy or OpenVDB are unavailable). + add_test(NAME pytest_nanovdb_examples + COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/test/TestExamples.py -v + WORKING_DIRECTORY "${NANOVDB_PYTHON_WORKING_DIR}") + set_tests_properties(pytest_nanovdb_examples PROPERTIES TIMEOUT 300) + if(WIN32) set(PYTHONPATH "$ENV{PYTHONPATH};${NANOVDB_PYTHON_WORKING_DIR}") string(REPLACE "\\;" ";" PYTHONPATH "${PYTHONPATH}") string(REPLACE ";" "\\;" PYTHONPATH "${PYTHONPATH}") - set_tests_properties(pytest_nanovdb PROPERTIES ENVIRONMENT "PYTHONPATH=${PYTHONPATH}") + set_tests_properties(pytest_nanovdb pytest_nanovdb_examples PROPERTIES ENVIRONMENT "PYTHONPATH=${PYTHONPATH}") else() - set_tests_properties(pytest_nanovdb PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${NANOVDB_PYTHON_WORKING_DIR}") + set_tests_properties(pytest_nanovdb pytest_nanovdb_examples PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${NANOVDB_PYTHON_WORKING_DIR}") endif() endif() diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index a635a359c0..a83a4e3d22 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -27,6 +27,21 @@ PYTHONPATH=. python /path/to/.py | [`bulk_leaf_numpy.py`](bulk_leaf_numpy.py) | Zero-copy `(N_leaves, 512)` NumPy view of every leaf's mValues via `grid.leaf_values()`. Includes a global-stats reduction and an in-place mutation that propagates back into the grid. Requires NumPy. | | [`quantize.py`](quantize.py) | Quantize a `NanoGrid` through `nanovdb.tools.createNanoGridFp{4,8,16,N}`. Shows fixed-width quantization with dithering and variable-width `FpN` with both `AbsDiff` and `RelDiff` oracles. | | [`validate.py`](validate.py) | `nanovdb.tools.validateGrid` / `validateGrids`, `checkGrid`, `isValid`, and the `evalChecksum` / `validateChecksum` / `updateChecksum` round-trip. | +| [`io_roundtrip.py`](io_roundtrip.py) | `nanovdb.io` write/read round-trip over five primitives: `writeGrids` with codec fallback, `readGridMetaData`, `hasGrid`, `readGrid` by name, `splitGrids`, and zero-copy point positions via `getBlindData`. Port of `ex_write_nanovdb_grids` + `ex_read_nanovdb_sphere_accessor`. | +| [`make_funny_nanovdb.py`](make_funny_nanovdb.py) | Functor-based construction: `tools.createFloatGrid(background, name, gridClass, func, bbox)` evaluates a Python callback at every voxel. Port of `ex_make_funny_nanovdb` on a reduced domain. | +| [`make_typed_grids.py`](make_typed_grids.py) | One solid sphere per value type via `tools.build.{Float,Double,Int16,Int32,Int64,UInt32,Vec3f}Grid`, written to a single file and re-read with polymorphic `handle.grid()` dispatch. Port of `ex_make_typed_grids`. | +| [`raytrace_level_set.py`](raytrace_level_set.py) | CPU sphere-traced level-set render to PGM using `worldToIndex`, the trilinear sampler, `zeroCrossing()`, and normalized `gradient()` shading. Port of `ex_raytrace_level_set` (host path). | +| [`raytrace_fog_volume.py`](raytrace_fog_volume.py) | CPU transmittance ray-march of a fog volume to PGM using a `ReadAccessor` and `Coord.Floor` — the accessor-based sampling idiom. Port of `ex_raytrace_fog_volume` (host path). | +| [`collide_level_set.py`](collide_level_set.py) | Particles colliding with a level set: `worldToIndexF`, `tree.isActive` narrow-band test, accessor distance reads, and `sampler.gradient()` collision normals. Port of `ex_collide_level_set` (host path). | +| [`index_grid_channels.py`](index_grid_channels.py) | `tools.createNanoGridOnIndex(src, channels=1)`, `grid.valueCount()`, coordinate reads through `createChannelAccessor`, and blind-data authoring with `tools.CreateNanoGrid.addBlindData` + the writable `getBlindData` view. Extends the host half of `ex_index_grid_cuda`. Requires NumPy for the authoring section. | +| [`node_manager.py`](node_manager.py) | Linearized node iteration with `createNodeManager`: per-level counts, `leaf(i)` / `lower(i)` access, node origins, masks, and stats. Port of the host half of `ex_nodemanager_cuda`. | +| [`openvdb_interop.py`](openvdb_interop.py) | `tools.openToNanoVDB` / `nanoToOpenVDB` round-trip with accessor comparison on both sides. Self-skips unless built with `NANOVDB_USE_OPENVDB` and `openvdb` is importable. Port of `ex_openvdb_to_nanovdb_accessor`. | + +Scripts that produce files write them to a fresh temporary directory +and print its path, so the source tree stays clean. The whole set is +smoke-tested by [`../test/TestExamples.py`](../test/TestExamples.py) +(ctest name `pytest_nanovdb_examples`) when the module is configured +with `NANOVDB_BUILD_PYTHON_UNITTESTS=ON`. For full API signatures and per-argument docstrings, use Python's `help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`. diff --git a/nanovdb/nanovdb/python/examples/collide_level_set.py b/nanovdb/nanovdb/python/examples/collide_level_set.py new file mode 100644 index 0000000000..3a3bbc29fe --- /dev/null +++ b/nanovdb/nanovdb/python/examples/collide_level_set.py @@ -0,0 +1,85 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Collide falling particles against a level-set surface on the CPU. + +Python port of the host path of ex_collide_level_set: particles fall +under gravity; each step transforms the candidate position to index +space with grid.worldToIndexF, tests the narrow band with +tree.isActive, reads the signed distance through a ReadAccessor, and +on penetration reflects the velocity about the surface normal. The C++ +original builds the normal from 6-tap finite differences of the SDF; +here the bound sampler.gradient() provides the same quantity in one +call (index-space units — normalized before use). + +Run with: python collide_level_set.py +""" +import random + +import nanovdb + +NUM_PARTICLES = 500 +NUM_STEPS = 40 +DT = 0.1 +GRAVITY = -9.8 + + +def main(): + handle = nanovdb.tools.createLevelSetSphere(radius=100.0, name="sphere") + grid = handle.grid() + tree = grid.tree() + acc = grid.getAccessor() + sampler = nanovdb.math.createTrilinearSampler(grid) + + # Seed particles above the north pole of the sphere, falling down. + rng = random.Random(42) + particles = [] + for _ in range(NUM_PARTICLES): + p = [rng.uniform(-30.0, 30.0), rng.uniform(115.0, 140.0), + rng.uniform(-30.0, 30.0)] + particles.append((p, [0.0, -20.0, 0.0])) + + total_collisions = 0 + for step in range(NUM_STEPS): + collisions = 0 + for p, v in particles: + v[1] += GRAVITY * DT + next_p = [p[i] + v[i] * DT for i in range(3)] + + ijk = nanovdb.math.Coord.Floor(grid.worldToIndexF( + nanovdb.math.Vec3f(next_p[0], next_p[1], next_p[2]))) + if tree.isActive(ijk): # inside the narrow band? + d = acc.getValue(ijk) + if d <= 0.0: # inside the level set? + ipos = grid.worldToIndexF( + nanovdb.math.Vec3f(next_p[0], next_p[1], next_p[2])) + n = sampler.gradient(ipos) + n.normalize() + # Project the position back to the surface and + # reflect the velocity, as in the C++ example. + for i in range(3): + next_p[i] -= d * n[i] + v_dot_n = sum(v[i] * n[i] for i in range(3)) + for i in range(3): + v[i] -= 2.0 * v_dot_n * n[i] + collisions += 1 + p[:] = next_p + total_collisions += collisions + if collisions: + print(f"step {step:2d}: {collisions} collisions") + + print(f"total collisions over {NUM_STEPS} steps: {total_collisions}") + assert total_collisions > 0 + + # No particle should end up inside the surface. + worst = 0.0 + for p, _ in particles: + ijk = nanovdb.math.Coord.Floor(grid.worldToIndexF( + nanovdb.math.Vec3f(p[0], p[1], p[2]))) + if tree.isActive(ijk): + worst = min(worst, acc.getValue(ijk)) + print(f"deepest final penetration: {worst:.3f} world units") + assert worst > -2.0 * grid.voxelSize()[0] + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/index_grid_channels.py b/nanovdb/nanovdb/python/examples/index_grid_channels.py new file mode 100644 index 0000000000..91e421d08f --- /dev/null +++ b/nanovdb/nanovdb/python/examples/index_grid_channels.py @@ -0,0 +1,86 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Index grids, value channels, and blind-data authoring. + +Part A ports the host half of ex_index_grid_cuda: a float level set is +re-encoded as a NanoGrid whose voxels store sequential +uint64 indices, with the original float values copied into a blind-data +channel — then read back by coordinate through a ChannelAccessor, +which the C++ example only does in a CUDA kernel. Part B authors a +brand-new blind-data channel on a grid with tools.CreateNanoGrid and +fills it in place through the writable getBlindData() NumPy view. + +Run with: python index_grid_channels.py +""" +import nanovdb + + +def index_grid_with_channel(): + src = nanovdb.tools.createLevelSetSphere(radius=50.0, name="sphere") + src_grid = src.grid() + + # channels=1 copies the source values into blind-data channel 0, + # indexed by the per-voxel uint64 indices the OnIndex grid stores. + handle = nanovdb.tools.createNanoGridOnIndex(src_grid, channels=1) + grid = handle.grid() + print(f"OnIndex grid: valueCount={grid.valueCount()}, " + f"source activeVoxelCount={src_grid.activeVoxelCount()}") + assert grid.valueCount() >= src_grid.activeVoxelCount() + + # createChannelAccessor inspects the channel's recorded dataType and + # returns the matching typed accessor (here: OnIndexFloat...). + channel = nanovdb.createChannelAccessor(grid, 0) + print(f"channel accessor: {type(channel).__name__}, " + f"valueCount={channel.valueCount()}") + + src_acc = src_grid.getAccessor() + for ijk in (nanovdb.math.Coord(48, 0, 0), nanovdb.math.Coord(0, 50, 0), + nanovdb.math.Coord(0, 0, 52)): + via_channel = channel.getValue(ijk) + via_source = src_acc.getValue(ijk) + print(f" {ijk}: channel={via_channel:.3f} source={via_source:.3f} " + f"(linear offset {channel.getIndex(ijk)})") + assert via_channel == via_source + + +def author_blind_data(): + try: + import numpy as np + except ImportError: + print("NumPy not found. Skipping the blind-data authoring section.") + return + src = nanovdb.tools.build.FloatGrid(0.0, "authored", + nanovdb.GridClass.Unknown) + for i in range(8): + src.setValue(nanovdb.math.Coord(i, 0, 0), float(i)) + + # Declare a channel at conversion time; it is allocated zero-filled + # in the baked grid and filled afterwards through the writable view. + conv = nanovdb.tools.CreateNanoGrid(src) + ch = conv.addBlindData("temperature", count=64, + dataType=nanovdb.GridType.Float) + handle = conv.getHandle() + grid = handle.grid() + print(f"authored grid has {grid.blindDataCount()} blind-data channel(s)") + + view = grid.getBlindData(ch) + view[:] = np.linspace(273.0, 373.0, num=64, dtype=np.float32) + + # Re-resolve the channel by name and confirm the writes persisted. + n = grid.findBlindData("temperature") + meta = grid.blindMetaData(n) + stored = grid.getBlindData(n) + print(f" {meta.name()!r}: {meta.valueCount} x {meta.dataType}, " + f"range [{stored.min():.1f}, {stored.max():.1f}]") + assert n == ch + assert stored[0] == np.float32(273.0) + + +def main(): + index_grid_with_channel() + print() + author_blind_data() + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/io_roundtrip.py b/nanovdb/nanovdb/python/examples/io_roundtrip.py new file mode 100644 index 0000000000..3bd77f633a --- /dev/null +++ b/nanovdb/nanovdb/python/examples/io_roundtrip.py @@ -0,0 +1,103 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Write a bundle of primitive grids to a .nvdb file and read it back. + +Python port of ex_write_nanovdb_grids, ex_read_nanovdb_sphere, and +ex_read_nanovdb_sphere_accessor, consolidated into one self-contained +round trip: bake five primitives, write them to a single file (with +codec fallback), inspect the file metadata without loading the grids, +re-read one grid by name and all grids at once, and split a merged +handle back apart. The point-sphere section reads the world-space +point positions through the zero-copy getBlindData() NumPy view. + +Run with: python io_roundtrip.py +""" +import os +import tempfile + +import nanovdb + + +def write_primitives(path): + handles = [ + nanovdb.tools.createLevelSetSphere(radius=50.0, name="sphere_ls"), + nanovdb.tools.createLevelSetTorus(majorRadius=50.0, minorRadius=20.0, + name="torus_ls"), + nanovdb.tools.createLevelSetBox(width=40.0, height=60.0, depth=80.0, + name="box_ls"), + nanovdb.tools.createLevelSetBBox(width=40.0, height=60.0, depth=80.0, + thickness=10.0, name="bbox_ls"), + nanovdb.tools.createPointSphere(pointsPerVoxel=2, radius=50.0, + name="sphere_points"), + ] + # BLOSC gives the best compression but is a build-time option; fall + # back to an uncompressed file when this module was built without it. + try: + nanovdb.io.writeGrids(path, handles, codec=nanovdb.io.Codec.BLOSC) + codec = "BLOSC" + except RuntimeError: + nanovdb.io.writeGrids(path, handles, codec=nanovdb.io.Codec.NONE) + codec = "NONE" + print(f"Wrote {len(handles)} grids to {path} (codec={codec})") + + +def inspect_file(path): + # readGridMetaData parses the per-grid file headers without loading + # any voxel data — the cheap way to answer "what's in this file?". + for meta in nanovdb.io.readGridMetaData(path): + print(f" {meta.gridName!r}: type={meta.gridType}, " + f"class={meta.gridClass}, voxels={meta.voxelCount}") + assert nanovdb.io.hasGrid(path, "sphere_ls") + assert not nanovdb.io.hasGrid(path, "no_such_grid") + + +def read_back(path): + # Read a single grid by name, then print the recognizable + # ex_read_nanovdb_sphere_accessor cross-section along the x-axis. + handle = nanovdb.io.readGrid(path, "sphere_ls") + acc = handle.grid().getAccessor() + for i in range(47, 54): + ijk = nanovdb.math.Coord(i, 0, 0) + print(f" sphere_ls({i},0,0) = {acc.getValue(ijk):.2f}") + + # Read every grid, merge them into one multi-grid handle, and split + # that handle back into one handle per grid. + handles = nanovdb.io.readGrids(path) + merged = nanovdb.mergeGrids(handles) + print(f" merged handle holds {merged.gridCount()} grids") + parts = nanovdb.splitGrids(merged) + assert len(parts) == len(handles) + print(f" splitGrids -> {len(parts)} single-grid handles") + return handles + + +def point_positions(handles): + try: + import numpy as np + except ImportError: + print("NumPy not found. Skipping the point-positions section.") + return + # The point sphere stores its world-space positions as a blind-data + # channel; getBlindData() exposes it as a zero-copy (N, 3) view. + points = next(h.grid() for h in handles + if h.grid().gridName() == "sphere_points") + positions = points.getBlindData(0) + radii = np.linalg.norm(positions, axis=1) + print(f" {positions.shape[0]} points, " + f"|p| in [{radii.min():.2f}, {radii.max():.2f}]") + # Points are jittered within their voxel, so allow ~1.5 voxels. + assert np.all(np.abs(radii - 50.0) < 1.5) + + +def main(): + out_dir = tempfile.mkdtemp(prefix="nanovdb_") + path = os.path.join(out_dir, "primitives.nvdb") + write_primitives(path) + inspect_file(path) + handles = read_back(path) + point_positions(handles) + print(f"Output left in {out_dir}") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/make_funny_nanovdb.py b/nanovdb/nanovdb/python/examples/make_funny_nanovdb.py new file mode 100644 index 0000000000..fd9068586e --- /dev/null +++ b/nanovdb/nanovdb/python/examples/make_funny_nanovdb.py @@ -0,0 +1,61 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Build a level set from a Python function evaluated at every voxel. + +Python port of ex_make_funny_nanovdb: a trigonometric interference +pattern is CSG-intersected with a sphere and clamped to a narrow band, +using the functor-based nanovdb.tools.createFloatGrid factory. The +callback is one Python call per voxel in the bbox, so this port uses a +65^3 domain (~275k calls, a few seconds) where the C++ original fills +[-500,500]^3 — scale `SIZE` up only if you are willing to wait. + +Run with: python make_funny_nanovdb.py +""" +import math +import os +import tempfile + +import nanovdb + +SIZE = 32 # half-width of the cubic domain, in voxels +BACKGROUND = 5.0 # narrow-band half-width, in world units +FREQ = 0.8 # rescaled from the C++ 0.1 to fit the smaller domain + + +def funny(ijk): + v = 4.0 + 5.0 * (math.cos(ijk.x * FREQ) * math.sin(ijk.y * FREQ) + + math.cos(ijk.y * FREQ) * math.sin(ijk.z * FREQ) + + math.cos(ijk.z * FREQ) * math.sin(ijk.x * FREQ)) + # CSG intersection with a sphere of radius SIZE. + r = math.sqrt(ijk.x ** 2 + ijk.y ** 2 + ijk.z ** 2) + v = max(v, r - SIZE) + # Clamp to the narrow band. + return max(-BACKGROUND, min(BACKGROUND, v)) + + +def main(): + bbox = nanovdb.math.CoordBBox(nanovdb.math.Coord(-SIZE), + nanovdb.math.Coord(SIZE)) + print(f"Evaluating funny() over {bbox} ...") + handle = nanovdb.tools.createFloatGrid( + BACKGROUND, "funny", nanovdb.GridClass.LevelSet, funny, bbox) + + grid = handle.grid() + print(f"activeVoxelCount = {grid.activeVoxelCount()}") + acc = grid.getAccessor() + probe = nanovdb.math.Coord(0, 0, 0) + print(f"value at {probe} = {acc.getValue(probe):.3f}") + assert grid.isLevelSet() + assert grid.activeVoxelCount() > 0 + + out_dir = tempfile.mkdtemp(prefix="nanovdb_") + path = os.path.join(out_dir, "funny.nvdb") + try: + nanovdb.io.writeGrid(path, handle, codec=nanovdb.io.Codec.BLOSC) + except RuntimeError: + nanovdb.io.writeGrid(path, handle, codec=nanovdb.io.Codec.NONE) + print(f"Wrote {path}") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/make_typed_grids.py b/nanovdb/nanovdb/python/examples/make_typed_grids.py new file mode 100644 index 0000000000..6454d53a0d --- /dev/null +++ b/nanovdb/nanovdb/python/examples/make_typed_grids.py @@ -0,0 +1,72 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Build grids of many value types and read them back polymorphically. + +Python port of ex_make_typed_grids: a small solid sphere is baked once +per value type through the matching nanovdb.tools.build.Grid +mutable builder, all handles are written to a single .nvdb file, and +the file is re-read with handle.grid() returning the correct typed +subclass for each grid at runtime. + +Run with: python make_typed_grids.py +""" +import os +import tempfile + +import nanovdb + +RADIUS = 8 # voxels; ~2.1k active voxels per grid keeps this quick + + +def solid_sphere_coords(): + r2 = RADIUS * RADIUS + for i in range(-RADIUS, RADIUS + 1): + for j in range(-RADIUS, RADIUS + 1): + for k in range(-RADIUS, RADIUS + 1): + if i * i + j * j + k * k < r2: + yield nanovdb.math.Coord(i, j, k) + + +def build_typed_grids(): + # (builder class, grid name, background, voxel value) + specs = [ + (nanovdb.tools.build.FloatGrid, "float_grid", 0.0, 1.0), + (nanovdb.tools.build.DoubleGrid, "double_grid", 0.0, 1.0), + (nanovdb.tools.build.Int16Grid, "int16_grid", 0, 1), + (nanovdb.tools.build.Int32Grid, "int32_grid", 0, 1), + (nanovdb.tools.build.Int64Grid, "int64_grid", 0, 1), + (nanovdb.tools.build.UInt32Grid, "uint32_grid", 0, 1), + (nanovdb.tools.build.Vec3fGrid, "vec3f_grid", + nanovdb.math.Vec3f(0.0), nanovdb.math.Vec3f(1.0, 0.0, 0.0)), + ] + handles = [] + for cls, name, background, value in specs: + grid = cls(background, name, nanovdb.GridClass.Unknown) + for ijk in solid_sphere_coords(): + grid.setValue(ijk, value) + handles.append(grid.to_nanovdb()) + print(f"built {name} ({cls.__name__})") + return handles + + +def main(): + handles = build_typed_grids() + + out_dir = tempfile.mkdtemp(prefix="nanovdb_") + path = os.path.join(out_dir, "custom_types.nvdb") + nanovdb.io.writeGrids(path, handles) + print(f"Wrote {len(handles)} grids to {path}") + + # Re-read and dispatch: handle.grid() returns FloatGrid, Int16Grid, + # Vec3fGrid, ... according to the GridType each grid carries. + probe = nanovdb.math.Coord(0, 0, 0) + for handle in nanovdb.io.readGrids(path): + grid = handle.grid() + value = grid.getAccessor().getValue(probe) + print(f" {grid.gridName():<12} -> {type(grid).__name__:<12} " + f"active={grid.activeVoxelCount()} value(0,0,0)={value}") + assert grid.activeVoxelCount() > 0 + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/node_manager.py b/nanovdb/nanovdb/python/examples/node_manager.py new file mode 100644 index 0000000000..3ca3feffbe --- /dev/null +++ b/nanovdb/nanovdb/python/examples/node_manager.py @@ -0,0 +1,50 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Iterate a grid's nodes linearly through a NodeManager. + +Python port of the host half of ex_nodemanager_cuda: createNodeManager +builds linearized arrays of the tree's leaf, lower, and upper nodes so +they can be visited by index instead of by tree traversal. Each node +exposes its origin, per-node stats, and (on leaves) the raw 512-value +buffer. For bulk NumPy analytics over every leaf at once, see +bulk_leaf_numpy.py's grid.leaf_values() instead. + +Run with: python node_manager.py +""" +import nanovdb + + +def main(): + handle = nanovdb.tools.createLevelSetSphere(radius=50.0, name="sphere") + grid = handle.grid() + tree = grid.tree() + + nmh = nanovdb.createNodeManager(grid) + nm = nmh.mgr() + print(f"NodeManager over {grid.gridName()!r} (linear={nm.isLinear()}):") + print(f" leaves={nm.leafCount()}, lower={nm.lowerCount()}, " + f"upper={nm.upperCount()}") + + # The counts mirror the tree's per-level node counts. + assert nm.leafCount() == tree.nodeCount(0) + assert nm.lowerCount() == tree.nodeCount(1) + assert nm.upperCount() == tree.nodeCount(2) + + # Linear access agrees with tree traversal. + assert nm.leaf(0).origin() == tree.getFirstLeaf().origin() + + # Visit a few leaves by index: origin, activity, and value range. + for i in range(min(5, nm.leafCount())): + leaf = nm.leaf(i) + print(f" leaf[{i}] origin={leaf.origin()} " + f"on={leaf.valueMask().countOn()} " + f"min={leaf.minimum():.3f} max={leaf.maximum():.3f}") + + # Internal nodes are reachable the same way. + lower = nm.lower(0) + print(f" lower[0] origin={lower.origin()} " + f"children={lower.childMask().countOn()}") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/openvdb_interop.py b/nanovdb/nanovdb/python/examples/openvdb_interop.py new file mode 100644 index 0000000000..2871026e1a --- /dev/null +++ b/nanovdb/nanovdb/python/examples/openvdb_interop.py @@ -0,0 +1,53 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Round-trip a grid between OpenVDB and NanoVDB. + +Python port of ex_openvdb_to_nanovdb and ex_openvdb_to_nanovdb_accessor: +an OpenVDB level-set sphere is converted to NanoVDB with openToNanoVDB, +values are compared through both libraries' accessors, and the NanoVDB +grid is converted back with nanoToOpenVDB. Requires a build with +NANOVDB_USE_OPENVDB and an importable `openvdb` module; the script +skips (exit 0) when either is missing. + +Run with: python openvdb_interop.py +""" +import nanovdb + + +def main(): + if not hasattr(nanovdb.tools, "openToNanoVDB"): + print("nanovdb was built without NANOVDB_USE_OPENVDB. Skipping...") + return + try: + import openvdb + except ImportError: + print("openvdb not found. Skipping...") + return + + sphere = openvdb.createLevelSetSphere(100.0) + sphere.name = "sphere" + handle = nanovdb.tools.openToNanoVDB(sphere) + grid = handle.grid() + print(f"openToNanoVDB: {grid.gridName()!r}, class={grid.gridClass()}, " + f"active={grid.activeVoxelCount()}") + assert grid.gridClass() == nanovdb.GridClass.LevelSet + + # Compare a cross-section through both accessors, as the C++ + # accessor example does. + open_acc = sphere.getAccessor() + nano_acc = grid.getAccessor() + for i in range(97, 104): + open_v = open_acc.getValue((i, 0, 0)) + nano_v = nano_acc.getValue(nanovdb.math.Coord(i, 0, 0)) + print(f" ({i},0,0): openvdb={open_v:.3f} nanovdb={nano_v:.3f}") + assert abs(open_v - nano_v) < 1e-5 + + # And back: NanoVDB -> OpenVDB. + back = nanovdb.tools.nanoToOpenVDB(handle) + print(f"nanoToOpenVDB: {back.name!r}, empty={back.empty()}") + assert back.name == "sphere" + assert not back.empty() + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/raytrace_fog_volume.py b/nanovdb/nanovdb/python/examples/raytrace_fog_volume.py new file mode 100644 index 0000000000..ddb92be4a3 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/raytrace_fog_volume.py @@ -0,0 +1,95 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Ray-march a fog volume on the CPU and write a PGM image. + +Python port of the host path of ex_raytrace_fog_volume: each ray is +clipped to the grid's index bounding box, then integrated with fixed +steps, accumulating transmittance from the density sampled through a +ReadAccessor at Coord.Floor of the march position — the accessor-based +idiom, in contrast to the sampler-based raytrace_level_set.py. Every +step is one Python-to-C++ call, so the default RES is modest; raise it +for a nicer image. + +Run with: python raytrace_fog_volume.py +""" +import math +import os +import tempfile + +import nanovdb + +RES = 64 # image is RES x RES pixels +FOV_DEG = 45.0 +DT = 1.0 # march step, in voxels +SIGMA = 0.2 # extinction scale applied to the sampled density + + +def clip_to_bbox(eye, direction, bbox): + """Slab-clip a ray against a CoordBBox; returns (t0, t1) or None.""" + t0, t1 = 0.0, math.inf + for axis in range(3): + lo = bbox.min[axis] - eye[axis] + hi = bbox.max[axis] + 1.0 - eye[axis] + d = direction[axis] + if abs(d) < 1e-12: + if lo > 0.0 or hi < 0.0: + return None + continue + near, far = lo / d, hi / d + if near > far: + near, far = far, near + t0, t1 = max(t0, near), min(t1, far) + if t0 > t1: + return None + return t0, t1 + + +def main(): + handle = nanovdb.tools.createFogVolumeSphere(radius=50.0, name="fog") + grid = handle.grid() + bbox = grid.indexBBox() + acc = grid.getAccessor() + + # Perspective camera looking down -z, as in the C++ RayGenOp, + # working directly in index space (the grid transform is uniform). + dim = [bbox.max[i] + 1 - bbox.min[i] for i in range(3)] + center = [bbox.min[i] + 0.5 * dim[i] for i in range(3)] + eye = (center[0], center[1], center[2] + 2.0 * dim[2]) + tan_fov = math.tan(math.radians(FOV_DEG) * 0.5) + + pixels = bytearray(RES * RES) + for y in range(RES): + for x in range(RES): + px = (2.0 * (x + 0.5) / RES - 1.0) * tan_fov + py = (2.0 * (y + 0.5) / RES - 1.0) * tan_fov + norm = math.sqrt(px * px + py * py + 1.0) + direction = (px / norm, py / norm, -1.0 / norm) + + span = clip_to_bbox(eye, direction, bbox) + if span is None: + continue + transmittance = 1.0 + t = span[0] + while t < span[1]: + pos = nanovdb.math.Vec3d(eye[0] + t * direction[0], + eye[1] + t * direction[1], + eye[2] + t * direction[2]) + density = acc.getValue(nanovdb.math.Coord.Floor(pos)) + transmittance *= 1.0 - density * SIGMA * DT + if transmittance < 0.005: + break + t += DT + pixels[y * RES + x] = int(255 * (1.0 - transmittance)) + + out_dir = tempfile.mkdtemp(prefix="nanovdb_") + path = os.path.join(out_dir, "raytrace_fog_volume.pgm") + with open(path, "wb") as f: + f.write(f"P5\n{RES} {RES}\n255\n".encode("ascii")) + f.write(bytes(pixels)) + lit = sum(1 for p in pixels if p > 0) + print(f"Rendered {RES}x{RES} image, {lit} foggy pixels -> {path}") + assert lit > 0 + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/raytrace_level_set.py b/nanovdb/nanovdb/python/examples/raytrace_level_set.py new file mode 100644 index 0000000000..415787d031 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/raytrace_level_set.py @@ -0,0 +1,120 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Ray-trace a narrow-band level set on the CPU and write a PGM image. + +Python port of the host path of ex_raytrace_level_set. The C++ example +finds surface hits with math::Ray + HDDA ZeroCrossing, which are not +bound in Python; this port re-expresses the search as sphere tracing — +the clamped SDF value itself bounds the safe step size — using the +trilinear sampler, with sampler.zeroCrossing() confirming the interval +and sampler.gradient() shading the hit. Every sample is one +Python-to-C++ call, so the default RES is modest; raise it for a nicer +image. + +Run with: python raytrace_level_set.py +""" +import math +import os +import tempfile + +import nanovdb + +RES = 64 # image is RES x RES pixels +FOV_DEG = 45.0 +LIGHT = (0.577, 0.577, 0.577) # unit vector toward the light + + +def clip_to_bbox(eye, direction, bbox): + """Slab-clip a ray against a CoordBBox; returns (t0, t1) or None.""" + t0, t1 = 0.0, math.inf + for axis in range(3): + lo = bbox.min[axis] - eye[axis] + hi = bbox.max[axis] + 1.0 - eye[axis] + d = direction[axis] + if abs(d) < 1e-12: + if lo > 0.0 or hi < 0.0: + return None + continue + near, far = lo / d, hi / d + if near > far: + near, far = far, near + t0, t1 = max(t0, near), min(t1, far) + if t0 > t1: + return None + return t0, t1 + + +def trace(sampler, eye, direction, t0, t1, voxel_size): + """Sphere-trace from t0 to t1; returns (hit t, crossing seen) or None.""" + t = t0 + while t < t1: + pos = nanovdb.math.Vec3d(eye[0] + t * direction[0], + eye[1] + t * direction[1], + eye[2] + t * direction[2]) + d = sampler(pos) # world-unit SDF, clamped to the narrow band + if d <= 0.0: + # zeroCrossing() reports whether the reconstruction stencil + # here straddles the iso-surface — the sampler-level analog + # of the HDDA ZeroCrossing test in the C++ example. + return t, sampler.zeroCrossing(pos) + # The clamped SDF bounds the distance to the surface, so it is + # a safe (index-space) step size; never step below half a voxel. + t += max(d / voxel_size, 0.5) + return None + + +def main(): + handle = nanovdb.tools.createLevelSetSphere(radius=100.0, name="sphere") + grid = handle.grid() + bbox = grid.indexBBox() + voxel_size = grid.voxelSize()[0] + sampler = nanovdb.math.createTrilinearSampler(grid) + + # Perspective camera looking down -z, as in the C++ RayGenOp, + # working directly in index space (the grid transform is uniform). + dim = [bbox.max[i] + 1 - bbox.min[i] for i in range(3)] + center = [bbox.min[i] + 0.5 * dim[i] for i in range(3)] + eye = nanovdb.math.Vec3d(center[0], center[1], center[2] + 2.0 * dim[2]) + tan_fov = math.tan(math.radians(FOV_DEG) * 0.5) + + pixels = bytearray(RES * RES) + crossings = 0 + for y in range(RES): + for x in range(RES): + px = (2.0 * (x + 0.5) / RES - 1.0) * tan_fov + py = (2.0 * (y + 0.5) / RES - 1.0) * tan_fov + norm = math.sqrt(px * px + py * py + 1.0) + direction = (px / norm, py / norm, -1.0 / norm) + + span = clip_to_bbox(eye, direction, bbox) + if span is None: + continue + hit = trace(sampler, eye, direction, span[0], span[1], + voxel_size) + if hit is None: + continue + t_hit, crossed = hit + crossings += crossed + pos = nanovdb.math.Vec3d(eye[0] + t_hit * direction[0], + eye[1] + t_hit * direction[1], + eye[2] + t_hit * direction[2]) + # gradient() is in index-space units — normalize before use. + n = sampler.gradient(pos) + n.normalize() + shade = max(0.0, n[0] * LIGHT[0] + n[1] * LIGHT[1] + + n[2] * LIGHT[2]) + pixels[y * RES + x] = int(255 * shade) + + out_dir = tempfile.mkdtemp(prefix="nanovdb_") + path = os.path.join(out_dir, "raytrace_level_set.pgm") + with open(path, "wb") as f: + f.write(f"P5\n{RES} {RES}\n255\n".encode("ascii")) + f.write(bytes(pixels)) + lit = sum(1 for p in pixels if p > 0) + print(f"Rendered {RES}x{RES} image, {lit} lit pixels " + f"({crossings} confirmed zero-crossings) -> {path}") + assert lit > 0 and crossings > 0 + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/test/TestExamples.py b/nanovdb/nanovdb/python/test/TestExamples.py new file mode 100644 index 0000000000..3d6355b9fc --- /dev/null +++ b/nanovdb/nanovdb/python/test/TestExamples.py @@ -0,0 +1,77 @@ +#!/usr/bin/env python +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Smoke test: run every example script and require a clean exit. + +The examples are self-contained and degrade gracefully (exit 0 with a +skip message) when an optional dependency such as NumPy or OpenVDB is +missing, so a non-zero exit or an uncaught exception always indicates +a real breakage — typically the bindings drifting under the examples. + +The script list is deliberately explicit rather than a glob so that a +missing listed example fails loudly and unrelated files placed under +examples/ are not executed. +""" + +import os +import subprocess +import sys +import tempfile +import unittest + +EXAMPLES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), + os.pardir, "examples") + +EXAMPLE_SCRIPTS = [ + "build_grid.py", + "bulk_leaf_numpy.py", + "collide_level_set.py", + "index_grid_channels.py", + "io_roundtrip.py", + "load_inspect.py", + "make_funny_nanovdb.py", + "make_typed_grids.py", + "node_manager.py", + "openvdb_interop.py", + "quantize.py", + "raytrace_fog_volume.py", + "raytrace_level_set.py", + "validate.py", +] + + +class TestExamples(unittest.TestCase): + pass + + +def _make_test(script_name): + def test(self): + script = os.path.join(EXAMPLES_DIR, script_name) + self.assertTrue(os.path.isfile(script), + f"example script is missing: {script}") + # A fresh cwd per run keeps any output files out of the source + # tree (the examples themselves also write to tempdirs). + result = subprocess.run( + [sys.executable, script], + cwd=tempfile.mkdtemp(prefix="nanovdb_example_"), + env=os.environ.copy(), + capture_output=True, + text=True, + timeout=120, + ) + self.assertEqual( + result.returncode, 0, + f"{script_name} exited with {result.returncode}\n" + f"--- stdout ---\n{result.stdout}\n" + f"--- stderr ---\n{result.stderr}") + + return test + + +for _name in EXAMPLE_SCRIPTS: + _test_name = "test_" + _name.replace(".py", "") + setattr(TestExamples, _test_name, _make_test(_name)) + + +if __name__ == "__main__": + unittest.main() From 4d63d3bfc80cdb8c6a2c439c720be4a2940c9af4 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Tue, 7 Jul 2026 05:24:14 +0000 Subject: [PATCH 39/48] nanovdb python: GPU examples, MeshToGrid binding, device-tool fixes Add device/GPU Python examples exercising the nanovdb.cuda / nanovdb.tools.cuda surface, bind tools::cuda::MeshToGrid, and fix two device-tool issues surfaced while writing the examples. New GPU examples (nanovdb/nanovdb/python/examples/): - voxels_to_grid_cuda, device_topology_ops, sample_from_voxels_cuda, index_to_grid_cuda, signed_flood_fill_cuda, validate_cuda, collide_level_set_cuda: device grid construction, morphology/topology ops, trilinear sampling + gradients, index->value baking, signed flood fill, device QC, and vectorized particle collision. - raytrace_level_set_cuda, raytrace_fog_volume_cuda: cupy.RawKernel renderers using the in-kernel HDDA / accessor C++ API. - mesh_to_grid_cuda: triangle mesh -> narrow-band UDF via meshToGrid. Bindings: - Bind nanovdb::tools::cuda::MeshToGrid as tools.cuda.meshToGrid, returning a device OnIndex handle plus a per-value UDF sidecar buffer. - Bind device isValid for Point grids. Fixes: - IndexToGrid.cuh: the destination value grid inherited the source index grid's GridClass (Index), forming an invalid GridType/GridClass combination that failed grid validation; set GridClass::Unknown on the output value grid. Document the new GPU examples and meshToGrid in the examples README. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 1 + nanovdb/nanovdb/python/PyTools.cc | 6 + .../python/cuda/PyDeviceGridValidator.cu | 1 + nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu | 73 +++++++++ nanovdb/nanovdb/python/cuda/PyMeshToGrid.h | 19 +++ nanovdb/nanovdb/python/examples/README.md | 13 +- .../python/examples/collide_level_set_cuda.py | 109 +++++++++++++ .../python/examples/device_topology_ops.py | 93 +++++++++++ .../python/examples/index_to_grid_cuda.py | 82 ++++++++++ .../python/examples/mesh_to_grid_cuda.py | 94 +++++++++++ .../examples/raytrace_fog_volume_cuda.py | 139 ++++++++++++++++ .../examples/raytrace_level_set_cuda.py | 151 ++++++++++++++++++ .../examples/sample_from_voxels_cuda.py | 89 +++++++++++ .../python/examples/signed_flood_fill_cuda.py | 74 +++++++++ .../nanovdb/python/examples/validate_cuda.py | 73 +++++++++ .../python/examples/voxels_to_grid_cuda.py | 90 +++++++++++ nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh | 4 + 17 files changed, 1110 insertions(+), 1 deletion(-) create mode 100644 nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu create mode 100644 nanovdb/nanovdb/python/cuda/PyMeshToGrid.h create mode 100644 nanovdb/nanovdb/python/examples/collide_level_set_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/device_topology_ops.py create mode 100644 nanovdb/nanovdb/python/examples/index_to_grid_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/mesh_to_grid_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/raytrace_fog_volume_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/raytrace_level_set_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/sample_from_voxels_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/signed_flood_fill_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/validate_cuda.py create mode 100644 nanovdb/nanovdb/python/examples/voxels_to_grid_cuda.py diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index 1393770ef2..bf9f382463 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -51,6 +51,7 @@ nanobind_add_module(nanovdb_python NB_STATIC cuda/PyMergeGrids.cu cuda/PyInjectData.cu cuda/PyIndexToGrid.cu + cuda/PyMeshToGrid.cu cuda/PyAddBlindData.cu cuda/PyDeviceGridStats.cu cuda/PyDeviceGridValidator.cu diff --git a/nanovdb/nanovdb/python/PyTools.cc b/nanovdb/nanovdb/python/PyTools.cc index 5a9bbf58ea..fbc0e5fd99 100644 --- a/nanovdb/nanovdb/python/PyTools.cc +++ b/nanovdb/nanovdb/python/PyTools.cc @@ -27,6 +27,7 @@ #include "cuda/PyMergeGrids.h" #include "cuda/PyInjectData.h" #include "cuda/PyIndexToGrid.h" +#include "cuda/PyMeshToGrid.h" #include "cuda/PyAddBlindData.h" #include "cuda/PyDeviceGridStats.h" #include "cuda/PyDeviceGridValidator.h" @@ -88,6 +89,10 @@ void defineToolsModule(nb::module_& m) definePointsToGrid(cudaModule, "pointsToGrid"); definePointsToGrid(cudaModule, "pointsToGrid"); + // Triangle mesh -> narrow-band UDF (nanovdb::tools::cuda::MeshToGrid). + // Returns (ValueOnIndex device handle, per-value float UDF sidecar buffer). + defineMeshToGrid(cudaModule, "meshToGrid"); + // Multi-GPU voxel-coordinate -> grid builder (nanovdb::tools::cuda:: // DistributedPointsToGrid). Distributes an (N, 3) int32 unified-memory // array of index-space voxel coordinates over a DeviceMesh. Bound for the @@ -203,6 +208,7 @@ void defineToolsModule(nb::module_& m) defineDeviceIsValid(cudaModule, "isValid"); defineDeviceIsValid(cudaModule, "isValid"); defineDeviceIsValid(cudaModule, "isValid"); + defineDeviceIsValid(cudaModule, "isValid"); defineDeviceGridChecksum(cudaModule); defineDeviceGridChecksum(cudaModule); diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu index 84524fd50a..4ef9cac959 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu @@ -68,5 +68,6 @@ template void defineDeviceIsValid(nb::module_&, const char*); template void defineDeviceIsValid(nb::module_&, const char*); template void defineDeviceIsValid(nb::module_&, const char*); template void defineDeviceIsValid(nb::module_&, const char*); +template void defineDeviceIsValid(nb::module_&, const char*); } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu b/nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu new file mode 100644 index 0000000000..6c673e8b8d --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu @@ -0,0 +1,73 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#include "PyMeshToGrid.h" + +#include +#include +#include + +#include +#include + +#include + +namespace nb = nanobind; +using namespace nb::literals; +// NOTE: deliberately NOT `using namespace nanovdb;`. These tools instantiate +// CUB DeviceScan, whose nvcc-generated host stub references unqualified +// `cuda::std::...`; with `nanovdb::cuda` in scope that becomes ambiguous and +// fails to compile. Fully qualify nanovdb:: instead (matches PyPointsToGrid.cu). + +namespace pynanovdb { + +void defineMeshToGrid(nb::module_& m, const char* name) +{ + m.def( + name, + [](nb::ndarray, nb::c_contig, nb::device::cuda> points, + nb::ndarray, nb::c_contig, nb::device::cuda> triangles, + double voxelSize, float halfWidth, const std::string& gridName, + uintptr_t stream) { + cudaStream_t s = reinterpret_cast(stream); + // Vec3f / Vec3i are three contiguous scalars, so the c_contig + // (N, 3) tensors reinterpret element-for-element. + const nanovdb::Vec3f* d_points = + reinterpret_cast(points.data()); + const nanovdb::Vec3i* d_triangles = + reinterpret_cast(triangles.data()); + const uint32_t pointCount = static_cast(points.shape(0)); + const uint32_t triangleCount = static_cast(triangles.shape(0)); + nanovdb::Map map; + map.set(voxelSize, nanovdb::Vec3d(0.0, 0.0, 0.0), 1.0); + // MeshToGrid launches kernels and synchronizes the stream; pure + // CUDA touching no Python objects, so release the GIL. + nb::gil_scoped_release release; + nanovdb::tools::cuda::MeshToGrid converter( + d_points, pointCount, d_triangles, triangleCount, map, s); + converter.setNarrowBandWidth(halfWidth); + if (!gridName.empty()) + converter.setGridName(gridName); + // Compute a checksum during the build so the result validates. + converter.setChecksum(nanovdb::CheckMode::Full); + return converter.getHandleAndUDF(); + }, + "points"_a, + "triangles"_a, + "voxelSize"_a = 1.0, + "halfWidth"_a = 3.0, + "gridName"_a = "", + "stream"_a = 0, + "Rasterize a triangle mesh into a narrow-band unsigned distance field on " + "the device. points is an (N, 3) float32 CUDA array of vertex world " + "positions; triangles is an (M, 3) int32 CUDA array of vertex indices. " + "Returns a tuple (handle, udf): handle is a device ValueOnIndex " + "GridHandle holding the narrow-band topology, and udf is a " + "nanovdb.cuda.DeviceBuffer of (valueCount) float32 unsigned distances " + "(in voxel units) indexed by the grid's per-voxel value index -- feed it " + "straight to indexToGrid to bake a Float distance grid. voxelSize is the " + "world size of a voxel; halfWidth is the narrow-band half-width in voxel " + "units; stream is a raw CUDA stream handle (Python int; 0 = default " + "stream)."); +} + +} // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyMeshToGrid.h b/nanovdb/nanovdb/python/cuda/PyMeshToGrid.h new file mode 100644 index 0000000000..a3d4f11775 --- /dev/null +++ b/nanovdb/nanovdb/python/cuda/PyMeshToGrid.h @@ -0,0 +1,19 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_CUDA_PYMESHTOGRID_HAS_BEEN_INCLUDED +#define NANOVDB_CUDA_PYMESHTOGRID_HAS_BEEN_INCLUDED + +#include + +namespace nb = nanobind; + +namespace pynanovdb { + +// Bind nanovdb::tools::cuda::MeshToGrid: rasterize a triangle mesh (device +// vertex + triangle-index arrays) into a device ValueOnIndex GridHandle plus a +// per-value unsigned-distance-field sidecar buffer. Registered under `name`. +void defineMeshToGrid(nb::module_& m, const char* name); + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/examples/README.md b/nanovdb/nanovdb/python/examples/README.md index 9a79b42200..703cba2de0 100644 --- a/nanovdb/nanovdb/python/examples/README.md +++ b/nanovdb/nanovdb/python/examples/README.md @@ -55,6 +55,16 @@ GPU-array framework it uses is unavailable. | [`cupy_rawkernel.py`](cupy_rawkernel.py) | A `cupy.RawKernel` that `#include ` (compiled with `nanovdb.cuda.compile_options()`) and reads a `const nanovdb::NanoGrid*` straight from `grid.data_ptr()`. Documents the device-pointer ABI. Requires CuPy. | | [`numba_cuda.py`](numba_cuda.py) | Adopting a NanoVDB device buffer as a zero-copy `numba.cuda` array (Numba can't parse the C++ ABI, so it operates on the raw buffer). Requires Numba. | | [`triton_kernel.py`](triton_kernel.py) | Handing a NanoVDB device buffer to a Triton kernel via a zero-copy `torch.from_dlpack` tensor. Requires Triton + PyTorch. | +| [`voxels_to_grid_cuda.py`](voxels_to_grid_cuda.py) | Build a grid on the GPU from a CuPy coordinate array with `tools.cuda.voxelsToOnIndexGrid` and `tools.cuda.pointsToGrid` — construction entirely on the device (no host tree). GPU counterpart to `build_grid.py`; port of `ex_voxels_to_grid_cuda`. Requires CuPy. | +| [`device_topology_ops.py`](device_topology_ops.py) | `tools.cuda.dilateGrid` / `coarsenGrid` / `refineGrid` / `mergeGrids` / `pruneGrid` on a device OnIndex grid. Ports the device path of `ex_{dilate,coarsen,refine,merge}_nanovdb_cuda` (OpenVDB-free: the source grid is built on the device). Requires CuPy. | +| [`sample_from_voxels_cuda.py`](sample_from_voxels_cuda.py) | `tools.cuda.sampleFromVoxels` — batched trilinear samples plus analytic gradients at device query points; the device analog of `createTrilinearSampler`, cross-checked against the host sampler. Requires CuPy. | +| [`index_to_grid_cuda.py`](index_to_grid_cuda.py) | `tools.cuda.indexToGrid` bakes a CuPy-computed value array onto an index grid on the device, with `tools.cuda.activeVoxelCoords` recovering each value slot's coordinate. Device half of `ex_index_grid_cuda`. Requires CuPy. | +| [`signed_flood_fill_cuda.py`](signed_flood_fill_cuda.py) | `tools.cuda.signedFloodFill` propagates interior/exterior sign of a level set in place on the device; verified with `tools.cuda.sampleFromVoxels`. Requires CuPy. | +| [`validate_cuda.py`](validate_cuda.py) | Device quality control: `tools.cuda.isValid`, `evalChecksum` / `updateChecksum` / `validateChecksum`, and `updateGridStats`, all in place on a device grid. GPU counterpart to `validate.py`. Requires CuPy. | +| [`collide_level_set_cuda.py`](collide_level_set_cuda.py) | GPU counterpart to `collide_level_set.py`: a whole particle set collided against a level set via one `tools.cuda.sampleFromVoxels` call (distances + gradients), with the reflection response as vectorized CuPy. Port of the device path of `ex_collide_level_set`. Requires CuPy. | +| [`raytrace_level_set_cuda.py`](raytrace_level_set_cuda.py) | GPU counterpart to `raytrace_level_set.py`: one CUDA thread per pixel in a `cupy.RawKernel`, using the faithful `nanovdb::math::Ray` + HDDA `ZeroCrossing` surface search (bound only in C++). Port of the device path of `ex_raytrace_level_set`. Requires CuPy + NanoVDB headers. | +| [`raytrace_fog_volume_cuda.py`](raytrace_fog_volume_cuda.py) | GPU counterpart to `raytrace_fog_volume.py`: per-pixel transmittance ray-march in a `cupy.RawKernel` over the device grid accessor. Port of the device path of `ex_raytrace_fog_volume`. Requires CuPy + NanoVDB headers. | +| [`mesh_to_grid_cuda.py`](mesh_to_grid_cuda.py) | `tools.cuda.meshToGrid` rasterizes a triangle mesh into a narrow-band unsigned distance field on the device (returns an OnIndex grid + per-value UDF sidecar), then `indexToGrid` bakes it into a Float distance grid. Port of `ex_mesh_to_grid_cuda` (OpenVDB-free: the mesh is generated with NumPy). Requires CuPy. | #### One level-set filter, three compute backends @@ -137,7 +147,8 @@ TC = nanovdb.tools.cuda # ok `nanovdb::tools::cuda`): point / voxel rasterizers (`pointsToGrid`, `voxelsTo{OnIndex,Index,RGBA8}Grid`, `pointsToRGBA8Grid`), morphology / topology (`dilateGrid`, `coarsenGrid`, `refineGrid`, `pruneGrid`, - `mergeGrids`), index utilities (`indexToGrid`, `addBlindData`), in-place + `mergeGrids`), mesh rasterization (`meshToGrid`), index utilities + (`indexToGrid`, `addBlindData`), in-place device QC (`updateGridStats`, `updateChecksum`, `evalChecksum`, `validateChecksum`, `isValid`), `signedFloodFill`, `sampleFromVoxels`, `buildVoxelBlockManager`, and the multi-GPU `Distributed*PointsToGrid` diff --git a/nanovdb/nanovdb/python/examples/collide_level_set_cuda.py b/nanovdb/nanovdb/python/examples/collide_level_set_cuda.py new file mode 100644 index 0000000000..e2eaabcde8 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/collide_level_set_cuda.py @@ -0,0 +1,109 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Collide falling particles against a level set on the GPU. + +GPU counterpart to ``collide_level_set.py`` and a Python port of the +device path of ``ex_collide_level_set``. The CPU version loops over +particles one at a time, querying the SDF and its gradient per particle; +here the entire particle set is a pair of ``(N, 3)`` CuPy arrays and a +single ``nanovdb.tools.cuda.sampleFromVoxels`` call returns the signed +distance AND the analytic gradient for every particle at once — exactly +what a collision response needs (distance to push out along, normal to +reflect about). The per-step reflection is then plain vectorized CuPy. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python collide_level_set_cuda.py +""" +import os +import tempfile + +import nanovdb + +NUM_PARTICLES = 4000 +NUM_STEPS = 60 +DT = 0.1 +GRAVITY = -9.8 +RADIUS = 100.0 +BAND = 3.0 # narrow-band half-width in world units (default 3 voxels) + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + host_handle = nanovdb.tools.createLevelSetSphere( + radius=RADIUS, voxelSize=1.0, name="sphere") + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + nanovdb.io.writeGrid(tmp.name, host_handle) + try: + handle = nanovdb.io.deviceReadGrid(tmp.name) + finally: + os.unlink(tmp.name) + handle.deviceUpload(0, True) + device_grid = handle.deviceGrid(0) + + # Seed particles above the north pole of the sphere, falling down. + rng = cp.random.RandomState(42) + p = cp.empty((NUM_PARTICLES, 3), dtype=cp.float32) + p[:, 0] = rng.uniform(-30.0, 30.0, NUM_PARTICLES) + p[:, 1] = rng.uniform(RADIUS + 15.0, RADIUS + 40.0, NUM_PARTICLES) + p[:, 2] = rng.uniform(-30.0, 30.0, NUM_PARTICLES) + v = cp.zeros((NUM_PARTICLES, 3), dtype=cp.float32) + v[:, 1] = -20.0 + + values = cp.empty(NUM_PARTICLES, dtype=cp.float32) + grads = cp.empty((NUM_PARTICLES, 3), dtype=cp.float32) + total_collisions = 0 + + for step in range(NUM_STEPS): + v[:, 1] += GRAVITY * DT + next_p = cp.ascontiguousarray(p + v * DT) + + nanovdb.tools.cuda.sampleFromVoxels(next_p, device_grid, values, grads, 0) + cp.cuda.Stream.null.synchronize() + + # A collision is a point inside the surface but still within the + # meaningful narrow band (deep interior clamps to -background). + hit = (values <= 0.0) & (values > -BAND) + collisions = int(hit.sum()) + total_collisions += collisions + + # Normalized surface normals from the sampled gradients. + norm = cp.linalg.norm(grads, axis=1, keepdims=True) + n = grads / cp.maximum(norm, 1e-8) + mask = hit[:, None] + # Project penetrating particles back onto the surface... + next_p = cp.where(mask, next_p - values[:, None] * n, next_p) + # ...and reflect their velocity about the surface normal. + v_dot_n = (v * n).sum(axis=1, keepdims=True) + v = cp.where(mask, v - 2.0 * v_dot_n * n, v) + p = next_p + + if collisions: + print(f"step {step:2d}: {collisions} collisions") + + print(f"total collisions over {NUM_STEPS} steps: {total_collisions}") + assert total_collisions > 0 + + # No particle should end up deep inside the surface. + nanovdb.tools.cuda.sampleFromVoxels(cp.ascontiguousarray(p), device_grid, values, 0) + cp.cuda.Stream.null.synchronize() + in_band = (values > -BAND) & (values < BAND) + if bool(in_band.any()): + worst = float(values[in_band].min()) + print(f"min final SDF among in-band particles: {worst:.3f} world units " + f"(> 0 means all bounced clear of the surface)") + assert worst > -2.0 + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/device_topology_ops.py b/nanovdb/nanovdb/python/examples/device_topology_ops.py new file mode 100644 index 0000000000..5094ca3359 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/device_topology_ops.py @@ -0,0 +1,93 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Morphology and topology operators on device OnIndex grids. + +Python port of the device path shared by ``ex_dilate_nanovdb_cuda``, +``ex_coarsen_nanovdb_cuda``, ``ex_refine_nanovdb_cuda`` and +``ex_merge_nanovdb_cuda``. The C++ examples read a ``.vdb`` through +OpenVDB to source the grid; here the source OnIndex grid is built +on the GPU with ``voxelsToOnIndexGrid``, so the example needs no +OpenVDB and everything stays on the device. + +Each operator returns a fresh device ``GridHandle``: + +* ``dilateGrid(g, op)`` — grow active topology (op 6 = faces, + 26 = faces+edges+vertices). +* ``coarsenGrid(g)`` — 2x topological downsample. +* ``refineGrid(g)`` — 2x topological upsample. +* ``mergeGrids(a, b)`` — active-mask union (strictly binary). +* ``pruneGrid(g, mask)`` — keep only voxels flagged in a per-leaf + retain mask (one ``nanovdb::Mask<3>`` = 8 x uint64 per leaf). + +Requires CuPy and a CUDA-capable GPU. + +Run with: python device_topology_ops.py +""" +import nanovdb + + +def _active(handle): + """Download a device handle and return its active voxel count.""" + handle.deviceDownload(0, True) + return handle.grid(0).activeVoxelCount() + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + # Source grid: a solid 8x8x8 block of voxels, built on the GPU. + lin = cp.arange(8, dtype=cp.int32) + i, j, k = cp.meshgrid(lin, lin, lin, indexing="ij") + coords = cp.ascontiguousarray( + cp.stack([i.ravel(), j.ravel(), k.ravel()], axis=1)) + src = nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, 1.0, 0) + src_grid = src.deviceGrid(0) + src_active = _active(src) + print(f"source block: {src_active} active voxels") + + dil6 = nanovdb.tools.cuda.dilateGrid(src_grid, 6, 0) + dil26 = nanovdb.tools.cuda.dilateGrid(src_grid, 26, 0) + coarse = nanovdb.tools.cuda.coarsenGrid(src_grid, 0) + fine = nanovdb.tools.cuda.refineGrid(src_grid, 0) + a6, a26 = _active(dil6), _active(dil26) + ac, af = _active(coarse), _active(fine) + print(f"dilate(faces) -> {a6} active") + print(f"dilate(faces+e+v) -> {a26} active") + print(f"coarsen (2x down) -> {ac} active") + print(f"refine (2x up) -> {af} active") + assert a6 > src_active and a26 >= a6 + assert ac < src_active < af + + # Union with a shifted copy of the block; the merged topology must + # cover at least the larger of the two inputs. + shifted = nanovdb.tools.cuda.voxelsToOnIndexGrid( + cp.ascontiguousarray(coords + cp.asarray([4, 0, 0], dtype=cp.int32)), + 1.0, 0) + merged = nanovdb.tools.cuda.mergeGrids(src_grid, shifted.deviceGrid(0), 0) + am = _active(merged) + print(f"merge(block, block+4x) -> {am} active") + assert am > src_active + + # Prune with a retain-all mask (8 uint64 per leaf); topology is + # unchanged, demonstrating the mask-driven prune entry point. + src.deviceDownload(0, True) + leaf_count = src.grid(0).tree().nodeCount(0) + retain_all = cp.full(leaf_count * 8, 0xFFFFFFFFFFFFFFFF, dtype=cp.uint64) + pruned = nanovdb.tools.cuda.pruneGrid(src_grid, retain_all, 0) + ap = _active(pruned) + print(f"prune(retain-all, {leaf_count} leaves) -> {ap} active") + assert ap == src_active + + print("OK: dilate / coarsen / refine / merge / prune on the device") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/index_to_grid_cuda.py b/nanovdb/nanovdb/python/examples/index_to_grid_cuda.py new file mode 100644 index 0000000000..423ecced38 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/index_to_grid_cuda.py @@ -0,0 +1,82 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Bake a per-voxel value array onto an index grid, on the device. + +Python port of the device path of ``ex_index_grid_cuda`` (the GPU half +of ``index_grid_channels.py``). An OnIndex grid stores only topology; +the per-voxel payload lives in a separate linear array indexed by +``grid.valueCount()``. ``nanovdb.tools.cuda.indexToGrid`` fuses the two +on the device — an index grid plus a value array in, a fully-typed +value grid out: + + value_handle = tools.cuda.indexToGrid(d_index_grid, values, stream) + +To fill the value array meaningfully we use +``tools.cuda.activeVoxelCoords`` to recover each value slot's +index-space coordinate (the decode companion to the index grid), then +compute a value per voxel with plain CuPy — here the distance from the +origin, turning the shell into a scalar field. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python index_to_grid_cuda.py +""" +import nanovdb + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + # A hollow sphere shell of voxels, built on the GPU. + radius = 24.0 + lin = cp.arange(-30, 31, dtype=cp.float32) + x, y, z = cp.meshgrid(lin, lin, lin, indexing="ij") + r = cp.sqrt(x * x + y * y + z * z) + shell = cp.abs(r - radius) < 1.0 + coords = cp.ascontiguousarray( + cp.stack([x[shell], y[shell], z[shell]], axis=1).astype(cp.int32)) + index_handle = nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, 1.0, 0) + index_grid = index_handle.deviceGrid(0) + + index_handle.deviceDownload(0, True) + value_count = index_handle.grid(0).valueCount() + print(f"OnIndex grid: {index_handle.grid(0).activeVoxelCount()} voxels, " + f"valueCount={value_count}") + + # Recover the coordinate of every value slot, then compute a value + # (distance from origin) per voxel with CuPy. Row 0 is the background + # slot; leave it at zero. + voxel_coords = cp.zeros((value_count, 3), dtype=cp.int32) + nanovdb.tools.cuda.activeVoxelCoords(index_grid, voxel_coords, 9, 0) + cp.cuda.Stream.null.synchronize() + fc = voxel_coords.astype(cp.float32) + values = cp.sqrt((fc * fc).sum(axis=1)).astype(cp.float32) + values[0] = 0.0 # background slot + values = cp.ascontiguousarray(values) + + value_handle = nanovdb.tools.cuda.indexToGrid(index_grid, values, 0) + value_grid = value_handle.deviceGrid(0) + print(f"indexToGrid -> {value_handle.gridType(0)} grid") + assert nanovdb.tools.cuda.isValid(value_grid, nanovdb.CheckMode.Full) + + # Confirm the CuPy-computed values actually landed in the grid by + # reading one back on the host: a shell voxel holds its radius. + value_handle.deviceDownload(0, True) + sampler = nanovdb.math.createTrilinearSampler(value_handle.grid(0)) + baked = sampler(nanovdb.math.Vec3f(radius, 0.0, 0.0)) + print(f" baked value at (r,0,0) = {baked:.2f} (shell radius {radius:.0f})") + assert abs(baked - radius) < 1.0 + + print("OK: baked a CuPy-computed value array onto an index grid on device") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/mesh_to_grid_cuda.py b/nanovdb/nanovdb/python/examples/mesh_to_grid_cuda.py new file mode 100644 index 0000000000..80b285309e --- /dev/null +++ b/nanovdb/nanovdb/python/examples/mesh_to_grid_cuda.py @@ -0,0 +1,94 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Rasterize a triangle mesh into a narrow-band SDF on the GPU. + +Python port of the device path of ``ex_mesh_to_grid_cuda``. +``nanovdb.tools.cuda.meshToGrid`` takes device arrays of mesh vertices and +triangle indices and builds, entirely on the GPU, a narrow-band ``OnIndex`` +grid plus a per-value unsigned-distance-field (UDF) sidecar. The C++ example +sources its mesh through OpenVDB; here the mesh (a closed box) is generated +with NumPy, so the example needs no OpenVDB. + +The returned pieces chain naturally: + +* ``handle`` — a device ``OnIndex`` grid holding the narrow-band topology. +* ``udf`` — a ``nanovdb.cuda.DeviceBuffer`` of ``valueCount`` float32 unsigned + distances (voxel units), indexed by the grid's per-voxel value index. That + is exactly the value array ``tools.cuda.indexToGrid`` consumes, so we bake + the UDF into a Float distance grid and sample it back to verify. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python mesh_to_grid_cuda.py +""" +import nanovdb + + +def _box_mesh(cp, half): + """Return (vertices (8,3) float32, triangles (12,3) int32) for a cube.""" + corners = cp.asarray([ + [-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1], + [-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1], + ], dtype=cp.float32) * half + # 12 triangles (2 per face), outward winding. + tris = cp.asarray([ + [0, 3, 2], [0, 2, 1], # -z + [4, 5, 6], [4, 6, 7], # +z + [0, 1, 5], [0, 5, 4], # -y + [3, 7, 6], [3, 6, 2], # +y + [0, 4, 7], [0, 7, 3], # -x + [1, 2, 6], [1, 6, 5], # +x + ], dtype=cp.int32) + return cp.ascontiguousarray(corners), cp.ascontiguousarray(tris) + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + half = 40.0 + voxel_size = 1.0 + vertices, triangles = _box_mesh(cp, half) + print(f"Box mesh: {vertices.shape[0]} vertices, {triangles.shape[0]} " + f"triangles, half-extent {half:.0f}") + + # Build the narrow-band UDF on the device in one call. + handle, udf = nanovdb.tools.cuda.meshToGrid( + vertices, triangles, voxel_size, 3.0, "box_udf", 0) + index_grid = handle.deviceGrid(0) + assert nanovdb.tools.cuda.isValid(index_grid) + + handle.deviceDownload(0, True) + value_count = handle.grid(0).valueCount() + # The sidecar is a device buffer of `value_count` float32 UDF values. + udf_values = cp.asarray(udf).view(cp.float32)[:value_count] + udf_values = cp.ascontiguousarray(udf_values) + print(f"meshToGrid -> {handle.gridType(0)} grid, " + f"{handle.grid(0).activeVoxelCount()} narrow-band voxels, " + f"UDF range [{float(udf_values[1:].min()):.2f}, " + f"{float(udf_values[1:].max()):.2f}] voxels") + + # Bake the UDF into a Float distance grid on the device and sample it. + dist_handle = nanovdb.tools.cuda.indexToGrid(index_grid, udf_values, 0) + dist_grid = dist_handle.deviceGrid(0) + assert nanovdb.tools.cuda.isValid(dist_grid, nanovdb.CheckMode.Full) + + # A point right on a face should read a near-zero unsigned distance. + dist_handle.deviceDownload(0, True) + sampler = nanovdb.math.createTrilinearSampler(dist_handle.grid(0)) + on_face = sampler(nanovdb.math.Vec3f(half, 0.0, 0.0)) + print(f" baked UDF at a box face = {on_face:.3f} voxels (expect near 0)") + assert on_face < 2.0 + + print("OK: built a narrow-band SDF from a triangle mesh on the device") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/raytrace_fog_volume_cuda.py b/nanovdb/nanovdb/python/examples/raytrace_fog_volume_cuda.py new file mode 100644 index 0000000000..eb1bf3eb4b --- /dev/null +++ b/nanovdb/nanovdb/python/examples/raytrace_fog_volume_cuda.py @@ -0,0 +1,139 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Ray-march a fog volume on the GPU with a custom CUDA kernel. + +GPU counterpart to ``raytrace_fog_volume.py`` and a Python port of the +device path of ``ex_raytrace_fog_volume``. The host port integrates +transmittance one pixel at a time in Python; here a ``cupy.RawKernel`` +does one thread per pixel with the full C++ NanoVDB API: a +``nanovdb::math::Ray`` clipped to the grid, then a fixed-step march +accumulating optical depth from ``acc.getValue(Coord::Floor(...))`` — the +accessor-based sampling idiom, on the device. The whole image renders in +one launch. + +The kernel compiles against the bundled NanoVDB headers via +``nanovdb.cuda.compile_options()`` (with the ``NANOVDB_INCLUDE`` dev-tree +fallback from ``cupy_rawkernel.py``). + +Requires CuPy and a CUDA-capable GPU. + +Run with: python raytrace_fog_volume_cuda.py +""" +import os +import tempfile + +import nanovdb + +RES = 256 +FOV_DEG = 45.0 +STEP = 0.5 # index-space march step +DENSITY_SCALE = 40.0 # maps fog density to optical depth + +KERNEL_SRC = r""" +#include +#include + +extern "C" __global__ +void render_fog(const nanovdb::NanoGrid* d_grid, + unsigned char* image, int res, float tan_fov, + float eye_x, float eye_y, float eye_z, + float step, float density_scale) +{ + const int x = blockIdx.x * blockDim.x + threadIdx.x; + const int y = blockIdx.y * blockDim.y + threadIdx.y; + if (x >= res || y >= res) return; + + using Vec3T = nanovdb::math::Vec3f; + using RayT = nanovdb::math::Ray; + + auto acc = d_grid->tree().getAccessor(); + const float px = (2.0f * (x + 0.5f) / res - 1.0f) * tan_fov; + const float py = (2.0f * (y + 0.5f) / res - 1.0f) * tan_fov; + const float inv = 1.0f / sqrtf(px * px + py * py + 1.0f); + + RayT ray(Vec3T(eye_x, eye_y, eye_z), + Vec3T(px * inv, py * inv, -inv)); + float transmittance = 1.0f; + if (ray.clip(d_grid->indexBBox())) { + for (float t = ray.t0(); t < ray.t1(); t += step) { + const Vec3T pos = ray(t); + const float sigma = acc.getValue(nanovdb::Coord::Floor(pos)); + if (sigma > 0.0f) + transmittance *= expf(-sigma * step * density_scale); + } + } + // Opacity = 1 - transmittance, mapped to an 8-bit gray value. + image[y * res + x] = (unsigned char)(255.0f * (1.0f - transmittance)); +} +""" + + +def _include_options(): + """compile_options(), falling back to $NANOVDB_INCLUDE in a dev tree.""" + opts = list(nanovdb.cuda.compile_options("-std=c++17")) + inc_dir = opts[0][2:] + if not os.path.isdir(inc_dir): + env_inc = os.environ.get("NANOVDB_INCLUDE") + if env_inc and os.path.isdir(env_inc): + opts[0] = f"-I{env_inc}" + else: + print(f"NanoVDB headers not found at {inc_dir!r} (expected in an " + "installed wheel). Set NANOVDB_INCLUDE to the dir that " + "contains nanovdb/NanoVDB.h to run from a source tree.") + return None + return tuple(opts) + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + options = _include_options() + if options is None: + return + + handle = nanovdb.tools.cuda.createFogVolumeSphere( + nanovdb.GridType.Float, radius=100.0) + handle.deviceUpload(0, True) + handle.deviceDownload(0, True) + device_grid = handle.deviceGrid(0) + bbox = handle.grid(0).indexBBox() + + import math + dim = [bbox.max[i] + 1 - bbox.min[i] for i in range(3)] + center = [bbox.min[i] + 0.5 * dim[i] for i in range(3)] + eye = (center[0], center[1], center[2] + 2.0 * dim[2]) + tan_fov = math.tan(math.radians(FOV_DEG) * 0.5) + + kernel = cp.RawKernel( + KERNEL_SRC, "render_fog", options=options, backend="nvrtc") + image = cp.zeros(RES * RES, dtype=cp.uint8) + block = (16, 16) + grid = ((RES + 15) // 16, (RES + 15) // 16) + kernel(grid, block, + (device_grid.data_ptr(), image, RES, cp.float32(tan_fov), + cp.float32(eye[0]), cp.float32(eye[1]), cp.float32(eye[2]), + cp.float32(STEP), cp.float32(DENSITY_SCALE))) + cp.cuda.runtime.deviceSynchronize() + + host_image = cp.asnumpy(image) + lit = int((host_image > 0).sum()) + out_dir = tempfile.mkdtemp(prefix="nanovdb_") + path = os.path.join(out_dir, "raytrace_fog_volume_cuda.pgm") + with open(path, "wb") as f: + f.write(f"P5\n{RES} {RES}\n255\n".encode("ascii")) + f.write(host_image.tobytes()) + print(f"Rendered {RES}x{RES} fog volume on the GPU in one launch, " + f"{lit} lit pixels -> {path}") + assert lit > 0 + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/raytrace_level_set_cuda.py b/nanovdb/nanovdb/python/examples/raytrace_level_set_cuda.py new file mode 100644 index 0000000000..6419108496 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/raytrace_level_set_cuda.py @@ -0,0 +1,151 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Ray-trace a level set on the GPU with a custom CUDA kernel. + +GPU counterpart to ``raytrace_level_set.py`` and a Python port of the +device path of ``ex_raytrace_level_set``. The host port loops over +pixels in Python and had to approximate the C++ HDDA surface search with +sphere tracing (``math::Ray`` / ``ZeroCrossing`` are not bound in +Python). Inside a ``cupy.RawKernel`` we have the FULL C++ NanoVDB API, +so this version is the faithful original: one CUDA thread per pixel, +``nanovdb::math::Ray`` clipped to the grid, ``nanovdb::math::ZeroCrossing`` +(HDDA) for the exact surface hit, and a central-difference gradient for +Lambert shading. The whole image renders in one launch. + +The kernel is compiled against the bundled NanoVDB headers via +``nanovdb.cuda.compile_options()`` (with the ``NANOVDB_INCLUDE`` dev-tree +fallback from ``cupy_rawkernel.py``) and reads the device grid straight +from ``deviceGrid(0).data_ptr()``. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python raytrace_level_set_cuda.py +""" +import os +import tempfile + +import nanovdb + +RES = 256 +FOV_DEG = 45.0 +LIGHT = (0.577, 0.577, 0.577) + +KERNEL_SRC = r""" +#include +#include +#include + +extern "C" __global__ +void render_level_set(const nanovdb::NanoGrid* d_grid, + unsigned char* image, int res, float tan_fov, + float eye_x, float eye_y, float eye_z, + float lx, float ly, float lz) +{ + const int x = blockIdx.x * blockDim.x + threadIdx.x; + const int y = blockIdx.y * blockDim.y + threadIdx.y; + if (x >= res || y >= res) return; + + using Vec3T = nanovdb::math::Vec3f; + using RayT = nanovdb::math::Ray; + + auto acc = d_grid->tree().getAccessor(); + const float px = (2.0f * (x + 0.5f) / res - 1.0f) * tan_fov; + const float py = (2.0f * (y + 0.5f) / res - 1.0f) * tan_fov; + const float inv = 1.0f / sqrtf(px * px + py * py + 1.0f); + + RayT ray(Vec3T(eye_x, eye_y, eye_z), + Vec3T(px * inv, py * inv, -inv)); + unsigned char shade = 0; + if (ray.clip(d_grid->indexBBox())) { + nanovdb::Coord ijk; + float v = 0.0f, t = 0.0f; + if (nanovdb::math::ZeroCrossing(ray, acc, ijk, v, t)) { + float gx = acc.getValue(ijk.offsetBy(1, 0, 0)) + - acc.getValue(ijk.offsetBy(-1, 0, 0)); + float gy = acc.getValue(ijk.offsetBy(0, 1, 0)) + - acc.getValue(ijk.offsetBy(0, -1, 0)); + float gz = acc.getValue(ijk.offsetBy(0, 0, 1)) + - acc.getValue(ijk.offsetBy(0, 0, -1)); + const float gl = sqrtf(gx * gx + gy * gy + gz * gz); + if (gl > 0.0f) { gx /= gl; gy /= gl; gz /= gl; } + float s = gx * lx + gy * ly + gz * lz; + if (s < 0.0f) s = 0.0f; + shade = (unsigned char)(255.0f * s); + } + } + image[y * res + x] = shade; +} +""" + + +def _include_options(): + """compile_options(), falling back to $NANOVDB_INCLUDE in a dev tree.""" + opts = list(nanovdb.cuda.compile_options("-std=c++17")) + inc_dir = opts[0][2:] + if not os.path.isdir(inc_dir): + env_inc = os.environ.get("NANOVDB_INCLUDE") + if env_inc and os.path.isdir(env_inc): + opts[0] = f"-I{env_inc}" + else: + print(f"NanoVDB headers not found at {inc_dir!r} (expected in an " + "installed wheel). Set NANOVDB_INCLUDE to the dir that " + "contains nanovdb/NanoVDB.h to run from a source tree.") + return None + return tuple(opts) + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + options = _include_options() + if options is None: + return + + # Level-set sphere built on the device; download once for the host-side + # camera setup (index bounding box). + handle = nanovdb.tools.cuda.createLevelSetSphere( + nanovdb.GridType.Float, radius=100.0) + handle.deviceUpload(0, True) + handle.deviceDownload(0, True) + device_grid = handle.deviceGrid(0) + bbox = handle.grid(0).indexBBox() + + import math + dim = [bbox.max[i] + 1 - bbox.min[i] for i in range(3)] + center = [bbox.min[i] + 0.5 * dim[i] for i in range(3)] + eye = (center[0], center[1], center[2] + 2.0 * dim[2]) + tan_fov = math.tan(math.radians(FOV_DEG) * 0.5) + + kernel = cp.RawKernel( + KERNEL_SRC, "render_level_set", options=options, backend="nvrtc") + image = cp.zeros(RES * RES, dtype=cp.uint8) + block = (16, 16) + grid = ((RES + 15) // 16, (RES + 15) // 16) + kernel(grid, block, + (device_grid.data_ptr(), image, RES, cp.float32(tan_fov), + cp.float32(eye[0]), cp.float32(eye[1]), cp.float32(eye[2]), + cp.float32(LIGHT[0]), cp.float32(LIGHT[1]), cp.float32(LIGHT[2]))) + cp.cuda.runtime.deviceSynchronize() + + host_image = cp.asnumpy(image) + lit = int((host_image > 0).sum()) + out_dir = tempfile.mkdtemp(prefix="nanovdb_") + path = os.path.join(out_dir, "raytrace_level_set_cuda.pgm") + with open(path, "wb") as f: + f.write(f"P5\n{RES} {RES}\n255\n".encode("ascii")) + f.write(host_image.tobytes()) + print(f"Rendered {RES}x{RES} level set on the GPU in one launch, " + f"{lit} lit pixels -> {path}") + assert lit > 0 + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/sample_from_voxels_cuda.py b/nanovdb/nanovdb/python/examples/sample_from_voxels_cuda.py new file mode 100644 index 0000000000..e5362c3d06 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/sample_from_voxels_cuda.py @@ -0,0 +1,89 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Trilinearly sample a device grid at arbitrary points, with gradients. + +Demonstrates ``nanovdb.tools.cuda.sampleFromVoxels`` — the device +equivalent of the host ``createTrilinearSampler`` used by +``raytrace_level_set.py`` and ``collide_level_set.py``. Given a device +``FloatGrid`` and an ``(N, 3)`` CuPy array of WORLD-space query points, +it writes trilinear samples into an ``(N,)`` array and (optionally) +analytic gradients into an ``(N, 3)`` array, all on the GPU: + + tools.cuda.sampleFromVoxels(points, d_grid, values, stream) + tools.cuda.sampleFromVoxels(points, d_grid, values, gradients, stream) + +Here we sample a level-set sphere along a ray crossing its surface, +recover the surface normal from the normalized gradient, and cross-check +a few device samples against the host ``createTrilinearSampler``. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python sample_from_voxels_cuda.py +""" +import nanovdb + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + # Level-set sphere (radius 50), read straight onto the device. + radius = 50.0 + host_handle = nanovdb.tools.createLevelSetSphere( + radius=radius, voxelSize=1.0, name="sphere") + import os + import tempfile + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + nanovdb.io.writeGrid(tmp.name, host_handle) + try: + handle = nanovdb.io.deviceReadGrid(tmp.name) + finally: + os.unlink(tmp.name) + handle.deviceUpload(0, True) + device_grid = handle.deviceGrid(0) + + # Query points marching along +x across the surface (within the + # narrow band, where the SDF is meaningful). + xs = cp.linspace(radius - 3.0, radius + 3.0, 7, dtype=cp.float32) + points = cp.zeros((xs.size, 3), dtype=cp.float32) + points[:, 0] = xs + points = cp.ascontiguousarray(points) + + values = cp.empty(xs.size, dtype=cp.float32) + gradients = cp.empty((xs.size, 3), dtype=cp.float32) + nanovdb.tools.cuda.sampleFromVoxels(points, device_grid, values, gradients, 0) + cp.cuda.Stream.null.synchronize() + + print(f"Sampling a radius-{radius:.0f} level set along +x:") + v_host = cp.asnumpy(values) + g_host = cp.asnumpy(gradients) + for xi, vi, gi in zip(cp.asnumpy(xs), v_host, g_host): + gn = (gi[0] ** 2 + gi[1] ** 2 + gi[2] ** 2) ** 0.5 + print(f" x={xi:6.2f} sdf={vi:+.3f} |grad|={gn:.3f}") + + # SDF must increase monotonically as we move outward through the band. + assert (v_host[1:] >= v_host[:-1]).all() + # The sample straddling the surface should be ~0. + assert abs(v_host[xs.size // 2]) < 1.0 + + # Cross-check the device sampler against the host sampler. + host_sampler = nanovdb.math.createTrilinearSampler(host_handle.grid(0)) + mid = float(cp.asnumpy(xs)[xs.size // 2]) + host_val = host_sampler(nanovdb.math.Vec3f(mid, 0.0, 0.0)) + print(f"device vs host at x={mid:.2f}: " + f"{v_host[xs.size // 2]:+.4f} vs {host_val:+.4f}") + assert abs(v_host[xs.size // 2] - host_val) < 1e-3 + + print("OK: device trilinear sampling + gradients match the host sampler") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/signed_flood_fill_cuda.py b/nanovdb/nanovdb/python/examples/signed_flood_fill_cuda.py new file mode 100644 index 0000000000..a9c82bec76 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/signed_flood_fill_cuda.py @@ -0,0 +1,74 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Signed flood fill of a level set on the device. + +Demonstrates ``nanovdb.tools.cuda.signedFloodFill`` — the device tool +that propagates interior/exterior sign out of a narrow band so the +inactive tiles inside the surface read as negative background and those +outside read as positive background. It runs in place on a device +``FloatGrid`` (or ``DoubleGrid``): + + tools.cuda.signedFloodFill(d_grid, verbose, stream) + +We upload a level-set sphere, run the flood fill on the device, then use +``tools.cuda.sampleFromVoxels`` to confirm the field is consistently +signed (negative inside, positive outside) and that the grid still +validates. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python signed_flood_fill_cuda.py +""" +import os +import tempfile + +import nanovdb + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + radius = 40.0 + host_handle = nanovdb.tools.createLevelSetSphere( + radius=radius, voxelSize=1.0, name="sphere") + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + nanovdb.io.writeGrid(tmp.name, host_handle) + try: + handle = nanovdb.io.deviceReadGrid(tmp.name) + finally: + os.unlink(tmp.name) + handle.deviceUpload(0, True) + device_grid = handle.deviceGrid(0) + + # Propagate signs across the whole grid on the device, in place. + nanovdb.tools.cuda.signedFloodFill(device_grid, False, 0) + print("signedFloodFill: done on the device") + + # The interior sign is only meaningful right at the band; sample just + # inside and just outside the surface and confirm the signs. + points = cp.asarray([[radius - 2.0, 0.0, 0.0], + [radius + 2.0, 0.0, 0.0]], dtype=cp.float32) + values = cp.empty(2, dtype=cp.float32) + nanovdb.tools.cuda.sampleFromVoxels(cp.ascontiguousarray(points), device_grid, values, 0) + cp.cuda.Stream.null.synchronize() + inside, outside = (float(v) for v in cp.asnumpy(values)) + print(f" sdf just inside surface = {inside:+.3f} (expect < 0)") + print(f" sdf just outside surface = {outside:+.3f} (expect > 0)") + assert inside < 0.0 < outside + + # The flooded grid must still be structurally valid. + assert nanovdb.tools.cuda.isValid(device_grid, nanovdb.CheckMode.Full) + print("OK: device signed flood fill produced a consistent, valid SDF") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/validate_cuda.py b/nanovdb/nanovdb/python/examples/validate_cuda.py new file mode 100644 index 0000000000..e0e1fef549 --- /dev/null +++ b/nanovdb/nanovdb/python/examples/validate_cuda.py @@ -0,0 +1,73 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Validate a grid, and its checksum and stats, entirely on the device. + +GPU counterpart to ``validate.py``. Once a grid is resident on the GPU +(``deviceReadGrid`` + ``deviceUpload``), the quality-control tools run +in place on the device grid — no download required: + +* ``tools.cuda.isValid(g, mode)`` — structural + checksum check. +* ``tools.cuda.evalChecksum(g, mode)`` — compute the CRC checksum. +* ``tools.cuda.updateChecksum(g, mode)`` — recompute it in place. +* ``tools.cuda.validateChecksum(g, mode)`` — verify it matches. +* ``tools.cuda.updateGridStats(g, mode)`` — recompute min/max/avg/bbox. + +``CheckMode`` trades coverage for speed (``Partial`` vs ``Full``); +``StatsMode`` selects how much of the per-node stats to refresh. + +Requires CuPy and a CUDA-capable GPU. + +Run with: python validate_cuda.py +""" +import os +import tempfile + +import nanovdb + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy # noqa: F401 (only needed to confirm a usable GPU stack) + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + host_handle = nanovdb.tools.createLevelSetSphere( + radius=40.0, voxelSize=1.0, name="sphere") + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + nanovdb.io.writeGrid(tmp.name, host_handle) + try: + handle = nanovdb.io.deviceReadGrid(tmp.name) + finally: + os.unlink(tmp.name) + handle.deviceUpload(0, True) + device_grid = handle.deviceGrid(0) + + # Structural validation on the device, partial and full. + partial = nanovdb.tools.cuda.isValid(device_grid, nanovdb.CheckMode.Partial) + full = nanovdb.tools.cuda.isValid(device_grid, nanovdb.CheckMode.Full) + print(f"isValid: Partial={partial}, Full={full}") + assert partial and full + + # Checksum round-trip: recompute in place, then verify it matches. + nanovdb.tools.cuda.updateChecksum(device_grid, nanovdb.CheckMode.Full) + checksum = nanovdb.tools.cuda.evalChecksum(device_grid, nanovdb.CheckMode.Full) + ok = nanovdb.tools.cuda.validateChecksum(device_grid, nanovdb.CheckMode.Full) + print(f"checksum: {checksum} validateChecksum(Full)={ok}") + assert ok + + # Recompute per-node statistics on the device. + nanovdb.tools.cuda.updateGridStats(device_grid) + print("updateGridStats: OK") + assert nanovdb.tools.cuda.isValid(device_grid, nanovdb.CheckMode.Full) + + print("OK: device-side validation, checksum, and stats all pass") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/python/examples/voxels_to_grid_cuda.py b/nanovdb/nanovdb/python/examples/voxels_to_grid_cuda.py new file mode 100644 index 0000000000..266294de6b --- /dev/null +++ b/nanovdb/nanovdb/python/examples/voxels_to_grid_cuda.py @@ -0,0 +1,90 @@ +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Build a NanoVDB grid directly on the GPU from a CuPy coordinate array. + +GPU counterpart to ``build_grid.py`` and a Python port of +``ex_voxels_to_grid_cuda`` (and the device half of +``ex_make_custom_nanovdb_cuda``). Where the host builders write voxels +one at a time through a tree, the device rasterizers take a whole +``(N, 3)`` array already resident on the GPU and construct the grid in +one call, entirely on the device: + +* ``nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, voxelSize, stream)`` + rasterizes ``(N, 3)`` int32 INDEX-space voxel coordinates into a + device ``OnIndexGrid`` handle. +* ``nanovdb.tools.cuda.pointsToGrid(points, voxelSize, stream)`` + rasterizes ``(N, 3)`` float WORLD-space positions into a device + ``NanoGrid`` (the point coordinates are stored as blind data). + +Key facts demonstrated: + +* The returned handle is already DEVICE-resident: ``deviceGrid(0)`` is + valid immediately with no ``deviceUpload``. ``grid(0)`` (the host + grid) is ``None`` until you call ``deviceDownload``. +* The device grid feeds straight into other ``nanovdb.tools.cuda.*`` + ops (here, device validation). + +Requires CuPy and a CUDA-capable GPU. + +Run with: python voxels_to_grid_cuda.py +""" +import nanovdb + + +def main(): + if not (nanovdb.isCudaAvailable() and nanovdb.isGpuAvailable()): + print("This example requires a CUDA build of nanovdb and a GPU. " + "Skipping.") + return + try: + import cupy as cp + except ImportError: + print("This example requires CuPy. Install it with: pip install cupy") + return + + # Build the voxel coordinates of a hollow sphere shell directly on the + # GPU with CuPy, so nothing round-trips through the host. + radius = 32.0 + lin = cp.arange(-40, 41, dtype=cp.float32) + x, y, z = cp.meshgrid(lin, lin, lin, indexing="ij") + r = cp.sqrt(x * x + y * y + z * z) + shell = (cp.abs(r - radius) < 1.0) + coords = cp.stack([x[shell], y[shell], z[shell]], axis=1).astype(cp.int32) + coords = cp.ascontiguousarray(coords) + print(f"Generated {coords.shape[0]} shell voxel coords on the GPU") + + # Rasterize into a device OnIndex grid in one call. + handle = nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, 1.0, 0) + device_grid = handle.deviceGrid(0) + print(f"voxelsToOnIndexGrid -> {handle.gridType(0)} handle, " + f"device_grid={type(device_grid).__name__}") + print(f" grid(0) before download = {handle.grid(0)} (device-built)") + + # Validate entirely on the device, then download for host-side metadata. + print(f" tools.cuda.isValid(device_grid) = {nanovdb.tools.cuda.isValid(device_grid)}") + handle.deviceDownload(0, True) + host_grid = handle.grid(0) + active = host_grid.activeVoxelCount() + print(f" after deviceDownload: activeVoxelCount={active}, " + f"valueCount={host_grid.valueCount()}") + # OnIndex value 0 is the background slot, so valueCount == active + 1. + assert host_grid.valueCount() == active + 1 + assert active == int(coords.shape[0]) + + # pointsToGrid takes WORLD-space float positions and builds a Point grid. + world_points = coords.astype(cp.float32) * 0.5 # arbitrary world scale + world_points = cp.ascontiguousarray(world_points) + point_handle = nanovdb.tools.cuda.pointsToGrid(world_points, 0.5, 0) + print(f"pointsToGrid -> {point_handle.gridType(0)} handle " + f"(positions stored as blind data)") + assert nanovdb.tools.cuda.isValid(point_handle.deviceGrid(0)) + point_handle.deviceDownload(0, True) + point_active = point_handle.grid(0).activeVoxelCount() + print(f" point grid activeVoxelCount={point_active} (isValid on device)") + assert point_active > 0 + + print("OK: built OnIndex and Point grids on the device from CuPy arrays") + + +if __name__ == "__main__": + main() diff --git a/nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh b/nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh index 60bcc25d0d..1351c7e2cb 100644 --- a/nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh +++ b/nanovdb/nanovdb/tools/cuda/IndexToGrid.cuh @@ -146,6 +146,10 @@ __global__ void processGridTreeRootKernel(typename IndexToGrid::NodeA // process Grid *dstGrid.data() = *srcGrid.data(); dstGrid.mGridType = toGridType(); + // NOTE: The source is an index grid (GridClass::IndexGrid); a plain value + // grid must not inherit that class or it forms an invalid GridType/GridClass + // combination (e.g. Float + IndexGrid) that fails grid validation. + dstGrid.mGridClass = GridClass::Unknown; dstGrid.mData1 = 0u; dstGrid.mGridSize = nodeAcc->size; From 0bc279c38764b54f0acc613d47d2b6005c13f5a3 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 30 Jul 2026 15:58:34 +1200 Subject: [PATCH 40/48] fix example DLL lookup on Windows Run example smoke tests through a Windows-specific child launcher that registers dependent DLL directories before importing nanovdb. Keep the DLL directory handles alive while preserving subprocess isolation and normal script execution behavior. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/test/TestExamples.py | 47 +++++++++++++++++++-- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/nanovdb/nanovdb/python/test/TestExamples.py b/nanovdb/nanovdb/python/test/TestExamples.py index 3d6355b9fc..5bb9c1ef29 100644 --- a/nanovdb/nanovdb/python/test/TestExamples.py +++ b/nanovdb/nanovdb/python/test/TestExamples.py @@ -14,6 +14,7 @@ """ import os +import runpy import subprocess import sys import tempfile @@ -21,6 +22,7 @@ EXAMPLES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir, "examples") +RUN_EXAMPLE_ARG = "--run-example" EXAMPLE_SCRIPTS = [ "build_grid.py", @@ -44,17 +46,53 @@ class TestExamples(unittest.TestCase): pass +def _run_example(script): + """Run one example after configuring dependent-DLL lookup on Windows.""" + dll_directory_handles = [] + if hasattr(os, "add_dll_directory"): + for path in os.environ.get("PATH", "").split(os.pathsep): + if os.path.isdir(path): + try: + # Keep each handle alive until the example has finished. + dll_directory_handles.append(os.add_dll_directory(path)) + except OSError: + pass + + sys.argv = [script] + sys.path[0] = os.path.dirname(os.path.abspath(script)) + runpy.run_path(script, run_name="__main__") + + def _make_test(script_name): def test(self): script = os.path.join(EXAMPLES_DIR, script_name) self.assertTrue(os.path.isfile(script), f"example script is missing: {script}") + command = [sys.executable, script] + env = os.environ.copy() + if hasattr(os, "add_dll_directory"): + # Match TestNanoVDB.py's in-tree OpenVDB DLL lookup. The child + # starts in a temporary directory, so resolve this while the + # parent is still in the CMake configuration directory. + config = os.path.basename(os.getcwd()) + openvdb_dll_directory = os.path.abspath(os.path.join( + os.getcwd(), os.pardir, os.pardir, os.pardir, os.pardir, + "openvdb", "openvdb", config)) + env["PATH"] = os.pathsep.join( + (openvdb_dll_directory, env.get("PATH", ""))) + # os.add_dll_directory() registrations are process-local, so run + # through this file's child mode to register them in the process + # that imports nanovdb. + command = [ + sys.executable, os.path.abspath(__file__), + RUN_EXAMPLE_ARG, script, + ] # A fresh cwd per run keeps any output files out of the source # tree (the examples themselves also write to tempdirs). result = subprocess.run( - [sys.executable, script], + command, cwd=tempfile.mkdtemp(prefix="nanovdb_example_"), - env=os.environ.copy(), + env=env, capture_output=True, text=True, timeout=120, @@ -74,4 +112,7 @@ def test(self): if __name__ == "__main__": - unittest.main() + if len(sys.argv) == 3 and sys.argv[1] == RUN_EXAMPLE_ARG: + _run_example(sys.argv[2]) + else: + unittest.main() From 4a97f065af888fcdf0d3c438cc16a011fb8ce1b7 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 30 Jul 2026 17:10:53 +1200 Subject: [PATCH 41/48] fix Windows example test startup Register dependent DLL directories through a test-only sitecustomize module so example smoke tests can run as normal Python scripts. This avoids the native access violations caused by executing them through runpy while preserving subprocess isolation. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/test/TestExamples.py | 38 +++++--------------- nanovdb/nanovdb/python/test/sitecustomize.py | 18 ++++++++++ 2 files changed, 26 insertions(+), 30 deletions(-) create mode 100644 nanovdb/nanovdb/python/test/sitecustomize.py diff --git a/nanovdb/nanovdb/python/test/TestExamples.py b/nanovdb/nanovdb/python/test/TestExamples.py index 5bb9c1ef29..93789f337a 100644 --- a/nanovdb/nanovdb/python/test/TestExamples.py +++ b/nanovdb/nanovdb/python/test/TestExamples.py @@ -14,7 +14,6 @@ """ import os -import runpy import subprocess import sys import tempfile @@ -22,7 +21,6 @@ EXAMPLES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir, "examples") -RUN_EXAMPLE_ARG = "--run-example" EXAMPLE_SCRIPTS = [ "build_grid.py", @@ -46,23 +44,6 @@ class TestExamples(unittest.TestCase): pass -def _run_example(script): - """Run one example after configuring dependent-DLL lookup on Windows.""" - dll_directory_handles = [] - if hasattr(os, "add_dll_directory"): - for path in os.environ.get("PATH", "").split(os.pathsep): - if os.path.isdir(path): - try: - # Keep each handle alive until the example has finished. - dll_directory_handles.append(os.add_dll_directory(path)) - except OSError: - pass - - sys.argv = [script] - sys.path[0] = os.path.dirname(os.path.abspath(script)) - runpy.run_path(script, run_name="__main__") - - def _make_test(script_name): def test(self): script = os.path.join(EXAMPLES_DIR, script_name) @@ -80,13 +61,13 @@ def test(self): "openvdb", "openvdb", config)) env["PATH"] = os.pathsep.join( (openvdb_dll_directory, env.get("PATH", ""))) - # os.add_dll_directory() registrations are process-local, so run - # through this file's child mode to register them in the process - # that imports nanovdb. - command = [ - sys.executable, os.path.abspath(__file__), - RUN_EXAMPLE_ARG, script, - ] + # Python imports sitecustomize during startup. Add this test + # directory to the child's module path so sitecustomize.py can + # register the DLL directories before the example imports + # nanovdb, while still running the example as a normal script. + test_directory = os.path.dirname(os.path.abspath(__file__)) + env["PYTHONPATH"] = os.pathsep.join( + (test_directory, env.get("PYTHONPATH", ""))) # A fresh cwd per run keeps any output files out of the source # tree (the examples themselves also write to tempdirs). result = subprocess.run( @@ -112,7 +93,4 @@ def test(self): if __name__ == "__main__": - if len(sys.argv) == 3 and sys.argv[1] == RUN_EXAMPLE_ARG: - _run_example(sys.argv[2]) - else: - unittest.main() + unittest.main() diff --git a/nanovdb/nanovdb/python/test/sitecustomize.py b/nanovdb/nanovdb/python/test/sitecustomize.py new file mode 100644 index 0000000000..5e531634cf --- /dev/null +++ b/nanovdb/nanovdb/python/test/sitecustomize.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python +# Copyright Contributors to the OpenVDB Project +# SPDX-License-Identifier: Apache-2.0 +"""Configure dependent-DLL lookup for in-tree Windows Python tests.""" + +import os + + +# Keep these handles alive for the lifetime of the child Python process. +_dll_directory_handles = [] + +if hasattr(os, "add_dll_directory"): + for _path in os.environ.get("PATH", "").split(os.pathsep): + if os.path.isdir(_path): + try: + _dll_directory_handles.append(os.add_dll_directory(_path)) + except OSError: + pass From e72bf3e86faa86e5059361fc9f2bfa66720b65e9 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Thu, 30 Jul 2026 21:41:57 +1200 Subject: [PATCH 42/48] diagnose Windows example crashes Enable Python's fault handler and unbuffered output for example subprocesses so Windows access violations report the active Python frame and preserve output leading up to the crash. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/test/TestExamples.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/nanovdb/nanovdb/python/test/TestExamples.py b/nanovdb/nanovdb/python/test/TestExamples.py index 93789f337a..8a122601f1 100644 --- a/nanovdb/nanovdb/python/test/TestExamples.py +++ b/nanovdb/nanovdb/python/test/TestExamples.py @@ -51,6 +51,10 @@ def test(self): f"example script is missing: {script}") command = [sys.executable, script] env = os.environ.copy() + # Preserve output up to a native crash and ask Python to report the + # active frame for fatal signals and Windows exceptions. + env["PYTHONFAULTHANDLER"] = "1" + env["PYTHONUNBUFFERED"] = "1" if hasattr(os, "add_dll_directory"): # Match TestNanoVDB.py's in-tree OpenVDB DLL lookup. The child # starts in a temporary directory, so resolve this while the From 9102630227e2c62fba286ce7b56f9deabc792caa Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Fri, 31 Jul 2026 09:32:40 +1200 Subject: [PATCH 43/48] fix Windows example DLL search Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/test/TestExamples.py | 5 ++--- nanovdb/nanovdb/python/test/sitecustomize.py | 16 ++++++++-------- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/nanovdb/nanovdb/python/test/TestExamples.py b/nanovdb/nanovdb/python/test/TestExamples.py index 8a122601f1..6d8a1eb06c 100644 --- a/nanovdb/nanovdb/python/test/TestExamples.py +++ b/nanovdb/nanovdb/python/test/TestExamples.py @@ -63,11 +63,10 @@ def test(self): openvdb_dll_directory = os.path.abspath(os.path.join( os.getcwd(), os.pardir, os.pardir, os.pardir, os.pardir, "openvdb", "openvdb", config)) - env["PATH"] = os.pathsep.join( - (openvdb_dll_directory, env.get("PATH", ""))) + env["NANOVDB_TEST_DLL_DIRECTORY"] = openvdb_dll_directory # Python imports sitecustomize during startup. Add this test # directory to the child's module path so sitecustomize.py can - # register the DLL directories before the example imports + # register the OpenVDB DLL directory before the example imports # nanovdb, while still running the example as a normal script. test_directory = os.path.dirname(os.path.abspath(__file__)) env["PYTHONPATH"] = os.pathsep.join( diff --git a/nanovdb/nanovdb/python/test/sitecustomize.py b/nanovdb/nanovdb/python/test/sitecustomize.py index 5e531634cf..305163a84e 100644 --- a/nanovdb/nanovdb/python/test/sitecustomize.py +++ b/nanovdb/nanovdb/python/test/sitecustomize.py @@ -6,13 +6,13 @@ import os -# Keep these handles alive for the lifetime of the child Python process. -_dll_directory_handles = [] +# Keep this handle alive for the lifetime of the child Python process. +_dll_directory_handle = None if hasattr(os, "add_dll_directory"): - for _path in os.environ.get("PATH", "").split(os.pathsep): - if os.path.isdir(_path): - try: - _dll_directory_handles.append(os.add_dll_directory(_path)) - except OSError: - pass + _path = os.environ.get("NANOVDB_TEST_DLL_DIRECTORY") + if _path and os.path.isdir(_path): + try: + _dll_directory_handle = os.add_dll_directory(_path) + except OSError: + pass From 3296fd06e24da7c440202d5e4e2ab88635cceb90 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Fri, 31 Jul 2026 11:12:12 +1200 Subject: [PATCH 44/48] add CUDA DLL path to Windows examples Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/test/TestExamples.py | 12 +++++++++--- nanovdb/nanovdb/python/test/sitecustomize.py | 17 +++++++++-------- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/nanovdb/nanovdb/python/test/TestExamples.py b/nanovdb/nanovdb/python/test/TestExamples.py index 6d8a1eb06c..abc3584dc5 100644 --- a/nanovdb/nanovdb/python/test/TestExamples.py +++ b/nanovdb/nanovdb/python/test/TestExamples.py @@ -63,11 +63,17 @@ def test(self): openvdb_dll_directory = os.path.abspath(os.path.join( os.getcwd(), os.pardir, os.pardir, os.pardir, os.pardir, "openvdb", "openvdb", config)) - env["NANOVDB_TEST_DLL_DIRECTORY"] = openvdb_dll_directory + dll_directories = [openvdb_dll_directory] + cuda_root = env.get("CUDA_PATH") + if cuda_root: + dll_directories.append(os.path.join(cuda_root, "bin")) + env["NANOVDB_TEST_DLL_DIRECTORIES"] = os.pathsep.join( + dll_directories) # Python imports sitecustomize during startup. Add this test # directory to the child's module path so sitecustomize.py can - # register the OpenVDB DLL directory before the example imports - # nanovdb, while still running the example as a normal script. + # register only the required OpenVDB and CUDA DLL directories + # before the example imports nanovdb, while still running the + # example as a normal script. test_directory = os.path.dirname(os.path.abspath(__file__)) env["PYTHONPATH"] = os.pathsep.join( (test_directory, env.get("PYTHONPATH", ""))) diff --git a/nanovdb/nanovdb/python/test/sitecustomize.py b/nanovdb/nanovdb/python/test/sitecustomize.py index 305163a84e..e282fd4fca 100644 --- a/nanovdb/nanovdb/python/test/sitecustomize.py +++ b/nanovdb/nanovdb/python/test/sitecustomize.py @@ -6,13 +6,14 @@ import os -# Keep this handle alive for the lifetime of the child Python process. -_dll_directory_handle = None +# Keep these handles alive for the lifetime of the child Python process. +_dll_directory_handles = [] if hasattr(os, "add_dll_directory"): - _path = os.environ.get("NANOVDB_TEST_DLL_DIRECTORY") - if _path and os.path.isdir(_path): - try: - _dll_directory_handle = os.add_dll_directory(_path) - except OSError: - pass + for _path in os.environ.get( + "NANOVDB_TEST_DLL_DIRECTORIES", "").split(os.pathsep): + if os.path.isdir(_path): + try: + _dll_directory_handles.append(os.add_dll_directory(_path)) + except OSError: + pass From 5fccbab19a720cb47090a5eec13c36fb5e6ac408 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 19 Aug 2026 10:49:34 +1200 Subject: [PATCH 45/48] nanovdb python: use camelCase consistently for bound names MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Python bindings had drifted into two naming conventions. The method and function surface was already 208 camelCase names to 6 snake_case, but parameter names were split almost evenly (27 camelCase, 24 snake_case), with the snake_case ones confined to the CUDA device bindings and PyVoxelBlockManager.cc while the host bindings used camelCase throughout. That put the same concept under two spellings in one namespace, e.g. createNanoGridOnIndex(includeStats=) next to createOnIndexGrid(include_stats=), and left three files internally mixed. Rename the snake_case parameters (d_grid, log2_block_width, include_stats, first_offset, src_grid, leaf_masks, ...) and the five snake_case method/property outliers (block_width, log2_block_width, jump_map_length, leaf_values, is_divisible) to camelCase, and update every call site in the examples and tests along with the prose that referenced them. The interop surface keeps its snake_case names on purpose: data_ptr, device_ptr, host_ptr, from_external, from_buffer, jump_map_ptr and first_leaf_id_ptr sit in the zero-copy CuPy / PyTorch / Numba path, where data_ptr() is the spelling those libraries use. Default grid-name values such as "sphere_ls" are data rather than identifiers and are untouched. The bindings have never appeared in a release (they landed on master after v13.0.0), so no deprecation aliases are needed. Verified with TestNanoVDB.py (166), TestGpuInterop.py (50, 31 skipped for absent torch/cupy) and TestExamples.py (14) — all passing. Co-Authored-By: Claude Fable 5 Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/PyMath.cc | 2 +- nanovdb/nanovdb/python/PyTree.h | 2 +- nanovdb/nanovdb/python/PyVoxelBlockManager.cc | 168 +++++++++--------- nanovdb/nanovdb/python/cuda/PyAddBlindData.cu | 6 +- nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu | 6 +- nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc | 22 +-- .../python/cuda/PyDeviceGridChecksum.cu | 32 ++-- .../python/cuda/PyDeviceGridChecksum.h | 2 +- .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 12 +- .../nanovdb/python/cuda/PyDeviceGridStats.cu | 6 +- .../python/cuda/PyDeviceGridValidator.cu | 6 +- .../python/cuda/PyDeviceNodeManager.cu | 14 +- .../python/cuda/PyDeviceVoxelBlockManager.cu | 126 ++++++------- nanovdb/nanovdb/python/cuda/PyDilateGrid.cu | 6 +- nanovdb/nanovdb/python/cuda/PyInjectData.cu | 82 ++++----- nanovdb/nanovdb/python/cuda/PyMergeGrids.cu | 12 +- nanovdb/nanovdb/python/cuda/PyPruneGrid.cu | 6 +- nanovdb/nanovdb/python/cuda/PyRefineGrid.cu | 6 +- .../nanovdb/python/cuda/PySampleFromVoxels.cu | 26 +-- .../nanovdb/python/cuda/PySignedFloodFill.cu | 6 +- nanovdb/nanovdb/python/examples/build_grid.py | 2 +- .../python/examples/bulk_leaf_numpy.py | 10 +- .../python/examples/collide_level_set_cuda.py | 6 +- .../nanovdb/python/examples/cupy_rawkernel.py | 14 +- .../python/examples/device_topology_ops.py | 14 +- .../python/examples/gpu_load_inspect.py | 12 +- .../python/examples/index_grid_channels.py | 10 +- .../python/examples/levelset_filter.py | 18 +- .../examples/levelset_filter_rawkernel.py | 2 +- .../nanovdb/python/examples/node_manager.py | 2 +- nanovdb/nanovdb/python/examples/quantize.py | 10 +- .../examples/raytrace_fog_volume_cuda.py | 10 +- .../examples/raytrace_level_set_cuda.py | 10 +- .../examples/sample_from_voxels_cuda.py | 8 +- .../python/examples/signed_flood_fill_cuda.py | 10 +- .../nanovdb/python/examples/validate_cuda.py | 16 +- .../python/examples/voxels_to_grid_cuda.py | 6 +- nanovdb/nanovdb/python/test/TestGpuInterop.py | 10 +- nanovdb/nanovdb/python/test/TestNanoVDB.py | 68 +++---- 39 files changed, 393 insertions(+), 393 deletions(-) diff --git a/nanovdb/nanovdb/python/PyMath.cc b/nanovdb/nanovdb/python/PyMath.cc index 2e212f09b9..d759da8a34 100644 --- a/nanovdb/nanovdb/python/PyMath.cc +++ b/nanovdb/nanovdb/python/PyMath.cc @@ -486,7 +486,7 @@ template void defineBBoxInteger(nb::module_& m, const char* nam "Construct a cube of side dim voxels anchored at the min Coord.") .def_static("createCube", nb::overload_cast(&math::BBox::createCube), "min"_a, "max"_a, "Construct a cube spanning [min, max] in every axis.") - .def("is_divisible", &math::BBox::is_divisible, + .def("isDivisible", &math::BBox::is_divisible, "True iff this CoordBBox has more than one voxel in every axis.") .def("empty", &math::BBox::empty, "True iff this CoordBBox is empty (any min component > the matching max).") diff --git a/nanovdb/nanovdb/python/PyTree.h b/nanovdb/nanovdb/python/PyTree.h index d695ac85a2..5aa7c4e97f 100644 --- a/nanovdb/nanovdb/python/PyTree.h +++ b/nanovdb/nanovdb/python/PyTree.h @@ -444,7 +444,7 @@ struct PyLeafValuesBinder; using LeafT = nanovdb::NanoLeaf; using ValueT = typename LeafT::ValueType; - cls.def("leaf_values", + cls.def("leafValues", [](nb::handle py_self) -> nb::object { auto& grid = nb::cast(py_self); const auto& tree = grid.tree(); diff --git a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc index 5e61913050..3fa5de928e 100644 --- a/nanovdb/nanovdb/python/PyVoxelBlockManager.cc +++ b/nanovdb/nanovdb/python/PyVoxelBlockManager.cc @@ -44,15 +44,15 @@ static auto dispatchLog2BlockWidth(int log2BlockWidth, F&& fn) case 9: return fn(std::integral_constant{}); default: throw nb::value_error( - "VoxelBlockManager: log2_block_width must be 6, 7, 8, or 9 " + "VoxelBlockManager: log2BlockWidth must be 6, 7, 8, or 9 " "(BlockWidth = 64, 128, 256, or 512). Larger widths are not " "bound in Python by default."); } } // PyVBMHandle wraps the C++ VoxelBlockManagerHandle and carries the -// log2_block_width the handle was built with. The C++ handle does NOT store -// log2_block_width itself, so without this wrapper the Python binding would +// log2BlockWidth the handle was built with. The C++ handle does NOT store +// log2BlockWidth itself, so without this wrapper the Python binding would // have to ask the caller every time — which the user can lie about and // trigger out-of-bounds reads of the metadata buffers. Storing it once at // build time and consulting it in every accessor closes that hole. @@ -188,12 +188,12 @@ static void defineHandle(nb::module_& toolsModule) "by this handle.") .def("reset", &PyVBMHandle::reset, "Release this handle's buffers and reset it to the empty state.") - .def_prop_ro("log2_block_width", [](const PyVBMHandle& h) { return h.log2BlockWidth; }, - "The log2_block_width this handle was built with. The jumpMap " + .def_prop_ro("log2BlockWidth", [](const PyVBMHandle& h) { return h.log2BlockWidth; }, + "The log2BlockWidth this handle was built with. The jumpMap " "and decodeBlock outputs derive their shapes from this value.") - .def_prop_ro("block_width", &PyVBMHandle::blockWidth, - "BlockWidth = 1 << log2_block_width (64, 128, 256, or 512).") - .def_prop_ro("jump_map_length", &PyVBMHandle::jumpMapLength, + .def_prop_ro("blockWidth", &PyVBMHandle::blockWidth, + "BlockWidth = 1 << log2BlockWidth (64, 128, 256, or 512).") + .def_prop_ro("jumpMapLength", &PyVBMHandle::jumpMapLength, "JumpMapLength = BlockWidth / 64 (1, 2, 4, or 8).") .def( "__bool__", @@ -224,7 +224,7 @@ static void defineHandle(nb::module_& toolsModule) "default-constructed or reset() handle. The view keeps this " "handle alive.") // jumpMap is uint64_t[blockCount * JumpMapLength]. JumpMapLength is - // determined by the log2_block_width recorded on the handle, not by + // determined by the log2BlockWidth recorded on the handle, not by // the caller — that way the returned view always covers exactly the // allocated buffer, with no risk of OOB reads. .def("jumpMap", @@ -248,22 +248,22 @@ static void defineHandle(nb::module_& toolsModule) nb::keep_alive<0, 1>(), "Return a zero-copy (blockCount, jump_map_length) uint64 NumPy " "view of the jumpMap. The shape is determined by the " - "log2_block_width the handle was built with. Returns an empty " + "log2BlockWidth the handle was built with. Returns an empty " "(0, jump_map_length) array on a default-constructed or reset() " "handle. The view keeps this handle alive.") // Decode the inverse maps for a single block of this VBM. The - // log2_block_width is taken from the handle, so the caller cannot + // log2BlockWidth is taken from the handle, so the caller cannot // request a width that doesn't match what was built. .def("decodeBlock", [](PyVBMHandle& self, nb::handle py_grid, - uint64_t block_index) -> nb::object { + uint64_t blockIndex) -> nb::object { const auto* grid = castOnIndexGrid(py_grid, "VoxelBlockManagerHandle.decodeBlock"); - if (block_index >= self.blockCount()) { + if (blockIndex >= self.blockCount()) { throw nb::index_error( - "VoxelBlockManagerHandle.decodeBlock(block_index): " - "block_index out of range [0, blockCount)."); + "VoxelBlockManagerHandle.decodeBlock(blockIndex): " + "blockIndex out of range [0, blockCount)."); } // Defensive: NanoVDB's buildVoxelBlockManager doesn't always // initialize firstLeafID for blocks where no leaf starts at @@ -275,7 +275,7 @@ static void defineHandle(nb::module_& toolsModule) // read of tree.getFirstNode<0>()[garbage]. Catch the case // and raise rather than segfault. const uint32_t firstLeafID = - self.handle.hostFirstLeafID()[block_index]; + self.handle.hostFirstLeafID()[blockIndex]; const uint32_t nLeaves = grid->tree().nodeCount(0); if (firstLeafID >= nLeaves) { throw nb::value_error( @@ -295,19 +295,19 @@ static void defineHandle(nb::module_& toolsModule) constexpr int JumpMapLength = VoxelBlockManagerBase::JumpMapLength; const uint64_t blockFirstOffset = - self.firstOffset() + block_index * BlockWidth; + self.firstOffset() + blockIndex * BlockWidth; return pyDecodeInverseMapsImpl( *grid, firstLeafID, - self.handle.hostJumpMap() + block_index * JumpMapLength, + self.handle.hostJumpMap() + blockIndex * JumpMapLength, blockFirstOffset); }); }, - "grid"_a, "block_index"_a, - "Decode the inverse maps for the block_index-th block of this " + "grid"_a, "blockIndex"_a, + "Decode the inverse maps for the blockIndex-th block of this " "VBM. Returns (leaf_index, voxel_offset) uint32 / uint16 NumPy " - "arrays of length BlockWidth = 1< PyVBMHandle { + int log2BlockWidth, + uint64_t firstOffset, + uint64_t lastOffset, + uint64_t nBlocks) -> PyVBMHandle { const auto* grid = castOnIndexGrid(py_grid, "buildVoxelBlockManager"); // The C++ implementation only NANOVDB_ASSERTs these preconditions, // which makes them no-ops in release builds. Validate them here @@ -331,25 +331,25 @@ static void defineBuild(nb::module_& toolsModule) "layout). NanoVDB grids constructed via " "tools.createOnIndexGrid satisfy this by default."); } - return dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + return dispatchLog2BlockWidth(log2BlockWidth, [&](auto W) { constexpr int LBW = decltype(W)::value; using Base = VoxelBlockManagerBase; constexpr uint64_t BlockWidth = Base::BlockWidth; constexpr uint64_t JumpMapLength = Base::JumpMapLength; - // first_offset must be 1 (mod BlockWidth). The single-arg + // firstOffset must be 1 (mod BlockWidth). The single-arg // C++ helper would normalize a zero input to 1; we do the // same here so the in-place builder below sees a valid // value. Validate the nonzero case ourselves. - if (first_offset != 0 && - ((first_offset - 1) & (BlockWidth - 1)) != 0) { + if (firstOffset != 0 && + ((firstOffset - 1) & (BlockWidth - 1)) != 0) { throw nb::value_error( - "buildVoxelBlockManager: first_offset must satisfy " - "first_offset == 1 (mod BlockWidth). Pass 0 (the " + "buildVoxelBlockManager: firstOffset must satisfy " + "firstOffset == 1 (mod BlockWidth). Pass 0 (the " "default) to let the implementation use 1."); } - if (first_offset == 0) first_offset = 1; - if (last_offset == 0) last_offset = grid->activeVoxelCount(); - if (last_offset < first_offset) return PyVBMHandle(); + if (firstOffset == 0) firstOffset = 1; + if (lastOffset == 0) lastOffset = grid->activeVoxelCount(); + if (lastOffset < firstOffset) return PyVBMHandle(); // Capacity must hold at least ceil((last - first + 1) / // BlockWidth) blocks; otherwise the handle's lastOffset // would advertise more coverage than blockCount allows @@ -357,18 +357,18 @@ static void defineBuild(nb::module_& toolsModule) // below equals the ceil() above when BlockWidth is a // power of two. const uint64_t minBlocks = - (last_offset - first_offset + BlockWidth) >> LBW; - if (n_blocks != 0 && n_blocks < minBlocks) { + (lastOffset - firstOffset + BlockWidth) >> LBW; + if (nBlocks != 0 && nBlocks < minBlocks) { std::string msg( - "buildVoxelBlockManager: n_blocks must be at " - "least ceil((last_offset - first_offset + 1) / " + "buildVoxelBlockManager: nBlocks must be at " + "least ceil((lastOffset - firstOffset + 1) / " "BlockWidth) = "); msg += std::to_string(minBlocks); msg += ". Pass 0 (the default) to use the minimum " "required capacity."; throw nb::value_error(msg.c_str()); } - if (n_blocks == 0) n_blocks = minBlocks; + if (nBlocks == 0) nBlocks = minBlocks; // Allocate the metadata buffers ourselves so we can // pre-initialize firstLeafID with a sentinel value before // the in-place builder runs. The C++ allocating overload @@ -380,20 +380,20 @@ static void defineBuild(nb::module_& toolsModule) // up front, every untouched slot deterministically trips // the guard. auto firstLeafIDBuf = HostBuffer::create( - n_blocks * sizeof(uint32_t)); + nBlocks * sizeof(uint32_t)); auto jumpMapBuf = HostBuffer::create( - n_blocks * JumpMapLength * sizeof(uint64_t)); + nBlocks * JumpMapLength * sizeof(uint64_t)); const uint32_t nLeaves = grid->tree().nodeCount(0); { uint32_t* slots = static_cast( firstLeafIDBuf.data()); - for (uint64_t i = 0; i < n_blocks; ++i) { + for (uint64_t i = 0; i < nBlocks; ++i) { slots[i] = nLeaves; } } VoxelBlockManagerHandle handle( std::move(firstLeafIDBuf), std::move(jumpMapBuf), - n_blocks, first_offset, last_offset); + nBlocks, firstOffset, lastOffset); // In-place builder zeros the jumpMap itself and only // touches firstLeafID slots it actually visits. Release the // GIL around it — it's pure C++ (touches no Python objects) @@ -406,16 +406,16 @@ static void defineBuild(nb::module_& toolsModule) }); }, "grid"_a, - "log2_block_width"_a = 6, - "first_offset"_a = 0, - "last_offset"_a = 0, - "n_blocks"_a = 0, + "log2BlockWidth"_a = 6, + "firstOffset"_a = 0, + "lastOffset"_a = 0, + "nBlocks"_a = 0, "Build a host-side VoxelBlockManager from an OnIndexGrid. " - "log2_block_width selects the per-block active-voxel count " - "(6=64, 7=128, 8=256, 9=512). Pass 0 for first_offset / " - "last_offset / n_blocks to use the full grid (first active " + "log2BlockWidth selects the per-block active-voxel count " + "(6=64, 7=128, 8=256, 9=512). Pass 0 for firstOffset / " + "lastOffset / nBlocks to use the full grid (first active " "voxel through grid.activeVoxelCount(), minimum block count). " - "first_offset, if nonzero, must satisfy first_offset == 1 " + "firstOffset, if nonzero, must satisfy firstOffset == 1 " "(mod BlockWidth)."); } @@ -425,14 +425,14 @@ static void defineDecode(nb::module_& toolsModule) { toolsModule.def("decodeInverseMaps", [](nb::handle py_grid, - uint32_t first_leaf_id, + uint32_t firstLeafId, nb::ndarray, - nb::c_contig, nb::device::cpu> jump_map, - uint64_t block_first_offset, - int log2_block_width) -> nb::object { + nb::c_contig, nb::device::cpu> jumpMap, + uint64_t blockFirstOffset, + int log2BlockWidth) -> nb::object { const auto* grid = castOnIndexGrid(py_grid, "decodeInverseMaps"); - // The C++ helper indexes tree.getFirstNode<0>()[first_leaf_id] - // without a bounds check, so a stray first_leaf_id leads to an + // The C++ helper indexes tree.getFirstNode<0>()[firstLeafId] + // without a bounds check, so a stray firstLeafId leads to an // OOB read. Validate up front. (We also require isSequential(); // getFirstNode only makes sense on a sequential tree.) if (!grid->isSequential()) { @@ -441,37 +441,37 @@ static void defineDecode(nb::module_& toolsModule) "grid.isSequential()."); } const uint32_t nLeaves = grid->tree().nodeCount(0); - if (first_leaf_id >= nLeaves) { + if (firstLeafId >= nLeaves) { throw nb::index_error( - "decodeInverseMaps: first_leaf_id out of range " + "decodeInverseMaps: firstLeafId out of range " "[0, grid.tree().nodeCount(0))."); } - return dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + return dispatchLog2BlockWidth(log2BlockWidth, [&](auto W) { constexpr int LBW = decltype(W)::value; constexpr int JumpMapLength = VoxelBlockManagerBase::JumpMapLength; - if (jump_map.shape(0) != JumpMapLength) { - std::string msg("decodeInverseMaps: jump_map must have " + if (jumpMap.shape(0) != JumpMapLength) { + std::string msg("decodeInverseMaps: jumpMap must have " "length JumpMapLength = "); msg += std::to_string(JumpMapLength); - msg += " for log2_block_width="; + msg += " for log2BlockWidth="; msg += std::to_string(LBW); throw nb::value_error(msg.c_str()); } return pyDecodeInverseMapsImpl( - *grid, first_leaf_id, jump_map.data(), - block_first_offset); + *grid, firstLeafId, jumpMap.data(), + blockFirstOffset); }); }, "grid"_a, - "first_leaf_id"_a, - "jump_map"_a, - "block_first_offset"_a, - "log2_block_width"_a = 6, + "firstLeafId"_a, + "jumpMap"_a, + "blockFirstOffset"_a, + "log2BlockWidth"_a = 6, "Decode the inverse maps for a single voxel block of an OnIndexGrid. " "Returns a (leaf_index, voxel_offset) tuple of fresh NumPy arrays of " - "length BlockWidth = 1< static nb::object tryCreateOnIndexGrid(nb::handle py_grid, uint32_t channels, - bool include_stats, - bool include_tiles, + bool includeStats, + bool includeTiles, int verbose) { using SrcGridT = NanoGrid; @@ -494,7 +494,7 @@ static nb::object tryCreateOnIndexGrid(nb::handle py_grid, const SrcGridT& src = nb::cast(py_grid); return nb::cast( tools::createNanoGrid( - src, channels, include_stats, include_tiles, verbose)); + src, channels, includeStats, includeTiles, verbose)); } static void defineCreateOnIndexGrid(nb::module_& toolsModule) @@ -502,31 +502,31 @@ static void defineCreateOnIndexGrid(nb::module_& toolsModule) toolsModule.def("createOnIndexGrid", [](nb::handle py_grid, uint32_t channels, - bool include_stats, - bool include_tiles, + bool includeStats, + bool includeTiles, int verbose) -> nb::object { // Try every source BuildT we accept. if (auto r = tryCreateOnIndexGrid( - py_grid, channels, include_stats, include_tiles, verbose); + py_grid, channels, includeStats, includeTiles, verbose); r.is_valid()) return r; if (auto r = tryCreateOnIndexGrid( - py_grid, channels, include_stats, include_tiles, verbose); + py_grid, channels, includeStats, includeTiles, verbose); r.is_valid()) return r; if (auto r = tryCreateOnIndexGrid( - py_grid, channels, include_stats, include_tiles, verbose); + py_grid, channels, includeStats, includeTiles, verbose); r.is_valid()) return r; if (auto r = tryCreateOnIndexGrid( - py_grid, channels, include_stats, include_tiles, verbose); + py_grid, channels, includeStats, includeTiles, verbose); r.is_valid()) return r; throw nb::type_error( "createOnIndexGrid: source grid must be a FloatGrid, " "DoubleGrid, Int32Grid, or Vec3fGrid (other source BuildTs " "are not yet bound)."); }, - "src_grid"_a, + "srcGrid"_a, "channels"_a = 0u, - "include_stats"_a = true, - "include_tiles"_a = true, + "includeStats"_a = true, + "includeTiles"_a = true, "verbose"_a = 0, "Convert a source grid into a NanoGrid " "(OnIndexGrid). Accepts FloatGrid / DoubleGrid / Int32Grid / " diff --git a/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu b/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu index 259568d2b5..bf5d2ad088 100644 --- a/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu +++ b/nanovdb/nanovdb/python/cuda/PyAddBlindData.cu @@ -24,7 +24,7 @@ void defineAddBlindData(nb::module_& m, const char* name) { m.def( name, - [](nanovdb::NanoGrid* d_grid, + [](nanovdb::NanoGrid* dGrid, nb::ndarray, nb::c_contig, nb::device::cuda> blindData, nanovdb::GridBlindDataClass blindClass, nanovdb::GridBlindDataSemantic semantics, @@ -41,10 +41,10 @@ void defineAddBlindData(nb::module_& m, const char* name) // across the GIL release for the duration of this call frame. nb::gil_scoped_release release; return nanovdb::tools::cuda::addBlindData( - d_grid, d_blindData, valueCount, blindClass, semantics, + dGrid, d_blindData, valueCount, blindClass, semantics, dataName.c_str(), nanovdb::cuda::DeviceBuffer(), s); }, - "d_grid"_a, + "dGrid"_a, "blindData"_a, "blindClass"_a = nanovdb::GridBlindDataClass::Unknown, "semantics"_a = nanovdb::GridBlindDataSemantic::Unknown, diff --git a/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu b/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu index e76302358a..aa4e5200df 100644 --- a/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyCoarsenGrid.cu @@ -19,15 +19,15 @@ template void defineCoarsenGrid(nb::module_& m, const char* nam { m.def( name, - [](nanovdb::NanoGrid* d_grid, uintptr_t stream) { + [](nanovdb::NanoGrid* dGrid, uintptr_t stream) { cudaStream_t s = reinterpret_cast(stream); // CoarsenGrid::getHandle launches kernels and synchronizes the // stream; pure CUDA touching no Python objects, so release the GIL. nb::gil_scoped_release release; - nanovdb::tools::cuda::CoarsenGrid coarsener(d_grid, s); + nanovdb::tools::cuda::CoarsenGrid coarsener(dGrid, s); return coarsener.getHandle(); }, - "d_grid"_a, + "dGrid"_a, "stream"_a = 0, "Topologically coarsen (2x downsample) a device OnIndex grid and return " "a fresh device GridHandle of the coarsened grid. stream is a raw CUDA " diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc index d5533cb436..fad468080b 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc @@ -20,29 +20,29 @@ void defineDeviceBuffer(nb::module_& m) defineDeviceBufferLike(m, "DeviceBuffer") .def_static( "from_external", - [](uint64_t size, uintptr_t gpu_ptr, uintptr_t cpu_ptr) { + [](uint64_t size, uintptr_t gpuPtr, uintptr_t cpuPtr) { // Wrap externally-managed host + device memory in a NON-OWNING // DeviceBuffer (mManaged == 0). The buffer will NOT free either // pointer on destruction, upload, or download — the caller // retains ownership of both allocations. - if (gpu_ptr == 0) + if (gpuPtr == 0) throw nb::value_error( - "from_external: gpu_ptr must be a non-null device pointer."); - if (cpu_ptr == 0) + "from_external: gpuPtr must be a non-null device pointer."); + if (cpuPtr == 0) throw nb::value_error( - "from_external: cpu_ptr must be a non-null host pointer; the " + "from_external: cpuPtr must be a non-null host pointer; the " "externally-managed DeviceBuffer constructor requires both a " "host and a device pointer."); return BufferT::create(size, - reinterpret_cast(cpu_ptr), - reinterpret_cast(gpu_ptr)); + reinterpret_cast(cpuPtr), + reinterpret_cast(gpuPtr)); }, "size"_a, - "gpu_ptr"_a, - "cpu_ptr"_a, + "gpuPtr"_a, + "cpuPtr"_a, "Wrap externally-managed host and device memory in a NON-OWNING " - "DeviceBuffer. size is the byte size of both allocations; gpu_ptr " - "and cpu_ptr are raw pointers (Python ints). The returned buffer " + "DeviceBuffer. size is the byte size of both allocations; gpuPtr " + "and cpuPtr are raw pointers (Python ints). The returned buffer " "does NOT take ownership: it will never free either pointer, so " "the caller must keep both allocations alive for the buffer's " "lifetime. The device pointer is associated with the current CUDA " diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu index f66badcbf0..bb64e87da0 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.cu @@ -30,16 +30,16 @@ void defineDeviceGridChecksum(nb::module_& m) { m.def( "evalChecksum", - [](const nanovdb::NanoGrid* d_grid, nanovdb::CheckMode mode, + [](const nanovdb::NanoGrid* dGrid, nanovdb::CheckMode mode, uintptr_t stream) -> nanovdb::Checksum { cudaStream_t s = reinterpret_cast(stream); const nanovdb::GridData* d_gridData = - reinterpret_cast(d_grid); + reinterpret_cast(dGrid); // Pure CUDA (CRC32 on device, header copied D2H); release the GIL. nb::gil_scoped_release release; return nanovdb::tools::cuda::evalChecksum(d_gridData, mode, s); }, - "d_grid"_a, + "dGrid"_a, "mode"_a = nanovdb::CheckMode::Default, "stream"_a = 0, "Compute and return the Checksum of the device grid for the given " @@ -48,15 +48,15 @@ void defineDeviceGridChecksum(nb::module_& m) m.def( "validateChecksum", - [](const nanovdb::NanoGrid* d_grid, nanovdb::CheckMode mode, + [](const nanovdb::NanoGrid* dGrid, nanovdb::CheckMode mode, uintptr_t stream) -> bool { cudaStream_t s = reinterpret_cast(stream); const nanovdb::GridData* d_gridData = - reinterpret_cast(d_grid); + reinterpret_cast(dGrid); nb::gil_scoped_release release; return nanovdb::tools::cuda::validateChecksum(d_gridData, mode, s); }, - "d_grid"_a, + "dGrid"_a, "mode"_a = nanovdb::CheckMode::Default, "stream"_a = 0, "Return True iff the device grid's stored checksum matches a freshly " @@ -66,15 +66,15 @@ void defineDeviceGridChecksum(nb::module_& m) m.def( "updateChecksum", - [](nanovdb::NanoGrid* d_grid, nanovdb::CheckMode mode, + [](nanovdb::NanoGrid* dGrid, nanovdb::CheckMode mode, uintptr_t stream) { cudaStream_t s = reinterpret_cast(stream); nanovdb::GridData* d_gridData = - reinterpret_cast(d_grid); + reinterpret_cast(dGrid); nb::gil_scoped_release release; nanovdb::tools::cuda::updateChecksum(d_gridData, mode, s); }, - "d_grid"_a, + "dGrid"_a, "mode"_a = nanovdb::CheckMode::Default, "stream"_a = 0, "Recompute and write the checksum of the device grid in place using " @@ -85,10 +85,10 @@ void defineDeviceGridChecksum(nb::module_& m) // One-thread kernel that overwrites the grid's GridClass field in place. Grid // publicly derives GridData, so mGridClass is reachable on the device pointer. template -__global__ void setGridClassKernel(nanovdb::NanoGrid* d_grid, +__global__ void setGridClassKernel(nanovdb::NanoGrid* dGrid, nanovdb::GridClass gridClass) { - if (blockIdx.x == 0 && threadIdx.x == 0) d_grid->mGridClass = gridClass; + if (blockIdx.x == 0 && threadIdx.x == 0) dGrid->mGridClass = gridClass; } // Device-side mutable grid-header metadata (the read-only counterparts already @@ -98,20 +98,20 @@ void defineDeviceGridMetadata(nb::module_& m) { m.def( "setGridClass", - [](nanovdb::NanoGrid* d_grid, nanovdb::GridClass gridClass, + [](nanovdb::NanoGrid* dGrid, nanovdb::GridClass gridClass, uintptr_t stream) { - if (!d_grid) throw nb::value_error("setGridClass: d_grid is None."); + if (!dGrid) throw nb::value_error("setGridClass: dGrid is None."); cudaStream_t s = reinterpret_cast(stream); nb::gil_scoped_release release; - setGridClassKernel<<<1, 1, 0, s>>>(d_grid, gridClass); + setGridClassKernel<<<1, 1, 0, s>>>(dGrid, gridClass); // Refresh the checksum (preserving its existing mode) so that // checksum-validating readers still accept the grid after the // class field changed; a no-op for grids with checksum disabled. nanovdb::tools::cuda::updateChecksum( - reinterpret_cast(d_grid), s); + reinterpret_cast(dGrid), s); cudaStreamSynchronize(s); }, - "d_grid"_a, + "dGrid"_a, "gridClass"_a, "stream"_a = 0, "Set the device grid's GridClass (e.g. GridClass.LevelSet) in place and " diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h index 58565f8c8d..633b110cb6 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridChecksum.h @@ -17,7 +17,7 @@ template void defineDeviceGridChecksum(nb::module_& m); // Bind mutable grid-header metadata setters for one grid BuildT. Currently -// registers nanovdb.tools.cuda.setGridClass(d_grid, gridClass, stream), which +// registers nanovdb.tools.cuda.setGridClass(dGrid, gridClass, stream), which // overwrites the device grid's GridClass in place and refreshes its checksum. template void defineDeviceGridMetadata(nb::module_& m); diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index 515aaa8d9c..ed897bef2f 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -67,14 +67,14 @@ void defineDeviceGridHandle(nb::module_& m) .def( "__init__", [](GridHandle& handle, - nb::ndarray, nb::device::cpu> cpu_t, - nb::ndarray, nb::device::cuda> cuda_t) { - assert(cpu_t.size() == cuda_t.size()); - BufferT buffer(cpu_t.size() * sizeof(uint32_t), cpu_t.data(), cuda_t.data()); + nb::ndarray, nb::device::cpu> cpuT, + nb::ndarray, nb::device::cuda> cudaT) { + assert(cpuT.size() == cudaT.size()); + BufferT buffer(cpuT.size() * sizeof(uint32_t), cpuT.data(), cudaT.data()); new (&handle) GridHandle(std::move(buffer)); }, - "cpu_t"_a.noconvert(), - "cuda_t"_a.noconvert(), + "cpuT"_a.noconvert(), + "cudaT"_a.noconvert(), "Construct a DeviceGridHandle that wraps an existing pair of " "host and device uint32 arrays of equal length.") .def("deviceGrid", &pyDeviceGrid, "n"_a = 0, diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.cu index 718fa90a12..aaa577d46f 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridStats.cu @@ -27,15 +27,15 @@ void defineDeviceUpdateGridStats(nb::module_& m, const char* name) { m.def( name, - [](nanovdb::NanoGrid* d_grid, nanovdb::tools::StatsMode mode, uintptr_t stream) { + [](nanovdb::NanoGrid* dGrid, nanovdb::tools::StatsMode mode, uintptr_t stream) { cudaStream_t s = reinterpret_cast(stream); // updateGridStats launches kernels and synchronizes the stream; // pure CUDA touching no Python objects, so release the GIL. The // operation mutates the device grid in place. nb::gil_scoped_release release; - nanovdb::tools::cuda::updateGridStats(d_grid, mode, s); + nanovdb::tools::cuda::updateGridStats(dGrid, mode, s); }, - "d_grid"_a, + "dGrid"_a, "mode"_a = nanovdb::tools::StatsMode::Default, "stream"_a = 0, "Recompute and write per-node statistics into the given device grid " diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu index 4ef9cac959..8a734b6926 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridValidator.cu @@ -26,7 +26,7 @@ void defineDeviceIsValid(nb::module_& m, const char* name) { m.def( name, - [](const nanovdb::NanoGrid* d_grid, nanovdb::CheckMode mode, + [](const nanovdb::NanoGrid* dGrid, nanovdb::CheckMode mode, bool verbose, uintptr_t stream) -> bool { cudaStream_t s = reinterpret_cast(stream); // isValid runs structural checks in a device kernel plus a device @@ -34,9 +34,9 @@ void defineDeviceIsValid(nb::module_& m, const char* name) // optional verbose diagnostic goes to std::cerr), so release the // GIL. nb::gil_scoped_release release; - return nanovdb::tools::cuda::isValid(d_grid, mode, verbose, s); + return nanovdb::tools::cuda::isValid(dGrid, mode, verbose, s); }, - "d_grid"_a, + "dGrid"_a, "mode"_a = nanovdb::CheckMode::Default, "verbose"_a = false, "stream"_a = 0, diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu index e4e9aeb2a4..34d819c5f1 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu @@ -75,7 +75,7 @@ static void defineDeviceNodeManagerHandle(nb::module_& m) // kernel use. nb::class_(m, "DeviceNodeManagerHandle", "Owns the device memory backing a device-resident NodeManager. " - "Move-only. Obtain via nanovdb.cuda.createDeviceNodeManager(device_grid). " + "Move-only. Obtain via nanovdb.cuda.createDeviceNodeManager(deviceGrid). " "The NodeManager returned by mgr() is a device pointer: its node " "accessors must only be used from CUDA kernels, never dereferenced on " "the host.") @@ -96,8 +96,8 @@ static void defineDeviceNodeManagerHandle(nb::module_& m) } // cuda::createNodeManager has one template instantiation per BuildT. We expose -// a single polymorphic createDeviceNodeManager(device_grid, stream) that picks -// the right one based on the runtime type of `device_grid` (any bound +// a single polymorphic createDeviceNodeManager(deviceGrid, stream) that picks +// the right one based on the runtime type of `deviceGrid` (any bound // NanoGrid whose underlying pointer is a device pointer, e.g. from // DeviceGridHandle.deviceGrid(n)). The created handle stores a raw pointer back // to the device grid, so the handle must keep the grid alive. @@ -109,12 +109,12 @@ static nb::object tryCreateDeviceNodeManager(nb::handle py_grid, cudaStream_t st return nb::object(); // sentinel: "not this BuildT, try next" } // &grid is the device pointer (the NanoGrid object wraps a device this). - auto* d_grid = &nb::cast(py_grid); + auto* dGrid = &nb::cast(py_grid); NodeManagerHandle handle; { nb::gil_scoped_release release; handle = nanovdb::cuda::createNodeManager( - d_grid, nanovdb::cuda::DeviceBuffer(), stream); + dGrid, nanovdb::cuda::DeviceBuffer(), stream); } return nb::cast(std::move(handle)); } @@ -147,14 +147,14 @@ static void defineCreateDeviceNodeManager(nb::module_& m) "grid of any bound BuildT. Pass a device grid obtained from " "DeviceGridHandle.deviceGrid(n)."); }, - "device_grid"_a, "stream"_a = 0, + "deviceGrid"_a, "stream"_a = 0, // The constructed NodeManager stores a raw pointer back to the device // grid; the handle must therefore keep the grid (and transitively the // DeviceGridHandle that owns the grid's device buffer) alive. nb::keep_alive<0, 1>(), "Build a device-resident NodeManager for the given DEVICE grid, " "returning a DeviceNodeManagerHandle that owns the underlying device " - "buffer. device_grid MUST be a device grid (from " + "buffer. deviceGrid MUST be a device grid (from " "DeviceGridHandle.deviceGrid(n)); passing a host grid is a usage " "error. stream is a raw CUDA stream handle (Python int; 0 = default " "stream). The handle's mgr() returns the typed device NodeManager and " diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu index 077e9bf14e..780576abaf 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu @@ -28,7 +28,7 @@ namespace pynanovdb { // ----------------------- Log2BlockWidth dispatch -------------------------- // // Mirrors the host dispatchLog2BlockWidth (PyVoxelBlockManager.cc): turn the -// runtime log2_block_width into one of the four compile-time widths the +// runtime log2BlockWidth into one of the four compile-time widths the // device builder is instantiated for (BlockWidth = 64, 128, 256, 512). template static auto dispatchLog2BlockWidth(int log2BlockWidth, F&& fn) @@ -40,15 +40,15 @@ static auto dispatchLog2BlockWidth(int log2BlockWidth, F&& fn) case 9: return fn(std::integral_constant{}); default: throw nb::value_error( - "VoxelBlockManager: log2_block_width must be 6, 7, 8, or 9 " + "VoxelBlockManager: log2BlockWidth must be 6, 7, 8, or 9 " "(BlockWidth = 64, 128, 256, or 512). Larger widths are not " "bound in Python by default."); } } // PyDeviceVBMHandle wraps the device VoxelBlockManagerHandle and carries the -// log2_block_width it was built with (parallel to the host PyVBMHandle). The -// C++ handle does NOT store log2_block_width itself, so recording it once at +// log2BlockWidth it was built with (parallel to the host PyVBMHandle). The +// C++ handle does NOT store log2BlockWidth itself, so recording it once at // build time keeps the jumpMap view shape derivable from the handle rather // than from a caller who could spoof it. struct PyDeviceVBMHandle @@ -81,7 +81,7 @@ static NanoGrid* castOnIndexDeviceGrid(nb::handle py_grid, { if (!nb::isinstance>(py_grid)) { std::string msg(fn_name); - msg += ": device_grid must be a NanoVDB device grid of build type " + msg += ": deviceGrid must be a NanoVDB device grid of build type " "ValueOnIndex (OnIndexGrid), obtained from " "DeviceGridHandle.deviceGrid(n)"; throw nb::type_error(msg.c_str()); @@ -100,12 +100,12 @@ static NanoGrid* castOnIndexDeviceGrid(nb::handle py_grid, // value indices beyond activeVoxelCount, so those writes run out of bounds (an // illegal memory access). Detect it cheaply (two D2H header reads) and raise a // clear error instead of crashing. -static void requireContiguousIndexing(const NanoGrid* d_grid, +static void requireContiguousIndexing(const NanoGrid* dGrid, const char* fn_name) { using Traits = nanovdb::util::cuda::DeviceGridTraits; - const uint64_t valueCount = Traits::getValueCount(d_grid); - const uint64_t activeCount = Traits::getActiveVoxelCount(d_grid); + const uint64_t valueCount = Traits::getValueCount(dGrid); + const uint64_t activeCount = Traits::getActiveVoxelCount(dGrid); if (valueCount != activeCount + 1) { std::string msg(fn_name); msg += ": requires an OnIndex grid with contiguous active-voxel indexing " @@ -113,7 +113,7 @@ static void requireContiguousIndexing(const NanoGrid* d_grid, "grid has " + std::to_string(valueCount) + " values for " + std::to_string(activeCount) + " active voxels -- it carries " "per-node statistics and/or tile values. Rebuild it with " - "createOnIndexGrid(grid, include_stats=False, include_tiles=False) " + "createOnIndexGrid(grid, includeStats=False, includeTiles=False) " "or voxelsToOnIndexGrid."; throw nb::value_error(msg.c_str()); } @@ -140,13 +140,13 @@ static void defineHandle(nb::module_& m) "by this handle.") .def("reset", &PyDeviceVBMHandle::reset, "Release this handle's device buffers and reset it to the empty state.") - .def_prop_ro("log2_block_width", + .def_prop_ro("log2BlockWidth", [](const PyDeviceVBMHandle& h) { return h.log2BlockWidth; }, - "The log2_block_width this handle was built with. The jumpMap " + "The log2BlockWidth this handle was built with. The jumpMap " "view derives its shape from this value.") - .def_prop_ro("block_width", &PyDeviceVBMHandle::blockWidth, - "BlockWidth = 1 << log2_block_width (64, 128, 256, or 512).") - .def_prop_ro("jump_map_length", &PyDeviceVBMHandle::jumpMapLength, + .def_prop_ro("blockWidth", &PyDeviceVBMHandle::blockWidth, + "BlockWidth = 1 << log2BlockWidth (64, 128, 256, or 512).") + .def_prop_ro("jumpMapLength", &PyDeviceVBMHandle::jumpMapLength, "JumpMapLength = BlockWidth / 64 (1, 2, 4, or 8).") .def( "__bool__", @@ -200,7 +200,7 @@ static void defineHandle(nb::module_& m) // ------------------- jumpMap device view ------------------- // Zero-copy DEVICE view of the (blockCount, jumpMapLength) uint64 // jumpMap. jumpMapLength derives from the handle's recorded - // log2_block_width, never the caller, so the view exactly covers the + // log2BlockWidth, never the caller, so the view exactly covers the // allocated buffer. .def( "jumpMap", @@ -219,7 +219,7 @@ static void defineHandle(nb::module_& m) // not weak-referenceable; the py_self owner arg anchors lifetime. "Return a zero-copy (blockCount, jump_map_length) uint64 DEVICE " "array view of the jumpMap, consumable by CuPy / PyTorch / Numba. " - "The shape is determined by the log2_block_width the handle was " + "The shape is determined by the log2BlockWidth the handle was " "built with. Returns an empty (0, jump_map_length) array on a " "default-constructed or reset() handle. The view keeps this handle " "alive.") @@ -279,25 +279,25 @@ static void defineBuild(nb::module_& m) { m.def("buildVoxelBlockManager", [](nb::handle py_grid, - int log2_block_width, - uint64_t first_offset, - uint64_t last_offset, - uint64_t n_blocks, + int log2BlockWidth, + uint64_t firstOffset, + uint64_t lastOffset, + uint64_t nBlocks, uintptr_t stream) -> PyDeviceVBMHandle { - auto* d_grid = castOnIndexDeviceGrid(py_grid, "buildVoxelBlockManager"); + auto* dGrid = castOnIndexDeviceGrid(py_grid, "buildVoxelBlockManager"); cudaStream_t s = reinterpret_cast(stream); - return dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + return dispatchLog2BlockWidth(log2BlockWidth, [&](auto W) { constexpr int LBW = decltype(W)::value; using Base = VoxelBlockManagerBase; constexpr uint64_t BlockWidth = Base::BlockWidth; - // first_offset, if nonzero, must satisfy first_offset == 1 + // firstOffset, if nonzero, must satisfy firstOffset == 1 // (mod BlockWidth); the C++ builder only NANOVDB_ASSERTs this // (a no-op in release), so validate it here for a clear error. - if (first_offset != 0 && - ((first_offset - 1) & (BlockWidth - 1)) != 0) { + if (firstOffset != 0 && + ((firstOffset - 1) & (BlockWidth - 1)) != 0) { throw nb::value_error( - "buildVoxelBlockManager: first_offset must satisfy " - "first_offset == 1 (mod BlockWidth). Pass 0 (the " + "buildVoxelBlockManager: firstOffset must satisfy " + "firstOffset == 1 (mod BlockWidth). Pass 0 (the " "default) to let the implementation use 1."); } VoxelBlockManagerHandle handle; @@ -308,29 +308,29 @@ static void defineBuild(nb::module_& m) nb::gil_scoped_release release; handle = nanovdb::tools::cuda::buildVoxelBlockManager< LBW, nanovdb::cuda::DeviceBuffer>( - d_grid, first_offset, last_offset, n_blocks, s); + dGrid, firstOffset, lastOffset, nBlocks, s); } return PyDeviceVBMHandle(std::move(handle), LBW); }); }, - "device_grid"_a, - "log2_block_width"_a = 6, - "first_offset"_a = 0, - "last_offset"_a = 0, - "n_blocks"_a = 0, + "deviceGrid"_a, + "log2BlockWidth"_a = 6, + "firstOffset"_a = 0, + "lastOffset"_a = 0, + "nBlocks"_a = 0, "stream"_a = 0, // The device builder reads the grid from device memory; keep the // device grid (and its owning DeviceGridHandle) alive for the duration // of the build. The returned handle owns its own device metadata // buffers, so it does NOT need to keep the grid alive afterwards. "Build a device-side VoxelBlockManager from an OnIndex DEVICE grid. " - "device_grid MUST be a device grid (from " + "deviceGrid MUST be a device grid (from " "DeviceGridHandle.deviceGrid(n)); passing a host grid is a usage " - "error. log2_block_width selects the per-block active-voxel count " - "(6=64, 7=128, 8=256, 9=512). Pass 0 for first_offset / last_offset / " - "n_blocks to use the full grid (first active voxel through " + "error. log2BlockWidth selects the per-block active-voxel count " + "(6=64, 7=128, 8=256, 9=512). Pass 0 for firstOffset / lastOffset / " + "nBlocks to use the full grid (first active voxel through " "activeVoxelCount, minimum block count); these are read from device " - "memory. first_offset, if nonzero, must satisfy first_offset == 1 " + "memory. firstOffset, if nonzero, must satisfy firstOffset == 1 " "(mod BlockWidth). stream is a raw CUDA stream handle (Python int; 0 = " "default stream)."); } @@ -385,40 +385,40 @@ template void defineGatherBoxStencil(nb::module_& m, const char* nam [](nb::handle py_grid, nb::ndarray, nb::c_contig, nb::device::cuda> values, nb::ndarray, nb::c_contig, nb::device::cuda> out, - int log2_block_width, uintptr_t stream) { - auto* d_grid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencil"); - requireContiguousIndexing(d_grid, "gatherBoxStencil"); + int log2BlockWidth, uintptr_t stream) { + auto* dGrid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencil"); + requireContiguousIndexing(dGrid, "gatherBoxStencil"); cudaStream_t s = reinterpret_cast(stream); const T* dVals = values.data(); T* dOut = out.data(); // Build a transient VBM, then one block per VBM block decodes and // gathers the 27 neighbour values; pure CUDA, so release the GIL. nb::gil_scoped_release release; - dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + dispatchLog2BlockWidth(log2BlockWidth, [&](auto W) { constexpr int LBW = decltype(W)::value; auto handle = nanovdb::tools::cuda::buildVoxelBlockManager< - LBW, nanovdb::cuda::DeviceBuffer>(d_grid, 0, 0, 0, s); + LBW, nanovdb::cuda::DeviceBuffer>(dGrid, 0, 0, 0, s); const uint32_t bc = static_cast(handle.blockCount()); if (bc) gatherBoxStencilKernel<<>>( - d_grid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), + dGrid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), handle.firstOffset(), dVals, dOut); cudaCheck(cudaStreamSynchronize(s)); return 0; }); }, - "device_grid"_a, "values"_a, "out"_a, "log2_block_width"_a = 9, "stream"_a = 0, + "deviceGrid"_a, "values"_a, "out"_a, "log2BlockWidth"_a = 9, "stream"_a = 0, "Gather the 3x3x3 box-stencil neighbour values of every active voxel " "into a dense (valueCount, 27) array -- the bridge that lets tile / " "array frameworks (CuPy, cuTile, ...) run VDB stencils without pointer-" - "chasing the tree. device_grid is an OnIndex device grid from " + "chasing the tree. deviceGrid is an OnIndex device grid from " "DeviceGridHandle.deviceGrid(n); values is a 1-D device array indexed by " "value index (the per-voxel sidecar; entry 0 is the background slot); out " "is a 2-D device array of shape (rows, 27) with rows >= valueCount, " "filled so out[k, j] is voxel k's neighbour value at 3x3x3 spoke " "j = (di+1)*9+(dj+1)*3+(dk+1) (centre j=13; the six faces are " "j = 4, 10, 12, 14, 16, 22). Inactive neighbours read values[0]. A " - "transient VoxelBlockManager is built internally at log2_block_width " + "transient VoxelBlockManager is built internally at log2BlockWidth " "(6/7/8/9). stream is a raw CUDA stream handle (Python int; 0 = default " "stream)."); } @@ -465,9 +465,9 @@ template void defineGatherBoxStencilColumns(nb::module_& m, const ch nb::ndarray, nb::c_contig, nb::device::cuda> values, nb::ndarray, nb::c_contig, nb::device::cuda> out, nb::ndarray, nb::c_contig> spokes, - int log2_block_width, uintptr_t stream) { - auto* d_grid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencilColumns"); - requireContiguousIndexing(d_grid, "gatherBoxStencilColumns"); + int log2BlockWidth, uintptr_t stream) { + auto* dGrid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencilColumns"); + requireContiguousIndexing(dGrid, "gatherBoxStencilColumns"); const int K = static_cast(spokes.shape(0)); if (K < 1 || K > 27) throw nb::value_error("gatherBoxStencilColumns: len(spokes) must be in [1, 27]."); @@ -484,20 +484,20 @@ template void defineGatherBoxStencilColumns(nb::module_& m, const ch const T* dVals = values.data(); T* dOut = out.data(); nb::gil_scoped_release release; - dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + dispatchLog2BlockWidth(log2BlockWidth, [&](auto W) { constexpr int LBW = decltype(W)::value; auto handle = nanovdb::tools::cuda::buildVoxelBlockManager< - LBW, nanovdb::cuda::DeviceBuffer>(d_grid, 0, 0, 0, s); + LBW, nanovdb::cuda::DeviceBuffer>(dGrid, 0, 0, 0, s); const uint32_t bc = static_cast(handle.blockCount()); if (bc) gatherBoxStencilColumnsKernel<<>>( - d_grid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), + dGrid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), handle.firstOffset(), dVals, dOut, sp, K); cudaCheck(cudaStreamSynchronize(s)); return 0; }); }, - "device_grid"_a, "values"_a, "out"_a, "spokes"_a, "log2_block_width"_a = 9, "stream"_a = 0, + "deviceGrid"_a, "values"_a, "out"_a, "spokes"_a, "log2BlockWidth"_a = 9, "stream"_a = 0, "Like gatherBoxStencil, but gathers only a chosen SUBSET of the 27 box-" "stencil spokes into a dense (valueCount, K) array: out[k, col] is voxel " "k's neighbour value at spoke spokes[col], where spoke " @@ -547,35 +547,35 @@ void defineActiveVoxelCoords(nb::module_& m, const char* name) name, [](nb::handle py_grid, nb::ndarray, nb::c_contig, nb::device::cuda> out, - int log2_block_width, uintptr_t stream) { - auto* d_grid = castOnIndexDeviceGrid(py_grid, "activeVoxelCoords"); - requireContiguousIndexing(d_grid, "activeVoxelCoords"); + int log2BlockWidth, uintptr_t stream) { + auto* dGrid = castOnIndexDeviceGrid(py_grid, "activeVoxelCoords"); + requireContiguousIndexing(dGrid, "activeVoxelCoords"); cudaStream_t s = reinterpret_cast(stream); int32_t* dOut = out.data(); nb::gil_scoped_release release; - dispatchLog2BlockWidth(log2_block_width, [&](auto W) { + dispatchLog2BlockWidth(log2BlockWidth, [&](auto W) { constexpr int LBW = decltype(W)::value; auto handle = nanovdb::tools::cuda::buildVoxelBlockManager< - LBW, nanovdb::cuda::DeviceBuffer>(d_grid, 0, 0, 0, s); + LBW, nanovdb::cuda::DeviceBuffer>(dGrid, 0, 0, 0, s); const uint32_t bc = static_cast(handle.blockCount()); if (bc) activeVoxelCoordsKernel<<>>( - d_grid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), + dGrid, handle.deviceFirstLeafID(), handle.deviceJumpMap(), handle.firstOffset(), dOut); cudaCheck(cudaStreamSynchronize(s)); return 0; }); }, - "device_grid"_a, "out"_a, "log2_block_width"_a = 9, "stream"_a = 0, + "deviceGrid"_a, "out"_a, "log2BlockWidth"_a = 9, "stream"_a = 0, "Write each active voxel's index-space coordinate into a dense " "(rows, 3) int32 device array keyed by value index (rows >= valueCount); " "out[k] is the (i, j, k) coordinate of value index k (row 0, the " "background slot, is left untouched). The decode companion to " "gatherBoxStencil -- recovers per-voxel coordinates without a " "hand-written decode kernel (e.g. to bake a sidecar result back into a " - "grid). device_grid is an OnIndex device grid from " + "grid). deviceGrid is an OnIndex device grid from " "DeviceGridHandle.deviceGrid(n); a transient VoxelBlockManager is built " - "internally at log2_block_width (6/7/8/9). stream is a raw CUDA stream " + "internally at log2BlockWidth (6/7/8/9). stream is a raw CUDA stream " "handle (Python int; 0 = default stream)."); } diff --git a/nanovdb/nanovdb/python/cuda/PyDilateGrid.cu b/nanovdb/nanovdb/python/cuda/PyDilateGrid.cu index b7e110bff2..b62ec517b4 100644 --- a/nanovdb/nanovdb/python/cuda/PyDilateGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyDilateGrid.cu @@ -20,16 +20,16 @@ template void defineDilateGrid(nb::module_& m, const char* name { m.def( name, - [](nanovdb::NanoGrid* d_grid, int op, uintptr_t stream) { + [](nanovdb::NanoGrid* dGrid, int op, uintptr_t stream) { cudaStream_t s = reinterpret_cast(stream); // DilateGrid::getHandle launches kernels and synchronizes the // stream; pure CUDA touching no Python objects, so release the GIL. nb::gil_scoped_release release; - nanovdb::tools::cuda::DilateGrid dilator(d_grid, s); + nanovdb::tools::cuda::DilateGrid dilator(dGrid, s); dilator.setOperation(static_cast(op)); return dilator.getHandle(); }, - "d_grid"_a, + "dGrid"_a, "op"_a = static_cast(nanovdb::tools::morphology::NN_FACE_EDGE_VERTEX), "stream"_a = 0, "Morphologically dilate a device OnIndex grid and return a fresh device " diff --git a/nanovdb/nanovdb/python/cuda/PyInjectData.cu b/nanovdb/nanovdb/python/cuda/PyInjectData.cu index a6f69efb2c..bb35e508fb 100644 --- a/nanovdb/nanovdb/python/cuda/PyInjectData.cu +++ b/nanovdb/nanovdb/python/cuda/PyInjectData.cu @@ -41,10 +41,10 @@ castOnIndexDeviceGrid(nb::handle py_grid, const char* fn_name) } // Leaf-node count read from device memory (one D2H copy of the tree header). -uint32_t leafCountOf(const nanovdb::NanoGrid* d_grid) +uint32_t leafCountOf(const nanovdb::NanoGrid* dGrid) { using Traits = nanovdb::util::cuda::DeviceGridTraits; - return Traits::getTreeData(d_grid).mNodeCount[0]; + return Traits::getTreeData(dGrid).mNodeCount[0]; } } // anonymous namespace @@ -53,15 +53,15 @@ template void defineInject(nb::module_& m, const char* name) { m.def( name, - [](nb::handle src_grid, nb::handle dst_grid, - nb::ndarray, nb::c_contig, nb::device::cuda> src_sidecar, - nb::ndarray, nb::c_contig, nb::device::cuda> dst_sidecar, + [](nb::handle srcGrid, nb::handle dstGrid, + nb::ndarray, nb::c_contig, nb::device::cuda> srcSidecar, + nb::ndarray, nb::c_contig, nb::device::cuda> dstSidecar, uintptr_t stream) { - auto* src = castOnIndexDeviceGrid(src_grid, "inject"); - auto* dst = castOnIndexDeviceGrid(dst_grid, "inject"); + auto* src = castOnIndexDeviceGrid(srcGrid, "inject"); + auto* dst = castOnIndexDeviceGrid(dstGrid, "inject"); cudaStream_t s = reinterpret_cast(stream); - const T* dSrc = src_sidecar.data(); - T* dDst = dst_sidecar.data(); + const T* dSrc = srcSidecar.data(); + T* dDst = dstSidecar.data(); const uint32_t srcLeafCount = leafCountOf(src); using Op = nanovdb::util::cuda::InjectGridDataFunctor; // operatorKernel launches one block per SOURCE leaf and copies the @@ -72,15 +72,15 @@ template void defineInject(nb::module_& m, const char* name) <<>>(src, dst, dSrc, dDst); cudaCheck(cudaStreamSynchronize(s)); }, - "src_grid"_a, "dst_grid"_a, "src_sidecar"_a, "dst_sidecar"_a, "stream"_a = 0, + "srcGrid"_a, "dstGrid"_a, "srcSidecar"_a, "dstSidecar"_a, "stream"_a = 0, "Inject sidecar values from a source OnIndex device grid onto a " "destination OnIndex device grid (injectData; NanoVDB 2.0 paper, " "section 3.4). For every voxel present in BOTH grids the source sidecar " "value is copied to that voxel's slot in the destination sidecar; " "destination voxels with no source counterpart are left unchanged " "(so the source need not be a subset -- the copy is over the " - "intersection). src_grid / dst_grid are device grids from " - "DeviceGridHandle.deviceGrid(n); src_sidecar / dst_sidecar are 1-D " + "intersection). srcGrid / dstGrid are device grids from " + "DeviceGridHandle.deviceGrid(n); srcSidecar / dstSidecar are 1-D " "device arrays indexed by each grid's value index (entry 0 is the " "background slot, untouched). Wraps " "nanovdb::util::cuda::InjectGridDataFunctor. stream is a raw CUDA " @@ -91,20 +91,20 @@ template void defineInjectFeatures(nb::module_& m, const char* name) { m.def( name, - [](nb::handle src_grid, nb::handle dst_grid, - nb::ndarray, nb::c_contig, nb::device::cuda> src_sidecar, - nb::ndarray, nb::c_contig, nb::device::cuda> dst_sidecar, + [](nb::handle srcGrid, nb::handle dstGrid, + nb::ndarray, nb::c_contig, nb::device::cuda> srcSidecar, + nb::ndarray, nb::c_contig, nb::device::cuda> dstSidecar, uintptr_t stream) { - auto* src = castOnIndexDeviceGrid(src_grid, "inject"); - auto* dst = castOnIndexDeviceGrid(dst_grid, "inject"); - if (src_sidecar.shape(1) != dst_sidecar.shape(1)) + auto* src = castOnIndexDeviceGrid(srcGrid, "inject"); + auto* dst = castOnIndexDeviceGrid(dstGrid, "inject"); + if (srcSidecar.shape(1) != dstSidecar.shape(1)) throw nb::value_error( "inject: src and dst feature sidecars must share the same " "feature dimension (shape[1])."); cudaStream_t s = reinterpret_cast(stream); - const T* dSrc = src_sidecar.data(); - T* dDst = dst_sidecar.data(); - const size_t dim = src_sidecar.shape(1); + const T* dSrc = srcSidecar.data(); + T* dDst = dstSidecar.data(); + const size_t dim = srcSidecar.shape(1); const uint32_t srcLeafCount = leafCountOf(src); using Op = nanovdb::util::cuda::InjectGridFeatureFunctor; nb::gil_scoped_release release; @@ -113,9 +113,9 @@ template void defineInjectFeatures(nb::module_& m, const char* name) <<>>(src, dst, dSrc, dDst, dim); cudaCheck(cudaStreamSynchronize(s)); }, - "src_grid"_a, "dst_grid"_a, "src_sidecar"_a, "dst_sidecar"_a, "stream"_a = 0, + "srcGrid"_a, "dstGrid"_a, "srcSidecar"_a, "dstSidecar"_a, "stream"_a = 0, "Inject vector-valued (feature) sidecar data across OnIndex device " - "grids -- the multi-channel form of inject. src_sidecar / dst_sidecar " + "grids -- the multi-channel form of inject. srcSidecar / dstSidecar " "are 2-D device arrays of shape (value count, dim), row-major per voxel " "(row 0 is the background slot); the feature dimension dim is taken " "from shape[1] and must match. Values are copied for the src/dst voxel " @@ -130,36 +130,36 @@ void defineInjectPredicateToMask(nb::module_& m, const char* name) name, [](nb::handle grid, nb::ndarray, nb::c_contig, nb::device::cuda> predicate, - nb::ndarray, nb::c_contig, nb::device::cuda> leaf_masks, + nb::ndarray, nb::c_contig, nb::device::cuda> leafMasks, uintptr_t stream) { - auto* d_grid = castOnIndexDeviceGrid(grid, "injectPredicateToMask"); + auto* dGrid = castOnIndexDeviceGrid(grid, "injectPredicateToMask"); cudaStream_t s = reinterpret_cast(stream); - const uint32_t leafCount = leafCountOf(d_grid); + const uint32_t leafCount = leafCountOf(dGrid); constexpr size_t W = nanovdb::Mask<3>::WORD_COUNT; // 8 uint64 / leaf - if (leaf_masks.size() < static_cast(leafCount) * W) + if (leafMasks.size() < static_cast(leafCount) * W) throw nb::value_error( - "injectPredicateToMask: leaf_masks length must be at least " + "injectPredicateToMask: leafMasks length must be at least " "(leaf count) * 8 uint64 (one Mask<3> per leaf). A safe " "upper bound is activeVoxelCount * 8, since every leaf " "holds at least one active voxel."); const bool* dPred = predicate.data(); nanovdb::Mask<3>* dMask = - reinterpret_cast*>(leaf_masks.data()); + reinterpret_cast*>(leafMasks.data()); using Op = nanovdb::util::cuda::InjectPredicateToMaskFunctor; // One block per leaf; the functor zeroes each leaf mask, then sets // the bit of every active voxel whose predicate slot is true. nb::gil_scoped_release release; if (leafCount) nanovdb::util::cuda::operatorKernel - <<>>(d_grid, dPred, dMask); + <<>>(dGrid, dPred, dMask); cudaCheck(cudaStreamSynchronize(s)); }, - "grid"_a, "predicate"_a, "leaf_masks"_a, "stream"_a = 0, + "grid"_a, "predicate"_a, "leafMasks"_a, "stream"_a = 0, "Build a per-leaf retain mask for pruneGrid from a boolean predicate " "over an OnIndex device grid's value indices. grid is a device grid " "from DeviceGridHandle.deviceGrid(n); predicate is a 1-D device bool " "array indexed by value index (entry n true => keep that voxel); " - "leaf_masks is a 1-D device uint64 output of length at least " + "leafMasks is a 1-D device uint64 output of length at least " "(leaf count) * 8 (one nanovdb::Mask<3> per leaf, in leaf order), " "ready to pass straight to pruneGrid; activeVoxelCount * 8 is a safe " "size since every leaf holds at least one active voxel. Wraps " @@ -171,21 +171,21 @@ void defineInjectGridMask(nb::module_& m, const char* name) { m.def( name, - [](nb::handle src_grid, nb::handle dst_grid, - nb::ndarray, nb::c_contig, nb::device::cuda> leaf_masks, + [](nb::handle srcGrid, nb::handle dstGrid, + nb::ndarray, nb::c_contig, nb::device::cuda> leafMasks, uintptr_t stream) { - auto* src = castOnIndexDeviceGrid(src_grid, "injectGridMask"); - auto* dst = castOnIndexDeviceGrid(dst_grid, "injectGridMask"); + auto* src = castOnIndexDeviceGrid(srcGrid, "injectGridMask"); + auto* dst = castOnIndexDeviceGrid(dstGrid, "injectGridMask"); cudaStream_t s = reinterpret_cast(stream); const uint32_t dstLeafCount = leafCountOf(dst); constexpr size_t W = nanovdb::Mask<3>::WORD_COUNT; // 8 uint64 / leaf - if (leaf_masks.size() < static_cast(dstLeafCount) * W) + if (leafMasks.size() < static_cast(dstLeafCount) * W) throw nb::value_error( - "injectGridMask: leaf_masks length must be at least " + "injectGridMask: leafMasks length must be at least " "(dst leaf count) * 8 uint64 (one Mask<3> per leaf). A safe " "upper bound is the destination grid's activeVoxelCount * 8."); nanovdb::Mask<3>* dMask = - reinterpret_cast*>(leaf_masks.data()); + reinterpret_cast*>(leafMasks.data()); using Op = nanovdb::util::cuda::InjectGridMaskFunctor; constexpr unsigned threads = 128; nb::gil_scoped_release release; @@ -195,11 +195,11 @@ void defineInjectGridMask(nb::module_& m, const char* name) threads, 0, s>>>(dstLeafCount, Op{}, src, dst, dMask); cudaCheck(cudaStreamSynchronize(s)); }, - "src_grid"_a, "dst_grid"_a, "leaf_masks"_a, "stream"_a = 0, + "srcGrid"_a, "dstGrid"_a, "leafMasks"_a, "stream"_a = 0, "Build a per-leaf mask over the DESTINATION grid marking the voxels " "that are ALSO active in the source grid (the src/dst intersection). " "grid args are device grids from DeviceGridHandle.deviceGrid(n); " - "leaf_masks is a 1-D device uint64 output of length at least " + "leafMasks is a 1-D device uint64 output of length at least " "(dst leaf count) * 8 (one nanovdb::Mask<3> per leaf, in leaf order; " "the destination activeVoxelCount * 8 is a safe size). Pass it to " "pruneGrid to keep only the intersection. Wraps " diff --git a/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu b/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu index aa3e789b63..32c9dde8a1 100644 --- a/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu +++ b/nanovdb/nanovdb/python/cuda/PyMergeGrids.cu @@ -21,23 +21,23 @@ template void defineMergeGrids(nb::module_& m, const char* name { m.def( name, - [](nanovdb::NanoGrid* d_grid1, - nanovdb::NanoGrid* d_grid2, + [](nanovdb::NanoGrid* dGrid1, + nanovdb::NanoGrid* dGrid2, uintptr_t stream) { cudaStream_t s = reinterpret_cast(stream); // MergeGrids::getHandle launches kernels and synchronizes the // stream; pure CUDA touching no Python objects, so release the GIL. nb::gil_scoped_release release; - nanovdb::tools::cuda::MergeGrids merger(d_grid1, d_grid2, s); + nanovdb::tools::cuda::MergeGrids merger(dGrid1, dGrid2, s); return merger.getHandle(); }, - "d_grid1"_a, - "d_grid2"_a, + "dGrid1"_a, + "dGrid2"_a, "stream"_a = 0, "Topologically merge (active-mask union) two device OnIndex grids and " "return a fresh device GridHandle of the union. The operation is " "strictly binary; chain calls to union more than two grids. Output " - "metadata is taken from d_grid1. stream is a raw CUDA stream handle " + "metadata is taken from dGrid1. stream is a raw CUDA stream handle " "(Python int; 0 = default stream)."); // List overload: N-ary merge diff --git a/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu b/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu index d857e86dbc..bddbcdbdb0 100644 --- a/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu @@ -22,7 +22,7 @@ template void definePruneGrid(nb::module_& m, const char* name) { m.def( name, - [](nanovdb::NanoGrid* d_grid, + [](nanovdb::NanoGrid* dGrid, nb::ndarray leafMask, uintptr_t stream) { // The sidecar is a device array of nanovdb::Mask<3> (one 512-bit / @@ -42,10 +42,10 @@ template void definePruneGrid(nb::module_& m, const char* name) // PruneGrid::getHandle launches kernels and synchronizes the // stream; pure CUDA touching no Python objects, so release the GIL. nb::gil_scoped_release release; - nanovdb::tools::cuda::PruneGrid pruner(d_grid, d_mask, s); + nanovdb::tools::cuda::PruneGrid pruner(dGrid, d_mask, s); return pruner.getHandle(); }, - "d_grid"_a, + "dGrid"_a, "leafMask"_a, "stream"_a = 0, "Morphologically prune a device OnIndex grid against a per-leaf retain " diff --git a/nanovdb/nanovdb/python/cuda/PyRefineGrid.cu b/nanovdb/nanovdb/python/cuda/PyRefineGrid.cu index 316e05ff6c..c53fbae2b3 100644 --- a/nanovdb/nanovdb/python/cuda/PyRefineGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyRefineGrid.cu @@ -19,15 +19,15 @@ template void defineRefineGrid(nb::module_& m, const char* name { m.def( name, - [](nanovdb::NanoGrid* d_grid, uintptr_t stream) { + [](nanovdb::NanoGrid* dGrid, uintptr_t stream) { cudaStream_t s = reinterpret_cast(stream); // RefineGrid::getHandle launches kernels and synchronizes the // stream; pure CUDA touching no Python objects, so release the GIL. nb::gil_scoped_release release; - nanovdb::tools::cuda::RefineGrid refiner(d_grid, s); + nanovdb::tools::cuda::RefineGrid refiner(dGrid, s); return refiner.getHandle(); }, - "d_grid"_a, + "dGrid"_a, "stream"_a = 0, "Topologically refine (2x upsample) a device OnIndex grid and return a " "fresh device GridHandle of the refined grid. stream is a raw CUDA " diff --git a/nanovdb/nanovdb/python/cuda/PySampleFromVoxels.cu b/nanovdb/nanovdb/python/cuda/PySampleFromVoxels.cu index 4805a93d6e..9c13cdcff8 100644 --- a/nanovdb/nanovdb/python/cuda/PySampleFromVoxels.cu +++ b/nanovdb/nanovdb/python/cuda/PySampleFromVoxels.cu @@ -15,34 +15,34 @@ using namespace nanovdb; namespace { -template __global__ void sampleFromVoxels(unsigned int numPoints, const BuildT* points, const NanoGrid* d_grid, BuildT* values) +template __global__ void sampleFromVoxels(unsigned int numPoints, const BuildT* points, const NanoGrid* dGrid, BuildT* values) { using TreeT = NanoTree; using Vec3T = math::Vec3; for (unsigned int i = threadIdx.x + blockIdx.x * blockDim.x; i < numPoints; i += blockDim.x * gridDim.x) { Vec3T worldPos(points[3 * i], points[3 * i + 1], points[3 * i + 2]); - Vec3T indexPos = d_grid->worldToIndex(worldPos); + Vec3T indexPos = dGrid->worldToIndex(worldPos); - math::SampleFromVoxels sampler(d_grid->tree()); + math::SampleFromVoxels sampler(dGrid->tree()); values[i] = sampler(indexPos); } } template -__global__ void sampleFromVoxels(unsigned int numPoints, const BuildT* points, const NanoGrid* d_grid, BuildT* values, BuildT* gradients) +__global__ void sampleFromVoxels(unsigned int numPoints, const BuildT* points, const NanoGrid* dGrid, BuildT* values, BuildT* gradients) { using TreeT = NanoTree; using Vec3T = math::Vec3; for (unsigned int i = threadIdx.x + blockIdx.x * blockDim.x; i < numPoints; i += blockDim.x * gridDim.x) { Vec3T worldPos(points[3 * i], points[3 * i + 1], points[3 * i + 2]); - Vec3T indexPos = d_grid->worldToIndex(worldPos); + Vec3T indexPos = dGrid->worldToIndex(worldPos); - math::SampleFromVoxels sampler(d_grid->tree()); + math::SampleFromVoxels sampler(dGrid->tree()); values[i] = sampler(indexPos); - Vec3T inv2Dx = (BuildT).5 / d_grid->voxelSize(); + Vec3T inv2Dx = (BuildT).5 / dGrid->voxelSize(); Vec3T gradient = Vec3T(sampler(indexPos + Vec3T(1, 0, 0)) - sampler(indexPos - Vec3T(1, 0, 0)), sampler(indexPos + Vec3T(0, 1, 0)) - sampler(indexPos - Vec3T(0, 1, 0)), sampler(indexPos + Vec3T(0, 0, 1)) - sampler(indexPos - Vec3T(0, 0, 1))) * @@ -62,7 +62,7 @@ template void defineSampleFromVoxels(nb::module_& m, const char m.def( name, [](nb::ndarray, nb::c_contig, nb::device::cuda> points, - NanoGrid* d_grid, + NanoGrid* dGrid, nb::ndarray, nb::device::cuda> values, uintptr_t stream) { cudaStream_t s = reinterpret_cast(stream); @@ -71,16 +71,16 @@ template void defineSampleFromVoxels(nb::module_& m, const char // Raw kernel launch on the supplied stream; touches no Python // objects, so release the GIL. nb::gil_scoped_release release; - sampleFromVoxels<<>>(points.shape(0), points.data(), d_grid, values.data()); + sampleFromVoxels<<>>(points.shape(0), points.data(), dGrid, values.data()); }, "points"_a, - "d_grid"_a, + "dGrid"_a, "values"_a, "stream"_a = 0); m.def( name, [](nb::ndarray, nb::c_contig, nb::device::cuda> points, - NanoGrid* d_grid, + NanoGrid* dGrid, nb::ndarray, nb::device::cuda> values, nb::ndarray, nb::device::cuda> gradients, uintptr_t stream) { @@ -90,10 +90,10 @@ template void defineSampleFromVoxels(nb::module_& m, const char // Raw kernel launch on the supplied stream; touches no Python // objects, so release the GIL. nb::gil_scoped_release release; - sampleFromVoxels<<>>(points.shape(0), points.data(), d_grid, values.data(), gradients.data()); + sampleFromVoxels<<>>(points.shape(0), points.data(), dGrid, values.data(), gradients.data()); }, "points"_a, - "d_grid"_a, + "dGrid"_a, "values"_a, "gradients"_a, "stream"_a = 0); diff --git a/nanovdb/nanovdb/python/cuda/PySignedFloodFill.cu b/nanovdb/nanovdb/python/cuda/PySignedFloodFill.cu index eb6fe7ea8b..16efa1b583 100644 --- a/nanovdb/nanovdb/python/cuda/PySignedFloodFill.cu +++ b/nanovdb/nanovdb/python/cuda/PySignedFloodFill.cu @@ -16,14 +16,14 @@ template void defineSignedFloodFill(nb::module_& m, const char* { m.def( name, - [](NanoGrid* d_grid, bool verbose, uintptr_t stream) { + [](NanoGrid* dGrid, bool verbose, uintptr_t stream) { cudaStream_t s = reinterpret_cast(stream); // signedFloodFill launches kernels and synchronizes the stream; // pure CUDA touching no Python objects, so release the GIL. nb::gil_scoped_release release; - tools::cuda::signedFloodFill(d_grid, verbose, s); + tools::cuda::signedFloodFill(dGrid, verbose, s); }, - "d_grid"_a, + "dGrid"_a, "verbose"_a = false, "stream"_a = 0, "Perform a signed flood fill on a device float/double grid in place. " diff --git a/nanovdb/nanovdb/python/examples/build_grid.py b/nanovdb/nanovdb/python/examples/build_grid.py index d4c3f27bed..2e63c9a0bd 100644 --- a/nanovdb/nanovdb/python/examples/build_grid.py +++ b/nanovdb/nanovdb/python/examples/build_grid.py @@ -7,7 +7,7 @@ three construction loops you'll typically reach for (setValue directly, the cached ValueAccessor, and the thread-safe WriteAccessor), then bakes each build grid into a host NanoGrid via -.to_nanovdb() and reads it back through the regular polymorphic +.toNanoVDB() and reads it back through the regular polymorphic handle.grid() API. Run with: python build_grid.py diff --git a/nanovdb/nanovdb/python/examples/bulk_leaf_numpy.py b/nanovdb/nanovdb/python/examples/bulk_leaf_numpy.py index f967a2ae4a..6b34487910 100644 --- a/nanovdb/nanovdb/python/examples/bulk_leaf_numpy.py +++ b/nanovdb/nanovdb/python/examples/bulk_leaf_numpy.py @@ -2,7 +2,7 @@ # SPDX-License-Identifier: Apache-2.0 """Bulk per-leaf value access as a zero-copy NumPy array. -grid.leaf_values() is the highest-bandwidth path from NanoVDB into +grid.leafValues() is the highest-bandwidth path from NanoVDB into NumPy. It returns an (N_leaves, 512) view of every leaf's mValues without copying — modify it, slice it, feed it into a PyTorch tensor, hash it for cache lookup, whatever you need. @@ -21,18 +21,18 @@ def main(): # Build a fog volume sphere with stats so the leaves have meaningful # min/max attached (just for the printing below — not required by - # leaf_values itself). + # leafValues itself). handle = nanovdb.tools.createFogVolumeSphere( radius=20.0, name="bulk_demo") grid = handle.grid() print(f"Grid: {grid.gridType()}, active voxels = {grid.activeVoxelCount()}, " f"leaves = {grid.tree().nodeCount(0)}") - # leaf_values() is the zero-copy view. Modifying it modifies the grid. - bulk = grid.leaf_values() + # leafValues() is the zero-copy view. Modifying it modifies the grid. + bulk = grid.leafValues() # np.asarray adds a NumPy wrapper but doesn't copy. arr = np.asarray(bulk) - print(f"leaf_values: shape={arr.shape}, dtype={arr.dtype}, " + print(f"leafValues: shape={arr.shape}, dtype={arr.dtype}, " f"backed by grid memory (no copy).") # Global statistics across every voxel in every leaf, computed in C. diff --git a/nanovdb/nanovdb/python/examples/collide_level_set_cuda.py b/nanovdb/nanovdb/python/examples/collide_level_set_cuda.py index e2eaabcde8..7d9fb28920 100644 --- a/nanovdb/nanovdb/python/examples/collide_level_set_cuda.py +++ b/nanovdb/nanovdb/python/examples/collide_level_set_cuda.py @@ -49,7 +49,7 @@ def main(): finally: os.unlink(tmp.name) handle.deviceUpload(0, True) - device_grid = handle.deviceGrid(0) + deviceGrid = handle.deviceGrid(0) # Seed particles above the north pole of the sphere, falling down. rng = cp.random.RandomState(42) @@ -68,7 +68,7 @@ def main(): v[:, 1] += GRAVITY * DT next_p = cp.ascontiguousarray(p + v * DT) - nanovdb.tools.cuda.sampleFromVoxels(next_p, device_grid, values, grads, 0) + nanovdb.tools.cuda.sampleFromVoxels(next_p, deviceGrid, values, grads, 0) cp.cuda.Stream.null.synchronize() # A collision is a point inside the surface but still within the @@ -95,7 +95,7 @@ def main(): assert total_collisions > 0 # No particle should end up deep inside the surface. - nanovdb.tools.cuda.sampleFromVoxels(cp.ascontiguousarray(p), device_grid, values, 0) + nanovdb.tools.cuda.sampleFromVoxels(cp.ascontiguousarray(p), deviceGrid, values, 0) cp.cuda.Stream.null.synchronize() in_band = (values > -BAND) & (values < BAND) if bool(in_band.any()): diff --git a/nanovdb/nanovdb/python/examples/cupy_rawkernel.py b/nanovdb/nanovdb/python/examples/cupy_rawkernel.py index 09bef2008c..e092f84d94 100644 --- a/nanovdb/nanovdb/python/examples/cupy_rawkernel.py +++ b/nanovdb/nanovdb/python/examples/cupy_rawkernel.py @@ -37,14 +37,14 @@ KERNEL_SRC = r""" #include -// d_grid is the raw device pointer from FloatGrid.data_ptr(); out is a +// dGrid is the raw device pointer from FloatGrid.data_ptr(); out is a // 2-float device buffer that receives [value@origin, activeVoxelCount]. extern "C" __global__ -void inspect_float_grid(const nanovdb::NanoGrid* d_grid, float* out) +void inspect_float_grid(const nanovdb::NanoGrid* dGrid, float* out) { - auto acc = d_grid->getAccessor(); + auto acc = dGrid->getAccessor(); out[0] = acc.getValue(nanovdb::Coord(0, 0, 0)); - out[1] = static_cast(d_grid->activeVoxelCount()); + out[1] = static_cast(dGrid->activeVoxelCount()); } """ @@ -83,15 +83,15 @@ def main(): # Build a float level-set sphere directly on the device. handle = nanovdb.tools.cuda.createLevelSetSphere(nanovdb.GridType.Float, 20) handle.deviceUpload(0, True) - device_grid = handle.deviceGrid(0) - print(f"Device FloatGrid at {hex(device_grid.data_ptr())}") + deviceGrid = handle.deviceGrid(0) + print(f"Device FloatGrid at {hex(deviceGrid.data_ptr())}") kernel = cp.RawKernel( KERNEL_SRC, "inspect_float_grid", options=options, backend="nvrtc") out = cp.zeros(2, dtype=cp.float32) # Launch with the raw device-grid pointer as the first argument. - kernel((1,), (1,), (device_grid.data_ptr(), out.data.ptr)) + kernel((1,), (1,), (deviceGrid.data_ptr(), out.data.ptr)) cp.cuda.runtime.deviceSynchronize() value, active = cp.asnumpy(out) diff --git a/nanovdb/nanovdb/python/examples/device_topology_ops.py b/nanovdb/nanovdb/python/examples/device_topology_ops.py index 5094ca3359..18381e323f 100644 --- a/nanovdb/nanovdb/python/examples/device_topology_ops.py +++ b/nanovdb/nanovdb/python/examples/device_topology_ops.py @@ -49,14 +49,14 @@ def main(): coords = cp.ascontiguousarray( cp.stack([i.ravel(), j.ravel(), k.ravel()], axis=1)) src = nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, 1.0, 0) - src_grid = src.deviceGrid(0) + srcGrid = src.deviceGrid(0) src_active = _active(src) print(f"source block: {src_active} active voxels") - dil6 = nanovdb.tools.cuda.dilateGrid(src_grid, 6, 0) - dil26 = nanovdb.tools.cuda.dilateGrid(src_grid, 26, 0) - coarse = nanovdb.tools.cuda.coarsenGrid(src_grid, 0) - fine = nanovdb.tools.cuda.refineGrid(src_grid, 0) + dil6 = nanovdb.tools.cuda.dilateGrid(srcGrid, 6, 0) + dil26 = nanovdb.tools.cuda.dilateGrid(srcGrid, 26, 0) + coarse = nanovdb.tools.cuda.coarsenGrid(srcGrid, 0) + fine = nanovdb.tools.cuda.refineGrid(srcGrid, 0) a6, a26 = _active(dil6), _active(dil26) ac, af = _active(coarse), _active(fine) print(f"dilate(faces) -> {a6} active") @@ -71,7 +71,7 @@ def main(): shifted = nanovdb.tools.cuda.voxelsToOnIndexGrid( cp.ascontiguousarray(coords + cp.asarray([4, 0, 0], dtype=cp.int32)), 1.0, 0) - merged = nanovdb.tools.cuda.mergeGrids(src_grid, shifted.deviceGrid(0), 0) + merged = nanovdb.tools.cuda.mergeGrids(srcGrid, shifted.deviceGrid(0), 0) am = _active(merged) print(f"merge(block, block+4x) -> {am} active") assert am > src_active @@ -81,7 +81,7 @@ def main(): src.deviceDownload(0, True) leaf_count = src.grid(0).tree().nodeCount(0) retain_all = cp.full(leaf_count * 8, 0xFFFFFFFFFFFFFFFF, dtype=cp.uint64) - pruned = nanovdb.tools.cuda.pruneGrid(src_grid, retain_all, 0) + pruned = nanovdb.tools.cuda.pruneGrid(srcGrid, retain_all, 0) ap = _active(pruned) print(f"prune(retain-all, {leaf_count} leaves) -> {ap} active") assert ap == src_active diff --git a/nanovdb/nanovdb/python/examples/gpu_load_inspect.py b/nanovdb/nanovdb/python/examples/gpu_load_inspect.py index 98611da35d..19517e9278 100644 --- a/nanovdb/nanovdb/python/examples/gpu_load_inspect.py +++ b/nanovdb/nanovdb/python/examples/gpu_load_inspect.py @@ -80,19 +80,19 @@ def main(): # grid's data_ptr is a device address. The grid object itself cannot # tell them apart. host_grid = handle.grid(0) - device_grid = handle.deviceGrid(0) + deviceGrid = handle.deviceGrid(0) print(f" host grid.data_ptr() = {hex(host_grid.data_ptr())} (CPU)") - print(f" device grid.data_ptr() = {hex(device_grid.data_ptr())} (GPU)") + print(f" device grid.data_ptr() = {hex(deviceGrid.data_ptr())} (GPU)") print(f" device grid.data_ptr() == handle.device_ptr(): " - f"{device_grid.data_ptr() == handle.device_ptr()}") + f"{deviceGrid.data_ptr() == handle.device_ptr()}") # The device grid is the input to nanovdb.tools.cuda.* ops. Validate # it entirely on the device. - print(f" tools.cuda.isValid(device_grid) = " - f"{nanovdb.tools.cuda.isValid(device_grid)}") + print(f" tools.cuda.isValid(deviceGrid) = " + f"{nanovdb.tools.cuda.isValid(deviceGrid)}") print("WARNING: calling a host-side accessor (e.g. " - "device_grid.getAccessor().getValue(...)) on a DEVICE grid " + "deviceGrid.getAccessor().getValue(...)) on a DEVICE grid " "dereferences GPU memory on the CPU and SEGFAULTS. Use host_grid " "for host reads.") diff --git a/nanovdb/nanovdb/python/examples/index_grid_channels.py b/nanovdb/nanovdb/python/examples/index_grid_channels.py index 91e421d08f..dc54833867 100644 --- a/nanovdb/nanovdb/python/examples/index_grid_channels.py +++ b/nanovdb/nanovdb/python/examples/index_grid_channels.py @@ -17,15 +17,15 @@ def index_grid_with_channel(): src = nanovdb.tools.createLevelSetSphere(radius=50.0, name="sphere") - src_grid = src.grid() + srcGrid = src.grid() # channels=1 copies the source values into blind-data channel 0, # indexed by the per-voxel uint64 indices the OnIndex grid stores. - handle = nanovdb.tools.createNanoGridOnIndex(src_grid, channels=1) + handle = nanovdb.tools.createNanoGridOnIndex(srcGrid, channels=1) grid = handle.grid() print(f"OnIndex grid: valueCount={grid.valueCount()}, " - f"source activeVoxelCount={src_grid.activeVoxelCount()}") - assert grid.valueCount() >= src_grid.activeVoxelCount() + f"source activeVoxelCount={srcGrid.activeVoxelCount()}") + assert grid.valueCount() >= srcGrid.activeVoxelCount() # createChannelAccessor inspects the channel's recorded dataType and # returns the matching typed accessor (here: OnIndexFloat...). @@ -33,7 +33,7 @@ def index_grid_with_channel(): print(f"channel accessor: {type(channel).__name__}, " f"valueCount={channel.valueCount()}") - src_acc = src_grid.getAccessor() + src_acc = srcGrid.getAccessor() for ijk in (nanovdb.math.Coord(48, 0, 0), nanovdb.math.Coord(0, 50, 0), nanovdb.math.Coord(0, 0, 52)): via_channel = channel.getValue(ijk) diff --git a/nanovdb/nanovdb/python/examples/levelset_filter.py b/nanovdb/nanovdb/python/examples/levelset_filter.py index 525c2e7f91..eae92fc9e7 100644 --- a/nanovdb/nanovdb/python/examples/levelset_filter.py +++ b/nanovdb/nanovdb/python/examples/levelset_filter.py @@ -92,7 +92,7 @@ def setup(self, handle): handle.deviceUpload(0, True) grid = handle.deviceGrid(0) n = int(nanovdb.tools.cuda.buildVoxelBlockManager( - grid, log2_block_width=LOG2_BLOCK_WIDTH).lastOffset()) + grid, log2BlockWidth=LOG2_BLOCK_WIDTH).lastOffset()) return {"handle": handle, "grid": grid, "n": n} def active_coords(self, g): @@ -141,7 +141,7 @@ def read_to_device(backend, path, band): tmp = None if gtype == nanovdb.GridType.Float: onh = T.createOnIndexGrid(host.grid(0), channels=1, - include_stats=False, include_tiles=False) + includeStats=False, includeTiles=False) sdf = np.array(onh.grid(0).getBlindData(0), dtype=np.float32) tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() io.writeGrid(tmp.name, onh) @@ -157,7 +157,7 @@ def read_to_device(backend, path, band): if sdf.shape[0] != g["n"] + 1: raise SystemExit(f"{path}: SDF channel length {sdf.shape[0]} != activeVoxelCount+1 " f"({g['n'] + 1}); the OnIndex grid must use contiguous voxel indexing " - "(built with include_stats=False, include_tiles=False).") + "(built with includeStats=False, includeTiles=False).") phi = cp.asarray(sdf) phi[0] = SENTINEL # inactive-neighbour marker if tmp is not None: @@ -172,7 +172,7 @@ def sphere_on_device(backend, radius, voxel_size=1.0, band=BAND, name="sphere"): io, T = nanovdb.io, nanovdb.tools fg = T.createLevelSetSphere(radius=radius, voxelSize=voxel_size, name=name) onh = T.createOnIndexGrid(fg.grid(0), channels=1, - include_stats=False, include_tiles=False) + includeStats=False, includeTiles=False) sdf = np.array(onh.grid(0).getBlindData(0), dtype=np.float32) tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False); tmp.close() io.writeGrid(tmp.name, onh) @@ -194,9 +194,9 @@ def rebuild(backend, g, phi, vx, half_width): TC.inject(g["grid"], gd["grid"], phi, phi_d) # carry old phi (intersection) phi_d = backend.extrapolate(gd, phi_d, vx) # fill the freshly-dilated ring predicate = cp.abs(phi_d) <= half_width # phi_d[0]=SENTINEL -> False - leaf_masks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) - TC.injectPredicateToMask(gd["grid"], predicate, leaf_masks) - gp = backend.setup(TC.pruneGrid(gd["grid"], leaf_masks)) + leafMasks = cp.zeros(gd["n"] * 8, dtype=cp.uint64) + TC.injectPredicateToMask(gd["grid"], predicate, leafMasks) + gp = backend.setup(TC.pruneGrid(gd["grid"], leafMasks)) phi_p = cp.full(gp["n"] + 1, SENTINEL, dtype=cp.float32) TC.inject(gd["grid"], gp["grid"], phi_d, phi_p) return gp, phi_p @@ -231,7 +231,7 @@ def write_output(backend, g, phi, vx, path, style, band, name="filtered"): fh = builder.toNanoVDB() if style == nanovdb.GridType.OnIndex: io.writeGrid(path, T.createOnIndexGrid(fh.grid(0), channels=1, - include_stats=False, include_tiles=False)) + includeStats=False, includeTiles=False)) else: io.writeGrid(path, fh) @@ -294,7 +294,7 @@ def self_test(backend): sphere = T.createLevelSetSphere(radius=18.0, voxelSize=1.0, name="sphere") io.writeGrid(o_in, T.createOnIndexGrid(sphere.grid(0), channels=1, - include_stats=False, include_tiles=False)) + includeStats=False, includeTiles=False)) print(f"self-test 2 [{backend.NAME}]: OnIndex+SDF sphere -> filter -> OnIndex+SDF") r0b, rb, style2 = run_filter(backend, o_in, o_out, outer_iters=4) ro = io.readGrid(o_out) diff --git a/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py b/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py index 1e2c85e833..0f20503b24 100644 --- a/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py +++ b/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py @@ -247,7 +247,7 @@ def setup(self, handle): if grid is None or grid.data_ptr() == 0: handle.deviceUpload(0, True) grid = handle.deviceGrid(0) - vbm = self.tc.buildVoxelBlockManager(grid, log2_block_width=LOG2_BLOCK_WIDTH) + vbm = self.tc.buildVoxelBlockManager(grid, log2BlockWidth=LOG2_BLOCK_WIDTH) n, bc = int(vbm.lastOffset()), int(vbm.blockCount()) coords = cp.zeros((n + 1, 3), dtype=cp.int32) self.k_decode((bc,), (BLOCK_WIDTH,), diff --git a/nanovdb/nanovdb/python/examples/node_manager.py b/nanovdb/nanovdb/python/examples/node_manager.py index 3ca3feffbe..3e15de87e4 100644 --- a/nanovdb/nanovdb/python/examples/node_manager.py +++ b/nanovdb/nanovdb/python/examples/node_manager.py @@ -7,7 +7,7 @@ they can be visited by index instead of by tree traversal. Each node exposes its origin, per-node stats, and (on leaves) the raw 512-value buffer. For bulk NumPy analytics over every leaf at once, see -bulk_leaf_numpy.py's grid.leaf_values() instead. +bulk_leaf_numpy.py's grid.leafValues() instead. Run with: python node_manager.py """ diff --git a/nanovdb/nanovdb/python/examples/quantize.py b/nanovdb/nanovdb/python/examples/quantize.py index 83935f3b68..229d4ce211 100644 --- a/nanovdb/nanovdb/python/examples/quantize.py +++ b/nanovdb/nanovdb/python/examples/quantize.py @@ -20,9 +20,9 @@ def gridSize_in_kb(handle): def main(): # Source: a 50-radius sphere fog volume at full float precision. src_handle = nanovdb.tools.createFogVolumeSphere(radius=50.0) - src_grid = src_handle.grid() + srcGrid = src_handle.grid() print(f"Source FloatGrid: {gridSize_in_kb(src_handle):.1f} KB, " - f"active voxels = {src_grid.activeVoxelCount()}") + f"active voxels = {srcGrid.activeVoxelCount()}") # Fixed-width quantization. Each subsequent format roughly halves # the per-voxel storage cost; dithering optional. @@ -31,7 +31,7 @@ def main(): ("createNanoGridFp8", "Fp8 (8-bit fixed)"), ("createNanoGridFp4", "Fp4 (4-bit fixed)"), ]: - h = getattr(nanovdb.tools, fn_name)(src_grid, ditherOn=True) + h = getattr(nanovdb.tools, fn_name)(srcGrid, ditherOn=True) print(f" {label}: {gridSize_in_kb(h):.1f} KB, " f"active voxels = {h.grid().activeVoxelCount()}") @@ -40,11 +40,11 @@ def main(): # relative error. -1 tolerance means "uninitialized" so we pass # an explicit value. abs_oracle = nanovdb.tools.AbsDiff(0.05) # ±0.05 per voxel - h_fpn_abs = nanovdb.tools.createNanoGridFpN(src_grid, abs_oracle) + h_fpn_abs = nanovdb.tools.createNanoGridFpN(srcGrid, abs_oracle) print(f" FpN (AbsDiff 0.05): {gridSize_in_kb(h_fpn_abs):.1f} KB") rel_oracle = nanovdb.tools.RelDiff(0.1) # 10% relative error - h_fpn_rel = nanovdb.tools.createNanoGridFpN(src_grid, rel_oracle) + h_fpn_rel = nanovdb.tools.createNanoGridFpN(srcGrid, rel_oracle) print(f" FpN (RelDiff 0.10): {gridSize_in_kb(h_fpn_rel):.1f} KB") # The output is a regular NanoGrid — read-only, but exposes diff --git a/nanovdb/nanovdb/python/examples/raytrace_fog_volume_cuda.py b/nanovdb/nanovdb/python/examples/raytrace_fog_volume_cuda.py index eb1bf3eb4b..e9800d8cdb 100644 --- a/nanovdb/nanovdb/python/examples/raytrace_fog_volume_cuda.py +++ b/nanovdb/nanovdb/python/examples/raytrace_fog_volume_cuda.py @@ -34,7 +34,7 @@ #include extern "C" __global__ -void render_fog(const nanovdb::NanoGrid* d_grid, +void render_fog(const nanovdb::NanoGrid* dGrid, unsigned char* image, int res, float tan_fov, float eye_x, float eye_y, float eye_z, float step, float density_scale) @@ -46,7 +46,7 @@ using Vec3T = nanovdb::math::Vec3f; using RayT = nanovdb::math::Ray; - auto acc = d_grid->tree().getAccessor(); + auto acc = dGrid->tree().getAccessor(); const float px = (2.0f * (x + 0.5f) / res - 1.0f) * tan_fov; const float py = (2.0f * (y + 0.5f) / res - 1.0f) * tan_fov; const float inv = 1.0f / sqrtf(px * px + py * py + 1.0f); @@ -54,7 +54,7 @@ RayT ray(Vec3T(eye_x, eye_y, eye_z), Vec3T(px * inv, py * inv, -inv)); float transmittance = 1.0f; - if (ray.clip(d_grid->indexBBox())) { + if (ray.clip(dGrid->indexBBox())) { for (float t = ray.t0(); t < ray.t1(); t += step) { const Vec3T pos = ray(t); const float sigma = acc.getValue(nanovdb::Coord::Floor(pos)); @@ -103,7 +103,7 @@ def main(): nanovdb.GridType.Float, radius=100.0) handle.deviceUpload(0, True) handle.deviceDownload(0, True) - device_grid = handle.deviceGrid(0) + deviceGrid = handle.deviceGrid(0) bbox = handle.grid(0).indexBBox() import math @@ -118,7 +118,7 @@ def main(): block = (16, 16) grid = ((RES + 15) // 16, (RES + 15) // 16) kernel(grid, block, - (device_grid.data_ptr(), image, RES, cp.float32(tan_fov), + (deviceGrid.data_ptr(), image, RES, cp.float32(tan_fov), cp.float32(eye[0]), cp.float32(eye[1]), cp.float32(eye[2]), cp.float32(STEP), cp.float32(DENSITY_SCALE))) cp.cuda.runtime.deviceSynchronize() diff --git a/nanovdb/nanovdb/python/examples/raytrace_level_set_cuda.py b/nanovdb/nanovdb/python/examples/raytrace_level_set_cuda.py index 6419108496..9a5bda953c 100644 --- a/nanovdb/nanovdb/python/examples/raytrace_level_set_cuda.py +++ b/nanovdb/nanovdb/python/examples/raytrace_level_set_cuda.py @@ -36,7 +36,7 @@ #include extern "C" __global__ -void render_level_set(const nanovdb::NanoGrid* d_grid, +void render_level_set(const nanovdb::NanoGrid* dGrid, unsigned char* image, int res, float tan_fov, float eye_x, float eye_y, float eye_z, float lx, float ly, float lz) @@ -48,7 +48,7 @@ using Vec3T = nanovdb::math::Vec3f; using RayT = nanovdb::math::Ray; - auto acc = d_grid->tree().getAccessor(); + auto acc = dGrid->tree().getAccessor(); const float px = (2.0f * (x + 0.5f) / res - 1.0f) * tan_fov; const float py = (2.0f * (y + 0.5f) / res - 1.0f) * tan_fov; const float inv = 1.0f / sqrtf(px * px + py * py + 1.0f); @@ -56,7 +56,7 @@ RayT ray(Vec3T(eye_x, eye_y, eye_z), Vec3T(px * inv, py * inv, -inv)); unsigned char shade = 0; - if (ray.clip(d_grid->indexBBox())) { + if (ray.clip(dGrid->indexBBox())) { nanovdb::Coord ijk; float v = 0.0f, t = 0.0f; if (nanovdb::math::ZeroCrossing(ray, acc, ijk, v, t)) { @@ -115,7 +115,7 @@ def main(): nanovdb.GridType.Float, radius=100.0) handle.deviceUpload(0, True) handle.deviceDownload(0, True) - device_grid = handle.deviceGrid(0) + deviceGrid = handle.deviceGrid(0) bbox = handle.grid(0).indexBBox() import math @@ -130,7 +130,7 @@ def main(): block = (16, 16) grid = ((RES + 15) // 16, (RES + 15) // 16) kernel(grid, block, - (device_grid.data_ptr(), image, RES, cp.float32(tan_fov), + (deviceGrid.data_ptr(), image, RES, cp.float32(tan_fov), cp.float32(eye[0]), cp.float32(eye[1]), cp.float32(eye[2]), cp.float32(LIGHT[0]), cp.float32(LIGHT[1]), cp.float32(LIGHT[2]))) cp.cuda.runtime.deviceSynchronize() diff --git a/nanovdb/nanovdb/python/examples/sample_from_voxels_cuda.py b/nanovdb/nanovdb/python/examples/sample_from_voxels_cuda.py index e5362c3d06..2603f58188 100644 --- a/nanovdb/nanovdb/python/examples/sample_from_voxels_cuda.py +++ b/nanovdb/nanovdb/python/examples/sample_from_voxels_cuda.py @@ -9,8 +9,8 @@ it writes trilinear samples into an ``(N,)`` array and (optionally) analytic gradients into an ``(N, 3)`` array, all on the GPU: - tools.cuda.sampleFromVoxels(points, d_grid, values, stream) - tools.cuda.sampleFromVoxels(points, d_grid, values, gradients, stream) + tools.cuda.sampleFromVoxels(points, dGrid, values, stream) + tools.cuda.sampleFromVoxels(points, dGrid, values, gradients, stream) Here we sample a level-set sphere along a ray crossing its surface, recover the surface normal from the normalized gradient, and cross-check @@ -48,7 +48,7 @@ def main(): finally: os.unlink(tmp.name) handle.deviceUpload(0, True) - device_grid = handle.deviceGrid(0) + deviceGrid = handle.deviceGrid(0) # Query points marching along +x across the surface (within the # narrow band, where the SDF is meaningful). @@ -59,7 +59,7 @@ def main(): values = cp.empty(xs.size, dtype=cp.float32) gradients = cp.empty((xs.size, 3), dtype=cp.float32) - nanovdb.tools.cuda.sampleFromVoxels(points, device_grid, values, gradients, 0) + nanovdb.tools.cuda.sampleFromVoxels(points, deviceGrid, values, gradients, 0) cp.cuda.Stream.null.synchronize() print(f"Sampling a radius-{radius:.0f} level set along +x:") diff --git a/nanovdb/nanovdb/python/examples/signed_flood_fill_cuda.py b/nanovdb/nanovdb/python/examples/signed_flood_fill_cuda.py index a9c82bec76..86f8e34e2f 100644 --- a/nanovdb/nanovdb/python/examples/signed_flood_fill_cuda.py +++ b/nanovdb/nanovdb/python/examples/signed_flood_fill_cuda.py @@ -8,7 +8,7 @@ outside read as positive background. It runs in place on a device ``FloatGrid`` (or ``DoubleGrid``): - tools.cuda.signedFloodFill(d_grid, verbose, stream) + tools.cuda.signedFloodFill(dGrid, verbose, stream) We upload a level-set sphere, run the flood fill on the device, then use ``tools.cuda.sampleFromVoxels`` to confirm the field is consistently @@ -47,10 +47,10 @@ def main(): finally: os.unlink(tmp.name) handle.deviceUpload(0, True) - device_grid = handle.deviceGrid(0) + deviceGrid = handle.deviceGrid(0) # Propagate signs across the whole grid on the device, in place. - nanovdb.tools.cuda.signedFloodFill(device_grid, False, 0) + nanovdb.tools.cuda.signedFloodFill(deviceGrid, False, 0) print("signedFloodFill: done on the device") # The interior sign is only meaningful right at the band; sample just @@ -58,7 +58,7 @@ def main(): points = cp.asarray([[radius - 2.0, 0.0, 0.0], [radius + 2.0, 0.0, 0.0]], dtype=cp.float32) values = cp.empty(2, dtype=cp.float32) - nanovdb.tools.cuda.sampleFromVoxels(cp.ascontiguousarray(points), device_grid, values, 0) + nanovdb.tools.cuda.sampleFromVoxels(cp.ascontiguousarray(points), deviceGrid, values, 0) cp.cuda.Stream.null.synchronize() inside, outside = (float(v) for v in cp.asnumpy(values)) print(f" sdf just inside surface = {inside:+.3f} (expect < 0)") @@ -66,7 +66,7 @@ def main(): assert inside < 0.0 < outside # The flooded grid must still be structurally valid. - assert nanovdb.tools.cuda.isValid(device_grid, nanovdb.CheckMode.Full) + assert nanovdb.tools.cuda.isValid(deviceGrid, nanovdb.CheckMode.Full) print("OK: device signed flood fill produced a consistent, valid SDF") diff --git a/nanovdb/nanovdb/python/examples/validate_cuda.py b/nanovdb/nanovdb/python/examples/validate_cuda.py index e0e1fef549..ea2dfdc9a9 100644 --- a/nanovdb/nanovdb/python/examples/validate_cuda.py +++ b/nanovdb/nanovdb/python/examples/validate_cuda.py @@ -46,25 +46,25 @@ def main(): finally: os.unlink(tmp.name) handle.deviceUpload(0, True) - device_grid = handle.deviceGrid(0) + deviceGrid = handle.deviceGrid(0) # Structural validation on the device, partial and full. - partial = nanovdb.tools.cuda.isValid(device_grid, nanovdb.CheckMode.Partial) - full = nanovdb.tools.cuda.isValid(device_grid, nanovdb.CheckMode.Full) + partial = nanovdb.tools.cuda.isValid(deviceGrid, nanovdb.CheckMode.Partial) + full = nanovdb.tools.cuda.isValid(deviceGrid, nanovdb.CheckMode.Full) print(f"isValid: Partial={partial}, Full={full}") assert partial and full # Checksum round-trip: recompute in place, then verify it matches. - nanovdb.tools.cuda.updateChecksum(device_grid, nanovdb.CheckMode.Full) - checksum = nanovdb.tools.cuda.evalChecksum(device_grid, nanovdb.CheckMode.Full) - ok = nanovdb.tools.cuda.validateChecksum(device_grid, nanovdb.CheckMode.Full) + nanovdb.tools.cuda.updateChecksum(deviceGrid, nanovdb.CheckMode.Full) + checksum = nanovdb.tools.cuda.evalChecksum(deviceGrid, nanovdb.CheckMode.Full) + ok = nanovdb.tools.cuda.validateChecksum(deviceGrid, nanovdb.CheckMode.Full) print(f"checksum: {checksum} validateChecksum(Full)={ok}") assert ok # Recompute per-node statistics on the device. - nanovdb.tools.cuda.updateGridStats(device_grid) + nanovdb.tools.cuda.updateGridStats(deviceGrid) print("updateGridStats: OK") - assert nanovdb.tools.cuda.isValid(device_grid, nanovdb.CheckMode.Full) + assert nanovdb.tools.cuda.isValid(deviceGrid, nanovdb.CheckMode.Full) print("OK: device-side validation, checksum, and stats all pass") diff --git a/nanovdb/nanovdb/python/examples/voxels_to_grid_cuda.py b/nanovdb/nanovdb/python/examples/voxels_to_grid_cuda.py index 266294de6b..4c9c579d0b 100644 --- a/nanovdb/nanovdb/python/examples/voxels_to_grid_cuda.py +++ b/nanovdb/nanovdb/python/examples/voxels_to_grid_cuda.py @@ -55,13 +55,13 @@ def main(): # Rasterize into a device OnIndex grid in one call. handle = nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, 1.0, 0) - device_grid = handle.deviceGrid(0) + deviceGrid = handle.deviceGrid(0) print(f"voxelsToOnIndexGrid -> {handle.gridType(0)} handle, " - f"device_grid={type(device_grid).__name__}") + f"deviceGrid={type(deviceGrid).__name__}") print(f" grid(0) before download = {handle.grid(0)} (device-built)") # Validate entirely on the device, then download for host-side metadata. - print(f" tools.cuda.isValid(device_grid) = {nanovdb.tools.cuda.isValid(device_grid)}") + print(f" tools.cuda.isValid(deviceGrid) = {nanovdb.tools.cuda.isValid(deviceGrid)}") handle.deviceDownload(0, True) host_grid = handle.grid(0) active = host_grid.activeVoxelCount() diff --git a/nanovdb/nanovdb/python/test/TestGpuInterop.py b/nanovdb/nanovdb/python/test/TestGpuInterop.py index 326eb46f26..b85e93dbeb 100644 --- a/nanovdb/nanovdb/python/test/TestGpuInterop.py +++ b/nanovdb/nanovdb/python/test/TestGpuInterop.py @@ -478,9 +478,9 @@ def test_block_manager_properties(self): dh, dg = _build_device_onindex_grid(100.0) vbm = nanovdb.tools.cuda.buildVoxelBlockManager(dg, 6, 0, 0, 0, 0) self.assertGreater(vbm.blockCount(), 0) - # block_width / log2_block_width / jump_map_length are PROPERTIES. - self.assertEqual(vbm.block_width, 64) - self.assertEqual(vbm.log2_block_width, 6) + # blockWidth / log2BlockWidth / jumpMapLength are PROPERTIES. + self.assertEqual(vbm.blockWidth, 64) + self.assertEqual(vbm.log2BlockWidth, 6) # firstOffset == 1 (mod BlockWidth) for a full grid. self.assertEqual(vbm.firstOffset(), 1) self.assertGreater(vbm.lastOffset(), 0) @@ -538,8 +538,8 @@ def test_gather_box_stencil_dtypes(self): def test_gather_rejects_noncontiguous_grid(self): """gatherBoxStencil / activeVoxelCoords raise a clear ValueError (rather than an illegal memory access) on a grid whose active-voxel indexing is - NOT contiguous -- e.g. createOnIndexGrid's default include_stats / - include_tiles, which add value slots past activeVoxelCount.""" + NOT contiguous -- e.g. createOnIndexGrid's default includeStats / + includeTiles, which add value slots past activeVoxelCount.""" cp = _require_cupy(self) dh, dg = _build_device_onindex_grid(20.0) # createOnIndexGrid defaults: stats+tiles N = int(nanovdb.tools.cuda.buildVoxelBlockManager(dg, 9, 0, 0, 0, 0).lastOffset()) + 1 diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index df35609221..f465efb233 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -641,7 +641,7 @@ def test_readonly_accessors_have_neither_setvoxel_nor_nodeinfo(self): class TestTreeNodeWalking(unittest.TestCase): """Walk a grid's tree from Python: Grid.tree(), Root/Upper/Lower/Leaf node access and metadata, per-leaf zero-copy values() and bulk - grid.leaf_values() NumPy views, NodeManager + createNodeManager.""" + grid.leafValues() NumPy views, NodeManager + createNodeManager.""" @classmethod def setUpClass(cls): @@ -708,7 +708,7 @@ def test_bulk_leaf_values(self): import numpy as np except ImportError: self.skipTest("numpy not installed") - bulk = self.g.leaf_values() + bulk = self.g.leafValues() self.assertEqual(bulk.shape, (self.tree.nodeCount(0), 512)) self.assertEqual(bulk.dtype, np.float32) # First row should match per-leaf values(). @@ -727,7 +727,7 @@ def test_bulk_leaf_values_empty_grid_returns_empty_array(self): empty_h = nanovdb.tools.createFloatGrid( 0.0, "empty", nanovdb.GridClass.Unknown, lambda ijk: 0.0, empty_bbox) - bulk = empty_h.grid().leaf_values() + bulk = empty_h.grid().leafValues() self.assertEqual(bulk.shape, (0, 512)) self.assertEqual(bulk.dtype, np.float32) @@ -849,7 +849,7 @@ def test_grid_leaf_values_temporary(self): import numpy # noqa: F401 except ImportError: self.skipTest("numpy not installed") - bulk = nanovdb.tools.createFogVolumeSphere().grid().leaf_values() + bulk = nanovdb.tools.createFogVolumeSphere().grid().leafValues() self._force_gc() self.assertEqual(bulk.shape[1], 512) _ = float(bulk[0, 0]) @@ -899,7 +899,7 @@ def _make_cube_on_index_grid(self): 0.0, "cube", nanovdb.GridClass.Unknown, lambda ijk: 1.0, bbox) return nanovdb.tools.createOnIndexGrid( - float_h.grid(), include_stats=False, include_tiles=False) + float_h.grid(), includeStats=False, includeTiles=False) def test_create_on_index_grid(self): h = self._make_cube_on_index_grid() @@ -919,7 +919,7 @@ def test_create_on_index_grid_rejects_unsupported_source(self): def test_build_voxel_block_manager_handle(self): h = self._make_cube_on_index_grid() g = h.grid() - vbm = nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=6) + vbm = nanovdb.tools.buildVoxelBlockManager(g, log2BlockWidth=6) self.assertGreater(vbm.blockCount(), 0) self.assertEqual(vbm.firstOffset(), 1) self.assertEqual(vbm.lastOffset(), g.activeVoxelCount()) @@ -931,21 +931,21 @@ def test_buffers_zero_copy_shape_and_dtype(self): except ImportError: self.skipTest("numpy not installed") h = self._make_cube_on_index_grid() - # log2_block_width=6 -> JumpMapLength=1. - vbm6 = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2_block_width=6) - self.assertEqual(vbm6.log2_block_width, 6) - self.assertEqual(vbm6.block_width, 64) - self.assertEqual(vbm6.jump_map_length, 1) + # log2BlockWidth=6 -> JumpMapLength=1. + vbm6 = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2BlockWidth=6) + self.assertEqual(vbm6.log2BlockWidth, 6) + self.assertEqual(vbm6.blockWidth, 64) + self.assertEqual(vbm6.jumpMapLength, 1) fl = vbm6.firstLeafID() self.assertEqual(fl.shape, (vbm6.blockCount(),)) self.assertEqual(fl.dtype, np.uint32) jm = vbm6.jumpMap() self.assertEqual(jm.shape, (vbm6.blockCount(), 1)) self.assertEqual(jm.dtype, np.uint64) - # log2_block_width=7 -> JumpMapLength=2 (independent build, separate + # log2BlockWidth=7 -> JumpMapLength=2 (independent build, separate # allocation; the jumpMap shape comes from the handle, not the caller). - vbm7 = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2_block_width=7) - self.assertEqual(vbm7.jump_map_length, 2) + vbm7 = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2BlockWidth=7) + self.assertEqual(vbm7.jumpMapLength, 2) self.assertEqual(vbm7.jumpMap().shape, (vbm7.blockCount(), 2)) def test_decode_block_zero(self): @@ -955,7 +955,7 @@ def test_decode_block_zero(self): self.skipTest("numpy not installed") h = self._make_cube_on_index_grid() g = h.grid() - vbm = nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=6) + vbm = nanovdb.tools.buildVoxelBlockManager(g, log2BlockWidth=6) leaf_index, voxel_offset = vbm.decodeBlock(g, 0) self.assertEqual(leaf_index.shape, (64,)) self.assertEqual(leaf_index.dtype, np.uint32) @@ -965,7 +965,7 @@ def test_decode_block_zero(self): fl = np.asarray(vbm.firstLeafID()) jm = np.asarray(vbm.jumpMap()) li_free, vo_free = nanovdb.tools.decodeInverseMaps( - g, int(fl[0]), jm[0], vbm.firstOffset(), log2_block_width=6) + g, int(fl[0]), jm[0], vbm.firstOffset(), log2BlockWidth=6) self.assertTrue(np.array_equal(leaf_index, li_free)) self.assertTrue(np.array_equal(voxel_offset, vo_free)) @@ -980,24 +980,24 @@ def test_log2_block_width_out_of_range(self): h = self._make_cube_on_index_grid() g = h.grid() with self.assertRaises(ValueError): - nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=5) + nanovdb.tools.buildVoxelBlockManager(g, log2BlockWidth=5) with self.assertRaises(ValueError): - nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=10) + nanovdb.tools.buildVoxelBlockManager(g, log2BlockWidth=10) def test_build_voxel_block_manager_rejects_misaligned_first_offset(self): - # first_offset must satisfy first_offset == 1 (mod BlockWidth). - # For log2_block_width=6, BlockWidth=64, so 1, 65, 129, ... are valid + # firstOffset must satisfy firstOffset == 1 (mod BlockWidth). + # For log2BlockWidth=6, BlockWidth=64, so 1, 65, 129, ... are valid # but 2 is not. h = self._make_cube_on_index_grid() g = h.grid() with self.assertRaises(ValueError): nanovdb.tools.buildVoxelBlockManager( - g, log2_block_width=6, first_offset=2) - # And the wider-block case: log2_block_width=7 -> BlockWidth=128, - # first_offset=65 is valid for width=6 but misaligned for width=7. + g, log2BlockWidth=6, firstOffset=2) + # And the wider-block case: log2BlockWidth=7 -> BlockWidth=128, + # firstOffset=65 is valid for width=6 but misaligned for width=7. with self.assertRaises(ValueError): nanovdb.tools.buildVoxelBlockManager( - g, log2_block_width=7, first_offset=65) + g, log2BlockWidth=7, firstOffset=65) def test_decode_inverse_maps_rejects_bad_first_leaf_id(self): try: @@ -1006,25 +1006,25 @@ def test_decode_inverse_maps_rejects_bad_first_leaf_id(self): self.skipTest("numpy not installed") h = self._make_cube_on_index_grid() g = h.grid() - vbm = nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=6) + vbm = nanovdb.tools.buildVoxelBlockManager(g, log2BlockWidth=6) jm0 = np.asarray(vbm.jumpMap())[0] n_leaves = g.tree().nodeCount(0) with self.assertRaises(IndexError): nanovdb.tools.decodeInverseMaps( - g, n_leaves, jm0, vbm.firstOffset(), log2_block_width=6) + g, n_leaves, jm0, vbm.firstOffset(), log2BlockWidth=6) def test_build_voxel_block_manager_rejects_undersized_n_blocks(self): - # Caller-supplied n_blocks must hold at least - # ceil((last_offset - first_offset + 1) / BlockWidth) blocks; + # Caller-supplied nBlocks must hold at least + # ceil((lastOffset - firstOffset + 1) / BlockWidth) blocks; # smaller values would silently truncate coverage. h = self._make_cube_on_index_grid() g = h.grid() - # The cube grid has ~9261 active voxels, so at log2_block_width=6 + # The cube grid has ~9261 active voxels, so at log2BlockWidth=6 # (BlockWidth=64) the minimum is roughly 145 blocks. Passing 1 # must be rejected. with self.assertRaises(ValueError): nanovdb.tools.buildVoxelBlockManager( - g, log2_block_width=6, n_blocks=1) + g, log2BlockWidth=6, nBlocks=1) def test_build_voxel_block_manager_rejects_non_on_index_grid(self): # FloatGrid is not an OnIndexGrid. @@ -1040,7 +1040,7 @@ def test_untouched_blocks_trip_sentinel_guard(self): # decodeBlock (raising ValueError). The sweep must therefore see # only two outcomes per block: a successful decode or a sentinel # ValueError — no segfaults, no IndexError (those would indicate - # block_index out of range, not sentinel), and no silent wrong + # blockIndex out of range, not sentinel), and no silent wrong # decode. try: import numpy as np # noqa: F401 @@ -1048,7 +1048,7 @@ def test_untouched_blocks_trip_sentinel_guard(self): self.skipTest("numpy not installed") h = self._make_cube_on_index_grid() g = h.grid() - vbm = nanovdb.tools.buildVoxelBlockManager(g, log2_block_width=6) + vbm = nanovdb.tools.buildVoxelBlockManager(g, log2BlockWidth=6) n_leaves = g.tree().nodeCount(0) fl = vbm.firstLeafID() for b in range(vbm.blockCount()): @@ -1082,7 +1082,7 @@ def test_default_constructed_handle_returns_empty_arrays(self): self.assertEqual(fl.shape, (0,)) self.assertEqual(fl.dtype, np.uint32) jm = np.asarray(vbm.jumpMap()) - # Default-constructed handle uses log2_block_width=6 -> JumpMapLength=1. + # Default-constructed handle uses log2BlockWidth=6 -> JumpMapLength=1. self.assertEqual(jm.shape, (0, 1)) self.assertEqual(jm.dtype, np.uint64) @@ -1093,7 +1093,7 @@ def test_reset_handle_returns_empty_arrays(self): except ImportError: self.skipTest("numpy not installed") h = self._make_cube_on_index_grid() - vbm = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2_block_width=6) + vbm = nanovdb.tools.buildVoxelBlockManager(h.grid(), log2BlockWidth=6) self.assertGreater(vbm.blockCount(), 0) vbm.reset() self.assertEqual(vbm.blockCount(), 0) From 1214a4ee869957ecb23b629ea9bfccdcbaad4ff9 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 19 Aug 2026 12:08:58 +1200 Subject: [PATCH 46/48] nanovdb python: bind DeviceBuffer/DeviceGridHandle.recordUse for external streams MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The bindings' interop surface hands raw device pointers to CuPy / PyTorch / Numba via device_ptr(), __cuda_array_interface__ and __dlpack__, and those frameworks launch kernels on their own non-blocking streams. Such work is invisible to the buffer's automatic upload/download use tracking, so the cudaFreeAsync issued when the buffer is cleared or destroyed could race it — the C++ side grew DeviceBuffer::recordUse(device, stream) for exactly this caller, but Python had no way to reach it. Bind recordUse(stream, device=-1) on DeviceBuffer and on DeviceGridHandle (which owns its buffer internally and is where every real workflow's owning buffer lives). Both forward to a shared recordUseChecked helper that validates the device id — the C++ method indexes per-device tracking state unchecked — raising IndexError on an out-of-range id, with -1 selecting the current CUDA device. The device_ptr and __cuda_array_interface__ docstrings now point callers at recordUse. Non-owning (from_external) buffers accept the call as a no-op, matching the C++ behavior. Add TestRecordUse to TestGpuInterop.py: a CuPy reduction enqueued on a non-blocking stream against a zero-copy CAI view, recorded, and validated after the handle is destroyed while the work may still be in flight; plus default-stream/explicit-device acceptance, IndexError on a bad device id, and the non-owning no-op. Also normalize the cuda/ sources' includes of shared binding headers: add the python source dir to the target's private include paths so they use plain names ("PyGridHandle.h", "BuildTypes.def") instead of "../"-relative paths, which the rest of the codebase does not use. Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/CMakeLists.txt | 5 +- nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc | 4 +- nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h | 42 +++++++++++++- .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 22 ++++++-- .../python/cuda/PyDeviceNodeManager.cu | 6 +- .../python/cuda/PyUnifiedGridHandle.cu | 4 +- nanovdb/nanovdb/python/test/TestGpuInterop.py | 56 +++++++++++++++++++ 7 files changed, 127 insertions(+), 12 deletions(-) diff --git a/nanovdb/nanovdb/python/CMakeLists.txt b/nanovdb/nanovdb/python/CMakeLists.txt index bf9f382463..899d5cb095 100644 --- a/nanovdb/nanovdb/python/CMakeLists.txt +++ b/nanovdb/nanovdb/python/CMakeLists.txt @@ -58,7 +58,10 @@ nanobind_add_module(nanovdb_python NB_STATIC cuda/PyDeviceGridChecksum.cu ) -target_include_directories(nanovdb_python PRIVATE ${CUDA_INCLUDE_DIRECTORY}) +# CMAKE_CURRENT_SOURCE_DIR lets sources in cuda/ include the shared binding +# headers by their plain names ("PyGridHandle.h", "BuildTypes.def") instead of +# "../"-relative paths, matching the include style used across the codebase. +target_include_directories(nanovdb_python PRIVATE ${CMAKE_CURRENT_SOURCE_DIR} ${CUDA_INCLUDE_DIRECTORY}) target_link_libraries(nanovdb_python PRIVATE nanovdb ${CUDA_LIBRARIES} ${NANOVDB_BLOSC} ${NANOVDB_ZLIB} ${NANOVDB_OPENVDB} ${NANOVDB_TBB}) target_compile_definitions(nanovdb_python PRIVATE ${NANOVDB_USE_CUDA_FLAG} ${NANOVDB_USE_BLOSC_FLAG} ${NANOVDB_USE_ZLIB_FLAG} ${NANOVDB_USE_OPENVDB_FLAG} ${NANOVDB_USE_TBB_FLAG}) set_target_properties(nanovdb_python PROPERTIES OUTPUT_NAME "nanovdb") diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc index fad468080b..b3d967e364 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.cc @@ -46,7 +46,9 @@ void defineDeviceBuffer(nb::module_& m) "does NOT take ownership: it will never free either pointer, so " "the caller must keep both allocations alive for the buffer's " "lifetime. The device pointer is associated with the current CUDA " - "device."); + "device.") + .def("recordUse", &recordUseChecked, "stream"_a, "device"_a = -1, + kRecordUseDoc); } } // namespace pynanovdb diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h index d8cec8de14..7372edbcd8 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h @@ -9,8 +9,11 @@ #include #include +#include #include + +#include #endif namespace nb = nanobind; @@ -19,6 +22,39 @@ namespace pynanovdb { #ifdef NANOVDB_USE_CUDA +/// @brief Bounds-checked forwarder for DeviceBuffer::recordUse, shared by the +/// DeviceBuffer and DeviceGridHandle bindings. device == -1 selects the +/// current CUDA device. recordUse itself indexes per-device tracking +/// state unchecked, so validate here where we can raise a Python +/// exception instead. +inline void recordUseChecked(nanovdb::cuda::DeviceBuffer& buf, uintptr_t stream, int device) +{ + int count = 0; + cudaCheck(cudaGetDeviceCount(&count)); + if (device < 0) cudaCheck(cudaGetDevice(&device)); + if (device >= count) { + const std::string msg = "recordUse: device id " + std::to_string(device) + + " out of range [0, " + std::to_string(count) + ")."; + throw nb::index_error(msg.c_str()); + } + nb::gil_scoped_release release; + buf.recordUse(device, reinterpret_cast(stream)); +} + +/// @brief Docstring shared by the DeviceBuffer and DeviceGridHandle recordUse +/// bindings (the two forward to the same underlying buffer method). +inline constexpr char kRecordUseDoc[] = + "Record that this buffer's device data was just used on `stream` (a raw " + "cudaStream_t as a Python int, e.g. cupy.cuda.Stream.ptr), so the device " + "free issued when the buffer is cleared or destroyed is ordered after " + "that work. Uploads/downloads record themselves automatically; call this " + "after enqueuing your own kernels or copies against device_ptr() / " + "__cuda_array_interface__ / __dlpack__ on a non-blocking stream — " + "without it such work is only safe if you synchronize before dropping " + "the buffer. device selects which device's buffer was used (-1 = the " + "current CUDA device). No-op on non-owning (from_external) buffers, " + "which never free their pointers."; + /// @brief Bind the device-interop surface (CUDA Array Interface / DLPack, raw /// device/host pointers, streams) onto a device-buffer-like class. /// @@ -40,7 +76,11 @@ void addDeviceInterop(nb::class_& cls) return reinterpret_cast(buf.deviceData()); }, "Raw device pointer to the current device's buffer as a Python int " - "(0 if no device allocation exists yet)."); + "(0 if no device allocation exists yet). Work you enqueue against " + "this pointer on a non-blocking stream is invisible to the buffer's " + "lifetime tracking: record it afterwards where the buffer supports " + "it (DeviceBuffer.recordUse / DeviceGridHandle.recordUse), or " + "synchronize before the buffer is destroyed."); cls.def( "host_ptr", diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index ed897bef2f..e3c73d934c 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -2,7 +2,8 @@ // SPDX-License-Identifier: Apache-2.0 #ifdef NANOVDB_USE_CUDA -#include "../PyGridHandle.h" +#include "PyGridHandle.h" +#include "PyDeviceBuffer.h" // for recordUseChecked / kRecordUseDoc #include #include @@ -54,7 +55,7 @@ static nb::object pyDeviceGrid(nb::handle py_handle, uint32_t n) return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ : nb::none(); \ } -#include "../BuildTypes.def" +#include "BuildTypes.def" default: return nb::none(); } @@ -114,7 +115,17 @@ void defineDeviceGridHandle(nb::module_& m) }, "Raw device pointer to the base of the whole device buffer as a " "Python int (0 if the handle has not been uploaded to the device " - "yet).") + "yet). Work you enqueue against this pointer on a non-blocking " + "stream is invisible to the buffer's lifetime tracking: call " + "recordUse(stream) afterwards, or synchronize before the handle " + "is destroyed.") + .def( + "recordUse", + [](GridHandle& handle, uintptr_t stream, int device) { + recordUseChecked(handle.buffer(), stream, device); + }, + "stream"_a, "device"_a = -1, + kRecordUseDoc) .def_prop_ro( "__cuda_array_interface__", [](GridHandle& handle) { @@ -133,7 +144,10 @@ void defineDeviceGridHandle(nb::module_& m) }, "CUDA Array Interface (v3) view of the whole device buffer as 1-D " "uint8 — lets CuPy / Numba / PyTorch consume the serialized grid " - "bytes zero-copy. Returns a null data pointer until deviceUpload.") + "bytes zero-copy. Returns a null data pointer until deviceUpload. " + "After enqueuing work on this view from a non-blocking stream, " + "call recordUse(stream) so the buffer's device free is ordered " + "after it.") .def( "__dlpack_device__", [](GridHandle&) { diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu index 34d819c5f1..495a87e903 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceNodeManager.cu @@ -2,7 +2,7 @@ // SPDX-License-Identifier: Apache-2.0 #ifdef NANOVDB_USE_CUDA -#include "../PyTree.h" +#include "PyTree.h" #include @@ -56,7 +56,7 @@ static nb::object pyDeviceNodeMgr(nb::handle py_self) if (auto* m = handle.template deviceMgr()) { \ return nb::cast(m, nb::rv_policy::reference, py_self); \ } -#include "../BuildTypes.def" +#include "BuildTypes.def" return nb::none(); } @@ -141,7 +141,7 @@ static void defineCreateDeviceNodeManager(nb::module_& m) if (auto obj = tryCreateDeviceNodeManager(py_grid, s); obj.is_valid()) { \ return obj; \ } -#include "../BuildTypes.def" +#include "BuildTypes.def" throw nb::type_error( "createDeviceNodeManager: argument is not a NanoVDB device " "grid of any bound BuildT. Pass a device grid obtained from " diff --git a/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.cu index ac0657a5b8..93402974cd 100644 --- a/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyUnifiedGridHandle.cu @@ -3,7 +3,7 @@ #ifdef NANOVDB_USE_CUDA #include "PyUnifiedGridHandle.h" -#include "../PyGridHandle.h" +#include "PyGridHandle.h" #include @@ -51,7 +51,7 @@ static nb::object pyUnifiedDeviceGrid(nb::handle py_handle, uint32_t n) return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \ : nb::none(); \ } -#include "../BuildTypes.def" +#include "BuildTypes.def" default: return nb::none(); } diff --git a/nanovdb/nanovdb/python/test/TestGpuInterop.py b/nanovdb/nanovdb/python/test/TestGpuInterop.py index b85e93dbeb..a649105c60 100644 --- a/nanovdb/nanovdb/python/test/TestGpuInterop.py +++ b/nanovdb/nanovdb/python/test/TestGpuInterop.py @@ -295,6 +295,62 @@ def test_from_buffer_rejects_garbage(self): cp.cuda.set_allocator(prev) +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestRecordUse(unittest.TestCase): + """DeviceBuffer/DeviceGridHandle.recordUse: order the buffer's device free + after work enqueued on external non-blocking streams (the CAI/DLPack + interop pattern, where kernels launched against device_ptr() are invisible + to the buffer's automatic upload/download tracking).""" + + def test_free_ordered_after_external_stream_work(self): + cp = _require_cupy(self) + dh, _ = _build_device_onindex_grid(20.0) + # Baseline: reduce the serialized grid bytes while the handle is alive. + view = cp.asarray(dh) # zero-copy CAI view of the device buffer + expected = int(view.sum()) + # Enqueue the same reduction on a NON-BLOCKING stream, record the use, + # and drop the handle while the reduction may still be in flight. The + # buffer's cudaFreeAsync must be ordered after the recorded event; if + # it were not, the reduction would race the free and read freed memory. + s = cp.cuda.Stream(non_blocking=True) + with s: + pending = view.sum() # device scalar; no host sync yet + dh.recordUse(s.ptr) + del dh, view + s.synchronize() + self.assertEqual(int(pending), expected) + + def test_default_stream_and_explicit_device(self): + dh, _ = _build_device_onindex_grid(10.0) + # Default stream (0) and the current device, both explicit and implied. + dh.recordUse(0) + dh.recordUse(0, device=0) + + def test_rejects_out_of_range_device(self): + dh, _ = _build_device_onindex_grid(10.0) + with self.assertRaises(IndexError): + dh.recordUse(0, device=1_000_000) + + def test_noop_on_non_owning_buffer(self): + cp = _require_cupy(self) + prev = cp.cuda.get_allocator() + cp.cuda.set_allocator(cp.cuda.malloc_managed) + try: + mbuf = cp.zeros(256, dtype=cp.uint8) + ptr = int(mbuf.data.ptr) + ext = nanovdb.cuda.DeviceBuffer.from_external(256, ptr, ptr) + # Non-owning buffers never free their pointers, so recordUse has + # nothing to order — it must be accepted and be a no-op. + ext.recordUse(0) + finally: + cp.cuda.set_allocator(prev) + + @unittest.skipIf( not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" ) From 34bafa9519bf17dd193938b71b0a700d850a2c1a Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Wed, 19 Aug 2026 19:34:16 +1200 Subject: [PATCH 47/48] nanovdb python: validate value-indexed arrays and geometry; order CAI/DLPack exports MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address the Copilot review on #2225: every kernel-facing binding that indexes a user array by grid value index or leaf ID now checks the array covers the grid's count (ValueError instead of an out-of-bounds device access), geometric parameters (voxelSize, halfWidth) are rejected unless finite and positive before reaching Map::set (which only debug-asserts), and the CAI / DLPack exports order the consumer against the buffer's tracked prior uses — the CAI export backs its stream=1 claim by ordering the legacy default stream, and __dlpack__ honors the protocol's consumer-stream argument. The ordering activates once DeviceBuffer::orderAfterPriorUses becomes public upstream; the call sites are written against those semantics already. Co-Authored-By: Claude Fable 5 Signed-off-by: Jonathan Swartz --- nanovdb/nanovdb/python/NanoVDBModule.cc | 4 + nanovdb/nanovdb/python/PyValidate.h | 33 ++++ nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h | 62 +++++++- .../nanovdb/python/cuda/PyDeviceGridHandle.cu | 13 +- .../python/cuda/PyDeviceVoxelBlockManager.cu | 36 ++++- nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu | 29 ++++ nanovdb/nanovdb/python/cuda/PyInjectData.cu | 26 +++ nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu | 6 + nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu | 7 + nanovdb/nanovdb/python/cuda/PyPruneGrid.cu | 14 ++ nanovdb/nanovdb/python/test/TestGpuInterop.py | 150 ++++++++++++++++++ nanovdb/nanovdb/python/test/TestNanoVDB.py | 10 ++ 12 files changed, 381 insertions(+), 9 deletions(-) create mode 100644 nanovdb/nanovdb/python/PyValidate.h diff --git a/nanovdb/nanovdb/python/NanoVDBModule.cc b/nanovdb/nanovdb/python/NanoVDBModule.cc index 3ff9a2ba63..010d4259da 100644 --- a/nanovdb/nanovdb/python/NanoVDBModule.cc +++ b/nanovdb/nanovdb/python/NanoVDBModule.cc @@ -26,6 +26,7 @@ #endif #include "PyBuildGrid.h" #include "PyGridHandle.h" +#include "PyValidate.h" #include "PyHostBuffer.h" #include "PyIO.h" #include "PyMath.h" @@ -285,6 +286,9 @@ void defineGrid(nb::module_& m) "the checksum. Host twin of tools.cuda.setGridClass for device grids.") .def("setTransform", [](GridData& g, double voxelSize, const Vec3d& translation) { + // Map::set only debug-asserts positivity, so validate here to + // keep a singular / non-finite transform out of the header. + requirePositiveFinite(voxelSize, "setTransform", "voxelSize"); // Uniform scale + translation index->world map, then recompute the // world-space AABB from the index bbox under the new map (mirrors // how GridStats sets mWorldBBox). diff --git a/nanovdb/nanovdb/python/PyValidate.h b/nanovdb/nanovdb/python/PyValidate.h new file mode 100644 index 0000000000..0efbdfa377 --- /dev/null +++ b/nanovdb/nanovdb/python/PyValidate.h @@ -0,0 +1,33 @@ +// Copyright Contributors to the OpenVDB Project +// SPDX-License-Identifier: Apache-2.0 +#ifndef NANOVDB_PYVALIDATE_HAS_BEEN_INCLUDED +#define NANOVDB_PYVALIDATE_HAS_BEEN_INCLUDED + +#include + +#include +#include + +namespace pynanovdb { + +/// @brief Raise a Python ValueError unless @a value is a finite, strictly +/// positive number. Used to validate geometric parameters (voxelSize, +/// narrow-band halfWidth) before they reach nanovdb::Map::set and the +/// grid builders, which only debug-assert positivity — release builds +/// would otherwise persist a singular / non-finite transform in the +/// grid header. +inline void requirePositiveFinite(double value, const char* fnName, const char* paramName) +{ + if (!(std::isfinite(value) && value > 0.0)) { + std::string msg(fnName); + msg += ": "; + msg += paramName; + msg += " must be a finite, strictly positive number; got "; + msg += std::to_string(value); + throw nanobind::value_error(msg.c_str()); + } +} + +} // namespace pynanovdb + +#endif diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h index 7372edbcd8..c727c0b008 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h +++ b/nanovdb/nanovdb/python/cuda/PyDeviceBuffer.h @@ -41,6 +41,53 @@ inline void recordUseChecked(nanovdb::cuda::DeviceBuffer& buf, uintptr_t stream, buf.recordUse(device, reinterpret_cast(stream)); } +/// @brief Order the buffer's tracked prior uses (uploads / downloads / +/// recordUse'd kernels) before work subsequently issued on @a stream, +/// when the buffer type exposes use tracking. No-op overload for buffer +/// types without it (UnifiedBuffer). Call with a trailing 0 so the +/// tracking overload is preferred when available. +/// @note DeviceBuffer::orderAfterPriorUses is still private upstream, so the +/// SFINAE currently selects the no-op for DeviceBuffer too — the +/// CAI/DLPack exports below only gain real event ordering once the +/// upstream change making it public (and chaining recordUse) lands and +/// is merged in. The call sites are written against the final +/// semantics so no binding change is needed at that point. +template +inline auto orderPriorUsesBefore(const BufferT& buf, cudaStream_t stream, int) + -> decltype(buf.orderAfterPriorUses(0, stream)) +{ + int device = 0; + cudaCheck(cudaGetDevice(&device)); + buf.orderAfterPriorUses(device, stream); +} +template +inline void orderPriorUsesBefore(const BufferT&, cudaStream_t, long) {} + +/// @brief Resolve the DLPack-protocol `stream` argument to the cudaStream_t +/// the export must be ordered on. Per the protocol: None/0/1 = the +/// legacy default stream, 2 = the per-thread default stream, -1 = the +/// consumer does its own synchronization (no ordering requested; return +/// false), any other integer = a raw cudaStream_t handle. +inline bool resolveDlpackStream(nb::handle stream, cudaStream_t& out) +{ + if (stream.is_none()) { + out = nullptr; // legacy default stream + return true; + } + const long long v = nb::cast(stream); + if (v == -1) return false; + if (v == 2) { + out = cudaStreamPerThread; + return true; + } + if (v == 0 || v == 1) { + out = nullptr; // legacy default stream + return true; + } + out = reinterpret_cast(static_cast(v)); + return true; +} + /// @brief Docstring shared by the DeviceBuffer and DeviceGridHandle recordUse /// bindings (the two forward to the same underlying buffer method). inline constexpr char kRecordUseDoc[] = @@ -95,7 +142,11 @@ void addDeviceInterop(nb::class_& cls) [](const BufferT& buf) { // Hand-rolled CUDA Array Interface (version 3) describing the whole // device buffer as a 1-D contiguous uint8 array. stream=1 selects - // the legacy default stream per the CAI v3 spec. + // the legacy default stream per the CAI v3 spec — make that claim + // true by ordering the legacy default stream after the buffer's + // tracked prior uses (async uploads, recordUse'd kernels), so a + // consumer that synchronizes on it per the spec sees complete data. + orderPriorUsesBefore(buf, cudaStream_t(0), 0); nb::dict iface; iface["shape"] = nb::make_tuple(buf.size()); iface["typestr"] = "|u1"; @@ -123,8 +174,15 @@ void addDeviceInterop(nb::class_& cls) cls.def( "__dlpack__", - [](nb::handle self, nb::handle /*stream*/) { + [](nb::handle self, nb::handle stream) { const BufferT& buf = nb::cast(self); + // Honor the consumer-provided stream per the DLPack protocol: + // order it after the buffer's tracked prior uses so the consumer + // cannot read a partially-written buffer (e.g. after an async + // deviceUpload on another stream). + cudaStream_t consumer; + if (resolveDlpackStream(stream, consumer)) + orderPriorUsesBefore(buf, consumer, 0); size_t shape[1] = {static_cast(buf.size())}; // Delegate the capsule construction to nanobind: build a device // ndarray view parented to this buffer (keep_alive via owner) and diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu index e3c73d934c..a4bf3fa73f 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceGridHandle.cu @@ -131,7 +131,10 @@ void defineDeviceGridHandle(nb::module_& m) [](GridHandle& handle) { // CUDA Array Interface (v3) over the whole device buffer as a // 1-D contiguous uint8 array. stream=1 selects the legacy - // default stream per the CAI v3 spec. + // default stream per the CAI v3 spec — make that claim true by + // ordering the legacy default stream after the buffer's tracked + // prior uses (async uploads, recordUse'd kernels). + orderPriorUsesBefore(handle.buffer(), cudaStream_t(0), 0); nb::dict iface; iface["shape"] = nb::make_tuple(handle.buffer().size()); iface["typestr"] = "|u1"; @@ -158,8 +161,14 @@ void defineDeviceGridHandle(nb::module_& m) "DLPack device tuple (kDLCUDA, device_id) for the device buffer.") .def( "__dlpack__", - [](nb::handle self, nb::handle /*stream*/) { + [](nb::handle self, nb::handle stream) { auto& handle = nb::cast&>(self); + // Honor the consumer-provided stream per the DLPack protocol: + // order it after the buffer's tracked prior uses so the + // consumer cannot read a partially-uploaded buffer. + cudaStream_t consumer; + if (resolveDlpackStream(stream, consumer)) + orderPriorUsesBefore(handle.buffer(), consumer, 0); size_t shape[1] = {static_cast(handle.buffer().size())}; nb::ndarray> arr( handle.buffer().deviceData(), 1, shape, self); diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu index 780576abaf..df11076e0d 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu @@ -100,8 +100,10 @@ static NanoGrid* castOnIndexDeviceGrid(nb::handle py_grid, // value indices beyond activeVoxelCount, so those writes run out of bounds (an // illegal memory access). Detect it cheaply (two D2H header reads) and raise a // clear error instead of crashing. -static void requireContiguousIndexing(const NanoGrid* dGrid, - const char* fn_name) +// Returns the grid's value count so callers can also bounds-check their +// value-indexed arrays without a second D2H read. +static uint64_t requireContiguousIndexing(const NanoGrid* dGrid, + const char* fn_name) { using Traits = nanovdb::util::cuda::DeviceGridTraits; const uint64_t valueCount = Traits::getValueCount(dGrid); @@ -117,6 +119,24 @@ static void requireContiguousIndexing(const NanoGrid* dGrid, "or voxelsToOnIndexGrid."; throw nb::value_error(msg.c_str()); } + return valueCount; +} + +// The gather / decode kernels index their value-keyed arrays by every value +// index in [0, valueCount), so a shorter array is an out-of-bounds device +// access; reject it with a Python exception instead. +static void requireValueCountRows(size_t rows, uint64_t valueCount, + const char* fn_name, const char* array_name) +{ + if (rows < valueCount) { + std::string msg(fn_name); + msg += ": "; + msg += array_name; + msg += " covers " + std::to_string(rows) + " rows but the grid stores " + + std::to_string(valueCount) + + " value indices (activeVoxelCount + 1); it must cover all of them."; + throw nb::value_error(msg.c_str()); + } } // ------------------- DeviceVoxelBlockManagerHandle binding ----------------- @@ -387,7 +407,9 @@ template void defineGatherBoxStencil(nb::module_& m, const char* nam nb::ndarray, nb::c_contig, nb::device::cuda> out, int log2BlockWidth, uintptr_t stream) { auto* dGrid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencil"); - requireContiguousIndexing(dGrid, "gatherBoxStencil"); + const uint64_t vc = requireContiguousIndexing(dGrid, "gatherBoxStencil"); + requireValueCountRows(values.size(), vc, "gatherBoxStencil", "values"); + requireValueCountRows(out.shape(0), vc, "gatherBoxStencil", "out"); cudaStream_t s = reinterpret_cast(stream); const T* dVals = values.data(); T* dOut = out.data(); @@ -467,7 +489,10 @@ template void defineGatherBoxStencilColumns(nb::module_& m, const ch nb::ndarray, nb::c_contig> spokes, int log2BlockWidth, uintptr_t stream) { auto* dGrid = castOnIndexDeviceGrid(py_grid, "gatherBoxStencilColumns"); - requireContiguousIndexing(dGrid, "gatherBoxStencilColumns"); + const uint64_t vc = + requireContiguousIndexing(dGrid, "gatherBoxStencilColumns"); + requireValueCountRows(values.size(), vc, "gatherBoxStencilColumns", "values"); + requireValueCountRows(out.shape(0), vc, "gatherBoxStencilColumns", "out"); const int K = static_cast(spokes.shape(0)); if (K < 1 || K > 27) throw nb::value_error("gatherBoxStencilColumns: len(spokes) must be in [1, 27]."); @@ -549,7 +574,8 @@ void defineActiveVoxelCoords(nb::module_& m, const char* name) nb::ndarray, nb::c_contig, nb::device::cuda> out, int log2BlockWidth, uintptr_t stream) { auto* dGrid = castOnIndexDeviceGrid(py_grid, "activeVoxelCoords"); - requireContiguousIndexing(dGrid, "activeVoxelCoords"); + const uint64_t vc = requireContiguousIndexing(dGrid, "activeVoxelCoords"); + requireValueCountRows(out.shape(0), vc, "activeVoxelCoords", "out"); cudaStream_t s = reinterpret_cast(stream); int32_t* dOut = out.data(); nb::gil_scoped_release release; diff --git a/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu b/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu index 1824832f01..96fe5087fc 100644 --- a/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyIndexToGrid.cu @@ -5,8 +5,10 @@ #include #include +#include #include +#include // value count of a device grid namespace nb = nanobind; using namespace nb::literals; @@ -17,6 +19,31 @@ using namespace nb::literals; namespace pynanovdb { +namespace { + +// indexToGrid reads d_srcValues[i] for every value index stored in the source +// grid (voxels, plus any per-node stats and tile slots), so the values array +// must cover the grid's full value count. Verify with a D2H header read and +// raise a Python exception instead of an out-of-bounds device read. +template +void requireValueCountRows(const nanovdb::NanoGrid* d_srcGrid, + size_t rows, const char* fnName) +{ + using Traits = nanovdb::util::cuda::DeviceGridTraits; + const uint64_t valueCount = Traits::getValueCount(d_srcGrid); + if (rows < valueCount) { + std::string msg(fnName); + msg += ": values array covers " + std::to_string(rows) + + " entries but the source IndexGrid stores " + + std::to_string(valueCount) + + " value indices (grid.valueCount()); the array must cover all " + "of them."; + throw nb::value_error(msg.c_str()); + } +} + +} // anonymous namespace + // Scalar destination value type (e.g. float, double): d_srcValues is a flat, // 1-D device array indexed by the IndexGrid's per-element uint64 indices. template @@ -27,6 +54,7 @@ void defineIndexToGridScalar(nb::module_& m, const char* name) [](nanovdb::NanoGrid* d_srcGrid, nb::ndarray, nb::c_contig, nb::device::cuda> values, uintptr_t stream) { + requireValueCountRows(d_srcGrid, values.size(), "indexToGrid"); cudaStream_t s = reinterpret_cast(stream); const DstBuildT* d_values = values.data(); // indexToGrid launches kernels and synchronizes the stream; pure @@ -58,6 +86,7 @@ void defineIndexToGridVec3(nb::module_& m, const char* name) [](nanovdb::NanoGrid* d_srcGrid, nb::ndarray, nb::c_contig, nb::device::cuda> values, uintptr_t stream) { + requireValueCountRows(d_srcGrid, values.shape(0), "indexToGrid"); cudaStream_t s = reinterpret_cast(stream); const DstBuildT* d_values = reinterpret_cast(values.data()); // indexToGrid launches kernels and synchronizes the stream; pure diff --git a/nanovdb/nanovdb/python/cuda/PyInjectData.cu b/nanovdb/nanovdb/python/cuda/PyInjectData.cu index bb35e508fb..bc2b23cd1b 100644 --- a/nanovdb/nanovdb/python/cuda/PyInjectData.cu +++ b/nanovdb/nanovdb/python/cuda/PyInjectData.cu @@ -47,6 +47,26 @@ uint32_t leafCountOf(const nanovdb::NanoGrid* dGrid) return Traits::getTreeData(dGrid).mNodeCount[0]; } +// The injection kernels index the sidecar / predicate arrays by each grid's +// value indices, so an array shorter than the grid's value count is an +// out-of-bounds device access. Verify (one D2H header read per grid) and +// raise a Python exception instead. +void requireValueCountRows(const nanovdb::NanoGrid* dGrid, + size_t rows, const char* fnName, const char* arrayName) +{ + using Traits = nanovdb::util::cuda::DeviceGridTraits; + const uint64_t valueCount = Traits::getValueCount(dGrid); + if (rows < valueCount) { + std::string msg(fnName); + msg += ": "; + msg += arrayName; + msg += " covers " + std::to_string(rows) + " entries but its grid stores " + + std::to_string(valueCount) + + " value indices (grid.valueCount()); the array must cover all of them."; + throw nb::value_error(msg.c_str()); + } +} + } // anonymous namespace template void defineInject(nb::module_& m, const char* name) @@ -59,6 +79,8 @@ template void defineInject(nb::module_& m, const char* name) uintptr_t stream) { auto* src = castOnIndexDeviceGrid(srcGrid, "inject"); auto* dst = castOnIndexDeviceGrid(dstGrid, "inject"); + requireValueCountRows(src, srcSidecar.size(), "inject", "srcSidecar"); + requireValueCountRows(dst, dstSidecar.size(), "inject", "dstSidecar"); cudaStream_t s = reinterpret_cast(stream); const T* dSrc = srcSidecar.data(); T* dDst = dstSidecar.data(); @@ -101,6 +123,8 @@ template void defineInjectFeatures(nb::module_& m, const char* name) throw nb::value_error( "inject: src and dst feature sidecars must share the same " "feature dimension (shape[1])."); + requireValueCountRows(src, srcSidecar.shape(0), "inject", "srcSidecar"); + requireValueCountRows(dst, dstSidecar.shape(0), "inject", "dstSidecar"); cudaStream_t s = reinterpret_cast(stream); const T* dSrc = srcSidecar.data(); T* dDst = dstSidecar.data(); @@ -133,6 +157,8 @@ void defineInjectPredicateToMask(nb::module_& m, const char* name) nb::ndarray, nb::c_contig, nb::device::cuda> leafMasks, uintptr_t stream) { auto* dGrid = castOnIndexDeviceGrid(grid, "injectPredicateToMask"); + requireValueCountRows(dGrid, predicate.size(), + "injectPredicateToMask", "predicate"); cudaStream_t s = reinterpret_cast(stream); const uint32_t leafCount = leafCountOf(dGrid); constexpr size_t W = nanovdb::Mask<3>::WORD_COUNT; // 8 uint64 / leaf diff --git a/nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu b/nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu index 6c673e8b8d..4b8fdca1a0 100644 --- a/nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyMeshToGrid.cu @@ -1,6 +1,7 @@ // Copyright Contributors to the OpenVDB Project // SPDX-License-Identifier: Apache-2.0 #include "PyMeshToGrid.h" +#include "PyValidate.h" #include #include @@ -28,6 +29,11 @@ void defineMeshToGrid(nb::module_& m, const char* name) nb::ndarray, nb::c_contig, nb::device::cuda> triangles, double voxelSize, float halfWidth, const std::string& gridName, uintptr_t stream) { + // Map::set only debug-asserts a positive voxel size, and a + // non-positive halfWidth reverses the raster bounds; validate both + // before any CUDA work. + requirePositiveFinite(voxelSize, "meshToGrid", "voxelSize"); + requirePositiveFinite(halfWidth, "meshToGrid", "halfWidth"); cudaStream_t s = reinterpret_cast(stream); // Vec3f / Vec3i are three contiguous scalars, so the c_contig // (N, 3) tensors reinterpret element-for-element. diff --git a/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu b/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu index 7129822251..1265ccd795 100644 --- a/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyPointsToGrid.cu @@ -1,6 +1,7 @@ // Copyright Contributors to the OpenVDB Project // SPDX-License-Identifier: Apache-2.0 #include "PyPointsToGrid.h" +#include "PyValidate.h" #include @@ -82,6 +83,9 @@ template void defineVoxelsToGrid(nb::module_& m, const char* na [](nb::ndarray, nb::c_contig, nb::device::cuda> tensor, double voxelSize, uintptr_t stream) { + // Map::set only debug-asserts positivity; reject a singular / + // non-finite transform before any CUDA work. + requirePositiveFinite(voxelSize, "voxelsToGrid", "voxelSize"); cudaStream_t s = reinterpret_cast(stream); NdArrayCoordPtr points(tensor.data(), tensor.stride(0), tensor.stride(1)); const size_t count = tensor.shape(0); @@ -107,6 +111,9 @@ template void definePointsToGrid(nb::module_& m, const char* na [](nb::ndarray, nb::c_contig, nb::device::cuda> tensor, double voxelSize, uintptr_t stream) { + // Map::set only debug-asserts positivity; reject a singular / + // non-finite transform before any CUDA work. + requirePositiveFinite(voxelSize, "pointsToGrid", "voxelSize"); cudaStream_t s = reinterpret_cast(stream); NdArrayVec3Ptr points(tensor.data(), tensor.stride(0), tensor.stride(1)); const size_t count = tensor.shape(0); diff --git a/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu b/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu index bddbcdbdb0..e1ac8c631e 100644 --- a/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu +++ b/nanovdb/nanovdb/python/cuda/PyPruneGrid.cu @@ -6,8 +6,10 @@ #include #include +#include #include +#include // leaf count of a device grid namespace nb = nanobind; using namespace nb::literals; @@ -36,6 +38,18 @@ template void definePruneGrid(nb::module_& m, const char* name) throw std::invalid_argument( "pruneGrid: leafMask uint64 word count must be a multiple of " "Mask<3>::WORD_COUNT (8); supply one 512-bit mask per leaf node"); + // PruneGrid indexes one Mask<3> per SOURCE leaf, so the sidecar + // must cover every leaf; a shorter multiple of 8 would pass the + // divisibility check but read out of bounds on the device. + using Traits = nanovdb::util::cuda::DeviceGridTraits; + const uint64_t leafCount = Traits::getTreeData(dGrid).mNodeCount[0]; + if (totalWords < leafCount * wordsPerMask) + throw nb::value_error( + ("pruneGrid: leafMask holds " + + std::to_string(totalWords / wordsPerMask) + + " masks but the grid has " + std::to_string(leafCount) + + " leaf nodes; supply one Mask<3> (8 uint64) per leaf.") + .c_str()); cudaStream_t s = reinterpret_cast(stream); const nanovdb::Mask<3>* d_mask = reinterpret_cast*>(leafMask.data()); diff --git a/nanovdb/nanovdb/python/test/TestGpuInterop.py b/nanovdb/nanovdb/python/test/TestGpuInterop.py index a649105c60..2299dd6ba1 100644 --- a/nanovdb/nanovdb/python/test/TestGpuInterop.py +++ b/nanovdb/nanovdb/python/test/TestGpuInterop.py @@ -351,6 +351,156 @@ def test_noop_on_non_owning_buffer(self): cp.cuda.set_allocator(prev) +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestBindingValidation(unittest.TestCase): + """Negative tests: undersized value-indexed arrays and invalid geometric + parameters must raise Python exceptions instead of out-of-bounds device + accesses or silently-invalid grid transforms.""" + + BAD_SIZES = (0.0, -1.0, float("nan"), float("inf")) + + def _grid(self, cp): + import numpy as np + ijk = np.stack(np.meshgrid(*([np.arange(8)] * 3), indexing="ij"), + axis=-1).reshape(-1, 3).astype(np.int32) + return _device_onindex_from_coords(cp, ijk) # (dh, dg, n, coords) + + def test_voxels_to_grid_rejects_invalid_voxel_size(self): + cp = _require_cupy(self) + coords = cp.zeros((4, 3), dtype=cp.int32) + for bad in self.BAD_SIZES: + with self.assertRaises(ValueError): + nanovdb.tools.cuda.voxelsToOnIndexGrid(coords, bad) + + def test_mesh_to_grid_rejects_invalid_geometry(self): + cp = _require_cupy(self) + pts = cp.asarray( + [[0.0, 0.0, 0.0], [8.0, 0.0, 0.0], [0.0, 8.0, 0.0]], dtype=cp.float32) + tris = cp.asarray([[0, 1, 2]], dtype=cp.int32) + for bad in self.BAD_SIZES: + with self.assertRaises(ValueError): + nanovdb.tools.cuda.meshToGrid(pts, tris, voxelSize=bad) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.meshToGrid(pts, tris, halfWidth=bad) + + def test_index_to_grid_rejects_short_values(self): + cp = _require_cupy(self) + dh, dg, n, _ = self._grid(cp) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.indexToGrid(dg, cp.zeros(n, dtype=cp.float32)) + # Exact valueCount (= n + 1) is accepted. + out = nanovdb.tools.cuda.indexToGrid(dg, cp.zeros(n + 1, dtype=cp.float32)) + self.assertEqual(out.gridCount(), 1) + + def test_inject_rejects_short_sidecars(self): + cp = _require_cupy(self) + dh, dg, n, _ = self._grid(cp) + good = cp.zeros(n + 1, dtype=cp.float32) + short = cp.zeros(n, dtype=cp.float32) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.inject(dg, dg, short, good) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.inject(dg, dg, good, short) + # 2-D (feature) overload: row counts are checked the same way. + good2 = cp.zeros((n + 1, 2), dtype=cp.float32) + short2 = cp.zeros((n, 2), dtype=cp.float32) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.inject(dg, dg, short2, good2) + + def test_predicate_to_mask_rejects_short_predicate(self): + cp = _require_cupy(self) + dh, dg, n, _ = self._grid(cp) + masks = cp.zeros((n + 1) * 8, dtype=cp.uint64) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.injectPredicateToMask( + dg, cp.zeros(n, dtype=cp.bool_), masks) + + def test_prune_grid_rejects_short_mask(self): + cp = _require_cupy(self) + import numpy as np + # 16^3 coords span 8 leaf nodes (2 per axis), so one 8-word Mask<3> is + # a valid multiple of 8 but covers only 1 of the 8 leaves — previously + # an out-of-bounds device read rather than a Python error. + ijk = np.stack(np.meshgrid(*([np.arange(16)] * 3), indexing="ij"), + axis=-1).reshape(-1, 3).astype(np.int32) + dh, dg, n, _ = _device_onindex_from_coords(cp, ijk) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.pruneGrid(dg, cp.zeros(8, dtype=cp.uint64)) + + def test_gather_and_coords_reject_short_arrays(self): + cp = _require_cupy(self) + dh, dg, n, _ = self._grid(cp) + good_vals = cp.zeros(n + 1, dtype=cp.float32) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.gatherBoxStencil( + dg, cp.zeros(n, dtype=cp.float32), cp.zeros((n + 1, 27), dtype=cp.float32)) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.gatherBoxStencil( + dg, good_vals, cp.zeros((n, 27), dtype=cp.float32)) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.gatherBoxStencilColumns( + dg, good_vals, cp.zeros((n, 2), dtype=cp.float32), + cp.asnumpy(cp.asarray([13, 4], dtype=cp.int32))) + with self.assertRaises(ValueError): + nanovdb.tools.cuda.activeVoxelCoords(dg, cp.zeros((n, 3), dtype=cp.int32)) + + +@unittest.skipIf( + not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" +) +@unittest.skipIf( + not nanovdb.isGpuAvailable(), "No CUDA-capable GPU available" +) +class TestExportStreamOrdering(unittest.TestCase): + """CAI / DLPack exports must order the consumer against the buffer's + tracked prior uses (async uploads on non-blocking streams).""" + + def test_cai_after_async_upload(self): + cp = _require_cupy(self) + h = nanovdb.tools.createLevelSetSphere(radius=20.0, voxelSize=1.0) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + try: + nanovdb.io.writeGrid(tmp.name, h) + dh = nanovdb.io.deviceReadGrid(tmp.name) + finally: + os.unlink(tmp.name) + s = cp.cuda.Stream(non_blocking=True) + # Async upload on a non-blocking stream, then consume via CAI without + # ever synchronizing s explicitly: the export's stream ordering (CAI + # stream=1 backed by the tracked upload event) must make this safe. + dh.deviceUpload(s.ptr, False) + view = cp.asarray(dh) + host = bytes(cp.asnumpy(view)) + self.assertEqual(len(host), dh.size()) + self.assertNotEqual(host.count(0), len(host)) # real grid bytes arrived + + def test_dlpack_accepts_protocol_stream_values(self): + cp = _require_cupy(self) + h = nanovdb.tools.createLevelSetSphere(radius=10.0, voxelSize=1.0) + tmp = tempfile.NamedTemporaryFile(suffix=".nvdb", delete=False) + tmp.close() + try: + nanovdb.io.writeGrid(tmp.name, h) + dh = nanovdb.io.deviceReadGrid(tmp.name) + finally: + os.unlink(tmp.name) + dh.deviceUpload(0, True) + s = cp.cuda.Stream(non_blocking=True) + # None / legacy default / per-thread / no-sync / raw handle must all + # be accepted per the DLPack protocol. + for stream in (None, 1, 2, -1, s.ptr): + capsule = dh.__dlpack__(stream=stream) + self.assertIsNotNone(capsule) + arr = cp.from_dlpack(dh) + self.assertEqual(arr.nbytes, dh.size()) + + @unittest.skipIf( not nanovdb.isCudaAvailable(), "nanovdb module was compiled without CUDA support" ) diff --git a/nanovdb/nanovdb/python/test/TestNanoVDB.py b/nanovdb/nanovdb/python/test/TestNanoVDB.py index f465efb233..e05ce69e59 100644 --- a/nanovdb/nanovdb/python/test/TestNanoVDB.py +++ b/nanovdb/nanovdb/python/test/TestNanoVDB.py @@ -478,6 +478,16 @@ def test_set_transform(self): self.assertEqual(g.voxelSize()[0], 0.5) self.assertEqual(g.applyMap(nanovdb.math.Vec3d(0, 0, 0)), nanovdb.math.Vec3d(0, 0, 0)) + def test_set_transform_rejects_invalid_voxel_size(self): + # Map::set only debug-asserts positivity, so the binding must reject a + # singular / non-finite transform before it reaches the grid header. + h, g = self._sphere() + for bad in (0.0, -1.0, float("nan"), float("inf")): + with self.assertRaises(ValueError): + g.setTransform(bad) + # The transform is unchanged after the rejected calls. + self.assertEqual(g.voxelSize()[0], 1.0) + def test_flag_and_name_setters_refresh_checksum(self): h, g = self._sphere() nanovdb.tools.updateChecksum(g, nanovdb.CheckMode.Full) From 240b1885556dd24a7865651ffab1c47893f75243 Mon Sep 17 00:00:00 2001 From: Jonathan Swartz Date: Fri, 28 Aug 2026 23:00:30 +1200 Subject: [PATCH 48/48] NanoVDB Python: follow thread-local VBM stencil API Signed-off-by: Jonathan Swartz --- .../python/cuda/PyDeviceVoxelBlockManager.cu | 28 +++++++++---------- .../examples/levelset_filter_rawkernel.py | 9 ++++-- 2 files changed, 20 insertions(+), 17 deletions(-) diff --git a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu index df11076e0d..dfb243fdd2 100644 --- a/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu +++ b/nanovdb/nanovdb/python/cuda/PyDeviceVoxelBlockManager.cu @@ -382,17 +382,17 @@ __global__ void gatherBoxStencilKernel( constexpr int BW = 1 << Log2BlockWidth; constexpr int JML = BW / 64; using VBM = nanovdb::tools::cuda::VoxelBlockManager; - __shared__ uint32_t smem_leafIndex[BW]; - __shared__ uint16_t smem_voxelOffset[BW]; const uint64_t blockFirstOffset = firstOffset + uint64_t(blockIdx.x) * BW; - VBM::template decodeInverseMaps( - grid, firstLeafID[blockIdx.x], &jumpMap[uint64_t(blockIdx.x) * JML], - blockFirstOffset, smem_leafIndex, smem_voxelOffset); const int tID = threadIdx.x; - if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + uint32_t leafIndex; + uint16_t voxelOffset; + VBM::template decodeInverseMap( + grid, firstLeafID[blockIdx.x], &jumpMap[uint64_t(blockIdx.x) * JML], + blockFirstOffset, tID, leafIndex, voxelOffset); + if (leafIndex == VBM::UnusedLeafIndex) return; uint64_t st[27]; VBM::template computeBoxStencil( - grid, smem_leafIndex, smem_voxelOffset, st); + grid, leafIndex, voxelOffset, st); const uint64_t c = st[13]; // centre value index #pragma unroll for (int j = 0; j < 27; ++j) out[c * 27 + j] = values[st[j]]; // 0 -> background @@ -464,17 +464,17 @@ __global__ void gatherBoxStencilColumnsKernel( constexpr int BW = 1 << Log2BlockWidth; constexpr int JML = BW / 64; using VBM = nanovdb::tools::cuda::VoxelBlockManager; - __shared__ uint32_t smem_leafIndex[BW]; - __shared__ uint16_t smem_voxelOffset[BW]; const uint64_t blockFirstOffset = firstOffset + uint64_t(blockIdx.x) * BW; - VBM::template decodeInverseMaps( - grid, firstLeafID[blockIdx.x], &jumpMap[uint64_t(blockIdx.x) * JML], - blockFirstOffset, smem_leafIndex, smem_voxelOffset); const int tID = threadIdx.x; - if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; + uint32_t leafIndex; + uint16_t voxelOffset; + VBM::template decodeInverseMap( + grid, firstLeafID[blockIdx.x], &jumpMap[uint64_t(blockIdx.x) * JML], + blockFirstOffset, tID, leafIndex, voxelOffset); + if (leafIndex == VBM::UnusedLeafIndex) return; uint64_t st[27]; VBM::template computeBoxStencil( - grid, smem_leafIndex, smem_voxelOffset, st); + grid, leafIndex, voxelOffset, st); const uint64_t c = st[13]; // centre value index for (int col = 0; col < K; ++col) out[c * K + col] = values[st[spokes.s[col]]]; } diff --git a/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py b/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py index 0f20503b24..ec0e6432d5 100644 --- a/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py +++ b/nanovdb/nanovdb/python/examples/levelset_filter_rawkernel.py @@ -103,7 +103,8 @@ if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; uint64_t st[27]; - VBM::computeBoxStencil(grid, smem_leafIndex, smem_voxelOffset, st); + VBM::computeBoxStencil( + grid, smem_leafIndex[tID], smem_voxelOffset[tID], st); const uint64_t c = st[13]; const float vc = vin[c]; @@ -131,7 +132,8 @@ if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; uint64_t st[27]; - VBM::computeBoxStencil(grid, smem_leafIndex, smem_voxelOffset, st); + VBM::computeBoxStencil( + grid, smem_leafIndex[tID], smem_voxelOffset[tID], st); const uint64_t c = st[13]; const float vc = vin[c]; @@ -176,7 +178,8 @@ if (smem_leafIndex[tID] == VBM::UnusedLeafIndex) return; uint64_t st[27]; - VBM::computeBoxStencil(grid, smem_leafIndex, smem_voxelOffset, st); + VBM::computeBoxStencil( + grid, smem_leafIndex[tID], smem_voxelOffset[tID], st); const uint64_t c = st[13]; const float vc = vin[c];