Skip to content

Nanobind to dev#71

Open
DasVinch wants to merge 29 commits into
devfrom
nanobind-to-dev
Open

Nanobind to dev#71
DasVinch wants to merge 29 commits into
devfrom
nanobind-to-dev

Conversation

@DasVinch
Copy link
Copy Markdown
Member

@DasVinch DasVinch commented May 3, 2026

Complex PR with a number of things developed in the nanobind branch (a few of which reverted for dev) in synergy with the nanobind branch of pyMilk (https://github.com/milk-org/pyMilk/tree/nanobind) (which is to become pyMilk's dev branch soon).

The pyMilk nanobind branch wraps all of that in gcovr and provides testing and decent coverage (ImageStreamIO.c 43%; ImageStreamIOWrap.cpp 74%) of this merge.

Included:

  • migrate python_module to nanobind away from pybind11

    • Same author, same philosophy
    • Compiles faster, less runtime overhead passing up and down from C
    • Buffer protocol doesn't exit anymore so we now have symmetric functions IMAGE.copy() and IMAGE.view() (zerocopy), and np.array(IMAGE) is disallowed.
    • The way time structs work, we may have lost some precision in some timestamps.
    • Overall binding is 200 lines shorter. Whatever but hey!
  • Increase package version to 2.1 (not ImageStruct, still 1.0.3) [NO: that's the milk version, I got mixed up.]

  • A couple missing const in some ISIO function signatures for strings

  • Atomic creation of SHM files (creating under tmp filename, then atomic mv)

  • refactor of destroyIm / closeIm so that destroyIm actually equals closeIm + extra work.

  • Bug where image->used = 1; was never set in ImageStreamIO_read_sharedmem_image_toIMAGE (e.g. ImageStreamIO_openIm)

  • GPU SHM: registry with refcount to manage multiple handles to the same GPU IPC segment within one process.

  • GPU compilation: int IMAGESTRUCT_COMPILED_HAVE_CUDA = 0/1 for runtime checking of compilation if needed.

  • Fix checkCudaError error print function

  • Coverage build triggered with -DCMAKE_BUILD_TYPE=Coverage. For integration with the nox session "tests_run_coverage" of pyMilk which performs coverage over python and C/C++ files

Reverted - FYI (but present on nanobind brach):

  • Addition of CLIcore_signals.h, 'milkDebugTools.h', timeutils.{c,h} --> helpful for restructuring milk v1 into a standalone package with processinfo and fps (see pyMilk under milkengine/ path).
  • Changing the target to ImageStreamIOpyMilk for disambiguation at runtime linking.
  • Changing the export symbols in CMakeLists. In the course of packaging a milkengine for pyMilk, I switched some target_include_directories from PUBLIC to PRIVATE. This means that #include "ImageStreamIO.h" is generally disallowed in the nanobind branch used by pyMilk, with #include "../ImageStreamIO/ImageStreamIO.h" required from sibling folders, or "ImageStreamIO/ImageStreamIO.h"` since the overall project root is on the include path.
    The current is:
target_include_directories(${LIBNAME} PUBLIC $<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}>
                                              $<INSTALL_INTERFACE:include/${SRCNAME}>
                                              $<INSTALL_INTERFACE:include>)

But I'd prefer to use:

target_include_directories(${LIBNAME} PRIVATE ${PROJECT_SOURCE_DIR})
target_include_directories(${LIBNAME} PUBLIC $<BUILD_INTERFACE:${CMAKE_SOURCE_DIR}>
                                             $<INSTALL_INTERFACE:include/${SRCNAME}>
                                             $<INSTALL_INTERFACE:include>)

…e of opened IPC handles with refcount (otherwise cannot open the same GPU SHM twice in the same process)
@DasVinch
Copy link
Copy Markdown
Member Author

DasVinch commented May 3, 2026

No ABI changes.

@a-sevin @floferreira you may be interested.

This comment was marked as outdated.

@DasVinch DasVinch force-pushed the nanobind-to-dev branch from f215748 to 373545b Compare May 3, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants