NanoVDB: SyncFromAsync, ResourceRef, and resource seams for MeshToGrid and TempPool (CUDA)

[harrism]

Follow-up to #2231 and #2251, part of #2232. This is B2 of step 2's retrofit.

Stacked on #2268 — it branches from that PR, so the diff shown here includes its commits until it merges. Review only the commit titled "add SyncFromAsync, and give MeshToGrid a resource seam".

SyncFromAsync<Derived>

A stream-ordered resource must also model the synchronous concept (is_async_resource<R> implies is_resource<R>, matching cuda::mr's refinement), so a custom resource has to write four methods where two would do. This CRTP base supplies the synchronous pair in terms of the stream-ordered one:

struct MyResource : nanovdb::cuda::SyncFromAsync<MyResource> {
    static constexpr size_t DEFAULT_ALIGNMENT = 256;
    void* allocate_async(size_t bytes, size_t alignment, cudaStream_t stream);
    void  deallocate_async(void* p, size_t bytes, size_t alignment, cudaStream_t stream);
};

The synchronous pair is deliberately not a bare delegate. Memory returned by allocate must be usable immediately on any stream, so the null-stream allocation is synchronized before it is returned — which is what DeviceResource::allocate already did, in a place nobody else could reuse. Omitting that synchronization produces memory that satisfies the concept but is not actually synchronous: a race, not a compile error. deallocate does not synchronize, because the synchronous concept's contract is that the memory is already quiescent.

The two test resources in TestMemoryResource.cu are the first users, and were already wrong in exactly this way — they provide only the async pair, so they never modelled is_async_resource. Nothing caught it because TempPool duck-types rather than asserting. They now derive from the mixin and conform.

MeshToGrid resource seam

MeshToGrid was the last builder allocating from a hard-wired DeviceResource, via the TempDevicePool alias. It now takes a ResourceT parameter (defaulted, so existing callers are unaffected) and threads it into both its TopologyBuilder and its pool.

As with Data in #2268, BoxTrianglePair is hoisted out of the class as MeshToGridBoxTrianglePair, with an in-class alias. It carries no dependence on the resource, and leaving it nested would give every ResourceT instantiation its own incompatible type — which the device functors, templated on BuildT alone, could not name.

ResourceRef, and the TempPool conversion it unblocks

The first attempt at converting TempPool failed its own unit tests, and the failure exposed a real gap rather than a bug in the conversion:

• TempPool holds Resource* — non-owning, documented "must outlive this pool" — and TestMemoryResource's resources carry their counters inline, so they are stateful and lossy to copy.
• cuda::Buffer holds R by value, so a pool holding a Buffer had to copy the resource, and the pool then recorded traffic into its copy while the tests asserted on the original.

Checking the model we borrowed showed we had adopted half of a two-part design. cuda::buffer genuinely owns its resource — its constructor says so in a static_assert: "Buffer owns a copy of the memory resource…" — but in CCCL the ownership semantics are selected by what is placed in the by-value slot: an owning any_resource, a borrowing resource_ref, or a refcounted shared_resource. We shipped the slot without the borrowing type, so a non-owning container had no legitimate way to express itself.

nanovdb::cuda::ResourceRef<R> is the missing piece — a non-owning reference that is itself a resource:

• copying the ref shares the underlying instance, so a by-value Buffer over a ref borrows;
• its async methods exist only when R models AsyncResource (enable_if-gated, the same pattern as Buffer's stream API), so a ref over a synchronous resource does not misreport its tier — covered by static_asserts over ResourceRef<PinnedResource>;
• two refs compare equal exactly when they reference the same resource, which is the equality_comparable semantics CCCL's concepts require ("memory allocated by x may be deallocated by y");
• it is the static, dependency-free analog of cuda::mr::resource_ref, and the same shape as std::pmr::polymorphic_allocator over memory_resource*.

TempPool now keeps its bytes in a Buffer<std::byte, ResourceRef<Resource>>: same Resource* contract, same stream retention, same discard-on-growth reallocation (via destroy(stream) + move-assignment — scratch is never prefix-copied), but the block is freed by ownership rather than by hand. The TempPool unit tests — the ones the by-value attempt failed — pass unchanged, which is the regression guard for the borrow semantics.

The ownership rule of record is written up on #2232: Buffer stays by-value; a concrete resource in the slot is owned, a ResourceRef borrows; a refcounted SharedResource analog is deferred until the Python-bindings need materializes.

Testing

New tests in TestBuffer.cu cover the mixin (supplies a working synchronous pair; a Buffer over a mixin-based resource round-trips; a mixin user models both concepts) and ResourceRef (traffic reaches the original stateful resource, not a copy; tier gating via PinnedResource; equality is identity). A pendingchanges note covers the builder seams, the mixin, and the ref.

Verified locally on an RTX 6000 Ada, CUDA 12.6:

• nanovdb_cuda_buffer_unit_test — 27/27 (two new: a stateful resource observes all traffic through a by-value Buffer over a ref; ref equality is identity)
• nanovdb_cuda_memory_resource_unit_test — 9/9, including the two TempPool tests the by-value conversion failed, unchanged
• nanovdb_test_cuda — 52/53; the failure is UnifiedBuffer_IO, a missing test-data file (data/3_spheres.nvdb) in an unrelated component, which fails identically on an unmodified baseline.

Per #2264 the CUDA tests are excluded from CI, so CI will build but not run these.

[harrism]

@kmuseth ready for review after #2268 (stacked on it — the two commits after #2268's set are the payload). Adds SyncFromAsync, ResourceRef (the resource-ownership reconciliation recorded on #2232), MeshToGrid's resource seam, and TempPool onto cuda::Buffer. The TempPool unit tests pass unchanged, which is the regression guard for the borrow semantics.

[harrism]

Copilot cycles (fork mirror: harrism#2). Net changes: TopologyBuilder's scratch now borrows its resource through ResourceRef instead of copying it — a rule-of-record alignment found while triaging (f619056); the builders assert the stream-ordered-resource requirement directly with a diagnostic naming the builder; a DEFAULT_ALIGNMENT >= alignof(uint64_t) assert closes the custom-resource alignment corner (2608ea8); and SyncFromAsync documents its synchronize cost.

[kmuseth]

looks good but I have two questions

[harrism]

@kmuseth #2268 merged — thanks! I've merged latest master into this branch (no rebase, so the commits you looked at are unchanged) and the diff now shows only this PR's payload: SyncFromAsync, ResourceRef, TempPool/TopologyBuilder/MeshToGrid resource injection, and their tests.

Both of your comment threads were on Buffer.h lines that belonged to #2268's diff; that code merged there with the doxygen @deprecated tag on clear() included. Buffer.h is no longer part of this diff. Marking ready for review.

[kmuseth]

Approved!

Thanks for you changes/comments

I cloned your repo, build the branch locally and ran all nanovdb_test_cuda* unit-test on a Blackwell 6000. nanovdb_test_mgpu (still) fails, but it's clearly unrelated to your changes. The other tests passed, so I'll approve and merge