Skip to content

Initial dynamic input support for gpu codegen#45

Draft
harz05 wants to merge 13 commits into
ML4EP:gpu/alpakafrom
harz05:feat/dynamic-input-alpaka
Draft

Initial dynamic input support for gpu codegen#45
harz05 wants to merge 13 commits into
ML4EP:gpu/alpakafrom
harz05:feat/dynamic-input-alpaka

Conversation

@harz05

@harz05 harz05 commented Jun 30, 2026

Copy link
Copy Markdown

This PR adds dynamic shape (symbolic batch dimension N) input support to the gpu code generator.

While generating a dynamic model on the GPU path it failed to compile. The dynamic intermediate tensor buffers were emitted with a type alias built on the runtime device object devAcc where a type is needed, declared under the name bufDev_<name> while every operator references deviceBuf_<name> (so the buffer was undefined at use), and allocated into a discarded auto local in the constructor with a hardcoded float type, so the member was never set. After fixing those, a second error showed up because alpaka::Buf has no default constructor, so the bare member declaration would not compile. The fix declares the dynamic buffers as Session members with the correct Buf type and deviceBuf_ name according to the tensor dtype, default initializes each member to a one element placeholder buffer to satisfy the no default constructor requirement, and allocates the real buffer in the constructor sized by the runtime length once N is known.

Modified operators that now support dynamic input-

  • Tile: Dim shapes, dynamic Initialize, kernel index math uses the runtime dimension.
  • Comparison / Equal / Greater / Less: dual size_t and Dim representation mirroring the ROOT CPU operator, dynamic Initialize branch that registers the output as a dynamic tensor without materializing, kernel handles both static and dynamic shapes via Dim.

Kernel argument convention

Index math kernels that reference a runtime dimension now receive the model shape parameters (for example N) as size_t kernel arguments, supplied at the createTaskKernel call site. This matches the existing ScatterElements kernel. Flagging it for review, if there could be some different methods to do it instead of this

Added gtests(DynamicTile and DynamicEqual) to verify the changes made. For this test two different batch sizes of N = 1 and N = 8 were chosen and the output was compared against an independent host reference.

Tests verified on NVIDIA T4.

@harz05 harz05 force-pushed the feat/dynamic-input-alpaka branch from 2946253 to 78eb28e Compare July 5, 2026 17:38
@harz05 harz05 force-pushed the feat/dynamic-input-alpaka branch from 01ddd4c to e8623a1 Compare July 8, 2026 16:15
@harz05 harz05 marked this pull request as draft July 10, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant