#15 - Add reusable FFT/GEMM GPU workloads to receive-path benchmarks

[RamyaGuru]

Closes #15

What

• examples/bench_workload.{h,cu} — reusable, engine-agnostic GPU workload run per received burst on its own GPU scratch (no payload pointer → drops into any engine; no-op for none, validated by an init-time warmup).
• --workload none|fft|gemm|gemm_fp16: batched cuFFT C2C; FP32 cublasSgemm; and the same-size mixed-precision FP16/tensor-core cublasGemmEx (inference-style). Wired into DPDK/HDS (daqiri_bench_raw_gpudirect, daqiri_bench_raw_hds) and RoCE (daqiri_bench_rdma); run_spark_bench.sh WORKLOAD env fills the post_process CSV column.
• Fixes the Spark netns RoCE bench config, which Rename manager/backend abstraction to engine (DAQIRI_MGR → DAQIRI_ENGINE) #156 left on the pre-rename client schema (socket_config.remote_addr is now rejected for RoCE clients).

Results (DGX Spark, 3-rep mean ± std, all drop-free)

	none	FFT	GEMM FP32	GEMM FP16
DPDK 8 KB	98.5 / ~0%	95.1 / 9%	61.5 / 82%	98.5 / 29%
RoCE 8 MB	101.4 / ~0%	99.4 / 27%	77.8 / 77%	100.1 / 35%

(Gb/s / GPU SM%.) The receive path backpressures against GPU load without dropping packets, and the same matmul at FP16/tensor-core precision returns to line rate at far lower SM than FP32.

How workloads run

Each backend's RX path now feeds the actual received payload to the GPU workload, assembled into one contiguous device buffer per batch by a shared ReorderPipeline (examples/bench_pipeline.{h,cu}) that shares the workload's CUDA stream:

• DPDK raw / UDP (out-of-order capable): a sequence-number reorder kernel (packet_reorder_copy_payload_by_sequence) places each packet's payload at slot = seq % packets_per_batch, producing an ordered contiguous buffer. TX injects a per-packet seq number.
• RoCE (RC) / TCP (reliable, in-order): an arrival-order gather (packet_gather_copy_payload). A single large device-resident message is a zero-copy pass-through (the recv buffer is the contiguous buffer).
• Sockets additionally stage host→device first, since the kernel hands payloads to the app in pageable host memory — that copy is on the measured path (the honest cost of a socket receiver doing GPU work).

The assembled buffer then feeds GpuWorkload (examples/bench_workload.{h,cu}): cuFFT C2C, or cuBLAS FP32 Sgemm / FP16 tensor-core GemmEx (the buffer is the FFT input / GEMM A operand). To overlap compute with ingest, each path holds a batch of received data and drains the GPU stream once per batch (raw: a ~10-window burst; RoCE: a small batch of held completions whose recv buffers stay live during the pass-through), so the receiver backpressures against GPU load rather than dropping.

--workload-batch-bytes decouples the compute working set from the I/O unit (RoCE sub-chunks each message; raw sizes its reorder window), enabling the batch-size sweep.

Socket (UDP/TCP) results are deferred to a follow-up PR: the netns wire-loopback harness has a pre-existing receive-path regression (server binds but receives nothing), independent of this change, so socket numbers will land with that fix on fix/socket-netns-rx.

[greptile-apps]

Greptile Summary

Adds examples/bench_workload.{h,cu} and examples/bench_pipeline.{h,cu} — a reusable, engine-agnostic GPU workload layer (batched cuFFT C2C, FP32 cublasSgemm, and FP16/tensor-core cublasGemmEx) wired into all four bench binaries, with a new src/kernels.cu gather kernel for in-order transports. Also fixes the RoCE netns config's post-#156 client schema (remote_addr → local_addr) and adds --workload-batch-bytes to decouple the GPU compute working set from the I/O unit.

• GpuWorkload / ReorderPipeline: init-time warmup catches misconfigured cuFFT/cuBLAS calls before the hot path; all CUDA handles and buffers are freed in destroy() on every exit; BurstParams free contract is maintained — the burst is freed only after the stream drains so the reorder/staging copies finish reading DMA buffers first.
• RoCE batch-drain pipelining (rdma_bench.cpp): holds a small window of receive completions so compute overlaps with subsequent receives, mirroring the raw path; flush_held_recv() is called on all exit paths (batch-full, idle, shutdown).
• run_spark_bench.sh: WORKLOAD / WORKLOAD_BATCH validated at startup; post_process CSV column now records the actual workload name instead of the hardcoded \"none\"; new post_process_batch column appended last to preserve existing column offsets.

Confidence Score: 5/5

Safe to merge — the BurstParams free contract is preserved on every exit path across all three backends, and the new GPU compute layer is fully isolated from the packet I/O path.

The workload and pipeline code is well-structured: warmup validation at init time, stream-synchronized frees before buffer recycling, and destroy() covering all failure paths. The RoCE batch-drain pipelining correctly flushes held completions on every exit. The only gap is a missing doc update for docs/tutorials/configuration-walkthrough.md, which does not affect runtime behavior.

No files require special attention for correctness; docs/tutorials/configuration-walkthrough.md should be updated before the next external-facing release to document the new CLI flags and the YAML schema fix.

Important Files Changed

Filename	Overview
examples/bench_workload.cu	New GpuWorkload class (cuFFT C2C + FP32/FP16-tensor-core GEMM); init-time warmup validates the op; stream ownership and handle lifetime are cleanly managed with destroy() covering every exit path.
examples/bench_workload.h	Public header for GpuWorkload and BenchWorkload enum; CUDA types hidden behind void* so it is includable from plain .cpp mains — clean design matching existing patterns.
examples/bench_pipeline.cu	ReorderPipeline correctly handles both SeqReorder and GatherOnly modes; the single-packet zero-copy pass-through avoids a redundant kernel launch for RoCE; all CUDA allocations freed in destroy().
examples/raw_bench_common.cpp	Wires workload into rx_count_worker; per-burst gpu_workload.sync() before free_all_packets_and_burst_rx correctly prevents the reorder kernel from reading freed DMA buffers.
examples/rdma_bench.cpp	Batch-drain pipelining for RoCE is well-structured; flush_held_recv() is called on every exit (batch full, idle, shutdown), preserving the BurstParams free contract on all paths.
examples/socket_bench.cpp	Socket workload correctly stages host payloads to the GPU and syncs the stream before freeing the burst; per-burst sync is necessary here since the staging copy reads the burst's pageable memory.
src/kernels.cu	New packet_gather_copy_payload kernel follows the same vectorized-copy pattern as the existing reorder kernel; one block per packet, 128 threads, input null-check present.
examples/CMakeLists.txt	Adds bench_workload.cu + bench_pipeline.cu to all four bench targets and links CUDA::cufft + CUDA::cublas; also correctly adds CUDA_ARCHITECTURES to rdma and socket benches that were previously missing it.
examples/run_spark_bench.sh	WORKLOAD / WORKLOAD_BATCH env vars validated at startup with case/regex guards; CSV post_process column now records the actual workload instead of the hardcoded 'none'.
examples/raw_gpudirect_bench.cpp	Injects per-packet network-byte-order seq numbers into TX templates and wires ReorderGeometry into rx_count_worker; packets_per_batch capped to tx.batch_size.
examples/raw_hds_bench.cpp	HDS reorder uses segment 1 (GPU payload) with zero payload_byte_offset; no seq injection since HDS TX does not write seq numbers — acknowledged in comment as acceptable for a throughput benchmark.
examples/daqiri_bench_rdma_tx_rx_spark_netns.yaml	Fixes post-#156 RoCE client schema by replacing rejected remote_addr with local_addr; comment explains the reason for the change.
docs/benchmarks/performance-dgx-spark.md	Adds full GPU-workload results section with methodology, per-transport tables, and batch-size sweep; prose and numbers are internally consistent.

_{Reviews (5): Last reviewed commit: "#15 - Parse --workload-batch-bytes in so..." | Re-trigger Greptile}