Add CUDA backend for NVIDIA GPU acceleration#20

Open

ServeurpersoCom wants to merge 32 commits into

antirez:mainfrom

ServeurpersoCom:cuda

Commits on Feb 4, 2026

wip: add CUDA/cuBLAS backend for NVIDIA GPU acceleration

ServeurpersoCom
and
claude
committed
wip: fix CUDA initialization and linking

ServeurpersoCom
and
claude
committed
wip: first working version, fix cuBLAS row-major to column-major conversion

ServeurpersoCom
and
claude
committed
fix: use caller-provided lda/ldb/ldc in cuBLAS sgemm
ServeurpersoCom
committed
perf: weight cache - 26% faster (5.9s vs 8.0s denoising)
ServeurpersoCom
committed
wip: Scratch Buffers - Reusable GPU memory for activations
ServeurpersoCom
committed
feat(cuda): GPU tensor pool for single_block

ServeurpersoCom
and
claude
committed
feat(cuda): GPU batched attention - 53% total speedup from baseline

ServeurpersoCom
and
claude
committed
feat(cuda): GPU joint attention for double blocks - 60% total speedup

ServeurpersoCom
and
claude
committed
perf(cuda): tensor pool for attention - reduce malloc overhead

ServeurpersoCom
and
claude
committed
feat(cuda): full GPU double_block_forward - 64% total speedup

ServeurpersoCom
and
claude
committed
feat(cuda): GPU conv2d for VAE decoder - 82% faster VAE

ServeurpersoCom
and
claude
committed
feat(cuda): GPU RoPE + VAE conv2d - 66% total speedup (2.96x)

ServeurpersoCom
and
claude
committed
feat(cuda): Qwen3 causal attention on GPU - 42% faster text encoding

ServeurpersoCom
and
claude
committed
fix(cuda): 64-bit indexing in im2col for 1024x1024 VAE decode

ServeurpersoCom
and
claude
committed
doc: update
ServeurpersoCom
committed
test: add CUDA test runner with GPU-appropriate tolerance

ServeurpersoCom
and
claude
committed
fix(cuda): disable weight cache in mmap mode to fix stale weight bug

Pascal
authored and
ServeurpersoCom
committed
fix(cli): silence GCC truncation warnings in flux_cli.c

authored and
ServeurpersoCom
committed
fix(cuda): auto-detect GPU architecture, fix TF32 for Turing

ServeurpersoCom
and
claude
committed
test script
ServeurpersoCom
committed
refactor(cuda): remove dead code, optimize small buffer allocations

ServeurpersoCom
and
claude
committed
feat(cuda): Add BF16 weight caching for single blocks

ServeurpersoCom
and
claude
committed
perf(cuda): Add chained single block path to reduce CPU/GPU transfers
ServeurpersoCom
committed
fix: Don't free bf16 mmap pointers in transformer cleanup

ServeurpersoCom
and
claude
committed
perf(cuda): Pre-compute AdaLN modulation once for all single blocks
ServeurpersoCom
committed
cuda: use cublasGemmEx with CUBLAS_COMPUTE_32F_FAST_16F

ServeurpersoCom
and
claude
committed
fix(cuda): disable BF16 weight cache in no-mmap mode

ServeurpersoCom
and
claude
committed
nit
ServeurpersoCom
committed
remove dead code
ServeurpersoCom
committed
doc
ServeurpersoCom
committed
feat(windows): add Windows build support with secure temp file handling

committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CUDA backend for NVIDIA GPU acceleration#20

Add CUDA backend for NVIDIA GPU acceleration#20
ServeurpersoCom wants to merge 32 commits into
antirez:mainfrom
ServeurpersoCom:cuda

Commits on Feb 4, 2026

wip: add CUDA/cuBLAS backend for NVIDIA GPU acceleration

wip: fix CUDA initialization and linking

wip: first working version, fix cuBLAS row-major to column-major conversion

fix: use caller-provided lda/ldb/ldc in cuBLAS sgemm

perf: weight cache - 26% faster (5.9s vs 8.0s denoising)

wip: Scratch Buffers - Reusable GPU memory for activations

feat(cuda): GPU tensor pool for single_block

feat(cuda): GPU batched attention - 53% total speedup from baseline

feat(cuda): GPU joint attention for double blocks - 60% total speedup

perf(cuda): tensor pool for attention - reduce malloc overhead

feat(cuda): full GPU double_block_forward - 64% total speedup

feat(cuda): GPU conv2d for VAE decoder - 82% faster VAE

feat(cuda): GPU RoPE + VAE conv2d - 66% total speedup (2.96x)

feat(cuda): Qwen3 causal attention on GPU - 42% faster text encoding

fix(cuda): 64-bit indexing in im2col for 1024x1024 VAE decode

doc: update

test: add CUDA test runner with GPU-appropriate tolerance

fix(cuda): disable weight cache in mmap mode to fix stale weight bug

fix(cli): silence GCC truncation warnings in flux_cli.c

fix(cuda): auto-detect GPU architecture, fix TF32 for Turing

test script

refactor(cuda): remove dead code, optimize small buffer allocations

feat(cuda): Add BF16 weight caching for single blocks

perf(cuda): Add chained single block path to reduce CPU/GPU transfers

fix: Don't free bf16 mmap pointers in transformer cleanup

perf(cuda): Pre-compute AdaLN modulation once for all single blocks

cuda: use cublasGemmEx with CUBLAS_COMPUTE_32F_FAST_16F

fix(cuda): disable BF16 weight cache in no-mmap mode

nit

remove dead code

doc

feat(windows): add Windows build support with secure temp file handling