Add allocation-free raw copy/axpy kernels#140
Conversation
copy_scale_raw / copy_scale_conj_raw / axpy_raw / axpy_conj_raw take RawStridedRef/RawStridedMut (borrowed metadata) and run a stack-allocated fused loop nest (axes ordered by destination stride, adjacent axes merged, contiguous inner runs as plain slice loops), so small prepared-replay copies avoid the per-call Arc metadata and plan building of the view kernels. Ranks above RAW_FUSED_RANK_LIMIT fall back to the view-based kernels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The complex `FusedScalar` implementation had no test coverage — every existing fused test used f64 — which kept `strided-kernel/src/fused.rs` below the 80% per-file coverage threshold. Real negate/conj/abs were the only real scalar ops not reached by the existing op chains. Add single-instruction plans that iterate every op over Complex64 (both the static-specialization and interpreter paths dispatch through the scalar methods), cover the real negate/conj/abs ops, and add direct tests for the plan/layout validation error branches. fused.rs line coverage 80% -> 92%. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Update: to unblock CI without waiting on #141, this branch now cherry-picks #141's |
|
Overall, the direction and layering look good. The rank <= 8 stack-backed fast path plus a higher-rank fallback seems like a reasonable first direct-execution API; reusable prepared plans and the remaining add/map/reduce operations can follow in #139. I do see one API-consistency issue that would be better to fix before this becomes public: the existing view APIs are mixed-scalar generic, copy_scale<D, S, A, Op>(...)
axpy<D, S, A, Op>(...)but the raw variants require destination, source, and scale to all be the same Could these signatures be generalized to A few smaller points:
With the scalar genericity/trait-bound mismatch addressed, this looks like a good incremental step toward #139. |
|
Gentle bump — this has been green and mergeable for a week (the coverage note above is self-contained via the cherry-picked #141 commit). Happy to rebase or split if reviewers prefer a different shape. A follow-up building the prepared-plan layer from #139 on top of these raw entry points is ready to go once this lands, so a review here unblocks that next step too. |
Add allocation-free raw copy/axpy kernels
Summary
This PR adds allocation-free copy/scale/axpy entry points over borrowed raw
strided layouts (
RawStridedRef/RawStridedMut), the elementwisecounterpart to the raw bgemm API added in #134.
The normal
StridedView/StridedViewMutkernels remain unchanged. The newfunctions are for prepared replay paths that already hold validated
dims/strides/offsetdescriptors and re-run the same small stridedcopies many times.
Motivation
StridedView/StridedViewMutown their metadata (Arc<[usize]>/Arc<[isize]>) and the map/zip kernels build a traversal plan on every call.For small replay copies that fixed cost — two
Arcallocations per view plusper-call planning — dominates the actual data movement.
A replay consumer already has the borrowed layout in hand:
Rebuilding an owning view and re-planning for each such copy is avoidable
overhead. This PR gives that case a raw borrowed-layout path, mirroring the
motivation in #134 for GEMM.
API
Re-exported from
strided-kernel:Each fuses the (dst, src) stride pair into a stack-allocated loop nest and runs
plain loops — no heap allocation on any call path with rank at most
RAW_FUSED_RANK_LIMIT(8). Higher ranks fall back to the existing view kernels(
copy_scale/axpy), so behaviour and results are unchanged there.Tests
strided-kerneltests cover parity against the view kernels and the raw-onlypaths:
raw_copy_scale_matches_view_kernelraw_copy_scale_conjugates_complex_sourcesraw_axpy_accumulatescargo test -p strided-kernelpasses; the full workspace builds.Scope
This covers the copy/scale/axpy families only. Raw borrowed-layout
reduce/partial-trace and a reusable compiled plan (fuse/order/block computed once,
executed many times) are follow-ups tracked in #139.