Skip to content

Add allocation-free raw copy/axpy kernels#140

Merged
shinaoka merged 2 commits into
tensor4all:mainfrom
Ryo-wtnb11:feat/raw-copy-axpy-kernels
Jul 15, 2026
Merged

Add allocation-free raw copy/axpy kernels#140
shinaoka merged 2 commits into
tensor4all:mainfrom
Ryo-wtnb11:feat/raw-copy-axpy-kernels

Conversation

@Ryo-wtnb11

Copy link
Copy Markdown
Contributor

Add allocation-free raw copy/axpy kernels

Summary

This PR adds allocation-free copy/scale/axpy entry points over borrowed raw
strided layouts (RawStridedRef / RawStridedMut), the elementwise
counterpart to the raw bgemm API added in #134.

The normal StridedView / StridedViewMut kernels remain unchanged. The new
functions are for prepared replay paths that already hold validated
dims / strides / offset descriptors and re-run the same small strided
copies many times.

Motivation

StridedView / StridedViewMut own their metadata (Arc<[usize]> /
Arc<[isize]>) and the map/zip kernels build a traversal plan on every call.
For small replay copies that fixed cost — two Arc allocations per view plus
per-call planning — dominates the actual data movement.

A replay consumer already has the borrowed layout in hand:

let dims: &[usize] = &[m, n];
let dst_strides: &[isize] = &[n as isize, 1];
let src_strides: &[isize] = &[1, m as isize];

Rebuilding an owning view and re-planning for each such copy is avoidable
overhead. This PR gives that case a raw borrowed-layout path, mirroring the
motivation in #134 for GEMM.

API

Re-exported from strided-kernel:

use strided_kernel::{
    copy_scale_raw, copy_scale_conj_raw, axpy_raw, axpy_conj_raw,
    RawStridedMut, RawStridedRef,
};

// dest = scale * src
copy_scale_raw(&mut dest, &src, scale)?;
// dest = scale * conj(src)
copy_scale_conj_raw(&mut dest, &src, scale)?;
// dest = alpha * src + dest
axpy_raw(&mut dest, &src, alpha)?;
// dest = alpha * conj(src) + dest
axpy_conj_raw(&mut dest, &src, alpha)?;

Each fuses the (dst, src) stride pair into a stack-allocated loop nest and runs
plain loops — no heap allocation on any call path with rank at most
RAW_FUSED_RANK_LIMIT (8). Higher ranks fall back to the existing view kernels
(copy_scale / axpy), so behaviour and results are unchanged there.

Tests

strided-kernel tests cover parity against the view kernels and the raw-only
paths:

  • raw_copy_scale_matches_view_kernel
  • raw_copy_scale_conjugates_complex_sources
  • raw_axpy_accumulates

cargo test -p strided-kernel passes; the full workspace builds.

Scope

This covers the copy/scale/axpy families only. Raw borrowed-layout reduce /
partial-trace and a reusable compiled plan (fuse/order/block computed once,
executed many times) are follow-ups tracked in #139.

copy_scale_raw / copy_scale_conj_raw / axpy_raw / axpy_conj_raw take
RawStridedRef/RawStridedMut (borrowed metadata) and run a
stack-allocated fused loop nest (axes ordered by destination stride,
adjacent axes merged, contiguous inner runs as plain slice loops), so
small prepared-replay copies avoid the per-call Arc metadata and plan
building of the view kernels. Ranks above RAW_FUSED_RANK_LIMIT fall
back to the view-based kernels.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Ryo-wtnb11

Copy link
Copy Markdown
Contributor Author

The failing coverage check here is pre-existing on main (strided-kernel/src/fused.rs is below the 80% per-file threshold, unrelated to this PR — it touches only raw_ops.rs/lib.rs). #141 restores that file's coverage; this PR should go green once #141 lands and it's rebased.

The complex `FusedScalar` implementation had no test coverage — every
existing fused test used f64 — which kept `strided-kernel/src/fused.rs`
below the 80% per-file coverage threshold. Real negate/conj/abs were the
only real scalar ops not reached by the existing op chains.

Add single-instruction plans that iterate every op over Complex64 (both
the static-specialization and interpreter paths dispatch through the
scalar methods), cover the real negate/conj/abs ops, and add direct tests
for the plan/layout validation error branches.

fused.rs line coverage 80% -> 92%.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Ryo-wtnb11

Copy link
Copy Markdown
Contributor Author

Update: to unblock CI without waiting on #141, this branch now cherry-picks #141's fused.rs coverage commit (304147d) so the coverage check passes here on its own. It's a test-only change touching only fused.rs (no overlap with raw_ops.rs). When #141 merges first, rebasing this onto main drops the duplicate automatically.

@shinaoka

Copy link
Copy Markdown
Member

Overall, the direction and layering look good. The rank <= 8 stack-backed fast path plus a higher-rank fallback seems like a reasonable first direct-execution API; reusable prepared plans and the remaining add/map/reduce operations can follow in #139.

I do see one API-consistency issue that would be better to fix before this becomes public: the existing view APIs are mixed-scalar generic,

copy_scale<D, S, A, Op>(...)
axpy<D, S, A, Op>(...)

but the raw variants require destination, source, and scale to all be the same T. In addition, the non-conjugating copy_scale_raw and axpy_raw unnecessarily require ElementOpApply, although their identity paths never call conj. This means the raw API rejects mixed scalar and custom scalar types accepted by the corresponding view API.

Could these signatures be generalized to D, S, A, with A: Mul<S, Output = D>, and reserve ElementOpApply for the conjugating variants? The internal apply_fused_pair is already parameterized over D and S, so this looks compatible with the implementation structure.

A few smaller points:

  • The allocation claim should probably say “successful rank <= 8 execution path”: shape-mismatch errors allocate their Vec payloads, and rank > 8 deliberately allocates through as_view().
  • It would be useful to cover axpy_conj_raw, negative strides with a nonzero offset, rank 0 / zero extents, and the rank > 8 fallback. A counting-allocator test would directly protect the main performance contract.
  • Since test(fused): restore fused.rs coverage above threshold #141 has merged, this branch should be rebased so its duplicated fused.rs coverage commit drops out.

With the scalar genericity/trait-bound mismatch addressed, this looks like a good incremental step toward #139.

@Ryo-wtnb11

Copy link
Copy Markdown
Contributor Author

Gentle bump — this has been green and mergeable for a week (the coverage note above is self-contained via the cherry-picked #141 commit). Happy to rebase or split if reviewers prefer a different shape. A follow-up building the prepared-plan layer from #139 on top of these raw entry points is ready to go once this lands, so a review here unblocks that next step too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants