Skip to content

[Feature] l0_swimlane: restore real tensor payloads so metadata-driven kernels are replayable #1532

Description

@ChaoWao

Summary

l0_swimlane reconstructs a task's args from a level-3 dump — descriptor
metadata only, no payload (l0_swimlane.py:264 runs --dump-args 3). Tensor
buffers are therefore aclrtMemset to 0 in the replay, and --set-arg is the
only way to give one content: it writes a single integer into every element
(l0_swimlane.py:643-652).

That covers a scalar or a uniform control tensor, but not a kernel whose control
input is a struct. Proposal: let the tool restore real bytes for selected
tensors from a --dump-args 2 dump, which already captures them.

Motivation / Use Case

examples/a2a3/tensormap_and_ringbuffer/qwen3_14b_decode is currently
unreplayable for this reason, and the l0 doc's "real SPMD workload" category has
no representative because of it (see #1529).

Its attention is a CANN FusedInferAttentionScore extern that reads a
FAInferTilingData struct out of a UINT8 metadata buffer filled at runtime by
paged_attention_tiling_cce. Uniform fill cannot synthesize a coherent struct —
and UINT8 is in INTEGER_DTYPES, so --set-arg does not even refuse it
(l0_swimlane.py:104, :1231); it just produces nonsense. With metadata
zeroed, fai_body.hpp reads tiling->needCoreNum == 0 and takes the degenerate
branch, so the trace comes out unrepresentative rather than empty — the
worse of the two failures, because it looks like a result.

Being an extern is not the blocker: the replay feeds kernel_entry and does not
care whether the source was generated or hand-written. The blocker is purely
that the tool cannot deliver a structured buffer. Any kernel taking its work
distribution from a runtime-built descriptor hits the same wall, which is the
normal shape for CANN operator wrappers.

Proposed API / Behavior

Add an opt-in that sources a tensor's initializer from a payload-carrying dump
rather than a constant, e.g.:

$L0a --func-id 11,12 --restore-arg 6 --test $E/qwen3_14b_decode/test_qwen3_14b_decode.py
  • --restore-arg SLOT (repeatable): emit that tensor's captured bytes into the
    generated workspace and aclrtMemcpy them in, instead of aclrtMemset(0).
  • Implies a level-2 dump for the run (or errors clearly if reusing a level-3
    dump that has no .bin).
  • Composes with --set-arg; conflicting slots should be rejected.

Scope note: only worth restoring for control inputs. Restoring every tensor
would drag the full working set (weights, KV pool) into the workspace, which is
why level 3 exists in the first place (#1020).

Alternatives Considered

  • --set-arg on the metadata buffer — what exists today. Cannot express a
    struct; silently yields a degenerate trace.
  • Hand-author the replay workspace (the insight-trace path) — works, but
    gives up exactly what l0_swimlane was built for ([Feature] Generate MindStudio Insight replay traces from kernel args dump #836): real captured args
    with no shape-guessing, and it must be redone whenever the tiling layout moves.
  • Synthesize the tiling in the replay by calling the tiling kernel first
    turns a single-op replay into a two-op one and reintroduces the ordering the
    replay recipe deliberately flattens.
  • Leave the category unrepresented — the status quo after Docs: correct two claims left behind by the qwen3 re-harvest #1529, and the
    reason for filing this.

Additional Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions