Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,6 @@ This project is licensed under the **CANN Open Software License Agreement Versio

- [src/a2a3/platform/](src/a2a3/platform/) - Platform implementations
- [src/a2a3/runtime/](src/a2a3/runtime/) - Runtime implementations
- [examples/a2a3/](examples/a2a3/) - Examples organized by runtime
- [examples/](examples/README.md) - Runnable examples, indexed: `workers/` for the raw `Worker` API, `a2a3/` and `a5/` for `@scene_test` kernels
- [simpler_setup/](simpler_setup/) - SceneTestCase framework, runtime builder, kernel compiler
- [python/](python/) - Python bindings and user-facing runtime API
72 changes: 72 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Examples

Two ways in, split by what you are trying to learn.

| Directory | Entry point | Read it to learn |
| --------- | ----------- | ---------------- |
| [`workers/`](workers/README.md) | the raw `Worker` class | **How the runtime works.** Every step is explicit — build a `ChipCallable` from `.cpp` sources, pack `TaskArgs`, compose chips and sub-workers into a DAG. Organized by level: [`l2/`](workers/l2/README.md) is one chip, [`l3/`](workers/l3/README.md) is a host driving several. |
| [`a2a3/`](a2a3/tensormap_and_ringbuffer/README.md), [`a5/`](a5/tensormap_and_ringbuffer/README.md) | the `@scene_test` framework | **How to write and validate a kernel.** Case parametrization, golden comparison, and pytest integration are handled for you. Organized by architecture, then runtime. |

Start with `workers/l2/hello_worker/` if the runtime is new to you; start with
your architecture's `vector_example/` if kernels are new to you.

## Running them

Everything under `examples/` is collected by pytest, and CI runs the whole tree
on both simulators:

```bash
pytest examples --platform a2a3sim
pytest examples --platform a2a3 --device 0-1 # hardware
Comment thread
coderabbitai[bot] marked this conversation as resolved.
```

A single example:

```bash
pytest examples/a2a3/tensormap_and_ringbuffer/vector_example --platform a2a3sim
```

Most `workers/` examples are also plain scripts, which is the quickest way to
read one end to end:

```bash
python examples/workers/l2/hello_worker/main.py -p a2a3sim -d 0
```

Each example declares the platforms it supports and the number of devices it
needs; pytest deselects the ones that do not apply to your `--platform`. On a
shared box, wrap hardware runs in `task-submit` — see
[`.claude/rules/running-onboard.md`](../.claude/rules/running-onboard.md).

Requires the venv and `pip install .` described in
[`docs/getting-started.md`](../docs/getting-started.md).

## Layout

```text
examples/
├── workers/ # raw Worker API, organized by level
│ ├── l2/ # one chip
│ └── l3/ # one host, several chips (+ SubWorkers)
├── a2a3/tensormap_and_ringbuffer/ # @scene_test kernels, a2a3
└── a5/tensormap_and_ringbuffer/ # @scene_test kernels, a5
```

An example directory holds its test file, a `kernels/` tree (`aic/`, `aiv/`,
`orchestration/`), and a README:

```text
my_example/
├── kernels/
│ ├── aic/ # AICore-CUBE sources (optional)
│ ├── aiv/ # AICore-VECTOR sources (optional)
│ └── orchestration/ # orchestration C++
├── test_my_example.py
└── README.md
```

## Related

- [`docs/user/how-to/write-and-run-a-kernel.md`](../docs/user/how-to/write-and-run-a-kernel.md) — the guided version of what these examples show
- [`docs/testing.md`](../docs/testing.md) — the `@scene_test` framework and the wider test suite
- [`tests/st/`](../tests/st/) — scene tests proper, including the full collective-algorithm corpus
79 changes: 79 additions & 0 deletions examples/a2a3/tensormap_and_ringbuffer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# a2a3 — `tensormap_and_ringbuffer` examples

Kernels written against the `@scene_test` framework for the a2a3 architecture
and the `tensormap_and_ringbuffer` runtime.

**Read [`docs/INCORE_ORCHESTRATION_GUIDE.md`](docs/INCORE_ORCHESTRATION_GUIDE.md)
first.** It explains the one thing every example here has in common: the
orchestration function runs *on the AICPU* and builds the task graph on device,
dependencies are discovered by TensorMap from tensor overlap rather than
declared, and task memory comes from ring buffers. Nothing below makes much
sense without it.

For the `Worker` API underneath the framework, see
[`examples/workers/`](../../workers/README.md).

## Start here

| Example | What it teaches |
| ------- | --------------- |
| [`vector_example/`](vector_example/) | The smallest complete kernel: `f = (a+b+1)*(a+b+2) + (a+b)`. Runs on sim. |
| [`scalar_data/`](scalar_data/) | Orchestration-level data manipulation — `GetTensorData` / `SetTensorData` round-trips, `add_inout`, and automatic WAW / WAR waits including on external tensors. |

## Compute

| Example | What it teaches |
| ------- | --------------- |
| [`benchmark_bgemm/`](benchmark_bgemm/) | Runtime-configurable tiled matmul `C = sum(k) A[k] @ B[k]`, shaped for measurement. Runs on sim. |
| [`paged_attention/`](paged_attention/) | Online softmax with AIC/AIV subgraph splitting, bfloat16. The baseline the three variants below are compared against. |
| [`paged_attention_manual_scope/`](paged_attention_manual_scope/) | The same computation with explicit scope control — see [`docs/manual-scope.md`](../../../docs/manual-scope.md). |
| [`paged_attention_unroll_manual_scope/`](paged_attention_unroll_manual_scope/) | Manual scope plus loop unrolling. |
| [`paged_attention_ringbuffer/`](paged_attention_ringbuffer/) | Deliberately undersized rings, driven per case through `config.runtime_env` (`ring_task_window` / `ring_heap` / `ring_dep_pool`) rather than process-global env. A stress test for rotation and reclamation. |
| [`merge_pipeline_barrier/`](merge_pipeline_barrier/) | Three pipeline stages merged into **one** `block_num=8` SPMD task, ordered by an intra-task cross-core barrier instead of by three scheduled tasks. |
| [`qwen3_14b_decode/`](qwen3_14b_decode/README.md) | The whole Qwen3-14B 40-layer decode stack as a single fused dispatch, using CANN fused attention. The largest example in the repo. |

## Asynchronous completion and cross-card transfer

Each of these registers an async event and lets the consumer wait on deferred
completion rather than on task end. All but `deferred_notify_demo`, which runs
the simulator path, are **onboard-only**; every one except `prefetch_async_demo`
needs two dies.

| Example | Mechanism | Devices |
| ------- | --------- | ------- |
| [`prefetch_async_demo/`](prefetch_async_demo/) | `TPREFETCH_ASYNC` over the runtime-injected SDMA workspace, provisioned once by `Worker(enable_sdma=True)` and injected into every kernel's `GlobalContext`. | 1 |
| [`sdma_async_completion_demo/`](sdma_async_completion_demo/) | `TGET_ASYNC` from a peer's window slot, completion registered via `defer_pto_async_event`. | 2 |
| [`async_notify_demo/`](async_notify_demo/) | Notification counters alongside deferred completion. | 2 |
| [`deferred_notify_demo/`](deferred_notify_demo/) | The same shape on the simulator. | 2 |

## Running

```bash
# Everything that runs on the simulator
pytest examples/a2a3/tensormap_and_ringbuffer --platform a2a3sim

# One example, on hardware
pytest examples/a2a3/tensormap_and_ringbuffer/paged_attention --platform a2a3 --device 0
```

Most examples here are marked `platforms=["a2a3"]` — onboard only — because
they exercise real AIC/AIV timing or cross-card transfer. `vector_example` and
`benchmark_bgemm` are marked for `a2a3sim` as well, and `deferred_notify_demo`
always runs the simulator path.

Wrap hardware runs in `task-submit` on a shared box; see
[`.claude/rules/running-onboard.md`](../../../.claude/rules/running-onboard.md).

## Relationship to `examples/a5/`

Seven examples exist under both architectures with the same name:
`vector_example`, `paged_attention`, `paged_attention_manual_scope`,
`paged_attention_unroll_manual_scope`, `async_notify_demo`,
`deferred_notify_demo`, `sdma_async_completion_demo`. They are ports of each
other and differ mainly in tile shapes and platform strings — `vector_example`
differs by two lines. When you change one, check whether its sibling needs the
same change.

Only here: `benchmark_bgemm` (a5 has `bgemm` instead),
`merge_pipeline_barrier`, `paged_attention_ringbuffer`, `prefetch_async_demo`,
`qwen3_14b_decode`, `scalar_data`.
73 changes: 73 additions & 0 deletions examples/a5/tensormap_and_ringbuffer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# a5 — `tensormap_and_ringbuffer` examples

Kernels written against the `@scene_test` framework for the a5 architecture and
the `tensormap_and_ringbuffer` runtime.

The orchestration model is the same as on a2a3 — the orchestration function
runs on the AICPU, TensorMap discovers dependencies from tensor overlap, task
memory comes from ring buffers. That is written up once, under a2a3:
[`examples/a2a3/tensormap_and_ringbuffer/docs/INCORE_ORCHESTRATION_GUIDE.md`](../../a2a3/tensormap_and_ringbuffer/docs/INCORE_ORCHESTRATION_GUIDE.md).

For the `Worker` API underneath the framework, see
[`examples/workers/`](../../workers/README.md).

## Start here

| Example | What it teaches |
| ------- | --------------- |
| [`vector_example/`](vector_example/) | The smallest complete kernel: `f = (a+b+1)*(a+b+2) + (a+b)`. Runs on sim. |
| [`bgemm/`](bgemm/) | Batched tiled matmul `C = A @ B` — a fixed 4×4×4 grid of 64×64 tiles over 2 batches, with AIC doing the matmul and AIV the accumulation. The clearest two-core-type example here. Runs on sim. |

## Paged attention

| Example | What it teaches |
| ------- | --------------- |
| [`paged_attention/`](paged_attention/) | Online softmax with mixed AIC/AIV execution, bfloat16, at production scale. Onboard only. |
| [`paged_attention_manual_scope/`](paged_attention_manual_scope/) | The same computation with explicit scope control — see [`docs/manual-scope.md`](../../../docs/manual-scope.md). Also runs on sim. |
| [`paged_attention_unroll_manual_scope/`](paged_attention_unroll_manual_scope/) | Manual scope plus loop unrolling. Onboard only. |

## Asynchronous completion and cross-card transfer

Each registers an async event and lets the consumer wait on deferred completion
rather than on task end. All need two dies.

| Example | Mechanism |
| ------- | --------- |
| [`sdma_async_completion_demo/`](sdma_async_completion_demo/) | `TGET_ASYNC` from a peer's window slot over SDMA, completion registered via `defer_pto_async_event`. |
| [`urma_deferred_completion_demo/`](urma_deferred_completion_demo/) | The same protocol over **URMA** instead — the a5-only counterpart, and the reason this pair is worth reading side by side. Carries an extra `skipif` guard beyond the platform marker. |
| [`async_notify_demo/`](async_notify_demo/) | Notification counters alongside deferred completion. |
| [`deferred_notify_demo/`](deferred_notify_demo/) | The same shape on the simulator. |

## Running

```bash
# Everything that runs on the simulator
pytest examples/a5/tensormap_and_ringbuffer --platform a5sim

# One example, on hardware
pytest examples/a5/tensormap_and_ringbuffer/bgemm --platform a5 --device 0
```

`vector_example`, `bgemm`, `paged_attention_manual_scope`, and
`deferred_notify_demo` run under `a5sim`; the rest are onboard-only.

Wrap hardware runs in `task-submit` on a shared box; see
[`.claude/rules/running-onboard.md`](../../../.claude/rules/running-onboard.md).
Check the box's silicon first — `.claude/skills/onboard-arch-precheck/check.sh a5`
refuses a wrong-arch invocation before any device lock is taken.

## Relationship to `examples/a2a3/`

Seven examples exist under both architectures with the same name:
`vector_example`, `paged_attention`, `paged_attention_manual_scope`,
`paged_attention_unroll_manual_scope`, `async_notify_demo`,
`deferred_notify_demo`, `sdma_async_completion_demo`. They are ports of each
other and differ mainly in tile shapes and platform strings — `vector_example`
differs by two lines. When you change one, check whether its sibling needs the
same change.

Only here: `bgemm` (a2a3 has `benchmark_bgemm` instead) and
`urma_deferred_completion_demo`. a2a3 additionally carries
`merge_pipeline_barrier`, `paged_attention_ringbuffer`, `prefetch_async_demo`,
`qwen3_14b_decode`, and `scalar_data`, none of which have an a5 port —
`tests/st` tracks that gap separately (PR #1450 ports the `tests/st` side).
Loading