diff --git a/README.md b/README.md index 021b31bd9..00c796910 100644 --- a/README.md +++ b/README.md @@ -121,6 +121,6 @@ This project is licensed under the **CANN Open Software License Agreement Versio - [src/a2a3/platform/](src/a2a3/platform/) - Platform implementations - [src/a2a3/runtime/](src/a2a3/runtime/) - Runtime implementations -- [examples/a2a3/](examples/a2a3/) - Examples organized by runtime +- [examples/](examples/README.md) - Runnable examples, indexed: `workers/` for the raw `Worker` API, `a2a3/` and `a5/` for `@scene_test` kernels - [simpler_setup/](simpler_setup/) - SceneTestCase framework, runtime builder, kernel compiler - [python/](python/) - Python bindings and user-facing runtime API diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 000000000..acd0c309c --- /dev/null +++ b/examples/README.md @@ -0,0 +1,72 @@ +# Examples + +Two ways in, split by what you are trying to learn. + +| Directory | Entry point | Read it to learn | +| --------- | ----------- | ---------------- | +| [`workers/`](workers/README.md) | the raw `Worker` class | **How the runtime works.** Every step is explicit — build a `ChipCallable` from `.cpp` sources, pack `TaskArgs`, compose chips and sub-workers into a DAG. Organized by level: [`l2/`](workers/l2/README.md) is one chip, [`l3/`](workers/l3/README.md) is a host driving several. | +| [`a2a3/`](a2a3/tensormap_and_ringbuffer/README.md), [`a5/`](a5/tensormap_and_ringbuffer/README.md) | the `@scene_test` framework | **How to write and validate a kernel.** Case parametrization, golden comparison, and pytest integration are handled for you. Organized by architecture, then runtime. | + +Start with `workers/l2/hello_worker/` if the runtime is new to you; start with +your architecture's `vector_example/` if kernels are new to you. + +## Running them + +Everything under `examples/` is collected by pytest, and CI runs the whole tree +on both simulators: + +```bash +pytest examples --platform a2a3sim +pytest examples --platform a2a3 --device 0-1 # hardware +``` + +A single example: + +```bash +pytest examples/a2a3/tensormap_and_ringbuffer/vector_example --platform a2a3sim +``` + +Most `workers/` examples are also plain scripts, which is the quickest way to +read one end to end: + +```bash +python examples/workers/l2/hello_worker/main.py -p a2a3sim -d 0 +``` + +Each example declares the platforms it supports and the number of devices it +needs; pytest deselects the ones that do not apply to your `--platform`. On a +shared box, wrap hardware runs in `task-submit` — see +[`.claude/rules/running-onboard.md`](../.claude/rules/running-onboard.md). + +Requires the venv and `pip install .` described in +[`docs/getting-started.md`](../docs/getting-started.md). + +## Layout + +```text +examples/ +├── workers/ # raw Worker API, organized by level +│ ├── l2/ # one chip +│ └── l3/ # one host, several chips (+ SubWorkers) +├── a2a3/tensormap_and_ringbuffer/ # @scene_test kernels, a2a3 +└── a5/tensormap_and_ringbuffer/ # @scene_test kernels, a5 +``` + +An example directory holds its test file, a `kernels/` tree (`aic/`, `aiv/`, +`orchestration/`), and a README: + +```text +my_example/ +├── kernels/ +│ ├── aic/ # AICore-CUBE sources (optional) +│ ├── aiv/ # AICore-VECTOR sources (optional) +│ └── orchestration/ # orchestration C++ +├── test_my_example.py +└── README.md +``` + +## Related + +- [`docs/user/how-to/write-and-run-a-kernel.md`](../docs/user/how-to/write-and-run-a-kernel.md) — the guided version of what these examples show +- [`docs/testing.md`](../docs/testing.md) — the `@scene_test` framework and the wider test suite +- [`tests/st/`](../tests/st/) — scene tests proper, including the full collective-algorithm corpus diff --git a/examples/a2a3/tensormap_and_ringbuffer/README.md b/examples/a2a3/tensormap_and_ringbuffer/README.md new file mode 100644 index 000000000..ad725573b --- /dev/null +++ b/examples/a2a3/tensormap_and_ringbuffer/README.md @@ -0,0 +1,79 @@ +# a2a3 — `tensormap_and_ringbuffer` examples + +Kernels written against the `@scene_test` framework for the a2a3 architecture +and the `tensormap_and_ringbuffer` runtime. + +**Read [`docs/INCORE_ORCHESTRATION_GUIDE.md`](docs/INCORE_ORCHESTRATION_GUIDE.md) +first.** It explains the one thing every example here has in common: the +orchestration function runs *on the AICPU* and builds the task graph on device, +dependencies are discovered by TensorMap from tensor overlap rather than +declared, and task memory comes from ring buffers. Nothing below makes much +sense without it. + +For the `Worker` API underneath the framework, see +[`examples/workers/`](../../workers/README.md). + +## Start here + +| Example | What it teaches | +| ------- | --------------- | +| [`vector_example/`](vector_example/) | The smallest complete kernel: `f = (a+b+1)*(a+b+2) + (a+b)`. Runs on sim. | +| [`scalar_data/`](scalar_data/) | Orchestration-level data manipulation — `GetTensorData` / `SetTensorData` round-trips, `add_inout`, and automatic WAW / WAR waits including on external tensors. | + +## Compute + +| Example | What it teaches | +| ------- | --------------- | +| [`benchmark_bgemm/`](benchmark_bgemm/) | Runtime-configurable tiled matmul `C = sum(k) A[k] @ B[k]`, shaped for measurement. Runs on sim. | +| [`paged_attention/`](paged_attention/) | Online softmax with AIC/AIV subgraph splitting, bfloat16. The baseline the three variants below are compared against. | +| [`paged_attention_manual_scope/`](paged_attention_manual_scope/) | The same computation with explicit scope control — see [`docs/manual-scope.md`](../../../docs/manual-scope.md). | +| [`paged_attention_unroll_manual_scope/`](paged_attention_unroll_manual_scope/) | Manual scope plus loop unrolling. | +| [`paged_attention_ringbuffer/`](paged_attention_ringbuffer/) | Deliberately undersized rings, driven per case through `config.runtime_env` (`ring_task_window` / `ring_heap` / `ring_dep_pool`) rather than process-global env. A stress test for rotation and reclamation. | +| [`merge_pipeline_barrier/`](merge_pipeline_barrier/) | Three pipeline stages merged into **one** `block_num=8` SPMD task, ordered by an intra-task cross-core barrier instead of by three scheduled tasks. | +| [`qwen3_14b_decode/`](qwen3_14b_decode/README.md) | The whole Qwen3-14B 40-layer decode stack as a single fused dispatch, using CANN fused attention. The largest example in the repo. | + +## Asynchronous completion and cross-card transfer + +Each of these registers an async event and lets the consumer wait on deferred +completion rather than on task end. All but `deferred_notify_demo`, which runs +the simulator path, are **onboard-only**; every one except `prefetch_async_demo` +needs two dies. + +| Example | Mechanism | Devices | +| ------- | --------- | ------- | +| [`prefetch_async_demo/`](prefetch_async_demo/) | `TPREFETCH_ASYNC` over the runtime-injected SDMA workspace, provisioned once by `Worker(enable_sdma=True)` and injected into every kernel's `GlobalContext`. | 1 | +| [`sdma_async_completion_demo/`](sdma_async_completion_demo/) | `TGET_ASYNC` from a peer's window slot, completion registered via `defer_pto_async_event`. | 2 | +| [`async_notify_demo/`](async_notify_demo/) | Notification counters alongside deferred completion. | 2 | +| [`deferred_notify_demo/`](deferred_notify_demo/) | The same shape on the simulator. | 2 | + +## Running + +```bash +# Everything that runs on the simulator +pytest examples/a2a3/tensormap_and_ringbuffer --platform a2a3sim + +# One example, on hardware +pytest examples/a2a3/tensormap_and_ringbuffer/paged_attention --platform a2a3 --device 0 +``` + +Most examples here are marked `platforms=["a2a3"]` — onboard only — because +they exercise real AIC/AIV timing or cross-card transfer. `vector_example` and +`benchmark_bgemm` are marked for `a2a3sim` as well, and `deferred_notify_demo` +always runs the simulator path. + +Wrap hardware runs in `task-submit` on a shared box; see +[`.claude/rules/running-onboard.md`](../../../.claude/rules/running-onboard.md). + +## Relationship to `examples/a5/` + +Seven examples exist under both architectures with the same name: +`vector_example`, `paged_attention`, `paged_attention_manual_scope`, +`paged_attention_unroll_manual_scope`, `async_notify_demo`, +`deferred_notify_demo`, `sdma_async_completion_demo`. They are ports of each +other and differ mainly in tile shapes and platform strings — `vector_example` +differs by two lines. When you change one, check whether its sibling needs the +same change. + +Only here: `benchmark_bgemm` (a5 has `bgemm` instead), +`merge_pipeline_barrier`, `paged_attention_ringbuffer`, `prefetch_async_demo`, +`qwen3_14b_decode`, `scalar_data`. diff --git a/examples/a5/tensormap_and_ringbuffer/README.md b/examples/a5/tensormap_and_ringbuffer/README.md new file mode 100644 index 000000000..714f7b1db --- /dev/null +++ b/examples/a5/tensormap_and_ringbuffer/README.md @@ -0,0 +1,73 @@ +# a5 — `tensormap_and_ringbuffer` examples + +Kernels written against the `@scene_test` framework for the a5 architecture and +the `tensormap_and_ringbuffer` runtime. + +The orchestration model is the same as on a2a3 — the orchestration function +runs on the AICPU, TensorMap discovers dependencies from tensor overlap, task +memory comes from ring buffers. That is written up once, under a2a3: +[`examples/a2a3/tensormap_and_ringbuffer/docs/INCORE_ORCHESTRATION_GUIDE.md`](../../a2a3/tensormap_and_ringbuffer/docs/INCORE_ORCHESTRATION_GUIDE.md). + +For the `Worker` API underneath the framework, see +[`examples/workers/`](../../workers/README.md). + +## Start here + +| Example | What it teaches | +| ------- | --------------- | +| [`vector_example/`](vector_example/) | The smallest complete kernel: `f = (a+b+1)*(a+b+2) + (a+b)`. Runs on sim. | +| [`bgemm/`](bgemm/) | Batched tiled matmul `C = A @ B` — a fixed 4×4×4 grid of 64×64 tiles over 2 batches, with AIC doing the matmul and AIV the accumulation. The clearest two-core-type example here. Runs on sim. | + +## Paged attention + +| Example | What it teaches | +| ------- | --------------- | +| [`paged_attention/`](paged_attention/) | Online softmax with mixed AIC/AIV execution, bfloat16, at production scale. Onboard only. | +| [`paged_attention_manual_scope/`](paged_attention_manual_scope/) | The same computation with explicit scope control — see [`docs/manual-scope.md`](../../../docs/manual-scope.md). Also runs on sim. | +| [`paged_attention_unroll_manual_scope/`](paged_attention_unroll_manual_scope/) | Manual scope plus loop unrolling. Onboard only. | + +## Asynchronous completion and cross-card transfer + +Each registers an async event and lets the consumer wait on deferred completion +rather than on task end. All need two dies. + +| Example | Mechanism | +| ------- | --------- | +| [`sdma_async_completion_demo/`](sdma_async_completion_demo/) | `TGET_ASYNC` from a peer's window slot over SDMA, completion registered via `defer_pto_async_event`. | +| [`urma_deferred_completion_demo/`](urma_deferred_completion_demo/) | The same protocol over **URMA** instead — the a5-only counterpart, and the reason this pair is worth reading side by side. Carries an extra `skipif` guard beyond the platform marker. | +| [`async_notify_demo/`](async_notify_demo/) | Notification counters alongside deferred completion. | +| [`deferred_notify_demo/`](deferred_notify_demo/) | The same shape on the simulator. | + +## Running + +```bash +# Everything that runs on the simulator +pytest examples/a5/tensormap_and_ringbuffer --platform a5sim + +# One example, on hardware +pytest examples/a5/tensormap_and_ringbuffer/bgemm --platform a5 --device 0 +``` + +`vector_example`, `bgemm`, `paged_attention_manual_scope`, and +`deferred_notify_demo` run under `a5sim`; the rest are onboard-only. + +Wrap hardware runs in `task-submit` on a shared box; see +[`.claude/rules/running-onboard.md`](../../../.claude/rules/running-onboard.md). +Check the box's silicon first — `.claude/skills/onboard-arch-precheck/check.sh a5` +refuses a wrong-arch invocation before any device lock is taken. + +## Relationship to `examples/a2a3/` + +Seven examples exist under both architectures with the same name: +`vector_example`, `paged_attention`, `paged_attention_manual_scope`, +`paged_attention_unroll_manual_scope`, `async_notify_demo`, +`deferred_notify_demo`, `sdma_async_completion_demo`. They are ports of each +other and differ mainly in tile shapes and platform strings — `vector_example` +differs by two lines. When you change one, check whether its sibling needs the +same change. + +Only here: `bgemm` (a2a3 has `benchmark_bgemm` instead) and +`urma_deferred_completion_demo`. a2a3 additionally carries +`merge_pipeline_barrier`, `paged_attention_ringbuffer`, `prefetch_async_demo`, +`qwen3_14b_decode`, and `scalar_data`, none of which have an a5 port — +`tests/st` tracks that gap separately (PR #1450 ports the `tests/st` side).