Skip to content

Add: partial-good-aware AICPU thread resolution (a2a3/a5) #1521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yanghaoran29 wants to merge 1 commit into
base: main
Choose a base branch
from
+120 −182
Open

Add: partial-good-aware AICPU thread resolution (a2a3/a5) #1521

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions examples/a2a3/tensormap_and_ringbuffer/benchmark_bgemm/test_benchmark_bgemm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,41 +52,35 @@ class TestBenchmarkBgemm(SceneTestCase):
{
"name": "Case0",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {"matmul_add_task_num": 500, "incore_data_size": 128, "incore_loop": 4, "grid_k": 2},
},
{
"name": "Case1",
"manual": True,
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {"matmul_add_task_num": 64, "incore_data_size": 128, "incore_loop": 4, "grid_k": 2},
},
{
"name": "Case2",
"manual": True,
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {"matmul_add_task_num": 256, "incore_data_size": 128, "incore_loop": 4, "grid_k": 2},
},
{
"name": "Case3",
"manual": True,
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {"matmul_add_task_num": 64, "incore_data_size": 128, "incore_loop": 16, "grid_k": 2},
},
{
"name": "Case4",
"manual": True,
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {"matmul_add_task_num": 64, "incore_data_size": 128, "incore_loop": 4, "grid_k": 4},
},
{
"name": "Bgemm64",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {"matmul_add_task_num": 32, "incore_data_size": 64, "incore_loop": 1, "grid_k": 4},
},
]
Expand Down
1 change: 0 additions & 1 deletion examples/a2a3/tensormap_and_ringbuffer/merge_pipeline_barrier/test_merge_pipeline_barrier.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,6 @@ class TestMergePipelineBarrier(SceneTestCase):
# per-task via launch_spec.set_block_num in merge_orch.cpp).
"name": "merge",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {},
},
]
Expand Down
7 changes: 0 additions & 7 deletions examples/a2a3/tensormap_and_ringbuffer/paged_attention/test_paged_attention.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "Case1",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 256,
"num_heads": 16,
Expand All @@ -79,7 +78,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "Case2",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -95,7 +93,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "Case3",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -111,7 +108,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "CaseSmall1",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 1,
"num_heads": 16,
Expand All @@ -126,7 +122,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "CaseSmall2",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 1,
Expand All @@ -142,7 +137,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "CaseVarSeq2",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 2,
Expand All @@ -159,7 +153,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "CaseVarSeq4",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 4,
Expand Down
8 changes: 1 addition & 7 deletions ...ensormap_and_ringbuffer/paged_attention_manual_scope/test_paged_attention_manual_scope.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ class TestPagedAttentionManualScope(SceneTestCase):
# MANUAL scope; the default per-ring task window (16384) can fill
# before the oldest task retires and wedge the orchestrator
# (FLOW_CONTROL_DEADLOCK / code 3). Double the window for headroom.
"config": {"aicpu_thread_num": 4, "runtime_env": {"ring_task_window": 32768}},
"config": {"runtime_env": {"ring_task_window": 32768}},
"params": {
"batch": 256,
"num_heads": 16,
Expand All @@ -83,7 +83,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "Case2",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -99,7 +98,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "Case3",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -115,7 +113,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "CaseSmall1",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 1,
"num_heads": 16,
Expand All @@ -130,7 +127,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "CaseSmall2",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 1,
Expand All @@ -146,7 +142,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "CaseVarSeq2",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 2,
Expand All @@ -163,7 +158,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "CaseVarSeq4",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 4,
Expand Down
1 change: 0 additions & 1 deletion ...a3/tensormap_and_ringbuffer/paged_attention_ringbuffer/test_paged_attention_ringbuffer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,6 @@ class TestPagedAttentionRingbuffer(SceneTestCase):
# ring_heap is bytes per ring. Non power-of-2 sizes are accepted,
# but 4 MiB keeps the small-ring stress intent compact.
"config": {
"aicpu_thread_num": 4,
"runtime_env": {
"ring_task_window": 64,
"ring_heap": 4 * 1024 * 1024,
Expand Down
3 changes: 0 additions & 3 deletions ...ingbuffer/paged_attention_unroll_manual_scope/test_paged_attention_unroll_manual_scope.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,6 @@ class TestPagedAttentionUnrollManualScope(SceneTestCase):
{
"name": "Case1",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 256,
"num_heads": 16,
Expand All @@ -79,7 +78,6 @@ class TestPagedAttentionUnrollManualScope(SceneTestCase):
{
"name": "Case2",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -95,7 +93,6 @@ class TestPagedAttentionUnrollManualScope(SceneTestCase):
{
"name": "Case3",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand Down
1 change: 0 additions & 1 deletion examples/a2a3/tensormap_and_ringbuffer/qwen3_14b_decode/test_qwen3_14b_decode.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,6 @@ class TestQwen314BDecode(SceneTestCase):
"name": "StressBatch16Seq3500",
"platforms": ["a2a3"],
# A run takes the whole device, matching the lib default.
"config": {"aicpu_thread_num": 4},
"params": {"seed": 1234, "seq_len": 3500},
},
]
Expand Down
1 change: 0 additions & 1 deletion examples/a2a3/tensormap_and_ringbuffer/scalar_data/test_scalar_data.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ class TestScalarData(SceneTestCase):
{
"name": "default",
"platforms": ["a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {},
},
]
Expand Down
1 change: 0 additions & 1 deletion examples/a2a3/tensormap_and_ringbuffer/vector_example/test_vector_example.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ class TestVectorExample(SceneTestCase):
{
"name": "default",
"platforms": ["a2a3sim", "a2a3"],
"config": {"aicpu_thread_num": 4},
"params": {},
},
]
Expand Down
1 change: 0 additions & 1 deletion examples/a5/tensormap_and_ringbuffer/bgemm/test_bgemm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,6 @@ class TestBgemm(SceneTestCase):
{
"name": "default",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"params": {},
}
]
Expand Down
7 changes: 0 additions & 7 deletions examples/a5/tensormap_and_ringbuffer/paged_attention/test_paged_attention.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "Case1",
"platforms": ["a5"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 256,
"num_heads": 16,
Expand All @@ -85,7 +84,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "Case2",
"platforms": ["a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -101,7 +99,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "Case3",
"platforms": ["a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -117,7 +114,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "SmallCase1",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 1,
"num_heads": 16,
Expand All @@ -132,7 +128,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "SmallCase2",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 1,
Expand All @@ -148,7 +143,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "SmallCaseVarSeq2",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 2,
Expand All @@ -165,7 +159,6 @@ class TestPagedAttention(SceneTestCase):
{
"name": "SmallCaseVarSeq4",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 4,
Expand Down
4 changes: 0 additions & 4 deletions ...ensormap_and_ringbuffer/paged_attention_manual_scope/test_paged_attention_manual_scope.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "SmallCase1",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 1,
"num_heads": 16,
Expand All @@ -81,7 +80,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "SmallCase2",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 1,
Expand All @@ -97,7 +95,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "SmallCaseVarSeq2",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 2,
Expand All @@ -114,7 +111,6 @@ class TestPagedAttentionManualScope(SceneTestCase):
{
"name": "SmallCaseVarSeq4",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 4,
Expand Down
5 changes: 0 additions & 5 deletions ...ingbuffer/paged_attention_unroll_manual_scope/test_paged_attention_unroll_manual_scope.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ class TestPagedAttentionUnrollManualScope(SceneTestCase):
{
"name": "Case1",
"platforms": ["a5"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 256,
"num_heads": 16,
Expand All @@ -81,7 +80,6 @@ class TestPagedAttentionUnrollManualScope(SceneTestCase):
{
"name": "Case2",
"platforms": ["a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -97,7 +95,6 @@ class TestPagedAttentionUnrollManualScope(SceneTestCase):
{
"name": "Case3",
"platforms": ["a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 64,
Expand All @@ -113,7 +110,6 @@ class TestPagedAttentionUnrollManualScope(SceneTestCase):
{
"name": "SmallCase1",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"params": {
"batch": 1,
"num_heads": 16,
Expand All @@ -128,7 +124,6 @@ class TestPagedAttentionUnrollManualScope(SceneTestCase):
{
"name": "SmallCase2",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"manual": True,
"params": {
"batch": 1,
Expand Down
1 change: 0 additions & 1 deletion examples/a5/tensormap_and_ringbuffer/vector_example/test_vector_example.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ class TestVectorExample(SceneTestCase):
{
"name": "default",
"platforms": ["a5sim", "a5"],
"config": {"aicpu_thread_num": 4},
"params": {},
},
]
Expand Down
2 changes: 1 addition & 1 deletion python/simpler/task_interface.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1284,7 +1284,7 @@ def run(
args: ChipStorageTaskArgs for this invocation.
config: Optional CallConfig. If None, a default is created.
**kwargs: Overrides applied to config (e.g.
``aicpu_thread_num=4``). A run always takes the whole device;
``aicpu_thread_num=2``). A run always takes the whole device;
orchestration reads the resulting width back through
``rt_available_cluster_count()``.
Expand Down
4 changes: 3 additions & 1 deletion simpler_setup/scene_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1127,7 +1127,9 @@ def _build_config(
from simpler.task_interface import CallConfig # noqa: PLC0415

config = CallConfig()
config.aicpu_thread_num = config_dict.get("aicpu_thread_num", 3)
# 0 = auto: DeviceRunner resolves to the arch default (1 orch + N sched)
# and clamps to the probed usable count (partial-good tolerance).
config.aicpu_thread_num = config_dict.get("aicpu_thread_num", 0)
# Per-task ring sizing (tensormap_and_ringbuffer only; 0 = unset),
# nested under the "runtime_env" key. Takes precedence over the
# PTO2_RING_* env vars / RUNTIME_ENV. Each value is either a scalar
Expand Down
4 changes: 2 additions & 2 deletions src/a2a3/docs/hardware.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,11 +145,11 @@ correctness, prefer ACL or CANN ini.

| You are doing… | Use |
| -------------- | --- |
| Configuring runtime `aicpu_thread_num` (per-case config) | **user-accessible** (6 — never request more) |
| Configuring runtime `aicpu_thread_num` (per-case config) | **0 = auto** -> 1 orch + 3 sched (=4, `PLATFORM_DEFAULT_AICPU_THREAD_NUM`), clamped to the probed usable pool (6); active cap 4 (`PLATFORM_MAX_AICPU_THREADS`); errors if usable < 2 |
| Setting kernel `block_dim` for AICore launch | **user-accessible** (24 — runtime rejects > 24) |
| Reading the spec sheet / product datasheet | **spec / user-accessible** (24 AIC, 6 AICPU) |
| Asking "what is in silicon" | **HAL physical** (25 AIC, 8 AICPU slots) — 1 PG-disabled + 1 OS-reserved on AICPU; 1 PG-disabled on AICore |
| Debugging "I set `aicpu_thread_num=8`, got error" | gap is **cpu_id 0 (AICPU OS) + cpu_id 1 (PG)** — both unreachable from runtime; cap is 6 |
| Debugging "I set `aicpu_thread_num=8`, got error" | active cap is **4** (`PLATFORM_MAX_AICPU_THREADS`); 6 is the launch pool (popcount OCCUPY). Gap to 8 AICPU slots = **cpu_id 0 (OS) + cpu_id 1 (PG)**, both unreachable from runtime |
| Debugging "I set `block_dim=25`, got error" | runtime cap is **24**; the +1 slot is PG fab-disabled |
| Building a chip-management dashboard | use HAL OCCUPY + OS_SCHED to expose the split; cite `tools/cann-examples/aicpu-device-query/` for the OS_SCHED query |

Expand Down
Loading
Loading