From 5a8c47cf634f6e0175a9df5408f6ce5d5e9c696f Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Fri, 17 Jul 2026 08:16:13 +0000 Subject: [PATCH 001/214] Add: document A5 FDWIC Submit atomic analysis --- tests/atomic_probe/README.zh-CN.md | 300 +++++++++++++++++++++++++++++ 1 file changed, 300 insertions(+) create mode 100644 tests/atomic_probe/README.zh-CN.md diff --git a/tests/atomic_probe/README.zh-CN.md b/tests/atomic_probe/README.zh-CN.md new file mode 100644 index 0000000000..5397debd87 --- /dev/null +++ b/tests/atomic_probe/README.zh-CN.md @@ -0,0 +1,300 @@ +# A5 FDWIC Paged Attention Submit 原子操作与优化记录 + +## 1. 范围与当前结论 + +本文记录 `TestPagedAttentionUnroll::Case1` 在真实 A5 上的 FDWIC AICore +Submit 路径,供后续继续优化。快照日期为 2026-07-17,当前代码提交为 +`e3b748b43c7f226c025c4dcdfc2eb2805cec7f21`。 + +范围限定为: + +- 用例:`examples/a5/fully_distributed_within_core/paged_attention_unroll/`; +- Case:`Case1`; +- runtime:`fully_distributed_within_core`; +- 平台:真实 A5,A5Sim 只用于功能回归; +- 性能口径:所有 worker 中最早的 `Submit.start` 到最晚的 `Submit.end`; +- 原子操作清单:该用例实际经过的 FDWIC Submit、执行完成和外围生命周期路径。 + +当前结论: + +- 96 个 worker(32 AIC + 64 AIV)分别回放 1280 次 Submit,共有 + 122880 个完整 Submit 事件; +- 本轮已经消除了纯单 lane 图中 146944 次无效的 BlockWon + `atomic_load(any_pub)`,在 A5 上实际对应 `atomicAdd(addr, 0)`; +- 仍在 Submit 热路径中执行且数量最大的原子操作是 Claim:固定 + 73728 次 `atomicMax`; +- 三轮最终版本的首末 Submit 中位数为 5.115620 ms,相比 + 5.642245 ms 基线下降 0.526625 ms,即 9.33%;最好单轮为 + 5.096685 ms; +- 优化没有改变通用 atomic 语义,也没有把任务推迟到最终 drain 来制造 + 表面收益;下一优先级应是 Claim 竞争,其次是 completion/frontier 和 + fanin ready 轮询。 + +环境安装、编译和基线复现过程见 +[A5 FDWIC Paged Attention 安装与复现指南](A5_FDWIC_PAGED_ATTENTION_REPRO.md)。 + +## 2. Case1 工作量与 atomic 语义 + +Case1 的关键参数是 `batch=256`、`num_heads=16`、`block_size=128`、 +`context_len=8192`。当前 orchestration 中 `N_UNROLL=64`,因此每个 batch +只有一个 block group,且 `q_loop=1`。每个 worker 回放的 task id 分布如下: + +| task id | 每 batch 操作 | 执行 lane | fanin 数 | 新分配输出 | +| ------- | ------------- | --------- | -------: | ---------- | +| `5b+0` | Alloc | 任一 worker 竞争 winner | 0 | 有 | +| `5b+1` | QK | AIC | 0 | 有 | +| `5b+2` | SF | AIV | 1 | 有 | +| `5b+3` | PV | AIC | 1 | 有 | +| `5b+4` | UP | AIV | 3 | 无,仅 INOUT | + +其中 `b=0..255`,所以每核恰好回放 1280 个 task;全局实际执行 +256 个 Alloc 和 1024 个 kernel task。 + +A5 CCEC 下的封装位于 +`src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/atomic.h`: + +| C++ 封装 | A5 指令封装 | 说明 | +| -------- | ----------- | ---- | +| `atomic_load(x)` | `atomicAdd(&x, 0)` | 不是普通 load,而是 read-modify-write | +| `atomic_exchange(x, v)` | `atomicExch(&x, v)` | 发布或重置共享状态 | +| `atomic_fetch_add(x, v)` | `atomicAdd(&x, v)` | 计数器递增 | +| `atomic_fetch_sub(x, v)` | `atomicSub(&x, v)` | joint task 完成计数 | +| `atomic_fetch_max(x, v)` | `atomicMax(&x, v)` | Claim 和 frontier 前推 | + +因此,即使源码写的是 `atomic_load`,大量 worker 读取同一个地址时仍会形成 +RMW 竞争。本文中的次数分为三类: + +- “固定”表示可以由 Case1 拓扑和源码精确推出; +- “下界”表示每次正确执行至少发生这些操作,竞争和未就绪轮询会增加次数; +- “条件”表示 trace 只能确认分支是否等待,不能直接得到 atomic 指令计数。 + +## 3. 当前原子操作分布 + +### 3.1 Claim cursor:当前最大固定项 + +调用链为 +`submit_runtime.h::dist_submit_claim_*()` -> +`submit_helpers.h::claim()` -> `atomic_fetch_max(cursor, task_id)`。 +Claim span 会在所有 worker 上记录,但只有符合 lane 的 worker 执行 +`atomicMax`;Alloc 则由全部 96 个 worker 竞争。 + +| 操作 | task 数 | 每 task 竞争 worker | 固定 atomicMax 次数 | +| ---- | ------: | ------------------: | ------------------: | +| Alloc | 256 | 96 | 24576 | +| QK | 256 | 32 AIC | 8192 | +| SF | 256 | 64 AIV | 16384 | +| PV | 256 | 32 AIC | 8192 | +| UP | 256 | 64 AIV | 16384 | +| 合计 | 1280 | - | **73728** | + +每核累计 Claim 中位数中,AIC 从基线 0.517 ms 上升到最终三轮中位数 +0.608 ms,AIV 从 0.515 ms 上升到 0.613 ms。跳过 BlockWon 轮询后 +worker 更同步,cursor 瞬时竞争反而变强。总时间仍然下降,但 Claim 已成为 +最明确的剩余原子热点。 + +### 3.2 Heap 容量保护:winner action tail + +位置为 `submit_runtime.h::dist_submit_wait_heap_capacity()`。只有 winner 且 +`output_bytes>0` 时进入检查;本 Case 每 batch 的 Alloc、QK、SF、PV 满足, +UP 不满足,所以共有 1024 次有效调用。 + +最好一轮没有 `RingBp` 事件,表示每次调用都在第一次循环检查后返回: + +- `fatal_set()`:固定 1024 次 `atomic_load(g_dist.fatal)`; +- frontier:固定 1024 次 `atomic_load(g_dist.frontier)`; +- `load_task_vend(F-H)`:当索引非负时执行 `atomic_load(cell.vend)`,本轮 + 次数介于 0 和 1024,现有 trace 不能给出精确值; +- 如果未来出现 heap backpressure,上述三项会在循环中重复,耗时会归入 + `RingBp`。 + +无等待时,这些 atomic 位于 winner 的 Build/Alloc action 中。对 kernel +Submit 来说,泳道上表现为 `Build.end` 到 `Submit.end` 的尾段。 + +### 3.3 fanin ready 与 task completion + +依赖轮询位于 `submit_core.h::drain_phase_b()`:每个 fanin 通过 +`task_flag_ready()` 执行一次 `atomic_load(cell.flag)`。QK、SF、PV、UP 的 +fanin 分别为 0、1、1、3,因此任务真正开始执行前至少有: + +~~~text +256 * (0 + 1 + 1 + 3) = 1280 次 atomic_load(flag) +~~~ + +依赖未就绪时会提前退出并在下一次 drain 重试,所以 1280 只是成功检查的 +下界,不包含失败轮询。它们发生在执行 kernel 的 drain 位置:通常是下一个 +Submit 的 `EfDrain`,有背压时可能在 `RingBp`,剩余任务则在最终 drain。 + +每个 task 完成一次,路径为 +`execute_slot()/dist_submit_complete_alloc()` -> +`complete_executed_task()` -> `advance_frontier()`: + +| 共享状态 | 操作 | 次数性质 | Case1 次数 | +| -------- | ---- | -------- | ---------: | +| `task.vend` | `atomic_exchange` | 每 task 固定一次 | 1280 | +| `task.flag` 发布 | `atomic_exchange` | 每 task 固定一次 | 1280 | +| `frontier` 初始读取 | `atomic_load` | 每 task 固定一次 | 1280 | +| 后续 `task.flag` ready | `atomic_load` | 下界 | >=1280 | +| `frontier` 前推 | `atomic_fetch_max` | 下界,竞争时可重复 | >=1280 | + +最终最好一轮的 1024 个 kernel 中,1011 个在某个 Submit 的 `EfDrain` +中执行,0 个在 `RingBp` 中执行,13 个在本核最后一个 Submit 之后的最终 +drain 中执行。completion atomic 紧随 kernel,分布也基本相同;Alloc 的 +completion 则在 Alloc winner 的 Submit 内完成。基线对应分布是 +979 个 `EfDrain`、29 个 `RingBp`、16 个最终 drain。 + +### 3.4 BlockWon:代码仍保留,当前 Case 动态次数为零 + +BlockWon 用于同一物理 block 内至少两个 lane 联合执行一个 task。相关位置: + +- `submit_helpers.h::alloc_won_slot()`:`atomicMax(state)`; +- `submit_helpers.h::populate_won_slot_from_submit()`:`atomicExch(drained)`; +- `submit_core.h::publish_won_slot()`、`clear_won_slot_state()`: + `atomicExch(state)`; +- `submit_core.h::claim_won_lane()`:`atomicExch(drained)`; +- `submit_core.h::decrement_won_remaining_is_last()`:`atomicSub(remaining)`; +- `submit_runtime.h::publish_joint_deposits()`:`atomicExch(any_pub)`; +- `submit_core.h::drain_block_won()`、`has_pending_won()`:读取 + `any_pub/state/drained`,在 A5 上均为 `atomicAdd(addr, 0)`。 + +PA Case1 的 QK/PV 为单 AIC lane,SF/UP 为单 AIV lane,没有 joint task, +trace 中也没有 `DrainWon`。当前由 worker-local 的 +`g_fdwic_joint_submit_seen` 关闭这些轮询,所以以上 BlockWon atomic 在该 Case 的 AICore +Submit/执行路径中动态次数为 **0**;AICPU 初始化仍会重置相关状态,joint 用例也仍走完整原协议。 + +### 3.5 Submit 窗口外和非本路径 atomic + +这些操作不属于用户关注的首末 Submit 耗时,但做端到端优化时不能忽略: + +- worker 启动屏障,`core_main.h`:96 次 + `atomic_fetch_add(started_count)`,以及等待期间重复读取 + `started_count/fatal`; +- worker 最终 drain,`submit_runtime.h`:96 次 + `atomic_fetch_add(replay_done)`,以及等待期间重复读取 `replay_done`; +- AICPU 初始化,`aicpu/control_plane.h`:12 个分片 cursor、frontier、 + fatal、replay_done、started_count、32 个 BlockWon 的 `any_pub` 和 + 128 个 won slot state 均以 atomic exchange 重置;65536 个 task cell + 各重置 flag/vend,共 131072 次 atomic exchange; +- AICPU 外层生命周期,`aicpu/aicpu_executor.cpp`:线程状态使用 + `std::atomic`,并轮询 `runtime->dist.done_count`。真实 A5 的 AICore 通过 + COND 寄存器发布完成,不走 A5Sim 中的 `done_count` atomic add; +- `debug_dump.h` 中的 atomic load 只在诊断 dump 时运行;centralized + scheduler 的 completion mailbox 和 SDMA completion atomic 不属于本 + FDWIC PA 执行路径,未计入本文。 + +异常路径中的 `set_fatal()` 使用 `atomic_exchange(g_dist.fatal, 1)`;上述 +成功运行没有触发。由于 fanin/frontier/屏障轮询次数依赖时序,不能把静态 +下界当作整次运行的 atomic 总数。若需要精确动态计数,应使用 PMU 或独立 +诊断构建;直接在热路径增加共享计数器会反过来改变竞争形态。 + +## 4. 当前优化逻辑与效果 + +### 4.1 首个 joint task 前跳过 BlockWon 轮询 + +涉及文件: + +- `dist_engine/common/worker_state.h`:增加 worker-local、单调的 + `g_fdwic_joint_submit_seen`; +- `dist_engine/aicore/core_main.h`:每次运行开始时重置为 `false`; +- `dist_engine/aicore/submit_runtime.h`:当当前 active core mask 的 + popcount 大于等于 2 时,在该 Submit 的首次 drain 前置为 `true`; +- `dist_engine/aicore/submit_core.h`:flag 为 `false` 时, + `drain_block_won()` 和 `has_pending_won()` 直接返回。 + +flag 一旦变为 `true` 就不再清零。不能只根据“当前 task 不是 joint”跳过, +因为较快 worker 可能已经发布后续 joint slot,而较慢 worker 仍在处理前一 +task。当前方案利用所有 worker 回放相同 task stream 的条件:较慢 worker +到达自己的首个 joint task 时,会在首次 drain 之前打开轮询;此后保留原 +协议,所以不会漏掉已发布或稍后发布的 won slot。 + +对纯单 lane 的 PA Case1,flag 始终为 `false`。因此删除的是: + +- 每次 Submit 开头、归入 `EfDrain` 的 AIV `any_pub` load: + `64 * 1280 = 81920` 次; +- kernel loser 在 `Replay.end` 到 `Submit.end` 尾段的 load: + QK/PV 的 `512 * 64 = 32768` 次,加 SF/UP 的 + `512 * 63 = 32256` 次,共 65024 次; +- Submit 内合计 146944 次无效 `atomicAdd(addr, 0)`。 + +所以本次 atomic 优化直接缩短的是 `EfDrain` 和 loser 的 +`Replay.end -> Submit.end` 尾段,不是 Claim。三轮 joint-skip 版本的全局 +中位数为 5.171330 ms,相对基线下降 0.470915 ms(8.35%)。AIV 每核累计 +`EfDrain` 中位数由约 0.765 ms 降至约 0.409 ms;Replay 后尾段也明显 +缩短。同时 Claim 竞争上升,抵消了一部分收益。 + +### 4.2 复用参数 tag 扫描结果 + +该项不是 atomic 优化,目标是继续减少每核重复执行的 Submit 前端工作: + +- `calculate_output_layout()` 第一次扫描 tensor tag 时同时生成 + `output_mask` 和 `register_mask`; +- materialize 用 `output_mask` 只访问 OUTPUT 参数; +- register 用保存在 `DistSubmitCtx` 中的 `register_mask` 只访问 INOUT 和 + OUTPUT_EXISTING 参数; +- 不改变参数 tag、tensor map 行为或对外接口。 + +只加入 register mask 的三轮中位数为 5.186679 ms,相比 joint-skip 的 +5.171330 ms 没有稳定的全局收益;加入 output mask 后,最终三轮中位数为 +5.115620 ms。最终版本的每核 `Materialize+Register` 累计中位数约为 +AIC 1.416 ms、AIV 1.426 ms,基线分别约为 1.453 ms、1.451 ms。 +设备未锁独占且每组只有三轮,因此 mask 的独立收益只能视为方向性证据, +不能按两组中位数差值作严格因果拆分。 + +### 4.3 性能结果和证据文件 + +| 版本 | 三轮首末 Submit(ms) | 中位数(ms) | +| ---- | --------------------- | -----------: | +| atomic baseline | 5.642245(稳定样本) | 5.642245 | +| joint polling skip | 5.171330 / 5.239468 / 5.167902 | 5.171330 | +| + register mask | 5.201349 / 5.128588 / 5.186679 | 5.186679 | +| + output/register masks | 5.115620 / 5.145057 / 5.096685 | **5.115620** | + +最终中位数比基线快 9.33%,最好单轮比基线快 9.67%。最后一个 task 的 +96-worker Submit 起点波宽从基线 463.913 us 降至最终三轮中位数 +71.919 us,表明 worker 长尾明显收敛。1024 个 kernel span 的累计时长 +从基线 32.518 ms 到最终三轮中位数 32.607 ms,未缩短,证明收益来自 +Submit 调度而非 kernel 计算变快。 + +本地证据文件: + +- 基线:`outputs/TestPagedAttentionUnroll_Case1_20260717_023809/merged_swimlane_atomic_load.json`; +- `ld_dev()+nop(100)`:`outputs/TestPagedAttentionUnroll_Case1_20260717_035341/merged_swimlane_nop100.json`; +- `ld_dev()+nop(10)`:`outputs/TestPagedAttentionUnroll_Case1_20260717_035954/merged_swimlane_nop10.json`; +- 最终最好一轮:`outputs/TestPagedAttentionUnroll_Case1_20260717_055638/merged_swimlane_best_joint_poll_skip_arg_masks_5.096685ms.json`; +- 对应原始记录均为相同目录下的 `l2_swimlane_records.json`。 + +`ld_dev()+nop(100)` 和 `ld_dev()+nop(10)` 的完整 Submit 样本分别为 +5.343592 ms 和 5.401034 ms,但它们仅用于定位 atomic load 成本,已回退, +不能作为安全方案。普通 device load 加固定 NOP 不具备 atomic RMW 的同步 +语义,也不能保证 cache 可见性或顺序;延迟长短不能修复协议正确性。 + +## 5. 后续优化顺序与验证要求 + +建议按以下顺序继续,且每次只改一个变量: + +1. **Claim cursor 竞争。** 73728 次 `atomicMax` 是最大固定项。可以研究 + 确定性 owner、分层/分片 Claim 或减少无胜算 worker 参与,但必须保持 + AIC/AIV 负载均衡、ring slot 容量和 joint placement 语义。当前四分片是按 + task id 分片,不能假设再加 shard 一定能降低同一 task 的竞争。 +2. **completion/frontier。** 研究减少重复 `atomicMax(frontier)` 和连续 flag + 扫描,例如单 advancer、批量前推或分层 frontier;最终判断必须保持“只有 + 连续完成的 task 才能释放 heap”的约束。 +3. **fanin ready 轮询。** 统计失败 load 次数,再判断是否能利用已知拓扑、 + 本地完成缓存或更少的 drain 扫描。不能跳过 acquire-ready 条件后直接执行。 +4. **heap guard。** 当前无 RingBp,优先级较低。可以缓存 frontier/vend 做 + advisory fast path,但真正覆盖 ring 前仍需可靠的共享状态确认。 +5. **非 atomic 前端。** 继续观察 tensor map、PrepareMap、参数复制及每核约 + 0.8 ms 的 inter-submit gap;这些不能由替换 atomic load 解决。 + +每个候选改动至少验证: + +- PA Case1:A5Sim 正确性、真实 A5 正确性和 10 轮以上首末 Submit A/B; +- joint mixed(AIC+AIV)和 dual-AIV 的重复 task/slot 复用; +- heap/ring 压力、依赖未就绪、不同 kernel 时长和 worker 到达顺序; +- 96 核均有 1280 个 Submit,task id 完整为 0..1279; +- `DrainWon`、`RingBp`、kernel placement 和最终 drain 数量没有异常漂移; +- 同时报告中位数、离散度、最好/最差值,并记录设备是否独占。 + +泳道统计时,`Build` 和 `Replay` 是 lap marker,会覆盖前面的阶段,不能与 +`Materialize/PrepareMap/Claim/Fanin/Register` 相加。可加总的是显式互斥 +span,或单独定义 `Submit.end - Build/Replay/Alloc.end` 为 action tail。 +后续文档和脚本都应沿用这一口径。 From 3a3de54db0a900e489a5a5496cbee1dc5b76be7a Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Fri, 17 Jul 2026 08:19:22 +0000 Subject: [PATCH 002/214] =?UTF-8?q?Update:=20=E9=87=8D=E5=91=BD=E5=90=8D?= =?UTF-8?q?=20PA=20atomic=20=E6=83=85=E5=86=B5=E5=88=86=E6=9E=90=E6=96=87?= =?UTF-8?q?=E6=A1=A3?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename tests/atomic_probe/README.zh-CN.md => "tests/atomic_probe/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" (100%) diff --git a/tests/atomic_probe/README.zh-CN.md "b/tests/atomic_probe/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" similarity index 100% rename from tests/atomic_probe/README.zh-CN.md rename to "tests/atomic_probe/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" From a3f5ecc2186fb8a06cded6d26886a4ab802009c0 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Fri, 17 Jul 2026 09:47:15 +0000 Subject: [PATCH 003/214] test(a5): isolate AICore submit inline boundary Inline runtime submit helpers into the caller translation unit, add empty-runtime and weak-submit-noinline controls, and harden object, symbol, layout, pytest, and device-matrix validation. --- ...03\347\224\250\351\252\214\350\257\201.md" | 1074 ++++++----------- .../ccec/nested_lambda_cross_tu.cpp | 94 +- .../ccec/nested_lambda_cross_tu_api.h | 114 +- .../ccec/nested_lambda_cross_tu_host.cpp | 68 +- .../ccec/nested_lambda_cross_tu_layout.h | 71 +- .../ccec/nested_lambda_cross_tu_runtime.cpp | 96 +- ...ested_lambda_inline_plus_empty_runtime.cpp | 11 + ...ested_lambda_only_weak_submit_noinline.cpp | 12 + tests/atomic_probe/ccec/run_all.sh | 112 +- .../cpu/nested_lambda_args_runtime_read.cpp | 23 +- tests/atomic_probe/run_nested_lambda.sh | 22 +- tests/atomic_probe/test_atomic_probe.py | 15 +- 12 files changed, 707 insertions(+), 1005 deletions(-) create mode 100644 tests/atomic_probe/ccec/nested_lambda_inline_plus_empty_runtime.cpp create mode 100644 tests/atomic_probe/ccec/nested_lambda_only_weak_submit_noinline.cpp diff --git "a/tests/atomic_probe/AICore\345\265\214\345\245\227Lambda\346\215\225\350\216\267\344\270\216\346\250\241\346\235\277\350\260\203\347\224\250\351\252\214\350\257\201.md" "b/tests/atomic_probe/AICore\345\265\214\345\245\227Lambda\346\215\225\350\216\267\344\270\216\346\250\241\346\235\277\350\260\203\347\224\250\351\252\214\350\257\201.md" index 68adce5af7..824d40c10b 100644 --- "a/tests/atomic_probe/AICore\345\265\214\345\245\227Lambda\346\215\225\350\216\267\344\270\216\346\250\241\346\235\277\350\260\203\347\224\250\351\252\214\350\257\201.md" +++ "b/tests/atomic_probe/AICore\345\265\214\345\245\227Lambda\346\215\225\350\216\267\344\270\216\346\250\241\346\235\277\350\260\203\347\224\250\351\252\214\350\257\201.md" @@ -1,52 +1,66 @@ -# AICore 嵌套 Lambda、捕获与 caller 栈地址验证 +# AICore 嵌套 Lambda、caller 栈地址与 inline 调用边界验证 > 最后验证日期:2026-07-17 > -> 验证仓库:`simpler`,基线 HEAD `52ca4f5eba343c2f7b7a3a743e575cb9308d128f` +> 验证基线 HEAD:`3a3de54db0a900e489a5a5496cbee1dc5b76be7a` > > 测试目录:`tests/atomic_probe` > -> 验证平台:A5 `Ascend950PR_9599`、CANN 9.1.0 +> 验证平台:A5 device 0、CANN 9.1.0 ## 1. 文档目的和最终结论 -本文是一份可以独立阅读和执行的复现手册。读者不需要先了解此前的排查过程, -按本文即可完成以下验证: +本文验证四件事: 1. CPU、AscendC 和纯 CCEC 是否支持嵌套 lambda、捕获和模板调用; -2. 是否能在 A5 上复现 caller 栈地址经过 runtime 调用后使用时的异常; -3. “删除独立 context,把 capture 写入现有 `L0TaskArgs`,由 runtime TU - 直接读取”的数据驱动方案是否可行; -4. CPU 对等实现是否具有相同功能,以及是否存在明显生命周期或未定义行为。 - -本次结果为: - -- CPU、AscendC AIV、纯 CCEC AIV 的嵌套 lambda/捕获/模板基线全部通过; -- 纯 CCEC AIC 的 `weak-context-materialize-0` 在最终测试版本上 5/5 次触发 - `507015 AICore exception`; -- 同一版探针中的 `args-runtime-read` 在 A5 上 11/11 次通过,累计 - 704 轮、2816 次 submit; -- CPU caller 与 AIC 共用同一份 runtime TU 源码,功能验证通过;GCC - `-O0/-O2/-O3` 各 20/20 次通过,ASan+UBSan 通过。 - -必须保留的结论边界:业务用例是 1/2 条无业务地址物化仍失败,第 3 条 -`block_table` 地址物化后通过;当前纯 CCEC 探针是 0 条失败、1/2/3 条通过。 -因此本文已经复现“caller 栈地址传递对 HiIPU 最终 codegen 敏感”的核心现象, -但不是与业务阈值和机器码完全相同的最小复现。 - -当前设备状态也必须单独记录:最后一次负向复核之后,后续正向检查在 -`aclInit(nullptr)` 阶段返回 500000;用户已确认当前 NPU 设备不存在。本文此前 -的 A5 数据是在设备仍存在时取得,现在不能继续上板验证。没有芯片或驱动侧 -故障日志,不能仅凭时间先后断言是本 probe 导致设备物理掉线。 +2. caller 栈上的 `Tensor` 地址经过未内联 submit 调用后,是否会触发 + AICore 异常; +3. 异常是否由“链接两个 `.o`”或 `ld.lld` 跨对象重定位直接导致; +4. runtime submit 全 inline,以及把 capture 写入既有 `L0TaskArgs`, + 是否可作为当前工具链的规避形态。 -## 2. 最短复现路径 +本轮得到的核心矩阵如下: + +| AIC 构建形态 | submit 形态 | 语义变体 | 本轮结果 | +| ------------ | ----------- | -------- | -------- | +| 原始双 `.o` | 外部 submit | m0 | 5/5 507015 | +| 双 `.o`,第二个 `.text=0` | 全 inline | m0 | 5/5 PASS | +| 单 `.o` | 全 inline | m0 | 5/5 PASS | +| 单 `.o` | 仅 weak-context submit noinline | m0 | 5/5 507015 | +| 同一 noinline fixture | 仅 weak-context submit noinline | m1 | 3/3 PASS | +| 同一 noinline fixture | strong 路径保持 inline | strong | 3/3 PASS | + +因此可以排除“只要链接两个 `.o` 就会失败”。单 `.o` 保留 +`nested_probe_submit_weak_context` 未内联时仍然失败,而双 `.o` +在第二个对象没有代码、submit 已完全内联时通过。 + +当前最小、可重复的触发边界是: -如果只想最快确认结果,执行本节即可。后续章节解释原理、源码和每个测试的 -完整判定规则。 +```text +weak-context-materialize-0 ++ nested_probe_submit_weak_context 未内联 ++ caller 栈地址跨该调用边界 +-> 507015 AICore exception +``` -### 2.1 设置路径 +全 inline 后 m0 通过;同一 noinline 产物增加一次无业务地址物化后也通过。 +这仍然是对 HiIPU 最终 codegen 形态的收敛,不是对某个具体后端 pass 或 +某条机器指令的根因定位。 -将前两行替换为本机实际路径: +需要严格保留以下结论边界: + +- 不是 C++ lambda、捕获或模板语言语义不受支持; +- 没有证据表明 `ld.lld` 链接两个对象本身有错误; +- 不能把 weak、noinline 或零地址物化任一单因素写成充分根因; +- all-inline 是已验证规避,不是 CCEC 编译器修复; +- `args-runtime-read` 仍是避免独立 caller context 的数据驱动方案, + 与“submit 全 inline”是两个不同维度。 + +## 2. 最短复现路径 + +以下命令从仓库根目录执行。 + +### 2.1 环境 ```bash export REPO=/path/to/simpler @@ -58,19 +72,18 @@ export PTO_ISA_ROOT="$CANN_ROOT/x86_64-linux" export ATOMIC_PROBE_DEVICE=0 ``` -确认关键文件和工具存在: +确认工具和源码存在: ```bash test -x "$ASCEND_HOME_PATH/bin/ccec" -test -x "$ASCEND_HOME_PATH/bin/bisheng" test -x "$ASCEND_HOME_PATH/bin/ld.lld" test -f "$PTO_ISA_ROOT/include/pto/common/kernel_meta.hpp" test -f tests/atomic_probe/ccec/nested_lambda_cross_tu.cpp -test -f tests/atomic_probe/ccec/nested_lambda_cross_tu_runtime.cpp -test -f tests/atomic_probe/cpu/nested_lambda_args_runtime_read.cpp +test -f tests/atomic_probe/ccec/nested_lambda_cross_tu_api.h +test -f tests/atomic_probe/ccec/nested_lambda_only_weak_submit_noinline.cpp ``` -### 2.2 CPU 对等功能验证 +### 2.2 CPU 语义和 inline runtime-read ```bash tests/atomic_probe/run_nested_lambda.sh cpu @@ -80,837 +93,474 @@ tests/atomic_probe/run_nested_lambda.sh cpu ```text [ASSERT] CPU nested capture/template semantics PASS -[VALUES] rounds=64 mismatches=0 checksum=0x3e6cd1b792bff0e0 L0TaskArgs=1024B +[VALUES] rounds=64 mismatches=0 checksum=0x3e6cd1b792bff0e0 [ASSERT] CPU L0TaskArgs args-runtime-read semantics PASS [SUMMARY] semantic_failures=0 ``` -### 2.3 编译 AIC 双 TU 探针 +CPU 和 AIC 都从 `nested_lambda_cross_tu_api.h` 使用同一份 inline +runtime 实现。CPU 不再链接独立 runtime TU。 + +### 2.3 默认单对象、全 inline 正向组 + +构建: ```bash tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu build ``` -命令必须以 0 退出,并生成: +构建过程会检查: ```text -tests/atomic_probe/ccec/build/nested_lambda_cross_tu_kernel_caller_aic.o -tests/atomic_probe/ccec/build/nested_lambda_cross_tu_kernel_runtime_aic.o -tests/atomic_probe/ccec/build/nested_lambda_cross_tu_kernel.o -tests/atomic_probe/ccec/build/nested_lambda_cross_tu_host +[ASSERT] CCEC caller-capture runtime symbol shape PASS +[VALUES] aic_input_objects=1 runtime_text=n/a submit_symbols=none ``` -### 2.4 运行可落地方案组 +运行默认正向组: ```bash -ATOMIC_PROBE_MODE=args-runtime-read \ - tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run +tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run ``` -关键期望输出和退出码: +固定顺序和期望结果: ```text -[ASSERT] CCEC AIC cross-TU ABI variant=args-runtime-read PASS -[VALUES] rounds=64 mismatches=0 dispatches=0 materializations=3 \ -checksum=0x3e6cd1b792bff0e0 L0TaskArgs=1024B -[SUMMARY] semantic_failures=0 +args-runtime-read PASS +weak-context-materialize-0 PASS +run_failures=0 ``` -退出码必须为 0。 - -### 2.5 运行故障对照组 - -该组会故意触发 AICore 异常。必须使用独立进程,并遵守所在环境的设备独占和 -异常恢复规则。 +也可以单独运行 m0: ```bash -set +e ATOMIC_PROBE_MODE=weak-context-materialize-0 \ tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run -bad_rc=$? -set -e -echo "bad_rc=$bad_rc" ``` -本次受影响编译器上的期望结果: +严格 oracle 为: ```text -ACL error 507015 from aclrtSynchronizeStream(stream) ... -CCEC [nested_lambda_cross_tu_kernel] failed runs: 1 -=== Done. run_failures=1 === -bad_rc=1 -``` - -此处退出码 1 是“故障复现成功”,不是 launcher 构建失败。如果该组返回 0, -说明当前编译器或当前代码布局没有命中此复现边界,不能据此认为方案组失败。 - -## 3. 被验证的问题是什么 - -### 3.1 原始业务现象 - -业务中的 private-lazy 调用会在 orchestration 栈上构造多个 `Tensor`,再把 -这些对象的地址放入一个 caller context,经过 runtime 调用后继续使用: - -```text -orchestration caller - -> runtime submit - -> dispatcher/recipe bind - -> 使用 caller 栈上的 Tensor* -``` - -已有业务证据为: - -- 优化后的 LLVM IR 正确写入三个 `Tensor *` context 字段; -- orchestration CFA 为 `reg93 + 1952`,栈帧只有 1952B; -- 32 KiB 和 64 KiB 栈配置都失败,所以不是普通栈容量不足; -- 不参与业务的地址写会改变 PASS/FAIL; -- 去掉 `-cce-aicore-addr-transform` 后业务仍失败,不能归罪于单一 pass; -- 同 TU、可内联 dispatcher 仍失败,说明跨 TU 外部回调不是根因; -- 业务原发日志包含 code 264:scalar 访问使用了无效 GM 地址;外层最终表现为 - `507018 AICPU exception`。 - -### 3.2 当前最合理的出错原理 - -坏路径可以简化为: - -```text -reg93 栈 - ├─ Tensor first - ├─ Tensor second - ├─ Tensor third - ├─ CallerContext {&first, &second, &third, salt} - └─ L0TaskArgs args - │ - └─ runtime(site_id, &context, &args) - │ - └─ 再解引用 context 中的 Tensor* +rounds=64 +mismatches=0 +dispatches=128 +materializations=0 +checksum=0x3e6cd1b792bff0e0 +L0TaskArgs=1024B +进程退出码=0 ``` -LLVM IR 已经正确,栈容量也足够。可疑区间位于优化后 LLVM IR 到 HiIPU 最终 -机器码之间,包括栈地址物化、指令选择、活跃区间、调度和寄存器分配。 -`&Tensor` 来自 `reg93 + offset`;如果其地址寄存器在调用边界被错误复用、 -覆盖或以错误偏移传递,runtime 后续解引用就会访问无效地址。 - -无业务用途的 `ptrtoint/store` 会改变地址的活跃区间和寄存器选择,所以即使 -LLVM 业务语义完全不变,最终机器码也可能从失败形态切换到通过形态。这是 -Heisenbug 诊断特征,不是合法修复方式。 - -目前尚未证明是某一个具体 CCEC pass。本文只把问题边界收敛到 HiIPU 后端的 -地址物化/调度/寄存器分配组合,不声称已经完成编译器根因定位。 +### 2.4 双对象数量控制 -### 3.3 数据驱动方案为什么可能绕开问题 +该 fixture 仍向 `ld.lld` 传入两个 AIC 对象,但第二个对象的 +`.text` 大小必须为零,且所有 submit 已进入 caller 对象并被内联。 -方案组不再构造独立 `CallerContext`: +```bash +tests/atomic_probe/ccec/run_all.sh \ + nested_lambda_inline_plus_empty_runtime build -```text -caller 栈上的 Tensor 地址 - -> 写入既有 L0TaskArgs 固定 slot - -> 跨调用只传稳定使用的 args_ptr - -> runtime TU 直接读取 slot - -> submit 返回前 add_input 并完成物化 +tests/atomic_probe/ccec/run_all.sh \ + nested_lambda_inline_plus_empty_runtime run ``` -它减少了一条独立 caller 栈指针传参链,并迫使三个地址在调用前形成明确的 -内存表示。它是对易错 codegen 形态的规避,不是对 CCEC 后端的修复。 - -## 4. 测试分层与调用结构 - -### 4.1 第一层:嵌套 lambda/捕获/模板语言基线 - -共享测试形态为: +构建期望: ```text -自由函数模板 Submit(outer_lambda) - -> outer lambda 按引用捕获 caller 状态 - -> SubmitBuilder::AddInput(inner_lambda) - -> SubmitBuilder::AddOutput(inner_lambda) - -> SubmitBuilder::AddScalar(inner_lambda) +[VALUES] aic_input_objects=2 runtime_text=000000 submit_symbols=none ``` -具体覆盖: - -- outer lambda:`[&]`; -- input lambda:`[&]`,修改引用捕获; -- output lambda:`[outer_local, &output_calls]`,混合按值/按引用捕获; -- scalar lambda:按值和按引用混合捕获; -- 自由函数模板 `Submit`; -- 成员函数模板 `AddInput/AddOutput/AddScalar`; -- AICore 版本的 lambda 显式标注 `__aicore__`。 - -这一层分别由 CPU、AscendC AIV 和纯 CCEC AIV 实现。它只能回答“语言和 -单 TU 语义是否支持”,不能单独复现 caller 栈地址问题。 +运行期望:m0 PASS、退出码 0。该控制证明两个链接输入对象本身不足以触发 +异常。 -### 4.2 第二层:caller capture 传输 A/B +### 2.5 only-weak-submit-noinline 故障控制 -第二层直接复用仓库中的真实类型: +该 fixture 仍只编译一个有代码的 AIC 对象,但仅保留 +`nested_probe_submit_weak_context` 为 noinline。其他 runtime submit、 +digest、consume 和 control 均保持 `always_inline`。 -```text -L0TaskArgs -Tensor -TaskOutputTensors +```bash +tests/atomic_probe/ccec/run_all.sh \ + nested_lambda_only_weak_submit_noinline build ``` -没有另造简化容器,也没有链接完整 Simpler runtime。探针只复用类型定义, -用单独的 `nested_lambda_cross_tu_runtime.cpp` 模拟待测 runtime 调用边界。 -ACL 只负责加载 raw AICore ELF、启动 kernel 和回读结果,不参与被测 ABI。 - -每个变体执行: +构建期望: ```text -64 轮 - 每轮在 caller 栈上新建 3 个 Tensor - 复用同一个 L0TaskArgs - 第 1 次 submit:待测 lazy/capture 路径 - 第 2~4 次 submit:控制路径 +[VALUES] aic_input_objects=1 runtime_text=n/a +submit_symbols=*nested_probe_submit_weak_context* ``` -这保留了“同一容器四次 submit”和 caller 栈对象反复创建/销毁的关键形态。 - -### 4.3 `L0TaskArgs` recipe slot 布局 - -测试专用布局为: - -| slot | 内容 | 读取者 | -| ---: | --- | --- | -| scalar 8 | `&first` | runtime 或 dispatcher | -| scalar 9 | `&second` | runtime 或 dispatcher | -| scalar 10 | `&third` | runtime 或 dispatcher | -| scalar 11 | `salt` | runtime 或 dispatcher | -| scalar 0 | runtime 计算结果 | caller oracle | -| scalar 5 | dispatcher 次数 | caller oracle | -| scalar 6 | dispatcher 缺失标志 | caller oracle | - -8~11 是本探针的 recipe/诊断 slot,不属于生产 ABI。真实落地必须为 recipe -定义正式布局、容量和版本,不能直接把这些编号当成业务规范。 - -### 4.4 七个 AIC 变体 - -所有变体都是独立 global AIC kernel,地址物化数量在编译期固定,不是运行时 -分支。数字 entry 必须唯一。 - -| entry | 运行参数 | 独立 context | capture 读取位置 | dispatcher | 地址写 | 本次 A5 | -| ---: | --- | --- | --- | --- | ---: | --- | -| 0 | `weak-context-materialize-0` | 有 | caller dispatcher | weak | 0 | 507015 | -| 1 | `weak-context-materialize-1` | 有 | caller dispatcher | weak | 1 | PASS | -| 2 | `weak-context-materialize-2` | 有 | caller dispatcher | weak | 2 | PASS | -| 3 | `weak-context-materialize-3` | 有 | caller dispatcher | weak | 3 | PASS | -| 4 | `weak-args-storage` | 无 | caller dispatcher | weak | 3 个实际 capture | PASS | -| 5 | `strong-context` | 有 | caller dispatcher | strong | 0 | PASS | -| 6 | `args-runtime-read` | 无 | runtime TU 直接读取 | 无回调 | 3 个实际 capture | PASS | - -四个 `materialize-N` 组中的 scalar 8~10 不参与业务读取,只用于改变地址物化。 -`weak-args-storage` 中这些 slot 是实际 capture,但 runtime 仍回调 weak -dispatcher。`args-runtime-read` 才是最终待验证方案:没有第二个 context, -也没有反向回调。 - -### 4.5 精确 oracle - -语言基线固定 `seed=0x120`: - -| 字段 | 期望值 | -| --- | ---: | -| outer/input/output/scalar 调用次数 | 各 1 | -| `reference_state` | 300 | -| `input.value` | 591 | -| `output.value` | 323 | -| `scalar` | 337 | -| `combined` | 1251 | - -caller capture 组要求: - -- `completed_rounds == 64`; -- `mismatches == 0`; -- 64×4 次 submit 的 checksum 为 `0x3e6cd1b792bff0e0`; -- 回调组 dispatcher 次数为 128;`args-runtime-read` 为 0; -- 地址写数量和 variant echo 与 entry 完全一致; -- `sizeof(L0TaskArgs) == 1024`,小于 32 KiB 且按 64B 对齐。 - -## 5. 源码清单和复用关系 - -所有测试代码位于 `simpler/tests/atomic_probe`。 - -### 5.1 语言基线 - -| 文件 | 作用 | -| --- | --- | -| `nested_lambda_probe.h` | 三端共用的 builder、模板、字段布局和 oracle | -| `cpu/nested_lambda.cpp` | 标准 C++17 语言对照 | -| `ascendc/nested_lambda.asc` | AscendC AIV kernel 和 ACL host | -| `ccec/nested_lambda.cpp` | 纯 CCEC AIV kernel,无 AscendC API | -| `ccec/nested_lambda_host.cpp` | 纯 CCEC raw ELF launcher 和校验 | +先运行通过控制: -### 5.2 caller capture 与 CPU 对等用例 - -| 文件 | 作用 | -| --- | --- | -| `ccec/nested_lambda_cross_tu.cpp` | AIC caller、dispatcher、七个 kernel entry | -| `ccec/nested_lambda_cross_tu_runtime.cpp` | 独立 runtime TU;含回调组和直接读取组 | -| `ccec/nested_lambda_cross_tu_api.h` | caller/runtime 共用函数签名 | -| `ccec/nested_lambda_cross_tu_layout.h` | 变体、entry、结果字段和 checksum oracle | -| `ccec/nested_lambda_cross_tu_host.cpp` | 按 entry 启动 raw ELF 并回读校验 | -| `cpu/nested_lambda_args_runtime_read.cpp` | CPU caller 对等实现 | -| `run_nested_lambda.sh` | CPU、AscendC、CCEC 统一入口 | -| `ccec/run_all.sh` | CCEC 编译、链接和独立进程矩阵 runner | -| `test_atomic_probe.py` | CPU 和 A5 pytest 入口 | - -CPU 用例没有复制 runtime 实现。g++ 把 -`cpu/nested_lambda_args_runtime_read.cpp` 与同一份 -`ccec/nested_lambda_cross_tu_runtime.cpp` 作为两个 TU 编译,再通过 -`--gc-sections` 裁掉 CPU 未调用的 dispatcher 控制函数。CPU 和 AIC 实际调用 -的是同一个 `nested_probe_submit_args_runtime_read()` 源码函数。 - -## 6. 环境与构建前置条件 - -### 6.1 CPU - -最低要求: +```bash +ATOMIC_PROBE_MODE=strong-context \ + tests/atomic_probe/ccec/run_all.sh \ + nested_lambda_only_weak_submit_noinline run -```text -bash -git -g++,支持 C++17 +ATOMIC_PROBE_MODE=weak-context-materialize-1 \ + tests/atomic_probe/ccec/run_all.sh \ + nested_lambda_only_weak_submit_noinline run ``` -CPU 功能测试不需要 CANN runtime,也不访问设备,但会包含仓库现有的 -`L0TaskArgs/Tensor` 头文件。 - -### 6.2 A5/CANN - -要求: - -```text -CANN 9.1.0 -ccec -bisheng -ld.lld -g++ -libascendcl.so -PTO kernel_meta.hpp -至少 1 个可用 A5 device -``` +两条命令都必须退出 0。 -推荐检查: +最后运行故障控制: ```bash -echo "ASCEND_HOME_PATH=$ASCEND_HOME_PATH" -"$ASCEND_HOME_PATH/bin/ccec" --version | head -n 3 -"$ASCEND_HOME_PATH/bin/bisheng" --version | head -n 1 -g++ --version | head -n 1 -test -f "$ASCEND_HOME_PATH/x86_64-linux/lib64/libascendcl.so" -test -f "$PTO_ISA_ROOT/include/pto/common/kernel_meta.hpp" +set +e +ATOMIC_PROBE_MODE=weak-context-materialize-0 \ + tests/atomic_probe/ccec/run_all.sh \ + nested_lambda_only_weak_submit_noinline run +bad_rc=$? +set -e +echo "bad_rc=$bad_rc" ``` -### 6.3 负向用例执行约束 - -故障组会触发 AICore exception,必须一次只运行一个变体,并在独立 host 进程 -中启动。host 强制要求 ` ` 两个参数;runner 也为每个变体 -创建独立进程,避免一个异常污染其他控制组的判定。 - -早期两组交替实验中,故障进程退出后方案组可以立即再次通过;但最后一次 -负向复核后,稍后的正向检查在 `aclInit(nullptr)` 就返回 500000,用户随后 -确认 NPU 设备已经不存在。因此不能把“进程退出即可恢复”写成通用结论;没有 -故障恢复条件时不要执行负向组。当前环境不再执行任何硬件命令。 - -### 6.4 本探针与 32K/64K 配置的关系 - -本探针不是 `aclInit` 栈配置测试:host 使用 `aclInit(nullptr)`,不读取额外 -ACL JSON。CCEC runner 的 AIC 编译参数包含: +受影响 CCEC 上的期望结果: ```text --O3 -g -x cce -std=c++17 --cce-aicore-only --mllvm -cce-aicore-stack-size=0x8000 --mllvm -cce-aicore-function-stack-size=0x8000 --mllvm -cce-aicore-record-overflow=false --mllvm -cce-aicore-addr-transform --mllvm -cce-aicore-dcci-insert-for-scalar=false --mllvm -cce-aicore-dcci-before-kernel-end=false ---cce-aicore-arch=dav-c310-cube +ACL error 507015 from aclrtSynchronizeStream(stream) ... +CCEC [...] failed runs: 1 +bad_rc=1 ``` -`0x8000` 是 32 KiB 上限,但实际 orchestration CFA 只有 1952B。业务侧已经 -独立验证 32K/64K 都失败,因此本文不再通过增大栈来解释或掩盖问题。 - -## 7. 逐项测试用例 - -以下命令除特别说明外,都从 `simpler` 仓库根目录执行。 +这里的退出码 1 表示故障控制命中,不是构建失败。该组必须在独立 host +进程中运行,并放在所有通过控制之后。 -### TC-CPU-01:CPU 语言基线与 runtime-read 对等实现 +## 3. 被验证的问题 -目的: +### 3.1 原始调用形态 -- 验证标准 C++17 的嵌套 lambda/捕获/模板语义; -- 验证真实 `L0TaskArgs/Tensor/TaskOutputTensors`; -- 用与 AIC 相同的 runtime TU 验证数据驱动 recipe 功能。 +业务中的 private-lazy 路径会在 orchestration 栈上构造多个 `Tensor`, +再把对象地址写入 caller context: -命令: - -```bash -tests/atomic_probe/run_nested_lambda.sh cpu +```text +orchestration caller + -> caller 栈上的 Tensor first/second/third + -> CallerContext {&first, &second, &third, salt} + -> submit(context, args) + -> dispatcher 解引用 context 中的 Tensor* ``` -判定:两个 `[ASSERT]` 均为 PASS,两个 summary 均为 0;runtime-read 的 -rounds、mismatch、checksum 和 `L0TaskArgs` 大小必须与第 2.2 节一致。 +已有证据包括: -pytest: +- 优化后的 LLVM IR 正确写入三个 `Tensor *`; +- orchestration CFA 为 `reg93 + 1952`,不是普通栈容量不足; +- 32 KiB 和 64 KiB 栈配置都失败; +- 无业务用途的地址写会改变 PASS/FAIL; +- 去掉 `-cce-aicore-addr-transform` 后业务仍失败; +- 业务原发日志包含无效 GM 地址访问。 -```bash -export PYTHONPATH=python -.venv/bin/python -m pytest \ - tests/atomic_probe/test_atomic_probe.py::test_cpu_nested_lambda_compiler_probe \ - -q -s -``` - -本次结果:`1 passed in 1.73s`。 +### 3.2 本轮新增的判别证据 -### TC-CPU-02:CPU 优化级别和 sanitizer +本轮把“对象数量”和“函数调用边界”拆开: -目的:排除该 CPU 实现仅在某个优化级别偶然通过,以及明显的越界、 -use-after-scope 或未定义行为。 - -下面的命令与本次实测一致: +```text +两个对象 + 空 runtime + submit 全 inline + -> PASS -```bash -INCLUDES=( - -Itests/atomic_probe/ccec - -Isrc/a5/platform/onboard/aicore - -Isrc/a5/platform/include - -Isrc/common/platform/include - -Isrc/common/task_interface - -Isrc/common/log/include - -Isrc/common - -Isrc/a5/runtime/fully_distributed_within_core/runtime - -Isrc/a5/runtime/fully_distributed_within_core/common - -Isrc/a5/runtime/fully_distributed_within_core/orchestration - -Isrc/a5/runtime -) -SOURCES=( - tests/atomic_probe/cpu/nested_lambda_args_runtime_read.cpp - tests/atomic_probe/ccec/nested_lambda_cross_tu_runtime.cpp -) - -for opt in 0 2 3; do - out="/tmp/cpu_args_runtime_read_O${opt}" - g++ "-O${opt}" -std=c++17 -Wall -Wextra -Werror -ffunction-sections \ - "${INCLUDES[@]}" "${SOURCES[@]}" -Wl,--gc-sections -o "$out" - for i in $(seq 1 20); do "$out" >/dev/null; done - echo "g++ -O${opt}: 20/20 PASS" -done - -out=/tmp/cpu_args_runtime_read_sanitize -g++ -O2 -std=c++17 -Wall -Wextra -Werror -ffunction-sections \ - -fsanitize=address,undefined -fno-omit-frame-pointer \ - "${INCLUDES[@]}" "${SOURCES[@]}" -Wl,--gc-sections -o "$out" -ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=halt_on_error=1 "$out" +一个对象 + weak-context submit noinline + -> 507015 ``` -本次 GCC 13.3.0 结果:O0/O2/O3 各 20/20 PASS;ASan+UBSan PASS。本机没有 -`clang++`,所以未完成第二种 CPU 后端交叉验证。 +因此多对象链接不是必要条件,未内联 submit 才是当前最小 fixture 中的 +必要构建形态。它仍不是单独的充分解释,因为: -### TC-ASCENDC-01:AscendC AIV 语言基线 +- 同一 noinline fixture 的 m1 通过; +- 同一 fixture 的 strong 路径通过; +- 全 inline 的 weak m0 通过。 -目的:验证 `bisheng -xasc` 对设备 lambda、捕获和模板调用的支持。 +最合理的范围仍是优化 LLVM IR 到 HiIPU 最终机器码之间的地址物化、 +活跃区间、调度和寄存器分配组合。 -编译并上板: +当前安装的 `llvm-objdump` 能读取 `elf64-hiipu` 符号和 DWARF,但指令 +反汇编显示 ``。没有可用 fault PC,因此本文不声称已经定位 +具体机器指令。 -```bash -ATOMIC_PROBE_DEVICE=0 tests/atomic_probe/run_nested_lambda.sh ascendc -``` +### 3.3 两种规避形态 -仅编译、不访问设备: +全 inline 形态: -```bash -bisheng -xasc tests/atomic_probe/ascendc/nested_lambda.asc \ - --npu-arch=dav-3510 \ - -o /tmp/nested_lambda_ascendc +```text +caller 栈地址 + -> inline submit/helper + -> 不跨 nested_probe_submit_weak_context 函数边界 + -> dispatcher/consume ``` -上板期望: +数据驱动形态: ```text -[ASSERT] AscendC nested capture/template semantics PASS -[ASSERT] AscendC ACL cleanup PASS -[SUMMARY] semantic_failures=0 +caller 栈地址 + -> 写入既有 L0TaskArgs 固定 slot + -> inline args-runtime-read 直接读取 + -> submit 返回前完成 add_input 和结果物化 ``` -### TC-CCEC-AIV-01:纯 CCEC AIV 语言基线 +两者都在当前工具链上通过。生产实现仍必须保证 submit 返回前完成复制或 +物化,不能让异步阶段继续保存 caller 栈裸地址。 -目的:不包含 AscendC header,只用 CCEC 和 lowercase builtin 验证同一语言 -形态。 +## 4. 测试结构和 oracle -编译并上板: +### 4.1 语言语义层 -```bash -ATOMIC_PROBE_DEVICE=0 \ - tests/atomic_probe/ccec/run_all.sh nested_lambda -``` +CPU、AscendC AIV 和纯 CCEC AIV 共同覆盖: -拆分执行: +- outer lambda 按引用捕获; +- nested lambda 按值、按引用和混合捕获; +- 自由函数模板 `Submit`; +- 成员函数模板 `AddInput/AddOutput/AddScalar`; +- AICore lambda 的 `__aicore__` 标注。 -```bash -tests/atomic_probe/ccec/run_all.sh nested_lambda build -ATOMIC_PROBE_DEVICE=0 \ - tests/atomic_probe/ccec/run_all.sh nested_lambda run -``` +固定 `seed=0x120` 的期望值为: -期望: +| 字段 | 期望值 | +| ---- | -----: | +| outer/input/output/scalar 调用次数 | 各 1 | +| `reference_state` | 300 | +| `input.value` | 591 | +| `output.value` | 323 | +| `scalar` | 337 | +| `combined` | 1251 | -```text -[ASSERT] CCEC nested capture/template semantics PASS -[ASSERT] CCEC ACL cleanup PASS -[SUMMARY] semantic_failures=0 -``` +### 4.2 caller-capture 语义层 -### TC-CCEC-AIC-01:双 TU build-only 和 ELF 结构 +每个变体执行 64 轮,每轮执行 4 次 submit: -目的:确认 caller/runtime 分开编译、weak/strong 符号和七个入口都真实保留。 +- 第一次为待测 caller-capture 或 args-runtime-read 路径; +- 后三次为固定 control 路径; +- 每轮重新创建三个 caller 栈 `Tensor`; +- 同一轮复用一个 `L0TaskArgs`。 -构建: +精确 oracle: -```bash -tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu build -``` +| 字段 | 期望 | +| ---- | ---: | +| completed rounds | 64 | +| mismatches | 0 | +| submits | 256 | +| checksum | `0x3e6cd1b792bff0e0` | +| `sizeof(L0TaskArgs)` | 1024B | +| callback dispatcher 次数 | 128 | +| args-runtime-read dispatcher 次数 | 0 | -静态核对: +`L0TaskArgs` 的测试 slot: -```bash -BUILD=tests/atomic_probe/ccec/build +| slot | 内容 | +| ---: | ---- | +| scalar 8 | `&first` | +| scalar 9 | `&second` | +| scalar 10 | `&third` | +| scalar 11 | `salt` | +| scalar 0 | 计算结果 | +| scalar 5 | dispatcher 次数 | +| scalar 6 | dispatcher 缺失标志 | -readelf -S -W "$BUILD/nested_lambda_cross_tu_kernel.o" \ - | rg '\.ascend\.meta\.' +这些编号仅属于探针,不是生产 ABI。 -readelf -Ws -W "$BUILD/nested_lambda_cross_tu_kernel_runtime_aic.o" \ - | rg 'nested_probe_(weak_.*dispatch|strong_context_dispatch)' +### 4.3 七个语义变体 -readelf -Ws -W "$BUILD/nested_lambda_cross_tu_kernel.o" \ - | rg 'nested_probe_orchestration|nested_lambda_cross_tu_.*mix_aic' +| entry | 运行参数 | 独立 context | 地址物化 | dispatcher | +| ----: | -------- | ------------ | -------: | ---------- | +| 0 | `weak-context-materialize-0` | 有 | 0 | weak | +| 1 | `weak-context-materialize-1` | 有 | 1 | weak | +| 2 | `weak-context-materialize-2` | 有 | 2 | weak | +| 3 | `weak-context-materialize-3` | 有 | 3 | weak | +| 4 | `weak-args-storage` | 无 | 3 | weak | +| 5 | `strong-context` | 有 | 0 | strong | +| 6 | `args-runtime-read` | 无 | 3 | 无回调 | -"$ASCEND_HOME_PATH/bin/llvm-objdump" --dwarf=frames \ - "$BUILD/nested_lambda_cross_tu_kernel.o" \ - | rg 'reg93 \+1952' -``` +变体本身不再绑定固定 PASS/FAIL。结果必须同时写明构建 fixture。尤其 m0 +在 all-inline fixture 中通过,在 only-weak-submit-noinline fixture 中失败。 -期望: +### 4.4 三个持久构建 fixture -- 七个 `.ascend.meta.*` section; -- runtime object 中两个 dispatcher 是 `WEAK UND`,strong dispatcher 是 - `GLOBAL UND`; -- context orchestration text 为 1432/1436/1440/1444B; -- `args-runtime-read` orchestration text 为 1484B; -- 七个 orchestration 的 CFA 均为 `reg93 + 1952`; -- 七个 global wrapper 均为 128B。 +| target | AIC 输入对象 | submit FUNC 符号 | 默认运行 | +| ------ | -----------: | ---------------- | -------- | +| `nested_lambda_cross_tu` | 1 | 0 | args-runtime-read、m0 | +| `nested_lambda_inline_plus_empty_runtime` | 2 | 0 | m0 | +| `nested_lambda_only_weak_submit_noinline` | 1 | 仅 weak-context submit | strong、m1、m0 | -### TC-CCEC-AIC-02:数据驱动方案正向验证 +第三个 target 的 m0 放在最后;预期 runner 退出 1。默认 pytest 不运行该 +故障组。 -目的:验证没有独立 context、没有反向 dispatcher 的 runtime-read 路径。 +## 5. 源码和测试入口 -```bash -ATOMIC_PROBE_MODE=args-runtime-read \ -ATOMIC_PROBE_DEVICE=0 \ - tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run -``` +| 文件 | 作用 | +| ---- | ---- | +| `nested_lambda_probe.h` | 三端共享的语言语义和 oracle | +| `cpu/nested_lambda.cpp` | 标准 C++17 语言对照 | +| `cpu/nested_lambda_args_runtime_read.cpp` | CPU inline runtime-read 对照 | +| `ccec/nested_lambda_cross_tu.cpp` | AIC caller、dispatcher 和七个入口 | +| `ccec/nested_lambda_cross_tu_api.h` | inline runtime 实现和 caller context | +| `ccec/nested_lambda_cross_tu_runtime.cpp` | `.text=0` 的第二对象控制 | +| `ccec/nested_lambda_inline_plus_empty_runtime.cpp` | 双对象数量控制 wrapper | +| `ccec/nested_lambda_only_weak_submit_noinline.cpp` | only-weak-submit-noinline wrapper | +| `ccec/nested_lambda_cross_tu_layout.h` | 变体、字段和精确 oracle | +| `ccec/nested_lambda_cross_tu_host.cpp` | raw ELF launcher 和结果校验 | +| `ccec/run_all.sh` | 三种 fixture 的构建、ELF 断言和运行 | +| `run_nested_lambda.sh` | CPU、AscendC、CCEC 统一入口 | +| `test_atomic_probe.py` | CPU 和默认 A5 正向 pytest | -严格判定: +文件名中的 `cross_tu` 是历史命名。当前默认 target 是单个有代码的 AIC +输入对象,不能再根据文件名推断构建形态。 -```text -rounds=64 -mismatches=0 -dispatches=0 -materializations=3 -checksum=0x3e6cd1b792bff0e0 -L0TaskArgs=1024B -semantic_failures=0 -进程退出码=0 -``` +## 6. 自动化测试 -pytest 会重新 build 再运行方案组: +### 6.1 CPU pytest ```bash export PYTHONPATH=python -ATOMIC_PROBE_DEVICE=0 .venv/bin/python -m pytest \ - tests/atomic_probe/test_atomic_probe.py::test_a5_ccec_nested_lambda_args_runtime_read \ +.venv/bin/python -m pytest \ + tests/atomic_probe/test_atomic_probe.py::test_cpu_nested_lambda_compiler_probe \ -q -s ``` -### TC-CCEC-AIC-03:独立 context 故障复现 - -目的:证明同一编译器和同一测试版本仍能触发 caller-context 异常,避免因为 -所有变体都通过而错误宣称方案有效。 +### 6.2 A5 默认正向 pytest ```bash -set +e -ATOMIC_PROBE_MODE=weak-context-materialize-0 \ -ATOMIC_PROBE_DEVICE=0 \ - tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run -rc=$? -set -e -echo "rc=$rc" -``` - -本机期望:`aclrtSynchronizeStream` 返回 507015,runner 和最终命令退出码都为 -1。由于异常发生在同步阶段,host 的普通成功清理路径不会执行;必须依赖独立 -进程和平台认可的异常恢复流程。 - -最新一次文档复核确实再次得到 507015;之后的正向恢复检查在 -`aclInit(nullptr)` 返回 500000,当前 NPU 已不再存在。只有具备平台恢复能力 -时才应继续执行本负向用例。 - -### TC-CCEC-AIC-04:完整七变体矩阵 - -runner 会把每个变体放在独立进程中,并把故障组放到最后,防止一个 AICore -exception 遮住其他控制组。 - -```bash -unset ATOMIC_PROBE_MODE -set +e -ATOMIC_PROBE_DEVICE=0 \ - tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run -matrix_rc=$? -set -e -echo "matrix_rc=$matrix_rc" -``` - -固定运行顺序: - -```text -strong-context PASS -args-runtime-read PASS -weak-args-storage PASS -weak-context-materialize-3 PASS -weak-context-materialize-2 PASS -weak-context-materialize-1 PASS -weak-context-materialize-0 507015 +export PYTHONPATH=python +.venv/bin/python -m pytest \ + tests/atomic_probe/test_atomic_probe.py::test_a5_ccec_nested_lambda_call_boundary_controls \ + --platform a5 --device 0 \ + -q -s ``` -本机 `matrix_rc=1` 是期望结果。该显式诊断 probe 不在 `ccec/run_all.sh` 默认 -cache-line suite 中。 +该 pytest 会: -### TC-CCEC-AIC-05:方案组稳定性 +1. build 单对象 all-inline target,并验证 submit FUNC 符号为零; +2. 先运行 args-runtime-read,要求 PASS; +3. 再运行 m0,要求 PASS。 -先完成 build-only,再用独立进程重复运行: +它不会运行 only-weak-submit-noinline m0,避免默认 CI 故意制造 AICore +exception。双对象数量控制和 noinline 故障控制使用第 2.4、2.5 节的显式 +命令。 -```bash -for i in $(seq 1 10); do - echo "=== args-runtime-read iteration=$i/10 ===" - ATOMIC_PROBE_MODE=args-runtime-read \ - ATOMIC_PROBE_DEVICE=0 \ - tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run -done -``` - -要求 10/10 进程退出码为 0,且每次 checksum 完全一致。正式稳定性批次为 -10/10;文档复核又成功运行 1 次,因此累计记录为 11/11、704 轮、2816 次 -submit。 - -可选的交替 A/B: +### 6.3 完整语言入口 ```bash -for cycle in 1 2; do - set +e - ATOMIC_PROBE_MODE=weak-context-materialize-0 \ - ATOMIC_PROBE_DEVICE=0 \ - tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run - bad_rc=$? - set -e - - if [ "$bad_rc" -eq 0 ]; then - echo "bad control unexpectedly passed" >&2 - exit 1 - fi - - # 若所在平台要求 reset,请在这里执行平台批准的恢复流程。 - ATOMIC_PROBE_MODE=args-runtime-read \ - ATOMIC_PROBE_DEVICE=0 \ - tests/atomic_probe/ccec/run_all.sh nested_lambda_cross_tu run -done +tests/atomic_probe/run_nested_lambda.sh all ``` -早期两组交替均为“context 507015,随后方案组立即 PASS”。这只证明当时两次 -可以恢复,不覆盖最后一次设备消失的状态。 +`all` 依次运行 CPU、AscendC、纯 CCEC AIV 和默认 AIC caller-capture +正向组。 -## 8. 实测环境和结果记录 +## 7. 本轮实测结果 -### 8.1 软件和设备 +软件环境: ```text -Repo HEAD: 52ca4f5eba343c2f7b7a3a743e575cb9308d128f -Device: Ascend950PR_9599 -Short SoC: Ascend950 -CANN: 9.1.0 -ccec: clang 15.0.5, build 2026-06-10T11:29:46+08:00 +Repo HEAD: 3a3de54db0a900e489a5a5496cbee1dc5b76be7a +CCEC: clang 15.0.5, build 2026-07-07T20:35:46+08:00 GCC: 13.3.0 -AscendC arch: dav-3510 -CCEC AIV arch: dav-c310-vec -CCEC AIC arch: dav-c310-cube +AIC arch: dav-c310-cube +Device: A5 device 0 ``` -### 8.2 A5 矩阵 +持久 fixture 结果: -| 变体 | 次数 | 结果 | rounds | mismatches | dispatcher | checksum | -| --- | ---: | --- | ---: | ---: | ---: | --- | -| `args-runtime-read` | 11 | 11/11 PASS | 64/次 | 0 | 0 | 固定正确 | -| `weak-context-materialize-0` | 5 | 5/5 507015 | 未回读 | 未回读 | 未回读 | 未回读 | -| 其他五个控制组 | 各至少 1 | PASS | 64 | 0 | 128 | 固定正确 | +| fixture / variant | 次数 | 结果 | +| ----------------- | ---: | ---- | +| 单对象 all-inline / m0 | 5 | 5/5 PASS | +| 双对象、空 runtime / m0 | 5 | 5/5 PASS | +| only-weak-submit-noinline / m1 | 3 | 3/3 PASS | +| only-weak-submit-noinline / strong | 3 | 3/3 PASS | +| only-weak-submit-noinline / m0 | 5 | 5/5 507015 | +| 单对象 all-inline / args-runtime-read | 1 | PASS | +| CPU 语言语义 | 1 | PASS | +| CPU inline args-runtime-read | 1 | PASS | -故障组在 `aclrtSynchronizeStream` 就返回,因此不能伪造 rounds/mismatch 等设备 -结果;表中明确写“未回读”。 +此外,本轮从当前 HEAD 临时恢复原始 external runtime 源码,重新生成双 +`.o` 产物;m0 为 5/5 507015。该临时产物只用于确认历史基线,不是当前 +runner 的持久 target。 -最新状态:第 5 次负向命中后,后续 `args-runtime-read` 没有进入 kernel, -而是在 `aclInit(nullptr)` 返回 500000。用户确认当前 NPU 设备已经不存在。 -因此 A5 数据到此冻结,后续仅做离线审查。 +故障组在 `aclrtSynchronizeStream` 返回 507015,不能回读 rounds、 +mismatch 或 checksum。表中不为故障组伪造设备结果。 -### 8.3 最终 build 产物哈希 +## 8. 如何解释结果 -哈希用于确认当前机器上的复现产物,不应假设不同绝对源码路径或不同工具版本 -一定生成同一哈希。 +| 观察 | 可以支持 | 不能推出 | +| ---- | -------- | -------- | +| 双对象空 runtime 通过 | 两个链接输入不足以触发 | `ld.lld` 已被全面证明无缺陷 | +| 单对象 noinline m0 失败 | 多对象链接不是必要条件 | 任意 noinline 调用都会失败 | +| all-inline m0 通过 | inline 可规避当前形态 | inline 修复了 CCEC 后端 | +| noinline m1 通过 | 地址物化会改变最终 codegen | 增加一次 store 是生产修复 | +| 同 fixture strong 通过 | strong 路径是有效控制 | weak 单独就是根因 | +| args-runtime-read 通过 | 数据驱动方案机制可行 | 真实 PA 业务已经完成修复 | -```text -caller AIC object: -2412dd11bf0d8b07d8c9f9fd08179cc4821e95335616f314a2f55b4b710f52b0 +507015 在 CANN 中表示 AICore exception。业务经过 AICPU 外层时可能报告 +507018;错误层级与调用路径不同,不能只凭错误码断言 fault PC 相同。 -runtime AIC object: -ec365301d026dbefec93d562102f7b85b44c72aa8ade2cea4956eb6e3f09470e +## 9. 生产落地约束 -linked raw AICore ELF: -74167ad081799b93daa29a7b974746339cc4a3cd488dd2463514097696fb900b +1. submit 返回前必须复制或物化 caller 栈数据; +2. 不得让异步阶段保存 caller 栈裸地址; +3. all-inline 必须检查最终 ELF 不含 `nested_probe_submit_*` FUNC 符号; +4. inline 会改变代码体积,真实业务必须检查指令空间和性能; +5. recipe slot 必须定义正式布局、容量、版本和边界检查; +6. 真实 PA 必须保留 eager、原 lazy 和候选方案三组 A/B; +7. 无业务地址写只能用于诊断,不能作为正式修复; +8. only-weak-submit-noinline m0 只属于显式诊断,不进入默认 CI。 -host launcher: -未记录。host 入口已收紧为必须显式指定单个变体,修改后未重新执行 build-only。 -``` - -## 9. 如何解释结果 +## 10. 常见问题 -| 观察 | 可以支持的结论 | 不能推出的结论 | -| --- | --- | --- | -| CPU/AscendC/CCEC AIV 全通过 | 前端和单 TU 语言语义支持 | 真实跨调用 ABI 一定正确 | -| context 组 507015、方案组通过 | 数据驱动方案避开当前易错形态 | 已定位某个具体 CCEC pass | -| 地址写数量改变 PASS/FAIL | 最终 codegen 对地址活跃区间敏感 | “加一条/三条 store”是合法修复 | -| strong 通过、weak 失败 | weak 会影响当前最小代码布局 | weak 是业务根因;同 TU 实验已反证 | -| 所有七组都通过 | 当前工具链/布局未命中 probe | 原业务问题不存在 | -| `args-runtime-read` 失败 | 当前方案在该工具链下不可用或测试有回归 | 一定与原业务是同一 fault PC | +### 10.1 build 目录中混入旧产物 -### 9.1 507015 与 507018 +每个 fixture 使用独立 kernel、caller object 和 host 文件名。不要拿 +`nested_lambda_cross_tu_host` 启动 noinline kernel,也不要只看 build +目录中是否残留历史 runtime object。 -CANN 安装头文件 `include/acl/error_codes/rt_error_codes.h` 定义: +以 runner 的构建输出为准: ```text -507015 = ACL_ERROR_RT_AICORE_EXCEPTION -507018 = ACL_ERROR_RT_AICPU_EXCEPTION +aic_input_objects=... +runtime_text=... +submit_symbols=... ``` -本文探针由 ACL 直接启动 AIC kernel,所以原发异常在 host 侧表现为 507015。 -业务经过 AICPU 外层时报告 507018。错误层级符合调用路径差异,但仅凭两个 -错误码不能证明 fault PC 完全相同。 - -### 9.2 CPU 是否也有栈地址物化缺陷 - -CPU 同样需要物化栈地址,但当前 x86-64 GCC 后端在这条双 TU 调用链上没有 -观察到缺陷:多优化级别和 sanitizer 都通过。这支持问题是 CCEC HiIPU 后端 -特有,而不是通用 C++ 语义错误。 - -这不是对所有 CPU 编译器的普遍证明。本机没有 `clang++`,也没有覆盖所有 -代码布局。如果 runtime 在 submit 返回后仍保存并异步解引用 caller 栈地址, -CPU 也会发生 use-after-return;那是生命周期错误,不是地址物化缺陷。 - -## 10. 方案落地约束 - -当前结果证明“机制可行”,不等于真实 PA 业务修复已经完成。生产改造至少要 -满足: +### 10.2 noinline 完整运行退出 1 -1. runtime 必须在 submit 返回前把 recipe/capture 复制或物化到自己的稳定 - 存储,不能把 caller 栈裸地址留给异步阶段; -2. recipe slot 必须定义正式布局、容量、版本和边界检查; -3. 不能让 recipe slot 与正常 `scalar_count`、tensor slot 或后续 ABI 演进冲突; -4. 必须在真实 PA 上保留 eager control、原 lazy bad 和数据驱动 recipe 三组 - A/B,并检查 golden; -5. 必须覆盖真实的 64 轮、同容器四次 submit、多进程稳定性和异常恢复; -6. 无业务地址写只能用于诊断,不能作为正式修复提交; -7. 负向 507015 probe 只留在显式诊断矩阵,默认 CI 只运行正向 - `args-runtime-read`。 - -## 11. 常见问题与排查 - -### 11.1 `ASCEND_HOME_PATH` 或 `PTO_ISA_ROOT` 未设置 - -重新执行: +以下命令默认按 strong、m1、m0 顺序运行: ```bash -source "$CANN_ROOT/set_env.sh" -export PTO_ISA_ROOT="$CANN_ROOT/x86_64-linux" -``` - -不要把 `PTO_ISA_ROOT` 指到不含 -`include/pto/common/kernel_meta.hpp` 的目录。 - -### 11.2 `aclrtBinaryGetFunction` 返回 107000 - -raw AICore ELF 的多入口不要使用带 `_mix_aic` 的完整符号名查找。本文已经按 -CANN runtime 头文件的定义处理:`funcEntry` 是 kernel 名中的数字后缀,七个 -入口使用唯一的 `0..6`,host 调用 `aclrtBinaryGetFunctionByEntry`。 - -当前正确命名示例: - -```text -nested_lambda_cross_tu_ctx_m0_0_mix_aic -nested_lambda_cross_tu_ctx_m1_1_mix_aic -... -nested_lambda_cross_tu_runtime_args_6_mix_aic -``` - -本机官方依据: - -```text -$ASCEND_HOME_PATH/x86_64-linux/pkg_inc/runtime/runtime/rts/rts_kernel.h -funcEntry: the suffix number; kernel_foo_123 -> 123 +tests/atomic_probe/ccec/run_all.sh \ + nested_lambda_only_weak_submit_noinline run ``` -### 11.3 完整矩阵退出码为 1 - -先检查是否只有最后的 `weak-context-materialize-0` 返回 507015。如果前六组 -都是 PASS,那么退出码 1 是预期诊断结果。若 build、入口获取或方案组也失败, -则不是预期结果。 - -### 11.4 CCEC 报 `tensor.h` 的 unused variable warning - -当前构建会从仓库已有 `tensor.h` 报一个 `buffer_elems` unused warning;本次 -编译仍成功。不要把 warning 当作本 probe 的 semantic failure,也不要为了 -本测试批量修改无关生产头文件。 - -### 11.5 故障组后设备不可用 - -停止继续运行,执行所在平台批准的设备 reset/recovery,再先跑 -`strong-context` 或 `args-runtime-read` 控制组。本文不提供未经本机文档验证的 -reset 命令。本文最终一次复核后正处于该状态:`aclInit=500000`,NPU 已不再 -存在/暴露,所以没有继续执行任何硬件命令。 +如果前两组 PASS,最后 m0 返回 507015,则最终退出 1 是预期诊断结果。 -### 11.6 故障组意外通过 +### 10.3 m0 意外通过或失败 先确认: -- 使用的是 `dav-c310-cube`,不是 AIV; -- caller/runtime 确实分开编译; -- 没有复用旧 build; -- CCEC 版本和 flags 与第 6.4 节一致; -- entry 0 启动的是 `weak-context-materialize-0`; -- source tree 包含本文列出的测试版本。 - -重新执行 build-only 后再运行。如果仍通过,应记录为“当前环境未复现”,不能 -人为添加无关代码强迫其失败。 - -## 12. 最终复现检查清单 - -完成下列检查即可认为复现记录完整: - -- [ ] 记录 repo HEAD、CANN/ccec/GCC 版本和设备型号; -- [ ] CPU 嵌套 lambda 基线 PASS; -- [ ] CPU runtime-read 对等实现 PASS; -- [ ] AscendC AIV 基线 PASS; -- [ ] 纯 CCEC AIV 基线 PASS; -- [ ] AIC caller/runtime 双 TU build PASS; -- [ ] 七个 metadata entry 和 weak/strong 符号符合预期; -- [ ] orchestration CFA 为 `reg93 + 1952`; -- [ ] `args-runtime-read` 为 PASS、checksum 精确匹配; -- [ ] `weak-context-materialize-0` 在受影响环境返回 507015; -- [ ] 明确记录完整矩阵退出码 1 是预期负向命中; -- [ ] 异常后按平台规范完成恢复; -- [ ] 没有把地址物化条数或 weak 属性误写成最终根因; -- [ ] 没有把机制验证误写成真实业务修复完成。 +- 使用了正确 fixture 的 kernel; +- all-inline target 没有 submit FUNC 符号; +- noinline target 只保留 weak-context submit; +- CCEC arch 为 `dav-c310-cube`; +- caller orchestration 和七个 metadata entry 仍存在; +- 没有复用另一 fixture 的 host/kernel 组合。 + +若 noinline m0 仍通过,应记录“当前工具链未命中”,不能添加无关代码强迫 +失败。 + +### 10.4 `tensor.h` unused warning + +CCEC 当前会报告 `buffer_elems` 未使用。它不属于本 probe 的 semantic +failure,不应为了该测试修改无关生产头文件。 + +## 11. 最终检查清单 + +- [ ] CPU 嵌套 lambda 语义 PASS; +- [ ] CPU inline args-runtime-read PASS; +- [ ] 默认 AIC target 只有一个输入对象; +- [ ] 默认 ELF 没有 `nested_probe_submit_*` FUNC 符号; +- [ ] 默认 all-inline m0 为 5/5 PASS; +- [ ] 双对象控制的第二个对象 `.text=0`; +- [ ] 双对象控制 m0 为 5/5 PASS; +- [ ] noinline target 只保留 weak-context submit; +- [ ] noinline m1 为 3/3 PASS; +- [ ] 同 fixture strong 为 3/3 PASS; +- [ ] noinline m0 为 5/5 507015; +- [ ] args-runtime-read checksum 精确匹配; +- [ ] `sizeof(L0TaskArgs) == 1024`; +- [ ] 没有把 `.o` 数量、`ld.lld`、weak 或某个 pass 写成已定位根因; +- [ ] 默认 pytest 不运行故障组。 diff --git a/tests/atomic_probe/ccec/nested_lambda_cross_tu.cpp b/tests/atomic_probe/ccec/nested_lambda_cross_tu.cpp index dbdbd955b5..0d8caae84b 100644 --- a/tests/atomic_probe/ccec/nested_lambda_cross_tu.cpp +++ b/tests/atomic_probe/ccec/nested_lambda_cross_tu.cpp @@ -28,10 +28,9 @@ using nested_lambda_cross_tu_probe::Field; using nested_lambda_cross_tu_probe::Variant; constexpr int32_t kSiteId = 7; +static_assert(sizeof(L0TaskArgs) == nested_lambda_cross_tu_probe::kExpectedL0TaskArgsBytes); -PTO_DEVICE_FUNC void BindContext( - int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args) -{ +PTO_DEVICE_FUNC void BindContext(int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args) { if (site_id != kSiteId) return; const auto *context = reinterpret_cast(caller_context); if (phase == static_cast(DispatchPhase::Prepare)) { @@ -41,23 +40,21 @@ PTO_DEVICE_FUNC void BindContext( args->add_input(*context->first, *context->second, *context->third); } -PTO_DEVICE_FUNC void InitTensor(Tensor &tensor, uint32_t round, uint32_t tensor_index) -{ +PTO_DEVICE_FUNC void InitTensor(Tensor &tensor, uint32_t round, uint32_t tensor_index) { tensor.buffer.addr = nested_lambda_cross_tu_probe::TensorAddress(round, tensor_index); tensor.start_offset = nested_lambda_cross_tu_probe::TensorOffset(round, tensor_index); tensor.version = nested_lambda_cross_tu_probe::TensorVersion(round, tensor_index); tensor.shapes[0] = nested_lambda_cross_tu_probe::TensorShape(round, tensor_index); } -PTO_DEVICE_FUNC void StoreField(__gm__ uint32_t *storage, Field field, uint32_t value) -{ +PTO_DEVICE_FUNC void StoreField(__gm__ uint32_t *storage, Field field, uint32_t value) { st_dev_b32(&storage[nested_lambda_cross_tu_probe::FieldIndex(field)], value); } PTO_DEVICE_FUNC void StoreResults( __gm__ uint32_t *storage, Variant variant, uint32_t completed_rounds, uint32_t mismatches, - uint32_t dispatcher_calls, uint32_t materializations, uint64_t checksum) -{ + uint32_t dispatcher_calls, uint32_t materializations, uint64_t checksum +) { StoreField(storage, Field::CompletedRounds, completed_rounds); StoreField(storage, Field::MismatchCount, mismatches); StoreField(storage, Field::DispatcherCalls, dispatcher_calls); @@ -70,8 +67,7 @@ PTO_DEVICE_FUNC void StoreResults( } template -PTO_DEVICE_FUNC TaskOutputTensors SubmitContext(uint64_t caller_context, L0TaskArgs *args) -{ +PTO_DEVICE_FUNC TaskOutputTensors SubmitContext(uint64_t caller_context, L0TaskArgs *args) { if constexpr (StrongDispatcher) { return nested_probe_submit_strong_context(kSiteId, caller_context, args); } @@ -79,8 +75,7 @@ PTO_DEVICE_FUNC TaskOutputTensors SubmitContext(uint64_t caller_context, L0TaskA } template -PTO_DEVICE_FUNC void RunContextVariant(__gm__ uint32_t *storage, Variant variant) -{ +PTO_DEVICE_FUNC void RunContextVariant(__gm__ uint32_t *storage, Variant variant) { L0TaskArgs args; uint32_t completed_rounds = 0; uint32_t mismatches = 0; @@ -129,8 +124,7 @@ PTO_DEVICE_FUNC void RunContextVariant(__gm__ uint32_t *storage, Variant variant } template -PTO_DEVICE_FUNC void RunArgsStorageVariant(__gm__ uint32_t *storage, Variant variant) -{ +PTO_DEVICE_FUNC void RunArgsStorageVariant(__gm__ uint32_t *storage, Variant variant) { L0TaskArgs args; uint32_t completed_rounds = 0; uint32_t mismatches = 0; @@ -178,21 +172,18 @@ PTO_DEVICE_FUNC void RunArgsStorageVariant(__gm__ uint32_t *storage, Variant var completed_rounds++; } - StoreResults( - storage, variant, completed_rounds, mismatches, dispatcher_calls, 3, checksum); + StoreResults(storage, variant, completed_rounds, mismatches, dispatcher_calls, 3, checksum); } -} // namespace +} // namespace -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_weak_context_dispatch( - int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void +nested_probe_weak_context_dispatch(int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args) { BindContext(site_id, phase, caller_context, args); } -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_weak_args_dispatch( - int32_t site_id, int32_t phase, L0TaskArgs *args) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void +nested_probe_weak_args_dispatch(int32_t site_id, int32_t phase, L0TaskArgs *args) { if (site_id != kSiteId) return; if (phase == static_cast(DispatchPhase::Prepare)) { args->scalar(4) = args->scalar(11); @@ -204,9 +195,8 @@ extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_weak_args_dis args->add_input(*first, *second, *third); } -extern "C" PTO_DEVICE_FUNC void nested_probe_strong_context_dispatch( - int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args) -{ +extern "C" PTO_DEVICE_FUNC void +nested_probe_strong_context_dispatch(int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args) { BindContext(site_id, phase, caller_context, args); } @@ -214,79 +204,59 @@ extern "C" PTO_DEVICE_FUNC void nested_probe_strong_context_dispatch( // aicpu_orchestration_entry. Putting it directly in a __global__ wrapper would // move its locals from the orchestration function stack to the kernel stack and // miss the suspected reg93 stack-address materialization path. -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_ctx_m0( - __gm__ uint32_t *storage) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_ctx_m0(__gm__ uint32_t *storage) { RunContextVariant<0, false>(storage, Variant::WeakContextMaterialize0); } -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_ctx_m1( - __gm__ uint32_t *storage) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_ctx_m1(__gm__ uint32_t *storage) { RunContextVariant<1, false>(storage, Variant::WeakContextMaterialize1); } -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_ctx_m2( - __gm__ uint32_t *storage) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_ctx_m2(__gm__ uint32_t *storage) { RunContextVariant<2, false>(storage, Variant::WeakContextMaterialize2); } -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_ctx_m3( - __gm__ uint32_t *storage) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_ctx_m3(__gm__ uint32_t *storage) { RunContextVariant<3, false>(storage, Variant::WeakContextMaterialize3); } -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_args( - __gm__ uint32_t *storage) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_args(__gm__ uint32_t *storage) { RunArgsStorageVariant(storage, Variant::WeakArgsStorage); } -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_strong( - __gm__ uint32_t *storage) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_strong(__gm__ uint32_t *storage) { RunContextVariant<0, true>(storage, Variant::StrongContext); } -extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void nested_probe_orchestration_runtime_args( - __gm__ uint32_t *storage) -{ +extern "C" __attribute__((weak)) PTO_DEVICE_FUNC void +nested_probe_orchestration_runtime_args(__gm__ uint32_t *storage) { RunArgsStorageVariant(storage, Variant::ArgsRuntimeRead); } -extern "C" __global__ __aicore__ void nested_lambda_cross_tu_ctx_m0_0_mix_aic(__gm__ uint32_t *storage) -{ +extern "C" __global__ __aicore__ void nested_lambda_cross_tu_ctx_m0_0_mix_aic(__gm__ uint32_t *storage) { if (get_block_idx() == 0) nested_probe_orchestration_ctx_m0(storage); } -extern "C" __global__ __aicore__ void nested_lambda_cross_tu_ctx_m1_1_mix_aic(__gm__ uint32_t *storage) -{ +extern "C" __global__ __aicore__ void nested_lambda_cross_tu_ctx_m1_1_mix_aic(__gm__ uint32_t *storage) { if (get_block_idx() == 0) nested_probe_orchestration_ctx_m1(storage); } -extern "C" __global__ __aicore__ void nested_lambda_cross_tu_ctx_m2_2_mix_aic(__gm__ uint32_t *storage) -{ +extern "C" __global__ __aicore__ void nested_lambda_cross_tu_ctx_m2_2_mix_aic(__gm__ uint32_t *storage) { if (get_block_idx() == 0) nested_probe_orchestration_ctx_m2(storage); } -extern "C" __global__ __aicore__ void nested_lambda_cross_tu_ctx_m3_3_mix_aic(__gm__ uint32_t *storage) -{ +extern "C" __global__ __aicore__ void nested_lambda_cross_tu_ctx_m3_3_mix_aic(__gm__ uint32_t *storage) { if (get_block_idx() == 0) nested_probe_orchestration_ctx_m3(storage); } -extern "C" __global__ __aicore__ void nested_lambda_cross_tu_args_4_mix_aic(__gm__ uint32_t *storage) -{ +extern "C" __global__ __aicore__ void nested_lambda_cross_tu_args_4_mix_aic(__gm__ uint32_t *storage) { if (get_block_idx() == 0) nested_probe_orchestration_args(storage); } -extern "C" __global__ __aicore__ void nested_lambda_cross_tu_strong_5_mix_aic(__gm__ uint32_t *storage) -{ +extern "C" __global__ __aicore__ void nested_lambda_cross_tu_strong_5_mix_aic(__gm__ uint32_t *storage) { if (get_block_idx() == 0) nested_probe_orchestration_strong(storage); } -extern "C" __global__ __aicore__ void nested_lambda_cross_tu_runtime_args_6_mix_aic(__gm__ uint32_t *storage) -{ +extern "C" __global__ __aicore__ void nested_lambda_cross_tu_runtime_args_6_mix_aic(__gm__ uint32_t *storage) { if (get_block_idx() == 0) nested_probe_orchestration_runtime_args(storage); } diff --git a/tests/atomic_probe/ccec/nested_lambda_cross_tu_api.h b/tests/atomic_probe/ccec/nested_lambda_cross_tu_api.h index a4db3dc4c9..48ef4606ff 100644 --- a/tests/atomic_probe/ccec/nested_lambda_cross_tu_api.h +++ b/tests/atomic_probe/ccec/nested_lambda_cross_tu_api.h @@ -11,6 +11,7 @@ #ifndef TESTS_ATOMIC_PROBE_CCEC_NESTED_LAMBDA_CROSS_TU_API_H #define TESTS_ATOMIC_PROBE_CCEC_NESTED_LAMBDA_CROSS_TU_API_H +#include "nested_lambda_cross_tu_layout.h" #include "pto_types.h" #include @@ -29,15 +30,108 @@ struct CallerContext { uint64_t salt; }; -} // namespace nested_lambda_cross_tu_probe +} // namespace nested_lambda_cross_tu_probe -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_weak_context( - int32_t site_id, uint64_t caller_context, L0TaskArgs *args); -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_weak_args( - int32_t site_id, L0TaskArgs *args); -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_strong_context( - int32_t site_id, uint64_t caller_context, L0TaskArgs *args); -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_control(L0TaskArgs *args); -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_args_runtime_read(L0TaskArgs *args); +extern "C" PTO_DEVICE_FUNC void +nested_probe_weak_context_dispatch(int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args) + __attribute__((weak)); +extern "C" PTO_DEVICE_FUNC void nested_probe_weak_args_dispatch(int32_t site_id, int32_t phase, L0TaskArgs *args) + __attribute__((weak)); +extern "C" PTO_DEVICE_FUNC void +nested_probe_strong_context_dispatch(int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args); -#endif // TESTS_ATOMIC_PROBE_CCEC_NESTED_LAMBDA_CROSS_TU_API_H +#define NESTED_PROBE_RUNTIME_INLINE PTO_DEVICE_FUNC __attribute__((always_inline)) inline + +#if defined(NESTED_PROBE_WEAK_CONTEXT_NOINLINE) +#define NESTED_PROBE_WEAK_CONTEXT_FUNC PTO_DEVICE_FUNC __attribute__((noinline)) +#else +#define NESTED_PROBE_WEAK_CONTEXT_FUNC NESTED_PROBE_RUNTIME_INLINE +#endif + +namespace nested_lambda_cross_tu_probe::detail { + +NESTED_PROBE_RUNTIME_INLINE uint64_t TensorDigest(const Tensor &tensor) { + return tensor.buffer.addr + tensor.start_offset * 17ULL + + static_cast(static_cast(tensor.version)) * 257ULL + tensor.shapes[0] * 65537ULL; +} + +NESTED_PROBE_RUNTIME_INLINE void ConsumeBoundArguments(L0TaskArgs *args) { + const Tensor &first = args->tensor(0).ref(); + const Tensor &second = args->tensor(1).ref(); + const Tensor &third = args->tensor(2).ref(); + args->scalar(0) = + TensorDigest(first) * 3ULL + TensorDigest(second) * 5ULL + TensorDigest(third) * 7ULL + args->scalar(4); + args->scalar(5) = 2; + args->scalar(6) = 0; +} + +} // namespace nested_lambda_cross_tu_probe::detail + +NESTED_PROBE_WEAK_CONTEXT_FUNC TaskOutputTensors +nested_probe_submit_weak_context(int32_t site_id, uint64_t caller_context, L0TaskArgs *args) { + TaskOutputTensors outputs; + if (nested_probe_weak_context_dispatch == nullptr) { + args->scalar(6) = 1; + return outputs; + } + nested_probe_weak_context_dispatch( + site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::Prepare), caller_context, args + ); + nested_probe_weak_context_dispatch( + site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::WinnerBind), caller_context, args + ); + nested_lambda_cross_tu_probe::detail::ConsumeBoundArguments(args); + return outputs; +} + +NESTED_PROBE_RUNTIME_INLINE TaskOutputTensors nested_probe_submit_weak_args(int32_t site_id, L0TaskArgs *args) { + TaskOutputTensors outputs; + if (nested_probe_weak_args_dispatch == nullptr) { + args->scalar(6) = 1; + return outputs; + } + nested_probe_weak_args_dispatch( + site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::Prepare), args + ); + nested_probe_weak_args_dispatch( + site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::WinnerBind), args + ); + nested_lambda_cross_tu_probe::detail::ConsumeBoundArguments(args); + return outputs; +} + +NESTED_PROBE_RUNTIME_INLINE TaskOutputTensors +nested_probe_submit_strong_context(int32_t site_id, uint64_t caller_context, L0TaskArgs *args) { + TaskOutputTensors outputs; + nested_probe_strong_context_dispatch( + site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::Prepare), caller_context, args + ); + nested_probe_strong_context_dispatch( + site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::WinnerBind), caller_context, args + ); + nested_lambda_cross_tu_probe::detail::ConsumeBoundArguments(args); + return outputs; +} + +NESTED_PROBE_RUNTIME_INLINE TaskOutputTensors nested_probe_submit_control(L0TaskArgs *args) { + TaskOutputTensors outputs; + args->scalar(0) ^= nested_lambda_cross_tu_probe::kControlXor; + return outputs; +} + +NESTED_PROBE_RUNTIME_INLINE TaskOutputTensors nested_probe_submit_args_runtime_read(L0TaskArgs *args) { + TaskOutputTensors outputs; + const auto *first = reinterpret_cast(args->scalar(8)); + const auto *second = reinterpret_cast(args->scalar(9)); + const auto *third = reinterpret_cast(args->scalar(10)); + args->scalar(4) = args->scalar(11); + args->add_input(*first, *second, *third); + nested_lambda_cross_tu_probe::detail::ConsumeBoundArguments(args); + args->scalar(5) = 0; + return outputs; +} + +#undef NESTED_PROBE_RUNTIME_INLINE +#undef NESTED_PROBE_WEAK_CONTEXT_FUNC + +#endif // TESTS_ATOMIC_PROBE_CCEC_NESTED_LAMBDA_CROSS_TU_API_H diff --git a/tests/atomic_probe/ccec/nested_lambda_cross_tu_host.cpp b/tests/atomic_probe/ccec/nested_lambda_cross_tu_host.cpp index cac501d91e..b789d28dce 100644 --- a/tests/atomic_probe/ccec/nested_lambda_cross_tu_host.cpp +++ b/tests/atomic_probe/ccec/nested_lambda_cross_tu_host.cpp @@ -26,13 +26,11 @@ struct KernelArgs { uint64_t storage_pointer; }; -uint32_t ReadField(const std::vector &storage, Field field) -{ +uint32_t ReadField(const std::vector &storage, Field field) { return storage[nested_lambda_cross_tu_probe::FieldIndex(field)]; } -bool ParseVariant(const char *text, Variant *variant) -{ +bool ParseVariant(const char *text, Variant *variant) { for (uint32_t raw = 0; raw < static_cast(Variant::Count); raw++) { const Variant candidate = static_cast(raw); if (std::strcmp(text, nested_lambda_cross_tu_probe::VariantName(candidate)) == 0) { @@ -43,39 +41,38 @@ bool ParseVariant(const char *text, Variant *variant) return false; } -void Validate(const std::vector &storage, Variant variant, atomic_probe::Result &result) -{ +void Validate(const std::vector &storage, Variant variant, atomic_probe::Result &result) { const uint64_t checksum = static_cast(ReadField(storage, Field::ChecksumLow)) | - (static_cast(ReadField(storage, Field::ChecksumHigh)) << 32); + (static_cast(ReadField(storage, Field::ChecksumHigh)) << 32); const uint32_t args_bytes = ReadField(storage, Field::L0TaskArgsBytes); bool exact = true; exact &= ReadField(storage, Field::CompletedRounds) == nested_lambda_cross_tu_probe::kRounds; exact &= ReadField(storage, Field::MismatchCount) == 0; - exact &= ReadField(storage, Field::DispatcherCalls) == - nested_lambda_cross_tu_probe::ExpectedDispatcherCalls(variant); + exact &= + ReadField(storage, Field::DispatcherCalls) == nested_lambda_cross_tu_probe::ExpectedDispatcherCalls(variant); exact &= ReadField(storage, Field::AddressMaterializations) == - nested_lambda_cross_tu_probe::ExpectedMaterializations(variant); + nested_lambda_cross_tu_probe::ExpectedMaterializations(variant); exact &= checksum == nested_lambda_cross_tu_probe::ExpectedTotalChecksum(); - exact &= args_bytes > 0 && args_bytes < 32U * 1024U && args_bytes % 64U == 0; + exact &= args_bytes == nested_lambda_cross_tu_probe::kExpectedL0TaskArgsBytes; exact &= ReadField(storage, Field::VariantEcho) == static_cast(variant); char label[120]; std::snprintf( - label, sizeof(label), "CCEC AIC cross-TU ABI variant=%s", - nested_lambda_cross_tu_probe::VariantName(variant)); + label, sizeof(label), "CCEC AIC caller-capture variant=%s", nested_lambda_cross_tu_probe::VariantName(variant) + ); result.Expect(exact, label); std::printf( "[VALUES] rounds=%u mismatches=%u dispatches=%u materializations=%u " "checksum=0x%016llx L0TaskArgs=%uB\n", ReadField(storage, Field::CompletedRounds), ReadField(storage, Field::MismatchCount), ReadField(storage, Field::DispatcherCalls), ReadField(storage, Field::AddressMaterializations), - static_cast(checksum), args_bytes); + static_cast(checksum), args_bytes + ); } -} // namespace +} // namespace -int main(int argc, char *argv[]) -{ +int main(int argc, char *argv[]) { if (argc != 3) { std::fprintf(stderr, "Usage: %s \n", argv[0]); return EXIT_FAILURE; @@ -83,7 +80,7 @@ int main(int argc, char *argv[]) const char *kernel_path = argv[1]; Variant variant = Variant::Count; if (!ParseVariant(argv[2], &variant)) { - std::fprintf(stderr, "Unknown cross-TU variant: %s\n", argv[2]); + std::fprintf(stderr, "Unknown caller-capture variant: %s\n", argv[2]); return EXIT_FAILURE; } @@ -110,32 +107,35 @@ int main(int argc, char *argv[]) void *storage_device = nullptr; PROBE_ACL_CHECK(aclrtMalloc(&storage_device, storage_bytes, ACL_MEM_MALLOC_HUGE_FIRST)); - std::printf("=== Pure CCEC AIC Caller-Capture Transport Probe ===\n"); + std::printf("=== Pure CCEC AIC Caller-Capture Call-Boundary Probe ===\n"); atomic_probe::Result result; aclrtFuncHandle function_handle = nullptr; PROBE_ACL_CHECK(aclrtBinaryGetFunctionByEntry( - binary_handle, nested_lambda_cross_tu_probe::KernelEntry(variant), &function_handle)); + binary_handle, nested_lambda_cross_tu_probe::KernelEntry(variant), &function_handle + )); std::vector storage(nested_lambda_cross_tu_probe::kStorageWords, 0); - PROBE_ACL_CHECK(aclrtMemcpy( - storage_device, storage_bytes, storage.data(), storage_bytes, ACL_MEMCPY_HOST_TO_DEVICE)); + PROBE_ACL_CHECK( + aclrtMemcpy(storage_device, storage_bytes, storage.data(), storage_bytes, ACL_MEMCPY_HOST_TO_DEVICE) + ); KernelArgs args{reinterpret_cast(storage_device)}; - PROBE_ACL_CHECK(aclrtLaunchKernelWithHostArgs( - function_handle, 1, stream, nullptr, &args, sizeof(args), nullptr, 0)); + PROBE_ACL_CHECK( + aclrtLaunchKernelWithHostArgs(function_handle, 1, stream, nullptr, &args, sizeof(args), nullptr, 0) + ); PROBE_ACL_CHECK(aclrtSynchronizeStream(stream)); - PROBE_ACL_CHECK(aclrtMemcpy( - storage.data(), storage_bytes, storage_device, storage_bytes, ACL_MEMCPY_DEVICE_TO_HOST)); + PROBE_ACL_CHECK( + aclrtMemcpy(storage.data(), storage_bytes, storage_device, storage_bytes, ACL_MEMCPY_DEVICE_TO_HOST) + ); Validate(storage, variant, result); bool cleanup_ok = true; + cleanup_ok &= atomic_probe::CheckAcl(aclrtFree(storage_device), "aclrtFree(storage_device)", __FILE__, __LINE__); cleanup_ok &= atomic_probe::CheckAcl( - aclrtFree(storage_device), "aclrtFree(storage_device)", __FILE__, __LINE__); - cleanup_ok &= atomic_probe::CheckAcl( - aclrtBinaryUnLoad(binary_handle), "aclrtBinaryUnLoad(binary_handle)", __FILE__, __LINE__); - cleanup_ok &= atomic_probe::CheckAcl( - aclrtDestroyStream(stream), "aclrtDestroyStream(stream)", __FILE__, __LINE__); - cleanup_ok &= atomic_probe::CheckAcl( - aclrtResetDevice(device_id), "aclrtResetDevice(device_id)", __FILE__, __LINE__); + aclrtBinaryUnLoad(binary_handle), "aclrtBinaryUnLoad(binary_handle)", __FILE__, __LINE__ + ); + cleanup_ok &= atomic_probe::CheckAcl(aclrtDestroyStream(stream), "aclrtDestroyStream(stream)", __FILE__, __LINE__); + cleanup_ok &= + atomic_probe::CheckAcl(aclrtResetDevice(device_id), "aclrtResetDevice(device_id)", __FILE__, __LINE__); cleanup_ok &= atomic_probe::CheckAcl(aclFinalize(), "aclFinalize()", __FILE__, __LINE__); - result.Expect(cleanup_ok, "CCEC cross-TU ACL cleanup"); + result.Expect(cleanup_ok, "CCEC caller-capture ACL cleanup"); return result.ExitCode(); } diff --git a/tests/atomic_probe/ccec/nested_lambda_cross_tu_layout.h b/tests/atomic_probe/ccec/nested_lambda_cross_tu_layout.h index 1713b0111a..a7c4e420d0 100644 --- a/tests/atomic_probe/ccec/nested_lambda_cross_tu_layout.h +++ b/tests/atomic_probe/ccec/nested_lambda_cross_tu_layout.h @@ -46,66 +46,52 @@ enum class Field : uint32_t { constexpr uint32_t kRounds = 64; constexpr uint32_t kSubmitsPerRound = 4; +constexpr uint32_t kExpectedL0TaskArgsBytes = 1024; constexpr uint32_t kCacheLineWords = 16; constexpr uint32_t kStorageWords = static_cast(Field::Count) * kCacheLineWords; constexpr uint64_t kControlXor = 0x6A09E667F3BCC909ULL; -CROSS_TU_DEVICE constexpr uint32_t FieldIndex(Field field) -{ - return static_cast(field) * kCacheLineWords; -} +CROSS_TU_DEVICE constexpr uint32_t FieldIndex(Field field) { return static_cast(field) * kCacheLineWords; } -CROSS_TU_DEVICE constexpr uint64_t TensorAddress(uint32_t round, uint32_t tensor_index) -{ - return 0x100000000ULL + static_cast(round) * 0x1000ULL + - static_cast(tensor_index) * 0x100ULL + 0x55ULL; +CROSS_TU_DEVICE constexpr uint64_t TensorAddress(uint32_t round, uint32_t tensor_index) { + return 0x100000000ULL + static_cast(round) * 0x1000ULL + static_cast(tensor_index) * 0x100ULL + + 0x55ULL; } -CROSS_TU_DEVICE constexpr uint64_t TensorOffset(uint32_t round, uint32_t tensor_index) -{ +CROSS_TU_DEVICE constexpr uint64_t TensorOffset(uint32_t round, uint32_t tensor_index) { return static_cast(round) * 8ULL + tensor_index; } -CROSS_TU_DEVICE constexpr int32_t TensorVersion(uint32_t round, uint32_t tensor_index) -{ +CROSS_TU_DEVICE constexpr int32_t TensorVersion(uint32_t round, uint32_t tensor_index) { return static_cast(100U + round * 3U + tensor_index); } -CROSS_TU_DEVICE constexpr uint32_t TensorShape(uint32_t round, uint32_t tensor_index) -{ +CROSS_TU_DEVICE constexpr uint32_t TensorShape(uint32_t round, uint32_t tensor_index) { return 17U + round + tensor_index; } -CROSS_TU_DEVICE constexpr uint64_t ContextSalt(uint32_t round) -{ - return 0xBADC000000000000ULL + round; -} +CROSS_TU_DEVICE constexpr uint64_t ContextSalt(uint32_t round) { return 0xBADC000000000000ULL + round; } -CROSS_TU_DEVICE constexpr uint64_t TensorDigest(uint32_t round, uint32_t tensor_index) -{ +CROSS_TU_DEVICE constexpr uint64_t TensorDigest(uint32_t round, uint32_t tensor_index) { return TensorAddress(round, tensor_index) + TensorOffset(round, tensor_index) * 17ULL + - static_cast(static_cast(TensorVersion(round, tensor_index))) * 257ULL + - TensorShape(round, tensor_index) * 65537ULL; + static_cast(static_cast(TensorVersion(round, tensor_index))) * 257ULL + + TensorShape(round, tensor_index) * 65537ULL; } -CROSS_TU_DEVICE constexpr uint64_t ExpectedLazyDigest(uint32_t round) -{ - return TensorDigest(round, 0) * 3ULL + TensorDigest(round, 1) * 5ULL + - TensorDigest(round, 2) * 7ULL + ContextSalt(round); +CROSS_TU_DEVICE constexpr uint64_t ExpectedLazyDigest(uint32_t round) { + return TensorDigest(round, 0) * 3ULL + TensorDigest(round, 1) * 5ULL + TensorDigest(round, 2) * 7ULL + + ContextSalt(round); } -CROSS_TU_DEVICE constexpr uint64_t ControlInput(uint32_t round, uint32_t submit_index) -{ +CROSS_TU_DEVICE constexpr uint64_t ControlInput(uint32_t round, uint32_t submit_index) { return 0xC000000000000000ULL + static_cast(round) * kSubmitsPerRound + submit_index; } -CROSS_TU_DEVICE constexpr uint64_t ExpectedControlResult(uint32_t round, uint32_t submit_index) -{ +CROSS_TU_DEVICE constexpr uint64_t ExpectedControlResult(uint32_t round, uint32_t submit_index) { return ControlInput(round, submit_index) ^ kControlXor; } -CROSS_TU_DEVICE constexpr uint64_t ExpectedTotalChecksum() -{ +CROSS_TU_DEVICE constexpr uint64_t ExpectedTotalChecksum() { uint64_t checksum = 0; for (uint32_t round = 0; round < kRounds; round++) { checksum += ExpectedLazyDigest(round); @@ -116,8 +102,7 @@ CROSS_TU_DEVICE constexpr uint64_t ExpectedTotalChecksum() return checksum; } -constexpr uint32_t ExpectedMaterializations(Variant variant) -{ +constexpr uint32_t ExpectedMaterializations(Variant variant) { switch (variant) { case Variant::WeakContextMaterialize0: case Variant::StrongContext: @@ -136,8 +121,7 @@ constexpr uint32_t ExpectedMaterializations(Variant variant) return 0; } -constexpr const char *VariantName(Variant variant) -{ +constexpr const char *VariantName(Variant variant) { switch (variant) { case Variant::WeakContextMaterialize0: return "weak-context-materialize-0"; @@ -159,8 +143,7 @@ constexpr const char *VariantName(Variant variant) return "invalid"; } -constexpr const char *KernelName(Variant variant) -{ +constexpr const char *KernelName(Variant variant) { switch (variant) { case Variant::WeakContextMaterialize0: return "nested_lambda_cross_tu_ctx_m0_0_mix_aic"; @@ -184,18 +167,14 @@ constexpr const char *KernelName(Variant variant) // CANN defines funcEntry as the numeric suffix in kernel_foo_. // Keep entries unique inside this multi-kernel raw ELF. -constexpr uint64_t KernelEntry(Variant variant) -{ - return static_cast(variant); -} +constexpr uint64_t KernelEntry(Variant variant) { return static_cast(variant); } -constexpr uint32_t ExpectedDispatcherCalls(Variant variant) -{ +constexpr uint32_t ExpectedDispatcherCalls(Variant variant) { return variant == Variant::ArgsRuntimeRead ? 0 : kRounds * 2; } #undef CROSS_TU_DEVICE -} // namespace nested_lambda_cross_tu_probe +} // namespace nested_lambda_cross_tu_probe -#endif // TESTS_ATOMIC_PROBE_CCEC_NESTED_LAMBDA_CROSS_TU_LAYOUT_H +#endif // TESTS_ATOMIC_PROBE_CCEC_NESTED_LAMBDA_CROSS_TU_LAYOUT_H diff --git a/tests/atomic_probe/ccec/nested_lambda_cross_tu_runtime.cpp b/tests/atomic_probe/ccec/nested_lambda_cross_tu_runtime.cpp index 249c3c93bb..53ea6e332d 100644 --- a/tests/atomic_probe/ccec/nested_lambda_cross_tu_runtime.cpp +++ b/tests/atomic_probe/ccec/nested_lambda_cross_tu_runtime.cpp @@ -9,97 +9,7 @@ * ----------------------------------------------------------------------------------------------------------- */ #include "nested_lambda_cross_tu_api.h" -#include "nested_lambda_cross_tu_layout.h" -extern "C" PTO_DEVICE_FUNC void nested_probe_weak_context_dispatch( - int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args) __attribute__((weak)); -extern "C" PTO_DEVICE_FUNC void nested_probe_weak_args_dispatch( - int32_t site_id, int32_t phase, L0TaskArgs *args) __attribute__((weak)); -extern "C" PTO_DEVICE_FUNC void nested_probe_strong_context_dispatch( - int32_t site_id, int32_t phase, uint64_t caller_context, L0TaskArgs *args); - -namespace { - -PTO_DEVICE_FUNC uint64_t TensorDigest(const Tensor &tensor) -{ - return tensor.buffer.addr + tensor.start_offset * 17ULL + - static_cast(static_cast(tensor.version)) * 257ULL + - tensor.shapes[0] * 65537ULL; -} - -PTO_DEVICE_FUNC void ConsumeBoundArguments(L0TaskArgs *args) -{ - const Tensor &first = args->tensor(0).ref(); - const Tensor &second = args->tensor(1).ref(); - const Tensor &third = args->tensor(2).ref(); - args->scalar(0) = TensorDigest(first) * 3ULL + TensorDigest(second) * 5ULL + - TensorDigest(third) * 7ULL + args->scalar(4); - args->scalar(5) = 2; - args->scalar(6) = 0; -} - -} // namespace - -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_weak_context( - int32_t site_id, uint64_t caller_context, L0TaskArgs *args) -{ - TaskOutputTensors outputs; - if (nested_probe_weak_context_dispatch == nullptr) { - args->scalar(6) = 1; - return outputs; - } - nested_probe_weak_context_dispatch( - site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::Prepare), caller_context, args); - nested_probe_weak_context_dispatch( - site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::WinnerBind), caller_context, args); - ConsumeBoundArguments(args); - return outputs; -} - -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_weak_args( - int32_t site_id, L0TaskArgs *args) -{ - TaskOutputTensors outputs; - if (nested_probe_weak_args_dispatch == nullptr) { - args->scalar(6) = 1; - return outputs; - } - nested_probe_weak_args_dispatch( - site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::Prepare), args); - nested_probe_weak_args_dispatch( - site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::WinnerBind), args); - ConsumeBoundArguments(args); - return outputs; -} - -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_strong_context( - int32_t site_id, uint64_t caller_context, L0TaskArgs *args) -{ - TaskOutputTensors outputs; - nested_probe_strong_context_dispatch( - site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::Prepare), caller_context, args); - nested_probe_strong_context_dispatch( - site_id, static_cast(nested_lambda_cross_tu_probe::DispatchPhase::WinnerBind), caller_context, args); - ConsumeBoundArguments(args); - return outputs; -} - -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_control(L0TaskArgs *args) -{ - TaskOutputTensors outputs; - args->scalar(0) ^= nested_lambda_cross_tu_probe::kControlXor; - return outputs; -} - -PTO_DEVICE_FUNC TaskOutputTensors nested_probe_submit_args_runtime_read(L0TaskArgs *args) -{ - TaskOutputTensors outputs; - const auto *first = reinterpret_cast(args->scalar(8)); - const auto *second = reinterpret_cast(args->scalar(9)); - const auto *third = reinterpret_cast(args->scalar(10)); - args->scalar(4) = args->scalar(11); - args->add_input(*first, *second, *third); - ConsumeBoundArguments(args); - args->scalar(5) = 0; - return outputs; -} +// This translation unit intentionally emits no runtime submit functions. It is +// linked only by the two-object control to separate object count from an actual +// cross-object call boundary. diff --git a/tests/atomic_probe/ccec/nested_lambda_inline_plus_empty_runtime.cpp b/tests/atomic_probe/ccec/nested_lambda_inline_plus_empty_runtime.cpp new file mode 100644 index 0000000000..434b5f1d6a --- /dev/null +++ b/tests/atomic_probe/ccec/nested_lambda_inline_plus_empty_runtime.cpp @@ -0,0 +1,11 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ +#include "nested_lambda_cross_tu.cpp" diff --git a/tests/atomic_probe/ccec/nested_lambda_only_weak_submit_noinline.cpp b/tests/atomic_probe/ccec/nested_lambda_only_weak_submit_noinline.cpp new file mode 100644 index 0000000000..98ab040a89 --- /dev/null +++ b/tests/atomic_probe/ccec/nested_lambda_only_weak_submit_noinline.cpp @@ -0,0 +1,12 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ +#define NESTED_PROBE_WEAK_CONTEXT_NOINLINE +#include "nested_lambda_cross_tu.cpp" diff --git a/tests/atomic_probe/ccec/run_all.sh b/tests/atomic_probe/ccec/run_all.sh index f33d931099..051a07758c 100755 --- a/tests/atomic_probe/ccec/run_all.sh +++ b/tests/atomic_probe/ccec/run_all.sh @@ -1,10 +1,18 @@ #!/bin/bash +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- # Build & run ALL ccec atomic probes on A5 onboard hardware. # # For each probe: compiles the kernel .cpp with ccec -x cce, links it into # an AICore binary with ld.lld, then compiles the host launcher with g++ and -# runs it. Probes are AIV-only except the explicit cross-TU compiler/ABI probe, -# which targets AIC to match the affected orchestration build. +# runs it. Probes are AIV-only except the explicit caller-capture compiler +# probes, which target AIC to match the affected orchestration build. # # All kernels are pure-CCEC (ccec_utils.h + lowercase builtins); no # kernel_operator.h, no AscendC APIs. @@ -75,11 +83,13 @@ PROBES=( "cacheline_matrix.cpp:cacheline_matrix_kernel.o:cacheline_matrix_host.cpp:cacheline_matrix_host" ) -# This compiler-regression probe can intentionally trigger an AICore exception -# on affected CCEC builds, so it is selectable by name but is not part of the -# default cache-line suite. +# These caller-capture build-shape probes are selectable by name but are not +# part of the default cache-line suite. The noinline target can intentionally +# trigger an AICore exception on affected CCEC builds. MANUAL_PROBES=( "nested_lambda_cross_tu.cpp:nested_lambda_cross_tu_kernel.o:nested_lambda_cross_tu_host.cpp:nested_lambda_cross_tu_host" + "nested_lambda_inline_plus_empty_runtime.cpp:nested_lambda_inline_plus_empty_runtime_kernel.o:nested_lambda_cross_tu_host.cpp:nested_lambda_inline_plus_empty_runtime_host" + "nested_lambda_only_weak_submit_noinline.cpp:nested_lambda_only_weak_submit_noinline_kernel.o:nested_lambda_cross_tu_host.cpp:nested_lambda_only_weak_submit_noinline_host" ) REQUESTED="${1:-all}" @@ -127,7 +137,9 @@ build_one() { fi local kernel_objects=() - if [[ "$ks" == "nested_lambda_cross_tu.cpp" ]]; then + if [[ "$ks" == "nested_lambda_cross_tu.cpp" || + "$ks" == "nested_lambda_inline_plus_empty_runtime.cpp" || + "$ks" == "nested_lambda_only_weak_submit_noinline.cpp" ]]; then local repo_root repo_root="$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel)" local cross_tu_inc_flags=( @@ -147,14 +159,33 @@ build_one() { "$CCEC" "${CCEC_FLAGS[@]}" --cce-aicore-arch=dav-c310-cube \ "${cross_tu_inc_flags[@]}" \ -o "$BUILD_DIR/${tag}_caller_aic.o" "$SCRIPT_DIR/$ks" - echo "=== [$tag] Compiling runtime TU for AIC (dav-c310-cube) ===" - "$CCEC" "${CCEC_FLAGS[@]}" --cce-aicore-arch=dav-c310-cube \ - "${cross_tu_inc_flags[@]}" \ - -o "$BUILD_DIR/${tag}_runtime_aic.o" "$SCRIPT_DIR/nested_lambda_cross_tu_runtime.cpp" - kernel_objects+=( - "$BUILD_DIR/${tag}_caller_aic.o" - "$BUILD_DIR/${tag}_runtime_aic.o" - ) + kernel_objects+=("$BUILD_DIR/${tag}_caller_aic.o") + if [[ "$ks" == "nested_lambda_inline_plus_empty_runtime.cpp" ]]; then + echo "=== [$tag] Compiling empty runtime TU for AIC (dav-c310-cube) ===" + "$CCEC" "${CCEC_FLAGS[@]}" --cce-aicore-arch=dav-c310-cube \ + "${cross_tu_inc_flags[@]}" \ + -o "$BUILD_DIR/${tag}_runtime_empty_aic.o" \ + "$SCRIPT_DIR/nested_lambda_cross_tu_runtime.cpp" + kernel_objects+=("$BUILD_DIR/${tag}_runtime_empty_aic.o") + fi + local expected_objects=1 + if [[ "$ks" == "nested_lambda_inline_plus_empty_runtime.cpp" ]]; then + expected_objects=2 + local section_table runtime_text_size + if ! section_table="$(readelf -S -W "$BUILD_DIR/${tag}_runtime_empty_aic.o")"; then + echo "Error: failed to read empty runtime object sections" >&2 + exit 1 + fi + runtime_text_size="$(awk '$3 == ".text" {print $7}' <<< "$section_table")" + if [[ "$runtime_text_size" != "000000" ]]; then + echo "Error: runtime object control must have an empty .text section" >&2 + exit 1 + fi + fi + if [[ "${#kernel_objects[@]}" -ne "$expected_objects" ]]; then + echo "Error: unexpected caller-capture AIC input object count" >&2 + exit 1 + fi else echo "=== [$tag] Compiling AIV-only (dav-c310-vec) ===" "$CCEC" "${CCEC_FLAGS[@]}" --cce-aicore-arch=dav-c310-vec \ @@ -167,6 +198,34 @@ build_one() { "$LD" -m aicorelinux -Ttext=0 -static --allow-multiple-definition \ -o "$BUILD_DIR/$ko" "${kernel_objects[@]}" + if [[ "$ks" == "nested_lambda_cross_tu.cpp" || + "$ks" == "nested_lambda_inline_plus_empty_runtime.cpp" || + "$ks" == "nested_lambda_only_weak_submit_noinline.cpp" ]]; then + local symbol_table submit_symbols + if ! symbol_table="$(readelf -Ws -W "$BUILD_DIR/$ko")"; then + echo "Error: failed to read caller-capture symbol table" >&2 + exit 1 + fi + submit_symbols="$( + awk '$4 == "FUNC" && $8 ~ /nested_probe_submit_/ {sub(/\$local$/, "", $8); print $8}' \ + <<< "$symbol_table" | + sort -u + )" + if [[ "$ks" == "nested_lambda_only_weak_submit_noinline.cpp" ]]; then + if [[ "$(printf '%s\n' "$submit_symbols" | sed '/^$/d' | wc -l)" -ne 1 || + "$submit_symbols" != *nested_probe_submit_weak_context* ]]; then + echo "Error: noinline control must retain only nested_probe_submit_weak_context" >&2 + exit 1 + fi + elif [[ -n "$submit_symbols" ]]; then + echo "Error: inline caller-capture probe retained runtime submit symbols" >&2 + exit 1 + fi + echo "[ASSERT] CCEC caller-capture runtime symbol shape PASS" + echo "[VALUES] aic_input_objects=${#kernel_objects[@]}" \ + "runtime_text=${runtime_text_size:-n/a} submit_symbols=${submit_symbols:-none}" + fi + echo "=== [$tag] Compiling host ===" g++ -O2 -std=c++17 \ -I"$ASCEND_HOME_PATH/include" \ @@ -185,24 +244,27 @@ run_one() { local probe_failures=0 tag="$(basename "$ko" .o)" echo "=== Running [$tag] ===" - if [[ "$tag" == "nested_lambda_cross_tu_kernel" ]]; then + if [[ "$tag" == "nested_lambda_cross_tu_kernel" || + "$tag" == "nested_lambda_inline_plus_empty_runtime_kernel" || + "$tag" == "nested_lambda_only_weak_submit_noinline_kernel" ]]; then if [[ -n "${ATOMIC_PROBE_MODE:-}" ]]; then probe_modes=("$ATOMIC_PROBE_MODE") - else - # Run each variant in its own process: an expected AICore exception - # in a bad compiler variant must not hide the remaining controls. + elif [[ "$tag" == "nested_lambda_only_weak_submit_noinline_kernel" ]]; then probe_modes=( strong-context - args-runtime-read - weak-args-storage - weak-context-materialize-3 - weak-context-materialize-2 weak-context-materialize-1 weak-context-materialize-0 ) + elif [[ "$tag" == "nested_lambda_inline_plus_empty_runtime_kernel" ]]; then + probe_modes=(weak-context-materialize-0) + else + probe_modes=( + args-runtime-read + weak-context-materialize-0 + ) fi for probe_mode in "${probe_modes[@]}"; do - echo "--- CCEC cross-TU variant=$probe_mode ---" + echo "--- CCEC caller-capture variant=$probe_mode ---" if ! timeout "$RUN_TIMEOUT" \ "$BUILD_DIR/$hb" "$BUILD_DIR/$ko" "$probe_mode"; then probe_failures=$((probe_failures + 1)) @@ -370,7 +432,9 @@ export LD_LIBRARY_PATH="$ASCEND_HOME_PATH/x86_64-linux/lib64:${LD_LIBRARY_PATH:- selected=0 suite_run_failures=0 entries=("${PROBES[@]}") -if [[ "$SELECT" == "nested_lambda_cross_tu" ]]; then +if [[ "$SELECT" == "nested_lambda_cross_tu" || + "$SELECT" == "nested_lambda_inline_plus_empty_runtime" || + "$SELECT" == "nested_lambda_only_weak_submit_noinline" ]]; then entries+=("${MANUAL_PROBES[@]}") fi for entry in "${entries[@]}"; do diff --git a/tests/atomic_probe/cpu/nested_lambda_args_runtime_read.cpp b/tests/atomic_probe/cpu/nested_lambda_args_runtime_read.cpp index 83213282f4..3547ed71b2 100644 --- a/tests/atomic_probe/cpu/nested_lambda_args_runtime_read.cpp +++ b/tests/atomic_probe/cpu/nested_lambda_args_runtime_read.cpp @@ -16,18 +16,16 @@ namespace { -void InitTensor(Tensor &tensor, uint32_t round, uint32_t tensor_index) -{ +void InitTensor(Tensor &tensor, uint32_t round, uint32_t tensor_index) { tensor.buffer.addr = nested_lambda_cross_tu_probe::TensorAddress(round, tensor_index); tensor.start_offset = nested_lambda_cross_tu_probe::TensorOffset(round, tensor_index); tensor.version = nested_lambda_cross_tu_probe::TensorVersion(round, tensor_index); tensor.shapes[0] = nested_lambda_cross_tu_probe::TensorShape(round, tensor_index); } -} // namespace +} // namespace -int main() -{ +int main() { L0TaskArgs args; uint32_t completed_rounds = 0; uint32_t mismatches = 0; @@ -68,16 +66,15 @@ int main() completed_rounds++; } - const bool exact = completed_rounds == nested_lambda_cross_tu_probe::kRounds && - mismatches == 0 && checksum == nested_lambda_cross_tu_probe::ExpectedTotalChecksum() && - sizeof(L0TaskArgs) > 0 && sizeof(L0TaskArgs) < 32U * 1024U && sizeof(L0TaskArgs) % 64U == 0; + const bool exact = completed_rounds == nested_lambda_cross_tu_probe::kRounds && mismatches == 0 && + checksum == nested_lambda_cross_tu_probe::ExpectedTotalChecksum() && + sizeof(L0TaskArgs) == nested_lambda_cross_tu_probe::kExpectedL0TaskArgsBytes; std::printf("=== CPU L0TaskArgs Runtime-Read Probe ===\n"); std::printf( - "[VALUES] rounds=%u mismatches=%u checksum=0x%016llx L0TaskArgs=%zuB\n", - completed_rounds, mismatches, static_cast(checksum), sizeof(L0TaskArgs)); - std::printf( - "[ASSERT] CPU L0TaskArgs args-runtime-read semantics %s\n", - exact ? "PASS" : "FAIL"); + "[VALUES] rounds=%u mismatches=%u checksum=0x%016llx L0TaskArgs=%zuB\n", completed_rounds, mismatches, + static_cast(checksum), sizeof(L0TaskArgs) + ); + std::printf("[ASSERT] CPU L0TaskArgs args-runtime-read semantics %s\n", exact ? "PASS" : "FAIL"); std::printf("[SUMMARY] semantic_failures=%u\n", exact ? 0U : 1U); return exact ? 0 : 1; } diff --git a/tests/atomic_probe/run_nested_lambda.sh b/tests/atomic_probe/run_nested_lambda.sh index 2babab2a27..22b83c50e6 100755 --- a/tests/atomic_probe/run_nested_lambda.sh +++ b/tests/atomic_probe/run_nested_lambda.sh @@ -1,8 +1,16 @@ #!/usr/bin/env bash +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- # Run the minimal nested-lambda/capture/template compiler probe on one or all # of the CPU, AscendC, and pure-CCEC paths. # -# Usage: ./run_nested_lambda.sh [cpu|ascendc|ccec|ccec-cross-tu|all] +# Usage: ./run_nested_lambda.sh [cpu|ascendc|ccec|ccec-caller-capture|all] set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" @@ -33,7 +41,6 @@ run_cpu() { -I"$REPO_ROOT/src/a5/runtime/fully_distributed_within_core/orchestration" \ -I"$REPO_ROOT/src/a5/runtime" \ "$SCRIPT_DIR/cpu/nested_lambda_args_runtime_read.cpp" \ - "$SCRIPT_DIR/ccec/nested_lambda_cross_tu_runtime.cpp" \ -Wl,--gc-sections \ -o "$BUILD_DIR/nested_lambda_args_runtime_read_cpu" "$BUILD_DIR/nested_lambda_args_runtime_read_cpu" @@ -62,13 +69,13 @@ run_ccec() { "$SCRIPT_DIR/ccec/run_all.sh" nested_lambda } -run_ccec_cross_tu() { +run_ccec_caller_capture() { require_cann if [[ -z "${PTO_ISA_ROOT:-}" ]] && \ [[ -f "$ASCEND_HOME_PATH/x86_64-linux/include/pto/common/kernel_meta.hpp" ]]; then export PTO_ISA_ROOT="$ASCEND_HOME_PATH/x86_64-linux" fi - echo "=== Pure CCEC AIC cross-TU ABI path ===" + echo "=== Pure CCEC AIC inline caller-capture path ===" "$SCRIPT_DIR/ccec/run_all.sh" nested_lambda_cross_tu } @@ -82,16 +89,17 @@ ascendc) ccec) run_ccec ;; -ccec-cross-tu) - run_ccec_cross_tu +ccec-caller-capture | ccec-cross-tu) + run_ccec_caller_capture ;; all) run_cpu run_ascendc run_ccec + run_ccec_caller_capture ;; *) - echo "Usage: $0 [cpu|ascendc|ccec|ccec-cross-tu|all]" >&2 + echo "Usage: $0 [cpu|ascendc|ccec|ccec-caller-capture|all]" >&2 exit 2 ;; esac diff --git a/tests/atomic_probe/test_atomic_probe.py b/tests/atomic_probe/test_atomic_probe.py index 65a8dc22cf..b6b5f934f8 100644 --- a/tests/atomic_probe/test_atomic_probe.py +++ b/tests/atomic_probe/test_atomic_probe.py @@ -87,12 +87,19 @@ def test_a5_ccec_cacheline_probes(st_device_ids: list[int]) -> None: @pytest.mark.platforms(["a5"]) @pytest.mark.device_count(1) @pytest.mark.timeout(900) -def test_a5_ccec_nested_lambda_args_runtime_read(st_device_ids: list[int]) -> None: +def test_a5_ccec_nested_lambda_call_boundary_controls(st_device_ids: list[int]) -> None: environment = _onboard_environment(st_device_ids) - environment["ATOMIC_PROBE_MODE"] = "args-runtime-read" subprocess.run( - [str(HERE / "run_nested_lambda.sh"), "ccec-cross-tu"], - cwd=HERE, + [str(HERE / "ccec" / "run_all.sh"), "nested_lambda_cross_tu", "build"], + cwd=HERE / "ccec", env=environment, check=True, ) + for mode in ("args-runtime-read", "weak-context-materialize-0"): + environment["ATOMIC_PROBE_MODE"] = mode + subprocess.run( + [str(HERE / "ccec" / "run_all.sh"), "nested_lambda_cross_tu", "run"], + cwd=HERE / "ccec", + env=environment, + check=True, + ) From 290dbda08666d9cf7f19e1fb46c375d6d16081ac Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Fri, 17 Jul 2026 15:38:04 +0000 Subject: [PATCH 004/214] =?UTF-8?q?test(a5):=20=E5=A2=9E=E5=8A=A0=E7=8B=AC?= =?UTF-8?q?=E7=AB=8B=20PA=20=E8=B0=83=E5=BA=A6=E6=80=A7=E8=83=BD=E5=A4=8D?= =?UTF-8?q?=E7=8E=B0=E7=94=A8=E4=BE=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 增加共享 PA 调度模型及 CCEC、AscendC、CPU 三种独立后端。 支持可配置 NOP、阶段 profiling、本地泳道采集与标准库转换。 迁移 PA atomic 分析,补充独立复现指南和关键路径中文注释。 --- ...05\345\206\265\345\210\206\346\236\220.md" | 42 +- ...77\347\224\250\346\214\207\345\215\227.md" | 328 +++++ .../pa_scheduler/ascendc/build.sh | 53 + .../pa_scheduler/ascendc/pa_scheduler.asc | 362 +++++ tests/atomic_probe/pa_scheduler/ccec/build.sh | 114 ++ tests/atomic_probe/pa_scheduler/ccec/host.cpp | 272 ++++ .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 154 +++ .../pa_scheduler/common/host_support.h | 965 ++++++++++++++ .../pa_scheduler/common/pa_frontend.h | 1162 +++++++++++++++++ .../pa_scheduler/common/pa_model.h | 567 ++++++++ .../pa_scheduler/common/pa_scheduler_core.h | 788 +++++++++++ .../pa_scheduler/common/pa_trace.h | 158 +++ tests/atomic_probe/pa_scheduler/cpu/build.sh | 35 + tests/atomic_probe/pa_scheduler/cpu/main.cpp | 243 ++++ tests/atomic_probe/pa_scheduler/run.sh | 195 +++ .../pa_scheduler/swimlane_converter.py | 289 ++++ 16 files changed, 5726 insertions(+), 1 deletion(-) rename "tests/atomic_probe/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" => "tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" (85%) create mode 100644 "tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" create mode 100755 tests/atomic_probe/pa_scheduler/ascendc/build.sh create mode 100644 tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc create mode 100755 tests/atomic_probe/pa_scheduler/ccec/build.sh create mode 100644 tests/atomic_probe/pa_scheduler/ccec/host.cpp create mode 100644 tests/atomic_probe/pa_scheduler/ccec/kernel.cpp create mode 100644 tests/atomic_probe/pa_scheduler/common/host_support.h create mode 100644 tests/atomic_probe/pa_scheduler/common/pa_frontend.h create mode 100644 tests/atomic_probe/pa_scheduler/common/pa_model.h create mode 100644 tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h create mode 100644 tests/atomic_probe/pa_scheduler/common/pa_trace.h create mode 100755 tests/atomic_probe/pa_scheduler/cpu/build.sh create mode 100644 tests/atomic_probe/pa_scheduler/cpu/main.cpp create mode 100755 tests/atomic_probe/pa_scheduler/run.sh create mode 100755 tests/atomic_probe/pa_scheduler/swimlane_converter.py diff --git "a/tests/atomic_probe/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" similarity index 85% rename from "tests/atomic_probe/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" rename to "tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 5397debd87..98d46bbbcf 100644 --- "a/tests/atomic_probe/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -31,7 +31,7 @@ Submit 路径,供后续继续优化。快照日期为 2026-07-17,当前代 fanin ready 轮询。 环境安装、编译和基线复现过程见 -[A5 FDWIC Paged Attention 安装与复现指南](A5_FDWIC_PAGED_ATTENTION_REPRO.md)。 +[A5 FDWIC Paged Attention 安装与复现指南](../A5_FDWIC_PAGED_ATTENTION_REPRO.md)。 ## 2. Case1 工作量与 atomic 语义 @@ -298,3 +298,43 @@ Submit 调度而非 kernel 计算变快。 `Materialize/PrepareMap/Claim/Fanin/Register` 相加。可加总的是显式互斥 span,或单独定义 `Submit.end - Build/Replay/Alloc.end` 为 action tail。 后续文档和脚本都应沿用这一口径。 + +## 6. 独立 PA 调度复现的对应关系 + +`pa_scheduler/` 下的 CCEC、AscendC 和 CPU 用例以本文的真实 PA 路径为模型, +目标是脱离 simpler 的编译和链接依赖后,仍能单独研究 Submit 调度性能。这里的 +“脱离 simpler”不表示删减 PA 调度逻辑。当前独立模型保留: + +- Case1 的 256 batch、Alloc/QK/SF/PV/UP 五 task 拓扑和 AIC/AIV active mask; +- 96 worker 全量回放、四分片 Claim cursor 和固定 73728 次 Claim atomicMax; +- TaskArgs/Tensor/TaskPayload/DistSubmitCtx 的关键 ABI 和真实 tag 扫描; +- materialize、TensorMap retire/lookup/insert、register、fanin、slot payload; +- EfDrain、Replay、WaitForSlot、HeapGuard、flag/vend/frontier 和最终 drain; +- 单 lane Case1 的 BlockWon 动态次数为零,以及真实泳道记录格式。 + +只有 QK/SF/PV/UP 的计算体由可控 NOP 模拟;NOP 默认值按本文最好真实泳道的 +44.170/53.729/27.626/1.565 us 校准。独立用例不会在 Claim、Register、 +PrepareMap 或等待路径中增加 NOP 来硬凑 5 ms。 + +当前严格校验覆盖 73,728 次 Claim、每 task 唯一 winner、1,024 个 kernel、 +TensorMap/heap 最终状态、fanin、flag、vend、frontier、cursor、ring placement +和每 worker 的前端操作次数。2026-07-17 三个 CCEC 独立进程首轮为 +4.846431/4.798260/4.830184 ms,中位数 4.830184 ms;AscendC 独立进程首轮为 +4.917014 ms,均为 PASS;真实最好泳道为 5.096685 ms。 +差异主要来自真实 orchestration 与 `dist_submit_impl` 跨翻译单元,而 standalone +共享实现会和固定任务图一起被编译器优化。为制造编译边界而做的强制 noinline、 +拆设备目标和全局 memory clobber 实验曾分别触发状态破坏、device exception 或 +明显 RingBp,均已回退。 + +四阶段诊断通过 `--profile-phases` 输出:Claim 和 EfDrain 每 worker 调用 +1280 次;WaitForSlot 由 1024 个 kernel winner 调用;HeapGuard 由每 batch 的 +Alloc/QK/SF/PV winner 调用,共 1024 次。当前代表性 CCEC 轮次的累计中位数为: + +| role | Claim | EfDrain | WaitForSlot | HeapGuard | +| ---- | ----: | ------: | ----------: | --------: | +| AIC | 470.503 us | 711.005 us | 234.063 us(全 AIC 43 次等待) | 21.349 us(约 1 次 heap 等待) | +| AIV | 533.755 us | 401.962 us | 0.067 us(无等待) | 3.723 us(约 1 次 heap 等待) | + +完整构建、参数、内存占用、冷热运行口径和脱仓复制方法见同目录 +[PA 调度器独立复现与泳道使用指南](PA调度器独立复现与泳道使用指南.md)。后续调度优化应先在该 +独立用例做协议回归和阶段定位,再回到真实 PA Case1 做最终性能确认。 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" new file mode 100644 index 0000000000..e0ad51a21a --- /dev/null +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -0,0 +1,328 @@ +# PA 调度器独立复现与泳道使用指南 + +## 1. 目标与边界 + +本目录复现的是 A5 FDWIC Paged Attention Case1 的 **PA Submit 调度行为**, +不是把 PA 简化成普通 NOP 并发压测。整个目录复制到 `simpler` 代码仓之外后, +不再包含对 simpler 头文件、库、Python、PyTorch 或虚拟环境的编译和链接依赖。 +泳道转换脚本也位于本目录,仅使用 Python 标准库,不 import `simpler_setup` +或任何目录外模块。 + +独立实现保留了当前 PA Case1 与调度性能相关的完整路径: + +- 32 个 AIC worker、64 个 AIV worker,物理启动比例为 1:2; +- 默认 256 个 batch,每 batch 依次提交 Alloc、QK、SF、PV、UP 五个 task; +- 96 个 worker 各自回放 1,280 次 Submit,共 122,880 次 Submit; +- Alloc 由 96 个 worker 竞争,QK/PV 由 32 个 AIC 竞争,SF/UP 由 64 个 AIV 竞争; +- 4 路 Alloc/cube/vector Claim cursor,以及实际 `atomicMax` Claim; +- PA 的 TaskArgs、Tensor、TaskPayload、DistSubmitCtx、DistCore/DistGlobal 关键 ABI 布局; +- tensor tag 扫描、输出 layout、materialize、TensorMap retire/lookup/insert、register mask; +- fanin 收集、winner/loser、Replay、私有 ring slot 构造和 tensor/scalar payload 拷贝; +- EfDrain、WaitForSlot、HeapGuard、completion flag、vend、frontier、最终 drain; +- 与真实 PA 相同的单 lane 优化:Case1 不执行 BlockWon 轮询; +- 与真实泳道格式对齐的阶段记录及严格的结束状态校验。 + +默认工作量固定产生 73,728 次 Claim、1,280 个 winner 和 1,024 次 kernel +执行。每次运行都会校验这些数量以及最终 TensorMap、heap、cursor、flag、vend、 +frontier 和 worker 状态,任一不符都会返回失败。 + +有意保留的替代只有两类: + +1. QK/SF/PV/UP 的真实计算体由可控 NOP 数模拟,使每个 task 的占用时间接近 + 真实 PA;调度前后路径没有用 NOP 补时。 +2. simpler 的 AICPU/runtime 装载链路由本目录 host runner 代替;测试关注的 + 首个 Submit 到最后一个 Submit 区间不含 AICPU 初始化和最终回收。 + +该对等范围只覆盖当前 PA Case1 的全单-lane 图,不宣称覆盖通用 FDWIC 的 +joint/mixed task。若未来加入需要两个及以上 lane 联合执行的 task,必须补回 +BlockWon 发布、claim、drain 和剩余计数协议后再谈语义对等。 + +因此,约 5 ms 是性能参考,不是通过条件。不能为了命中 5 ms 而在 Claim、 +Register、PrepareMap 或等待路径中插入虚假延时。 + +## 2. 三种实现 + +三种后端共用 `common/` 中完全相同的 PA 模型和调度器,只分别实现原子指令、 +时钟、NOP 和启动入口。 + +| 后端 | 启动形式 | atomic load | Claim fetch-max | +| --- | --- | --- | --- | +| CCEC | 1:2 mixed AIC/AIV ELF | `atomicAdd(addr, 0)` | `atomicMax` | +| AscendC | `__mix__(1, 2)` | `AtomicAdd(addr, 0)` | `AtomicMax` | +| CPU | 96 个 pthread | `fetch_add(0)` | C++17 CAS loop | + +AscendC 对 64 位 vend 使用 signed `AtomicAdd(addr, 0)`。CANN 9.1 虽提供 +unsigned overload,但它在本 mixed kernel 的 64 MiB heap wrap 位置发生过 +稳定停滞;PA vend 小于 `INT64_MAX`,因此 signed add-zero 返回的位模式与比较 +语义不变。CPU 版本用于协议和边界检查,host 线程调度耗时不能与 A5 比较。 + +## 3. 默认 kernel 时间 + +真实 PA 最好泳道中四类 kernel 的 1 GHz counter 均值和当前 A5 NOP 校准值为: + +| Kernel | 目标时间 | 默认 NOP 数 | +| --- | ---: | ---: | +| QK | 44.170 us | 129,600 | +| SF | 53.729 us | 157,900 | +| PV | 27.626 us | 79,950 | +| UP | 1.565 us | 2,400 | + +Alloc 没有模拟 kernel body。NOP 数是 A5 实测校准量,不应解释为 A5 cycle 数。 +可以在运行时覆盖: + +```bash +./run.sh run ccec --nop-count 100000 +./run.sh run ccec --nop-counts 129600,157900,79950,2400 +``` + +`--nop-count N` 同时设置四类 kernel;`--nop-counts` 的顺序固定为 +QK、SF、PV、UP,允许范围为 0 到 10,000,000。 + +## 4. 本机依赖和构建 + +CCEC 与 AscendC 使用本用户安装的 CANN 9.1。非交互 shell 不保证读取 +`~/.bashrc`,复现时建议显式执行: + +```bash +cd /path/to/pa_scheduler + +source /home/q00473782/Ascend/cann-9.1.0-weekly-20260708/cann/set_env.sh + +export GCC15_ROOT=/home/q00473782/.local/gcc-15/root +export PATH="$GCC15_ROOT/usr/bin:$PATH" +export LD_LIBRARY_PATH="$GCC15_ROOT/usr/lib/x86_64-linux-gnu:$GCC15_ROOT/usr/lib/gcc/x86_64-linux-gnu/15${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" +export CXX="$GCC15_ROOT/usr/bin/g++-15" + +./run.sh build all +``` + +组合构建严格按 CCEC、AscendC、CPU 的顺序执行。CCEC 构建会检查 1:2 mixed +的两个入口和 metadata section;CANN 9.1 自带的 PTO 头可直接使用。若换用单独 +安装的 PTO ISA,可把 `PTO_ISA_ROOT` 指向包含 +`include/pto/common/kernel_meta.hpp` 的目录。 + +## 5. 使用说明:运行、测量与泳道查看 + +以下命令均在 `pa_scheduler` 目录执行。首次使用应先按第 4 节 source CANN +环境并完成构建。四个 action 的用途如下: + +| action | 用途 | 是否生成泳道文件 | +| --- | --- | --- | +| `build` | 构建指定后端 | 否 | +| `smoke` | 1 batch、1 run、零 NOP 的快速语义回归 | 否,只做内存记录校验 | +| `run` | 自行控制 batch、run、NOP 和诊断参数 | 仅显式传入 `--swimlane-json` 时生成 raw | +| `swimlane` | 单轮运行并自动生成 raw 和 Perfetto merged JSON | 是 | + +`ccec|ascendc|cpu|all` 用于选择后端;`all` 始终按 CCEC、AscendC、CPU +的顺序执行。 + +### 5.1 首次回归 + +先做三后端快速语义回归: + +```bash +./run.sh smoke all --device 0 +``` + +通过标准是所有 `[ASSERT]` 为 `PASS`,最终同时出现: + +```text +semantic_status=PASS postprocess_status=PASS +``` + +`semantic_status` 表示 PA 调度协议和终态校验结果;`postprocess_status` 表示 +泳道读取、导出或转换等后处理结果。任何一个为 `FAIL`,进程都会返回非零。 + +### 5.2 只运行 benchmark 和文字诊断 + +再在真实 A5 上测完整 CCEC 或 AscendC: + +```bash +./run.sh run ccec \ + --device 0 --batches 256 --runs 5 \ + --profile-phases --analyze-swimlane + +./run.sh run ascendc \ + --device 0 --batches 256 --runs 5 \ + --profile-phases --analyze-swimlane +``` + +此模式输出指标和泳道统计,但不会自动落盘 JSON。若只关注性能且不需要完整 +泳道逐事件分析,可去掉 `--analyze-swimlane`;若也不需要记录泳道,可使用 +`--no-swimlane` 进一步节省约 384 MiB device 内存。 + +### 5.3 生成并查看泳道 + +生成可直接载入 Perfetto 的泳道文件时使用独立的 `swimlane` action: + +```bash +./run.sh swimlane ccec \ + --device 0 --batches 256 --profile-phases --analyze-swimlane +``` + +`swimlane` action 会管理 `--runs 1` 和输出路径,因此不要再传 +`--runs`、`--swimlane-json` 或 `--no-swimlane`。转换器默认使用 `python3`;如需 +固定到用户自己的 Python,可在命令前设置: + +```bash +export PYTHON=/path/to/venv/bin/python +``` + +转换器只使用 Python 标准库,不需要 PyTorch,也不需要安装 simpler Python 包。 + +该 action 固定执行一轮,产物全部位于本目录的 +`outputs/pa_scheduler_swimlane__/ccec/`: + +- `l2_swimlane_records.json`:与真实 PA 相同的十列 `fdwic_events` 原始格式; +- `merged_swimlane.json`:Chrome Trace Event 格式,拖入 + 即可查看泳道。 + +runner 结束时会打印准确目录: + +```text +[SWIMLANE] output_root=.../outputs/pa_scheduler_swimlane__ +``` + +查看步骤: + +1. 打开 ; +2. 将 `merged_swimlane.json` 拖入页面,不要拖原始的 + `l2_swimlane_records.json`; +3. 每个 `block0` 至 `block31` 是一个物理 1AIC+2AIV block; +4. `AIC`、`AIV0`、`AIV1` 轨展示 Submit、Claim、EfDrain、Replay、RingBp 等 + runtime 阶段,带 `·kernel` 的轨展示 QK、SF、PV、UP; +5. 点击事件可查看 `task_id`、`func_id`、`core`、`mc` 和 `aux`。 + +WaitForSlot 和 HeapGuard 没有可伪造的逐事件起止时间,因此不单独生成 Perfetto +事件;它们由 `--profile-phases` 的 `[PHASE]` 累计统计呈现。实际发生等待时, +泳道中会出现 RingBp 事件。 + +`outputs/` 已被 Git 忽略,生成的几十至数百 MiB 泳道文件不会被普通 +`git add` 意外纳入提交。 + +### 5.4 手工导出或重新转换 + +转换完全由本目录脚本完成,不依赖 simpler 的 Python 包或虚拟环境。已有原始 +文件也可单独转换: + +```bash +python3 ./swimlane_converter.py \ + ./outputs//ccec/l2_swimlane_records.json \ + -o ./outputs//ccec/merged_swimlane.json +``` + +若只需要原始记录,可通过通用 `run` action 显式指定文件;导出为避免多轮覆盖 +而要求 `--runs 1`: + +```bash +mkdir -p ./outputs/manual +./run.sh run ccec --device 0 --batches 256 --runs 1 \ + --swimlane-json ./outputs/manual/l2_swimlane_records.json +``` + +手工 `--swimlane-json` 只生成 raw,不会自动生成 merged;需要随后调用上面的 +`swimlane_converter.py`。该参数强制要求 `--runs 1`,避免多轮静默覆盖同一文件。 + +### 5.5 CPU 回归、参数与测量口径 + +CPU 完整协议回归建议关闭大泳道缓冲区: + +```bash +./run.sh run cpu \ + --batches 256 --runs 1 --nop-count 0 \ + --profile-phases --no-swimlane +``` + +主要选项: + +- `--profile-phases`:分别统计 Claim、EfDrain、WaitForSlot、HeapGuard; +- `--analyze-swimlane`:读取完整记录,输出各阶段的 per-worker 累计分布以及 + EfDrain/Materialize/Claim/Register 的 per-role、per-task-kind 单事件分布; +- `--swimlane-json FILE`:流式导出原始 `fdwic_events` JSON,要求单轮运行; +- `--no-swimlane`:关闭泳道记录,不能与 `--analyze-swimlane` 或 + `--swimlane-json` 同时使用; +- `--runs N`:同一进程和同一已装载 kernel 中连续运行 N 次; +- `smoke`:1 batch、1 run、零 NOP,但仍启动全部 96 个 worker 并执行全部校验。 + +`submit_span_us` 的定义与当前 PA 基线口径一致: + +```text +96 个 worker 中最早的第一个 Submit.begin + -> 96 个 worker 中最晚的最后一个 Submit.end +``` + +它排除启动屏障和最终 drain。`host_launch_us` 另外包含 host launch、最终 drain +和 stream/thread 同步。不同版本比较时,必须同时开启或同时关闭 +`--profile-phases` 和泳道,因为计时与记录本身会影响竞争时序。 + +`[PHASE]` 的每个阶段都是每 worker 在 1,280 次 Submit 中的累计时间: + +- Claim:当前 worker 实际参与或跳过对应 lane Claim 的完整 span; +- EfDrain:每次 Submit 开头执行已就绪私有 slot 的时间; +- WaitForSlot:仅 1,024 个 kernel winner 调用,额外给出发生等待的事件数; +- HeapGuard:Alloc/QK/SF/PV 的 1,024 个输出 winner 调用,额外给出 heap + 等待事件数。 + +WaitForSlot 和 HeapGuard 的 `calls_total` 会按 winner 所在角色分布,AIC/AIV +相加必须分别等于 1,024;等待事件则对应额外的 RingBp 泳道记录。 + +## 6. 当前 A5 结果与真实 PA 的差异 + +2026-07-17 当前源码的一轮代表性结果如下。所有严格校验均为 PASS,kernel +时间使用上表附近的校准 NOP: + +| 实现 | 首轮 `submit_span_us` | EfDrain/RingBp/FinalDrain | +| --- | ---: | ---: | +| 真实 simpler PA 最好泳道 | 5,096.685 us | 1011 / 0 / 13 | +| standalone CCEC,3 个独立进程首轮中位数 | 4,830.184 us | 961 / 50 / 12 | +| standalone AscendC,独立进程首轮 | 4,917.014 us | 1009 / 4 / 11 | + +三次 CCEC 独立进程首轮分别为 4,846.431、4,798.260、4,830.184 us;这里 +报告中位数,不挑最好值。`--runs N` 的后续轮次会复用进程、device binary 和 +已分配内存,而真实 PA 的 5.1 ms 来自完整测试进程的一轮泳道,因此做基线 +比较时应优先比较独立进程首轮,不能把热运行中位数混作同一口径。 + +四个重点阶段的一轮 CCEC 代表值为: + +| role | Claim | EfDrain | WaitForSlot | HeapGuard | +| --- | ---: | ---: | ---: | ---: | +| AIC 每 worker 累计中位数 | 470.503 us | 711.005 us | 234.063 us,43 次等待 | 21.349 us,约 1 次等待 | +| AIV 每 worker 累计中位数 | 533.755 us | 401.962 us | 0.067 us,0 次等待 | 3.723 us,约 1 次等待 | + +这里的“等待次数”是三轮中对应角色的全局中位数,不是每 worker 次数。Claim +和 EfDrain 已与真实 PA 同量级;HeapGuard 只有 0 至 3 次偶发等待。当前主要 +残差来自编译边界: +真实 orchestration 和 `dist_submit_impl` 位于不同翻译单元,TaskArgs 对 Submit +编译器是运行时数据;standalone 为了保持可复制构建,共享实现会与固定 PA +任务图一起优化,导致 PrepareMap/Register 等前端阶段的指令生成不同。CCEC +的 AIC 到达顺序还会让约 39 至 54 个 kernel 进入 WaitForSlot/RingBp;AscendC +同轮只有 4 个,说明这部分主要是后端 codegen 触发的时序放大,不是缺少完成协议。 + +已经验证过的强制 `noinline`、拆设备目标文件和全局 compiler memory clobber +分别造成状态破坏、A5 device exception 或明显 RingBp,均未保留。它们不是 +可靠的 PA 语义模拟。当前选择是保持源级协议和 ABI 对等,坦诚记录约 +0.1 至 0.4 ms 的后端/冷热差异,不用虚假 NOP 填平调度阶段。 + +## 7. 内存占用和脱仓复制 + +为保持真实 DistGlobal/DistCore 偏移、65,536 个 task cell、每 worker payload +和 TensorMap,`SchedulerState` 为 1,007,092,544 bytes。默认泳道缓冲区另占 +402,660,160 bytes,所以 A5 device 侧总计约 1.31 GiB,host 侧也需分配相近 +内存。`smoke` 不缩小 State;只有 `--no-swimlane` 能省去泳道缓冲区。 +256 batch 的正常采集约有 86 万条事件;原始 JSON 和 merged JSON 都可能达到 +数十至数百 MiB。writer 与 converter 都使用临时文件后原子替换,失败时不会把 +半截文件冒充完整产物。 + +脱离 simpler 时必须复制整个目录,因为三个后端共用 `common/`: + +```bash +cp -a tests/atomic_probe/pa_scheduler /tmp/pa_scheduler +cd /tmp/pa_scheduler +./run.sh build cpu +./run.sh smoke cpu +``` + +CCEC/AscendC 只需再 source CANN 环境。本目录的构建脚本不会搜索 Git 根目录, +也不会引用 `simpler/src`、`simpler/examples` 或其他仓内文件。泳道转换只需 +Python 3 标准库;复制后的 `./run.sh swimlane ...` 仍使用当前目录内的 +`swimlane_converter.py`。 diff --git a/tests/atomic_probe/pa_scheduler/ascendc/build.sh b/tests/atomic_probe/pa_scheduler/ascendc/build.sh new file mode 100755 index 0000000000..d97517229e --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ascendc/build.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" +BUILD_DIR="$ROOT_DIR/build/ascendc" + +# 所有源码和公共头都从 pa_scheduler 目录解析;外部只需要用户安装的 +# CANN 工具链和运行库,不搜索 simpler 仓库根目录。 + +if [[ -z "${ASCEND_HOME_PATH:-}" ]]; then + # 非交互 shell 不保证自动 source CANN 环境,缺失时直接失败,避免误用 + # PATH 中其他版本的 bisheng。 + echo "ASCEND_HOME_PATH is not set; source the CANN 9.1 set_env.sh first." >&2 + exit 1 +fi + +BISHENG="$ASCEND_HOME_PATH/bin/bisheng" +if [[ ! -x "$BISHENG" ]]; then + # 只接受当前 ASCEND_HOME_PATH 下的编译器,确保构建与运行库来自同一套 CANN。 + echo "bisheng is missing under ASCEND_HOME_PATH=$ASCEND_HOME_PATH" >&2 + exit 1 +fi + +# build 产物固定放回 pa_scheduler/build/ascendc;目录可重复创建,重新构建会 +# 原位替换可执行文件,不向源码目录散落中间产物。 +mkdir -p "$BUILD_DIR" + +echo "[BUILD] AscendC 1:2 mixed host + kernel executable" +# -xasc 会把本文件中的 host main 与 AscendC kernel 构建为一个可执行文件。 +# __MIX_CORE_AIC_RATION__ 的拼写来自 bisheng mixed-core ABI;值 2 必须与 +# kernel 上的 __mix__(1, 2) 一致,否则 launch metadata 不能表达 32+64 拓扑。 +# 调度器已经在协议边界显式执行 dcci,关闭自动 scalar DCCI 可避免编译器 +# 额外插入 cache 操作并扰动待测的 atomic/Submit 时序。 +"$BISHENG" -O3 -xasc \ + "$SCRIPT_DIR/pa_scheduler.asc" \ + --npu-arch=dav-3510 \ + -D__MIX_CORE_AIC_RATION__=2 \ + -I"$ROOT_DIR/common" \ + -mllvm -cce-aicore-dcci-insert-for-scalar=false \ + -mllvm -cce-aicore-dcci-before-kernel-end=false \ + -o "$BUILD_DIR/pa_scheduler_ascendc" + +echo "[BUILD] complete: $BUILD_DIR/pa_scheduler_ascendc" diff --git a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc new file mode 100644 index 0000000000..e2207bc2ed --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc @@ -0,0 +1,362 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#include "../common/host_support.h" + +#include "acl/acl.h" +#include "kernel_operator.h" + +#include +#include +#include +#include +#include + +using namespace AscendC; + +#define PA_DEVICE __aicore__ inline +#define PA_GM __gm__ +// 通过两个宏把公共调度器实例化为 AscendC 设备代码;公共头本身不依赖 +// kernel_operator.h,也不会为 AscendC 复制一套调度协议。 +#include "../common/pa_scheduler_core.h" + +namespace { + +template +__aicore__ inline void EmitNops() { +// Count 必须是编译期常量,展开后才能稳定保留对应数量的 AICore nop 指令。 +#pragma unroll + for (uint32_t index = 0; index < Count; ++index) { + asm volatile("nop"); + } +} + +__aicore__ inline void RuntimeNop(uint32_t count) { + // AscendC 的 Nop() 不能接收运行时参数,因此先按 256 指令分块, + // 再用二进制尾块拼出任意 count。两端 PIPE_ALL 屏障把模拟计算体与 + // 前后调度访存隔开;count 是本机校准量,不等同于硬件 cycle 数。 + PipeBarrier(); + while (count >= 256) { + EmitNops<256>(); + count -= 256; + } + if ((count & 128U) != 0) EmitNops<128>(); + if ((count & 64U) != 0) EmitNops<64>(); + if ((count & 32U) != 0) EmitNops<32>(); + if ((count & 16U) != 0) EmitNops<16>(); + if ((count & 8U) != 0) EmitNops<8>(); + if ((count & 4U) != 0) EmitNops<4>(); + if ((count & 2U) != 0) EmitNops<2>(); + if ((count & 1U) != 0) EmitNops<1>(); + PipeBarrier(); +} + +struct AscendcOps { + // 公共调度器把 Load 定义为“具有原子一致性的读”。这里坚持使用 + // AtomicAdd(addr, 0),以保留真实 PA 在热点 cache line 上的竞争形态。 + __aicore__ static inline int32_t Load(__gm__ volatile int32_t *address) { + return AtomicAdd(const_cast<__gm__ int32_t *>(address), static_cast(0)); + } + + __aicore__ static inline int64_t Load(__gm__ volatile int64_t *address) { + return AtomicAdd(const_cast<__gm__ int64_t *>(address), static_cast(0)); + } + + __aicore__ static inline uint64_t Load(__gm__ volatile uint64_t *address) { + // CANN 9.1 虽接受 AtomicAdd,但该重载在本 mixed kernel + // 第一次跨过 64 MiB heap wrap 时可稳定停滞。PA vend 始终小于 + // INT64_MAX,因此对同一 64 位地址执行有符号 add-zero,不改变位模式 + // 和比较语义,同时仍保留原子读路径。 + __gm__ int64_t *signed_address = reinterpret_cast<__gm__ int64_t *>(const_cast<__gm__ uint64_t *>(address)); + return static_cast(AtomicAdd(signed_address, static_cast(0))); + } + + __aicore__ static inline int32_t Exchange(__gm__ volatile int32_t *address, int32_t value) { + // AtomicExch 公开重载使用无符号类型;这里只转换位表示,不做数值换算。 + __gm__ uint32_t *unsigned_address = reinterpret_cast<__gm__ uint32_t *>(const_cast<__gm__ int32_t *>(address)); + return static_cast(AtomicExch(unsigned_address, static_cast(value))); + } + + __aicore__ static inline int64_t Exchange(__gm__ volatile int64_t *address, int64_t value) { + __gm__ uint64_t *unsigned_address = reinterpret_cast<__gm__ uint64_t *>(const_cast<__gm__ int64_t *>(address)); + return static_cast(AtomicExch(unsigned_address, static_cast(value))); + } + + __aicore__ static inline uint64_t Exchange(__gm__ volatile uint64_t *address, uint64_t value) { + return AtomicExch(const_cast<__gm__ uint64_t *>(address), value); + } + + __aicore__ static inline int64_t FetchAdd(__gm__ volatile int64_t *address, int64_t value) { + return AtomicAdd(const_cast<__gm__ int64_t *>(address), value); + } + + __aicore__ static inline int64_t FetchMax(__gm__ volatile int64_t *address, int64_t value, uint64_t &retries) { + // A5 直接提供硬件 AtomicMax,不需要像 CPU CAS loop 那样统计软件重试。 + retries = 0; + return AtomicMax(const_cast<__gm__ int64_t *>(address), value); + } + + // A5 PA 契约中的 OUT_OF_ORDER_STORE_BARRIER 在此为空:completion 的 vend/flag + // 依靠前后 AtomicExch 协议,config 与 trace 的 cache 可见性则分别由下方 dcci/dsb 处理。 + // 不额外插入全局屏障,以免改变真实 PA 的热路径。 + __aicore__ static inline void StoreBarrier() {} + + // A5 SYS_CNT 为 1 GHz;公共模型也以 1 GHz tick 记录泳道时间。 + __aicore__ static inline uint64_t Now() { return static_cast(GetSystemCycle()); } + + __aicore__ static inline void Nop(uint32_t count) { RuntimeNop(count); } + + __aicore__ static inline void SpinHint() {} + + __aicore__ static inline void InvalidateRegion(__gm__ const void *address, uint64_t bytes) { + // 当前用于失效 host 写入的 standalone config cache line,再以 dsb 保证 + // 后续普通 GM 读看到配置;worker 间共享 task 状态走 atomic 路径。 + if (bytes == 0) return; + const uint64_t start = reinterpret_cast(address) & ~uint64_t{63}; + const uint64_t end = (reinterpret_cast(address) + bytes + 63) & ~uint64_t{63}; + for (uint64_t current = start; current < end; current += 64) { + dcci(reinterpret_cast<__gm__ uint8_t *>(current), SINGLE_CACHE_LINE); + } + dsb((mem_dsb_t)0); + } + + __aicore__ static inline void FlushRegion(__gm__ void *address, uint64_t bytes) { + // 泳道记录通过普通 GM cache 写入,kernel 结束前逐 cache line clean-out, + // 随后的 dsb 保证 host 在 stream 同步后可以读取完整记录。 + if (bytes == 0) return; + __asm__ volatile("" ::: "memory"); + const uint64_t start = reinterpret_cast(address) & ~uint64_t{63}; + const uint64_t end = (reinterpret_cast(address) + bytes + 63) & ~uint64_t{63}; + for (uint64_t current = start; current < end; current += 64) { + dcci(reinterpret_cast<__gm__ uint8_t *>(current), SINGLE_CACHE_LINE, CACHELINE_OUT); + } + dsb((mem_dsb_t)0); + } + + __aicore__ static inline void Publish(__gm__ uint64_t *address, uint64_t value) { + // worker 结果使用 bypass-DCache 写,避免与仍驻留在标量 cache 中的旧值混合。 + WriteGmByPassDCache(address, value); + } +}; + +// ACL host API 的统一错误门:任何资源创建、拷贝、同步失败都立即带标签返回, +// 上层 main 不会在半初始化的 device 状态上继续启动 kernel。 +bool CheckAcl(aclError error, const char *label) { + if (error == ACL_SUCCESS) return true; + std::fprintf(stderr, "ACL error %d: %s\n", static_cast(error), label); + return false; +} + +} // namespace + +// 一个 mixed block 固定包含 1 个 AIC 与 2 个 AIV。host 启动 32 个 block, +// 因而形成 32 AIC + 64 AIV。编译时还必须由 build.sh 设置 +// __MIX_CORE_AIC_RATION__=2,使 binary metadata 与 __mix__(1, 2) 一致。 +__schedmode__(1) __global__ __mix__(1, 2) void pa_scheduler_ascendc(__gm__ pa_scheduler::SchedulerState *state) { +#if defined(__DAV_VEC__) + // AIV 侧 GetBlockIdx() 已展平为 0..63,平移 32 后得到公共协议中的 + // worker 32..95;AIC 侧保持原始 block 号 0..31。 + const uint32_t worker_id = pa_scheduler::kAicWorkers + static_cast(GetBlockIdx()); + pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aiv); +#else + const uint32_t worker_id = static_cast(GetBlockIdx()); + pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aic); +#endif +} + +// AscendC 可执行文件同时承载 host runner 和 mixed kernel。正常执行路径为: +// 解析参数 -> 初始化 ACL/stream -> 分配 GM -> 逐轮 launch/校验/后处理 -> +// 释放 GM/stream/device;初始化或传输等早期错误仍会就地返回。性能区间只包围 +// launch 与 stream synchronize。 +int32_t main(int32_t argc, char **argv) { + pa_scheduler::host::Options options; + const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions(argc, argv, false, &options); + if (parse_status != pa_scheduler::host::ParseStatus::Ok) { + return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; + } + pa_scheduler::host::PrintBanner("AscendC", options); + + // ACL 初始化和选卡必须先于 stream、GM 以及 kernel stub 的任何使用。 + if (!CheckAcl(aclInit(nullptr), "aclInit") || !CheckAcl(aclrtSetDevice(options.device), "aclrtSetDevice")) { + return EXIT_FAILURE; + } + aclrtStream stream = nullptr; + if (!CheckAcl(aclrtCreateStream(&stream), "aclrtCreateStream")) return EXIT_FAILURE; + + // SchedulerState 保留被测关键 offset、DistCore ABI 和生产总跨度;单独检查 + // 64 字节对齐,保证其中每条 atomic/cache-line 状态线不会错位。 + pa_scheduler::SchedulerState *state_device = nullptr; + if (!CheckAcl( + aclrtMalloc( + reinterpret_cast(&state_device), sizeof(pa_scheduler::SchedulerState), + ACL_MEM_MALLOC_HUGE_FIRST + ), + "aclrtMalloc(state)" + )) { + return EXIT_FAILURE; + } + if ((reinterpret_cast(state_device) & 63U) != 0) { + std::fprintf(stderr, "Device state is not 64-byte aligned: %p\n", static_cast(state_device)); + return EXIT_FAILURE; + } + + // trace 是独立 GM 区域,关闭泳道时完全不分配;开启时 header 与每个 + // worker 的固定跨度 records 共用同一块连续内存。 + void *trace_device = nullptr; + if (options.trace_enabled && + !CheckAcl( + aclrtMalloc(&trace_device, pa_scheduler::kTraceBytes, ACL_MEM_MALLOC_HUGE_FIRST), + "aclrtMalloc(swimlane trace)" + )) { + return EXIT_FAILURE; + } + if (options.trace_enabled && (reinterpret_cast(trace_device) & 63U) != 0) { + std::fprintf(stderr, "Device swimlane trace is not 64-byte aligned: %p\n", trace_device); + return EXIT_FAILURE; + } + + std::unique_ptr state(new pa_scheduler::SchedulerState); + pa_scheduler::TraceHeader trace_header{}; + std::vector spans; + bool all_passed = true; + bool postprocess_ok = true; + // 多轮复用已加载的 binary、stream 和 GM 分配,只重置协议状态与 trace + // header;因此热轮次不能冒充独立进程首轮性能。 + for (uint32_t run = 1; run <= options.runs; ++run) { + pa_scheduler::host::InitializeState(state.get(), options); + pa_scheduler::host::ConfigureTrace(state.get(), options, trace_device); + if (options.trace_enabled) { + pa_scheduler::host::InitializeTraceHeader(&trace_header); + if (!CheckAcl( + aclrtMemcpy( + trace_device, sizeof(trace_header), &trace_header, sizeof(trace_header), + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D swimlane header)" + )) { + return EXIT_FAILURE; + } + } + // 只传输初始化所需的 state prefix 和 standalone controls,避免每轮 + // 把约 1 GiB 的 worker 私有区从 host 全量拷入 device。 + if (!CheckAcl( + aclrtMemcpy( + state_device, pa_scheduler::host::StatePrefixBytes(), state.get(), + pa_scheduler::host::StatePrefixBytes(), ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D state prefix)" + ) || + !CheckAcl( + aclrtMemcpy( + &state_device->config, pa_scheduler::host::ControlBytes(), &state->config, + pa_scheduler::host::ControlBytes(), ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D standalone controls)" + )) { + return EXIT_FAILURE; + } + + // launch 维度是 mixed block 数而不是总 worker 数;__mix__(1, 2) + // 会从这 32 个 block 派生出全部 96 个参与者。 + const auto wall_begin = std::chrono::steady_clock::now(); + pa_scheduler_ascendc<<>>(state_device); + if (!CheckAcl(aclrtSynchronizeStream(stream), "aclrtSynchronizeStream")) return EXIT_FAILURE; + const auto wall_end = std::chrono::steady_clock::now(); + const double host_us = std::chrono::duration(wall_end - wall_begin).count(); + // kernel 完成后只回读全局前缀和 96 份结果;大 worker 私有区仍留在 + // device。trace 也先回读小 header,再按实际 count 分块读取 records。 + if (!CheckAcl( + aclrtMemcpy( + state.get(), pa_scheduler::host::StatePrefixBytes(), state_device, + pa_scheduler::host::StatePrefixBytes(), ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H state prefix)" + ) || + !CheckAcl( + aclrtMemcpy( + state->results, pa_scheduler::host::ResultBytes(), &state_device->results[0], + pa_scheduler::host::ResultBytes(), ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H worker results)" + )) { + return EXIT_FAILURE; + } + if (options.trace_enabled && + !CheckAcl( + aclrtMemcpy( + &trace_header, sizeof(trace_header), trace_device, sizeof(trace_header), + ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H swimlane header)" + )) { + return EXIT_FAILURE; + } + // trace buffer 按 worker 分成固定跨度;分析和 JSON 导出复用同一个 + // 逐 worker D2H 回调,不依赖 simpler 的采集器实现。 + const auto read_trace_records = + [trace_device](uint32_t worker, uint32_t count, pa_scheduler::TraceRecord *records) { + const uint64_t offset = sizeof(pa_scheduler::TraceHeader) + + static_cast(worker) * pa_scheduler::kTraceRecordsPerCore * + sizeof(pa_scheduler::TraceRecord); + return CheckAcl( + aclrtMemcpy( + records, static_cast(count) * sizeof(pa_scheduler::TraceRecord), + static_cast(trace_device) + offset, + static_cast(count) * sizeof(pa_scheduler::TraceRecord), + ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H swimlane records)" + ); + }; + // 语义校验是产物门禁:拓扑、cursor、flag、vend、frontier 和每 worker + // 结果全部通过后,才允许把本轮 records 导出成 raw JSON。 + const pa_scheduler::host::Metrics metrics = pa_scheduler::host::Validate( + *state, run, host_us, options.trace_enabled ? &trace_header : nullptr + ); + all_passed &= metrics.passed; + spans.push_back(metrics.submit_span_us); + if (options.analyze_swimlane && + !pa_scheduler::host::AnalyzeSwimlaneRecords(trace_header, *state, read_trace_records)) { + postprocess_ok = false; + break; + } + if (!options.swimlane_json.empty()) { + if (!metrics.passed) { + std::fprintf(stderr, "Skipping swimlane export because semantic validation failed.\n"); + postprocess_ok = false; + break; + } + if (!pa_scheduler::host::ExportSwimlaneRecords( + trace_header, options.swimlane_json, read_trace_records + )) { + postprocess_ok = false; + break; + } + } + } + + std::printf( + "[SUMMARY] runs=%u median_submit_span_us=%.3f semantic_status=%s postprocess_status=%s\n", options.runs, + pa_scheduler::host::Median(spans), all_passed ? "PASS" : "FAIL", postprocess_ok ? "PASS" : "FAIL" + ); + + // 进入统一清理阶段后继续累计错误而不提前返回,尽量释放本阶段持有的资源; + // 最终退出码同时包含语义、后处理和清理三类状态。 + bool cleanup_ok = true; + if (trace_device != nullptr) { + cleanup_ok &= CheckAcl(aclrtFree(trace_device), "aclrtFree(swimlane trace)"); + } + cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(state)"); + cleanup_ok &= CheckAcl(aclrtDestroyStream(stream), "aclrtDestroyStream"); + cleanup_ok &= CheckAcl(aclrtResetDevice(options.device), "aclrtResetDevice"); + cleanup_ok &= CheckAcl(aclFinalize(), "aclFinalize"); + return all_passed && postprocess_ok && cleanup_ok ? EXIT_SUCCESS : EXIT_FAILURE; +} diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh new file mode 100755 index 0000000000..f6d5e7d6ea --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +# 任一工具失败、未定义变量或管道中间失败都立即终止,避免继续使用半成品 device ELF。 +set -euo pipefail + +# 所有输入和产物都从脚本自身位置解析,调用者无需位于仓库根目录。 +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" +BUILD_DIR="$ROOT_DIR/build/ccec" + +# 编译只依赖本目录源码与用户安装的 CANN/PTO 头,不引用 pa_scheduler 目录外的 simpler 构建产物。 +if [[ -z "${ASCEND_HOME_PATH:-}" ]]; then + echo "ASCEND_HOME_PATH is not set; source the CANN 9.1 set_env.sh first." >&2 + exit 1 +fi + +CCEC="$ASCEND_HOME_PATH/bin/ccec" +LD="$ASCEND_HOME_PATH/bin/ld.lld" +CXX_BIN="${CXX:-g++}" +READELF_BIN="${READELF:-readelf}" +PTO_INCLUDE_ROOT="${PTO_ISA_ROOT:-$ASCEND_HOME_PATH/x86_64-linux}" + +# ccec/ld.lld 必须来自当前已 source 的 CANN;host 编译器和 readelf 允许用户通过环境变量替换。 +if [[ ! -x "$CCEC" || ! -x "$LD" ]]; then + echo "CCEC or ld.lld is missing under ASCEND_HOME_PATH=$ASCEND_HOME_PATH" >&2 + exit 1 +fi +if ! command -v "$READELF_BIN" >/dev/null 2>&1; then + echo "readelf is required to verify the mixed AICore ELF." >&2 + exit 1 +fi +if [[ ! -f "$PTO_INCLUDE_ROOT/include/pto/common/kernel_meta.hpp" ]]; then + echo "PTO kernel metadata header is missing under $PTO_INCLUDE_ROOT/include" >&2 + exit 1 +fi + +mkdir -p "$BUILD_DIR" + +# 关闭编译器自动插入的 scalar DCCI,由 kernel.cpp 中与 PA 对齐的显式失效/回写协议负责 cache 可见性。 +# 两种架构共用这些 ABI、栈和优化参数,避免 AIC/AIV 对共享 SchedulerState 产生不同解释。 +COMMON_FLAGS=( + -c -O3 -g -x cce -Wall -std=c++17 + --cce-aicore-only + -mllvm -cce-aicore-stack-size=0x8000 + -mllvm -cce-aicore-function-stack-size=0x8000 + -mllvm -cce-aicore-record-overflow=false + -mllvm -cce-aicore-addr-transform + -mllvm -cce-aicore-dcci-insert-for-scalar=false + -mllvm -cce-aicore-dcci-before-kernel-end=false + -I"$ROOT_DIR/common" + -I"$PTO_INCLUDE_ROOT/include" +) + +# 同一入口源码分别面向 cube 与 vector ISA 编译,宏只选择各自的全局入口和 mixed metadata。 +echo "[BUILD] CCEC AIC entry (dav-c310-cube)" +"$CCEC" "${COMMON_FLAGS[@]}" \ + --cce-aicore-arch=dav-c310-cube \ + -DPA_BUILD_AIC \ + -o "$BUILD_DIR/pa_scheduler_aic.o" \ + "$SCRIPT_DIR/kernel.cpp" + +echo "[BUILD] CCEC AIV entry (dav-c310-vec)" +"$CCEC" "${COMMON_FLAGS[@]}" \ + --cce-aicore-arch=dav-c310-vec \ + -DPA_BUILD_AIV \ + -o "$BUILD_DIR/pa_scheduler_aiv.o" \ + "$SCRIPT_DIR/kernel.cpp" + +# 静态链接把两个 device object 合成一个可由 runtime 按 1:2 比例启动的 mixed AICore ELF。 +echo "[BUILD] Static 1:2 mixed AICore ELF" +"$LD" -m aicorelinux -Ttext=0 -static \ + -o "$BUILD_DIR/pa_scheduler_kernel.o" \ + "$BUILD_DIR/pa_scheduler_aic.o" \ + "$BUILD_DIR/pa_scheduler_aiv.o" + +SYMBOL_TABLE="$("$READELF_BIN" --symbols --wide "$BUILD_DIR/pa_scheduler_kernel.o")" +SECTION_TABLE="$("$READELF_BIN" --sections --wide "$BUILD_DIR/pa_scheduler_kernel.o")" +# 构建成功不等于 mixed launch 可用:同时检查两个入口符号及其 metadata section,缺一即拒绝产物。 +# `set -e` 同时保证 readelf 自身失败时不会拿空字符串继续做伪检查。 +for entry in pa_scheduler_0_mix_aic pa_scheduler_0_mix_aiv; do + if [[ "$SYMBOL_TABLE" != *" $entry"* ]]; then + echo "Missing mixed-kernel entry: $entry" >&2 + exit 1 + fi + if [[ "$SECTION_TABLE" != *".ascend.meta.$entry"* ]]; then + echo "Missing mixed-kernel metadata section: .ascend.meta.$entry" >&2 + exit 1 + fi +done +echo "[CHECK] both 1:2 mixed entries and metadata sections are present" + +# host runner 只链接用户 CANN 9.1 的 ACL/runtime,并写入同一安装目录的 rpath,运行时不需要 simpler 动态库。 +# `-Werror` 让 host API 签名或尺寸类型变化在构建期暴露,避免到上板阶段才出现参数截断。 +echo "[BUILD] CCEC host runner" +"$CXX_BIN" -O2 -std=c++17 -Wall -Wextra -Werror -Wno-deprecated-declarations \ + -I"$ROOT_DIR/common" \ + -I"$ASCEND_HOME_PATH/include" \ + -I"$ASCEND_HOME_PATH/pkg_inc" \ + -I"$ASCEND_HOME_PATH/pkg_inc/runtime" \ + "$SCRIPT_DIR/host.cpp" \ + -L"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ + -Wl,-rpath,"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ + -lascendcl -lruntime \ + -o "$BUILD_DIR/pa_scheduler_host" + +echo "[BUILD] complete: $BUILD_DIR" diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp new file mode 100644 index 0000000000..360383d6aa --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -0,0 +1,272 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#include "../common/host_support.h" + +#include "acl/acl.h" +#include "runtime/rt.h" + +#include +#include +#include +#include +#include +#include +#include + +namespace { + +bool CheckAcl(aclError error, const char *label) { + if (error == ACL_SUCCESS) return true; + std::fprintf(stderr, "ACL error %d: %s\n", static_cast(error), label); + return false; +} + +bool CheckRt(rtError_t error, const char *label) { + if (error == RT_ERROR_NONE) return true; + std::fprintf(stderr, "RT error %d: %s\n", static_cast(error), label); + return false; +} + +std::vector ReadBinary(const std::string &path) { + // ELF 整体保存在 vector 中直到 runtime 卸载完成,保证 rtDevBinary_t.data 在整个注册生命周期内有效。 + std::ifstream file(path, std::ios::binary | std::ios::ate); + if (!file) return {}; + const std::streamsize size = file.tellg(); + if (size <= 0) return {}; + std::vector data(static_cast(size)); + file.seekg(0, std::ios::beg); + if (!file.read(data.data(), size)) return {}; + return data; +} + +} // namespace + +int main(int argc, char **argv) { + // 参数和 ELF 在创建 ACL 资源前完成校验,早期错误不会留下 device、stream 或 kernel handle。 + pa_scheduler::host::Options options; + const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions(argc, argv, true, &options); + if (parse_status != pa_scheduler::host::ParseStatus::Ok) { + return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; + } + const std::vector binary_data = ReadBinary(options.kernel_path); + if (binary_data.empty()) { + std::fprintf(stderr, "Cannot read kernel binary: %s\n", options.kernel_path.c_str()); + return EXIT_FAILURE; + } + pa_scheduler::host::PrintBanner("CCEC", options); + + // 正常及后处理路径依次完成 ACL 初始化、选卡、stream/ELF/设备区创建、launch/D2H + // 和尾部清理;初始化、传输或 launch 的早期错误仍按当前实现就地返回。 + if (!CheckAcl(aclInit(nullptr), "aclInit") || !CheckAcl(aclrtSetDevice(options.device), "aclrtSetDevice")) { + return EXIT_FAILURE; + } + aclrtStream stream = nullptr; + if (!CheckAcl(aclrtCreateStream(&stream), "aclrtCreateStream")) return EXIT_FAILURE; + + rtDevBinary_t binary{RT_DEV_BINARY_MAGIC_ELF, 0, binary_data.data(), binary_data.size()}; + void *kernel_handle = nullptr; + bool registered_all = true; + // 先尝试注册带 mixed metadata 的 ELF;若 rtRegisterAllKernel 报错或未返回 handle, + // 再尝试无 tiling-key 装载。这里仅描述实际回退条件,不假设具体运行时原因。 + rtError_t register_error = rtRegisterAllKernel(&binary, &kernel_handle); + if (register_error != RT_ERROR_NONE || kernel_handle == nullptr) { + registered_all = false; + register_error = rtBinaryLoadWithoutTilingKey(binary_data.data(), binary_data.size(), &kernel_handle); + } + if (!CheckRt(register_error, "register mixed AICore ELF") || kernel_handle == nullptr) return EXIT_FAILURE; + + // SchedulerState 保留被测关键 offset、DistCore ABI 和约 1 GiB 生产总跨度; + // 使用 HUGE_FIRST 降低大块设备内存碎片风险。 + void *state_device = nullptr; + if (!CheckAcl( + aclrtMalloc(&state_device, sizeof(pa_scheduler::SchedulerState), ACL_MEM_MALLOC_HUGE_FIRST), + "aclrtMalloc(state)" + )) { + return EXIT_FAILURE; + } + if ((reinterpret_cast(state_device) & 63U) != 0) { + std::fprintf(stderr, "Device state is not 64-byte aligned: %p\n", state_device); + return EXIT_FAILURE; + } + + // 泳道区按 96 worker 各 65536 条记录预留,约 384 MiB;关闭泳道时不申请,也不会传递有效 base。 + void *trace_device = nullptr; + if (options.trace_enabled && + !CheckAcl( + aclrtMalloc(&trace_device, pa_scheduler::kTraceBytes, ACL_MEM_MALLOC_HUGE_FIRST), + "aclrtMalloc(swimlane trace)" + )) { + return EXIT_FAILURE; + } + if (options.trace_enabled && (reinterpret_cast(trace_device) & 63U) != 0) { + std::fprintf(stderr, "Device swimlane trace is not 64-byte aligned: %p\n", trace_device); + return EXIT_FAILURE; + } + + // host shadow 保留约 1 GiB 总跨度以便按关键 offset 寻址,但每轮传输只选择 + // 共享前缀、控制区和结果区。 + std::unique_ptr state(new pa_scheduler::SchedulerState); + pa_scheduler::TraceHeader trace_header{}; + std::vector spans; + bool all_passed = true; + bool postprocess_ok = true; + for (uint32_t run = 1; run <= options.runs; ++run) { + pa_scheduler::host::InitializeState(state.get(), options); + pa_scheduler::host::ConfigureTrace(state.get(), options, trace_device); + if (options.trace_enabled) { + // 每轮只需重置约 7 KiB header;各 worker 会从 count=0 覆盖自己的记录区,无需清零整块 384 MiB。 + pa_scheduler::host::InitializeTraceHeader(&trace_header); + if (!CheckAcl( + aclrtMemcpy( + trace_device, sizeof(trace_header), &trace_header, sizeof(trace_header), + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D swimlane header)" + )) { + return EXIT_FAILURE; + } + } + // 为避免每轮搬运约 1 GiB,只 H2D 被测共享前缀和位于生产总跨度之后的 + // standalone 控制区; + // 每个 worker 的大块私有状态由 device kernel 自行初始化。 + if (!CheckAcl( + aclrtMemcpy( + state_device, pa_scheduler::host::StatePrefixBytes(), state.get(), + pa_scheduler::host::StatePrefixBytes(), ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D state prefix)" + ) || + !CheckAcl( + aclrtMemcpy( + &static_cast(state_device)->config, + pa_scheduler::host::ControlBytes(), &state->config, pa_scheduler::host::ControlBytes(), + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D standalone controls)" + )) { + return EXIT_FAILURE; + } + + void *kernel_args[] = {state_device}; + rtArgsEx_t args_info{}; + args_info.args = kernel_args; + args_info.argsSize = sizeof(kernel_args); + rtTaskCfgInfo_t task_config{}; + // launch 维度是 32 个物理 mixed block;ELF metadata 让每个 block 同时产生 1 AIC + 2 AIV,共 96 worker。 + // wall time 在同步完成处截止,包含 launch、完整调度、最终 drain 和 stream 同步,但不包含后续 D2H/JSON。 + const auto wall_begin = std::chrono::steady_clock::now(); + if (!CheckRt( + rtKernelLaunchWithHandleV2( + kernel_handle, 0, pa_scheduler::kAicWorkers, &args_info, nullptr, stream, &task_config + ), + "rtKernelLaunchWithHandleV2" + ) || + !CheckAcl(aclrtSynchronizeStream(stream), "aclrtSynchronizeStream")) { + return EXIT_FAILURE; + } + const auto wall_end = std::chrono::steady_clock::now(); + const double host_us = std::chrono::duration(wall_end - wall_begin).count(); + // D2H 同样避开约 1 GiB 的 worker arena:共享前缀用于 flag/vend/frontier 校验,末尾 results 单独回传。 + if (!CheckAcl( + aclrtMemcpy( + state.get(), pa_scheduler::host::StatePrefixBytes(), state_device, + pa_scheduler::host::StatePrefixBytes(), ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H state prefix)" + ) || + !CheckAcl( + aclrtMemcpy( + state->results, pa_scheduler::host::ResultBytes(), + &static_cast(state_device)->results[0], + pa_scheduler::host::ResultBytes(), ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H worker results)" + )) { + return EXIT_FAILURE; + } + if (options.trace_enabled && + !CheckAcl( + aclrtMemcpy( + &trace_header, sizeof(trace_header), trace_device, sizeof(trace_header), + ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H swimlane header)" + )) { + return EXIT_FAILURE; + } + // 常规校验只需 header 中的 per-worker count;真实 records 在分析或导出时才按核、按实际 count 懒加载。 + const auto read_trace_records = + [trace_device](uint32_t worker, uint32_t count, pa_scheduler::TraceRecord *records) { + // 每核记录区采用固定容量 stride;只复制 header 声明的实际 count,避免 D2H 未使用的尾部空间。 + const uint64_t offset = sizeof(pa_scheduler::TraceHeader) + + static_cast(worker) * pa_scheduler::kTraceRecordsPerCore * + sizeof(pa_scheduler::TraceRecord); + return CheckAcl( + aclrtMemcpy( + records, static_cast(count) * sizeof(pa_scheduler::TraceRecord), + static_cast(trace_device) + offset, + static_cast(count) * sizeof(pa_scheduler::TraceRecord), + ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H swimlane records)" + ); + }; + // 先完成共享状态、拓扑、计数和 trace header 的语义校验,再允许 raw JSON 成为性能证据。 + const pa_scheduler::host::Metrics metrics = pa_scheduler::host::Validate( + *state, run, host_us, options.trace_enabled ? &trace_header : nullptr + ); + all_passed &= metrics.passed; + spans.push_back(metrics.submit_span_us); + if (options.analyze_swimlane && + !pa_scheduler::host::AnalyzeSwimlaneRecords(trace_header, *state, read_trace_records)) { + // 后处理错误使用 break 汇入统一 cleanup;与初始化/launch 失败的进程级立即返回语义区分开。 + postprocess_ok = false; + break; + } + if (!options.swimlane_json.empty()) { + // 只有语义校验通过才把 raw JSON 经“临时文件写完后 rename”发布, + // 避免把截断或错误调度结果误当成可用性能证据。 + if (!metrics.passed) { + std::fprintf(stderr, "Skipping swimlane export because semantic validation failed.\n"); + postprocess_ok = false; + break; + } + if (!pa_scheduler::host::ExportSwimlaneRecords( + trace_header, options.swimlane_json, read_trace_records + )) { + postprocess_ok = false; + break; + } + } + } + + std::printf( + "[SUMMARY] runs=%u median_submit_span_us=%.3f semantic_status=%s postprocess_status=%s\n", options.runs, + pa_scheduler::host::Median(spans), all_passed ? "PASS" : "FAIL", postprocess_ok ? "PASS" : "FAIL" + ); + + // 后处理失败也统一走设备资源释放、ELF 卸载和 ACL 收尾,避免文件系统错误遗留运行时上下文。 + bool cleanup_ok = true; + // 先释放依赖当前 device/context 的大块内存,再卸载 ELF、销毁 stream,最后 reset device 与 finalize ACL。 + if (trace_device != nullptr) { + cleanup_ok &= CheckAcl(aclrtFree(trace_device), "aclrtFree(swimlane trace)"); + } + cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(state)"); + const rtError_t unload_error = + registered_all ? rtDevBinaryUnRegister(kernel_handle) : rtBinaryUnLoad(kernel_handle); + cleanup_ok &= CheckRt(unload_error, "unload mixed AICore ELF"); + cleanup_ok &= CheckAcl(aclrtDestroyStream(stream), "aclrtDestroyStream"); + cleanup_ok &= CheckAcl(aclrtResetDevice(options.device), "aclrtResetDevice"); + cleanup_ok &= CheckAcl(aclFinalize(), "aclFinalize"); + // 运行语义、后处理和资源清理三者全部成功,进程才返回成功,脚本据此决定是否继续生成 merged 泳道。 + return all_passed && postprocess_ok && cleanup_ok ? EXIT_SUCCESS : EXIT_FAILURE; +} diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp new file mode 100644 index 0000000000..e9c95ddc7c --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -0,0 +1,154 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#include "cce_aicore_intrinsics.h" +#include + +#define PA_DEVICE __aicore__ inline +#define PA_GM __gm__ +#include "../common/pa_scheduler_core.h" + +namespace { + +template +__aicore__ inline void EmitNops() { +#pragma unroll + for (uint32_t index = 0; index < Count; ++index) { + asm volatile("nop"); + } +} + +__aicore__ inline void RuntimeNop(uint32_t count) { + // 两侧全流水屏障把可调 NOP 段限定为 kernel 模拟体,避免前后调度访存进入被测计算区间。 + __builtin_cce_pipe_barrier(PIPE_ALL); + while (count >= 256) { + EmitNops<256>(); + count -= 256; + } + if ((count & 128U) != 0) EmitNops<128>(); + if ((count & 64U) != 0) EmitNops<64>(); + if ((count & 32U) != 0) EmitNops<32>(); + if ((count & 16U) != 0) EmitNops<16>(); + if ((count & 8U) != 0) EmitNops<8>(); + if ((count & 4U) != 0) EmitNops<4>(); + if ((count & 2U) != 0) EmitNops<2>(); + if ((count & 1U) != 0) EmitNops<1>(); + __builtin_cce_pipe_barrier(PIPE_ALL); +} + +struct CcecOps { + // 该适配层把平台无关调度器需要的原子、计时、NOP 和 cache 操作逐一映射到 CCEC intrinsic。 + // A5 上 PA 的共享“读取”使用 atomicAdd(addr, 0),不是普通 GM load;这里保留其 RMW 竞争语义。 + __aicore__ static inline int32_t Load(__gm__ volatile int32_t *address) { + // atomicAdd 返回加法发生前的值;加数为 0,因此它就是本次共享读取的结果。 + return atomicAdd(const_cast<__gm__ int32_t *>(address), static_cast(0)); + } + + __aicore__ static inline int64_t Load(__gm__ volatile int64_t *address) { + return atomicAdd(const_cast<__gm__ int64_t *>(address), static_cast(0)); + } + + __aicore__ static inline uint64_t Load(__gm__ volatile uint64_t *address) { + return atomicAdd(const_cast<__gm__ uint64_t *>(address), static_cast(0)); + } + + __aicore__ static inline int32_t Exchange(__gm__ volatile int32_t *address, int32_t value) { + // atomicExch 同样返回旧值;当前 completion/fatal 发布只需要其原子写入副作用。 + return atomicExch(const_cast<__gm__ int32_t *>(address), value); + } + + __aicore__ static inline int64_t Exchange(__gm__ volatile int64_t *address, int64_t value) { + return atomicExch(const_cast<__gm__ int64_t *>(address), value); + } + + __aicore__ static inline uint64_t Exchange(__gm__ volatile uint64_t *address, uint64_t value) { + return atomicExch(const_cast<__gm__ uint64_t *>(address), value); + } + + __aicore__ static inline int64_t FetchAdd(__gm__ volatile int64_t *address, int64_t value) { + // 返回递增前的计数;启动和 replay 屏障只关心全局累加结果,因此调用方不使用该返回值。 + return atomicAdd(const_cast<__gm__ int64_t *>(address), value); + } + + __aicore__ static inline int64_t FetchMax(__gm__ volatile int64_t *address, int64_t value, uint64_t &retries) { + // CCEC 直接生成单条硬件 atomicMax,不存在 CPU CAS 循环可观测的重试次数。 + retries = 0; + // 返回更新前的 cursor/frontier,Claim 用它判定 winner,frontier 扫描用它吸收其他核的进度。 + return atomicMax(const_cast<__gm__ int64_t *>(address), value); + } + + // PA's A5 OUT_OF_ORDER_STORE_BARRIER is intentionally a no-op; cache + // coherency is handled by the runtime's DCCI protocol. + // 这是对生产 A5 契约的刻意复刻,不是遗漏 barrier;若在这里额外插入 dsb, + // 会改变待测 Submit 热路径。completion 使用 atomic,config/trace 的 cache + // 可见性则由各自既有的 DCCI 路径处理。 + __aicore__ static inline void StoreBarrier() {} + + __aicore__ static inline uint64_t Now() { return static_cast(get_sys_cnt()); } + + __aicore__ static inline void Nop(uint32_t count) { RuntimeNop(count); } + + // SPIN_WAIT_HINT is also a no-op in the real A5 inner-kernel contract. + // 同理不额外插入 nop,让等待循环保留真实 PA 内核“不主动退避”的指令成本。 + __aicore__ static inline void SpinHint() {} + + __aicore__ static inline void InvalidateRegion(__gm__ const void *address, uint64_t bytes) { + // 逐 cache line 失效并以 dsb 收口,供 worker 在启动时读取 host 刚写入的 standalone 控制区。 + if (bytes == 0) return; + const uint64_t start = reinterpret_cast(address) & ~uint64_t{63}; + const uint64_t end = (reinterpret_cast(address) + bytes + 63) & ~uint64_t{63}; + for (uint64_t current = start; current < end; current += 64) { + dcci(reinterpret_cast<__gm__ uint8_t *>(current), SINGLE_CACHE_LINE); + } + dsb((mem_dsb_t)0); + } + + __aicore__ static inline void FlushRegion(__gm__ void *address, uint64_t bytes) { + // 泳道记录先写普通 GM cache,kernel 结束前显式 CACHELINE_OUT,确保 host D2H 能看到完整记录。 + if (bytes == 0) return; + __asm__ volatile("" ::: "memory"); + const uint64_t start = reinterpret_cast(address) & ~uint64_t{63}; + const uint64_t end = (reinterpret_cast(address) + bytes + 63) & ~uint64_t{63}; + for (uint64_t current = start; current < end; current += 64) { + dcci(reinterpret_cast<__gm__ uint8_t *>(current), SINGLE_CACHE_LINE, CACHELINE_OUT); + } + dsb((mem_dsb_t)0); + } + + __aicore__ static inline void Publish(__gm__ uint64_t *address, uint64_t value) { + // 每核独占的 WorkerResult 用 bypass-DCache store 发布,host 同步后可直接 D2H,无需共享原子竞争。 + __builtin_cce_st_dev(value, address, 0); + } +}; + +} // namespace + +#if defined(PA_BUILD_AIC) +// 同一源码分别按 cube/vec 架构编译;metadata 声明每个物理 block 静态组合 1 个 AIC 与 2 个 AIV。 +PTO_SYNCALL_MIX_AIC_KERNEL_META(pa_scheduler_0_mix_aic, 1, 2); + +extern "C" __global__ __aicore__ void pa_scheduler_0_mix_aic(__gm__ pa_scheduler::SchedulerState *state) { + // 32 个物理 block 的 AIC 直接使用 block_idx,形成连续 worker 0..31。 + const uint32_t worker_id = static_cast(get_block_idx()); + pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aic); +} +#elif defined(PA_BUILD_AIV) +PTO_SYNCALL_MIX_AIC_KERNEL_META(pa_scheduler_0_mix_aiv, 1, 2); + +extern "C" __global__ __aicore__ void pa_scheduler_0_mix_aiv(__gm__ pa_scheduler::SchedulerState *state) { + // 每个 block 的两个 vector sub-block 展平为 vector_id=2*b+subblock,偏移 32 后形成 worker 32..95。 + const uint32_t vector_id = static_cast(get_block_idx() * get_subblockdim() + get_subblockid()); + const uint32_t worker_id = pa_scheduler::kAicWorkers + vector_id; + pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aiv); +} +#else +#error "Compile with PA_BUILD_AIC or PA_BUILD_AIV" +#endif diff --git a/tests/atomic_probe/pa_scheduler/common/host_support.h b/tests/atomic_probe/pa_scheduler/common/host_support.h new file mode 100644 index 0000000000..6b6bf42556 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/common/host_support.h @@ -0,0 +1,965 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_COMMON_HOST_SUPPORT_H +#define PA_SCHEDULER_COMMON_HOST_SUPPORT_H + +#include "pa_model.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +namespace pa_scheduler::host { + +// 三种后端共用同一套命令行配置,保证 CPU 语义回归与 A5 上板使用完全相同的工作量。 +struct Options { + std::string kernel_path; + std::string swimlane_json; + uint32_t device = 0; + uint32_t batches = kDefaultBatches; + uint32_t runs = 5; + NopCounts nops{kDefaultQkNops, kDefaultSfNops, kDefaultPvNops, kDefaultUpNops}; + bool profile_phases = false; + bool trace_enabled = true; + bool analyze_swimlane = false; +}; + +enum class ParseStatus { + Ok, + Help, + Error, +}; + +inline bool ParseUint(const char *raw, uint32_t minimum, uint32_t maximum, uint32_t *value) { + // 要求整串都能被 strtoul 解析且结果落在给定范围内,拒绝尾随字符和溢出值, + // 避免参数被部分解析后悄悄改变工作量。 + errno = 0; + char *end = nullptr; + const unsigned long parsed = std::strtoul(raw, &end, 10); + if (errno != 0 || end == raw || *end != '\0' || parsed < minimum || parsed > maximum) { + return false; + } + *value = static_cast(parsed); + return true; +} + +inline bool ParseNopCounts(const char *raw, NopCounts *counts) { + // 四类 kernel 的 NOP 数必须一次性完整给出,顺序固定为 QK、SF、PV、UP。 + unsigned int qk = 0; + unsigned int sf = 0; + unsigned int pv = 0; + unsigned int up = 0; + char tail = '\0'; + if (std::sscanf(raw, "%u,%u,%u,%u%c", &qk, &sf, &pv, &up, &tail) != 4) { + return false; + } + constexpr uint32_t kMaxNopCount = 10000000; + if (qk > kMaxNopCount || sf > kMaxNopCount || pv > kMaxNopCount || up > kMaxNopCount) { + return false; + } + *counts = NopCounts{qk, sf, pv, up}; + return true; +} + +inline void PrintUsage(const char *program, bool require_kernel) { + // require_kernel 只影响 CCEC host 的用法文本,其余 benchmark 参数在三后端完全一致。 + std::fprintf( + stderr, "Usage: %s%s [--device N] [--batches 1..256] [--runs N] ", program, + require_kernel ? " --kernel FILE" : "" + ); + std::fprintf( + stderr, + "[--nop-count N | --nop-counts QK,SF,PV,UP] [--profile-phases] [--analyze-swimlane] " + "[--swimlane-json FILE] [--no-swimlane]\n" + ); +} + +inline ParseStatus ParseOptions(int argc, char **argv, bool require_kernel, Options *options) { + // CCEC host 需要外部 kernel ELF;AscendC 和 CPU 的可执行文件已包含 kernel,因此不需要该参数。 + bool nop_override_seen = false; + bool swimlane_json_seen = false; + for (int index = 1; index < argc; ++index) { + // 无值开关先处理;其余参数统一在消费下一个 argv 前检查缺值,保证错误位置明确。 + const std::string argument = argv[index]; + if (argument == "--help" || argument == "-h") { + PrintUsage(argv[0], require_kernel); + return ParseStatus::Help; + } + if (argument == "--profile-phases") { + options->profile_phases = true; + continue; + } + if (argument == "--no-swimlane") { + options->trace_enabled = false; + continue; + } + if (argument == "--analyze-swimlane") { + options->analyze_swimlane = true; + continue; + } + if (index + 1 >= argc) { + std::fprintf(stderr, "Missing value after %s\n", argument.c_str()); + return ParseStatus::Error; + } + const char *value = argv[++index]; + if (argument == "--kernel" && require_kernel) { + options->kernel_path = value; + } else if (argument == "--device") { + if (!ParseUint(value, 0, INT32_MAX, &options->device)) return ParseStatus::Error; + } else if (argument == "--batches") { + if (!ParseUint(value, 1, kMaxBatches, &options->batches)) return ParseStatus::Error; + } else if (argument == "--runs") { + if (!ParseUint(value, 1, 1000, &options->runs)) return ParseStatus::Error; + } else if (argument == "--swimlane-json") { + if (swimlane_json_seen) { + std::fprintf(stderr, "Specify --swimlane-json only once.\n"); + return ParseStatus::Error; + } + if (*value == '\0') { + std::fprintf(stderr, "--swimlane-json requires a non-empty path.\n"); + return ParseStatus::Error; + } + options->swimlane_json = value; + swimlane_json_seen = true; + } else if (argument == "--nop-count") { + if (nop_override_seen) { + std::fprintf(stderr, "Specify only one NOP override.\n"); + return ParseStatus::Error; + } + uint32_t count = 0; + if (!ParseUint(value, 0, 10000000, &count)) return ParseStatus::Error; + options->nops = NopCounts{count, count, count, count}; + nop_override_seen = true; + } else if (argument == "--nop-counts") { + if (nop_override_seen) { + std::fprintf(stderr, "Specify only one NOP override.\n"); + return ParseStatus::Error; + } + if (!ParseNopCounts(value, &options->nops)) return ParseStatus::Error; + nop_override_seen = true; + } else { + std::fprintf(stderr, "Unknown argument: %s\n", argument.c_str()); + return ParseStatus::Error; + } + } + if (require_kernel && options->kernel_path.empty()) { + std::fprintf(stderr, "--kernel is required\n"); + return ParseStatus::Error; + } + if (options->analyze_swimlane && !options->trace_enabled) { + // 分析和导出都依赖完整 record 缓冲,不能与节省内存的 --no-swimlane 同时使用。 + std::fprintf(stderr, "--analyze-swimlane requires swimlane tracing.\n"); + return ParseStatus::Error; + } + if (!options->swimlane_json.empty() && !options->trace_enabled) { + std::fprintf(stderr, "--swimlane-json requires swimlane tracing.\n"); + return ParseStatus::Error; + } + if (!options->swimlane_json.empty() && options->runs != 1) { + // 一个文件只对应一次完整采集,禁止多轮运行反复覆盖而丢失轮次边界。 + std::fprintf(stderr, "--swimlane-json requires --runs 1 to avoid overwriting captures.\n"); + return ParseStatus::Error; + } + return ParseStatus::Ok; +} + +inline void InitializeState(SchedulerState *state, const Options &options) { + // WorkerState 有意保持真实 PA 每核约 9 MiB 的布局。若 host 每轮清空全部 worker, + // 会额外触碰并拷贝近 1 GiB 内存;因此只初始化全局前缀和结果区,worker 的活跃字段 + // 由各自 kernel 在启动后复位,这也与真实 PA 的生命周期一致。 + std::memset(state, 0, offsetof(SchedulerState, workers)); + std::memset(&state->config, 0, offsetof(SchedulerState, results) - offsetof(SchedulerState, config)); + std::memset(state->results, 0, sizeof(state->results)); + state->heap_window = kHeapWindow; + state->heap_base = kSyntheticHeapBase; + state->heap_size = kHeapBytes; + state->num_workers = kWorkers; + state->num_blocks = kAicWorkers; + state->config.batches = options.batches; + state->config.workers = kWorkers; + state->config.nops = options.nops; + state->config.profile_phases = options.profile_phases ? 1U : 0U; + for (uint32_t batch = 0; batch < options.batches; ++batch) { + state->context_lens[batch] = 8192; + } + for (uint32_t shard = 0; shard < kCursorShards; ++shard) { + // -1 表示尚无 task 被 claim;task 0 的 atomicMax 因而也能正常判定唯一 winner。 + state->cube_cursor[shard].value = -1; + state->vector_cursor[shard].value = -1; + state->alloc_cursor[shard].value = -1; + } + state->frontier.value = -1; +} + +inline void ConfigureTrace(SchedulerState *state, const Options &options, const void *trace_base) { + // device 只持有裸地址和每核容量;TraceHeader/record 缓冲区由 host 单独分配并初始化。 + state->config.trace_enabled = options.trace_enabled ? 1U : 0U; + state->config.trace_base = options.trace_enabled ? reinterpret_cast(trace_base) : 0; + state->config.trace_records_per_core = options.trace_enabled ? kTraceRecordsPerCore : 0; +} + +inline void InitializeTraceHeader(TraceHeader *header) { + // magic、版本、108 槽 header 布局和 1 GHz 频率均与真实 FDWIC 泳道 ABI 对齐。 + std::memset(header, 0, sizeof(*header)); + header->magic = 0x4653574cU; + header->version = 1; + header->num_cores = kWorkers; + header->records_per_core = kTraceRecordsPerCore; + header->frequency_hz = kSystemCounterHz; +} + +// 巨大的 WorkerState 不参与每轮 H2D/D2H;以下三个范围只搬运运行所需的前缀、控制量和结果。 +inline constexpr size_t StatePrefixBytes() { return offsetof(SchedulerState, workers); } + +inline constexpr size_t ControlBytes() { + // control sidecar 位于为生产 DistGlobal 保留的总跨度之后,到 results 之前为止。 + return offsetof(SchedulerState, results) - offsetof(SchedulerState, config); +} + +inline constexpr size_t ResultBytes() { return sizeof(WorkerResult) * kWorkers; } + +struct Metrics { + // passed 是全部语义断言的合取;submit_span_us 是本用例唯一用于对比 PA 的性能口径。 + bool passed = true; + double submit_span_us = 0; +}; + +inline void Expect(bool condition, const char *label, Metrics *metrics) { + // 所有断言都继续执行,以便一次失败运行尽可能暴露完整状态,而不是遇到首错立即退出。 + std::printf("[ASSERT] %-48s %s\n", label, condition ? "PASS" : "FAIL"); + if (!condition) metrics->passed = false; +} + +struct Uint64Distribution { + uint64_t total = 0; + double median = 0.0; + uint64_t p95 = 0; + uint64_t maximum = 0; +}; + +inline Uint64Distribution SummarizeUint64(std::vector values) { + // 这里按 worker 维度统计累计周期,p95 使用 nearest-rank,避免插值掩盖慢核。 + Uint64Distribution summary; + if (values.empty()) return summary; + + std::sort(values.begin(), values.end()); + for (uint64_t value : values) summary.total += value; + const size_t middle = values.size() / 2; + summary.median = (values.size() & 1U) != 0 + ? static_cast(values[middle]) + : (static_cast(values[middle - 1]) + static_cast(values[middle])) / 2.0; + const size_t p95_rank = (95U * values.size() + 99U) / 100U; + summary.p95 = values[p95_rank - 1]; + summary.maximum = values.back(); + return summary; +} + +inline void PrintPhaseDiagnostics(const SchedulerState &state) { + if (state.config.profile_phases == 0) return; + + // WaitForSlot/HeapGuard 没有各自独立命名的 TracePhase;实际发生等待时会写 + // RingBp 记录,汇总诊断则使用 WorkerResult 中的累计周期和等待次数。 + struct PhaseSpec { + ProfilePhase phase; + const char *name; + int32_t wait_event_index; + }; + const PhaseSpec phases[] = { + {ProfilePhase::Claim, "Claim", -1}, + {ProfilePhase::EfDrain, "EfDrain", -1}, + {ProfilePhase::WaitForSlot, "WaitForSlot", 0}, + {ProfilePhase::HeapGuard, "HeapGuard", 1}, + }; + const CoreRole roles[] = {CoreRole::Aic, CoreRole::Aiv}; + const char *role_names[] = {"AIC", "AIV"}; + + for (uint32_t role_index = 0; role_index < 2; ++role_index) { + // AIC/AIV 分开统计,避免 32:64 的参与者数量差异掩盖某一类核上的长尾。 + for (const PhaseSpec &phase : phases) { + std::vector cycles; + std::vector calls; + std::vector wait_events; + const uint32_t phase_index = static_cast(phase.phase); + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + const WorkerResult &result = state.results[worker]; + if (result.role != static_cast(roles[role_index])) continue; + cycles.push_back(result.phase_cycles[phase_index]); + calls.push_back(result.phase_calls[phase_index]); + wait_events.push_back( + phase.wait_event_index < 0 ? 0 : result.wait_events[static_cast(phase.wait_event_index)] + ); + } + const Uint64Distribution cycle_summary = SummarizeUint64(cycles); + const Uint64Distribution call_summary = SummarizeUint64(calls); + const Uint64Distribution wait_summary = SummarizeUint64(wait_events); + std::printf( + "[PHASE] role=%s phase=%s workers=%zu accumulated_us_median=%.3f " + "accumulated_us_p95=%.3f accumulated_us_max=%.3f calls_total=%llu " + "calls_per_worker_median=%.1f calls_per_worker_p95=%llu calls_per_worker_max=%llu " + "wait_events_total=%llu wait_events_per_worker_median=%.1f " + "wait_events_per_worker_p95=%llu wait_events_per_worker_max=%llu\n", + role_names[role_index], phase.name, cycles.size(), cycle_summary.median / 1000.0, + static_cast(cycle_summary.p95) / 1000.0, + static_cast(cycle_summary.maximum) / 1000.0, + static_cast(call_summary.total), call_summary.median, + static_cast(call_summary.p95), + static_cast(call_summary.maximum), + static_cast(wait_summary.total), wait_summary.median, + static_cast(wait_summary.p95), + static_cast(wait_summary.maximum) + ); + } + } +} + +inline const char *TracePhaseName(uint32_t phase) { + // 名称必须与 l2_swimlane_records.json 的 fdwic_events schema 保持一致。 + const char *names[] = { + "Kernel", "Alloc", "Build", "DrainWon", "Replay", "RingBp", "EfDrain", "Commit", + "Submit", "Materialize", "PrepareMap", "Claim", "Fanin", "Register", + }; + return phase < sizeof(names) / sizeof(names[0]) ? names[phase] : "Unknown"; +} + +inline bool ValidateTraceHeader(const TraceHeader &header, const char *operation) { + // 在任何 D2H record 搬运前先验证容量和 dropped,防止损坏 header 导致 scratch 越界或导出残缺泳道。 + // 频率也要求精确为 1 GHz,否则后续 ns/us 换算即使 JSON 合法也没有性能意义。 + const bool valid = header.magic == 0x4653574cU && header.version == 1 && + header.num_cores == kWorkers && header.records_per_core == kTraceRecordsPerCore && + header.frequency_hz == kSystemCounterHz; + bool core_states_valid = true; + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + core_states_valid &= header.cores[worker].count <= kTraceRecordsPerCore; + core_states_valid &= header.cores[worker].dropped == 0; + } + if (!valid || !core_states_valid) { + std::fprintf( + stderr, + "%s rejected an invalid trace header: magic=0x%08x version=%u cores=%u " + "records_per_core=%u frequency_hz=%llu core_states_valid=%s\n", + operation, header.magic, header.version, header.num_cores, header.records_per_core, + static_cast(header.frequency_hz), core_states_valid ? "yes" : "no" + ); + } + return valid && core_states_valid; +} + +template +inline bool ExportSwimlaneRecords( + const TraceHeader &header, const std::string &output_path, ReadRecords read_records +) { + if (!ValidateTraceHeader(header, "swimlane export")) return false; + + // 先写同目录临时文件,全部记录写完并关闭后再 rename 替换,避免把半截 JSON + // 当成有效采集;这里没有 fsync 文件和目录,不承诺掉电后的持久化原子性。 + const std::string temporary_path = output_path + ".tmp"; + std::FILE *output = std::fopen(temporary_path.c_str(), "wb"); + if (output == nullptr) { + std::fprintf( + stderr, "Cannot open swimlane output %s: %s\n", temporary_path.c_str(), std::strerror(errno) + ); + return false; + } + + // 采用固定 1 MiB stdio 缓冲并逐核流式写出;默认 256 batch 时约 86 万条, + // 无论实际 batch 数是多少都不在 host 侧一次性聚合全部 JSON 记录。 + std::vector output_buffer(1U << 20); + std::setvbuf(output, output_buffer.data(), _IOFBF, output_buffer.size()); + std::fprintf( + output, + "{\n\"l2_swimlane_level\":1,\n" + "\"metadata\":{\"clock_freq_hz\":%llu,\"num_cores\":%u,\"core_types\":[", + static_cast(header.frequency_hz), kWorkers + ); + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + std::fprintf(output, "%s\"%s\"", worker == 0 ? "" : ",", worker < kAicWorkers ? "aic" : "aiv"); + } + std::fprintf( + output, + "]},\n\"aicore_tasks\":[],\n\"aicpu_tasks\":[],\n" + "\"aicpu_scheduler_phases\":[],\n\"aicpu_orchestrator_phases\":[],\n\"fdwic_events\":[\n" + ); + // fdwic_events 每行固定十列:core、block、lane、task、function、phase、起止周期、flags、aux。 + + bool success = true; + bool first_record = true; + uint64_t exported_records = 0; + std::vector scratch(kTraceRecordsPerCore); + constexpr int32_t kTracePhaseCount = static_cast(TracePhase::Register) + 1; + for (uint32_t worker = 0; worker < kWorkers && success; ++worker) { + // 每次只读取一个 worker 的有效区间;完整 384 MiB trace 缓冲无需整体回拷。 + const uint32_t available = header.cores[worker].count; + if (available > header.records_per_core) { + std::fprintf( + stderr, "Trace core %u count %u exceeds capacity %u.\n", worker, available, + header.records_per_core + ); + success = false; + break; + } + if (available != 0 && !read_records(worker, available, scratch.data())) { + success = false; + break; + } + for (uint32_t index = 0; index < available; ++index) { + const TraceRecord &record = scratch[index]; + // 这里只检查 lane/block 的合法范围以及 core_idx 是否落在所属 worker 槽, + // 不把生产者应遵守的 worker↔block/lane 精确映射误说成 exporter 已完成的校验。 + const bool record_valid = record.end_cycle >= record.start_cycle && record.phase >= 0 && + record.phase < kTracePhaseCount && record.lane >= 0 && record.lane <= 2 && + record.block_id >= 0 && record.block_id < static_cast(kAicWorkers) && + record.core_idx == static_cast(worker); + if (!record_valid) { + std::fprintf( + stderr, + "Invalid trace record at worker=%u index=%u: phase=%d lane=%d block=%d core=%d " + "start=%llu end=%llu\n", + worker, index, record.phase, record.lane, record.block_id, record.core_idx, + static_cast(record.start_cycle), + static_cast(record.end_cycle) + ); + success = false; + break; + } + std::fprintf( + output, + "%s[%d,%d,%d,%d,%d,\"%s\",%llu,%llu,%u,%u]", + first_record ? "" : ",\n", record.core_idx, record.block_id, record.lane, record.task_id, + record.function_id, TracePhaseName(static_cast(record.phase)), + static_cast(record.start_cycle), + static_cast(record.end_cycle), record.flags, record.auxiliary + ); + first_record = false; + ++exported_records; + } + } + if (success) std::fprintf(output, "\n]}\n"); + if (std::ferror(output) != 0) { + std::fprintf(stderr, "Failed while writing swimlane output %s.\n", temporary_path.c_str()); + success = false; + } + if (std::fclose(output) != 0) { + std::fprintf(stderr, "Failed to close swimlane output %s: %s\n", temporary_path.c_str(), std::strerror(errno)); + success = false; + } + if (success && std::rename(temporary_path.c_str(), output_path.c_str()) != 0) { + std::fprintf( + stderr, "Cannot finalize swimlane output %s: %s\n", output_path.c_str(), std::strerror(errno) + ); + success = false; + } + if (!success) { + std::remove(temporary_path.c_str()); + return false; + } + std::printf( + "[SWIMLANE] raw_json=%s events=%llu\n", output_path.c_str(), + static_cast(exported_records) + ); + return true; +} + +template +inline bool AnalyzeSwimlaneRecords( + const TraceHeader &header, const SchedulerState &state, ReadRecords read_records +) { + if (!ValidateTraceHeader(header, "swimlane analysis")) return false; + + // 第一组数组统计“每个 worker 在某阶段的累计时间”;task_durations 则保留重点阶段的单事件分布。 + constexpr uint32_t kTracePhaseCount = static_cast(TracePhase::Register) + 1; + constexpr TracePhase kDetailedPhases[] = { + TracePhase::EfDrain, TracePhase::Materialize, TracePhase::Claim, TracePhase::Register, + }; + uint64_t cycles[kWorkers][kTracePhaseCount] = {}; + uint64_t counts[kWorkers][kTracePhaseCount] = {}; + std::vector task_durations[2][kTasksPerBatch][sizeof(kDetailedPhases) / sizeof(kDetailedPhases[0])]; + std::vector scratch(kTraceRecordsPerCore); + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + const uint32_t available = header.cores[worker].count; + const uint32_t count = std::min(available, header.records_per_core); + if (count != 0 && !read_records(worker, count, scratch.data())) { + return false; + } + for (uint32_t index = 0; index < count; ++index) { + const TraceRecord &record = scratch[index]; + if (record.phase < 0 || record.phase >= static_cast(kTracePhaseCount) || + record.end_cycle < record.start_cycle) { + // 分析器面对单条坏记录选择跳过;严格导出路径会直接拒绝,二者服务于不同诊断目的。 + continue; + } + const uint32_t phase = static_cast(record.phase); + const uint64_t duration = record.end_cycle - record.start_cycle; + cycles[worker][phase] += duration; + ++counts[worker][phase]; + if (record.task_id >= 0) { + // task_id % 5 恰好对应 Alloc/QK/SF/PV/UP,这是固定 PA Case1 图的拓扑约束。 + const uint32_t role_index = + state.results[worker].role == static_cast(CoreRole::Aic) ? 0U : 1U; + const uint32_t kind = static_cast(record.task_id) % kTasksPerBatch; + for (uint32_t detail = 0; detail < sizeof(kDetailedPhases) / sizeof(kDetailedPhases[0]); ++detail) { + if (phase == static_cast(kDetailedPhases[detail])) { + task_durations[role_index][kind][detail].push_back(duration); + } + } + } + } + } + + const CoreRole roles[] = {CoreRole::Aic, CoreRole::Aiv}; + const char *role_names[] = {"AIC", "AIV"}; + for (uint32_t role_index = 0; role_index < 2; ++role_index) { + for (uint32_t phase = 0; phase < kTracePhaseCount; ++phase) { + std::vector role_cycles; + uint64_t record_count = 0; + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + if (state.results[worker].role != static_cast(roles[role_index])) continue; + role_cycles.push_back(cycles[worker][phase]); + record_count += counts[worker][phase]; + } + const Uint64Distribution summary = SummarizeUint64(role_cycles); + std::printf( + "[TRACE_PHASE] role=%s phase=%s records=%llu accumulated_us_median=%.3f " + "accumulated_us_p95=%.3f accumulated_us_max=%.3f\n", + role_names[role_index], TracePhaseName(phase), + static_cast(record_count), summary.median / 1000.0, + static_cast(summary.p95) / 1000.0, + static_cast(summary.maximum) / 1000.0 + ); + } + } + const char *kind_names[] = {"Alloc", "QK", "SF", "PV", "UP"}; + // 单事件统计按 role 与 task kind 展开,可区分“该 role 真实参与”与“只回放前端”的成本。 + for (uint32_t role_index = 0; role_index < 2; ++role_index) { + for (uint32_t kind = 0; kind < kTasksPerBatch; ++kind) { + for (uint32_t detail = 0; detail < sizeof(kDetailedPhases) / sizeof(kDetailedPhases[0]); ++detail) { + const std::vector &durations = task_durations[role_index][kind][detail]; + const Uint64Distribution summary = SummarizeUint64(durations); + std::printf( + "[TRACE_TASK] role=%s kind=%s phase=%s events=%zu median_ns=%.1f p95_ns=%llu max_ns=%llu\n", + role_names[role_index], kind_names[kind], + TracePhaseName(static_cast(kDetailedPhases[detail])), durations.size(), summary.median, + static_cast(summary.p95), + static_cast(summary.maximum) + ); + } + } + } + return true; +} + +inline Metrics Validate( + const SchedulerState &state, uint32_t run, double host_us, const TraceHeader *trace_header = nullptr +) { + Metrics metrics; + // 每个 worker 都回放全部 task。Alloc 由 96 个 worker 全部执行 atomicMax Claim; + // 其余 kernel task 只有与 active role 匹配的 AIC 或 AIV 参与 Claim。 + const uint32_t batches = state.config.batches; + const uint32_t task_count = batches * kTasksPerBatch; + const uint64_t expected_submits = static_cast(kWorkers) * task_count; + const uint64_t expected_claims = + static_cast(batches) * (kWorkers + kAicWorkers + kAivWorkers + kAicWorkers + kAivWorkers); + // 上式依次对应 Alloc、QK、SF、PV、UP 的 active worker 数,默认 256 batch 时为 73728。 + + // 聚合量分为调度核心计数、kernel 分布、前端操作数和最终状态四组,便于定位语义偏差。 + uint64_t first_submit = UINT64_MAX; + uint64_t last_submit = 0; + uint64_t submits = 0; + uint64_t claims = 0; + uint64_t wins = 0; + uint64_t heap_guards = 0; + uint64_t fanin_loads = 0; + uint64_t duplicates = 0; + uint64_t cas_retries = 0; + uint64_t joint_polls = 0; + uint64_t trace_wait_records = 0; + uint64_t wins_by_kind[5] = {}; + uint64_t kernel_counts[4] = {}; + uint64_t kernel_cycles[4] = {}; + uint64_t kernel_min[4] = {}; + uint64_t kernel_max[4] = {}; + uint64_t placements[3] = {}; + uint64_t phase_calls[static_cast(ProfilePhase::Count)] = {}; + uint64_t context_reads = 0; + uint64_t views_created = 0; + uint64_t dynamic_create_infos = 0; + uint64_t arg_resets = 0; + uint64_t tensor_args_added = 0; + uint64_t scalar_args_added = 0; + uint64_t materialized_outputs = 0; + uint64_t map_inserts = 0; + uint64_t map_lookups = 0; + uint64_t slot_tensor_copies = 0; + uint64_t slot_scalar_copies = 0; + uint64_t fanin_edges = 0; + bool worker_ids[kWorkers] = {}; + uint32_t aic_count = 0; + uint32_t aiv_count = 0; + uint32_t winning_workers = 0; + uint64_t max_worker_wins = 0; + bool worker_shape_ok = true; + bool submit_timestamps_ok = true; + bool vend_values_ok = true; + bool frontend_worker_counts_ok = true; + bool final_worker_state_ok = true; + bool worker_checksums_ok = true; + + // 按真实输出大小、1 KiB 对齐和 256 MiB 环回规则重算每个 task 可接受的最小 vend。 + uint64_t expected_heap_next = 0; + bool vend_progress_bounds_ok = true; + uint32_t first_bad_vend = task_count; + uint64_t first_bad_vend_minimum = 0; + uint64_t first_bad_vend_actual = 0; + std::vector minimum_vends(task_count); + const uint64_t output_bytes_by_kind[] = {10240, 524288, 264192, 8192, 0}; + for (uint32_t task_id = 0; task_id < task_count; ++task_id) { + const uint64_t output_bytes = output_bytes_by_kind[task_id % kTasksPerBatch]; + uint64_t task_base = (expected_heap_next + kOutputAlignment - 1) / kOutputAlignment * kOutputAlignment; + if (output_bytes != 0 && (task_base % state.heap_size) + output_bytes > state.heap_size) { + task_base = (task_base / state.heap_size + 1) * state.heap_size; + } + expected_heap_next = task_base + output_bytes; + minimum_vends[task_id] = expected_heap_next; + } + for (uint32_t task_id = 0; task_id < task_count; ++task_id) { + // vend 可以大于本 task 的最小末端,因为 winner 发布的是其本地 heap_cursor 快照; + // 但不能超过该 worker 完整回放所有 task 后的最终 heap_next。 + if (state.tasks[task_id].vend < minimum_vends[task_id] || + state.tasks[task_id].vend > expected_heap_next) { + vend_progress_bounds_ok = false; + if (first_bad_vend == task_count) { + first_bad_vend = task_id; + first_bad_vend_minimum = minimum_vends[task_id]; + first_bad_vend_actual = state.tasks[task_id].vend; + } + } + } + // TensorMap 只保留 heap window 内仍可能被依赖的四类输出;floor 对应已安全退休的 task 边界。 + const uint64_t expected_map_live = 4ULL * std::min(batches, 13); + const uint64_t expected_map_floor = task_count > kHeapWindow + 1 ? task_count - kHeapWindow - 1 : 0; + + for (uint32_t index = 0; index < kWorkers; ++index) { + // 每核只写自己独占且按 cache line 隔离的 WorkerResult;host 在 kernel 完成后统一汇总,不引入额外 atomic。 + const WorkerResult &result = state.results[index]; + if (result.worker_id < kWorkers) { + worker_ids[result.worker_id] = true; + } else { + worker_shape_ok = false; + } + aic_count += result.role == static_cast(CoreRole::Aic); + aiv_count += result.role == static_cast(CoreRole::Aiv); + worker_shape_ok &= result.submits == task_count; + worker_shape_ok &= result.max_occupied <= kUsableSlots; + worker_shape_ok &= result.final_occupied == 0; + submit_timestamps_ok &= result.submit_begin != 0; + submit_timestamps_ok &= result.submit_end >= result.submit_begin; + submit_timestamps_ok &= result.finish_cycle >= result.submit_end; + first_submit = std::min(first_submit, result.submit_begin); + last_submit = std::max(last_submit, result.submit_end); + submits += result.submits; + claims += result.claim_attempts; + wins += result.claim_wins; + if (result.claim_wins != 0) ++winning_workers; + max_worker_wins = std::max(max_worker_wins, result.claim_wins); + heap_guards += result.heap_guards; + fanin_loads += result.fanin_loads; + duplicates += result.completion_duplicates; + cas_retries += result.cas_retries; + joint_polls += result.joint_polls; + trace_wait_records += result.wait_events[0] + result.wait_events[1]; + context_reads += result.context_reads; + views_created += result.views_created; + dynamic_create_infos += result.dynamic_create_infos; + arg_resets += result.arg_resets; + tensor_args_added += result.tensor_args_added; + scalar_args_added += result.scalar_args_added; + materialized_outputs += result.materialized_outputs; + map_inserts += result.map_inserts; + map_lookups += result.map_lookups; + slot_tensor_copies += result.slot_tensor_copies; + slot_scalar_copies += result.slot_scalar_copies; + fanin_edges += result.fanin_edges; + frontend_worker_counts_ok &= result.context_reads == batches; + frontend_worker_counts_ok &= result.views_created == static_cast(batches) * 2; + frontend_worker_counts_ok &= result.dynamic_create_infos == static_cast(batches) * 2; + frontend_worker_counts_ok &= result.arg_resets == static_cast(batches) * 4; + frontend_worker_counts_ok &= result.tensor_args_added == static_cast(batches) * 22; + frontend_worker_counts_ok &= result.scalar_args_added == static_cast(batches) * 9; + frontend_worker_counts_ok &= result.materialized_outputs == static_cast(batches) * 8; + frontend_worker_counts_ok &= result.map_inserts == static_cast(batches) * 4; + final_worker_state_ok &= result.final_heap_next == expected_heap_next; + final_worker_state_ok &= result.map_high_water == expected_map_live; + final_worker_state_ok &= result.map_live_entries == expected_map_live; + final_worker_state_ok &= result.map_alive_floor == expected_map_floor; + final_worker_state_ok &= result.map_cleaned_upto == expected_map_floor; + worker_checksums_ok &= result.checksum == (0xcbf29ce484222325ULL ^ result.worker_id); + for (uint32_t kind = 0; kind < 5; ++kind) + wins_by_kind[kind] += result.wins[kind]; + for (uint32_t kind = 0; kind < 4; ++kind) { + kernel_counts[kind] += result.kernel_counts[kind]; + kernel_cycles[kind] += result.kernel_cycles[kind]; + if (result.kernel_min_cycles[kind] != 0 && + (kernel_min[kind] == 0 || result.kernel_min_cycles[kind] < kernel_min[kind])) { + kernel_min[kind] = result.kernel_min_cycles[kind]; + } + kernel_max[kind] = std::max(kernel_max[kind], result.kernel_max_cycles[kind]); + } + for (uint32_t place = 0; place < 3; ++place) + placements[place] += result.placement[place]; + for (uint32_t phase = 0; phase < static_cast(ProfilePhase::Count); ++phase) + phase_calls[phase] += result.phase_calls[phase]; + } + for (bool seen : worker_ids) + worker_shape_ok &= seen; + + uint32_t ready_flags = 0; + for (uint32_t task_id = 0; task_id < task_count; ++task_id) { + // ready flag 和 vend 是跨核 completion 的最终外部可见状态,不能只依赖 worker 私有计数判断完成。 + ready_flags += state.tasks[task_id].flag == 1; + vend_values_ok &= state.tasks[task_id].vend != 0; + vend_values_ok &= state.tasks[task_id].vend % kOutputAlignment == 0; + } + const uint64_t kernel_total = kernel_counts[0] + kernel_counts[1] + kernel_counts[2] + kernel_counts[3]; + const uint64_t placement_total = placements[0] + placements[1] + placements[2]; + + // 第一组断言覆盖参与者拓扑、Claim/winner、completion 和最终 drain 等调度主协议。 + Expect(aic_count == kAicWorkers && aiv_count == kAivWorkers, "participant topology is 32 AIC + 64 AIV", &metrics); + Expect(worker_shape_ok, "all 96 worker markers and private rings are valid", &metrics); + Expect(submit_timestamps_ok, "all Submit timing markers are valid", &metrics); + Expect(state.started_count.value == kWorkers, "started_count is 96", &metrics); + Expect(submits == expected_submits, "replay count is workers * tasks", &metrics); + Expect(claims == expected_claims, "Claim attempt count matches PA topology", &metrics); + Expect(wins == task_count, "exactly one winner per task", &metrics); + Expect( + wins_by_kind[0] == batches && wins_by_kind[1] == batches && wins_by_kind[2] == batches && + wins_by_kind[3] == batches && wins_by_kind[4] == batches, + "Alloc/QK/SF/PV/UP winners are one per batch", &metrics + ); + Expect(kernel_total == static_cast(batches) * 4, "kernel count is four per batch", &metrics); + Expect( + kernel_counts[0] == batches && kernel_counts[1] == batches && kernel_counts[2] == batches && + kernel_counts[3] == batches, + "each kernel kind executes once per batch", &metrics + ); + Expect(heap_guards == static_cast(batches) * 4, "heap guard count matches output winners", &metrics); + Expect(fanin_loads >= static_cast(batches) * 5, "successful fanin checks meet PA lower bound", &metrics); + Expect(duplicates == 0, "completion flags are published once", &metrics); + Expect(ready_flags == task_count, "all task flags are ready", &metrics); + Expect(vend_values_ok, "all published vend values are nonzero and aligned", &metrics); + Expect(vend_progress_bounds_ok, "every task vend is within PA worker heap progress bounds", &metrics); + Expect(state.frontier.value == static_cast(task_count) - 1, "frontier reaches the final task", &metrics); + Expect(state.replay_done.value == kWorkers, "replay_done is 96", &metrics); + Expect(state.fatal.value == 0, "fatal remains clear", &metrics); + Expect(placement_total == kernel_total, "EfDrain + RingBp + final placement covers every kernel", &metrics); + // joint_polls 是为未来 BlockWon 模拟预留的结果字段,当前调度路径没有递增点; + // 此断言只确认现有输出保持零,不能单独证明 active_count>=2 分支不可达。 + Expect(joint_polls == 0, "single-lane PA performs no BlockWon polling", &metrics); + // 第二组断言锁定 scalar 前端工作量,防止编译器优化或后续改动悄悄删掉 PA 模拟步骤。 + Expect(frontend_worker_counts_ok, "every worker replays the exact PA frontend operation counts", &metrics); + Expect( + context_reads == static_cast(kWorkers) * batches && + views_created == static_cast(kWorkers) * batches * 2 && + dynamic_create_infos == static_cast(kWorkers) * batches * 2 && + arg_resets == static_cast(kWorkers) * batches * 4 && + tensor_args_added == static_cast(kWorkers) * batches * 22 && + scalar_args_added == static_cast(kWorkers) * batches * 9 && + materialized_outputs == static_cast(kWorkers) * batches * 8 && + map_inserts == static_cast(kWorkers) * batches * 4, + "global PA frontend operation totals are exact", &metrics + ); + Expect( + map_lookups == static_cast(batches) * 14 && + slot_tensor_copies == static_cast(batches) * 19 && + slot_scalar_copies == static_cast(batches) * 9 && + fanin_edges == static_cast(batches) * 5, + "winner-only map, slot-copy, and fanin totals are exact", &metrics + ); + Expect(final_worker_state_ok, "every worker final heap and TensorMap state is exact", &metrics); + Expect(worker_checksums_ok, "all frontend registration checksums remain clean", &metrics); + + // 三类 Claim cursor 各有四个 shard;按 task_id 重新推导每个 shard 应停留的最后任务。 + int64_t expected_cube[kCursorShards] = {-1, -1, -1, -1}; + int64_t expected_vector[kCursorShards] = {-1, -1, -1, -1}; + int64_t expected_alloc[kCursorShards] = {-1, -1, -1, -1}; + for (uint32_t task_id = 0; task_id < task_count; ++task_id) { + const TaskKind kind = static_cast(task_id % kTasksPerBatch); + int64_t *cursors = kind == TaskKind::Alloc + ? expected_alloc + : (kind == TaskKind::Qk || kind == TaskKind::Pv ? expected_cube : expected_vector); + cursors[task_id % kCursorShards] = task_id; + } + bool cursors_ok = true; + for (uint32_t shard = 0; shard < kCursorShards; ++shard) { + cursors_ok &= state.cube_cursor[shard].value == expected_cube[shard]; + cursors_ok &= state.vector_cursor[shard].value == expected_vector[shard]; + cursors_ok &= state.alloc_cursor[shard].value == expected_alloc[shard]; + } + Expect(cursors_ok, "all sharded Claim cursors reach their exact final task", &metrics); + + if (state.config.profile_phases != 0) { + // profile 开关关闭时这些字段允许保持零,避免把可选诊断本身变成语义门禁。 + Expect( + phase_calls[static_cast(ProfilePhase::Claim)] == expected_submits && + phase_calls[static_cast(ProfilePhase::EfDrain)] == expected_submits && + phase_calls[static_cast(ProfilePhase::WaitForSlot)] == + static_cast(batches) * 4 && + phase_calls[static_cast(ProfilePhase::HeapGuard)] == + static_cast(batches) * 4, + "profile call counts match Claim/EfDrain/WaitForSlot/HeapGuard flow", &metrics + ); + } + + if (state.config.trace_enabled != 0) { + // 固定阶段记录数加上动态等待记录数,应与所有 worker 的 header count 精确相等。 + bool trace_shape_ok = trace_header != nullptr; + uint64_t trace_records = 0; + uint64_t trace_dropped = 0; + bool per_worker_trace_counts_ok = true; + if (trace_header != nullptr) { + trace_shape_ok &= trace_header->magic == 0x4653574cU; + trace_shape_ok &= trace_header->version == 1; + trace_shape_ok &= trace_header->num_cores == kWorkers; + trace_shape_ok &= trace_header->records_per_core == kTraceRecordsPerCore; + trace_shape_ok &= trace_header->frequency_hz == kSystemCounterHz; + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + trace_records += trace_header->cores[worker].count; + trace_dropped += trace_header->cores[worker].dropped; + trace_shape_ok &= trace_header->cores[worker].count <= kTraceRecordsPerCore; + const WorkerResult &result = state.results[worker]; + const uint64_t worker_kernels = result.kernel_counts[0] + result.kernel_counts[1] + + result.kernel_counts[2] + result.kernel_counts[3]; + const uint64_t worker_expected = 7 * result.submits + + result.claim_wins - result.wins[0] + + 2 * worker_kernels + result.wait_events[0] + + result.wait_events[1]; + per_worker_trace_counts_ok &= trace_header->cores[worker].count == worker_expected; + } + } + const uint64_t expected_trace_records = + static_cast(batches) * (static_cast(kWorkers) * 35 + 12) + trace_wait_records; + // 每 batch 固定记录为 96*35+12;RingBp 等真实等待按运行时次数额外加入。 + Expect(trace_shape_ok, "swimlane header and per-worker capacities are valid", &metrics); + Expect(trace_dropped == 0, "swimlane records fit without drops", &metrics); + Expect(trace_records == expected_trace_records, "swimlane record count matches PA phase flow", &metrics); + Expect(per_worker_trace_counts_ok, "every worker swimlane record count is exact", &metrics); + std::printf( + "[TRACE] records=%llu expected=%llu dropped=%llu bytes=%zu\n", + static_cast(trace_records), + static_cast(expected_trace_records), + static_cast(trace_dropped), kTraceBytes + ); + } + + if (first_submit != UINT64_MAX && last_submit >= first_submit) { + // 性能口径只覆盖最早 Submit.begin 到最晚 Submit.end,不含启动屏障、最终 drain 和 host 同步。 + metrics.submit_span_us = static_cast(last_submit - first_submit) / 1000.0; + } + std::printf( + "[METRIC] run=%u submit_span_us=%.3f host_launch_us=%.3f claims=%llu fanin_loads=%llu cas_retries=%llu\n", run, + metrics.submit_span_us, host_us, static_cast(claims), + static_cast(fanin_loads), static_cast(cas_retries) + ); + std::printf( + "[WINNERS] active_workers=%u max_wins_per_worker=%llu\n", winning_workers, + static_cast(max_worker_wins) + ); + std::printf( + "[PLACEMENT] EfDrain=%llu RingBp=%llu FinalDrain=%llu\n", + static_cast(placements[static_cast(DrainPlace::EfDrain)]), + static_cast(placements[static_cast(DrainPlace::RingBackpressure)]), + static_cast(placements[static_cast(DrainPlace::FinalDrain)]) + ); + // placement 统计回答 kernel 最终在哪个 drain 点执行,与 TracePhase 的累计 span 互补。 + const char *kernel_names[] = {"QK", "SF", "PV", "UP"}; + const uint32_t targets[] = {kTargetQkTicks, kTargetSfTicks, kTargetPvTicks, kTargetUpTicks}; + for (uint32_t kind = 0; kind < 4; ++kind) { + const double mean = + kernel_counts[kind] == 0 ? 0.0 : static_cast(kernel_cycles[kind]) / kernel_counts[kind]; + std::printf( + "[KERNEL] %-2s count=%llu mean_us=%.3f min_us=%.3f max_us=%.3f target_us=%.3f\n", kernel_names[kind], + static_cast(kernel_counts[kind]), mean / 1000.0, kernel_min[kind] / 1000.0, + kernel_max[kind] / 1000.0, targets[kind] / 1000.0 + ); + } + PrintPhaseDiagnostics(state); + if (!metrics.passed) { + // 失败时补充第一处未完成 task、vend 边界和 worker 进度,避免只有笼统的 ASSERT FAIL。 + uint32_t first_not_ready = task_count; + for (uint32_t task_id = 0; task_id < task_count; ++task_id) { + if (state.tasks[task_id].flag != 1) { + first_not_ready = task_id; + break; + } + } + uint64_t min_worker_submits = UINT64_MAX; + uint64_t max_worker_submits = 0; + uint32_t incomplete_workers = 0; + uint32_t occupied_workers = 0; + uint64_t max_final_occupied = 0; + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + const WorkerResult &result = state.results[worker]; + min_worker_submits = std::min(min_worker_submits, result.submits); + max_worker_submits = std::max(max_worker_submits, result.submits); + incomplete_workers += result.submits != task_count; + occupied_workers += result.final_occupied != 0; + max_final_occupied = std::max(max_final_occupied, result.final_occupied); + } + const int64_t retire = state.frontier.value - static_cast(kHeapWindow); + const uint64_t retire_vend = + retire >= 0 && retire < static_cast(task_count) ? state.tasks[retire].vend : 0; + std::printf( + "[FAILURE_STATE] fatal=%d frontier=%lld first_not_ready=%u first_bad_vend=%u " + "vend_minimum=%llu vend_actual=%llu retire=%lld retire_vend=%llu " + "worker_submits_min=%llu worker_submits_max=%llu incomplete_workers=%u " + "final_occupied_workers=%u max_final_occupied=%llu\n", + state.fatal.value, static_cast(state.frontier.value), first_not_ready, first_bad_vend, + static_cast(first_bad_vend_minimum), + static_cast(first_bad_vend_actual), + static_cast(retire), static_cast(retire_vend), + static_cast(min_worker_submits), + static_cast(max_worker_submits), incomplete_workers, occupied_workers, + static_cast(max_final_occupied) + ); + } + return metrics; +} + +inline double Median(std::vector values) { + // 多轮 benchmark 只报告中位数;上板基线比较仍应优先采用独立进程首轮。 + std::sort(values.begin(), values.end()); + const size_t middle = values.size() / 2; + if ((values.size() & 1U) != 0) return values[middle]; + return (values[middle - 1] + values[middle]) / 2.0; +} + +inline void PrintBanner(const char *backend, const Options &options) { + // 开始运行前完整打印工作量和大内存占用,便于确认比较口径没有混用。 + std::printf("=== Standalone PA Scheduler Benchmark: %s ===\n", backend); + std::printf( + "device=%u batches=%u tasks=%u workers=%u runs=%u nops=%u,%u,%u,%u state_bytes=%zu " + "swimlane=%s trace_bytes=%zu\n", options.device, + options.batches, options.batches * kTasksPerBatch, kWorkers, options.runs, options.nops.qk, options.nops.sf, + options.nops.pv, options.nops.up, sizeof(SchedulerState), options.trace_enabled ? "on" : "off", + options.trace_enabled ? kTraceBytes : 0 + ); + if (!options.swimlane_json.empty()) { + std::printf("swimlane_json=%s\n", options.swimlane_json.c_str()); + } +} + +} // namespace pa_scheduler::host + +#endif // PA_SCHEDULER_COMMON_HOST_SUPPORT_H diff --git a/tests/atomic_probe/pa_scheduler/common/pa_frontend.h b/tests/atomic_probe/pa_scheduler/common/pa_frontend.h new file mode 100644 index 0000000000..6b874eeaae --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/common/pa_frontend.h @@ -0,0 +1,1162 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_COMMON_PA_FRONTEND_H +#define PA_SCHEDULER_COMMON_PA_FRONTEND_H + +#include "pa_model.h" + +namespace pa_scheduler { + +// 这些基址只充当稳定的 tensor identity,供 descriptor、区间重叠和 heap 地址 +// 计算使用;NOP kernel 不读取其中的数据。context_lens 是唯一按真实 GM 指针读取的输入。 +constexpr uint64_t kInvalidTaskId = UINT64_MAX; +constexpr uint64_t kSyntheticQueryBase = 0x200000000ULL; +constexpr uint64_t kSyntheticKeyBase = 0x300000000ULL; +constexpr uint64_t kSyntheticValueBase = 0x400000000ULL; +constexpr uint64_t kSyntheticBlockTableBase = 0x500000000ULL; +constexpr uint64_t kSyntheticOutputBase = 0x600000000ULL; +constexpr uint64_t kSyntheticContextLensBase = 0x700000000ULL; +constexpr uint32_t kPaHeads = 16; +constexpr uint32_t kPaHeadDim = 128; +constexpr uint32_t kPaBlockSize = 128; +constexpr uint32_t kPaBlocksPerRequest = 64; +constexpr uint32_t kPaMaxBlocksPerRequest = 256; +constexpr uint64_t kPaScaleBits = 0x3F800000ULL; +constexpr uint32_t kSpmdLocalContextIndex = kMaxTaskTensors + kMaxTaskScalars; +constexpr uint32_t kSpmdGlobalContextIndex = kSpmdLocalContextIndex + 1; +static_assert(kMaxTaskTensors == 32, "PA frontend requires the real 32 tensor slots"); +static_assert(kMaxTaskScalars == 16, "PA frontend requires the real 16 scalar slots"); +static_assert(kSpmdLocalContextIndex == 48, "PA local-context dispatch index mismatch"); +static_assert(kSpmdGlobalContextIndex == 49, "PA global-context dispatch index mismatch"); +static_assert(kMaxFanin == 16, "PA frontend requires the real 16 fanin slots"); + +enum class TensorRefKind : uint8_t { + LocalTensor = 0, + GmTensor = 1, + CreateInfo = 2, +}; + +// TaskArgs 同时容纳 orchestration 栈上的 descriptor、GM 中已物化的 descriptor, +// 以及尚待 Materialize 的 CreateInfo。显式 kind 保留生产 TensorRef 的地址空间分支。 +union TensorPointer { + const TensorDesc *local_tensor; + PA_GM const TensorDesc *gm_tensor; + const TensorCreateInfo *create_info; +}; + +struct TaskTensorRef { + TensorPointer pointer; + TensorRefKind kind; +}; +static_assert(sizeof(TaskTensorRef) == 16, "TaskTensorRef must match the PA TensorRef ABI"); +static_assert(offsetof(TaskTensorRef, pointer) == 0, "TaskTensorRef pointer offset mismatch"); +static_assert(offsetof(TaskTensorRef, kind) == 8, "TaskTensorRef kind offset mismatch"); + +struct PaLaunchSpec { + int16_t core_num; + bool require_sync_start; +}; +static_assert(sizeof(PaLaunchSpec) == 4, "PA launch spec ABI mismatch"); + +struct PaAsyncContext { + uint64_t completion_count; + uint64_t completion_error_code; + uint64_t completion_entries; + uint32_t completion_capacity; + uint32_t alignment_padding; + uint64_t task_token; +}; +static_assert(sizeof(PaAsyncContext) == 40, "PA async context ABI mismatch"); + +struct PaLocalContext { + int32_t block_index; + int32_t block_count; + PaAsyncContext async; +}; +// Local/GlobalContext 最终放进 RingSlot 的固定 dispatch 参数位 48/49;它们不是 +// standalone 自定义参数,offset 必须与真实 SPMD kernel 调用约定一致。 +static_assert(sizeof(PaLocalContext) == 48, "PA local context ABI mismatch"); + +struct PaGlobalContext { + int32_t sub_block_id; +}; +static_assert(sizeof(PaGlobalContext) == 4, "PA global context ABI mismatch"); + +// PTO2 profiling is enabled in the PA baseline. reset() clears all 160 bytes +// below on every QK/SF/PV/UP argument rebuild, even though Case1 does not ask +// to dump an argument. Keeping this storage and write stream matters to the +// spacing between consecutive Claim operations. +// 这段看似未使用的清零属于真实前端成本,删除会改变各 worker 到达 +// Claim 的波形与竞争强度,因此仍按生产构造/reset 顺序执行。 +struct PaDumpArgSelection { + uint64_t dump_arg_mask; + uint64_t dump_arg_index_ambiguous_mask; + uint64_t scalar_source_ptrs[kMaxTaskScalars]; + uint8_t scalar_dtypes[kMaxTaskScalars]; +}; +static_assert(sizeof(PaDumpArgSelection) == 160, "PA dump-selection ABI mismatch"); + +struct TaskArgs { + // The real TaskArgsTpl inherits its tag mixin first. TensorArgType is an + // int32 enum in the PA ABI; keeping tags first also reproduces its offsets. + // tag 数组位于对象首部不是任意排布;Materialize、fanin 与 register + // 都会重复扫描/复用这些 tag,错误 offset 会同时改变语义和前端访存成本。 + int32_t tags[kMaxTaskTensors]; + TaskTensorRef tensors[kMaxTaskTensors]; + uint64_t scalars[kMaxTaskScalars]; + int32_t tensor_count; + int32_t scalar_count; + + bool has_error; + uint64_t error_msg; + PaLaunchSpec launch_spec; + PaDumpArgSelection dump_arg_selection; + uint64_t explicit_deps; + uint32_t explicit_dep_count; + uint8_t cacheline_pad[48]; +}; +static_assert(sizeof(TaskArgs) == 1024, "TaskArgs must match the PA L0TaskArgs ABI size"); +static_assert(offsetof(TaskArgs, tags) == 0, "TaskArgs tag offset mismatch"); +static_assert(offsetof(TaskArgs, tensors) == 128, "TaskArgs tensor-ref offset mismatch"); +static_assert(offsetof(TaskArgs, scalars) == 640, "TaskArgs scalar offset mismatch"); +static_assert(offsetof(TaskArgs, tensor_count) == 768, "TaskArgs tensor-count offset mismatch"); +static_assert(offsetof(TaskArgs, scalar_count) == 772, "TaskArgs scalar-count offset mismatch"); +static_assert(offsetof(TaskArgs, has_error) == 776, "TaskArgs error flag offset mismatch"); +static_assert(offsetof(TaskArgs, error_msg) == 784, "TaskArgs error pointer offset mismatch"); +static_assert(offsetof(TaskArgs, launch_spec) == 792, "TaskArgs launch-spec offset mismatch"); +static_assert(offsetof(TaskArgs, dump_arg_selection) == 800, "TaskArgs dump-selection offset mismatch"); +static_assert(offsetof(TaskArgs, explicit_deps) == 960, "TaskArgs dependency pointer offset mismatch"); +static_assert(offsetof(TaskArgs, explicit_dep_count) == 968, "TaskArgs dependency count offset mismatch"); + +struct TaskOutputs { + uint64_t task_id; + uint32_t count; + PA_GM TensorDesc *tensors[kMaxTaskTensors]; +}; +static_assert(sizeof(TaskOutputs) == 272, "TaskOutputs must match the PA TaskOutputTensors ABI size"); +static_assert(offsetof(TaskOutputs, tensors) == 16, "TaskOutputs tensor pointer offset mismatch"); + +struct SubmitContext { + PA_GM WorkerState *self; + PA_GM TaskPayload *payload; + int32_t task_id; + int32_t tensor_count; + int32_t scalar_count; + uint32_t register_mask; + uint64_t output_bytes; + TaskOutputs result; + int32_t fanin[kMaxFanin]; + int32_t fanin_count; + int32_t kernel_id; + bool won; + bool joint; + bool joint_init; + int32_t joint_block; + int32_t joint_slot; + int32_t joint_count; +}; +// SubmitContext 贯穿一次 Submit:Begin 绑定 task/payload,Materialize 填充输出与 +// register_mask,winner 收集 fanin 并构建 slot。它复刻 DistSubmitCtx 而非诊断结构。 +static_assert(sizeof(SubmitContext) == 400, "SubmitContext must match DistSubmitCtx"); +static_assert(offsetof(SubmitContext, output_bytes) == 32, "SubmitContext output-byte offset mismatch"); +static_assert(offsetof(SubmitContext, result) == 40, "SubmitContext result offset mismatch"); +static_assert(offsetof(SubmitContext, fanin) == 312, "SubmitContext fanin offset mismatch"); + +struct OutputLayout { + uint64_t buffer_sizes[kMaxTaskTensors]; + uint64_t total_output_size; +}; +// 只有 tag=Output 的槽位拥有有效 buffer_sizes;总大小按 1 KiB 对齐累计,随后 +// 作为 HeapGuard 的 output_bytes 和本 worker heap_next 的推进量。 +static_assert(sizeof(OutputLayout) == 264, "OutputLayout must match DistOutputLayout"); + +// 该状态保存真实 PA orchestration 在五个 Submit 之间传递的 descriptor。输出指针 +// 指向每个 worker 自己 materialize 的 payload,不能跨 worker 共享或简化为全局对象。 +struct PaOrchestrationState { + TensorDesc query; + TensorDesc key_cache; + TensorDesc value_cache; + TensorDesc block_table; + TensorDesc context_lens; + TensorDesc output; + TensorDesc query_view; + TensorDesc output_view; + + TensorCreateInfo tile_create_info; + TensorCreateInfo scalar_create_info; + TensorCreateInfo qk_create_info; + TensorCreateInfo sf_create_info; + + // The pointer is supplied by the standalone backend. On A5 it must point + // at GM so every batch performs the same descriptor-based load as PA. + PA_GM const volatile int32_t *context_lens_data; + uint64_t scale_bits; + uint64_t current_sequence; + uint64_t current_blocks; + uint64_t current_block_offset; + uint64_t current_nblocks; + uint64_t current_valid_len; + uint32_t current_batch; + + PA_GM TensorDesc *accumulated_output; + PA_GM TensorDesc *accumulated_sum; + PA_GM TensorDesc *accumulated_max; + PA_GM TensorDesc *qk_scores; + PA_GM TensorDesc *sf_probs; + PA_GM TensorDesc *sf_max; + PA_GM TensorDesc *sf_sum; + PA_GM TensorDesc *pv_output; +}; + +PA_DEVICE uint64_t ElementSize(DataType dtype) { + // 输入 dtype 来自已通过 PA ABI 构造的 descriptor/create-info,必须落在 Count 前; + // 输出字节数同时用于外部 tensor range 与新 Output 的 heap 大小计算。 + constexpr static uint64_t sizes[static_cast(DataType::Count)] = { + 4, 2, 4, 2, 1, 1, 2, 8, 8, 2, 4, 1, + }; + return sizes[static_cast(dtype)]; +} + +PA_DEVICE int32_t TagValue(TensorArgType tag) { return static_cast(tag); } + +PA_DEVICE TensorArgType TaskTag(const TaskArgs &args, uint32_t index) { + // index 的有效范围由 tensor_count 保证;集中转换避免各阶段对 int32 ABI tag + // 做不同解释,Materialize/CollectFanin/Register 因而共享同一分类结果。 + return static_cast(args.tags[index]); +} + +PA_DEVICE void ClearDumpArgSelection(PaDumpArgSelection &selection) { + // Volatile stores intentionally preserve the profiling-enabled PA reset + // traffic even when the standalone NOP kernels never consume dump data. + // volatile 的目的不是同步,而是阻止编译器删掉这段生产基线中存在的写流量。 + volatile uint64_t *masks = &selection.dump_arg_mask; + masks[0] = 0; + masks[1] = 0; + volatile uint64_t *sources = &selection.scalar_source_ptrs[0]; + for (uint32_t index = 0; index < kMaxTaskScalars; ++index) { + sources[index] = 0; + } + volatile uint8_t *dtypes = &selection.scalar_dtypes[0]; + for (uint32_t index = 0; index < kMaxTaskScalars; ++index) { + dtypes[index] = 0; + } +} + +PA_DEVICE void ConstructTaskArgs(TaskArgs &args) { + // TensorTagMixin::tags_{} is value-initialized by the + // real L0TaskArgs constructor. TensorRef/scalar slots remain lazy. + // 构造只初始化真实构造函数会触碰的字段,未使用的 tensor/scalar + // 槽保持惰性;整对象 memset 会引入 PA 本身没有的额外前端开销。 + volatile int32_t *tags = &args.tags[0]; + for (uint32_t index = 0; index < kMaxTaskTensors; ++index) { + tags[index] = 0; + } + args.tensor_count = 0; + args.scalar_count = 0; + args.has_error = false; + args.error_msg = 0; + args.launch_spec.core_num = 1; + args.launch_spec.require_sync_start = false; + ClearDumpArgSelection(args.dump_arg_selection); + args.explicit_deps = 0; + args.explicit_dep_count = 0; +} + +PA_DEVICE void ResetTaskArgs(TaskArgs &args) { + // reset 的输出是不含 tensor/scalar/显式依赖的新逻辑参数表,但保留已分配对象及 + // launch_spec;QK/SF/PV/UP 在同一个 1 KiB TaskArgs 上依次复用这一状态。 + args.tensor_count = 0; + args.scalar_count = 0; + ClearDumpArgSelection(args.dump_arg_selection); + args.explicit_deps = 0; + args.explicit_dep_count = 0; + args.has_error = false; + args.error_msg = 0; +} + +PA_DEVICE bool ReserveTensorArgs(TaskArgs &args, int32_t count) { + // tensor 必须先于 scalar 追加,以保持 dispatch args 的 [tensor..., scalar...] + // 排列;失败只置 has_error,不发生部分追加。 + if (args.scalar_count != 0 || count < 0 || + args.tensor_count + count > static_cast(kMaxTaskTensors)) { + args.has_error = true; + return false; + } + return true; +} + +PA_DEVICE void AppendLocalTensor(TaskArgs &args, const TensorDesc &tensor, TensorArgType tag) { + const uint32_t index = static_cast(args.tensor_count++); + args.tensors[index].pointer.local_tensor = &tensor; + args.tensors[index].kind = TensorRefKind::LocalTensor; + args.tags[index] = TagValue(tag); +} + +PA_DEVICE void AppendGmTensor(TaskArgs &args, PA_GM const TensorDesc &tensor, TensorArgType tag) { + const uint32_t index = static_cast(args.tensor_count++); + args.tensors[index].pointer.gm_tensor = &tensor; + args.tensors[index].kind = TensorRefKind::GmTensor; + args.tags[index] = TagValue(tag); +} + +PA_DEVICE void AppendOutput(TaskArgs &args, const TensorCreateInfo &create_info) { + const uint32_t index = static_cast(args.tensor_count++); + args.tensors[index].pointer.create_info = &create_info; + args.tensors[index].kind = TensorRefKind::CreateInfo; + args.tags[index] = TagValue(TensorArgType::Output); +} + +PA_DEVICE bool ReserveScalarArgs(TaskArgs &args, int32_t count) { + // 先整体校验容量再由 AddTwo/AddThree 连续写入,保证多 scalar 操作全有或全无。 + if (count < 0 || args.scalar_count + count > static_cast(kMaxTaskScalars)) { + args.has_error = true; + return false; + } + return true; +} + +PA_DEVICE void AppendScalar(TaskArgs &args, uint64_t value) { + args.scalars[static_cast(args.scalar_count++)] = value; +} + +PA_DEVICE void AddLocalTensor(TaskArgs &args, const TensorDesc &tensor, TensorArgType tag) { + if (ReserveTensorArgs(args, 1)) AppendLocalTensor(args, tensor, tag); +} + +PA_DEVICE void AddGmTensor(TaskArgs &args, PA_GM const TensorDesc &tensor, TensorArgType tag) { + if (ReserveTensorArgs(args, 1)) AppendGmTensor(args, tensor, tag); +} + +PA_DEVICE void AddOutput(TaskArgs &args, const TensorCreateInfo &create_info) { + if (ReserveTensorArgs(args, 1)) AppendOutput(args, create_info); +} + +PA_DEVICE void AddScalar(TaskArgs &args, uint64_t value) { + if (ReserveScalarArgs(args, 1)) AppendScalar(args, value); +} + +PA_DEVICE void AddTwoScalars(TaskArgs &args, uint64_t value0, uint64_t value1) { + if (!ReserveScalarArgs(args, 2)) return; + AppendScalar(args, value0); + AppendScalar(args, value1); +} + +PA_DEVICE void AddThreeScalars(TaskArgs &args, uint64_t value0, uint64_t value1, uint64_t value2) { + if (!ReserveScalarArgs(args, 3)) return; + AppendScalar(args, value0); + AppendScalar(args, value1); + AppendScalar(args, value2); +} + +PA_DEVICE void InitCreateInfo( + TensorCreateInfo &info, const uint32_t shapes[kMaxTensorDims], uint32_t ndims, DataType dtype +) { + info.initial_value = 0; + info.has_initial_value = false; + info.reserved0 = 0; + info.start_offset = 0; + info.version = 0; + info.ndims = ndims; + info.dtype = dtype; + info.manual_dep = false; + info.is_contiguous = true; + info.child_memory = 0; + // TensorCreateInfo's real constructor only writes active dimensions. + // 只写 ndims 个 shape,保留生产构造器的写入范围,不能为方便把五维全清零。 + for (uint32_t index = 0; index < ndims; ++index) { + info.shapes[index] = shapes[index]; + } +} + +PA_DEVICE void ClearCreateInfo(TensorCreateInfo &info) { + volatile uint8_t *bytes = reinterpret_cast(&info); + for (uint32_t index = 0; index < sizeof(TensorCreateInfo); ++index) { + bytes[index] = 0; + } +} + +PA_DEVICE void InitExternalTensor( + TensorDesc &tensor, uint64_t address, const uint32_t shapes[kMaxTensorDims], uint32_t ndims, DataType dtype, + bool manual_dep +) { + // 输入为稳定 backing address、逻辑 shape 和依赖属性;输出是完整连续 descriptor, + // owner 无效表示它不是本轮 task 产生,row-major stride 从末维向前计算。 + uint64_t elements = 1; + for (uint32_t index = 0; index < ndims; ++index) { + elements *= shapes[index]; + } + tensor.buffer_addr = address; + tensor.buffer_size = elements * ElementSize(dtype); + tensor.owner_task_id = kInvalidTaskId; + tensor.start_offset = 0; + tensor.version = 0; + tensor.ndims = ndims; + tensor.dtype = dtype; + tensor.manual_dep = manual_dep; + tensor.is_contiguous = true; + tensor.child_memory = 0; + for (uint32_t index = 0; index < kMaxTensorDims; ++index) { + tensor.shapes[index] = shapes[index]; + tensor.strides[index] = 0; + } + uint32_t stride = 1; + for (int32_t index = static_cast(ndims) - 1; index >= 0; --index) { + tensor.strides[index] = stride; + stride *= tensor.shapes[index]; + } + tensor.extent_elem_cache = stride; +} + +PA_DEVICE bool InitTensorFromCreateInfo( + PA_GM TensorDesc &tensor, const TensorCreateInfo &info, uint64_t address, uint64_t buffer_size +) { + tensor.buffer_addr = address; + tensor.buffer_size = buffer_size; + tensor.owner_task_id = kInvalidTaskId; + tensor.start_offset = info.start_offset; + tensor.version = info.version; + tensor.ndims = info.ndims; + tensor.dtype = info.dtype; + tensor.manual_dep = info.manual_dep; + tensor.is_contiguous = info.is_contiguous; + tensor.child_memory = info.child_memory; + for (uint32_t index = 0; index < kMaxTensorDims; ++index) { + tensor.shapes[index] = info.shapes[index]; + } + uint32_t stride = 1; + for (int32_t index = static_cast(tensor.ndims) - 1; index >= 0; --index) { + tensor.strides[index] = stride; + stride *= tensor.shapes[index]; + } + tensor.extent_elem_cache = stride; + // PA can initialize the backing allocation here. Case1 never requests it; + // the standalone uses synthetic heap addresses and therefore rejects that + // unsupported path instead of writing to a fabricated GM pointer. + // Case1 的 has_initial_value 恒为 false;返回 false 是对未模拟分支的 + // 明确保护,不会在合成地址上伪造初始化写入。 + return !info.has_initial_value; +} + +PA_DEVICE uint64_t CreateInfoBytes(const TensorCreateInfo &info) { + uint64_t elements = 1; + for (uint32_t index = 0; index < info.ndims; ++index) { + elements *= info.shapes[index]; + } + return elements * ElementSize(info.dtype); +} + +template +PA_DEVICE void CopyTensorLine1(TensorDesc &destination, const Source &source) { + // view 只需复制 descriptor 第一条 cache line 的身份/shape 字段,随后由调用方 + // 覆盖 offset、shape、stride 与 extent;不做整 128-byte 拷贝以匹配 PA 写流。 + destination.buffer_addr = source.buffer_addr; + destination.buffer_size = source.buffer_size; + destination.owner_task_id = source.owner_task_id; + destination.start_offset = source.start_offset; + destination.version = source.version; + destination.ndims = source.ndims; + destination.dtype = source.dtype; + destination.manual_dep = source.manual_dep; + destination.is_contiguous = source.is_contiguous; + destination.child_memory = source.child_memory; + for (uint32_t index = 0; index < kMaxTensorDims; ++index) { + destination.shapes[index] = source.shapes[index]; + } +} + +PA_DEVICE void MakeBatchViews(PaOrchestrationState &orch, uint32_t batch) { + // query/output view 共享原 backing buffer,仅通过 start_offset 切出当前 batch。 + // output_view 保留真实 manual_dep 标记;UP 的生产者依赖由 Alloc、SF、PV 返回 descriptor 的 owner 闭合。 + CopyTensorLine1(orch.query_view, orch.query); + orch.query_view.start_offset = static_cast(batch) * kPaHeads * kPaHeadDim; + orch.query_view.ndims = 2; + orch.query_view.shapes[0] = kPaHeads; + orch.query_view.shapes[1] = kPaHeadDim; + orch.query_view.strides[0] = kPaHeadDim; + orch.query_view.strides[1] = 1; + orch.query_view.extent_elem_cache = kPaHeads * kPaHeadDim; + + CopyTensorLine1(orch.output_view, orch.output); + orch.output_view.start_offset = static_cast(batch) * kPaHeads * kPaHeadDim; + orch.output_view.ndims = 2; + orch.output_view.manual_dep = true; + orch.output_view.shapes[0] = kPaHeads; + orch.output_view.shapes[1] = kPaHeadDim; + orch.output_view.strides[0] = kPaHeadDim; + orch.output_view.strides[1] = 1; + orch.output_view.extent_elem_cache = kPaHeads * kPaHeadDim; +} + +PA_DEVICE uint64_t MinU64(uint64_t lhs, uint64_t rhs) { return lhs < rhs ? lhs : rhs; } + +PA_DEVICE uint64_t ReadPaContextLength(const PaOrchestrationState &orch, uint32_t batch) { + if (orch.context_lens_data == nullptr) { + // Compatibility fallback for a backend that has not yet supplied the + // 256-int GM buffer. Exact PA runs must pass a non-null pointer. + // 正式对等运行必须走下方 descriptor+stride 的 GM load;fallback + // 只用于不具备该缓冲区的兼容后端。 + return kPaBlocksPerRequest * kPaBlockSize; + } + const uint64_t flat_index = orch.context_lens.start_offset + + static_cast(batch) * orch.context_lens.strides[0]; + PA_GM const volatile int32_t *value = reinterpret_cast( + orch.context_lens.buffer_addr + flat_index * ElementSize(DataType::Int32) + ); + return static_cast(*value); +} + +PA_DEVICE void PreparePaBlockGroup(PaOrchestrationState &orch, uint64_t block_offset) { + // 输入 block_offset 位于 [0,current_blocks);输出 nblocks 最多64,并计算最后 + // 一个 block 的有效 token 数。Case1 只有一个 group,但仍执行通用边界算术。 + orch.current_block_offset = block_offset; + orch.current_nblocks = MinU64(kPaBlocksPerRequest, orch.current_blocks - block_offset); + const uint64_t last_block_sequence_start = + (block_offset + orch.current_nblocks - 1) * kPaBlockSize; + orch.current_valid_len = MinU64(kPaBlockSize, orch.current_sequence - last_block_sequence_start); +} + +PA_DEVICE void BeginPaBatch(PaOrchestrationState &orch, uint32_t batch) { + // Match paged_attention_orch.cpp: context GM load and block arithmetic + // happen before entering the q scope and before constructing either view. + // 该顺序会影响 Submit 前的指令与访存间隔,故不把长度读取挪进 QK 构参。 + orch.current_batch = batch; + orch.current_sequence = ReadPaContextLength(orch, batch); + orch.current_blocks = (orch.current_sequence + kPaBlockSize - 1) / kPaBlockSize; + MakeBatchViews(orch, batch); +} + +PA_DEVICE void InitPaOrchestration( + PaOrchestrationState &orch, uint32_t batches, PA_GM const volatile int32_t *context_lens_data +) { + // 初始化只建立整轮回放共享的外部 descriptor/create-info 模板;每 batch 的 view、 + // context length、动态 QK/SF shape 和返回 descriptor 留给五阶段流按原顺序更新。 + const uint32_t query_shape[kMaxTensorDims] = {batches * kPaHeads, kPaHeadDim, 0, 0, 0}; + const uint32_t cache_shape[kMaxTensorDims] = { + batches * kPaBlocksPerRequest * kPaBlockSize, kPaHeadDim, 0, 0, 0 + }; + const uint32_t table_shape[kMaxTensorDims] = {batches, kPaMaxBlocksPerRequest, 0, 0, 0}; + const uint32_t context_shape[kMaxTensorDims] = {batches, 0, 0, 0, 0}; + InitExternalTensor(orch.query, kSyntheticQueryBase, query_shape, 2, DataType::Bfloat16, false); + InitExternalTensor(orch.key_cache, kSyntheticKeyBase, cache_shape, 2, DataType::Bfloat16, false); + InitExternalTensor(orch.value_cache, kSyntheticValueBase, cache_shape, 2, DataType::Bfloat16, false); + InitExternalTensor(orch.block_table, kSyntheticBlockTableBase, table_shape, 2, DataType::Int32, false); + const uint64_t context_address = context_lens_data == nullptr + ? kSyntheticContextLensBase + : reinterpret_cast(context_lens_data); + InitExternalTensor(orch.context_lens, context_address, context_shape, 1, DataType::Int32, false); + InitExternalTensor(orch.output, kSyntheticOutputBase, query_shape, 2, DataType::Float32, false); + + const uint32_t tile_shape[kMaxTensorDims] = {kPaHeads, kPaHeadDim, 0, 0, 0}; + const uint32_t scalar_shape[kMaxTensorDims] = {kPaHeads, 0, 0, 0, 0}; + ClearCreateInfo(orch.tile_create_info); + ClearCreateInfo(orch.scalar_create_info); + ClearCreateInfo(orch.qk_create_info); + ClearCreateInfo(orch.sf_create_info); + InitCreateInfo(orch.tile_create_info, tile_shape, 2, DataType::Float32); + InitCreateInfo(orch.scalar_create_info, scalar_shape, 1, DataType::Float32); + + // QK/SF create infos are deliberately not constructed here: in PA they are + // constructed inside the group after Alloc and QK respectively. + // 动态 shape 依赖当前 block group,提前构造既不符合业务数据流,也会 + // 把真实发生在两个 Submit 之间的前端工作错误搬到初始化阶段。 + orch.context_lens_data = context_lens_data; + orch.scale_bits = kPaScaleBits; + orch.current_sequence = 0; + orch.current_blocks = 0; + orch.current_block_offset = 0; + orch.current_nblocks = 0; + orch.current_valid_len = 0; + orch.current_batch = 0; + + orch.accumulated_output = nullptr; + orch.accumulated_sum = nullptr; + orch.accumulated_max = nullptr; + orch.qk_scores = nullptr; + orch.sf_probs = nullptr; + orch.sf_max = nullptr; + orch.sf_sum = nullptr; + orch.pv_output = nullptr; +} + +PA_DEVICE void InitPaOrchestration(PaOrchestrationState &orch, uint32_t batches) { + InitPaOrchestration(orch, batches, nullptr); +} + +PA_DEVICE void BuildAllocArgs(PaOrchestrationState &orch, TaskArgs &args, uint32_t batch) { + BeginPaBatch(orch, batch); + // PA constructs a fresh L0TaskArgs after its two views; Alloc is populated + // without calling reset(). + // 三个 Output 分别成为累计 output/sum/max;Alloc 无 kernel slot,winner 在 + // HeapGuard 后直接发布 task completion。 + ConstructTaskArgs(args); + if (!ReserveTensorArgs(args, 3)) return; + AppendOutput(args, orch.tile_create_info); + AppendOutput(args, orch.scalar_create_info); + AppendOutput(args, orch.scalar_create_info); +} + +PA_DEVICE void BuildQkArgs(PaOrchestrationState &orch, TaskArgs &args, uint32_t batch) { + (void)batch; + // PA computes the block group after Alloc returns, immediately before it + // constructs the dynamic QK output create-info. + // QK 消费 query/key/block-table,产出 score;其 active role 为 AIC。 + PreparePaBlockGroup(orch, 0); + const uint32_t score_shape[kMaxTensorDims] = { + kPaHeads, static_cast(orch.current_nblocks * kPaBlockSize), 0, 0, 0 + }; + // Constructed after Alloc submit and immediately before QK reset/adds. + InitCreateInfo(orch.qk_create_info, score_shape, 2, DataType::Float32); + ResetTaskArgs(args); + if (!ReserveTensorArgs(args, 3)) return; + AppendLocalTensor(args, orch.query_view, TensorArgType::Input); + AppendLocalTensor(args, orch.key_cache, TensorArgType::Input); + AppendLocalTensor(args, orch.block_table, TensorArgType::Input); + AddOutput(args, orch.qk_create_info); + AddTwoScalars( + args, orch.current_nblocks, + static_cast(orch.current_batch) * kPaMaxBlocksPerRequest + orch.current_block_offset + ); +} + +PA_DEVICE void BuildSfArgs(PaOrchestrationState &orch, TaskArgs &args) { + const uint32_t probability_shape[kMaxTensorDims] = { + kPaHeads, static_cast(orch.current_nblocks * kPaBlockSize), 0, 0, 0 + }; + // Constructed only after QK submit returns its sij descriptor. + // SF 通过 qk_scores.owner 得到 QK fanin,产出 probability/max/sum;active role 为 AIV。 + InitCreateInfo(orch.sf_create_info, probability_shape, 2, DataType::Bfloat16); + ResetTaskArgs(args); + AddGmTensor(args, *orch.qk_scores, TensorArgType::Input); + if (!ReserveTensorArgs(args, 3)) return; + AppendOutput(args, orch.sf_create_info); + AppendOutput(args, orch.scalar_create_info); + AppendOutput(args, orch.scalar_create_info); + AddThreeScalars(args, orch.scale_bits, orch.current_nblocks, orch.current_valid_len); +} + +PA_DEVICE void BuildPvArgs(PaOrchestrationState &orch, TaskArgs &args, uint32_t batch) { + (void)batch; + // PV 消费 SF probability 与 value/block-table,owner 形成一条 SF->PV 依赖; + // 结果 pv_output 供最后的 UP 使用,active role 回到 AIC。 + ResetTaskArgs(args); + if (!ReserveTensorArgs(args, 3)) return; + AppendGmTensor(args, *orch.sf_probs, TensorArgType::Input); + AppendLocalTensor(args, orch.value_cache, TensorArgType::Input); + AppendLocalTensor(args, orch.block_table, TensorArgType::Input); + AddOutput(args, orch.tile_create_info); + AddTwoScalars( + args, orch.current_nblocks, + static_cast(orch.current_batch) * kPaMaxBlocksPerRequest + orch.current_block_offset + ); +} + +PA_DEVICE void BuildUpdateArgs(PaOrchestrationState &orch, TaskArgs &args) { + // UP 的 SF max/sum 共享一个 SF owner,PV output 提供一个 PV owner,三个累计 + // Inout 共享 Alloc owner,去重后共 3 条 fanin;output_view 把更新写回当前 batch。 + ResetTaskArgs(args); + if (!ReserveTensorArgs(args, 3)) return; + AppendGmTensor(args, *orch.sf_max, TensorArgType::Input); + AppendGmTensor(args, *orch.sf_sum, TensorArgType::Input); + AppendGmTensor(args, *orch.pv_output, TensorArgType::Input); + if (!ReserveTensorArgs(args, 4)) return; + AppendGmTensor(args, *orch.accumulated_max, TensorArgType::Inout); + AppendGmTensor(args, *orch.accumulated_sum, TensorArgType::Inout); + AppendGmTensor(args, *orch.accumulated_output, TensorArgType::Inout); + AppendLocalTensor(args, orch.output_view, TensorArgType::Inout); + AddTwoScalars( + args, orch.current_block_offset == 0 ? 1 : 0, + orch.current_block_offset + orch.current_nblocks >= orch.current_blocks ? 1 : 0 + ); +} + +PA_DEVICE void AcceptTaskOutputs(PaOrchestrationState &orch, TaskKind kind, const TaskOutputs &outputs) { + // 每个 worker 都完整回放并接收自己 materialize 的 descriptor;只有 Claim winner + // 会执行 kernel,但 loser 的后续 orchestration 仍使用相同 task_id/owner 拓扑。 + switch (kind) { + case TaskKind::Alloc: + orch.accumulated_output = outputs.tensors[0]; + orch.accumulated_sum = outputs.tensors[1]; + orch.accumulated_max = outputs.tensors[2]; + break; + case TaskKind::Qk: + orch.qk_scores = outputs.tensors[0]; + break; + case TaskKind::Sf: + orch.sf_probs = outputs.tensors[0]; + orch.sf_max = outputs.tensors[1]; + orch.sf_sum = outputs.tensors[2]; + break; + case TaskKind::Pv: + orch.pv_output = outputs.tensors[0]; + break; + default: + // UP 只更新既有 Inout,没有新 Output descriptor 需要传给下一阶段。 + break; + } +} + +PA_DEVICE void ResetTensorMap(PA_GM TensorMap &map) { + // TensorMap 完全属于当前 worker,初始化和后续增删都不需要 atomic。bucket 与 + // task_heads 置空后,entry 存储按 high_water 首次分配、再经 free_head 复用。 + map.free_head = -1; + map.high_water = 0; + map.alive_floor = 0; + map.cleaned_upto = 0; + for (uint32_t index = 0; index < kMapBuckets; ++index) { + map.buckets[index] = -1; + } + for (uint32_t index = 0; index < kTaskWindow; ++index) { + map.task_heads[index] = -1; + } +} + +PA_DEVICE uint32_t TensorMapHash(uint64_t address) { + address *= 0x9E3779B97F4A7C15ULL; + return static_cast(address >> (64 - kMapBucketShift)); +} + +template +PA_DEVICE void TensorByteRange(const TensorReference &tensor, uint64_t &address, uint64_t &lo, uint64_t &hi) { + // identity 先按 backing buffer 地址分桶,再用半开字节区间 [lo, hi) 判断 view + // 是否重叠。连续 tensor 由 shape 现算 extent,非连续 tensor 使用缓存 extent。 + const uint64_t element_size = ElementSize(tensor.dtype); + address = tensor.buffer_addr; + lo = tensor.start_offset * element_size; + uint64_t extent; + if (tensor.is_contiguous) { + extent = 1; + for (uint32_t index = 0; index < tensor.ndims; ++index) { + extent *= tensor.shapes[index]; + } + } else { + extent = tensor.extent_elem_cache; + } + hi = (tensor.start_offset + extent) * element_size; +} + +PA_DEVICE int32_t AllocateMapEntry(PA_GM TensorMap &map) { + // 输出为可写 entry 下标:优先复用退休链,之后增长 high_water;返回 -1 表示 + // 固定容量耗尽。分配动作尚未把 entry 接入任何 bucket/task 链。 + if (map.free_head >= 0) { + const int32_t slot = map.free_head; + map.free_head = map.entries[slot].next_in_bucket; + return slot; + } + if (map.high_water < static_cast(kMapCapacity)) { + return map.high_water++; + } + return -1; +} + +PA_DEVICE void FreeMapEntry(PA_GM TensorMap &map, int32_t index) { + // 输入 index 必须仍位于其 bucket 链。输出状态是从双向 bucket 链完整摘除, + // 再把 next_in_bucket 改作 free-list next;task 链由 AdvanceTensorMap 顺序遍历。 + PA_GM MapEntry &entry = map.entries[index]; + if (entry.prev_in_bucket < 0) { + map.buckets[entry.bucket] = entry.next_in_bucket; + } else { + map.entries[entry.prev_in_bucket].next_in_bucket = entry.next_in_bucket; + } + if (entry.next_in_bucket >= 0) { + map.entries[entry.next_in_bucket].prev_in_bucket = entry.prev_in_bucket; + } + entry.bucket = -1; + entry.next_in_bucket = map.free_head; + map.free_head = index; +} + +PA_DEVICE void AdvanceTensorMap(PA_GM TensorMap &map, uint32_t task_id, int32_t heap_window) { + // PrepareMap 在 Claim 前把存活下界推进到 task_id-H。离开窗口的 producer 先按 + // task_heads 找到其全部 entry,再从 bucket 链摘除并进入 free list。TensorMap 与 + // heap 共享窗口宽度 H,但前者按本 worker 的 task_id 推进,后者按跨核连续 + // frontier 推进,二者并不要求同步到达同一位置。 + const int32_t new_floor = static_cast(task_id) - heap_window; + if (new_floor <= map.cleaned_upto) { + if (new_floor > map.alive_floor) { + map.alive_floor = new_floor; + } + return; + } + for (int32_t id = map.cleaned_upto; id < new_floor; ++id) { + int32_t current = map.task_heads[static_cast(id) & kTaskWindowMask]; + while (current >= 0) { + const int32_t next = map.entries[current].next_in_task; + FreeMapEntry(map, current); + current = next; + } + map.task_heads[static_cast(id) & kTaskWindowMask] = -1; + } + map.cleaned_upto = new_floor; + map.alive_floor = new_floor; +} + +template +PA_DEVICE void InsertTensor(PA_GM TensorMap &map, const TensorReference &tensor, int32_t producer) { + // 新 producer 同时插入地址 bucket 的表头与 producer 对应 task 链的表头。 + // map 满时生产语义是静默放弃登记;standalone 保持该行为,不新增异常分支。 + uint64_t address = 0; + uint64_t lo = 0; + uint64_t hi = 0; + TensorByteRange(tensor, address, lo, hi); + const int32_t slot = AllocateMapEntry(map); + if (slot < 0) { + return; + } + const uint32_t bucket = TensorMapHash(address); + PA_GM MapEntry &entry = map.entries[slot]; + entry.buffer_addr = address; + entry.lo = lo; + entry.hi = hi; + entry.producer = producer; + entry.bucket = static_cast(bucket); + entry.prev_in_bucket = -1; + entry.next_in_bucket = map.buckets[bucket]; + if (map.buckets[bucket] >= 0) { + map.entries[map.buckets[bucket]].prev_in_bucket = slot; + } + map.buckets[bucket] = slot; + const uint32_t task_slot = static_cast(producer) & kTaskWindowMask; + entry.next_in_task = map.task_heads[task_slot]; + map.task_heads[task_slot] = slot; +} + +template +PA_DEVICE int32_t LookupTensor(PA_GM const TensorMap &map, const TensorReference &tensor) { + uint64_t address = 0; + uint64_t lo = 0; + uint64_t hi = 0; + TensorByteRange(tensor, address, lo, hi); + int32_t best = -1; + // 同一 buffer 可能存在多个历史写者;只考虑仍存活且区间重叠的 entry,并选择 + // task_id 最大的最新 producer,防止依赖回退到更老版本。 + for (int32_t current = map.buckets[TensorMapHash(address)]; current >= 0; + current = map.entries[current].next_in_bucket) { + PA_GM const MapEntry &entry = map.entries[current]; + if (entry.producer < map.alive_floor) { + continue; + } + if (entry.buffer_addr == address && lo < entry.hi && entry.lo < hi && entry.producer > best) { + best = entry.producer; + } + } + return best; +} + +PA_DEVICE uint64_t TensorOwner(const TaskTensorRef &reference) { + // CreateInfo 只会出现在 tag=Output 且在 fanin 前已被跳过;这里的输入不变量是 + // LocalTensor/GmTensor,输出为显式 owner 或 kInvalidTaskId。 + if (reference.kind == TensorRefKind::GmTensor) { + return reference.pointer.gm_tensor->owner_task_id; + } + return reference.pointer.local_tensor->owner_task_id; +} + +PA_DEVICE int32_t LookupTensorRef(PA_GM const TensorMap &map, const TaskTensorRef &reference) { + // 与 TensorOwner 相同,此辅助入口只接收已存在 descriptor;返回最新重叠 producer, + // 未登记或已退休则返回 -1。 + if (reference.kind == TensorRefKind::GmTensor) { + return LookupTensor(map, *reference.pointer.gm_tensor); + } + return LookupTensor(map, *reference.pointer.local_tensor); +} + +PA_DEVICE void AddFanin(int32_t fanin[kMaxFanin], uint32_t &count, int32_t producer) { + // owner 与 TensorMap lookup 可能指向同一 producer,先去重再写固定 16 槽数组; + // Case1 的最大 fanin 为 UP 的 3,正常路径不会截断。 + if (producer < 0) { + return; + } + for (uint32_t index = 0; index < count; ++index) { + if (fanin[index] == producer) { + return; + } + } + if (count < kMaxFanin) { + fanin[count++] = producer; + } +} + +PA_DEVICE uint32_t CollectFanin( + PA_GM const TensorMap &map, const TaskArgs &args, int32_t fanin[kMaxFanin] +) { + // fanin 只由 winner 收集:先吸收 descriptor 的显式 owner,再对 Input/Inout + // 查询最新重叠写者;纯 Output 尚未存在,不应成为本次 task 的输入依赖。 + uint32_t count = 0; + for (int32_t index = 0; index < args.tensor_count; ++index) { + const TensorArgType tag = TaskTag(args, static_cast(index)); + if (tag == TensorArgType::Output) { + continue; + } + const TaskTensorRef &reference = args.tensors[index]; + // Keep the two address spaces in separate control-flow arms. CCEC's + // O2/O3 backend rejects a merged pointer phi even when both arms only + // feed scalar field loads; this is also how PA's production helper is + // written. + // 分支重复是后端约束与生产写法的一部分,不应抽成一个混合地址空间指针。 + if (reference.kind == TensorRefKind::GmTensor) { + PA_GM const TensorDesc &tensor = *reference.pointer.gm_tensor; + const uint64_t owner = tensor.owner_task_id; + if (owner != kInvalidTaskId) { + AddFanin(fanin, count, static_cast(owner & 0xFFFFFFFFU)); + } + if (tag == TensorArgType::Input || tag == TensorArgType::Inout) { + AddFanin(fanin, count, LookupTensor(map, tensor)); + } + } else { + const TensorDesc &tensor = *reference.pointer.local_tensor; + const uint64_t owner = tensor.owner_task_id; + if (owner != kInvalidTaskId) { + AddFanin(fanin, count, static_cast(owner & 0xFFFFFFFFU)); + } + if (tag == TensorArgType::Input || tag == TensorArgType::Inout) { + AddFanin(fanin, count, LookupTensor(map, tensor)); + } + } + } + return count; +} + +PA_DEVICE void InsertExistingTensor(SubmitContext &context, const TaskArgs &args, int32_t index) { + // 输入 index 来自 register_mask,故必为已有 descriptor 而非 CreateInfo;写入结果 + // 只影响 context.self 对应 worker 的 map,并把当前 task_id 登记为新的 hazard 版本。 + const TaskTensorRef &reference = args.tensors[index]; + if (reference.kind == TensorRefKind::GmTensor) { + InsertTensor(context.self->map, *reference.pointer.gm_tensor, context.task_id); + } else { + InsertTensor(context.self->map, *reference.pointer.local_tensor, context.task_id); + } +} + +PA_DEVICE void RegisterOutputs(SubmitContext &context, const TaskArgs &args, bool include_existing) { + // register_mask 只覆盖 Inout/OutputExisting。新 Output 已带本次 owner;现有 + // backing buffer 的新写者则必须登记到本 worker TensorMap,供后继 task 查 hazard。 + if (!include_existing) { + return; + } + uint32_t register_mask = context.register_mask; + for (uint32_t index = 0; register_mask != 0; ++index, register_mask >>= 1) { + if ((register_mask & 1U) != 0) { + InsertExistingTensor(context, args, static_cast(index)); + } + } +} + +PA_DEVICE uint64_t FrontendAlignUp(uint64_t value, uint64_t alignment) { + // alignment 在本模型中固定为2的幂1 KiB;返回逻辑 heap 地址,不做 ring 取模。 + return (value + alignment - 1) & ~(alignment - 1); +} + +PA_DEVICE void BeginSubmit( + PA_GM WorkerState &worker, const TaskArgs &args, SubmitContext &context +) { + // Mirrors dist_submit_begin(). The production Submit and Materialize spans + // both start after this per-call context initialization. + // local_index 在所有 worker 上按同一 orchestration 顺序递增,因此 + // task_id 一致;该初始化位于 Submit 计时起点之前,不能误计进阶段耗时。 + const uint32_t task_id = static_cast(worker.local_index++); + context.self = &worker; + context.payload = &worker.payloads[task_id & kPayloadMask]; + context.task_id = static_cast(task_id); + context.tensor_count = args.tensor_count; + context.scalar_count = args.scalar_count; + context.result.task_id = task_id; + context.result.count = 0; + context.register_mask = 0; + context.output_bytes = 0; + context.fanin_count = 0; + context.kernel_id = -1; + context.won = false; + context.joint = false; + context.joint_init = false; + context.joint_block = -1; + context.joint_slot = -1; + context.joint_count = 0; +} + +PA_DEVICE bool MaterializeTask( + PA_GM WorkerState &worker, uint32_t task_id, const TaskArgs &args, SubmitContext &context, + uint64_t heap_base, uint64_t heap_size +) { + // 输入是 BeginSubmit 已绑定的 payload/context 与当前 worker.heap_next;成功输出 + // 包括本 task 的 GM TensorDesc 指针、output_bytes 和推进后的单调 heap_next。 + // 失败不得进入 Claim/slot 流程,由上层设置 fatal 并终止该 worker 回放。 + if (context.payload == nullptr) { + return false; + } + context.tensor_count = args.tensor_count; + context.scalar_count = args.scalar_count; + context.register_mask = 0; + + // DistOutputLayout leaves non-output slots lazy and writes only the sizes + // selected by output_mask. + // 第一次 tag 扫描同时产生 output_mask/register_mask;第二次只遍历 + // Output 位,避免读取未初始化的非输出 buffer_sizes。 + OutputLayout layout; + layout.total_output_size = 0; + uint32_t output_mask = 0; + for (int32_t index = 0; index < args.tensor_count; ++index) { + const TensorArgType tag = TaskTag(args, static_cast(index)); + if (tag == TensorArgType::Inout || tag == TensorArgType::OutputExisting) { + context.register_mask |= 1U << index; + } + if (tag != TensorArgType::Output) { + continue; + } + output_mask |= 1U << index; + layout.buffer_sizes[index] = CreateInfoBytes(*args.tensors[index].pointer.create_info); + layout.total_output_size += FrontendAlignUp(layout.buffer_sizes[index], kOutputAlignment); + } + + uint64_t task_base = FrontendAlignUp(worker.heap_next, kOutputAlignment); + const uint64_t total = layout.total_output_size; + if (total > heap_size || (total != 0 && heap_base == 0)) { + return false; + } + if (total != 0 && (task_base % heap_size) + total > heap_size) { + // 单个 task 的输出必须物理连续;若跨 ring 尾部则把逻辑 task_base 推到 + // 下一圈起点。heap_next 仍保持单调,不在这里取模。 + task_base = (task_base / heap_size + 1) * heap_size; + } + + uint64_t output_offset = 0; + // 各 Output 在同一 task_base 内按参数顺序排布;result 只收集 Output,索引与 + // TaskArgs 中非输出槽无关,而 payload 仍按原参数 index 保存 descriptor。 + for (int32_t index = 0; output_mask != 0; ++index, output_mask >>= 1) { + if ((output_mask & 1U) == 0) { + continue; + } + const uint64_t physical = (task_base + output_offset) % heap_size; + PA_GM TensorDesc &tensor = context.payload->tensors[index]; + if (!InitTensorFromCreateInfo( + tensor, *args.tensors[index].pointer.create_info, heap_base + physical, layout.buffer_sizes[index] + )) { + return false; + } + tensor.owner_task_id = task_id; + context.result.tensors[context.result.count++] = &tensor; + output_offset += FrontendAlignUp(layout.buffer_sizes[index], kOutputAlignment); + } + worker.heap_next = task_base + total; + context.output_bytes = total; + return true; +} + +PA_DEVICE void CopyTensorFromRef(PA_GM TensorDesc &destination, const TaskTensorRef &reference) { + // slot 必须拥有 descriptor 快照,不能保存指向 orchestration 栈对象的引用; + // 按 byte volatile copy 同时兼容 local/GM 源并保留真实 128-byte 搬运量。 + PA_GM volatile uint8_t *destination_bytes = reinterpret_cast(&destination); + if (reference.kind == TensorRefKind::GmTensor) { + PA_GM const volatile uint8_t *source_bytes = + reinterpret_cast(reference.pointer.gm_tensor); + for (uint32_t byte = 0; byte < sizeof(TensorDesc); ++byte) { + destination_bytes[byte] = source_bytes[byte]; + } + return; + } + const volatile uint8_t *source_bytes = + reinterpret_cast(reference.pointer.local_tensor); + for (uint32_t byte = 0; byte < sizeof(TensorDesc); ++byte) { + destination_bytes[byte] = source_bytes[byte]; + } +} + +PA_DEVICE void CopyGmTensor(PA_GM TensorDesc &destination, PA_GM const TensorDesc &source) { + // 新 Output 的源 descriptor 已位于 GM payload;单独入口避免把 GM 指针误走 + // local 地址空间分支,输出仍是 slot 内独立副本。 + PA_GM volatile uint8_t *destination_bytes = reinterpret_cast(&destination); + PA_GM const volatile uint8_t *source_bytes = reinterpret_cast(&source); + for (uint32_t byte = 0; byte < sizeof(TensorDesc); ++byte) { + destination_bytes[byte] = source_bytes[byte]; + } +} + +PA_DEVICE void PopulateSlotPayload( + PA_GM LocalSlot &slot, const TaskArgs &args, const SubmitContext &context, const int32_t fanin[kMaxFanin], + uint32_t fanin_count, int32_t sub_block_id, bool is_multicore, int32_t won_block, int32_t won_slot +) { + // winner 将活动 descriptor/scalar 复制进私有 slot,dispatch args 指向 slot 内 + // 副本而非 orchestration 临时对象;fanin 随 slot 保存,kernel 执行前逐 flag 检查。 + slot.tensor_count = context.tensor_count; + slot.scalar_count = context.scalar_count; + for (int32_t index = 0; index < context.tensor_count; ++index) { + if (TaskTag(args, static_cast(index)) == TensorArgType::Output) { + CopyGmTensor(slot.tensors[index], context.payload->tensors[index]); + } else { + CopyTensorFromRef(slot.tensors[index], args.tensors[index]); + } + slot.args[index] = static_cast(reinterpret_cast(&slot.tensors[index])); + } + for (int32_t index = 0; index < context.scalar_count; ++index) { + slot.scalars[index] = args.scalars[index]; + slot.args[context.tensor_count + index] = args.scalars[index]; + } + + PA_GM PaLocalContext &local = + *reinterpret_cast(&slot.local_context[0]); + // standalone 每个 task 只由一个 lane kernel 执行,故 block_index/count 固定0/1; + // async completion 未启用,task_token 保持 invalid,与 PA 普通同步 slot 一致。 + local.block_index = 0; + local.block_count = 1; + local.async.completion_count = 0; + local.async.completion_error_code = 0; + local.async.completion_entries = 0; + local.async.completion_capacity = 0; + local.async.task_token = kInvalidTaskId; + slot.global_context = static_cast(sub_block_id); + slot.args[kSpmdLocalContextIndex] = + static_cast(reinterpret_cast(&slot.local_context[0])); + slot.args[kSpmdGlobalContextIndex] = + static_cast(reinterpret_cast(&slot.global_context)); + slot.fanin_count = fanin_count; + // fanin 数组只复制有效前缀;执行端以 fanin_count 为边界,未使用尾部保持惰性。 + for (uint32_t index = 0; index < fanin_count; ++index) { + slot.fanin[index] = fanin[index]; + } + slot.is_multicore = is_multicore; + slot.won_block = won_block; + slot.won_slot = won_slot; +} + +PA_DEVICE void BuildSlotPayload( + PA_GM LocalSlot &slot, uint32_t task_id, uint32_t function_id, uint64_t function_address, const TaskArgs &args, + const SubmitContext &context, const int32_t fanin[kMaxFanin], uint32_t fanin_count, int32_t sub_block_id = 0, + bool is_multicore = false, int32_t won_block = -1, int32_t won_slot = -1 +) { + // Match build_ring_slot_from_submit ordering: publish the header first, + // then copy the active descriptors/scalars and construct dispatch payload. + // slot 仅由所属 worker 消费,这里的写入次序用于复刻真实构建成本与 + // 状态机;跨核可见性由 task completion 的 flag/vend 协议承担。 + slot.occupied = true; + slot.task_id = task_id; + slot.kind = function_id; + slot.function_address = function_address; + slot.built = 1; + PopulateSlotPayload( + slot, args, context, fanin, fanin_count, sub_block_id, is_multicore, won_block, won_slot + ); +} + +// Compatibility overload for a core that has already populated the slot +// header before calling the PA frontend. +// 该入口只补 payload,不改变既有 task/function 头;输出不变量与完整 +// BuildSlotPayload 相同,均得到 built 且可由 DrainReady 检查 fanin 的私有 slot。 +PA_DEVICE void BuildSlotPayload( + PA_GM LocalSlot &slot, const TaskArgs &args, const SubmitContext &context, const int32_t fanin[kMaxFanin], + uint32_t fanin_count +) { + slot.built = 1; + PopulateSlotPayload(slot, args, context, fanin, fanin_count, 0, false, -1, -1); +} + +} // namespace pa_scheduler + +#endif // PA_SCHEDULER_COMMON_PA_FRONTEND_H diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h new file mode 100644 index 0000000000..ec7f0b9a22 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -0,0 +1,567 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_COMMON_PA_MODEL_H +#define PA_SCHEDULER_COMMON_PA_MODEL_H + +#include +#include + +namespace pa_scheduler { + +// 这里固定的是 PA Case1 的调度拓扑,而不是为了缩小 standalone 人为选择的规模: +// 每个 batch 依次回放 Alloc/QK/SF/PV/UP 五个 task,32 个 AIC 与 64 个 AIV +// 都执行同一条 orchestration 流,只在 Claim 时按 task 的 active role 分流。 +constexpr uint32_t kDefaultBatches = 256; +constexpr uint32_t kMaxBatches = 256; +constexpr uint32_t kTasksPerBatch = 5; +constexpr uint32_t kMaxTasks = kMaxBatches * kTasksPerBatch; +constexpr uint32_t kTaskCellCapacity = 1U << 16; + +constexpr uint32_t kAicWorkers = 32; +constexpr uint32_t kAivWorkers = 64; +constexpr uint32_t kWorkers = kAicWorkers + kAivWorkers; +constexpr uint32_t kRuntimeMaxWorkers = 108; +constexpr uint32_t kCursorShards = 4; +// 每个 worker 私有 ring 有 4 个物理 slot,其中 2 个为 BlockWon 协议预留; +// 单 lane Case1 虽不进入 BlockWon,普通 kernel 仍只能占用剩余 2 个 slot。 +constexpr uint32_t kPrivateSlots = 4; +constexpr uint32_t kWonReserve = 2; +constexpr uint32_t kUsableSlots = kPrivateSlots - kWonReserve; +constexpr uint32_t kMaxFanin = 16; +// H=64 同时约束 heap 可回收 frontier 和 TensorMap producer 的存活下界。 +// heap_next 使用单调逻辑地址;真正落到 256 MiB 环形 heap 时才取模,因而可判断覆盖风险。 +constexpr uint32_t kHeapWindow = 64; +constexpr uint64_t kHeapBytes = 256ULL << 20; +constexpr uint64_t kSyntheticHeapBase = 0x100000000ULL; +constexpr uint64_t kOutputAlignment = 1024; +constexpr uint32_t kMaxTensorDims = 5; +constexpr uint32_t kMaxTaskTensors = 32; +constexpr uint32_t kMaxTaskScalars = 16; +constexpr uint32_t kPayloadSlots = 2048; +constexpr uint32_t kPayloadMask = kPayloadSlots - 1; +constexpr uint32_t kPayloadStride = 4096; +constexpr uint32_t kMapCapacity = 16384; +constexpr uint32_t kMapBuckets = 1 << 13; +constexpr uint32_t kMapBucketShift = 13; +constexpr uint32_t kTaskWindow = 1 << 10; +constexpr uint32_t kTaskWindowMask = kTaskWindow - 1; +constexpr uint64_t kSystemCounterHz = 1000000000ULL; +constexpr uint64_t kWatchdogTicks = 2 * kSystemCounterHz; +// 下列 offset/size 来自真实 DistGlobal/DistCore ABI。standalone 保留被测关键字段的 +// offset、DistCore ABI 和 kRealDistGlobalBytes 总跨度;其余区域可用 opaque padding, +// 并不是对生产结构全部字段的逐一镜像。 +constexpr size_t kRealDistCoreOffset = 10043904; +constexpr size_t kRealDistGlobalBytes = 1007023872; +constexpr size_t kRealTasksOffset = 896; +constexpr size_t kRealFatalOffset = 4195264; +constexpr size_t kRealReplayDoneOffset = 10043776; +constexpr size_t kRealStartedCountOffset = 10043840; +constexpr uint32_t kTraceRecordsPerCore = 1U << 16; +static_assert((kPayloadSlots & kPayloadMask) == 0, "payload slots must be a power of two"); + +// These are the measured means from the best PA A5 trace, in 1 GHz ticks. +// The CCEC stage calibrates the NOP counts against these targets before the +// defaults are considered final. +// 这里只用 NOP 代替四个计算 kernel 的执行体;Submit、依赖、heap 与 +// completion 路径均不靠 NOP 补时。target 是真实泳道均值,不是调度阶段预算。 +constexpr uint32_t kTargetQkTicks = 44170; +constexpr uint32_t kTargetSfTicks = 53729; +constexpr uint32_t kTargetPvTicks = 27626; +constexpr uint32_t kTargetUpTicks = 1565; + +// Calibrated on the local A5 with the CCEC RuntimeNop implementation. These +// counts resolve to the measured targets above; they are not cycle guesses. +constexpr uint32_t kDefaultQkNops = 129600; +constexpr uint32_t kDefaultSfNops = 157900; +constexpr uint32_t kDefaultPvNops = 79950; +constexpr uint32_t kDefaultUpNops = 2400; + +enum class CoreRole : uint32_t { + Aic = 0, + Aiv = 1, +}; + +// task_id % 5 即 kind;该周期性是不变量,既决定 Claim cursor/active role, +// 也决定输出大小、fanin 拓扑和 NOP kernel 的选择。 +enum class TaskKind : uint32_t { + Alloc = 0, + Qk = 1, + Sf = 2, + Pv = 3, + Up = 4, + Count = 5, +}; + +// 记录 kernel 最终在哪次 drain 中落地:Submit 开头、slot/heap 背压期间,或 +// 所有 worker 回放结束后的最终清空。三者之和必须等于实际 kernel 数。 +enum class DrainPlace : uint32_t { + EfDrain = 0, + RingBackpressure = 1, + FinalDrain = 2, + Count = 3, +}; + +enum class TensorArgType : int32_t { + Input = 0, + Output = 1, + Inout = 2, + OutputExisting = 3, + NoDependency = 4, +}; +// Input 作为 kernel 输入并参与依赖、但不登记为写者;Output 由本次 Submit 在 heap 中物化; +// Inout 与 OutputExisting 还需登记进每 worker 私有 TensorMap,供后续重叠区间查询 producer。 +static_assert(sizeof(TensorArgType) == sizeof(int32_t), "TensorArgType must match the PA tag ABI"); + +enum class DataType : uint8_t { + Float32 = 0, + Float16 = 1, + Int32 = 2, + Int16 = 3, + Int8 = 4, + Uint8 = 5, + Bfloat16 = 6, + Int64 = 7, + Uint64 = 8, + Uint16 = 9, + Uint32 = 10, + Bool = 11, + Count = 12, +}; + +// ProfilePhase 是聚合计数下标,TracePhase 是原始泳道事件 ABI;二者故意分离, +// 不能假设枚举值相同。一次 trace 写入可同时归入一个不同命名的 profile 阶段。 +enum class ProfilePhase : uint32_t { + Orchestration = 0, + Submit = 1, + EfDrain = 2, + Materialize = 3, + PrepareMap = 4, + Claim = 5, + Fanin = 6, + Register = 7, + WaitForSlot = 8, + HeapGuard = 9, + Build = 10, + ReplayTail = 11, + Count = 12, +}; + +struct NopCounts { + uint32_t qk; + uint32_t sf; + uint32_t pv; + uint32_t up; +}; + +// RunConfig 是 host 在 launch 前写、worker 启动时只读的控制 cache line。 +// 输入为 batch/NOP/诊断开关;输出不回写这里,而发布到独立 WorkerResult。 +struct alignas(64) RunConfig { + uint32_t batches; + uint32_t workers; + NopCounts nops; + uint32_t profile_phases; + uint32_t trace_enabled; + uint64_t trace_base; + uint32_t trace_records_per_core; + uint32_t reserved[5]; +}; +static_assert(sizeof(RunConfig) == 64, "RunConfig must occupy one cache line"); + +enum class TracePhase : int32_t { + Kernel = 0, + Alloc = 1, + Build = 2, + DrainWon = 3, + Replay = 4, + RingBp = 5, + EfDrain = 6, + Commit = 7, + Submit = 8, + Materialize = 9, + PrepareMap = 10, + Claim = 11, + Fanin = 12, + Register = 13, +}; + +struct alignas(64) TraceCoreState { + volatile uint32_t count; + volatile uint32_t dropped; + uint32_t padding[14]; +}; +// 每个 worker 独占一个计数 cache line 和一段定长 records,不需要为了写 trace +// 再引入跨核 atomic;满容量后只增加本 worker 的 dropped。 +static_assert(sizeof(TraceCoreState) == 64, "trace core state must occupy one cache line"); + +struct alignas(64) TraceHeader { + uint32_t magic; + uint32_t version; + uint32_t num_cores; + uint32_t records_per_core; + uint64_t frequency_hz; + TraceCoreState cores[kRuntimeMaxWorkers]; +}; +// 本 benchmark 固定物理分配 kWorkers=96 个定长 record 分区,合法 header 也要求 +// num_cores==96;其 header/record 布局和 phase 编号可转换为真实泳道使用的 JSON。 +static_assert(sizeof(TraceHeader) == 6976, "trace header must match PA swimlane layout"); + +struct alignas(64) TraceRecord { + uint64_t start_cycle; + uint64_t end_cycle; + int32_t task_id; + int32_t function_id; + int32_t phase; + int32_t lane; + int32_t block_id; + int32_t core_idx; + uint32_t flags; + uint32_t auxiliary; +}; +// start/end 保留原始 1 GHz counter;task/function/物理 lane 用于离线还原轨道。 +// flags/aux 的含义由 phase 决定,例如 winner、Alloc 或 RingBp 类型,不参与调度决策。 +static_assert(sizeof(TraceRecord) == 64, "trace record must occupy one cache line"); + +constexpr size_t kTraceBytes = + sizeof(TraceHeader) + static_cast(kWorkers) * kTraceRecordsPerCore * sizeof(TraceRecord); + +struct alignas(64) AtomicLine { + volatile int64_t value; + uint8_t padding[64 - sizeof(int64_t)]; +}; +// 热点共享量各占一个 cache line,保持生产代码的地址隔离,避免 standalone +// 因伪共享额外放大 Claim/frontier/start barrier 的竞争。 +static_assert(sizeof(AtomicLine) == 64, "AtomicLine must occupy one cache line"); + +struct alignas(64) AtomicFlagLine { + volatile int32_t value; + uint8_t padding[64 - sizeof(int32_t)]; +}; +// 32-bit fatal 与 64-bit cursor 使用不同封装,但都独占 cache line;成功路径中 +// fatal 始终为零,任何写一都表示协议已终止,不能作为普通等待条件清除。 +static_assert(sizeof(AtomicFlagLine) == 64, "AtomicFlagLine must occupy one cache line"); + +struct alignas(64) TaskCell { + volatile int64_t flag; + volatile uint64_t vend; + uint8_t padding[64 - 2 * sizeof(int64_t)]; +}; +// flag 是依赖就绪与 frontier 连续前推的发布位;vend 是该 task 完成时 worker 的 +// 单调 heap_next 快照。HeapGuard 读取 frontier-H 对应 vend,判断环形 heap 是否可覆盖。 +static_assert(sizeof(TaskCell) == 64, "TaskCell must occupy one cache line"); + +// TensorDesc 保留真实 Tensor 的两条 64-byte 数据线。owner_task_id 表达显式生产者, +// buffer_addr + 字节区间用于 TensorMap 发现同一 backing buffer 上的读写依赖。 +struct TensorDesc { + uint64_t buffer_addr; + uint64_t buffer_size; + uint64_t owner_task_id; + uint64_t start_offset; + int32_t version; + uint32_t ndims; + DataType dtype; + bool manual_dep; + bool is_contiguous; + uint8_t child_memory; + uint32_t shapes[kMaxTensorDims]; + + uint64_t extent_elem_cache; + uint32_t strides[kMaxTensorDims]; + uint8_t padding[36]; +}; +static_assert(sizeof(TensorDesc) == 128, "TensorDesc must match the PA Tensor ABI size"); +static_assert(offsetof(TensorDesc, buffer_addr) == 0, "TensorDesc buffer offset mismatch"); +static_assert(offsetof(TensorDesc, owner_task_id) == 16, "TensorDesc owner offset mismatch"); +static_assert(offsetof(TensorDesc, start_offset) == 24, "TensorDesc view offset mismatch"); +static_assert(offsetof(TensorDesc, version) == 32, "TensorDesc version offset mismatch"); +static_assert(offsetof(TensorDesc, shapes) == 44, "TensorDesc shape offset mismatch"); +static_assert(offsetof(TensorDesc, extent_elem_cache) == 64, "TensorDesc extent offset mismatch"); +static_assert(offsetof(TensorDesc, strides) == 72, "TensorDesc stride offset mismatch"); + +struct TensorCreateInfo { + uint64_t initial_value; + bool has_initial_value; + uint8_t padding0[7]; + uint64_t reserved0; + uint64_t start_offset; + int32_t version; + uint32_t ndims; + DataType dtype; + bool manual_dep; + bool is_contiguous; + uint8_t child_memory; + uint32_t shapes[kMaxTensorDims]; +}; +// CreateInfo 只描述尚未分配的 Output;Materialize 根据形状和 dtype 计算大小, +// 再把它变成位于 worker 逻辑 heap 上的 TensorDesc。 +static_assert(sizeof(TensorCreateInfo) == 64, "TensorCreateInfo must match the PA create-info ABI size"); +static_assert(offsetof(TensorCreateInfo, start_offset) == 24, "TensorCreateInfo start offset mismatch"); +static_assert(offsetof(TensorCreateInfo, version) == 32, "TensorCreateInfo version offset mismatch"); +static_assert(offsetof(TensorCreateInfo, shapes) == 44, "TensorCreateInfo shape offset mismatch"); + +struct MapEntry { + uint64_t buffer_addr; + uint64_t lo; + uint64_t hi; + int32_t producer; + int32_t bucket; + int32_t next_in_bucket; + int32_t prev_in_bucket; + int32_t next_in_task; +}; +// 同一 entry 同时挂在两条链上:bucket 链按 buffer 地址查询重叠区间,task 链按 +// producer 批量退休。next_in_bucket 在空闲状态下复用为 free-list 链接。 +static_assert(sizeof(MapEntry) == 48, "MapEntry must match the PA tensor-map entry ABI"); +static_assert(offsetof(MapEntry, producer) == 24, "MapEntry producer offset mismatch"); +static_assert(offsetof(MapEntry, next_in_task) == 40, "MapEntry task-link offset mismatch"); + +struct TensorMap { + MapEntry entries[kMapCapacity]; + int32_t buckets[kMapBuckets]; + int32_t task_heads[kTaskWindow]; + int32_t free_head; + int32_t high_water; + int32_t alive_floor; + int32_t cleaned_upto; +}; +// TensorMap 是 worker 私有状态,不在多核间共享。alive_floor 表达查询存活下界, +// cleaned_upto 表达已物理摘链的进度;即使 Case1 中通常同步推进,也不能合并其 ABI 字段。 +static_assert(sizeof(TensorMap) == 823312, "TensorMap must match the PA fixed-capacity layout"); +static_assert(offsetof(TensorMap, buckets) == 786432, "TensorMap bucket offset mismatch"); +static_assert(offsetof(TensorMap, task_heads) == 819200, "TensorMap task-head offset mismatch"); +static_assert(offsetof(TensorMap, free_head) == 823296, "TensorMap control offset mismatch"); + +struct TaskPayload { + TensorDesc tensors[kMaxTaskTensors]; +}; +// task_id 通过 kPayloadMask 映射到 2048 个 4 KiB payload;Case1 只有 1280 个 task, +// 本轮不会回绕,但仍保留生产容量、寻址方式和 4 KiB stride。 +static_assert(sizeof(TaskPayload) == kPayloadStride, "TaskPayload must preserve the real 4 KiB task stride"); +static_assert(alignof(TaskPayload) == 8, "TaskPayload alignment must match DistTaskPayload"); +static_assert(offsetof(TaskPayload, tensors) == 0, "TaskPayload tensor offset mismatch"); + +struct LocalSlot { + // occupied 先保留容量,built 表示 payload 已按生产顺序构建;task/function + // 标识决定执行哪个 NOP 体,后续大数组则是 kernel 真正看到的参数快照。 + bool occupied; + bool built; + uint8_t header_padding[2]; + uint32_t task_id; + uint32_t kind; + uint32_t function_padding; + uint64_t function_address; + uint32_t tensor_count; + uint32_t scalar_count; + uint8_t tensor_padding[32]; + + TensorDesc tensors[kMaxTaskTensors]; + uint64_t scalars[kMaxTaskScalars]; + uint64_t args[kMaxTaskTensors + kMaxTaskScalars + 2]; + union { + struct { + uint8_t local_context[48]; + uint32_t global_context; + int32_t fanin[kMaxFanin]; + uint32_t fanin_count; + }; + // Compatibility view used by the standalone NOP payload builder. The + // first six words are the real 48-byte LocalContext; the remaining + // words overlap GlobalContext and the beginning of fanin, exactly as + // dictated by the real RingSlot offsets. + // 该视图只用于按真实 offset 填充 dispatch context,不增加另一份 + // 存储;修改其中后两字会同步覆盖 GlobalContext/fanin 的对应 ABI 字节。 + uint64_t context_words[8]; + }; + bool is_multicore; + int32_t won_block; + int32_t won_slot; +}; +// LocalSlot 是每个 winner 写入自己私有 ring 的完整 dispatch 包。fanin 在执行前 +// 逐项检查 task.flag;occupied/built 与计数共同约束最多两个普通 kernel 在途。 +static_assert(sizeof(LocalSlot) == 4824, "LocalSlot must match the PA RingSlot ABI size"); +static_assert(alignof(LocalSlot) == 8, "LocalSlot alignment must match RingSlot"); +static_assert(offsetof(LocalSlot, occupied) == 0, "LocalSlot occupied offset mismatch"); +static_assert(offsetof(LocalSlot, built) == 1, "LocalSlot built offset mismatch"); +static_assert(offsetof(LocalSlot, task_id) == 4, "LocalSlot task offset mismatch"); +static_assert(offsetof(LocalSlot, kind) == 8, "LocalSlot function-id offset mismatch"); +static_assert(offsetof(LocalSlot, function_address) == 16, "LocalSlot function address offset mismatch"); +static_assert(offsetof(LocalSlot, tensor_count) == 24, "LocalSlot tensor-count offset mismatch"); +static_assert(offsetof(LocalSlot, tensors) == 64, "LocalSlot tensor payload offset mismatch"); +static_assert(offsetof(LocalSlot, scalars) == 4160, "LocalSlot scalar payload offset mismatch"); +static_assert(offsetof(LocalSlot, args) == 4288, "LocalSlot dispatch-args offset mismatch"); +static_assert(offsetof(LocalSlot, local_context) == 4688, "LocalSlot local-context offset mismatch"); +static_assert(offsetof(LocalSlot, global_context) == 4736, "LocalSlot global-context offset mismatch"); +static_assert(offsetof(LocalSlot, fanin) == 4740, "LocalSlot fanin offset mismatch"); +static_assert(offsetof(LocalSlot, fanin_count) == 4804, "LocalSlot fanin-count offset mismatch"); +static_assert(offsetof(LocalSlot, is_multicore) == 4808, "LocalSlot multicore offset mismatch"); +static_assert(offsetof(LocalSlot, won_block) == 4812, "LocalSlot won-block offset mismatch"); +static_assert(offsetof(LocalSlot, won_slot) == 4816, "LocalSlot won-slot offset mismatch"); + +struct WorkerState { + CoreRole role; + int32_t core_idx; + int32_t block_id; + int32_t lane; + int32_t sub_block_id; + int32_t local_index; + uint64_t heap_next; + TensorMap map; + uint8_t slot_padding[16]; + LocalSlot slots[kPrivateSlots]; + uint32_t occupied_count; + uint32_t owned_total; + uint64_t swimlane_last_cycle; + uint8_t payload_padding[16]; + TaskPayload payloads[kPayloadSlots]; +}; +// 每个物理 worker 都持有独立 heap cursor、TensorMap、ring 与 task payload arena; +// 多核共享的只有 SchedulerState 前缀中的 cursor/task/frontier 等协议状态。 +static_assert(sizeof(WorkerState) == 9231296, "WorkerState must match the PA DistCore ABI size"); +static_assert(alignof(WorkerState) == 8, "WorkerState alignment must match DistCore"); +static_assert(offsetof(WorkerState, role) == 0, "WorkerState role offset mismatch"); +static_assert(offsetof(WorkerState, local_index) == 20, "WorkerState replay-index offset mismatch"); +static_assert(offsetof(WorkerState, heap_next) == 24, "WorkerState heap cursor offset mismatch"); +static_assert(offsetof(WorkerState, map) == 32, "WorkerState tensor-map offset mismatch"); +static_assert(offsetof(WorkerState, slots) == 823360, "WorkerState ring-slot offset mismatch"); +static_assert(offsetof(WorkerState, occupied_count) == 842656, "WorkerState occupancy offset mismatch"); +static_assert(offsetof(WorkerState, owned_total) == 842660, "WorkerState owned-count offset mismatch"); +static_assert(offsetof(WorkerState, swimlane_last_cycle) == 842664, "WorkerState trace clock offset mismatch"); +static_assert(offsetof(WorkerState, payloads) == 842688, "WorkerState task-payload offset mismatch"); + +struct alignas(64) WorkerResult { + // 时间边界:Submit 口径不含启动屏障和最终 drain,finish_cycle 则覆盖完整 worker 生命周期。 + uint64_t submit_begin; + uint64_t submit_end; + uint64_t finish_cycle; + uint64_t checksum; + + // 协议计数用于验证固定 Claim 拓扑及等待/依赖动态次数;joint_polls 是为未来 + // BlockWon 模拟保留的兼容字段,当前实现没有递增点,不能据其检测 joint 分支。 + uint64_t submits; + uint64_t claim_attempts; + uint64_t claim_wins; + uint64_t heap_guards; + uint64_t fanin_loads; + uint64_t completion_duplicates; + uint64_t cas_retries; + uint64_t joint_polls; + + // 默认 256 batch 时 winner、kernel 分别闭合到 1280 task 和 1024 kernel; + // 非默认配置按 5*batches、4*batches 计算,placement 仍闭合到全部 kernel。 + uint64_t wins[static_cast(TaskKind::Count)]; + uint64_t kernel_counts[4]; + uint64_t kernel_cycles[4]; + uint64_t kernel_min_cycles[4]; + uint64_t kernel_max_cycles[4]; + uint64_t placement[static_cast(DrainPlace::Count)]; + uint64_t phase_cycles[static_cast(ProfilePhase::Count)]; + uint64_t phase_calls[static_cast(ProfilePhase::Count)]; + uint64_t wait_events[2]; + uint64_t wait_iterations[2]; + + // 前端工作量计数不是性能填充:构参、materialize 和 map insert 用于核对全部 + // 96 个 worker 的回放;map lookup、slot copy 与 fanin 则是 winner-only 全局计数。 + uint64_t context_reads; + uint64_t views_created; + uint64_t dynamic_create_infos; + uint64_t arg_resets; + uint64_t tensor_args_added; + uint64_t scalar_args_added; + uint64_t materialized_outputs; + uint64_t map_inserts; + uint64_t map_lookups; + uint64_t slot_tensor_copies; + uint64_t slot_scalar_copies; + uint64_t fanin_edges; + + // 最终快照用于跨 worker 比较逻辑 heap 与 TensorMap 回收状态是否完全一致。 + uint64_t final_heap_next; + uint64_t map_high_water; + uint64_t map_alive_floor; + uint64_t map_cleaned_upto; + uint64_t map_live_entries; + + uint64_t worker_id; + uint64_t role; + uint64_t max_occupied; + uint64_t final_occupied; +}; +// WorkerResult 是 standalone 尾部的诊断 sidecar,不属于真实 DistCore ABI;按 +// cache line 隔离后,各 worker 发布统计不会相互覆盖或污染被测共享状态。 +static_assert(sizeof(WorkerResult) % 64 == 0, "WorkerResult must not share cache lines"); + +// 从 cube_cursor 到 workers 结束保留关键字段 offset、DistCore ABI 和生产总字节跨度, +// 并非字段级完整镜像。RunConfig、输入 context_lens 与校验结果追加在该跨度之后, +// 因此测试控制信息不会改变被测字段 offset。 +struct alignas(64) SchedulerState { + // 三组四分片 cursor 分别服务 AIC kernel、AIV kernel 和 Alloc;同 task 的 + // eligible workers 竞争同一 shard,只有旧值小于 task_id 的调用成为 winner。 + AtomicLine cube_cursor[kCursorShards]; + AtomicLine vector_cursor[kCursorShards]; + AtomicLine alloc_cursor[kCursorShards]; + AtomicLine frontier; + int32_t heap_window; + uint8_t tasks_padding[60]; + TaskCell tasks[kTaskCellCapacity]; + // heap_base/size 描述共享物理环,worker.heap_next 则是各 worker 一致推进的逻辑游标。 + uint64_t heap_base; + uint64_t heap_size; + uint64_t orchestration_args; + uint64_t runtime_state; + uint64_t runtime; + uint8_t fatal_padding[24]; + AtomicFlagLine fatal; + int32_t num_workers; + int32_t num_blocks; + // Case1 never enters BlockWon, but the inactive layout and BlockWon arena + // remains byte-for-byte reserved so every subsequent PA atomic line keeps + // its production offset. + // 此处不能因 Case1 动态次数为零而删减,否则 replay_done、started_count + // 和 DistCore 数组整体前移,便不再是对真实 PA 地址布局的等价测试。 + uint8_t layout_and_block_won[5848440]; + AtomicLine replay_done; + AtomicLine started_count; + // started_count 形成 launch 屏障;replay_done 只用于最终 drain 判定所有 worker + // 已不再产生新 slot。两者都位于 Submit 性能口径之外,但属于完整协议。 + WorkerState workers[kRuntimeMaxWorkers]; + // Standalone-only controls live after the complete DistGlobal image. They + // therefore do not shift any cursor/task/fatal/worker address under test. + RunConfig config; + // Context lengths are the only PA input elements read by orchestration; + // keeping them in GM preserves the per-batch descriptor-based load. + // 除这 256 个长度值外,其余 tensor 仅需稳定的合成地址来复现 + // descriptor、依赖和 heap 行为,不会解引用成真实计算数据。 + volatile int32_t context_lens[kMaxBatches]; + WorkerResult results[kWorkers]; +}; +static_assert(offsetof(SchedulerState, cube_cursor) == 0, "cube cursor offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, vector_cursor) == 256, "vector cursor offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, alloc_cursor) == 512, "alloc cursor offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, frontier) == 768, "frontier offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, heap_window) == 832, "H offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, tasks) == kRealTasksOffset, "task table offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, heap_base) == 4195200, "heap base offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, heap_size) == 4195208, "heap size offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, fatal) == kRealFatalOffset, "fatal offset must match PA DistGlobal"); +static_assert(offsetof(SchedulerState, replay_done) == kRealReplayDoneOffset, "replay offset must match PA DistGlobal"); +static_assert( + offsetof(SchedulerState, started_count) == kRealStartedCountOffset, + "started-count offset must match PA DistGlobal" +); +static_assert(offsetof(SchedulerState, tasks) % 64 == 0, "task table must be cache-line aligned"); +static_assert(offsetof(SchedulerState, workers) % 64 == 0, "worker table must be cache-line aligned"); +static_assert(offsetof(SchedulerState, results) % 64 == 0, "result table must be cache-line aligned"); +static_assert(offsetof(SchedulerState, workers) == kRealDistCoreOffset, "DistCore table offset must match PA"); +static_assert(offsetof(SchedulerState, config) == kRealDistGlobalBytes, "DistGlobal byte size must match PA"); + +} // namespace pa_scheduler + +#endif // PA_SCHEDULER_COMMON_PA_MODEL_H diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h new file mode 100644 index 0000000000..0737fd7ac4 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -0,0 +1,788 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_COMMON_PA_SCHEDULER_CORE_H +#define PA_SCHEDULER_COMMON_PA_SCHEDULER_CORE_H + +#ifndef PA_DEVICE +#define PA_DEVICE inline +#endif + +#ifndef PA_GM +#define PA_GM +#endif + +#include "pa_frontend.h" +#include "pa_trace.h" + +namespace pa_scheduler { + +struct LocalStats { + WorkerResult result; + uint32_t max_occupied; + TraceContext trace; +}; + +PA_DEVICE uint32_t KindIndex(TaskKind kind) { return static_cast(kind); } + +PA_DEVICE TaskKind GetTaskKind(uint32_t task_id) { return static_cast(task_id % kTasksPerBatch); } + +PA_DEVICE int32_t FunctionId(TaskKind kind) { + return kind == TaskKind::Alloc ? -1 : static_cast(KindIndex(kind) - 1); +} + +PA_DEVICE uint32_t NopCountForKind(PA_GM const NopCounts &nops, TaskKind kind) { + switch (kind) { + case TaskKind::Qk: + return nops.qk; + case TaskKind::Sf: + return nops.sf; + case TaskKind::Pv: + return nops.pv; + case TaskKind::Up: + return nops.up; + default: + return 0; + } +} + +PA_DEVICE uint32_t CountBits(uint32_t value) { + uint32_t count = 0; + while (value != 0) { + count += value & 1U; + value >>= 1; + } + return count; +} + +template +PA_DEVICE int64_t LoadLine(PA_GM AtomicLine &line) { + // Ops::Load 在 A5 后端是 atomicAdd(0);返回值是该 RMW 线性化时观察到的共享值。 + return Ops::Load(&line.value); +} + +template +PA_DEVICE int32_t LoadLine(PA_GM AtomicFlagLine &line) { + return Ops::Load(&line.value); +} + +template +PA_DEVICE void SetFatal(PA_GM SchedulerState *state) { + // fatal 只从 0 单调置 1,重复 Exchange 不会把其他 worker 已观察到的失败状态清除。 + Ops::Exchange(&state->fatal.value, static_cast(1)); +} + +template +PA_DEVICE bool IsFatal(PA_GM SchedulerState *state) { + return LoadLine(state->fatal) != 0; +} + +template +PA_DEVICE bool WatchdogExpired(PA_GM SchedulerState *state, uint64_t begin, uint32_t &polls) { + // 每 1024 次自旋才读取系统计数器,降低正常启动屏障上的计时开销;超时后向所有 worker 广播 fatal。 + ++polls; + if ((polls & 1023U) != 0 || Ops::Now() - begin <= kWatchdogTicks) { + return false; + } + SetFatal(state); + return true; +} + +template +PA_DEVICE void AdvanceFrontier(PA_GM SchedulerState *state, LocalStats &stats) { + // frontier 只表示“从 task 0 开始已经连续完成”的最高 task id,不能越过尚未发布 flag 的空洞。 + // 多个完成者可以同时扫描同一段连续区间,FetchMax 保证共享 frontier 只前进、不回退。 + int64_t frontier = LoadLine(state->frontier); + while (true) { + const int64_t next = frontier + 1; + if (next < 0 || next >= static_cast(kTaskCellCapacity)) { + break; + } + if (Ops::Load(&state->tasks[next].flag) == 0) { + break; + } + uint64_t retries = 0; + // FetchMax 返回更新前的值;若其他核已经走得更远,就从其 old 值继续扫描,避免重复从 next 起步。 + const int64_t old = Ops::FetchMax(&state->frontier.value, next, retries); + stats.result.cas_retries += retries; + frontier = old > next ? old : next; + } +} + +template +PA_DEVICE void CompleteTask( + PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_id, LocalStats &stats +) { + // 完成发布顺序与 PA 一致:先公布该 worker 的 heap 游标,再发布 ready flag,最后推进连续 frontier。 + // fanin 和 heap 回收方以 flag/frontier 为可见性条件,因此不能交换 vend 与 flag 的先后关系。 + Ops::Exchange(&state->tasks[task_id].vend, worker.heap_next); + Ops::StoreBarrier(); + Ops::Exchange(&state->tasks[task_id].flag, 1); + AdvanceFrontier(state, stats); +} + +template +PA_DEVICE bool SlotReady(PA_GM SchedulerState *state, PA_GM LocalSlot &slot, LocalStats &stats) { + // 每个 fanin flag 都是跨核共享的完成条件;遇到第一个未就绪依赖即返回,后续 drain 会再次轮询。 + for (uint32_t index = 0; index < slot.fanin_count; ++index) { + ++stats.result.fanin_loads; + if (Ops::Load(&state->tasks[slot.fanin[index]].flag) == 0) { + return false; + } + } + return true; +} + +PA_DEVICE void RecordKernelCycles(LocalStats &stats, TaskKind kind, uint64_t cycles) { + const uint32_t index = KindIndex(kind) - 1; + ++stats.result.kernel_counts[index]; + stats.result.kernel_cycles[index] += cycles; + if (stats.result.kernel_min_cycles[index] == 0 || cycles < stats.result.kernel_min_cycles[index]) { + stats.result.kernel_min_cycles[index] = cycles; + } + if (cycles > stats.result.kernel_max_cycles[index]) { + stats.result.kernel_max_cycles[index] = cycles; + } +} + +template +PA_DEVICE uint32_t DrainReady( + PA_GM SchedulerState *state, PA_GM WorkerState &worker, DrainPlace place, LocalStats &stats +) { + // 同一套 drain 被三个位置复用:每次 Submit 开头的 EfDrain、ring 背压等待和所有 Submit 后的最终 drain。 + // slot 属于当前 worker;只有其全部跨核 fanin 已 ready 时才执行 NOP 模拟的 kernel、发布完成并释放 slot。 + if (worker.occupied_count == 0) { + return 0; + } + uint32_t freed = 0; + // 一次调用遍历完本核全部私有 slot;未就绪项保留 occupied/built,已完成项立即清槽并继续扫描。 + for (uint32_t index = 0; index < kPrivateSlots; ++index) { + PA_GM LocalSlot &slot = worker.slots[index]; + if (!slot.occupied || !slot.built || !SlotReady(state, slot, stats)) { + continue; + } + const TaskKind kind = static_cast(slot.kind + 1); + const uint64_t kernel_begin = Ops::Now(); + Ops::Nop(NopCountForKind(state->config.nops, kind)); + const uint64_t kernel_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(slot.task_id), static_cast(slot.kind), + TracePhase::Kernel, ProfilePhase::ReplayTail, kernel_begin, kernel_end + ); + RecordKernelCycles(stats, kind, kernel_end - kernel_begin); + CompleteTask(state, worker, slot.task_id, stats); + const uint64_t commit_cycle = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(slot.task_id), static_cast(slot.kind), + TracePhase::Commit, ProfilePhase::ReplayTail, commit_cycle, commit_cycle + ); + slot.built = false; + slot.occupied = false; + --worker.occupied_count; + ++stats.result.placement[static_cast(place)]; + ++freed; + } + return freed; +} + +PA_DEVICE int32_t FindFreeSlot(PA_GM WorkerState &worker) { + for (uint32_t index = 0; index < kPrivateSlots; ++index) { + if (!worker.slots[index].occupied) { + return static_cast(index); + } + } + return -1; +} + +template +PA_DEVICE void WaitForSlot( + PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_id, LocalStats &stats +) { + // 四个物理 slot 中预留两个 won slot 语义位,仅有 kUsableSlots 个可供本图使用;满时靠 drain 取得进展。 + const uint64_t wait_begin = Ops::Now(); + bool waited = false; + // 退出条件只有 occupied_count 重新低于可用容量;依赖尚未 ready 时 SpinHint 后继续重试。 + while (worker.occupied_count >= kUsableSlots) { + waited = true; + ++stats.result.wait_iterations[0]; + if (DrainReady( + state, worker, DrainPlace::RingBackpressure, stats + ) == 0) { + Ops::SpinHint(); + } + } + if (waited) { + ++stats.result.wait_events[0]; + const uint64_t wait_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, TracePhase::RingBp, + ProfilePhase::WaitForSlot, wait_begin, wait_end, 0, 0 + ); + } else if constexpr (Profile) { + AccumulatePhase(stats.result, ProfilePhase::WaitForSlot, wait_begin, Ops::Now()); + } +} + +template +PA_DEVICE bool HeapGuard( + PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_id, uint64_t output_bytes, + LocalStats &stats +) { + // 只有产生新输出的 winner 需要保护环形 heap;retire=frontier-H 对应已经允许复用的最老任务 vend。 + // 等待期间也主动 drain 本核已就绪 slot,避免只自旋而阻塞能够推动 frontier 的 kernel。 + if (output_bytes == 0 || state->heap_base == 0) { + return true; + } + const uint64_t ring = state->heap_size; + ++stats.result.heap_guards; + const uint64_t wait_begin = Ops::Now(); + bool waited = false; + // 正常出口是 heap_next-vend 落入一个 ring;检测到不可能释放的覆盖或其他核 fatal 时返回失败。 + while (!IsFatal(state)) { + const int64_t frontier = LoadLine(state->frontier); + const int64_t retire = frontier - static_cast(state->heap_window); + const uint64_t vend = retire < 0 ? 0 : Ops::Load(&state->tasks[retire].vend); + if (worker.heap_next - vend <= ring) { + if (waited) { + ++stats.result.wait_events[1]; + const uint64_t wait_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, TracePhase::RingBp, + ProfilePhase::HeapGuard, wait_begin, wait_end, 0, 1 + ); + } else if constexpr (Profile) { + AccumulatePhase(stats.result, ProfilePhase::HeapGuard, wait_begin, Ops::Now()); + } + return true; + } + if (frontier >= static_cast(task_id) - 1) { + SetFatal(state); + return false; + } + waited = true; + ++stats.result.wait_iterations[1]; + if (DrainReady( + state, worker, DrainPlace::RingBackpressure, stats + ) == 0) { + Ops::SpinHint(); + } + } + if (waited) { + ++stats.result.wait_events[1]; + const uint64_t wait_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, TracePhase::RingBp, + ProfilePhase::HeapGuard, wait_begin, wait_end, 0, 1 + ); + } + return false; +} + +struct ClaimOutcome { + bool attempted; + bool won; + uint64_t retries; + int32_t function_id; +}; + +template +PA_DEVICE ClaimOutcome Claim( + PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_id, TaskKind kind +) { + // Claim 在四个 shard 的单调 cursor 上执行 atomicMax:同一 task 只有观察到旧值更小的竞争者获胜。 + // Alloc 由全部 96 个 worker 竞争;QK/PV 仅 32 个 AIC,SF/UP 仅 64 个 AIV 进入真正的 atomicMax。 + ClaimOutcome outcome{false, false, 0, -1}; + if (task_id >= kTaskCellCapacity) { + return outcome; + } + PA_GM AtomicLine *cursor = nullptr; + if (kind == TaskKind::Alloc) { + cursor = &state->alloc_cursor[task_id % kCursorShards]; + } else { + // Mirror MixedKernels::to_active_mask(), core_mask(), popcount(), + // lane_active(), and self->role routing inside the real Claim span. + const int32_t aic_kernel = kind == TaskKind::Qk || kind == TaskKind::Pv ? FunctionId(kind) : -1; + const int32_t aiv0_kernel = kind == TaskKind::Sf || kind == TaskKind::Up ? FunctionId(kind) : -1; + const int32_t aiv1_kernel = -1; + uint8_t active_mask = 0; + if (aic_kernel >= 0) active_mask |= 1U; + if (aiv0_kernel >= 0) active_mask |= 2U; + if (aiv1_kernel >= 0) active_mask |= 4U; + const uint8_t core_mask = active_mask & 0x07U; + const int32_t active_count = __builtin_popcount(static_cast(core_mask)); + // 这里保留生产 Claim 的 lane-mask 路由边界。当前固定 PA 图按构造只生成单 lane + // 的 QK/PV 或 SF/UP;需要两个及以上 lane 协作的 joint task 本应进入 BlockWon + // 协议,本独立用例没有实现该动态路径,因此显式拒绝而不把它误当成单 lane task。 + if (active_count >= 2) { + return outcome; + } + if ((core_mask & 1U) != 0) { + if (worker.role != CoreRole::Aic) return outcome; + cursor = &state->cube_cursor[task_id % kCursorShards]; + outcome.function_id = aic_kernel; + } else if ((core_mask & 6U) != 0) { + if (worker.role != CoreRole::Aiv) return outcome; + cursor = &state->vector_cursor[task_id % kCursorShards]; + outcome.function_id = (core_mask & 2U) != 0 ? aiv0_kernel : aiv1_kernel; + } else { + return outcome; + } + } + outcome.attempted = true; + // atomicMax 返回写入前的 cursor:old=task_id 则必须 Replay。 + const int64_t old = Ops::FetchMax(&cursor->value, static_cast(task_id), outcome.retries); + outcome.won = old < static_cast(task_id); + if (!outcome.won) outcome.function_id = -1; + return outcome; +} + +PA_DEVICE void RecordClaimOutcome(LocalStats &stats, TaskKind kind, const ClaimOutcome &outcome) { + if (outcome.attempted) ++stats.result.claim_attempts; + stats.result.cas_retries += outcome.retries; + if (outcome.won) { + ++stats.result.claim_wins; + ++stats.result.wins[KindIndex(kind)]; + } +} + +template +PA_DEVICE bool BuildWinner( + PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_id, TaskKind kind, + const TaskArgs &args, const SubmitContext &context, + const int32_t fanin[kMaxFanin], uint32_t fanin_count, LocalStats &stats +) { + // kernel winner 不在 Submit 内立即执行计算,而是把完整 payload 和 fanin 存入自己的私有 ring slot。 + // 后续 EfDrain/背压 drain/最终 drain 在依赖满足后执行它,这正是 PA 的 Submit 与执行解耦点。 + WaitForSlot(state, worker, task_id, stats); + if (!HeapGuard( + state, worker, task_id, context.output_bytes, stats + )) { + return false; + } + const int32_t slot_index = FindFreeSlot(worker); + if (slot_index < 0) { + SetFatal(state); + return false; + } + PA_GM LocalSlot &slot = worker.slots[slot_index]; + // Match dist_submit_alloc_slot(): reserve and account the private slot + // before build_ring_slot_from_submit publishes its completed payload. + // 状态按“occupied 占位 -> built 清零 -> 计入占用 -> BuildSlotPayload”推进;后者会先 + // 置 built,再填充 payload。slot 为 worker 私有、没有跨核发布竞争,所以此处的 built + // 只是复刻生产状态机与构建成本,不承担对其他核发布完整 payload 的同步语义。 + slot.occupied = true; + slot.built = 0; + ++worker.occupied_count; + if (worker.occupied_count > stats.max_occupied) { + stats.max_occupied = worker.occupied_count; + } + const int32_t sub_block_id = worker.lane == 2 ? 1 : 0; + BuildSlotPayload( + slot, task_id, static_cast(FunctionId(kind)), 0, args, context, fanin, fanin_count, + sub_block_id + ); + stats.result.slot_tensor_copies += static_cast(context.tensor_count); + stats.result.slot_scalar_copies += static_cast(context.scalar_count); + stats.result.fanin_edges += fanin_count; + return true; +} + +template +PA_DEVICE bool SubmitTask( + PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_count, TaskKind kind, + const TaskArgs &args, SubmitContext &context, LocalStats &stats +) { + // 每个 worker 都完整回放相同 task stream。主流程为:EfDrain -> materialize -> TensorMap retire + // -> Claim -> winner 收集 fanin -> 全员 register -> winner Build / loser Replay。Alloc 在 Claim 前 register, + // 且 winner 不入 kernel slot,而是在 heap guard 后直接发布完成。 + BeginSubmit(worker, args, context); + const uint32_t task_id = static_cast(context.task_id); + const uint64_t submit_begin = Ops::Now(); + if (task_id == 0) { + stats.result.submit_begin = submit_begin; + } + + ResetTraceLap(worker); + // EfDrain 在当前 Submit 的参数物化前执行上一批已就绪 slot,是绝大多数 kernel 的正常落点。 + DrainReady(state, worker, DrainPlace::EfDrain, stats); + WriteTraceLap( + stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::EfDrain, + ProfilePhase::EfDrain + ); + + // dist_submit_materialize_and_prepare_map resets the lap origin before its + // two independently traced spans. Build/Replay later consumes this origin. + // lap 起点在 materialize 前重置;Materialize/PrepareMap 各自取独立绝对区间,而后续 + // Build/Replay 会从这个起点形成覆盖式 span。因此泳道上的这些阶段不能直接相加。 + ResetTraceLap(worker); + const uint64_t materialize_begin = Ops::Now(); + if (!MaterializeTask(worker, task_id, args, context, state->heap_base, state->heap_size)) { + SetFatal(state); + return false; + } + const uint64_t materialize_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Materialize, + ProfilePhase::Materialize, materialize_begin, materialize_end, 0, + kind == TaskKind::Alloc ? 1U : 0U + ); + stats.result.materialized_outputs += context.result.count; + + const uint64_t prepare_begin = Ops::Now(); + AdvanceTensorMap(worker.map, task_id, static_cast(state->heap_window)); + const uint64_t prepare_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, TracePhase::PrepareMap, + ProfilePhase::PrepareMap, prepare_begin, prepare_end, 0, kind == TaskKind::Alloc ? 1U : 0U + ); + + bool winner = false; + int32_t function_id = -1; + + if (kind == TaskKind::Alloc) { + // Alloc 没有 kernel lane,96 个 worker 都维护本地物化/heap 状态,但只有 Claim winner 发布全局完成。 + const uint64_t register_begin = Ops::Now(); + RegisterOutputs(context, args, false); + const uint64_t register_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Register, + ProfilePhase::Register, register_begin, register_end, 0, 0 + ); + + const uint64_t claim_begin = Ops::Now(); + const ClaimOutcome claim = Claim(state, worker, task_id, kind); + winner = claim.won; + context.won = winner; + context.kernel_id = claim.function_id; + const uint64_t claim_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Claim, + ProfilePhase::Claim, claim_begin, claim_end, winner ? 1U : 0U, 1 + ); + RecordClaimOutcome(stats, kind, claim); + if (winner) { + if (!HeapGuard( + state, worker, task_id, context.output_bytes, stats + )) { + return false; + } + CompleteTask(state, worker, task_id, stats); + WriteTraceLap( + stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::Alloc, + ProfilePhase::ReplayTail + ); + } else { + // Replay 表示该 worker 输掉 Claim;前面的物化、TensorMap 和 register 仍已执行,以保持本地状态同步。 + WriteTraceLap( + stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::Replay, + ProfilePhase::ReplayTail + ); + } + } else { + const uint64_t claim_begin = Ops::Now(); + const ClaimOutcome claim = Claim(state, worker, task_id, kind); + winner = claim.won; + function_id = claim.function_id; + context.won = winner; + context.kernel_id = function_id; + const uint64_t claim_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Claim, + ProfilePhase::Claim, claim_begin, claim_end, winner ? 1U : 0U, 0 + ); + RecordClaimOutcome(stats, kind, claim); + + if (winner) { + const uint64_t fanin_begin = Ops::Now(); + context.fanin_count = static_cast(CollectFanin(worker.map, args, context.fanin)); + const uint64_t fanin_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Fanin, + ProfilePhase::Fanin, fanin_begin, fanin_end, 0, static_cast(context.fanin_count) + ); + stats.result.map_lookups += static_cast(args.tensor_count) - context.result.count; + } + + const uint64_t register_begin = Ops::Now(); + RegisterOutputs(context, args, true); + const uint64_t register_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Register, + ProfilePhase::Register, register_begin, register_end, 0, 1 + ); + stats.result.map_inserts += CountBits(context.register_mask); + + if (winner) { + WriteTraceLap( + stats.trace, worker, stats.result, static_cast(task_id), function_id, TracePhase::Build, + ProfilePhase::ReplayTail + ); + if (!BuildWinner( + state, worker, task_id, kind, args, context, context.fanin, + static_cast(context.fanin_count), stats + )) { + return false; + } + } else { + // 非 winner 不占用私有 ring slot,也不执行 kernel;Replay marker 覆盖本次前端回放的尾段。 + WriteTraceLap( + stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::Replay, + ProfilePhase::ReplayTail + ); + // drain_block_won() is a local boolean early-return for this + // single-lane PA graph, so it intentionally performs no GM access. + } + } + + ++stats.result.submits; + const uint64_t submit_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Submit, + ProfilePhase::Submit, submit_begin, submit_end, winner ? 1U : 0U, kind == TaskKind::Alloc ? 1U : 0U + ); + if (task_id + 1 == task_count) { + stats.result.submit_end = submit_end; + } + return true; +} + +PA_DEVICE uint32_t CountLiveMapEntries(PA_GM const TensorMap &map) { + uint32_t free_entries = 0; + for (int32_t current = map.free_head; current >= 0; current = map.entries[current].next_in_bucket) { + ++free_entries; + } + return static_cast(map.high_water) - free_entries; +} + +template +PA_DEVICE void PublishResult(PA_GM WorkerResult &destination, const WorkerResult &source) { + // 每个 worker 只写自己独占、覆盖多条 cache line 的 WorkerResult 分区;逐字段 + // bypass 保证结果对 host 可见,而独立 sidecar 允许 D2H 只搬结果、不搬约 9 MiB WorkerState。 +#define PA_PUBLISH_FIELD(field) Ops::Publish(&destination.field, source.field) + PA_PUBLISH_FIELD(submit_begin); + PA_PUBLISH_FIELD(submit_end); + PA_PUBLISH_FIELD(finish_cycle); + PA_PUBLISH_FIELD(checksum); + PA_PUBLISH_FIELD(submits); + PA_PUBLISH_FIELD(claim_attempts); + PA_PUBLISH_FIELD(claim_wins); + PA_PUBLISH_FIELD(heap_guards); + PA_PUBLISH_FIELD(fanin_loads); + PA_PUBLISH_FIELD(completion_duplicates); + PA_PUBLISH_FIELD(cas_retries); + PA_PUBLISH_FIELD(joint_polls); + for (uint32_t index = 0; index < static_cast(TaskKind::Count); ++index) { + Ops::Publish(&destination.wins[index], source.wins[index]); + } + for (uint32_t index = 0; index < 4; ++index) { + Ops::Publish(&destination.kernel_counts[index], source.kernel_counts[index]); + Ops::Publish(&destination.kernel_cycles[index], source.kernel_cycles[index]); + Ops::Publish(&destination.kernel_min_cycles[index], source.kernel_min_cycles[index]); + Ops::Publish(&destination.kernel_max_cycles[index], source.kernel_max_cycles[index]); + } + for (uint32_t index = 0; index < static_cast(DrainPlace::Count); ++index) { + Ops::Publish(&destination.placement[index], source.placement[index]); + } + for (uint32_t index = 0; index < static_cast(ProfilePhase::Count); ++index) { + Ops::Publish(&destination.phase_cycles[index], source.phase_cycles[index]); + Ops::Publish(&destination.phase_calls[index], source.phase_calls[index]); + } + for (uint32_t index = 0; index < 2; ++index) { + Ops::Publish(&destination.wait_events[index], source.wait_events[index]); + Ops::Publish(&destination.wait_iterations[index], source.wait_iterations[index]); + } + PA_PUBLISH_FIELD(context_reads); + PA_PUBLISH_FIELD(views_created); + PA_PUBLISH_FIELD(dynamic_create_infos); + PA_PUBLISH_FIELD(arg_resets); + PA_PUBLISH_FIELD(tensor_args_added); + PA_PUBLISH_FIELD(scalar_args_added); + PA_PUBLISH_FIELD(materialized_outputs); + PA_PUBLISH_FIELD(map_inserts); + PA_PUBLISH_FIELD(map_lookups); + PA_PUBLISH_FIELD(slot_tensor_copies); + PA_PUBLISH_FIELD(slot_scalar_copies); + PA_PUBLISH_FIELD(fanin_edges); + PA_PUBLISH_FIELD(final_heap_next); + PA_PUBLISH_FIELD(map_high_water); + PA_PUBLISH_FIELD(map_alive_floor); + PA_PUBLISH_FIELD(map_cleaned_upto); + PA_PUBLISH_FIELD(map_live_entries); + PA_PUBLISH_FIELD(worker_id); + PA_PUBLISH_FIELD(role); + PA_PUBLISH_FIELD(max_occupied); + PA_PUBLISH_FIELD(final_occupied); +#undef PA_PUBLISH_FIELD + Ops::StoreBarrier(); +} + +template +PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, CoreRole role) { + // 一个入口实例只拥有 state->workers[worker_id] 的私有 map/ring/payload;cursor、task cell 和屏障为跨核共享区。 + if (worker_id >= kWorkers) { + return; + } + PA_GM WorkerState &worker = state->workers[worker_id]; + worker.role = role; + worker.core_idx = static_cast(worker_id); + // standalone 使用连续 worker 编号:AIC 为 0..31;AIV 为 32..95。 + // 每个物理 block b 对应 AIC(b, lane0)、AIV(32+2b, lane1)、AIV(33+2b, lane2)。 + if (role == CoreRole::Aic) { + worker.block_id = static_cast(worker_id); + worker.lane = 0; + } else { + const uint32_t vector_id = worker_id - kAicWorkers; + worker.block_id = static_cast(vector_id / 2); + worker.lane = static_cast(1 + vector_id % 2); + } + worker.sub_block_id = worker.lane == 2 ? 1 : 0; + worker.local_index = 0; + worker.heap_next = 0; + ResetTensorMap(worker.map); + worker.occupied_count = 0; + worker.owned_total = 0; + worker.swimlane_last_cycle = 0; + for (uint32_t index = 0; index < kPrivateSlots; ++index) { + worker.slots[index].occupied = false; + worker.slots[index].built = false; + } + + LocalStats stats{}; + stats.result.worker_id = worker_id; + stats.result.role = static_cast(role); + stats.result.checksum = 0xcbf29ce484222325ULL ^ worker_id; + stats.trace = AttachTrace(state, worker, worker_id); + + // 96 个参与者全部完成本地状态初始化后再进入 task 0,主要用于压低启动偏斜对 + // winner 分布和 Submit 时序的干扰;atomicMax 的唯一 winner 正确性本身不依赖该屏障。 + Ops::FetchAdd(&state->started_count.value, 1); + const uint64_t start_wait = Ops::Now(); + uint32_t start_polls = 0; + // 全员到齐或任一核发布 fatal 即退出启动等待;watchdog 防止缺失参与者造成永久挂死。 + while (LoadLine(state->started_count) < static_cast(state->config.workers) && + !IsFatal(state)) { + Ops::SpinHint(); + if (WatchdogExpired(state, start_wait, start_polls)) { + break; + } + } + + const uint32_t batches = state->config.batches; + const uint32_t task_count = batches * kTasksPerBatch; + PaOrchestrationState orchestration; + TaskArgs args; + SubmitContext context; + if (!IsFatal(state)) { + // Case1 每个 batch 固定回放 Alloc/QK/SF/PV/UP 五个 task;所有 worker 顺序相同,执行 lane 由 Claim 筛选。 + ResetTraceLap(worker); + InitPaOrchestration(orchestration, batches, &state->context_lens[0]); + for (uint32_t batch = 0; batch < batches; ++batch) { + BuildAllocArgs(orchestration, args, batch); + ++stats.result.context_reads; + stats.result.views_created += 2; + stats.result.tensor_args_added += 3; + if (!SubmitTask( + state, worker, task_count, TaskKind::Alloc, args, context, stats + )) { + break; + } + AcceptTaskOutputs(orchestration, TaskKind::Alloc, context.result); + + BuildQkArgs(orchestration, args, batch); + ++stats.result.dynamic_create_infos; + ++stats.result.arg_resets; + stats.result.tensor_args_added += 4; + stats.result.scalar_args_added += 2; + if (!SubmitTask( + state, worker, task_count, TaskKind::Qk, args, context, stats + )) { + break; + } + AcceptTaskOutputs(orchestration, TaskKind::Qk, context.result); + + BuildSfArgs(orchestration, args); + ++stats.result.dynamic_create_infos; + ++stats.result.arg_resets; + stats.result.tensor_args_added += 4; + stats.result.scalar_args_added += 3; + if (!SubmitTask( + state, worker, task_count, TaskKind::Sf, args, context, stats + )) { + break; + } + AcceptTaskOutputs(orchestration, TaskKind::Sf, context.result); + + BuildPvArgs(orchestration, args, batch); + ++stats.result.arg_resets; + stats.result.tensor_args_added += 4; + stats.result.scalar_args_added += 2; + if (!SubmitTask( + state, worker, task_count, TaskKind::Pv, args, context, stats + )) { + break; + } + AcceptTaskOutputs(orchestration, TaskKind::Pv, context.result); + + BuildUpdateArgs(orchestration, args); + ++stats.result.arg_resets; + stats.result.tensor_args_added += 7; + stats.result.scalar_args_added += 2; + if (!SubmitTask( + state, worker, task_count, TaskKind::Up, args, context, stats + )) { + break; + } + } + } + + // replay_done 表示所有 worker 已退出回放循环(成功路径即完整提交);之后仍需 drain 到本核 slot 为空。 + Ops::FetchAdd(&state->replay_done.value, 1); + while (true) { + const uint32_t freed = + DrainReady(state, worker, DrainPlace::FinalDrain, stats); + const bool all_replayed = + LoadLine(state->replay_done) >= static_cast(state->config.workers); + // 必须同时满足“无人再生产新 slot”和“本核旧 slot 全部完成”,否则继续帮助系统推进 completion。 + if (all_replayed && worker.occupied_count == 0) { + break; + } + if (freed == 0) { + Ops::SpinHint(); + } + } + + // PA writes swimlane records through the ordinary GM cache and explicitly + // cleans each worker's record range before the kernel finishes. + FlushTraceCore(stats.trace); + stats.result.finish_cycle = Ops::Now(); + stats.result.max_occupied = stats.max_occupied; + stats.result.final_occupied = worker.occupied_count; + stats.result.final_heap_next = worker.heap_next; + stats.result.map_high_water = static_cast(worker.map.high_water); + stats.result.map_alive_floor = static_cast(worker.map.alive_floor); + stats.result.map_cleaned_upto = static_cast(worker.map.cleaned_upto); + stats.result.map_live_entries = CountLiveMapEntries(worker.map); + PublishResult(state->results[worker_id], stats.result); +} + +template +PA_DEVICE void RunScheduler(PA_GM SchedulerState *state, uint32_t worker_id, CoreRole role) { + // Profile 作为编译期模板参数,只在显式开启时保留阶段累计代码,关闭时不在热路径增加运行时分支。 + if (state->config.profile_phases != 0) { + RunSchedulerImpl(state, worker_id, role); + } else { + RunSchedulerImpl(state, worker_id, role); + } +} + +} // namespace pa_scheduler + +#endif // PA_SCHEDULER_COMMON_PA_SCHEDULER_CORE_H diff --git a/tests/atomic_probe/pa_scheduler/common/pa_trace.h b/tests/atomic_probe/pa_scheduler/common/pa_trace.h new file mode 100644 index 0000000000..5bd77387b1 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/common/pa_trace.h @@ -0,0 +1,158 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_COMMON_PA_TRACE_H +#define PA_SCHEDULER_COMMON_PA_TRACE_H + +#include "pa_model.h" + +namespace pa_scheduler { + +// 记录区与真实 PA 一样直接拼在定长 Header 后面;worker_id 只选择自己的 +// records 分区,避免记录动作本身制造跨核共享写热点。 +PA_DEVICE PA_GM TraceRecord *GetTraceRecords(PA_GM TraceHeader *header) { + // 输入必须指向完整且按 64 byte 对齐的 trace buffer;返回值只是首条记录, + // 调用方还需按 worker_id * capacity 选择自己的分区。 + return reinterpret_cast(reinterpret_cast(header) + sizeof(TraceHeader)); +} + +struct TraceContext { + PA_GM TraceCoreState *core; + PA_GM TraceRecord *records; + uint32_t capacity; + int32_t lane; + int32_t block_id; + int32_t core_idx; +}; + +// Attach 只缓存本 worker 的 header 状态、分区首址和物理 lane 信息。配置先做 +// cache invalidate,确保 A5 worker 看到 host 在 launch 前写入的 trace 开关与地址。 +template +PA_DEVICE TraceContext AttachTrace( + PA_GM SchedulerState *state, PA_GM const WorkerState &worker, uint32_t worker_id +) { + TraceContext trace{nullptr, nullptr, 0, worker.lane, worker.block_id, static_cast(worker_id)}; + Ops::InvalidateRegion(&state->config, sizeof(state->config)); + const uint64_t base = state->config.trace_base; + const uint32_t capacity = state->config.trace_records_per_core; + if (state->config.trace_enabled == 0 || base == 0 || capacity == 0 || worker_id >= kWorkers) { + return trace; + } + PA_GM TraceHeader *header = reinterpret_cast(base); + if (worker_id >= header->num_cores) { + return trace; + } + trace.core = &header->cores[worker_id]; + trace.records = &GetTraceRecords(header)[static_cast(worker_id) * capacity]; + // 成功返回的不变量是 core/records/capacity 同时有效;任一前置条件失败则三者 + // 保持空值,后续 WriteTrace/FlushTraceCore 可无分支地安全退化为 no-op。 + trace.capacity = capacity; + trace.core->count = 0; + trace.core->dropped = 0; + return trace; +} + +template +PA_DEVICE void AccumulatePhase( + WorkerResult &result, ProfilePhase phase, uint64_t start_cycle, uint64_t end_cycle +) { + // phase profile 与完整泳道是两套正交机制:即使关闭 records,Profile=true + // 仍会累计用户当前关注的 Claim/EfDrain/WaitForSlot/HeapGuard 四段。 + if constexpr (Profile) { + if (phase != ProfilePhase::Claim && phase != ProfilePhase::EfDrain && + phase != ProfilePhase::WaitForSlot && phase != ProfilePhase::HeapGuard) { + return; + } + const uint32_t index = static_cast(phase); + // 调用方保证 end_cycle>=start_cycle;各后端把 Now() 归一到每 tick 1 ns 的 + // 数值标度,聚合持续时间可直接相加并在 host 侧按 1000 换算为微秒。 + const uint64_t duration = end_cycle - start_cycle; + result.phase_cycles[index] += duration; + ++result.phase_calls[index]; + } +} + +template +PA_DEVICE void WriteTrace( + TraceContext &trace, WorkerResult &result, int32_t task_id, int32_t function_id, TracePhase trace_phase, + ProfilePhase profile_phase, uint64_t start_cycle, uint64_t end_cycle, uint32_t flags = 0, + uint32_t auxiliary = 0 +) { + // 每段先更新轻量 phase 统计,再按需写 64-byte 原始记录。一个分区只有对应 + // worker 写入,因此 count/dropped 保持普通单写者更新,不额外引入 atomic。 + AccumulatePhase(result, profile_phase, start_cycle, end_cycle); + if (trace.core == nullptr || trace.records == nullptr || trace.capacity == 0) { + return; + } + PA_GM TraceCoreState &core = *trace.core; + const uint32_t slot = core.count; + if (slot >= trace.capacity) { + core.dropped = core.dropped + 1; + return; + } + PA_GM TraceRecord &record = trace.records[slot]; + record.start_cycle = start_cycle; + record.end_cycle = end_cycle; + record.task_id = task_id; + record.function_id = function_id; + record.phase = static_cast(trace_phase); + record.lane = trace.lane; + record.block_id = trace.block_id; + record.core_idx = trace.core_idx; + record.flags = flags; + record.auxiliary = auxiliary; + // count 最后更新,使其始终指向下一空槽;单写者条件下无需 reserve/commit 两阶段。 + // 最终 FlushTraceCore 会把记录体先于该计数一并导出。 + core.count = slot + 1; +} + +template +PA_DEVICE void ResetTraceLap(PA_GM WorkerState &worker) { + // lap 是后续 Build/Replay/Alloc 等覆盖式阶段的共同起点,不代表新增嵌套 span。 + // 因此分析时不能把 lap 时长再与其中的 Materialize/Claim/Register 直接相加。 + worker.swimlane_last_cycle = Ops::Now(); +} + +template +PA_DEVICE void FlushTraceCore(TraceContext &trace) { + if (trace.core == nullptr || trace.records == nullptr || trace.capacity == 0) { + return; + } + PA_GM TraceCoreState &core = *trace.core; + const uint32_t count = core.count < trace.capacity ? core.count : trace.capacity; + // A5 侧记录经普通 GM cache 写入,kernel 结束前必须把有效 records 与最后的 + // count/dropped cache line 显式 clean,host 的 D2H 才能得到完整且自洽的快照。 + if (count != 0) { + Ops::FlushRegion(trace.records, static_cast(count) * sizeof(TraceRecord)); + } + Ops::FlushRegion(&core, sizeof(core)); +} + +template +PA_DEVICE uint64_t WriteTraceLap( + TraceContext &trace, PA_GM WorkerState &worker, WorkerResult &result, int32_t task_id, + int32_t function_id, TracePhase trace_phase, ProfilePhase profile_phase, + uint32_t flags = 0, uint32_t auxiliary = 0 +) { + // lap 记录区间 [上一次 Reset/WriteTraceLap, 当前时刻],写完立即推进起点。 + // 显式 WriteTrace span 不会修改该起点,这正是生产泳道中阶段可重叠的原因。 + const uint64_t end_cycle = Ops::Now(); + WriteTrace( + trace, result, task_id, function_id, trace_phase, profile_phase, worker.swimlane_last_cycle, end_cycle, + flags, auxiliary + ); + worker.swimlane_last_cycle = end_cycle; + return end_cycle; +} + +} // namespace pa_scheduler + +#endif // PA_SCHEDULER_COMMON_PA_TRACE_H diff --git a/tests/atomic_probe/pa_scheduler/cpu/build.sh b/tests/atomic_probe/pa_scheduler/cpu/build.sh new file mode 100755 index 0000000000..d3fd5f0f75 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/cpu/build.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" +BUILD_DIR="$ROOT_DIR/build/cpu" +CXX_BIN="${CXX:-g++}" + +# CPU 后端只依赖 C++17、pthread 和本目录 common/,不需要 CANN。 +# CXX 可显式指向用户目录下的 g++-15,未设置时沿用当前 PATH 中的 g++。 + +# CPU build 与设备 build 使用平行目录,便于 run.sh 根据 backend 做严格选择, +# 也避免把 host 回归二进制误当成 A5 产物。 +mkdir -p "$BUILD_DIR" + +echo "[BUILD] CPU scheduler executable" +# -pthread 同时提供编译期线程宏和链接期 pthread 支持;严格告警用于防止 +# CPU 等价层因类型或原子接口变化而静默偏离设备端公共协议。 +"$CXX_BIN" -O3 -std=c++17 -pthread -Wall -Wextra -Werror \ + -I"$ROOT_DIR/common" \ + "$SCRIPT_DIR/main.cpp" \ + -o "$BUILD_DIR/pa_scheduler_cpu" + +# set -e 保证编译或链接失败时不会打印 complete,也不会在组合构建中继续 +# 后续步骤;只有成功退出的构建才被本脚本声明为可运行产物。 +echo "[BUILD] complete: $BUILD_DIR/pa_scheduler_cpu" diff --git a/tests/atomic_probe/pa_scheduler/cpu/main.cpp b/tests/atomic_probe/pa_scheduler/cpu/main.cpp new file mode 100644 index 0000000000..f01adba2b8 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/cpu/main.cpp @@ -0,0 +1,243 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#include "../common/host_support.h" + +#define PA_DEVICE inline +#define PA_GM +// CPU 后端直接实例化与设备端相同的公共调度器;这里只消去 AICore 地址空间 +// 修饰符,不另写一套简化状态机,因此它可以承担协议和边界回归。 +#include "../common/pa_scheduler_core.h" + +#include +#include +#include +#include +#include +#include +#include + +namespace { + +template +inline void EmitNops() { + // 编译期 Count 配合强制展开,避免编译器把空循环折叠掉。 +#if defined(__clang__) +#pragma clang loop unroll(full) +#elif defined(__GNUC__) +#pragma GCC unroll 256 +#endif + for (uint32_t index = 0; index < Count; ++index) { + asm volatile("nop"); + } +} + +inline void RuntimeNop(uint32_t count) { + // 与 AscendC 后端采用相同的 256 + 二进制尾块分解,保证入参含义一致。 + // x86 nop 的吞吐和 CPU 线程调度都不同于 A5,所以这里只复现指令数量, + // 不能把 CPU 测得时间解释为 A5 kernel 时间。 + while (count >= 256) { + EmitNops<256>(); + count -= 256; + } + if ((count & 128U) != 0) EmitNops<128>(); + if ((count & 64U) != 0) EmitNops<64>(); + if ((count & 32U) != 0) EmitNops<32>(); + if ((count & 16U) != 0) EmitNops<16>(); + if ((count & 8U) != 0) EmitNops<8>(); + if ((count & 4U) != 0) EmitNops<4>(); + if ((count & 2U) != 0) EmitNops<2>(); + if ((count & 1U) != 0) EmitNops<1>(); +} + +struct CpuOps { + // 用 fetch_add(0) 模拟 A5 atomicAdd(addr, 0) 原子读,而不是退化为普通 + // CPU load。Acquire/AcqRel 只建立本 CPU 协议回归需要的发布/观察关系, + // 不模拟 A5 cache 或设备内存模型细节。 + static inline int32_t Load(volatile int32_t *address) { + return __atomic_fetch_add(address, static_cast(0), __ATOMIC_ACQUIRE); + } + + static inline int64_t Load(volatile int64_t *address) { + // 保留原子 add-zero 路径,让 96 个 pthread 仍在同一批热点地址上竞争。 + return __atomic_fetch_add(address, static_cast(0), __ATOMIC_ACQUIRE); + } + + static inline uint64_t Load(volatile uint64_t *address) { + return __atomic_fetch_add(address, static_cast(0), __ATOMIC_ACQUIRE); + } + + static inline int32_t Exchange(volatile int32_t *address, int32_t value) { + return __atomic_exchange_n(address, value, __ATOMIC_ACQ_REL); + } + + static inline int64_t Exchange(volatile int64_t *address, int64_t value) { + return __atomic_exchange_n(address, value, __ATOMIC_ACQ_REL); + } + + static inline uint64_t Exchange(volatile uint64_t *address, uint64_t value) { + return __atomic_exchange_n(address, value, __ATOMIC_ACQ_REL); + } + + static inline int64_t FetchAdd(volatile int64_t *address, int64_t value) { + return __atomic_fetch_add(address, value, __ATOMIC_ACQ_REL); + } + + static inline int64_t FetchMax(volatile int64_t *address, int64_t value, uint64_t &retries) { + // CPU 没有直接对应本测试签名的 fetch-max,用 CAS loop 实现同一返回值 + // 语义;retries 仅用于诊断软件竞争,不能与 A5 硬件 AtomicMax 对比。 + int64_t current = __atomic_load_n(address, __ATOMIC_ACQUIRE); + retries = 0; + while (value > current) { + if (__atomic_compare_exchange_n(address, ¤t, value, true, __ATOMIC_ACQ_REL, __ATOMIC_ACQUIRE)) { + break; + } + ++retries; + } + return current; + } + + // 前后的 Exchange 已使用 AcqRel,Publish 使用 Release,因此这里不重复 + // 插入 fence,保持与设备适配层相同的调用边界。 + static inline void StoreBarrier() {} + + // 将 steady_clock 统一换算成纳秒,数值上适配公共模型的 1 GHz tick 标度; + // 这不表示 CPU 物理时钟为 1 GHz,也不保证实际分辨率达到 1 ns。 + static inline uint64_t Now() { + return static_cast( + std::chrono::duration_cast(std::chrono::steady_clock::now().time_since_epoch()) + .count() + ); + } + + static inline void Nop(uint32_t count) { RuntimeNop(count); } + + static inline void SpinHint() {} + + static inline void InvalidateRegion(const void *, uint64_t) { + // CPU 没有 A5 DCache line 失效指令;这里仅提供保守的本线程顺序边界, + // 接口占位但不模拟设备 cache line 行为。共享状态本身仍使用 atomic。 + std::atomic_thread_fence(std::memory_order_seq_cst); + } + + static inline void FlushRegion(void *, uint64_t) { std::atomic_thread_fence(std::memory_order_seq_cst); } + + static inline void Publish(uint64_t *address, uint64_t value) { + // Release store 对应设备端 bypass-DCache 结果发布的可见性边界。 + __atomic_store_n(address, value, __ATOMIC_RELEASE); + } +}; + +} // namespace + +// CPU runner 不承担 A5 性能对比,只负责用同一份 SchedulerState 和公共调度器 +// 做协议回归。生命周期为参数解析、host 内存准备、逐轮 96 线程执行、严格校验、 +// 可选泳道后处理,最后统一释放 trace buffer。 +int main(int argc, char **argv) { + pa_scheduler::host::Options options; + const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions(argc, argv, false, &options); + if (parse_status != pa_scheduler::host::ParseStatus::Ok) { + return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; + } + pa_scheduler::host::PrintBanner("CPU", options); + std::printf("[NOTE] CPU NOP counts preserve instruction count, not A5 microseconds.\n"); + + // SchedulerState 很大,放到 heap 而不是主线程栈;trace 继续保持独立的 + // 64 字节对齐区域,以复用设备端完全相同的二进制布局。 + std::unique_ptr state(new pa_scheduler::SchedulerState); + void *trace_memory = nullptr; + if (options.trace_enabled) { + trace_memory = std::aligned_alloc(64, pa_scheduler::kTraceBytes); + if (trace_memory == nullptr) { + std::fprintf(stderr, "Cannot allocate %zu-byte swimlane trace buffer.\n", pa_scheduler::kTraceBytes); + return EXIT_FAILURE; + } + } + auto *trace_header = static_cast(trace_memory); + std::vector spans; + bool all_passed = true; + bool postprocess_ok = true; + // 每轮复用大块 host 分配,只重置公共状态和 trace header。与设备后端一样, + // runs>1 表示同进程热运行,不等价于多个独立首轮。 + for (uint32_t run = 1; run <= options.runs; ++run) { + pa_scheduler::host::InitializeState(state.get(), options); + pa_scheduler::host::ConfigureTrace(state.get(), options, trace_memory); + if (options.trace_enabled) { + pa_scheduler::host::InitializeTraceHeader(trace_header); + } + const auto wall_begin = std::chrono::steady_clock::now(); + // 固定创建 96 个参与者:worker 0..31 扮演 AIC,32..95 扮演 AIV。 + // 每个线程仍会进入公共 started_count 屏障,再共同回放完整 task 流。 + std::vector workers; + workers.reserve(pa_scheduler::kWorkers); + for (uint32_t worker_id = 0; worker_id < pa_scheduler::kWorkers; ++worker_id) { + const pa_scheduler::CoreRole role = + worker_id < pa_scheduler::kAicWorkers ? pa_scheduler::CoreRole::Aic : pa_scheduler::CoreRole::Aiv; + workers.emplace_back([state_pointer = state.get(), worker_id, role]() { + pa_scheduler::RunScheduler(state_pointer, worker_id, role); + }); + } + // join 是本后端的 kernel 完成屏障;所有 worker 退出后才能读取最终状态, + // 对应设备 runner 的 aclrtSynchronizeStream。 + for (std::thread &worker : workers) + worker.join(); + const auto wall_end = std::chrono::steady_clock::now(); + const double host_us = std::chrono::duration(wall_end - wall_begin).count(); + // host 内存沿用与 A5 相同的 TraceHeader + 每 worker 固定跨度 ABI; + // 分析器和 raw JSON writer 因而可以与设备后端共用同一回调接口。 + const auto read_trace_records = + [trace_memory](uint32_t worker, uint32_t count, pa_scheduler::TraceRecord *records) { + const uint64_t offset = sizeof(pa_scheduler::TraceHeader) + + static_cast(worker) * pa_scheduler::kTraceRecordsPerCore * + sizeof(pa_scheduler::TraceRecord); + std::memcpy( + records, static_cast(trace_memory) + offset, + static_cast(count) * sizeof(pa_scheduler::TraceRecord) + ); + return true; + }; + // 先完成严格语义校验,再允许写出;失败运行不会生成可误认成有效 + // 基线的泳道 JSON。 + const pa_scheduler::host::Metrics metrics = pa_scheduler::host::Validate( + *state, run, host_us, options.trace_enabled ? trace_header : nullptr + ); + all_passed &= metrics.passed; + spans.push_back(metrics.submit_span_us); + // 分析只打印统计,导出则写 raw JSON;两者失败都标记 postprocess, + // 与调度语义失败分开报告,便于区分协议问题和产物问题。 + if (options.analyze_swimlane && + !pa_scheduler::host::AnalyzeSwimlaneRecords(*trace_header, *state, read_trace_records)) { + postprocess_ok = false; + break; + } + if (!options.swimlane_json.empty()) { + if (!metrics.passed) { + std::fprintf(stderr, "Skipping swimlane export because semantic validation failed.\n"); + postprocess_ok = false; + break; + } + if (!pa_scheduler::host::ExportSwimlaneRecords( + *trace_header, options.swimlane_json, read_trace_records + )) { + postprocess_ok = false; + break; + } + } + } + + std::printf( + "[SUMMARY] runs=%u median_submit_span_us=%.3f semantic_status=%s postprocess_status=%s\n", options.runs, + pa_scheduler::host::Median(spans), all_passed ? "PASS" : "FAIL", postprocess_ok ? "PASS" : "FAIL" + ); + // std::free(nullptr) 合法,因此关闭泳道时也走同一条收尾路径。 + std::free(trace_memory); + return all_passed && postprocess_ok ? EXIT_SUCCESS : EXIT_FAILURE; +} diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh new file mode 100755 index 0000000000..69d6ed3ca7 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -0,0 +1,195 @@ +#!/usr/bin/env bash +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +set -euo pipefail + +# 用脚本自身位置锚定所有构建产物、转换器和输出目录;从任意 cwd 调用都 +# 不会回退到 simpler 仓库中的同名工具。 +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" + +usage() { + cat <<'EOF' +Usage: + ./run.sh build ccec|ascendc|cpu|all + ./run.sh run ccec|ascendc|cpu|all [benchmark options] + ./run.sh smoke ccec|ascendc|cpu|all [--device N] + ./run.sh swimlane ccec|ascendc|cpu|all [benchmark options] + +Benchmark options: + --device N + --batches 1..256 + --runs N + --nop-count N + --nop-counts QK,SF,PV,UP + --profile-phases + --analyze-swimlane + --swimlane-json FILE + --no-swimlane + +The swimlane action performs exactly one run and writes both the raw capture +and merged Perfetto JSON below this directory's outputs/ folder. It rejects +--runs, --swimlane-json, and --no-swimlane because those are managed by the action. + +The all target always uses the requested implementation order: +CCEC, AscendC, then CPU. +EOF +} + +require_file() { + # run/smoke/swimlane 都只消费本目录 build/ 下已经生成的后端产物, + # 缺失时明确提示对应 build action,而不是临时猜测编译命令。 + if [[ ! -f "$1" ]]; then + echo "Missing build artifact: $1" >&2 + echo "Run: $0 build $2" >&2 + exit 1 + fi +} + +build_backend() { + # 后端自己的 build.sh 是唯一构建入口;all 的先后顺序由下方 BACKENDS + # 固定为 CCEC、AscendC、CPU,便于按用户要求分阶段复现。 + case "$1" in + ccec|ascendc|cpu) + "$SCRIPT_DIR/$1/build.sh" + ;; + *) + echo "Unknown backend: $1" >&2 + exit 1 + ;; + esac +} + +run_backend() { + local backend="$1" + shift + # 所有 benchmark 参数原样透传给同一套 host option parser。CCEC 额外 + # 传入本目录内的 mixed ELF,其余两个后端直接启动本地可执行文件。 + case "$backend" in + ccec) + local host="$SCRIPT_DIR/build/ccec/pa_scheduler_host" + local kernel="$SCRIPT_DIR/build/ccec/pa_scheduler_kernel.o" + require_file "$host" ccec + require_file "$kernel" ccec + "$host" --kernel "$kernel" "$@" + ;; + ascendc) + local executable="$SCRIPT_DIR/build/ascendc/pa_scheduler_ascendc" + require_file "$executable" ascendc + "$executable" "$@" + ;; + cpu) + local executable="$SCRIPT_DIR/build/cpu/pa_scheduler_cpu" + require_file "$executable" cpu + "$executable" "$@" + ;; + *) + echo "Unknown backend: $backend" >&2 + exit 1 + ;; + esac +} + +reject_managed_swimlane_options() { + # swimlane action 必须独占轮数、raw 路径和 trace 开关,才能保证每个 + # backend 恰好对应一对 raw/merged 文件且不会发生多轮覆盖。 + for argument in "$@"; do + case "$argument" in + --runs|--swimlane-json|--no-swimlane) + echo "The swimlane action manages $argument; do not pass it explicitly." >&2 + exit 1 + ;; + esac + done +} + +# 顶层先解释 action/backend;run 的其余参数交给共享 parser,build、smoke 和 +# swimlane 再分别处理自己的约束或默认注入项。参数不足会在创建目录前失败。 +if [[ $# -lt 2 ]]; then + usage >&2 + exit 1 +fi + +ACTION="$1" +BACKEND="$2" +shift 2 + +if [[ "$BACKEND" == "all" ]]; then + # 该顺序也是组合构建、运行和泳道采集的稳定对外约定。 + BACKENDS=(ccec ascendc cpu) +else + BACKENDS=("$BACKEND") +fi + +case "$ACTION" in + build) + # build 只选择后端,不接收 benchmark 参数;这样编译配置不会被运行时 + # 选项暗中改变,三种实现的构建命令也保持可独立复现。 + if [[ $# -ne 0 ]]; then + echo "The build action does not accept benchmark options." >&2 + exit 1 + fi + for backend in "${BACKENDS[@]}"; do + build_backend "$backend" + done + ;; + run) + # run 不替用户补默认覆盖项,完整参数校验交给各后端共享的 Options parser。 + for backend in "${BACKENDS[@]}"; do + run_backend "$backend" "$@" + done + ;; + smoke) + # smoke 仍启动全部 96 个 worker,并默认注入 1 batch、1 run、0 NOP; + # 后置用户参数仍由共享 parser 处理。它用于快速检查原子协议、拓扑和 + # 最终状态,不作为性能数据。 + for backend in "${BACKENDS[@]}"; do + run_backend "$backend" --batches 1 --runs 1 --nop-count 0 "$@" + done + ;; + swimlane) + # swimlane 是“采集 + 转换”的事务边界:runner 失败则不转换,converter + # 失败则 action 非零退出;已成功写完的 raw 会保留用于排查转换问题。 + reject_managed_swimlane_options "$@" + # 仅需要 Python 标准库;允许用户用 PYTHON 指向自己的虚拟环境, + # 但转换脚本始终取自当前 pa_scheduler 目录。 + PYTHON_BIN="${PYTHON:-python3}" + if ! command -v "$PYTHON_BIN" >/dev/null 2>&1; then + echo "Python executable not found: $PYTHON_BIN" >&2 + exit 1 + fi + if [[ ! -f "$SCRIPT_DIR/swimlane_converter.py" ]]; then + echo "Missing local converter: $SCRIPT_DIR/swimlane_converter.py" >&2 + exit 1 + fi + # UTC 秒级时间加当前 shell PID 避免并行采集目录冲突;所有产物保持 + # 在本目录 outputs/ 下,复制 pa_scheduler 后仍可原样工作。 + OUTPUT_ROOT="$SCRIPT_DIR/outputs/pa_scheduler_swimlane_$(date -u +%Y%m%d_%H%M%S)_$$" + mkdir -p "$OUTPUT_ROOT" + # all 模式下每个 backend 使用独立子目录,避免同名 raw/merged 互相覆盖; + # 某一后端失败后 set -e 停止,之前已完成后端的产物仍可单独检查。 + for backend in "${BACKENDS[@]}"; do + BACKEND_OUTPUT="$OUTPUT_ROOT/$backend" + RAW_JSON="$BACKEND_OUTPUT/l2_swimlane_records.json" + MERGED_JSON="$BACKEND_OUTPUT/merged_swimlane.json" + mkdir -p "$BACKEND_OUTPUT" + # runner 先执行单轮严格语义校验并流式写 raw;成功后才调用本地 + # converter 生成 Perfetto 文件。set -e 保证任一步失败即停止。 + run_backend "$backend" --runs 1 --swimlane-json "$RAW_JSON" "$@" + "$PYTHON_BIN" "$SCRIPT_DIR/swimlane_converter.py" "$RAW_JSON" -o "$MERGED_JSON" + done + echo "[SWIMLANE] output_root=$OUTPUT_ROOT" + ;; + *) + # 未知 action 不尝试推断用户意图,也不会触发任何构建或设备操作。 + echo "Unknown action: $ACTION" >&2 + usage >&2 + exit 1 + ;; +esac diff --git a/tests/atomic_probe/pa_scheduler/swimlane_converter.py b/tests/atomic_probe/pa_scheduler/swimlane_converter.py new file mode 100755 index 0000000000..611d3c84b6 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/swimlane_converter.py @@ -0,0 +1,289 @@ +#!/usr/bin/env python3 +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- +"""把 standalone PA 调度器的原始 FDWIC 记录转换为 Perfetto 泳道。 + +脚本只使用 Python 标准库和调用者给出的本地 JSON,不 import ``simpler_setup`` +或仓库外模块。输出遵循 Chrome Trace Event 格式,可直接载入 Perfetto。 +""" + +from __future__ import annotations + +import argparse +import json +import os +import sys +from pathlib import Path +from typing import Any, TextIO + + +# 与真实 PA swimlane converter 使用同一套阶段命名。这里保留 ringbp、 +# efdrain 等既有拼写,避免同一阶段在两类泳道中被 Perfetto 分成不同名称。 +PHASE_NAMES = { + "Kernel": "kernel", + "Alloc": "alloc", + "Build": "build", + "DrainWon": "drain_won", + "Replay": "replay", + "RingBp": "ringbp", + "EfDrain": "efdrain", + "Commit": "commit", + "Submit": "submit", + "Materialize": "materialize", + "PrepareMap": "prepare_map", + "Claim": "claim", + "Fanin": "fanin", + "Register": "register", +} +KERNEL_NAMES = {0: "QK", 1: "SF", 2: "PV", 3: "UP"} +# 一个物理 mixed block 的三条 runtime lane:AIC、AIV0、AIV1。 +LANE_NAMES = {0: "AIC", 1: "AIV0", 2: "AIV1"} + + +# 把可转为整数的 raw 标量归一为 int,并在错误中保留精确字段路径。 +def _integer(value: Any, label: str) -> int: + # 这是兼容 JSON 数值/数值字符串的宽松归一,不负责强制原始 JSON 类型必须为 int。 + try: + return int(value) + except (TypeError, ValueError) as error: + raise ValueError(f"{label} is not an integer: {value!r}") from error + + +# 读取 raw JSON,校验十列结构、字段范围与可转整数值,并返回规范化视图。 +def _load_and_validate(input_path: Path) -> tuple[int, list[tuple[Any, ...]], dict[tuple[int, int], int], int]: + # raw 文件沿用真实 l2_swimlane_records.json 的十列 fdwic_events ABI: + # core、block、lane、task、func、phase、start、end、flags、aux。 + with input_path.open("r", encoding="utf-8") as input_file: + data = json.load(input_file) + if not isinstance(data, dict): + raise ValueError("capture root must be a JSON object") + + # 先验证顶层 schema 和时钟元数据;时钟频率是 cycle 转时间的唯一依据, + # 不允许由 converter 根据平台名称猜测。 + level = _integer(data.get("l2_swimlane_level"), "l2_swimlane_level") + if level not in (1, 2, 3, 4): + raise ValueError(f"unsupported l2_swimlane_level: {level}") + metadata = data.get("metadata") + if not isinstance(metadata, dict): + raise ValueError("metadata must be a JSON object") + frequency_hz = _integer(metadata.get("clock_freq_hz"), "metadata.clock_freq_hz") + if frequency_hz <= 0: + raise ValueError("metadata.clock_freq_hz must be positive") + num_cores = _integer(metadata.get("num_cores"), "metadata.num_cores") + if num_cores <= 0: + raise ValueError("metadata.num_cores must be positive") + core_types = metadata.get("core_types") + if not isinstance(core_types, list) or len(core_types) != num_cores: + raise ValueError("metadata.core_types length must equal metadata.num_cores") + + rows = data.get("fdwic_events") + if not isinstance(rows, list) or not rows: + raise ValueError("fdwic_events must be a non-empty array") + + # Perfetto metadata 需要从 (block, lane) 找回稳定的 core 编号;同一 lane + # 若在 raw 中映射到两个 core,说明采集已损坏,不能继续生成误导性泳道。 + core_by_block_lane: dict[tuple[int, int], int] = {} + base_cycle: int | None = None + # 逐行在写输出前检查列数、范围和可转整数的字段。任一行不满足这些约束 + # 都会整体拒绝输入,不生成缺少关键阶段的“部分可看”泳道。 + for index, row in enumerate(rows): + if not isinstance(row, (list, tuple)) or len(row) != 10: + raise ValueError(f"fdwic_events[{index}] must contain exactly 10 fields") + core_id = _integer(row[0], f"fdwic_events[{index}].core_id") + block_id = _integer(row[1], f"fdwic_events[{index}].block_id") + lane = _integer(row[2], f"fdwic_events[{index}].lane") + task_id = _integer(row[3], f"fdwic_events[{index}].task_id") + function_id = _integer(row[4], f"fdwic_events[{index}].function_id") + phase = str(row[5]) + start_cycle = _integer(row[6], f"fdwic_events[{index}].start_cycle") + end_cycle = _integer(row[7], f"fdwic_events[{index}].end_cycle") + flags = _integer(row[8], f"fdwic_events[{index}].flags") + auxiliary = _integer(row[9], f"fdwic_events[{index}].auxiliary") + if not 0 <= core_id < num_cores: + raise ValueError(f"fdwic_events[{index}] has out-of-range core_id {core_id}") + if block_id < 0: + raise ValueError(f"fdwic_events[{index}] has negative block_id {block_id}") + if lane not in LANE_NAMES: + raise ValueError(f"fdwic_events[{index}] has invalid lane {lane}") + if phase not in PHASE_NAMES: + raise ValueError(f"fdwic_events[{index}] has unknown phase {phase!r}") + if start_cycle <= 0 or end_cycle < start_cycle: + raise ValueError( + f"fdwic_events[{index}] has invalid cycles start={start_cycle} end={end_cycle}" + ) + key = (block_id, lane) + previous_core = core_by_block_lane.setdefault(key, core_id) + if previous_core != core_id: + raise ValueError( + f"block {block_id} lane {lane} maps to both core {previous_core} and core {core_id}" + ) + # 所有 X 事件共同减去最早 start,既避免大整数转 float 的精度损失, + # 也让设备 SYS_CNT 的绝对值不影响 Perfetto 横轴。 + base_cycle = start_cycle if base_cycle is None else min(base_cycle, start_cycle) + rows[index] = ( + core_id, + block_id, + lane, + task_id, + function_id, + phase, + start_cycle, + end_cycle, + flags, + auxiliary, + ) + + assert base_cycle is not None + return frequency_hz, rows, core_by_block_lane, base_cycle + + +# 写一个 Chrome Trace Event,并统一处理数组元素间的逗号。 +def _emit_event(output: TextIO, event: dict[str, Any], first: bool) -> bool: + # 逐事件写出,避免再在内存中构造一份体积可达数百 MiB 的 merged 列表。 + if not first: + output.write(",\n") + json.dump(event, output, ensure_ascii=False, separators=(",", ":")) + return False + + +# 完成一次 raw 到 merged 的转换,成功时返回事件数、block 数和基准 cycle。 +def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: + frequency_hz, rows, core_by_block_lane, base_cycle = _load_and_validate(input_path) + # 禁止原地转换;否则创建临时文件或最终 replace 时可能破坏唯一一份 raw。 + if input_path.resolve() == output_path.resolve(): + raise ValueError("input and output paths must differ") + + # 始终先写同目录临时文件,完整 flush/fsync 后再原子替换目标;转换失败 + # 时删除临时文件,不把半截 JSON 留作可加载的正式产物。 + output_path.parent.mkdir(parents=True, exist_ok=True) + temporary_path = output_path.with_name(output_path.name + ".tmp") + # Chrome Trace Event 的 ts/dur 约定使用微秒;displayTimeUnit="ns" 只控制 + # Perfetto 的显示精度。1 GHz A5 counter 因此每 tick 对应 0.001 us。 + factor = 1_000_000.0 / float(frequency_hz) + blocks = sorted({block_id for block_id, _ in core_by_block_lane}) + first = True + emitted = 0 + # 临时文件的整个生命周期都在 try 内;包括 Ctrl-C 在内的异常都会先清理 + # .tmp 再向上传播。格式/IO 错误由 main 简短报告,Ctrl-C 保留默认中断行为。 + try: + with temporary_path.open("w", encoding="utf-8") as output: + output.write('{"displayTimeUnit":"ns","traceEvents":[\n') + # 每个物理 block 建一个 process;每条硬件 lane 再拆成 runtime + # 与 kernel 两个 thread,避免等待/提交阶段覆盖 kernel 执行条。 + for block_id in blocks: + first = _emit_event( + output, + {"ph": "M", "name": "process_name", "pid": block_id, "args": {"name": f"block{block_id}"}}, + first, + ) + first = _emit_event( + output, + { + "ph": "M", + "name": "process_sort_index", + "pid": block_id, + "args": {"sort_index": block_id}, + }, + first, + ) + for lane, lane_name in LANE_NAMES.items(): + core_id = core_by_block_lane.get((block_id, lane)) + if core_id is None: + continue + for thread_id, thread_name in ( + (lane, f"{lane_name} (core{core_id})"), + (lane + 3, f"{lane_name}·kernel (core{core_id})"), + ): + first = _emit_event( + output, + { + "ph": "M", + "name": "thread_name", + "pid": block_id, + "tid": thread_id, + "args": {"name": thread_name}, + }, + first, + ) + + for row in rows: + core_id, block_id, lane, task_id, function_id, phase_raw, start, end, flags, auxiliary = row + phase = PHASE_NAMES[phase_raw] + # Kernel 和 Commit 放到 lane+3 的 kernel 子泳道;其他阶段留在 + # lane 0..2 的 runtime 泳道。这与真实 PA merged 文件的布局一致。 + if phase == "kernel" and function_id >= 0: + name = f"{KERNEL_NAMES.get(function_id, f'f{function_id}')}#{task_id}" + thread_id = lane + 3 + elif phase == "commit": + name = f"commit#{task_id}" + thread_id = lane + 3 + else: + name = f"{phase}#{task_id}" + thread_id = lane + event = { + "ph": "X", + "name": name, + "pid": block_id, + "tid": thread_id, + "ts": round((start - base_cycle) * factor, 3), + "dur": round((end - start) * factor, 3), + "args": { + "phase": phase, + "task_id": task_id, + "func_id": function_id, + "core": core_id, + # mc 是兼容真实 merged schema 的字段名,只原样承载 flags + # bit0;它与 aux 的实际含义均需结合 phase 解读,例如 Claim + # 可表示 winner,而 Fanin/HeapGuard 的 aux 各有自己的计数语义。 + "mc": flags & 1, + "aux": auxiliary, + }, + } + first = _emit_event(output, event, first) + emitted += 1 + output.write("\n]}\n") + output.flush() + os.fsync(output.fileno()) + os.replace(temporary_path, output_path) + except BaseException: + temporary_path.unlink(missing_ok=True) + raise + return emitted, len(blocks), base_cycle + + +# 只解析显式 input/output,不扫描仓库 outputs,也不选择“最新”文件。 +def _parse_args() -> argparse.Namespace: + # 强制 -o 使覆盖目标可审查,避免脱仓后因 cwd 不同写到意外目录。 + parser = argparse.ArgumentParser( + description="Convert standalone PA fdwic_events JSON to a Chrome/Perfetto swimlane trace." + ) + parser.add_argument("input", type=Path, help="l2_swimlane_records.json produced by the standalone runner") + parser.add_argument("-o", "--output", type=Path, required=True, help="merged_swimlane.json output path") + return parser.parse_args() + + +# 命令行错误边界:预期的输入、格式和文件系统错误统一返回 1。 +def main() -> int: + args = _parse_args() + try: + events, blocks, base_cycle = convert(args.input, args.output) + except (OSError, ValueError, json.JSONDecodeError) as error: + # 不吞掉错误原因,但也不向普通使用者输出长 traceback;convert 已保证 + # 失败路径不会留下临时 merged 文件。 + print(f"swimlane conversion failed: {error}", file=sys.stderr) + return 1 + print( + f"[SWIMLANE] merged_json={args.output} events={events} blocks={blocks} base_cycle={base_cycle}" + ) + print(f"Open https://ui.perfetto.dev/ and load {args.output}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From 76df85ced5a43099d268cd6bcd5a988d03d9cca7 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 13:22:06 +0800 Subject: [PATCH 005/214] =?UTF-8?q?test(atomic=5Fprobe):=20=E9=AA=8C?= =?UTF-8?q?=E8=AF=81=20atomic=20=E4=B8=8E=20I-cache=20=E7=AD=89=E5=BE=85?= =?UTF-8?q?=E5=91=A8=E6=9C=9F=E5=BD=92=E7=B1=BB?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- tests/atomic_probe/ccec/atomic_scalar_pmu.cpp | 179 +++++ .../ccec/atomic_scalar_pmu_host.cpp | 615 ++++++++++++++++++ .../ccec/atomic_scalar_pmu_shared.h | 74 +++ tests/atomic_probe/ccec/icache_scalar_pmu.cpp | 226 +++++++ .../ccec/icache_scalar_pmu_host.cpp | 402 ++++++++++++ .../ccec/icache_scalar_pmu_shared.h | 112 ++++ tests/atomic_probe/ccec/pmu_probe_aicpu.cpp | 179 +++++ tests/atomic_probe/ccec/pmu_probe_control.h | 59 ++ .../ccec/pmu_probe_host_support.h | 309 +++++++++ .../ccec/run_atomic_scalar_pmu.sh | 147 +++++ .../ccec/run_icache_scalar_pmu.sh | 219 +++++++ tests/atomic_probe/test_case.md | 92 +++ 12 files changed, 2613 insertions(+) create mode 100644 tests/atomic_probe/ccec/atomic_scalar_pmu.cpp create mode 100644 tests/atomic_probe/ccec/atomic_scalar_pmu_host.cpp create mode 100644 tests/atomic_probe/ccec/atomic_scalar_pmu_shared.h create mode 100644 tests/atomic_probe/ccec/icache_scalar_pmu.cpp create mode 100644 tests/atomic_probe/ccec/icache_scalar_pmu_host.cpp create mode 100644 tests/atomic_probe/ccec/icache_scalar_pmu_shared.h create mode 100644 tests/atomic_probe/ccec/pmu_probe_aicpu.cpp create mode 100644 tests/atomic_probe/ccec/pmu_probe_control.h create mode 100644 tests/atomic_probe/ccec/pmu_probe_host_support.h create mode 100755 tests/atomic_probe/ccec/run_atomic_scalar_pmu.sh create mode 100755 tests/atomic_probe/ccec/run_icache_scalar_pmu.sh diff --git a/tests/atomic_probe/ccec/atomic_scalar_pmu.cpp b/tests/atomic_probe/ccec/atomic_scalar_pmu.cpp new file mode 100644 index 0000000000..7214e86cb7 --- /dev/null +++ b/tests/atomic_probe/ccec/atomic_scalar_pmu.cpp @@ -0,0 +1,179 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +// 测试目标:用单个 AIV、无竞争、无轮询的固定次数窗口,精确区分 +// atomicAdd 等待返回的时间是否计入 PMU scalar busy。Host/AICPU 在 launch 前已把 +// PMU slot0/1/2 配置为 scalar busy(0x1)、I-cache request(0x34)、I-cache miss(0x35)。 +// +// 每次 kernel 的完整时序: +// 1. 先关闭 PMU,再对 host 写入的 control 单独 cache line 做 DCCI + DSB,避免复用 +// ProbeState 时沿用上一次 kernel 的旧 control。Atomic target 不做普通 load/store +// 或 DCCI,始终保持为一条独占 cache line 的 raw atomic 目标。 +// 2. 按 physical core id 从 host 传入的寄存器基址表取本 AIV PMU base,然后用 +// read-to-clear 清空所有 counter。 +// 3. 同一个 get_sys_cnt 时间窗内执行 metrics_prof_start/stop;三种 mode 只替换 +// gate 内部的固定 rounds 工作负载: +// EMPTY:不做工作,测 gate 与计时固有开销; +// SCALAR_CONTROL:只在 scalar 寄存器中执行与 atomic 相同的数据依赖递推; +// DEPENDENT_ATOMIC_ADD:对独占 target 执行 atomicAdd,下一轮 addend 由上一轮 +// atomicAdd 的返回值计算,不允许多条 atomic 并行隐藏单条等待。 +// 4. 关闭 PMU 后才读 total/scalar/request/miss,最后仅用 st_dev 把结果发布到 +// result 独占 cache line,并用 DSB 收口。 +// +// CONTROL 与 ATOMIC 共用可由 host 精确复算的递推(uint64_t 模 2^64): +// value=seed, delta=1, checksum=0; +// 每轮 old=value/atomicAdd(target, delta) 的返回值,checksum+=old, +// delta=1+(old&1);CONTROL 另执行 value+=本轮 delta。 +// 解读时应对多个 rounds 取斜率并扣除 EMPTY/CONTROL:若 atomic 的 total 斜率显著 +// 增长而 scalar 斜率不同比例增长,atomic 等待不属于 scalar busy;若两者同比例增长, +// 则等待被计入 scalar busy。 + +#include "atomic_scalar_pmu_shared.h" +#include "ccec_utils.h" + +CCEC_PROBE_KERNEL_META(atomic_scalar_pmu); + +namespace { + +constexpr uint32_t kPmuPhysicalSubcores = 108; +constexpr uint64_t kPmuCtrl0Offset = 0x4200ULL; + +__aicore__ __attribute__((always_inline)) inline int32_t *PmuCounterBase(uint64_t register_base) +{ + // 以 PMU_CTRL_0(0x4200) 为基准后,所有 counter 都落在 ld_dev 的 12-bit immediate 范围内。 + return reinterpret_cast(register_base + kPmuCtrl0Offset); +} + +__aicore__ __attribute__((always_inline)) inline void ClearPmuCounters(uint64_t register_base) +{ + int32_t *base = PmuCounterBase(register_base); + // A5 PMU counter 是 read-to-clear;显式展开保证每个 ld_dev offset 为编译期常量。 + (void)ld_dev(base, 0x10); + (void)ld_dev(base, 0x18); + (void)ld_dev(base, 0x20); + (void)ld_dev(base, 0x28); + (void)ld_dev(base, 0x30); + (void)ld_dev(base, 0x38); + (void)ld_dev(base, 0x40); + (void)ld_dev(base, 0x48); + (void)ld_dev(base, 0x50); + (void)ld_dev(base, 0x54); + (void)ld_dev(base, 0x60); + (void)ld_dev(base, 0x64); +} + +__aicore__ __attribute__((always_inline)) inline uint64_t ReadPmuScalar(uint64_t register_base) +{ + return static_cast(ld_dev(PmuCounterBase(register_base), 0x10)); +} + +__aicore__ __attribute__((always_inline)) inline uint64_t ReadPmuIcacheRequest(uint64_t register_base) +{ + return static_cast(ld_dev(PmuCounterBase(register_base), 0x18)); +} + +__aicore__ __attribute__((always_inline)) inline uint64_t ReadPmuIcacheMiss(uint64_t register_base) +{ + return static_cast(ld_dev(PmuCounterBase(register_base), 0x20)); +} + +__aicore__ __attribute__((always_inline)) inline uint64_t ReadPmuTotal(uint64_t register_base) +{ + int32_t *base = PmuCounterBase(register_base); + const uint64_t low = static_cast(ld_dev(base, 0x60)); + const uint64_t high = static_cast(ld_dev(base, 0x64)); + return low | (high << 32); +} + +__aicore__ __attribute__((always_inline)) inline void Publish64(__gm__ uint64_t *address, uint64_t value) +{ + __builtin_cce_st_dev(value, address, 0); +} + +} // namespace + +extern "C" __global__ __aicore__ void KERNEL_ENTRY(atomic_scalar_pmu)( + __gm__ atomic_scalar_pmu::ProbeState *state) +{ + using atomic_scalar_pmu::Mode; + + // task-based profiler 可能在入口前已打开 PMU;先关闭,确保 control DCCI 和准备阶段不入窗。 + bisheng::cce::metrics_prof_stop(); + dcci(&state->control, SINGLE_CACHE_LINE); + dsb(DSB_ALL); + + const uint32_t mode_value = state->control.mode; + const uint32_t rounds = state->control.rounds; + const uint64_t seed = state->control.seed; + const uint32_t physical_core_id = static_cast(get_coreid()) & 0x0fffU; + + uint64_t register_base = 0; + if (state->control.pmu_register_bases != 0 && physical_core_id < kPmuPhysicalSubcores) { + __gm__ uint64_t *register_bases = + reinterpret_cast<__gm__ uint64_t *>(state->control.pmu_register_bases); + register_base = register_bases[physical_core_id]; + } + if (register_base != 0) { + ClearPmuCounters(register_base); + } + + uint64_t checksum = 0; + uint64_t delta = 1; + uint64_t scalar_value = seed; + + const uint64_t sys_begin = static_cast(get_sys_cnt()); + bisheng::cce::metrics_prof_start(); + + if (mode_value == static_cast(Mode::ScalarControl)) { + for (uint32_t round = 0; round < rounds; ++round) { + const uint64_t old = scalar_value; + scalar_value += delta; + checksum += old; + delta = 1 + (old & 1U); + } + } else if (mode_value == static_cast(Mode::DependentAtomicAdd)) { + __gm__ uint64_t *target = const_cast<__gm__ uint64_t *>(&state->target.value); + for (uint32_t round = 0; round < rounds; ++round) { + // delta 直接依赖上一轮 old;除第一轮外,后一条 atomic 必须等前一条返回。 + const uint64_t old = atomicAdd(target, delta); + checksum += old; + delta = 1 + (old & 1U); + } + } + // Empty 和非法 mode 都保持空窗;host 仅会发布 enum 中的三个合法值。 + + bisheng::cce::metrics_prof_stop(); + const uint64_t sys_end = static_cast(get_sys_cnt()); + const uint64_t ctrl_after_stop = static_cast(get_ctrl()); + + uint64_t pmu_total = 0; + uint64_t pmu_scalar = 0; + uint64_t pmu_icache_request = 0; + uint64_t pmu_icache_miss = 0; + if (register_base != 0) { + // counter 为 read-to-clear,每项只读一次,且必须在 stop 后执行。 + pmu_scalar = ReadPmuScalar(register_base); + pmu_icache_request = ReadPmuIcacheRequest(register_base); + pmu_icache_miss = ReadPmuIcacheMiss(register_base); + pmu_total = ReadPmuTotal(register_base); + } + + __gm__ atomic_scalar_pmu::ProbeResult *result = &state->result; + Publish64(&result->sys_cycles, sys_end - sys_begin); + Publish64(&result->pmu_total_cycles, pmu_total); + Publish64(&result->pmu_scalar_busy, pmu_scalar); + Publish64(&result->pmu_icache_request, pmu_icache_request); + Publish64(&result->pmu_icache_miss, pmu_icache_miss); + Publish64(&result->checksum, checksum); + Publish64(&result->pmu_ctrl_after_stop, ctrl_after_stop); + Publish64(&result->physical_core_id, physical_core_id); + dsb(DSB_ALL); +} diff --git a/tests/atomic_probe/ccec/atomic_scalar_pmu_host.cpp b/tests/atomic_probe/ccec/atomic_scalar_pmu_host.cpp new file mode 100644 index 0000000000..4ce419b1f6 --- /dev/null +++ b/tests/atomic_probe/ccec/atomic_scalar_pmu_host.cpp @@ -0,0 +1,615 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +// 单 AIV、无轮询、固定操作数的 atomic/标量 PMU 对照 host。 +// 每个 rounds 都重复 EMPTY -> SCALAR_CONTROL -> DEPENDENT_ATOMIC_ADD: +// 1. EMPTY 给出同位置 PMU gate/read 的固定成本; +// 2. SCALAR_CONTROL 执行与 atomic 路径完全相同的返回值递推和 checksum; +// 3. DEPENDENT_ATOMIC_ADD 让下一条 atomicAdd 的 addend 依赖上一条返回值。 +// 因而 (ATOMIC-CONTROL)/rounds 直接回答 atomic 等待周期落在 PMU total、scalar busy +// 中的哪一项,而不混入多核竞争、轮询次数变化或未消费返回值的并行发射。 + +#include "atomic_scalar_pmu_shared.h" +#include "pmu_probe_control.h" +#include "../probe_host.h" + +#include "aicpu_loader/host/load_aicpu_op.h" +#include "common/kernel_args.h" +#include "driver/ascend_hal.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +constexpr uint32_t kPhysicalAicoreCount = 36; +constexpr uint32_t kSubcoresPerAicore = 3; +constexpr uint32_t kPhysicalSubcoreCount = kPhysicalAicoreCount * kSubcoresPerAicore; +constexpr uint32_t kAicorePerDie = 18; +constexpr uint32_t kSubcoresPerDie = kAicorePerDie * kSubcoresPerAicore; +constexpr uint32_t kAivBaseInDie = kAicorePerDie; +constexpr uint64_t kSubcoreStride = 0x100000ULL; +constexpr uint32_t kAicoreMapBytes = 0x300000U; + +static_assert( + kPhysicalSubcoreCount == atomic_probe::pmu::kPmuPhysicalSubcores, "PMU table size mismatch" +); + +bool CheckAcl(aclError error, const char *label) { + if (error == ACL_SUCCESS) return true; + std::fprintf(stderr, "ACL error %d: %s\n", static_cast(error), label); + return false; +} + +std::vector ReadBinary(const std::string &path) { + std::ifstream file(path, std::ios::binary | std::ios::ate); + if (!file) return {}; + const std::streamsize size = file.tellg(); + if (size <= 0) return {}; + std::vector bytes(static_cast(size)); + file.seekg(0, std::ios::beg); + if (!file.read(bytes.data(), size)) return {}; + return bytes; +} + +std::string ArtifactBesideKernel(const std::string &kernel_path, const char *name) { + const size_t slash = kernel_path.find_last_of('/'); + return slash == std::string::npos ? std::string(name) : kernel_path.substr(0, slash + 1) + name; +} + +class PmuResources { +public: + using MapFn = drvError_t (*)(unsigned int, struct res_map_info *, unsigned long *, unsigned int *); + using UnmapFn = drvError_t (*)(unsigned int, struct res_map_info *); + + ~PmuResources() { RestoreAndUnmap(); } + + bool Initialize(uint32_t device) { + device_ = device; + map_ = reinterpret_cast(dlsym(RTLD_DEFAULT, "halResMap")); + unmap_ = reinterpret_cast(dlsym(RTLD_DEFAULT, "halResUnmap")); + if (map_ == nullptr || unmap_ == nullptr) { + hal_handle_ = dlopen("libascend_hal.so", RTLD_NOW | RTLD_GLOBAL); + if (hal_handle_ != nullptr) { + map_ = reinterpret_cast(dlsym(hal_handle_, "halResMap")); + unmap_ = reinterpret_cast(dlsym(hal_handle_, "halResUnmap")); + } + } + if (map_ == nullptr || unmap_ == nullptr) { + std::fprintf(stderr, "Cannot resolve halResMap/halResUnmap.\n"); + return false; + } + + for (uint32_t physical = 0; physical < kPhysicalAicoreCount; ++physical) { + res_map_info &info = map_info_[physical]; + std::memset(&info, 0, sizeof(info)); + info.target_proc_type = PROCESS_CP1; + info.res_type = RES_AICORE; + info.res_id = physical; + unsigned long map_address = 0; + unsigned int map_length = kAicoreMapBytes; + const drvError_t error = map_(device_, &info, &map_address, &map_length); + if (error != 0 || map_address == 0 || map_length < kAicoreMapBytes) { + std::fprintf( + stderr, "halResMap failed: physical=%u error=%d address=0x%lx length=%u\n", physical, + static_cast(error), map_address, map_length + ); + Unmap(); + return false; + } + ++mapped_count_; + const uint64_t base = static_cast(map_address); + const uint32_t die = physical / kAicorePerDie; + const uint32_t local = physical % kAicorePerDie; + const uint32_t die_base = die * kSubcoresPerDie; + register_bases_[die_base + local] = base; + const uint32_t aiv0 = die_base + kAivBaseInDie + local * 2; + register_bases_[aiv0] = base + kSubcoreStride; + register_bases_[aiv0 + 1] = base + 2 * kSubcoreStride; + } + return true; + } + + void RestoreAndUnmap() { + Unmap(); + if (hal_handle_ != nullptr) { + dlclose(hal_handle_); + hal_handle_ = nullptr; + } + } + + const std::array &RegisterBases() const { return register_bases_; } + +private: + void Unmap() { + while (mapped_count_ != 0) { + --mapped_count_; + const drvError_t error = unmap_(device_, &map_info_[mapped_count_]); + if (error != 0) { + std::fprintf( + stderr, "halResUnmap failed: physical=%u error=%d\n", mapped_count_, + static_cast(error) + ); + } + } + } + + uint32_t device_ = 0; + uint32_t mapped_count_ = 0; + void *hal_handle_ = nullptr; + MapFn map_ = nullptr; + UnmapFn unmap_ = nullptr; + std::array map_info_{}; + std::array register_bases_{}; +}; + +bool RunPmuCommand( + host::LoadAicpuOp &loader, aclrtStream stream, KernelArgs *kernel_args, void *control_device, + atomic_probe::pmu::PmuControl *control, atomic_probe::pmu::PmuCommand command +) { + control->command = static_cast(command); + control->status = atomic_probe::pmu::kPmuStatusPending; + kernel_args->enable_profiling_flag = static_cast(command); + const int launch_error = loader.LaunchBuiltInOp(stream, kernel_args, 1, host::KernelNames::RunName); + if (launch_error != 0) { + std::fprintf(stderr, "AICPU PMU helper launch failed: %d\n", launch_error); + return false; + } + if (!CheckAcl(aclrtSynchronizeStream(stream), "aclrtSynchronizeStream(PMU helper)") || + !CheckAcl( + aclrtMemcpy(control, sizeof(*control), control_device, sizeof(*control), ACL_MEMCPY_DEVICE_TO_HOST), + "aclrtMemcpy(D2H PMU control)" + )) { + return false; + } + const bool expected_state = command == atomic_probe::pmu::PmuCommand::Configure + ? control->configured == 1 && control->processed_subcores == atomic_probe::pmu::kPmuPhysicalSubcores + : control->configured == 0 && control->processed_subcores == 0; + if (control->status != 0 || !expected_state) { + std::fprintf( + stderr, "PMU helper failed: command=%u status=%d configured=%u processed=%u\n", control->command, + static_cast(control->status), control->configured, control->processed_subcores + ); + return false; + } + return true; +} + +const char *ModeName(atomic_scalar_pmu::Mode mode) { + switch (mode) { + case atomic_scalar_pmu::Mode::Empty: return "EMPTY"; + case atomic_scalar_pmu::Mode::ScalarControl: return "SCALAR_CONTROL"; + case atomic_scalar_pmu::Mode::DependentAtomicAdd: return "DEPENDENT_ATOMIC_ADD"; + default: return "UNKNOWN"; + } +} + +bool ParseUint64(const char *text, uint64_t maximum, uint64_t *value) { + if (text == nullptr || text[0] == '\0') return false; + errno = 0; + char *end = nullptr; + const unsigned long long parsed = std::strtoull(text, &end, 0); + if (errno != 0 || end == text || *end != '\0' || parsed > maximum) return false; + *value = static_cast(parsed); + return true; +} + +uint32_t RepeatsFromEnv() { + const char *raw = std::getenv("ATOMIC_SCALAR_PMU_REPEATS"); + if (raw == nullptr || raw[0] == '\0') return 7; + uint64_t value = 0; + if (!ParseUint64(raw, 100, &value) || value == 0) { + std::fprintf(stderr, "ATOMIC_SCALAR_PMU_REPEATS must be in 1..100: %s\n", raw); + return 0; + } + return static_cast(value); +} + +uint64_t SeedFromEnv(bool *ok) { + const char *raw = std::getenv("ATOMIC_SCALAR_PMU_SEED"); + if (raw == nullptr || raw[0] == '\0') return 0x1234ULL; + uint64_t value = 0; + const bool parsed = ParseUint64(raw, std::numeric_limits::max(), &value); + *ok &= parsed; + if (!parsed) std::fprintf(stderr, "Invalid ATOMIC_SCALAR_PMU_SEED: %s\n", raw); + return value; +} + +std::vector RoundsFromEnv(bool *ok) { + const char *raw = std::getenv("ATOMIC_SCALAR_PMU_ROUNDS"); + if (raw == nullptr || raw[0] == '\0') return {0, 1, 4, 16, 64, 256, 1024, 4096, 8192}; + std::vector rounds; + const std::string input(raw); + size_t begin = 0; + while (begin <= input.size()) { + const size_t comma = input.find(',', begin); + const std::string token = input.substr(begin, comma == std::string::npos ? comma : comma - begin); + uint64_t value = 0; + if (!ParseUint64(token.c_str(), 1000000, &value)) { + std::fprintf(stderr, "Invalid ATOMIC_SCALAR_PMU_ROUNDS item: %s\n", token.c_str()); + *ok = false; + return {}; + } + rounds.push_back(static_cast(value)); + if (comma == std::string::npos) break; + begin = comma + 1; + } + return rounds; +} + +struct Oracle { + uint64_t final_value; + uint64_t checksum; +}; + +Oracle Simulate(uint64_t seed, uint32_t rounds) { + uint64_t value = seed; + uint64_t delta = 1; + uint64_t checksum = 0; + for (uint32_t round = 0; round < rounds; ++round) { + const uint64_t old = value; + value += delta; + checksum += old; + delta = 1 + (old & 1ULL); + } + return {value, checksum}; +} + +struct Sample { + atomic_scalar_pmu::ProbeResult result{}; + uint64_t final_value = 0; +}; + +bool ValidateSample( + const Sample &sample, atomic_scalar_pmu::Mode mode, uint32_t rounds, uint64_t seed, std::string *reason +) { + const Oracle oracle = Simulate(seed, rounds); + const uint64_t expected_checksum = mode == atomic_scalar_pmu::Mode::Empty ? 0 : oracle.checksum; + const uint64_t expected_final = mode == atomic_scalar_pmu::Mode::DependentAtomicAdd ? oracle.final_value : seed; + if (sample.result.checksum != expected_checksum) { + *reason = "checksum"; + return false; + } + if (sample.final_value != expected_final) { + *reason = "target-final"; + return false; + } + if (sample.result.physical_core_id >= kPhysicalSubcoreCount) { + *reason = "physical-core-id"; + return false; + } + if ((sample.result.pmu_ctrl_after_stop & 1ULL) != 0) { + *reason = "pmu-gate-still-enabled"; + return false; + } + if (sample.result.sys_cycles == 0 || sample.result.pmu_total_cycles == 0) { + *reason = "zero-cycle-window"; + return false; + } + if (sample.result.pmu_icache_miss > sample.result.pmu_icache_request) { + *reason = "icache-miss-exceeds-request"; + return false; + } + return true; +} + +bool RunOne( + aclrtFuncHandle function, aclrtStream stream, void *state_device, uint64_t pmu_register_bases, + atomic_scalar_pmu::Mode mode, uint32_t rounds, uint32_t repeat, uint64_t seed, Sample *sample +) { + atomic_scalar_pmu::ProbeState state{}; + state.control.pmu_register_bases = pmu_register_bases; + state.control.mode = static_cast(mode); + state.control.rounds = rounds; + state.control.seed = seed; + state.target.value = seed; + if (!CheckAcl( + aclrtMemcpy( + state_device, sizeof(state), &state, sizeof(state), ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D probe state)" + )) { + return false; + } + + struct KernelArgs { + uint64_t state_pointer; + } args{reinterpret_cast(state_device)}; + static_assert(sizeof(KernelArgs) == sizeof(uint64_t), "unexpected CCEC kernel argument ABI"); + if (!CheckAcl( + aclrtLaunchKernelWithHostArgs(function, 1, stream, nullptr, &args, sizeof(args), nullptr, 0), + "aclrtLaunchKernelWithHostArgs" + ) || + !CheckAcl(aclrtSynchronizeStream(stream), "aclrtSynchronizeStream(AIV probe)") || + !CheckAcl( + aclrtMemcpy(&state, sizeof(state), state_device, sizeof(state), ACL_MEMCPY_DEVICE_TO_HOST), + "aclrtMemcpy(D2H probe state)" + )) { + return false; + } + + sample->result = state.result; + sample->final_value = state.target.value; + std::string reason; + const bool semantic_ok = ValidateSample(*sample, mode, rounds, seed, &reason); + std::printf( + "[RAW] repeat=%u rounds=%u mode=%s sys_cycles=%llu total=%llu scalar=%llu " + "icache_req=%llu icache_miss=%llu checksum=%llu final=%llu physical=%llu ctrl=0x%llx status=%s%s%s\n", + repeat, rounds, ModeName(mode), static_cast(sample->result.sys_cycles), + static_cast(sample->result.pmu_total_cycles), + static_cast(sample->result.pmu_scalar_busy), + static_cast(sample->result.pmu_icache_request), + static_cast(sample->result.pmu_icache_miss), + static_cast(sample->result.checksum), + static_cast(sample->final_value), + static_cast(sample->result.physical_core_id), + static_cast(sample->result.pmu_ctrl_after_stop), semantic_ok ? "PASS" : "FAIL", + semantic_ok ? "" : " reason=", semantic_ok ? "" : reason.c_str() + ); + return semantic_ok; +} + +uint64_t Median(std::vector values) { + std::sort(values.begin(), values.end()); + const size_t middle = values.size() / 2; + if ((values.size() & 1U) != 0) return values[middle]; + return values[middle - 1] + (values[middle] - values[middle - 1]) / 2; +} + +double Median(std::vector values) { + std::sort(values.begin(), values.end()); + const size_t middle = values.size() / 2; + return (values.size() & 1U) != 0 ? values[middle] : (values[middle - 1] + values[middle]) / 2.0; +} + +using CounterMember = uint64_t atomic_scalar_pmu::ProbeResult::*; + +double PairedDeltaPerOperation( + const std::vector &minuend, const std::vector &subtrahend, + CounterMember member, uint32_t rounds +) { + std::vector deltas; + for (size_t index = 0; index < minuend.size(); ++index) { + deltas.push_back( + (static_cast(minuend[index].result.*member) - + static_cast(subtrahend[index].result.*member)) / + rounds + ); + } + return Median(std::move(deltas)); +} + +void PrintRoundSummary( + uint32_t rounds, const std::array, 3> &samples +) { + struct Metric { + const char *name; + CounterMember member; + }; + constexpr Metric metrics[] = { + {"sys_cycles", &atomic_scalar_pmu::ProbeResult::sys_cycles}, + {"total", &atomic_scalar_pmu::ProbeResult::pmu_total_cycles}, + {"scalar", &atomic_scalar_pmu::ProbeResult::pmu_scalar_busy}, + {"icache_req", &atomic_scalar_pmu::ProbeResult::pmu_icache_request}, + {"icache_miss", &atomic_scalar_pmu::ProbeResult::pmu_icache_miss}, + }; + for (uint32_t mode_index = 0; mode_index < static_cast(atomic_scalar_pmu::Mode::Count); + ++mode_index) { + const auto mode = static_cast(mode_index); + std::printf("[MEDIAN] rounds=%u mode=%s", rounds, ModeName(mode)); + for (const Metric &metric : metrics) { + std::vector values; + for (const Sample &sample : samples[mode_index]) values.push_back(sample.result.*(metric.member)); + std::printf(" %s=%llu", metric.name, static_cast(Median(std::move(values)))); + } + std::printf("\n"); + } + if (rounds == 0) return; + + const auto &empty = samples[static_cast(atomic_scalar_pmu::Mode::Empty)]; + const auto &control = samples[static_cast(atomic_scalar_pmu::Mode::ScalarControl)]; + const auto &atomic = samples[static_cast(atomic_scalar_pmu::Mode::DependentAtomicAdd)]; + for (const Metric &metric : metrics) { + const double control_minus_empty = + PairedDeltaPerOperation(control, empty, metric.member, rounds); + const double atomic_minus_control = + PairedDeltaPerOperation(atomic, control, metric.member, rounds); + std::printf( + "[DELTA_PER_OP] rounds=%u metric=%s control_minus_empty=%.6f atomic_minus_control=%.6f\n", rounds, + metric.name, control_minus_empty, atomic_minus_control + ); + } + + const double atomic_sys_ns = PairedDeltaPerOperation( + atomic, control, &atomic_scalar_pmu::ProbeResult::sys_cycles, rounds + ); + const double atomic_total_cycles = PairedDeltaPerOperation( + atomic, control, &atomic_scalar_pmu::ProbeResult::pmu_total_cycles, rounds + ); + const double atomic_scalar_cycles = PairedDeltaPerOperation( + atomic, control, &atomic_scalar_pmu::ProbeResult::pmu_scalar_busy, rounds + ); + const double scalar_share = atomic_total_cycles == 0.0 ? 0.0 : atomic_scalar_cycles / atomic_total_cycles; + std::printf( + "[ATOMIC_CLASSIFICATION] rounds=%u completion_ns_per_op=%.6f " + "pmu_total_cycles_per_op=%.6f scalar_busy_cycles_per_op=%.6f scalar_share=%.9f\n", + rounds, atomic_sys_ns, atomic_total_cycles, atomic_scalar_cycles, scalar_share + ); +} + +} // namespace + +int main(int argc, char **argv) { + const std::string kernel_path = argc > 1 ? argv[1] : "./atomic_scalar_pmu_kernel.o"; + if (argc > 2) { + std::fprintf(stderr, "Usage: %s [atomic_scalar_pmu_kernel.o]\n", argv[0]); + return EXIT_FAILURE; + } + const uint32_t repeats = RepeatsFromEnv(); + bool options_ok = repeats != 0; + const uint64_t seed = SeedFromEnv(&options_ok); + const std::vector round_values = RoundsFromEnv(&options_ok); + if (!options_ok || round_values.empty()) return EXIT_FAILURE; + + const int32_t device = atomic_probe::DeviceId(); + if (device < 0) return EXIT_FAILURE; + const std::vector kernel_data = ReadBinary(kernel_path); + if (kernel_data.empty()) { + std::fprintf(stderr, "Cannot read kernel binary: %s\n", kernel_path.c_str()); + return EXIT_FAILURE; + } + + if (!CheckAcl(aclInit(nullptr), "aclInit") || !CheckAcl(aclrtSetDevice(device), "aclrtSetDevice")) { + return EXIT_FAILURE; + } + aclrtStream stream = nullptr; + if (!CheckAcl(aclrtCreateStream(&stream), "aclrtCreateStream")) return EXIT_FAILURE; + + aclrtBinHandle binary_handle = nullptr; + if (!CheckAcl( + atomic_probe::LoadAicoreBinaryFromData(kernel_data.data(), kernel_data.size(), &binary_handle), + "LoadAicoreBinaryFromData" + )) { + return EXIT_FAILURE; + } + aclrtFuncHandle function = nullptr; + if (!CheckAcl(aclrtBinaryGetFunctionByEntry(binary_handle, 0, &function), "aclrtBinaryGetFunctionByEntry")) { + return EXIT_FAILURE; + } + + void *state_device = nullptr; + void *pmu_regs_device = nullptr; + void *pmu_control_device = nullptr; + if (!CheckAcl( + aclrtMalloc(&state_device, sizeof(atomic_scalar_pmu::ProbeState), ACL_MEM_MALLOC_NORMAL_ONLY), + "aclrtMalloc(probe state)" + )) { + return EXIT_FAILURE; + } + PmuResources pmu_resources; + if (!pmu_resources.Initialize(static_cast(device))) return EXIT_FAILURE; + const size_t pmu_regs_bytes = sizeof(pmu_resources.RegisterBases()); + if (!CheckAcl(aclrtMalloc(&pmu_regs_device, pmu_regs_bytes, ACL_MEM_MALLOC_NORMAL_ONLY), "aclrtMalloc(PMU regs)") || + !CheckAcl( + aclrtMemcpy( + pmu_regs_device, pmu_regs_bytes, pmu_resources.RegisterBases().data(), pmu_regs_bytes, + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D PMU regs)" + ) || + !CheckAcl( + aclrtMalloc(&pmu_control_device, sizeof(atomic_probe::pmu::PmuControl), ACL_MEM_MALLOC_NORMAL_ONLY), + "aclrtMalloc(PMU control)" + )) { + return EXIT_FAILURE; + } + + atomic_probe::pmu::PmuControl pmu_control{}; + pmu_control.magic = atomic_probe::pmu::kPmuControlMagic; + pmu_control.version = atomic_probe::pmu::kPmuControlVersion; + pmu_control.expected_subcores = atomic_probe::pmu::kPmuPhysicalSubcores; + if (!CheckAcl( + aclrtMemcpy( + pmu_control_device, sizeof(pmu_control), &pmu_control, sizeof(pmu_control), + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D initial PMU control)" + )) { + return EXIT_FAILURE; + } + + const std::string dispatcher_path = ArtifactBesideKernel(kernel_path, "libsimpler_aicpu_dispatcher.so"); + const std::string helper_path = ArtifactBesideKernel(kernel_path, "libatomic_scalar_pmu_aicpu.so"); + const std::vector dispatcher_data = ReadBinary(dispatcher_path); + const std::vector helper_data = ReadBinary(helper_path); + if (dispatcher_data.empty() || helper_data.empty()) { + std::fprintf(stderr, "Cannot read PMU artifacts: %s %s\n", dispatcher_path.c_str(), helper_path.c_str()); + return EXIT_FAILURE; + } + host::LoadAicpuOp pmu_loader; + if (pmu_loader.BootstrapDispatcher( + dispatcher_data.data(), dispatcher_data.size(), helper_data.data(), helper_data.size(), stream, device + ) != 0 || + pmu_loader.Init() != 0) { + std::fprintf(stderr, "Cannot initialize PMU AICPU helper.\n"); + return EXIT_FAILURE; + } + KernelArgs pmu_kernel_args{}; + pmu_kernel_args.runtime_args = reinterpret_cast(pmu_control_device); + pmu_kernel_args.regs = reinterpret_cast(pmu_regs_device); + pmu_kernel_args.device_id = static_cast(device); + if (!RunPmuCommand( + pmu_loader, stream, &pmu_kernel_args, pmu_control_device, &pmu_control, + atomic_probe::pmu::PmuCommand::Configure + )) { + return EXIT_FAILURE; + } + + std::printf( + "=== Single-AIV dependent atomicAdd scalar-busy PMU probe ===\n" + "device=%d repeats=%u seed=0x%llx events=total,scalar_busy(0x1),icache_req(0x34),icache_miss(0x35)\n", + device, repeats, static_cast(seed) + ); + bool all_passed = true; + for (const uint32_t rounds : round_values) { + std::array, 3> samples; + for (uint32_t repeat = 1; repeat <= repeats; ++repeat) { + for (uint32_t mode_index = 0; mode_index < static_cast(atomic_scalar_pmu::Mode::Count); + ++mode_index) { + Sample sample; + const auto mode = static_cast(mode_index); + const bool passed = RunOne( + function, stream, state_device, reinterpret_cast(pmu_regs_device), mode, rounds, + repeat, seed, &sample + ); + all_passed &= passed; + samples[mode_index].push_back(sample); + if (!passed) break; + } + if (!all_passed) break; + } + if (!all_passed) break; + PrintRoundSummary(rounds, samples); + } + + const bool restored = RunPmuCommand( + pmu_loader, stream, &pmu_kernel_args, pmu_control_device, &pmu_control, + atomic_probe::pmu::PmuCommand::Restore + ); + bool cleanup_ok = restored; + pmu_loader.Finalize(); + pmu_resources.RestoreAndUnmap(); + cleanup_ok &= CheckAcl(aclrtFree(pmu_control_device), "aclrtFree(PMU control)"); + cleanup_ok &= CheckAcl(aclrtFree(pmu_regs_device), "aclrtFree(PMU regs)"); + cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(probe state)"); + cleanup_ok &= CheckAcl(aclrtBinaryUnLoad(binary_handle), "aclrtBinaryUnLoad"); + cleanup_ok &= CheckAcl(aclrtDestroyStream(stream), "aclrtDestroyStream"); + cleanup_ok &= CheckAcl(aclrtResetDevice(device), "aclrtResetDevice"); + cleanup_ok &= CheckAcl(aclFinalize(), "aclFinalize"); + std::printf( + "[SUMMARY] semantic_status=%s pmu_restore_and_cleanup=%s\n", all_passed ? "PASS" : "FAIL", + cleanup_ok ? "PASS" : "FAIL" + ); + return all_passed && cleanup_ok ? EXIT_SUCCESS : EXIT_FAILURE; +} diff --git a/tests/atomic_probe/ccec/atomic_scalar_pmu_shared.h b/tests/atomic_probe/ccec/atomic_scalar_pmu_shared.h new file mode 100644 index 0000000000..415449058e --- /dev/null +++ b/tests/atomic_probe/ccec/atomic_scalar_pmu_shared.h @@ -0,0 +1,74 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef TESTS_ATOMIC_PROBE_CCEC_ATOMIC_SCALAR_PMU_SHARED_H_ +#define TESTS_ATOMIC_PROBE_CCEC_ATOMIC_SCALAR_PMU_SHARED_H_ + +#include +#include + +namespace atomic_scalar_pmu { + +// 三条路径使用同一份 kernel 和同一个 PMU 读数协议,只替换 gate 内的固定次数工作负载: +// EMPTY 量 gate/read 固有开销;SCALAR_CONTROL 量与 atomic 路径相同的标量递推; +// DEPENDENT_ATOMIC_ADD 让后一条 atomicAdd 的加数依赖前一条返回值,避免多条 atomic 并行掩盖等待时间。 +enum class Mode : uint32_t { + Empty = 0, + ScalarControl = 1, + DependentAtomicAdd = 2, + Count = 3, +}; + +// Host launch 前只写本 cache line;kernel 在测量窗口中只读。PMU MMIO base 表 +// 按 get_coreid() 的低 12 bit 索引,布局与公共 pmu_probe AICPU helper 完全一致。 +struct alignas(64) ProbeControl { + uint64_t pmu_register_bases; + uint32_t mode; + uint32_t rounds; + uint64_t seed; + uint64_t reserved[5]; +}; + +// Atomic 目标独占 cache line,排除 result/control 的普通 GM 写或 DCCI 对原子值的影响。 +struct alignas(64) AtomicTarget { + volatile uint64_t value; + uint64_t reserved[7]; +}; + +// 单 AIV 独占写结果 cache line。sys_cycles 是 get_sys_cnt() 前后差;其余四项是同一 +// gate 窗口内的 PMU total、scalar busy、I-cache request 和 I-cache miss 原始计数。 +struct alignas(64) ProbeResult { + uint64_t sys_cycles; + uint64_t pmu_total_cycles; + uint64_t pmu_scalar_busy; + uint64_t pmu_icache_request; + uint64_t pmu_icache_miss; + uint64_t checksum; + uint64_t pmu_ctrl_after_stop; + uint64_t physical_core_id; +}; + +struct alignas(64) ProbeState { + ProbeControl control; + AtomicTarget target; + ProbeResult result; +}; + +static_assert(sizeof(ProbeControl) == 64, "probe control must occupy one cache line"); +static_assert(sizeof(AtomicTarget) == 64, "atomic target must occupy one cache line"); +static_assert(sizeof(ProbeResult) == 64, "probe result must occupy one cache line"); +static_assert(offsetof(ProbeState, target) == 64, "atomic target must start on its own cache line"); +static_assert(offsetof(ProbeState, result) == 128, "probe result must start on its own cache line"); +static_assert(sizeof(ProbeState) == 192, "probe state ABI changed unexpectedly"); + +} // namespace atomic_scalar_pmu + +#endif // TESTS_ATOMIC_PROBE_CCEC_ATOMIC_SCALAR_PMU_SHARED_H_ diff --git a/tests/atomic_probe/ccec/icache_scalar_pmu.cpp b/tests/atomic_probe/ccec/icache_scalar_pmu.cpp new file mode 100644 index 0000000000..2a2ddb7e6f --- /dev/null +++ b/tests/atomic_probe/ccec/icache_scalar_pmu.cpp @@ -0,0 +1,226 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +// 测试目标:用单个 AIV 精确核实“等待 I-cache miss 回填的周期是否计入 PMU +// scalar busy”。Host/AICPU 在 launch 前把 PMU slot0/1/2 配置为 scalar +// busy(0x1)、I-cache request(0x34)、I-cache miss(0x35)。本用例不含 atomic、 +// GM 轮询、Vector/Cube/MTE 计算,因此 WARM/COLD 的差值只来自 target 的取指状态。 +// +// 每次 kernel 的完整时序: +// 1. 关闭 PMU;对 host 写入的 control 独立 cache line 执行 DCCI + DSB;读取 +// mode、seed、PMU MMIO base,并用 read-to-clear 清空所有 PMU counter。 +// 2. 在 PMU 窗口外准备 I-cache: +// WARM:先调用一次被测 noinline target,使其约 8 KiB 指令体进入 I-cache; +// COLD:调用约 32 KiB 的 noinline evictor,以超过 16 KiB 容量的顺序 +// 指令流替换 I-cache 内容。 +// 两条准备路径都返回可精确复算的 checksum,确保调用不能被编译器删除。 +// 3. 两条路径在分支后汇合。get_sys_cnt 后打开 PMU,窗口内只从同一个调用点 +// 调用同一个 noinline target 一次,然后立即关闭 PMU。两种 mode 的窗内动态 +// 指令完全相同,仅 target 调用前的 I-cache 冷热状态不同。 +// 4. PMU 关闭后才读取 total/scalar/request/miss;最后仅用 st_dev 发布结果, +// 并用 DSB 收口。Host 必须同时校验 target checksum、准备 checksum、mode echo、 +// physical core id 和 PMU gate 状态。 +// +// 预期与判读: +// - COLD 的 I-cache miss 必须显著高于 WARM,先证明冷热对照确实成立; +// - 以 COLD-WARM 扣除同一 target 的固定执行成本。若 total 增量与 scalar busy +// 增量近似相同,miss 回填等待计入 scalar busy;若 total 显著增加而 scalar +// busy 不同比例增加,则该等待形成 scalar-busy gap。 +// - 构建后还必须按最终 ELF 符号大小核实 target >= 8 KiB、evictor >= 32 KiB; +// 否则本用例只能算源码意图,不能算有效的 I-cache 驱逐实验。 + +#include "icache_scalar_pmu_shared.h" +#include "ccec_utils.h" + +CCEC_PROBE_KERNEL_META(icache_scalar_pmu); + +namespace { + +constexpr uint32_t kPmuPhysicalSubcores = 108; +constexpr uint64_t kPmuCtrl0Offset = 0x4200ULL; + +__aicore__ __attribute__((always_inline)) inline int32_t *PmuCounterBase(uint64_t register_base) +{ + // 以 PMU_CTRL_0(0x4200) 为基准后,所有 counter 都落在 ld_dev 的 12-bit immediate 范围内。 + return reinterpret_cast(register_base + kPmuCtrl0Offset); +} + +__aicore__ __attribute__((always_inline)) inline void ClearPmuCounters(uint64_t register_base) +{ + int32_t *base = PmuCounterBase(register_base); + // A5 PMU counter 是 read-to-clear;逐项显式展开,保持 ld_dev offset 为编译期常量。 + (void)ld_dev(base, 0x10); + (void)ld_dev(base, 0x18); + (void)ld_dev(base, 0x20); + (void)ld_dev(base, 0x28); + (void)ld_dev(base, 0x30); + (void)ld_dev(base, 0x38); + (void)ld_dev(base, 0x40); + (void)ld_dev(base, 0x48); + (void)ld_dev(base, 0x50); + (void)ld_dev(base, 0x54); + (void)ld_dev(base, 0x60); + (void)ld_dev(base, 0x64); +} + +__aicore__ __attribute__((always_inline)) inline uint64_t ReadPmuScalar(uint64_t register_base) +{ + return static_cast(ld_dev(PmuCounterBase(register_base), 0x10)); +} + +__aicore__ __attribute__((always_inline)) inline uint64_t ReadPmuIcacheRequest(uint64_t register_base) +{ + return static_cast(ld_dev(PmuCounterBase(register_base), 0x18)); +} + +__aicore__ __attribute__((always_inline)) inline uint64_t ReadPmuIcacheMiss(uint64_t register_base) +{ + return static_cast(ld_dev(PmuCounterBase(register_base), 0x20)); +} + +__aicore__ __attribute__((always_inline)) inline uint64_t ReadPmuTotal(uint64_t register_base) +{ + int32_t *base = PmuCounterBase(register_base); + const uint64_t low = static_cast(ld_dev(base, 0x60)); + const uint64_t high = static_cast(ld_dev(base, 0x64)); + return low | (high << 32); +} + +__aicore__ __attribute__((always_inline)) inline void Publish64(__gm__ uint64_t *address, uint64_t value) +{ + __builtin_cce_st_dev(value, address, 0); +} + +} // namespace + +// 递归宏最终展开成固定数量的独立 volatile NOP。这里不用运行时循环,是为了让 +// target/evictor 的静态指令 footprint 本身达到指定大小,而不是反复执行一个热循环。 +#define ICACHE_PMU_NOPS_1() asm volatile("nop"); +#define ICACHE_PMU_NOPS_2() ICACHE_PMU_NOPS_1() ICACHE_PMU_NOPS_1() +#define ICACHE_PMU_NOPS_4() ICACHE_PMU_NOPS_2() ICACHE_PMU_NOPS_2() +#define ICACHE_PMU_NOPS_8() ICACHE_PMU_NOPS_4() ICACHE_PMU_NOPS_4() +#define ICACHE_PMU_NOPS_16() ICACHE_PMU_NOPS_8() ICACHE_PMU_NOPS_8() +#define ICACHE_PMU_NOPS_32() ICACHE_PMU_NOPS_16() ICACHE_PMU_NOPS_16() +#define ICACHE_PMU_NOPS_64() ICACHE_PMU_NOPS_32() ICACHE_PMU_NOPS_32() +#define ICACHE_PMU_NOPS_128() ICACHE_PMU_NOPS_64() ICACHE_PMU_NOPS_64() +#define ICACHE_PMU_NOPS_256() ICACHE_PMU_NOPS_128() ICACHE_PMU_NOPS_128() +#define ICACHE_PMU_NOPS_512() ICACHE_PMU_NOPS_256() ICACHE_PMU_NOPS_256() +#define ICACHE_PMU_NOPS_1024() ICACHE_PMU_NOPS_512() ICACHE_PMU_NOPS_512() +#define ICACHE_PMU_NOPS_2048() ICACHE_PMU_NOPS_1024() ICACHE_PMU_NOPS_1024() +#define ICACHE_PMU_NOPS_4096() ICACHE_PMU_NOPS_2048() ICACHE_PMU_NOPS_2048() +#define ICACHE_PMU_NOPS_8192() ICACHE_PMU_NOPS_4096() ICACHE_PMU_NOPS_4096() + +// 保持外部可见、used、noinline:WARM 预热调用和 gate 内测量调用必须指向同一符号, +// 不能被内联成两份物理代码。ELF 检查还会验证本函数的最终符号大小至少为 8 KiB。 +extern "C" __aicore__ __attribute__((noinline, used, aligned(128))) uint64_t +icache_scalar_pmu_target(uint64_t seed) +{ + ICACHE_PMU_NOPS_2048() + return icache_scalar_pmu::TargetOracle(seed); +} + +// evictor 同样保持外部可见、used、noinline;8192 条 volatile NOP 形成约 32 KiB +// 顺序指令流。返回值由 host 复算,额外证明 COLD 准备调用确实完成。 +extern "C" __aicore__ __attribute__((noinline, used, aligned(128))) uint64_t +icache_scalar_pmu_evictor(uint64_t seed) +{ + ICACHE_PMU_NOPS_8192() + return icache_scalar_pmu::EvictorOracle(seed); +} + +// 把 mode 分支封装在另一个 noinline 函数内:kernel 本体在准备调用返回以后没有 +// WARM/COLD 控制流,防止 O3 对共同测量尾部做 tail duplication,进而在 gate 内 +// 生成两个物理调用点。该函数本身完全位于 PMU start 之前。 +extern "C" __aicore__ __attribute__((noinline, used)) uint64_t +icache_scalar_pmu_prepare(uint32_t mode_value, uint64_t seed) +{ + if (mode_value == static_cast(icache_scalar_pmu::Mode::WarmTarget)) { + return icache_scalar_pmu_target(seed); + } + if (mode_value == static_cast(icache_scalar_pmu::Mode::ColdTarget)) { + return icache_scalar_pmu_evictor(seed); + } + return 0; +} + +extern "C" __global__ __aicore__ void KERNEL_ENTRY(icache_scalar_pmu)( + __gm__ icache_scalar_pmu::ProbeState *state) +{ + // task-based profiler 可能在入口前已打开 PMU;先关闭,确保准备阶段绝不入窗。 + bisheng::cce::metrics_prof_stop(); + dcci(&state->control, SINGLE_CACHE_LINE); + dsb(DSB_ALL); + + const uint32_t mode_value = state->control.mode; + const uint64_t seed = state->control.seed; + const uint32_t physical_core_id = static_cast(get_coreid()) & 0x0fffU; + + uint64_t register_base = 0; + if (state->control.pmu_register_bases != 0 && physical_core_id < kPmuPhysicalSubcores) { + __gm__ uint64_t *register_bases = + reinterpret_cast<__gm__ uint64_t *>(state->control.pmu_register_bases); + register_base = register_bases[physical_core_id]; + } + if (register_base != 0) { + ClearPmuCounters(register_base); + } + + const uint64_t preparation_checksum = icache_scalar_pmu_prepare(mode_value, seed); + + // mode 分支在这里结束。以下测量窗口对 WARM/COLD 是同一个静态调用点和同一 + // target 符号,动态指令序列不再依赖 mode。 + const uint64_t sys_begin = static_cast(get_sys_cnt()); + bisheng::cce::metrics_prof_start(); + const uint64_t target_checksum = icache_scalar_pmu_target(seed); + bisheng::cce::metrics_prof_stop(); + const uint64_t sys_end = static_cast(get_sys_cnt()); + const uint64_t ctrl_after_stop = static_cast(get_ctrl()); + + uint64_t pmu_total = 0; + uint64_t pmu_scalar = 0; + uint64_t pmu_icache_request = 0; + uint64_t pmu_icache_miss = 0; + if (register_base != 0) { + // counter 为 read-to-clear,每项只读一次,且必须在 stop 后执行。 + pmu_scalar = ReadPmuScalar(register_base); + pmu_icache_request = ReadPmuIcacheRequest(register_base); + pmu_icache_miss = ReadPmuIcacheMiss(register_base); + pmu_total = ReadPmuTotal(register_base); + } + + __gm__ icache_scalar_pmu::ProbeResult *result = &state->result; + Publish64(&result->sys_cycles, sys_end - sys_begin); + Publish64(&result->pmu_total_cycles, pmu_total); + Publish64(&result->pmu_scalar_busy, pmu_scalar); + Publish64(&result->pmu_icache_request, pmu_icache_request); + Publish64(&result->pmu_icache_miss, pmu_icache_miss); + Publish64(&result->target_checksum, target_checksum); + Publish64(&result->preparation_checksum, preparation_checksum); + Publish64(&result->pmu_ctrl_after_stop, ctrl_after_stop); + Publish64(&result->physical_core_id, physical_core_id); + Publish64(&result->mode_echo, mode_value); + dsb(DSB_ALL); +} + +#undef ICACHE_PMU_NOPS_8192 +#undef ICACHE_PMU_NOPS_4096 +#undef ICACHE_PMU_NOPS_2048 +#undef ICACHE_PMU_NOPS_1024 +#undef ICACHE_PMU_NOPS_512 +#undef ICACHE_PMU_NOPS_256 +#undef ICACHE_PMU_NOPS_128 +#undef ICACHE_PMU_NOPS_64 +#undef ICACHE_PMU_NOPS_32 +#undef ICACHE_PMU_NOPS_16 +#undef ICACHE_PMU_NOPS_8 +#undef ICACHE_PMU_NOPS_4 +#undef ICACHE_PMU_NOPS_2 +#undef ICACHE_PMU_NOPS_1 diff --git a/tests/atomic_probe/ccec/icache_scalar_pmu_host.cpp b/tests/atomic_probe/ccec/icache_scalar_pmu_host.cpp new file mode 100644 index 0000000000..6b881dcf42 --- /dev/null +++ b/tests/atomic_probe/ccec/icache_scalar_pmu_host.cpp @@ -0,0 +1,402 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +// 单 AIV、同一 target 调用点的 WARM/COLD I-cache PMU 配对 host。 +// 每一对样本使用同一个 seed;WARM 在窗外预热 target,COLD 在窗外执行 32 KiB +// evictor。Host 逐样本复算两种 checksum,并要求配对样本落在同一物理 AIV。 +// 最终只报告 COLD-WARM 的原始差值和比例,不在代码中预设“miss 是否计入 +// scalar busy”的结论。 + +#include "icache_scalar_pmu_shared.h" +#include "pmu_probe_host_support.h" +#include "../probe_host.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using atomic_probe::pmu::CheckAcl; +using atomic_probe::pmu::ReadBinary; + +constexpr uint32_t kMinimumRepeats = 11; +constexpr uint32_t kMaximumRepeats = 101; +constexpr uint64_t kRepeatSeedStride = 0x9e3779b97f4a7c15ULL; + +const char *ModeName(icache_scalar_pmu::Mode mode) +{ + switch (mode) { + case icache_scalar_pmu::Mode::WarmTarget: return "WARM_TARGET"; + case icache_scalar_pmu::Mode::ColdTarget: return "COLD_TARGET"; + default: return "UNKNOWN"; + } +} + +bool ParseUint64(const char *text, uint64_t maximum, uint64_t *value) +{ + if (text == nullptr || text[0] == '\0') return false; + errno = 0; + char *end = nullptr; + const unsigned long long parsed = std::strtoull(text, &end, 0); + if (errno != 0 || end == text || *end != '\0' || parsed > maximum) return false; + *value = static_cast(parsed); + return true; +} + +uint32_t RepeatsFromEnv() +{ + const char *raw = std::getenv("ICACHE_SCALAR_PMU_REPEATS"); + if (raw == nullptr || raw[0] == '\0') return kMinimumRepeats; + uint64_t value = 0; + if (!ParseUint64(raw, kMaximumRepeats, &value) || value < kMinimumRepeats) { + std::fprintf( + stderr, "ICACHE_SCALAR_PMU_REPEATS must be in %u..%u: %s\n", kMinimumRepeats, + kMaximumRepeats, raw + ); + return 0; + } + return static_cast(value); +} + +uint64_t SeedFromEnv(bool *ok) +{ + const char *raw = std::getenv("ICACHE_SCALAR_PMU_SEED"); + if (raw == nullptr || raw[0] == '\0') return 0x123456789abcdef0ULL; + uint64_t value = 0; + const bool parsed = ParseUint64(raw, std::numeric_limits::max(), &value); + *ok &= parsed; + if (!parsed) std::fprintf(stderr, "Invalid ICACHE_SCALAR_PMU_SEED: %s\n", raw); + return value; +} + +struct Sample { + icache_scalar_pmu::ProbeResult result{}; +}; + +bool ValidateSample( + const Sample &sample, icache_scalar_pmu::Mode mode, uint64_t seed, std::string *reason +) +{ + const uint64_t expected_preparation = mode == icache_scalar_pmu::Mode::WarmTarget + ? icache_scalar_pmu::TargetOracle(seed) + : icache_scalar_pmu::EvictorOracle(seed); + if (sample.result.target_checksum != icache_scalar_pmu::TargetOracle(seed)) { + *reason = "target-checksum"; + return false; + } + if (sample.result.preparation_checksum != expected_preparation) { + *reason = "preparation-checksum"; + return false; + } + if (sample.result.mode_echo != static_cast(mode)) { + *reason = "mode-echo"; + return false; + } + if (sample.result.physical_core_id >= atomic_probe::pmu::kPhysicalSubcoreCount) { + *reason = "physical-core-id"; + return false; + } + if ((sample.result.pmu_ctrl_after_stop & 1ULL) != 0) { + *reason = "pmu-gate-still-enabled"; + return false; + } + if (sample.result.sys_cycles == 0 || sample.result.pmu_total_cycles == 0) { + *reason = "zero-cycle-window"; + return false; + } + if (sample.result.pmu_icache_miss > sample.result.pmu_icache_request) { + *reason = "icache-miss-exceeds-request"; + return false; + } + return true; +} + +bool RunOne( + aclrtFuncHandle function, aclrtStream stream, void *state_device, uint64_t pmu_register_bases, + icache_scalar_pmu::Mode mode, uint32_t repeat, uint64_t seed, Sample *sample +) +{ + icache_scalar_pmu::ProbeState state{}; + state.control.pmu_register_bases = pmu_register_bases; + state.control.mode = static_cast(mode); + state.control.seed = seed; + if (!CheckAcl( + aclrtMemcpy(state_device, sizeof(state), &state, sizeof(state), ACL_MEMCPY_HOST_TO_DEVICE), + "aclrtMemcpy(H2D probe state)" + )) { + return false; + } + + struct KernelArgs { + uint64_t state_pointer; + } args{reinterpret_cast(state_device)}; + static_assert(sizeof(KernelArgs) == sizeof(uint64_t), "unexpected CCEC kernel argument ABI"); + if (!CheckAcl( + aclrtLaunchKernelWithHostArgs(function, 1, stream, nullptr, &args, sizeof(args), nullptr, 0), + "aclrtLaunchKernelWithHostArgs" + ) || + !CheckAcl(aclrtSynchronizeStream(stream), "aclrtSynchronizeStream(AIV probe)") || + !CheckAcl( + aclrtMemcpy(&state, sizeof(state), state_device, sizeof(state), ACL_MEMCPY_DEVICE_TO_HOST), + "aclrtMemcpy(D2H probe state)" + )) { + return false; + } + + sample->result = state.result; + std::string reason; + const bool semantic_ok = ValidateSample(*sample, mode, seed, &reason); + std::printf( + "[RAW] repeat=%u mode=%s seed=0x%llx sys_cycles=%llu total=%llu scalar=%llu " + "icache_req=%llu icache_miss=%llu target=0x%llx preparation=0x%llx " + "physical=%llu ctrl=0x%llx status=%s%s%s\n", + repeat, ModeName(mode), static_cast(seed), + static_cast(sample->result.sys_cycles), + static_cast(sample->result.pmu_total_cycles), + static_cast(sample->result.pmu_scalar_busy), + static_cast(sample->result.pmu_icache_request), + static_cast(sample->result.pmu_icache_miss), + static_cast(sample->result.target_checksum), + static_cast(sample->result.preparation_checksum), + static_cast(sample->result.physical_core_id), + static_cast(sample->result.pmu_ctrl_after_stop), semantic_ok ? "PASS" : "FAIL", + semantic_ok ? "" : " reason=", semantic_ok ? "" : reason.c_str() + ); + return semantic_ok; +} + +uint64_t Median(std::vector values) +{ + std::sort(values.begin(), values.end()); + const size_t middle = values.size() / 2; + if ((values.size() & 1U) != 0) return values[middle]; + return values[middle - 1] + (values[middle] - values[middle - 1]) / 2; +} + +double Median(std::vector values) +{ + std::sort(values.begin(), values.end()); + const size_t middle = values.size() / 2; + return (values.size() & 1U) != 0 ? values[middle] : (values[middle - 1] + values[middle]) / 2.0; +} + +using CounterMember = uint64_t icache_scalar_pmu::ProbeResult::*; + +double PairedDelta( + const std::vector &cold, const std::vector &warm, CounterMember member +) +{ + std::vector deltas; + deltas.reserve(cold.size()); + for (size_t index = 0; index < cold.size(); ++index) { + deltas.push_back( + static_cast(cold[index].result.*member) - + static_cast(warm[index].result.*member) + ); + } + return Median(std::move(deltas)); +} + +struct Metric { + const char *name; + CounterMember member; +}; + +constexpr Metric kMetrics[] = { + {"sys_cycles", &icache_scalar_pmu::ProbeResult::sys_cycles}, + {"total", &icache_scalar_pmu::ProbeResult::pmu_total_cycles}, + {"scalar", &icache_scalar_pmu::ProbeResult::pmu_scalar_busy}, + {"icache_req", &icache_scalar_pmu::ProbeResult::pmu_icache_request}, + {"icache_miss", &icache_scalar_pmu::ProbeResult::pmu_icache_miss}, +}; + +void PrintSummary(const std::array, 2> &samples) +{ + for (uint32_t mode_index = 0; mode_index < static_cast(icache_scalar_pmu::Mode::Count); + ++mode_index) { + const auto mode = static_cast(mode_index); + std::printf("[MEDIAN] mode=%s", ModeName(mode)); + for (const Metric &metric : kMetrics) { + std::vector values; + values.reserve(samples[mode_index].size()); + for (const Sample &sample : samples[mode_index]) values.push_back(sample.result.*(metric.member)); + std::printf(" %s=%llu", metric.name, static_cast(Median(std::move(values)))); + } + std::printf("\n"); + } + + const auto &warm = samples[static_cast(icache_scalar_pmu::Mode::WarmTarget)]; + const auto &cold = samples[static_cast(icache_scalar_pmu::Mode::ColdTarget)]; + for (const Metric &metric : kMetrics) { + std::printf( + "[PAIRED_DELTA] metric=%s cold_minus_warm=%.6f\n", metric.name, + PairedDelta(cold, warm, metric.member) + ); + } + + const double sys_delta = PairedDelta(cold, warm, &icache_scalar_pmu::ProbeResult::sys_cycles); + const double total_delta = PairedDelta(cold, warm, &icache_scalar_pmu::ProbeResult::pmu_total_cycles); + const double scalar_delta = PairedDelta(cold, warm, &icache_scalar_pmu::ProbeResult::pmu_scalar_busy); + const double request_delta = PairedDelta(cold, warm, &icache_scalar_pmu::ProbeResult::pmu_icache_request); + const double miss_delta = PairedDelta(cold, warm, &icache_scalar_pmu::ProbeResult::pmu_icache_miss); + const double scalar_share = total_delta == 0.0 ? 0.0 : scalar_delta / total_delta; + const double scalar_gap = total_delta - scalar_delta; + const double total_per_miss = miss_delta == 0.0 ? 0.0 : total_delta / miss_delta; + const double scalar_per_miss = miss_delta == 0.0 ? 0.0 : scalar_delta / miss_delta; + const double gap_per_miss = miss_delta == 0.0 ? 0.0 : scalar_gap / miss_delta; + std::printf( + "[ICACHE_CLASSIFICATION] completion_delta_sys_cycles=%.6f pmu_total_delta_cycles=%.6f " + "scalar_busy_delta_cycles=%.6f request_delta=%.6f miss_delta=%.6f " + "scalar_share_of_total_delta=%.9f scalar_gap_cycles=%.6f " + "total_cycles_per_extra_miss=%.6f scalar_cycles_per_extra_miss=%.6f " + "gap_cycles_per_extra_miss=%.6f\n", + sys_delta, total_delta, scalar_delta, request_delta, miss_delta, scalar_share, scalar_gap, + total_per_miss, scalar_per_miss, gap_per_miss + ); +} + +} // namespace + +int main(int argc, char **argv) +{ + const std::string kernel_path = argc > 1 ? argv[1] : "./icache_scalar_pmu_kernel.o"; + if (argc > 2) { + std::fprintf(stderr, "Usage: %s [icache_scalar_pmu_kernel.o]\n", argv[0]); + return EXIT_FAILURE; + } + const uint32_t repeats = RepeatsFromEnv(); + bool options_ok = repeats != 0; + const uint64_t base_seed = SeedFromEnv(&options_ok); + if (!options_ok) return EXIT_FAILURE; + + const int32_t device = atomic_probe::DeviceId(); + if (device < 0) return EXIT_FAILURE; + const std::vector kernel_data = ReadBinary(kernel_path); + if (kernel_data.empty()) { + std::fprintf(stderr, "Cannot read kernel binary: %s\n", kernel_path.c_str()); + return EXIT_FAILURE; + } + + if (!CheckAcl(aclInit(nullptr), "aclInit") || !CheckAcl(aclrtSetDevice(device), "aclrtSetDevice")) { + return EXIT_FAILURE; + } + aclrtStream stream = nullptr; + if (!CheckAcl(aclrtCreateStream(&stream), "aclrtCreateStream")) return EXIT_FAILURE; + + aclrtBinHandle binary_handle = nullptr; + if (!CheckAcl( + atomic_probe::LoadAicoreBinaryFromData(kernel_data.data(), kernel_data.size(), &binary_handle), + "LoadAicoreBinaryFromData" + )) { + return EXIT_FAILURE; + } + aclrtFuncHandle function = nullptr; + if (!CheckAcl(aclrtBinaryGetFunctionByEntry(binary_handle, 0, &function), "aclrtBinaryGetFunctionByEntry")) { + return EXIT_FAILURE; + } + + void *state_device = nullptr; + if (!CheckAcl( + aclrtMalloc(&state_device, sizeof(icache_scalar_pmu::ProbeState), ACL_MEM_MALLOC_NORMAL_ONLY), + "aclrtMalloc(probe state)" + )) { + return EXIT_FAILURE; + } + + atomic_probe::pmu::PmuSession pmu_session; + if (!pmu_session.Initialize( + static_cast(device), stream, kernel_path, "libicache_scalar_pmu_aicpu.so" + ) || + !pmu_session.Configure()) { + (void)pmu_session.Finalize(); + return EXIT_FAILURE; + } + + std::printf( + "=== Single-AIV paired WARM/COLD I-cache scalar-busy PMU probe ===\n" + "device=%d pairs=%u base_seed=0x%llx " + "events=total,scalar_busy(0x1),icache_req(0x34),icache_miss(0x35)\n", + device, repeats, static_cast(base_seed) + ); + + std::array, 2> samples; + for (auto &mode_samples : samples) mode_samples.reserve(repeats); + bool all_passed = true; + for (uint32_t repeat = 1; repeat <= repeats; ++repeat) { + const uint64_t seed = base_seed + static_cast(repeat - 1) * kRepeatSeedStride; + const bool warm_first = (repeat & 1U) != 0; + const std::array order = warm_first + ? std::array{ + icache_scalar_pmu::Mode::WarmTarget, icache_scalar_pmu::Mode::ColdTarget} + : std::array{ + icache_scalar_pmu::Mode::ColdTarget, icache_scalar_pmu::Mode::WarmTarget}; + + std::array pair; + bool pair_samples_ok = true; + for (const icache_scalar_pmu::Mode mode : order) { + const uint32_t mode_index = static_cast(mode); + pair_samples_ok &= RunOne( + function, stream, state_device, pmu_session.RegisterBasesDeviceAddress(), mode, repeat, seed, + &pair[mode_index] + ); + if (!pair_samples_ok) break; + } + if (!pair_samples_ok) { + all_passed = false; + break; + } + + const Sample &warm = pair[static_cast(icache_scalar_pmu::Mode::WarmTarget)]; + const Sample &cold = pair[static_cast(icache_scalar_pmu::Mode::ColdTarget)]; + const bool same_physical_core = warm.result.physical_core_id == cold.result.physical_core_id; + const bool cold_has_more_misses = cold.result.pmu_icache_miss > warm.result.pmu_icache_miss; + std::printf( + "[PAIR] repeat=%u order=%s physical_warm=%llu physical_cold=%llu " + "miss_warm=%llu miss_cold=%llu same_physical=%s cold_gt_warm_miss=%s status=%s\n", + repeat, warm_first ? "WARM,COLD" : "COLD,WARM", + static_cast(warm.result.physical_core_id), + static_cast(cold.result.physical_core_id), + static_cast(warm.result.pmu_icache_miss), + static_cast(cold.result.pmu_icache_miss), same_physical_core ? "PASS" : "FAIL", + cold_has_more_misses ? "PASS" : "FAIL", + same_physical_core && cold_has_more_misses ? "PASS" : "FAIL" + ); + if (!same_physical_core || !cold_has_more_misses) { + all_passed = false; + break; + } + samples[static_cast(icache_scalar_pmu::Mode::WarmTarget)].push_back(warm); + samples[static_cast(icache_scalar_pmu::Mode::ColdTarget)].push_back(cold); + } + + if (all_passed) PrintSummary(samples); + + bool cleanup_ok = pmu_session.Finalize(); + cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(probe state)"); + cleanup_ok &= CheckAcl(aclrtBinaryUnLoad(binary_handle), "aclrtBinaryUnLoad"); + cleanup_ok &= CheckAcl(aclrtDestroyStream(stream), "aclrtDestroyStream"); + cleanup_ok &= CheckAcl(aclrtResetDevice(device), "aclrtResetDevice"); + cleanup_ok &= CheckAcl(aclFinalize(), "aclFinalize"); + std::printf( + "[SUMMARY] completed_pairs=%zu requested_pairs=%u semantic_status=%s " + "pmu_restore_and_cleanup=%s\n", + samples[0].size(), repeats, all_passed ? "PASS" : "FAIL", cleanup_ok ? "PASS" : "FAIL" + ); + return all_passed && cleanup_ok ? EXIT_SUCCESS : EXIT_FAILURE; +} diff --git a/tests/atomic_probe/ccec/icache_scalar_pmu_shared.h b/tests/atomic_probe/ccec/icache_scalar_pmu_shared.h new file mode 100644 index 0000000000..43e8d98243 --- /dev/null +++ b/tests/atomic_probe/ccec/icache_scalar_pmu_shared.h @@ -0,0 +1,112 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef TESTS_ATOMIC_PROBE_CCEC_ICACHE_SCALAR_PMU_SHARED_H_ +#define TESTS_ATOMIC_PROBE_CCEC_ICACHE_SCALAR_PMU_SHARED_H_ + +#include +#include + +namespace icache_scalar_pmu { + +#if defined(__CCE_AICORE__) +#define ICACHE_SCALAR_PMU_SHARED_FN __aicore__ +#else +#define ICACHE_SCALAR_PMU_SHARED_FN +#endif + +// 两条路径的 PMU 窗口内都只调用一次、且调用同一个 noinline target: +// WARM 在窗外先调用一次 target;COLD 在窗外执行大于 16 KiB I-cache 容量的 evictor。 +// 模式分支本身不进入 PMU gate,因而不会改变窗内 target 的动态指令序列。 +enum class Mode : uint32_t { + WarmTarget = 0, + ColdTarget = 1, + Count = 2, +}; + +// A5 scalar I-cache 容量为 16 KiB。AICore 标量指令为 4B;target 用 2048 条 +// volatile NOP 形成约 8 KiB 指令体,evictor 用 8192 条形成约 32 KiB 顺序指令流。 +// 构建脚本仍应从最终 ELF 的符号大小复核这两个下界,不能只相信源码常量。 +constexpr uint32_t kTargetNopCount = 2048; +constexpr uint32_t kEvictorNopCount = 8192; +constexpr uint64_t kTargetXor = 0xd6e8feb86659fd93ULL; +constexpr uint64_t kTargetMultiplier = 0x9e3779b185ebca87ULL; +constexpr uint64_t kTargetAddend = 0xa0761d6478bd642fULL; +constexpr uint64_t kEvictorXor = 0xe7037ed1a0b428dbULL; +constexpr uint64_t kEvictorMultiplier = 0x8ebc6af09c88c6e3ULL; + +// Host 与 device 共用完全相同的无符号递推;uint64_t 溢出按模 2^64 定义。 +// checksum 只负责证明 target/evictor 的调用确实发生,不参与 PMU 窗口分类。 +ICACHE_SCALAR_PMU_SHARED_FN constexpr uint64_t TargetOracle(uint64_t seed) +{ + uint64_t value = seed ^ kTargetXor; + value ^= value >> 29; + value *= kTargetMultiplier; + value += kTargetAddend; + value ^= value >> 31; + return value; +} + +ICACHE_SCALAR_PMU_SHARED_FN constexpr uint64_t EvictorOracle(uint64_t seed) +{ + uint64_t value = seed ^ kEvictorXor; + value ^= value >> 23; + value *= kEvictorMultiplier; + value ^= value >> 27; + return value; +} + +// Host launch 前只写本 cache line;kernel 在 PMU 窗口外 DCCI 后读取。 +// pmu_register_bases 按 get_coreid() 的低 12 bit 索引,两个 scalar PMU 探针 +// 共用同一份 108 physical sub-core MMIO base 表协议。 +struct alignas(64) ProbeControl { + uint64_t pmu_register_bases; + uint32_t mode; + uint32_t reserved0; + uint64_t seed; + uint64_t reserved[5]; +}; + +// 前五项是同一 target 单次调用窗口内的时间和 PMU 原始计数。 +// target_checksum 在两种 mode 下都必须等于 TargetOracle(seed):这是窗内动态 +// 工作负载一致的功能 oracle。preparation_checksum 在 WARM 下也等于 TargetOracle, +// 在 COLD 下等于 EvictorOracle,用来证明对应的窗外准备路径没有被编译器删除。 +// 两条 cache line 均由唯一 AIV 用 st_dev 发布,kernel 末尾统一 DSB。 +struct alignas(64) ProbeResult { + uint64_t sys_cycles; + uint64_t pmu_total_cycles; + uint64_t pmu_scalar_busy; + uint64_t pmu_icache_request; + uint64_t pmu_icache_miss; + uint64_t target_checksum; + uint64_t preparation_checksum; + uint64_t pmu_ctrl_after_stop; + + uint64_t physical_core_id; + uint64_t mode_echo; + uint64_t reserved[6]; +}; + +struct alignas(64) ProbeState { + ProbeControl control; + ProbeResult result; +}; + +static_assert(sizeof(ProbeControl) == 64, "probe control must occupy one cache line"); +static_assert(sizeof(ProbeResult) == 128, "probe result must occupy two cache lines"); +static_assert(offsetof(ProbeState, result) == 64, "probe result must start on its own cache line"); +static_assert(sizeof(ProbeState) == 192, "probe state ABI changed unexpectedly"); + +#undef ICACHE_SCALAR_PMU_SHARED_FN + +} // namespace icache_scalar_pmu + +#endif // TESTS_ATOMIC_PROBE_CCEC_ICACHE_SCALAR_PMU_SHARED_H_ diff --git a/tests/atomic_probe/ccec/pmu_probe_aicpu.cpp b/tests/atomic_probe/ccec/pmu_probe_aicpu.cpp new file mode 100644 index 0000000000..1d36c00384 --- /dev/null +++ b/tests/atomic_probe/ccec/pmu_probe_aicpu.cpp @@ -0,0 +1,179 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#include "pmu_probe_control.h" + +#include "aicpu/platform_regs.h" +#include "common/kernel_args.h" + +#include +#include + +namespace { + +using atomic_probe::pmu::PmuControl; + +void FlushControl(const PmuControl *control) +{ + const uintptr_t begin = reinterpret_cast(control); + const uintptr_t end = begin + sizeof(*control); + for (uintptr_t address = begin; address < end; address += 64) { + __asm__ volatile("dc cvac, %0" : : "r"(address) : "memory"); + } + __asm__ volatile("dsb sy" ::: "memory"); + __asm__ volatile("isb" ::: "memory"); +} + +bool SavedConfigurationMatches(const PmuControl *control, uint64_t base, uint32_t index) +{ + return read_reg(base, RegId::PMU_CTRL_0) == control->saved_ctrl0[index] && + read_reg(base, RegId::PMU_CTRL_1) == control->saved_ctrl1[index] && + read_reg(base, RegId::PMU_CNT0_IDX) == control->saved_selector0[index] && + read_reg(base, RegId::PMU_CNT1_IDX) == control->saved_selector1[index] && + read_reg(base, RegId::PMU_CNT2_IDX) == control->saved_selector2[index] && + read_reg(base, RegId::PMU_START_CYC0) == control->saved_start_cycle_low[index] && + read_reg(base, RegId::PMU_START_CYC1) == control->saved_start_cycle_high[index] && + read_reg(base, RegId::PMU_STOP_CYC0) == control->saved_stop_cycle_low[index] && + read_reg(base, RegId::PMU_STOP_CYC1) == control->saved_stop_cycle_high[index]; +} + +bool ProbeConfigurationMatches(uint64_t base) +{ + return read_reg(base, RegId::PMU_CTRL_0) == REG_MMIO_PMU_CTRL_0_ENABLE_VAL && + read_reg(base, RegId::PMU_CTRL_1) == REG_MMIO_PMU_CTRL_1_ENABLE_VAL && + read_reg(base, RegId::PMU_CNT0_IDX) == 0x1U && read_reg(base, RegId::PMU_CNT1_IDX) == 0x34U && + read_reg(base, RegId::PMU_CNT2_IDX) == 0x35U && read_reg(base, RegId::PMU_START_CYC0) == 0 && + read_reg(base, RegId::PMU_START_CYC1) == 0 && read_reg(base, RegId::PMU_STOP_CYC0) == 0xffffffffU && + read_reg(base, RegId::PMU_STOP_CYC1) == 0xffffffffU; +} + +bool RestoreOne(PmuControl *control, uint64_t base, uint32_t index) +{ + write_reg(base, RegId::PMU_CTRL_0, 0); + write_reg(base, RegId::PMU_CTRL_1, 0); + write_reg(base, RegId::PMU_CNT0_IDX, control->saved_selector0[index]); + write_reg(base, RegId::PMU_CNT1_IDX, control->saved_selector1[index]); + write_reg(base, RegId::PMU_CNT2_IDX, control->saved_selector2[index]); + write_reg(base, RegId::PMU_START_CYC0, control->saved_start_cycle_low[index]); + write_reg(base, RegId::PMU_START_CYC1, control->saved_start_cycle_high[index]); + write_reg(base, RegId::PMU_STOP_CYC0, control->saved_stop_cycle_low[index]); + write_reg(base, RegId::PMU_STOP_CYC1, control->saved_stop_cycle_high[index]); + write_reg(base, RegId::PMU_CTRL_0, control->saved_ctrl0[index]); + write_reg(base, RegId::PMU_CTRL_1, control->saved_ctrl1[index]); + return SavedConfigurationMatches(control, base, index); +} + +int Configure(PmuControl *control, const uint64_t *register_bases) +{ + if (control->configured != 0) return -10; + control->processed_subcores = 0; + for (uint32_t index = 0; index < atomic_probe::pmu::kPmuPhysicalSubcores; ++index) { + const uint64_t base = register_bases[index]; + if (base == 0) { + while (control->processed_subcores != 0) { + --control->processed_subcores; + const uint32_t rollback = control->processed_subcores; + (void)RestoreOne(control, register_bases[rollback], rollback); + } + return -11; + } + + control->saved_ctrl0[index] = static_cast(read_reg(base, RegId::PMU_CTRL_0)); + control->saved_ctrl1[index] = static_cast(read_reg(base, RegId::PMU_CTRL_1)); + control->saved_selector0[index] = static_cast(read_reg(base, RegId::PMU_CNT0_IDX)); + control->saved_selector1[index] = static_cast(read_reg(base, RegId::PMU_CNT1_IDX)); + control->saved_selector2[index] = static_cast(read_reg(base, RegId::PMU_CNT2_IDX)); + control->saved_start_cycle_low[index] = static_cast(read_reg(base, RegId::PMU_START_CYC0)); + control->saved_start_cycle_high[index] = static_cast(read_reg(base, RegId::PMU_START_CYC1)); + control->saved_stop_cycle_low[index] = static_cast(read_reg(base, RegId::PMU_STOP_CYC0)); + control->saved_stop_cycle_high[index] = static_cast(read_reg(base, RegId::PMU_STOP_CYC1)); + + // 先冻结框架,再配置 Custom 三事件及完整计数周期;最后启用 + // GLB_PMU_EN | USER_PMU_MODE_EN | SAMPLE_PMU_MODE_EN。 + write_reg(base, RegId::PMU_CTRL_0, 0); + write_reg(base, RegId::PMU_CTRL_1, 0); + write_reg(base, RegId::PMU_CNT0_IDX, 0x1U); + write_reg(base, RegId::PMU_CNT1_IDX, 0x34U); + write_reg(base, RegId::PMU_CNT2_IDX, 0x35U); + for (int counter = 0; counter < 10; ++counter) { + (void)read_reg(base, reg_index(RegId::PMU_CNT0, counter)); + } + (void)read_reg(base, RegId::PMU_CNT_TOTAL0); + (void)read_reg(base, RegId::PMU_CNT_TOTAL1); + write_reg(base, RegId::PMU_START_CYC0, 0); + write_reg(base, RegId::PMU_START_CYC1, 0); + write_reg(base, RegId::PMU_STOP_CYC0, 0xffffffffU); + write_reg(base, RegId::PMU_STOP_CYC1, 0xffffffffU); + write_reg(base, RegId::PMU_CTRL_0, REG_MMIO_PMU_CTRL_0_ENABLE_VAL); + write_reg(base, RegId::PMU_CTRL_1, REG_MMIO_PMU_CTRL_1_ENABLE_VAL); + if (!ProbeConfigurationMatches(base)) { + (void)RestoreOne(control, base, index); + while (control->processed_subcores != 0) { + --control->processed_subcores; + const uint32_t rollback = control->processed_subcores; + (void)RestoreOne(control, register_bases[rollback], rollback); + } + return -12; + } + control->processed_subcores = index + 1; + } + control->configured = 1; + return 0; +} + +int Restore(PmuControl *control, const uint64_t *register_bases) +{ + if (control->configured == 0 || control->processed_subcores != atomic_probe::pmu::kPmuPhysicalSubcores) { + return -20; + } + bool all_restored = true; + while (control->processed_subcores != 0) { + --control->processed_subcores; + const uint32_t index = control->processed_subcores; + all_restored &= RestoreOne(control, register_bases[index], index); + } + control->configured = 0; + return all_restored ? 0 : -21; +} + +} // namespace + +extern "C" __attribute__((visibility("default"))) int simpler_aicpu_exec(void *argument) +{ + if (argument == nullptr) return -1; + auto *kernel_args = reinterpret_cast(argument); + auto *control = reinterpret_cast(kernel_args->runtime_args); + auto *register_bases = reinterpret_cast(kernel_args->regs); + if (control == nullptr || register_bases == nullptr) return -2; + + // command 位于每次 launch 都由 CANN 重新复制的 inline KernelArgs 中; + // PmuControl 初始化后只由 AICPU 写,因此这里不依赖 EL0 cache invalidate。 + control->command = kernel_args->enable_profiling_flag; + control->status = atomic_probe::pmu::kPmuStatusPending; + if (control->magic != atomic_probe::pmu::kPmuControlMagic || + control->version != atomic_probe::pmu::kPmuControlVersion || + control->expected_subcores != atomic_probe::pmu::kPmuPhysicalSubcores) { + control->status = -3; + FlushControl(control); + return -3; + } + + int status = -4; + const auto command = static_cast(kernel_args->enable_profiling_flag); + if (command == atomic_probe::pmu::PmuCommand::Configure) { + status = Configure(control, register_bases); + } else if (command == atomic_probe::pmu::PmuCommand::Restore) { + status = Restore(control, register_bases); + } + control->status = status; + FlushControl(control); + return status; +} diff --git a/tests/atomic_probe/ccec/pmu_probe_control.h b/tests/atomic_probe/ccec/pmu_probe_control.h new file mode 100644 index 0000000000..62f8406af0 --- /dev/null +++ b/tests/atomic_probe/ccec/pmu_probe_control.h @@ -0,0 +1,59 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef TESTS_ATOMIC_PROBE_CCEC_PMU_PROBE_CONTROL_H_ +#define TESTS_ATOMIC_PROBE_CCEC_PMU_PROBE_CONTROL_H_ + +#include +#include + +namespace atomic_probe::pmu { + +constexpr uint32_t kPmuControlMagic = 0x504d5551U; // "PMUQ" +constexpr uint32_t kPmuControlVersion = 1; +constexpr uint32_t kPmuPhysicalSubcores = 108; +constexpr int32_t kPmuStatusPending = 0x7fffffff; + +enum class PmuCommand : uint32_t { + Configure = 1, + Restore = 2, +}; + +// Host 与单线程 AICPU helper 共享的 PMU 所有权记录。helper 在 Configure +// 阶段保存被改寄存器,在 Restore 阶段先关 PMU、恢复 selector/range,最后恢复 CTRL。 +// counter 本身是 read-to-clear,旧计数内容无法恢复,因此 PMU session 必须独占。 +struct alignas(64) PmuControl { + uint32_t magic; + uint32_t version; + uint32_t command; + volatile int32_t status; + uint32_t configured; + uint32_t processed_subcores; + uint32_t expected_subcores; + uint32_t reserved[9]; + + uint32_t saved_ctrl0[kPmuPhysicalSubcores]; + uint32_t saved_ctrl1[kPmuPhysicalSubcores]; + uint32_t saved_selector0[kPmuPhysicalSubcores]; + uint32_t saved_selector1[kPmuPhysicalSubcores]; + uint32_t saved_selector2[kPmuPhysicalSubcores]; + uint32_t saved_start_cycle_low[kPmuPhysicalSubcores]; + uint32_t saved_start_cycle_high[kPmuPhysicalSubcores]; + uint32_t saved_stop_cycle_low[kPmuPhysicalSubcores]; + uint32_t saved_stop_cycle_high[kPmuPhysicalSubcores]; +}; + +static_assert(offsetof(PmuControl, saved_ctrl0) == 64, "PMU control header must occupy one cache line"); +static_assert(sizeof(PmuControl) % 64 == 0, "PMU control must use complete cache lines"); + +} // namespace atomic_probe::pmu + +#endif // TESTS_ATOMIC_PROBE_CCEC_PMU_PROBE_CONTROL_H_ diff --git a/tests/atomic_probe/ccec/pmu_probe_host_support.h b/tests/atomic_probe/ccec/pmu_probe_host_support.h new file mode 100644 index 0000000000..6755edd82d --- /dev/null +++ b/tests/atomic_probe/ccec/pmu_probe_host_support.h @@ -0,0 +1,309 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef TESTS_ATOMIC_PROBE_CCEC_PMU_PROBE_HOST_SUPPORT_H_ +#define TESTS_ATOMIC_PROBE_CCEC_PMU_PROBE_HOST_SUPPORT_H_ + +#include "pmu_probe_control.h" + +#include "acl/acl.h" +#include "aicpu_loader/host/load_aicpu_op.h" +#include "common/kernel_args.h" +#include "driver/ascend_hal.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +namespace atomic_probe::pmu { + +inline bool CheckAcl(aclError error, const char *label) +{ + if (error == ACL_SUCCESS) return true; + std::fprintf(stderr, "ACL error %d: %s\n", static_cast(error), label); + return false; +} + +inline std::vector ReadBinary(const std::string &path) +{ + std::ifstream file(path, std::ios::binary | std::ios::ate); + if (!file) return {}; + const std::streamsize size = file.tellg(); + if (size <= 0) return {}; + std::vector bytes(static_cast(size)); + file.seekg(0, std::ios::beg); + if (!file.read(bytes.data(), size)) return {}; + return bytes; +} + +inline std::string ArtifactBesideKernel(const std::string &kernel_path, const char *name) +{ + const size_t slash = kernel_path.find_last_of('/'); + return slash == std::string::npos ? std::string(name) : kernel_path.substr(0, slash + 1) + name; +} + +constexpr uint32_t kPhysicalAicoreCount = 36; +constexpr uint32_t kSubcoresPerAicore = 3; +constexpr uint32_t kPhysicalSubcoreCount = kPhysicalAicoreCount * kSubcoresPerAicore; +constexpr uint32_t kAicorePerDie = 18; +constexpr uint32_t kSubcoresPerDie = kAicorePerDie * kSubcoresPerAicore; +constexpr uint32_t kAivBaseInDie = kAicorePerDie; +constexpr uint64_t kSubcoreStride = 0x100000ULL; +constexpr uint32_t kAicoreMapBytes = 0x300000U; + +static_assert(kPhysicalSubcoreCount == kPmuPhysicalSubcores, "PMU table size mismatch"); + +// halResMap 只负责把 36 个物理 AICore 展开成 AIC/AIV 共 108 项 MMIO base。 +// 它不负责恢复 PMU selector/CTRL;恢复必须先由 PmuSession::Restore 完成。 +class RegisterMappings { +public: + using MapFn = drvError_t (*)(unsigned int, struct res_map_info *, unsigned long *, unsigned int *); + using UnmapFn = drvError_t (*)(unsigned int, struct res_map_info *); + + ~RegisterMappings() { Release(); } + + bool Initialize(uint32_t device) + { + device_ = device; + map_ = reinterpret_cast(dlsym(RTLD_DEFAULT, "halResMap")); + unmap_ = reinterpret_cast(dlsym(RTLD_DEFAULT, "halResUnmap")); + if (map_ == nullptr || unmap_ == nullptr) { + hal_handle_ = dlopen("libascend_hal.so", RTLD_NOW | RTLD_GLOBAL); + if (hal_handle_ != nullptr) { + map_ = reinterpret_cast(dlsym(hal_handle_, "halResMap")); + unmap_ = reinterpret_cast(dlsym(hal_handle_, "halResUnmap")); + } + } + if (map_ == nullptr || unmap_ == nullptr) { + std::fprintf(stderr, "Cannot resolve halResMap/halResUnmap.\n"); + return false; + } + + for (uint32_t physical = 0; physical < kPhysicalAicoreCount; ++physical) { + res_map_info &info = map_info_[physical]; + std::memset(&info, 0, sizeof(info)); + info.target_proc_type = PROCESS_CP1; + info.res_type = RES_AICORE; + info.res_id = physical; + unsigned long map_address = 0; + unsigned int map_length = kAicoreMapBytes; + const drvError_t error = map_(device_, &info, &map_address, &map_length); + if (error != 0 || map_address == 0 || map_length < kAicoreMapBytes) { + std::fprintf( + stderr, "halResMap failed: physical=%u error=%d address=0x%lx length=%u\n", physical, + static_cast(error), map_address, map_length + ); + Release(); + return false; + } + ++mapped_count_; + const uint64_t base = static_cast(map_address); + const uint32_t die = physical / kAicorePerDie; + const uint32_t local = physical % kAicorePerDie; + const uint32_t die_base = die * kSubcoresPerDie; + register_bases_[die_base + local] = base; + const uint32_t aiv0 = die_base + kAivBaseInDie + local * 2; + register_bases_[aiv0] = base + kSubcoreStride; + register_bases_[aiv0 + 1] = base + 2 * kSubcoreStride; + } + return true; + } + + void Release() + { + while (mapped_count_ != 0) { + --mapped_count_; + const drvError_t error = unmap_(device_, &map_info_[mapped_count_]); + if (error != 0) { + std::fprintf( + stderr, "halResUnmap failed: physical=%u error=%d\n", mapped_count_, + static_cast(error) + ); + } + } + if (hal_handle_ != nullptr) { + dlclose(hal_handle_); + hal_handle_ = nullptr; + } + } + + const std::array &RegisterBases() const { return register_bases_; } + +private: + uint32_t device_ = 0; + uint32_t mapped_count_ = 0; + void *hal_handle_ = nullptr; + MapFn map_ = nullptr; + UnmapFn unmap_ = nullptr; + std::array map_info_{}; + std::array register_bases_{}; +}; + +// 两个 scalar PMU probe 共用唯一配置会话:selector/range 只配置一次,所有 +// 单 AIV 样本完成后统一恢复。command 始终经 inline KernelArgs 传给 AICPU; +// Configure/Restore 之间绝不由 host 再 H2D 覆盖 PmuControl cache line。 +class PmuSession { +public: + bool Initialize( + uint32_t device, aclrtStream stream, const std::string &kernel_path, const char *helper_name + ) + { + stream_ = stream; + device_ = device; + if (!mappings_.Initialize(device)) return false; + + const size_t register_bytes = sizeof(mappings_.RegisterBases()); + if (!CheckAcl( + aclrtMalloc(®ister_bases_device_, register_bytes, ACL_MEM_MALLOC_NORMAL_ONLY), + "aclrtMalloc(PMU regs)" + ) || + !CheckAcl( + aclrtMemcpy( + register_bases_device_, register_bytes, mappings_.RegisterBases().data(), register_bytes, + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D PMU regs)" + ) || + !CheckAcl( + aclrtMalloc(&control_device_, sizeof(PmuControl), ACL_MEM_MALLOC_NORMAL_ONLY), + "aclrtMalloc(PMU control)" + )) { + return false; + } + + control_.magic = kPmuControlMagic; + control_.version = kPmuControlVersion; + control_.expected_subcores = kPmuPhysicalSubcores; + if (!CheckAcl( + aclrtMemcpy( + control_device_, sizeof(control_), &control_, sizeof(control_), ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D initial PMU control)" + )) { + return false; + } + + const std::string dispatcher_path = ArtifactBesideKernel(kernel_path, "libsimpler_aicpu_dispatcher.so"); + const std::string helper_path = ArtifactBesideKernel(kernel_path, helper_name); + const std::vector dispatcher_data = ReadBinary(dispatcher_path); + const std::vector helper_data = ReadBinary(helper_path); + if (dispatcher_data.empty() || helper_data.empty()) { + std::fprintf(stderr, "Cannot read PMU artifacts: %s %s\n", dispatcher_path.c_str(), helper_path.c_str()); + return false; + } + if (loader_.BootstrapDispatcher( + dispatcher_data.data(), dispatcher_data.size(), helper_data.data(), helper_data.size(), stream, + static_cast(device) + ) != 0 || + loader_.Init() != 0) { + std::fprintf(stderr, "Cannot initialize PMU AICPU helper.\n"); + return false; + } + kernel_args_.runtime_args = reinterpret_cast(control_device_); + kernel_args_.regs = reinterpret_cast(register_bases_device_); + kernel_args_.device_id = device; + initialized_ = true; + return true; + } + + bool Configure() + { + if (!initialized_ || configured_) return false; + configured_ = RunCommand(PmuCommand::Configure); + return configured_; + } + + bool Restore() + { + if (!configured_) return true; + if (!RunCommand(PmuCommand::Restore)) return false; + configured_ = false; + return true; + } + + bool Finalize() + { + bool ok = Restore(); + loader_.Finalize(); + if (control_device_ != nullptr) { + ok &= CheckAcl(aclrtFree(control_device_), "aclrtFree(PMU control)"); + control_device_ = nullptr; + } + if (register_bases_device_ != nullptr) { + ok &= CheckAcl(aclrtFree(register_bases_device_), "aclrtFree(PMU regs)"); + register_bases_device_ = nullptr; + } + mappings_.Release(); + initialized_ = false; + return ok; + } + + uint64_t RegisterBasesDeviceAddress() const + { + return reinterpret_cast(register_bases_device_); + } + +private: + bool RunCommand(PmuCommand command) + { + control_.command = static_cast(command); + control_.status = kPmuStatusPending; + kernel_args_.enable_profiling_flag = static_cast(command); + const int launch_error = loader_.LaunchBuiltInOp( + stream_, &kernel_args_, 1, host::KernelNames::RunName + ); + if (launch_error != 0) { + std::fprintf(stderr, "AICPU PMU helper launch failed: %d\n", launch_error); + return false; + } + if (!CheckAcl(aclrtSynchronizeStream(stream_), "aclrtSynchronizeStream(PMU helper)") || + !CheckAcl( + aclrtMemcpy( + &control_, sizeof(control_), control_device_, sizeof(control_), ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H PMU control)" + )) { + return false; + } + const bool expected_state = command == PmuCommand::Configure + ? control_.configured == 1 && control_.processed_subcores == kPmuPhysicalSubcores + : control_.configured == 0 && control_.processed_subcores == 0; + if (control_.status != 0 || !expected_state) { + std::fprintf( + stderr, "PMU helper failed: command=%u status=%d configured=%u processed=%u\n", + control_.command, static_cast(control_.status), control_.configured, + control_.processed_subcores + ); + return false; + } + return true; + } + + uint32_t device_ = 0; + aclrtStream stream_ = nullptr; + RegisterMappings mappings_; + host::LoadAicpuOp loader_; + void *register_bases_device_ = nullptr; + void *control_device_ = nullptr; + PmuControl control_{}; + KernelArgs kernel_args_{}; + bool initialized_ = false; + bool configured_ = false; +}; + +} // namespace atomic_probe::pmu + +#endif // TESTS_ATOMIC_PROBE_CCEC_PMU_PROBE_HOST_SUPPORT_H_ diff --git a/tests/atomic_probe/ccec/run_atomic_scalar_pmu.sh b/tests/atomic_probe/ccec/run_atomic_scalar_pmu.sh new file mode 100755 index 0000000000..3db203cf76 --- /dev/null +++ b/tests/atomic_probe/ccec/run_atomic_scalar_pmu.sh @@ -0,0 +1,147 @@ +#!/usr/bin/env bash +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +# 独立构建并运行单 AIV atomic/scalar PMU 对照;不修改 run_all.sh,也不下载 PTO_ISA。 +# 用法: +# ./run_atomic_scalar_pmu.sh # build + run +# ./run_atomic_scalar_pmu.sh build # 仅构建 +# ./run_atomic_scalar_pmu.sh run # 仅运行已有产物 +# 可选环境:ATOMIC_PROBE_DEVICE、ATOMIC_SCALAR_PMU_REPEATS、 +# ATOMIC_SCALAR_PMU_ROUNDS(逗号分隔)、ATOMIC_SCALAR_PMU_SEED。 +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_ROOT="$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel)" +BUILD_DIR="$SCRIPT_DIR/build/atomic_scalar_pmu" +ACTION="${1:-all}" +if [[ "$ACTION" != "all" && "$ACTION" != "build" && "$ACTION" != "run" ]]; then + echo "Usage: $0 [all|build|run]" >&2 + exit 1 +fi +if [[ $# -gt 1 ]]; then + echo "Usage: $0 [all|build|run]" >&2 + exit 1 +fi +if [[ -z "${ASCEND_HOME_PATH:-}" ]]; then + echo "ASCEND_HOME_PATH is not set; source the local CANN environment first." >&2 + exit 1 +fi + +CCEC="$ASCEND_HOME_PATH/bin/ccec" +LD="$ASCEND_HOME_PATH/bin/ld.lld" +HCC="$ASCEND_HOME_PATH/tools/hcc/bin/aarch64-target-linux-gnu-g++" +CXX_BIN="${CXX:-g++}" +READELF_BIN="${READELF:-readelf}" +# 优先复用调用者指定的本机 PTO_ISA;未指定时使用 CANN 安装包自带的 metadata header。 +PTO_INCLUDE_ROOT="${PTO_ISA_ROOT:-$ASCEND_HOME_PATH/x86_64-linux}" + +if [[ ! -x "$CCEC" || ! -x "$LD" || ! -x "$HCC" ]]; then + echo "CCEC, ld.lld, or AICPU cross compiler is missing under $ASCEND_HOME_PATH" >&2 + exit 1 +fi +if [[ ! -f "$PTO_INCLUDE_ROOT/include/pto/common/kernel_meta.hpp" ]]; then + echo "Local PTO metadata header is missing: $PTO_INCLUDE_ROOT/include/pto/common/kernel_meta.hpp" >&2 + exit 1 +fi +if ! command -v "$READELF_BIN" >/dev/null 2>&1; then + echo "readelf is required." >&2 + exit 1 +fi + +KERNEL="$BUILD_DIR/atomic_scalar_pmu_kernel.o" +HOST="$BUILD_DIR/atomic_scalar_pmu_host" + +build_probe() { + mkdir -p "$BUILD_DIR" + local common_flags=( + -c -O3 -g -x cce -Wall -std=c++17 + --cce-aicore-only + --cce-aicore-arch=dav-c310-vec + -mllvm -cce-aicore-stack-size=0x8000 + -mllvm -cce-aicore-function-stack-size=0x8000 + -mllvm -cce-aicore-record-overflow=false + -mllvm -cce-aicore-addr-transform + -mllvm -cce-aicore-dcci-insert-for-scalar=false + -mllvm -cce-aicore-dcci-before-kernel-end=false + -DCCEC_SYNC_AIV_ONLY + -I"$SCRIPT_DIR" + -I"$PTO_INCLUDE_ROOT/include" + ) + + echo "[BUILD] single-AIV CCEC kernel" + "$CCEC" "${common_flags[@]}" \ + -o "$BUILD_DIR/atomic_scalar_pmu_vec.o" \ + "$SCRIPT_DIR/atomic_scalar_pmu.cpp" + "$LD" -m aicorelinux -Ttext=0 -static \ + -o "$KERNEL" "$BUILD_DIR/atomic_scalar_pmu_vec.o" + + local symbols sections entry="atomic_scalar_pmu_0_mix_aiv" + symbols="$("$READELF_BIN" --symbols --wide "$KERNEL")" + sections="$("$READELF_BIN" --sections --wide "$KERNEL")" + if [[ "$symbols" != *" $entry"* || "$sections" != *".ascend.meta.$entry"* ]]; then + echo "Missing AIV entry or metadata for $entry" >&2 + exit 1 + fi + echo "[CHECK] AIV-only entry and metadata present" + + # 两个 scalar PMU 探针复用同一份 108-subcore 配置/恢复 helper。 + echo "[BUILD] AICPU PMU configure/restore helper" + "$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra -Werror \ + -Wl,--build-id \ + -I"$SCRIPT_DIR" \ + -I"$REPO_ROOT/src/a5/platform/include" \ + "$SCRIPT_DIR/pmu_probe_aicpu.cpp" \ + "$REPO_ROOT/src/a5/platform/onboard/aicpu/inner_platform_regs.cpp" \ + -o "$BUILD_DIR/libatomic_scalar_pmu_aicpu.so" + + echo "[BUILD] AICPU bootstrap dispatcher" + "$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra \ + -Wl,--build-id \ + -I"$REPO_ROOT/src/common" \ + "$REPO_ROOT/src/common/aicpu_loader/device/aicpu_dispatcher.cpp" \ + -ldl \ + -o "$BUILD_DIR/libsimpler_aicpu_dispatcher.so" + + echo "[BUILD] host runner" + "$CXX_BIN" -O2 -std=c++17 -Wall -Wextra -Werror -Wno-deprecated-declarations \ + -I"$REPO_ROOT/src/common" \ + -I"$REPO_ROOT/src/common/log/include" \ + -I"$REPO_ROOT/src/a5/platform/include" \ + -I"$ASCEND_HOME_PATH/include" \ + -I"$ASCEND_HOME_PATH/pkg_inc" \ + -I"$ASCEND_HOME_PATH/pkg_inc/runtime" \ + -I"$ASCEND_HOME_PATH/pkg_inc/runtime/runtime" \ + "$SCRIPT_DIR/atomic_scalar_pmu_host.cpp" \ + "$REPO_ROOT/src/common/aicpu_loader/host/load_aicpu_op.cpp" \ + "$REPO_ROOT/src/common/log/host_log.cpp" \ + "$REPO_ROOT/src/common/log/unified_log_host.cpp" \ + -L"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ + -Wl,-rpath,"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ + -lascendcl -lruntime -ldl -pthread \ + -o "$HOST" + echo "[BUILD] complete: $BUILD_DIR" +} + +run_probe() { + if [[ ! -x "$HOST" || ! -f "$KERNEL" || ! -f "$BUILD_DIR/libatomic_scalar_pmu_aicpu.so" || + ! -f "$BUILD_DIR/libsimpler_aicpu_dispatcher.so" ]]; then + echo "Build artifacts are incomplete; run '$0 build' first." >&2 + exit 1 + fi + echo "[RUN] device=${ATOMIC_PROBE_DEVICE:-${TASK_DEVICE:-0}}" + timeout "${ATOMIC_PROBE_TIMEOUT:-120}" "$HOST" "$KERNEL" +} + +if [[ "$ACTION" == "all" || "$ACTION" == "build" ]]; then + build_probe +fi +if [[ "$ACTION" == "all" || "$ACTION" == "run" ]]; then + run_probe +fi diff --git a/tests/atomic_probe/ccec/run_icache_scalar_pmu.sh b/tests/atomic_probe/ccec/run_icache_scalar_pmu.sh new file mode 100755 index 0000000000..14159538a7 --- /dev/null +++ b/tests/atomic_probe/ccec/run_icache_scalar_pmu.sh @@ -0,0 +1,219 @@ +#!/usr/bin/env bash +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +# 独立构建并运行单 AIV WARM/COLD I-cache PMU 配对;不修改 run_all.sh, +# 也不下载 PTO_ISA。最终 ELF 的符号尺寸、128B 对齐和区间不重叠是运行前硬条件。 +# 用法: +# ./run_icache_scalar_pmu.sh # build + run(默认 11 对) +# ./run_icache_scalar_pmu.sh build # 仅构建和检查 ELF +# ./run_icache_scalar_pmu.sh run # 仅检查已有 ELF 后运行 +# 可选环境:ATOMIC_PROBE_DEVICE、ICACHE_SCALAR_PMU_REPEATS(11..101)、 +# ICACHE_SCALAR_PMU_SEED、ICACHE_SCALAR_PMU_TIMEOUT。 +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_ROOT="$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel)" +BUILD_DIR="$SCRIPT_DIR/build/icache_scalar_pmu" +ACTION="${1:-all}" +if [[ "$ACTION" != "all" && "$ACTION" != "build" && "$ACTION" != "run" ]]; then + echo "Usage: $0 [all|build|run]" >&2 + exit 1 +fi +if [[ $# -gt 1 ]]; then + echo "Usage: $0 [all|build|run]" >&2 + exit 1 +fi +if [[ -z "${ASCEND_HOME_PATH:-}" ]]; then + echo "ASCEND_HOME_PATH is not set; source the local CANN environment first." >&2 + exit 1 +fi + +CCEC="$ASCEND_HOME_PATH/bin/ccec" +LD="$ASCEND_HOME_PATH/bin/ld.lld" +HCC="$ASCEND_HOME_PATH/tools/hcc/bin/aarch64-target-linux-gnu-g++" +CXX_BIN="${CXX:-g++}" +READELF_BIN="${READELF:-readelf}" +# 优先复用调用者指定的本机 PTO_ISA;未指定时使用 CANN 安装包自带的 metadata header。 +PTO_INCLUDE_ROOT="${PTO_ISA_ROOT:-$ASCEND_HOME_PATH/x86_64-linux}" + +if [[ ! -x "$CCEC" || ! -x "$LD" || ! -x "$HCC" ]]; then + echo "CCEC, ld.lld, or AICPU cross compiler is missing under $ASCEND_HOME_PATH" >&2 + exit 1 +fi +if [[ ! -f "$PTO_INCLUDE_ROOT/include/pto/common/kernel_meta.hpp" ]]; then + echo "Local PTO metadata header is missing: $PTO_INCLUDE_ROOT/include/pto/common/kernel_meta.hpp" >&2 + exit 1 +fi +if ! command -v "$CXX_BIN" >/dev/null 2>&1; then + echo "Host C++ compiler is missing: $CXX_BIN" >&2 + exit 1 +fi +if ! command -v "$READELF_BIN" >/dev/null 2>&1; then + echo "readelf is required: $READELF_BIN" >&2 + exit 1 +fi + +KERNEL="$BUILD_DIR/icache_scalar_pmu_kernel.o" +HOST="$BUILD_DIR/icache_scalar_pmu_host" + +symbol_address_hex="" +symbol_size="" +read_unique_function_symbol() { + local elf="$1" + local name="$2" + local -a matches=() + mapfile -t matches < <( + "$READELF_BIN" --symbols --wide "$elf" | + awk -v symbol="$name" '$4 == "FUNC" && $8 == symbol {print $2 " " $3}' + ) + if [[ ${#matches[@]} -ne 1 ]]; then + echo "Expected exactly one FUNC symbol named $name; found ${#matches[@]}" >&2 + exit 1 + fi + read -r symbol_address_hex symbol_size <<<"${matches[0]}" + if [[ ! "$symbol_address_hex" =~ ^[0-9a-fA-F]+$ || ! "$symbol_size" =~ ^[0-9]+$ ]]; then + echo "Cannot parse ELF symbol $name: ${matches[0]}" >&2 + exit 1 + fi +} + +check_kernel_elf() { + if [[ ! -f "$KERNEL" ]]; then + echo "Kernel ELF is missing: $KERNEL" >&2 + exit 1 + fi + + local symbols sections entry="icache_scalar_pmu_0_mix_aiv" + symbols="$("$READELF_BIN" --symbols --wide "$KERNEL")" + sections="$("$READELF_BIN" --sections --wide "$KERNEL")" + if [[ "$symbols" != *" $entry"* || "$sections" != *".ascend.meta.$entry"* ]]; then + echo "Missing AIV entry or metadata for $entry" >&2 + exit 1 + fi + + read_unique_function_symbol "$KERNEL" "icache_scalar_pmu_target" + local target_address_hex="$symbol_address_hex" + local target_size="$symbol_size" + read_unique_function_symbol "$KERNEL" "icache_scalar_pmu_evictor" + local evictor_address_hex="$symbol_address_hex" + local evictor_size="$symbol_size" + + local target_address=$((16#$target_address_hex)) + local evictor_address=$((16#$evictor_address_hex)) + local target_end=$((target_address + target_size)) + local evictor_end=$((evictor_address + evictor_size)) + if ((target_size < 8192)); then + echo "target is smaller than 8192B: $target_size" >&2 + exit 1 + fi + if ((evictor_size < 32768)); then + echo "evictor is smaller than 32768B: $evictor_size" >&2 + exit 1 + fi + if ((target_address % 128 != 0)); then + echo "target address is not 128B-aligned: 0x$target_address_hex" >&2 + exit 1 + fi + if ((evictor_address % 128 != 0)); then + echo "evictor address is not 128B-aligned: 0x$evictor_address_hex" >&2 + exit 1 + fi + if ! ((target_end <= evictor_address || evictor_end <= target_address)); then + printf 'target/evictor ranges overlap: target=[0x%x,0x%x) evictor=[0x%x,0x%x)\n' \ + "$target_address" "$target_end" "$evictor_address" "$evictor_end" >&2 + exit 1 + fi + printf '[CHECK] AIV entry/meta PASS; target address=0x%x size=%u; evictor address=0x%x size=%u; ' \ + "$target_address" "$target_size" "$evictor_address" "$evictor_size" + echo "alignment=128B ranges=non-overlap" +} + +build_probe() { + mkdir -p "$BUILD_DIR" + local common_flags=( + -c -O3 -g -x cce -Wall -std=c++17 + --cce-aicore-only + --cce-aicore-arch=dav-c310-vec + -mllvm -cce-aicore-stack-size=0x8000 + -mllvm -cce-aicore-function-stack-size=0x8000 + -mllvm -cce-aicore-record-overflow=false + -mllvm -cce-aicore-addr-transform + -mllvm -cce-aicore-dcci-insert-for-scalar=false + -mllvm -cce-aicore-dcci-before-kernel-end=false + -DCCEC_SYNC_AIV_ONLY + -I"$SCRIPT_DIR" + -I"$PTO_INCLUDE_ROOT/include" + ) + + echo "[BUILD] single-AIV CCEC WARM/COLD I-cache kernel" + "$CCEC" "${common_flags[@]}" \ + -o "$BUILD_DIR/icache_scalar_pmu_vec.o" \ + "$SCRIPT_DIR/icache_scalar_pmu.cpp" + "$LD" -m aicorelinux -Ttext=0 -static \ + -o "$KERNEL" "$BUILD_DIR/icache_scalar_pmu_vec.o" + check_kernel_elf + + # 与 atomic_scalar_pmu 复用同一份 108-subcore PMU 配置/恢复源码; + # 仅产物名独立,避免两个 probe 的 build 目录相互依赖。 + echo "[BUILD] AICPU PMU configure/restore helper" + "$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra -Werror \ + -Wl,--build-id \ + -I"$SCRIPT_DIR" \ + -I"$REPO_ROOT/src/a5/platform/include" \ + "$SCRIPT_DIR/pmu_probe_aicpu.cpp" \ + "$REPO_ROOT/src/a5/platform/onboard/aicpu/inner_platform_regs.cpp" \ + -o "$BUILD_DIR/libicache_scalar_pmu_aicpu.so" + + echo "[BUILD] AICPU bootstrap dispatcher" + "$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra \ + -Wl,--build-id \ + -I"$REPO_ROOT/src/common" \ + "$REPO_ROOT/src/common/aicpu_loader/device/aicpu_dispatcher.cpp" \ + -ldl \ + -o "$BUILD_DIR/libsimpler_aicpu_dispatcher.so" + + echo "[BUILD] host runner" + "$CXX_BIN" -O2 -std=c++17 -Wall -Wextra -Werror -Wno-deprecated-declarations \ + -I"$REPO_ROOT/src/common" \ + -I"$REPO_ROOT/src/common/log/include" \ + -I"$REPO_ROOT/src/a5/platform/include" \ + -I"$ASCEND_HOME_PATH/include" \ + -I"$ASCEND_HOME_PATH/pkg_inc" \ + -I"$ASCEND_HOME_PATH/pkg_inc/runtime" \ + -I"$ASCEND_HOME_PATH/pkg_inc/runtime/runtime" \ + "$SCRIPT_DIR/icache_scalar_pmu_host.cpp" \ + "$REPO_ROOT/src/common/aicpu_loader/host/load_aicpu_op.cpp" \ + "$REPO_ROOT/src/common/log/host_log.cpp" \ + "$REPO_ROOT/src/common/log/unified_log_host.cpp" \ + -L"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ + -Wl,-rpath,"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ + -lascendcl -lruntime -ldl -pthread \ + -o "$HOST" + echo "[BUILD] complete: $BUILD_DIR" +} + +run_probe() { + if [[ ! -x "$HOST" || ! -f "$KERNEL" || ! -f "$BUILD_DIR/libicache_scalar_pmu_aicpu.so" || + ! -f "$BUILD_DIR/libsimpler_aicpu_dispatcher.so" ]]; then + echo "Build artifacts are incomplete; run '$0 build' first." >&2 + exit 1 + fi + # run-only 也重新检查 ELF,不能绕开尺寸/对齐/区间 oracle。 + check_kernel_elf + echo "[RUN] device=${ATOMIC_PROBE_DEVICE:-${TASK_DEVICE:-0}} pairs=${ICACHE_SCALAR_PMU_REPEATS:-11}" + timeout "${ICACHE_SCALAR_PMU_TIMEOUT:-120}" "$HOST" "$KERNEL" +} + +if [[ "$ACTION" == "all" || "$ACTION" == "build" ]]; then + build_probe +fi +if [[ "$ACTION" == "all" || "$ACTION" == "run" ]]; then + run_probe +fi diff --git a/tests/atomic_probe/test_case.md b/tests/atomic_probe/test_case.md index 4a9f330374..c0fd9b758b 100644 --- a/tests/atomic_probe/test_case.md +++ b/tests/atomic_probe/test_case.md @@ -36,6 +36,13 @@ DCCI、`st_dev` 与 atomic 的 API 功能、隔离规则和代码评审清单见 - 2026-07-14:新增 CCEC `ld_dev_fanout_publish`。24 AIV 受控对照中,ordinary+DSB 为 `0/4416` 可见,st_dev+DSB 与 AtomicExch 均为 `4416/4416`;72 AIV 持续读压力同时破坏独立 control 对照,单列记录为高压力进展失败,不能外推为某个 data writer 的独立语义结论。 +- 2026-07-18:新增 CCEC 单 AIV `atomic_scalar_pmu`,以 EMPTY/SCALAR_CONTROL 扣除 + gate 和同构标量递推开销。三个独立会话的 8192 次 dependent `atomicAdd` 均显示: + atomic 额外 PMU total 几乎 100% 同步增加到 `scalar_instr_busy(0x1)`。 +- 2026-07-18:新增 CCEC 单 AIV `icache_scalar_pmu`,在同一静态调用点配对执行 + WARM/COLD 同一 target。三个独立会话共 33 对都是 `WARM miss=0`、 + `COLD miss=68`;COLD-WARM 只增加 48 scalar busy cycle,但增加 `2309..2312` + total cycle,证明本场景中 I-cache refill 等待的绝大多数周期不计入 scalar busy。 - 原始环境与定量结果记录在 `tests/ATOMIC_MINIBENCH_ONBOARD_LOG.md` 的 2026-07-11 与 2026-07-13 小节。 ## 权威覆盖矩阵 @@ -356,6 +363,89 @@ ATOMIC_PROBE_AIVS=24 ATOMIC_PROBE_FANOUT_LAUNCHES=3 \ tests/atomic_probe/ccec/run_all.sh ld_dev_fanout_publish ``` +## PMU 对 atomic 与 I-cache miss 等待周期的精确归类 + +`ccec/atomic_scalar_pmu.cpp` 与 `ccec/icache_scalar_pmu.cpp` 是两个独立单 AIV 微基准。 +它们共用 `pmu_probe_control.h` / `pmu_probe_aicpu.cpp` 的 108 physical sub-core +MMIO 表与 PMU 所有权协议;I-cache host 另用 `pmu_probe_host_support.h` 封装同一协议。 +AICPU helper 会保存并读回核验 +CTRL、slot 0/1/2 selector 和 START/STOP range,配置: + +| PMU 计数 | 事件 | 用途 | +|---|---:|---| +| slot 0 | `0x1` | `scalar_instr_busy` | +| slot 1 | `0x34` | `icache_req` | +| slot 2 | `0x35` | `icache_miss` | +| total | 固定总周期计数器 | PMU gate 内的总 AICore cycle | + +每个用例在待测段前后只执行 `metrics_prof_start/stop`,关窗后才用 +`ld_dev` 读取 total/scalar/request/miss,最后恢复进程进入用例前的 PMU 配置。 +它们不依赖 `msprof task-based` 的整任务 context 计数,也不与另一个 PMU session +并发执行。 + +### dependent atomic 等待计入 scalar busy + +Atomic 用例对每个 rounds 依次执行: + +1. `EMPTY`:只量 gate/read 固定开销; +2. `SCALAR_CONTROL`:使用 scalar 寄存器执行与 atomic 路径同构的 + `old/delta/checksum` 递推; +3. `DEPENDENT_ATOMIC_ADD`:`old` 改由 64-bit `atomicAdd` 返回,下一次 addend + 依赖上一次返回值,强制测量 atomic 完成延迟而不是无依赖吞吐。 + +Host 对 CONTROL/ATOMIC 复算完全相同的 checksum,并检查 atomic 终值、 +physical core id 和 `CTRL.bit0 == 0`。2026-07-18 在 A5 device 0 上的三个独立 +进程会话均执行 `8192 次 × 7 组`: + +| 会话 | `(ATOMIC-CONTROL)` 完成延迟 | PMU total cycle/op | scalar busy cycle/op | scalar / total | +|---:|---:|---:|---:|---:| +| 1 | 182.922729 ns | 301.804321 | 301.803955 | 0.999998787 | +| 2 | 251.687622 ns | 415.253540 | 415.253174 | 0.999999119 | +| 3 | 271.009888 ns | 447.110718 | 447.110352 | 0.999999181 | + +完成延迟在三个会话中处于不同档位,当前证据不足以归因;但计数归类完全一致: +**dependent `atomicAdd` 增加的 PMU total 周期几乎 100% 同步计入 +`scalar_instr_busy(0x1)`。** `get_sys_cnt` 是 1 GHz 时基,表中 PMU total/scalar +则是 AICore 核时钟 cycle,两者不能直接按同一单位比较。 + +### I-cache miss 回填等待的绝大多数周期不计入 scalar busy + +I-cache 用例的 WARM/COLD 两条路径在 PMU 开窗前汇合,窗口内只从同一个 +静态调用点执行一次同一个 target: + +- WARM 在窗外先调用一次 target; +- COLD 在窗外先执行超过 16 KiB AIV scalar I-cache 容量的 evictor; +- 最终 ELF 硬校验 target 为 `8280 B @ 0x0`、evictor 为 `32836 B @ 0x2080`, + 两者均按 128 B 对齐且区间不重叠; +- Host 逐样本复算 target/prepare checksum,并要求每对 WARM/COLD 使用同一 + physical AIV、`COLD miss > WARM miss`。 + +2026-07-18 在 A5 device 0 执行三个独立进程会话,每个会话 11 对, +交替使用 WARM,COLD 和 COLD,WARM 顺序。全部 33 对均为 +`WARM miss=0`、`COLD miss=68`,且 checksum、mode echo、PMU 关窗与恢复全部通过: + +| 会话 | WARM `total/scalar/req/miss` | COLD `total/scalar/req/miss` | total 增量 | scalar 增量 | miss 增量 | scalar / total 增量 | +|---:|---|---|---:|---:|---:|---:| +| 1 | `1068/1060/520/0` | `3377/1108/588/68` | 2309 | 48 | 68 | 2.078822% | +| 2 | `1068/1060/520/0` | `3379/1108/588/68` | 2311 | 48 | 68 | 2.077023% | +| 3 | `1068/1060/520/0` | `3380/1108/588/68` | 2312 | 48 | 68 | 2.076125% | + +因此准确结论是:**本场景中 I-cache miss 回填等待的绝大多数周期不计入 +scalar busy,但不是 scalar 增量严格为零。** 额外 68 次 miss 产生 `2309..2312` +total cycle,scalar busy 只增加 48 cycle,其余 `2261..2264` cycle 形成 +scalar-busy gap。target 实际覆盖 65 条 cache line,另 3 次 miss 与顺序预取相符; +因此 `total_delta / 68` 只能作为本窗口归一化值,不能称为单次阻塞 +I-cache miss 的精确延迟。 + +两个用例均只使用本机 CANN/PTO-ISA,不下载外部 PTO-ISA: + +```bash +source /home/q00473782/cann/cann-9.1.0/set_env.sh +cd tests/atomic_probe/ccec +./run_atomic_scalar_pmu.sh +./run_icache_scalar_pmu.sh +``` + ## 其余探针 | 文件 | 类型 | 验证内容 | @@ -371,6 +461,8 @@ ATOMIC_PROBE_AIVS=24 ATOMIC_PROBE_FANOUT_LAUNCHES=3 \ | `ascendc/st_dev_separate_line_stress.asc` / `ccec/st_dev_separate_line_stress.cpp` | regression gating | 只含分-line 数据路径;四模式覆盖两组活跃 block 与两种 allocation 内 line offset,100000 次精确终值检查 | | `ascendc/atomic_exch_same_line.asc` / `ccec/atomic_exch_same_line.cpp` | gating + control | 与 st_dev 同构的 AtomicExch 末值顺序对照;三组路径均精确通过 | | `ccec/ld_dev_fanout_publish.cpp` | regression gating + timing | 唯一 writer 以 ordinary+DSB、st_dev+DSB、AtomicExch 三种方式逐轮发布;其余全部 AIV 只用 ld_dev 读取完整序列,并记录 writer/全读者周期 | +| `ccec/atomic_scalar_pmu.cpp` | gating + PMU classification | 单 AIV dependent atomicAdd 完成延迟与同构 scalar control 对照;核实 atomic 等待是否计入 scalar busy | +| `ccec/icache_scalar_pmu.cpp` | gating + PMU classification | 单 AIV 同一 target 的 WARM/COLD I-cache 对照;核实 miss 回填等待是否计入 scalar busy | | `ascendc/dcci_atomic_stress.asc` | legacy observation | 旧的混合 stress;不再作为 DCCI selector 语义证据 | | `ccec/dcci_clean_clobber.cpp` | gating | 有序 dirty/clean line 的 dcci clobber 与 control | | `ascendc/mb2_flags_clobber.asc` | gating + observation | AtomicMax flags 无丢失;store+dcci 仅统计 | From 04ec9b955a54a6634a5dabfe70874816170d2fa6 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 00:50:13 +0000 Subject: [PATCH 006/214] =?UTF-8?q?perf(a5):=20=E8=B7=B3=E8=BF=87HeapGuard?= =?UTF-8?q?=E9=A6=96=E5=9C=88=E5=86=97=E4=BD=99=E5=8E=9F=E5=AD=90=E8=AF=BB?= =?UTF-8?q?=E5=8F=96?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../dist_engine/aicore/submit_runtime.h | 3 + ...05\345\206\265\345\210\206\346\236\220.md" | 310 ++++++++++++++++++ .../pa_scheduler/common/pa_scheduler_core.h | 8 + 3 files changed, 321 insertions(+) diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_runtime.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_runtime.h index 97f75eb968..56e46aae7b 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_runtime.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_runtime.h @@ -110,6 +110,9 @@ PTO_DEVICE_FUNC bool dist_submit_wait_heap_capacity(DistSubmitCtx &ctx, DistSubm bool waited = false; TRACE_SPAN_BEGIN(heap_bp_trace); while (!fatal_set()) { + // 逻辑 heap 尚未走完第一圈时,物理地址还没有发生环形复用;保留 fatal + // 原子检查后,可直接跳过 frontier/vend 原子读取。 + if (ctx.self->heap_next <= ring) return true; const int32_t f = static_cast(atomic_load(g_dist.frontier)); const int32_t R = f - g_dist.H; const uint64_t vstart_live = load_task_vend(R); diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 98d46bbbcf..ed752e26a9 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -269,6 +269,9 @@ Submit 调度而非 kernel 计算变快。 ## 5. 后续优化顺序与验证要求 +本节是首轮分析时给出的候选方向。实际推进采用“单变量、低风险到高风险”的 +阶段门禁,并持续记录在第 7 节;若两处顺序不同,以第 7 节已经完成验证的结论为准。 + 建议按以下顺序继续,且每次只改一个变量: 1. **Claim cursor 竞争。** 73728 次 `atomicMax` 是最大固定项。可以研究 @@ -338,3 +341,310 @@ Alloc/QK/SF/PV winner 调用,共 1024 次。当前代表性 CCEC 轮次的累 完整构建、参数、内存占用、冷热运行口径和脱仓复制方法见同目录 [PA 调度器独立复现与泳道使用指南](PA调度器独立复现与泳道使用指南.md)。后续调度优化应先在该 独立用例做协议回归和阶段定位,再回到真实 PA Case1 做最终性能确认。 + +## 7. Atomic 消减阶段日志 + +从 2026-07-17 起,每个候选必须按以下节奏更新本节:先写静态证明和直接消减 +口径,再记录 standalone 三后端结果、压力/反例结果和性能 A/B,最后写是否允许 +进入真实 PA。失败、超时和回退同样保留,不能只记录正向数据。一次只验证一个 +变量,后续阶段不得把前一阶段的间接调度变化冒充为本阶段的直接 atomic 收益。 + +| 阶段 | 单变量 | 当前状态 | +| ---- | ------ | -------- | +| H1 | HeapGuard 首圈 fast path | 正确性与真实性能完成,保留并本地提交 | +| F1 | fanin 依赖检查顺序 | H1 提交后开始 | +| 后续 | ready cache、退避、frontier、Claim | 尚未开始,必须逐项验证 | + +### 7.1 阶段 H1:HeapGuard 首圈 fast path + +#### 7.1.1 修改范围与正确性证明 + +本阶段只在 standalone 的 `HeapGuard()` 中增加以下判断;原 slow path 未改: + +```cpp +while (!IsFatal(state)) { + if (worker.heap_next <= ring) { + return true; + } + // 原 frontier/vend 容量检查。 +} +``` + +判断放在 `IsFatal()` 之后,因此仍保留每次 HeapGuard 的 fatal 原子检查。profile +版本在 fast path 返回前仍累计 HeapGuard 时间,没有改变统计调用次数。 + +该判断依赖以下由现有代码确认的不变量: + +1. 每个 worker 的 `heap_next` 从 0 开始,materialize 只按单调逻辑地址推进; +2. 只有生成物理输出地址时才对 ring 取模; +3. 每个输出按 1 KiB 对齐,单 task 输出超过 ring 会直接失败; +4. 当前 task 若跨越 ring 尾部,会先把 `task_base` 推到下一圈; +5. HeapGuard 在当前 task 完成 materialize 后执行。 + +所以在 `uint64_t` 未溢出的前提下,`heap_next <= ring` 表示所有已分配逻辑区间 +仍位于 `[0, ring)`,取模后一一对应,尚不可能覆盖旧输出。`heap_next == ring` +也安全,因为已分配区间右端为开区间;第一个真正进入第二圈的非零输出会使 +`heap_next > ring`。不能用 `output_bytes <= ring` 替代该条件,后者无法证明 +历史分配没有 wrap。 + +本阶段不顺手修改 slow path 中 `heap_next - vend` 的无符号下溢边界,也不修改 +frontier、fanin、Claim 或 completion 协议,避免把两个正确性问题混在一起。 + +#### 7.1.2 直接 atomic 消减口径 + +默认 256 batch 的 standalone 共调用 1024 次 HeapGuard。默认 256 MiB ring 下, +每个 worker 的最终状态为: + +```text +heap_next = 206,569,472 B +ring = 268,435,456 B +剩余 = 61,865,984 B +``` + +1024 次调用全部处于第一圈,直接消减如下: + +| 操作 | 修改前 | 修改后 | 直接消减 | +| ---- | -----: | -----: | -------: | +| fatal atomic load | 1024 | 1024 | 0 | +| frontier atomic load | 1024 | 0 | 固定 1024 | +| vend atomic load | `V` | 0 | 动态 `V` | + +只有 `retire = frontier - H >= 0` 时原逻辑才读取 vend。按 1280 task、`H=64` +和输出 task 分布,`0 <= V <= 972`。所以默认 workload 直接删除的是 +`1024 + V` 次 HeapGuard 原子读取,不是所有观测指标的变化量。 + +#### 7.1.3 默认配置正确性门禁 + +恢复默认 256 MiB 配置前,本阶段已经完成一次 CCEC、AscendC、CPU 的全量构建, +以及 256 batch、零 NOP、开启四阶段统计、关闭泳道大缓冲区的完整运行: + +```bash +./run.sh run all --device 0 --batches 256 --runs 1 \ + --nop-count 0 --profile-phases --no-swimlane +``` + +三个后端均为 `semantic_status=PASS`、`postprocess_status=PASS`。严格校验包括 +73,728 次 Claim、1280 个唯一 winner、1024 个 kernel、1024 次 HeapGuard、 +fanin、flag、vend、frontier、cursor、TensorMap/heap 最终状态和每 worker +前端操作次数。该结果只证明 standalone 协议,没有替代真实输出 golden。 + +#### 7.1.4 16 MiB wrap 压力与 CPU 对照 + +为覆盖首圈之外的原 slow path,曾临时把 `kHeapBytes` 从 256 MiB 改为 16 MiB +并全量重编。256 batch 的静态状态为: + +```text +ring = 16,777,216 B +最终逻辑 heap_next = 209,385,472 B +跨 ring 尾部跳转 = 12 次 +首圈 fast path = 82 次 HeapGuard +原 slow path = 942 次 HeapGuard +``` + +CCEC 和 AscendC 的 256 batch 完整语义与阶段统计均 PASS。设备结果确实进入慢路径: +CCEC 的 HeapGuard wait event 为 `629 + 3 = 632`,AscendC 为 +`542 + 6 = 548`,不是仍停留在首圈的伪压力测试。 + +CPU 结果需要单独解释:fast 版的 32、64、128 batch 均快速 PASS;256 batch +在开启阶段统计时持续运行超过 8 分钟未结束,关闭阶段统计后也未在观察窗口结束。 +为排除 fast path 回归,保持 16 MiB 和 256 batch 不变,只临时撤掉 fast path、 +重编 CPU 原版,并用统一的 120 秒门限运行;原版同样超时。由此只能得出: + +- 该 CPU synthetic-heap 模型在 16 MiB、多圈、256 batch 下存在原有的长程活性 + 或 host 线程调度问题; +- 当前证据不能把该超时归因于 H1,也不能把 CPU 256 batch 记为 PASS; +- 真实 tiny-ring 数据 golden 仍是生产迁移前不可省略的门禁。 + +对照完成后已恢复 `kHeapBytes = 256 MiB`,临时原版构建不保留为源码改动。 + +恢复后再次执行 `build all`,并用与 7.1.3 相同的 256 batch 命令完成终验; +CCEC、AscendC、CPU 再次全部 PASS。该轮 HeapGuard wait event 均为 0,符合默认 +配置全程不 wrap 的静态结论。CCEC/AscendC/CPU 的 Submit span 分别为 +3499.426 us、3523.687 us、229235.747 us;CPU 时间仅反映 host pthread 调度, +不参与 A5 性能比较。 + +#### 7.1.5 CCEC A5 standalone 十样本 A/B + +baseline 和 H1 均使用独立进程首轮,各保留 10 个具有完整 PASS 结果的有效样本; +单位为微秒: + +```text +baseline = [ + 4451.574, 4847.797, 5089.239, 5403.477, 5227.148, + 5217.446, 4929.629, 4084.562, 5268.516, 5246.626 +] + +H1 = [ + 3759.041, 4142.396, 4582.740, 4134.989, 5670.639, + 4348.529, 4503.739, 5507.320, 4286.019, 4088.046 +] +``` + +| 指标 | baseline(us) | H1(us) | 相对变化 | +| ---- | ------------: | -------: | -------: | +| 样本数 | 10 | 10 | - | +| 中位数 | 5153.3425 | 4317.2740 | -16.2238% | +| 均值 | 4976.6014 | 4502.3458 | -9.5297% | +| p90 | 5268.516 | 5507.320 | +4.5327% | +| 最小值 | 4084.562 | 3759.041 | - | +| 最大值 | 5403.477 | 5670.639 | - | + +九组可按采集顺序配对的样本中 H1 有 7 组更快,配对变化中位数为 `-14.55%`, +两组反向样本为 `+8.48%` 和 `+34.83%`。动态指标也随调度时序变化:fanin +load 中位数从 91,254 降至 66,318.5(-27.33%),但 p90 上升 11.47%; +RingBp placement 中位数从 118.5 降至 101(-14.77%),p90 下降 3.76%。 + +另有两次没有生成完整结果的 baseline 异常:首次外层 launch 约四分钟无结果后 +人工中断;后续 baseline `r9` 在 60 秒门限超时。二者均未计入耗时分布,也不能 +记为 PASS 或 FAIL;异常后设备 smoke 恢复正常。正式 A/B 必须让两边使用相同 +timeout,并把 timeout 率单独报告,避免只统计成功样本造成幸存者偏差。 + +fanin 和 RingBp 代码并未在 H1 修改,其变化幅度又远大于固定 1024 次 frontier +读取,因此只能解释为 worker 到达、依赖完成和 drain 相对时序改变后的间接效应。 +当前中心趋势显示方向性收益,但 H1 的 p90 变差且离散较大,不能声称尾延迟稳定 +改善,更不能把 16.22% 全部归因于 `1024 + V` 次直接 atomic 消减。 + +#### 7.1.6 阶段结论与下一门禁 + +H1 当前状态是“standalone 默认配置通过,允许进入真实 PA 正确性验证”,不是 +“真实 PA 已优化完成”。继续推进前按以下顺序执行: + +1. 默认 256 MiB 恢复后重建三后端并再次执行完整回归,确认临时压力常量无残留; +2. 对真实 `dist_submit_wait_heap_capacity()` 做一次单点、等价修改; +3. 运行 fully-distributed PA Case1 的 A5Sim golden; +4. 运行已有 68 KiB `AllocFillRunAhead`、`AllocHeapBackPressure` 和 MB6 heap + reclaim 用例,验证真实输出没有 premature reuse; +5. A5 正确性通过后,再做双方相同门限、独立进程、交错顺序的至少 10 轮 A/B; +6. H1 通过真实门禁后,下一项低风险候选是只调整 fanin 检查顺序;ready cache、 + backoff、frontier 合并和 Claim 参与者缩减必须分别作为后续单变量阶段。 + +standalone 使用 synthetic heap 且计算 kernel 由 NOP 模拟,它能验证控制协议、 +分支和 atomic 计数,但不能用 synthetic 地址证明真实 tensor 内容未被提前覆盖。 + +#### 7.1.7 真实 fully-distributed 迁移与正确性结果 + +standalone 门禁通过后,只在真实实现的 +`dist_submit_wait_heap_capacity()` 中加入同形判断: + +```cpp +while (!fatal_set()) { + if (ctx.self->heap_next <= ring) return true; + // 原 slow path 不变。 +} +``` + +生产实现没有 standalone 的 HeapGuard phase profile,因此没有搬入额外统计字段。 +判断仍位于 `TRACE_SPAN_BEGIN` 之后和 fatal 检查之内:原本不等待的成功路径就不会 +生成 RingBp 事件,新 fast path 的 trace 语义不变。真实函数共被 1280 个 winner +调用,UP 因 `output_bytes == 0` 在进入 atomic-bearing loop 前返回;实际消减口径 +仍是 Alloc/QK/SF/PV 的 1024 次 guard。 + +测试统一使用用户环境 `/home/q00473782/.venv`、CANN 9.1、本地 GCC 15,并按 +仓库 CI 固定 PTO-ISA 到: + +```text +ddafa8da9c760ecd13fe9fe2833d6ee55fb20bd8 +``` + +未传 SHA 时测试会把 managed clone 更新到当时的 `origin/main ea90d400`,该组合的 +CPU stub/`pto_instr.hpp` 出现大量重复定义,kernel 未能编译;这不是 H1 结果, +也没有通过修改 PTO-ISA 头文件规避。非交互 shell 还必须显式加入用户本地 +`g++-15` 路径,不能假设会自动 source `.bashrc`。 + +真实数据门禁结果如下: + +| 平台 | 用例 | 覆盖点 | 结果 | +| ---- | ---- | ------ | ---- | +| A5Sim | `AllocFillRunAhead67`,68 KiB | 首圈边界 | PASS | +| A5Sim | `AllocFillRunAhead128`,68 KiB | 跨圈 slow path | PASS | +| A5Sim | `AllocHeapBackPressure`,68 KiB | 等待、回收、真实数据 golden | PASS | +| A5Sim | PA Case1,真实 kernel | 256 batch 数值 golden | PASS | +| A5 | `AllocFillRunAhead67`,68 KiB | 首圈边界 | PASS | +| A5 | `AllocFillRunAhead128`,68 KiB | 跨圈 slow path | PASS | +| A5 | `AllocHeapBackPressure`,68 KiB | 等待、回收、真实数据 golden | PASS | +| A5 | PA Case1,真实 kernel | 256 batch 数值 golden | PASS | + +PA A5Sim 的真实 kernel orchestration 约为 38.824 s。另一次带 +`--use-example-exec-time` 的 A5Sim 调度运行也 PASS,但该选项会跳过数值 golden, +所以只作为控制流证据,不计入上表的正确性依据。 + +MB6 需要保留两个基线问题,不能写成 PASS: + +1. `Normal` 已完成底层数值运行,但 `DistRuntimeContractMixin` 随后因没有捕获到 + `[dist] DEPSIG` 而失败;当前 `src/`、`simpler_setup/` 和 runtime 构建源码中 + 查不到 `PTO_DIST_DEPSIG/DEPSIG` 实现,因此属于测试契约与当前 runtime 不匹配, + 不能靠加 shell 环境变量伪造 oracle。 +2. `Heavy` 的 8 MiB/H=64 A5Sim 压力在运行中触发 native abort。保持全部参数 + 不变、只撤销生产 H1 后,基线同样以相同调用路径 abort;所以该失败不是 H1 + 引入,但也不能作为 H1 的通过项。`FullCore36` 在 Heavy 基线已经失败后未继续跑。 + +综合现有证据,H1 已通过目标 PA 与真实 68 KiB 回卷/回收的 A5Sim+A5 golden, +可以进入真实 PA 性能 A/B。MB6 的两项既有问题作为未关闭风险保留,不能被其他 +PASS 掩盖,也不在 H1 中顺手修改测试框架或 heap slow path。 + +#### 7.1.8 真实 A5 PA Case1 十对性能 A/B + +真实性能使用提交前基线 `290dbda0` 的 detached 临时 worktree 和带 H1 的主 +worktree。两边均预先完成一次不计入统计的 warm run,随后按 +`baseline -> H1` 顺序采集 10 对独立 pytest 进程。统一条件为: + +- A5 device 0,真实 PA kernel,256 batch; +- 用户 `/home/q00473782/.venv`、CANN 9.1、GCC 15; +- PTO-ISA 固定为 `ddafa8da9c760ecd13fe9fe2833d6ee55fb20bd8`; +- 开启 L2 swimlane,不使用 `--use-example-exec-time` 或 `--skip-golden`; +- 每个进程统一 180 s timeout;双方均 10/10 PASS、0 timeout; +- 为避免自动生成数百 MiB merged JSON 影响采集周转,双方仅在测试进程内对称地 + 关闭 post-case converter;设备执行和原始 `l2_swimlane_records.json` 不变; +- 每个原始文件均有 122,880 个 Submit 事件,指标直接取最早 start 到最晚 end。 + +预热样本为 baseline 5.134950 ms、H1 5.116055 ms,只验证两边构建和设备状态, +不进入下面统计。10 对正式样本为: + +| 对次 | baseline(ms) | H1(ms) | 配对变化 | +| ---: | ------------: | -------: | -------: | +| 1 | 5.149955 | 5.301485 | +2.942% | +| 2 | 5.331087 | 5.110060 | -4.146% | +| 3 | 5.137861 | 5.098696 | -0.762% | +| 4 | 5.150200 | 5.119599 | -0.594% | +| 5 | 5.129350 | 5.112860 | -0.321% | +| 6 | 5.146475 | 5.129671 | -0.327% | +| 7 | 5.111929 | 5.105638 | -0.123% | +| 8 | 5.274224 | 5.229773 | -0.843% | +| 9 | 5.114477 | 5.137014 | +0.441% | +| 10 | 5.125086 | 5.125040 | -0.001% | + +| 统计 | baseline(ms) | H1(ms) | 相对变化 | +| ---- | ------------: | -------: | -------: | +| 中位数 | 5.142168 | 5.122320 | -0.386% | +| 均值 | 5.167064 | 5.146984 | -0.389% | +| nearest-rank p90 | 5.274224 | 5.229773 | -0.843% | +| 最小值 | 5.111929 | 5.098696 | - | +| 最大值 | 5.331087 | 5.301485 | - | +| 样本标准差 | 0.073960 | 0.065764 | - | + +H1 在 10 对中 8 胜 2 负;配对变化中位数为 -0.324%,均值为 -0.373%。第 1、2 +对分别有 +2.942% 和 -4.146% 的反向大波动,且采集顺序固定为 baseline 在前, +因此这 10 对只能支持“小幅方向性收益”,不能声称统计上已经稳定到每轮必胜。 +与 standalone 的 -16.22% 中位数不同,真实 PA 的约 0.3%~0.4% 中心改善更符合 +固定删除少量 HeapGuard atomic 的规模;standalone 的巨大间接调度变化不应外推。 + +最佳 H1 样本为第 3 对的 5.098696 ms,单独生成的泳道文件为: + +```text +outputs/TestPagedAttentionUnroll_Case1_20260717_173313/ +merged_swimlane_heapguard_first_lap_fastpath_5.098696ms.json +``` + +原始 A/B 文件分别位于临时 baseline worktree 和主 worktree 的对应 timestamp +目录。自动 converter 被关闭的正式样本仍保留 raw JSON;上述最佳样本在统计完成后 +单独补做了 converter,未重新执行 device workload。 + +#### 7.1.9 H1 最终阶段决定 + +H1 满足保留条件:正确性证明成立,standalone 三后端默认配置 PASS,真实 +A5Sim/A5 的 68 KiB 首圈/跨圈/回收 golden 与 PA Case1 golden 全部 PASS,真实 +A5 十对中心趋势和 p90 均未回退,并且直接 atomic 数量确定下降。因此将 standalone +与生产同形改动、测试证据文档作为一个本地 commit 保存,不 push。 + +真实性能收益较小且仍有单轮反向,不把 H1 宣传成大幅优化。下一阶段 F1 只研究 +fanin 依赖检查顺序;不得同时加入 ready cache 或退避,避免失去因果归属。 diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index 0737fd7ac4..670731c5ad 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -247,6 +247,14 @@ PA_DEVICE bool HeapGuard( bool waited = false; // 正常出口是 heap_next-vend 落入一个 ring;检测到不可能释放的覆盖或其他核 fatal 时返回失败。 while (!IsFatal(state)) { + // 逻辑 heap 尚未走完第一圈时,所有物理输出区间都位于 [0, heap_next), + // 不可能覆盖此前分配;保留上面的 fatal 原子检查后,可直接跳过 frontier/vend 读取。 + if (worker.heap_next <= ring) { + if constexpr (Profile) { + AccumulatePhase(stats.result, ProfilePhase::HeapGuard, wait_begin, Ops::Now()); + } + return true; + } const int64_t frontier = LoadLine(state->frontier); const int64_t retire = frontier - static_cast(state->heap_window); const uint64_t vend = retire < 0 ? 0 : Ops::Load(&state->tasks[retire].vend); From 3d174a082ecb8459781ac99374a31fb6512a5d7c Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 01:12:04 +0000 Subject: [PATCH 007/214] =?UTF-8?q?=E6=96=87=E6=A1=A3(a5):=20=E8=AE=B0?= =?UTF-8?q?=E5=BD=95fanin=E9=A1=BA=E5=BA=8F=E5=AE=9E=E9=AA=8C=E4=B8=8E?= =?UTF-8?q?=E5=9B=9E=E9=80=80=E7=BB=93=E8=AE=BA?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...05\345\206\265\345\210\206\346\236\220.md" | 77 ++++++++++++++++++- 1 file changed, 75 insertions(+), 2 deletions(-) diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index ed752e26a9..622a47467c 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -352,8 +352,8 @@ Alloc/QK/SF/PV winner 调用,共 1024 次。当前代表性 CCEC 轮次的累 | 阶段 | 单变量 | 当前状态 | | ---- | ------ | -------- | | H1 | HeapGuard 首圈 fast path | 正确性与真实性能完成,保留并本地提交 | -| F1 | fanin 依赖检查顺序 | H1 提交后开始 | -| 后续 | ready cache、退避、frontier、Claim | 尚未开始,必须逐项验证 | +| F1 | fanin 依赖检查顺序 | standalone 性能未改善,已回退 | +| 后续 | ready cache、退避、frontier、Claim | 按风险从低到高逐项验证 | ### 7.1 阶段 H1:HeapGuard 首圈 fast path @@ -648,3 +648,76 @@ A5 十对中心趋势和 p90 均未回退,并且直接 atomic 数量确定下 真实性能收益较小且仍有单轮反向,不把 H1 宣传成大幅优化。下一阶段 F1 只研究 fanin 依赖检查顺序;不得同时加入 ready cache 或退避,避免失去因果归属。 + +### 7.2 阶段 F1:fanin 依赖检查顺序 + +#### 7.2.1 候选改动与静态分析 + +F1 只在 standalone `CollectFanin()` 完成原有去重和 16 项截断后,将有效 +fanin 前缀按 producer task id 降序排列。排序放在截断之后,所以不改变 +原实现选中的依赖集合、`kMaxFanin=16` 语义或 slot ABI。fanin ready 是 AND +条件,仅改变检查顺序不改变“所有 producer 都完成后才可执行”的结果。 + +PA 中只有 UP 包含 3 个 fanin,原顺序是 `[SF, PV, Alloc]`,降序后为 +`[PV, SF, Alloc]`。PV 依赖 SF,因此在其他调度状态不变时: + +- SF、PV 均未完成:两种顺序都在第 1 次 ready load 后返回; +- SF 已完成、PV 未完成:原顺序读 2 次,新顺序读 1 次; +- PV 已完成:由依赖关系可知 SF 已完成,两种顺序都成功读完 3 项。 + +所以在固定调度状态下,UP 每次失败检查的 ready atomic load 不会增加, +且在 SF 已完成而 PV 尚未完成的窗口可减少 1 次。QK 的 fanin 为 0, +SF/PV 各为 1,本改动对它们不产生重排。不过,排序本身会改变标量指令和 +worker 到达时序;真实动态调度不能由上述固定状态推导出必然性能收益。 + +#### 7.2.2 standalone 正确性与完整运行 + +F1 修改后重建 CCEC、AscendC 和 CPU 三后端,并分别运行 smoke、256 batch 零 +NOP、256 batch 默认 NOP。全部结果的语义断言和后处理都为 PASS;完整用例 +仍为 73,728 次 Claim、1,280 个唯一 winner、1,024 个 kernel,placement 总数为 +1,024。关键单轮结果如下: + +| 场景 | CCEC `fanin / Submit us` | AscendC | CPU | +| ---- | -----------------------: | ------: | --: | +| smoke | `6 / 34.912` | `5 / 40.142` | `6 / 156383.064` | +| 256 batch、零 NOP | `42546 / 3415.839` | `25515 / 3498.038` | `1983 / 163840.941` | +| 256 batch、默认 NOP | `40958 / 4019.835` | `47692 / 4283.306` | `1051412 / 185456.323` | + +CPU 只用于语义对照,不作为 A5 性能依据。单轮 fanin load 对 worker 调度非常 +敏感,不能用上表三个数值直接归因,因此又执行了独立进程的交错 A/B。 + +#### 7.2.3 CCEC 十对交错 A/B + +基线固定在 H1 提交 `2c3dd1e2`,使用 detached worktree +`/tmp/simpler-f1-baseline`。两端都用 256 batch、默认 NOP、关闭泳道和 phase profile; +每个样本是独立进程首轮,统一 60 s timeout。前 5 对为 baseline -> F1, +后 5 对为 F1 -> baseline。双方均 10/10 PASS、0 timeout、Claim=73,728、CAS retry=0。 + +| 指标 | baseline | F1 | 相对变化 | +| ---- | -------: | -: | -------: | +| Submit 中位数(us) | 4290.401 | 4623.944 | **+7.774%** | +| Submit 均值(us) | 4498.804 | 4684.302 | **+4.123%** | +| Submit nearest-rank p90(us) | 5303.373 | 5219.956 | -1.573% | +| Submit 样本标准差(us) | 944.376 | 475.794 | - | +| fanin load 中位数 | 67914 | 65433.5 | -3.652% | +| fanin load 均值 | 82844.5 | 72219.2 | -12.826% | +| fanin load nearest-rank p90 | 118955 | 94088 | -20.905% | + +F1 的 Submit 为 4 胜 6 负;配对相对变化中位数为 `+6.439%`,均值为 +`+8.410%`。交换顺序后,后 5 对的配对变化中位数仍为 `+3.423%`,没有把 +中心趋势变成收益。fanin load 的配对变化中位数为 `-13.582%`,但均值为 +`+0.665%`,单对范围从 `-58.082%` 到 `+89.165%`,说明其仍强烈受动态调度影响。 + +F1 的 p90 略好主要由 baseline 的 5.303 ms 和 6.640 ms 慢样本抬高;在中位数、 +均值和配对中心全部回退时,不能单独用这个 p90 宣称收益。原始日志保留在: + +```text +/tmp/f1_standalone_ab_20260718_010300/ +``` + +#### 7.2.4 阶段决定 + +F1 在固定状态下的 atomic load 不增证明成立,三后端语义也全部通过;但 +standalone 的 Submit 中位数、均值和配对中心都没有改善。因此不将这个启发式 +重排迁移到真实 `dist_submit_collect_fanin()`,也不进行真实 PA A/B。standalone +候选代码已撤回,本节保留负结果,防止后续重复同一实验。 From 3d0aaea7e883028b73782ce2a71482f3a0cd7a4c Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 02:21:30 +0000 Subject: [PATCH 008/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E5=BB=BA?= =?UTF-8?q?=E7=AB=8B=E7=8B=AC=E7=AB=8BPA=E6=AF=8F=E6=A0=B8=E6=A0=87?= =?UTF-8?q?=E9=87=8FPMU=E8=A7=82=E5=AF=9F=E9=93=BE=E8=B7=AF?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...05\345\206\265\345\210\206\346\236\220.md" | 102 ++++++- ...77\347\224\250\346\214\207\345\215\227.md" | 89 ++++++ tests/atomic_probe/pa_scheduler/ccec/build.sh | 1 + tests/atomic_probe/pa_scheduler/ccec/host.cpp | 279 +++++++++++++++++- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 107 +++++++ .../pa_scheduler/ccec/pmu_probe.h | 94 ++++++ .../pa_scheduler/common/pa_model.h | 12 +- tests/atomic_probe/pa_scheduler/run.sh | 4 + 8 files changed, 676 insertions(+), 12 deletions(-) create mode 100644 tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 622a47467c..b0b20c8dd3 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -3,8 +3,8 @@ ## 1. 范围与当前结论 本文记录 `TestPagedAttentionUnroll::Case1` 在真实 A5 上的 FDWIC AICore -Submit 路径,供后续继续优化。快照日期为 2026-07-17,当前代码提交为 -`e3b748b43c7f226c025c4dcdfc2eb2805cec7f21`。 +Submit 路径,供后续继续优化。快照日期更新至 2026-07-18;当前保留的生产 +优化基线为 `2c3dd1e2`,F1 负结果记录提交为 `c93c3666`。 范围限定为: @@ -26,9 +26,11 @@ Submit 路径,供后续继续优化。快照日期为 2026-07-17,当前代 - 三轮最终版本的首末 Submit 中位数为 5.115620 ms,相比 5.642245 ms 基线下降 0.526625 ms,即 9.33%;最好单轮为 5.096685 ms; +- H1 又消除了默认 256 MiB heap 第一圈的 1024 次 frontier atomic load, + 十对真实 A5 的配对变化中位数为 -0.324%,最好单轮为 5.098696 ms; - 优化没有改变通用 atomic 语义,也没有把任务推迟到最终 drain 来制造 - 表面收益;下一优先级应是 Claim 竞争,其次是 completion/frontier 和 - fanin ready 轮询。 + 表面收益。F1 的 fanin 顺序重排已经证明性能回退并撤销;下一步先精确区分 + fanin 成功/失败 load 与 frontier 重复前推,再进行单变量消减。 环境安装、编译和基线复现过程见 [A5 FDWIC Paged Attention 安装与复现指南](../A5_FDWIC_PAGED_ATTENTION_REPRO.md)。 @@ -98,12 +100,14 @@ worker 更同步,cursor 瞬时竞争反而变强。总时间仍然下降,但 `output_bytes>0` 时进入检查;本 Case 每 batch 的 Alloc、QK、SF、PV 满足, UP 不满足,所以共有 1024 次有效调用。 -最好一轮没有 `RingBp` 事件,表示每次调用都在第一次循环检查后返回: +H1 后最好一轮没有 `RingBp` 事件,且默认 256 MiB heap 的 1024 次有效调用都 +命中第一圈 fast path: - `fatal_set()`:固定 1024 次 `atomic_load(g_dist.fatal)`; -- frontier:固定 1024 次 `atomic_load(g_dist.frontier)`; -- `load_task_vend(F-H)`:当索引非负时执行 `atomic_load(cell.vend)`,本轮 - 次数介于 0 和 1024,现有 trace 不能给出精确值; +- frontier:第一圈为 0 次;只有逻辑 heap 超过一整圈后才恢复原 + `atomic_load(g_dist.frontier)`; +- `load_task_vend(F-H)`:第一圈为 0 次;跨圈后仍按原 frontier/vend 容量 + 规则执行; - 如果未来出现 heap backpressure,上述三项会在循环中重复,耗时会归入 `RingBp`。 @@ -133,8 +137,25 @@ Submit 的 `EfDrain`,有背压时可能在 `RingBp`,剩余任务则在最终 | `task.vend` | `atomic_exchange` | 每 task 固定一次 | 1280 | | `task.flag` 发布 | `atomic_exchange` | 每 task 固定一次 | 1280 | | `frontier` 初始读取 | `atomic_load` | 每 task 固定一次 | 1280 | -| 后续 `task.flag` ready | `atomic_load` | 下界 | >=1280 | -| `frontier` 前推 | `atomic_fetch_max` | 下界,竞争时可重复 | >=1280 | +| 后续 `task.flag` ready | `atomic_load` | 若前推次数为 A,则准确为 A+1280 | >=2560 | +| `frontier` 前推 | `atomic_fetch_max` | 记为 A,竞争时可重复 | >=1280 | + +原因是每次 `advance_frontier()` 都以一次未就绪 flag load 结束;每次成功 +`atomic_fetch_max(frontier)` 前又恰有一次 ready flag load。因此完整成功运行中, +该处 flag load 不是笼统的“至少 1280”,而是准确的 `A + 1280`。 + +若把 fanin ready load 记为 `G`,H1 后完整 Submit + completion 路径的 atomic +总数为: + +~~~text +73728 Claim + 1024 HeapGuard fatal + G fanin ++ 1280 vend exchange + 1280 flag exchange + 1280 frontier initial load ++ A frontier atomicMax + (A + 1280) frontier flag load += 79872 + G + 2A +~~~ + +由于 `G>=1280`、`A>=1280`,硬下界为 83712 次。`G` 和 `A` 受真实调度时序 +影响,不能用静态下界代替动态样本。 最终最好一轮的 1024 个 kernel 中,1011 个在某个 Submit 的 `EfDrain` 中执行,0 个在 `RingBp` 中执行,13 个在本核最后一个 Submit 之后的最终 @@ -721,3 +742,64 @@ F1 在固定状态下的 atomic load 不增证明成立,三后端语义也全 standalone 的 Submit 中位数、均值和配对中心都没有改善。因此不将这个启发式 重排迁移到真实 `dist_submit_collect_fanin()`,也不进行真实 PA A/B。standalone 候选代码已撤回,本节保留负结果,防止后续重复同一实验。 + +### 7.3 阶段 O1:建立 CCEC 每核 scalar PMU 观察链路 + +#### 7.3.1 为什么不再使用 external task 汇总冒充局部数据 + +控制实验已经证明:即使在 kernel 内调用 CANN 正式 `metrics_prof_stop()`, +external task-based `msprof` 的整任务 raw cycles 仍不会随局部门控骤降。因此该 +汇总只能描述整个 task,不能作为 Claim、EfDrain、WaitForSlot 或 HeapGuard 的 +局部取数依据。 + +O1 在 standalone CCEC 同一 runtime TU 内完成门控、读取和发布:host 复用正式 +A5 runtime 的 `halResMap(PROCESS_CP1, RES_AICORE)` 布局,将 36 个 AICore 展开为 +108 个物理子核 MMIO base;kernel 用真实 `get_coreid()` 索引,逐核核对 +PipeUtilization 的 `CNT2=0x1`、`CNT6=0x34`、`CNT7=0x35` selector,最后只读取 +一次 `CNT_TOTAL/CNT2/CNT6/CNT7` 并写入每 worker 独占结果区。 + +这套 PMU 事件不直接给出 atomic 操作条数。atomic 条数继续由源码不变量和 +worker-local 软件计数精确核对;PMU 用于观察这些 atomic 及周边 scalar 指令造成 +的周期、I-cache request/miss 和竞争时序变化,两种证据不能相互冒充。 + +#### 7.3.2 A5 动态验证结果 + +在 CANN 9.1 task-based PipeUtilization 配置下,分别执行 10 轮 empty、单段 +100000 NOP 和双段 `2×100000` NOP。三组每轮都满足 96/96 selector/MMIO 记录 +可信、96/96 物理子核 id 唯一;CCEC、AscendC、CPU 原语义 smoke 也全部 PASS。 + +| 窗口 | 96 核 total 中位数 | 稳定性与响应 | +| ---- | -----------------: | ------------ | +| empty | 预热后约 419 | 轮间约 ±1,给出空窗口 gate 固定开销 | +| scalar 100000 | 预热后约 56775 | 轮间约 ±2,CNT2/req/miss 同步增加 | +| scalar-double 2×100000 | 预热后约 113113 | 轮间约 ±8,扣除 empty 后为单段 1.9997 倍 | + +I-cache request 的每核量级约为 empty 22、单段 26245、双段 52476;miss 约为 +4、23、45。窗口前 snapshot 会消费/清除此前累计;双段中间只执行 +`stop/start`、不读取 MMIO,末尾一次 snapshot 仍得到近似精确的两倍响应。 +这证明多窗口续积机制成立,而不需要在每次 Submit 中插入高扰动 MMIO read; +扩展到上千个真实小窗口仍需进一步核对 gate 次数、同次数 empty 对照和 counter +是否溢出,不能由双段结果直接宣称已经精确覆盖任意窗口数。 + +原始日志保留在: + +~~~text +tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ + empty10_20260718_015533_console.log + scalar100k_10_20260718_015558_console.log + scalar2x100k_10_20260718_021035_console.log +~~~ + +#### 7.3.3 阶段决定与后续使用边界 + +O1 已达到“可用”的门槛:物理核映射、事件 selector、正向敏感性、A/A 重复性 +和双 gate 累计全部在真实 A5 上得到动态验证。因此后续可以恢复 atomic 优化, +但一次运行只选择一个阶段,A/B 两侧保持完全相同的 gate 次数和代码布局,并用 +同次数 empty gate 对照门控成本。真实 vector/cube kernel 不能跳过;测 EfDrain +时应在唯一 kernel 调用点暂时 stop,返回后仅在原 gate 活跃时 resume,从而保留 +真实依赖时序而排除计算体计数。 + +下一步先在 standalone 增加 `G` 的成功/失败分类、`A`、frontier ready/终止 load +等 worker-local 诊断计数,不增加共享 atomic。得到动态规模后,首个低风险单变量 +候选是每个私有 slot 缓存已经观察为 ready 的 fanin 前缀;slot 重用时必须重置, +且先完成 CCEC 交错 A/B,再决定是否迁移到真实 FDWIC。 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index e0ad51a21a..858d8e87b7 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -266,6 +266,95 @@ CPU 完整协议回归建议关闭大泳道缓冲区: WaitForSlot 和 HeapGuard 的 `calls_total` 会按 winner 所在角色分布,AIC/AIV 相加必须分别等于 1,024;等待事件则对应额外的 RingBp 泳道记录。 +### 5.6 CCEC 每核 scalar PMU 与 I-cache 诊断 + +CCEC 后端提供一个显式诊断模式,用来验证局部 scalar 性能观察链路。它不读取 +`msprof` 导出的整任务 PMU 汇总作为局部结果,而是由 kernel 在每个物理子核上 +直接读取 `CNT_TOTAL/CNT2/CNT6/CNT7`,最后写入该 worker 独占的 +`WorkerResult`。对应含义为: + +| 字段 | PipeUtilization selector | 含义 | +| --- | ---: | --- | +| `pmu_total_cycles` | PMU total | gate 窗口累计周期 | +| `pmu_scalar_busy` | `CNT2 = 0x1` | scalar 原始事件计数 | +| `pmu_icache_requests` | `CNT6 = 0x34` | I-cache request | +| `pmu_icache_misses` | `CNT7 = 0x35` | I-cache miss | + +selector 和 PMU framework 仍由 CANN 9.1 的 task-based profiler 配置;CCEC +只做门控与读取。一次 snapshot 会消费/清除此前累计,因此窗口前先冻结并做一次 +baseline read-clear,窗口中间只切 gate,末尾再做唯一一次结果 snapshot。host +按正式 A5 runtime 的方式调用 `halResMap`,构造 36 个 +物理 AICore展开后的 108 项子核 MMIO 表;kernel 使用真实 `get_coreid()` 索引, +不能用逻辑 `worker_id` 猜物理核。 + +在本目录先构建 CCEC,再直接让 `msprof` 包住 host runner: + +```bash +./run.sh build ccec + +OUT="./outputs/pmu_validation/empty" +msprof \ + --output="$OUT" \ + --type=db \ + --ai-core=on \ + --aic-mode=task-based \ + --aic-metrics=PipeUtilization \ + ./build/ccec/pa_scheduler_host \ + --kernel ./build/ccec/pa_scheduler_kernel.o \ + --device 0 --batches 1 --runs 10 --nop-count 0 --no-swimlane \ + --pmu-window empty +``` + +三个诊断窗口的用途是: + +- `empty`:只执行一次 start/stop,量空窗口的 gate 固定开销;末尾 snapshot 在 + stop 后执行,其 MMIO 读取开销不计入 total; +- `scalar`:在同一窗口内执行 `--pmu-scalar-nops N` 个受控 NOP; +- `scalar-double`:执行两段相同 NOP,中间只 stop/start、不读 counter,用于验证 + 多个局部窗口能否暂停后继续累计; +- `off`:默认值,不申请 MMIO 表,也不要求由 `msprof` 启动。 + +例如 10 万 NOP 的正向敏感性测试只需把末尾参数换成: + +```bash +--pmu-window scalar --pmu-scalar-nops 100000 +``` + +每轮必须同时看到: + +```text +trusted=96/96 unique_coreids=96/96 +[ASSERT] all PMU records have configured selectors and data PASS +[ASSERT] all 96 PMU physical subcore ids are unique PASS +``` + +`trusted` 会逐核检查 MMIO 映射、三个 selector 和非零 total;因此不能用“外部 +profiler 已经启动”代替实际 selector 读回。AIC 与 AIV 分开输出,I-cache miss +rate 使用 `sum(miss) / sum(request)`,不平均逐核百分比。`pmu_scalar_busy` 是原始 +事件计数;empty 窗口中它可以大于 total,不能把两者比值直接宣传成利用率。 + +2026-07-18 的 A5 验证结果为: + +| 窗口 | 轮次 | 96 核 total 中位数 | 结论 | +| --- | ---: | ---: | --- | +| empty | 10 | 预热后约 419,轮间约 ±1 | 空窗口 gate 开销稳定 | +| scalar 100,000 | 10 | 预热后约 56,774,轮间约 ±2 | scalar/req/miss 均稳定响应 | +| scalar-double 2×100,000 | 10 | 预热后约 113,113,轮间约 ±8 | 扣除 empty 后为单段的 1.9997 倍,多 gate 可续积 | + +同一轮中,I-cache request 从 empty 的每核约 22 增至单段的约 26,245,双段约 +52,476;miss 也从每核约 4 增至约 23/45。双段十轮中 96 核均满足 +`trusted=96/96` 和物理子核 id 唯一,说明本链路能直接看到 scalar 和 I-cache +变化,也证明 stop/start 之间不读 counter 时,多段窗口会累计到末尾唯一一次 +snapshot。首次热身的 p95 偶有偏高,正式比较应丢弃首轮,并保留 A/A +重复性数据。 + +这里验证的是观察手段,不是 PA 优化本身。后续用于 Claim、EfDrain、 +WaitForSlot 或 HeapGuard 时,应先做一次 baseline read-clear,多个局部窗口之间只切 +gate,最后每核读取一次;禁止每个 Submit 都读 MMIO。当前双段只证明多窗口续积 +机制成立,扩展到上千个真实小窗口时仍要核对 gate 次数、empty 对照和 counter +是否溢出。empty 开销和诊断代码布局必须在 A/B 两边完全相同,真实 PA 的最终 +结论仍需撤回诊断代码后再跑原始泳道与 golden。 + ## 6. 当前 A5 结果与真实 PA 的差异 2026-07-17 当前源码的一轮代表性结果如下。所有严格校验均为 PASS,kernel diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh index f6d5e7d6ea..574f2cb940 100755 --- a/tests/atomic_probe/pa_scheduler/ccec/build.sh +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -109,6 +109,7 @@ echo "[BUILD] CCEC host runner" -L"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ -Wl,-rpath,"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ -lascendcl -lruntime \ + -ldl \ -o "$BUILD_DIR/pa_scheduler_host" echo "[BUILD] complete: $BUILD_DIR" diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index 360383d6aa..c19f24d07d 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -10,13 +10,16 @@ */ #include "../common/host_support.h" +#include "pmu_probe.h" #include "acl/acl.h" +#include "driver/ascend_hal.h" #include "runtime/rt.h" #include #include #include +#include #include #include #include @@ -48,13 +51,256 @@ std::vector ReadBinary(const std::string &path) { return data; } +struct PmuOptions { + pa_scheduler::ccec_pmu::WindowMode mode = pa_scheduler::ccec_pmu::WindowMode::Off; + uint32_t scalar_nops = 100000; +}; + +const char *PmuModeName(pa_scheduler::ccec_pmu::WindowMode mode) { + switch (mode) { + case pa_scheduler::ccec_pmu::WindowMode::Off: + return "off"; + case pa_scheduler::ccec_pmu::WindowMode::Empty: + return "empty"; + case pa_scheduler::ccec_pmu::WindowMode::Scalar: + return "scalar"; + case pa_scheduler::ccec_pmu::WindowMode::ScalarDouble: + return "scalar-double"; + } + return "invalid"; +} + +bool ParsePmuOptions(int argc, char **argv, PmuOptions *pmu, std::vector *common_argv) { + // PMU 参数只属于 CCEC 验证分支;先摘出再交给三后端共享 parser,避免 CPU/AscendC 静默接受却不生效。 + bool mode_seen = false; + bool nops_seen = false; + common_argv->clear(); + common_argv->push_back(argv[0]); + for (int index = 1; index < argc; ++index) { + const std::string argument = argv[index]; + if (argument != "--pmu-window" && argument != "--pmu-scalar-nops") { + common_argv->push_back(argv[index]); + continue; + } + if (index + 1 >= argc) { + std::fprintf(stderr, "Missing value after %s\n", argument.c_str()); + return false; + } + const char *value = argv[++index]; + if (argument == "--pmu-window") { + if (mode_seen) { + std::fprintf(stderr, "Specify --pmu-window only once.\n"); + return false; + } + const std::string name = value; + if (name == "off") { + pmu->mode = pa_scheduler::ccec_pmu::WindowMode::Off; + } else if (name == "empty") { + pmu->mode = pa_scheduler::ccec_pmu::WindowMode::Empty; + } else if (name == "scalar") { + pmu->mode = pa_scheduler::ccec_pmu::WindowMode::Scalar; + } else if (name == "scalar-double") { + pmu->mode = pa_scheduler::ccec_pmu::WindowMode::ScalarDouble; + } else { + std::fprintf( + stderr, "Invalid --pmu-window value: %s (expected off|empty|scalar|scalar-double)\n", value + ); + return false; + } + mode_seen = true; + } else { + if (nops_seen || !pa_scheduler::host::ParseUint(value, 0, 10000000, &pmu->scalar_nops)) { + std::fprintf(stderr, "Invalid or duplicate --pmu-scalar-nops value: %s\n", value); + return false; + } + nops_seen = true; + } + } + if (nops_seen && pmu->mode != pa_scheduler::ccec_pmu::WindowMode::Scalar && + pmu->mode != pa_scheduler::ccec_pmu::WindowMode::ScalarDouble) { + std::fprintf(stderr, "--pmu-scalar-nops requires a scalar PMU window.\n"); + return false; + } + return true; +} + +using HalResMapFn = int (*)(uint32_t, struct res_map_info *, unsigned long *, uint32_t *); +using HalResUnmapFn = int (*)(uint32_t, struct res_map_info *); + +struct PmuRegisterMappings { + HalResUnmapFn unmap = nullptr; + std::vector mapped_resources; + std::vector register_bases; +}; + +bool UnmapPmuRegisters(uint32_t device, PmuRegisterMappings *mappings) { + bool ok = true; + if (mappings->unmap != nullptr) { + for (auto iterator = mappings->mapped_resources.rbegin(); iterator != mappings->mapped_resources.rend(); + ++iterator) { + const int error = mappings->unmap(device, &*iterator); + if (error != 0) { + std::fprintf(stderr, "halResUnmap failed for core %u (rc=%d)\n", iterator->res_id, error); + ok = false; + } + } + } + mappings->mapped_resources.clear(); + mappings->register_bases.clear(); + return ok; +} + +bool MapPmuRegisters(uint32_t device, PmuRegisterMappings *mappings) { + using namespace pa_scheduler::ccec_pmu; + const auto map = reinterpret_cast(dlsym(RTLD_DEFAULT, "halResMap")); + mappings->unmap = reinterpret_cast(dlsym(RTLD_DEFAULT, "halResUnmap")); + if (map == nullptr || mappings->unmap == nullptr) { + std::fprintf(stderr, "halResMap/halResUnmap is unavailable in the current CANN driver process.\n"); + return false; + } + + mappings->register_bases.assign(kPhysicalSubcoreCount, 0); + mappings->mapped_resources.reserve(kPhysicalAicoreCount); + for (uint32_t aicore = 0; aicore < kPhysicalAicoreCount; ++aicore) { + res_map_info info{}; + info.target_proc_type = PROCESS_CP1; + info.res_type = RES_AICORE; + info.res_id = aicore; + unsigned long map_address = 0; + uint32_t map_bytes = kAicoreMapBytes; + const int error = map(device, &info, &map_address, &map_bytes); + if (error != 0 || map_address == 0 || map_bytes < kAicoreMapBytes) { + std::fprintf( + stderr, "halResMap failed for core %u (rc=%d address=0x%lx bytes=%u)\n", aicore, error, + map_address, map_bytes + ); + (void)UnmapPmuRegisters(device, mappings); + return false; + } + mappings->mapped_resources.push_back(info); + + // 与正式 A5 host_regs.cpp 相同:每个 die 的布局为 18 AIC,随后是 36 AIV。 + const uint32_t die = aicore / kAicorePerDie; + const uint32_t local = aicore % kAicorePerDie; + const uint32_t die_base = die * kSubcoresPerDie; + mappings->register_bases[die_base + local] = static_cast(map_address); + mappings->register_bases[die_base + kAicorePerDie + local * 2] = + static_cast(map_address) + kAivFirstOffset; + mappings->register_bases[die_base + kAicorePerDie + local * 2 + 1] = + static_cast(map_address) + kAivSecondOffset; + } + return true; +} + +void ConfigurePmu(pa_scheduler::SchedulerState *state, const PmuOptions &pmu, const void *register_table) { + using namespace pa_scheduler::ccec_pmu; + state->config.reserved[kConfigMode] = static_cast(pmu.mode); + state->config.reserved[kConfigScalarNops] = pmu.scalar_nops; + StorePointer(state->config.reserved, register_table); + state->config.reserved[kConfigMagic] = pmu.mode == WindowMode::Off ? 0 : kConfigMagicValue; +} + +struct PmuAggregate { + std::vector total_cycles; + uint64_t scalar_busy = 0; + uint64_t icache_requests = 0; + uint64_t icache_misses = 0; +}; + +void AddPmuSample(const pa_scheduler::WorkerResult &result, PmuAggregate *aggregate) { + aggregate->total_cycles.push_back(result.pmu_total_cycles); + aggregate->scalar_busy += result.pmu_scalar_busy; + aggregate->icache_requests += result.pmu_icache_requests; + aggregate->icache_misses += result.pmu_icache_misses; +} + +void PrintPmuAggregate(const char *name, const PmuAggregate &aggregate) { + const pa_scheduler::host::Uint64Distribution total = + pa_scheduler::host::SummarizeUint64(aggregate.total_cycles); + const double miss_rate = aggregate.icache_requests == 0 + ? 0.0 + : 100.0 * aggregate.icache_misses / aggregate.icache_requests; + std::printf( + "[PMU-%s] cores=%zu total_sum=%llu total_median=%.1f total_p95=%llu scalar_busy=%llu " + "icache_req=%llu icache_miss=%llu miss_rate=%.4f%%\n", + name, aggregate.total_cycles.size(), static_cast(total.total), total.median, + static_cast(total.p95), static_cast(aggregate.scalar_busy), + static_cast(aggregate.icache_requests), + static_cast(aggregate.icache_misses), miss_rate + ); +} + +bool ValidatePmu(const pa_scheduler::SchedulerState &state, uint32_t run, const PmuOptions &pmu) { + using namespace pa_scheduler::ccec_pmu; + if (pmu.mode == WindowMode::Off) return true; + + bool seen[kPhysicalSubcoreCount] = {}; + uint32_t trusted = 0; + uint32_t unique = 0; + uint32_t prior_larger = 0; + uint32_t bad_printed = 0; + PmuAggregate all; + PmuAggregate aic; + PmuAggregate aiv; + for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { + const pa_scheduler::WorkerResult &result = state.results[worker]; + const uint32_t status = result.pmu_status; + const uint32_t core_id = StatusCoreId(status); + const bool record_trusted = (status & kStatusRequired) == kStatusRequired; + trusted += record_trusted; + prior_larger += (status & kStatusPriorSnapshotLarger) != 0; + if (core_id < kPhysicalSubcoreCount && !seen[core_id]) { + seen[core_id] = true; + ++unique; + } + if (!record_trusted && bad_printed < 8) { + std::printf( + "[PMU-BAD] worker=%u role=%llu coreid=%u status=0x%08x total=%llu scalar=%u req=%u miss=%u\n", + worker, static_cast(result.role), core_id, status, + static_cast(result.pmu_total_cycles), result.pmu_scalar_busy, + result.pmu_icache_requests, result.pmu_icache_misses + ); + ++bad_printed; + } + AddPmuSample(result, &all); + AddPmuSample(result, result.role == static_cast(pa_scheduler::CoreRole::Aic) ? &aic : &aiv); + } + + PrintPmuAggregate("ALL", all); + PrintPmuAggregate("AIC", aic); + PrintPmuAggregate("AIV", aiv); + const bool records_ok = trusted == pa_scheduler::kWorkers; + const bool core_ids_ok = unique == pa_scheduler::kWorkers; + std::printf( + "[PMU] run=%u window=%s scalar_nops=%u trusted=%u/%u unique_coreids=%u/%u prior_larger=%u/%u\n", run, + PmuModeName(pmu.mode), pmu.scalar_nops, trusted, pa_scheduler::kWorkers, unique, pa_scheduler::kWorkers, + prior_larger, pa_scheduler::kWorkers + ); + std::printf("[ASSERT] %-48s %s\n", "all PMU records have configured selectors and data", + records_ok ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "all 96 PMU physical subcore ids are unique", + core_ids_ok ? "PASS" : "FAIL"); + return records_ok && core_ids_ok; +} + } // namespace int main(int argc, char **argv) { // 参数和 ELF 在创建 ACL 资源前完成校验,早期错误不会留下 device、stream 或 kernel handle。 pa_scheduler::host::Options options; - const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions(argc, argv, true, &options); + PmuOptions pmu_options; + std::vector common_argv; + if (!ParsePmuOptions(argc, argv, &pmu_options, &common_argv)) return EXIT_FAILURE; + const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions( + static_cast(common_argv.size()), common_argv.data(), true, &options + ); if (parse_status != pa_scheduler::host::ParseStatus::Ok) { + if (parse_status == pa_scheduler::host::ParseStatus::Help) { + std::fprintf( + stderr, + "CCEC PMU options: [--pmu-window off|empty|scalar|scalar-double] [--pmu-scalar-nops N]\n" + ); + } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } const std::vector binary_data = ReadBinary(options.kernel_path); @@ -63,6 +309,10 @@ int main(int argc, char **argv) { return EXIT_FAILURE; } pa_scheduler::host::PrintBanner("CCEC", options); + std::printf( + "[PMU-CONFIG] window=%s scalar_nops=%u source=direct-per-core requires=msprof-PipeUtilization\n", + PmuModeName(pmu_options.mode), pmu_options.scalar_nops + ); // 正常及后处理路径依次完成 ACL 初始化、选卡、stream/ELF/设备区创建、launch/D2H // 和尾部清理;初始化、传输或 launch 的早期错误仍按当前实现就地返回。 @@ -98,6 +348,27 @@ int main(int argc, char **argv) { return EXIT_FAILURE; } + PmuRegisterMappings pmu_mappings; + void *pmu_registers_device = nullptr; + if (pmu_options.mode != pa_scheduler::ccec_pmu::WindowMode::Off) { + if (!MapPmuRegisters(options.device, &pmu_mappings)) return EXIT_FAILURE; + const size_t register_bytes = pmu_mappings.register_bases.size() * sizeof(uint64_t); + if (!CheckAcl( + aclrtMalloc(&pmu_registers_device, register_bytes, ACL_MEM_MALLOC_NORMAL_ONLY), + "aclrtMalloc(PMU register table)" + ) || + !CheckAcl( + aclrtMemcpy( + pmu_registers_device, register_bytes, pmu_mappings.register_bases.data(), register_bytes, + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D PMU register table)" + )) { + (void)UnmapPmuRegisters(options.device, &pmu_mappings); + return EXIT_FAILURE; + } + } + // 泳道区按 96 worker 各 65536 条记录预留,约 384 MiB;关闭泳道时不申请,也不会传递有效 base。 void *trace_device = nullptr; if (options.trace_enabled && @@ -122,6 +393,7 @@ int main(int argc, char **argv) { for (uint32_t run = 1; run <= options.runs; ++run) { pa_scheduler::host::InitializeState(state.get(), options); pa_scheduler::host::ConfigureTrace(state.get(), options, trace_device); + ConfigurePmu(state.get(), pmu_options, pmu_registers_device); if (options.trace_enabled) { // 每轮只需重置约 7 KiB header;各 worker 会从 count=0 覆盖自己的记录区,无需清零整块 384 MiB。 pa_scheduler::host::InitializeTraceHeader(&trace_header); @@ -225,6 +497,7 @@ int main(int argc, char **argv) { *state, run, host_us, options.trace_enabled ? &trace_header : nullptr ); all_passed &= metrics.passed; + all_passed &= ValidatePmu(*state, run, pmu_options); spans.push_back(metrics.submit_span_us); if (options.analyze_swimlane && !pa_scheduler::host::AnalyzeSwimlaneRecords(trace_header, *state, read_trace_records)) { @@ -260,6 +533,10 @@ int main(int argc, char **argv) { if (trace_device != nullptr) { cleanup_ok &= CheckAcl(aclrtFree(trace_device), "aclrtFree(swimlane trace)"); } + if (pmu_registers_device != nullptr) { + cleanup_ok &= CheckAcl(aclrtFree(pmu_registers_device), "aclrtFree(PMU register table)"); + cleanup_ok &= UnmapPmuRegisters(options.device, &pmu_mappings); + } cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(state)"); const rtError_t unload_error = registered_all ? rtDevBinaryUnRegister(kernel_handle) : rtBinaryUnLoad(kernel_handle); diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index e9c95ddc7c..5724866900 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -12,6 +12,8 @@ #include "cce_aicore_intrinsics.h" #include +#include "pmu_probe.h" + #define PA_DEVICE __aicore__ inline #define PA_GM __gm__ #include "../common/pa_scheduler_core.h" @@ -127,8 +129,111 @@ struct CcecOps { // 每核独占的 WorkerResult 用 bypass-DCache store 发布,host 同步后可直接 D2H,无需共享原子竞争。 __builtin_cce_st_dev(value, address, 0); } + + __aicore__ static inline void Publish(__gm__ uint32_t *address, uint32_t value) { + __builtin_cce_st_dev(value, address, 0); + } }; +struct PmuSnapshot { + uint64_t total_cycles = 0; + uint32_t scalar_busy = 0; + uint32_t icache_requests = 0; + uint32_t icache_misses = 0; + uint32_t status = 0; +}; + +template +__aicore__ inline uint32_t ReadPmuRegister(uint64_t reg_base) { + // 传给 ld_dev 的是重基址后的 __gm__ 指针;相对 offset 均落在编译器允许的 [-2048, 2047]。 + int32_t *block = reinterpret_cast(reg_base + BlockOffset); + return static_cast(ld_dev(block, static_cast(RegisterOffset - BlockOffset))); +} + +__aicore__ inline PmuSnapshot ReadObservedCounters(uint64_t reg_base) { + PmuSnapshot sample; + sample.scalar_busy = ReadPmuRegister(reg_base); + sample.icache_requests = ReadPmuRegister(reg_base); + sample.icache_misses = ReadPmuRegister(reg_base); + const uint64_t low = ReadPmuRegister(reg_base); + const uint64_t high = ReadPmuRegister(reg_base); + sample.total_cycles = low | (high << 32); + return sample; +} + +__aicore__ inline void RunPmuProbe(__gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id) { + using namespace pa_scheduler::ccec_pmu; + __gm__ pa_scheduler::WorkerResult &result = state->results[worker_id]; + PmuSnapshot sample; + + const WindowMode mode = static_cast(state->config.reserved[kConfigMode]); + if (mode != WindowMode::Off) { + sample.status |= kStatusRequested; + const uint32_t physical_core_id = static_cast(get_coreid()) & kStatusCoreIdMask; + sample.status |= physical_core_id << kStatusCoreIdShift; + const uint64_t table_address = + static_cast(state->config.reserved[kConfigRegTableLow]) | + (static_cast(state->config.reserved[kConfigRegTableHigh]) << 32); + if (state->config.reserved[kConfigMagic] == kConfigMagicValue && table_address != 0 && + physical_core_id < kPhysicalSubcoreCount) { + sample.status |= kStatusCoreIdValid; + __gm__ const uint64_t *register_bases = reinterpret_cast<__gm__ const uint64_t *>(table_address); + const uint64_t reg_base = register_bases[physical_core_id]; + if (reg_base != 0) { + sample.status |= kStatusRegMapped; + const uint32_t selector2 = + ReadPmuRegister(reg_base); + const uint32_t selector6 = + ReadPmuRegister(reg_base); + const uint32_t selector7 = + ReadPmuRegister(reg_base); + if (selector2 == kScalarBusyEvent) sample.status |= kStatusCnt2Selector; + if (selector6 == kIcacheRequestEvent) sample.status |= kStatusCnt6Selector; + if (selector7 == kIcacheMissEvent) sample.status |= kStatusCnt7Selector; + + // 外部 profiler 启动 task 时可能已经累计了 scheduler;先冻结并读取一次窗口前快照。 + // A5 的 snapshot 会消费/清除当前累计;stop/start 只负责门控,中间不读取才能续积多段窗口。 + bisheng::cce::metrics_prof_stop(); + const PmuSnapshot prior = ReadObservedCounters(reg_base); + bisheng::cce::metrics_prof_start(); + if (mode == WindowMode::Scalar || mode == WindowMode::ScalarDouble) { + RuntimeNop(state->config.reserved[kConfigScalarNops]); + } + if (mode == WindowMode::ScalarDouble) { + // 两段相同工作量之间只切 gate、不读取 counter,用于确认 resume 是累计还是重置。 + bisheng::cce::metrics_prof_stop(); + bisheng::cce::metrics_prof_start(); + RuntimeNop(state->config.reserved[kConfigScalarNops]); + } + bisheng::cce::metrics_prof_stop(); + sample = ReadObservedCounters(reg_base); + sample.status |= kStatusRequested | kStatusRegMapped | kStatusCoreIdValid | + (physical_core_id << kStatusCoreIdShift); + if (selector2 == kScalarBusyEvent) sample.status |= kStatusCnt2Selector; + if (selector6 == kIcacheRequestEvent) sample.status |= kStatusCnt6Selector; + if (selector7 == kIcacheMissEvent) sample.status |= kStatusCnt7Selector; + if (sample.total_cycles != 0) sample.status |= kStatusTotalNonzero; + + // 窗口前 read-clear 之后只做这一次最终 snapshot;重复读取会看到读取路径自身的残余。 + // prior 更大只作为“已从此前 scheduler 累计中隔离”的辅助证据,不作为长负载通用门禁。 + if (sample.total_cycles < prior.total_cycles) sample.status |= kStatusPriorSnapshotLarger; + } + } + } + + // 结果位于每核独占 sidecar,五次 bypass store 不参与 Submit 时间口径。 + CcecOps::Publish(&result.pmu_total_cycles, sample.total_cycles); + CcecOps::Publish(&result.pmu_scalar_busy, sample.scalar_busy); + CcecOps::Publish(&result.pmu_icache_requests, sample.icache_requests); + CcecOps::Publish(&result.pmu_icache_misses, sample.icache_misses); + CcecOps::Publish(&result.pmu_status, sample.status); +} + } // namespace #if defined(PA_BUILD_AIC) @@ -139,6 +244,7 @@ extern "C" __global__ __aicore__ void pa_scheduler_0_mix_aic(__gm__ pa_scheduler // 32 个物理 block 的 AIC 直接使用 block_idx,形成连续 worker 0..31。 const uint32_t worker_id = static_cast(get_block_idx()); pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aic); + RunPmuProbe(state, worker_id); } #elif defined(PA_BUILD_AIV) PTO_SYNCALL_MIX_AIC_KERNEL_META(pa_scheduler_0_mix_aiv, 1, 2); @@ -148,6 +254,7 @@ extern "C" __global__ __aicore__ void pa_scheduler_0_mix_aiv(__gm__ pa_scheduler const uint32_t vector_id = static_cast(get_block_idx() * get_subblockdim() + get_subblockid()); const uint32_t worker_id = pa_scheduler::kAicWorkers + vector_id; pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aiv); + RunPmuProbe(state, worker_id); } #else #error "Compile with PA_BUILD_AIC or PA_BUILD_AIV" diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h new file mode 100644 index 0000000000..a19765b9a0 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h @@ -0,0 +1,94 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_CCEC_PMU_PROBE_H +#define PA_SCHEDULER_CCEC_PMU_PROBE_H + +#include + +namespace pa_scheduler::ccec_pmu { + +// 第一阶段只验证观察链路,不把门控下沉到真实 Submit 热路径。 +// Empty 用于量 start/stop 本身的底噪,Scalar 在同一窗口内执行可控 NOP 段。 +enum class WindowMode : uint32_t { + Off = 0, + Empty = 1, + Scalar = 2, + ScalarDouble = 3, +}; + +// RunConfig::reserved 保持既有 64B ABI;CCEC 独占解释以下五个槽位,其他后端仍看到全零。 +constexpr uint32_t kConfigMode = 0; +constexpr uint32_t kConfigScalarNops = 1; +constexpr uint32_t kConfigRegTableLow = 2; +constexpr uint32_t kConfigRegTableHigh = 3; +constexpr uint32_t kConfigMagic = 4; +constexpr uint32_t kConfigMagicValue = 0x504d5531U; // "PMU1" + +// DAV_3510 有 36 个物理 AICore,每个 AICore 展开为 1 AIC + 2 AIV,共 108 个物理子核编号。 +constexpr uint32_t kPhysicalAicoreCount = 36; +constexpr uint32_t kPhysicalSubcoreCount = 108; +constexpr uint32_t kAicorePerDie = 18; +constexpr uint32_t kSubcoresPerDie = 54; +constexpr uint64_t kAivFirstOffset = 0x100000ULL; +constexpr uint64_t kAivSecondOffset = 0x200000ULL; +constexpr uint32_t kAicoreMapBytes = 0x300000U; + +// PIPE_UTILIZATION 的三个 scalar 前端事件。事件配置仍由 msprof/AICPU 完成,kernel 只读回核对。 +constexpr uint32_t kScalarBusyEvent = 0x1U; +constexpr uint32_t kIcacheRequestEvent = 0x34U; +constexpr uint32_t kIcacheMissEvent = 0x35U; + +// DAV_3510 PMU MMIO offset。ld_dev 的立即数只有 12 bit,因此 kernel 会分别重基址到 0x2400/0x4200。 +constexpr uint32_t kSelectorBlockOffset = 0x2400U; +constexpr uint32_t kCounterBlockOffset = 0x4200U; +constexpr uint32_t kCnt2Offset = 0x4220U; +constexpr uint32_t kCnt6Offset = 0x4240U; +constexpr uint32_t kCnt7Offset = 0x4248U; +constexpr uint32_t kTotalLowOffset = 0x4260U; +constexpr uint32_t kTotalHighOffset = 0x4264U; +constexpr uint32_t kCnt2SelectorOffset = 0x2508U; +constexpr uint32_t kCnt6SelectorOffset = 0x2518U; +constexpr uint32_t kCnt7SelectorOffset = 0x251cU; + +// pmu_status 的低位描述本条记录是否可信,高 16 bit 保存 get_coreid(),便于 host 检查 96 核唯一性。 +constexpr uint32_t kStatusRequested = 1U << 0; +constexpr uint32_t kStatusRegMapped = 1U << 1; +constexpr uint32_t kStatusCoreIdValid = 1U << 2; +constexpr uint32_t kStatusCnt2Selector = 1U << 3; +constexpr uint32_t kStatusCnt6Selector = 1U << 4; +constexpr uint32_t kStatusCnt7Selector = 1U << 5; +constexpr uint32_t kStatusTotalNonzero = 1U << 7; +constexpr uint32_t kStatusPriorSnapshotLarger = 1U << 8; +constexpr uint32_t kStatusRequired = kStatusRequested | kStatusRegMapped | kStatusCoreIdValid | + kStatusCnt2Selector | kStatusCnt6Selector | kStatusCnt7Selector | + kStatusTotalNonzero; +constexpr uint32_t kStatusCoreIdShift = 16; +constexpr uint32_t kStatusCoreIdMask = 0x0fffU; + +inline uint64_t PackPointer(const uint32_t *words) { + return static_cast(words[kConfigRegTableLow]) | + (static_cast(words[kConfigRegTableHigh]) << 32); +} + +inline void StorePointer(uint32_t *words, const void *pointer) { + const uint64_t raw = reinterpret_cast(pointer); + words[kConfigRegTableLow] = static_cast(raw); + words[kConfigRegTableHigh] = static_cast(raw >> 32); +} + +inline uint32_t StatusCoreId(uint32_t status) { + return (status >> kStatusCoreIdShift) & kStatusCoreIdMask; +} + +} // namespace pa_scheduler::ccec_pmu + +#endif // PA_SCHEDULER_CCEC_PMU_PROBE_H diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index ec7f0b9a22..e7d055e1e3 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -493,10 +493,20 @@ struct alignas(64) WorkerResult { uint64_t role; uint64_t max_occupied; uint64_t final_occupied; + + // CCEC 标量 PMU 取证复用 WorkerResult 原有的 24B 尾部 padding,不增加结果区大小。 + // 该诊断只在显式开启时有效;CNT2/CNT6/CNT7 分别对应 scalar busy、I-cache req/miss。 + uint64_t pmu_total_cycles; + uint32_t pmu_scalar_busy; + uint32_t pmu_icache_requests; + uint32_t pmu_icache_misses; + uint32_t pmu_status; }; // WorkerResult 是 standalone 尾部的诊断 sidecar,不属于真实 DistCore ABI;按 // cache line 隔离后,各 worker 发布统计不会相互覆盖或污染被测共享状态。 -static_assert(sizeof(WorkerResult) % 64 == 0, "WorkerResult must not share cache lines"); +static_assert(sizeof(WorkerResult) == 704, "WorkerResult PMU fields must only consume existing tail padding"); +static_assert(offsetof(WorkerResult, pmu_total_cycles) == 680, "WorkerResult PMU offset mismatch"); +static_assert(offsetof(WorkerResult, pmu_status) == 700, "WorkerResult PMU status offset mismatch"); // 从 cube_cursor 到 workers 结束保留关键字段 offset、DistCore ABI 和生产总字节跨度, // 并非字段级完整镜像。RunConfig、输入 context_lens 与校验结果追加在该跨度之后, diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index 69d6ed3ca7..7fc881173f 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -33,6 +33,10 @@ Benchmark options: --swimlane-json FILE --no-swimlane +CCEC-only PMU probe options (the host must be launched by msprof PipeUtilization): + --pmu-window off|empty|scalar|scalar-double + --pmu-scalar-nops N + The swimlane action performs exactly one run and writes both the raw capture and merged Perfetto JSON below this directory's outputs/ folder. It rejects --runs, --swimlane-json, and --no-swimlane because those are managed by the action. From 67407cc436d651c97a6337a31417623b5e4be534 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 02:43:30 +0000 Subject: [PATCH 009/214] =?UTF-8?q?=E6=B5=8B=E8=AF=95(a5):=20=E7=BB=86?= =?UTF-8?q?=E5=88=86PA=E4=BE=9D=E8=B5=96=E4=B8=8Efrontier=E5=8E=9F?= =?UTF-8?q?=E5=AD=90=E8=AE=A1=E6=95=B0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...05\345\206\265\345\210\206\346\236\220.md" | 78 +++++++++++++++++++ ...77\347\224\250\346\214\207\345\215\227.md" | 24 +++++- .../pa_scheduler/common/host_support.h | 55 ++++++++++++- .../pa_scheduler/common/pa_model.h | 13 +++- .../pa_scheduler/common/pa_scheduler_core.h | 12 ++- 5 files changed, 174 insertions(+), 8 deletions(-) diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index b0b20c8dd3..f18083bb15 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -803,3 +803,81 @@ O1 已达到“可用”的门槛:物理核映射、事件 selector、正向 等 worker-local 诊断计数,不增加共享 atomic。得到动态规模后,首个低风险单变量 候选是每个私有 slot 缓存已经观察为 ready 的 fanin 前缀;slot 重用时必须重置, 且先完成 CCEC 交错 A/B,再决定是否迁移到真实 FDWIC。 + +### 7.4 阶段 D1:精确分类 fanin 与 frontier 动态原子次数 + +#### 7.4.1 计数实现与闭合关系 + +D1 已在 standalone 公共调度器中增加 worker-local 软件计数,不新增共享 atomic: + +- fanin 每次 flag load 只递增一个分类:`ready` 或 `not_ready`; +- 每次 completion 递增一个 frontier initial load; +- ready flag 与紧随其后的 FetchMax 共用一个一一对应计数 `A`; +- 扫描遇到 not-ready flag 退出时递增 terminal load。 + +计数先保存在本核 `LocalStats`,kernel 结束时才发布到独占 `WorkerResult`。 +`WorkerResult` 从 704 B 扩展到 768 B,但原 PMU 字段 offset、生产 DistGlobal/DistCore +offset 和 `LocalSlot` ABI 都不变。三后端完整重建后,smoke 和 256 batch 默认 NOP +均通过全部语义断言。 + +对任一 worker,若其完成数为 `Cw`、fanin 边数为 `Ew`、失败 load 为 `Fw`,则 +逐核检查: + +~~~text +frontier_initial_w == Cw +frontier_terminal_w == Cw +fanin_ready_w >= Ew +fanin_ready_w - Ew <= 2 * Fw +~~~ + +最后一个上界来自 PA 最大 fanin 为 3:一次失败检查最多先重读两个 ready 前缀, +然后在一个 not-ready 依赖上返回。全局 `T=1280` 时还必须满足: + +~~~text +frontier_initial = frontier_terminal = T +frontier_ready = frontier_FetchMax = A >= T +frontier_flag_loads = A + T +Submit+completion ops = 79872 + G + 2A +~~~ + +其中 `G=fanin_ready+fanin_not_ready`。CCEC/AscendC 的 `A` 对应真实 A5 +atomicMax;CPU 的 FetchMax 是 load/CAS 实现,`A` 只能解释为逻辑调用数。 + +#### 7.4.2 CCEC 十轮动态基线 + +在 256 batch、默认 NOP、关闭泳道和 phase profile 的同一进程十轮基线中,全部 +语义断言和上述计数恒等式均 PASS: + +| 指标 | 中位数 | 均值 | nearest-rank p90 | 范围 | +| ---- | -----: | ---: | ----------------: | ---: | +| Submit(us) | 4776.940 | 4802.821 | 5335.931 | 3805.757~5535.473 | +| fanin 总 load `G` | 93201.5 | 99442.6 | 145045 | 56620~151260 | +| fanin ready | 7323.5 | 7418.0 | 9370 | 4191~9933 | +| fanin not-ready | 86675.5 | 92024.6 | 136347 | 49038~141327 | +| frontier FetchMax `A` | 15365 | 14957.8 | 18957 | 5587~20026 | +| Submit+completion ops | 203803.5 | 209230.2 | 264969 | 164508~269046 | + +日志位于: + +~~~text +tests/atomic_probe/pa_scheduler/outputs/atomic_diagnostics/ + ccec_baseline_10_20260718_023551.log +~~~ + +该十轮不是独立进程交错 A/B,因此只用于确认动态规模,不用于宣布性能收益。 +分类代码新增约 `2T+A` 次 worker 私有 scalar 增量,且结果 sidecar 扩大了一条 +cache line;本节绝对 Submit 不能与 D1 之前的二进制直接归因比较。 + +#### 7.4.3 对下一候选的约束 + +当前动态最大项不是 ready 前缀,而是 not-ready 重试:其中位数约 86676 次; +frontier helping 的额外 FetchMax 中位数为 `A-T=14085` 次。ready-prefix cache 在 +固定轮询序列下最多删除 `fanin_ready-E=6043.5` 次中位重复 load,约占 fanin 总量 +6.5%、Submit+completion ops 3.0%,预期只能是小幅候选。 + +仍先做该候选,因为它不改变依赖集合、flag 发布或跨核共享状态,正确性风险最低。 +但 probe 变短后可能在同一等待期间发起更多 not-ready 重试,所以整轮 `G` 不保证 +静态单调下降。保留门槛是:优化后 `fanin_ready == fanin_edges == 1280`,CCEC +独立进程交错十对中 fanin 总 load 配对中心下降、Submit 中心不回退;否则记录负 +结果并撤回,不迁移真实 FDWIC。standalone 第一版只验证当前单-lane PA Case1, +不能拿它的 PASS 代替 joint/BlockWon 覆盖。 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 858d8e87b7..434fd9c319 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -266,6 +266,28 @@ CPU 完整协议回归建议关闭大泳道缓冲区: WaitForSlot 和 HeapGuard 的 `calls_total` 会按 winner 所在角色分布,AIC/AIV 相加必须分别等于 1,024;等待事件则对应额外的 RingBp 泳道记录。 +每轮还会输出一行不依赖泳道的动态原子分类: + +```text +[ATOMIC] submit_completion_ops=... fanin_ready=... fanin_not_ready=... \ +frontier_initial=... frontier_flag=... frontier_ready_fetch_max=... frontier_terminal=... +``` + +- `fanin_ready/not_ready` 分别是依赖 flag 返回 1/0 的次数,两者之和等于 + `[METRIC] fanin_loads`; +- `frontier_initial` 是每个 completion 对 frontier 的首次 load; +- `frontier_ready_fetch_max` 同时计数 ready flag 和紧随其后的 FetchMax,两者在 + 这条控制流中一一对应;CCEC/AscendC 上它是一条真实 A5 atomicMax,CPU 上只是 + 一次逻辑 FetchMax 调用; +- `frontier_terminal` 是每次扫描最终遇到的 not-ready flag,当前工作量下应与 + completion 数相等;`frontier_flag = ready + terminal`; +- `submit_completion_ops` 覆盖 Claim、第一圈 HeapGuard、fanin、completion 发布和 + frontier,不包含 started/replay_done 生命周期屏障。 + +这些字段在每个 worker 的私有 `LocalStats` 中递增,kernel 结束时才发布到独占 +结果区,不为诊断新增共享 atomic。它们仍会增加少量 scalar 指令,因此优化 A/B +必须使用相同的计数布局;不能把启用分类后的绝对时间直接与旧二进制比较。 + ### 5.6 CCEC 每核 scalar PMU 与 I-cache 诊断 CCEC 后端提供一个显式诊断模式,用来验证局部 scalar 性能观察链路。它不读取 @@ -284,7 +306,7 @@ selector 和 PMU framework 仍由 CANN 9.1 的 task-based profiler 配置;CCEC 只做门控与读取。一次 snapshot 会消费/清除此前累计,因此窗口前先冻结并做一次 baseline read-clear,窗口中间只切 gate,末尾再做唯一一次结果 snapshot。host 按正式 A5 runtime 的方式调用 `halResMap`,构造 36 个 -物理 AICore展开后的 108 项子核 MMIO 表;kernel 使用真实 `get_coreid()` 索引, +物理 AICore 展开后的 108 项子核 MMIO 表;kernel 使用真实 `get_coreid()` 索引, 不能用逻辑 `worker_id` 猜物理核。 在本目录先构建 CCEC,再直接让 `msprof` 包住 host runner: diff --git a/tests/atomic_probe/pa_scheduler/common/host_support.h b/tests/atomic_probe/pa_scheduler/common/host_support.h index 6b6bf42556..0f5e054efe 100644 --- a/tests/atomic_probe/pa_scheduler/common/host_support.h +++ b/tests/atomic_probe/pa_scheduler/common/host_support.h @@ -581,7 +581,11 @@ inline Metrics Validate( uint64_t claims = 0; uint64_t wins = 0; uint64_t heap_guards = 0; - uint64_t fanin_loads = 0; + uint64_t fanin_ready_loads = 0; + uint64_t fanin_not_ready_loads = 0; + uint64_t frontier_initial_loads = 0; + uint64_t frontier_updates = 0; + uint64_t frontier_terminal_loads = 0; uint64_t duplicates = 0; uint64_t cas_retries = 0; uint64_t joint_polls = 0; @@ -616,6 +620,8 @@ inline Metrics Validate( bool frontend_worker_counts_ok = true; bool final_worker_state_ok = true; bool worker_checksums_ok = true; + bool fanin_worker_counts_ok = true; + bool frontier_worker_counts_ok = true; // 按真实输出大小、1 KiB 对齐和 256 MiB 环回规则重算每个 task 可接受的最小 vend。 uint64_t expected_heap_next = 0; @@ -675,7 +681,11 @@ inline Metrics Validate( if (result.claim_wins != 0) ++winning_workers; max_worker_wins = std::max(max_worker_wins, result.claim_wins); heap_guards += result.heap_guards; - fanin_loads += result.fanin_loads; + fanin_ready_loads += result.fanin_ready_loads; + fanin_not_ready_loads += result.fanin_not_ready_loads; + frontier_initial_loads += result.frontier_initial_loads; + frontier_updates += result.frontier_updates; + frontier_terminal_loads += result.frontier_terminal_loads; duplicates += result.completion_duplicates; cas_retries += result.cas_retries; joint_polls += result.joint_polls; @@ -706,6 +716,17 @@ inline Metrics Validate( final_worker_state_ok &= result.map_alive_floor == expected_map_floor; final_worker_state_ok &= result.map_cleaned_upto == expected_map_floor; worker_checksums_ok &= result.checksum == (0xcbf29ce484222325ULL ^ result.worker_id); + const uint64_t worker_kernel_completions = result.kernel_counts[0] + result.kernel_counts[1] + + result.kernel_counts[2] + result.kernel_counts[3]; + const uint64_t worker_completions = result.wins[0] + worker_kernel_completions; + frontier_worker_counts_ok &= result.frontier_initial_loads == worker_completions; + frontier_worker_counts_ok &= result.frontier_terminal_loads == result.frontier_initial_loads; + fanin_worker_counts_ok &= result.fanin_ready_loads >= result.fanin_edges; + if (result.fanin_ready_loads >= result.fanin_edges) { + // PA 最大 fanin 为 3;每次失败检查最多先重读两个 ready 前缀,再遇到一个 not-ready。 + fanin_worker_counts_ok &= + result.fanin_ready_loads - result.fanin_edges <= 2 * result.fanin_not_ready_loads; + } for (uint32_t kind = 0; kind < 5; ++kind) wins_by_kind[kind] += result.wins[kind]; for (uint32_t kind = 0; kind < 4; ++kind) { @@ -734,6 +755,8 @@ inline Metrics Validate( } const uint64_t kernel_total = kernel_counts[0] + kernel_counts[1] + kernel_counts[2] + kernel_counts[3]; const uint64_t placement_total = placements[0] + placements[1] + placements[2]; + const uint64_t fanin_loads = fanin_ready_loads + fanin_not_ready_loads; + const uint64_t frontier_flag_loads = frontier_updates + frontier_terminal_loads; // 第一组断言覆盖参与者拓扑、Claim/winner、completion 和最终 drain 等调度主协议。 Expect(aic_count == kAicWorkers && aiv_count == kAivWorkers, "participant topology is 32 AIC + 64 AIV", &metrics); @@ -755,7 +778,19 @@ inline Metrics Validate( "each kernel kind executes once per batch", &metrics ); Expect(heap_guards == static_cast(batches) * 4, "heap guard count matches output winners", &metrics); - Expect(fanin_loads >= static_cast(batches) * 5, "successful fanin checks meet PA lower bound", &metrics); + Expect( + fanin_worker_counts_ok && fanin_ready_loads >= fanin_edges && + fanin_ready_loads - fanin_edges <= 2 * fanin_not_ready_loads, + "fanin ready/failure load classification is complete", &metrics + ); + Expect( + frontier_worker_counts_ok && frontier_initial_loads == task_count, + "frontier initial loads match completed tasks", &metrics + ); + Expect( + frontier_terminal_loads == task_count && frontier_updates >= task_count, + "frontier ready/update/terminal load identity is exact", &metrics + ); Expect(duplicates == 0, "completion flags are published once", &metrics); Expect(ready_flags == task_count, "all task flags are ready", &metrics); Expect(vend_values_ok, "all published vend values are nonzero and aligned", &metrics); @@ -872,6 +907,20 @@ inline Metrics Validate( metrics.submit_span_us, host_us, static_cast(claims), static_cast(fanin_loads), static_cast(cas_retries) ); + const uint64_t submit_completion_ops = + claims + heap_guards + fanin_loads + 2ULL * task_count + frontier_initial_loads + + frontier_flag_loads + frontier_updates; + std::printf( + "[ATOMIC] submit_completion_ops=%llu fanin_ready=%llu fanin_not_ready=%llu frontier_initial=%llu " + "frontier_flag=%llu frontier_ready_fetch_max=%llu frontier_terminal=%llu\n", + static_cast(submit_completion_ops), + static_cast(fanin_ready_loads), + static_cast(fanin_not_ready_loads), + static_cast(frontier_initial_loads), + static_cast(frontier_flag_loads), + static_cast(frontier_updates), + static_cast(frontier_terminal_loads) + ); std::printf( "[WINNERS] active_workers=%u max_wins_per_worker=%llu\n", winning_workers, static_cast(max_worker_wins) diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index e7d055e1e3..93f87bca30 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -67,6 +67,7 @@ constexpr size_t kRealReplayDoneOffset = 10043776; constexpr size_t kRealStartedCountOffset = 10043840; constexpr uint32_t kTraceRecordsPerCore = 1U << 16; static_assert((kPayloadSlots & kPayloadMask) == 0, "payload slots must be a power of two"); +static_assert(kMaxTasks < kTaskCellCapacity, "every frontier scan must terminate on an in-range not-ready flag"); // These are the measured means from the best PA A5 trace, in 1 GHz ticks. // The CCEC stage calibrates the NOP counts against these targets before the @@ -449,7 +450,7 @@ struct alignas(64) WorkerResult { uint64_t claim_attempts; uint64_t claim_wins; uint64_t heap_guards; - uint64_t fanin_loads; + uint64_t fanin_ready_loads; uint64_t completion_duplicates; uint64_t cas_retries; uint64_t joint_polls; @@ -501,12 +502,20 @@ struct alignas(64) WorkerResult { uint32_t pmu_icache_requests; uint32_t pmu_icache_misses; uint32_t pmu_status; + + // 这些计数只在 worker 私有 LocalStats 中递增,结束时一次性发布;它们把动态 + // fanin 重试和 frontier helping 展开为准确次数,不为取数再增加共享 atomic。 + uint64_t fanin_not_ready_loads; + uint64_t frontier_initial_loads; + uint64_t frontier_updates; + uint64_t frontier_terminal_loads; }; // WorkerResult 是 standalone 尾部的诊断 sidecar,不属于真实 DistCore ABI;按 // cache line 隔离后,各 worker 发布统计不会相互覆盖或污染被测共享状态。 -static_assert(sizeof(WorkerResult) == 704, "WorkerResult PMU fields must only consume existing tail padding"); +static_assert(sizeof(WorkerResult) == 768, "WorkerResult diagnostics must occupy whole cache lines"); static_assert(offsetof(WorkerResult, pmu_total_cycles) == 680, "WorkerResult PMU offset mismatch"); static_assert(offsetof(WorkerResult, pmu_status) == 700, "WorkerResult PMU status offset mismatch"); +static_assert(offsetof(WorkerResult, fanin_not_ready_loads) == 704, "WorkerResult atomic diagnostic offset mismatch"); // 从 cube_cursor 到 workers 结束保留关键字段 offset、DistCore ABI 和生产总字节跨度, // 并非字段级完整镜像。RunConfig、输入 context_lens 与校验结果追加在该跨度之后, diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index 670731c5ad..64672629ec 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -100,6 +100,7 @@ template PA_DEVICE void AdvanceFrontier(PA_GM SchedulerState *state, LocalStats &stats) { // frontier 只表示“从 task 0 开始已经连续完成”的最高 task id,不能越过尚未发布 flag 的空洞。 // 多个完成者可以同时扫描同一段连续区间,FetchMax 保证共享 frontier 只前进、不回退。 + ++stats.result.frontier_initial_loads; int64_t frontier = LoadLine(state->frontier); while (true) { const int64_t next = frontier + 1; @@ -107,10 +108,12 @@ PA_DEVICE void AdvanceFrontier(PA_GM SchedulerState *state, LocalStats &stats) { break; } if (Ops::Load(&state->tasks[next].flag) == 0) { + ++stats.result.frontier_terminal_loads; break; } uint64_t retries = 0; // FetchMax 返回更新前的值;若其他核已经走得更远,就从其 old 值继续扫描,避免重复从 next 起步。 + ++stats.result.frontier_updates; const int64_t old = Ops::FetchMax(&state->frontier.value, next, retries); stats.result.cas_retries += retries; frontier = old > next ? old : next; @@ -133,10 +136,11 @@ template PA_DEVICE bool SlotReady(PA_GM SchedulerState *state, PA_GM LocalSlot &slot, LocalStats &stats) { // 每个 fanin flag 都是跨核共享的完成条件;遇到第一个未就绪依赖即返回,后续 drain 会再次轮询。 for (uint32_t index = 0; index < slot.fanin_count; ++index) { - ++stats.result.fanin_loads; if (Ops::Load(&state->tasks[slot.fanin[index]].flag) == 0) { + ++stats.result.fanin_not_ready_loads; return false; } + ++stats.result.fanin_ready_loads; } return true; } @@ -583,7 +587,7 @@ PA_DEVICE void PublishResult(PA_GM WorkerResult &destination, const WorkerResult PA_PUBLISH_FIELD(claim_attempts); PA_PUBLISH_FIELD(claim_wins); PA_PUBLISH_FIELD(heap_guards); - PA_PUBLISH_FIELD(fanin_loads); + PA_PUBLISH_FIELD(fanin_ready_loads); PA_PUBLISH_FIELD(completion_duplicates); PA_PUBLISH_FIELD(cas_retries); PA_PUBLISH_FIELD(joint_polls); @@ -628,6 +632,10 @@ PA_DEVICE void PublishResult(PA_GM WorkerResult &destination, const WorkerResult PA_PUBLISH_FIELD(role); PA_PUBLISH_FIELD(max_occupied); PA_PUBLISH_FIELD(final_occupied); + PA_PUBLISH_FIELD(fanin_not_ready_loads); + PA_PUBLISH_FIELD(frontier_initial_loads); + PA_PUBLISH_FIELD(frontier_updates); + PA_PUBLISH_FIELD(frontier_terminal_loads); #undef PA_PUBLISH_FIELD Ops::StoreBarrier(); } From 8deefdefa4920354670c639844ecaaf84ad28ee8 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 07:31:41 +0000 Subject: [PATCH 010/214] =?UTF-8?q?=E4=BF=AE=E5=A4=8DPMU=20owner=E6=B4=BB?= =?UTF-8?q?=E8=B7=83=E5=AD=90=E6=A0=B8=E9=85=8D=E7=BD=AE=E4=B8=8E=E6=81=A2?= =?UTF-8?q?=E5=A4=8D=E9=97=AD=E7=8E=AF?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../ccec/atomic_scalar_pmu_host.cpp | 59 +++- tests/atomic_probe/ccec/pmu_probe_aicpu.cpp | 277 +++++++++++++----- tests/atomic_probe/ccec/pmu_probe_control.h | 76 ++++- .../ccec/pmu_probe_host_support.h | 93 +++++- 4 files changed, 414 insertions(+), 91 deletions(-) diff --git a/tests/atomic_probe/ccec/atomic_scalar_pmu_host.cpp b/tests/atomic_probe/ccec/atomic_scalar_pmu_host.cpp index 4ce419b1f6..8c44ac4b66 100644 --- a/tests/atomic_probe/ccec/atomic_scalar_pmu_host.cpp +++ b/tests/atomic_probe/ccec/atomic_scalar_pmu_host.cpp @@ -18,7 +18,7 @@ // 中的哪一项,而不混入多核竞争、轮询次数变化或未消费返回值的并行发射。 #include "atomic_scalar_pmu_shared.h" -#include "pmu_probe_control.h" +#include "pmu_probe_host_support.h" #include "../probe_host.h" #include "aicpu_loader/host/load_aicpu_op.h" @@ -183,16 +183,45 @@ bool RunPmuCommand( )) { return false; } + const uint32_t bitmap_count = atomic_probe::pmu::CountPmuConfiguredSubcores(*control); const bool expected_state = command == atomic_probe::pmu::PmuCommand::Configure - ? control->configured == 1 && control->processed_subcores == atomic_probe::pmu::kPmuPhysicalSubcores - : control->configured == 0 && control->processed_subcores == 0; + ? control->configured == 1 && control->processed_subcores == control->expected_subcores && + bitmap_count == control->expected_subcores && + control->skipped_subcores + bitmap_count == atomic_probe::pmu::kPmuPhysicalSubcores + : control->configured == 0 && control->processed_subcores == 0 && bitmap_count == 0; if (control->status != 0 || !expected_state) { + const auto failed_field = + static_cast(control->first_failed_field); std::fprintf( - stderr, "PMU helper failed: command=%u status=%d configured=%u processed=%u\n", control->command, - static_cast(control->status), control->configured, control->processed_subcores + stderr, + "PMU helper failed: command=%u status=%d configured=%u processed=%u bitmap_count=%u " + "expected_subcores=%u skipped=%u failed_index=%u failed_field=%s(%u) " + "observed=0x%x expected=0x%x\n", + control->command, static_cast(control->status), control->configured, + control->processed_subcores, bitmap_count, control->expected_subcores, + control->skipped_subcores, control->first_failed_index, + atomic_probe::pmu::PmuRegisterFieldName(failed_field), control->first_failed_field, + control->first_failed_observed, control->first_failed_expected ); return false; } + if (command == atomic_probe::pmu::PmuCommand::Configure) { + const auto failed_field = + static_cast(control->first_failed_field); + std::printf( + "[PMU_OWNER] configured=%u bitmap_count=%u expected=%u skipped=%u " + "first_skipped_index=%u first_skipped_field=%s(%u) observed=0x%x expected_value=0x%x\n", + control->processed_subcores, bitmap_count, control->expected_subcores, + control->skipped_subcores, control->first_failed_index, + atomic_probe::pmu::PmuRegisterFieldName(failed_field), control->first_failed_field, + control->first_failed_observed, control->first_failed_expected + ); + std::printf( + "[PMU_OWNER] configured_bitmap=%08x:%08x:%08x:%08x\n", + control->configured_bitmap[3], control->configured_bitmap[2], + control->configured_bitmap[1], control->configured_bitmap[0] + ); + } return true; } @@ -282,7 +311,8 @@ struct Sample { }; bool ValidateSample( - const Sample &sample, atomic_scalar_pmu::Mode mode, uint32_t rounds, uint64_t seed, std::string *reason + const Sample &sample, atomic_scalar_pmu::Mode mode, uint32_t rounds, uint64_t seed, + const atomic_probe::pmu::PmuControl &pmu_control, std::string *reason ) { const Oracle oracle = Simulate(seed, rounds); const uint64_t expected_checksum = mode == atomic_scalar_pmu::Mode::Empty ? 0 : oracle.checksum; @@ -299,6 +329,12 @@ bool ValidateSample( *reason = "physical-core-id"; return false; } + if (!atomic_probe::pmu::IsPmuSubcoreConfigured( + pmu_control, static_cast(sample.result.physical_core_id) + )) { + *reason = "physical-core-not-in-configured-bitmap"; + return false; + } if ((sample.result.pmu_ctrl_after_stop & 1ULL) != 0) { *reason = "pmu-gate-still-enabled"; return false; @@ -316,7 +352,8 @@ bool ValidateSample( bool RunOne( aclrtFuncHandle function, aclrtStream stream, void *state_device, uint64_t pmu_register_bases, - atomic_scalar_pmu::Mode mode, uint32_t rounds, uint32_t repeat, uint64_t seed, Sample *sample + atomic_scalar_pmu::Mode mode, uint32_t rounds, uint32_t repeat, uint64_t seed, + const atomic_probe::pmu::PmuControl &pmu_control, Sample *sample ) { atomic_scalar_pmu::ProbeState state{}; state.control.pmu_register_bases = pmu_register_bases; @@ -352,7 +389,7 @@ bool RunOne( sample->result = state.result; sample->final_value = state.target.value; std::string reason; - const bool semantic_ok = ValidateSample(*sample, mode, rounds, seed, &reason); + const bool semantic_ok = ValidateSample(*sample, mode, rounds, seed, pmu_control, &reason); std::printf( "[RAW] repeat=%u rounds=%u mode=%s sys_cycles=%llu total=%llu scalar=%llu " "icache_req=%llu icache_miss=%llu checksum=%llu final=%llu physical=%llu ctrl=0x%llx status=%s%s%s\n", @@ -485,6 +522,8 @@ int main(int argc, char **argv) { } aclrtStream stream = nullptr; if (!CheckAcl(aclrtCreateStream(&stream), "aclrtCreateStream")) return EXIT_FAILURE; + atomic_probe::pmu::ActiveSubcoreLimits active_limits; + if (!atomic_probe::pmu::QueryActiveSubcoreLimits(stream, &active_limits)) return EXIT_FAILURE; aclrtBinHandle binary_handle = nullptr; if (!CheckAcl( @@ -528,7 +567,7 @@ int main(int argc, char **argv) { atomic_probe::pmu::PmuControl pmu_control{}; pmu_control.magic = atomic_probe::pmu::kPmuControlMagic; pmu_control.version = atomic_probe::pmu::kPmuControlVersion; - pmu_control.expected_subcores = atomic_probe::pmu::kPmuPhysicalSubcores; + pmu_control.expected_subcores = active_limits.total; if (!CheckAcl( aclrtMemcpy( pmu_control_device, sizeof(pmu_control), &pmu_control, sizeof(pmu_control), @@ -581,7 +620,7 @@ int main(int argc, char **argv) { const auto mode = static_cast(mode_index); const bool passed = RunOne( function, stream, state_device, reinterpret_cast(pmu_regs_device), mode, rounds, - repeat, seed, &sample + repeat, seed, pmu_control, &sample ); all_passed &= passed; samples[mode_index].push_back(sample); diff --git a/tests/atomic_probe/ccec/pmu_probe_aicpu.cpp b/tests/atomic_probe/ccec/pmu_probe_aicpu.cpp index 1d36c00384..6f01a985bb 100644 --- a/tests/atomic_probe/ccec/pmu_probe_aicpu.cpp +++ b/tests/atomic_probe/ccec/pmu_probe_aicpu.cpp @@ -20,6 +20,7 @@ namespace { using atomic_probe::pmu::PmuControl; +using atomic_probe::pmu::PmuRegisterField; void FlushControl(const PmuControl *control) { @@ -32,30 +33,104 @@ void FlushControl(const PmuControl *control) __asm__ volatile("isb" ::: "memory"); } -bool SavedConfigurationMatches(const PmuControl *control, uint64_t base, uint32_t index) +void ResetFailureDiagnostic(PmuControl *control) { - return read_reg(base, RegId::PMU_CTRL_0) == control->saved_ctrl0[index] && - read_reg(base, RegId::PMU_CTRL_1) == control->saved_ctrl1[index] && - read_reg(base, RegId::PMU_CNT0_IDX) == control->saved_selector0[index] && - read_reg(base, RegId::PMU_CNT1_IDX) == control->saved_selector1[index] && - read_reg(base, RegId::PMU_CNT2_IDX) == control->saved_selector2[index] && - read_reg(base, RegId::PMU_START_CYC0) == control->saved_start_cycle_low[index] && - read_reg(base, RegId::PMU_START_CYC1) == control->saved_start_cycle_high[index] && - read_reg(base, RegId::PMU_STOP_CYC0) == control->saved_stop_cycle_low[index] && - read_reg(base, RegId::PMU_STOP_CYC1) == control->saved_stop_cycle_high[index]; + control->first_failed_index = atomic_probe::pmu::kPmuDiagnosticUnset; + control->first_failed_field = static_cast(PmuRegisterField::None); + control->first_failed_observed = 0; + control->first_failed_expected = 0; } -bool ProbeConfigurationMatches(uint64_t base) +void RecordFirstFailure( + PmuControl *control, uint32_t index, PmuRegisterField field, uint32_t observed, uint32_t expected +) { - return read_reg(base, RegId::PMU_CTRL_0) == REG_MMIO_PMU_CTRL_0_ENABLE_VAL && - read_reg(base, RegId::PMU_CTRL_1) == REG_MMIO_PMU_CTRL_1_ENABLE_VAL && - read_reg(base, RegId::PMU_CNT0_IDX) == 0x1U && read_reg(base, RegId::PMU_CNT1_IDX) == 0x34U && - read_reg(base, RegId::PMU_CNT2_IDX) == 0x35U && read_reg(base, RegId::PMU_START_CYC0) == 0 && - read_reg(base, RegId::PMU_START_CYC1) == 0 && read_reg(base, RegId::PMU_STOP_CYC0) == 0xffffffffU && - read_reg(base, RegId::PMU_STOP_CYC1) == 0xffffffffU; + if (control->first_failed_index != atomic_probe::pmu::kPmuDiagnosticUnset) return; + control->first_failed_index = index; + control->first_failed_field = static_cast(field); + control->first_failed_observed = observed; + control->first_failed_expected = expected; } -bool RestoreOne(PmuControl *control, uint64_t base, uint32_t index) +bool CheckProbeRegister( + uint64_t base, RegId reg, uint32_t expected, PmuRegisterField field, + PmuRegisterField *failed_field, uint32_t *observed +) +{ + const uint32_t actual = static_cast(read_reg(base, reg)); + if (actual == expected) return true; + *failed_field = field; + *observed = actual; + return false; +} + +bool ProbeConfigurationMatches( + uint64_t base, PmuRegisterField *failed_field, uint32_t *observed, uint32_t *expected +) +{ + struct RegisterExpectation { + RegId reg; + uint32_t value; + PmuRegisterField field; + }; + const RegisterExpectation expectations[] = { + {RegId::PMU_CTRL_0, REG_MMIO_PMU_CTRL_0_ENABLE_VAL, PmuRegisterField::Ctrl0}, + {RegId::PMU_CTRL_1, REG_MMIO_PMU_CTRL_1_ENABLE_VAL, PmuRegisterField::Ctrl1}, + {RegId::PMU_CNT0_IDX, 0x1U, PmuRegisterField::Selector0}, + {RegId::PMU_CNT1_IDX, 0x34U, PmuRegisterField::Selector1}, + {RegId::PMU_CNT2_IDX, 0x35U, PmuRegisterField::Selector2}, + {RegId::PMU_START_CYC0, 0U, PmuRegisterField::StartCycleLow}, + {RegId::PMU_START_CYC1, 0U, PmuRegisterField::StartCycleHigh}, + {RegId::PMU_STOP_CYC0, 0xffffffffU, PmuRegisterField::StopCycleLow}, + {RegId::PMU_STOP_CYC1, 0xffffffffU, PmuRegisterField::StopCycleHigh}, + }; + for (const RegisterExpectation &expectation : expectations) { + if (!CheckProbeRegister( + base, expectation.reg, expectation.value, expectation.field, failed_field, observed + )) { + *expected = expectation.value; + return false; + } + } + return true; +} + +bool SavedConfigurationMatches( + const PmuControl *control, uint64_t base, uint32_t index, + PmuRegisterField *failed_field, uint32_t *observed, uint32_t *expected +) +{ + struct RegisterExpectation { + RegId reg; + uint32_t value; + PmuRegisterField field; + }; + const RegisterExpectation expectations[] = { + {RegId::PMU_CTRL_0, control->saved_ctrl0[index], PmuRegisterField::Ctrl0}, + {RegId::PMU_CTRL_1, control->saved_ctrl1[index], PmuRegisterField::Ctrl1}, + {RegId::PMU_CNT0_IDX, control->saved_selector0[index], PmuRegisterField::Selector0}, + {RegId::PMU_CNT1_IDX, control->saved_selector1[index], PmuRegisterField::Selector1}, + {RegId::PMU_CNT2_IDX, control->saved_selector2[index], PmuRegisterField::Selector2}, + {RegId::PMU_START_CYC0, control->saved_start_cycle_low[index], PmuRegisterField::StartCycleLow}, + {RegId::PMU_START_CYC1, control->saved_start_cycle_high[index], PmuRegisterField::StartCycleHigh}, + {RegId::PMU_STOP_CYC0, control->saved_stop_cycle_low[index], PmuRegisterField::StopCycleLow}, + {RegId::PMU_STOP_CYC1, control->saved_stop_cycle_high[index], PmuRegisterField::StopCycleHigh}, + }; + for (const RegisterExpectation &expectation : expectations) { + if (!CheckProbeRegister( + base, expectation.reg, expectation.value, expectation.field, failed_field, observed + )) { + *expected = expectation.value; + return false; + } + } + return true; +} + +bool RestoreOne( + PmuControl *control, uint64_t base, uint32_t index, + PmuRegisterField *failed_field, uint32_t *observed, uint32_t *expected +) { write_reg(base, RegId::PMU_CTRL_0, 0); write_reg(base, RegId::PMU_CTRL_1, 0); @@ -68,62 +143,119 @@ bool RestoreOne(PmuControl *control, uint64_t base, uint32_t index) write_reg(base, RegId::PMU_STOP_CYC1, control->saved_stop_cycle_high[index]); write_reg(base, RegId::PMU_CTRL_0, control->saved_ctrl0[index]); write_reg(base, RegId::PMU_CTRL_1, control->saved_ctrl1[index]); - return SavedConfigurationMatches(control, base, index); + return SavedConfigurationMatches(control, base, index, failed_field, observed, expected); +} + +void SaveOne(PmuControl *control, uint64_t base, uint32_t index) +{ + control->saved_ctrl0[index] = static_cast(read_reg(base, RegId::PMU_CTRL_0)); + control->saved_ctrl1[index] = static_cast(read_reg(base, RegId::PMU_CTRL_1)); + control->saved_selector0[index] = static_cast(read_reg(base, RegId::PMU_CNT0_IDX)); + control->saved_selector1[index] = static_cast(read_reg(base, RegId::PMU_CNT1_IDX)); + control->saved_selector2[index] = static_cast(read_reg(base, RegId::PMU_CNT2_IDX)); + control->saved_start_cycle_low[index] = static_cast(read_reg(base, RegId::PMU_START_CYC0)); + control->saved_start_cycle_high[index] = static_cast(read_reg(base, RegId::PMU_START_CYC1)); + control->saved_stop_cycle_low[index] = static_cast(read_reg(base, RegId::PMU_STOP_CYC0)); + control->saved_stop_cycle_high[index] = static_cast(read_reg(base, RegId::PMU_STOP_CYC1)); +} + +void ConfigureOne(uint64_t base) +{ + // 先冻结框架,再配置 Custom 三事件及完整计数周期;最后启用 + // GLB_PMU_EN | USER_PMU_MODE_EN | SAMPLE_PMU_MODE_EN。 + write_reg(base, RegId::PMU_CTRL_0, 0); + write_reg(base, RegId::PMU_CTRL_1, 0); + write_reg(base, RegId::PMU_CNT0_IDX, 0x1U); + write_reg(base, RegId::PMU_CNT1_IDX, 0x34U); + write_reg(base, RegId::PMU_CNT2_IDX, 0x35U); + for (int counter = 0; counter < 10; ++counter) { + (void)read_reg(base, reg_index(RegId::PMU_CNT0, counter)); + } + (void)read_reg(base, RegId::PMU_CNT_TOTAL0); + (void)read_reg(base, RegId::PMU_CNT_TOTAL1); + write_reg(base, RegId::PMU_START_CYC0, 0); + write_reg(base, RegId::PMU_START_CYC1, 0); + write_reg(base, RegId::PMU_STOP_CYC0, 0xffffffffU); + write_reg(base, RegId::PMU_STOP_CYC1, 0xffffffffU); + write_reg(base, RegId::PMU_CTRL_0, REG_MMIO_PMU_CTRL_0_ENABLE_VAL); + write_reg(base, RegId::PMU_CTRL_1, REG_MMIO_PMU_CTRL_1_ENABLE_VAL); +} + +bool RestoreConfiguredBitmap(PmuControl *control, const uint64_t *register_bases) +{ + bool all_restored = true; + for (uint32_t next = atomic_probe::pmu::kPmuPhysicalSubcores; next != 0; --next) { + const uint32_t index = next - 1; + if (!atomic_probe::pmu::IsPmuSubcoreConfigured(*control, index)) continue; + PmuRegisterField failed_field = PmuRegisterField::None; + uint32_t observed = 0; + uint32_t expected = 0; + if (RestoreOne( + control, register_bases[index], index, &failed_field, &observed, &expected + )) { + atomic_probe::pmu::ClearPmuSubcoreConfigured(control, index); + if (control->processed_subcores != 0) --control->processed_subcores; + } else { + RecordFirstFailure(control, index, failed_field, observed, expected); + all_restored = false; + } + } + return all_restored && control->processed_subcores == 0 && + atomic_probe::pmu::CountPmuConfiguredSubcores(*control) == 0; } int Configure(PmuControl *control, const uint64_t *register_bases) { if (control->configured != 0) return -10; control->processed_subcores = 0; + control->skipped_subcores = 0; + for (uint32_t word = 0; word < atomic_probe::pmu::kPmuBitmapWords; ++word) { + control->configured_bitmap[word] = 0; + } + ResetFailureDiagnostic(control); for (uint32_t index = 0; index < atomic_probe::pmu::kPmuPhysicalSubcores; ++index) { const uint64_t base = register_bases[index]; if (base == 0) { - while (control->processed_subcores != 0) { - --control->processed_subcores; - const uint32_t rollback = control->processed_subcores; - (void)RestoreOne(control, register_bases[rollback], rollback); - } - return -11; + RecordFirstFailure(control, index, PmuRegisterField::RegisterBase, 0, 1); + ++control->skipped_subcores; + continue; } - control->saved_ctrl0[index] = static_cast(read_reg(base, RegId::PMU_CTRL_0)); - control->saved_ctrl1[index] = static_cast(read_reg(base, RegId::PMU_CTRL_1)); - control->saved_selector0[index] = static_cast(read_reg(base, RegId::PMU_CNT0_IDX)); - control->saved_selector1[index] = static_cast(read_reg(base, RegId::PMU_CNT1_IDX)); - control->saved_selector2[index] = static_cast(read_reg(base, RegId::PMU_CNT2_IDX)); - control->saved_start_cycle_low[index] = static_cast(read_reg(base, RegId::PMU_START_CYC0)); - control->saved_start_cycle_high[index] = static_cast(read_reg(base, RegId::PMU_START_CYC1)); - control->saved_stop_cycle_low[index] = static_cast(read_reg(base, RegId::PMU_STOP_CYC0)); - control->saved_stop_cycle_high[index] = static_cast(read_reg(base, RegId::PMU_STOP_CYC1)); - - // 先冻结框架,再配置 Custom 三事件及完整计数周期;最后启用 - // GLB_PMU_EN | USER_PMU_MODE_EN | SAMPLE_PMU_MODE_EN。 - write_reg(base, RegId::PMU_CTRL_0, 0); - write_reg(base, RegId::PMU_CTRL_1, 0); - write_reg(base, RegId::PMU_CNT0_IDX, 0x1U); - write_reg(base, RegId::PMU_CNT1_IDX, 0x34U); - write_reg(base, RegId::PMU_CNT2_IDX, 0x35U); - for (int counter = 0; counter < 10; ++counter) { - (void)read_reg(base, reg_index(RegId::PMU_CNT0, counter)); - } - (void)read_reg(base, RegId::PMU_CNT_TOTAL0); - (void)read_reg(base, RegId::PMU_CNT_TOTAL1); - write_reg(base, RegId::PMU_START_CYC0, 0); - write_reg(base, RegId::PMU_START_CYC1, 0); - write_reg(base, RegId::PMU_STOP_CYC0, 0xffffffffU); - write_reg(base, RegId::PMU_STOP_CYC1, 0xffffffffU); - write_reg(base, RegId::PMU_CTRL_0, REG_MMIO_PMU_CTRL_0_ENABLE_VAL); - write_reg(base, RegId::PMU_CTRL_1, REG_MMIO_PMU_CTRL_1_ENABLE_VAL); - if (!ProbeConfigurationMatches(base)) { - (void)RestoreOne(control, base, index); - while (control->processed_subcores != 0) { - --control->processed_subcores; - const uint32_t rollback = control->processed_subcores; - (void)RestoreOne(control, register_bases[rollback], rollback); + SaveOne(control, base, index); + ConfigureOne(base); + PmuRegisterField failed_field = PmuRegisterField::None; + uint32_t observed = 0; + uint32_t expected = 0; + if (!ProbeConfigurationMatches(base, &failed_field, &observed, &expected)) { + RecordFirstFailure(control, index, failed_field, observed, expected); + ++control->skipped_subcores; + PmuRegisterField restore_failed_field = PmuRegisterField::None; + uint32_t restore_observed = 0; + uint32_t restore_expected = 0; + if (!RestoreOne( + control, base, index, &restore_failed_field, &restore_observed, &restore_expected + )) { + // 该项没有进入成功 bitmap,无法在后续 Restore 命令中重试; + // 先回滚此前成功项,再用独立状态区分“探测失败且现场恢复失败”。 + const bool rollback_ok = RestoreConfiguredBitmap(control, register_bases); + control->configured = control->processed_subcores == 0 ? 0U : 1U; + return rollback_ok ? -13 : -14; } - return -12; + continue; } - control->processed_subcores = index + 1; + atomic_probe::pmu::SetPmuSubcoreConfigured(control, index); + ++control->processed_subcores; + } + + if (control->processed_subcores != control->expected_subcores || + atomic_probe::pmu::CountPmuConfiguredSubcores(*control) != control->expected_subcores) { + RecordFirstFailure( + control, atomic_probe::pmu::kPmuPhysicalSubcores, PmuRegisterField::ConfiguredCount, + control->processed_subcores, control->expected_subcores + ); + const bool rollback_ok = RestoreConfiguredBitmap(control, register_bases); + control->configured = control->processed_subcores == 0 ? 0U : 1U; + return rollback_ok ? -12 : -14; } control->configured = 1; return 0; @@ -131,16 +263,13 @@ int Configure(PmuControl *control, const uint64_t *register_bases) int Restore(PmuControl *control, const uint64_t *register_bases) { - if (control->configured == 0 || control->processed_subcores != atomic_probe::pmu::kPmuPhysicalSubcores) { + if (control->configured == 0 || control->processed_subcores == 0 || + control->processed_subcores != atomic_probe::pmu::CountPmuConfiguredSubcores(*control)) { return -20; } - bool all_restored = true; - while (control->processed_subcores != 0) { - --control->processed_subcores; - const uint32_t index = control->processed_subcores; - all_restored &= RestoreOne(control, register_bases[index], index); - } - control->configured = 0; + ResetFailureDiagnostic(control); + const bool all_restored = RestoreConfiguredBitmap(control, register_bases); + control->configured = all_restored ? 0U : 1U; return all_restored ? 0 : -21; } @@ -160,10 +289,11 @@ extern "C" __attribute__((visibility("default"))) int simpler_aicpu_exec(void *a control->status = atomic_probe::pmu::kPmuStatusPending; if (control->magic != atomic_probe::pmu::kPmuControlMagic || control->version != atomic_probe::pmu::kPmuControlVersion || - control->expected_subcores != atomic_probe::pmu::kPmuPhysicalSubcores) { + control->expected_subcores == 0 || + control->expected_subcores > atomic_probe::pmu::kPmuPhysicalSubcores) { control->status = -3; FlushControl(control); - return -3; + return 0; } int status = -4; @@ -175,5 +305,8 @@ extern "C" __attribute__((visibility("default"))) int simpler_aicpu_exec(void *a } control->status = status; FlushControl(control); - return status; + // AICPU entry 的非零返回会被 runtime 升格为 stream 异常,host 从而无法 + // D2H 读取上面的精确状态。协议级成败统一由 control->status 传递, + // entry 只报告“命令已执行并已发布状态”。 + return 0; } diff --git a/tests/atomic_probe/ccec/pmu_probe_control.h b/tests/atomic_probe/ccec/pmu_probe_control.h index 62f8406af0..24258870c9 100644 --- a/tests/atomic_probe/ccec/pmu_probe_control.h +++ b/tests/atomic_probe/ccec/pmu_probe_control.h @@ -18,9 +18,26 @@ namespace atomic_probe::pmu { constexpr uint32_t kPmuControlMagic = 0x504d5551U; // "PMUQ" -constexpr uint32_t kPmuControlVersion = 1; +constexpr uint32_t kPmuControlVersion = 2; constexpr uint32_t kPmuPhysicalSubcores = 108; +constexpr uint32_t kPmuBitmapWords = (kPmuPhysicalSubcores + 31U) / 32U; constexpr int32_t kPmuStatusPending = 0x7fffffff; +constexpr uint32_t kPmuDiagnosticUnset = 0xffffffffU; + +enum class PmuRegisterField : uint32_t { + None = 0, + RegisterBase, + Ctrl0, + Ctrl1, + Selector0, + Selector1, + Selector2, + StartCycleLow, + StartCycleHigh, + StopCycleLow, + StopCycleHigh, + ConfiguredCount, +}; enum class PmuCommand : uint32_t { Configure = 1, @@ -36,9 +53,16 @@ struct alignas(64) PmuControl { uint32_t command; volatile int32_t status; uint32_t configured; + // v2 中这是 bitmap 内成功配置的数量,不再表示 0..N 连续前缀。 uint32_t processed_subcores; + // 由 host 从同一 stream 的 cube/vector resource limit 求和后填入。 uint32_t expected_subcores; - uint32_t reserved[9]; + uint32_t configured_bitmap[kPmuBitmapWords]; + uint32_t first_failed_index; + uint32_t first_failed_field; + uint32_t first_failed_observed; + uint32_t first_failed_expected; + uint32_t skipped_subcores; uint32_t saved_ctrl0[kPmuPhysicalSubcores]; uint32_t saved_ctrl1[kPmuPhysicalSubcores]; @@ -54,6 +78,54 @@ struct alignas(64) PmuControl { static_assert(offsetof(PmuControl, saved_ctrl0) == 64, "PMU control header must occupy one cache line"); static_assert(sizeof(PmuControl) % 64 == 0, "PMU control must use complete cache lines"); +inline bool IsPmuSubcoreConfigured(const PmuControl &control, uint32_t index) +{ + return index < kPmuPhysicalSubcores && + (control.configured_bitmap[index / 32U] & (1U << (index % 32U))) != 0; +} + +inline void SetPmuSubcoreConfigured(PmuControl *control, uint32_t index) +{ + if (index < kPmuPhysicalSubcores) { + control->configured_bitmap[index / 32U] |= 1U << (index % 32U); + } +} + +inline void ClearPmuSubcoreConfigured(PmuControl *control, uint32_t index) +{ + if (index < kPmuPhysicalSubcores) { + control->configured_bitmap[index / 32U] &= ~(1U << (index % 32U)); + } +} + +inline uint32_t CountPmuConfiguredSubcores(const PmuControl &control) +{ + uint32_t count = 0; + for (uint32_t index = 0; index < kPmuPhysicalSubcores; ++index) { + count += IsPmuSubcoreConfigured(control, index) ? 1U : 0U; + } + return count; +} + +inline const char *PmuRegisterFieldName(PmuRegisterField field) +{ + switch (field) { + case PmuRegisterField::None: return "none"; + case PmuRegisterField::RegisterBase: return "register-base"; + case PmuRegisterField::Ctrl0: return "ctrl0"; + case PmuRegisterField::Ctrl1: return "ctrl1"; + case PmuRegisterField::Selector0: return "selector0"; + case PmuRegisterField::Selector1: return "selector1"; + case PmuRegisterField::Selector2: return "selector2"; + case PmuRegisterField::StartCycleLow: return "start-cycle-low"; + case PmuRegisterField::StartCycleHigh: return "start-cycle-high"; + case PmuRegisterField::StopCycleLow: return "stop-cycle-low"; + case PmuRegisterField::StopCycleHigh: return "stop-cycle-high"; + case PmuRegisterField::ConfiguredCount: return "configured-count"; + default: return "unknown"; + } +} + } // namespace atomic_probe::pmu #endif // TESTS_ATOMIC_PROBE_CCEC_PMU_PROBE_CONTROL_H_ diff --git a/tests/atomic_probe/ccec/pmu_probe_host_support.h b/tests/atomic_probe/ccec/pmu_probe_host_support.h index 6755edd82d..f2a0b088ec 100644 --- a/tests/atomic_probe/ccec/pmu_probe_host_support.h +++ b/tests/atomic_probe/ccec/pmu_probe_host_support.h @@ -55,6 +55,49 @@ inline std::string ArtifactBesideKernel(const std::string &kernel_path, const ch return slash == std::string::npos ? std::string(name) : kernel_path.substr(0, slash + 1) + name; } +struct ActiveSubcoreLimits { + uint32_t cube = 0; + uint32_t vector = 0; + uint32_t total = 0; +}; + +// PMU MMIO 表覆盖芯片的 108 个物理槽,但当前 stream 只会调度 ACL +// resource limit 报告的活跃 cube/vector 子核。A5 当前典型值为 32+64=96; +// 不把 96 写死,以免把 stream 分区变化误报成寄存器配置失败。 +inline bool QueryActiveSubcoreLimits(aclrtStream stream, ActiveSubcoreLimits *limits) +{ + if (stream == nullptr || limits == nullptr) { + std::fprintf(stderr, "Cannot query PMU active subcores with a null stream/result.\n"); + return false; + } + uint32_t cube = 0; + uint32_t vector = 0; + const aclError cube_error = aclrtGetStreamResLimit(stream, ACL_RT_DEV_RES_CUBE_CORE, &cube); + const aclError vector_error = aclrtGetStreamResLimit(stream, ACL_RT_DEV_RES_VECTOR_CORE, &vector); + const uint64_t total = static_cast(cube) + vector; + constexpr uint32_t kMaximumCubeSubcores = kPmuPhysicalSubcores / 3U; + constexpr uint32_t kMaximumVectorSubcores = kPmuPhysicalSubcores - kMaximumCubeSubcores; + if (cube_error != ACL_SUCCESS || vector_error != ACL_SUCCESS || cube == 0 || vector == 0 || + cube > kMaximumCubeSubcores || vector > kMaximumVectorSubcores || total > kPmuPhysicalSubcores) { + std::fprintf( + stderr, + "Cannot determine stream PMU subcores: cube_error=%d vector_error=%d " + "cube=%u vector=%u total=%llu physical_slots=%u\n", + static_cast(cube_error), static_cast(vector_error), cube, vector, + static_cast(total), kPmuPhysicalSubcores + ); + return false; + } + limits->cube = cube; + limits->vector = vector; + limits->total = static_cast(total); + std::printf( + "[PMU_OWNER] stream_active cube=%u vector=%u total=%u physical_slots=%u\n", + limits->cube, limits->vector, limits->total, kPmuPhysicalSubcores + ); + return true; +} + constexpr uint32_t kPhysicalAicoreCount = 36; constexpr uint32_t kSubcoresPerAicore = 3; constexpr uint32_t kPhysicalSubcoreCount = kPhysicalAicoreCount * kSubcoresPerAicore; @@ -163,6 +206,7 @@ class PmuSession { { stream_ = stream; device_ = device; + if (!QueryActiveSubcoreLimits(stream, &active_limits_)) return false; if (!mappings_.Initialize(device)) return false; const size_t register_bytes = sizeof(mappings_.RegisterBases()); @@ -186,7 +230,7 @@ class PmuSession { control_.magic = kPmuControlMagic; control_.version = kPmuControlVersion; - control_.expected_subcores = kPmuPhysicalSubcores; + control_.expected_subcores = active_limits_.total; if (!CheckAcl( aclrtMemcpy( control_device_, sizeof(control_), &control_, sizeof(control_), ACL_MEMCPY_HOST_TO_DEVICE @@ -222,8 +266,11 @@ class PmuSession { bool Configure() { if (!initialized_ || configured_) return false; - configured_ = RunCommand(PmuCommand::Configure); - return configured_; + const bool command_ok = RunCommand(PmuCommand::Configure); + // 若 Configure 报错且设备侧回滚也失败,bitmap 仍保存待恢复项; + // 不能因为业务状态失败就丢失 Restore 所有权。 + configured_ = control_.configured != 0; + return command_ok; } bool Restore() @@ -256,6 +303,11 @@ class PmuSession { return reinterpret_cast(register_bases_device_); } + bool IsConfiguredSubcore(uint32_t index) const + { + return configured_ && IsPmuSubcoreConfigured(control_, index); + } + private: bool RunCommand(PmuCommand command) { @@ -278,17 +330,43 @@ class PmuSession { )) { return false; } + const uint32_t bitmap_count = CountPmuConfiguredSubcores(control_); const bool expected_state = command == PmuCommand::Configure - ? control_.configured == 1 && control_.processed_subcores == kPmuPhysicalSubcores - : control_.configured == 0 && control_.processed_subcores == 0; + ? control_.configured == 1 && control_.processed_subcores == control_.expected_subcores && + bitmap_count == control_.expected_subcores && + control_.skipped_subcores + bitmap_count == kPmuPhysicalSubcores + : control_.configured == 0 && control_.processed_subcores == 0 && bitmap_count == 0; if (control_.status != 0 || !expected_state) { + const auto failed_field = static_cast(control_.first_failed_field); std::fprintf( - stderr, "PMU helper failed: command=%u status=%d configured=%u processed=%u\n", + stderr, + "PMU helper failed: command=%u status=%d configured=%u processed=%u bitmap_count=%u " + "expected_subcores=%u skipped=%u failed_index=%u failed_field=%s(%u) " + "observed=0x%x expected=0x%x\n", control_.command, static_cast(control_.status), control_.configured, - control_.processed_subcores + control_.processed_subcores, bitmap_count, control_.expected_subcores, + control_.skipped_subcores, control_.first_failed_index, + PmuRegisterFieldName(failed_field), control_.first_failed_field, + control_.first_failed_observed, control_.first_failed_expected ); return false; } + if (command == PmuCommand::Configure) { + const auto failed_field = static_cast(control_.first_failed_field); + std::printf( + "[PMU_OWNER] configured=%u bitmap_count=%u expected=%u skipped=%u " + "first_skipped_index=%u first_skipped_field=%s(%u) observed=0x%x expected_value=0x%x\n", + control_.processed_subcores, bitmap_count, control_.expected_subcores, + control_.skipped_subcores, control_.first_failed_index, + PmuRegisterFieldName(failed_field), control_.first_failed_field, + control_.first_failed_observed, control_.first_failed_expected + ); + std::printf( + "[PMU_OWNER] configured_bitmap=%08x:%08x:%08x:%08x\n", + control_.configured_bitmap[3], control_.configured_bitmap[2], + control_.configured_bitmap[1], control_.configured_bitmap[0] + ); + } return true; } @@ -300,6 +378,7 @@ class PmuSession { void *control_device_ = nullptr; PmuControl control_{}; KernelArgs kernel_args_{}; + ActiveSubcoreLimits active_limits_{}; bool initialized_ = false; bool configured_ = false; }; From 13431a23154c6d15c4dc12f5e408259515bf60aa Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 08:50:21 +0000 Subject: [PATCH 011/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E8=A1=A5?= =?UTF-8?q?=E9=BD=90=20PA=20=E8=B0=83=E5=BA=A6=E5=99=A8=E9=80=90=20atomic?= =?UTF-8?q?=20=E6=B3=B3=E9=81=93=E8=A7=82=E6=B5=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 覆盖 standalone PA 的共享 atomic 调用点,记录 site、op、任务归属和动态次数。 按原调用语义区分 return-ready 与 source-issue 边界;CCEC 通过返回值依赖 MOV 后读取 SYS_CNT,不插入 DSB,也不强制发布型 atomic 消费旧值。 将 Atomic 与 ClockBaseline 合并到 AIC/AIV scalar 泳道,补齐 Claim attempted/won schema,并以 atomic_trace_calls、逐核 record 数和 dropped 建立闭环。 泳道采集默认开启逐 atomic 记录,转换器补充边界标识、标量泳道布局和 schema 回归测试。 验证:转换器单元测试 4/4 通过;CPU、CCEC、AscendC 三后端构建通过;A5 CCEC b1 调度语义、atomic 计数、记录容量和泳道后处理全部通过,记录丢失为 0。 --- .../pa_scheduler/ascendc/pa_scheduler.asc | 10 + .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 18 ++ .../pa_scheduler/common/host_support.h | 185 ++++++++++++++++-- .../pa_scheduler/common/pa_model.h | 60 ++++++ .../pa_scheduler/common/pa_scheduler_core.h | 141 +++++++++---- .../pa_scheduler/common/pa_trace.h | 114 ++++++++++- tests/atomic_probe/pa_scheduler/cpu/main.cpp | 11 ++ tests/atomic_probe/pa_scheduler/run.sh | 9 +- .../pa_scheduler/swimlane_converter.py | 180 ++++++++++++++++- .../pa_scheduler/test_swimlane_converter.py | 182 +++++++++++++++++ 10 files changed, 850 insertions(+), 60 deletions(-) create mode 100644 tests/atomic_probe/pa_scheduler/test_swimlane_converter.py diff --git a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc index e2207bc2ed..3972af5517 100644 --- a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc +++ b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc @@ -60,6 +60,10 @@ __aicore__ inline void RuntimeNop(uint32_t count) { } struct AscendcOps { + // AscendC 路径尚未完成同构机器码核验,因此保留源码括号并在 raw flags 中 + // 明确标记为未建立 return-ready 边界。 + static constexpr bool kAtomicReturnReadyObserved = false; + // 公共调度器把 Load 定义为“具有原子一致性的读”。这里坚持使用 // AtomicAdd(addr, 0),以保留真实 PA 在热点 cache line 上的竞争形态。 __aicore__ static inline int32_t Load(__gm__ volatile int32_t *address) { @@ -112,6 +116,12 @@ struct AscendcOps { // A5 SYS_CNT 为 1 GHz;公共模型也以 1 GHz tick 记录泳道时间。 __aicore__ static inline uint64_t Now() { return static_cast(GetSystemCycle()); } + template + __aicore__ static inline uint64_t NowAfterAtomicResult(T value) { + (void)value; + return Now(); + } + __aicore__ static inline void Nop(uint32_t count) { RuntimeNop(count); } __aicore__ static inline void SpinHint() {} diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index 5724866900..3ccbdcd025 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -47,6 +47,8 @@ __aicore__ inline void RuntimeNop(uint32_t count) { } struct CcecOps { + static constexpr bool kAtomicReturnReadyObserved = true; + // 该适配层把平台无关调度器需要的原子、计时、NOP 和 cache 操作逐一映射到 CCEC intrinsic。 // A5 上 PA 的共享“读取”使用 atomicAdd(addr, 0),不是普通 GM load;这里保留其 RMW 竞争语义。 __aicore__ static inline int32_t Load(__gm__ volatile int32_t *address) { @@ -96,6 +98,22 @@ struct CcecOps { __aicore__ static inline uint64_t Now() { return static_cast(get_sys_cnt()); } + template + __aicore__ static inline uint64_t NowAfterAtomicResult(T value) { + static_assert(sizeof(T) == 4 || sizeof(T) == 8, "atomic dependency expects a scalar result"); + uint64_t cycle = 0; + // 同一个 inline asm 块先真正消费 atomic 返回寄存器,再读取 + // SYS_CNT;编译器不能把 t1 拆到依赖 MOV 之前。AIC/AIV 对该序列 + // 生成相同指令字节,且不增加 DSB/ISB/GM 访存。该边界仍只表示 + // 返回值已可被本核 scalar 消费,不表示跨核全局可见。 + asm volatile( + "MOV %0, %0\n" + "MOV %1, SYS_CNT\n" + : "+l"(value), "=&l"(cycle) + ); + return cycle; + } + __aicore__ static inline void Nop(uint32_t count) { RuntimeNop(count); } // SPIN_WAIT_HINT is also a no-op in the real A5 inner-kernel contract. diff --git a/tests/atomic_probe/pa_scheduler/common/host_support.h b/tests/atomic_probe/pa_scheduler/common/host_support.h index 0f5e054efe..02b0bd0d60 100644 --- a/tests/atomic_probe/pa_scheduler/common/host_support.h +++ b/tests/atomic_probe/pa_scheduler/common/host_support.h @@ -35,6 +35,7 @@ struct Options { NopCounts nops{kDefaultQkNops, kDefaultSfNops, kDefaultPvNops, kDefaultUpNops}; bool profile_phases = false; bool trace_enabled = true; + bool trace_atomics = false; bool analyze_swimlane = false; }; @@ -84,7 +85,7 @@ inline void PrintUsage(const char *program, bool require_kernel) { std::fprintf( stderr, "[--nop-count N | --nop-counts QK,SF,PV,UP] [--profile-phases] [--analyze-swimlane] " - "[--swimlane-json FILE] [--no-swimlane]\n" + "[--trace-atomics] [--swimlane-json FILE] [--no-swimlane]\n" ); } @@ -107,6 +108,10 @@ inline ParseStatus ParseOptions(int argc, char **argv, bool require_kernel, Opti options->trace_enabled = false; continue; } + if (argument == "--trace-atomics") { + options->trace_atomics = true; + continue; + } if (argument == "--analyze-swimlane") { options->analyze_swimlane = true; continue; @@ -165,6 +170,10 @@ inline ParseStatus ParseOptions(int argc, char **argv, bool require_kernel, Opti std::fprintf(stderr, "--analyze-swimlane requires swimlane tracing.\n"); return ParseStatus::Error; } + if (options->trace_atomics && !options->trace_enabled) { + std::fprintf(stderr, "--trace-atomics cannot be combined with --no-swimlane.\n"); + return ParseStatus::Error; + } if (!options->swimlane_json.empty() && !options->trace_enabled) { std::fprintf(stderr, "--swimlane-json requires swimlane tracing.\n"); return ParseStatus::Error; @@ -207,7 +216,9 @@ inline void InitializeState(SchedulerState *state, const Options &options) { inline void ConfigureTrace(SchedulerState *state, const Options &options, const void *trace_base) { // device 只持有裸地址和每核容量;TraceHeader/record 缓冲区由 host 单独分配并初始化。 - state->config.trace_enabled = options.trace_enabled ? 1U : 0U; + state->config.trace_enabled = options.trace_enabled + ? kTracePhasesEnabled | (options.trace_atomics ? kTraceAtomicsEnabled : 0U) + : 0U; state->config.trace_base = options.trace_enabled ? reinterpret_cast(trace_base) : 0; state->config.trace_records_per_core = options.trace_enabled ? kTraceRecordsPerCore : 0; } @@ -330,11 +341,45 @@ inline const char *TracePhaseName(uint32_t phase) { // 名称必须与 l2_swimlane_records.json 的 fdwic_events schema 保持一致。 const char *names[] = { "Kernel", "Alloc", "Build", "DrainWon", "Replay", "RingBp", "EfDrain", "Commit", - "Submit", "Materialize", "PrepareMap", "Claim", "Fanin", "Register", + "Submit", "Materialize", "PrepareMap", "Claim", "Fanin", "Register", "Atomic", + "ClockBaseline", }; return phase < sizeof(names) / sizeof(names[0]) ? names[phase] : "Unknown"; } +inline const char *AtomicSiteName(uint32_t site) { + // 顺序与 pa_model.h::AtomicSite 的稳定 raw ABI 完全一致。 + const char *names[] = { + "StartupIncrement", "StartupPoll", "FatalPoll", "FatalSet", "ClaimMax", + "FaninFlagLoad", "CompletionVendExchange", "CompletionFlagExchange", + "FrontierInitialLoad", "FrontierFlagLoad", "FrontierMax", "HeapFrontierLoad", + "HeapVendLoad", "ReplayDoneIncrement", "ReplayDonePoll", + }; + return site < sizeof(names) / sizeof(names[0]) ? names[site] : "Unknown"; +} + +inline const char *AtomicOpName(uint32_t op) { + const char *names[] = {"Load", "Exchange", "FetchAdd", "FetchMax"}; + return op < sizeof(names) / sizeof(names[0]) ? names[op] : "Unknown"; +} + +inline AtomicOp AtomicSiteOp(AtomicSite site) { + switch (site) { + case AtomicSite::StartupIncrement: + case AtomicSite::ReplayDoneIncrement: + return AtomicOp::FetchAdd; + case AtomicSite::FatalSet: + case AtomicSite::CompletionVendExchange: + case AtomicSite::CompletionFlagExchange: + return AtomicOp::Exchange; + case AtomicSite::ClaimMax: + case AtomicSite::FrontierMax: + return AtomicOp::FetchMax; + default: + return AtomicOp::Load; + } +} + inline bool ValidateTraceHeader(const TraceHeader &header, const char *operation) { // 在任何 D2H record 搬运前先验证容量和 dropped,防止损坏 header 导致 scratch 越界或导出残缺泳道。 // 频率也要求精确为 1 GHz,否则后续 ns/us 换算即使 JSON 合法也没有性能意义。 @@ -382,7 +427,8 @@ inline bool ExportSwimlaneRecords( std::fprintf( output, "{\n\"l2_swimlane_level\":1,\n" - "\"metadata\":{\"clock_freq_hz\":%llu,\"num_cores\":%u,\"core_types\":[", + "\"metadata\":{\"clock_freq_hz\":%llu,\"num_cores\":%u," + "\"trace_schema_version\":2,\"core_types\":[", static_cast(header.frequency_hz), kWorkers ); for (uint32_t worker = 0; worker < kWorkers; ++worker) { @@ -399,7 +445,7 @@ inline bool ExportSwimlaneRecords( bool first_record = true; uint64_t exported_records = 0; std::vector scratch(kTraceRecordsPerCore); - constexpr int32_t kTracePhaseCount = static_cast(TracePhase::Register) + 1; + constexpr int32_t kTracePhaseCount = static_cast(TracePhase::ClockBaseline) + 1; for (uint32_t worker = 0; worker < kWorkers && success; ++worker) { // 每次只读取一个 worker 的有效区间;完整 384 MiB trace 缓冲无需整体回拷。 const uint32_t available = header.cores[worker].count; @@ -419,10 +465,30 @@ inline bool ExportSwimlaneRecords( const TraceRecord &record = scratch[index]; // 这里只检查 lane/block 的合法范围以及 core_idx 是否落在所属 worker 槽, // 不把生产者应遵守的 worker↔block/lane 精确映射误说成 exporter 已完成的校验。 + const bool atomic_record = record.phase == static_cast(TracePhase::Atomic); + const bool claim_record = record.phase == static_cast(TracePhase::Claim); + const uint32_t atomic_op = record.flags & kAtomicOpMask; + const bool atomic_result_used = (record.flags & kAtomicResultUsed) != 0; + const bool atomic_return_ready = (record.flags & kAtomicReturnReady) != 0; + const bool atomic_return_ready_valid = !atomic_return_ready || + (atomic_result_used && + (atomic_op == static_cast(AtomicOp::Load) || + atomic_op == static_cast(AtomicOp::FetchMax))); + const bool atomic_schema_valid = !atomic_record || + (record.auxiliary < static_cast(AtomicSite::Count) && + atomic_op <= static_cast(AtomicOp::FetchMax) && + atomic_return_ready_valid && + atomic_op == static_cast( + AtomicSiteOp(static_cast(record.auxiliary)) + )); + const bool claim_schema_valid = !claim_record || + ((record.flags & ~(kClaimWon | kClaimAttempted)) == 0 && + ((record.flags & kClaimWon) == 0 || (record.flags & kClaimAttempted) != 0)); const bool record_valid = record.end_cycle >= record.start_cycle && record.phase >= 0 && record.phase < kTracePhaseCount && record.lane >= 0 && record.lane <= 2 && record.block_id >= 0 && record.block_id < static_cast(kAicWorkers) && - record.core_idx == static_cast(worker); + record.core_idx == static_cast(worker) && atomic_schema_valid && + claim_schema_valid; if (!record_valid) { std::fprintf( stderr, @@ -480,13 +546,18 @@ inline bool AnalyzeSwimlaneRecords( if (!ValidateTraceHeader(header, "swimlane analysis")) return false; // 第一组数组统计“每个 worker 在某阶段的累计时间”;task_durations 则保留重点阶段的单事件分布。 - constexpr uint32_t kTracePhaseCount = static_cast(TracePhase::Register) + 1; + constexpr uint32_t kTracePhaseCount = static_cast(TracePhase::ClockBaseline) + 1; constexpr TracePhase kDetailedPhases[] = { TracePhase::EfDrain, TracePhase::Materialize, TracePhase::Claim, TracePhase::Register, }; uint64_t cycles[kWorkers][kTracePhaseCount] = {}; uint64_t counts[kWorkers][kTracePhaseCount] = {}; std::vector task_durations[2][kTasksPerBatch][sizeof(kDetailedPhases) / sizeof(kDetailedPhases[0])]; + std::vector atomic_durations[2][static_cast(AtomicSite::Count)]; + uint64_t atomic_return_ready_counts[2][static_cast(AtomicSite::Count)] = {}; + std::vector clock_baselines[2]; + std::vector clock_dependency_baselines[2]; + uint64_t clock_dependency_applied[2] = {}; std::vector scratch(kTraceRecordsPerCore); for (uint32_t worker = 0; worker < kWorkers; ++worker) { const uint32_t available = header.cores[worker].count; @@ -505,6 +576,25 @@ inline bool AnalyzeSwimlaneRecords( const uint64_t duration = record.end_cycle - record.start_cycle; cycles[worker][phase] += duration; ++counts[worker][phase]; + if (record.phase == static_cast(TracePhase::Atomic) && + record.auxiliary < static_cast(AtomicSite::Count)) { + const uint32_t role_index = + state.results[worker].role == static_cast(CoreRole::Aic) ? 0U : 1U; + atomic_durations[role_index][record.auxiliary].push_back(duration); + atomic_return_ready_counts[role_index][record.auxiliary] += + (record.flags & kAtomicReturnReady) != 0; + } + if (record.phase == static_cast(TracePhase::ClockBaseline)) { + const uint32_t role_index = + state.results[worker].role == static_cast(CoreRole::Aic) ? 0U : 1U; + if ((record.flags & kClockAtomicDependency) != 0) { + clock_dependency_baselines[role_index].push_back(duration); + clock_dependency_applied[role_index] += + (record.flags & kClockAtomicDependencyApplied) != 0; + } else { + clock_baselines[role_index].push_back(duration); + } + } if (record.task_id >= 0) { // task_id % 5 恰好对应 Alloc/QK/SF/PV/UP,这是固定 PA Case1 图的拓扑约束。 const uint32_t role_index = @@ -541,6 +631,56 @@ inline bool AnalyzeSwimlaneRecords( ); } } + for (uint32_t role_index = 0; role_index < 2; ++role_index) { + const Uint64Distribution summary = SummarizeUint64(clock_baselines[role_index]); + if (!clock_baselines[role_index].empty()) { + std::printf( + "[TRACE_CLOCK] role=%s samples=%zu definition=consecutive-sys-cnt-reads " + "median_ns=%.1f p95_ns=%llu max_ns=%llu\n", + role_names[role_index], clock_baselines[role_index].size(), summary.median, + static_cast(summary.p95), + static_cast(summary.maximum) + ); + } + const Uint64Distribution dependency_summary = + SummarizeUint64(clock_dependency_baselines[role_index]); + if (!clock_dependency_baselines[role_index].empty()) { + std::printf( + "[TRACE_CLOCK] role=%s samples=%zu definition=atomic-return-dependency-hook " + "dependency_applied=%llu/%zu median_ns=%.1f p95_ns=%llu max_ns=%llu\n", + role_names[role_index], clock_dependency_baselines[role_index].size(), + static_cast(clock_dependency_applied[role_index]), + clock_dependency_baselines[role_index].size(), dependency_summary.median, + static_cast(dependency_summary.p95), + static_cast(dependency_summary.maximum) + ); + } + } + // Atomic 只报告原始括号分布,不扣除计时底噪,也不把 total_cycles + // 解释成可与 Submit 墙钟直接相加的“atomic 占比”。return-ready 只表示 + // 本核可消费返回值,不表示其他核已经观察到更新。 + for (uint32_t role_index = 0; role_index < 2; ++role_index) { + for (uint32_t site = 0; site < static_cast(AtomicSite::Count); ++site) { + const std::vector &durations = atomic_durations[role_index][site]; + if (durations.empty()) continue; + const Uint64Distribution summary = SummarizeUint64(durations); + const AtomicOp op = AtomicSiteOp(static_cast(site)); + const uint64_t return_ready_count = atomic_return_ready_counts[role_index][site]; + const char *boundary = return_ready_count == durations.size() + ? "return-ready" + : (return_ready_count == 0 ? "source-issue" : "mixed"); + std::printf( + "[TRACE_ATOMIC] role=%s site=%s op=%s events=%zu boundary=%s " + "return_ready=%llu/%zu bracket_cycles_total=%llu median_ns=%.1f " + "p95_ns=%llu max_ns=%llu\n", + role_names[role_index], AtomicSiteName(site), AtomicOpName(static_cast(op)), + durations.size(), boundary, static_cast(return_ready_count), + durations.size(), static_cast(summary.total), summary.median, + static_cast(summary.p95), + static_cast(summary.maximum) + ); + } + } const char *kind_names[] = {"Alloc", "QK", "SF", "PV", "UP"}; // 单事件统计按 role 与 task kind 展开,可区分“该 role 真实参与”与“只回放前端”的成本。 for (uint32_t role_index = 0; role_index < 2; ++role_index) { @@ -586,6 +726,7 @@ inline Metrics Validate( uint64_t frontier_initial_loads = 0; uint64_t frontier_updates = 0; uint64_t frontier_terminal_loads = 0; + uint64_t atomic_trace_calls = 0; uint64_t duplicates = 0; uint64_t cas_retries = 0; uint64_t joint_polls = 0; @@ -686,6 +827,7 @@ inline Metrics Validate( frontier_initial_loads += result.frontier_initial_loads; frontier_updates += result.frontier_updates; frontier_terminal_loads += result.frontier_terminal_loads; + atomic_trace_calls += result.atomic_trace_calls; duplicates += result.completion_duplicates; cas_retries += result.cas_retries; joint_polls += result.joint_polls; @@ -876,26 +1018,42 @@ inline Metrics Validate( const WorkerResult &result = state.results[worker]; const uint64_t worker_kernels = result.kernel_counts[0] + result.kernel_counts[1] + result.kernel_counts[2] + result.kernel_counts[3]; - const uint64_t worker_expected = 7 * result.submits + - result.claim_wins - result.wins[0] + - 2 * worker_kernels + result.wait_events[0] + - result.wait_events[1]; + const uint64_t worker_expected = + 7 * result.submits + result.claim_wins - result.wins[0] + + 2 * worker_kernels + result.wait_events[0] + result.wait_events[1] + + (((state.config.trace_enabled & kTraceAtomicsEnabled) != 0) + ? result.atomic_trace_calls + 2 + : 0); per_worker_trace_counts_ok &= trace_header->cores[worker].count == worker_expected; } } const uint64_t expected_trace_records = - static_cast(batches) * (static_cast(kWorkers) * 35 + 12) + trace_wait_records; + static_cast(batches) * (static_cast(kWorkers) * 35 + 12) + + trace_wait_records + + (((state.config.trace_enabled & kTraceAtomicsEnabled) != 0) + ? atomic_trace_calls + 2 * kWorkers + : 0); // 每 batch 固定记录为 96*35+12;RingBp 等真实等待按运行时次数额外加入。 Expect(trace_shape_ok, "swimlane header and per-worker capacities are valid", &metrics); Expect(trace_dropped == 0, "swimlane records fit without drops", &metrics); Expect(trace_records == expected_trace_records, "swimlane record count matches PA phase flow", &metrics); Expect(per_worker_trace_counts_ok, "every worker swimlane record count is exact", &metrics); + if ((state.config.trace_enabled & kTraceAtomicsEnabled) != 0) { + Expect(atomic_trace_calls != 0, "atomic trace captured source-level calls", &metrics); + } else { + Expect(atomic_trace_calls == 0, "atomic trace counters stay zero when disabled", &metrics); + } std::printf( "[TRACE] records=%llu expected=%llu dropped=%llu bytes=%zu\n", static_cast(trace_records), static_cast(expected_trace_records), static_cast(trace_dropped), kTraceBytes ); + std::printf( + "[ATOMIC_TRACE] enabled=%s calls=%llu definition=per-record-boundary-flags\n", + (state.config.trace_enabled & kTraceAtomicsEnabled) != 0 ? "yes" : "no", + static_cast(atomic_trace_calls) + ); } if (first_submit != UINT64_MAX && last_submit >= first_submit) { @@ -999,9 +1157,10 @@ inline void PrintBanner(const char *backend, const Options &options) { std::printf("=== Standalone PA Scheduler Benchmark: %s ===\n", backend); std::printf( "device=%u batches=%u tasks=%u workers=%u runs=%u nops=%u,%u,%u,%u state_bytes=%zu " - "swimlane=%s trace_bytes=%zu\n", options.device, + "swimlane=%s trace_atomics=%s trace_bytes=%zu\n", options.device, options.batches, options.batches * kTasksPerBatch, kWorkers, options.runs, options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up, sizeof(SchedulerState), options.trace_enabled ? "on" : "off", + options.trace_atomics ? "on" : "off", options.trace_enabled ? kTraceBytes : 0 ); if (!options.swimlane_json.empty()) { diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index 93f87bca30..d22bdf86a7 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -56,6 +56,16 @@ constexpr uint32_t kTaskWindow = 1 << 10; constexpr uint32_t kTaskWindowMask = kTaskWindow - 1; constexpr uint64_t kSystemCounterHz = 1000000000ULL; constexpr uint64_t kWatchdogTicks = 2 * kSystemCounterHz; +// trace_enabled 是位图而不是 bool:bit0 保持既有阶段泳道,bit1 额外开启 +// 逐条 atomic 源码括号记录。atomic 记录依赖同一份 trace buffer,因此 bit1 +// 只能与 bit0 一起配置。 +constexpr uint32_t kTracePhasesEnabled = 1U << 0; +constexpr uint32_t kTraceAtomicsEnabled = 1U << 1; +// Claim trace flags 是独立 raw ABI:bit0 表示获胜,bit1 表示已经通过 +// AIC/AIV role 路由并真正执行 atomicMax。未 attempted 的 Claim 仍保留 +// role-selection 开销,但转换器会明确标成 claim.not_attempted。 +constexpr uint32_t kClaimWon = 1U << 0; +constexpr uint32_t kClaimAttempted = 1U << 1; // 下列 offset/size 来自真实 DistGlobal/DistCore ABI。standalone 保留被测关键字段的 // offset、DistCore ABI 和 kRealDistGlobalBytes 总跨度;其余区域可用 opaque padding, // 并不是对生产结构全部字段的逐一镜像。 @@ -192,8 +202,53 @@ enum class TracePhase : int32_t { Claim = 11, Fanin = 12, Register = 13, + Atomic = 14, + // 逐 atomic 诊断构建中,每个 worker 只记录一次连续两次 SYS_CNT 的 + // 空括号,用来给出同一二进制、同一物理核上的计时分辨率下限。 + ClockBaseline = 15, }; +// AtomicSite 按 standalone PA 中真实出现的源码调用点分类。编号写入 TraceRecord::auxiliary, +// 是离线泳道 schema 的一部分;追加新位置只能在 Count 前扩展,不能重排既有值。 +enum class AtomicSite : uint32_t { + StartupIncrement = 0, + StartupPoll = 1, + FatalPoll = 2, + FatalSet = 3, + ClaimMax = 4, + FaninFlagLoad = 5, + CompletionVendExchange = 6, + CompletionFlagExchange = 7, + FrontierInitialLoad = 8, + FrontierFlagLoad = 9, + FrontierMax = 10, + HeapFrontierLoad = 11, + HeapVendLoad = 12, + ReplayDoneIncrement = 13, + ReplayDonePoll = 14, + Count = 15, +}; + +// Atomic 记录 flags 的低四位保存操作种类;bit4 表示返回值参与后续判断, +// bit5 表示 Load 观察到零,bit6 表示结束时间已由返回值依赖推进到 +// return-ready 边界,bits[31:8] 保存 FetchMax 的软件重试数(饱和)。 +enum class AtomicOp : uint32_t { + Load = 0, + Exchange = 1, + FetchAdd = 2, + FetchMax = 3, +}; +constexpr uint32_t kAtomicOpMask = 0x0fU; +constexpr uint32_t kAtomicResultUsed = 1U << 4; +constexpr uint32_t kAtomicValueZero = 1U << 5; +constexpr uint32_t kAtomicReturnReady = 1U << 6; +constexpr uint32_t kAtomicRetriesShift = 8; + +// ClockBaseline 的 bit0 区分普通连续 SYS_CNT 与后端的 atomic 返回依赖 +// 计时钩子;后者用于量化那一条依赖 MOV 自身带来的固定底噪。 +constexpr uint32_t kClockAtomicDependency = 1U << 0; +constexpr uint32_t kClockAtomicDependencyApplied = 1U << 1; + struct alignas(64) TraceCoreState { volatile uint32_t count; volatile uint32_t dropped; @@ -509,6 +564,10 @@ struct alignas(64) WorkerResult { uint64_t frontier_initial_loads; uint64_t frontier_updates; uint64_t frontier_terminal_loads; + + // 仅在 trace_enabled bit1 开启时递增;每次源码 atomic 调用恰好增加一, + // host 用它与 Atomic span 数逐 worker 闭合,禁止把丢记录的泳道当成完整结果。 + uint64_t atomic_trace_calls; }; // WorkerResult 是 standalone 尾部的诊断 sidecar,不属于真实 DistCore ABI;按 // cache line 隔离后,各 worker 发布统计不会相互覆盖或污染被测共享状态。 @@ -516,6 +575,7 @@ static_assert(sizeof(WorkerResult) == 768, "WorkerResult diagnostics must occupy static_assert(offsetof(WorkerResult, pmu_total_cycles) == 680, "WorkerResult PMU offset mismatch"); static_assert(offsetof(WorkerResult, pmu_status) == 700, "WorkerResult PMU status offset mismatch"); static_assert(offsetof(WorkerResult, fanin_not_ready_loads) == 704, "WorkerResult atomic diagnostic offset mismatch"); +static_assert(offsetof(WorkerResult, atomic_trace_calls) == 736, "WorkerResult atomic trace offset mismatch"); // 从 cube_cursor 到 workers 结束保留关键字段 offset、DistCore ABI 和生产总字节跨度, // 并非字段级完整镜像。RunConfig、输入 context_lens 与校验结果追加在该跨度之后, diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index 64672629ec..4353557b4b 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -64,35 +64,44 @@ PA_DEVICE uint32_t CountBits(uint32_t value) { } template -PA_DEVICE int64_t LoadLine(PA_GM AtomicLine &line) { +PA_DEVICE int64_t LoadLine( + PA_GM AtomicLine &line, LocalStats &stats, AtomicSite site, int32_t task_id = -1 +) { // Ops::Load 在 A5 后端是 atomicAdd(0);返回值是该 RMW 线性化时观察到的共享值。 - return Ops::Load(&line.value); + return TraceAtomicLoad(stats.trace, stats.result, task_id, site, &line.value); } template -PA_DEVICE int32_t LoadLine(PA_GM AtomicFlagLine &line) { - return Ops::Load(&line.value); +PA_DEVICE int32_t LoadLine( + PA_GM AtomicFlagLine &line, LocalStats &stats, AtomicSite site, int32_t task_id = -1 +) { + return TraceAtomicLoad(stats.trace, stats.result, task_id, site, &line.value); } template -PA_DEVICE void SetFatal(PA_GM SchedulerState *state) { +PA_DEVICE void SetFatal(PA_GM SchedulerState *state, LocalStats &stats, int32_t task_id = -1) { // fatal 只从 0 单调置 1,重复 Exchange 不会把其他 worker 已观察到的失败状态清除。 - Ops::Exchange(&state->fatal.value, static_cast(1)); + TraceAtomicExchange( + stats.trace, stats.result, task_id, AtomicSite::FatalSet, &state->fatal.value, + static_cast(1) + ); } template -PA_DEVICE bool IsFatal(PA_GM SchedulerState *state) { - return LoadLine(state->fatal) != 0; +PA_DEVICE bool IsFatal(PA_GM SchedulerState *state, LocalStats &stats, int32_t task_id = -1) { + return LoadLine(state->fatal, stats, AtomicSite::FatalPoll, task_id) != 0; } template -PA_DEVICE bool WatchdogExpired(PA_GM SchedulerState *state, uint64_t begin, uint32_t &polls) { +PA_DEVICE bool WatchdogExpired( + PA_GM SchedulerState *state, LocalStats &stats, uint64_t begin, uint32_t &polls +) { // 每 1024 次自旋才读取系统计数器,降低正常启动屏障上的计时开销;超时后向所有 worker 广播 fatal。 ++polls; if ((polls & 1023U) != 0 || Ops::Now() - begin <= kWatchdogTicks) { return false; } - SetFatal(state); + SetFatal(state, stats); return true; } @@ -101,20 +110,28 @@ PA_DEVICE void AdvanceFrontier(PA_GM SchedulerState *state, LocalStats &stats) { // frontier 只表示“从 task 0 开始已经连续完成”的最高 task id,不能越过尚未发布 flag 的空洞。 // 多个完成者可以同时扫描同一段连续区间,FetchMax 保证共享 frontier 只前进、不回退。 ++stats.result.frontier_initial_loads; - int64_t frontier = LoadLine(state->frontier); + int64_t frontier = LoadLine( + state->frontier, stats, AtomicSite::FrontierInitialLoad + ); while (true) { const int64_t next = frontier + 1; if (next < 0 || next >= static_cast(kTaskCellCapacity)) { break; } - if (Ops::Load(&state->tasks[next].flag) == 0) { + if (TraceAtomicLoad( + stats.trace, stats.result, static_cast(next), AtomicSite::FrontierFlagLoad, + &state->tasks[next].flag + ) == 0) { ++stats.result.frontier_terminal_loads; break; } uint64_t retries = 0; // FetchMax 返回更新前的值;若其他核已经走得更远,就从其 old 值继续扫描,避免重复从 next 起步。 ++stats.result.frontier_updates; - const int64_t old = Ops::FetchMax(&state->frontier.value, next, retries); + const int64_t old = TraceAtomicFetchMax( + stats.trace, stats.result, static_cast(next), AtomicSite::FrontierMax, + &state->frontier.value, next, retries + ); stats.result.cas_retries += retries; frontier = old > next ? old : next; } @@ -126,9 +143,15 @@ PA_DEVICE void CompleteTask( ) { // 完成发布顺序与 PA 一致:先公布该 worker 的 heap 游标,再发布 ready flag,最后推进连续 frontier。 // fanin 和 heap 回收方以 flag/frontier 为可见性条件,因此不能交换 vend 与 flag 的先后关系。 - Ops::Exchange(&state->tasks[task_id].vend, worker.heap_next); + TraceAtomicExchange( + stats.trace, stats.result, static_cast(task_id), AtomicSite::CompletionVendExchange, + &state->tasks[task_id].vend, worker.heap_next + ); Ops::StoreBarrier(); - Ops::Exchange(&state->tasks[task_id].flag, 1); + TraceAtomicExchange( + stats.trace, stats.result, static_cast(task_id), AtomicSite::CompletionFlagExchange, + &state->tasks[task_id].flag, static_cast(1) + ); AdvanceFrontier(state, stats); } @@ -136,7 +159,11 @@ template PA_DEVICE bool SlotReady(PA_GM SchedulerState *state, PA_GM LocalSlot &slot, LocalStats &stats) { // 每个 fanin flag 都是跨核共享的完成条件;遇到第一个未就绪依赖即返回,后续 drain 会再次轮询。 for (uint32_t index = 0; index < slot.fanin_count; ++index) { - if (Ops::Load(&state->tasks[slot.fanin[index]].flag) == 0) { + const int32_t dependency = slot.fanin[index]; + if (TraceAtomicLoad( + stats.trace, stats.result, dependency, AtomicSite::FaninFlagLoad, + &state->tasks[dependency].flag + ) == 0) { ++stats.result.fanin_not_ready_loads; return false; } @@ -250,7 +277,7 @@ PA_DEVICE bool HeapGuard( const uint64_t wait_begin = Ops::Now(); bool waited = false; // 正常出口是 heap_next-vend 落入一个 ring;检测到不可能释放的覆盖或其他核 fatal 时返回失败。 - while (!IsFatal(state)) { + while (!IsFatal(state, stats, static_cast(task_id))) { // 逻辑 heap 尚未走完第一圈时,所有物理输出区间都位于 [0, heap_next), // 不可能覆盖此前分配;保留上面的 fatal 原子检查后,可直接跳过 frontier/vend 读取。 if (worker.heap_next <= ring) { @@ -259,9 +286,16 @@ PA_DEVICE bool HeapGuard( } return true; } - const int64_t frontier = LoadLine(state->frontier); + const int64_t frontier = LoadLine( + state->frontier, stats, AtomicSite::HeapFrontierLoad, static_cast(task_id) + ); const int64_t retire = frontier - static_cast(state->heap_window); - const uint64_t vend = retire < 0 ? 0 : Ops::Load(&state->tasks[retire].vend); + const uint64_t vend = retire < 0 + ? 0 + : TraceAtomicLoad( + stats.trace, stats.result, static_cast(task_id), AtomicSite::HeapVendLoad, + &state->tasks[retire].vend + ); if (worker.heap_next - vend <= ring) { if (waited) { ++stats.result.wait_events[1]; @@ -276,7 +310,7 @@ PA_DEVICE bool HeapGuard( return true; } if (frontier >= static_cast(task_id) - 1) { - SetFatal(state); + SetFatal(state, stats, static_cast(task_id)); return false; } waited = true; @@ -307,7 +341,8 @@ struct ClaimOutcome { template PA_DEVICE ClaimOutcome Claim( - PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_id, TaskKind kind + PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_id, TaskKind kind, + LocalStats &stats ) { // Claim 在四个 shard 的单调 cursor 上执行 atomicMax:同一 task 只有观察到旧值更小的竞争者获胜。 // Alloc 由全部 96 个 worker 竞争;QK/PV 仅 32 个 AIC,SF/UP 仅 64 个 AIV 进入真正的 atomicMax。 @@ -350,7 +385,10 @@ PA_DEVICE ClaimOutcome Claim( } outcome.attempted = true; // atomicMax 返回写入前的 cursor:old=task_id 则必须 Replay。 - const int64_t old = Ops::FetchMax(&cursor->value, static_cast(task_id), outcome.retries); + const int64_t old = TraceAtomicFetchMax( + stats.trace, stats.result, static_cast(task_id), AtomicSite::ClaimMax, + &cursor->value, static_cast(task_id), outcome.retries + ); outcome.won = old < static_cast(task_id); if (!outcome.won) outcome.function_id = -1; return outcome; @@ -381,7 +419,7 @@ PA_DEVICE bool BuildWinner( } const int32_t slot_index = FindFreeSlot(worker); if (slot_index < 0) { - SetFatal(state); + SetFatal(state, stats, static_cast(task_id)); return false; } PA_GM LocalSlot &slot = worker.slots[slot_index]; @@ -437,7 +475,7 @@ PA_DEVICE bool SubmitTask( ResetTraceLap(worker); const uint64_t materialize_begin = Ops::Now(); if (!MaterializeTask(worker, task_id, args, context, state->heap_base, state->heap_size)) { - SetFatal(state); + SetFatal(state, stats, static_cast(task_id)); return false; } const uint64_t materialize_end = Ops::Now(); @@ -470,14 +508,15 @@ PA_DEVICE bool SubmitTask( ); const uint64_t claim_begin = Ops::Now(); - const ClaimOutcome claim = Claim(state, worker, task_id, kind); + const ClaimOutcome claim = Claim(state, worker, task_id, kind, stats); winner = claim.won; context.won = winner; context.kernel_id = claim.function_id; const uint64_t claim_end = Ops::Now(); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Claim, - ProfilePhase::Claim, claim_begin, claim_end, winner ? 1U : 0U, 1 + ProfilePhase::Claim, claim_begin, claim_end, + (winner ? kClaimWon : 0U) | (claim.attempted ? kClaimAttempted : 0U), 1 ); RecordClaimOutcome(stats, kind, claim); if (winner) { @@ -500,7 +539,7 @@ PA_DEVICE bool SubmitTask( } } else { const uint64_t claim_begin = Ops::Now(); - const ClaimOutcome claim = Claim(state, worker, task_id, kind); + const ClaimOutcome claim = Claim(state, worker, task_id, kind, stats); winner = claim.won; function_id = claim.function_id; context.won = winner; @@ -508,7 +547,8 @@ PA_DEVICE bool SubmitTask( const uint64_t claim_end = Ops::Now(); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Claim, - ProfilePhase::Claim, claim_begin, claim_end, winner ? 1U : 0U, 0 + ProfilePhase::Claim, claim_begin, claim_end, + (winner ? kClaimWon : 0U) | (claim.attempted ? kClaimAttempted : 0U), 0 ); RecordClaimOutcome(stats, kind, claim); @@ -636,6 +676,7 @@ PA_DEVICE void PublishResult(PA_GM WorkerResult &destination, const WorkerResult PA_PUBLISH_FIELD(frontier_initial_loads); PA_PUBLISH_FIELD(frontier_updates); PA_PUBLISH_FIELD(frontier_terminal_loads); + PA_PUBLISH_FIELD(atomic_trace_calls); #undef PA_PUBLISH_FIELD Ops::StoreBarrier(); } @@ -679,14 +720,18 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, // 96 个参与者全部完成本地状态初始化后再进入 task 0,主要用于压低启动偏斜对 // winner 分布和 Submit 时序的干扰;atomicMax 的唯一 winner 正确性本身不依赖该屏障。 - Ops::FetchAdd(&state->started_count.value, 1); + TraceAtomicFetchAdd( + stats.trace, stats.result, -1, AtomicSite::StartupIncrement, + &state->started_count.value, 1 + ); const uint64_t start_wait = Ops::Now(); uint32_t start_polls = 0; // 全员到齐或任一核发布 fatal 即退出启动等待;watchdog 防止缺失参与者造成永久挂死。 - while (LoadLine(state->started_count) < static_cast(state->config.workers) && - !IsFatal(state)) { + while (LoadLine(state->started_count, stats, AtomicSite::StartupPoll) < + static_cast(state->config.workers) && + !IsFatal(state, stats)) { Ops::SpinHint(); - if (WatchdogExpired(state, start_wait, start_polls)) { + if (WatchdogExpired(state, stats, start_wait, start_polls)) { break; } } @@ -696,7 +741,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, PaOrchestrationState orchestration; TaskArgs args; SubmitContext context; - if (!IsFatal(state)) { + if (!IsFatal(state, stats)) { // Case1 每个 batch 固定回放 Alloc/QK/SF/PV/UP 五个 task;所有 worker 顺序相同,执行 lane 由 Claim 筛选。 ResetTraceLap(worker); InitPaOrchestration(orchestration, batches, &state->context_lens[0]); @@ -760,12 +805,16 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, } // replay_done 表示所有 worker 已退出回放循环(成功路径即完整提交);之后仍需 drain 到本核 slot 为空。 - Ops::FetchAdd(&state->replay_done.value, 1); + TraceAtomicFetchAdd( + stats.trace, stats.result, -1, AtomicSite::ReplayDoneIncrement, + &state->replay_done.value, 1 + ); while (true) { const uint32_t freed = DrainReady(state, worker, DrainPlace::FinalDrain, stats); const bool all_replayed = - LoadLine(state->replay_done) >= static_cast(state->config.workers); + LoadLine(state->replay_done, stats, AtomicSite::ReplayDonePoll) >= + static_cast(state->config.workers); // 必须同时满足“无人再生产新 slot”和“本核旧 slot 全部完成”,否则继续帮助系统推进 completion。 if (all_replayed && worker.occupied_count == 0) { break; @@ -775,6 +824,28 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, } } + if (stats.trace.atomics_enabled) { + // 两条基线都放在最终 drain 之后。第一条量连续 + // SYS_CNT,第二条量返回依赖钩子的固定成本;它们只描述计时底噪,不能 + // 从每条 atomic 中机械相减后宣称得到跨核全局可见性延迟。 + const uint64_t clock_begin = Ops::Now(); + const uint64_t clock_end = Ops::Now(); + WriteTrace( + stats.trace, stats.result, -1, -1, TracePhase::ClockBaseline, + ProfilePhase::ReplayTail, clock_begin, clock_end + ); + const uint64_t dependency_begin = Ops::Now(); + const uint64_t dependency_end = Ops::NowAfterAtomicResult( + static_cast(worker_id) + ); + WriteTrace( + stats.trace, stats.result, -1, -1, TracePhase::ClockBaseline, + ProfilePhase::ReplayTail, dependency_begin, dependency_end, + kClockAtomicDependency | + (Ops::kAtomicReturnReadyObserved ? kClockAtomicDependencyApplied : 0U) + ); + } + // PA writes swimlane records through the ordinary GM cache and explicitly // cleans each worker's record range before the kernel finishes. FlushTraceCore(stats.trace); diff --git a/tests/atomic_probe/pa_scheduler/common/pa_trace.h b/tests/atomic_probe/pa_scheduler/common/pa_trace.h index 5bd77387b1..07270f6d25 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_trace.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_trace.h @@ -28,6 +28,7 @@ struct TraceContext { PA_GM TraceCoreState *core; PA_GM TraceRecord *records; uint32_t capacity; + bool atomics_enabled; int32_t lane; int32_t block_id; int32_t core_idx; @@ -39,11 +40,12 @@ template PA_DEVICE TraceContext AttachTrace( PA_GM SchedulerState *state, PA_GM const WorkerState &worker, uint32_t worker_id ) { - TraceContext trace{nullptr, nullptr, 0, worker.lane, worker.block_id, static_cast(worker_id)}; + TraceContext trace{nullptr, nullptr, 0, false, worker.lane, worker.block_id, static_cast(worker_id)}; Ops::InvalidateRegion(&state->config, sizeof(state->config)); const uint64_t base = state->config.trace_base; const uint32_t capacity = state->config.trace_records_per_core; - if (state->config.trace_enabled == 0 || base == 0 || capacity == 0 || worker_id >= kWorkers) { + if ((state->config.trace_enabled & kTracePhasesEnabled) == 0 || base == 0 || capacity == 0 || + worker_id >= kWorkers) { return trace; } PA_GM TraceHeader *header = reinterpret_cast(base); @@ -55,11 +57,115 @@ PA_DEVICE TraceContext AttachTrace( // 成功返回的不变量是 core/records/capacity 同时有效;任一前置条件失败则三者 // 保持空值,后续 WriteTrace/FlushTraceCore 可无分支地安全退化为 no-op。 trace.capacity = capacity; + trace.atomics_enabled = (state->config.trace_enabled & kTraceAtomicsEnabled) != 0; trace.core->count = 0; trace.core->dropped = 0; return trace; } +template +PA_DEVICE void WriteTrace( + TraceContext &trace, WorkerResult &result, int32_t task_id, int32_t function_id, + TracePhase trace_phase, ProfilePhase profile_phase, uint64_t start_cycle, uint64_t end_cycle, + uint32_t flags = 0, uint32_t auxiliary = 0 +); + +PA_DEVICE uint32_t AtomicTraceFlags( + AtomicOp op, bool result_used, bool return_ready, bool value_zero = false, + uint64_t retries = 0 +) { + // 高 24 bit 只能容纳有限重试次数;A5 硬件 atomicMax 当前报告 0,CPU CAS + // 回归若超过范围则饱和,避免溢出覆盖低位的 op/语义标志。 + constexpr uint64_t kMaxRetries = (1ULL << (32 - kAtomicRetriesShift)) - 1; + const uint32_t encoded_retries = static_cast(retries > kMaxRetries ? kMaxRetries : retries); + return static_cast(op) | (result_used ? kAtomicResultUsed : 0U) | + (value_zero ? kAtomicValueZero : 0U) | (return_ready ? kAtomicReturnReady : 0U) | + (encoded_retries << kAtomicRetriesShift); +} + +template +PA_DEVICE void WriteAtomicTrace( + TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, AtomicOp op, + uint64_t start_cycle, uint64_t end_cycle, bool result_used, bool return_ready, + bool value_zero = false, uint64_t retries = 0 +) { + // 一次源码 atomic 只写一条同时含 start/end 的 span;结束时间先于 64B record + // 写入,因此本条区间不直接包含自己的记录写开销,但下一次竞争到达会受它影响。 + ++result.atomic_trace_calls; + WriteTrace( + trace, result, task_id, -1, TracePhase::Atomic, ProfilePhase::ReplayTail, + start_cycle, end_cycle, + AtomicTraceFlags(op, result_used, return_ready, value_zero, retries), + static_cast(site) + ); +} + +template +PA_DEVICE T TraceAtomicLoad( + TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, + PA_GM volatile T *address, bool result_used = true +) { + if (!trace.atomics_enabled) return Ops::Load(address); + const uint64_t begin = Ops::Now(); + const T old = Ops::Load(address); + // CCEC 只在返回值本来就参与协议判断时插入一条依赖 MOV,再读 SYS_CNT。 + // 这样不会把未消费返回值的 RED/no-return 路径强制改成返回型 ATOM。 + const bool return_ready = result_used && Ops::kAtomicReturnReadyObserved; + const uint64_t end = result_used ? Ops::NowAfterAtomicResult(old) : Ops::Now(); + WriteAtomicTrace( + trace, result, task_id, site, AtomicOp::Load, begin, end, result_used, return_ready, + old == static_cast(0) + ); + return old; +} + +template +PA_DEVICE T TraceAtomicExchange( + TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, + PA_GM volatile T *address, T value, bool result_used = false +) { + if (!trace.atomics_enabled) return Ops::Exchange(address, value); + const uint64_t begin = Ops::Now(); + const T old = Ops::Exchange(address, value); + const uint64_t end = Ops::Now(); + WriteAtomicTrace( + trace, result, task_id, site, AtomicOp::Exchange, begin, end, result_used, false + ); + return old; +} + +template +PA_DEVICE int64_t TraceAtomicFetchAdd( + TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, + PA_GM volatile int64_t *address, int64_t value, bool result_used = false +) { + if (!trace.atomics_enabled) return Ops::FetchAdd(address, value); + const uint64_t begin = Ops::Now(); + const int64_t old = Ops::FetchAdd(address, value); + const uint64_t end = Ops::Now(); + WriteAtomicTrace( + trace, result, task_id, site, AtomicOp::FetchAdd, begin, end, result_used, false + ); + return old; +} + +template +PA_DEVICE int64_t TraceAtomicFetchMax( + TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, + PA_GM volatile int64_t *address, int64_t value, uint64_t &retries, bool result_used = true +) { + if (!trace.atomics_enabled) return Ops::FetchMax(address, value, retries); + const uint64_t begin = Ops::Now(); + const int64_t old = Ops::FetchMax(address, value, retries); + const bool return_ready = result_used && Ops::kAtomicReturnReadyObserved; + const uint64_t end = result_used ? Ops::NowAfterAtomicResult(old) : Ops::Now(); + WriteAtomicTrace( + trace, result, task_id, site, AtomicOp::FetchMax, begin, end, result_used, + return_ready, false, retries + ); + return old; +} + template PA_DEVICE void AccumulatePhase( WorkerResult &result, ProfilePhase phase, uint64_t start_cycle, uint64_t end_cycle @@ -83,8 +189,8 @@ PA_DEVICE void AccumulatePhase( template PA_DEVICE void WriteTrace( TraceContext &trace, WorkerResult &result, int32_t task_id, int32_t function_id, TracePhase trace_phase, - ProfilePhase profile_phase, uint64_t start_cycle, uint64_t end_cycle, uint32_t flags = 0, - uint32_t auxiliary = 0 + ProfilePhase profile_phase, uint64_t start_cycle, uint64_t end_cycle, uint32_t flags, + uint32_t auxiliary ) { // 每段先更新轻量 phase 统计,再按需写 64-byte 原始记录。一个分区只有对应 // worker 写入,因此 count/dropped 保持普通单写者更新,不额外引入 atomic。 diff --git a/tests/atomic_probe/pa_scheduler/cpu/main.cpp b/tests/atomic_probe/pa_scheduler/cpu/main.cpp index f01adba2b8..98b03678d7 100644 --- a/tests/atomic_probe/pa_scheduler/cpu/main.cpp +++ b/tests/atomic_probe/pa_scheduler/cpu/main.cpp @@ -59,6 +59,10 @@ inline void RuntimeNop(uint32_t count) { } struct CpuOps { + // CPU 原子 built-in 在函数返回前已产生旧值;该后端只做协议回归,不把 + // x86 时间分布外推到 A5。 + static constexpr bool kAtomicReturnReadyObserved = true; + // 用 fetch_add(0) 模拟 A5 atomicAdd(addr, 0) 原子读,而不是退化为普通 // CPU load。Acquire/AcqRel 只建立本 CPU 协议回归需要的发布/观察关系, // 不模拟 A5 cache 或设备内存模型细节。 @@ -118,6 +122,13 @@ struct CpuOps { ); } + template + static inline uint64_t NowAfterAtomicResult(T value) { + // 空 asm 让编译器保留返回值到计时点的数据依赖,不额外插入 CPU fence。 + asm volatile("" : "+r"(value)); + return Now(); + } + static inline void Nop(uint32_t count) { RuntimeNop(count); } static inline void SpinHint() {} diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index 7fc881173f..bba2cdba9c 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -30,6 +30,7 @@ Benchmark options: --nop-counts QK,SF,PV,UP --profile-phases --analyze-swimlane + --trace-atomics --swimlane-json FILE --no-swimlane @@ -37,6 +38,10 @@ CCEC-only PMU probe options (the host must be launched by msprof PipeUtilization --pmu-window off|empty|scalar|scalar-double --pmu-scalar-nops N +The swimlane action enables atomic tracing by default. For the lower-level run +action, --trace-atomics still requires swimlane tracing; add +--analyze-swimlane to print the per-role/per-site timing distributions. + The swimlane action performs exactly one run and writes both the raw capture and merged Perfetto JSON below this directory's outputs/ folder. It rejects --runs, --swimlane-json, and --no-swimlane because those are managed by the action. @@ -185,7 +190,9 @@ case "$ACTION" in mkdir -p "$BACKEND_OUTPUT" # runner 先执行单轮严格语义校验并流式写 raw;成功后才调用本地 # converter 生成 Perfetto 文件。set -e 保证任一步失败即停止。 - run_backend "$backend" --runs 1 --swimlane-json "$RAW_JSON" "$@" + # 用户要求泳道默认带齐逐 atomic 性能。重复传入 --trace-atomics + # 只是幂等布尔开关,不会产生两份记录;直接 run 仍可选择 phase-only。 + run_backend "$backend" --runs 1 --trace-atomics --swimlane-json "$RAW_JSON" "$@" "$PYTHON_BIN" "$SCRIPT_DIR/swimlane_converter.py" "$RAW_JSON" -o "$MERGED_JSON" done echo "[SWIMLANE] output_root=$OUTPUT_ROOT" diff --git a/tests/atomic_probe/pa_scheduler/swimlane_converter.py b/tests/atomic_probe/pa_scheduler/swimlane_converter.py index 611d3c84b6..b18edbc76b 100755 --- a/tests/atomic_probe/pa_scheduler/swimlane_converter.py +++ b/tests/atomic_probe/pa_scheduler/swimlane_converter.py @@ -40,11 +40,40 @@ "Claim": "claim", "Fanin": "fanin", "Register": "register", + "Atomic": "atomic", + "ClockBaseline": "clock_baseline", } KERNEL_NAMES = {0: "QK", 1: "SF", 2: "PV", 3: "UP"} # 一个物理 mixed block 的三条 runtime lane:AIC、AIV0、AIV1。 LANE_NAMES = {0: "AIC", 1: "AIV0", 2: "AIV1"} +# Atomic raw ABI:auxiliary 存放调用点,flags 低 4 位存放操作类型。这里的 +# 数值必须与 standalone C++ AtomicSite/AtomicOp 枚举保持一致;未知值仍会 +# 以 site_/op_ 完整导出,便于识别版本不匹配,不会伪装成已知操作。 +ATOMIC_SITE_NAMES = { + 0: "startup_increment", + 1: "startup_poll", + 2: "fatal_poll", + 3: "fatal_set", + 4: "claim_max", + 5: "fanin_flag_load", + 6: "completion_vend_exchange", + 7: "completion_flag_exchange", + 8: "frontier_initial_load", + 9: "frontier_flag_load", + 10: "frontier_max", + 11: "heap_frontier_load", + 12: "heap_vend_load", + 13: "replay_done_increment", + 14: "replay_done_poll", +} +ATOMIC_OP_NAMES = { + 0: "load", + 1: "exchange", + 2: "fetch_add", + 3: "fetch_max", +} + # 把可转为整数的 raw 标量归一为 int,并在错误中保留精确字段路径。 def _integer(value: Any, label: str) -> int: @@ -56,7 +85,9 @@ def _integer(value: Any, label: str) -> int: # 读取 raw JSON,校验十列结构、字段范围与可转整数值,并返回规范化视图。 -def _load_and_validate(input_path: Path) -> tuple[int, list[tuple[Any, ...]], dict[tuple[int, int], int], int]: +def _load_and_validate( + input_path: Path, +) -> tuple[int, int, list[tuple[Any, ...]], dict[tuple[int, int], int], int]: # raw 文件沿用真实 l2_swimlane_records.json 的十列 fdwic_events ABI: # core、block、lane、task、func、phase、start、end、flags、aux。 with input_path.open("r", encoding="utf-8") as input_file: @@ -75,6 +106,11 @@ def _load_and_validate(input_path: Path) -> tuple[int, list[tuple[Any, ...]], di frequency_hz = _integer(metadata.get("clock_freq_hz"), "metadata.clock_freq_hz") if frequency_hz <= 0: raise ValueError("metadata.clock_freq_hz must be positive") + # v1 是旧 raw,Claim flags 只有 winner bit;v2 追加 attempted bit。 + # 不认识的新版本直接拒绝,避免把新 flags 按旧语义误读。 + trace_schema_version = _integer(metadata.get("trace_schema_version", 1), "metadata.trace_schema_version") + if trace_schema_version not in (1, 2): + raise ValueError(f"unsupported metadata.trace_schema_version: {trace_schema_version}") num_cores = _integer(metadata.get("num_cores"), "metadata.num_cores") if num_cores <= 0: raise ValueError("metadata.num_cores must be positive") @@ -113,6 +149,11 @@ def _load_and_validate(input_path: Path) -> tuple[int, list[tuple[Any, ...]], di raise ValueError(f"fdwic_events[{index}] has invalid lane {lane}") if phase not in PHASE_NAMES: raise ValueError(f"fdwic_events[{index}] has unknown phase {phase!r}") + if phase == "Claim" and trace_schema_version >= 2: + if flags & ~0x3 or (flags & 0x1 and not flags & 0x2): + raise ValueError( + f"fdwic_events[{index}] has invalid Claim flags 0x{flags:x}" + ) if start_cycle <= 0 or end_cycle < start_cycle: raise ValueError( f"fdwic_events[{index}] has invalid cycles start={start_cycle} end={end_cycle}" @@ -140,7 +181,7 @@ def _load_and_validate(input_path: Path) -> tuple[int, list[tuple[Any, ...]], di ) assert base_cycle is not None - return frequency_hz, rows, core_by_block_lane, base_cycle + return frequency_hz, trace_schema_version, rows, core_by_block_lane, base_cycle # 写一个 Chrome Trace Event,并统一处理数组元素间的逗号。 @@ -154,7 +195,7 @@ def _emit_event(output: TextIO, event: dict[str, Any], first: bool) -> bool: # 完成一次 raw 到 merged 的转换,成功时返回事件数、block 数和基准 cycle。 def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: - frequency_hz, rows, core_by_block_lane, base_cycle = _load_and_validate(input_path) + frequency_hz, trace_schema_version, rows, core_by_block_lane, base_cycle = _load_and_validate(input_path) # 禁止原地转换;否则创建临时文件或最终 replace 时可能破坏唯一一份 raw。 if input_path.resolve() == output_path.resolve(): raise ValueError("input and output paths must differ") @@ -167,6 +208,20 @@ def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: # Perfetto 的显示精度。1 GHz A5 counter 因此每 tick 对应 0.001 us。 factor = 1_000_000.0 / float(frequency_hz) blocks = sorted({block_id for block_id, _ in core_by_block_lane}) + # v1 raw 没有 Claim attempted bit。若同一份 capture 确实含逐 atomic + # 记录,则可以用同核、同 task 且时间被 Claim 完整包含的 + # claim_max 作为实测证据恢复 attempted;不含 atomic 时保留 unknown, + # 绝不根据 task kind 或 AIC/AIV role 在转换器中猜业务路由。 + legacy_claim_max_spans: dict[tuple[int, int, int, int], list[tuple[int, int]]] = {} + has_atomic_trace = False + if trace_schema_version == 1: + for core_id, block_id, lane, task_id, _, phase, start, end, _, auxiliary in rows: + if phase != "Atomic": + continue + has_atomic_trace = True + if auxiliary == 4: # AtomicSite::ClaimMax + key = (core_id, block_id, lane, task_id) + legacy_claim_max_spans.setdefault(key, []).append((start, end)) first = True emitted = 0 # 临时文件的整个生命周期都在 try 内;包括 Ctrl-C 在内的异常都会先清理 @@ -211,13 +266,63 @@ def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: }, first, ) - for row in rows: core_id, block_id, lane, task_id, function_id, phase_raw, start, end, flags, auxiliary = row phase = PHASE_NAMES[phase_raw] - # Kernel 和 Commit 放到 lane+3 的 kernel 子泳道;其他阶段留在 - # lane 0..2 的 runtime 泳道。这与真实 PA merged 文件的布局一致。 - if phase == "kernel" and function_id >= 0: + # Kernel/Commit 放到 lane+3 的计算单元子泳道;Atomic/ClockBaseline + # 都是 AIC/AIV 对应 scalar 上执行的指令,必须与 runtime 阶段共用 + # lane 0..2。这样 atomic span 作为 Claim/Fanin/轮询等阶段的子区间 + # 叠加显示,不会伪装成 AIC/AIV 之外的第三类执行单元。 + if phase == "claim": + claim_attempted: bool | None + claim_attempted_source: str + if trace_schema_version >= 2: + claim_attempted = bool(flags & 0x2) + claim_attempted_source = "raw_flag" + elif has_atomic_trace: + key = (core_id, block_id, lane, task_id) + matched_claim_max = any( + atomic_start >= start and atomic_end <= end + for atomic_start, atomic_end in legacy_claim_max_spans.get(key, []) + ) + # 命中的 claim_max 能正向证明 attempted;但 v1 raw 没有 + # 显式“atomic 记录完整”元数据,未命中不能反向证明 + # not_attempted,因此保留 unknown。 + claim_attempted = True if matched_claim_max else None + claim_attempted_source = ( + "contained_claim_max" + if matched_claim_max + else "unknown_v1_without_matching_claim_max" + ) + else: + claim_attempted = None + claim_attempted_source = "unknown_v1_without_atomic_trace" + claim_won = bool(flags & 0x1) + if claim_attempted is False: + name = f"claim.not_attempted#{task_id}" + elif claim_attempted is True: + name = f"claim.{'won' if claim_won else 'lost'}#{task_id}" + else: + name = f"claim#{task_id}" + thread_id = lane + elif phase == "atomic": + atomic_site_id = auxiliary + atomic_op_id = flags & 0xF + atomic_site = ATOMIC_SITE_NAMES.get(atomic_site_id, f"site_{atomic_site_id}") + atomic_op = ATOMIC_OP_NAMES.get(atomic_op_id, f"op_{atomic_op_id}") + # 边界直接写入 span 名称,打开泳道后无需点开 args + # 就能区分“本核返回值可消费”和“只包围源码发射”。 + atomic_boundary_tag = "return_ready" if flags & (1 << 6) else "source_issue" + name = f"atomic.{atomic_boundary_tag}.{atomic_site}.{atomic_op}#{task_id}" + thread_id = lane + elif phase == "clock_baseline": + name = ( + "clock.atomic_return_dependency_hook" + if flags & 1 + else "clock.consecutive_sys_cnt_reads" + ) + thread_id = lane + elif phase == "kernel" and function_id >= 0: name = f"{KERNEL_NAMES.get(function_id, f'f{function_id}')}#{task_id}" thread_id = lane + 3 elif phase == "commit": @@ -245,6 +350,67 @@ def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: "aux": auxiliary, }, } + if phase == "atomic": + # Atomic 的 flags/aux 有独立 ABI,不沿用普通 phase 的 mc 语义。 + # cycles 保留原始整数,避免 Perfetto dur 的微秒浮点换算丢失短 atomic 精度。 + event["args"] = { + "phase": phase, + "task_id": task_id, + "func_id": function_id, + "core": core_id, + "site": atomic_site, + "site_id": atomic_site_id, + "op": atomic_op, + "op_id": atomic_op_id, + "cycles": end - start, + "result_used": bool(flags & (1 << 4)), + "return_ready_observed": bool(flags & (1 << 6)), + "completion_boundary": ( + "return_value_ready" + if flags & (1 << 6) + else "source_issue_bracket" + ), + "flags": flags, + "execution_unit": "scalar", + } + # 分类同样带边界,便于 Perfetto 过滤和分组;二者 + # 仍在同一 AIC/AIV scalar lane,不伪造并行执行单元。 + event["cat"] = f"atomic.{atomic_boundary_tag}" + # bit5 只对 Load 有意义;bits8..31 只对 FetchMax 表示饱和后的 retry 数。 + if atomic_op_id == 0: + event["args"]["value_zero"] = bool(flags & (1 << 5)) + if atomic_op_id == 3: + event["args"]["retries"] = (flags >> 8) & 0xFFFFFF + elif phase == "claim": + event["args"] = { + "phase": phase, + "task_id": task_id, + "func_id": function_id, + "core": core_id, + "claim_attempted": claim_attempted, + "claim_won": claim_won, + "claim_attempted_source": claim_attempted_source, + "claim_path": "alloc" if auxiliary == 1 else "kernel", + "execution_unit": "scalar", + "flags": flags, + } + event["cat"] = "scalar_scheduler" + elif phase == "clock_baseline": + dependency_hook = bool(flags & 1) + event["args"] = { + "phase": phase, + "core": core_id, + "ticks": end - start, + "clock_freq_hz": frequency_hz, + "definition": ( + "atomic-return-dependency-hook" + if dependency_hook + else "consecutive-sys-cnt-reads" + ), + "dependency_applied": bool(flags & 2) if dependency_hook else False, + "execution_unit": "scalar", + } + event["cat"] = "scalar_clock" first = _emit_event(output, event, first) emitted += 1 output.write("\n]}\n") diff --git a/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py b/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py new file mode 100644 index 0000000000..a2c1bc5e6c --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py @@ -0,0 +1,182 @@ +#!/usr/bin/env python3 +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +"""standalone 泳道转换器的最小布局回归。""" + +from __future__ import annotations + +import json +import tempfile +import unittest +from pathlib import Path + +try: + # `python -m unittest tests.atomic_probe...` 以 namespace package 导入。 + from .swimlane_converter import convert +except ImportError: + # 也保留从本目录直接执行脚本的用法。 + from swimlane_converter import convert + + +class SwimlaneConverterLayoutTest(unittest.TestCase): + def test_atomic_and_clock_share_the_scalar_lane(self) -> None: + # 同一 mixed block 放一条 AIC 和一条 AIV0;Atomic 是 AIC scalar + # 上 Claim 的子区间,ClockBaseline 也是 AIV0 scalar 指令,而 + # Kernel 是 AIV0 计算单元上的独立区间。 + capture = { + "l2_swimlane_level": 4, + "metadata": { + "clock_freq_hz": 1_000_000_000, + "num_cores": 2, + "trace_schema_version": 2, + "core_types": ["AIC", "AIV"], + }, + "fdwic_events": [ + # Claim flags: attempted(bit1),本核输了所以 winner(bit0)=0。 + [0, 0, 0, 7, -1, "Claim", 100, 200, 0x2, 0], + # flags: FetchMax(3) | result-used(bit4) | return-ready(bit6)。 + [0, 0, 0, 7, -1, "Atomic", 120, 160, 0x53, 4], + # Exchange(1) 的旧值未消费,只能标 source-issue。 + [0, 0, 0, 7, -1, "Atomic", 161, 162, 0x01, 7], + # flags: dependency-hook(bit0) | dependency-applied(bit1)。 + [1, 0, 1, -1, -1, "ClockBaseline", 101, 102, 0x3, 0], + [1, 0, 1, 7, 0, "Kernel", 140, 180, 0, 0], + # 同一 AIV0 上的 role-filtered Claim,没有 atomic。 + [1, 0, 1, 8, -1, "Claim", 201, 220, 0x0, 0], + ], + } + + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + emitted, blocks, base_cycle = convert(input_path, output_path) + merged = json.loads(output_path.read_text(encoding="utf-8")) + + self.assertEqual((emitted, blocks, base_cycle), (6, 1, 100)) + events = merged["traceEvents"] + thread_names = { + (event["pid"], event["tid"]): event["args"]["name"] + for event in events + if event.get("ph") == "M" and event.get("name") == "thread_name" + } + self.assertEqual(thread_names[(0, 0)], "AIC (core0)") + self.assertEqual(thread_names[(0, 1)], "AIV0 (core1)") + self.assertEqual(thread_names[(0, 4)], "AIV0·kernel (core1)") + self.assertFalse(any("·atomic" in name for name in thread_names.values())) + + ready_atomic = next(event for event in events if event.get("cat") == "atomic.return_ready") + issue_atomic = next(event for event in events if event.get("cat") == "atomic.source_issue") + clock = next(event for event in events if event.get("cat") == "scalar_clock") + kernel = next(event for event in events if event.get("name") == "QK#7") + self.assertEqual((ready_atomic["pid"], ready_atomic["tid"]), (0, 0)) + self.assertEqual((issue_atomic["pid"], issue_atomic["tid"]), (0, 0)) + self.assertEqual((clock["pid"], clock["tid"]), (0, 1)) + self.assertEqual((kernel["pid"], kernel["tid"]), (0, 4)) + self.assertEqual( + ready_atomic["name"], "atomic.return_ready.claim_max.fetch_max#7" + ) + self.assertEqual( + issue_atomic["name"], "atomic.source_issue.completion_flag_exchange.exchange#7" + ) + self.assertEqual(ready_atomic["args"]["execution_unit"], "scalar") + self.assertEqual(issue_atomic["args"]["execution_unit"], "scalar") + self.assertEqual(clock["args"]["execution_unit"], "scalar") + self.assertEqual(ready_atomic["args"]["completion_boundary"], "return_value_ready") + self.assertEqual(issue_atomic["args"]["completion_boundary"], "source_issue_bracket") + attempted_claim = next(event for event in events if event.get("name") == "claim.lost#7") + skipped_claim = next(event for event in events if event.get("name") == "claim.not_attempted#8") + self.assertTrue(attempted_claim["args"]["claim_attempted"]) + self.assertFalse(skipped_claim["args"]["claim_attempted"]) + self.assertEqual(attempted_claim["args"]["claim_attempted_source"], "raw_flag") + + def test_v1_claim_attempt_uses_contained_atomic_evidence(self) -> None: + # 历史 raw 没有 attempted bit。只在同一 capture 真有 claim_max 记录时 + # 恢复该语义,不根据 task_id 或 core role 猜测。 + capture = { + "l2_swimlane_level": 1, + "metadata": { + "clock_freq_hz": 1_000_000_000, + "num_cores": 1, + "core_types": ["AIC"], + }, + "fdwic_events": [ + [0, 0, 0, 1, -1, "Claim", 100, 200, 0, 0], + [0, 0, 0, 1, -1, "Atomic", 120, 160, 0x50, 4], + [0, 0, 0, 2, -1, "Claim", 210, 230, 0, 0], + ], + } + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + convert(input_path, output_path) + events = json.loads(output_path.read_text(encoding="utf-8"))["traceEvents"] + + attempted = next(event for event in events if event.get("name") == "claim.lost#1") + unknown = next(event for event in events if event.get("name") == "claim#2") + self.assertTrue(attempted["args"]["claim_attempted"]) + self.assertIsNone(unknown["args"]["claim_attempted"]) + self.assertEqual(attempted["args"]["claim_attempted_source"], "contained_claim_max") + self.assertEqual( + unknown["args"]["claim_attempted_source"], + "unknown_v1_without_matching_claim_max", + ) + + def test_v2_claim_states_do_not_require_atomic_records(self) -> None: + # v2 raw 直接携带 attempted/won,因此关闭 --trace-atomics 后仍能 + # 区分三种 Claim 状态,不依赖 converter 从业务拓扑推断。 + capture = { + "l2_swimlane_level": 1, + "metadata": { + "clock_freq_hz": 1_000_000_000, + "num_cores": 1, + "trace_schema_version": 2, + "core_types": ["AIC"], + }, + "fdwic_events": [ + [0, 0, 0, 1, -1, "Claim", 100, 110, 0x0, 0], + [0, 0, 0, 2, -1, "Claim", 120, 140, 0x2, 0], + [0, 0, 0, 3, 0, "Claim", 150, 180, 0x3, 0], + ], + } + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + convert(input_path, output_path) + events = json.loads(output_path.read_text(encoding="utf-8"))["traceEvents"] + + names = {event.get("name") for event in events} + self.assertIn("claim.not_attempted#1", names) + self.assertIn("claim.lost#2", names) + self.assertIn("claim.won#3", names) + + def test_v2_rejects_winner_without_attempt(self) -> None: + capture = { + "l2_swimlane_level": 1, + "metadata": { + "clock_freq_hz": 1_000_000_000, + "num_cores": 1, + "trace_schema_version": 2, + "core_types": ["AIC"], + }, + "fdwic_events": [[0, 0, 0, 1, 0, "Claim", 100, 110, 0x1, 0]], + } + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "invalid Claim flags"): + convert(input_path, output_path) + + +if __name__ == "__main__": + unittest.main() From c93bd65d15d9a48df51fe3810cc096fac092a762 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 09:14:33 +0000 Subject: [PATCH 012/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E6=8E=A5?= =?UTF-8?q?=E9=80=9A=E7=8B=AC=E7=AB=8BPA=E6=A0=87=E9=87=8FPMU=E6=89=80?= =?UTF-8?q?=E6=9C=89=E6=9D=83=E9=97=AD=E7=8E=AF?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 在 pa_scheduler standalone 目录内复刻已验证的 Main AICPU Path-A:dispatcher 负责安装内容指纹命名的 owner SO,mode0 JSON 注册统一 simpler_aicpu_exec 入口。 owner 在配置前保存 108 个物理子核的 PMU 控制与 selector 状态,只持有实际开放的 32 AIC + 64 AIV,并按 bitmap 完成失败回滚和退出恢复。 host 直接复用 owner control 内嵌的寄存器基址表,固定 Restore、MMIO unmap、device reset 的清理顺序,不再依赖外部 msprof 配置 selector。 增加 owner bitmap、1 AIC + 2 AIV 三元组、worker 槽位、物理角色和实际运行核集合闭环校验。 验证:精确暂存快照 CCEC 全量构建通过;A5 empty、100000 NOP、2x100000 NOP 三组均为 96/96 可信记录且 Restore PASS;total 中位数约 214、56568、112994 cycles。 --- tests/atomic_probe/pa_scheduler/ccec/build.sh | 45 +- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 81 ++- .../pa_scheduler/ccec/pmu_owner_aicpu.cpp | 661 ++++++++++++++++++ .../pa_scheduler/ccec/pmu_owner_control.h | 211 ++++++ .../ccec/pmu_owner_dispatcher.cpp | 186 +++++ .../pa_scheduler/ccec/pmu_owner_host.h | 345 +++++++++ .../pa_scheduler/ccec/pmu_owner_main_abi.h | 75 ++ .../pa_scheduler/ccec/pmu_owner_main_loader.h | 389 +++++++++++ 8 files changed, 1974 insertions(+), 19 deletions(-) create mode 100644 tests/atomic_probe/pa_scheduler/ccec/pmu_owner_aicpu.cpp create mode 100644 tests/atomic_probe/pa_scheduler/ccec/pmu_owner_control.h create mode 100644 tests/atomic_probe/pa_scheduler/ccec/pmu_owner_dispatcher.cpp create mode 100644 tests/atomic_probe/pa_scheduler/ccec/pmu_owner_host.h create mode 100644 tests/atomic_probe/pa_scheduler/ccec/pmu_owner_main_abi.h create mode 100644 tests/atomic_probe/pa_scheduler/ccec/pmu_owner_main_loader.h diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh index 574f2cb940..8d5c5e2a83 100755 --- a/tests/atomic_probe/pa_scheduler/ccec/build.sh +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -24,13 +24,14 @@ fi CCEC="$ASCEND_HOME_PATH/bin/ccec" LD="$ASCEND_HOME_PATH/bin/ld.lld" +HCC="$ASCEND_HOME_PATH/tools/hcc/bin/aarch64-target-linux-gnu-g++" CXX_BIN="${CXX:-g++}" READELF_BIN="${READELF:-readelf}" PTO_INCLUDE_ROOT="${PTO_ISA_ROOT:-$ASCEND_HOME_PATH/x86_64-linux}" # ccec/ld.lld 必须来自当前已 source 的 CANN;host 编译器和 readelf 允许用户通过环境变量替换。 -if [[ ! -x "$CCEC" || ! -x "$LD" ]]; then - echo "CCEC or ld.lld is missing under ASCEND_HOME_PATH=$ASCEND_HOME_PATH" >&2 +if [[ ! -x "$CCEC" || ! -x "$LD" || ! -x "$HCC" ]]; then + echo "CCEC, ld.lld, or the AICPU HCC compiler is missing under ASCEND_HOME_PATH=$ASCEND_HOME_PATH" >&2 exit 1 fi if ! command -v "$READELF_BIN" >/dev/null 2>&1; then @@ -97,6 +98,45 @@ for entry in pa_scheduler_0_mix_aic pa_scheduler_0_mix_aiv; do done echo "[CHECK] both 1:2 mixed entries and metadata sections are present" +# PMU selector/CTRL 的所有权必须由主 aicpu_scheduler 配置并在退出前恢复。 +# standalone 目录内自带 Path-A dispatcher 与 owner:前者负责把 owner SO +# 落到设备预安装目录,后者由 mode=0 JSON 注册并通过统一入口执行命令。 +echo "[BUILD] self-contained AICPU PMU dispatcher" +"$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra -Werror \ + -Wl,--build-id \ + "$SCRIPT_DIR/pmu_owner_dispatcher.cpp" \ + -o "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so" + +echo "[BUILD] self-contained AICPU PMU owner" +"$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra -Werror \ + -Wl,--build-id \ + -I"$SCRIPT_DIR" \ + "$SCRIPT_DIR/pmu_owner_aicpu.cpp" \ + -o "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so" + +OWNER_HEADER="$("$READELF_BIN" --file-header "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so")" +OWNER_SYMBOLS="$("$READELF_BIN" --dyn-syms --wide "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so")" +DISPATCHER_HEADER="$("$READELF_BIN" --file-header "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so")" +DISPATCHER_SYMBOLS="$("$READELF_BIN" --dyn-syms --wide "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so")" +if [[ "$OWNER_HEADER" != *"Type: DYN"* || + "$OWNER_HEADER" != *"Machine: AArch64"* || + "$DISPATCHER_HEADER" != *"Type: DYN"* || + "$DISPATCHER_HEADER" != *"Machine: AArch64"* ]]; then + echo "PMU dispatcher and owner must both be AArch64 shared objects." >&2 + exit 1 +fi +if [[ "$OWNER_SYMBOLS" != *" simpler_aicpu_exec"* ]]; then + echo "Missing main AICPU PMU owner entry: simpler_aicpu_exec" >&2 + exit 1 +fi +for entry in StaticTileFwkBackendKernelServer DynTileFwkBackendKernelServerInit DynTileFwkBackendKernelServer; do + if [[ "$DISPATCHER_SYMBOLS" != *" $entry"* ]]; then + echo "Missing AICPU PMU dispatcher entry: $entry" >&2 + exit 1 + fi +done +echo "[CHECK] Path-A dispatcher and main AICPU PMU owner exports are present" + # host runner 只链接用户 CANN 9.1 的 ACL/runtime,并写入同一安装目录的 rpath,运行时不需要 simpler 动态库。 # `-Werror` 让 host API 签名或尺寸类型变化在构建期暴露,避免到上板阶段才出现参数截断。 echo "[BUILD] CCEC host runner" @@ -105,6 +145,7 @@ echo "[BUILD] CCEC host runner" -I"$ASCEND_HOME_PATH/include" \ -I"$ASCEND_HOME_PATH/pkg_inc" \ -I"$ASCEND_HOME_PATH/pkg_inc/runtime" \ + -I"$ASCEND_HOME_PATH/pkg_inc/runtime/runtime" \ "$SCRIPT_DIR/host.cpp" \ -L"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ -Wl,-rpath,"$ASCEND_HOME_PATH/x86_64-linux/lib64" \ diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index c19f24d07d..1f361b1698 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -10,6 +10,7 @@ */ #include "../common/host_support.h" +#include "pmu_owner_host.h" #include "pmu_probe.h" #include "acl/acl.h" @@ -230,13 +231,19 @@ void PrintPmuAggregate(const char *name, const PmuAggregate &aggregate) { ); } -bool ValidatePmu(const pa_scheduler::SchedulerState &state, uint32_t run, const PmuOptions &pmu) { +bool ValidatePmu( + const pa_scheduler::SchedulerState &state, uint32_t run, const PmuOptions &pmu, + const pa_scheduler::pmu_owner::PmuOwnerControl *owner +) { using namespace pa_scheduler::ccec_pmu; if (pmu.mode == WindowMode::Off) return true; bool seen[kPhysicalSubcoreCount] = {}; uint32_t trusted = 0; uint32_t unique = 0; + uint32_t owner_members = 0; + uint32_t exact_worker_slots = 0; + uint32_t physical_role_matches = 0; uint32_t prior_larger = 0; uint32_t bad_printed = 0; PmuAggregate all; @@ -247,7 +254,12 @@ bool ValidatePmu(const pa_scheduler::SchedulerState &state, uint32_t run, const const uint32_t status = result.pmu_status; const uint32_t core_id = StatusCoreId(status); const bool record_trusted = (status & kStatusRequired) == kStatusRequired; + const bool logical_aic = worker < pa_scheduler::kAicWorkers; + const bool physical_aic = pa_scheduler::pmu_owner::IsAicPhysicalSlot(core_id); trusted += record_trusted; + owner_members += owner != nullptr && pa_scheduler::pmu_owner::IsConfigured(*owner, core_id); + exact_worker_slots += result.worker_id == worker; + physical_role_matches += logical_aic == physical_aic; prior_larger += (status & kStatusPriorSnapshotLarger) != 0; if (core_id < kPhysicalSubcoreCount && !seen[core_id]) { seen[core_id] = true; @@ -266,11 +278,32 @@ bool ValidatePmu(const pa_scheduler::SchedulerState &state, uint32_t run, const AddPmuSample(result, result.role == static_cast(pa_scheduler::CoreRole::Aic) ? &aic : &aiv); } + uint32_t mixed_triplet_matches = 0U; + for (uint32_t block = 0U; block < pa_scheduler::kAicWorkers; ++block) { + const uint32_t aic_id = StatusCoreId(state.results[block].pmu_status); + if (!pa_scheduler::pmu_owner::IsAicPhysicalSlot(aic_id)) continue; + const uint32_t die_base = (aic_id / pa_scheduler::pmu_owner::kSubcoresPerDie) * + pa_scheduler::pmu_owner::kSubcoresPerDie; + const uint32_t local = aic_id % pa_scheduler::pmu_owner::kSubcoresPerDie; + const uint32_t expected_aiv0 = die_base + pa_scheduler::pmu_owner::kAicPerDie + local * 2U; + const uint32_t aiv0_id = StatusCoreId( + state.results[pa_scheduler::kAicWorkers + block * 2U].pmu_status + ); + const uint32_t aiv1_id = StatusCoreId( + state.results[pa_scheduler::kAicWorkers + block * 2U + 1U].pmu_status + ); + mixed_triplet_matches += aiv0_id == expected_aiv0 && aiv1_id == expected_aiv0 + 1U; + } + PrintPmuAggregate("ALL", all); PrintPmuAggregate("AIC", aic); PrintPmuAggregate("AIV", aiv); const bool records_ok = trusted == pa_scheduler::kWorkers; const bool core_ids_ok = unique == pa_scheduler::kWorkers; + const bool owner_members_ok = owner_members == pa_scheduler::kWorkers; + const bool worker_slots_ok = exact_worker_slots == pa_scheduler::kWorkers; + const bool physical_roles_ok = physical_role_matches == pa_scheduler::kWorkers; + const bool mixed_triplets_ok = mixed_triplet_matches == pa_scheduler::kAicWorkers; std::printf( "[PMU] run=%u window=%s scalar_nops=%u trusted=%u/%u unique_coreids=%u/%u prior_larger=%u/%u\n", run, PmuModeName(pmu.mode), pmu.scalar_nops, trusted, pa_scheduler::kWorkers, unique, pa_scheduler::kWorkers, @@ -280,7 +313,16 @@ bool ValidatePmu(const pa_scheduler::SchedulerState &state, uint32_t run, const records_ok ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "all 96 PMU physical subcore ids are unique", core_ids_ok ? "PASS" : "FAIL"); - return records_ok && core_ids_ok; + std::printf("[ASSERT] %-48s %s\n", "all PMU physical ids belong to the owner bitmap", + owner_members_ok ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "worker result slots and ids match exactly", + worker_slots_ok ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "logical AIC/AIV roles match physical subcores", + physical_roles_ok ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "all 32 mixed blocks map to physical 1:2 triplets", + mixed_triplets_ok ? "PASS" : "FAIL"); + return records_ok && core_ids_ok && owner_members_ok && worker_slots_ok && + physical_roles_ok && mixed_triplets_ok; } } // namespace @@ -310,7 +352,7 @@ int main(int argc, char **argv) { } pa_scheduler::host::PrintBanner("CCEC", options); std::printf( - "[PMU-CONFIG] window=%s scalar_nops=%u source=direct-per-core requires=msprof-PipeUtilization\n", + "[PMU-CONFIG] window=%s scalar_nops=%u source=direct-per-core owner=main-aicpu-path-a\n", PmuModeName(pmu_options.mode), pmu_options.scalar_nops ); @@ -349,24 +391,25 @@ int main(int argc, char **argv) { } PmuRegisterMappings pmu_mappings; - void *pmu_registers_device = nullptr; + pa_scheduler::pmu_owner::PmuOwnerSession pmu_owner; + const void *pmu_registers_device = nullptr; if (pmu_options.mode != pa_scheduler::ccec_pmu::WindowMode::Off) { if (!MapPmuRegisters(options.device, &pmu_mappings)) return EXIT_FAILURE; - const size_t register_bytes = pmu_mappings.register_bases.size() * sizeof(uint64_t); - if (!CheckAcl( - aclrtMalloc(&pmu_registers_device, register_bytes, ACL_MEM_MALLOC_NORMAL_ONLY), - "aclrtMalloc(PMU register table)" + const std::string dispatcher_path = pa_scheduler::pmu_owner::ArtifactBesideKernel( + options.kernel_path, "libpa_scheduler_pmu_owner_dispatcher.so" + ); + const std::string owner_path = pa_scheduler::pmu_owner::ArtifactBesideKernel( + options.kernel_path, "libpa_scheduler_pmu_owner_aicpu.so" + ); + if (!pmu_owner.Initialize( + options.device, stream, dispatcher_path, owner_path, pmu_mappings.register_bases ) || - !CheckAcl( - aclrtMemcpy( - pmu_registers_device, register_bytes, pmu_mappings.register_bases.data(), register_bytes, - ACL_MEMCPY_HOST_TO_DEVICE - ), - "aclrtMemcpy(H2D PMU register table)" - )) { + !pmu_owner.Configure()) { + (void)pmu_owner.Finalize(); (void)UnmapPmuRegisters(options.device, &pmu_mappings); return EXIT_FAILURE; } + pmu_registers_device = reinterpret_cast(pmu_owner.RegisterTableDeviceAddress()); } // 泳道区按 96 worker 各 65536 条记录预留,约 384 MiB;关闭泳道时不申请,也不会传递有效 base。 @@ -497,7 +540,10 @@ int main(int argc, char **argv) { *state, run, host_us, options.trace_enabled ? &trace_header : nullptr ); all_passed &= metrics.passed; - all_passed &= ValidatePmu(*state, run, pmu_options); + all_passed &= ValidatePmu( + *state, run, pmu_options, + pmu_options.mode == pa_scheduler::ccec_pmu::WindowMode::Off ? nullptr : &pmu_owner.Control() + ); spans.push_back(metrics.submit_span_us); if (options.analyze_swimlane && !pa_scheduler::host::AnalyzeSwimlaneRecords(trace_header, *state, read_trace_records)) { @@ -534,7 +580,8 @@ int main(int argc, char **argv) { cleanup_ok &= CheckAcl(aclrtFree(trace_device), "aclrtFree(swimlane trace)"); } if (pmu_registers_device != nullptr) { - cleanup_ok &= CheckAcl(aclrtFree(pmu_registers_device), "aclrtFree(PMU register table)"); + // owner 必须在 MMIO 映射、device context 和 ACL runtime 仍有效时恢复。 + cleanup_ok &= pmu_owner.Finalize(); cleanup_ok &= UnmapPmuRegisters(options.device, &pmu_mappings); } cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(state)"); diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_aicpu.cpp b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_aicpu.cpp new file mode 100644 index 0000000000..31237bfc12 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_aicpu.cpp @@ -0,0 +1,661 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#include "pmu_owner_control.h" +#include "pmu_owner_main_abi.h" + +#include +#include + +namespace { + +using pa_scheduler::pmu_owner::PmuOwnerControl; +using pa_scheduler::pmu_owner::PmuOwnerField; +using pa_scheduler::pmu_owner::PmuOwnerMainCommand; +using pa_scheduler::pmu_owner::PmuOwnerMainKernelArgs; +using pa_scheduler::pmu_owner::PmuOwnerStatus; +using pa_scheduler::pmu_owner::PmuSavedRegisters; + +constexpr size_t kCacheLineBytes = 64U; + +// 以下 offset 逐项核对自当前 A5 的 platform_config.h 与 onboard +// inner_platform_regs.cpp。standalone helper 在本目录重写这些常量,绝不 +// include pa_scheduler 目录外的 Simpler platform 实现。 +constexpr uint32_t kCtrl0Offset = 0x4200U; +constexpr uint32_t kCtrl1Offset = 0x2400U; +constexpr uint32_t kCounterOffsets[pa_scheduler::pmu_owner::kPmuCounterCount] = { + 0x4210U, 0x4218U, 0x4220U, 0x4228U, 0x4230U, + 0x4238U, 0x4240U, 0x4248U, 0x4250U, 0x4254U, +}; +constexpr uint32_t kSelectorOffsets[pa_scheduler::pmu_owner::kPmuCounterCount] = { + 0x2500U, 0x2504U, 0x2508U, 0x250cU, 0x2510U, + 0x2514U, 0x2518U, 0x251cU, 0x2520U, 0x2524U, +}; +constexpr uint32_t kTotalLowOffset = 0x4260U; +constexpr uint32_t kTotalHighOffset = 0x4264U; +constexpr uint32_t kStartCycleLowOffset = 0x42a0U; +constexpr uint32_t kStartCycleHighOffset = 0x42a4U; +constexpr uint32_t kStopCycleLowOffset = 0x42a8U; +constexpr uint32_t kStopCycleHighOffset = 0x42acU; +constexpr uint32_t kCtrl0Enabled = 0x7U; +constexpr uint32_t kCtrl1Enabled = 0x1U; + +static_assert( + sizeof(kCounterOffsets) / sizeof(kCounterOffsets[0]) == pa_scheduler::pmu_owner::kPmuCounterCount, + "PMU counter offset count changed" +); +static_assert( + sizeof(kSelectorOffsets) / sizeof(kSelectorOffsets[0]) == pa_scheduler::pmu_owner::kPmuCounterCount, + "PMU selector offset count changed" +); + +inline void InstructionBarrier() +{ + __asm__ volatile("isb" ::: "memory"); +} + +inline void FullSystemBarrier() +{ + __asm__ volatile("dsb sy" ::: "memory"); +} + +// Host 的 H2D/D2H 不会维护 AICPU L1。一条 Configure/Restore 命令开始前 +// 对 control 全区间执行 CIVAC,避免复用同一 GM 地址时读到上一轮 cache 内容。 +void InvalidateControl(const PmuOwnerControl *control) +{ + const uintptr_t begin = reinterpret_cast(control) & ~(kCacheLineBytes - 1U); + const uintptr_t end = + (reinterpret_cast(control) + sizeof(*control) + kCacheLineBytes - 1U) & + ~(kCacheLineBytes - 1U); + for (uintptr_t address = begin; address < end; address += kCacheLineBytes) { + __asm__ volatile("dc civac, %0" : : "r"(address) : "memory"); + } + FullSystemBarrier(); + InstructionBarrier(); +} + +// 命令完成后 clean 整个 control:不仅发布 status,也发布 bitmap、诊断以及 +// Configure 保存区。最后的 DSB/ISB 保证 runtime 同步返回后 host D2H 可见。 +void CleanControl(const PmuOwnerControl *control) +{ + const uintptr_t begin = reinterpret_cast(control) & ~(kCacheLineBytes - 1U); + const uintptr_t end = + (reinterpret_cast(control) + sizeof(*control) + kCacheLineBytes - 1U) & + ~(kCacheLineBytes - 1U); + for (uintptr_t address = begin; address < end; address += kCacheLineBytes) { + __asm__ volatile("dc cvac, %0" : : "r"(address) : "memory"); + } + FullSystemBarrier(); + InstructionBarrier(); +} + +inline volatile uint32_t *MmioPointer(uint64_t base, uint32_t offset) +{ + return reinterpret_cast(static_cast(base + offset)); +} + +inline uint32_t ReadMmio(uint64_t base, uint32_t offset) +{ + return *MmioPointer(base, offset); +} + +inline void WriteMmio(uint64_t base, uint32_t offset, uint32_t value) +{ + *MmioPointer(base, offset) = value; +} + +constexpr PmuOwnerField SelectorField(uint32_t counter) +{ + return static_cast( + static_cast(PmuOwnerField::Selector0) + counter + ); +} + +void ResetDiagnostics(PmuOwnerControl *control) +{ + control->first_failed_index = pa_scheduler::pmu_owner::kDiagnosticIndexUnset; + control->first_failed_field = static_cast(PmuOwnerField::None); + control->first_failed_observed = 0U; + control->first_failed_expected = 0U; + control->restore_failures = 0U; + control->first_restore_failed_index = pa_scheduler::pmu_owner::kDiagnosticIndexUnset; + control->first_restore_failed_field = static_cast(PmuOwnerField::None); + control->first_restore_failed_observed = 0U; + control->first_restore_failed_expected = 0U; +} + +void RecordFirstFailure( + PmuOwnerControl *control, uint32_t index, PmuOwnerField field, + uint32_t observed, uint32_t expected +) +{ + if (control->first_failed_index != pa_scheduler::pmu_owner::kDiagnosticIndexUnset) return; + control->first_failed_index = index; + control->first_failed_field = static_cast(field); + control->first_failed_observed = observed; + control->first_failed_expected = expected; +} + +void RecordRestoreFailure( + PmuOwnerControl *control, uint32_t index, PmuOwnerField field, + uint32_t observed, uint32_t expected +) +{ + ++control->restore_failures; + if (control->first_restore_failed_index != pa_scheduler::pmu_owner::kDiagnosticIndexUnset) return; + control->first_restore_failed_index = index; + control->first_restore_failed_field = static_cast(field); + control->first_restore_failed_observed = observed; + control->first_restore_failed_expected = expected; +} + +bool CheckRegister( + uint64_t base, uint32_t offset, uint32_t expected, PmuOwnerField field, + PmuOwnerField *failed_field, uint32_t *observed +) +{ + const uint32_t actual = ReadMmio(base, offset); + if (actual == expected) return true; + *failed_field = field; + *observed = actual; + return false; +} + +void SaveOne(PmuOwnerControl *control, uint64_t base, uint32_t index) +{ + PmuSavedRegisters &saved = control->saved[index]; + saved.ctrl0 = ReadMmio(base, kCtrl0Offset); + saved.ctrl1 = ReadMmio(base, kCtrl1Offset); + for (uint32_t counter = 0U; counter < pa_scheduler::pmu_owner::kPmuCounterCount; ++counter) { + saved.selectors[counter] = ReadMmio(base, kSelectorOffsets[counter]); + } + saved.start_cycle_low = ReadMmio(base, kStartCycleLowOffset); + saved.start_cycle_high = ReadMmio(base, kStartCycleHighOffset); + saved.stop_cycle_low = ReadMmio(base, kStopCycleLowOffset); + saved.stop_cycle_high = ReadMmio(base, kStopCycleHighOffset); +} + +void ConfigureOne(uint64_t base) +{ + // 先冻结 PMU,再写完整 10 槽 selector。CNT9=0 是正式配置中的未用策略, + // 不是遗漏;它和另外九槽一样会在 Restore 时恢复旧值。 + WriteMmio(base, kCtrl0Offset, 0U); + WriteMmio(base, kCtrl1Offset, 0U); + for (uint32_t counter = 0U; counter < pa_scheduler::pmu_owner::kPmuCounterCount; ++counter) { + WriteMmio(base, kSelectorOffsets[counter], pa_scheduler::pmu_owner::kConfiguredSelectors[counter]); + } + + // A5 counter 是 read-to-clear。旧计数无法保存,所以先清十个事件 counter + // 和 64-bit total,之后整个 configure→restore 区间由本 owner 独占。 + for (uint32_t counter = 0U; counter < pa_scheduler::pmu_owner::kPmuCounterCount; ++counter) { + (void)ReadMmio(base, kCounterOffsets[counter]); + } + (void)ReadMmio(base, kTotalLowOffset); + (void)ReadMmio(base, kTotalHighOffset); + + WriteMmio(base, kStartCycleLowOffset, 0U); + WriteMmio(base, kStartCycleHighOffset, 0U); + WriteMmio(base, kStopCycleLowOffset, 0xffffffffU); + WriteMmio(base, kStopCycleHighOffset, 0xffffffffU); + WriteMmio(base, kCtrl0Offset, kCtrl0Enabled); + WriteMmio(base, kCtrl1Offset, kCtrl1Enabled); + // Device-nGnRnE 会维持同一区域顺序;额外 DSB 只位于 owner 冷路径,用于 + // 保证下面的逐寄存器读回发生在所有配置写真正完成之后。 + FullSystemBarrier(); +} + +bool ConfigurationMatches( + uint64_t base, PmuOwnerField *failed_field, uint32_t *observed, uint32_t *expected +) +{ + if (!CheckRegister(base, kCtrl0Offset, kCtrl0Enabled, PmuOwnerField::Ctrl0, failed_field, observed)) { + *expected = kCtrl0Enabled; + return false; + } + if (!CheckRegister(base, kCtrl1Offset, kCtrl1Enabled, PmuOwnerField::Ctrl1, failed_field, observed)) { + *expected = kCtrl1Enabled; + return false; + } + for (uint32_t counter = 0U; counter < pa_scheduler::pmu_owner::kPmuCounterCount; ++counter) { + const uint32_t selector = pa_scheduler::pmu_owner::kConfiguredSelectors[counter]; + if (!CheckRegister( + base, kSelectorOffsets[counter], selector, SelectorField(counter), failed_field, observed + )) { + *expected = selector; + return false; + } + } + if (!CheckRegister(base, kStartCycleLowOffset, 0U, PmuOwnerField::StartCycleLow, failed_field, observed)) { + *expected = 0U; + return false; + } + if (!CheckRegister(base, kStartCycleHighOffset, 0U, PmuOwnerField::StartCycleHigh, failed_field, observed)) { + *expected = 0U; + return false; + } + if (!CheckRegister( + base, kStopCycleLowOffset, 0xffffffffU, PmuOwnerField::StopCycleLow, failed_field, observed + )) { + *expected = 0xffffffffU; + return false; + } + if (!CheckRegister( + base, kStopCycleHighOffset, 0xffffffffU, PmuOwnerField::StopCycleHigh, failed_field, observed + )) { + *expected = 0xffffffffU; + return false; + } + return true; +} + +void WriteSavedConfiguration(const PmuOwnerControl &control, uint64_t base, uint32_t index) +{ + const PmuSavedRegisters &saved = control.saved[index]; + WriteMmio(base, kCtrl0Offset, 0U); + WriteMmio(base, kCtrl1Offset, 0U); + for (uint32_t counter = 0U; counter < pa_scheduler::pmu_owner::kPmuCounterCount; ++counter) { + WriteMmio(base, kSelectorOffsets[counter], saved.selectors[counter]); + } + WriteMmio(base, kStartCycleLowOffset, saved.start_cycle_low); + WriteMmio(base, kStartCycleHighOffset, saved.start_cycle_high); + WriteMmio(base, kStopCycleLowOffset, saved.stop_cycle_low); + WriteMmio(base, kStopCycleHighOffset, saved.stop_cycle_high); + // CTRL 最后恢复,避免在 selector/range 尚未回到原值时短暂恢复旧计数状态。 + WriteMmio(base, kCtrl0Offset, saved.ctrl0); + WriteMmio(base, kCtrl1Offset, saved.ctrl1); + FullSystemBarrier(); +} + +bool SavedConfigurationMatches( + const PmuOwnerControl &control, uint64_t base, uint32_t index, + PmuOwnerField *failed_field, uint32_t *observed, uint32_t *expected +) +{ + const PmuSavedRegisters &saved = control.saved[index]; + if (!CheckRegister(base, kCtrl0Offset, saved.ctrl0, PmuOwnerField::Ctrl0, failed_field, observed)) { + *expected = saved.ctrl0; + return false; + } + if (!CheckRegister(base, kCtrl1Offset, saved.ctrl1, PmuOwnerField::Ctrl1, failed_field, observed)) { + *expected = saved.ctrl1; + return false; + } + for (uint32_t counter = 0U; counter < pa_scheduler::pmu_owner::kPmuCounterCount; ++counter) { + if (!CheckRegister( + base, kSelectorOffsets[counter], saved.selectors[counter], SelectorField(counter), + failed_field, observed + )) { + *expected = saved.selectors[counter]; + return false; + } + } + if (!CheckRegister( + base, kStartCycleLowOffset, saved.start_cycle_low, PmuOwnerField::StartCycleLow, + failed_field, observed + )) { + *expected = saved.start_cycle_low; + return false; + } + if (!CheckRegister( + base, kStartCycleHighOffset, saved.start_cycle_high, PmuOwnerField::StartCycleHigh, + failed_field, observed + )) { + *expected = saved.start_cycle_high; + return false; + } + if (!CheckRegister( + base, kStopCycleLowOffset, saved.stop_cycle_low, PmuOwnerField::StopCycleLow, + failed_field, observed + )) { + *expected = saved.stop_cycle_low; + return false; + } + if (!CheckRegister( + base, kStopCycleHighOffset, saved.stop_cycle_high, PmuOwnerField::StopCycleHigh, + failed_field, observed + )) { + *expected = saved.stop_cycle_high; + return false; + } + return true; +} + +bool RestoreOne( + PmuOwnerControl *control, uint32_t index, + PmuOwnerField *failed_field, uint32_t *observed, uint32_t *expected +) +{ + const uint64_t base = control->register_bases[index]; + if (base == 0U) { + *failed_field = PmuOwnerField::RegisterBase; + *observed = 0U; + *expected = 1U; + return false; + } + WriteSavedConfiguration(*control, base, index); + return SavedConfigurationMatches(*control, base, index, failed_field, observed, expected); +} + +void IncrementRoleCount(uint32_t index, uint32_t *aic, uint32_t *aiv) +{ + if (pa_scheduler::pmu_owner::IsAicPhysicalSlot(index)) { + ++(*aic); + } else { + ++(*aiv); + } +} + +void DecrementActiveRole(PmuOwnerControl *control, uint32_t index) +{ + if (control->active_total != 0U) --control->active_total; + if (pa_scheduler::pmu_owner::IsAicPhysicalSlot(index)) { + if (control->active_aic != 0U) --control->active_aic; + } else if (control->active_aiv != 0U) { + --control->active_aiv; + } +} + +// 恢复除 skip_index 外的 owned 槽。Configure 的“当前失败槽”若当场恢复失败, +// rollback 会跳过它,确保该 bit 留给 host 随后的幂等 Restore 再次重试。 +bool RestoreOwnedBitmapExcept(PmuOwnerControl *control, uint32_t skip_index) +{ + bool all_restored = true; + for (uint32_t next = pa_scheduler::pmu_owner::kPhysicalSubcoreCount; next != 0U; --next) { + const uint32_t index = next - 1U; + if (!pa_scheduler::pmu_owner::IsConfigured(*control, index)) continue; + if (index == skip_index) continue; + + PmuOwnerField failed_field = PmuOwnerField::None; + uint32_t observed = 0U; + uint32_t expected = 0U; + if (RestoreOne(control, index, &failed_field, &observed, &expected)) { + pa_scheduler::pmu_owner::ClearConfigured(control, index); + DecrementActiveRole(control, index); + } else { + RecordRestoreFailure(control, index, failed_field, observed, expected); + all_restored = false; + } + } + return all_restored; +} + +// 只消费 owner bitmap,严格按 107→0 恢复。某槽只有在完整读回一致后才清 bit; +// 因而 Restore 失败后可再次调用,下一次只重试仍由 owner 持有的槽。 +bool RestoreConfiguredBitmap(PmuOwnerControl *control) +{ + const bool all_restored = RestoreOwnedBitmapExcept( + control, pa_scheduler::pmu_owner::kDiagnosticIndexUnset + ); + const uint32_t bitmap_count = pa_scheduler::pmu_owner::CountConfigured(*control); + return all_restored && bitmap_count == 0U && control->active_total == 0U && + control->active_aic == 0U && control->active_aiv == 0U; +} + +bool ValidateControlHeader(PmuOwnerControl *control) +{ + if (control->magic != pa_scheduler::pmu_owner::kPmuOwnerControlMagic) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::ControlMagic, + control->magic, pa_scheduler::pmu_owner::kPmuOwnerControlMagic + ); + return false; + } + if (control->version != pa_scheduler::pmu_owner::kPmuOwnerControlVersion) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::ControlVersion, + control->version, pa_scheduler::pmu_owner::kPmuOwnerControlVersion + ); + return false; + } + if (control->struct_bytes != sizeof(PmuOwnerControl)) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::ControlSize, + control->struct_bytes, static_cast(sizeof(PmuOwnerControl)) + ); + return false; + } + return true; +} + +bool ValidateExpectedTopology(PmuOwnerControl *control) +{ + if (control->expected_total != pa_scheduler::pmu_owner::kExpectedSubcoreCount) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::TotalCount, + control->expected_total, pa_scheduler::pmu_owner::kExpectedSubcoreCount + ); + return false; + } + if (control->expected_aic != pa_scheduler::pmu_owner::kExpectedAicCount) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::AicCount, + control->expected_aic, pa_scheduler::pmu_owner::kExpectedAicCount + ); + return false; + } + if (control->expected_aiv != pa_scheduler::pmu_owner::kExpectedAivCount) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::AivCount, + control->expected_aiv, pa_scheduler::pmu_owner::kExpectedAivCount + ); + return false; + } + return true; +} + +PmuOwnerStatus Configure(PmuOwnerControl *control) +{ + if (control->configured != 0U || control->active_total != 0U || + pa_scheduler::pmu_owner::CountConfigured(*control) != 0U) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::State, + control->active_total, 0U + ); + return PmuOwnerStatus::AlreadyConfigured; + } + + control->active_total = 0U; + control->active_aic = 0U; + control->active_aiv = 0U; + control->discovered_total = 0U; + control->discovered_aic = 0U; + control->discovered_aiv = 0U; + control->skipped_total = 0U; + for (uint32_t word = 0U; word < pa_scheduler::pmu_owner::kConfiguredBitmapWords; ++word) { + control->configured_bitmap[word] = 0U; + } + ResetDiagnostics(control); + + for (uint32_t index = 0U; index < pa_scheduler::pmu_owner::kPhysicalSubcoreCount; ++index) { + const uint64_t base = control->register_bases[index]; + if (base == 0U) { + RecordFirstFailure(control, index, PmuOwnerField::RegisterBase, 0U, 1U); + ++control->skipped_total; + continue; + } + + SaveOne(control, base, index); + // 从保存完成到首次 MMIO 改写之间先取得所有权。即使后面的配置读回 + // 失败且当场恢复也失败,该槽仍留在 bitmap 中供后续 Restore 重试。 + pa_scheduler::pmu_owner::SetConfigured(control, index); + ++control->active_total; + IncrementRoleCount(index, &control->active_aic, &control->active_aiv); + ConfigureOne(base); + PmuOwnerField failed_field = PmuOwnerField::None; + uint32_t observed = 0U; + uint32_t expected = 0U; + if (!ConfigurationMatches(base, &failed_field, &observed, &expected)) { + RecordFirstFailure(control, index, failed_field, observed, expected); + ++control->skipped_total; + + // 配置未通过时必须当场恢复;只有原值完整读回一致才能释放该槽 + // 的所有权。若恢复失败,保留它的 bit 并回滚其余 owned 槽。 + PmuOwnerField restore_field = PmuOwnerField::None; + uint32_t restore_observed = 0U; + uint32_t restore_expected = 0U; + if (!RestoreOne(control, index, &restore_field, &restore_observed, &restore_expected)) { + RecordRestoreFailure(control, index, restore_field, restore_observed, restore_expected); + const bool rollback_ok = RestoreOwnedBitmapExcept(control, index); + control->configured = control->active_total == 0U ? 0U : 1U; + return rollback_ok ? PmuOwnerStatus::ConfigureSlotRestoreFailed : + PmuOwnerStatus::ConfigureRollbackFailed; + } + pa_scheduler::pmu_owner::ClearConfigured(control, index); + DecrementActiveRole(control, index); + continue; + } + + // discovered 只统计配置值全部读回一致的物理槽;bitmap/active 则表达 + // 更严格的“仍持有原值快照、尚未恢复”所有权,两者不可混用。 + ++control->discovered_total; + IncrementRoleCount(index, &control->discovered_aic, &control->discovered_aiv); + } + + const uint32_t bitmap_count = pa_scheduler::pmu_owner::CountConfigured(*control); + const bool topology_matches = + bitmap_count == pa_scheduler::pmu_owner::kExpectedSubcoreCount && + control->active_total == pa_scheduler::pmu_owner::kExpectedSubcoreCount && + control->active_aic == pa_scheduler::pmu_owner::kExpectedAicCount && + control->active_aiv == pa_scheduler::pmu_owner::kExpectedAivCount && + control->discovered_total == pa_scheduler::pmu_owner::kExpectedSubcoreCount && + control->discovered_aic == pa_scheduler::pmu_owner::kExpectedAicCount && + control->discovered_aiv == pa_scheduler::pmu_owner::kExpectedAivCount; + if (!topology_matches) { + if (bitmap_count != control->active_total) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::BitmapCount, + bitmap_count, control->active_total + ); + } else if (control->active_total != pa_scheduler::pmu_owner::kExpectedSubcoreCount) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::TotalCount, + control->active_total, pa_scheduler::pmu_owner::kExpectedSubcoreCount + ); + } else if (control->active_aic != pa_scheduler::pmu_owner::kExpectedAicCount) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::AicCount, + control->active_aic, pa_scheduler::pmu_owner::kExpectedAicCount + ); + } else if (control->active_aiv != pa_scheduler::pmu_owner::kExpectedAivCount) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::AivCount, + control->active_aiv, pa_scheduler::pmu_owner::kExpectedAivCount + ); + } else if (control->discovered_total != pa_scheduler::pmu_owner::kExpectedSubcoreCount) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::TotalCount, + control->discovered_total, pa_scheduler::pmu_owner::kExpectedSubcoreCount + ); + } else if (control->discovered_aic != pa_scheduler::pmu_owner::kExpectedAicCount) { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::AicCount, + control->discovered_aic, pa_scheduler::pmu_owner::kExpectedAicCount + ); + } else { + RecordFirstFailure( + control, pa_scheduler::pmu_owner::kPhysicalSubcoreCount, PmuOwnerField::AivCount, + control->discovered_aiv, pa_scheduler::pmu_owner::kExpectedAivCount + ); + } + const bool rollback_ok = RestoreConfiguredBitmap(control); + control->configured = control->active_total == 0U ? 0U : 1U; + return rollback_ok ? PmuOwnerStatus::ConfigureCountMismatch : + PmuOwnerStatus::ConfigureRollbackFailed; + } + + control->configured = 1U; + return PmuOwnerStatus::Success; +} + +PmuOwnerStatus Restore(PmuOwnerControl *control) +{ + // 幂等恢复:bitmap 已空时,无论是一次正常 Restore 后的重入,还是 host + // 失败路径的兜底调用,都统一收口为“未持有任何 PMU 槽”。 + const uint32_t bitmap_count = pa_scheduler::pmu_owner::CountConfigured(*control); + if (bitmap_count == 0U) { + control->configured = 0U; + control->active_total = 0U; + control->active_aic = 0U; + control->active_aiv = 0U; + ResetDiagnostics(control); + return PmuOwnerStatus::Success; + } + + // 若上一次命令在更新计数字段后异常退出,bitmap 才是唯一权威所有权源。 + // Restore 先由 bitmap 重建 active 计数,再逆序恢复,避免陈旧计数阻塞清理。 + control->active_total = bitmap_count; + control->active_aic = 0U; + control->active_aiv = 0U; + for (uint32_t index = 0U; index < pa_scheduler::pmu_owner::kPhysicalSubcoreCount; ++index) { + if (pa_scheduler::pmu_owner::IsConfigured(*control, index)) { + IncrementRoleCount(index, &control->active_aic, &control->active_aiv); + } + } + ResetDiagnostics(control); + const bool restored = RestoreConfiguredBitmap(control); + control->configured = restored ? 0U : 1U; + return restored ? PmuOwnerStatus::Success : PmuOwnerStatus::RestoreFailed; +} + +// 主 aicpu_scheduler 会复制完整 152B KernelArgs;其中 runtime_args_device +// 指向跨 Configure/Restore 持续存在的 GM control。参数副本本身不需要 cache 维护。 +PmuOwnerControl *ResolveControl(const PmuOwnerMainKernelArgs *arguments) +{ + if (arguments == nullptr || arguments->runtime_args_device == 0U || + (arguments->runtime_args_device % alignof(PmuOwnerControl)) != 0U) { + return nullptr; + } + return reinterpret_cast( + static_cast(arguments->runtime_args_device) + ); +} + +void ExecuteOwnerCommand(void *argument) +{ + const auto *arguments = reinterpret_cast(argument); + PmuOwnerControl *control = ResolveControl(arguments); + if (control == nullptr) return; + + InvalidateControl(control); + // 从这一行起,即使协议校验失败也把精确业务状态 clean 回 GM。 + control->status = pa_scheduler::pmu_owner::kStatusPending; + ResetDiagnostics(control); + if (!ValidateControlHeader(control)) { + control->status = static_cast(PmuOwnerStatus::InvalidControl); + CleanControl(control); + return; + } + const auto command = static_cast(arguments->command); + if (command == PmuOwnerMainCommand::Configure && !ValidateExpectedTopology(control)) { + control->status = static_cast(PmuOwnerStatus::UnexpectedTopology); + CleanControl(control); + return; + } + + PmuOwnerStatus status = PmuOwnerStatus::InvalidArguments; + if (command == PmuOwnerMainCommand::Configure) { + status = Configure(control); + } else if (command == PmuOwnerMainCommand::Restore) { + // Restore 以 bitmap 为唯一所有权依据。即使 expected_* 诊断字段被局部 + // 覆盖,也优先尝试恢复已经保存的寄存器,避免清理被无关字段阻塞。 + status = Restore(control); + } + control->status = static_cast(status); + CleanControl(control); +} + +} // namespace + +extern "C" __attribute__((visibility("default"))) int simpler_aicpu_exec(void *argument) +{ + ExecuteOwnerCommand(argument); + return 0; +} diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_control.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_control.h new file mode 100644 index 0000000000..f8a7c0d371 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_control.h @@ -0,0 +1,211 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_CCEC_PMU_OWNER_CONTROL_H_ +#define PA_SCHEDULER_CCEC_PMU_OWNER_CONTROL_H_ + +#include +#include + +namespace pa_scheduler::pmu_owner { + +// 这份头文件同时供 x86 host 与 AArch64 AICPU helper 使用。所有跨端字段都采用 +// 固定宽度整数;禁止在 ABI 中放 host 指针、bool、STL 容器或编译器相关位域。 +constexpr uint32_t kPmuOwnerControlMagic = 0x504d554fU; // "PMUO" +constexpr uint32_t kPmuOwnerControlVersion = 1U; + +// DAV_3510 一共有 2 die;每个 die 依次放 18 个 AIC 和 36 个 AIV 物理槽。 +// 当前 A5 stream 实际开放 32 个 AIC 与 64 个 AIV,其余 12 个槽的 MMIO +// 读回不会匹配配置值,因此 owner 必须扫描 108 槽,最终取得 96 个可用槽。 +constexpr uint32_t kPhysicalSubcoreCount = 108U; +constexpr uint32_t kExpectedSubcoreCount = 96U; +constexpr uint32_t kExpectedAicCount = 32U; +constexpr uint32_t kExpectedAivCount = 64U; +constexpr uint32_t kAicPerDie = 18U; +constexpr uint32_t kSubcoresPerDie = 54U; +constexpr uint32_t kConfiguredBitmapWords = 4U; +constexpr uint32_t kDiagnosticIndexUnset = 0xffffffffU; + +static_assert(kExpectedAicCount + kExpectedAivCount == kExpectedSubcoreCount, "active topology count mismatch"); +static_assert(kAicPerDie * 2U == 36U, "physical AIC topology changed"); +static_assert(kSubcoresPerDie * 2U == kPhysicalSubcoreCount, "physical subcore topology changed"); + +// A5 PIPE_UTILIZATION 的正式 counter 槽位布局。CNT9 的事件号为 0,明确 +// 表示本轮不消费该槽;仍保存并恢复其旧 selector,也仍读一次 counter 清零。 +constexpr uint32_t kPmuCounterCount = 10U; +constexpr uint32_t kConfiguredSelectors[kPmuCounterCount] = { + 0x501U, // CNT0: vector busy + 0x301U, // CNT1: cube busy + 0x001U, // CNT2: scalar busy + 0x701U, // CNT3: MTE1 busy + 0x202U, // CNT4: MTE2 busy + 0x203U, // CNT5: MTE3 busy + 0x034U, // CNT6: I-cache request + 0x035U, // CNT7: I-cache miss + 0x714U, // CNT8: fix-pipe busy + 0x000U, // CNT9: 未使用 +}; + +constexpr int32_t kStatusPending = 0x7fffffff; + +// AICPU entry 始终向 runtime 返回 0;协议结果只通过 control.status 回传, +// 从而避免一次可诊断的配置失败被 runtime 升格成整条 stream 异常。 +enum class PmuOwnerStatus : int32_t { + Success = 0, + InvalidArguments = -1, + InvalidControl = -2, + UnexpectedTopology = -3, + AlreadyConfigured = -4, + ConfigureCountMismatch = -5, + ConfigureRollbackFailed = -6, + ConfigureSlotRestoreFailed = -7, + RestoreFailed = -8, +}; + +// 首个异常寄存器使用稳定的枚举编号,host 不需要解析 AICPU 日志即可定位 +// 是基址、selector、计数范围还是 enable 控制读回不一致。 +enum class PmuOwnerField : uint32_t { + None = 0, + Arguments, + ControlMagic, + ControlVersion, + ControlSize, + State, + RegisterBase, + Ctrl0, + Ctrl1, + Selector0, + Selector1, + Selector2, + Selector3, + Selector4, + Selector5, + Selector6, + Selector7, + Selector8, + Selector9, + StartCycleLow, + StartCycleHigh, + StopCycleLow, + StopCycleHigh, + BitmapCount, + TotalCount, + AicCount, + AivCount, +}; + +// 单个物理子核被 owner 改动的完整可恢复状态恰好占一条 cache line。 +// PMU counter 是 read-to-clear,旧 counter 值无法恢复;owner 会话必须独占。 +struct alignas(64) PmuSavedRegisters { + uint32_t ctrl0; + uint32_t ctrl1; + uint32_t selectors[kPmuCounterCount]; + uint32_t start_cycle_low; + uint32_t start_cycle_high; + uint32_t stop_cycle_low; + uint32_t stop_cycle_high; +}; + +// Host 与 AICPU 共享的 owner 状态。前 128B 是命令结果和诊断,随后内嵌 +// 108 个 MMIO 基址、4-word 所有权 bitmap,以及每槽 64B 的 Configure 快照。 +// bitmap 的严格语义是“原值已保存、且 owner 可能已经改写 MMIO、但尚未 +// 完整恢复”的槽;它在 Configure 写第一项 MMIO 前置位,仅在恢复读回完整 +// 一致后清位。Restore 期间不得清零或重建 saved[],只能按 bitmap 逆序消费。 +struct alignas(64) PmuOwnerControl { + uint32_t magic; + uint32_t version; + uint32_t struct_bytes; + volatile int32_t status; + + uint32_t configured; + uint32_t expected_total; + uint32_t expected_aic; + uint32_t expected_aiv; + + // active_* 与 bitmap 表示仍由本 owner 持有、尚未恢复的物理槽。 + uint32_t active_total; + uint32_t active_aic; + uint32_t active_aiv; + // discovered_* 保留本次 Configure 扫描结果;即使计数不匹配后回滚, + // host 仍能看到回滚前究竟探测到了多少 AIC/AIV。 + uint32_t discovered_total; + uint32_t discovered_aic; + uint32_t discovered_aiv; + uint32_t skipped_total; + + uint32_t first_failed_index; + uint32_t first_failed_field; + uint32_t first_failed_observed; + uint32_t first_failed_expected; + + uint32_t restore_failures; + uint32_t first_restore_failed_index; + uint32_t first_restore_failed_field; + uint32_t first_restore_failed_observed; + uint32_t first_restore_failed_expected; + uint32_t reserved_header[8]; + + uint64_t register_bases[kPhysicalSubcoreCount]; + // 字段名保留 configured_bitmap 以稳定 host/device ABI;失败路径中它还会 + // 临时包含“配置未通过但恢复仍待重试”的 owned 槽。 + uint32_t configured_bitmap[kConfiguredBitmapWords]; + // 让 saved[] 从新的 64B cache line 开始;该 padding 不承载协议含义。 + uint32_t reserved_bitmap[4]; + PmuSavedRegisters saved[kPhysicalSubcoreCount]; +}; + +static_assert(sizeof(PmuSavedRegisters) == 64U, "one saved PMU slot must occupy one cache line"); +static_assert(alignof(PmuSavedRegisters) == 64U, "saved PMU slot alignment changed"); +static_assert(offsetof(PmuOwnerControl, status) == 12U, "PMU owner status offset changed"); +static_assert(offsetof(PmuOwnerControl, register_bases) == 128U, "PMU owner header must occupy two cache lines"); +static_assert(offsetof(PmuOwnerControl, configured_bitmap) == 992U, "PMU owner bitmap offset changed"); +static_assert(offsetof(PmuOwnerControl, saved) == 1024U, "PMU owner saved area must be cache-line aligned"); +static_assert(sizeof(PmuOwnerControl) == 7936U, "PMU owner control ABI changed"); +static_assert(sizeof(PmuOwnerControl) % 64U == 0U, "PMU owner control must use complete cache lines"); +static_assert(alignof(PmuOwnerControl) == 64U, "PMU owner control alignment changed"); + +inline bool IsAicPhysicalSlot(uint32_t index) +{ + return index < kPhysicalSubcoreCount && (index % kSubcoresPerDie) < kAicPerDie; +} + +inline bool IsConfigured(const PmuOwnerControl &control, uint32_t index) +{ + return index < kPhysicalSubcoreCount && + (control.configured_bitmap[index / 32U] & (1U << (index % 32U))) != 0U; +} + +inline void SetConfigured(PmuOwnerControl *control, uint32_t index) +{ + if (index < kPhysicalSubcoreCount) { + control->configured_bitmap[index / 32U] |= 1U << (index % 32U); + } +} + +inline void ClearConfigured(PmuOwnerControl *control, uint32_t index) +{ + if (index < kPhysicalSubcoreCount) { + control->configured_bitmap[index / 32U] &= ~(1U << (index % 32U)); + } +} + +inline uint32_t CountConfigured(const PmuOwnerControl &control) +{ + uint32_t count = 0U; + for (uint32_t index = 0U; index < kPhysicalSubcoreCount; ++index) { + count += IsConfigured(control, index) ? 1U : 0U; + } + return count; +} + +} // namespace pa_scheduler::pmu_owner + +#endif // PA_SCHEDULER_CCEC_PMU_OWNER_CONTROL_H_ diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_dispatcher.cpp b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_dispatcher.cpp new file mode 100644 index 0000000000..563113ead8 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_dispatcher.cpp @@ -0,0 +1,186 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +// 这个 SO 只在初始化阶段由 libaicpu_extend_kernels.so 临时加载。它在主 +// aicpu_scheduler 有权限访问的预安装目录中落盘真正的 PMU owner SO;随后 +// host 通过 mode=0 JSON 注册直接调用 owner,不会在每次命令中再经过本文件。 + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +extern "C" void DlogRecord(int module_id, int level, const char *format, ...); + +namespace { + +constexpr int kDlogModuleCcecpu = 3; +constexpr int kDlogLevelError = 3; +constexpr uint64_t kFnvOffsetBasis = UINT64_C(14695981039346656037); +constexpr uint64_t kFnvPrime = UINT64_C(1099511628211); + +void Log(const char *format, ...) +{ + char buffer[1024] = {}; + va_list arguments; + va_start(arguments, format); + (void)vsnprintf(buffer, sizeof(buffer), format, arguments); + va_end(arguments); + DlogRecord(kDlogModuleCcecpu, kDlogLevelError, "[pa-pmu-dispatcher] %s", buffer); +} + +// libaicpu_extend_kernels 固定从 KernelArgs::device_args(offset 40)获取 +// DeviceArgs。后五个 qword 是本地 bootstrap 协议,offset 必须保持不变。 +struct BootstrapKernelArgs { + uint64_t unused[5]; + void *device_args; + void *runtime_args; + uint64_t regs; +}; + +struct BootstrapDeviceArgs { + uint64_t unused[12]; + uint64_t dispatcher_so_device; // offset 96,extend kernel 消费 + uint64_t dispatcher_so_bytes; // offset 104 + uint64_t device_id; // offset 112 + uint64_t owner_so_device; // offset 120,本 dispatcher 消费 + uint64_t owner_so_bytes; // offset 128 +}; + +static_assert(offsetof(BootstrapKernelArgs, device_args) == 40U, "bootstrap KernelArgs ABI changed"); +static_assert(offsetof(BootstrapDeviceArgs, dispatcher_so_device) == 96U, "dispatcher address offset changed"); +static_assert(offsetof(BootstrapDeviceArgs, dispatcher_so_bytes) == 104U, "dispatcher size offset changed"); +static_assert(offsetof(BootstrapDeviceArgs, device_id) == 112U, "device id offset changed"); +static_assert(offsetof(BootstrapDeviceArgs, owner_so_device) == 120U, "owner address offset changed"); +static_assert(offsetof(BootstrapDeviceArgs, owner_so_bytes) == 128U, "owner size offset changed"); + +// host 与 device 都对完整 SO 字节做 FNV-1a;不只散列 ELF header,避免同一 +// toolchain 产出的等长 SO 发生名字碰撞并误加载旧代码。 +uint64_t FingerprintBytes(const void *data, uint64_t bytes) +{ + const auto *input = static_cast(data); + uint64_t hash = kFnvOffsetBasis; + for (uint64_t index = 0U; index < bytes; ++index) { + hash ^= input[index]; + hash *= kFnvPrime; + } + return hash; +} + +std::string OwnerSoPath(uint64_t fingerprint, uint64_t device_id) +{ + char path[256] = {}; + (void)snprintf( + path, sizeof(path), + "/usr/lib64/aicpu_kernels/0/aicpu_kernels_device/pa_scheduler_pmu_owner_%016llx_d%llu.so", + static_cast(fingerprint), static_cast(device_id) + ); + return path; +} + +// 先写同目录临时文件,再原子 rename。目标名由内容、device 共同确定;临时 +// 名再加入进程和源地址,避免同进程并发 bootstrap 写同一个临时 inode。 +bool WriteOwnerSo(const std::string &target, const void *data, uint64_t bytes) +{ + if (data == nullptr || bytes == 0U || + bytes > static_cast(std::numeric_limits::max())) { + Log("invalid owner SO buffer: data=%p bytes=%llu", data, static_cast(bytes)); + return false; + } + + char temporary[384] = {}; + (void)snprintf( + temporary, sizeof(temporary), "%s.tmp.%d.%016llx", target.c_str(), static_cast(getpid()), + static_cast(reinterpret_cast(data)) + ); + { + std::ofstream output(temporary, std::ios::binary | std::ios::trunc); + if (!output.is_open()) { + Log("open %s failed: %s", temporary, strerror(errno)); + return false; + } + output.write(static_cast(data), static_cast(bytes)); + output.close(); + if (!output) { + Log("write %s failed", temporary); + (void)unlink(temporary); + return false; + } + } + if (chmod(temporary, 0755) != 0) { + Log("chmod %s failed: %s", temporary, strerror(errno)); + (void)unlink(temporary); + return false; + } + if (rename(temporary, target.c_str()) != 0) { + Log("rename %s -> %s failed: %s", temporary, target.c_str(), strerror(errno)); + (void)unlink(temporary); + return false; + } + return true; +} + +} // namespace + +extern "C" { + +// extend kernel 在 dlopen 后要求三个符号同时存在;本 dispatcher 只使用 Init, +// 另外两个入口保持无副作用成功返回,避免未来 runtime 的预探测变成故障。 +__attribute__((visibility("default"))) int StaticTileFwkBackendKernelServer(void *arguments) +{ + (void)arguments; + return 0; +} + +__attribute__((visibility("default"))) uint32_t DynTileFwkBackendKernelServer(void *arguments) +{ + (void)arguments; + return 0U; +} + +__attribute__((visibility("default"))) uint32_t DynTileFwkBackendKernelServerInit(void *arguments) +{ + if (arguments == nullptr) { + Log("Init received null KernelArgs"); + return 1U; + } + auto *kernel_args = static_cast(arguments); + auto *device_args = static_cast(kernel_args->device_args); + if (device_args == nullptr || device_args->owner_so_device == 0U || device_args->owner_so_bytes == 0U) { + Log( + "Init received invalid DeviceArgs: args=%p owner=%016llx bytes=%llu", device_args, + static_cast(device_args == nullptr ? 0U : device_args->owner_so_device), + static_cast(device_args == nullptr ? 0U : device_args->owner_so_bytes) + ); + return 2U; + } + + const void *owner_so = reinterpret_cast(static_cast(device_args->owner_so_device)); + const uint64_t fingerprint = FingerprintBytes(owner_so, device_args->owner_so_bytes); + const std::string target = OwnerSoPath(fingerprint, device_args->device_id); + if (!WriteOwnerSo(target, owner_so, device_args->owner_so_bytes)) return 3U; + + Log( + "installed %s (%llu bytes)", target.c_str(), + static_cast(device_args->owner_so_bytes) + ); + return 0U; +} + +} // extern "C" diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_host.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_host.h new file mode 100644 index 0000000000..68dfac042e --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_host.h @@ -0,0 +1,345 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_CCEC_PMU_OWNER_HOST_H_ +#define PA_SCHEDULER_CCEC_PMU_OWNER_HOST_H_ + +#include "pmu_owner_control.h" +#include "pmu_owner_main_abi.h" +#include "pmu_owner_main_loader.h" + +#include "acl/acl.h" + +#include +#include +#include +#include +#include +#include + +namespace pa_scheduler::pmu_owner { + +inline bool OwnerCheckAcl(aclError error, const char *label) +{ + if (error == ACL_SUCCESS) return true; + std::fprintf(stderr, "ACL error %d: %s\n", static_cast(error), label); + return false; +} + +inline std::string ArtifactBesideKernel(const std::string &kernel_path, const char *name) +{ + const size_t slash = kernel_path.find_last_of('/'); + return slash == std::string::npos ? std::string(name) : kernel_path.substr(0, slash + 1U) + name; +} + +inline const char *OwnerFieldName(PmuOwnerField field) +{ + switch (field) { + case PmuOwnerField::None: return "none"; + case PmuOwnerField::Arguments: return "arguments"; + case PmuOwnerField::ControlMagic: return "control-magic"; + case PmuOwnerField::ControlVersion: return "control-version"; + case PmuOwnerField::ControlSize: return "control-size"; + case PmuOwnerField::State: return "state"; + case PmuOwnerField::RegisterBase: return "register-base"; + case PmuOwnerField::Ctrl0: return "ctrl0"; + case PmuOwnerField::Ctrl1: return "ctrl1"; + case PmuOwnerField::Selector0: return "selector0"; + case PmuOwnerField::Selector1: return "selector1"; + case PmuOwnerField::Selector2: return "selector2"; + case PmuOwnerField::Selector3: return "selector3"; + case PmuOwnerField::Selector4: return "selector4"; + case PmuOwnerField::Selector5: return "selector5"; + case PmuOwnerField::Selector6: return "selector6"; + case PmuOwnerField::Selector7: return "selector7"; + case PmuOwnerField::Selector8: return "selector8"; + case PmuOwnerField::Selector9: return "selector9"; + case PmuOwnerField::StartCycleLow: return "start-cycle-low"; + case PmuOwnerField::StartCycleHigh: return "start-cycle-high"; + case PmuOwnerField::StopCycleLow: return "stop-cycle-low"; + case PmuOwnerField::StopCycleHigh: return "stop-cycle-high"; + case PmuOwnerField::BitmapCount: return "bitmap-count"; + case PmuOwnerField::TotalCount: return "total-count"; + case PmuOwnerField::AicCount: return "aic-count"; + case PmuOwnerField::AivCount: return "aiv-count"; + } + return "unknown"; +} + +struct ActiveSubcoreLimits { + uint32_t aic = 0U; + uint32_t aiv = 0U; + uint32_t total = 0U; +}; + +inline bool QueryActiveSubcoreLimits(aclrtStream scheduling_stream, ActiveSubcoreLimits *limits) +{ + if (scheduling_stream == nullptr || limits == nullptr) { + std::fprintf(stderr, "Cannot query active PMU subcores with a null stream/result.\n"); + return false; + } + uint32_t aic = 0U; + uint32_t aiv = 0U; + const aclError aic_error = + aclrtGetStreamResLimit(scheduling_stream, ACL_RT_DEV_RES_CUBE_CORE, &aic); + const aclError aiv_error = + aclrtGetStreamResLimit(scheduling_stream, ACL_RT_DEV_RES_VECTOR_CORE, &aiv); + const uint64_t total = static_cast(aic) + aiv; + if (aic_error != ACL_SUCCESS || aiv_error != ACL_SUCCESS || + aic != kExpectedAicCount || aiv != kExpectedAivCount || total != kExpectedSubcoreCount) { + std::fprintf( + stderr, + "Unexpected stream PMU topology: aic_error=%d aiv_error=%d " + "aic=%u/%u aiv=%u/%u total=%llu/%u\n", + static_cast(aic_error), static_cast(aiv_error), aic, kExpectedAicCount, + aiv, kExpectedAivCount, static_cast(total), kExpectedSubcoreCount + ); + return false; + } + limits->aic = aic; + limits->aiv = aiv; + limits->total = static_cast(total); + std::printf( + "[PMU_OWNER] stream_active aic=%u aiv=%u total=%u physical_slots=%u\n", + limits->aic, limits->aiv, limits->total, kPhysicalSubcoreCount + ); + return true; +} + +// mixed launch 的一个物理 block 必须同时拥有 1 个 AIC 和相邻的 2 个 AIV。 +// 只检查 32/64 总数仍可能放过孤立 AIV;这里直接按两 die 的真实编号布局验闭包。 +inline bool ValidateConfiguredTripletTopology(const PmuOwnerControl &control) +{ + uint32_t complete_triplets = 0U; + uint32_t broken_triplets = 0U; + const uint32_t dies = kPhysicalSubcoreCount / kSubcoresPerDie; + for (uint32_t die = 0U; die < dies; ++die) { + const uint32_t die_base = die * kSubcoresPerDie; + for (uint32_t local = 0U; local < kAicPerDie; ++local) { + const bool aic = IsConfigured(control, die_base + local); + const bool aiv0 = IsConfigured(control, die_base + kAicPerDie + local * 2U); + const bool aiv1 = IsConfigured(control, die_base + kAicPerDie + local * 2U + 1U); + if (aic == aiv0 && aic == aiv1) { + complete_triplets += aic ? 1U : 0U; + } else { + ++broken_triplets; + } + } + } + const bool passed = complete_triplets == kExpectedAicCount && broken_triplets == 0U; + std::printf( + "[ASSERT] %-48s %s (complete=%u broken=%u)\n", + "PMU owner bitmap is complete 1-AIC + 2-AIV triplets", + passed ? "PASS" : "FAIL", complete_triplets, broken_triplets + ); + return passed; +} + +// owner 命令使用独立 stream,但通过 mode=0 JSON 在主 aicpu_scheduler 中执行。 +// Configure 同步完成后才允许启动 AICore;AICore 正常或异常退出后,Restore +// 都不会依赖业务 stream。MMIO 映射必须保持到 Finalize 完成之后。 +class PmuOwnerSession { +public: + PmuOwnerSession() = default; + PmuOwnerSession(const PmuOwnerSession &) = delete; + PmuOwnerSession &operator=(const PmuOwnerSession &) = delete; + + ~PmuOwnerSession() + { + if (HasResources()) (void)Finalize(); + } + + bool Initialize( + uint32_t device, aclrtStream scheduling_stream, const std::string &dispatcher_path, + const std::string &owner_path, const std::vector ®ister_bases + ) + { + if (HasResources() || register_bases.size() != kPhysicalSubcoreCount) { + std::fprintf( + stderr, "Invalid PMU owner initialization state or register table size: %zu\n", + register_bases.size() + ); + return false; + } + device_ = device; + if (!QueryActiveSubcoreLimits(scheduling_stream, &limits_)) return false; + if (!OwnerCheckAcl(aclrtCreateStream(&owner_stream_), "aclrtCreateStream(PMU owner)")) return false; + if (loader_.Initialize( + dispatcher_path, owner_path, owner_stream_, static_cast(device_) + ) != 0) { + return false; + } + if (!OwnerCheckAcl( + aclrtMalloc(&control_device_, sizeof(PmuOwnerControl), ACL_MEM_MALLOC_NORMAL_ONLY), + "aclrtMalloc(PMU owner control)" + )) { + return false; + } + if ((reinterpret_cast(control_device_) & (alignof(PmuOwnerControl) - 1U)) != 0U) { + std::fprintf(stderr, "PMU owner control is not 64-byte aligned: %p\n", control_device_); + return false; + } + + control_ = PmuOwnerControl{}; + control_.magic = kPmuOwnerControlMagic; + control_.version = kPmuOwnerControlVersion; + control_.struct_bytes = sizeof(PmuOwnerControl); + control_.status = kStatusPending; + control_.expected_total = limits_.total; + control_.expected_aic = limits_.aic; + control_.expected_aiv = limits_.aiv; + std::memcpy(control_.register_bases, register_bases.data(), sizeof(control_.register_bases)); + if (!OwnerCheckAcl( + aclrtMemcpy( + control_device_, sizeof(control_), &control_, sizeof(control_), ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D initial PMU owner control)" + )) { + return false; + } + ready_ = true; + return true; + } + + bool Configure() + { + if (!ready_) return false; + const bool command_ok = RunCommand(PmuOwnerMainCommand::Configure, "Configure"); + configured_ = CountConfigured(control_) != 0U; + const uint32_t bitmap_count = CountConfigured(control_); + const bool triplets_ok = ValidateConfiguredTripletTopology(control_); + const bool state_ok = control_.status == static_cast(PmuOwnerStatus::Success) && + control_.configured == 1U && control_.active_total == limits_.total && + control_.active_aic == limits_.aic && control_.active_aiv == limits_.aiv && + control_.discovered_total == limits_.total && control_.discovered_aic == limits_.aic && + control_.discovered_aiv == limits_.aiv && bitmap_count == limits_.total && + control_.skipped_total + bitmap_count == kPhysicalSubcoreCount && triplets_ok; + PrintControl("Configure", bitmap_count); + return command_ok && state_ok; + } + + bool Restore() + { + if (control_device_ == nullptr || owner_stream_ == nullptr || !loader_.IsInitialized()) { + return !configured_; + } + const bool command_ok = RunCommand(PmuOwnerMainCommand::Restore, "Restore"); + const uint32_t bitmap_count = CountConfigured(control_); + const bool state_ok = control_.status == static_cast(PmuOwnerStatus::Success) && + control_.configured == 0U && control_.active_total == 0U && + control_.active_aic == 0U && control_.active_aiv == 0U && bitmap_count == 0U; + configured_ = bitmap_count != 0U; + PrintControl("Restore", bitmap_count); + return command_ok && state_ok; + } + + bool Finalize() + { + bool ok = true; + if (control_device_ != nullptr && owner_stream_ != nullptr && loader_.IsInitialized()) { + bool restored = Restore(); + if (!restored) restored = Restore(); + ok &= restored; + } else { + ok &= !configured_; + } + ok &= loader_.Finalize() == 0; + if (control_device_ != nullptr) { + ok &= OwnerCheckAcl(aclrtFree(control_device_), "aclrtFree(PMU owner control)"); + control_device_ = nullptr; + } + if (owner_stream_ != nullptr) { + ok &= OwnerCheckAcl(aclrtDestroyStream(owner_stream_), "aclrtDestroyStream(PMU owner)"); + owner_stream_ = nullptr; + } + ready_ = false; + configured_ = false; + std::printf("[PMU_OWNER] restore_and_cleanup=%s\n", ok ? "PASS" : "FAIL"); + return ok; + } + + uint64_t RegisterTableDeviceAddress() const + { + if (control_device_ == nullptr) return 0U; + return reinterpret_cast(control_device_) + offsetof(PmuOwnerControl, register_bases); + } + + const PmuOwnerControl &Control() const { return control_; } + + bool IsConfiguredSubcore(uint32_t index) const + { + return ready_ && IsConfigured(control_, index); + } + +private: + bool HasResources() const + { + return owner_stream_ != nullptr || control_device_ != nullptr || loader_.IsInitialized(); + } + + bool RunCommand(PmuOwnerMainCommand command, const char *label) + { + const PmuOwnerMainKernelArgs arguments = MakePmuOwnerMainKernelArgs( + reinterpret_cast(control_device_), command, device_ + ); + const std::string sync_label = std::string("aclrtSynchronizeStream(PMU ") + label + ")"; + const std::string copy_label = std::string("aclrtMemcpy(D2H PMU ") + label + ")"; + if (loader_.Launch(owner_stream_, const_cast(&arguments), sizeof(arguments)) != 0 || + !OwnerCheckAcl(aclrtSynchronizeStream(owner_stream_), sync_label.c_str()) || + !OwnerCheckAcl( + aclrtMemcpy( + &control_, sizeof(control_), control_device_, sizeof(control_), ACL_MEMCPY_DEVICE_TO_HOST + ), + copy_label.c_str() + )) { + return false; + } + return true; + } + + void PrintControl(const char *command, uint32_t bitmap_count) const + { + const auto failed_field = static_cast(control_.first_failed_field); + const auto restore_field = static_cast(control_.first_restore_failed_field); + std::printf( + "[PMU_OWNER] command=%s status=%d configured=%u active=%u/%u/%u " + "discovered=%u/%u/%u bitmap=%u skipped=%u first_failed=%u:%s(%u):0x%x/0x%x " + "restore_failures=%u first_restore=%u:%s(%u):0x%x/0x%x\n", + command, static_cast(control_.status), control_.configured, + control_.active_total, control_.active_aic, control_.active_aiv, + control_.discovered_total, control_.discovered_aic, control_.discovered_aiv, + bitmap_count, control_.skipped_total, control_.first_failed_index, + OwnerFieldName(failed_field), control_.first_failed_field, + control_.first_failed_observed, control_.first_failed_expected, + control_.restore_failures, control_.first_restore_failed_index, + OwnerFieldName(restore_field), control_.first_restore_failed_field, + control_.first_restore_failed_observed, control_.first_restore_failed_expected + ); + std::printf( + "[PMU_OWNER] configured_bitmap=%08x:%08x:%08x:%08x\n", + control_.configured_bitmap[3], control_.configured_bitmap[2], + control_.configured_bitmap[1], control_.configured_bitmap[0] + ); + } + + uint32_t device_ = 0U; + aclrtStream owner_stream_ = nullptr; + MainAicpuLoader loader_; + void *control_device_ = nullptr; + PmuOwnerControl control_{}; + ActiveSubcoreLimits limits_{}; + bool ready_ = false; + bool configured_ = false; +}; + +} // namespace pa_scheduler::pmu_owner + +#endif // PA_SCHEDULER_CCEC_PMU_OWNER_HOST_H_ diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_main_abi.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_main_abi.h new file mode 100644 index 0000000000..d6ba9e2239 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_main_abi.h @@ -0,0 +1,75 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_CCEC_PMU_OWNER_MAIN_ABI_H_ +#define PA_SCHEDULER_CCEC_PMU_OWNER_MAIN_ABI_H_ + +#include +#include + +namespace pa_scheduler::pmu_owner { + +// 主 aicpu_scheduler 的统一入口 simpler_aicpu_exec 根据该命令选择配置或 +// 恢复。0 特意保留为 Invalid,避免零初始化参数意外改写 PMU 寄存器。 +enum class PmuOwnerMainCommand : uint32_t { + Invalid = 0U, + Configure = 1U, + Restore = 2U, +}; + +// 该结构逐字段复刻 A5 KernelArgs 的 152B ABI,但只使用固定宽度整数,因而 +// 不依赖 Simpler 的 DeviceArgs/Runtime C++ 类型。runtime_args_device 指向 +// PmuOwnerControl;command 位于原 enable_profiling_flag 的 offset 128。 +// 其余字段保持为零,既满足主 aicpu_scheduler 固定布局,也不引入外部依赖。 +struct PmuOwnerMainKernelArgs { + uint64_t unused[5]; // 0..39 + uint64_t device_args_device; // 40,当前 owner 不使用 + uint64_t runtime_args_device; // 48,PmuOwnerControl 的 GM 地址 + uint64_t register_bases_device; // 56,当前 control 已内嵌基址,保持为零 + uint64_t dump_data_base; // 64 + uint64_t l2_swimlane_data_base; // 72 + uint64_t pmu_data_base; // 80 + uint64_t dep_gen_data_base; // 88 + uint64_t l2_swimlane_rotation_table; // 96 + uint64_t aicore_pmu_ring_addrs; // 104 + uint64_t scope_stats_data_base; // 112 + uint32_t log_level; // 120 + uint32_t log_info_v; // 124 + uint32_t command; // 128,PmuOwnerMainCommand + uint32_t reserved_alignment; // 132 + uint64_t device_wall_data_base; // 136 + uint32_t device_id; // 144 + uint32_t force_simt_anchor; // 148 +}; + +static_assert(sizeof(PmuOwnerMainCommand) == sizeof(uint32_t), "PMU owner command ABI changed"); +static_assert(offsetof(PmuOwnerMainKernelArgs, device_args_device) == 40U, "device args offset changed"); +static_assert(offsetof(PmuOwnerMainKernelArgs, runtime_args_device) == 48U, "control pointer offset changed"); +static_assert(offsetof(PmuOwnerMainKernelArgs, register_bases_device) == 56U, "register pointer offset changed"); +static_assert(offsetof(PmuOwnerMainKernelArgs, command) == 128U, "PMU owner command offset changed"); +static_assert(offsetof(PmuOwnerMainKernelArgs, device_id) == 144U, "device id offset changed"); +static_assert(sizeof(PmuOwnerMainKernelArgs) == 152U, "main aicpu_scheduler KernelArgs ABI changed"); +static_assert(alignof(PmuOwnerMainKernelArgs) == 8U, "KernelArgs alignment changed"); + +inline PmuOwnerMainKernelArgs MakePmuOwnerMainKernelArgs( + uint64_t control_device, PmuOwnerMainCommand command, uint32_t device_id +) +{ + PmuOwnerMainKernelArgs arguments{}; + arguments.runtime_args_device = control_device; + arguments.command = static_cast(command); + arguments.device_id = device_id; + return arguments; +} + +} // namespace pa_scheduler::pmu_owner + +#endif // PA_SCHEDULER_CCEC_PMU_OWNER_MAIN_ABI_H_ diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_main_loader.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_main_loader.h new file mode 100644 index 0000000000..ea34f5fe59 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_main_loader.h @@ -0,0 +1,389 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_CCEC_PMU_OWNER_MAIN_LOADER_H_ +#define PA_SCHEDULER_CCEC_PMU_OWNER_MAIN_LOADER_H_ + +// PMU owner 的自包含 host 装载器: +// 1. 通过 libaicpu_extend_kernels bootstrap 临时 dispatcher; +// 2. dispatcher 将 owner SO 落到主 aicpu_scheduler 的预安装目录; +// 3. 用 cpuKernelMode=0 JSON 注册 owner 的 simpler_aicpu_exec; +// 4. 后续 Configure/Restore 都用缓存的 rtFuncHandle 直接下发。 +// +// 本头文件故意不定义 owner 命令字段。Launch 接受调用方构造的完整参数块, +// 从而让装载 ABI 与 PMU 状态机 ABI 解耦,也便于先独立验证 Path-A。 + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "acl/acl.h" +#include "runtime/rt.h" +#include "runtime/runtime/rts/rts_kernel.h" + +namespace pa_scheduler::pmu_owner { + +class MainAicpuLoader { +public: + MainAicpuLoader() = default; + MainAicpuLoader(const MainAicpuLoader &) = delete; + MainAicpuLoader &operator=(const MainAicpuLoader &) = delete; + MainAicpuLoader(MainAicpuLoader &&) = delete; + MainAicpuLoader &operator=(MainAicpuLoader &&) = delete; + + ~MainAicpuLoader() { (void)Finalize(); } + + // stream 必须属于当前 device,并且调用期间当前 ACL device 不能切换。 + // 成功后 owner SO 已注册到主 aicpu_scheduler,但尚未执行任何 PMU 命令。 + int Initialize( + const std::string &dispatcher_so_path, const std::string &owner_so_path, + aclrtStream stream, int32_t device_id + ) + { + if (IsInitialized() || stream == nullptr || device_id < 0) { + return Fail("Initialize received invalid state, stream, or device id", kInvalidArgument); + } + + const std::vector dispatcher = ReadBinary(dispatcher_so_path); + const std::vector owner = ReadBinary(owner_so_path); + if (dispatcher.empty() || owner.empty()) { + std::fprintf( + stderr, "[PMU_OWNER_LOADER] cannot read dispatcher/owner: %s (%zu B), %s (%zu B)\n", + dispatcher_so_path.c_str(), dispatcher.size(), owner_so_path.c_str(), owner.size() + ); + return kFileError; + } + + device_id_ = device_id; + owner_fingerprint_ = FingerprintBytes(owner.data(), owner.size()); + owner_so_basename_ = MakeOwnerSoBasename(owner_fingerprint_, device_id_); + op_type_ = MakeOpType(owner_fingerprint_, device_id_); + + int result = Bootstrap(dispatcher, owner, stream); + if (result == 0) result = RegisterOwner(); + if (result != 0) { + (void)Finalize(); + return result; + } + return 0; + } + + // 参数块由 runtime 在 launch 时复制;调用方只需保证本函数返回前 host + // buffer 有效。参数中的 GM 指针仍必须在设备命令同步结束前保持有效。 + int Launch( + aclrtStream stream, void *kernel_arguments, size_t argument_bytes, + uint32_t aicpu_blocks = 1U + ) const + { + if (!IsInitialized() || stream == nullptr || kernel_arguments == nullptr || + argument_bytes == 0U || argument_bytes > std::numeric_limits::max() || + aicpu_blocks == 0U) { + return Fail("Launch received invalid state or arguments", kInvalidArgument); + } + + rtCpuKernelArgs_t cpu_arguments = {}; + cpu_arguments.baseArgs.args = kernel_arguments; + cpu_arguments.baseArgs.argsSize = static_cast(argument_bytes); + rtKernelLaunchCfg_t launch_config = {}; + rtLaunchKernelAttr_t launch_attribute = {}; + launch_config.attrs = &launch_attribute; + launch_config.numAttrs = 0U; + + const rtError_t result = rtsLaunchCpuKernel( + function_handle_, aicpu_blocks, static_cast(stream), + &launch_config, &cpu_arguments + ); + if (result != RT_ERROR_NONE) { + std::fprintf(stderr, "[PMU_OWNER_LOADER] rtsLaunchCpuKernel failed: %d\n", result); + } + return static_cast(result); + } + + // Finalize 只释放 host/runtime 注册资源,不删除设备侧预安装 SO;后者按内容 + // 指纹命名,可由同一设备上的后续进程原子覆盖。 + int Finalize() + { + int result = 0; + function_handle_ = nullptr; + if (binary_handle_ != nullptr) { + const rtError_t unload_result = rtsBinaryUnload(binary_handle_); + if (unload_result != RT_ERROR_NONE) { + std::fprintf(stderr, "[PMU_OWNER_LOADER] rtsBinaryUnload failed: %d\n", unload_result); + result = static_cast(unload_result); + } + binary_handle_ = nullptr; + } + if (!json_path_.empty()) { + if (std::remove(json_path_.c_str()) != 0 && result == 0) { + std::fprintf(stderr, "[PMU_OWNER_LOADER] remove JSON failed: %s\n", json_path_.c_str()); + result = kFileError; + } + json_path_.clear(); + } + device_id_ = -1; + owner_fingerprint_ = 0U; + owner_so_basename_.clear(); + op_type_.clear(); + return result; + } + + bool IsInitialized() const { return binary_handle_ != nullptr && function_handle_ != nullptr; } + uint64_t OwnerFingerprint() const { return owner_fingerprint_; } + const std::string &OwnerSoBasename() const { return owner_so_basename_; } + const std::string &OpType() const { return op_type_; } + +private: + static constexpr int kInvalidArgument = -1; + static constexpr int kFileError = -2; + static constexpr int kBootstrapError = -3; + static constexpr uint64_t kFnvOffsetBasis = UINT64_C(14695981039346656037); + static constexpr uint64_t kFnvPrime = UINT64_C(1099511628211); + static constexpr const char *kOwnerFunction = "simpler_aicpu_exec"; + + struct DeviceBuffer { + void *address = nullptr; + DeviceBuffer() = default; + DeviceBuffer(const DeviceBuffer &) = delete; + DeviceBuffer &operator=(const DeviceBuffer &) = delete; + ~DeviceBuffer() + { + if (address != nullptr) (void)aclrtFree(address); + } + aclError Allocate(size_t bytes) + { + return aclrtMalloc(&address, bytes, ACL_MEM_MALLOC_HUGE_FIRST); + } + }; + + static int Fail(const char *message, int code) + { + std::fprintf(stderr, "[PMU_OWNER_LOADER] %s\n", message); + return code; + } + + static std::vector ReadBinary(const std::string &path) + { + std::ifstream input(path, std::ios::binary | std::ios::ate); + if (!input.is_open()) return {}; + const std::streampos end = input.tellg(); + if (end <= std::streampos(0) || + static_cast(end) > static_cast(std::numeric_limits::max())) { + return {}; + } + std::vector bytes(static_cast(end)); + input.seekg(0, std::ios::beg); + if (!input.read(reinterpret_cast(bytes.data()), static_cast(bytes.size()))) { + return {}; + } + return bytes; + } + + static uint64_t FingerprintBytes(const void *data, size_t bytes) + { + const auto *input = static_cast(data); + uint64_t hash = kFnvOffsetBasis; + for (size_t index = 0U; index < bytes; ++index) { + hash ^= input[index]; + hash *= kFnvPrime; + } + return hash; + } + + static std::string MakeOwnerSoBasename(uint64_t fingerprint, int32_t device_id) + { + char name[128] = {}; + (void)snprintf( + name, sizeof(name), "pa_scheduler_pmu_owner_%016llx_d%d.so", + static_cast(fingerprint), device_id + ); + return name; + } + + static std::string MakeOpType(uint64_t fingerprint, int32_t device_id) + { + char name[160] = {}; + (void)snprintf( + name, sizeof(name), "pa_scheduler_pmu_owner_%016llx_d%d", + static_cast(fingerprint), device_id + ); + return name; + } + + int Bootstrap( + const std::vector &dispatcher, const std::vector &owner, + aclrtStream stream + ) const + { + DeviceBuffer dispatcher_device; + DeviceBuffer owner_device; + DeviceBuffer device_args; + aclError acl_result = dispatcher_device.Allocate(dispatcher.size()); + if (acl_result != ACL_SUCCESS) return ReportAcl("aclrtMalloc(dispatcher)", acl_result); + acl_result = aclrtMemcpy( + dispatcher_device.address, dispatcher.size(), dispatcher.data(), dispatcher.size(), + ACL_MEMCPY_HOST_TO_DEVICE + ); + if (acl_result != ACL_SUCCESS) return ReportAcl("aclrtMemcpy(dispatcher H2D)", acl_result); + + acl_result = owner_device.Allocate(owner.size()); + if (acl_result != ACL_SUCCESS) return ReportAcl("aclrtMalloc(owner)", acl_result); + acl_result = aclrtMemcpy( + owner_device.address, owner.size(), owner.data(), owner.size(), ACL_MEMCPY_HOST_TO_DEVICE + ); + if (acl_result != ACL_SUCCESS) return ReportAcl("aclrtMemcpy(owner H2D)", acl_result); + + constexpr size_t kDeviceArgsBytes = 160U; + uint8_t host_device_args[kDeviceArgsBytes] = {}; + const auto write_qword = [&](size_t offset, uint64_t value) { + std::memcpy(host_device_args + offset, &value, sizeof(value)); + }; + write_qword(96U, reinterpret_cast(dispatcher_device.address)); + write_qword(104U, static_cast(dispatcher.size())); + write_qword(112U, static_cast(device_id_)); + write_qword(120U, reinterpret_cast(owner_device.address)); + write_qword(128U, static_cast(owner.size())); + + acl_result = device_args.Allocate(kDeviceArgsBytes); + if (acl_result != ACL_SUCCESS) return ReportAcl("aclrtMalloc(bootstrap args)", acl_result); + acl_result = aclrtMemcpy( + device_args.address, kDeviceArgsBytes, host_device_args, kDeviceArgsBytes, + ACL_MEMCPY_HOST_TO_DEVICE + ); + if (acl_result != ACL_SUCCESS) return ReportAcl("aclrtMemcpy(bootstrap args H2D)", acl_result); + + // k_args 总长和三个字符串 offset 与仓内已上板的 Path-A 完全一致。 + struct BootstrapArguments { + struct { + uint64_t unused[5]; + uint64_t device_args_address; + uint64_t padding[20]; + } kernel_args; + char kernel_name[32]; + char so_name[32]; + char op_name[32]; + } arguments = {}; + static_assert(offsetof(BootstrapArguments, kernel_args.device_args_address) == 40U, "bootstrap ABI changed"); + arguments.kernel_args.device_args_address = reinterpret_cast(device_args.address); + constexpr char kBootstrapKernel[] = "DynTileFwkKernelServerInit"; + constexpr char kBootstrapSo[] = "libaicpu_extend_kernels.so"; + static_assert(sizeof(kBootstrapKernel) <= sizeof(arguments.kernel_name), "bootstrap kernel name too long"); + static_assert(sizeof(kBootstrapSo) <= sizeof(arguments.so_name), "bootstrap SO name too long"); + std::memcpy(arguments.kernel_name, kBootstrapKernel, sizeof(kBootstrapKernel)); + std::memcpy(arguments.so_name, kBootstrapSo, sizeof(kBootstrapSo)); + + rtAicpuArgsEx_t runtime_arguments = {}; + runtime_arguments.args = &arguments; + runtime_arguments.argsSize = sizeof(arguments); + runtime_arguments.kernelNameAddrOffset = offsetof(BootstrapArguments, kernel_name); + runtime_arguments.soNameAddrOffset = offsetof(BootstrapArguments, so_name); + + const rtError_t launch_result = rtAicpuKernelLaunchExWithArgs( + rtKernelType_t::KERNEL_TYPE_AICPU_KFC, "AST_DYN_AICPU", 1U, + &runtime_arguments, nullptr, static_cast(stream), 0U + ); + if (launch_result != RT_ERROR_NONE) { + std::fprintf( + stderr, "[PMU_OWNER_LOADER] rtAicpuKernelLaunchExWithArgs failed: %d\n", + launch_result + ); + return static_cast(launch_result); + } + acl_result = aclrtSynchronizeStream(stream); + if (acl_result != ACL_SUCCESS) return ReportAcl("aclrtSynchronizeStream(bootstrap)", acl_result); + return 0; + } + + int RegisterOwner() + { + char path[256] = {}; + (void)snprintf( + path, sizeof(path), "/tmp/pa_scheduler_pmu_owner_%016llx_d%d_p%d_i%016llx.json", + static_cast(owner_fingerprint_), device_id_, static_cast(getpid()), + static_cast(reinterpret_cast(this)) + ); + json_path_ = path; + if (!WriteJson()) return kFileError; + + rtLoadBinaryOption_t option = {}; + option.optionId = RT_LOAD_BINARY_OPT_CPU_KERNEL_MODE; + option.value.cpuKernelMode = 0; + rtLoadBinaryConfig_t configuration = {}; + configuration.options = &option; + configuration.numOpt = 1U; + + rtError_t result = rtsBinaryLoadFromFile(json_path_.c_str(), &configuration, &binary_handle_); + if (result != RT_ERROR_NONE) { + std::fprintf(stderr, "[PMU_OWNER_LOADER] rtsBinaryLoadFromFile failed: %d\n", result); + return static_cast(result); + } + result = rtsFuncGetByName(binary_handle_, op_type_.c_str(), &function_handle_); + if (result != RT_ERROR_NONE) { + std::fprintf(stderr, "[PMU_OWNER_LOADER] rtsFuncGetByName(%s) failed: %d\n", op_type_.c_str(), result); + return static_cast(result); + } + return 0; + } + + bool WriteJson() const + { + std::ofstream json(json_path_, std::ios::out | std::ios::trunc); + if (!json.is_open()) { + std::fprintf(stderr, "[PMU_OWNER_LOADER] cannot create JSON: %s\n", json_path_.c_str()); + return false; + } + // 所有动态字段仅含固定前缀、十六进制、十进制和下划线,不需要 JSON 转义。 + json << "{\n" + << " \"" << op_type_ << "\": {\n" + << " \"opInfo\": {\n" + << " \"functionName\": \"" << kOwnerFunction << "\",\n" + << " \"kernelSo\": \"" << owner_so_basename_ << "\",\n" + << " \"opKernelLib\": \"AICPUKernel\",\n" + << " \"computeCost\": \"100\",\n" + << " \"engine\": \"DNN_VM_AICPU\",\n" + << " \"flagAsync\": \"False\",\n" + << " \"flagPartial\": \"False\",\n" + << " \"userDefined\": \"False\"\n" + << " }\n" + << " }\n" + << "}\n"; + json.close(); + if (!json) { + std::fprintf(stderr, "[PMU_OWNER_LOADER] writing JSON failed: %s\n", json_path_.c_str()); + return false; + } + return true; + } + + static int ReportAcl(const char *operation, aclError result) + { + std::fprintf(stderr, "[PMU_OWNER_LOADER] %s failed: %d\n", operation, static_cast(result)); + return static_cast(result == ACL_SUCCESS ? kBootstrapError : result); + } + + int32_t device_id_ = -1; + uint64_t owner_fingerprint_ = 0U; + std::string owner_so_basename_; + std::string op_type_; + std::string json_path_; + rtBinHandle binary_handle_ = nullptr; + rtFuncHandle function_handle_ = nullptr; +}; + +} // namespace pa_scheduler::pmu_owner + +#endif // PA_SCHEDULER_CCEC_PMU_OWNER_MAIN_LOADER_H_ From 640efe50538785cead6c84ef0166c57e448b8383 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 09:30:19 +0000 Subject: [PATCH 013/214] =?UTF-8?q?=E6=B5=8B=E8=AF=95(a5):=20=E5=BB=BA?= =?UTF-8?q?=E7=AB=8B=E5=8D=95=E6=AC=A1I-cache=20miss=E6=80=A7=E8=83=BD?= =?UTF-8?q?=E6=A0=87=E5=B0=BA?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 为PA scalar性能分析增加可与单次atomic约160ns直接对照的I-cache miss量级,避免继续从热循环aggregate residual臆测单次开销。 测试方法: - 新增CCEC icache-single PMU窗口和可配置的每核trial数 - 使用8B、128B line对齐的唯一目标函数,并在窗外执行64KiB指令sweep制造capacity eviction - cold/warm复用同一target、harness与PMU gate;warm预取放在read-clear前,避免窗外事件污染 - 每个phase丢弃一次分支训练样本,并在各角色内平衡cold-first与warm-first顺序 - 同时读取CNT_TOTAL、CNT6、CNT7和1GHz sys counter,以cold-warm差值计算cycles/ns per miss - 构建期强制校验target、harness、thrash的符号尺寸、128B对齐和链接顺序 正确性门禁: - 逐核要求cold CNT7等于trials、warm CNT7等于0 - 要求cold-warm周期和时间差为正,且96个物理子核全部完成配对校准 - host分别输出ALL、AIC、AIV统计及可直接用于scalar归因的换算公式 - 与远端atomic_trace_calls合并后,WorkerResult诊断sidecar按64B扩展到832B;生产DistCore ABI及既有字段offset不变 A5实测结果: - 64 trials/core连续10轮全部通过,ALL中位86.596ns/miss,范围86.532到86.792ns - 128 trials/core连续5轮全部通过,ALL中位89.629ns/miss,范围89.615到89.648ns - 15轮均满足每次cold严格增加一个CNT7 miss且warm miss为0 - AIC/AIV只相差数ns且方向随时段变化,不建立角色伪精确常数 - scalar分析统一采用T_icache_est_ns = CNT7_miss_total * 90;1000次miss约为90us 使用边界与文档: - compulsory、capacity、conflict miss都会计入CNT7总数 - 90ns是96核并发cold/warm校准的一阶等效标尺,不宣称为预取、重叠miss和不同下级命中条件下严格可加的stall - 同步更新PA分析、独立复现指南和atomic_probe测试数据表,记录原始日志、AIC/AIV分项及公式 验证: - 合并远端最新atomic泳道提交后,CCEC完整构建、mixed ELF及I-cache布局检查通过 - CPU构建与smoke全部通过 - 原始A5 icache-single长测及暂存快照短测通过96/96核和精确CNT7门禁 - git diff检查通过 --- ...05\345\206\265\345\210\206\346\236\220.md" | 55 ++++++- ...77\347\224\250\346\214\207\345\215\227.md" | 60 +++++++- tests/atomic_probe/pa_scheduler/ccec/build.sh | 45 ++++++ tests/atomic_probe/pa_scheduler/ccec/host.cpp | 132 ++++++++++++++-- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 143 ++++++++++++++++-- .../pa_scheduler/ccec/pmu_probe.h | 10 +- .../pa_scheduler/common/pa_model.h | 13 +- tests/atomic_probe/pa_scheduler/run.sh | 3 +- tests/atomic_probe/test_case.md | 24 +++ 9 files changed, 451 insertions(+), 34 deletions(-) diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index f18083bb15..12f476fadc 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -790,7 +790,53 @@ tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ scalar2x100k_10_20260718_021035_console.log ~~~ -#### 7.3.3 阶段决定与后续使用边界 +#### 7.3.3 单次 CNT7 I-cache miss 的时间标尺 + +此前 100000 NOP 的 aggregate residual 只能证明 I-cache 事件会响应,不能把一段 +热循环里的多次 request/miss 直接除成“单次 miss 延迟”。为给 scalar 性能分析 +建立与“单次 atomic 约 160 ns”同样直观的数量级,CCEC 新增 +`--pmu-window icache-single` 配对校准:目标函数只有 8 B 并按 128 B I-cache line +对齐;每个 cold trial 先在窗口外执行 64 KiB 指令 capacity sweep,warm trial 则在 +PMU read-clear 前额外调用一次相同目标。1 GHz sys counter 只包住最终目标调用, +两条路径的 PMU gate、harness 和目标符号完全相同。 + +64 trials/core × 10 轮以及 128 trials/core × 5 轮均满足: + +~~~text +cold CNT7 miss == trials +warm CNT7 miss == 0 +miss_delta == 96 * trials +calibrated_cores == 96/96 +~~~ + +总体系数的观测范围为 86.532~89.648 ns/miss;同一时段重跑 64 和 128 trials +都约为 89.6 ns/miss,说明几 ns 的变化属于并发环境下的有效 miss penalty 波动, +不应保留成伪精确常数。后续 scalar 分析使用下面的取整公式即可: + +~~~text +T_icache_est_ns = CNT7_miss_total * 90 +T_icache_est_us = CNT7_miss_total * 0.09 +~~~ + +即 1,000 个 miss 约解释 90 us,10,000 个约解释 0.9 ms;单次 I-cache miss 的 +量级约为当前 160 ns atomic 标尺的 56%。如果结果已经按角色拆开,也可以使用当次 +探针打印的 AIC/AIV 系数分别计算后相加;只有总 miss 时统一乘 90 ns。 + +compulsory、capacity、conflict miss 都会进入 `CNT7` 总数,PMU 本身不提供原因 +分类,所以上式三类全算。本探针刻意制造的是 capacity eviction,测得的是 96 核 +并发、cold 相对 warm 的一阶等效时间。真实 scalar 路径可能存在预取、多个 miss +重叠、不同下级命中位置或排队,因此乘积用于直观归因和数量级判断,不能宣称为 +逐次精确、完全可加的 stall 时间。 + +原始日志为: + +~~~text +tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ + icache_single_64x10_20260718_085929_3232836_console.log + icache_single_128x5_20260718_090151_3235468_console.log +~~~ + +#### 7.3.4 阶段决定与后续使用边界 O1 已达到“可用”的门槛:物理核映射、事件 selector、正向敏感性、A/A 重复性 和双 gate 累计全部在真实 A5 上得到动态验证。因此后续可以恢复 atomic 优化, @@ -816,9 +862,10 @@ D1 已在 standalone 公共调度器中增加 worker-local 软件计数,不新 - 扫描遇到 not-ready flag 退出时递增 terminal load。 计数先保存在本核 `LocalStats`,kernel 结束时才发布到独占 `WorkerResult`。 -`WorkerResult` 从 704 B 扩展到 768 B,但原 PMU 字段 offset、生产 DistGlobal/DistCore -offset 和 `LocalSlot` ABI 都不变。三后端完整重建后,smoke 和 256 batch 默认 NOP -均通过全部语义断言。 +`WorkerResult` 在 D1 从 704 B 扩展到 768 B;合入 atomic 泳道计数和 I-cache +cold/warm 配对字段后,standalone 诊断 sidecar 按完整 cache line 扩展到 832 B。 +原 PMU 字段 offset、生产 DistGlobal/DistCore offset 和 `LocalSlot` ABI 都不变。 +三后端完整重建后,smoke 和 256 batch 默认 NOP 均通过全部语义断言。 对任一 worker,若其完成数为 `Cw`、fanin 边数为 `Ew`、失败 load 为 `Fw`,则 逐核检查: diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 434fd9c319..804089727d 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -301,6 +301,10 @@ CCEC 后端提供一个显式诊断模式,用来验证局部 scalar 性能观 | `pmu_scalar_busy` | `CNT2 = 0x1` | scalar 原始事件计数 | | `pmu_icache_requests` | `CNT6 = 0x34` | I-cache request | | `pmu_icache_misses` | `CNT7 = 0x35` | I-cache miss | +| `pmu_window_ticks` | 1 GHz sys counter | cold 目标调用窗口,1 tick = 1 ns | +| `pmu_warm_total_cycles` | PMU total | 同核 warm 对照窗口累计周期 | +| `pmu_warm_window_ticks` | 1 GHz sys counter | warm 目标调用窗口 | +| `pmu_warm_icache_requests/misses` | `CNT6/CNT7` | 同核 warm 对照 request/miss | selector 和 PMU framework 仍由 CANN 9.1 的 task-based profiler 配置;CCEC 只做门控与读取。一次 snapshot 会消费/清除此前累计,因此窗口前先冻结并做一次 @@ -327,13 +331,18 @@ msprof \ --pmu-window empty ``` -三个诊断窗口的用途是: +各诊断模式的用途是: - `empty`:只执行一次 start/stop,量空窗口的 gate 固定开销;末尾 snapshot 在 stop 后执行,其 MMIO 读取开销不计入 total; - `scalar`:在同一窗口内执行 `--pmu-scalar-nops N` 个受控 NOP; - `scalar-double`:执行两段相同 NOP,中间只 stop/start、不读 counter,用于验证 多个局部窗口能否暂停后继续累计; +- `icache-single`:每核反复执行同一个 8 B、128 B line 对齐的目标函数。cold + 路径先在计时窗外执行 64 KiB 指令 sweep 逐出目标,warm 路径再在 read-clear + 前预取一次相同目标;target、harness、thrash 的链接顺序由构建门禁核对。每个 + phase 先丢弃一次分支训练样本,AIC/AIV 内各由一半 worker 采用 cold-first、 + 一半采用 warm-first; - `off`:默认值,不申请 MMIO 表,也不要求由 `msprof` 启动。 例如 10 万 NOP 的正向敏感性测试只需把末尾参数换成: @@ -342,6 +351,23 @@ msprof \ --pmu-window scalar --pmu-scalar-nops 100000 ``` +单次 I-cache miss 标尺使用: + +```bash +--pmu-window icache-single --pmu-icache-trials 64 +``` + +除通用 PMU 断言外,每轮还必须看到: + +```text +icache_pairs=96/96 calibrated_cores=96/96 +[ASSERT] each cold trial adds exactly one CNT7 I-cache miss PASS +``` + +门禁逐核要求 cold 的 `CNT7 == trials`、warm 的 `CNT7 == 0`,并要求 cold-warm +时间差为正。也就是说,最终系数的分母不是推测的循环次数,而是严格闭合的 +`CNT7` miss 差值。 + 每轮必须同时看到: ```text @@ -370,6 +396,38 @@ rate 使用 `sum(miss) / sum(request)`,不平均逐核百分比。`pmu_scalar_ snapshot。首次热身的 p95 偶有偏高,正式比较应丢弃首轮,并保留 A/A 重复性数据。 +#### 单次 CNT7 I-cache miss 的 scalar 估算标尺 + +2026-07-18 的 96 核并发 cold/warm 配对结果中,每个 cold trial 都严格增加一个 +`CNT7` miss,warm miss 恒为 0。64 trials/core 的 10 轮总体系数为 +86.532~86.792 ns/miss;同一日稍后 128 trials/core 的 5 轮为 +89.615~89.648 ns/miss。64 trials/core 在后一时段复测也约为 89.6 ns/miss, +因此 86 与 90 ns 的差异主要反映运行时段和共享下级存储状态,不值得作为 scalar +归因标尺保留小数精度。工程分析统一取整为: + +```text +估算的 scalar I-cache miss 时间(ns) = CNT7_I-cache_miss_total × 90 +估算的 scalar I-cache miss 时间(us) = CNT7_I-cache_miss_total × 0.09 +``` + +例如 1,000 次 miss 约为 90 us,10,000 次约为 0.9 ms。若已有 AIC/AIV 分项, +可以分别乘各自当次校准输出的 `[ICACHE-FORMULA-AIC/AIV]`;只需要总量级时直接 +使用 90 ns/miss,避免伪精确。 + +`CNT7` 只报告 I-cache miss 总数,不区分 compulsory、capacity 和 conflict 原因; +三类都应计入上式。本探针用 64 KiB sweep 明确制造 capacity eviction,因此得到的 +是 96 核并发条件下、cold 相对 warm 的有效 miss penalty。它适合回答“这些 miss +大约能解释多少 scalar 时间”,不是硬件逐次给出的可加 stall:真实代码中的预取、 +miss 重叠、下级命中位置和并发排队都会让实际关键路径与简单乘积有偏差。 + +原始输出保留在: + +```text +tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ + icache_single_64x10_20260718_085929_3232836_console.log + icache_single_128x5_20260718_090151_3235468_console.log +``` + 这里验证的是观察手段,不是 PA 优化本身。后续用于 Claim、EfDrain、 WaitForSlot 或 HeapGuard 时,应先做一次 baseline read-clear,多个局部窗口之间只切 gate,最后每核读取一次;禁止每个 Submit 都读 MMIO。当前双段只证明多窗口续积 diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh index 574f2cb940..21f9677a25 100755 --- a/tests/atomic_probe/pa_scheduler/ccec/build.sh +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -97,6 +97,51 @@ for entry in pa_scheduler_0_mix_aic pa_scheduler_0_mix_aiv; do done echo "[CHECK] both 1:2 mixed entries and metadata sections are present" +check_icache_probe_layout() { + local role="$1" + local target="pa_icache_target_${role}" + local harness="pa_icache_measure_${role}" + local thrash="pa_icache_thrash_${role}" + local target_record + local harness_record + local thrash_record + target_record="$(awk -v name="$target" '$4 == "FUNC" && index($NF, name) != 0 {print $2, $3; exit}' <<<"$SYMBOL_TABLE")" + harness_record="$(awk -v name="$harness" '$4 == "FUNC" && index($NF, name) != 0 {print $2, $3; exit}' <<<"$SYMBOL_TABLE")" + thrash_record="$(awk -v name="$thrash" '$4 == "FUNC" && index($NF, name) != 0 {print $2, $3; exit}' <<<"$SYMBOL_TABLE")" + if [[ -z "$target_record" || -z "$harness_record" || -z "$thrash_record" ]]; then + echo "Missing I-cache probe symbols for $role" >&2 + exit 1 + fi + + local target_hex target_size harness_hex harness_size thrash_hex thrash_size + read -r target_hex target_size <<<"$target_record" + read -r harness_hex harness_size <<<"$harness_record" + read -r thrash_hex thrash_size <<<"$thrash_record" + local target_address=$((16#$target_hex)) + local harness_address=$((16#$harness_hex)) + local thrash_address=$((16#$thrash_hex)) + if (( target_address % 128 != 0 || target_size == 0 || target_size > 16 )); then + echo "Invalid single-fetch-block I-cache target for $role: address=0x$target_hex size=$target_size" >&2 + exit 1 + fi + if (( thrash_size < 65536 )); then + echo "I-cache thrash body is smaller than 64 KiB for $role: size=$thrash_size" >&2 + exit 1 + fi + if (( harness_address % 128 != 0 || target_address + 128 > harness_address || + harness_address + harness_size > thrash_address )); then + echo "I-cache layout must be target -> harness -> thrash for $role" >&2 + exit 1 + fi + echo "[CHECK] $role I-cache target=0x$target_hex/$target_size harness=0x$harness_hex/$harness_size "\ + "thrash=0x$thrash_hex/$thrash_size" +} + +# 编译器不能把目标扩到两个 16B fetch block,也不能折叠 64 KiB 冲刷体;否则 +# cold/warm 虽然仍可能产生数字,却不再代表可解释的单次 I-cache miss。 +check_icache_probe_layout aic +check_icache_probe_layout aiv + # host runner 只链接用户 CANN 9.1 的 ACL/runtime,并写入同一安装目录的 rpath,运行时不需要 simpler 动态库。 # `-Werror` 让 host API 签名或尺寸类型变化在构建期暴露,避免到上板阶段才出现参数截断。 echo "[BUILD] CCEC host runner" diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index c19f24d07d..ab06280f7c 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -54,6 +54,7 @@ std::vector ReadBinary(const std::string &path) { struct PmuOptions { pa_scheduler::ccec_pmu::WindowMode mode = pa_scheduler::ccec_pmu::WindowMode::Off; uint32_t scalar_nops = 100000; + uint32_t icache_trials = 64; }; const char *PmuModeName(pa_scheduler::ccec_pmu::WindowMode mode) { @@ -66,6 +67,8 @@ const char *PmuModeName(pa_scheduler::ccec_pmu::WindowMode mode) { return "scalar"; case pa_scheduler::ccec_pmu::WindowMode::ScalarDouble: return "scalar-double"; + case pa_scheduler::ccec_pmu::WindowMode::IcacheSingle: + return "icache-single"; } return "invalid"; } @@ -74,11 +77,13 @@ bool ParsePmuOptions(int argc, char **argv, PmuOptions *pmu, std::vector // PMU 参数只属于 CCEC 验证分支;先摘出再交给三后端共享 parser,避免 CPU/AscendC 静默接受却不生效。 bool mode_seen = false; bool nops_seen = false; + bool icache_trials_seen = false; common_argv->clear(); common_argv->push_back(argv[0]); for (int index = 1; index < argc; ++index) { const std::string argument = argv[index]; - if (argument != "--pmu-window" && argument != "--pmu-scalar-nops") { + if (argument != "--pmu-window" && argument != "--pmu-scalar-nops" && + argument != "--pmu-icache-trials") { common_argv->push_back(argv[index]); continue; } @@ -101,19 +106,30 @@ bool ParsePmuOptions(int argc, char **argv, PmuOptions *pmu, std::vector pmu->mode = pa_scheduler::ccec_pmu::WindowMode::Scalar; } else if (name == "scalar-double") { pmu->mode = pa_scheduler::ccec_pmu::WindowMode::ScalarDouble; + } else if (name == "icache-single") { + pmu->mode = pa_scheduler::ccec_pmu::WindowMode::IcacheSingle; } else { std::fprintf( - stderr, "Invalid --pmu-window value: %s (expected off|empty|scalar|scalar-double)\n", value + stderr, + "Invalid --pmu-window value: %s " + "(expected off|empty|scalar|scalar-double|icache-single)\n", + value ); return false; } mode_seen = true; - } else { + } else if (argument == "--pmu-scalar-nops") { if (nops_seen || !pa_scheduler::host::ParseUint(value, 0, 10000000, &pmu->scalar_nops)) { std::fprintf(stderr, "Invalid or duplicate --pmu-scalar-nops value: %s\n", value); return false; } nops_seen = true; + } else { + if (icache_trials_seen || !pa_scheduler::host::ParseUint(value, 1, 10000, &pmu->icache_trials)) { + std::fprintf(stderr, "Invalid or duplicate --pmu-icache-trials value: %s\n", value); + return false; + } + icache_trials_seen = true; } } if (nops_seen && pmu->mode != pa_scheduler::ccec_pmu::WindowMode::Scalar && @@ -121,6 +137,10 @@ bool ParsePmuOptions(int argc, char **argv, PmuOptions *pmu, std::vector std::fprintf(stderr, "--pmu-scalar-nops requires a scalar PMU window.\n"); return false; } + if (icache_trials_seen && pmu->mode != pa_scheduler::ccec_pmu::WindowMode::IcacheSingle) { + std::fprintf(stderr, "--pmu-icache-trials requires the icache-single PMU window.\n"); + return false; + } return true; } @@ -195,23 +215,73 @@ bool MapPmuRegisters(uint32_t device, PmuRegisterMappings *mappings) { void ConfigurePmu(pa_scheduler::SchedulerState *state, const PmuOptions &pmu, const void *register_table) { using namespace pa_scheduler::ccec_pmu; state->config.reserved[kConfigMode] = static_cast(pmu.mode); - state->config.reserved[kConfigScalarNops] = pmu.scalar_nops; + state->config.reserved[kConfigWorkAmount] = + pmu.mode == WindowMode::IcacheSingle ? pmu.icache_trials : pmu.scalar_nops; StorePointer(state->config.reserved, register_table); state->config.reserved[kConfigMagic] = pmu.mode == WindowMode::Off ? 0 : kConfigMagicValue; } struct PmuAggregate { std::vector total_cycles; + std::vector warm_total_cycles; + uint64_t window_ticks = 0; + uint64_t warm_window_ticks = 0; uint64_t scalar_busy = 0; uint64_t icache_requests = 0; uint64_t icache_misses = 0; + uint64_t warm_icache_requests = 0; + uint64_t warm_icache_misses = 0; }; void AddPmuSample(const pa_scheduler::WorkerResult &result, PmuAggregate *aggregate) { aggregate->total_cycles.push_back(result.pmu_total_cycles); + aggregate->warm_total_cycles.push_back(result.pmu_warm_total_cycles); + aggregate->window_ticks += result.pmu_window_ticks; + aggregate->warm_window_ticks += result.pmu_warm_window_ticks; aggregate->scalar_busy += result.pmu_scalar_busy; aggregate->icache_requests += result.pmu_icache_requests; aggregate->icache_misses += result.pmu_icache_misses; + aggregate->warm_icache_requests += result.pmu_warm_icache_requests; + aggregate->warm_icache_misses += result.pmu_warm_icache_misses; +} + +bool PrintSingleIcacheAggregate(const char *name, const PmuAggregate &aggregate, uint32_t trials_per_core) { + const pa_scheduler::host::Uint64Distribution cold_cycles = + pa_scheduler::host::SummarizeUint64(aggregate.total_cycles); + const pa_scheduler::host::Uint64Distribution warm_cycles = + pa_scheduler::host::SummarizeUint64(aggregate.warm_total_cycles); + const int64_t cycle_delta = static_cast(cold_cycles.total) - static_cast(warm_cycles.total); + const int64_t tick_delta = + static_cast(aggregate.window_ticks) - static_cast(aggregate.warm_window_ticks); + const int64_t miss_delta = + static_cast(aggregate.icache_misses) - static_cast(aggregate.warm_icache_misses); + const uint64_t attempted = static_cast(trials_per_core) * aggregate.total_cycles.size(); + const double misses_per_trial = attempted == 0 ? 0.0 : static_cast(miss_delta) / attempted; + const double cycles_per_miss = miss_delta <= 0 ? 0.0 : static_cast(cycle_delta) / miss_delta; + // get_sys_cnt 是本用例既有的 1 GHz 时间基准,因此一个 tick 直接对应 1 ns。 + const double ns_per_miss = miss_delta <= 0 ? 0.0 : static_cast(tick_delta) / miss_delta; + std::printf( + "[ICACHE-SINGLE-%s] cores=%zu trials_per_core=%u attempted=%llu " + "cold_cycles=%llu warm_cycles=%llu cycle_delta=%lld cold_ticks=%llu warm_ticks=%llu tick_delta=%lld " + "cold_req=%llu warm_req=%llu cold_miss=%llu warm_miss=%llu miss_delta=%lld " + "misses_per_trial=%.6f cycles_per_miss=%.3f ns_per_miss=%.3f\n", + name, aggregate.total_cycles.size(), trials_per_core, static_cast(attempted), + static_cast(cold_cycles.total), static_cast(warm_cycles.total), + static_cast(cycle_delta), static_cast(aggregate.window_ticks), + static_cast(aggregate.warm_window_ticks), static_cast(tick_delta), + static_cast(aggregate.icache_requests), + static_cast(aggregate.warm_icache_requests), + static_cast(aggregate.icache_misses), + static_cast(aggregate.warm_icache_misses), static_cast(miss_delta), + misses_per_trial, cycles_per_miss, ns_per_miss + ); + std::printf( + "[ICACHE-FORMULA-%s] estimated_scalar_icache_time_ns = cnt7_icache_miss * %.3f\n", + name, ns_per_miss + ); + // 校准窗口必须做到每次 cold arm 恰好多一个 CNT7 miss;否则这个系数不能 + // 直接乘到 scalar 的总 CNT7 计数上。 + return cycle_delta > 0 && tick_delta > 0 && miss_delta == static_cast(attempted); } void PrintPmuAggregate(const char *name, const PmuAggregate &aggregate) { @@ -238,6 +308,8 @@ bool ValidatePmu(const pa_scheduler::SchedulerState &state, uint32_t run, const uint32_t trusted = 0; uint32_t unique = 0; uint32_t prior_larger = 0; + uint32_t icache_pairs = 0; + uint32_t icache_calibrated_cores = 0; uint32_t bad_printed = 0; PmuAggregate all; PmuAggregate aic; @@ -249,16 +321,34 @@ bool ValidatePmu(const pa_scheduler::SchedulerState &state, uint32_t run, const const bool record_trusted = (status & kStatusRequired) == kStatusRequired; trusted += record_trusted; prior_larger += (status & kStatusPriorSnapshotLarger) != 0; + icache_pairs += (status & kStatusIcachePairObserved) != 0; + if (pmu.mode == WindowMode::IcacheSingle) { + const int64_t worker_cycle_delta = + static_cast(result.pmu_total_cycles) - + static_cast(result.pmu_warm_total_cycles); + const int64_t worker_tick_delta = + static_cast(result.pmu_window_ticks) - + static_cast(result.pmu_warm_window_ticks); + const int64_t worker_miss_delta = + static_cast(result.pmu_icache_misses) - + static_cast(result.pmu_warm_icache_misses); + icache_calibrated_cores += worker_cycle_delta > 0 && worker_tick_delta > 0 && + worker_miss_delta == static_cast(pmu.icache_trials) && + result.pmu_warm_icache_misses == 0 && result.pmu_icache_misses == pmu.icache_trials; + } if (core_id < kPhysicalSubcoreCount && !seen[core_id]) { seen[core_id] = true; ++unique; } if (!record_trusted && bad_printed < 8) { std::printf( - "[PMU-BAD] worker=%u role=%llu coreid=%u status=0x%08x total=%llu scalar=%u req=%u miss=%u\n", + "[PMU-BAD] worker=%u role=%llu coreid=%u status=0x%08x total=%llu scalar=%u req=%u miss=%u " + "warm_total=%llu warm_req=%u warm_miss=%u\n", worker, static_cast(result.role), core_id, status, static_cast(result.pmu_total_cycles), result.pmu_scalar_busy, - result.pmu_icache_requests, result.pmu_icache_misses + result.pmu_icache_requests, result.pmu_icache_misses, + static_cast(result.pmu_warm_total_cycles), result.pmu_warm_icache_requests, + result.pmu_warm_icache_misses ); ++bad_printed; } @@ -269,18 +359,32 @@ bool ValidatePmu(const pa_scheduler::SchedulerState &state, uint32_t run, const PrintPmuAggregate("ALL", all); PrintPmuAggregate("AIC", aic); PrintPmuAggregate("AIV", aiv); + bool icache_measurement_ok = true; + if (pmu.mode == WindowMode::IcacheSingle) { + const bool all_ok = PrintSingleIcacheAggregate("ALL", all, pmu.icache_trials); + const bool aic_ok = PrintSingleIcacheAggregate("AIC", aic, pmu.icache_trials); + const bool aiv_ok = PrintSingleIcacheAggregate("AIV", aiv, pmu.icache_trials); + icache_measurement_ok = all_ok && aic_ok && aiv_ok && icache_pairs == pa_scheduler::kWorkers && + icache_calibrated_cores == pa_scheduler::kWorkers; + } const bool records_ok = trusted == pa_scheduler::kWorkers; const bool core_ids_ok = unique == pa_scheduler::kWorkers; std::printf( - "[PMU] run=%u window=%s scalar_nops=%u trusted=%u/%u unique_coreids=%u/%u prior_larger=%u/%u\n", run, - PmuModeName(pmu.mode), pmu.scalar_nops, trusted, pa_scheduler::kWorkers, unique, pa_scheduler::kWorkers, - prior_larger, pa_scheduler::kWorkers + "[PMU] run=%u window=%s scalar_nops=%u icache_trials=%u trusted=%u/%u unique_coreids=%u/%u " + "prior_larger=%u/%u icache_pairs=%u/%u calibrated_cores=%u/%u\n", + run, PmuModeName(pmu.mode), pmu.scalar_nops, pmu.icache_trials, trusted, pa_scheduler::kWorkers, unique, + pa_scheduler::kWorkers, prior_larger, pa_scheduler::kWorkers, icache_pairs, pa_scheduler::kWorkers, + icache_calibrated_cores, pa_scheduler::kWorkers ); std::printf("[ASSERT] %-48s %s\n", "all PMU records have configured selectors and data", records_ok ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "all 96 PMU physical subcore ids are unique", core_ids_ok ? "PASS" : "FAIL"); - return records_ok && core_ids_ok; + if (pmu.mode == WindowMode::IcacheSingle) { + std::printf("[ASSERT] %-48s %s\n", "each cold trial adds exactly one CNT7 I-cache miss", + icache_measurement_ok ? "PASS" : "FAIL"); + } + return records_ok && core_ids_ok && icache_measurement_ok; } } // namespace @@ -298,7 +402,8 @@ int main(int argc, char **argv) { if (parse_status == pa_scheduler::host::ParseStatus::Help) { std::fprintf( stderr, - "CCEC PMU options: [--pmu-window off|empty|scalar|scalar-double] [--pmu-scalar-nops N]\n" + "CCEC PMU options: [--pmu-window off|empty|scalar|scalar-double|icache-single] " + "[--pmu-scalar-nops N] [--pmu-icache-trials N]\n" ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; @@ -310,8 +415,9 @@ int main(int argc, char **argv) { } pa_scheduler::host::PrintBanner("CCEC", options); std::printf( - "[PMU-CONFIG] window=%s scalar_nops=%u source=direct-per-core requires=msprof-PipeUtilization\n", - PmuModeName(pmu_options.mode), pmu_options.scalar_nops + "[PMU-CONFIG] window=%s scalar_nops=%u icache_trials=%u source=direct-per-core " + "requires=msprof-PipeUtilization\n", + PmuModeName(pmu_options.mode), pmu_options.scalar_nops, pmu_options.icache_trials ); // 正常及后处理路径依次完成 ACL 初始化、选卡、stream/ELF/设备区创建、launch/D2H diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index 3ccbdcd025..1549ed2664 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -46,6 +46,30 @@ __aicore__ inline void RuntimeNop(uint32_t count) { __builtin_cce_pipe_barrier(PIPE_ALL); } +#if defined(PA_BUILD_AIC) +#define PA_ICACHE_TARGET_NAME pa_icache_target_aic +#define PA_ICACHE_MEASURE_NAME pa_icache_measure_aic +#define PA_ICACHE_THRASH_NAME pa_icache_thrash_aic +#elif defined(PA_BUILD_AIV) +#define PA_ICACHE_TARGET_NAME pa_icache_target_aiv +#define PA_ICACHE_MEASURE_NAME pa_icache_measure_aiv +#define PA_ICACHE_THRASH_NAME pa_icache_thrash_aiv +#endif + +// 目标函数的入口与 128B I-cache line 对齐,并保持在一个 16B IFU fetch block +// 内;cold/warm 两条路径调用完全相同的符号,唯一变量是计时窗前是否执行过它。 +__aicore__ static __attribute__((noinline, used, aligned(128), section(".text.pa_icache_target"))) +void PA_ICACHE_TARGET_NAME() { + asm volatile( + ".rept 1\n" + "nop\n" + ".endr\n" + ); +} + +__aicore__ static __attribute__((noinline, used, aligned(128), section(".text.pa_icache_thrash"))) +void PA_ICACHE_THRASH_NAME(); + struct CcecOps { static constexpr bool kAtomicReturnReadyObserved = true; @@ -161,6 +185,13 @@ struct PmuSnapshot { uint32_t status = 0; }; +struct IcachePairSnapshot { + PmuSnapshot cold; + PmuSnapshot warm; + uint64_t cold_window_ticks = 0; + uint64_t warm_window_ticks = 0; +}; + template __aicore__ inline uint32_t ReadPmuRegister(uint64_t reg_base) { // 传给 ld_dev 的是重基址后的 __gm__ 指针;相对 offset 均落在编译器允许的 [-2048, 2047]。 @@ -184,10 +215,86 @@ __aicore__ inline PmuSnapshot ReadObservedCounters(uint64_t reg_base) { return sample; } +__aicore__ inline void AccumulateObserved(const PmuSnapshot &sample, PmuSnapshot *total) { + total->total_cycles += sample.total_cycles; + total->scalar_busy += sample.scalar_busy; + total->icache_requests += sample.icache_requests; + total->icache_misses += sample.icache_misses; +} + +// 禁止内联,确保 cold/warm 经过同一个函数体、同一个 PMU gate 和同一个目标 +// callsite;否则 -O3 会复制两份测量代码,capacity sweep 后两份 harness 自身的 +// I-cache 状态不同,cold-warm 就不再是单变量实验。 +__aicore__ static __attribute__((noinline, used, aligned(128), section(".text.pa_icache_harness"))) +void PA_ICACHE_MEASURE_NAME( + uint64_t reg_base, bool warm, PmuSnapshot *total, uint64_t *window_ticks +) { + // 两个 arm 都先做同样的 capacity sweep。warm arm 只多一次窗外目标调用, + // 因而窗内 PMU 与 1 GHz sys counter 的差值对应同一目标行的 hit/miss 差异。 + bisheng::cce::metrics_prof_stop(); + PA_ICACHE_THRASH_NAME(); + if (warm) PA_ICACHE_TARGET_NAME(); + // CNT6/CNT7 在 stop 后仍可能留下窗外取指残余;warm 预热必须发生在本次 + // read-clear 之前,否则这次刻意制造的预热 miss 会被误记到 warm 窗口。 + (void)ReadObservedCounters(reg_base); + + bisheng::cce::metrics_prof_start(); + const uint64_t begin = static_cast(get_sys_cnt()); + PA_ICACHE_TARGET_NAME(); + const uint64_t end = static_cast(get_sys_cnt()); + bisheng::cce::metrics_prof_stop(); + + AccumulateObserved(ReadObservedCounters(reg_base), total); + *window_ticks += end - begin; +} + +// DAV_3510 的 scalar I-cache 最大为 32 KiB;定义刻意放在 target 与 measure +// 之后,使链接布局为 target -> harness -> thrash。窗外顺序执行 64 KiB 指令 +// 覆盖全部 set 多轮,返回较低地址的 harness 时不会向前预取更低地址的 target。 +__aicore__ static void PA_ICACHE_THRASH_NAME() { + asm volatile( + ".rept 16384\n" + "nop\n" + ".endr\n" + ); +} + +__aicore__ inline void RunIcachePhase( + uint64_t reg_base, bool warm, uint32_t trials, PmuSnapshot *total, uint64_t *window_ticks +) { + // 先丢弃一次同分支样本,让条件分支和目标取指预测进入本 phase 的稳定状态; + // 否则逐 trial 交替 warm/cold 会让 false 分支也提前取回目标行。 + PmuSnapshot discarded; + uint64_t discarded_ticks = 0; + PA_ICACHE_MEASURE_NAME(reg_base, warm, &discarded, &discarded_ticks); + for (uint32_t trial = 0; trial < trials; ++trial) { + PA_ICACHE_MEASURE_NAME(reg_base, warm, total, window_ticks); + } +} + +__aicore__ inline IcachePairSnapshot RunSingleIcacheProbe( + uint64_t reg_base, uint32_t trials, uint32_t worker_id +) { + IcachePairSnapshot pair; + // 每个角色内奇偶 worker 各占一半,分别采用 cold-first/warm-first,抵消两个 + // 连续 phase 的固定时间顺序;同一 phase 内保持分支历史稳定。 + if ((worker_id & 1U) == 0) { + RunIcachePhase(reg_base, false, trials, &pair.cold, &pair.cold_window_ticks); + RunIcachePhase(reg_base, true, trials, &pair.warm, &pair.warm_window_ticks); + } else { + RunIcachePhase(reg_base, true, trials, &pair.warm, &pair.warm_window_ticks); + RunIcachePhase(reg_base, false, trials, &pair.cold, &pair.cold_window_ticks); + } + return pair; +} + __aicore__ inline void RunPmuProbe(__gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id) { using namespace pa_scheduler::ccec_pmu; __gm__ pa_scheduler::WorkerResult &result = state->results[worker_id]; PmuSnapshot sample; + PmuSnapshot warm_sample; + uint64_t window_ticks = 0; + uint64_t warm_window_ticks = 0; const WindowMode mode = static_cast(state->config.reserved[kConfigMode]); if (mode != WindowMode::Off) { @@ -218,18 +325,29 @@ __aicore__ inline void RunPmuProbe(__gm__ pa_scheduler::SchedulerState *state, u // A5 的 snapshot 会消费/清除当前累计;stop/start 只负责门控,中间不读取才能续积多段窗口。 bisheng::cce::metrics_prof_stop(); const PmuSnapshot prior = ReadObservedCounters(reg_base); - bisheng::cce::metrics_prof_start(); - if (mode == WindowMode::Scalar || mode == WindowMode::ScalarDouble) { - RuntimeNop(state->config.reserved[kConfigScalarNops]); - } - if (mode == WindowMode::ScalarDouble) { - // 两段相同工作量之间只切 gate、不读取 counter,用于确认 resume 是累计还是重置。 - bisheng::cce::metrics_prof_stop(); + if (mode == WindowMode::IcacheSingle) { + const IcachePairSnapshot pair = + RunSingleIcacheProbe(reg_base, state->config.reserved[kConfigIcacheTrials], worker_id); + sample = pair.cold; + warm_sample = pair.warm; + window_ticks = pair.cold_window_ticks; + warm_window_ticks = pair.warm_window_ticks; + sample.status |= kStatusIcachePairObserved; + } else { bisheng::cce::metrics_prof_start(); - RuntimeNop(state->config.reserved[kConfigScalarNops]); + if (mode == WindowMode::Scalar || mode == WindowMode::ScalarDouble) { + RuntimeNop(state->config.reserved[kConfigScalarNops]); + } + if (mode == WindowMode::ScalarDouble) { + // 两段相同工作量之间只切 gate、不读取 counter,用于确认 resume 是累计还是重置。 + bisheng::cce::metrics_prof_stop(); + bisheng::cce::metrics_prof_start(); + RuntimeNop(state->config.reserved[kConfigScalarNops]); + } + bisheng::cce::metrics_prof_stop(); + sample = ReadObservedCounters(reg_base); } bisheng::cce::metrics_prof_stop(); - sample = ReadObservedCounters(reg_base); sample.status |= kStatusRequested | kStatusRegMapped | kStatusCoreIdValid | (physical_core_id << kStatusCoreIdShift); if (selector2 == kScalarBusyEvent) sample.status |= kStatusCnt2Selector; @@ -244,12 +362,17 @@ __aicore__ inline void RunPmuProbe(__gm__ pa_scheduler::SchedulerState *state, u } } - // 结果位于每核独占 sidecar,五次 bypass store 不参与 Submit 时间口径。 + // 结果位于每核独占 sidecar,这些 bypass store 不参与 Submit 或探针窗口时间口径。 CcecOps::Publish(&result.pmu_total_cycles, sample.total_cycles); CcecOps::Publish(&result.pmu_scalar_busy, sample.scalar_busy); CcecOps::Publish(&result.pmu_icache_requests, sample.icache_requests); CcecOps::Publish(&result.pmu_icache_misses, sample.icache_misses); CcecOps::Publish(&result.pmu_status, sample.status); + CcecOps::Publish(&result.pmu_window_ticks, window_ticks); + CcecOps::Publish(&result.pmu_warm_total_cycles, warm_sample.total_cycles); + CcecOps::Publish(&result.pmu_warm_window_ticks, warm_window_ticks); + CcecOps::Publish(&result.pmu_warm_icache_requests, warm_sample.icache_requests); + CcecOps::Publish(&result.pmu_warm_icache_misses, warm_sample.icache_misses); } } // namespace diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h index a19765b9a0..063fe5df36 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h @@ -16,18 +16,21 @@ namespace pa_scheduler::ccec_pmu { -// 第一阶段只验证观察链路,不把门控下沉到真实 Submit 热路径。 -// Empty 用于量 start/stop 本身的底噪,Scalar 在同一窗口内执行可控 NOP 段。 +// Empty 用于量 start/stop 本身的底噪,Scalar 在同一窗口内执行可控 NOP 段; +// IcacheSingle 在每核上成对累计隔离的 cold/warm 目标调用,用两者差值计算单次 miss 代价。 enum class WindowMode : uint32_t { Off = 0, Empty = 1, Scalar = 2, ScalarDouble = 3, + IcacheSingle = 4, }; // RunConfig::reserved 保持既有 64B ABI;CCEC 独占解释以下五个槽位,其他后端仍看到全零。 constexpr uint32_t kConfigMode = 0; -constexpr uint32_t kConfigScalarNops = 1; +constexpr uint32_t kConfigWorkAmount = 1; +constexpr uint32_t kConfigScalarNops = kConfigWorkAmount; +constexpr uint32_t kConfigIcacheTrials = kConfigWorkAmount; constexpr uint32_t kConfigRegTableLow = 2; constexpr uint32_t kConfigRegTableHigh = 3; constexpr uint32_t kConfigMagic = 4; @@ -66,6 +69,7 @@ constexpr uint32_t kStatusCoreIdValid = 1U << 2; constexpr uint32_t kStatusCnt2Selector = 1U << 3; constexpr uint32_t kStatusCnt6Selector = 1U << 4; constexpr uint32_t kStatusCnt7Selector = 1U << 5; +constexpr uint32_t kStatusIcachePairObserved = 1U << 6; constexpr uint32_t kStatusTotalNonzero = 1U << 7; constexpr uint32_t kStatusPriorSnapshotLarger = 1U << 8; constexpr uint32_t kStatusRequired = kStatusRequested | kStatusRegMapped | kStatusCoreIdValid | diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index d22bdf86a7..dc93a9ce45 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -550,7 +550,7 @@ struct alignas(64) WorkerResult { uint64_t max_occupied; uint64_t final_occupied; - // CCEC 标量 PMU 取证复用 WorkerResult 原有的 24B 尾部 padding,不增加结果区大小。 + // CCEC 标量 PMU 取证使用 WorkerResult 的诊断 sidecar,不改变生产 DistCore ABI。 // 该诊断只在显式开启时有效;CNT2/CNT6/CNT7 分别对应 scalar busy、I-cache req/miss。 uint64_t pmu_total_cycles; uint32_t pmu_scalar_busy; @@ -568,14 +568,23 @@ struct alignas(64) WorkerResult { // 仅在 trace_enabled bit1 开启时递增;每次源码 atomic 调用恰好增加一, // host 用它与 Atomic span 数逐 worker 闭合,禁止把丢记录的泳道当成完整结果。 uint64_t atomic_trace_calls; + + // I-cache 单 miss 探针把 cold 样本放在既有 PMU 字段中,并在扩展的诊断尾部 + // 保存同核配对的 warm 样本与 1 GHz 系统计数器窗口;非该模式均写零。 + uint64_t pmu_window_ticks; + uint64_t pmu_warm_total_cycles; + uint64_t pmu_warm_window_ticks; + uint32_t pmu_warm_icache_requests; + uint32_t pmu_warm_icache_misses; }; // WorkerResult 是 standalone 尾部的诊断 sidecar,不属于真实 DistCore ABI;按 // cache line 隔离后,各 worker 发布统计不会相互覆盖或污染被测共享状态。 -static_assert(sizeof(WorkerResult) == 768, "WorkerResult diagnostics must occupy whole cache lines"); +static_assert(sizeof(WorkerResult) == 832, "WorkerResult diagnostics must occupy whole cache lines"); static_assert(offsetof(WorkerResult, pmu_total_cycles) == 680, "WorkerResult PMU offset mismatch"); static_assert(offsetof(WorkerResult, pmu_status) == 700, "WorkerResult PMU status offset mismatch"); static_assert(offsetof(WorkerResult, fanin_not_ready_loads) == 704, "WorkerResult atomic diagnostic offset mismatch"); static_assert(offsetof(WorkerResult, atomic_trace_calls) == 736, "WorkerResult atomic trace offset mismatch"); +static_assert(offsetof(WorkerResult, pmu_window_ticks) == 744, "WorkerResult PMU timing offset mismatch"); // 从 cube_cursor 到 workers 结束保留关键字段 offset、DistCore ABI 和生产总字节跨度, // 并非字段级完整镜像。RunConfig、输入 context_lens 与校验结果追加在该跨度之后, diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index bba2cdba9c..9b7b2f2e55 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -35,8 +35,9 @@ Benchmark options: --no-swimlane CCEC-only PMU probe options (the host must be launched by msprof PipeUtilization): - --pmu-window off|empty|scalar|scalar-double + --pmu-window off|empty|scalar|scalar-double|icache-single --pmu-scalar-nops N + --pmu-icache-trials N The swimlane action enables atomic tracing by default. For the lower-level run action, --trace-atomics still requires swimlane tracing; add diff --git a/tests/atomic_probe/test_case.md b/tests/atomic_probe/test_case.md index c0fd9b758b..20dbfeeaec 100644 --- a/tests/atomic_probe/test_case.md +++ b/tests/atomic_probe/test_case.md @@ -468,8 +468,32 @@ cd tests/atomic_probe/ccec | `ascendc/mb2_flags_clobber.asc` | gating + observation | AtomicMax flags 无丢失;store+dcci 仅统计 | | `ascendc/mb8_dcci_seam.asc` / `ccec/dcci_seam.cpp` | gating | clean reader 的 DEFAULT/ALL/OUT/ATOMIC/no-DCCI 五模式精确对照 | | `ascendc/dcci_atomic_clobber.asc` / `ccec/dcci_atomic_clobber.cpp` | regression gating + control | 同-line 三 selector 当前明确失败;分-line 与 no-DCCI 五模式精确通过 | +| `pa_scheduler/ccec/kernel.cpp` | calibration | cold/warm 同核配对;每个 cold trial 严格增加一个 CNT7 I-cache miss,建立 scalar 时间标尺 | | `cpu/cpu_atomicity.cpp` | gating + observation | coherent CPU 同/异 cacheline 同构 control、atomic、snapshot、spinlock | +### PA I-cache 单 miss 实测数据 + +2026-07-18 在 device 0、32 AIC + 64 AIV 并发、`msprof PipeUtilization` 下, +`icache-single` 得到以下结果。时间列为多轮 `ns/miss` 中位数,括号内是最小值~最大值: + +| 配置 | 每轮 cold/warm CNT7 miss(ALL) | 严格门禁 | ALL | AIC | AIV | +|---|---:|---:|---:|---:|---:| +| 64 trials/core × 10 | 6,144 / 0(2,048 AIC + 4,096 AIV) | 10/10 PASS | 86.596(86.532~86.792) | 85.913(85.848~86.202) | 86.938(86.861~87.086) | +| 128 trials/core × 5 | 12,288 / 0(4,096 AIC + 8,192 AIV) | 5/5 PASS | 89.629(89.615~89.648) | 92.100(91.984~92.267) | 88.410(88.310~88.440) | + +两组每轮均为 `calibrated_cores=96/96`,并通过 “each cold trial adds exactly +one CNT7 I-cache miss” 断言。AIC/AIV 差值只有数 ns 且方向随运行时段变化, +因此不建立两个伪精确常数。原始日志为 +[`64×10`](pa_scheduler/outputs/pmu_validation/icache_single_64x10_20260718_085929_3232836_console.log) +和 +[`128×5`](pa_scheduler/outputs/pmu_validation/icache_single_128x5_20260718_090151_3235468_console.log)。 + +PA scalar 分析只需要数量级时,使用 `T_icache_est_ns = CNT7_miss_total * 90`;例如 +1,000 个 I-cache miss 约为 90 us。compulsory、capacity、conflict miss 都包含在 +`CNT7_miss_total` 内。该乘积是 cold/warm 校准得到的一阶等效时间,不是逐次精确 +可加的 stall;方法、角色分项和原始日志见 +[`PA调度器独立复现与泳道使用指南.md`](pa_scheduler/PA调度器独立复现与泳道使用指南.md#单次-cnt7-i-cache-miss-的-scalar-估算标尺)。 + ## 判定标准与退出码规则 1. **确定性安全契约必须 gating**:目标值、邻居值、参与核数、执行 marker 全部精确匹配;任一失败返回非零。 From 99971ac19d90daa64606843620007e26a9add151 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 09:49:08 +0000 Subject: [PATCH 014/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E8=A1=A5?= =?UTF-8?q?=E9=BD=90=E7=8B=AC=E7=AB=8BPA=20Submit=E5=85=A8=E7=AA=97PMU?= =?UTF-8?q?=E5=8F=96=E8=AF=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 在每个worker的参数构造前至最后一次Submit返回后门控PMU,采集CNT_TOTAL与CNT0..8并记录实际start/stop状态。 校验owner bitmap、worker映射、物理角色、triplet和计数器风险门槛,按ALL/AIC/AIV导出PMU-only JSON;仅在owner恢复与runtime清理成功后无覆盖发布。 补齐CPU/AscendC空hook与runner后端门禁。 --- .../pa_scheduler/ascendc/pa_scheduler.asc | 6 + tests/atomic_probe/pa_scheduler/ccec/host.cpp | 618 +++++++++++++++++- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 204 ++++-- .../pa_scheduler/ccec/pmu_probe.h | 43 +- .../pa_scheduler/common/pa_model.h | 10 + .../pa_scheduler/common/pa_scheduler_core.h | 7 + tests/atomic_probe/pa_scheduler/cpu/main.cpp | 4 + tests/atomic_probe/pa_scheduler/run.sh | 29 +- 8 files changed, 830 insertions(+), 91 deletions(-) diff --git a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc index 3972af5517..94f09dc875 100644 --- a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc +++ b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc @@ -124,6 +124,12 @@ struct AscendcOps { __aicore__ static inline void Nop(uint32_t count) { RuntimeNop(count); } + // 当前直接 PMU MMIO 观察链路只在 CCEC 分支验证;AscendC 保持公共 hook + // 的空实现,避免静默产出一套未核实寄存器读法。 + __aicore__ static inline bool PmuWindowStart(__gm__ pa_scheduler::SchedulerState *, uint32_t) { return false; } + + __aicore__ static inline void PmuWindowStop(__gm__ pa_scheduler::SchedulerState *, uint32_t, bool) {} + __aicore__ static inline void SpinHint() {} __aicore__ static inline void InvalidateRegion(__gm__ const void *address, uint64_t bytes) { diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index 1f361b1698..243871fdf4 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -17,7 +17,12 @@ #include "driver/ascend_hal.h" #include "runtime/rt.h" +#include +#include + #include +#include +#include #include #include #include @@ -55,6 +60,7 @@ std::vector ReadBinary(const std::string &path) { struct PmuOptions { pa_scheduler::ccec_pmu::WindowMode mode = pa_scheduler::ccec_pmu::WindowMode::Off; uint32_t scalar_nops = 100000; + std::string json_path; }; const char *PmuModeName(pa_scheduler::ccec_pmu::WindowMode mode) { @@ -67,6 +73,8 @@ const char *PmuModeName(pa_scheduler::ccec_pmu::WindowMode mode) { return "scalar"; case pa_scheduler::ccec_pmu::WindowMode::ScalarDouble: return "scalar-double"; + case pa_scheduler::ccec_pmu::WindowMode::SubmitAll: + return "submit-all"; } return "invalid"; } @@ -75,11 +83,12 @@ bool ParsePmuOptions(int argc, char **argv, PmuOptions *pmu, std::vector // PMU 参数只属于 CCEC 验证分支;先摘出再交给三后端共享 parser,避免 CPU/AscendC 静默接受却不生效。 bool mode_seen = false; bool nops_seen = false; + bool json_seen = false; common_argv->clear(); common_argv->push_back(argv[0]); for (int index = 1; index < argc; ++index) { const std::string argument = argv[index]; - if (argument != "--pmu-window" && argument != "--pmu-scalar-nops") { + if (argument != "--pmu-window" && argument != "--pmu-scalar-nops" && argument != "--pmu-json") { common_argv->push_back(argv[index]); continue; } @@ -102,19 +111,31 @@ bool ParsePmuOptions(int argc, char **argv, PmuOptions *pmu, std::vector pmu->mode = pa_scheduler::ccec_pmu::WindowMode::Scalar; } else if (name == "scalar-double") { pmu->mode = pa_scheduler::ccec_pmu::WindowMode::ScalarDouble; + } else if (name == "submit-all") { + pmu->mode = pa_scheduler::ccec_pmu::WindowMode::SubmitAll; } else { std::fprintf( - stderr, "Invalid --pmu-window value: %s (expected off|empty|scalar|scalar-double)\n", value + stderr, + "Invalid --pmu-window value: %s " + "(expected off|empty|scalar|scalar-double|submit-all)\n", + value ); return false; } mode_seen = true; - } else { + } else if (argument == "--pmu-scalar-nops") { if (nops_seen || !pa_scheduler::host::ParseUint(value, 0, 10000000, &pmu->scalar_nops)) { std::fprintf(stderr, "Invalid or duplicate --pmu-scalar-nops value: %s\n", value); return false; } nops_seen = true; + } else { + if (json_seen || *value == '\0') { + std::fprintf(stderr, "Invalid or duplicate --pmu-json path: %s\n", value); + return false; + } + pmu->json_path = value; + json_seen = true; } } if (nops_seen && pmu->mode != pa_scheduler::ccec_pmu::WindowMode::Scalar && @@ -203,40 +224,95 @@ void ConfigurePmu(pa_scheduler::SchedulerState *state, const PmuOptions &pmu, co struct PmuAggregate { std::vector total_cycles; - uint64_t scalar_busy = 0; - uint64_t icache_requests = 0; - uint64_t icache_misses = 0; + std::vector vector_busy; + std::vector cube_busy; + std::vector scalar_busy; + std::vector mte1_busy; + std::vector mte2_busy; + std::vector mte3_busy; + std::vector icache_requests; + std::vector icache_misses; + std::vector fix_busy; + uint32_t trusted = 0; }; void AddPmuSample(const pa_scheduler::WorkerResult &result, PmuAggregate *aggregate) { aggregate->total_cycles.push_back(result.pmu_total_cycles); - aggregate->scalar_busy += result.pmu_scalar_busy; - aggregate->icache_requests += result.pmu_icache_requests; - aggregate->icache_misses += result.pmu_icache_misses; + aggregate->vector_busy.push_back(result.pmu_vector_busy); + aggregate->cube_busy.push_back(result.pmu_cube_busy); + aggregate->scalar_busy.push_back(result.pmu_scalar_busy); + aggregate->mte1_busy.push_back(result.pmu_mte1_busy); + aggregate->mte2_busy.push_back(result.pmu_mte2_busy); + aggregate->mte3_busy.push_back(result.pmu_mte3_busy); + aggregate->icache_requests.push_back(result.pmu_icache_requests); + aggregate->icache_misses.push_back(result.pmu_icache_misses); + aggregate->fix_busy.push_back(result.pmu_fix_busy); + aggregate->trusted += + (result.pmu_status & pa_scheduler::ccec_pmu::kStatusRequired) == pa_scheduler::ccec_pmu::kStatusRequired; } void PrintPmuAggregate(const char *name, const PmuAggregate &aggregate) { const pa_scheduler::host::Uint64Distribution total = pa_scheduler::host::SummarizeUint64(aggregate.total_cycles); - const double miss_rate = aggregate.icache_requests == 0 - ? 0.0 - : 100.0 * aggregate.icache_misses / aggregate.icache_requests; + const pa_scheduler::host::Uint64Distribution scalar = + pa_scheduler::host::SummarizeUint64(aggregate.scalar_busy); + const pa_scheduler::host::Uint64Distribution vector = + pa_scheduler::host::SummarizeUint64(aggregate.vector_busy); + const pa_scheduler::host::Uint64Distribution cube = + pa_scheduler::host::SummarizeUint64(aggregate.cube_busy); + const pa_scheduler::host::Uint64Distribution mte1 = + pa_scheduler::host::SummarizeUint64(aggregate.mte1_busy); + const pa_scheduler::host::Uint64Distribution mte2 = + pa_scheduler::host::SummarizeUint64(aggregate.mte2_busy); + const pa_scheduler::host::Uint64Distribution mte3 = + pa_scheduler::host::SummarizeUint64(aggregate.mte3_busy); + const pa_scheduler::host::Uint64Distribution requests = + pa_scheduler::host::SummarizeUint64(aggregate.icache_requests); + const pa_scheduler::host::Uint64Distribution misses = + pa_scheduler::host::SummarizeUint64(aggregate.icache_misses); + const double miss_rate = requests.total == 0 ? 0.0 : 100.0 * misses.total / requests.total; std::printf( - "[PMU-%s] cores=%zu total_sum=%llu total_median=%.1f total_p95=%llu scalar_busy=%llu " - "icache_req=%llu icache_miss=%llu miss_rate=%.4f%%\n", + "[PMU-%s] cores=%zu total_sum=%llu total_median=%.1f total_p95=%llu " + "scalar_busy=%llu vector_busy=%llu cube_busy=%llu mte1_busy=%llu mte2_busy=%llu " + "mte3_busy=%llu icache_req=%llu icache_miss=%llu miss_rate=%.4f%%\n", name, aggregate.total_cycles.size(), static_cast(total.total), total.median, - static_cast(total.p95), static_cast(aggregate.scalar_busy), - static_cast(aggregate.icache_requests), - static_cast(aggregate.icache_misses), miss_rate + static_cast(total.p95), static_cast(scalar.total), + static_cast(vector.total), static_cast(cube.total), + static_cast(mte1.total), static_cast(mte2.total), + static_cast(mte3.total), + static_cast(requests.total), static_cast(misses.total), miss_rate ); } +struct PmuValidation { + uint32_t trusted = 0; + uint32_t unique_physical_core_ids = 0; + uint32_t owner_bitmap_members = 0; + uint32_t exact_worker_slots = 0; + uint32_t physical_role_matches = 0; + uint32_t mixed_triplet_matches = 0; + uint32_t window_started = 0; + uint32_t window_stopped = 0; + uint32_t prior_snapshot_larger = 0; + uint32_t maximum_programmable_counter = 0; + bool icache_order_valid = true; + bool counter_below_risk_threshold = true; + bool passed = true; +}; + +// 32-bit programmable counter 无法仅凭终值证明从未回卷。正式文件采用 25% +// 高水位作为保守拒绝阈值;它只降低风险,不把“未越线”表述成回卷证明。 +constexpr uint32_t kProgrammableCounterRiskThreshold = UINT32_MAX / 4U; + bool ValidatePmu( const pa_scheduler::SchedulerState &state, uint32_t run, const PmuOptions &pmu, - const pa_scheduler::pmu_owner::PmuOwnerControl *owner + const pa_scheduler::pmu_owner::PmuOwnerControl *owner, PmuValidation *validation ) { using namespace pa_scheduler::ccec_pmu; - if (pmu.mode == WindowMode::Off) return true; + if (pmu.mode == WindowMode::Off) { + *validation = PmuValidation{}; + return true; + } bool seen[kPhysicalSubcoreCount] = {}; uint32_t trusted = 0; @@ -244,7 +320,11 @@ bool ValidatePmu( uint32_t owner_members = 0; uint32_t exact_worker_slots = 0; uint32_t physical_role_matches = 0; + uint32_t window_started = 0; + uint32_t window_stopped = 0; uint32_t prior_larger = 0; + uint32_t maximum_programmable_counter = 0; + bool icache_order_valid = true; uint32_t bad_printed = 0; PmuAggregate all; PmuAggregate aic; @@ -260,7 +340,18 @@ bool ValidatePmu( owner_members += owner != nullptr && pa_scheduler::pmu_owner::IsConfigured(*owner, core_id); exact_worker_slots += result.worker_id == worker; physical_role_matches += logical_aic == physical_aic; + window_started += (status & kStatusWindowStarted) != 0U; + window_stopped += (status & kStatusWindowStopped) != 0U; prior_larger += (status & kStatusPriorSnapshotLarger) != 0; + icache_order_valid &= result.pmu_icache_misses <= result.pmu_icache_requests; + const uint32_t programmable[] = { + result.pmu_vector_busy, result.pmu_cube_busy, result.pmu_scalar_busy, + result.pmu_mte1_busy, result.pmu_mte2_busy, result.pmu_mte3_busy, + result.pmu_icache_requests, result.pmu_icache_misses, result.pmu_fix_busy, + }; + for (uint32_t value : programmable) { + maximum_programmable_counter = std::max(maximum_programmable_counter, value); + } if (core_id < kPhysicalSubcoreCount && !seen[core_id]) { seen[core_id] = true; ++unique; @@ -286,13 +377,14 @@ bool ValidatePmu( pa_scheduler::pmu_owner::kSubcoresPerDie; const uint32_t local = aic_id % pa_scheduler::pmu_owner::kSubcoresPerDie; const uint32_t expected_aiv0 = die_base + pa_scheduler::pmu_owner::kAicPerDie + local * 2U; + const uint32_t expected_aiv1 = expected_aiv0 + 1U; const uint32_t aiv0_id = StatusCoreId( state.results[pa_scheduler::kAicWorkers + block * 2U].pmu_status ); const uint32_t aiv1_id = StatusCoreId( state.results[pa_scheduler::kAicWorkers + block * 2U + 1U].pmu_status ); - mixed_triplet_matches += aiv0_id == expected_aiv0 && aiv1_id == expected_aiv0 + 1U; + mixed_triplet_matches += aiv0_id == expected_aiv0 && aiv1_id == expected_aiv1; } PrintPmuAggregate("ALL", all); @@ -304,10 +396,16 @@ bool ValidatePmu( const bool worker_slots_ok = exact_worker_slots == pa_scheduler::kWorkers; const bool physical_roles_ok = physical_role_matches == pa_scheduler::kWorkers; const bool mixed_triplets_ok = mixed_triplet_matches == pa_scheduler::kAicWorkers; + const bool windows_started_ok = window_started == pa_scheduler::kWorkers; + const bool windows_stopped_ok = window_stopped == pa_scheduler::kWorkers; + const bool counter_below_risk_threshold = + maximum_programmable_counter < kProgrammableCounterRiskThreshold; std::printf( - "[PMU] run=%u window=%s scalar_nops=%u trusted=%u/%u unique_coreids=%u/%u prior_larger=%u/%u\n", run, + "[PMU] run=%u window=%s calibration_scalar_nops=%u trusted=%u/%u unique_coreids=%u/%u " + "prior_larger=%u/%u programmable_max=%u headroom=%u\n", run, PmuModeName(pmu.mode), pmu.scalar_nops, trusted, pa_scheduler::kWorkers, unique, pa_scheduler::kWorkers, - prior_larger, pa_scheduler::kWorkers + prior_larger, pa_scheduler::kWorkers, maximum_programmable_counter, + UINT32_MAX - maximum_programmable_counter ); std::printf("[ASSERT] %-48s %s\n", "all PMU records have configured selectors and data", records_ok ? "PASS" : "FAIL"); @@ -321,8 +419,397 @@ bool ValidatePmu( physical_roles_ok ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "all 32 mixed blocks map to physical 1:2 triplets", mixed_triplets_ok ? "PASS" : "FAIL"); - return records_ok && core_ids_ok && owner_members_ok && worker_slots_ok && - physical_roles_ok && mixed_triplets_ok; + std::printf("[ASSERT] %-48s %s\n", "all 96 PMU windows executed start", + windows_started_ok ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "all 96 started PMU windows executed stop", + windows_stopped_ok ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "I-cache misses do not exceed requests", + icache_order_valid ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "programmable counters stay below 25% risk threshold", + counter_below_risk_threshold ? "PASS" : "FAIL"); + validation->trusted = trusted; + validation->unique_physical_core_ids = unique; + validation->owner_bitmap_members = owner_members; + validation->exact_worker_slots = exact_worker_slots; + validation->physical_role_matches = physical_role_matches; + validation->mixed_triplet_matches = mixed_triplet_matches; + validation->window_started = window_started; + validation->window_stopped = window_stopped; + validation->prior_snapshot_larger = prior_larger; + validation->maximum_programmable_counter = maximum_programmable_counter; + validation->icache_order_valid = icache_order_valid; + validation->counter_below_risk_threshold = counter_below_risk_threshold; + validation->passed = records_ok && core_ids_ok && owner_members_ok && worker_slots_ok && + physical_roles_ok && mixed_triplets_ok && windows_started_ok && windows_stopped_ok && + icache_order_valid && counter_below_risk_threshold; + return validation->passed; +} + +void WriteJsonString(std::FILE *output, const std::string &value) { + std::fputc('"', output); + for (unsigned char character : value) { + switch (character) { + case '"': + std::fputs("\\\"", output); + break; + case '\\': + std::fputs("\\\\", output); + break; + case '\b': + std::fputs("\\b", output); + break; + case '\f': + std::fputs("\\f", output); + break; + case '\n': + std::fputs("\\n", output); + break; + case '\r': + std::fputs("\\r", output); + break; + case '\t': + std::fputs("\\t", output); + break; + default: + if (character < 0x20U) { + std::fprintf(output, "\\u%04x", static_cast(character)); + } else { + std::fputc(character, output); + } + } + } + std::fputc('"', output); +} + +void WriteMetricDistribution(std::FILE *output, const std::vector &values) { + const pa_scheduler::host::Uint64Distribution summary = pa_scheduler::host::SummarizeUint64(values); + const double mean = values.empty() ? 0.0 : static_cast(summary.total) / values.size(); + std::fprintf( + output, "{\"sum\":%llu,\"mean\":%.17g,\"median\":%.17g,\"p95\":%llu,\"max\":%llu}", + static_cast(summary.total), mean, summary.median, + static_cast(summary.p95), static_cast(summary.maximum) + ); +} + +void WritePmuAggregateJson(std::FILE *output, const PmuAggregate &aggregate) { + const pa_scheduler::host::Uint64Distribution requests = + pa_scheduler::host::SummarizeUint64(aggregate.icache_requests); + const pa_scheduler::host::Uint64Distribution misses = + pa_scheduler::host::SummarizeUint64(aggregate.icache_misses); + uint32_t active_cores = 0; + for (uint64_t cycles : aggregate.total_cycles) active_cores += cycles != 0; + std::fprintf( + output, "{\"cores\":%zu,\"active_cores\":%u,\"trusted_cores\":%u,\"total_cycles\":", + aggregate.total_cycles.size(), active_cores, aggregate.trusted + ); + WriteMetricDistribution(output, aggregate.total_cycles); + std::fputs(",\"vector_busy\":", output); + WriteMetricDistribution(output, aggregate.vector_busy); + std::fputs(",\"cube_busy\":", output); + WriteMetricDistribution(output, aggregate.cube_busy); + std::fputs(",\"scalar_busy\":", output); + WriteMetricDistribution(output, aggregate.scalar_busy); + std::fputs(",\"mte1_busy\":", output); + WriteMetricDistribution(output, aggregate.mte1_busy); + std::fputs(",\"mte2_busy\":", output); + WriteMetricDistribution(output, aggregate.mte2_busy); + std::fputs(",\"mte3_busy\":", output); + WriteMetricDistribution(output, aggregate.mte3_busy); + std::fputs(",\"icache_requests\":", output); + WriteMetricDistribution(output, aggregate.icache_requests); + std::fputs(",\"icache_misses\":", output); + WriteMetricDistribution(output, aggregate.icache_misses); + std::fputs(",\"fix_busy\":", output); + WriteMetricDistribution(output, aggregate.fix_busy); + std::fputs(",\"icache_miss_rate\":", output); + if (requests.total == 0) { + std::fputs("null", output); + } else { + // 全局 miss rate 必须以总 miss/总 request 计算,不能平均逐核百分比。 + std::fprintf(output, "%.17g", static_cast(misses.total) / requests.total); + } + std::fputc('}', output); +} + +uint32_t PmuWindowSegments(pa_scheduler::ccec_pmu::WindowMode mode) { + if (mode == pa_scheduler::ccec_pmu::WindowMode::ScalarDouble) return 2U; + return 1U; +} + +uint32_t CountConfiguredMixedTriplets(const pa_scheduler::pmu_owner::PmuOwnerControl &owner) { + uint32_t complete = 0U; + for (uint32_t die_base = 0U; + die_base < pa_scheduler::pmu_owner::kPhysicalSubcoreCount; + die_base += pa_scheduler::pmu_owner::kSubcoresPerDie) { + for (uint32_t local = 0U; local < pa_scheduler::pmu_owner::kAicPerDie; ++local) { + const uint32_t aic = die_base + local; + const uint32_t aiv0 = die_base + pa_scheduler::pmu_owner::kAicPerDie + local * 2U; + const uint32_t aiv1 = aiv0 + 1U; + complete += pa_scheduler::pmu_owner::IsConfigured(owner, aic) && + pa_scheduler::pmu_owner::IsConfigured(owner, aiv0) && + pa_scheduler::pmu_owner::IsConfigured(owner, aiv1); + } + } + return complete; +} + +bool ExportPmuJson( + const pa_scheduler::SchedulerState &state, const pa_scheduler::host::Options &options, + const PmuOptions &pmu, uint32_t run, double host_us, double submit_span_us, + const PmuValidation &validation, bool semantic_passed, + const pa_scheduler::pmu_owner::PmuOwnerControl &owner, bool restore_passed, + const std::string &output_path +) { + using namespace pa_scheduler::ccec_pmu; + PmuAggregate all; + PmuAggregate aic; + PmuAggregate aiv; + for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { + const pa_scheduler::WorkerResult &result = state.results[worker]; + AddPmuSample(result, &all); + AddPmuSample(result, result.role == static_cast(pa_scheduler::CoreRole::Aic) ? &aic : &aiv); + } + + const auto generated = std::chrono::system_clock::now().time_since_epoch(); + const uint64_t generated_ns = static_cast( + std::chrono::duration_cast(generated).count() + ); + const std::string capture_id = "pa-pmu-" + std::to_string(generated_ns) + "-run" + std::to_string(run); + const std::string temporary_path = output_path + ".tmp"; + // 临时文件与最终文件都采用 no-replace 语义:并发采集不能截断同名 tmp, + // 也不能在最终发布时覆盖另一份已经完成的证据文件。 + const int output_fd = open(temporary_path.c_str(), O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC, 0644); + std::FILE *output = output_fd < 0 ? nullptr : fdopen(output_fd, "wb"); + if (output == nullptr) { + const int open_error = errno; + if (output_fd >= 0) { + (void)close(output_fd); + (void)std::remove(temporary_path.c_str()); + } + std::fprintf( + stderr, "Cannot exclusively create PMU JSON output %s: %s\n", temporary_path.c_str(), + std::strerror(open_error) + ); + return false; + } + std::vector output_buffer(1U << 20); + std::setvbuf(output, output_buffer.data(), _IOFBF, output_buffer.size()); + + const bool submit_window = IsSubmitWindow(pmu.mode); + const bool simulated_task_nops_nonzero = + options.nops.qk != 0U || options.nops.sf != 0U || options.nops.pv != 0U || options.nops.up != 0U; + const uint32_t owner_bitmap_count = pa_scheduler::pmu_owner::CountConfigured(owner); + const uint32_t owner_complete_triplets = CountConfiguredMixedTriplets(owner); + std::fputs("{\n\"schema\":{\"name\":\"pa_scheduler_pmu_phase_windows\",\"version\":2},\n", output); + std::fputs("\"capture\":{\"capture_id\":", output); + WriteJsonString(output, capture_id); + std::fprintf( + output, + ",\"generated_unix_time_ns\":%llu,\"run_index\":%u,\"accepted\":true," + "\"usable_for_same_configuration_submit_comparison\":%s," + "\"usable_as_absolute_real_pa_profile\":false,\"window_scope\":\"%s\"," + "\"pmu_probe_position\":\"%s\",\"scheduler_hot_path_included\":%s," + "\"total_sum_is_core_work_not_wall_time\":true," + "\"published_after_runtime_cleanup\":true,\"runtime_cleanup_passed\":true," + "\"owner_restore_passed\":%s},\n", + static_cast(generated_ns), run, + submit_window ? "true" : "false", + submit_window ? "per_worker_orchestration_to_last_submit_return" : "post_scheduler_calibration_probe", + submit_window ? "inside_RunScheduler" : "after_RunScheduler", + submit_window ? "true" : "false", restore_passed ? "true" : "false" + ); + std::fputs("\"configuration\":{\"kernel_path\":", output); + WriteJsonString(output, options.kernel_path); + std::fprintf( + output, + ",\"device\":%u,\"batches\":%u,\"workers\":%u,\"aic_workers\":%u,\"aiv_workers\":%u," + "\"trace_enabled\":%s,\"trace_atomics\":%s,\"profile_phases\":%s," + "\"nop_counts\":{\"qk\":%u,\"sf\":%u," + "\"pv\":%u,\"up\":%u},\"pmu_window\":", + options.device, options.batches, pa_scheduler::kWorkers, pa_scheduler::kAicWorkers, + pa_scheduler::kAivWorkers, options.trace_enabled ? "true" : "false", + options.trace_atomics ? "true" : "false", + options.profile_phases ? "true" : "false", options.nops.qk, options.nops.sf, options.nops.pv, + options.nops.up + ); + WriteJsonString(output, PmuModeName(pmu.mode)); + std::fputs(",\"calibration_scalar_nops_per_segment\":", output); + if (submit_window) { + std::fputs("null", output); + } else { + std::fprintf(output, "%u", pmu.scalar_nops); + } + std::fprintf( + output, + ",\"window_segments_are_mode_contract_per_record\":true," + "\"host_launch_to_sync_us\":%.17g,\"submit_span_us\":%.17g," + "\"selectors\":{\"cnt0_vector_busy\":%u,\"cnt1_cube_busy\":%u," + "\"cnt2_scalar_busy\":%u,\"cnt3_mte1_busy\":%u,\"cnt4_mte2_busy\":%u," + "\"cnt5_mte3_busy\":%u,\"cnt6_icache_request\":%u,\"cnt7_icache_miss\":%u," + "\"cnt8_fix_busy\":%u},\"counter_width_bits\":{\"total\":64,\"programmable\":32}," + "\"counter_wrap_not_directly_detectable\":true,\"counter_wrap_absence_proven\":false," + "\"programmable_counter_risk_threshold\":%u," + "\"gate_start_stop_have_pipe_all_barriers\":true," + "\"phase_timestamp_calls_present\":true,\"phase_record_writes\":false," + "\"atomic_trace\":false,\"profile_accumulation\":false," + "\"simulated_task_nop_mechanism_executes_on_scalar\":true," + "\"simulated_task_nops_nonzero\":%s," + "\"icache_miss_rate_definition\":\"sum(icache_misses)/sum(icache_requests)\"},\n", + host_us, submit_span_us, kVectorBusyEvent, kCubeBusyEvent, kScalarBusyEvent, + kMte1BusyEvent, kMte2BusyEvent, kMte3BusyEvent, kIcacheRequestEvent, kIcacheMissEvent, + kFixBusyEvent, kProgrammableCounterRiskThreshold, + simulated_task_nops_nonzero ? "true" : "false" + ); + std::fprintf( + output, + "\"validation\":{\"semantic_passed\":%s,\"pmu_passed\":%s,\"trusted_records\":%u," + "\"expected_records\":%u,\"unique_physical_core_ids\":%u,\"expected_unique_core_ids\":%u," + "\"owner_bitmap_member_records\":%u,\"expected_owner_bitmap_member_records\":%u," + "\"exact_worker_slot_records\":%u,\"expected_exact_worker_slot_records\":%u," + "\"physical_role_match_records\":%u,\"expected_physical_role_match_records\":%u," + "\"mixed_triplet_matches\":%u,\"expected_mixed_triplet_matches\":%u," + "\"window_started_records\":%u,\"window_stopped_records\":%u," + "\"expected_window_records\":%u,\"prior_snapshot_larger_records\":%u," + "\"icache_miss_le_request\":%s,\"counter_below_risk_threshold\":%s," + "\"maximum_programmable_counter\":%u,\"programmable_counter_risk_threshold\":%u," + "\"programmable_counter_headroom\":%u},\n", + semantic_passed ? "true" : "false", validation.passed ? "true" : "false", validation.trusted, + pa_scheduler::kWorkers, validation.unique_physical_core_ids, pa_scheduler::kWorkers, + validation.owner_bitmap_members, pa_scheduler::kWorkers, + validation.exact_worker_slots, pa_scheduler::kWorkers, + validation.physical_role_matches, pa_scheduler::kWorkers, + validation.mixed_triplet_matches, pa_scheduler::kAicWorkers, + validation.window_started, validation.window_stopped, pa_scheduler::kWorkers, + validation.prior_snapshot_larger, validation.icache_order_valid ? "true" : "false", + validation.counter_below_risk_threshold ? "true" : "false", + validation.maximum_programmable_counter, kProgrammableCounterRiskThreshold, + UINT32_MAX - validation.maximum_programmable_counter + ); + std::fprintf( + output, + "\"owner\":{\"mode\":\"main_aicpu_path_a\"," + "\"snapshot_phase\":\"after_configure_before_restore\"," + "\"control_magic\":%u,\"control_version\":%u,\"configure_status\":%d," + "\"configured_flag\":%u,\"configured_bitmap_count\":%u," + "\"expected\":{\"total\":%u,\"aic\":%u,\"aiv\":%u}," + "\"active\":{\"total\":%u,\"aic\":%u,\"aiv\":%u}," + "\"discovered\":{\"total\":%u,\"aic\":%u,\"aiv\":%u}," + "\"physical_slots_scanned\":%u,\"skipped_physical_slots\":%u," + "\"configured_bitmap_word_order\":\"least_significant_physical_ids_first\"," + "\"configured_bitmap_words\":[%u,%u,%u,%u]," + "\"configured_complete_mixed_triplets\":%u,\"expected_complete_mixed_triplets\":%u," + "\"configured_broken_mixed_triplets\":%u,\"restore_passed\":%s},\n", + owner.magic, owner.version, static_cast(owner.status), owner.configured, owner_bitmap_count, + owner.expected_total, owner.expected_aic, owner.expected_aiv, + owner.active_total, owner.active_aic, owner.active_aiv, + owner.discovered_total, owner.discovered_aic, owner.discovered_aiv, + pa_scheduler::pmu_owner::kPhysicalSubcoreCount, owner.skipped_total, + owner.configured_bitmap[0], owner.configured_bitmap[1], + owner.configured_bitmap[2], owner.configured_bitmap[3], + owner_complete_triplets, pa_scheduler::kAicWorkers, + owner_bitmap_count / 3U - owner_complete_triplets, restore_passed ? "true" : "false" + ); + std::fputs("\"records\":[\n", output); + for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { + const pa_scheduler::WorkerResult &result = state.results[worker]; + const uint32_t status = result.pmu_status; + const uint32_t physical_core_id = StatusCoreId(status); + const bool trusted = (status & kStatusRequired) == kStatusRequired; + const bool is_aic = result.role == static_cast(pa_scheduler::CoreRole::Aic); + const uint32_t vector_id = is_aic ? 0U : worker - pa_scheduler::kAicWorkers; + const uint32_t block_id = is_aic ? worker : vector_id / 2U; + const uint32_t lane = is_aic ? 0U : 1U + vector_id % 2U; + const uint32_t segments = PmuWindowSegments(pmu.mode); + const bool owner_bitmap_member = pa_scheduler::pmu_owner::IsConfigured(owner, physical_core_id); + const bool worker_slot_exact = result.worker_id == worker; + const bool physical_role_matches = + is_aic == pa_scheduler::pmu_owner::IsAicPhysicalSlot(physical_core_id); + const bool window_started = (status & kStatusWindowStarted) != 0U; + const bool window_stopped = (status & kStatusWindowStopped) != 0U; + std::fprintf( + output, + "%s{\"worker_id\":%u,\"physical_core_id\":%u,\"role\":\"%s\",\"block_id\":%u," + "\"lane\":%u,\"window_segments\":%u,\"window_segment_count_source\":\"mode_contract\"," + "\"window_started\":%s,\"window_stopped\":%s,\"total_cycles\":%llu,\"vector_busy\":%u," + "\"cube_busy\":%u,\"scalar_busy\":%u,\"mte1_busy\":%u,\"mte2_busy\":%u," + "\"mte3_busy\":%u,\"icache_requests\":%u,\"icache_misses\":%u,\"fix_busy\":%u," + "\"status\":%u,\"status_hex\":" + "\"0x%08x\",\"trusted\":%s,\"physical_core_id_valid\":%s,\"selectors_match\":%s," + "\"owner_bitmap_member\":%s,\"worker_slot_exact\":%s," + "\"physical_role_matches\":%s,\"prior_snapshot_larger\":%s}", + worker == 0 ? "" : ",\n", worker, physical_core_id, is_aic ? "aic" : "aiv", block_id, + lane, segments, window_started ? "true" : "false", window_stopped ? "true" : "false", + static_cast(result.pmu_total_cycles), result.pmu_vector_busy, + result.pmu_cube_busy, result.pmu_scalar_busy, result.pmu_mte1_busy, result.pmu_mte2_busy, + result.pmu_mte3_busy, result.pmu_icache_requests, result.pmu_icache_misses, + result.pmu_fix_busy, status, status, trusted ? "true" : "false", + (status & kStatusCoreIdValid) != 0 ? "true" : "false", + (status & (kStatusCnt0Selector | kStatusCnt1Selector | kStatusCnt2Selector | + kStatusCnt3Selector | kStatusCnt4Selector | kStatusCnt5Selector | + kStatusCnt6Selector | kStatusCnt7Selector | kStatusCnt8Selector)) == + (kStatusCnt0Selector | kStatusCnt1Selector | kStatusCnt2Selector | + kStatusCnt3Selector | kStatusCnt4Selector | kStatusCnt5Selector | + kStatusCnt6Selector | kStatusCnt7Selector | kStatusCnt8Selector) + ? "true" + : "false", + owner_bitmap_member ? "true" : "false", worker_slot_exact ? "true" : "false", + physical_role_matches ? "true" : "false", + (status & kStatusPriorSnapshotLarger) != 0 ? "true" : "false" + ); + } + std::fputs("\n],\n\"summary\":{\"all\":", output); + WritePmuAggregateJson(output, all); + std::fputs(",\"aic\":", output); + WritePmuAggregateJson(output, aic); + std::fputs(",\"aiv\":", output); + WritePmuAggregateJson(output, aiv); + std::fputs("}\n}\n", output); + + bool success = std::ferror(output) == 0; + int write_error = success ? 0 : EIO; + if (std::fflush(output) != 0) { + success = false; + write_error = errno; + } + if (success && fsync(fileno(output)) != 0) { + success = false; + write_error = errno; + } + if (std::fclose(output) != 0) { + success = false; + write_error = errno; + } + if (!success) { + std::fprintf(stderr, "Failed while writing PMU JSON output %s: %s\n", temporary_path.c_str(), + std::strerror(write_error)); + (void)std::remove(temporary_path.c_str()); + return false; + } + // 同目录 hard-link 在最终名称不存在时原子发布;EEXIST 时保留既有证据, + // 不采用会替换目标的 POSIX rename。 + if (link(temporary_path.c_str(), output_path.c_str()) != 0) { + std::fprintf( + stderr, "Cannot publish PMU JSON without replacement %s -> %s: %s\n", + temporary_path.c_str(), output_path.c_str(), + std::strerror(errno) + ); + (void)std::remove(temporary_path.c_str()); + return false; + } + if (unlink(temporary_path.c_str()) != 0) { + const int unlink_error = errno; + // 最终文件已链接但事务尚未完成;尽力撤回最终名称,避免失败返回时留下 + // 一份被调用方误认为成功发布的文件。 + (void)unlink(output_path.c_str()); + std::fprintf( + stderr, "Cannot remove PMU JSON temporary link %s: %s\n", temporary_path.c_str(), + std::strerror(unlink_error) + ); + return false; + } + std::printf("[PMU-JSON] capture_id=%s records=%u output=%s\n", capture_id.c_str(), pa_scheduler::kWorkers, + output_path.c_str()); + return true; } } // namespace @@ -340,11 +827,42 @@ int main(int argc, char **argv) { if (parse_status == pa_scheduler::host::ParseStatus::Help) { std::fprintf( stderr, - "CCEC PMU options: [--pmu-window off|empty|scalar|scalar-double] [--pmu-scalar-nops N]\n" + "CCEC PMU options: [--pmu-window " + "off|empty|scalar|scalar-double|submit-all] [--pmu-scalar-nops N] " + "[--pmu-json FILE]\n" ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } + if (!pmu_options.json_path.empty() && + pmu_options.mode == pa_scheduler::ccec_pmu::WindowMode::Off) { + std::fprintf(stderr, "--pmu-json requires a non-off --pmu-window.\n"); + return EXIT_FAILURE; + } + if (!pmu_options.json_path.empty() && options.runs != 1) { + // 一个 sidecar 对应一次采集,禁止多轮覆写后丢失逐轮边界。 + std::fprintf(stderr, "--pmu-json requires --runs 1 to avoid overwriting captures.\n"); + return EXIT_FAILURE; + } + if (!pmu_options.json_path.empty() && + (options.trace_enabled || options.trace_atomics || options.profile_phases || + options.analyze_swimlane || !options.swimlane_json.empty())) { + std::fprintf( + stderr, + "--pmu-json requires PMU-only collection: add --no-swimlane and do not enable " + "phase profiling, atomic tracing, swimlane analysis, or swimlane JSON.\n" + ); + return EXIT_FAILURE; + } + if (!pmu_options.json_path.empty() && + (access(pmu_options.json_path.c_str(), F_OK) == 0 || + access((pmu_options.json_path + ".tmp").c_str(), F_OK) == 0)) { + std::fprintf( + stderr, "Refusing to overwrite an existing PMU JSON or temporary file: %s\n", + pmu_options.json_path.c_str() + ); + return EXIT_FAILURE; + } const std::vector binary_data = ReadBinary(options.kernel_path); if (binary_data.empty()) { std::fprintf(stderr, "Cannot read kernel binary: %s\n", options.kernel_path.c_str()); @@ -352,7 +870,8 @@ int main(int argc, char **argv) { } pa_scheduler::host::PrintBanner("CCEC", options); std::printf( - "[PMU-CONFIG] window=%s scalar_nops=%u source=direct-per-core owner=main-aicpu-path-a\n", + "[PMU-CONFIG] window=%s calibration_scalar_nops=%u source=direct-per-core " + "owner=main-aicpu-path-a\n", PmuModeName(pmu_options.mode), pmu_options.scalar_nops ); @@ -392,6 +911,8 @@ int main(int argc, char **argv) { PmuRegisterMappings pmu_mappings; pa_scheduler::pmu_owner::PmuOwnerSession pmu_owner; + pa_scheduler::pmu_owner::PmuOwnerControl pmu_owner_evidence{}; + bool pmu_owner_evidence_valid = false; const void *pmu_registers_device = nullptr; if (pmu_options.mode != pa_scheduler::ccec_pmu::WindowMode::Off) { if (!MapPmuRegisters(options.device, &pmu_mappings)) return EXIT_FAILURE; @@ -409,6 +930,8 @@ int main(int argc, char **argv) { (void)UnmapPmuRegisters(options.device, &pmu_mappings); return EXIT_FAILURE; } + pmu_owner_evidence = pmu_owner.Control(); + pmu_owner_evidence_valid = true; pmu_registers_device = reinterpret_cast(pmu_owner.RegisterTableDeviceAddress()); } @@ -433,6 +956,12 @@ int main(int argc, char **argv) { std::vector spans; bool all_passed = true; bool postprocess_ok = true; + bool pmu_json_ready = false; + bool pmu_json_semantic_passed = false; + uint32_t pmu_json_run = 0U; + double pmu_json_host_us = 0.0; + double pmu_json_submit_span_us = 0.0; + PmuValidation pmu_json_validation; for (uint32_t run = 1; run <= options.runs; ++run) { pa_scheduler::host::InitializeState(state.get(), options); pa_scheduler::host::ConfigureTrace(state.get(), options, trace_device); @@ -540,11 +1069,27 @@ int main(int argc, char **argv) { *state, run, host_us, options.trace_enabled ? &trace_header : nullptr ); all_passed &= metrics.passed; - all_passed &= ValidatePmu( + PmuValidation pmu_validation; + const bool pmu_passed = ValidatePmu( *state, run, pmu_options, - pmu_options.mode == pa_scheduler::ccec_pmu::WindowMode::Off ? nullptr : &pmu_owner.Control() + pmu_options.mode == pa_scheduler::ccec_pmu::WindowMode::Off ? nullptr : &pmu_owner.Control(), + &pmu_validation ); + all_passed &= pmu_passed; spans.push_back(metrics.submit_span_us); + if (!pmu_options.json_path.empty()) { + if (!metrics.passed || !pmu_passed || !pmu_owner_evidence_valid) { + std::fprintf(stderr, "PMU JSON rejected because semantic, PMU, or owner validation failed.\n"); + postprocess_ok = false; + break; + } + pmu_json_ready = true; + pmu_json_semantic_passed = metrics.passed; + pmu_json_run = run; + pmu_json_host_us = host_us; + pmu_json_submit_span_us = metrics.submit_span_us; + pmu_json_validation = pmu_validation; + } if (options.analyze_swimlane && !pa_scheduler::host::AnalyzeSwimlaneRecords(trace_header, *state, read_trace_records)) { // 后处理错误使用 break 汇入统一 cleanup;与初始化/launch 失败的进程级立即返回语义区分开。 @@ -575,13 +1120,15 @@ int main(int argc, char **argv) { // 后处理失败也统一走设备资源释放、ELF 卸载和 ACL 收尾,避免文件系统错误遗留运行时上下文。 bool cleanup_ok = true; + bool pmu_owner_restore_ok = true; // 先释放依赖当前 device/context 的大块内存,再卸载 ELF、销毁 stream,最后 reset device 与 finalize ACL。 if (trace_device != nullptr) { cleanup_ok &= CheckAcl(aclrtFree(trace_device), "aclrtFree(swimlane trace)"); } if (pmu_registers_device != nullptr) { // owner 必须在 MMIO 映射、device context 和 ACL runtime 仍有效时恢复。 - cleanup_ok &= pmu_owner.Finalize(); + pmu_owner_restore_ok = pmu_owner.Finalize(); + cleanup_ok &= pmu_owner_restore_ok; cleanup_ok &= UnmapPmuRegisters(options.device, &pmu_mappings); } cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(state)"); @@ -591,6 +1138,19 @@ int main(int argc, char **argv) { cleanup_ok &= CheckAcl(aclrtDestroyStream(stream), "aclrtDestroyStream"); cleanup_ok &= CheckAcl(aclrtResetDevice(options.device), "aclrtResetDevice"); cleanup_ok &= CheckAcl(aclFinalize(), "aclFinalize"); + if (!pmu_options.json_path.empty()) { + if (!pmu_json_ready || !all_passed || !postprocess_ok || !cleanup_ok || !pmu_owner_restore_ok) { + std::fprintf(stderr, "PMU JSON was not published because the capture or restore transaction failed.\n"); + postprocess_ok = false; + } else if (!ExportPmuJson( + *state, options, pmu_options, pmu_json_run, pmu_json_host_us, + pmu_json_submit_span_us, pmu_json_validation, pmu_json_semantic_passed, + pmu_owner_evidence, + pmu_owner_restore_ok, pmu_options.json_path + )) { + postprocess_ok = false; + } + } // 运行语义、后处理和资源清理三者全部成功,进程才返回成功,脚本据此决定是否继续生成 merged 泳道。 return all_passed && postprocess_ok && cleanup_ok ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index 3ccbdcd025..48f67aa16b 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -116,6 +116,14 @@ struct CcecOps { __aicore__ static inline void Nop(uint32_t count) { RuntimeNop(count); } + __aicore__ static inline bool PmuWindowStart( + __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id + ); + + __aicore__ static inline void PmuWindowStop( + __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id, bool started + ); + // SPIN_WAIT_HINT is also a no-op in the real A5 inner-kernel contract. // 同理不额外插入 nop,让等待循环保留真实 PA 内核“不主动退避”的指令成本。 __aicore__ static inline void SpinHint() {} @@ -155,9 +163,15 @@ struct CcecOps { struct PmuSnapshot { uint64_t total_cycles = 0; + uint32_t vector_busy = 0; + uint32_t cube_busy = 0; uint32_t scalar_busy = 0; + uint32_t mte1_busy = 0; + uint32_t mte2_busy = 0; + uint32_t mte3_busy = 0; uint32_t icache_requests = 0; uint32_t icache_misses = 0; + uint32_t fix_busy = 0; uint32_t status = 0; }; @@ -170,12 +184,24 @@ __aicore__ inline uint32_t ReadPmuRegister(uint64_t reg_base) { __aicore__ inline PmuSnapshot ReadObservedCounters(uint64_t reg_base) { PmuSnapshot sample; + sample.vector_busy = ReadPmuRegister(reg_base); + sample.cube_busy = ReadPmuRegister(reg_base); sample.scalar_busy = ReadPmuRegister(reg_base); + sample.mte1_busy = ReadPmuRegister(reg_base); + sample.mte2_busy = ReadPmuRegister(reg_base); + sample.mte3_busy = ReadPmuRegister(reg_base); sample.icache_requests = ReadPmuRegister(reg_base); sample.icache_misses = ReadPmuRegister(reg_base); + sample.fix_busy = ReadPmuRegister(reg_base); const uint64_t low = ReadPmuRegister(reg_base); const uint64_t high = ReadPmuRegisterresults[worker_id]; + PmuRegisterContext context; + const uint32_t physical_core_id = static_cast(get_coreid()) & kStatusCoreIdMask; + context.status = kStatusRequested | (physical_core_id << kStatusCoreIdShift); + const uint64_t table_address = + static_cast(state->config.reserved[kConfigRegTableLow]) | + (static_cast(state->config.reserved[kConfigRegTableHigh]) << 32); + if (state->config.reserved[kConfigMagic] != kConfigMagicValue || table_address == 0 || + physical_core_id >= kPhysicalSubcoreCount) { + return context; + } + context.status |= kStatusCoreIdValid; + __gm__ const uint64_t *register_bases = reinterpret_cast<__gm__ const uint64_t *>(table_address); + context.reg_base = register_bases[physical_core_id]; + if (context.reg_base == 0) return context; + context.status |= kStatusRegMapped; + + // selector 与 standalone owner 的 A5 PIPE_UTILIZATION 事件表逐项核对,避免 + // 配置或 ABI 错位时仍把 CNT0..8 的数值按 vector/scalar/I-cache 名称导出。 + if (ReadPmuRegister(context.reg_base) == kVectorBusyEvent) + context.status |= kStatusCnt0Selector; + if (ReadPmuRegister(context.reg_base) == kCubeBusyEvent) + context.status |= kStatusCnt1Selector; + if (ReadPmuRegister(context.reg_base) == kScalarBusyEvent) + context.status |= kStatusCnt2Selector; + if (ReadPmuRegister(context.reg_base) == kMte1BusyEvent) + context.status |= kStatusCnt3Selector; + if (ReadPmuRegister(context.reg_base) == kMte2BusyEvent) + context.status |= kStatusCnt4Selector; + if (ReadPmuRegister(context.reg_base) == kMte3BusyEvent) + context.status |= kStatusCnt5Selector; + if (ReadPmuRegister(context.reg_base) == kIcacheRequestEvent) + context.status |= kStatusCnt6Selector; + if (ReadPmuRegister(context.reg_base) == kIcacheMissEvent) + context.status |= kStatusCnt7Selector; + if (ReadPmuRegister(context.reg_base) == kFixBusyEvent) + context.status |= kStatusCnt8Selector; + return context; +} + +__aicore__ inline void PublishPmuSnapshot( + __gm__ pa_scheduler::WorkerResult &result, const PmuSnapshot &sample +) { + // 每核独占 sidecar 通过 bypass store 一次性发布;这些写发生在 PMU stop/read 之后, + // 不进入被导出的 Submit 窗口。 + CcecOps::Publish(&result.pmu_total_cycles, sample.total_cycles); + CcecOps::Publish(&result.pmu_scalar_busy, sample.scalar_busy); + CcecOps::Publish(&result.pmu_icache_requests, sample.icache_requests); + CcecOps::Publish(&result.pmu_icache_misses, sample.icache_misses); + CcecOps::Publish(&result.pmu_status, sample.status); + CcecOps::Publish(&result.pmu_vector_busy, sample.vector_busy); + CcecOps::Publish(&result.pmu_cube_busy, sample.cube_busy); + CcecOps::Publish(&result.pmu_mte1_busy, sample.mte1_busy); + CcecOps::Publish(&result.pmu_mte2_busy, sample.mte2_busy); + CcecOps::Publish(&result.pmu_mte3_busy, sample.mte3_busy); + CcecOps::Publish(&result.pmu_fix_busy, sample.fix_busy); +} + +__aicore__ inline bool CcecOps::PmuWindowStart( + __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id +) { + using namespace pa_scheduler::ccec_pmu; + (void)worker_id; + const WindowMode mode = static_cast(state->config.reserved[kConfigMode]); + if (mode != WindowMode::SubmitAll) return false; + const PmuRegisterContext context = ResolvePmuRegisters(state); + if (context.reg_base == 0) return false; + // Main AICPU owner 已在 launch 前配置并开启计数;先 stop + snapshot/read-clear, + // 再从本 worker 的首个 orchestration 动作开始独立累计。 + bisheng::cce::metrics_prof_stop(); + (void)ReadObservedCounters(context.reg_base); + bisheng::cce::metrics_prof_start(); + return true; +} + +__aicore__ inline void CcecOps::PmuWindowStop( + __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id, bool started +) { + using namespace pa_scheduler::ccec_pmu; + if (!started) return; + // 先冻结 gate,再读取 selector 和 counter。若先 ResolvePmuRegisters,九次 + // selector ld_dev 会被误计入本 worker 的 Submit 窗口。 + bisheng::cce::metrics_prof_stop(); + const PmuRegisterContext context = ResolvePmuRegisters(state); PmuSnapshot sample; + sample.status = context.status; + if (context.reg_base != 0) { + sample = ReadObservedCounters(context.reg_base); + sample.status = context.status | kStatusWindowStarted | kStatusWindowStopped; + if (sample.total_cycles != 0) sample.status |= kStatusTotalNonzero; + } + PublishPmuSnapshot(state->results[worker_id], sample); +} +__aicore__ inline void RunPmuProbe(__gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id) { + using namespace pa_scheduler::ccec_pmu; + __gm__ pa_scheduler::WorkerResult &result = state->results[worker_id]; const WindowMode mode = static_cast(state->config.reserved[kConfigMode]); + // SubmitAll 已在公共调度器 hook 内完成 start/stop/read/publish;此处只保留 + // empty/scalar/scalar-double 校准,保证校准链路与正式窗口可独立复验。 + if (mode == WindowMode::SubmitAll) return; + + PmuSnapshot sample; if (mode != WindowMode::Off) { - sample.status |= kStatusRequested; - const uint32_t physical_core_id = static_cast(get_coreid()) & kStatusCoreIdMask; - sample.status |= physical_core_id << kStatusCoreIdShift; - const uint64_t table_address = - static_cast(state->config.reserved[kConfigRegTableLow]) | - (static_cast(state->config.reserved[kConfigRegTableHigh]) << 32); - if (state->config.reserved[kConfigMagic] == kConfigMagicValue && table_address != 0 && - physical_core_id < kPhysicalSubcoreCount) { - sample.status |= kStatusCoreIdValid; - __gm__ const uint64_t *register_bases = reinterpret_cast<__gm__ const uint64_t *>(table_address); - const uint64_t reg_base = register_bases[physical_core_id]; - if (reg_base != 0) { - sample.status |= kStatusRegMapped; - const uint32_t selector2 = - ReadPmuRegister(reg_base); - const uint32_t selector6 = - ReadPmuRegister(reg_base); - const uint32_t selector7 = - ReadPmuRegister(reg_base); - if (selector2 == kScalarBusyEvent) sample.status |= kStatusCnt2Selector; - if (selector6 == kIcacheRequestEvent) sample.status |= kStatusCnt6Selector; - if (selector7 == kIcacheMissEvent) sample.status |= kStatusCnt7Selector; - - // 外部 profiler 启动 task 时可能已经累计了 scheduler;先冻结并读取一次窗口前快照。 - // A5 的 snapshot 会消费/清除当前累计;stop/start 只负责门控,中间不读取才能续积多段窗口。 + const PmuRegisterContext context = ResolvePmuRegisters(state); + sample.status = context.status; + if (context.reg_base != 0) { + bisheng::cce::metrics_prof_stop(); + const PmuSnapshot prior = ReadObservedCounters(context.reg_base); + bisheng::cce::metrics_prof_start(); + if (mode == WindowMode::Scalar || mode == WindowMode::ScalarDouble) { + RuntimeNop(state->config.reserved[kConfigScalarNops]); + } + if (mode == WindowMode::ScalarDouble) { bisheng::cce::metrics_prof_stop(); - const PmuSnapshot prior = ReadObservedCounters(reg_base); bisheng::cce::metrics_prof_start(); - if (mode == WindowMode::Scalar || mode == WindowMode::ScalarDouble) { - RuntimeNop(state->config.reserved[kConfigScalarNops]); - } - if (mode == WindowMode::ScalarDouble) { - // 两段相同工作量之间只切 gate、不读取 counter,用于确认 resume 是累计还是重置。 - bisheng::cce::metrics_prof_stop(); - bisheng::cce::metrics_prof_start(); - RuntimeNop(state->config.reserved[kConfigScalarNops]); - } - bisheng::cce::metrics_prof_stop(); - sample = ReadObservedCounters(reg_base); - sample.status |= kStatusRequested | kStatusRegMapped | kStatusCoreIdValid | - (physical_core_id << kStatusCoreIdShift); - if (selector2 == kScalarBusyEvent) sample.status |= kStatusCnt2Selector; - if (selector6 == kIcacheRequestEvent) sample.status |= kStatusCnt6Selector; - if (selector7 == kIcacheMissEvent) sample.status |= kStatusCnt7Selector; - if (sample.total_cycles != 0) sample.status |= kStatusTotalNonzero; - - // 窗口前 read-clear 之后只做这一次最终 snapshot;重复读取会看到读取路径自身的残余。 - // prior 更大只作为“已从此前 scheduler 累计中隔离”的辅助证据,不作为长负载通用门禁。 - if (sample.total_cycles < prior.total_cycles) sample.status |= kStatusPriorSnapshotLarger; + RuntimeNop(state->config.reserved[kConfigScalarNops]); } + bisheng::cce::metrics_prof_stop(); + sample = ReadObservedCounters(context.reg_base); + sample.status = context.status | kStatusWindowStarted | kStatusWindowStopped; + if (sample.total_cycles != 0) sample.status |= kStatusTotalNonzero; + if (sample.total_cycles < prior.total_cycles) sample.status |= kStatusPriorSnapshotLarger; } } - - // 结果位于每核独占 sidecar,五次 bypass store 不参与 Submit 时间口径。 - CcecOps::Publish(&result.pmu_total_cycles, sample.total_cycles); - CcecOps::Publish(&result.pmu_scalar_busy, sample.scalar_busy); - CcecOps::Publish(&result.pmu_icache_requests, sample.icache_requests); - CcecOps::Publish(&result.pmu_icache_misses, sample.icache_misses); - CcecOps::Publish(&result.pmu_status, sample.status); + PublishPmuSnapshot(result, sample); } } // namespace diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h index a19765b9a0..ccea646362 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h @@ -16,15 +16,22 @@ namespace pa_scheduler::ccec_pmu { -// 第一阶段只验证观察链路,不把门控下沉到真实 Submit 热路径。 -// Empty 用于量 start/stop 本身的底噪,Scalar 在同一窗口内执行可控 NOP 段。 +// Empty/Scalar/ScalarDouble 在调度结束后校准门控底噪和 scalar 正向响应; +// SubmitAll 则在公共调度器 hook 内覆盖本 worker 的完整 Submit 回放窗口。 enum class WindowMode : uint32_t { Off = 0, Empty = 1, Scalar = 2, ScalarDouble = 3, + // SubmitAll 从本 worker 的 orchestration/Submit 回放前开始,到最后一次 + // Submit 返回后停止。 + SubmitAll = 4, }; +inline bool IsSubmitWindow(WindowMode mode) { + return mode == WindowMode::SubmitAll; +} + // RunConfig::reserved 保持既有 64B ABI;CCEC 独占解释以下五个槽位,其他后端仍看到全零。 constexpr uint32_t kConfigMode = 0; constexpr uint32_t kConfigScalarNops = 1; @@ -42,22 +49,40 @@ constexpr uint64_t kAivFirstOffset = 0x100000ULL; constexpr uint64_t kAivSecondOffset = 0x200000ULL; constexpr uint32_t kAicoreMapBytes = 0x300000U; -// PIPE_UTILIZATION 的三个 scalar 前端事件。事件配置仍由 msprof/AICPU 完成,kernel 只读回核对。 +// PIPE_UTILIZATION 事件由 standalone Main AICPU owner 配置,kernel 逐核读回并核对 selector。 constexpr uint32_t kScalarBusyEvent = 0x1U; constexpr uint32_t kIcacheRequestEvent = 0x34U; constexpr uint32_t kIcacheMissEvent = 0x35U; +constexpr uint32_t kVectorBusyEvent = 0x501U; +constexpr uint32_t kCubeBusyEvent = 0x301U; +constexpr uint32_t kMte1BusyEvent = 0x701U; +constexpr uint32_t kMte2BusyEvent = 0x202U; +constexpr uint32_t kMte3BusyEvent = 0x203U; +constexpr uint32_t kFixBusyEvent = 0x714U; // DAV_3510 PMU MMIO offset。ld_dev 的立即数只有 12 bit,因此 kernel 会分别重基址到 0x2400/0x4200。 constexpr uint32_t kSelectorBlockOffset = 0x2400U; constexpr uint32_t kCounterBlockOffset = 0x4200U; constexpr uint32_t kCnt2Offset = 0x4220U; +constexpr uint32_t kCnt0Offset = 0x4210U; +constexpr uint32_t kCnt1Offset = 0x4218U; +constexpr uint32_t kCnt3Offset = 0x4228U; +constexpr uint32_t kCnt4Offset = 0x4230U; +constexpr uint32_t kCnt5Offset = 0x4238U; constexpr uint32_t kCnt6Offset = 0x4240U; constexpr uint32_t kCnt7Offset = 0x4248U; +constexpr uint32_t kCnt8Offset = 0x4250U; constexpr uint32_t kTotalLowOffset = 0x4260U; constexpr uint32_t kTotalHighOffset = 0x4264U; constexpr uint32_t kCnt2SelectorOffset = 0x2508U; +constexpr uint32_t kCnt0SelectorOffset = 0x2500U; +constexpr uint32_t kCnt1SelectorOffset = 0x2504U; +constexpr uint32_t kCnt3SelectorOffset = 0x250cU; +constexpr uint32_t kCnt4SelectorOffset = 0x2510U; +constexpr uint32_t kCnt5SelectorOffset = 0x2514U; constexpr uint32_t kCnt6SelectorOffset = 0x2518U; constexpr uint32_t kCnt7SelectorOffset = 0x251cU; +constexpr uint32_t kCnt8SelectorOffset = 0x2520U; // pmu_status 的低位描述本条记录是否可信,高 16 bit 保存 get_coreid(),便于 host 检查 96 核唯一性。 constexpr uint32_t kStatusRequested = 1U << 0; @@ -66,11 +91,21 @@ constexpr uint32_t kStatusCoreIdValid = 1U << 2; constexpr uint32_t kStatusCnt2Selector = 1U << 3; constexpr uint32_t kStatusCnt6Selector = 1U << 4; constexpr uint32_t kStatusCnt7Selector = 1U << 5; +constexpr uint32_t kStatusWindowStarted = 1U << 6; constexpr uint32_t kStatusTotalNonzero = 1U << 7; constexpr uint32_t kStatusPriorSnapshotLarger = 1U << 8; +constexpr uint32_t kStatusCnt0Selector = 1U << 9; +constexpr uint32_t kStatusCnt1Selector = 1U << 10; +constexpr uint32_t kStatusCnt3Selector = 1U << 11; +constexpr uint32_t kStatusCnt4Selector = 1U << 12; +constexpr uint32_t kStatusCnt5Selector = 1U << 13; +constexpr uint32_t kStatusCnt8Selector = 1U << 14; +constexpr uint32_t kStatusWindowStopped = 1U << 15; constexpr uint32_t kStatusRequired = kStatusRequested | kStatusRegMapped | kStatusCoreIdValid | kStatusCnt2Selector | kStatusCnt6Selector | kStatusCnt7Selector | - kStatusTotalNonzero; + kStatusCnt0Selector | kStatusCnt1Selector | kStatusCnt3Selector | + kStatusCnt4Selector | kStatusCnt5Selector | kStatusCnt8Selector | + kStatusWindowStarted | kStatusWindowStopped | kStatusTotalNonzero; constexpr uint32_t kStatusCoreIdShift = 16; constexpr uint32_t kStatusCoreIdMask = 0x0fffU; diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index d22bdf86a7..5ef3e8cb42 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -568,6 +568,15 @@ struct alignas(64) WorkerResult { // 仅在 trace_enabled bit1 开启时递增;每次源码 atomic 调用恰好增加一, // host 用它与 Atomic span 数逐 worker 闭合,禁止把丢记录的泳道当成完整结果。 uint64_t atomic_trace_calls; + + // PIPE_UTILIZATION 已同时配置 CNT0/1/3/4/5/8;与上面的 scalar/I-cache + // 一样只保存每核原始累计值,AIC/AIV 汇总与比率统一在 host sidecar 中计算。 + uint32_t pmu_vector_busy; + uint32_t pmu_cube_busy; + uint32_t pmu_mte1_busy; + uint32_t pmu_mte2_busy; + uint32_t pmu_mte3_busy; + uint32_t pmu_fix_busy; }; // WorkerResult 是 standalone 尾部的诊断 sidecar,不属于真实 DistCore ABI;按 // cache line 隔离后,各 worker 发布统计不会相互覆盖或污染被测共享状态。 @@ -576,6 +585,7 @@ static_assert(offsetof(WorkerResult, pmu_total_cycles) == 680, "WorkerResult PMU static_assert(offsetof(WorkerResult, pmu_status) == 700, "WorkerResult PMU status offset mismatch"); static_assert(offsetof(WorkerResult, fanin_not_ready_loads) == 704, "WorkerResult atomic diagnostic offset mismatch"); static_assert(offsetof(WorkerResult, atomic_trace_calls) == 736, "WorkerResult atomic trace offset mismatch"); +static_assert(offsetof(WorkerResult, pmu_vector_busy) == 744, "WorkerResult extended PMU offset mismatch"); // 从 cube_cursor 到 workers 结束保留关键字段 offset、DistCore ABI 和生产总字节跨度, // 并非字段级完整镜像。RunConfig、输入 context_lens 与校验结果追加在该跨度之后, diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index 4353557b4b..c2fec18908 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -743,7 +743,13 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, SubmitContext context; if (!IsFatal(state, stats)) { // Case1 每个 batch 固定回放 Alloc/QK/SF/PV/UP 五个 task;所有 worker 顺序相同,执行 lane 由 Claim 筛选。 + // CCEC 可在这里开启本 worker 私有 PMU 窗口;CPU/AscendC 适配层是空实现。 + // 窗口覆盖从首个参数构造到末次 Submit 返回,与全局“首 Submit.begin~末 Submit.end” + // 口径接近但不相同,host sidecar 必须按 per-worker 累计解释。 ResetTraceLap(worker); + // lap 重置属于泳道观察自身,不应污染 PMU-only 的 Submit 取数;窗口从 + // orchestration 初始化(即首批参数构造)前一条边界开始。 + const bool pmu_window_started = Ops::PmuWindowStart(state, worker_id); InitPaOrchestration(orchestration, batches, &state->context_lens[0]); for (uint32_t batch = 0; batch < batches; ++batch) { BuildAllocArgs(orchestration, args, batch); @@ -802,6 +808,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, break; } } + Ops::PmuWindowStop(state, worker_id, pmu_window_started); } // replay_done 表示所有 worker 已退出回放循环(成功路径即完整提交);之后仍需 drain 到本核 slot 为空。 diff --git a/tests/atomic_probe/pa_scheduler/cpu/main.cpp b/tests/atomic_probe/pa_scheduler/cpu/main.cpp index 98b03678d7..53eec6d47d 100644 --- a/tests/atomic_probe/pa_scheduler/cpu/main.cpp +++ b/tests/atomic_probe/pa_scheduler/cpu/main.cpp @@ -131,6 +131,10 @@ struct CpuOps { static inline void Nop(uint32_t count) { RuntimeNop(count); } + static inline bool PmuWindowStart(pa_scheduler::SchedulerState *, uint32_t) { return false; } + + static inline void PmuWindowStop(pa_scheduler::SchedulerState *, uint32_t, bool) {} + static inline void SpinHint() {} static inline void InvalidateRegion(const void *, uint64_t) { diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index bba2cdba9c..5d248fba85 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -34,13 +34,16 @@ Benchmark options: --swimlane-json FILE --no-swimlane -CCEC-only PMU probe options (the host must be launched by msprof PipeUtilization): - --pmu-window off|empty|scalar|scalar-double +CCEC-only PMU probe options (selectors are owned by the standalone Main AICPU helper): + --pmu-window off|empty|scalar|scalar-double|submit-all --pmu-scalar-nops N + --pmu-json FILE The swimlane action enables atomic tracing by default. For the lower-level run action, --trace-atomics still requires swimlane tracing; add --analyze-swimlane to print the per-role/per-site timing distributions. +--pmu-json requires --runs 1 and a non-off PMU window. PMU probe options are +CCEC-only and cannot target all. The swimlane action performs exactly one run and writes both the raw capture and merged Perfetto JSON below this directory's outputs/ folder. It rejects @@ -118,6 +121,25 @@ reject_managed_swimlane_options() { done } +reject_pmu_options_for_non_ccec() { + local backend="$1" + shift + if [[ "$backend" == "ccec" ]]; then + return + fi + + # PMU selector、校准 NOP 和导出路径都由 CCEC Main AICPU 所有者消费。 + # 在顶层展开 all 之前拒绝,避免先启动 CCEC、再由其他后端迟到报错。 + for argument in "$@"; do + case "$argument" in + --pmu-window|--pmu-window=*|--pmu-scalar-nops|--pmu-scalar-nops=*|--pmu-json|--pmu-json=*) + echo "PMU option $argument is CCEC-only; backend '$backend' is not supported." >&2 + exit 1 + ;; + esac + done +} + # 顶层先解释 action/backend;run 的其余参数交给共享 parser,build、smoke 和 # swimlane 再分别处理自己的约束或默认注入项。参数不足会在创建目录前失败。 if [[ $# -lt 2 ]]; then @@ -136,6 +158,9 @@ else BACKENDS=("$BACKEND") fi +# 后端约束必须早于 build/run/smoke/swimlane 的任何文件创建、构建或设备动作。 +reject_pmu_options_for_non_ccec "$BACKEND" "$@" + case "$ACTION" in build) # build 只选择后端,不接收 benchmark 参数;这样编译配置不会被运行时 From c6daaeb774b3c899686e97927177c727768e767f Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 09:50:01 +0000 Subject: [PATCH 015/214] =?UTF-8?q?=E6=96=87=E6=A1=A3(a5):=20=E5=9B=BA?= =?UTF-8?q?=E5=8C=96PA=E5=8E=9F=E5=AD=90=E4=B8=8ESubmit=20PMU=E8=A7=82?= =?UTF-8?q?=E5=AF=9F=E5=8F=A3=E5=BE=84?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 记录逐atomic边界、计数闭环和256 batch ClaimMax定量归因。 补充自包含owner、submit-all PMU-only JSON合同、三轮256 batch原始取数及使用限制。 --- ...05\345\206\265\345\210\206\346\236\220.md" | 470 ++++++++++++++++-- ...77\347\224\250\346\214\207\345\215\227.md" | 291 ++++++++--- 2 files changed, 639 insertions(+), 122 deletions(-) diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index f18083bb15..1689b9cd8c 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -204,8 +204,9 @@ Submit/执行路径中动态次数为 **0**;AICPU 初始化仍会重置相关 异常路径中的 `set_fatal()` 使用 `atomic_exchange(g_dist.fatal, 1)`;上述 成功运行没有触发。由于 fanin/frontier/屏障轮询次数依赖时序,不能把静态 -下界当作整次运行的 atomic 总数。若需要精确动态计数,应使用 PMU 或独立 -诊断构建;直接在热路径增加共享计数器会反过来改变竞争形态。 +下界当作整次运行的 atomic 总数。若需要精确动态计数,应使用 worker-local 软件 +计数或独立诊断构建;当前 PMU 事件不直接给出 atomic 条数。直接在热路径增加共享 +计数器会反过来改变竞争形态。 ## 4. 当前优化逻辑与效果 @@ -743,20 +744,49 @@ standalone 的 Submit 中位数、均值和配对中心都没有改善。因此 重排迁移到真实 `dist_submit_collect_fanin()`,也不进行真实 PA A/B。standalone 候选代码已撤回,本节保留负结果,防止后续重复同一实验。 -### 7.3 阶段 O1:建立 CCEC 每核 scalar PMU 观察链路 - -#### 7.3.1 为什么不再使用 external task 汇总冒充局部数据 - -控制实验已经证明:即使在 kernel 内调用 CANN 正式 `metrics_prof_stop()`, -external task-based `msprof` 的整任务 raw cycles 仍不会随局部门控骤降。因此该 -汇总只能描述整个 task,不能作为 Claim、EfDrain、WaitForSlot 或 HeapGuard 的 -局部取数依据。 - -O1 在 standalone CCEC 同一 runtime TU 内完成门控、读取和发布:host 复用正式 -A5 runtime 的 `halResMap(PROCESS_CP1, RES_AICORE)` 布局,将 36 个 AICore 展开为 -108 个物理子核 MMIO base;kernel 用真实 `get_coreid()` 索引,逐核核对 -PipeUtilization 的 `CNT2=0x1`、`CNT6=0x34`、`CNT7=0x35` selector,最后只读取 -一次 `CNT_TOTAL/CNT2/CNT6/CNT7` 并写入每 worker 独占结果区。 +### 7.3 阶段 O1:建立 CCEC 每核 scalar/PIPE_UTIL PMU 观察链路 + +#### 7.3.1 直接 PMU owner 是唯一正式取数链路 + +整任务级 raw counter 无法由 kernel 内局部 gate 缩成 Submit 子窗口,因此不能作为 +Claim、EfDrain、WaitForSlot、HeapGuard 或 Submit 局部取数依据。当前正式链路不消费 +这类整任务汇总:CCEC host 使用本目录自带的 Main AICPU Path-A owner 保存、配置、 +读回并最终恢复 PMU 状态;kernel 在同一 runtime TU 内完成门控、读取和发布。 + +owner 的 Path-A loader、dispatcher 和 `simpler_aicpu_exec` 均随 standalone CCEC +构建,不依赖父目录探针。host 复用 A5 runtime 的 +`halResMap(PROCESS_CP1, RES_AICORE)` 布局,将 36 个 AICore 展开为 108 个物理子核 +MMIO base;kernel 用真实 `get_coreid()` 索引。观察链路逐核核对并读取 +`CNT0..CNT8`,最后一次性把 `CNT_TOTAL` 和九个 programmable counter 写入每 worker +独占结果区。 + +这里没有臆测事件编号。正式依据来自仓内 A5 platform 实现: + +- `src/a5/platform/include/common/platform_config.h` 给出 `CNT0..CNT8` 的 MMIO + offset `0x4210..0x4250`、selector offset `0x2500..0x2520`,以及 + `CNT_TOTAL` 的低/高 32 bit offset `0x4260/0x4264`; +- `src/a5/platform/include/common/pmu_profiling.h` 的 + `PMU_EVENTS_A5_PIPE_UTIL` 给出以下 event id 与正式名称; +- standalone `ccec/kernel.cpp` 在冻结 gate 后逐项读取 selector,只有九项都与 + 下表相等、物理子核映射有效且 total 非零时,host 才把该 worker 标为 trusted。 + +| counter | event id | 仓内正式名称 | 本文简写 | +| ------- | -------: | ------------ | -------- | +| CNT0 | `0x501` | `pmu_idc_aic_vec_busy_o` | vector busy | +| CNT1 | `0x301` | `cube_instr_busy` | cube busy | +| CNT2 | `0x001` | `scalar_instr_busy` | scalar busy | +| CNT3 | `0x701` | `mte1_instr_busy` | MTE1 busy | +| CNT4 | `0x202` | `mte2_instr_busy` | MTE2 busy | +| CNT5 | `0x203` | `mte3_instr_busy` | MTE3 busy | +| CNT6 | `0x034` | `icache_req` | I-cache request | +| CNT7 | `0x035` | `icache_miss` | I-cache miss | +| CNT8 | `0x714` | `pmu_fix_instr_busy` | fix busy | + +`CNT0..CNT8` 是 32 bit programmable counter,`CNT_TOTAL` 是由低/高两项组成的 +64 bit raw counter。正式门禁要求本轮最大 programmable counter 小于 +`UINT32_MAX/4`(25% 高水位),并报告剩余 headroom。这是缩短窗口后采用的保守 +风险阈值,只能降低未察觉回卷的风险;硬件没有随样本提供 wrap 次数,最终值即使 +远低于门槛,也不能证明此前没有恰好回卷一圈或多圈。 这套 PMU 事件不直接给出 atomic 操作条数。atomic 条数继续由源码不变量和 worker-local 软件计数精确核对;PMU 用于观察这些 atomic 及周边 scalar 指令造成 @@ -764,45 +794,65 @@ worker-local 软件计数精确核对;PMU 用于观察这些 atomic 及周边 #### 7.3.2 A5 动态验证结果 -在 CANN 9.1 task-based PipeUtilization 配置下,分别执行 10 轮 empty、单段 -100000 NOP 和双段 `2×100000` NOP。三组每轮都满足 96/96 selector/MMIO 记录 -可信、96/96 物理子核 id 唯一;CCEC、AscendC、CPU 原语义 smoke 也全部 PASS。 - -| 窗口 | 96 核 total 中位数 | 稳定性与响应 | -| ---- | -----------------: | ------------ | -| empty | 预热后约 419 | 轮间约 ±1,给出空窗口 gate 固定开销 | -| scalar 100000 | 预热后约 56775 | 轮间约 ±2,CNT2/req/miss 同步增加 | -| scalar-double 2×100000 | 预热后约 113113 | 轮间约 ±8,扣除 empty 后为单段 1.9997 倍 | - -I-cache request 的每核量级约为 empty 22、单段 26245、双段 52476;miss 约为 -4、23、45。窗口前 snapshot 会消费/清除此前累计;双段中间只执行 -`stop/start`、不读取 MMIO,末尾一次 snapshot 仍得到近似精确的两倍响应。 -这证明多窗口续积机制成立,而不需要在每次 Submit 中插入高扰动 MMIO read; -扩展到上千个真实小窗口仍需进一步核对 gate 次数、同次数 empty 对照和 counter -是否溢出,不能由双段结果直接宣称已经精确覆盖任意窗口数。 - -原始日志保留在: +2026-07-18 已用 standalone 自包含 owner 在真实 A5 上完成三种校准模式和一个 +`submit-all` 小样本。每种校准模式本轮各取一个样本,均为 96/96 trusted、物理核 id +唯一、32 AIC + 64 AIV、32 个完整 `1 AIC + 2 AIV` triplet,且 owner Restore PASS: + +| 窗口 | 96 核 PMU raw total 中位数 | 本轮可支持的结论 | +| ---- | -------------------------: | ---------------- | +| empty | 约 214 | 空 gate 路径能够闭环 | +| scalar 100,000 | 约 56,568 | scalar NOP 对 total/事件产生明确正向响应 | +| scalar-double 2×100,000 | 约 112,994 | 双段结果接近单段两倍,暂停后可继续累计 | + +这三个数是单次上板验收样本,不是多轮 A/A 稳定性统计,也不用于宣称某个 PMU raw +count 等于同数值的硬件 cycle 或纳秒。 + +同一版本还完成 `batches=1,nop-count=0,submit-all` 单次闭环:96 个 worker 都有 +实际 start/stop,九项 selector、owner bitmap membership、worker slot、物理 role、 +triplet 和 Restore 均通过。all/AIC/AIV 的 total 中位数约为 +36,066.5/28,708/39,745;96 核 scalar busy 求和约 2,661,612,I-cache +request/miss 求和为 210,399/30,283,按总和计算的 miss rate 约 14.3931%;host +`submit_span_us` 约 47.770。该样本只证明观察闭环可运行,不能替代 256 batch +约 5 ms 基线,也不能作为性能优化收益。 + +同一源码和默认 PA NOP 随后完成了 3 个独立进程的 +`batches=256,submit-all,PMU-only`。三轮均为 96/96 trusted,owner/slot/role/ +triplet/start/stop 门禁全部 PASS,Restore PASS,且原始 96 核记录重算 +ALL/AIC/AIV 的 `sum/mean/median/p95/max` 与 JSON summary 完全一致: + +| 独立进程 | Submit span | ALL/AIC/AIV total 中位数 | scalar busy 总和 | I-cache request/miss 总和 | miss rate | +| -------- | ----------: | ---------------------------: | -------------------: | -----------------------------: | --------: | +| run1 | 3,688.236 us | 5,812,276 / 5,546,285 / 5,835,567 | 448,463,772 | 69,812,583 / 5,854,421 | 8.3859% | +| run2 | 4,089.057 us | 6,624,292 / 6,365,728 / 6,640,326 | 523,142,035 | 69,451,706 / 5,847,256 | 8.4192% | +| run3 | 4,673.237 us | 7,251,930.5 / 7,230,620 / 7,272,650 | 594,774,017 | 70,065,443 / 5,830,645 | 8.3217% | + +三轮 Submit span 中位数为 4,089.057 us,仍处于 standalone 的毫秒级调度区间, +但不能与无 PMU 三轮的 4,904.346 us 做非配对单样本减法:`PIPE_ALL` +门控会改变多核到达与争用时序。本三轮内,I-cache request/miss 总和及 +miss rate 比 total/scalar busy 更稳定;AIC/AIV 分组 miss rate 的三轮中位数 +分别约为 3.1138% 和 12.0556%。这只是当前同配置三轮的可重复现象, +尚不能说明 miss 的来源,也不能换算成 stall 时间。三轮最大 programmable +counter 分别为 5,559,659、6,021,532 和 7,082,609,均低于 25% 风险门槛; +这仍不构成“已证明未回卷”。本机证据文件位于: ~~~text -tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ - empty10_20260718_015533_console.log - scalar100k_10_20260718_015558_console.log - scalar2x100k_10_20260718_021035_console.log +outputs/scalar_observation_final_20260718/pmu_submit_all_ccec_b256_v2/pmu_submit_all.json +outputs/scalar_observation_final_20260718/pmu_submit_all_ccec_b256_v2_run2/pmu_submit_all.json +outputs/scalar_observation_final_20260718/pmu_submit_all_ccec_b256_v2_run3/pmu_submit_all.json ~~~ #### 7.3.3 阶段决定与后续使用边界 -O1 已达到“可用”的门槛:物理核映射、事件 selector、正向敏感性、A/A 重复性 -和双 gate 累计全部在真实 A5 上得到动态验证。因此后续可以恢复 atomic 优化, -但一次运行只选择一个阶段,A/B 两侧保持完全相同的 gate 次数和代码布局,并用 -同次数 empty gate 对照门控成本。真实 vector/cube kernel 不能跳过;测 EfDrain -时应在唯一 kernel 调用点暂时 stop,返回后仅在原 gate 活跃时 resume,从而保留 -真实依赖时序而排除计算体计数。 +直接 owner、每核 selector/MMIO、实际 gate 状态、96 核拓扑和 Restore 已达到继续 +建设观察链路的正确性门槛。正式 Submit PMU 只保留一个窗口:`submit-all`。它从 +每 worker 的 orchestration 初始化前开始,在该 worker 最后一次 Submit 返回后停止; +不再通过热路径内反复 stop/start 尝试扣除模拟计算体。 -下一步先在 standalone 增加 `G` 的成功/失败分类、`A`、frontier ready/终止 load -等 worker-local 诊断计数,不增加共享 atomic。得到动态规模后,首个低风险单变量 -候选是每个私有 slot 缓存已经观察为 ready 的 fanin 前缀;slot 重用时必须重置, -且先完成 CCEC 交错 A/B,再决定是否迁移到真实 FDWIC。 +`metrics_prof_start/stop()` 自带 `PIPE_ALL` 屏障,门控会收口流水并可能改变多核 +到达时序。A/B 两侧必须保持相同 gate 次数、NOP 和代码布局,端到端收益仍由无 PMU、 +无泳道的独立进程交错 A/B 判断。standalone 的 QK/SF/PV/UP NOP 循环由 scalar +执行,会计入 scalar busy;它不是实际 Vector/Cube 计算,也不模拟真实 task 执行期间 +scalar 的等待状态。因此这里的 sidecar 只用于同配置调度分析,不能直接替代真实 PA。 ### 7.4 阶段 D1:精确分类 fanin 与 frontier 动态原子次数 @@ -881,3 +931,327 @@ frontier helping 的额外 FetchMax 中位数为 `A-T=14085` 次。ready-prefix 独立进程交错十对中 fanin 总 load 配对中心下降、Submit 中心不回退;否则记录负 结果并撤回,不迁移真实 FDWIC。standalone 第一版只验证当前单-lane PA Case1, 不能拿它的 PASS 代替 joint/BlockWon 覆盖。 + +### 7.5 阶段 O2:逐 atomic 泳道与 Submit scalar PMU + +#### 7.5.1 观察目标与证据拆分 + +O2 先完成观察链路,不在同一阶段继续消减 atomic。它回答两个不同问题: + +1. 逐 atomic 泳道回答“哪一个源码 atomic 调用发生在何时、属于哪个 site/op、 + 该调用按源码语义是 source-issue 还是本核 return-ready 边界、bracket 多宽”; +2. PMU sidecar 回答“指定 Submit 窗口内,每个物理子核累计了多少 + scalar/vector/cube/MTE/fix busy、I-cache request/miss 和 raw total”。 + +两类数据默认分开采集。逐 atomic trace 会增加两次 `get_sys_cnt()`、分支和一条 +64 B 私有 record 写入,进而改变代码布局、I-cache、worker 到达顺序、共享地址竞争 +和轮询次数;PMU-only 的扰动小于完整泳道,更适合取 AIC/AIV 平均和 PMU event +A/B。正式 PMU JSON 强制 `--no-swimlane`,不与 atomic trace 或 phase profile 合并; +两套结果只按同一源码版本和各自明确的 scope 交叉解释,不能声称逐 tick 精确对齐。 + +#### 7.5.2 十五个 site 与四种 op + +`AtomicSite` 是 raw/merged trace 的稳定编号,当前覆盖 standalone PA 公共调度器中 +所有显式共享 atomic 源码调用点。`AtomicOp` 只有 `Load/Exchange/FetchAdd/FetchMax` +四种;A5 CCEC 的 `Load` 仍是 `atomicAdd(address, 0)`,不是普通 GM load。 + +| id | AtomicSite | AtomicOp | 调度语义 | +| --: | ---------- | -------- | -------- | +| 0 | `StartupIncrement` | `FetchAdd` | worker 启动计数发布 | +| 1 | `StartupPoll` | `Load` | worker 启动屏障轮询 | +| 2 | `FatalPoll` | `Load` | 成功/异常路径 fatal 检查 | +| 3 | `FatalSet` | `Exchange` | 首次异常发布 | +| 4 | `ClaimMax` | `FetchMax` | 分片 cursor Claim | +| 5 | `FaninFlagLoad` | `Load` | 依赖 ready 检查 | +| 6 | `CompletionVendExchange` | `Exchange` | task vend 发布 | +| 7 | `CompletionFlagExchange` | `Exchange` | task flag 发布 | +| 8 | `FrontierInitialLoad` | `Load` | completion 开始时读取 frontier | +| 9 | `FrontierFlagLoad` | `Load` | 连续完成区间扫描 | +| 10 | `FrontierMax` | `FetchMax` | frontier 帮助前推 | +| 11 | `HeapFrontierLoad` | `Load` | HeapGuard slow path 读取 frontier | +| 12 | `HeapVendLoad` | `Load` | HeapGuard slow path 读取 retire vend | +| 13 | `ReplayDoneIncrement` | `FetchAdd` | worker 回放结束计数发布 | +| 14 | `ReplayDonePoll` | `Load` | 最终 drain 的 replay_done 轮询 | + +每次源码调用只生成一条同时包含 `start_cycle/end_cycle` 的 `Atomic` span,避免用 +start/end 两条记录把容量再放大一倍。raw schema 使用: + +- `auxiliary`:上表 site id; +- `flags[3:0]`:op id;bit 4:返回的旧值是否参与后续判断; +- 对 `Load`,bit 5 表示观察值是否为零; +- bit 6 表示本条结束时刻是否有“返回值本核可消费”依赖证据; +- 对 `FetchMax`,bits 8..31 保存饱和后的软件 retry 数;A5 单条硬件 + `atomicMax` 当前为 0,CPU CAS 回归可非零; +- `task_id`:能归属任务时写真实 task id,生命周期 atomic 写 `-1`。 + +converter 把这些事件放在对应 AIC/AIV 的原 scalar lane,atomic 不再 +伪装成与 scalar 并行的独立执行单元。名称显式带边界: +`atomic.return_ready..#` 或 +`atomic.source_issue..#`;category 也分别为 +`atomic.return_ready`/`atomic.source_issue`,无需点开 span 即可过滤区分。args 中 +保留整数 `cycles`、site/op、 +`result_used`、`return_ready_observed`、`completion_boundary`、Load 的 +`value_zero` 和 FetchMax 的 `retries`。Perfetto 的浮点微秒显示不用于 +替代 raw 整数 tick。 + +#### 7.5.3 按调用点语义区分的两种结束边界 + +不能按 `Exchange/FetchAdd/FetchMax` 指令名称一概选边界,必须看该源码 +调用点是否真正消费 atomic 返回的旧值。当前有两种口径: + +1. `source_issue_bracket`: + +~~~text +begin = get_sys_cnt() +old = atomic(...) +end = get_sys_cnt() +~~~ + +它用于返回旧值本来就不使用的发布型调用。`end` 与 `old` 无数据 +依赖,只表示源码发射包围区间;不表示返回值就绪、atomic retire 或 +其他核可见。 + +2. `return_value_ready`: + +~~~text +begin = get_sys_cnt() +old = atomic(...) +asm volatile("MOV old, old; MOV end, SYS_CNT" + : "+l"(old), "=&l"(end)) +~~~ + +该边界只用在协议本来就要判断 `old` 的 `Load/FetchMax` 调用。CCEC AIC +和 AIV 后端已确认生成紧邻的 `ATOM -> dependent MOV -> MOV SYS_CNT`; +`=&l` 防止时间戳输出与 atomic 返回寄存器重叠。它能证明 `old` 已可被 +本核 scalar 消费,仍不证明跨核全局可见。默认不加 DSB/ISB/额外 GM +地址依赖;这些操作要么本平台后端不支持,要么会明显改写被测路径。 + +standalone 的五个 `Exchange/FetchAdd` 调用点都不消费返回旧值,但共享 +新值仍由协议的后续 load 消费: + +| 发布调用点 | 旧值 | 新值消费者 | b1 热路实际情况 | +| ------------ | ---- | ------------ | ---------------- | +| `StartupIncrement` | 丢弃 | `StartupPoll` | 96 次发布,所有 worker 参与轮询 | +| `ReplayDoneIncrement` | 丢弃 | `ReplayDonePoll` | 96 次发布,所有 worker 参与轮询 | +| `CompletionFlagExchange` | 丢弃 | `FaninFlagLoad`/`FrontierFlagLoad` | 每 task 发布一次 | +| `CompletionVendExchange` | 丢弃 | wrap 后的 `HeapVendLoad` | b1 首圈 fast path 不读 vend | +| `FatalSet` | 丢弃 | `FatalPoll` | 成功 b1 不执行 `FatalSet` | + +所以强制这五处消费 `old` 会把原 no-return/发布路径改成等待返回型 +观测变体:Startup/Replay 会改变屏障到达和轮询次数,vend 会推迟 flag +发布,flag 会推迟 frontier helping。如果之后需要这种返回路径对照,必须 +做独立、单 site mask 的 A/B 诊断,不能把 B 的数据称为原 PA 热路。更贴近 +PA 的端到端 GAP 是“发布发射 -> 自然消费者首次观察到新值”;可以 +复用现有 load 记录做离线派生,但其包含互连可见性、消费者调度与轮询 +间隔,且在对外报告跨核时间差前还需单独验证各核 `SYS_CNT` 的对齐性。 + +两种边界的 `end` 都在本条 64 B trace record 写入之前取得,所以本条 +duration 不直接包含自己的 record 写入;但这次写入、附加指令和代码布局 +会影响后续 atomic 到达、竞争与 I-cache,整轮仍是插桩运行。 + +开启逐 atomic trace 时,每个 worker 在最终 drain 之后额外记录一个 +`ClockBaseline`:一条是连续两次 `get_sys_cnt()`,另一条是纯寄存器依赖 +hook 后读 `SYS_CNT`。它们只给出同一二进制、同一物理核上两类边界的 +计时分辨率和固定底噪分布,使用边界是: + +- AIC 与 AIV 分开报告 median/p95/max; +- 不把某个 role 的中位数逐条从 atomic duration 中相减; +- 不因某条 atomic 接近 ClockBaseline 就声称该 atomic 没有竞争或没有等待; +- 不用 ClockBaseline 推导 atomic retire、cache 一致性或跨核可见性时刻。 + +#### 7.5.4 计数闭环与容量门禁 + +逐 atomic 结果只有同时满足以下闭环才可进入正式分析: + +1. 每个 worker 的 raw `Atomic` 记录数等于其独占 + `WorkerResult.atomic_trace_calls`; +2. 全局 raw `Atomic` 记录数等于 96 个 worker 的软件计数之和; +3. 每个 worker 恰有 2 条 `ClockBaseline`(连续时钟与返回依赖 hook 各一条), + 因此全局固定为 192 条; +4. 原阶段记录、动态 wait 记录、atomic 记录和 192 条 ClockBaseline 的总和与 + trace header count 精确相等; +5. 每核 `dropped==0`,且 raw 到 merged 后逐 atomic 事件数不变; +6. 原有 Claim、winner、kernel、fanin、frontier、cursor、heap 和每 worker + 1280 Submit 语义断言仍全部 PASS。 + +trace 容量不足时必须明确报 overflow 并判该轮观察无效,不能只分析前缀,也不能 +根据静态公式补齐被丢弃的 duration。CPU pthread 启动轮询可能远多于 A5,CPU 的 +容量结果不能替代真实 A5 CCEC 的闭环;CPU/AscendC 主要用于公共 schema、编译和 +语义回归。 + +#### 7.5.5 Submit PMU 窗口与平均口径 + +CCEC 只保留一个正式逐 worker Submit 窗口,CPU/AscendC 对应 hook 是空实现,不伪造 +PMU 数据:`submit-all` 在本 worker 通过启动屏障、完成 `ResetTraceLap` 后,于 +orchestration 初始化前打开 gate,在该 worker 最后一次 Submit 返回后关闭。窗口包含 +参数构造、全部 Submit 调度,以及 standalone 用 scalar NOP 模拟的 QK/SF/PV/UP +计算体。 + +它是“每核从 orchestration 到最后一次 Submit 返回”的累计窗口,不是全局最早 +`Submit.start` 到最晚 `Submit.end` 的墙钟窗口,也不是逐次 Submit 窗口。 +启动屏障、`replay_done` 发布/轮询和最后一次 Submit 后的 final drain 不在该 PMU +窗口内;因此不能把十五个 atomic site 的全生命周期 trace 总和与 Submit PMU +直接一一对齐。 + +`metrics_prof_start/stop()` 都含 `PIPE_ALL` 屏障,门控边界会收口流水并可能改变 +worker 到达与争用时序。这个开销属于观察配置本身;只能比较 gate、NOP、构建布局 +完全相同的样本,不能把 PMU 样本的 Submit span 当成无观察性能基线。 + +PMU JSON sidecar 按 worker 保留 raw 记录,并分别汇总 32 AIC、64 AIV 和全部 96 核。 +各字段严格按下面口径解释: + +- `total_cycles`:gate 活跃期间的每核 64 bit PMU raw total;除非平台时钟/事件语义 + 另有正式证明,不直接按 1 GHz 换算成微秒;96 核求和是 core-work raw count, + 不是 Submit 墙钟时间; +- `scalar_busy`:`CNT2 scalar_instr_busy` 的事件累计,不等于窗口内全部时间, + 也不等于“纯调度耗时”; +- `icache_requests/misses`:`CNT6/CNT7` 的事件累计。整体 miss rate 必须用 + `sum(misses)/sum(requests)`,AIC/AIV 分开计算,不能平均 96 个逐核百分比; +- `vector/cube/MTE1/MTE2/MTE3/fix busy`:同一 PipeUtilization 配置的辅助证据, + 用来检查窗口内实际活跃单元,不能从事件名称反推出一条指令造成的精确 stall; +- `window_started/window_stopped`:来自每个 worker 实际执行 gate 的状态位,不用模式 + 配置推导“应该执行过”;所有 worker 还必须通过九项 selector、唯一物理核 id、 + owner bitmap membership、worker slot/物理 role/triplet、miss 不大于 request 和 + counter 风险门槛等门禁。 + +##### 7.5.5.1 自包含 Main AICPU Path-A owner 闭环 + +owner 已收入 `pa_scheduler/ccec`,所有构建和运行文件均位于 standalone 目录内。 +构建产出 Path-A dispatcher 与 owner AArch64 SO;host 通过已验证的 bootstrap、mode-0 +注册和 `simpler_aicpu_exec` 执行 Configure/Restore,不需要另一个进程提供 selector。 +本次上板闭环包括: + +- 对同一 stream 调用 `aclrtGetStreamResLimit`,A5 实报 AIC 32、AIV 64,活跃 + subcore 总数 96,不再把 108 个物理 MMIO slot 容量当成活跃数; +- AICPU 扫描 108 项,每项执行 save/configure/readback;成功 96 项置 bitmap, + 失败 12 项当场恢复并跳过,当前 bitmap 为 + `000003ff:fff3ff7f:f7cffcff:fffdf7ff`; +- 第一个跳过项是 index 11,`CTRL0` 读回 `0xffffffff`,期望 `0x7`;这是 + 不可配置 slot 的显式证据,不再泛化成 loader 失败; +- restore 只按 bitmap 从 107 到 0 逆序还原;dependent-atomic 最小样本和 11 对 + WARM/COLD I-cache 样本都位于成功 bitmap 的物理核 18,最终均为 + `pmu_restore_and_cleanup=PASS`; +- WARM/COLD 对照每轮均为 warm miss 0、cold miss 68,11/11 pair PASS,证明 + `CNT6/CNT7` 对受控 I-cache 工作量有稳定方向性。 + +standalone 本体还进一步通过 96/96 owner membership、精确 worker slot、物理 role、 +32 个完整 triplet、每核 window started/stopped 和 Restore 门禁。`batches=1,nop=0` +的 `submit-all` 单次样本见 7.3.2;它证明窗口可用,不代表 256 batch 的正式归因。 + +正式 PMU JSON 必须以 `--no-swimlane --runs 1 --pmu-window submit-all` 独立采集, +不能同时启用 atomic trace、phase profile、泳道分析或泳道 JSON。host 拒绝覆盖已有 +JSON/同名 `.tmp`,并只在协议、PMU/owner 门禁、Restore 和 runtime 清理全部成功后 +发布文件。`--no-swimlane` 会关闭 phase/atomic record 写,但普通调度路径中的阶段 +`SYS_CNT` 调用点仍存在;sidecar 会明确记录“有 timestamp call、无 record write、 +无 atomic trace、无 profile accumulation”,不能把该模式描述成编译期零插桩。 + +#### 7.5.6 当前证据状态与正式重采矩阵 + +O2 的 schema、逐 atomic wrapper、两条 ClockBaseline、scalar lane converter、Submit +PMU hook 和 CNT0..8 sidecar 已实现。2026-07-18 已用干净重编的 CCEC AIC/AIV +二进制完成 b1 atomic-trace-only 上板:全部协议断言 PASS,raw 共 4959 条、 +`expected=4959`、`dropped=0`,其中 Atomic 1395 条、ClockBaseline 192 条。 +Claim raw schema v2 也已直接闭环为 `won=5/lost=283/not_attempted=192`,不再 +借助 Atomic 记录反推 role-filtered Claim。converter 的直接脚本与 `python -m unittest` +两种入口均为 4/4 PASS。b1 的 Atomic 按边界闭环为 +`return_ready=1193/source_issue=202`。本轮产物为: + +~~~text +outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b1/raw.json +outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b1/merged_swimlane.json +~~~ + +b1 用于确认边界、schema 和泳道布局,不用它的插桩后 Submit span 替代 +256 batch 约 5 ms 的无诊断基线。同一二进制的 256 batch 随后也完成一轮: +全部断言 PASS,raw/merged 均为 963368 个 span,`expected=963368`、 +`dropped=0`,Atomic 99944 条、ClockBaseline 192 条;Claim 为 +`won=1280/lost=72448/not_attempted=49152`,恰好闭环 `96*1280`。合并泳道 +不存在旧 `AIC/AIV·atomic` 线程,atomic 全部位于对应 scalar lane。本轮 +边界计数为 `return_ready=97192/source_issue=2752`,与动态 site 计数之和一致; +插桩 Submit span 为 5.209261 ms,只证明观察构建未把数量级打坏;正式 +性能仍以关闭 trace/PMU 的独立进程 A/B 为准。产物为: + +~~~text +outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b256/raw.json +outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b256/merged_swimlane.json +~~~ + +同一源码随后关闭 swimlane/atomic-trace/PMU 做了三个独立进程,Submit 为 +3.729925/4.904346/5.563417 ms,中位数 4.904346 ms,三轮语义全部 PASS。 +这组数据说明当前 standalone 仍复现约 5 ms 量级,也显示了多核轮询、 +frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 ms 插桩轮 +不能与三轮中任一轮做单样本减法后归因 trace 成本。 + +仍未完成的正式重采如下: + +| 结果 | 当前状态 | +| ---- | -------- | +| 256 batch 逐 atomic raw/merged 事件数与 dropped | **已闭环** | +| 15 site 按 AIC/AIV 的 events/median/p95/max | **b1/256 batch 已采,未执行 site 明确为 0** | +| 192 条 ClockBaseline 的 AIC/AIV 分布 | **b1/256 batch 已闭环** | +| `submit-all` 的 owner/gate/CNT0..8/total 闭环 | **b1、零 NOP 单次上板已通过** | +| 256 batch `submit-all` 的 I-cache 与 pipe 分组分布 | **3 个 PMU-only 独立进程已采并完成 raw→summary 重算** | + +此前开发过程中的探索性输出不在这里引用为正式结论。最终按同一源码依次执行: + +1. 三后端全量重建及 no-trace/no-PMU 语义回归; +2. CCEC atomic-trace-only,检查上述计数闭环、ClockBaseline 和 raw→merged; +3. CCEC PMU-only 的 `submit-all` 已完成 3 个独立进程 A/A;后续每轮仍使用 + 唯一 JSON 路径,并检查 96 核、owner bitmap/triplet、start/stop、25% 风险门槛和 Restore; +4. 比较任何 atomic 优化前后时,双方使用相同观察模式、相同 gate/trace 配置和 + 独立进程交错 A/B;正式端到端 Submit 仍以无诊断构建复测。 + +#### 7.5.7 绝不能从 O2 声称的结论 + +即使上述重采全部通过,也不能声称: + +- 单条 source-issue 或 return-ready bracket 就是硬件 atomic retire 延迟、跨核 + 可见延迟或一致性完成时间; +- 所有 atomic duration 相加就是 Submit 中“atomic 占用时间”,或删除这些 atomic + 一定能等量缩短墙钟时间;不同核 bracket 会重叠,poll 数也会随插桩改变; +- `ClockBaseline` 可以逐事件相减并得到无偏的 atomic 硬件净耗时; +- trace 开启后的 Submit span 可以直接与无 trace 基线比较并宣布性能收益; +- `scalar_busy/total_cycles` 是墙钟时间,或 I-cache miss 数可以直接换算成 stall ns; +- standalone 的 scalar NOP 已经真实模拟了 vector/cube task 期间 scalar 的等待状态; + NOP 本身在 scalar 上执行并被 `submit-all` 统计,`PIPE_ALL` gate 还会收口流水; +- 最大 32-bit counter 低于 25% 风险门槛就证明本轮没有回卷;该门槛只降低风险, + 最终寄存器值不能直接检出恰好一圈或多圈的 wrap; +- standalone 的 PMU/atomic 分布可以直接替代真实 FDWIC PA。迁移真实用例时仍需 + 复用已验证的最小观察代码,并重新完成计数闭环、正确性和无诊断性能 A/B。 + +#### 7.5.8 256 batch loser ClaimMax 的定量归因 + +2026-07-18 的 `atomic_inlineasm_ccec_b256` 中,Claim 三态为 1280 个 winner、 +72448 个 attempted loser 和 49152 个 role-filtered not-attempted。每个 attempted Claim +恰好包含一条 `claim_max.fetch_max`,与 73728 条 ClaimMax 精确闭环。 + +| 口径 | 次数 | median | p95 | max | 多核累计 core-work | +| ---- | ---: | ---: | ---: | ---: | ---: | +| loser ClaimMax | 72448 | 280 ns | 637 ns | 4791 ns | 24.133954 ms | +| winner ClaimMax | 1280 | 268 ns | 508 ns | 1277 ns | 0.376479 ms | +| AIC loser ClaimMax | 23808 | 269 ns | 523 ns | 3434 ns | 7.058334 ms | +| AIV loser ClaimMax | 48640 | 290 ns | 677 ns | 4791 ns | 17.075620 ms | + +loser ClaimMax 占全部 99944 条 atomic bracket core-work 的 58.6%;去掉 Submit +窗口外的 startup/replay-done 生命周期 atomic 后,它占 Submit 内 atomic bracket +core-work 的 77.7%。因此它是“atomic 内部”的第一大项,但优势主要来自 +72448 次动态调用,不是 loser 单次比 winner 慢一个数量级。 + +不能由此把整个 Replay 或整个 Submit 都归因给 ClaimMax。`Replay` 是 lap +外层区间:从 materialize 前开始,包含 Materialize、PrepareMap、Claim、Register, +到 loser 分支结束,这些嵌套 span 不能相加。72448 个 attempted-loser Replay 的 +外层 core-work 为 192.977600 ms,其中 loser ClaimMax 直接 bracket 占 12.5%; +49152 个完全没有 FetchMax 的 not-attempted Replay 仍有 93.430073 ms core-work。 + +按每核累计看,loser ClaimMax 中位数为 0.258278 ms,占每核 Claim 中位 +累计的 45.7%、Replay 外层的 8.45%、Submit envelope 的 5.0%。本轮最晚结束的 +AIV core93 上,对应数字为 0.261731/5.191702 ms,也约 5.0%。这只是直接 +可见的本核返回等待占比;真正改动 Claim 协议后还会改变 winner 到达、 +frontier/fanin 时序和竞争形态,不能简单从 5.209261 ms 中减去 0.258 ms。 + +阶段性结论是:ClaimMax loser 是后续 atomic 消减的第一优先级,但尚未证明 +它是整个调度的主瓶颈。256 batch PMU-only 三轮 A/A 已补齐 Submit 全窗的 +I-cache 和 pipe 分组基线;它显示 AIV miss rate 在本三轮持续高于 AIC,但现有 +counter 仍不能把 miss 定位到 materialize/register 或某条 atomic。开始改 Claim 后, +应使用同配置的 PMU-only 交错 A/B 观察 request/miss 是否同步变化,端到端收益 +仍由无诊断的独立进程 A/B 确认。 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 434fd9c319..825a7c6443 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -191,7 +191,11 @@ runner 结束时会打印准确目录: 3. 每个 `block0` 至 `block31` 是一个物理 1AIC+2AIV block; 4. `AIC`、`AIV0`、`AIV1` 轨展示 Submit、Claim、EfDrain、Replay、RingBp 等 runtime 阶段,带 `·kernel` 的轨展示 QK、SF、PV、UP; -5. 点击事件可查看 `task_id`、`func_id`、`core`、`mc` 和 `aux`。 +5. 开启 `--trace-atomics` 时,Atomic 及 ClockBaseline 直接画在对应 + `AIC/AIV` scalar lane,名称与 category 显式区分 `return_ready` 和 + `source_issue`;不生成带 `·atomic` 的伪并行子轨; +6. 普通事件可查看 `task_id`、`func_id`、`core`、`mc` 和 `aux`;Atomic 的 + 字段和解读边界见 5.6 节。 WaitForSlot 和 HeapGuard 没有可伪造的逐事件起止时间,因此不单独生成 Perfetto 事件;它们由 `--profile-phases` 的 `[PHASE]` 累计统计呈现。实际发生等待时, @@ -238,6 +242,9 @@ CPU 完整协议回归建议关闭大泳道缓冲区: - `--profile-phases`:分别统计 Claim、EfDrain、WaitForSlot、HeapGuard; - `--analyze-swimlane`:读取完整记录,输出各阶段的 per-worker 累计分布以及 EfDrain/Materialize/Claim/Register 的 per-role、per-task-kind 单事件分布; +- `--trace-atomics`:在已开启的泳道中记录每次源码 atomic 调用括号;建议同时 + 传 `--analyze-swimlane` 输出按 AIC/AIV、调用点分组的分布。不能与 + `--no-swimlane` 同用; - `--swimlane-json FILE`:流式导出原始 `fdwic_events` JSON,要求单轮运行; - `--no-swimlane`:关闭泳道记录,不能与 `--analyze-swimlane` 或 `--swimlane-json` 同时使用; @@ -288,94 +295,230 @@ frontier_initial=... frontier_flag=... frontier_ready_fetch_max=... frontier_ter 结果区,不为诊断新增共享 atomic。它们仍会增加少量 scalar 指令,因此优化 A/B 必须使用相同的计数布局;不能把启用分类后的绝对时间直接与旧二进制比较。 -### 5.6 CCEC 每核 scalar PMU 与 I-cache 诊断 +### 5.6 逐 atomic 语义边界泳道 -CCEC 后端提供一个显式诊断模式,用来验证局部 scalar 性能观察链路。它不读取 -`msprof` 导出的整任务 PMU 汇总作为局部结果,而是由 kernel 在每个物理子核上 -直接读取 `CNT_TOTAL/CNT2/CNT6/CNT7`,最后写入该 worker 独占的 -`WorkerResult`。对应含义为: +`--trace-atomics` 记录 standalone 调度器中每一次动态 atomic 调用,不只记录 +winner 或慢样本。生成带文字分析的 CCEC 泳道可直接执行: -| 字段 | PipeUtilization selector | 含义 | -| --- | ---: | --- | -| `pmu_total_cycles` | PMU total | gate 窗口累计周期 | -| `pmu_scalar_busy` | `CNT2 = 0x1` | scalar 原始事件计数 | -| `pmu_icache_requests` | `CNT6 = 0x34` | I-cache request | -| `pmu_icache_misses` | `CNT7 = 0x35` | I-cache miss | +```bash +./run.sh swimlane ccec \ + --device 0 --batches 256 \ + --trace-atomics --analyze-swimlane +``` -selector 和 PMU framework 仍由 CANN 9.1 的 task-based profiler 配置;CCEC -只做门控与读取。一次 snapshot 会消费/清除此前累计,因此窗口前先冻结并做一次 -baseline read-clear,窗口中间只切 gate,末尾再做唯一一次结果 snapshot。host -按正式 A5 runtime 的方式调用 `halResMap`,构造 36 个 -物理 AICore 展开后的 108 项子核 MMIO 表;kernel 使用真实 `get_coreid()` 索引, -不能用逻辑 `worker_id` 猜物理核。 +转换后 Atomic 和 ClockBaseline 都放在对应 AIC/AIV 的原 scalar lane; +它们本来就是 scalar 指令,不再伪装成与 scalar 并行的独立子轨。 +Kernel 仍放在独立计算单元轨。Atomic 事件名显式区分两种边界: -在本目录先构建 CCEC,再直接让 `msprof` 包住 host runner: +```text +atomic.return_ready..# +atomic.source_issue..# +``` -```bash -./run.sh build ccec +category 也分别为 `atomic.return_ready` 与 `atomic.source_issue`,可在 +Perfetto 中直接过滤,无需逐条点开 args 才能区分。 + +当前固定 schema 共有 15 个调用点: + +| `site_id` | Perfetto `site` | `op` | 所属路径 | +| ---: | --- | --- | --- | +| 0 | `startup_increment` | `fetch_add` | 启动屏障到达计数 | +| 1 | `startup_poll` | `load` | 启动屏障轮询 | +| 2 | `fatal_poll` | `load` | fatal 状态检查 | +| 3 | `fatal_set` | `exchange` | fatal 状态发布 | +| 4 | `claim_max` | `fetch_max` | Submit lane Claim | +| 5 | `fanin_flag_load` | `load` | fanin 依赖 flag | +| 6 | `completion_vend_exchange` | `exchange` | completion vend 发布 | +| 7 | `completion_flag_exchange` | `exchange` | completion flag 发布 | +| 8 | `frontier_initial_load` | `load` | completion frontier 首次读取 | +| 9 | `frontier_flag_load` | `load` | frontier 扫描 flag | +| 10 | `frontier_max` | `fetch_max` | frontier 推进 | +| 11 | `heap_frontier_load` | `load` | HeapGuard frontier | +| 12 | `heap_vend_load` | `load` | HeapGuard vend | +| 13 | `replay_done_increment` | `fetch_add` | 回放完成屏障到达计数 | +| 14 | `replay_done_poll` | `load` | 最终 drain 中轮询回放完成 | + +上表是源码调用点集合,不代表每轮都会出现 15 类事件;例如正常成功路径不应 +执行 `fatal_set`。轮询点则会为每一次实际 load 生成独立事件。 + +原始 `fdwic_events` 仍是十列格式。对 `phase="Atomic"` 的记录, +`task_id` 是所属 PA task(启动/最终屏障等生命周期 atomic 为 -1), +`function_id` 固定为 -1,`auxiliary` 是上表 `site_id`,`flags` 使用独立 ABI:低 4 bit 是 +`op_id`(Load/Exchange/FetchAdd/FetchMax 依次为 0/1/2/3),bit 4 表示 +返回的旧值参与后续逻辑,bit 5 仅对 Load 表示本次读到零,bit 6 +表示是否取得了“返回值本核可消费”边界,bits 8..31 +仅对 FetchMax 保存饱和后的软件重试次数。CCEC/AscendC 的硬件 +`atomicMax` 当前报告重试数 0;CPU CAS loop 才有可观察的软件重试。 +merged 事件的 `args` 会显式导出 `site/site_id`、`op/op_id`、原始整数 +`cycles`、`result_used`、`return_ready_observed`、`completion_boundary`,以及操作 +适用时的 `value_zero` 或 `retries`。 + +边界必须按源码调用点语义解读: + +- `source_issue_bracket`:返回旧值本来就丢弃的发布型调用。当前五处是 + `startup_increment/replay_done_increment/completion_vend_exchange/` + `completion_flag_exchange/fatal_set`。结束时钟与旧值无依赖,只能表示 + 源码发射包围区间。 +- `return_value_ready`:协议本来就会判断返回值的 `Load/FetchMax`。CCEC + 在 atomic 后生成紧邻的 `dependent MOV -> MOV SYS_CNT`;这证明旧值已可被 + 本核 scalar 消费,不证明其他核已看到发布的新值。 + +两种终点都早于本条 64-byte trace record 写入。当前不为每条 atomic +加 DSB/ISB/额外 GM load;这些操作要么后端不支持,要么会明显改写 +被测路径。两种 bracket 都不能直接称为跨核可见或 atomic retire 延迟。 + +开启该诊断时,每个 worker 在最终 drain 之后还会写两条 +`ClockBaseline`:`clock.consecutive_sys_cnt_reads` 和 +`clock.atomic_return_dependency_hook`。前者量连续时钟读,后者量纯寄存器依赖 hook +的固定底噪。全局因此恰有 `96*2=192` 条,都只是分辨率参考,不是 +可以从每条 Atomic 机械相减的校正常数。`[TRACE_ATOMIC]` +按 AIC/AIV、site 和 op 输出事件数、源码括号原始累计、中位数、p95 和最大值。 + +记录写本身不在它自己的 span 内,但会改变后续指令布局、cache、多核到达顺序和 +atomic 争用;轮询次数也可能随之变化。所以不应将 `source_bracket_cycles_total` +与未插桩 Submit 时间相减,或用它计算 atomic 对 golden 的绝对占比。每核分区固定 +容纳 65,536 条记录;通过必须同时满足 `dropped=0`、总记录数闭合,以及每 worker +新增诊断记录数精确等于 `atomic_trace_calls + 2` 闭合。CPU 线程调度可能放大启动轮询并撑满分区;这种 +情况应视为本次 trace 无效,不能截断后继续分析。 + +### 5.7 CCEC 每核 PMU 与 I-cache sidecar + +CCEC 后端提供与泳道分离的 PMU sidecar。正式取数由本目录自带的 Main AICPU +Path-A owner 配置 selector、保存并恢复 PMU 状态;kernel 在每个物理子核内门控并 +读取 `CNT_TOTAL` 和 `CNT0..8`,再写入该 worker 独占的 `WorkerResult`。该链路不需要 +目录外探针或整任务 profiler 的计数结果。完整字段为: + +| sidecar 字段 | 寄存器 / selector | 原始事件含义 | +| --- | --- | --- | +| `total_cycles` | 64-bit PMU total | gate 窗口的 PMU total 原始计数 | +| `vector_busy` | `CNT0 = 0x501` | Vector pipe busy | +| `cube_busy` | `CNT1 = 0x301` | Cube pipe busy | +| `scalar_busy` | `CNT2 = 0x001` | Scalar pipe busy | +| `mte1_busy` | `CNT3 = 0x701` | MTE1 pipe busy | +| `mte2_busy` | `CNT4 = 0x202` | MTE2 pipe busy | +| `mte3_busy` | `CNT5 = 0x203` | MTE3 pipe busy | +| `icache_requests` | `CNT6 = 0x034` | I-cache request | +| `icache_misses` | `CNT7 = 0x035` | I-cache miss | +| `fix_busy` | `CNT8 = 0x714` | Fix pipe busy | + +#### Main AICPU Path-A owner + +owner 已自包含在 `ccec/`:构建会同时产出 dispatcher 和 owner AArch64 SO。host +通过 CANN 9.1 已验证的 Main AICPU Path-A 完成 bootstrap 和 mode-0 注册,运行时 +调用 `simpler_aicpu_exec` 执行 Configure/Restore;不要求用户另行启动 PMU 配置进程。 +owner 对本轮会话独占的 PMU 状态先保存、再配置并读回,结束时只按成功 bitmap 从 +107 到 0 逆序恢复。 + +host 对同一 stream 调用 `aclrtGetStreamResLimit`,当前 A5 实报 +`AIC=32/AIV=64/total=96`。owner 扫描 108 个物理 MMIO slot,只对完整读回一致的 +96 个置 bitmap,跳过 12 个不可配置 slot;32 个可用物理组都必须形成 +`1 AIC + 2 AIV` 完整 triplet。当前上板 bitmap 为: -OUT="./outputs/pmu_validation/empty" -msprof \ - --output="$OUT" \ - --type=db \ - --ai-core=on \ - --aic-mode=task-based \ - --aic-metrics=PipeUtilization \ - ./build/ccec/pa_scheduler_host \ - --kernel ./build/ccec/pa_scheduler_kernel.o \ - --device 0 --batches 1 --runs 10 --nop-count 0 --no-swimlane \ - --pmu-window empty +```text +000003ff:fff3ff7f:f7cffcff:fffdf7ff ``` -三个诊断窗口的用途是: +kernel 用真实 `get_coreid()` 查 host 通过 `halResMap` 建立的 MMIO 表,不用逻辑 +`worker_id` 猜物理核。正式结果必须同时满足:九个 selector 全部匹配、96 条记录 +可信且物理核 id 唯一、worker slot/role 与物理 triplet 对应、96 个核都实际执行过 +start/stop、`miss <= request`,以及 owner Restore 成功。 -- `empty`:只执行一次 start/stop,量空窗口的 gate 固定开销;末尾 snapshot 在 - stop 后执行,其 MMIO 读取开销不计入 total; -- `scalar`:在同一窗口内执行 `--pmu-scalar-nops N` 个受控 NOP; -- `scalar-double`:执行两段相同 NOP,中间只 stop/start、不读 counter,用于验证 - 多个局部窗口能否暂停后继续累计; -- `off`:默认值,不申请 MMIO 表,也不要求由 `msprof` 启动。 +`off` 是默认值,不建立 PMU owner 会话。四种非 off 模式中,前三种只用于观察链路 +校准;`submit-all` 是唯一正式的 Submit 取数窗口: -例如 10 万 NOP 的正向敏感性测试只需把末尾参数换成: +| `--pmu-window` | 位置与窗口边界 | 用途 | +| --- | --- | --- | +| `empty` | `RunScheduler` 完成后,baseline read-clear → start → stop → 末尾 snapshot | 量一段空 gate 底噪 | +| `scalar` | `RunScheduler` 完成后,一个 gate 中执行 `--pmu-scalar-nops N` | 验证 scalar/I-cache 正向敏感性 | +| `scalar-double` | `RunScheduler` 完成后,两段相同 NOP 中间只 stop/start、不读 counter | 验证暂停后继续累计 | +| `submit-all` | 每 worker 通过启动屏障后,在首次 PA 参数构造前 start,最后一次 Submit 返回后 stop | 累计 orchestration、Submit 和模拟 kernel NOP | -```bash ---pmu-window scalar --pmu-scalar-nops 100000 -``` +`submit-all` 不含启动屏障,也不含 `replay_done`、最终 drain、末尾 ClockBaseline +或 PMU 结果发布。它是 **每 worker 从 orchestration 初始化前到本核最后一次 Submit +返回后的累计窗口**,不是全局“最早 `Submit.begin` 到最晚 `Submit.end`”墙钟 span。 -每轮必须同时看到: +每次 `metrics_prof_start/stop()` 都带 `PIPE_ALL` 流水屏障。该屏障用于明确门控边界, +也会收口流水并可能改变多核到达时序;因此 PMU 样本只能与相同 gate、相同 NOP、 +相同构建配置的样本比较,不能把它当成无观察开销的端到端基线。当前模拟 QK/SF/PV/UP +的 NOP 循环实际在 scalar 上执行,`submit-all` 会把它计入 scalar 相关事件;这与真实 +Vector/Cube task 运行时 scalar 等待的状态并不等价,所以 sidecar 不声称是绝对的 +真实 PA profile。 -```text -trusted=96/96 unique_coreids=96/96 -[ASSERT] all PMU records have configured selectors and data PASS -[ASSERT] all 96 PMU physical subcore ids are unique PASS +#### 生成正式 PMU-only JSON + +正式 sidecar 使用单轮、独立进程和唯一输出路径: + +```bash +./run.sh build ccec + +OUT="./outputs/pmu_submit_all_$(date -u +%Y%m%dT%H%M%SZ)" +mkdir -p "$OUT" +./run.sh run ccec \ + --device 0 --batches 256 --runs 1 --no-swimlane \ + --pmu-window submit-all \ + --pmu-json "$OUT/pmu_submit_all.json" ``` -`trusted` 会逐核检查 MMIO 映射、三个 selector 和非零 total;因此不能用“外部 -profiler 已经启动”代替实际 selector 读回。AIC 与 AIV 分开输出,I-cache miss -rate 使用 `sum(miss) / sum(request)`,不平均逐核百分比。`pmu_scalar_busy` 是原始 -事件计数;empty 窗口中它可以大于 total,不能把两者比值直接宣传成利用率。 - -2026-07-18 的 A5 验证结果为: - -| 窗口 | 轮次 | 96 核 total 中位数 | 结论 | -| --- | ---: | ---: | --- | -| empty | 10 | 预热后约 419,轮间约 ±1 | 空窗口 gate 开销稳定 | -| scalar 100,000 | 10 | 预热后约 56,774,轮间约 ±2 | scalar/req/miss 均稳定响应 | -| scalar-double 2×100,000 | 10 | 预热后约 113,113,轮间约 ±8 | 扣除 empty 后为单段的 1.9997 倍,多 gate 可续积 | - -同一轮中,I-cache request 从 empty 的每核约 22 增至单段的约 26,245,双段约 -52,476;miss 也从每核约 4 增至约 23/45。双段十轮中 96 核均满足 -`trusted=96/96` 和物理子核 id 唯一,说明本链路能直接看到 scalar 和 I-cache -变化,也证明 stop/start 之间不读 counter 时,多段窗口会累计到末尾唯一一次 -snapshot。首次热身的 p95 偶有偏高,正式比较应丢弃首轮,并保留 A/A -重复性数据。 - -这里验证的是观察手段,不是 PA 优化本身。后续用于 Claim、EfDrain、 -WaitForSlot 或 HeapGuard 时,应先做一次 baseline read-clear,多个局部窗口之间只切 -gate,最后每核读取一次;禁止每个 Submit 都读 MMIO。当前双段只证明多窗口续积 -机制成立,扩展到上千个真实小窗口时仍要核对 gate 次数、empty 对照和 counter -是否溢出。empty 开销和诊断代码布局必须在 A/B 两边完全相同,真实 PA 的最终 -结论仍需撤回诊断代码后再跑原始泳道与 golden。 +`--pmu-json` 只允许 CCEC 的非 off 窗口并强制 `--runs 1`。正式 JSON 还要求 +`--no-swimlane`,且不能同时开启 `--trace-atomics`、`--profile-phases`、 +`--analyze-swimlane` 或 `--swimlane-json`。已有目标文件或同名 `.tmp` 会被拒绝, +不会静默覆盖。host 只在协议、96 核 PMU/owner 门禁、Restore 和 runtime 清理全部 +成功后,才把临时文件原子发布为最终 JSON。 + +`--no-swimlane` 会关闭 phase/atomic record 写和泳道后处理,但当前普通调度路径中的 +阶段 `SYS_CNT` 调用点仍存在;JSON 会显式记录“有 phase timestamp call、无 phase +record write、无 atomic trace、无 profile accumulation”。因此该模式是 PMU-only +采集口径,不等于编译期删除所有时间戳指令的零观察二进制。 + +JSON 包含: + +- `capture/configuration/validation`:采集边界、NOP 配置、`PIPE_ALL` 门控语义、九个 + selector、计数器位宽、实际 start/stop、96 条记录可信性及可编程 counter 风险门槛; +- `owner`:Main AICPU Path-A、配置 bitmap、AIC/AIV 数量、完整 triplet 和 Restore + 结果; +- `records`:每 worker 的物理子核 id、role、block/lane、原始 CNT0..8 和 total, + 以及本核 owner/slot/role、window started/stopped 断言; +- `summary.all/aic/aiv`:全部 96 核、32 个 AIC 和 64 个 AIV 分组后,各原始计数的 + `sum/mean/median/p95/max`。 + +控制台对应输出 `[PMU-ALL]`、`[PMU-AIC]`、`[PMU-AIV]`。I-cache miss rate 始终按 +`sum(icache_misses) / sum(icache_requests)` 计算,不平均逐核百分比。AIC 与 AIV +核数不同,比较每核强度时应看 mean/median 或 miss rate,不能直接比较两组 sum。 +`total_cycles` 是 PMU total 的原始值;96 核求和是 core-work,不是 Submit 墙钟 +时间,在没有额外核实其时钟/事件语义前不应直接换算成微秒。CNT0..CNT8 是 +32 bit,total 是 64 bit。正式门禁要求本轮最大可编程 counter 小于 `UINT32_MAX/4` +(25% 高水位),这只是缩短窗口后采用的保守风险阈值;最终 32-bit 值无法证明 +计数器没有恰好回卷一圈或多圈,因此通过该门禁也不能声称“已证明无回卷”。 + +正式观察保留三类互不混用的样本:关闭所有诊断的性能 golden、PMU-only +`submit-all` sidecar、Atomic-trace-only 泳道。PMU sidecar 只有整个窗口的每核累计, +没有可与单条 Atomic span 对齐的子窗口;不能把 AIC/AIV 平均 miss rate 回填成 +泳道中某条 atomic 的属性。优化前后需保持相同源码观察布局、NOP 和 owner 配置, +并用多个独立进程交错 A/B。 + +#### 2026-07-18 上板验收样本 + +自包含 owner 的本次验收中,`empty`、`scalar 100,000` 和 +`scalar-double 2×100,000` 的 96 核 total 中位数分别约为 214、56,568 和 +112,994;三个样本均为 `96/96 trusted`,owner 为 32 AIC + 64 AIV、32 个完整 triplet, +且每个样本 Restore PASS。它们是各模式的一次上板样本,不是多轮稳定性统计;只用于 +确认空窗底噪、scalar 正向响应和双段近似倍增。 + +同一版本的 `batches=1,nop-count=0,submit-all` 单次样本也通过 96 核 start/stop、 +selector、owner/slot/role/triplet 与 Restore 门禁:all/AIC/AIV 的 total 中位数约为 +36,066.5/28,708/39,745,96 核 scalar busy 求和约 2,661,612,I-cache request/miss +求和为 210,399/30,283(约 14.3931%),host `submit_span_us` 约 47.770。该样本只 +证明 `submit-all` 观察闭环可运行;`batches=1`、零模拟计算体和单次运行都不足以 +支持 256 batch 性能归因或真实 PA 绝对结论。 + +按上述正式命令和默认 PA NOP 还完成了 3 个独立进程的 256 batch +PMU-only 验收。Submit span 为 3,688.236/4,089.057/4,673.237 us,中位数 +4,089.057 us;I-cache request 总和为 69,812,583/69,451,706/70,065,443,miss +总和为 5,854,421/5,847,256/5,830,645,miss rate 为 8.3859%/8.4192%/8.3217%。 +三轮均通过 96 核、owner/slot/role/triplet、start/stop、counter 风险门槛和 Restore, +并使用本用户 `.venv` 从 raw 记录独立重算 summary 一致。这三轮证明的是 +同配置 PMU 取数可重复;由于 gate 包含 `PIPE_ALL` 且未与无 PMU 样本交错配对, +不应将它们与约 5 ms 无诊断基线直接相减。 ## 6. 当前 A5 结果与真实 PA 的差异 From e66001ff7fd08f37c93d70d7ad6b5df56c3b9c8e Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 12:09:38 +0000 Subject: [PATCH 016/214] =?UTF-8?q?=E6=96=B0=E5=A2=9E=20standalone=20CCEC?= =?UTF-8?q?=20=E7=9C=9F=E5=AE=9E=20Cube/Vector=20=E8=B4=9F=E8=BD=BD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 为 QK/PV 接入完整 128x128 Cube matmul 流水,为 SF/UP 接入 Vector add/mul 流水并等待 GM 写回完成 - 增加显式 real-compute 模式、分类型迭代次数、独立 workspace 与 active/sentinel 数值校验,默认仍保留三后端 scalar-nop 基线 - 将负载模式、次数、计算单元映射写入 PMU sidecar 与泳道元数据,并校验 Submit placement 和非零引擎计数闭合 - 加强 mixed ELF 入口与 LOCAL helper 构建门禁,补齐参数边界、转换器回归和中文复现文档 验证:CCEC scalar/real b1、real b256、b8 count1/count2 PMU、real b1 泳道、AscendC/CPU smoke、converter 5/5 均通过 --- ...05\345\206\265\345\210\206\346\236\220.md" | 106 +++- ...77\347\224\250\346\214\207\345\215\227.md" | 153 ++++- .../pa_scheduler/ascendc/pa_scheduler.asc | 14 +- tests/atomic_probe/pa_scheduler/ccec/build.sh | 53 +- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 564 ++++++++++++++++-- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 178 +++++- .../pa_scheduler/ccec/winner_workload.h | 57 ++ .../pa_scheduler/common/host_support.h | 37 +- .../pa_scheduler/common/pa_frontend.h | 5 +- .../pa_scheduler/common/pa_model.h | 55 +- .../pa_scheduler/common/pa_scheduler_core.h | 19 +- .../pa_scheduler/common/pa_trace.h | 6 +- tests/atomic_probe/pa_scheduler/cpu/main.cpp | 14 +- tests/atomic_probe/pa_scheduler/run.sh | 33 +- .../pa_scheduler/swimlane_converter.py | 76 ++- .../pa_scheduler/test_swimlane_converter.py | 41 ++ 16 files changed, 1286 insertions(+), 125 deletions(-) create mode 100644 tests/atomic_probe/pa_scheduler/ccec/winner_workload.h diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 876add4a61..c8c46989fc 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -337,9 +337,11 @@ span,或单独定义 `Submit.end - Build/Replay/Alloc.end` 为 action tail。 - EfDrain、Replay、WaitForSlot、HeapGuard、flag/vend/frontier 和最终 drain; - 单 lane Case1 的 BlockWon 动态次数为零,以及真实泳道记录格式。 -只有 QK/SF/PV/UP 的计算体由可控 NOP 模拟;NOP 默认值按本文最好真实泳道的 -44.170/53.729/27.626/1.565 us 校准。独立用例不会在 Claim、Register、 -PrepareMap 或等待路径中增加 NOP 来硬凑 5 ms。 +QK/SF/PV/UP 默认仍可由可控 scalar NOP 模拟,旧默认值按本文最好真实泳道的 +44.170/53.729/27.626/1.565 us 校准。2026-07-18 起 CCEC 另提供显式 +`real-compute`:QK/PV 运行真实 Cube matmul,SF/UP 运行真实 Vector add/mul, +每轮都含 GM load、计算、GM store 和完成等待;AscendC/CPU 尚按独立阶段迁移。 +两种模式都不会在 Claim、Register、PrepareMap 或等待路径中硬补 5 ms。 当前严格校验覆盖 73,728 次 Claim、每 task 唯一 winner、1,024 个 kernel、 TensorMap/heap 最终状态、fanin、flag、vend、frontier、cursor、ring placement @@ -869,10 +871,10 @@ calibrated_cores == 96/96 | 64 trials/core × 10 | 86.596(86.532~86.792)ns/miss | 85.913(85.848~86.202)ns/miss | 86.938(86.861~87.086)ns/miss | | 128 trials/core × 5 | 89.629(89.615~89.648)ns/miss | 92.100(91.984~92.267)ns/miss | 88.410(88.310~88.440)ns/miss | -总体系数观测范围为 86.532~89.648 ns/miss;同一时段重跑 64 和 128 trials -都约为 89.6 ns/miss。AIC/AIV 差值的方向在两组规模间改变,不建立跨时段 -的角色精确常数。几 ns 的变化属于并发环境下的有效 miss penalty 波动, -不应保留成伪精确常数。后续 scalar 分析使用下面的取整公式即可: +总体系数观测范围为 86.532~89.648 ns/miss;本表 64 和 128 trials 的中位数 +分别为 86.596 和 89.629 ns/miss。AIC/AIV 差值的方向在两组规模间改变,不建立 +跨时段的角色精确常数。几 ns 的变化属于并发环境下的有效 miss penalty 波动, +不应保留成伪精确常数;后续统一取 90 只用于量级估算: ~~~text T_icache_est_ns = CNT7_miss_total * 90 @@ -905,10 +907,10 @@ tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ 不再通过热路径内反复 stop/start 尝试扣除模拟计算体。 `metrics_prof_start/stop()` 自带 `PIPE_ALL` 屏障,门控会收口流水并可能改变多核 -到达时序。A/B 两侧必须保持相同 gate 次数、NOP 和代码布局,端到端收益仍由无 PMU、 -无泳道的独立进程交错 A/B 判断。standalone 的 QK/SF/PV/UP NOP 循环由 scalar -执行,会计入 scalar busy;它不是实际 Vector/Cube 计算,也不模拟真实 task 执行期间 -scalar 的等待状态。因此这里的 sidecar 只用于同配置调度分析,不能直接替代真实 PA。 +到达时序。A/B 两侧必须保持相同 gate 次数、winner mode/count 和代码布局,端到端 +收益仍由无 PMU、无泳道的独立进程交错 A/B 判断。`scalar-nop` 的 QK/SF/PV/UP +循环由 scalar 执行并计入 scalar busy;CCEC `real-compute` 则真实激活 +Cube/Vector/MTE/FIX。两种模式不能混比,且两者 sidecar 都不能直接替代真实 PA。 ### 7.4 阶段 D1:精确分类 fanin 与 frontier 动态原子次数 @@ -1108,7 +1110,7 @@ PA 的端到端 GAP 是“发布发射 -> 自然消费者首次观察到新值 duration 不直接包含自己的 record 写入;但这次写入、附加指令和代码布局 会影响后续 atomic 到达、竞争与 I-cache,整轮仍是插桩运行。 -开启逐 atomic trace 时,每个 worker 在最终 drain 之后额外记录一个 +开启逐 atomic trace 时,每个 worker 在最终 drain 之后额外记录两条 `ClockBaseline`:一条是连续两次 `get_sys_cnt()`,另一条是纯寄存器依赖 hook 后读 `SYS_CNT`。它们只给出同一二进制、同一物理核上两类边界的 计时分辨率和固定底噪分布,使用边界是: @@ -1143,8 +1145,8 @@ trace 容量不足时必须明确报 overflow 并判该轮观察无效,不能 CCEC 只保留一个正式逐 worker Submit 窗口,CPU/AscendC 对应 hook 是空实现,不伪造 PMU 数据:`submit-all` 在本 worker 通过启动屏障、完成 `ResetTraceLap` 后,于 orchestration 初始化前打开 gate,在该 worker 最后一次 Submit 返回后关闭。窗口包含 -参数构造、全部 Submit 调度,以及 standalone 用 scalar NOP 模拟的 QK/SF/PV/UP -计算体。 +参数构造、全部 Submit 调度,以及本 worker 在窗口内执行的 winner 计算体:默认模式 +为 scalar NOP,CCEC `real-compute` 模式为真实 Cube/Vector 与对应搬运流水。 它是“每核从 orchestration 到最后一次 Submit 返回”的累计窗口,不是全局最早 `Submit.start` 到最晚 `Submit.end` 的墙钟窗口,也不是逐次 Submit 窗口。 @@ -1153,8 +1155,8 @@ orchestration 初始化前打开 gate,在该 worker 最后一次 Submit 返回 直接一一对齐。 `metrics_prof_start/stop()` 都含 `PIPE_ALL` 屏障,门控边界会收口流水并可能改变 -worker 到达与争用时序。这个开销属于观察配置本身;只能比较 gate、NOP、构建布局 -完全相同的样本,不能把 PMU 样本的 Submit span 当成无观察性能基线。 +worker 到达与争用时序。这个开销属于观察配置本身;只能比较 gate、负载模式和构建 +布局完全相同的样本,不能把 PMU 样本的 Submit span 当成无观察性能基线。 PMU JSON sidecar 按 worker 保留 raw 记录,并分别汇总 32 AIC、64 AIV 和全部 96 核。 各字段严格按下面口径解释: @@ -1215,8 +1217,9 @@ PMU hook 和 CNT0..8 sidecar 已实现。2026-07-18 已用干净重编的 CCEC A 二进制完成 b1 atomic-trace-only 上板:全部协议断言 PASS,raw 共 4959 条、 `expected=4959`、`dropped=0`,其中 Atomic 1395 条、ClockBaseline 192 条。 Claim raw schema v2 也已直接闭环为 `won=5/lost=283/not_attempted=192`,不再 -借助 Atomic 记录反推 role-filtered Claim。converter 的直接脚本与 `python -m unittest` -两种入口均为 4/4 PASS。b1 的 Atomic 按边界闭环为 +借助 Atomic 记录反推 role-filtered Claim。当时 converter 原四项在直接脚本与 +`python -m unittest` 两种入口均为 4/4 PASS;增加 winner workload metadata +回归后,当前完整集合为 5/5 PASS。b1 的 Atomic 按边界闭环为 `return_ready=1193/source_issue=202`。本轮未入库产物为: ~~~text @@ -1245,7 +1248,7 @@ outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b256/merged_swim frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 ms 插桩轮 不能与三轮中任一轮做单样本减法后归因 trace 成本。 -仍未完成的正式重采如下: +正式重采状态如下: | 结果 | 当前状态 | | ---- | -------- | @@ -1254,6 +1257,7 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 | 192 条 ClockBaseline 的 AIC/AIV 分布 | **b1/256 batch 已闭环** | | `submit-all` 的 owner/gate/CNT0..8/total 闭环 | **b1、零 NOP 单次上板已通过** | | 256 batch `submit-all` 的 I-cache 与 pipe 分组分布 | **3 个 PMU-only 独立进程已采并完成 raw→summary 重算** | +| real-compute 引擎 PMU 与 placement | **b8 count1/count2 精确倍增并逐 worker 闭合** | 此前开发过程中的探索性输出不在这里引用为正式结论。最终按同一源码依次执行: @@ -1261,7 +1265,10 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 2. CCEC atomic-trace-only,检查上述计数闭环、ClockBaseline 和 raw→merged; 3. CCEC PMU-only 的 `submit-all` 已完成 3 个独立进程 A/A;后续每轮仍使用 唯一 JSON 路径,并检查 96 核、owner bitmap/triplet、start/stop、25% 风险门槛和 Restore; -4. 比较任何 atomic 优化前后时,双方使用相同观察模式、相同 gate/trace 配置和 +4. real-compute b8 count1/count2 已完成 Cube/Vector busy 精确倍增与 + `EfDrain` placement 闭合; +5. 比较任何 atomic 优化前后时,双方使用相同观察模式、相同 winner mode/count、 + 相同 gate/trace 配置和 独立进程交错 A/B;正式端到端 Submit 仍以无诊断构建复测。 #### 7.5.7 绝不能从 O2 声称的结论 @@ -1276,8 +1283,9 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 - trace 开启后的 Submit span 可以直接与无 trace 基线比较并宣布性能收益; - `scalar_busy/total_cycles` 是墙钟时间,或 I-cache miss 数可以直接换算成精确 stall ns;7.3.3 的 90 ns/miss 只是受控 cold/warm 探针得到的一阶等效估算; -- standalone 的 scalar NOP 已经真实模拟了 vector/cube task 期间 scalar 的等待状态; - NOP 本身在 scalar 上执行并被 `submit-all` 统计,`PIPE_ALL` gate 还会收口流水; +- standalone 的 `scalar-nop` 已经真实模拟了 vector/cube task 期间 scalar 的等待状态; + NOP 本身在 scalar 上执行并被 `submit-all` 统计。CCEC `real-compute` 虽已真实激活 + Cube/Vector/MTE/FIX,也仍不能由此推导完整真实 PA 的 scalar 等待与资源竞争; - 最大 32-bit counter 低于 25% 风险门槛就证明本轮没有回卷;该门槛只降低风险, 最终寄存器值不能直接检出恰好一圈或多圈的 wrap; - standalone 的 PMU/atomic 分布可以直接替代真实 FDWIC PA。迁移真实用例时仍需 @@ -1319,3 +1327,57 @@ I-cache 和 pipe 分组基线;它显示 AIV miss rate 在本三轮持续高于 counter 仍不能把 miss 定位到 materialize/register 或某条 atomic。开始改 Claim 后, 应使用同配置的 PMU-only 交错 A/B 观察 request/miss 是否同步变化,端到端收益 仍由无诊断的独立进程 A/B 确认。 + +#### 7.5.9 CCEC winner 负载从 scalar NOP 迁移到真实 Cube/Vector + +为避免 winner 执行期的 scalar NOP 污染 `scalar_busy`,2026-07-18 先在 +standalone CCEC 增加显式 `--winner-workload real-compute`,没有迁移真实 PA。 +整体无参数默认仍为 `scalar-nop`,保证旧三后端基线不静默变化;选择真计算而未 +指定次数时,QK/SF/PV/UP 使用 `6,28,4,1`。 + +实现与门禁如下: + +1. QK/PV 只在 AIC 执行完整 `128x128 float` Cube matmul,包含 + MTE2/MTE1、M、FIX、GM store 和最终完成等待; +2. SF/UP 只在 AIV 执行完整 Vector add/mul,包含 MTE2、V、MTE3、GM store + 和最终完成等待;每次 repeat 完成写回后才复用 tile; +3. 两个 GM 输入 tile 为所有 worker 只读共享,每 worker、每角色使用独占输出 + tile。host 在计时外初始化/传输,计时后逐 tile 验证 768/5/6 与 inactive + sentinel,共 12,713,984 bytes; +4. 最终 device ELF 严格限制为两个 mixed kernel GLOBAL 入口;冷路径 dispatcher + 与 Cube/add/mul 三个执行 helper 必须是非空 LOCAL 函数。这个门禁来自一次已 + 复现故障:错误暴露的 GLOBAL helper + 被 runtime 当入口启动,导致 scalar 模式也进入 Cube 路径;修正后同一 ELF 的 + scalar b1 与 real b1 均通过; +5. repeats 上限为 128,避免 b256 极端 winner 分布让 32-bit PMU 计数接近回卷 + 区域。真计算 count 与 NOP count 互斥,0 次被拒绝。 + +标定不是以凑齐 5.1 ms 为目标。三个独立 b256 进程的 QK/SF/PV 中位数约为 +41.336/54.039/27.971 us,最接近真实泳道目标 44.170/53.729/27.626 us; +UP 一次完整流水约 2.5 us,已经是当前正整数下限。三轮 Submit 为 +3.808/3.555/3.706 ms,中位数 3.706 ms;最终重建后的单轮为 3.683649 ms。 +Cube/Vector 分布在物理子核并行执行,与 scalar NOP 串行占用 scalar 的到达时序 +本来不同,不能通过增加无关 repeat 把总时间硬拉回 5.1 ms。 + +最终常量 tile 只证明某个 active worker-kind 至少完成一次,因为同一 tile 会被 +后续 repeat 覆盖;全部 repeat 的证据来自下面的受控 PMU 倍增,而不是数值结果本身。 + +`submit-all` PMU 做了独立倍增取证。b8、四类 count=1 时,窗口内 placement 为 +29 个 EfDrain 和 3 个 FinalDrain;恰有 14 个 AIC worker 的 Cube 非零且逐核 +`cube_busy=8281`,15 个 AIV worker 的 Vector 非零且逐核 +`vector_busy=936/937`,与 29 个 EfDrain 精确闭合。独立 count=2 样本为 +28 个 EfDrain、4 个 FinalDrain,14 个 AIC 与 14 个 AIV 非零计数档位分别为 +16562 和 1872/1874,精确两倍。获胜 worker 会随调度变化,不能把两轮按同一核 +强配对。FinalDrain 位于每核 PMU stop 之后,故 count1/count2 的 3/4 个 +FinalDrain kernel 不应出现在 Submit sidecar;这不是漏计。 + +泳道 raw/merged 的 `trace_schema_version` 仍为 2;本轮只扩展可选 metadata, +没有把它误升为 PMU JSON 的 schema v3。metadata 同时保存 mode、四类 count、迭代单位与 +QK/PV=Cube、SF/UP=Vector 映射,并增加可见的全局 capture metadata 事件。 +最终 b1 real-compute 泳道有 4964 条 raw data event;converter 产出 4965 条 +data event(增加一条 capture instant),再加 256 条 process/thread metadata, +最终 `traceEvents` 为 5221 条,`dropped=0`。转换器含旧 schema 兼容在内为 5/5 PASS。 + +阶段边界:本节只证明 CCEC standalone 的真实引擎负载、数值、角色、PMU 与 +泳道闭环。AscendC 真实计算、CPU 等价算术必须继续按后端分步实现和验证;在它们 +完成前,不能宣称三后端 winner 负载已经对等,更不能直接迁移真实 PA。 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 20af554d4c..7b3f9f355a 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -28,10 +28,10 @@ frontier 和 worker 状态,任一不符都会返回失败。 有意保留的替代只有两类: -1. QK/SF/PV/UP 的真实计算体由可控 NOP 数做时长标定,使每个 task 的 - 占用时间接近真实 PA;调度前后路径没有用 NOP 补时。NOP 实际由 - scalar 执行,只校准了时长,没有复现真实 Vector/Cube 计算单元及其间 - scalar 等待状态。 +1. 默认 `scalar-nop` 模式仍以可控 NOP 标定 QK/SF/PV/UP,供三后端沿用旧基线。 + CCEC 另有显式 `real-compute` 模式:QK/PV 运行完整 Cube matmul,SF/UP + 运行完整 Vector add/mul,并执行 GM load、引擎计算、GM store 和完成等待。 + AscendC 与 CPU 的对等真计算仍按阶段迁移,当前不能把 CCEC 结果泛化到三后端。 2. simpler 的 AICPU/runtime 装载链路由本目录 host runner 代替;测试关注的 首个 Submit 到最后一个 Submit 区间不含 AICPU 初始化和最终回收。 @@ -45,20 +45,22 @@ Register、PrepareMap 或等待路径中插入虚假延时。 ## 2. 三种实现 三种后端共用 `common/` 中完全相同的 PA 模型和调度器,只分别实现原子指令、 -时钟、NOP 和启动入口。 +时钟、winner 计算体和启动入口。 -| 后端 | 启动形式 | atomic load | Claim fetch-max | -| --- | --- | --- | --- | -| CCEC | 1:2 mixed AIC/AIV ELF | `atomicAdd(addr, 0)` | `atomicMax` | -| AscendC | `__mix__(1, 2)` | `AtomicAdd(addr, 0)` | `AtomicMax` | -| CPU | 96 个 pthread | `fetch_add(0)` | C++17 CAS loop | +| 后端 | 启动形式 | atomic load | Claim fetch-max | 当前 winner 负载 | +| --- | --- | --- | --- | --- | +| CCEC | 1:2 mixed AIC/AIV ELF | `atomicAdd(addr, 0)` | `atomicMax` | `scalar-nop` 或真实 Cube/Vector | +| AscendC | `__mix__(1, 2)` | `AtomicAdd(addr, 0)` | `AtomicMax` | 当前为 `scalar-nop` | +| CPU | 96 个 pthread | `fetch_add(0)` | C++17 CAS loop | 当前为 `scalar-nop` | AscendC 对 64 位 vend 使用 signed `AtomicAdd(addr, 0)`。CANN 9.1 虽提供 unsigned overload,但它在本 mixed kernel 的 64 MiB heap wrap 位置发生过 稳定停滞;PA vend 小于 `INT64_MAX`,因此 signed add-zero 返回的位模式与比较 语义不变。CPU 版本用于协议和边界检查,host 线程调度耗时不能与 A5 比较。 -## 3. 默认 kernel 时间 +## 3. Winner 计算负载 + +### 3.1 三后端默认 `scalar-nop` 真实 PA 最好泳道中四类 kernel 的 1 GHz counter 均值和当前 A5 NOP 校准值为: @@ -80,6 +82,44 @@ Alloc 没有模拟 kernel body。NOP 数是 A5 实测校准量,不应解释为 `--nop-count N` 同时设置四类 kernel;`--nop-counts` 的顺序固定为 QK、SF、PV、UP,允许范围为 0 到 10,000,000。 +### 3.2 CCEC 真实 Cube/Vector 模式 + +CCEC 可显式切换到真实计算: + +```bash +./run.sh run ccec --winner-workload real-compute --batches 256 --runs 1 --no-swimlane +``` + +未传 count 时,QK/SF/PV/UP 默认使用 `6,28,4,1` 次完整迭代。每次迭代均为 +`128x128 float` 的完整流水,不是 scalar NOP: + +- AIC 的 QK/PV:GM load → MTE2/MTE1 → Cube matmul → FIX → GM store → 完成等待; +- AIV 的 SF:GM load → Vector add → GM store → 完成等待; +- AIV 的 UP:GM load → Vector mul → GM store → 完成等待。 + +可以统一或分类型覆盖,范围为 1 到 128: + +```bash +./run.sh run ccec --winner-workload real-compute --real-compute-count 1 +./run.sh run ccec --winner-workload real-compute --real-compute-counts 6,28,4,1 +``` + +`--real-compute-count*` 与 `--nop-count*` 不能混用;当前这些选项只允许 CCEC。 +workspace 包含两个只读输入 tile,并为每个 worker 按其 role 对应的两个 task kind +各保留一个独占输出 tile,共 +12,713,984 bytes。host 在计时前初始化并 H2D,计时后 D2H;所有 active +worker-kind 的最终 tile 必须分别等于 QK/PV 的 768、SF 的 5、UP 的 6,未获胜 +输出必须保留 sentinel。同一 worker-kind 的 repeat 会覆盖同一 tile,因此最终常量 +结果只证明至少完成一次;全部 repeat 的完整性另由受控 PMU count1→2 倍增取证。 +输入使用常量 2 和 3,足以验证计算、角色路由和写回,但不单独证明转置或 stride +布局;这是一项有意保留的性能替身边界。 + +默认次数来自三个独立 b256 进程的标定:QK、SF、PV 中位数分别约 +41.336、54.039、27.971 us,接近真实 PA 的 44.170、53.729、27.626 us。 +UP 的一次完整 `128x128` 流水约 2.5 us,已经是正整数迭代下限;若后续要贴近 +1.565 us,应缩小 UP tile,不能用 0 次掩盖执行。真实计算下 Cube/Vector 会在 +不同物理子核并行,因此整体 Submit 约 3.6~3.8 ms,不应增加无关工作量硬凑 5.1 ms。 + ## 4. 本机依赖和构建 CCEC 与 AscendC 使用本用户安装的 CANN 9.1。非交互 shell 不保证读取 @@ -101,7 +141,8 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" 组合构建严格按 CCEC、AscendC、CPU 的顺序执行。CCEC 构建会检查 1:2 mixed 的两个入口和 metadata section;CANN 9.1 自带的 PTO 头可直接使用。若换用单独 安装的 PTO ISA,可把 `PTO_ISA_ROOT` 指向包含 -`include/pto/common/kernel_meta.hpp` 的目录。 +`include/pto/common/kernel_meta.hpp` 的目录;同一 include tree 还必须具有 +`pto/pto-inst.hpp`、`pto/common/constants.hpp` 和 `pto/common/pto_tile.hpp`。 ## 5. 使用说明:运行、测量与泳道查看 @@ -111,12 +152,15 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" | action | 用途 | 是否生成泳道文件 | | --- | --- | --- | | `build` | 构建指定后端 | 否 | -| `smoke` | 1 batch、1 run、零 NOP 的快速语义回归 | 否,只做内存记录校验 | -| `run` | 自行控制 batch、run、NOP 和诊断参数 | 仅显式传入 `--swimlane-json` 时生成 raw | +| `smoke` | 固定 b1/r1/`scalar-nop=0` 的快速语义回归 | 否,只做内存记录校验 | +| `run` | 自行控制 batch、run、winner 负载和诊断参数 | 仅显式传入 `--swimlane-json` 时生成 raw | | `swimlane` | 单轮运行并自动生成 raw 和 Perfetto merged JSON | 是 | `ccec|ascendc|cpu|all` 用于选择后端;`all` 始终按 CCEC、AscendC、CPU 的顺序执行。 +CCEC `run/swimlane` 可使用 `--winner-workload scalar-nop|real-compute`、 +`--real-compute-count N` 或 `--real-compute-counts QK,SF,PV,UP`;真计算 count +与 `--nop-count*` 互斥。`smoke` 有意固定 scalar-nop,不接受这些覆盖项。 ### 5.1 首次回归 @@ -149,6 +193,9 @@ semantic_status=PASS postprocess_status=PASS --profile-phases --analyze-swimlane ``` +这两条未传 `--winner-workload`,都使用默认 scalar-NOP;需要 CCEC 真计算时按 +3.2 节显式传入 `--winner-workload real-compute`。 + 此模式输出指标和泳道统计,但不会自动落盘 JSON。若只关注性能且不需要完整 泳道逐事件分析,可去掉 `--analyze-swimlane`;若也不需要记录泳道,可使用 `--no-swimlane` 进一步节省约 384 MiB device 内存。 @@ -160,6 +207,9 @@ semantic_status=PASS postprocess_status=PASS ```bash ./run.sh swimlane ccec \ --device 0 --batches 256 --profile-phases --analyze-swimlane + +./run.sh swimlane ccec \ + --device 0 --batches 1 --winner-workload real-compute --trace-atomics ``` `swimlane` action 会管理 `--runs 1` 和输出路径,因此不要再传 @@ -185,6 +235,9 @@ runner 结束时会打印准确目录: [SWIMLANE] output_root=.../outputs/pa_scheduler_swimlane__ ``` +真计算泳道必须同时检查 raw/merged 顶层的 `winner_workload` metadata;逻辑 +`·kernel` span 只说明 winner 执行区间,单凭轨道名称不能证明使用了硬件引擎。 + 查看步骤: 1. 打开 ; @@ -447,18 +500,17 @@ start/stop、`miss <= request`,以及 owner Restore 成功。 | `scalar` | `RunScheduler` 完成后,一个 gate 中执行 `--pmu-scalar-nops N` | 验证 scalar/I-cache 正向敏感性 | | `scalar-double` | `RunScheduler` 完成后,两段相同 NOP 中间只 stop/start、不读 counter | 验证暂停后继续累计 | | `icache-single` | `RunScheduler` 完成后,每 worker 做 cold/warm 配对试验;64 KiB sweep 或目标预热在目标 gate 外 | 标定受控单次 CNT7 miss 的一阶等效时间 | -| `submit-all` | 每 worker 通过启动屏障后,在首次 PA 参数构造前 start,最后一次 Submit 返回后 stop | 累计 orchestration、Submit 和模拟 kernel NOP | +| `submit-all` | 每 worker 通过启动屏障后,在首次 PA 参数构造前 start,最后一次 Submit 返回后 stop | 累计 orchestration、Submit,以及本 worker 在窗口内执行的 NOP 或真实引擎计算 | `submit-all` 不含启动屏障,也不含 `replay_done`、最终 drain、末尾 ClockBaseline 或 PMU 结果发布。它是 **每 worker 从 orchestration 初始化前到本核最后一次 Submit 返回后的累计窗口**,不是全局“最早 `Submit.begin` 到最晚 `Submit.end`”墙钟 span。 每次 `metrics_prof_start/stop()` 都带 `PIPE_ALL` 流水屏障。该屏障用于明确门控边界, -也会收口流水并可能改变多核到达时序;因此 PMU 样本只能与相同 gate、相同 NOP、 -相同构建配置的样本比较,不能把它当成无观察开销的端到端基线。当前模拟 QK/SF/PV/UP -的 NOP 循环实际在 scalar 上执行,`submit-all` 会把它计入 scalar 相关事件;这与真实 -Vector/Cube task 运行时 scalar 等待的状态并不等价,所以 sidecar 不声称是绝对的 -真实 PA profile。 +也会收口流水并可能改变多核到达时序;因此 PMU 样本只能与相同 gate、相同负载模式、 +相同构建配置的样本比较,不能把它当成无观察开销的端到端基线。`scalar-nop` 的循环 +实际在 scalar 上执行并计入 scalar 事件;CCEC `real-compute` 则真实激活 Cube/Vector +及其搬运流水。两种口径不能混比,且 standalone sidecar 都不声称是完整真实 PA profile。 `icache-single` 的可执行标定命令为: @@ -495,6 +547,16 @@ mkdir -p "$OUT" --pmu-json "$OUT/pmu_submit_all.json" ``` +上述命令没有选择 winner 模式,采集的是默认 `scalar-nop`。验证真实引擎计数时 +必须显式写出模式和次数,例如已用于 count 倍增取证的 b8 命令: + +```bash +./run.sh run ccec \ + --device 0 --batches 8 --runs 1 --no-swimlane \ + --winner-workload real-compute --real-compute-count 1 \ + --pmu-window submit-all --pmu-json "$OUT/pmu_real_b8_count1.json" +``` + `--pmu-json` 只允许 CCEC 的非 off 窗口并强制 `--runs 1`。正式 JSON 还要求 `--no-swimlane`,且不能同时开启 `--trace-atomics`、`--profile-phases`、 `--analyze-swimlane` 或 `--swimlane-json`。已有目标文件或同名 `.tmp` 会被拒绝, @@ -508,8 +570,9 @@ record write、无 atomic trace、无 profile accumulation”。因此该模式 JSON 包含: -- `capture/configuration/validation`:采集边界、NOP 配置、`PIPE_ALL` 门控语义、九个 - selector、计数器位宽、实际 start/stop、96 条记录可信性及可编程 counter 风险门槛; +- `capture/configuration/validation`:采集边界、winner mode/count/unit/角色引擎映射、 + `PIPE_ALL` 门控语义、九个 selector、计数器位宽、实际 start/stop、96 条记录可信性 + 及可编程 counter 风险门槛;real-compute 还记录数值输出与 placement/引擎闭合门禁; - `owner`:Main AICPU Path-A、配置 bitmap、AIC/AIV 数量、完整 triplet 和 Restore 结果; - `records`:每 worker 的物理子核 id、role、block/lane、原始 CNT0..8 和 total, @@ -529,7 +592,8 @@ JSON 包含: 正式观察保留三类互不混用的样本:关闭所有诊断的性能 golden、PMU-only `submit-all` sidecar、Atomic-trace-only 泳道。PMU sidecar 只有整个窗口的每核累计, 没有可与单条 Atomic span 对齐的子窗口;不能把 AIC/AIV 平均 miss rate 回填成 -泳道中某条 atomic 的属性。优化前后需保持相同源码观察布局、NOP 和 owner 配置, +泳道中某条 atomic 的属性。优化前后需保持相同源码观察布局、winner mode/count 和 +owner 配置, 并用多个独立进程交错 A/B。 #### 2026-07-18 上板验收样本 @@ -567,7 +631,8 @@ PMU-only 验收。Submit span 为 3,688.236/4,089.057/4,673.237 us,中位数 | 64 trials/core × 10 | 86.596(86.532~86.792)ns/miss | 85.913(85.848~86.202)ns/miss | 86.938(86.861~87.086)ns/miss | | 128 trials/core × 5 | 89.629(89.615~89.648)ns/miss | 92.100(91.984~92.267)ns/miss | 88.410(88.310~88.440)ns/miss | -同一时段重跑 64 与 128 trials 时,ALL 都约为 89.6 ns/miss。64-trial 样本中 +同一时段的 64 与 128 trials,ALL 分别约为 86.6 与 89.6 ns/miss。统一取 90 +只是便于总量级归因的保守取整,不是把两组实测改写成同一精确常数。64-trial 样本中 AIV 略高,128-trial 样本中则 AIC 更高;角色差值方向并不稳定,不建立 AIC/AIV 精确常数。只做总量级归因时,统一取整为: @@ -637,13 +702,47 @@ HeapGuard 只有 0 至 3 次偶发等待。当前主要 可靠的 PA 语义模拟。当前选择是保持源级协议和 ABI 对等,坦诚记录约 0.1 至 0.4 ms 的后端/冷热差异,不用虚假 NOP 填平调度阶段。 +### 6.1 2026-07-18 CCEC 真实计算阶段结果 + +CCEC `real-compute` 已按“构建门禁 → b1 数值 → b256 标定 → PMU 倍增 → +泳道元数据”的顺序上板闭环: + +- 最终 b256 默认 `6,28,4,1` 的 Submit 为 3,683.649 us,全部协议与 + 192 个 active output tile 数值断言通过;此前三个独立进程 Submit 为 + 3,808/3,555/3,706 ms,中位数 3.706 ms; +- b8、四类 count=1 时,Submit 窗口内 29 个 EfDrain 恰好对应 14 个非零 + AIC Cube worker 和 15 个非零 AIV Vector worker;AIC 每个 `cube_busy=8281`, + AIV 每个 `vector_busy=936/937`; +- 独立 count=2 样本的 placement 为 28 个 EfDrain、4 个 FinalDrain,14 个 AIC + 和 14 个 AIV worker 的非零计数档位分别为 16562 与 1872/1874,恰为 count=1 + 档位的两倍。获胜核会随调度变化,不能把两轮强行按同一 worker 配对; +- `submit-all` 明确排除 FinalDrain,所以 count1/count2 分别落在 FinalDrain 的 + 3/4 个 kernel 不应出现在该窗口的引擎 PMU 中。每轮都由 placement 与非零引擎 + worker 数量闭合; +- scalar-NOP 与 real-compute 使用同一个最终 ELF 分别跑 b1,避免把代码布局变化 + 误判为负载效果;最终 ELF 只暴露两个 mixed kernel 全局入口,冷路径 dispatcher + 以及 Cube/add/mul 三个执行 helper 均为非空 LOCAL 函数; +- real-compute b1 泳道 raw/merged 均记录 mode、`6,28,4,1`、完整迭代单位和 + QK/PV=Cube、SF/UP=Vector 的映射;4964 条 raw data event 转换为 4965 条 + data event(多一条 capture instant),最终 `traceEvents` 还含 256 条 + process/thread metadata,共 5221 条,且无 dropped record。 + +这组结果证明 standalone CCEC 的显式 `real-compute` 模式已不再用 scalar NOP +冒充 winner 计算,并能从 +数值、角色、PMU 和泳道四个方向闭环。它不证明 AscendC/CPU 已迁移,也不证明 +standalone 的 3.7 ms 应等于真实 PA 的 5.1 ms;真实引擎并行正是两者时序口径 +发生变化的原因之一。 + ## 7. 内存占用和脱仓复制 为保持真实 DistGlobal/DistCore 偏移、65,536 个 task cell、每 worker payload 和 TensorMap,当前 `WorkerResult` 为 832 bytes,用当前头文件实际编译得到的 -`SchedulerState` 为 1,007,104,832 bytes。默认泳道缓冲区另占 -402,660,160 bytes,所以 A5 device 侧总计约 1.31 GiB,host 侧也需分配相近 -内存。`smoke` 不缩小 State;只有 `--no-swimlane` 能省去泳道缓冲区。 +`SchedulerState` 为 1,007,104,896 bytes。新增的 64 bytes 是独立、对齐的 +winner workload 配置 cache line,生产 DistGlobal/DistCore 关键偏移保持不变。 +默认泳道缓冲区另占 402,660,160 bytes;CCEC `real-compute` 还分配 +12,713,984 bytes workspace。因此 scalar-nop+trace 的 A5 device 占用约 +1.313 GiB,real-compute+trace 约 1.325 GiB,host 侧也需分配相近内存。 +`smoke` 不缩小 State;只有 `--no-swimlane` 能省去泳道缓冲区。 256 batch 的正常采集约有 86 万条事件;原始 JSON 和 merged JSON 都可能达到 数十至数百 MiB。writer 与 converter 都使用临时文件后原子替换,失败时不会把 半截文件冒充完整产物。 diff --git a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc index 94f09dc875..d55ac10eff 100644 --- a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc +++ b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc @@ -122,7 +122,12 @@ struct AscendcOps { return Now(); } - __aicore__ static inline void Nop(uint32_t count) { RuntimeNop(count); } + __aicore__ static inline void ExecuteKernel( + __gm__ pa_scheduler::SchedulerState *, __gm__ pa_scheduler::WorkerState &, + pa_scheduler::TaskKind, uint32_t nop_count + ) { + RuntimeNop(nop_count); + } // 当前直接 PMU MMIO 观察链路只在 CCEC 分支验证;AscendC 保持公共 hook // 的空实现,避免静默产出一套未核实寄存器读法。 @@ -351,7 +356,12 @@ int32_t main(int32_t argc, char **argv) { break; } if (!pa_scheduler::host::ExportSwimlaneRecords( - trace_header, options.swimlane_json, read_trace_records + trace_header, options.swimlane_json, + pa_scheduler::WinnerWorkloadMode::ScalarNop, + pa_scheduler::WorkloadCounts{ + options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + }, + read_trace_records )) { postprocess_ok = false; break; diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh index 1e49d86704..1428f6568d 100755 --- a/tests/atomic_probe/pa_scheduler/ccec/build.sh +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -42,6 +42,12 @@ if [[ ! -f "$PTO_INCLUDE_ROOT/include/pto/common/kernel_meta.hpp" ]]; then echo "PTO kernel metadata header is missing under $PTO_INCLUDE_ROOT/include" >&2 exit 1 fi +for header in pto/pto-inst.hpp pto/common/constants.hpp pto/common/pto_tile.hpp; do + if [[ ! -f "$PTO_INCLUDE_ROOT/include/$header" ]]; then + echo "PTO real-compute header is missing: $PTO_INCLUDE_ROOT/include/$header" >&2 + exit 1 + fi +done mkdir -p "$BUILD_DIR" @@ -82,13 +88,15 @@ echo "[BUILD] Static 1:2 mixed AICore ELF" "$BUILD_DIR/pa_scheduler_aic.o" \ "$BUILD_DIR/pa_scheduler_aiv.o" -SYMBOL_TABLE="$("$READELF_BIN" --symbols --wide "$BUILD_DIR/pa_scheduler_kernel.o")" +SYMBOL_TABLE="$("$READELF_BIN" --symbols --wide --sym-base=10 "$BUILD_DIR/pa_scheduler_kernel.o")" SECTION_TABLE="$("$READELF_BIN" --sections --wide "$BUILD_DIR/pa_scheduler_kernel.o")" # 构建成功不等于 mixed launch 可用:同时检查两个入口符号及其 metadata section,缺一即拒绝产物。 # `set -e` 同时保证 readelf 自身失败时不会拿空字符串继续做伪检查。 for entry in pa_scheduler_0_mix_aic pa_scheduler_0_mix_aiv; do - if [[ "$SYMBOL_TABLE" != *" $entry"* ]]; then - echo "Missing mixed-kernel entry: $entry" >&2 + if ! awk -v name="$entry" \ + '$4 == "FUNC" && $5 == "GLOBAL" && $7 != "UND" && $NF == name && $3 != "0" && $3 != "0x0" {found = 1} END {exit !found}' \ + <<<"$SYMBOL_TABLE"; then + echo "Missing non-empty defined GLOBAL mixed-kernel entry: $entry" >&2 exit 1 fi if [[ "$SECTION_TABLE" != *".ascend.meta.$entry"* ]]; then @@ -98,6 +106,45 @@ for entry in pa_scheduler_0_mix_aic pa_scheduler_0_mix_aiv; do done echo "[CHECK] both 1:2 mixed entries and metadata sections are present" +# A5 runtime 会把已定义的 GLOBAL FUNC 当作可启动候选;最终 device ELF 只允许 +# 两个带 metadata 的 mixed 入口暴露为全局函数。任何新增 helper 都必须保持 LOCAL。 +while IFS= read -r global_func; do + case "$global_func" in + pa_scheduler_0_mix_aic|pa_scheduler_0_mix_aiv) ;; + *) + echo "Unexpected GLOBAL device function (possible kernel-entry pollution): $global_func" >&2 + exit 1 + ;; + esac +done < <(awk '$4 == "FUNC" && $5 == "GLOBAL" && $7 != "UND" {print $NF}' <<<"$SYMBOL_TABLE") +echo "[CHECK] only the two mixed entries are exported as GLOBAL device functions" + +# noinline/used 的三个本地函数是 CCEC 真计算模式的构建期证据;它们不得导出为 +# GLOBAL,否则 runtime 可能把 helper 误识别成可启动 kernel。运行期还必须用 +# 数值闭环和 PMU 的 AIC cube_busy/AIV vector_busy 共同证明它们确实执行。 +for workload_symbol in \ + pa_execute_real_winner_workload \ + pa_real_cube_workload_aic \ + pa_real_vector_add_workload_aiv \ + pa_real_vector_mul_workload_aiv; do + workload_size="$( + awk -v name="$workload_symbol" \ + '$4 == "FUNC" && $5 == "LOCAL" && $7 != "UND" && index($NF, name) != 0 && $3 + 0 > 0 {print $3; exit}' \ + <<<"$SYMBOL_TABLE" + )" + if [[ -z "$workload_size" ]]; then + echo "Missing non-empty LOCAL CCEC real-compute workload function: $workload_symbol" >&2 + exit 1 + fi + if awk -v name="$workload_symbol" \ + '$4 == "FUNC" && $5 == "GLOBAL" && $7 != "UND" && index($NF, name) != 0 {found = 1} END {exit !found}' \ + <<<"$SYMBOL_TABLE"; then + echo "CCEC real-compute helper must not be a GLOBAL kernel candidate: $workload_symbol" >&2 + exit 1 + fi +done +echo "[CHECK] CCEC cube/vector real-compute helpers are non-empty LOCAL functions" + check_icache_probe_layout() { local role="$1" local target="pa_icache_target_${role}" diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index 04f684484b..d7f9054b3b 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -12,6 +12,7 @@ #include "../common/host_support.h" #include "pmu_owner_host.h" #include "pmu_probe.h" +#include "winner_workload.h" #include "acl/acl.h" #include "driver/ascend_hal.h" @@ -45,6 +46,31 @@ bool CheckRt(rtError_t error, const char *label) { return false; } +class ScopedAclDeviceAllocation { +public: + ScopedAclDeviceAllocation() = default; + ScopedAclDeviceAllocation(const ScopedAclDeviceAllocation &) = delete; + ScopedAclDeviceAllocation &operator=(const ScopedAclDeviceAllocation &) = delete; + + ~ScopedAclDeviceAllocation() { + // 早退路径没有机会汇入末尾 cleanup;这里只负责尽力释放本类新增的 + // real-compute workspace。正常路径会先 Release,再检查 aclrtFree 返回值。 + if (pointer_ != nullptr) (void)aclrtFree(pointer_); + } + + void **Address() { return &pointer_; } + void *Get() const { return pointer_; } + + void *Release() { + void *pointer = pointer_; + pointer_ = nullptr; + return pointer; + } + +private: + void *pointer_ = nullptr; +}; + std::vector ReadBinary(const std::string &path) { // ELF 整体保存在 vector 中直到 runtime 卸载完成,保证 rtDevBinary_t.data 在整个注册生命周期内有效。 std::ifstream file(path, std::ios::binary | std::ios::ate); @@ -64,6 +90,140 @@ struct PmuOptions { std::string json_path; }; +struct WinnerWorkloadOptions { + pa_scheduler::WinnerWorkloadMode mode = pa_scheduler::WinnerWorkloadMode::ScalarNop; + pa_scheduler::WorkloadCounts repeats = pa_scheduler::ccec_workload::kDefaultRealComputeCounts; + bool counts_explicit = false; + bool nop_override_explicit = false; +}; + +const char *WinnerWorkloadModeName(pa_scheduler::WinnerWorkloadMode mode) { + switch (mode) { + case pa_scheduler::WinnerWorkloadMode::ScalarNop: + return "scalar-nop"; + case pa_scheduler::WinnerWorkloadMode::RealCompute: + return "real-compute"; + } + return "invalid"; +} + +bool ParseWorkloadCounts(const char *raw, pa_scheduler::WorkloadCounts *counts) { + unsigned int qk = 0; + unsigned int sf = 0; + unsigned int pv = 0; + unsigned int up = 0; + char tail = '\0'; + if (std::sscanf(raw, "%u,%u,%u,%u%c", &qk, &sf, &pv, &up, &tail) != 4) return false; + const uint32_t maximum = pa_scheduler::ccec_workload::kMaxRealComputeCount; + if (qk == 0 || sf == 0 || pv == 0 || up == 0 || + qk > maximum || sf > maximum || pv > maximum || up > maximum) { + return false; + } + *counts = pa_scheduler::WorkloadCounts{qk, sf, pv, up}; + return true; +} + +bool ParseWinnerWorkloadOptions( + int argc, char **argv, WinnerWorkloadOptions *workload, std::vector *remaining_argv +) { + bool mode_seen = false; + bool count_seen = false; + remaining_argv->clear(); + remaining_argv->push_back(argv[0]); + for (int index = 1; index < argc; ++index) { + const std::string argument = argv[index]; + if (argument == "--nop-count" || argument == "--nop-counts") { + workload->nop_override_explicit = true; + } + if (argument != "--winner-workload" && argument != "--real-compute-count" && + argument != "--real-compute-counts") { + remaining_argv->push_back(argv[index]); + continue; + } + if (index + 1 >= argc) { + std::fprintf(stderr, "Missing value after %s\n", argument.c_str()); + return false; + } + const char *value = argv[++index]; + if (argument == "--winner-workload") { + if (mode_seen) { + std::fprintf(stderr, "Specify --winner-workload only once.\n"); + return false; + } + const std::string name = value; + if (name == "scalar-nop") { + workload->mode = pa_scheduler::WinnerWorkloadMode::ScalarNop; + } else if (name == "real-compute") { + workload->mode = pa_scheduler::WinnerWorkloadMode::RealCompute; + } else { + std::fprintf( + stderr, + "Invalid --winner-workload value: %s (expected scalar-nop|real-compute)\n", + value + ); + return false; + } + mode_seen = true; + continue; + } + if (count_seen) { + std::fprintf(stderr, "Specify only one real-compute count override.\n"); + return false; + } + if (argument == "--real-compute-count") { + uint32_t count = 0; + if (!pa_scheduler::host::ParseUint( + value, 1, pa_scheduler::ccec_workload::kMaxRealComputeCount, &count + )) { + std::fprintf(stderr, "Invalid --real-compute-count value: %s\n", value); + return false; + } + workload->repeats = pa_scheduler::WorkloadCounts{count, count, count, count}; + } else if (!ParseWorkloadCounts(value, &workload->repeats)) { + std::fprintf(stderr, "Invalid --real-compute-counts value: %s\n", value); + return false; + } + count_seen = true; + workload->counts_explicit = true; + } + return true; +} + +bool ValidateWinnerWorkloadOptions(const WinnerWorkloadOptions &workload) { + if (workload.mode == pa_scheduler::WinnerWorkloadMode::RealCompute) { + if (workload.nop_override_explicit) { + std::fprintf( + stderr, + "--winner-workload real-compute cannot be combined with --nop-count or --nop-counts.\n" + ); + return false; + } + return true; + } + if (workload.counts_explicit) { + std::fprintf( + stderr, + "--real-compute-count(s) requires --winner-workload real-compute.\n" + ); + return false; + } + return true; +} + +void ConfigureWinnerWorkload( + pa_scheduler::SchedulerState *state, const WinnerWorkloadOptions &workload, + const void *workspace_device +) { + state->winner_workload.mode = static_cast(workload.mode); + state->winner_workload.version = pa_scheduler::kWinnerWorkloadConfigVersion; + state->winner_workload.repeats = workload.repeats; + state->winner_workload.workspace_base = reinterpret_cast(workspace_device); + state->winner_workload.workspace_bytes = + workload.mode == pa_scheduler::WinnerWorkloadMode::RealCompute + ? pa_scheduler::ccec_workload::kWorkspaceBytes + : 0; +} + const char *PmuModeName(pa_scheduler::ccec_pmu::WindowMode mode) { switch (mode) { case pa_scheduler::ccec_pmu::WindowMode::Off: @@ -241,6 +401,80 @@ void ConfigurePmu(pa_scheduler::SchedulerState *state, const PmuOptions &pmu, co state->config.reserved[kConfigMagic] = pmu.mode == WindowMode::Off ? 0 : kConfigMagicValue; } +const char *TaskKindName(pa_scheduler::TaskKind kind) { + switch (kind) { + case pa_scheduler::TaskKind::Qk: + return "QK"; + case pa_scheduler::TaskKind::Sf: + return "SF"; + case pa_scheduler::TaskKind::Pv: + return "PV"; + case pa_scheduler::TaskKind::Up: + return "UP"; + default: + return "invalid"; + } +} + +bool ValidateRealComputeOutputs( + const pa_scheduler::SchedulerState &state, const std::vector &outputs, uint32_t run +) { + using namespace pa_scheduler::ccec_workload; + const size_t expected_elements = static_cast(kOutputTiles) * kTileElements; + if (outputs.size() != expected_elements) { + std::fprintf( + stderr, "[ASSERT] real-compute output buffer size matches workspace layout FAIL\n" + ); + return false; + } + + uint32_t active_tiles = 0; + uint32_t inactive_tiles = 0; + for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { + const pa_scheduler::WorkerResult &result = state.results[worker]; + const bool aic = result.role == static_cast(pa_scheduler::CoreRole::Aic); + const pa_scheduler::TaskKind kinds[2] = { + aic ? pa_scheduler::TaskKind::Qk : pa_scheduler::TaskKind::Sf, + aic ? pa_scheduler::TaskKind::Pv : pa_scheduler::TaskKind::Up, + }; + for (uint32_t kind_slot = 0; kind_slot < 2; ++kind_slot) { + const pa_scheduler::TaskKind kind = kinds[kind_slot]; + const uint32_t kernel_index = static_cast(kind) - 1; + const bool active = result.kernel_counts[kernel_index] != 0; + const float expected = active + ? (aic ? kExpectedAicValue + : (kind == pa_scheduler::TaskKind::Sf ? kExpectedSfValue : kExpectedUpValue)) + : kOutputSentinel; + const size_t tile_index = + static_cast(worker) * kOutputTilesPerWorker + kind_slot; + const size_t begin = tile_index * kTileElements; + for (size_t element = 0; element < kTileElements; ++element) { + if (outputs[begin + element] == expected) continue; + std::fprintf( + stderr, + "[REAL-COMPUTE-FAIL] run=%u worker=%u kind=%s element=%zu expected=%.1f actual=%.9g\n", + run, worker, TaskKindName(kind), element, expected, + static_cast(outputs[begin + element]) + ); + std::fprintf( + stderr, "[ASSERT] real-compute output tiles match role-specific PTO results FAIL\n" + ); + return false; + } + active_tiles += active ? 1U : 0U; + inactive_tiles += active ? 0U : 1U; + } + } + const bool passed = active_tiles != 0 && + active_tiles + inactive_tiles == pa_scheduler::ccec_workload::kOutputTiles; + std::printf( + "[ASSERT] %-48s %s (active_tiles=%u inactive_sentinel_tiles=%u)\n", + "real-compute output tiles match role-specific PTO results", + passed ? "PASS" : "FAIL", active_tiles, inactive_tiles + ); + return passed; +} + struct PmuAggregate { std::vector total_cycles; std::vector window_ticks; @@ -378,9 +612,12 @@ struct PmuValidation { uint32_t icache_pairs = 0; uint32_t icache_calibrated_cores = 0; uint32_t prior_snapshot_larger = 0; + uint32_t submit_engine_workers_expected = 0; + uint32_t submit_engine_workers_matched = 0; uint32_t maximum_programmable_counter = 0; bool icache_order_valid = true; bool icache_measurement_valid = true; + bool submit_engine_observation_valid = true; bool counter_below_risk_threshold = true; bool passed = true; }; @@ -391,6 +628,7 @@ constexpr uint32_t kProgrammableCounterRiskThreshold = UINT32_MAX / 4U; bool ValidatePmu( const pa_scheduler::SchedulerState &state, uint32_t run, const PmuOptions &pmu, + const WinnerWorkloadOptions &workload, const pa_scheduler::pmu_owner::PmuOwnerControl *owner, PmuValidation *validation ) { using namespace pa_scheduler::ccec_pmu; @@ -410,6 +648,8 @@ bool ValidatePmu( uint32_t icache_pairs = 0; uint32_t icache_calibrated_cores = 0; uint32_t prior_larger = 0; + uint32_t submit_engine_workers_expected = 0; + uint32_t submit_engine_workers_matched = 0; uint32_t maximum_programmable_counter = 0; bool icache_order_valid = true; uint32_t bad_printed = 0; @@ -431,6 +671,17 @@ bool ValidatePmu( window_stopped += (status & kStatusWindowStopped) != 0U; icache_pairs += (status & kStatusIcachePairObserved) != 0U; prior_larger += (status & kStatusPriorSnapshotLarger) != 0; + if (pmu.mode == WindowMode::SubmitAll && + workload.mode == pa_scheduler::WinnerWorkloadMode::RealCompute) { + const uint64_t submit_engine_tasks = + result.placement[static_cast(pa_scheduler::DrainPlace::EfDrain)] + + result.placement[static_cast(pa_scheduler::DrainPlace::RingBackpressure)]; + const uint32_t relevant_busy = logical_aic ? result.pmu_cube_busy : result.pmu_vector_busy; + const bool engine_observation_matches = + (submit_engine_tasks == 0U) == (relevant_busy == 0U); + submit_engine_workers_expected += submit_engine_tasks != 0U; + submit_engine_workers_matched += engine_observation_matches; + } icache_order_valid &= result.pmu_icache_misses <= result.pmu_icache_requests && result.pmu_warm_icache_misses <= result.pmu_warm_icache_requests; if (pmu.mode == WindowMode::IcacheSingle) { @@ -512,6 +763,10 @@ bool ValidatePmu( const bool mixed_triplets_ok = mixed_triplet_matches == pa_scheduler::kAicWorkers; const bool windows_started_ok = window_started == pa_scheduler::kWorkers; const bool windows_stopped_ok = window_stopped == pa_scheduler::kWorkers; + const bool submit_engine_observation_ok = + pmu.mode != WindowMode::SubmitAll || + workload.mode != pa_scheduler::WinnerWorkloadMode::RealCompute || + submit_engine_workers_matched == pa_scheduler::kWorkers; const bool counter_below_risk_threshold = maximum_programmable_counter < kProgrammableCounterRiskThreshold; std::printf( @@ -544,6 +799,15 @@ bool ValidatePmu( icache_order_valid ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "programmable counters stay below 25% risk threshold", counter_below_risk_threshold ? "PASS" : "FAIL"); + if (pmu.mode == WindowMode::SubmitAll && + workload.mode == pa_scheduler::WinnerWorkloadMode::RealCompute) { + std::printf( + "[ASSERT] %-48s %s (active_workers=%u matched_workers=%u/%u)\n", + "Submit placement has matching AIC/AIV engine PMU", + submit_engine_observation_ok ? "PASS" : "FAIL", submit_engine_workers_expected, + submit_engine_workers_matched, pa_scheduler::kWorkers + ); + } if (pmu.mode == WindowMode::IcacheSingle) { std::printf("[ASSERT] %-48s %s\n", "each cold trial adds exactly one CNT7 I-cache miss", icache_measurement_ok ? "PASS" : "FAIL"); @@ -559,13 +823,17 @@ bool ValidatePmu( validation->icache_pairs = icache_pairs; validation->icache_calibrated_cores = icache_calibrated_cores; validation->prior_snapshot_larger = prior_larger; + validation->submit_engine_workers_expected = submit_engine_workers_expected; + validation->submit_engine_workers_matched = submit_engine_workers_matched; validation->maximum_programmable_counter = maximum_programmable_counter; validation->icache_order_valid = icache_order_valid; validation->icache_measurement_valid = icache_measurement_ok; + validation->submit_engine_observation_valid = submit_engine_observation_ok; validation->counter_below_risk_threshold = counter_below_risk_threshold; validation->passed = records_ok && core_ids_ok && owner_members_ok && worker_slots_ok && physical_roles_ok && mixed_triplets_ok && windows_started_ok && windows_stopped_ok && - icache_order_valid && icache_measurement_ok && counter_below_risk_threshold; + icache_order_valid && icache_measurement_ok && submit_engine_observation_ok && + counter_below_risk_threshold; return validation->passed; } @@ -727,8 +995,9 @@ uint32_t CountConfiguredMixedTriplets(const pa_scheduler::pmu_owner::PmuOwnerCon bool ExportPmuJson( const pa_scheduler::SchedulerState &state, const pa_scheduler::host::Options &options, - const PmuOptions &pmu, uint32_t run, double host_us, double submit_span_us, - const PmuValidation &validation, bool semantic_passed, + const PmuOptions &pmu, const WinnerWorkloadOptions &workload, + uint32_t run, double host_us, double submit_span_us, + const PmuValidation &validation, bool semantic_passed, bool workload_output_passed, const pa_scheduler::pmu_owner::PmuOwnerControl &owner, bool restore_passed, const std::string &output_path ) { @@ -736,10 +1005,25 @@ bool ExportPmuJson( PmuAggregate all; PmuAggregate aic; PmuAggregate aiv; + uint32_t active_output_tiles = 0; + uint64_t ef_drain_kernels = 0; + uint64_t ring_backpressure_kernels = 0; + uint64_t final_drain_kernels = 0; for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { const pa_scheduler::WorkerResult &result = state.results[worker]; AddPmuSample(result, &all); AddPmuSample(result, result.role == static_cast(pa_scheduler::CoreRole::Aic) ? &aic : &aiv); + if (result.role == static_cast(pa_scheduler::CoreRole::Aic)) { + active_output_tiles += result.kernel_counts[0] != 0; + active_output_tiles += result.kernel_counts[2] != 0; + } else if (result.role == static_cast(pa_scheduler::CoreRole::Aiv)) { + active_output_tiles += result.kernel_counts[1] != 0; + active_output_tiles += result.kernel_counts[3] != 0; + } + ef_drain_kernels += result.placement[static_cast(pa_scheduler::DrainPlace::EfDrain)]; + ring_backpressure_kernels += + result.placement[static_cast(pa_scheduler::DrainPlace::RingBackpressure)]; + final_drain_kernels += result.placement[static_cast(pa_scheduler::DrainPlace::FinalDrain)]; } const auto generated = std::chrono::system_clock::now().time_since_epoch(); @@ -769,11 +1053,18 @@ bool ExportPmuJson( const bool submit_window = IsSubmitWindow(pmu.mode); const bool icache_single = pmu.mode == WindowMode::IcacheSingle; - const bool simulated_task_nops_nonzero = - options.nops.qk != 0U || options.nops.sf != 0U || options.nops.pv != 0U || options.nops.up != 0U; + const bool real_compute = workload.mode == pa_scheduler::WinnerWorkloadMode::RealCompute; + const bool simulated_task_nops_nonzero = !real_compute && + (options.nops.qk != 0U || options.nops.sf != 0U || + options.nops.pv != 0U || options.nops.up != 0U); + const pa_scheduler::WorkloadCounts active_counts = real_compute + ? workload.repeats + : pa_scheduler::WorkloadCounts{ + options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + }; const uint32_t owner_bitmap_count = pa_scheduler::pmu_owner::CountConfigured(owner); const uint32_t owner_complete_triplets = CountConfiguredMixedTriplets(owner); - std::fputs("{\n\"schema\":{\"name\":\"pa_scheduler_pmu_phase_windows\",\"version\":2},\n", output); + std::fputs("{\n\"schema\":{\"name\":\"pa_scheduler_pmu_phase_windows\",\"version\":3},\n", output); std::fputs("\"capture\":{\"capture_id\":", output); WriteJsonString(output, capture_id); std::fprintf( @@ -783,13 +1074,15 @@ bool ExportPmuJson( "\"usable_as_absolute_real_pa_profile\":false,\"window_scope\":\"%s\"," "\"pmu_probe_position\":\"%s\",\"scheduler_hot_path_included\":%s," "\"total_sum_is_core_work_not_wall_time\":true," + "\"submit_window_excludes_final_drain\":%s," "\"published_after_runtime_cleanup\":true,\"runtime_cleanup_passed\":true," "\"owner_restore_passed\":%s},\n", static_cast(generated_ns), run, submit_window ? "true" : "false", submit_window ? "per_worker_orchestration_to_last_submit_return" : "post_scheduler_calibration_probe", submit_window ? "inside_RunScheduler" : "after_RunScheduler", - submit_window ? "true" : "false", restore_passed ? "true" : "false" + submit_window ? "true" : "false", submit_window ? "true" : "false", + restore_passed ? "true" : "false" ); std::fputs("\"configuration\":{\"kernel_path\":", output); WriteJsonString(output, options.kernel_path); @@ -797,14 +1090,56 @@ bool ExportPmuJson( output, ",\"device\":%u,\"batches\":%u,\"workers\":%u,\"aic_workers\":%u,\"aiv_workers\":%u," "\"trace_enabled\":%s,\"trace_atomics\":%s,\"profile_phases\":%s," - "\"nop_counts\":{\"qk\":%u,\"sf\":%u," - "\"pv\":%u,\"up\":%u},\"pmu_window\":", + "\"winner_workload\":{\"mode\":", options.device, options.batches, pa_scheduler::kWorkers, pa_scheduler::kAicWorkers, pa_scheduler::kAivWorkers, options.trace_enabled ? "true" : "false", options.trace_atomics ? "true" : "false", - options.profile_phases ? "true" : "false", options.nops.qk, options.nops.sf, options.nops.pv, - options.nops.up + options.profile_phases ? "true" : "false" ); + WriteJsonString(output, WinnerWorkloadModeName(workload.mode)); + std::fprintf( + output, + ",\"config_version\":%u,\"counts\":{\"qk\":%u,\"sf\":%u,\"pv\":%u,\"up\":%u}," + "\"unit\":", + pa_scheduler::kWinnerWorkloadConfigVersion, active_counts.qk, active_counts.sf, + active_counts.pv, active_counts.up + ); + WriteJsonString( + output, + real_compute ? "complete_128x128_engine_pipeline_iteration" : "scalar_nop_instruction" + ); + std::fprintf( + output, + ",\"tile_rows\":%u,\"tile_cols\":%u,\"shared_input_tiles\":%u," + "\"output_tiles_per_worker\":%u,\"workspace_bytes\":%zu,\"role_mapping\":", + real_compute ? pa_scheduler::ccec_workload::kTileRows : 0, + real_compute ? pa_scheduler::ccec_workload::kTileCols : 0, + real_compute ? pa_scheduler::ccec_workload::kSharedInputTiles : 0, + real_compute ? pa_scheduler::ccec_workload::kOutputTilesPerWorker : 0, + real_compute ? pa_scheduler::ccec_workload::kWorkspaceBytes : 0 + ); + if (real_compute) { + std::fputs( + "{\"qk\":\"cube_matmul\",\"pv\":\"cube_matmul\"," + "\"sf\":\"vector_add\",\"up\":\"vector_mul\"}", + output + ); + } else { + std::fputs("null", output); + } + std::fprintf( + output, ",\"engine_completion_waited_before_task_publish\":%s},\"nop_counts\":", + real_compute ? "true" : "false" + ); + if (real_compute) { + std::fputs("null", output); + } else { + std::fprintf( + output, "{\"qk\":%u,\"sf\":%u,\"pv\":%u,\"up\":%u}", + options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + ); + } + std::fputs(",\"pmu_window\":", output); WriteJsonString(output, PmuModeName(pmu.mode)); std::fputs(",\"calibration_scalar_nops_per_segment\":", output); if (submit_window || icache_single) { @@ -833,18 +1168,29 @@ bool ExportPmuJson( "\"gate_start_stop_have_pipe_all_barriers\":true," "\"phase_timestamp_calls_present\":true,\"phase_record_writes\":false," "\"atomic_trace\":false,\"profile_accumulation\":false," - "\"simulated_task_nop_mechanism_executes_on_scalar\":true," + "\"simulated_task_nop_mechanism_executes_on_scalar\":%s," "\"simulated_task_nops_nonzero\":%s," "\"icache_miss_rate_definition\":\"sum(icache_misses)/sum(icache_requests)\"},\n", icache_single ? 2U : 0U, icache_single ? "1" : "null", host_us, submit_span_us, kVectorBusyEvent, kCubeBusyEvent, kScalarBusyEvent, kMte1BusyEvent, kMte2BusyEvent, kMte3BusyEvent, kIcacheRequestEvent, kIcacheMissEvent, kFixBusyEvent, kProgrammableCounterRiskThreshold, + real_compute ? "false" : "true", simulated_task_nops_nonzero ? "true" : "false" ); std::fprintf( output, - "\"validation\":{\"semantic_passed\":%s,\"pmu_passed\":%s,\"trusted_records\":%u," + "\"validation\":{\"semantic_passed\":%s,\"pmu_passed\":%s," + "\"real_compute_output_validation_required\":%s," + "\"real_compute_output_validation_passed\":%s," + "\"real_compute_active_output_tiles\":%u," + "\"real_compute_inactive_sentinel_tiles\":%u," + "\"real_compute_output_mismatches\":%u," + "\"submit_engine_observation_valid\":%s," + "\"submit_engine_workers_expected\":%u," + "\"submit_engine_workers_matched\":%u," + "\"kernel_placement_counts\":{\"ef_drain\":%llu,\"ring_backpressure\":%llu," + "\"final_drain\":%llu},\"trusted_records\":%u," "\"expected_records\":%u,\"unique_physical_core_ids\":%u,\"expected_unique_core_ids\":%u," "\"owner_bitmap_member_records\":%u,\"expected_owner_bitmap_member_records\":%u," "\"exact_worker_slot_records\":%u,\"expected_exact_worker_slot_records\":%u," @@ -857,7 +1203,19 @@ bool ExportPmuJson( "\"icache_miss_le_request\":%s,\"counter_below_risk_threshold\":%s," "\"maximum_programmable_counter\":%u,\"programmable_counter_risk_threshold\":%u," "\"programmable_counter_headroom\":%u},\n", - semantic_passed ? "true" : "false", validation.passed ? "true" : "false", validation.trusted, + semantic_passed ? "true" : "false", validation.passed ? "true" : "false", + real_compute ? "true" : "false", + real_compute ? (workload_output_passed ? "true" : "false") : "null", + real_compute ? active_output_tiles : 0, + real_compute ? pa_scheduler::ccec_workload::kOutputTiles - active_output_tiles : 0, + real_compute && !workload_output_passed ? 1U : 0U, + validation.submit_engine_observation_valid ? "true" : "false", + validation.submit_engine_workers_expected, + validation.submit_engine_workers_matched, + static_cast(ef_drain_kernels), + static_cast(ring_backpressure_kernels), + static_cast(final_drain_kernels), + validation.trusted, pa_scheduler::kWorkers, validation.unique_physical_core_ids, pa_scheduler::kWorkers, validation.owner_bitmap_members, pa_scheduler::kWorkers, validation.exact_worker_slots, pa_scheduler::kWorkers, @@ -1014,8 +1372,15 @@ int main(int argc, char **argv) { // 参数和 ELF 在创建 ACL 资源前完成校验,早期错误不会留下 device、stream 或 kernel handle。 pa_scheduler::host::Options options; PmuOptions pmu_options; + WinnerWorkloadOptions workload_options; + std::vector pmu_argv; std::vector common_argv; - if (!ParsePmuOptions(argc, argv, &pmu_options, &common_argv)) return EXIT_FAILURE; + if (!ParseWinnerWorkloadOptions(argc, argv, &workload_options, &pmu_argv) || + !ParsePmuOptions( + static_cast(pmu_argv.size()), pmu_argv.data(), &pmu_options, &common_argv + )) { + return EXIT_FAILURE; + } const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions( static_cast(common_argv.size()), common_argv.data(), true, &options ); @@ -1027,9 +1392,15 @@ int main(int argc, char **argv) { "off|empty|scalar|scalar-double|icache-single|submit-all] " "[--pmu-scalar-nops N] [--pmu-icache-trials N] [--pmu-json FILE]\n" ); + std::fprintf( + stderr, + "CCEC winner workload options: [--winner-workload scalar-nop|real-compute] " + "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP]\n" + ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } + if (!ValidateWinnerWorkloadOptions(workload_options)) return EXIT_FAILURE; if (!pmu_options.json_path.empty() && pmu_options.mode == pa_scheduler::ccec_pmu::WindowMode::Off) { std::fprintf(stderr, "--pmu-json requires a non-off --pmu-window.\n"); @@ -1064,7 +1435,33 @@ int main(int argc, char **argv) { std::fprintf(stderr, "Cannot read kernel binary: %s\n", options.kernel_path.c_str()); return EXIT_FAILURE; } + const bool real_compute = + workload_options.mode == pa_scheduler::WinnerWorkloadMode::RealCompute; + std::vector workload_image; + std::vector workload_outputs; + if (real_compute) { + using namespace pa_scheduler::ccec_workload; + workload_image.assign(kWorkspaceTiles * kTileElements, kOutputSentinel); + std::fill_n(workload_image.begin(), kTileElements, kInputAValue); + std::fill_n(workload_image.begin() + kTileElements, kTileElements, kInputBValue); + workload_outputs.resize(static_cast(kOutputTiles) * kTileElements); + } pa_scheduler::host::PrintBanner("CCEC", options); + if (real_compute) { + std::printf( + "[WINNER-WORKLOAD] mode=real-compute counts=%u,%u,%u,%u " + "unit=complete_128x128_engine_pipeline_iteration workspace_bytes=%zu\n", + workload_options.repeats.qk, workload_options.repeats.sf, + workload_options.repeats.pv, workload_options.repeats.up, + pa_scheduler::ccec_workload::kWorkspaceBytes + ); + } else { + std::printf( + "[WINNER-WORKLOAD] mode=scalar-nop counts=%u,%u,%u,%u " + "unit=scalar_nop_instruction workspace_bytes=0\n", + options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + ); + } std::printf( "[PMU-CONFIG] window=%s calibration_scalar_nops=%u icache_trials=%u source=direct-per-core " "owner=main-aicpu-path-a\n", @@ -1105,6 +1502,41 @@ int main(int argc, char **argv) { return EXIT_FAILURE; } + // 真实 PTO 负载使用独立 GM,不解引用调度器中只用于依赖建模的 synthetic tensor 地址。 + // 这里先于 PMU owner 分配;每轮 H2D 初始化虽在 owner 配置之后,但仍位于 + // launch/wall 计时之前,因此两者都不进入 Submit 性能窗口。 + ScopedAclDeviceAllocation workload_allocation; + if (real_compute && + !CheckAcl( + aclrtMalloc( + workload_allocation.Address(), pa_scheduler::ccec_workload::kWorkspaceBytes, + ACL_MEM_MALLOC_HUGE_FIRST + ), + "aclrtMalloc(real-compute workspace)" + )) { + return EXIT_FAILURE; + } + void *workload_device = workload_allocation.Get(); + if (real_compute && (reinterpret_cast(workload_device) & 63U) != 0) { + std::fprintf(stderr, "Real-compute workspace is not 64-byte aligned: %p\n", workload_device); + return EXIT_FAILURE; + } + + // 泳道区按 96 worker 各 65536 条记录预留,约 384 MiB;关闭泳道时不申请,也不会传递有效 base。 + // 该分配先于 PMU owner 配置,失败时不会留下需要恢复的 selector/MMIO 会话。 + void *trace_device = nullptr; + if (options.trace_enabled && + !CheckAcl( + aclrtMalloc(&trace_device, pa_scheduler::kTraceBytes, ACL_MEM_MALLOC_HUGE_FIRST), + "aclrtMalloc(swimlane trace)" + )) { + return EXIT_FAILURE; + } + if (options.trace_enabled && (reinterpret_cast(trace_device) & 63U) != 0) { + std::fprintf(stderr, "Device swimlane trace is not 64-byte aligned: %p\n", trace_device); + return EXIT_FAILURE; + } + PmuRegisterMappings pmu_mappings; pa_scheduler::pmu_owner::PmuOwnerSession pmu_owner; pa_scheduler::pmu_owner::PmuOwnerControl pmu_owner_evidence{}; @@ -1131,29 +1563,17 @@ int main(int argc, char **argv) { pmu_registers_device = reinterpret_cast(pmu_owner.RegisterTableDeviceAddress()); } - // 泳道区按 96 worker 各 65536 条记录预留,约 384 MiB;关闭泳道时不申请,也不会传递有效 base。 - void *trace_device = nullptr; - if (options.trace_enabled && - !CheckAcl( - aclrtMalloc(&trace_device, pa_scheduler::kTraceBytes, ACL_MEM_MALLOC_HUGE_FIRST), - "aclrtMalloc(swimlane trace)" - )) { - return EXIT_FAILURE; - } - if (options.trace_enabled && (reinterpret_cast(trace_device) & 63U) != 0) { - std::fprintf(stderr, "Device swimlane trace is not 64-byte aligned: %p\n", trace_device); - return EXIT_FAILURE; - } - // host shadow 保留约 1 GiB 总跨度以便按关键 offset 寻址,但每轮传输只选择 // 共享前缀、控制区和结果区。 std::unique_ptr state(new pa_scheduler::SchedulerState); pa_scheduler::TraceHeader trace_header{}; std::vector spans; + bool execution_ok = true; bool all_passed = true; bool postprocess_ok = true; bool pmu_json_ready = false; bool pmu_json_semantic_passed = false; + bool pmu_json_workload_output_passed = false; uint32_t pmu_json_run = 0U; double pmu_json_host_us = 0.0; double pmu_json_submit_span_us = 0.0; @@ -1162,6 +1582,19 @@ int main(int argc, char **argv) { pa_scheduler::host::InitializeState(state.get(), options); pa_scheduler::host::ConfigureTrace(state.get(), options, trace_device); ConfigurePmu(state.get(), pmu_options, pmu_registers_device); + ConfigureWinnerWorkload(state.get(), workload_options, workload_device); + if (real_compute && + !CheckAcl( + aclrtMemcpy( + workload_device, pa_scheduler::ccec_workload::kWorkspaceBytes, + workload_image.data(), pa_scheduler::ccec_workload::kWorkspaceBytes, + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D real-compute workspace)" + )) { + execution_ok = false; + break; + } if (options.trace_enabled) { // 每轮只需重置约 7 KiB header;各 worker 会从 count=0 覆盖自己的记录区,无需清零整块 384 MiB。 pa_scheduler::host::InitializeTraceHeader(&trace_header); @@ -1172,7 +1605,8 @@ int main(int argc, char **argv) { ), "aclrtMemcpy(H2D swimlane header)" )) { - return EXIT_FAILURE; + execution_ok = false; + break; } } // 为避免每轮搬运约 1 GiB,只 H2D 被测共享前缀和位于生产总跨度之后的 @@ -1193,9 +1627,9 @@ int main(int argc, char **argv) { ), "aclrtMemcpy(H2D standalone controls)" )) { - return EXIT_FAILURE; + execution_ok = false; + break; } - void *kernel_args[] = {state_device}; rtArgsEx_t args_info{}; args_info.args = kernel_args; @@ -1211,7 +1645,8 @@ int main(int argc, char **argv) { "rtKernelLaunchWithHandleV2" ) || !CheckAcl(aclrtSynchronizeStream(stream), "aclrtSynchronizeStream")) { - return EXIT_FAILURE; + execution_ok = false; + break; } const auto wall_end = std::chrono::steady_clock::now(); const double host_us = std::chrono::duration(wall_end - wall_begin).count(); @@ -1231,7 +1666,26 @@ int main(int argc, char **argv) { ), "aclrtMemcpy(D2H worker results)" )) { - return EXIT_FAILURE; + execution_ok = false; + break; + } + if (real_compute && + !CheckAcl( + aclrtMemcpy( + workload_outputs.data(), + static_cast(pa_scheduler::ccec_workload::kOutputTiles) * + pa_scheduler::ccec_workload::kTileBytes, + static_cast(workload_device) + + pa_scheduler::ccec_workload::kSharedInputTiles * + pa_scheduler::ccec_workload::kTileBytes, + static_cast(pa_scheduler::ccec_workload::kOutputTiles) * + pa_scheduler::ccec_workload::kTileBytes, + ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H real-compute outputs)" + )) { + execution_ok = false; + break; } if (options.trace_enabled && !CheckAcl( @@ -1241,7 +1695,8 @@ int main(int argc, char **argv) { ), "aclrtMemcpy(D2H swimlane header)" )) { - return EXIT_FAILURE; + execution_ok = false; + break; } // 常规校验只需 header 中的 per-worker count;真实 records 在分析或导出时才按核、按实际 count 懒加载。 const auto read_trace_records = @@ -1265,22 +1720,26 @@ int main(int argc, char **argv) { *state, run, host_us, options.trace_enabled ? &trace_header : nullptr ); all_passed &= metrics.passed; + const bool workload_passed = + !real_compute || ValidateRealComputeOutputs(*state, workload_outputs, run); + all_passed &= workload_passed; PmuValidation pmu_validation; const bool pmu_passed = ValidatePmu( - *state, run, pmu_options, + *state, run, pmu_options, workload_options, pmu_options.mode == pa_scheduler::ccec_pmu::WindowMode::Off ? nullptr : &pmu_owner.Control(), &pmu_validation ); all_passed &= pmu_passed; spans.push_back(metrics.submit_span_us); if (!pmu_options.json_path.empty()) { - if (!metrics.passed || !pmu_passed || !pmu_owner_evidence_valid) { + if (!metrics.passed || !workload_passed || !pmu_passed || !pmu_owner_evidence_valid) { std::fprintf(stderr, "PMU JSON rejected because semantic, PMU, or owner validation failed.\n"); postprocess_ok = false; break; } pmu_json_ready = true; - pmu_json_semantic_passed = metrics.passed; + pmu_json_semantic_passed = metrics.passed && workload_passed; + pmu_json_workload_output_passed = workload_passed; pmu_json_run = run; pmu_json_host_us = host_us; pmu_json_submit_span_us = metrics.submit_span_us; @@ -1295,13 +1754,19 @@ int main(int argc, char **argv) { if (!options.swimlane_json.empty()) { // 只有语义校验通过才把 raw JSON 经“临时文件写完后 rename”发布, // 避免把截断或错误调度结果误当成可用性能证据。 - if (!metrics.passed) { + if (!metrics.passed || !workload_passed) { std::fprintf(stderr, "Skipping swimlane export because semantic validation failed.\n"); postprocess_ok = false; break; } if (!pa_scheduler::host::ExportSwimlaneRecords( - trace_header, options.swimlane_json, read_trace_records + trace_header, options.swimlane_json, workload_options.mode, + real_compute + ? workload_options.repeats + : pa_scheduler::WorkloadCounts{ + options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + }, + read_trace_records )) { postprocess_ok = false; break; @@ -1309,9 +1774,12 @@ int main(int argc, char **argv) { } } + const double median_submit_span_us = spans.empty() ? 0.0 : pa_scheduler::host::Median(spans); std::printf( - "[SUMMARY] runs=%u median_submit_span_us=%.3f semantic_status=%s postprocess_status=%s\n", options.runs, - pa_scheduler::host::Median(spans), all_passed ? "PASS" : "FAIL", postprocess_ok ? "PASS" : "FAIL" + "[SUMMARY] runs=%u completed_runs=%zu median_submit_span_us=%.3f " + "execution_status=%s semantic_status=%s postprocess_status=%s\n", + options.runs, spans.size(), median_submit_span_us, execution_ok ? "PASS" : "FAIL", + all_passed ? "PASS" : "FAIL", postprocess_ok ? "PASS" : "FAIL" ); // 后处理失败也统一走设备资源释放、ELF 卸载和 ACL 收尾,避免文件系统错误遗留运行时上下文。 @@ -1327,6 +1795,11 @@ int main(int argc, char **argv) { cleanup_ok &= pmu_owner_restore_ok; cleanup_ok &= UnmapPmuRegisters(options.device, &pmu_mappings); } + if (workload_device != nullptr) { + cleanup_ok &= CheckAcl( + aclrtFree(workload_allocation.Release()), "aclrtFree(real-compute workspace)" + ); + } cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(state)"); const rtError_t unload_error = registered_all ? rtDevBinaryUnRegister(kernel_handle) : rtBinaryUnLoad(kernel_handle); @@ -1339,8 +1812,9 @@ int main(int argc, char **argv) { std::fprintf(stderr, "PMU JSON was not published because the capture or restore transaction failed.\n"); postprocess_ok = false; } else if (!ExportPmuJson( - *state, options, pmu_options, pmu_json_run, pmu_json_host_us, - pmu_json_submit_span_us, pmu_json_validation, pmu_json_semantic_passed, + *state, options, pmu_options, workload_options, pmu_json_run, + pmu_json_host_us, pmu_json_submit_span_us, pmu_json_validation, + pmu_json_semantic_passed, pmu_json_workload_output_passed, pmu_owner_evidence, pmu_owner_restore_ok, pmu_options.json_path )) { @@ -1348,5 +1822,5 @@ int main(int argc, char **argv) { } } // 运行语义、后处理和资源清理三者全部成功,进程才返回成功,脚本据此决定是否继续生成 merged 泳道。 - return all_passed && postprocess_ok && cleanup_ok ? EXIT_SUCCESS : EXIT_FAILURE; + return execution_ok && all_passed && postprocess_ok && cleanup_ok ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index 2d695de7d0..dd1736cd51 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -10,9 +10,13 @@ */ #include "cce_aicore_intrinsics.h" +#include #include +#include +#include #include "pmu_probe.h" +#include "winner_workload.h" #define PA_DEVICE __aicore__ inline #define PA_GM __gm__ @@ -20,6 +24,8 @@ namespace { +using namespace pto; + template __aicore__ inline void EmitNops() { #pragma unroll @@ -46,6 +52,166 @@ __aicore__ inline void RuntimeNop(uint32_t count) { __builtin_cce_pipe_barrier(PIPE_ALL); } +#if defined(PA_BUILD_AIC) +// QK/PV 的首版真实负载使用完整的 128x128 float cube 路径。输入来自独立 GM +// workspace,输出属于当前 worker;每次迭代都等 FIX 写回 GM 后再复用 L0C, +// 因而函数返回就是该模拟 task 的完成边界,而不是单纯的指令发射边界。 +static __aicore__ __attribute__((noinline, used)) void pa_real_cube_workload_aic( + __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats +) { + constexpr int kTile = static_cast(pa_scheduler::ccec_workload::kTileRows); + constexpr int kBlockAlign = C0_SIZE_BYTE / sizeof(float); + static_assert(kTile % 16 == 0, "cube M must be 16-aligned"); + static_assert(kTile % kBlockAlign == 0, "cube K/N must satisfy C0 alignment"); + + using GlobalData = GlobalTensor< + float, Shape<1, 1, 1, kTile, kTile>, + pto::Stride>; + using TileMatA = Tile< + TileType::Mat, float, kTile, kTile, BLayout::ColMajor, + kTile, kTile, SLayout::RowMajor, 512>; + using TileMatB = Tile< + TileType::Mat, float, kTile, kTile, BLayout::ColMajor, + kTile, kTile, SLayout::RowMajor, 512>; + using LeftTile = TileLeft; + using RightTile = TileRight; + using AccTile = TileAcc; + + GlobalData input_a_global(input_a); + GlobalData input_b_global(input_b); + GlobalData output_global(output); + TileMatA input_a_mat; + TileMatB input_b_mat; + LeftTile input_a_l0; + RightTile input_b_l0; + AccTile output_l0; + TASSIGN(input_a_mat, 0x0); + TASSIGN(input_b_mat, 0x20000); + TASSIGN(input_a_l0, 0x0); + TASSIGN(input_b_l0, 0x0); + TASSIGN(output_l0, 0x0); + + for (uint32_t iteration = 0; iteration < repeats; ++iteration) { + TLOAD(input_a_mat, input_a_global); + TLOAD(input_b_mat, input_b_global); + set_flag(PIPE_MTE2, PIPE_MTE1, EVENT_ID0); + wait_flag(PIPE_MTE2, PIPE_MTE1, EVENT_ID0); + TMOV(input_a_l0, input_a_mat); + TMOV(input_b_l0, input_b_mat); + set_flag(PIPE_MTE1, PIPE_M, EVENT_ID0); + wait_flag(PIPE_MTE1, PIPE_M, EVENT_ID0); + TMATMUL(output_l0, input_a_l0, input_b_l0); + set_flag(PIPE_M, PIPE_FIX, EVENT_ID0); + wait_flag(PIPE_M, PIPE_FIX, EVENT_ID0); + TSTORE(output_global, output_l0); + set_flag(PIPE_FIX, PIPE_S, EVENT_ID7); + wait_flag(PIPE_FIX, PIPE_S, EVENT_ID7); + } +} +#elif defined(PA_BUILD_AIV) +template +__aicore__ inline void RunRealVectorWorkload( + __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats +) { + constexpr int kRows = static_cast(pa_scheduler::ccec_workload::kTileRows); + constexpr int kCols = static_cast(pa_scheduler::ccec_workload::kTileCols); + using GlobalData = GlobalTensor< + float, Shape<1, 1, 1, kRows, kCols>, pto::Stride<1, 1, 1, kCols, 1>>; + using TileData = Tile< + TileType::Vec, float, kRows, kCols, BLayout::RowMajor, -1, -1>; + + GlobalData input_a_global(input_a); + GlobalData input_b_global(input_b); + GlobalData output_global(output); + TileData input_a_tile(kRows, kCols); + TileData input_b_tile(kRows, kCols); + TileData output_tile(kRows, kCols); + TASSIGN(input_a_tile, 0x0); + TASSIGN(input_b_tile, 0x10000); + TASSIGN(output_tile, 0x20000); + + for (uint32_t iteration = 0; iteration < repeats; ++iteration) { + TLOAD(input_a_tile, input_a_global); + TLOAD(input_b_tile, input_b_global); + set_flag(PIPE_MTE2, PIPE_V, EVENT_ID0); + wait_flag(PIPE_MTE2, PIPE_V, EVENT_ID0); + if constexpr (Multiply) { + TMUL(output_tile, input_a_tile, input_b_tile); + } else { + TADD(output_tile, input_a_tile, input_b_tile); + } + set_flag(PIPE_V, PIPE_MTE3, EVENT_ID0); + wait_flag(PIPE_V, PIPE_MTE3, EVENT_ID0); + TSTORE(output_global, output_tile); + set_flag(PIPE_MTE3, PIPE_S, EVENT_ID7); + wait_flag(PIPE_MTE3, PIPE_S, EVENT_ID7); + } +} + +static __aicore__ __attribute__((noinline, used)) void pa_real_vector_add_workload_aiv( + __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats +) { + RunRealVectorWorkload(input_a, input_b, output, repeats); +} + +static __aicore__ __attribute__((noinline, used)) void pa_real_vector_mul_workload_aiv( + __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats +) { + RunRealVectorWorkload(input_a, input_b, output, repeats); +} +#endif + +// 整段真实负载分派保持为 LOCAL noinline 冷路径,避免 workspace 校验、地址计算和 +// kind 分支膨胀 scalar-NOP 的 Submit 热代码;正常 NOP 对照只多一次 mode 判断。 +static __aicore__ __attribute__((noinline, used)) void pa_execute_real_winner_workload( + __gm__ pa_scheduler::SchedulerState *state, __gm__ pa_scheduler::WorkerState &worker, + pa_scheduler::TaskKind kind +) { + const uint64_t workspace = state->winner_workload.workspace_base; + const uint32_t repeats = pa_scheduler::WorkloadCountForKind( + state->winner_workload.repeats, kind + ); + // 错版 host、截断 workspace 或越界 worker 不能继续解引用 GM。这里不额外 + // 写共享 fatal,避免在正常热路增加 atomic;host 的逐 kind sentinel/数值 + // 闭环会把这种配置错误判为失败。 + if (state->winner_workload.version != pa_scheduler::kWinnerWorkloadConfigVersion || + workspace == 0 || + state->winner_workload.workspace_bytes < pa_scheduler::ccec_workload::kWorkspaceBytes || + worker.core_idx < 0 || static_cast(worker.core_idx) >= pa_scheduler::kWorkers || + repeats == 0 || repeats > pa_scheduler::ccec_workload::kMaxRealComputeCount) { + return; + } +#if defined(PA_BUILD_AIC) + if (kind != pa_scheduler::TaskKind::Qk && kind != pa_scheduler::TaskKind::Pv) return; +#elif defined(PA_BUILD_AIV) + if (kind != pa_scheduler::TaskKind::Sf && kind != pa_scheduler::TaskKind::Up) return; +#endif + __gm__ float *input_a = reinterpret_cast<__gm__ float *>(workspace); + __gm__ float *input_b = reinterpret_cast<__gm__ float *>( + workspace + pa_scheduler::ccec_workload::kTileBytes + ); + const uint32_t kind_slot = + (kind == pa_scheduler::TaskKind::Pv || kind == pa_scheduler::TaskKind::Up) ? 1U : 0U; + const uint32_t output_tile = + pa_scheduler::ccec_workload::kSharedInputTiles + + static_cast(worker.core_idx) * + pa_scheduler::ccec_workload::kOutputTilesPerWorker + + kind_slot; + __gm__ float *output = reinterpret_cast<__gm__ float *>( + workspace + static_cast(output_tile) * + pa_scheduler::ccec_workload::kTileBytes + ); +#if defined(PA_BUILD_AIC) + pa_real_cube_workload_aic(input_a, input_b, output, repeats); +#elif defined(PA_BUILD_AIV) + if (kind == pa_scheduler::TaskKind::Sf) { + pa_real_vector_add_workload_aiv(input_a, input_b, output, repeats); + } else { + pa_real_vector_mul_workload_aiv(input_a, input_b, output, repeats); + } +#endif +} + #if defined(PA_BUILD_AIC) #define PA_ICACHE_TARGET_NAME pa_icache_target_aic #define PA_ICACHE_MEASURE_NAME pa_icache_measure_aic @@ -138,7 +304,17 @@ struct CcecOps { return cycle; } - __aicore__ static inline void Nop(uint32_t count) { RuntimeNop(count); } + __aicore__ static inline void ExecuteKernel( + __gm__ pa_scheduler::SchedulerState *state, __gm__ pa_scheduler::WorkerState &worker, + pa_scheduler::TaskKind kind, uint32_t nop_count + ) { + const auto mode = static_cast(state->winner_workload.mode); + if (mode != pa_scheduler::WinnerWorkloadMode::RealCompute) { + RuntimeNop(nop_count); + return; + } + pa_execute_real_winner_workload(state, worker, kind); + } __aicore__ static inline bool PmuWindowStart( __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id diff --git a/tests/atomic_probe/pa_scheduler/ccec/winner_workload.h b/tests/atomic_probe/pa_scheduler/ccec/winner_workload.h new file mode 100644 index 0000000000..2ea0978423 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/ccec/winner_workload.h @@ -0,0 +1,57 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_CCEC_WINNER_WORKLOAD_H +#define PA_SCHEDULER_CCEC_WINNER_WORKLOAD_H + +#include "../common/pa_model.h" + +#include +#include + +namespace pa_scheduler::ccec_workload { + +// 128x128 float 是仓内 A5 QK/PV cube 与 vector 示例共同验证过的基本形状。 +// 两个输入由 96 个 worker 只读共享;每个 worker 为本角色的两种 task kind +// 各占一个输出 tile,既避免写竞争,也能分别核对 QK/PV 与 SF/UP 的数值。 +constexpr uint32_t kTileRows = 128; +constexpr uint32_t kTileCols = 128; +constexpr size_t kTileElements = static_cast(kTileRows) * kTileCols; +constexpr size_t kTileBytes = kTileElements * sizeof(float); +constexpr uint32_t kSharedInputTiles = 2; +constexpr uint32_t kOutputTilesPerWorker = 2; +constexpr uint32_t kOutputTiles = kWorkers * kOutputTilesPerWorker; +constexpr size_t kWorkspaceTiles = kSharedInputTiles + kOutputTiles; +constexpr size_t kWorkspaceBytes = kWorkspaceTiles * kTileBytes; +constexpr float kInputAValue = 2.0F; +constexpr float kInputBValue = 3.0F; +constexpr float kExpectedAicValue = 768.0F; +constexpr float kExpectedSfValue = 5.0F; +constexpr float kExpectedUpValue = 6.0F; +constexpr float kOutputSentinel = -12345.0F; +// 256 batch 下,即使同一 AIC 极端地拿到全部 QK/PV,128 次完整 cube +// 迭代的实测 busy 上界仍低于 32-bit PMU 的 25% 门槛;更大的未经取证值 +// 既偏离约 50 us task 目标,也可能让 submit-all 计数整圈回绕后伪装成小值。 +constexpr uint32_t kMaxRealComputeCount = 128; + +// 1 次仅用于最小 b1 取证。默认次数来自本机 A5 的三个独立 256-batch +// 进程:QK/SF/PV 分别最接近 44.170/53.729/27.626 us。UP 的 1 次 +// 128x128 完整 load/vector/store/drain 已是合法下限(约 2.5 us),无法仅靠 +// 正整数 repeats 降到 1.565 us;若后续要继续贴准,应缩小 UP tile,而不是用 0 次。 +constexpr WorkloadCounts kRealComputeSmokeCounts{1, 1, 1, 1}; +constexpr WorkloadCounts kDefaultRealComputeCounts{6, 28, 4, 1}; + +static_assert(kTileBytes == 65536, "real-compute tile must occupy 64 KiB"); +static_assert(kWorkspaceBytes == 12713984, "real-compute workspace size changed unexpectedly"); + +} // namespace pa_scheduler::ccec_workload + +#endif // PA_SCHEDULER_CCEC_WINNER_WORKLOAD_H diff --git a/tests/atomic_probe/pa_scheduler/common/host_support.h b/tests/atomic_probe/pa_scheduler/common/host_support.h index 02b0bd0d60..a82f7fc929 100644 --- a/tests/atomic_probe/pa_scheduler/common/host_support.h +++ b/tests/atomic_probe/pa_scheduler/common/host_support.h @@ -405,9 +405,16 @@ inline bool ValidateTraceHeader(const TraceHeader &header, const char *operation template inline bool ExportSwimlaneRecords( - const TraceHeader &header, const std::string &output_path, ReadRecords read_records + const TraceHeader &header, const std::string &output_path, + WinnerWorkloadMode workload_mode, const WorkloadCounts &workload_counts, + ReadRecords read_records ) { if (!ValidateTraceHeader(header, "swimlane export")) return false; + if (workload_mode != WinnerWorkloadMode::ScalarNop && + workload_mode != WinnerWorkloadMode::RealCompute) { + std::fprintf(stderr, "swimlane export rejected invalid winner workload mode.\n"); + return false; + } // 先写同目录临时文件,全部记录写完并关闭后再 rename 替换,避免把半截 JSON // 当成有效采集;这里没有 fsync 文件和目录,不承诺掉电后的持久化原子性。 @@ -428,8 +435,20 @@ inline bool ExportSwimlaneRecords( output, "{\n\"l2_swimlane_level\":1,\n" "\"metadata\":{\"clock_freq_hz\":%llu,\"num_cores\":%u," - "\"trace_schema_version\":2,\"core_types\":[", - static_cast(header.frequency_hz), kWorkers + "\"trace_schema_version\":2," + "\"winner_workload\":{\"mode\":\"%s\"," + "\"counts\":{\"qk\":%u,\"sf\":%u,\"pv\":%u,\"up\":%u}," + "\"unit\":\"%s\",\"engine_mapping\":%s},\"core_types\":[", + static_cast(header.frequency_hz), kWorkers, + workload_mode == WinnerWorkloadMode::RealCompute ? "real-compute" : "scalar-nop", + workload_counts.qk, workload_counts.sf, workload_counts.pv, workload_counts.up, + workload_mode == WinnerWorkloadMode::RealCompute + ? "complete_128x128_engine_pipeline_iteration" + : "scalar_nop_instruction", + workload_mode == WinnerWorkloadMode::RealCompute + ? "{\"qk\":\"cube_matmul\",\"sf\":\"vector_add\"," + "\"pv\":\"cube_matmul\",\"up\":\"vector_mul\"}" + : "null" ); for (uint32_t worker = 0; worker < kWorkers; ++worker) { std::fprintf(output, "%s\"%s\"", worker == 0 ? "" : ",", worker < kAicWorkers ? "aic" : "aiv"); @@ -763,6 +782,7 @@ inline Metrics Validate( bool worker_checksums_ok = true; bool fanin_worker_counts_ok = true; bool frontier_worker_counts_ok = true; + bool role_kernel_routing_ok = true; // 按真实输出大小、1 KiB 对齐和 256 MiB 环回规则重算每个 task 可接受的最小 vend。 uint64_t expected_heap_next = 0; @@ -860,6 +880,13 @@ inline Metrics Validate( worker_checksums_ok &= result.checksum == (0xcbf29ce484222325ULL ^ result.worker_id); const uint64_t worker_kernel_completions = result.kernel_counts[0] + result.kernel_counts[1] + result.kernel_counts[2] + result.kernel_counts[3]; + if (result.role == static_cast(CoreRole::Aic)) { + role_kernel_routing_ok &= result.kernel_counts[1] == 0 && result.kernel_counts[3] == 0; + } else if (result.role == static_cast(CoreRole::Aiv)) { + role_kernel_routing_ok &= result.kernel_counts[0] == 0 && result.kernel_counts[2] == 0; + } else { + role_kernel_routing_ok = false; + } const uint64_t worker_completions = result.wins[0] + worker_kernel_completions; frontier_worker_counts_ok &= result.frontier_initial_loads == worker_completions; frontier_worker_counts_ok &= result.frontier_terminal_loads == result.frontier_initial_loads; @@ -919,6 +946,10 @@ inline Metrics Validate( kernel_counts[3] == batches, "each kernel kind executes once per batch", &metrics ); + Expect( + role_kernel_routing_ok, + "AIC executes only QK/PV and AIV executes only SF/UP", &metrics + ); Expect(heap_guards == static_cast(batches) * 4, "heap guard count matches output winners", &metrics); Expect( fanin_worker_counts_ok && fanin_ready_loads >= fanin_edges && diff --git a/tests/atomic_probe/pa_scheduler/common/pa_frontend.h b/tests/atomic_probe/pa_scheduler/common/pa_frontend.h index 6b874eeaae..797ca3e2b1 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_frontend.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_frontend.h @@ -17,7 +17,8 @@ namespace pa_scheduler { // 这些基址只充当稳定的 tensor identity,供 descriptor、区间重叠和 heap 地址 -// 计算使用;NOP kernel 不读取其中的数据。context_lens 是唯一按真实 GM 指针读取的输入。 +// 计算使用;winner workload 不解引用这些 synthetic 地址,real-compute 使用独立 +// workspace。context_lens 是唯一按真实 GM 指针读取的 PA 前端输入。 constexpr uint64_t kInvalidTaskId = UINT64_MAX; constexpr uint64_t kSyntheticQueryBase = 0x200000000ULL; constexpr uint64_t kSyntheticKeyBase = 0x300000000ULL; @@ -236,7 +237,7 @@ PA_DEVICE TensorArgType TaskTag(const TaskArgs &args, uint32_t index) { PA_DEVICE void ClearDumpArgSelection(PaDumpArgSelection &selection) { // Volatile stores intentionally preserve the profiling-enabled PA reset - // traffic even when the standalone NOP kernels never consume dump data. + // traffic even though the standalone winner workload never consumes dump data. // volatile 的目的不是同步,而是阻止编译器删掉这段生产基线中存在的写流量。 volatile uint64_t *masks = &selection.dump_arg_mask; masks[0] = 0; diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index 73b42454be..8f3f436433 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -80,10 +80,10 @@ static_assert((kPayloadSlots & kPayloadMask) == 0, "payload slots must be a powe static_assert(kMaxTasks < kTaskCellCapacity, "every frontier scan must terminate on an in-range not-ready flag"); // These are the measured means from the best PA A5 trace, in 1 GHz ticks. -// The CCEC stage calibrates the NOP counts against these targets before the -// defaults are considered final. -// 这里只用 NOP 代替四个计算 kernel 的执行体;Submit、依赖、heap 与 -// completion 路径均不靠 NOP 补时。target 是真实泳道均值,不是调度阶段预算。 +// The scalar-NOP baseline calibrates its counts against these targets. +// 默认模式只用 NOP 代替四个计算 kernel;CCEC 显式 real-compute 模式改用 +// 完整 Cube/Vector 流水。两种模式的 Submit、依赖、heap 与 completion 路径 +// 均不靠 NOP 补时。target 是真实泳道均值,不是调度阶段预算。 constexpr uint32_t kTargetQkTicks = 44170; constexpr uint32_t kTargetSfTicks = 53729; constexpr uint32_t kTargetPvTicks = 27626; @@ -102,7 +102,7 @@ enum class CoreRole : uint32_t { }; // task_id % 5 即 kind;该周期性是不变量,既决定 Claim cursor/active role, -// 也决定输出大小、fanin 拓扑和 NOP kernel 的选择。 +// 也决定输出大小、fanin 拓扑和 winner workload 的选择。 enum class TaskKind : uint32_t { Alloc = 0, Qk = 1, @@ -173,6 +173,41 @@ struct NopCounts { uint32_t up; }; +// winner 的计算负载与 NOP 校准量使用两套独立计数,禁止把同一个数字同时解释成 +// scalar 指令条数和 vector/cube 工作迭代数。首阶段只有 CCEC 实现 RealCompute; +// 该 ABI 放在公共模型中,便于后续按相同配置逐步迁移 AscendC 与 CPU。 +struct WorkloadCounts { + uint32_t qk; + uint32_t sf; + uint32_t pv; + uint32_t up; +}; +static_assert(sizeof(WorkloadCounts) == 16, "workload counts ABI changed"); + +enum class WinnerWorkloadMode : uint32_t { + ScalarNop = 0, + RealCompute = 1, +}; + +constexpr uint32_t kWinnerWorkloadConfigVersion = 1; + +// 真实计算工作区是 standalone sidecar,不属于生产 DistGlobal/DistCore ABI。 +// workspace_base 指向 host 单独申请并初始化的 GM;每个 worker 只写自己的输出片段。 +struct alignas(64) WinnerWorkloadConfig { + uint32_t mode; + uint32_t version; + WorkloadCounts repeats; + uint64_t workspace_base; + uint64_t workspace_bytes; + uint32_t reserved[6]; +}; +static_assert(sizeof(WinnerWorkloadConfig) == 64, "winner workload config must occupy one cache line"); +static_assert(offsetof(WinnerWorkloadConfig, mode) == 0, "winner workload mode offset changed"); +static_assert(offsetof(WinnerWorkloadConfig, version) == 4, "winner workload version offset changed"); +static_assert(offsetof(WinnerWorkloadConfig, repeats) == 8, "winner workload counts offset changed"); +static_assert(offsetof(WinnerWorkloadConfig, workspace_base) == 24, "winner workload base offset changed"); +static_assert(offsetof(WinnerWorkloadConfig, workspace_bytes) == 32, "winner workload bytes offset changed"); + // RunConfig 是 host 在 launch 前写、worker 启动时只读的控制 cache line。 // 输入为 batch/NOP/诊断开关;输出不回写这里,而发布到独立 WorkerResult。 struct alignas(64) RunConfig { @@ -634,6 +669,7 @@ struct alignas(64) SchedulerState { // Standalone-only controls live after the complete DistGlobal image. They // therefore do not shift any cursor/task/fatal/worker address under test. RunConfig config; + WinnerWorkloadConfig winner_workload; // Context lengths are the only PA input elements read by orchestration; // keeping them in GM preserves the per-batch descriptor-based load. // 除这 256 个长度值外,其余 tensor 仅需稳定的合成地址来复现 @@ -660,6 +696,15 @@ static_assert(offsetof(SchedulerState, workers) % 64 == 0, "worker table must be static_assert(offsetof(SchedulerState, results) % 64 == 0, "result table must be cache-line aligned"); static_assert(offsetof(SchedulerState, workers) == kRealDistCoreOffset, "DistCore table offset must match PA"); static_assert(offsetof(SchedulerState, config) == kRealDistGlobalBytes, "DistGlobal byte size must match PA"); +static_assert( + offsetof(SchedulerState, winner_workload) == kRealDistGlobalBytes + sizeof(RunConfig), + "winner workload sidecar must follow RunConfig" +); +static_assert( + offsetof(SchedulerState, context_lens) == + kRealDistGlobalBytes + sizeof(RunConfig) + sizeof(WinnerWorkloadConfig), + "context lengths must follow standalone controls" +); } // namespace pa_scheduler diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index c2fec18908..12df67f5ca 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -54,6 +54,21 @@ PA_DEVICE uint32_t NopCountForKind(PA_GM const NopCounts &nops, TaskKind kind) { } } +PA_DEVICE uint32_t WorkloadCountForKind(PA_GM const WorkloadCounts &counts, TaskKind kind) { + switch (kind) { + case TaskKind::Qk: + return counts.qk; + case TaskKind::Sf: + return counts.sf; + case TaskKind::Pv: + return counts.pv; + case TaskKind::Up: + return counts.up; + default: + return 0; + } +} + PA_DEVICE uint32_t CountBits(uint32_t value) { uint32_t count = 0; while (value != 0) { @@ -189,7 +204,7 @@ PA_DEVICE uint32_t DrainReady( PA_GM SchedulerState *state, PA_GM WorkerState &worker, DrainPlace place, LocalStats &stats ) { // 同一套 drain 被三个位置复用:每次 Submit 开头的 EfDrain、ring 背压等待和所有 Submit 后的最终 drain。 - // slot 属于当前 worker;只有其全部跨核 fanin 已 ready 时才执行 NOP 模拟的 kernel、发布完成并释放 slot。 + // slot 属于当前 worker;只有其全部跨核 fanin 已 ready 时才执行所选 winner 负载、发布完成并释放 slot。 if (worker.occupied_count == 0) { return 0; } @@ -202,7 +217,7 @@ PA_DEVICE uint32_t DrainReady( } const TaskKind kind = static_cast(slot.kind + 1); const uint64_t kernel_begin = Ops::Now(); - Ops::Nop(NopCountForKind(state->config.nops, kind)); + Ops::ExecuteKernel(state, worker, kind, NopCountForKind(state->config.nops, kind)); const uint64_t kernel_end = Ops::Now(); WriteTrace( stats.trace, stats.result, static_cast(slot.task_id), static_cast(slot.kind), diff --git a/tests/atomic_probe/pa_scheduler/common/pa_trace.h b/tests/atomic_probe/pa_scheduler/common/pa_trace.h index 07270f6d25..f6f88111fb 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_trace.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_trace.h @@ -35,13 +35,15 @@ struct TraceContext { }; // Attach 只缓存本 worker 的 header 状态、分区首址和物理 lane 信息。配置先做 -// cache invalidate,确保 A5 worker 看到 host 在 launch 前写入的 trace 开关与地址。 +// cache invalidate,确保 A5 worker 看到 host 在 launch 前写入的 trace 开关、地址与 winner 负载配置。 template PA_DEVICE TraceContext AttachTrace( PA_GM SchedulerState *state, PA_GM const WorkerState &worker, uint32_t worker_id ) { TraceContext trace{nullptr, nullptr, 0, false, worker.lane, worker.block_id, static_cast(worker_id)}; - Ops::InvalidateRegion(&state->config, sizeof(state->config)); + Ops::InvalidateRegion( + &state->config, sizeof(state->config) + sizeof(state->winner_workload) + ); const uint64_t base = state->config.trace_base; const uint32_t capacity = state->config.trace_records_per_core; if ((state->config.trace_enabled & kTracePhasesEnabled) == 0 || base == 0 || capacity == 0 || diff --git a/tests/atomic_probe/pa_scheduler/cpu/main.cpp b/tests/atomic_probe/pa_scheduler/cpu/main.cpp index 53eec6d47d..b3d26ee9b2 100644 --- a/tests/atomic_probe/pa_scheduler/cpu/main.cpp +++ b/tests/atomic_probe/pa_scheduler/cpu/main.cpp @@ -129,7 +129,12 @@ struct CpuOps { return Now(); } - static inline void Nop(uint32_t count) { RuntimeNop(count); } + static inline void ExecuteKernel( + pa_scheduler::SchedulerState *, pa_scheduler::WorkerState &, pa_scheduler::TaskKind, + uint32_t nop_count + ) { + RuntimeNop(nop_count); + } static inline bool PmuWindowStart(pa_scheduler::SchedulerState *, uint32_t) { return false; } @@ -240,7 +245,12 @@ int main(int argc, char **argv) { break; } if (!pa_scheduler::host::ExportSwimlaneRecords( - *trace_header, options.swimlane_json, read_trace_records + *trace_header, options.swimlane_json, + pa_scheduler::WinnerWorkloadMode::ScalarNop, + pa_scheduler::WorkloadCounts{ + options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + }, + read_trace_records )) { postprocess_ok = false; break; diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index 3c4e0985f6..2ffdb29c58 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -40,6 +40,16 @@ CCEC-only PMU probe options (selectors are owned by the standalone Main AICPU he --pmu-icache-trials N --pmu-json FILE +CCEC-only winner workload options: + --winner-workload scalar-nop|real-compute + --real-compute-count N + --real-compute-counts QK,SF,PV,UP + +real-compute is opt-in. Its calibrated A5 defaults are QK/SF/PV/UP=6/28/4/1; +one count is one complete 128x128 load/engine/store/completion-wait pipeline +per winner task, not a scalar NOP count. Explicit count options override those +four defaults. + The swimlane action enables atomic tracing by default. For the lower-level run action, --trace-atomics still requires swimlane tracing; add --analyze-swimlane to print the per-role/per-site timing distributions. @@ -122,20 +132,22 @@ reject_managed_swimlane_options() { done } -reject_pmu_options_for_non_ccec() { +reject_ccec_only_options_for_non_ccec() { local backend="$1" shift if [[ "$backend" == "ccec" ]]; then return fi - # PMU selector、校准 NOP 和导出路径都由 CCEC Main AICPU 所有者消费。 + # PMU selector、校准 NOP、导出路径和真实 winner 负载都只由 CCEC 分支消费。 # 在顶层展开 all 之前拒绝,避免先启动 CCEC、再由其他后端迟到报错。 for argument in "$@"; do case "$argument" in --pmu-window|--pmu-window=*|--pmu-scalar-nops|--pmu-scalar-nops=*|\ - --pmu-icache-trials|--pmu-icache-trials=*|--pmu-json|--pmu-json=*) - echo "PMU option $argument is CCEC-only; backend '$backend' is not supported." >&2 + --pmu-icache-trials|--pmu-icache-trials=*|--pmu-json|--pmu-json=*|\ + --winner-workload|--winner-workload=*|--real-compute-count|--real-compute-count=*|\ + --real-compute-counts|--real-compute-counts=*) + echo "CCEC-only option $argument is not supported by backend '$backend'." >&2 exit 1 ;; esac @@ -161,7 +173,7 @@ else fi # 后端约束必须早于 build/run/smoke/swimlane 的任何文件创建、构建或设备动作。 -reject_pmu_options_for_non_ccec "$BACKEND" "$@" +reject_ccec_only_options_for_non_ccec "$BACKEND" "$@" case "$ACTION" in build) @@ -185,6 +197,17 @@ case "$ACTION" in # smoke 仍启动全部 96 个 worker,并默认注入 1 batch、1 run、0 NOP; # 后置用户参数仍由共享 parser 处理。它用于快速检查原子协议、拓扑和 # 最终状态,不作为性能数据。 + for argument in "$@"; do + case "$argument" in + --batches|--batches=*|--runs|--runs=*|--nop-count|--nop-count=*|\ + --nop-counts|--nop-counts=*|--winner-workload|--winner-workload=*|\ + --real-compute-count|--real-compute-count=*|--real-compute-counts|\ + --real-compute-counts=*) + echo "The smoke action fixes b1/r1/scalar-nop=0; use 'run ccec' for real-compute." >&2 + exit 1 + ;; + esac + done for backend in "${BACKENDS[@]}"; do run_backend "$backend" --batches 1 --runs 1 --nop-count 0 "$@" done diff --git a/tests/atomic_probe/pa_scheduler/swimlane_converter.py b/tests/atomic_probe/pa_scheduler/swimlane_converter.py index b18edbc76b..7323cf64f0 100755 --- a/tests/atomic_probe/pa_scheduler/swimlane_converter.py +++ b/tests/atomic_probe/pa_scheduler/swimlane_converter.py @@ -87,7 +87,7 @@ def _integer(value: Any, label: str) -> int: # 读取 raw JSON,校验十列结构、字段范围与可转整数值,并返回规范化视图。 def _load_and_validate( input_path: Path, -) -> tuple[int, int, list[tuple[Any, ...]], dict[tuple[int, int], int], int]: +) -> tuple[int, int, list[tuple[Any, ...]], dict[tuple[int, int], int], int, dict[str, Any]]: # raw 文件沿用真实 l2_swimlane_records.json 的十列 fdwic_events ABI: # core、block、lane、task、func、phase、start、end、flags、aux。 with input_path.open("r", encoding="utf-8") as input_file: @@ -117,6 +117,49 @@ def _load_and_validate( core_types = metadata.get("core_types") if not isinstance(core_types, list) or len(core_types) != num_cores: raise ValueError("metadata.core_types length must equal metadata.num_cores") + winner_workload = metadata.get("winner_workload") + if winner_workload is not None: + if not isinstance(winner_workload, dict): + raise ValueError("metadata.winner_workload must be a JSON object") + workload_mode = winner_workload.get("mode") + if workload_mode not in ("scalar-nop", "real-compute"): + raise ValueError("metadata.winner_workload.mode must be scalar-nop or real-compute") + workload_counts = winner_workload.get("counts") + if not isinstance(workload_counts, dict): + raise ValueError("metadata.winner_workload.counts must be a JSON object") + normalized_counts: dict[str, int] = {} + for kind in ("qk", "sf", "pv", "up"): + value = _integer( + workload_counts.get(kind), f"metadata.winner_workload.counts.{kind}" + ) + if value < 0 or (workload_mode == "real-compute" and value == 0): + raise ValueError( + f"metadata.winner_workload.counts.{kind} is invalid for {workload_mode}" + ) + normalized_counts[kind] = value + expected_unit = ( + "complete_128x128_engine_pipeline_iteration" + if workload_mode == "real-compute" + else "scalar_nop_instruction" + ) + if winner_workload.get("unit") != expected_unit: + raise ValueError( + f"metadata.winner_workload.unit must be {expected_unit!r} for {workload_mode}" + ) + engine_mapping = winner_workload.get("engine_mapping") + if workload_mode == "real-compute": + expected_mapping = { + "qk": "cube_matmul", + "sf": "vector_add", + "pv": "cube_matmul", + "up": "vector_mul", + } + if engine_mapping != expected_mapping: + raise ValueError("metadata.winner_workload.engine_mapping is invalid") + elif engine_mapping is not None: + raise ValueError("scalar-nop metadata.winner_workload.engine_mapping must be null") + # 后续 merged 顶层与 instant event 使用经过整数归一的同一份配置。 + winner_workload["counts"] = normalized_counts rows = data.get("fdwic_events") if not isinstance(rows, list) or not rows: @@ -181,7 +224,7 @@ def _load_and_validate( ) assert base_cycle is not None - return frequency_hz, trace_schema_version, rows, core_by_block_lane, base_cycle + return frequency_hz, trace_schema_version, rows, core_by_block_lane, base_cycle, metadata # 写一个 Chrome Trace Event,并统一处理数组元素间的逗号。 @@ -195,7 +238,14 @@ def _emit_event(output: TextIO, event: dict[str, Any], first: bool) -> bool: # 完成一次 raw 到 merged 的转换,成功时返回事件数、block 数和基准 cycle。 def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: - frequency_hz, trace_schema_version, rows, core_by_block_lane, base_cycle = _load_and_validate(input_path) + ( + frequency_hz, + trace_schema_version, + rows, + core_by_block_lane, + base_cycle, + capture_metadata, + ) = _load_and_validate(input_path) # 禁止原地转换;否则创建临时文件或最终 replace 时可能破坏唯一一份 raw。 if input_path.resolve() == output_path.resolve(): raise ValueError("input and output paths must differ") @@ -228,7 +278,25 @@ def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: # .tmp 再向上传播。格式/IO 错误由 main 简短报告,Ctrl-C 保留默认中断行为。 try: with temporary_path.open("w", encoding="utf-8") as output: - output.write('{"displayTimeUnit":"ns","traceEvents":[\n') + output.write('{"displayTimeUnit":"ns","metadata":') + json.dump(capture_metadata, output, ensure_ascii=False, separators=(",", ":")) + output.write(',"traceEvents":[\n') + winner_workload = capture_metadata.get("winner_workload") + if winner_workload is not None: + first = _emit_event( + output, + { + "ph": "i", + "s": "g", + "name": "pa_scheduler.capture", + "pid": 0, + "tid": 0, + "ts": 0, + "args": {"winner_workload": winner_workload}, + }, + first, + ) + emitted += 1 # 每个物理 block 建一个 process;每条硬件 lane 再拆成 runtime # 与 kernel 两个 thread,避免等待/提交阶段覆盖 kernel 执行条。 for block_id in blocks: diff --git a/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py b/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py index a2c1bc5e6c..2e2184403d 100644 --- a/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py +++ b/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py @@ -26,6 +26,47 @@ class SwimlaneConverterLayoutTest(unittest.TestCase): + def test_real_compute_metadata_is_preserved_and_visible(self) -> None: + # raw 与 merged 都必须自描述真实 engine 负载;否则同名 QK/SF/PV/UP + # span 无法与历史 scalar-NOP 泳道区分。 + workload = { + "mode": "real-compute", + "counts": {"qk": 6, "sf": 28, "pv": 4, "up": 1}, + "unit": "complete_128x128_engine_pipeline_iteration", + "engine_mapping": { + "qk": "cube_matmul", + "sf": "vector_add", + "pv": "cube_matmul", + "up": "vector_mul", + }, + } + capture = { + "l2_swimlane_level": 1, + "metadata": { + "clock_freq_hz": 1_000_000_000, + "num_cores": 1, + "trace_schema_version": 2, + "winner_workload": workload, + "core_types": ["AIC"], + }, + "fdwic_events": [[0, 0, 0, 1, 0, "Kernel", 100, 200, 0, 0]], + } + + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + emitted, blocks, base_cycle = convert(input_path, output_path) + merged = json.loads(output_path.read_text(encoding="utf-8")) + + self.assertEqual((emitted, blocks, base_cycle), (2, 1, 100)) + self.assertEqual(merged["metadata"]["winner_workload"], workload) + capture_event = next( + event for event in merged["traceEvents"] + if event.get("name") == "pa_scheduler.capture" + ) + self.assertEqual(capture_event["args"]["winner_workload"], workload) + def test_atomic_and_clock_share_the_scalar_lane(self) -> None: # 同一 mixed block 放一条 AIC 和一条 AIV0;Atomic 是 AIC scalar # 上 Claim 的子区间,ClockBaseline 也是 AIV0 scalar 指令,而 From 9aeda0dd900f56ec70650bf4958d9b272bc3da80 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 12:46:06 +0000 Subject: [PATCH 017/214] =?UTF-8?q?=E6=B5=8B=E8=AF=95(a5):=20=E4=B8=BAstan?= =?UTF-8?q?dalone=20AscendC=E6=8E=A5=E5=85=A5=E7=9C=9F=E5=AE=9ECube?= =?UTF-8?q?=E4=B8=8EVector=E8=B4=9F=E8=BD=BD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 抽取三后端共享的winner workload布局、参数解析与数值校验\n- AscendC AIC执行完整128x128 Cube矩阵乘流水并等待GM写回\n- AscendC AIV执行Vector Add/Mul流水并等待GM写回\n- 修正A1/B1共享L1的地址重叠和普通KxN右矩阵的分形转置\n- 将真计算数值结果和模式计数纳入语义门禁及泳道元数据 --- .../pa_scheduler/ascendc/pa_scheduler.asc | 292 +++++++++++++++++- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 269 ++-------------- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 20 +- .../{ccec => common}/winner_workload.h | 32 +- .../common/winner_workload_host.h | 261 ++++++++++++++++ 5 files changed, 595 insertions(+), 279 deletions(-) rename tests/atomic_probe/pa_scheduler/{ccec => common}/winner_workload.h (63%) create mode 100644 tests/atomic_probe/pa_scheduler/common/winner_workload_host.h diff --git a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc index d55ac10eff..c857f18d53 100644 --- a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc +++ b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc @@ -10,6 +10,7 @@ */ #include "../common/host_support.h" +#include "../common/winner_workload_host.h" #include "acl/acl.h" #include "kernel_operator.h" @@ -59,6 +60,179 @@ __aicore__ inline void RuntimeNop(uint32_t count) { PipeBarrier(); } +#if defined(__DAV_VEC__) +template +__aicore__ __attribute__((noinline)) void RunRealVectorWorkload( + __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats +) { + using namespace pa_scheduler::winner_workload; + // 三个 64 KiB tile 固定占用 UB 的 [0, 192 KiB),与 CCEC 的 tile 尺寸 + // 完全一致;不构造 TPipe,避免 winner 热路径增加队列初始化/析构开销。 + LocalTensor input_a_local(TPosition::VECCALC, 0x00000, kTileElements); + LocalTensor input_b_local(TPosition::VECCALC, 0x10000, kTileElements); + LocalTensor output_local(TPosition::VECCALC, 0x20000, kTileElements); + GlobalTensor input_a_global; + GlobalTensor input_b_global; + GlobalTensor output_global; + input_a_global.SetGlobalBuffer(input_a, kTileElements); + input_b_global.SetGlobalBuffer(input_b, kTileElements); + output_global.SetGlobalBuffer(output, kTileElements); + const DataCopyParams copy_params{ + 1, static_cast(kTileBytes / 32), 0, 0 + }; + + for (uint32_t iteration = 0; iteration < repeats; ++iteration) { + DataCopy(input_a_local, input_a_global, copy_params); + DataCopy(input_b_local, input_b_global, copy_params); + SetFlag(EVENT_ID0); + WaitFlag(EVENT_ID0); + if constexpr (Multiply) { + Mul(output_local, input_a_local, input_b_local, static_cast(kTileElements)); + } else { + Add(output_local, input_a_local, input_b_local, static_cast(kTileElements)); + } + SetFlag(EVENT_ID0); + WaitFlag(EVENT_ID0); + DataCopy(output_global, output_local, copy_params); + // 返回前等待本轮 MTE3 真正完成 GM 写回;否则 Kernel span 只会包住发射。 + SetFlag(EVENT_ID7); + WaitFlag(EVENT_ID7); + } +} + +__aicore__ __attribute__((noinline)) void RealVectorAdd( + __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats +) { + RunRealVectorWorkload(input_a, input_b, output, repeats); +} + +__aicore__ __attribute__((noinline)) void RealVectorMul( + __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats +) { + RunRealVectorWorkload(input_a, input_b, output, repeats); +} +#else +__aicore__ __attribute__((noinline)) void RealCubeMatmul( + __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats +) { + using namespace pa_scheduler::winner_workload; + constexpr uint16_t kMFractals = kTileRows / 16; + constexpr uint16_t kKFractals = kTileCols / 8; + // CANN 9.1 arch35 Cube 示例把 unitFlag=3 定义为最终累加并触发最终写回。 + constexpr uint8_t kFinalAccumulation = 3; + // A1 与 B1 都映射到同一块物理 L1,不能像 L0A/L0B 那样都从地址 0 + // 开始。两块 64 KiB tile 显式错开,否则第二次搬入 B 会覆盖 A, + // 常量输入下会把 2*3*128 错算成 3*3*128。 + LocalTensor input_a_l1(TPosition::A1, 0, kTileElements); + LocalTensor input_b_l1(TPosition::B1, kTileBytes, kTileElements); + LocalTensor input_a_l0(TPosition::A2, 0, kTileElements); + LocalTensor input_b_l0(TPosition::B2, 0, kTileElements); + LocalTensor output_l0(TPosition::CO1, 0, kTileElements); + GlobalTensor input_a_global; + GlobalTensor input_b_global; + GlobalTensor output_global; + input_a_global.SetGlobalBuffer(input_a, kTileElements); + input_b_global.SetGlobalBuffer(input_b, kTileElements); + output_global.SetGlobalBuffer(output, kTileElements); + + Nd2NzParams nd_to_nz; + nd_to_nz.ndNum = 1; + nd_to_nz.nValue = kTileRows; + nd_to_nz.dValue = kTileCols; + nd_to_nz.srcNdMatrixStride = 0; + nd_to_nz.srcDValue = kTileCols; + nd_to_nz.dstNzC0Stride = kTileRows; + nd_to_nz.dstNzNStride = 1; + nd_to_nz.dstNzMatrixStride = 0; + LoadData2DParamsV2 a_l1_to_l0; + a_l1_to_l0.mStartPosition = 0; + a_l1_to_l0.kStartPosition = 0; + a_l1_to_l0.mStep = kMFractals; + a_l1_to_l0.kStep = kKFractals; + a_l1_to_l0.srcStride = kMFractals; + a_l1_to_l0.dstStride = kMFractals; + a_l1_to_l0.ifTranspose = false; + // GM 中的 B 是普通 KxN row-major。CANN 9.1 的 arch35 Matmul + // LoadDataToL0B 对这种右矩阵要求在 zN(L1)->nZ(L0B) 时转置分形; + // 常量 3 无法暴露这项差异,因此不能与 A 复用同一个 false 参数。 + LoadData2DParamsV2 b_l1_to_l0 = a_l1_to_l0; + b_l1_to_l0.ifTranspose = true; + MmadParams matmul; + matmul.m = kTileRows; + matmul.n = kTileCols; + matmul.k = kTileCols; + matmul.cmatrixInitVal = true; + matmul.cmatrixSource = false; + matmul.disableGemv = true; + matmul.unitFlag = kFinalAccumulation; + DataCopyCO12DstParams copy_out; + copy_out.nSize = kTileCols; + copy_out.mSize = kTileRows; + copy_out.dstStride = kTileCols; + copy_out.srcStride = kTileRows; + copy_out.quantPre = QuantMode_t::NoQuant; + copy_out.nz2ndEn = true; + copy_out.unitFlag = kFinalAccumulation; + + for (uint32_t iteration = 0; iteration < repeats; ++iteration) { + DataCopy(input_a_l1, input_a_global, nd_to_nz); + DataCopy(input_b_l1, input_b_global, nd_to_nz); + SetFlag(EVENT_ID0); + WaitFlag(EVENT_ID0); + LoadData(input_a_l0, input_a_l1, a_l1_to_l0); + LoadData(input_b_l0, input_b_l1, b_l1_to_l0); + SetFlag(EVENT_ID0); + WaitFlag(EVENT_ID0); + Mmad(output_l0, input_a_l0, input_b_l0, matmul); + SetFlag(EVENT_ID0); + WaitFlag(EVENT_ID0); + SetFixpipeNz2ndFlag(1, 1, 1); + DataCopy(output_global, output_l0, copy_out); + // FIX_S 是 task 完成边界;没有它会把 Cube 结果尚未写回 GM 的时间漏出 span。 + SetFlag(EVENT_ID7); + WaitFlag(EVENT_ID7); + } +} +#endif + +__aicore__ __attribute__((noinline)) void ExecuteRealWinnerWorkload( + __gm__ pa_scheduler::SchedulerState *state, __gm__ pa_scheduler::WorkerState &worker, + pa_scheduler::TaskKind kind +) { + using namespace pa_scheduler::winner_workload; + const uint64_t workspace = state->winner_workload.workspace_base; + const uint32_t repeats = pa_scheduler::WorkloadCountForKind(state->winner_workload.repeats, kind); + if (state->winner_workload.version != pa_scheduler::kWinnerWorkloadConfigVersion || + workspace == 0 || state->winner_workload.workspace_bytes < kWorkspaceBytes || + worker.core_idx < 0 || static_cast(worker.core_idx) >= pa_scheduler::kWorkers || + repeats == 0 || repeats > kMaxRealComputeCount) { + return; + } +#if defined(__DAV_VEC__) + if (kind != pa_scheduler::TaskKind::Sf && kind != pa_scheduler::TaskKind::Up) return; +#else + if (kind != pa_scheduler::TaskKind::Qk && kind != pa_scheduler::TaskKind::Pv) return; +#endif + __gm__ float *input_a = reinterpret_cast<__gm__ float *>(workspace); + __gm__ float *input_b = reinterpret_cast<__gm__ float *>(workspace + kTileBytes); + const uint32_t kind_slot = + (kind == pa_scheduler::TaskKind::Pv || kind == pa_scheduler::TaskKind::Up) ? 1U : 0U; + const uint32_t output_tile = kSharedInputTiles + + static_cast(worker.core_idx) * kOutputTilesPerWorker + kind_slot; + __gm__ float *output = reinterpret_cast<__gm__ float *>( + workspace + static_cast(output_tile) * kTileBytes + ); +#if defined(__DAV_VEC__) + if (kind == pa_scheduler::TaskKind::Sf) { + RealVectorAdd(input_a, input_b, output, repeats); + } else { + RealVectorMul(input_a, input_b, output, repeats); + } +#else + RealCubeMatmul(input_a, input_b, output, repeats); +#endif +} + struct AscendcOps { // AscendC 路径尚未完成同构机器码核验,因此保留源码括号并在 raw flags 中 // 明确标记为未建立 return-ready 边界。 @@ -123,9 +297,14 @@ struct AscendcOps { } __aicore__ static inline void ExecuteKernel( - __gm__ pa_scheduler::SchedulerState *, __gm__ pa_scheduler::WorkerState &, - pa_scheduler::TaskKind, uint32_t nop_count + __gm__ pa_scheduler::SchedulerState *state, __gm__ pa_scheduler::WorkerState &worker, + pa_scheduler::TaskKind kind, uint32_t nop_count ) { + if (state->winner_workload.mode == + static_cast(pa_scheduler::WinnerWorkloadMode::RealCompute)) { + ExecuteRealWinnerWorkload(state, worker, kind); + return; + } RuntimeNop(nop_count); } @@ -199,11 +378,43 @@ __schedmode__(1) __global__ __mix__(1, 2) void pa_scheduler_ascendc(__gm__ pa_sc // launch 与 stream synchronize。 int32_t main(int32_t argc, char **argv) { pa_scheduler::host::Options options; - const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions(argc, argv, false, &options); + pa_scheduler::host::WinnerWorkloadOptions workload_options; + std::vector common_argv; + if (!pa_scheduler::host::ParseWinnerWorkloadOptions( + argc, argv, &workload_options, &common_argv + )) { + return EXIT_FAILURE; + } + // false 只表示 AscendC 可执行文件内嵌 kernel、不要求 --kernel;与是否支持 + // real-compute 无关,不能为开启真计算而改成 true。 + const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions( + static_cast(common_argv.size()), common_argv.data(), false, &options + ); if (parse_status != pa_scheduler::host::ParseStatus::Ok) { + if (parse_status == pa_scheduler::host::ParseStatus::Help) { + std::fprintf( + stderr, + "AscendC winner workload options: " + "[--winner-workload scalar-nop|real-compute] " + "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP]\n" + ); + } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } + if (!pa_scheduler::host::ValidateWinnerWorkloadOptions(workload_options)) { + return EXIT_FAILURE; + } + const bool real_compute = + workload_options.mode == pa_scheduler::WinnerWorkloadMode::RealCompute; + std::vector workload_image; + std::vector workload_outputs; + if (real_compute) { + pa_scheduler::host::InitializeWinnerWorkloadBuffers( + &workload_image, &workload_outputs + ); + } pa_scheduler::host::PrintBanner("AscendC", options); + pa_scheduler::host::PrintWinnerWorkloadConfig(workload_options, options.nops); // ACL 初始化和选卡必须先于 stream、GM 以及 kernel stub 的任何使用。 if (!CheckAcl(aclInit(nullptr), "aclInit") || !CheckAcl(aclrtSetDevice(options.device), "aclrtSetDevice")) { @@ -229,6 +440,24 @@ int32_t main(int32_t argc, char **argv) { return EXIT_FAILURE; } + // 真计算只访问本目录分配的独立 GM workspace,不解引用 PA 前端用于 identity + // 建模的 synthetic tensor 地址。初始化和 H2D 都放在 launch 计时之前。 + void *workload_device = nullptr; + if (real_compute && + !CheckAcl( + aclrtMalloc( + &workload_device, pa_scheduler::winner_workload::kWorkspaceBytes, + ACL_MEM_MALLOC_HUGE_FIRST + ), + "aclrtMalloc(real-compute workspace)" + )) { + return EXIT_FAILURE; + } + if (real_compute && (reinterpret_cast(workload_device) & 63U) != 0) { + std::fprintf(stderr, "Real-compute workspace is not 64-byte aligned: %p\n", workload_device); + return EXIT_FAILURE; + } + // trace 是独立 GM 区域,关闭泳道时完全不分配;开启时 header 与每个 // worker 的固定跨度 records 共用同一块连续内存。 void *trace_device = nullptr; @@ -254,6 +483,20 @@ int32_t main(int32_t argc, char **argv) { for (uint32_t run = 1; run <= options.runs; ++run) { pa_scheduler::host::InitializeState(state.get(), options); pa_scheduler::host::ConfigureTrace(state.get(), options, trace_device); + pa_scheduler::host::ConfigureWinnerWorkload( + state.get(), workload_options, workload_device + ); + if (real_compute && + !CheckAcl( + aclrtMemcpy( + workload_device, pa_scheduler::winner_workload::kWorkspaceBytes, + workload_image.data(), pa_scheduler::winner_workload::kWorkspaceBytes, + ACL_MEMCPY_HOST_TO_DEVICE + ), + "aclrtMemcpy(H2D real-compute workspace)" + )) { + return EXIT_FAILURE; + } if (options.trace_enabled) { pa_scheduler::host::InitializeTraceHeader(&trace_header); if (!CheckAcl( @@ -310,6 +553,23 @@ int32_t main(int32_t argc, char **argv) { )) { return EXIT_FAILURE; } + if (real_compute && + !CheckAcl( + aclrtMemcpy( + workload_outputs.data(), + static_cast(pa_scheduler::winner_workload::kOutputTiles) * + pa_scheduler::winner_workload::kTileBytes, + static_cast(workload_device) + + pa_scheduler::winner_workload::kSharedInputTiles * + pa_scheduler::winner_workload::kTileBytes, + static_cast(pa_scheduler::winner_workload::kOutputTiles) * + pa_scheduler::winner_workload::kTileBytes, + ACL_MEMCPY_DEVICE_TO_HOST + ), + "aclrtMemcpy(D2H real-compute outputs)" + )) { + return EXIT_FAILURE; + } if (options.trace_enabled && !CheckAcl( aclrtMemcpy( @@ -342,7 +602,11 @@ int32_t main(int32_t argc, char **argv) { const pa_scheduler::host::Metrics metrics = pa_scheduler::host::Validate( *state, run, host_us, options.trace_enabled ? &trace_header : nullptr ); - all_passed &= metrics.passed; + const bool workload_passed = + !real_compute || pa_scheduler::host::ValidateRealComputeOutputs( + *state, workload_outputs, run + ); + all_passed &= metrics.passed && workload_passed; spans.push_back(metrics.submit_span_us); if (options.analyze_swimlane && !pa_scheduler::host::AnalyzeSwimlaneRecords(trace_header, *state, read_trace_records)) { @@ -350,17 +614,22 @@ int32_t main(int32_t argc, char **argv) { break; } if (!options.swimlane_json.empty()) { - if (!metrics.passed) { - std::fprintf(stderr, "Skipping swimlane export because semantic validation failed.\n"); + if (!metrics.passed || !workload_passed) { + std::fprintf( + stderr, + "Skipping swimlane export because semantic or winner-workload validation failed.\n" + ); postprocess_ok = false; break; } if (!pa_scheduler::host::ExportSwimlaneRecords( trace_header, options.swimlane_json, - pa_scheduler::WinnerWorkloadMode::ScalarNop, - pa_scheduler::WorkloadCounts{ - options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up - }, + workload_options.mode, + real_compute + ? workload_options.repeats + : pa_scheduler::WorkloadCounts{ + options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + }, read_trace_records )) { postprocess_ok = false; @@ -380,6 +649,9 @@ int32_t main(int32_t argc, char **argv) { if (trace_device != nullptr) { cleanup_ok &= CheckAcl(aclrtFree(trace_device), "aclrtFree(swimlane trace)"); } + if (workload_device != nullptr) { + cleanup_ok &= CheckAcl(aclrtFree(workload_device), "aclrtFree(real-compute workspace)"); + } cleanup_ok &= CheckAcl(aclrtFree(state_device), "aclrtFree(state)"); cleanup_ok &= CheckAcl(aclrtDestroyStream(stream), "aclrtDestroyStream"); cleanup_ok &= CheckAcl(aclrtResetDevice(options.device), "aclrtResetDevice"); diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index d7f9054b3b..9b9ac83a64 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -10,9 +10,9 @@ */ #include "../common/host_support.h" +#include "../common/winner_workload_host.h" #include "pmu_owner_host.h" #include "pmu_probe.h" -#include "winner_workload.h" #include "acl/acl.h" #include "driver/ascend_hal.h" @@ -90,139 +90,12 @@ struct PmuOptions { std::string json_path; }; -struct WinnerWorkloadOptions { - pa_scheduler::WinnerWorkloadMode mode = pa_scheduler::WinnerWorkloadMode::ScalarNop; - pa_scheduler::WorkloadCounts repeats = pa_scheduler::ccec_workload::kDefaultRealComputeCounts; - bool counts_explicit = false; - bool nop_override_explicit = false; -}; - -const char *WinnerWorkloadModeName(pa_scheduler::WinnerWorkloadMode mode) { - switch (mode) { - case pa_scheduler::WinnerWorkloadMode::ScalarNop: - return "scalar-nop"; - case pa_scheduler::WinnerWorkloadMode::RealCompute: - return "real-compute"; - } - return "invalid"; -} - -bool ParseWorkloadCounts(const char *raw, pa_scheduler::WorkloadCounts *counts) { - unsigned int qk = 0; - unsigned int sf = 0; - unsigned int pv = 0; - unsigned int up = 0; - char tail = '\0'; - if (std::sscanf(raw, "%u,%u,%u,%u%c", &qk, &sf, &pv, &up, &tail) != 4) return false; - const uint32_t maximum = pa_scheduler::ccec_workload::kMaxRealComputeCount; - if (qk == 0 || sf == 0 || pv == 0 || up == 0 || - qk > maximum || sf > maximum || pv > maximum || up > maximum) { - return false; - } - *counts = pa_scheduler::WorkloadCounts{qk, sf, pv, up}; - return true; -} - -bool ParseWinnerWorkloadOptions( - int argc, char **argv, WinnerWorkloadOptions *workload, std::vector *remaining_argv -) { - bool mode_seen = false; - bool count_seen = false; - remaining_argv->clear(); - remaining_argv->push_back(argv[0]); - for (int index = 1; index < argc; ++index) { - const std::string argument = argv[index]; - if (argument == "--nop-count" || argument == "--nop-counts") { - workload->nop_override_explicit = true; - } - if (argument != "--winner-workload" && argument != "--real-compute-count" && - argument != "--real-compute-counts") { - remaining_argv->push_back(argv[index]); - continue; - } - if (index + 1 >= argc) { - std::fprintf(stderr, "Missing value after %s\n", argument.c_str()); - return false; - } - const char *value = argv[++index]; - if (argument == "--winner-workload") { - if (mode_seen) { - std::fprintf(stderr, "Specify --winner-workload only once.\n"); - return false; - } - const std::string name = value; - if (name == "scalar-nop") { - workload->mode = pa_scheduler::WinnerWorkloadMode::ScalarNop; - } else if (name == "real-compute") { - workload->mode = pa_scheduler::WinnerWorkloadMode::RealCompute; - } else { - std::fprintf( - stderr, - "Invalid --winner-workload value: %s (expected scalar-nop|real-compute)\n", - value - ); - return false; - } - mode_seen = true; - continue; - } - if (count_seen) { - std::fprintf(stderr, "Specify only one real-compute count override.\n"); - return false; - } - if (argument == "--real-compute-count") { - uint32_t count = 0; - if (!pa_scheduler::host::ParseUint( - value, 1, pa_scheduler::ccec_workload::kMaxRealComputeCount, &count - )) { - std::fprintf(stderr, "Invalid --real-compute-count value: %s\n", value); - return false; - } - workload->repeats = pa_scheduler::WorkloadCounts{count, count, count, count}; - } else if (!ParseWorkloadCounts(value, &workload->repeats)) { - std::fprintf(stderr, "Invalid --real-compute-counts value: %s\n", value); - return false; - } - count_seen = true; - workload->counts_explicit = true; - } - return true; -} - -bool ValidateWinnerWorkloadOptions(const WinnerWorkloadOptions &workload) { - if (workload.mode == pa_scheduler::WinnerWorkloadMode::RealCompute) { - if (workload.nop_override_explicit) { - std::fprintf( - stderr, - "--winner-workload real-compute cannot be combined with --nop-count or --nop-counts.\n" - ); - return false; - } - return true; - } - if (workload.counts_explicit) { - std::fprintf( - stderr, - "--real-compute-count(s) requires --winner-workload real-compute.\n" - ); - return false; - } - return true; -} - -void ConfigureWinnerWorkload( - pa_scheduler::SchedulerState *state, const WinnerWorkloadOptions &workload, - const void *workspace_device -) { - state->winner_workload.mode = static_cast(workload.mode); - state->winner_workload.version = pa_scheduler::kWinnerWorkloadConfigVersion; - state->winner_workload.repeats = workload.repeats; - state->winner_workload.workspace_base = reinterpret_cast(workspace_device); - state->winner_workload.workspace_bytes = - workload.mode == pa_scheduler::WinnerWorkloadMode::RealCompute - ? pa_scheduler::ccec_workload::kWorkspaceBytes - : 0; -} +using pa_scheduler::host::ConfigureWinnerWorkload; +using pa_scheduler::host::ParseWinnerWorkloadOptions; +using pa_scheduler::host::ValidateRealComputeOutputs; +using pa_scheduler::host::ValidateWinnerWorkloadOptions; +using pa_scheduler::host::WinnerWorkloadModeName; +using pa_scheduler::host::WinnerWorkloadOptions; const char *PmuModeName(pa_scheduler::ccec_pmu::WindowMode mode) { switch (mode) { @@ -401,80 +274,6 @@ void ConfigurePmu(pa_scheduler::SchedulerState *state, const PmuOptions &pmu, co state->config.reserved[kConfigMagic] = pmu.mode == WindowMode::Off ? 0 : kConfigMagicValue; } -const char *TaskKindName(pa_scheduler::TaskKind kind) { - switch (kind) { - case pa_scheduler::TaskKind::Qk: - return "QK"; - case pa_scheduler::TaskKind::Sf: - return "SF"; - case pa_scheduler::TaskKind::Pv: - return "PV"; - case pa_scheduler::TaskKind::Up: - return "UP"; - default: - return "invalid"; - } -} - -bool ValidateRealComputeOutputs( - const pa_scheduler::SchedulerState &state, const std::vector &outputs, uint32_t run -) { - using namespace pa_scheduler::ccec_workload; - const size_t expected_elements = static_cast(kOutputTiles) * kTileElements; - if (outputs.size() != expected_elements) { - std::fprintf( - stderr, "[ASSERT] real-compute output buffer size matches workspace layout FAIL\n" - ); - return false; - } - - uint32_t active_tiles = 0; - uint32_t inactive_tiles = 0; - for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { - const pa_scheduler::WorkerResult &result = state.results[worker]; - const bool aic = result.role == static_cast(pa_scheduler::CoreRole::Aic); - const pa_scheduler::TaskKind kinds[2] = { - aic ? pa_scheduler::TaskKind::Qk : pa_scheduler::TaskKind::Sf, - aic ? pa_scheduler::TaskKind::Pv : pa_scheduler::TaskKind::Up, - }; - for (uint32_t kind_slot = 0; kind_slot < 2; ++kind_slot) { - const pa_scheduler::TaskKind kind = kinds[kind_slot]; - const uint32_t kernel_index = static_cast(kind) - 1; - const bool active = result.kernel_counts[kernel_index] != 0; - const float expected = active - ? (aic ? kExpectedAicValue - : (kind == pa_scheduler::TaskKind::Sf ? kExpectedSfValue : kExpectedUpValue)) - : kOutputSentinel; - const size_t tile_index = - static_cast(worker) * kOutputTilesPerWorker + kind_slot; - const size_t begin = tile_index * kTileElements; - for (size_t element = 0; element < kTileElements; ++element) { - if (outputs[begin + element] == expected) continue; - std::fprintf( - stderr, - "[REAL-COMPUTE-FAIL] run=%u worker=%u kind=%s element=%zu expected=%.1f actual=%.9g\n", - run, worker, TaskKindName(kind), element, expected, - static_cast(outputs[begin + element]) - ); - std::fprintf( - stderr, "[ASSERT] real-compute output tiles match role-specific PTO results FAIL\n" - ); - return false; - } - active_tiles += active ? 1U : 0U; - inactive_tiles += active ? 0U : 1U; - } - } - const bool passed = active_tiles != 0 && - active_tiles + inactive_tiles == pa_scheduler::ccec_workload::kOutputTiles; - std::printf( - "[ASSERT] %-48s %s (active_tiles=%u inactive_sentinel_tiles=%u)\n", - "real-compute output tiles match role-specific PTO results", - passed ? "PASS" : "FAIL", active_tiles, inactive_tiles - ); - return passed; -} - struct PmuAggregate { std::vector total_cycles; std::vector window_ticks; @@ -1112,11 +911,11 @@ bool ExportPmuJson( output, ",\"tile_rows\":%u,\"tile_cols\":%u,\"shared_input_tiles\":%u," "\"output_tiles_per_worker\":%u,\"workspace_bytes\":%zu,\"role_mapping\":", - real_compute ? pa_scheduler::ccec_workload::kTileRows : 0, - real_compute ? pa_scheduler::ccec_workload::kTileCols : 0, - real_compute ? pa_scheduler::ccec_workload::kSharedInputTiles : 0, - real_compute ? pa_scheduler::ccec_workload::kOutputTilesPerWorker : 0, - real_compute ? pa_scheduler::ccec_workload::kWorkspaceBytes : 0 + real_compute ? pa_scheduler::winner_workload::kTileRows : 0, + real_compute ? pa_scheduler::winner_workload::kTileCols : 0, + real_compute ? pa_scheduler::winner_workload::kSharedInputTiles : 0, + real_compute ? pa_scheduler::winner_workload::kOutputTilesPerWorker : 0, + real_compute ? pa_scheduler::winner_workload::kWorkspaceBytes : 0 ); if (real_compute) { std::fputs( @@ -1207,7 +1006,7 @@ bool ExportPmuJson( real_compute ? "true" : "false", real_compute ? (workload_output_passed ? "true" : "false") : "null", real_compute ? active_output_tiles : 0, - real_compute ? pa_scheduler::ccec_workload::kOutputTiles - active_output_tiles : 0, + real_compute ? pa_scheduler::winner_workload::kOutputTiles - active_output_tiles : 0, real_compute && !workload_output_passed ? 1U : 0U, validation.submit_engine_observation_valid ? "true" : "false", validation.submit_engine_workers_expected, @@ -1440,28 +1239,12 @@ int main(int argc, char **argv) { std::vector workload_image; std::vector workload_outputs; if (real_compute) { - using namespace pa_scheduler::ccec_workload; - workload_image.assign(kWorkspaceTiles * kTileElements, kOutputSentinel); - std::fill_n(workload_image.begin(), kTileElements, kInputAValue); - std::fill_n(workload_image.begin() + kTileElements, kTileElements, kInputBValue); - workload_outputs.resize(static_cast(kOutputTiles) * kTileElements); - } - pa_scheduler::host::PrintBanner("CCEC", options); - if (real_compute) { - std::printf( - "[WINNER-WORKLOAD] mode=real-compute counts=%u,%u,%u,%u " - "unit=complete_128x128_engine_pipeline_iteration workspace_bytes=%zu\n", - workload_options.repeats.qk, workload_options.repeats.sf, - workload_options.repeats.pv, workload_options.repeats.up, - pa_scheduler::ccec_workload::kWorkspaceBytes - ); - } else { - std::printf( - "[WINNER-WORKLOAD] mode=scalar-nop counts=%u,%u,%u,%u " - "unit=scalar_nop_instruction workspace_bytes=0\n", - options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + pa_scheduler::host::InitializeWinnerWorkloadBuffers( + &workload_image, &workload_outputs ); } + pa_scheduler::host::PrintBanner("CCEC", options); + pa_scheduler::host::PrintWinnerWorkloadConfig(workload_options, options.nops); std::printf( "[PMU-CONFIG] window=%s calibration_scalar_nops=%u icache_trials=%u source=direct-per-core " "owner=main-aicpu-path-a\n", @@ -1509,7 +1292,7 @@ int main(int argc, char **argv) { if (real_compute && !CheckAcl( aclrtMalloc( - workload_allocation.Address(), pa_scheduler::ccec_workload::kWorkspaceBytes, + workload_allocation.Address(), pa_scheduler::winner_workload::kWorkspaceBytes, ACL_MEM_MALLOC_HUGE_FIRST ), "aclrtMalloc(real-compute workspace)" @@ -1586,8 +1369,8 @@ int main(int argc, char **argv) { if (real_compute && !CheckAcl( aclrtMemcpy( - workload_device, pa_scheduler::ccec_workload::kWorkspaceBytes, - workload_image.data(), pa_scheduler::ccec_workload::kWorkspaceBytes, + workload_device, pa_scheduler::winner_workload::kWorkspaceBytes, + workload_image.data(), pa_scheduler::winner_workload::kWorkspaceBytes, ACL_MEMCPY_HOST_TO_DEVICE ), "aclrtMemcpy(H2D real-compute workspace)" @@ -1673,13 +1456,13 @@ int main(int argc, char **argv) { !CheckAcl( aclrtMemcpy( workload_outputs.data(), - static_cast(pa_scheduler::ccec_workload::kOutputTiles) * - pa_scheduler::ccec_workload::kTileBytes, + static_cast(pa_scheduler::winner_workload::kOutputTiles) * + pa_scheduler::winner_workload::kTileBytes, static_cast(workload_device) + - pa_scheduler::ccec_workload::kSharedInputTiles * - pa_scheduler::ccec_workload::kTileBytes, - static_cast(pa_scheduler::ccec_workload::kOutputTiles) * - pa_scheduler::ccec_workload::kTileBytes, + pa_scheduler::winner_workload::kSharedInputTiles * + pa_scheduler::winner_workload::kTileBytes, + static_cast(pa_scheduler::winner_workload::kOutputTiles) * + pa_scheduler::winner_workload::kTileBytes, ACL_MEMCPY_DEVICE_TO_HOST ), "aclrtMemcpy(D2H real-compute outputs)" diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index dd1736cd51..ef3751cdbd 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -16,7 +16,7 @@ #include #include "pmu_probe.h" -#include "winner_workload.h" +#include "../common/winner_workload.h" #define PA_DEVICE __aicore__ inline #define PA_GM __gm__ @@ -59,7 +59,7 @@ __aicore__ inline void RuntimeNop(uint32_t count) { static __aicore__ __attribute__((noinline, used)) void pa_real_cube_workload_aic( __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats ) { - constexpr int kTile = static_cast(pa_scheduler::ccec_workload::kTileRows); + constexpr int kTile = static_cast(pa_scheduler::winner_workload::kTileRows); constexpr int kBlockAlign = C0_SIZE_BYTE / sizeof(float); static_assert(kTile % 16 == 0, "cube M must be 16-aligned"); static_assert(kTile % kBlockAlign == 0, "cube K/N must satisfy C0 alignment"); @@ -113,8 +113,8 @@ template __aicore__ inline void RunRealVectorWorkload( __gm__ float *input_a, __gm__ float *input_b, __gm__ float *output, uint32_t repeats ) { - constexpr int kRows = static_cast(pa_scheduler::ccec_workload::kTileRows); - constexpr int kCols = static_cast(pa_scheduler::ccec_workload::kTileCols); + constexpr int kRows = static_cast(pa_scheduler::winner_workload::kTileRows); + constexpr int kCols = static_cast(pa_scheduler::winner_workload::kTileCols); using GlobalData = GlobalTensor< float, Shape<1, 1, 1, kRows, kCols>, pto::Stride<1, 1, 1, kCols, 1>>; using TileData = Tile< @@ -176,9 +176,9 @@ static __aicore__ __attribute__((noinline, used)) void pa_execute_real_winner_wo // 闭环会把这种配置错误判为失败。 if (state->winner_workload.version != pa_scheduler::kWinnerWorkloadConfigVersion || workspace == 0 || - state->winner_workload.workspace_bytes < pa_scheduler::ccec_workload::kWorkspaceBytes || + state->winner_workload.workspace_bytes < pa_scheduler::winner_workload::kWorkspaceBytes || worker.core_idx < 0 || static_cast(worker.core_idx) >= pa_scheduler::kWorkers || - repeats == 0 || repeats > pa_scheduler::ccec_workload::kMaxRealComputeCount) { + repeats == 0 || repeats > pa_scheduler::winner_workload::kMaxRealComputeCount) { return; } #if defined(PA_BUILD_AIC) @@ -188,18 +188,18 @@ static __aicore__ __attribute__((noinline, used)) void pa_execute_real_winner_wo #endif __gm__ float *input_a = reinterpret_cast<__gm__ float *>(workspace); __gm__ float *input_b = reinterpret_cast<__gm__ float *>( - workspace + pa_scheduler::ccec_workload::kTileBytes + workspace + pa_scheduler::winner_workload::kTileBytes ); const uint32_t kind_slot = (kind == pa_scheduler::TaskKind::Pv || kind == pa_scheduler::TaskKind::Up) ? 1U : 0U; const uint32_t output_tile = - pa_scheduler::ccec_workload::kSharedInputTiles + + pa_scheduler::winner_workload::kSharedInputTiles + static_cast(worker.core_idx) * - pa_scheduler::ccec_workload::kOutputTilesPerWorker + + pa_scheduler::winner_workload::kOutputTilesPerWorker + kind_slot; __gm__ float *output = reinterpret_cast<__gm__ float *>( workspace + static_cast(output_tile) * - pa_scheduler::ccec_workload::kTileBytes + pa_scheduler::winner_workload::kTileBytes ); #if defined(PA_BUILD_AIC) pa_real_cube_workload_aic(input_a, input_b, output, repeats); diff --git a/tests/atomic_probe/pa_scheduler/ccec/winner_workload.h b/tests/atomic_probe/pa_scheduler/common/winner_workload.h similarity index 63% rename from tests/atomic_probe/pa_scheduler/ccec/winner_workload.h rename to tests/atomic_probe/pa_scheduler/common/winner_workload.h index 2ea0978423..a3995fb50f 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/winner_workload.h +++ b/tests/atomic_probe/pa_scheduler/common/winner_workload.h @@ -9,19 +9,19 @@ * ----------------------------------------------------------------------------------------------------------- */ -#ifndef PA_SCHEDULER_CCEC_WINNER_WORKLOAD_H -#define PA_SCHEDULER_CCEC_WINNER_WORKLOAD_H +#ifndef PA_SCHEDULER_COMMON_WINNER_WORKLOAD_H +#define PA_SCHEDULER_COMMON_WINNER_WORKLOAD_H -#include "../common/pa_model.h" +#include "pa_model.h" #include #include -namespace pa_scheduler::ccec_workload { +namespace pa_scheduler::winner_workload { -// 128x128 float 是仓内 A5 QK/PV cube 与 vector 示例共同验证过的基本形状。 -// 两个输入由 96 个 worker 只读共享;每个 worker 为本角色的两种 task kind -// 各占一个输出 tile,既避免写竞争,也能分别核对 QK/PV 与 SF/UP 的数值。 +// 三种 standalone 后端共享完全相同的 GM/host 布局。128x128 float 是 CCEC +// 已在 A5 上验证过的基本形状;后端可以采用不同指令接口,但不能改变输入、 +// 输出 tile 编址或 host 数值校验口径。 constexpr uint32_t kTileRows = 128; constexpr uint32_t kTileCols = 128; constexpr size_t kTileElements = static_cast(kTileRows) * kTileCols; @@ -37,21 +37,21 @@ constexpr float kExpectedAicValue = 768.0F; constexpr float kExpectedSfValue = 5.0F; constexpr float kExpectedUpValue = 6.0F; constexpr float kOutputSentinel = -12345.0F; -// 256 batch 下,即使同一 AIC 极端地拿到全部 QK/PV,128 次完整 cube -// 迭代的实测 busy 上界仍低于 32-bit PMU 的 25% 门槛;更大的未经取证值 -// 既偏离约 50 us task 目标,也可能让 submit-all 计数整圈回绕后伪装成小值。 + +// 256 batch 下,即使同一 AIC 极端地拿到全部 QK/PV,128 次完整 Cube +// 迭代的 CCEC 实测 busy 上界仍低于 32-bit PMU 的 25% 门槛。其他后端也沿用 +// 此参数边界,避免相同命令在不同实现上产生不同含义。 constexpr uint32_t kMaxRealComputeCount = 128; -// 1 次仅用于最小 b1 取证。默认次数来自本机 A5 的三个独立 256-batch -// 进程:QK/SF/PV 分别最接近 44.170/53.729/27.626 us。UP 的 1 次 -// 128x128 完整 load/vector/store/drain 已是合法下限(约 2.5 us),无法仅靠 -// 正整数 repeats 降到 1.565 us;若后续要继续贴准,应缩小 UP tile,而不是用 0 次。 +// 1 次用于最小正确性取证。默认次数来自 CCEC 的三个独立 b256 A5 进程; +// AscendC 必须重新标定后才能宣称达到同样时长,不能仅因共享默认参数便沿用 +// CCEC 的性能结论。UP 的一次完整 128x128 流水是当前正整数下限。 constexpr WorkloadCounts kRealComputeSmokeCounts{1, 1, 1, 1}; constexpr WorkloadCounts kDefaultRealComputeCounts{6, 28, 4, 1}; static_assert(kTileBytes == 65536, "real-compute tile must occupy 64 KiB"); static_assert(kWorkspaceBytes == 12713984, "real-compute workspace size changed unexpectedly"); -} // namespace pa_scheduler::ccec_workload +} // namespace pa_scheduler::winner_workload -#endif // PA_SCHEDULER_CCEC_WINNER_WORKLOAD_H +#endif // PA_SCHEDULER_COMMON_WINNER_WORKLOAD_H diff --git a/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h b/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h new file mode 100644 index 0000000000..a23dce598e --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h @@ -0,0 +1,261 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#ifndef PA_SCHEDULER_COMMON_WINNER_WORKLOAD_HOST_H +#define PA_SCHEDULER_COMMON_WINNER_WORKLOAD_HOST_H + +#include "host_support.h" +#include "winner_workload.h" + +#include +#include +#include + +namespace pa_scheduler::host { + +// winner 负载参数在通用 benchmark parser 前单独剥离,CCEC 可在其后继续剥离 +// PMU 参数;AscendC/CPU 则直接把剩余 argv 交给 ParseOptions。这样不把后端 +// 私有功能塞入公共 PA 参数结构,也不会复制三套互斥规则。 +struct WinnerWorkloadOptions { + WinnerWorkloadMode mode = WinnerWorkloadMode::ScalarNop; + WorkloadCounts repeats = winner_workload::kDefaultRealComputeCounts; + bool counts_explicit = false; + bool nop_override_explicit = false; +}; + +inline const char *WinnerWorkloadModeName(WinnerWorkloadMode mode) { + switch (mode) { + case WinnerWorkloadMode::ScalarNop: + return "scalar-nop"; + case WinnerWorkloadMode::RealCompute: + return "real-compute"; + } + return "invalid"; +} + +inline bool ParseWorkloadCounts(const char *raw, WorkloadCounts *counts) { + unsigned int qk = 0; + unsigned int sf = 0; + unsigned int pv = 0; + unsigned int up = 0; + char tail = '\0'; + if (std::sscanf(raw, "%u,%u,%u,%u%c", &qk, &sf, &pv, &up, &tail) != 4) return false; + const uint32_t maximum = winner_workload::kMaxRealComputeCount; + if (qk == 0 || sf == 0 || pv == 0 || up == 0 || + qk > maximum || sf > maximum || pv > maximum || up > maximum) { + return false; + } + *counts = WorkloadCounts{qk, sf, pv, up}; + return true; +} + +inline bool ParseWinnerWorkloadOptions( + int argc, char **argv, WinnerWorkloadOptions *workload, std::vector *remaining_argv +) { + bool mode_seen = false; + bool count_seen = false; + remaining_argv->clear(); + remaining_argv->push_back(argv[0]); + for (int index = 1; index < argc; ++index) { + const std::string argument = argv[index]; + if (argument == "--nop-count" || argument == "--nop-counts") { + workload->nop_override_explicit = true; + } + if (argument != "--winner-workload" && argument != "--real-compute-count" && + argument != "--real-compute-counts") { + remaining_argv->push_back(argv[index]); + continue; + } + if (index + 1 >= argc) { + std::fprintf(stderr, "Missing value after %s\n", argument.c_str()); + return false; + } + const char *value = argv[++index]; + if (argument == "--winner-workload") { + if (mode_seen) { + std::fprintf(stderr, "Specify --winner-workload only once.\n"); + return false; + } + const std::string name = value; + if (name == "scalar-nop") { + workload->mode = WinnerWorkloadMode::ScalarNop; + } else if (name == "real-compute") { + workload->mode = WinnerWorkloadMode::RealCompute; + } else { + std::fprintf( + stderr, + "Invalid --winner-workload value: %s (expected scalar-nop|real-compute)\n", + value + ); + return false; + } + mode_seen = true; + continue; + } + if (count_seen) { + std::fprintf(stderr, "Specify only one real-compute count override.\n"); + return false; + } + if (argument == "--real-compute-count") { + uint32_t count = 0; + if (!ParseUint(value, 1, winner_workload::kMaxRealComputeCount, &count)) { + std::fprintf(stderr, "Invalid --real-compute-count value: %s\n", value); + return false; + } + workload->repeats = WorkloadCounts{count, count, count, count}; + } else if (!ParseWorkloadCounts(value, &workload->repeats)) { + std::fprintf(stderr, "Invalid --real-compute-counts value: %s\n", value); + return false; + } + count_seen = true; + workload->counts_explicit = true; + } + return true; +} + +inline bool ValidateWinnerWorkloadOptions(const WinnerWorkloadOptions &workload) { + if (workload.mode == WinnerWorkloadMode::RealCompute) { + if (workload.nop_override_explicit) { + std::fprintf( + stderr, + "--winner-workload real-compute cannot be combined with --nop-count or --nop-counts.\n" + ); + return false; + } + return true; + } + if (workload.counts_explicit) { + std::fprintf( + stderr, + "--real-compute-count(s) requires --winner-workload real-compute.\n" + ); + return false; + } + return true; +} + +inline void ConfigureWinnerWorkload( + SchedulerState *state, const WinnerWorkloadOptions &workload, const void *workspace_device +) { + state->winner_workload.mode = static_cast(workload.mode); + state->winner_workload.version = kWinnerWorkloadConfigVersion; + state->winner_workload.repeats = workload.repeats; + state->winner_workload.workspace_base = reinterpret_cast(workspace_device); + state->winner_workload.workspace_bytes = + workload.mode == WinnerWorkloadMode::RealCompute ? winner_workload::kWorkspaceBytes : 0; +} + +inline void InitializeWinnerWorkloadBuffers( + std::vector *workspace_image, std::vector *workspace_outputs +) { + using namespace winner_workload; + workspace_image->assign(kWorkspaceTiles * kTileElements, kOutputSentinel); + std::fill_n(workspace_image->begin(), kTileElements, kInputAValue); + std::fill_n(workspace_image->begin() + kTileElements, kTileElements, kInputBValue); + workspace_outputs->resize(static_cast(kOutputTiles) * kTileElements); +} + +inline const char *TaskKindName(TaskKind kind) { + switch (kind) { + case TaskKind::Qk: + return "QK"; + case TaskKind::Sf: + return "SF"; + case TaskKind::Pv: + return "PV"; + case TaskKind::Up: + return "UP"; + default: + return "invalid"; + } +} + +inline bool ValidateRealComputeOutputs( + const SchedulerState &state, const std::vector &outputs, uint32_t run +) { + using namespace winner_workload; + const size_t expected_elements = static_cast(kOutputTiles) * kTileElements; + if (outputs.size() != expected_elements) { + std::fprintf( + stderr, "[ASSERT] real-compute output buffer size matches workspace layout FAIL\n" + ); + return false; + } + + uint32_t active_tiles = 0; + uint32_t inactive_tiles = 0; + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + const WorkerResult &result = state.results[worker]; + const bool aic = result.role == static_cast(CoreRole::Aic); + const TaskKind kinds[2] = { + aic ? TaskKind::Qk : TaskKind::Sf, + aic ? TaskKind::Pv : TaskKind::Up, + }; + for (uint32_t kind_slot = 0; kind_slot < 2; ++kind_slot) { + const TaskKind kind = kinds[kind_slot]; + const uint32_t kernel_index = static_cast(kind) - 1; + const bool active = result.kernel_counts[kernel_index] != 0; + const float expected = active + ? (aic ? kExpectedAicValue : (kind == TaskKind::Sf ? kExpectedSfValue : kExpectedUpValue)) + : kOutputSentinel; + const size_t tile_index = + static_cast(worker) * kOutputTilesPerWorker + kind_slot; + const size_t begin = tile_index * kTileElements; + for (size_t element = 0; element < kTileElements; ++element) { + if (outputs[begin + element] == expected) continue; + std::fprintf( + stderr, + "[REAL-COMPUTE-FAIL] run=%u worker=%u kind=%s element=%zu " + "expected=%.1f actual=%.9g\n", + run, worker, TaskKindName(kind), element, expected, + static_cast(outputs[begin + element]) + ); + std::fprintf( + stderr, + "[ASSERT] real-compute output tiles match role-specific engine results FAIL\n" + ); + return false; + } + active_tiles += active ? 1U : 0U; + inactive_tiles += active ? 0U : 1U; + } + } + const bool passed = active_tiles != 0 && active_tiles + inactive_tiles == kOutputTiles; + std::printf( + "[ASSERT] %-48s %s (active_tiles=%u inactive_sentinel_tiles=%u)\n", + "real-compute output tiles match role-specific engine results", + passed ? "PASS" : "FAIL", active_tiles, inactive_tiles + ); + return passed; +} + +inline void PrintWinnerWorkloadConfig( + const WinnerWorkloadOptions &workload, const NopCounts &nops +) { + if (workload.mode == WinnerWorkloadMode::RealCompute) { + std::printf( + "[WINNER-WORKLOAD] mode=real-compute counts=%u,%u,%u,%u " + "unit=complete_128x128_engine_pipeline_iteration workspace_bytes=%zu\n", + workload.repeats.qk, workload.repeats.sf, workload.repeats.pv, + workload.repeats.up, winner_workload::kWorkspaceBytes + ); + return; + } + std::printf( + "[WINNER-WORKLOAD] mode=scalar-nop counts=%u,%u,%u,%u " + "unit=scalar_nop_instruction workspace_bytes=0\n", + nops.qk, nops.sf, nops.pv, nops.up + ); +} + +} // namespace pa_scheduler::host + +#endif // PA_SCHEDULER_COMMON_WINNER_WORKLOAD_HOST_H From 1d3a374ad0e88bd1faef4df7f19a96ec5f4b7020 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 12:47:30 +0000 Subject: [PATCH 018/214] =?UTF-8?q?=E6=B5=8B=E8=AF=95(a5):=20=E8=A1=A5?= =?UTF-8?q?=E9=BD=90standalone=20CPU=E5=AF=B9=E7=AD=89=E7=AE=97=E6=9C=AF?= =?UTF-8?q?=E8=B4=9F=E8=BD=BD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - CPU按相同workspace与角色路由执行完整矩阵乘、逐元素加法和乘法\n- 保留scalar-nop回归入口且不伪造A5引擎PMU\n- 将真计算输出和泳道元数据纳入CPU语义门禁\n- 顶层runner向三后端开放统一winner workload参数并继续限制CCEC专属PMU --- tests/atomic_probe/pa_scheduler/cpu/main.cpp | 156 +++++++++++++++++-- tests/atomic_probe/pa_scheduler/run.sh | 22 +-- 2 files changed, 157 insertions(+), 21 deletions(-) diff --git a/tests/atomic_probe/pa_scheduler/cpu/main.cpp b/tests/atomic_probe/pa_scheduler/cpu/main.cpp index b3d26ee9b2..c351b75892 100644 --- a/tests/atomic_probe/pa_scheduler/cpu/main.cpp +++ b/tests/atomic_probe/pa_scheduler/cpu/main.cpp @@ -10,6 +10,7 @@ */ #include "../common/host_support.h" +#include "../common/winner_workload_host.h" #define PA_DEVICE inline #define PA_GM @@ -58,6 +59,81 @@ inline void RuntimeNop(uint32_t count) { if ((count & 1U) != 0) EmitNops<1>(); } +// CPU 后端用相同的 128x128 float 输入、输出布局执行真实算术,以便在不依赖 +// CANN 的环境中回归任务分派和输出闭环。这里的普通 CPU 浮点循环只与数学 +// 结果对等,不冒充 A5 Cube/Vector 指令、流水线或 PMU 数据。 +__attribute__((noinline)) void RunRealMatrixWorkload( + const float *input_a, const float *input_b, float *output, uint32_t repeats +) { + using namespace pa_scheduler::winner_workload; + for (uint32_t iteration = 0; iteration < repeats; ++iteration) { + for (uint32_t row = 0; row < kTileRows; ++row) { + for (uint32_t column = 0; column < kTileCols; ++column) { + float accumulator = 0.0F; + for (uint32_t inner = 0; inner < kTileCols; ++inner) { + accumulator += input_a[static_cast(row) * kTileCols + inner] * + input_b[static_cast(inner) * kTileCols + column]; + } + output[static_cast(row) * kTileCols + column] = accumulator; + } + } + // 每轮都必须把完整结果物化到输出 tile,不能让 O3 因后续轮次覆盖同一 + // tile 而删除前一轮算术;这是编译器边界,不增加硬件 PMU 或 A5 屏障语义。 + asm volatile("" : : "r"(output) : "memory"); + } +} + +template +__attribute__((noinline)) void RunRealVectorWorkload( + const float *input_a, const float *input_b, float *output, uint32_t repeats +) { + using namespace pa_scheduler::winner_workload; + for (uint32_t iteration = 0; iteration < repeats; ++iteration) { + for (size_t element = 0; element < kTileElements; ++element) { + output[element] = Multiply ? input_a[element] * input_b[element] + : input_a[element] + input_b[element]; + } + // 与矩阵路径相同,明确保留每一次完整 elementwise 迭代。 + asm volatile("" : : "r"(output) : "memory"); + } +} + +__attribute__((noinline)) void ExecuteRealWinnerWorkload( + pa_scheduler::SchedulerState *state, pa_scheduler::WorkerState &worker, + pa_scheduler::TaskKind kind +) { + using namespace pa_scheduler; + using namespace pa_scheduler::winner_workload; + const WinnerWorkloadConfig &config = state->winner_workload; + const uint32_t repeats = WorkloadCountForKind(config.repeats, kind); + const bool role_matches = + (worker.role == CoreRole::Aic && (kind == TaskKind::Qk || kind == TaskKind::Pv)) || + (worker.role == CoreRole::Aiv && (kind == TaskKind::Sf || kind == TaskKind::Up)); + // 与设备实现采用同一组版本、范围和角色门禁;配置错误不解引用 workspace, + // host 的 active tile 数值/sentinel 校验会把本轮判为失败。 + if (config.version != kWinnerWorkloadConfigVersion || config.workspace_base == 0 || + config.workspace_bytes < kWorkspaceBytes || worker.core_idx < 0 || + static_cast(worker.core_idx) >= kWorkers || repeats == 0 || + repeats > kMaxRealComputeCount || !role_matches) { + return; + } + + float *workspace = reinterpret_cast(static_cast(config.workspace_base)); + const uint32_t kind_slot = (kind == TaskKind::Pv || kind == TaskKind::Up) ? 1U : 0U; + const size_t output_tile = + kSharedInputTiles + static_cast(worker.core_idx) * kOutputTilesPerWorker + kind_slot; + float *output = workspace + output_tile * kTileElements; + const float *input_a = workspace; + const float *input_b = workspace + kTileElements; + if (worker.role == CoreRole::Aic) { + RunRealMatrixWorkload(input_a, input_b, output, repeats); + } else if (kind == TaskKind::Sf) { + RunRealVectorWorkload(input_a, input_b, output, repeats); + } else { + RunRealVectorWorkload(input_a, input_b, output, repeats); + } +} + struct CpuOps { // CPU 原子 built-in 在函数返回前已产生旧值;该后端只做协议回归,不把 // x86 时间分布外推到 A5。 @@ -130,9 +206,14 @@ struct CpuOps { } static inline void ExecuteKernel( - pa_scheduler::SchedulerState *, pa_scheduler::WorkerState &, pa_scheduler::TaskKind, + pa_scheduler::SchedulerState *state, pa_scheduler::WorkerState &worker, pa_scheduler::TaskKind kind, uint32_t nop_count ) { + if (state->winner_workload.mode == + static_cast(pa_scheduler::WinnerWorkloadMode::RealCompute)) { + ExecuteRealWinnerWorkload(state, worker, kind); + return; + } RuntimeNop(nop_count); } @@ -163,12 +244,39 @@ struct CpuOps { // 可选泳道后处理,最后统一释放 trace buffer。 int main(int argc, char **argv) { pa_scheduler::host::Options options; - const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions(argc, argv, false, &options); + pa_scheduler::host::WinnerWorkloadOptions workload_options; + std::vector common_argv; + if (!pa_scheduler::host::ParseWinnerWorkloadOptions( + argc, argv, &workload_options, &common_argv + )) { + return EXIT_FAILURE; + } + const pa_scheduler::host::ParseStatus parse_status = pa_scheduler::host::ParseOptions( + static_cast(common_argv.size()), common_argv.data(), false, &options + ); if (parse_status != pa_scheduler::host::ParseStatus::Ok) { + if (parse_status == pa_scheduler::host::ParseStatus::Help) { + std::fprintf( + stderr, + "CPU winner workload options: [--winner-workload scalar-nop|real-compute] " + "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP]\n" + ); + } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } + if (!pa_scheduler::host::ValidateWinnerWorkloadOptions(workload_options)) { + return EXIT_FAILURE; + } + const bool real_compute = + workload_options.mode == pa_scheduler::WinnerWorkloadMode::RealCompute; + std::vector workload_image; + std::vector workload_outputs; pa_scheduler::host::PrintBanner("CPU", options); - std::printf("[NOTE] CPU NOP counts preserve instruction count, not A5 microseconds.\n"); + pa_scheduler::host::PrintWinnerWorkloadConfig(workload_options, options.nops); + std::printf( + "[NOTE] CPU scalar NOP preserves instruction count; real-compute preserves arithmetic " + "and workspace semantics. Neither represents A5 engine timing or PMU.\n" + ); // SchedulerState 很大,放到 heap 而不是主线程栈;trace 继续保持独立的 // 64 字节对齐区域,以复用设备端完全相同的二进制布局。 @@ -190,6 +298,16 @@ int main(int argc, char **argv) { for (uint32_t run = 1; run <= options.runs; ++run) { pa_scheduler::host::InitializeState(state.get(), options); pa_scheduler::host::ConfigureTrace(state.get(), options, trace_memory); + if (real_compute) { + // runs>1 必须恢复所有输出 sentinel,避免上一轮 winner 的 tile 被误认 + // 为本轮结果;输入也由公共 helper 恢复成与设备后端相同的 2/3。 + pa_scheduler::host::InitializeWinnerWorkloadBuffers( + &workload_image, &workload_outputs + ); + } + pa_scheduler::host::ConfigureWinnerWorkload( + state.get(), workload_options, real_compute ? workload_image.data() : nullptr + ); if (options.trace_enabled) { pa_scheduler::host::InitializeTraceHeader(trace_header); } @@ -211,6 +329,15 @@ int main(int argc, char **argv) { worker.join(); const auto wall_end = std::chrono::steady_clock::now(); const double host_us = std::chrono::duration(wall_end - wall_begin).count(); + if (real_compute) { + const size_t output_begin = + static_cast(pa_scheduler::winner_workload::kSharedInputTiles) * + pa_scheduler::winner_workload::kTileElements; + std::copy_n( + workload_image.begin() + output_begin, workload_outputs.size(), + workload_outputs.begin() + ); + } // host 内存沿用与 A5 相同的 TraceHeader + 每 worker 固定跨度 ABI; // 分析器和 raw JSON writer 因而可以与设备后端共用同一回调接口。 const auto read_trace_records = @@ -229,7 +356,11 @@ int main(int argc, char **argv) { const pa_scheduler::host::Metrics metrics = pa_scheduler::host::Validate( *state, run, host_us, options.trace_enabled ? trace_header : nullptr ); - all_passed &= metrics.passed; + const bool workload_passed = + !real_compute || pa_scheduler::host::ValidateRealComputeOutputs( + *state, workload_outputs, run + ); + all_passed &= metrics.passed && workload_passed; spans.push_back(metrics.submit_span_us); // 分析只打印统计,导出则写 raw JSON;两者失败都标记 postprocess, // 与调度语义失败分开报告,便于区分协议问题和产物问题。 @@ -239,17 +370,22 @@ int main(int argc, char **argv) { break; } if (!options.swimlane_json.empty()) { - if (!metrics.passed) { - std::fprintf(stderr, "Skipping swimlane export because semantic validation failed.\n"); + if (!metrics.passed || !workload_passed) { + std::fprintf( + stderr, + "Skipping swimlane export because semantic or winner-workload validation failed.\n" + ); postprocess_ok = false; break; } if (!pa_scheduler::host::ExportSwimlaneRecords( *trace_header, options.swimlane_json, - pa_scheduler::WinnerWorkloadMode::ScalarNop, - pa_scheduler::WorkloadCounts{ - options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up - }, + workload_options.mode, + real_compute + ? workload_options.repeats + : pa_scheduler::WorkloadCounts{ + options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up + }, read_trace_records )) { postprocess_ok = false; diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index 2ffdb29c58..0c0261d1e2 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -40,15 +40,16 @@ CCEC-only PMU probe options (selectors are owned by the standalone Main AICPU he --pmu-icache-trials N --pmu-json FILE -CCEC-only winner workload options: +Standalone winner workload options (CCEC, AscendC, and CPU): --winner-workload scalar-nop|real-compute --real-compute-count N --real-compute-counts QK,SF,PV,UP -real-compute is opt-in. Its calibrated A5 defaults are QK/SF/PV/UP=6/28/4/1; +real-compute is opt-in. Its CCEC-calibrated A5 defaults are QK/SF/PV/UP=6/28/4/1; one count is one complete 128x128 load/engine/store/completion-wait pipeline per winner task, not a scalar NOP count. Explicit count options override those -four defaults. +four defaults. AscendC must be calibrated independently; CPU only preserves +the arithmetic and routing semantics and is not an A5 timing reference. The swimlane action enables atomic tracing by default. For the lower-level run action, --trace-atomics still requires swimlane tracing; add @@ -132,21 +133,20 @@ reject_managed_swimlane_options() { done } -reject_ccec_only_options_for_non_ccec() { +reject_ccec_pmu_options_for_non_ccec() { local backend="$1" shift if [[ "$backend" == "ccec" ]]; then return fi - # PMU selector、校准 NOP、导出路径和真实 winner 负载都只由 CCEC 分支消费。 - # 在顶层展开 all 之前拒绝,避免先启动 CCEC、再由其他后端迟到报错。 + # PMU selector、校准 NOP 和导出路径只由 CCEC 分支消费;winner workload + # 已由三个后端共同解析。这里在展开 all 前拒绝 PMU,避免先启动 CCEC、 + # 再由其他后端迟到报错。 for argument in "$@"; do case "$argument" in --pmu-window|--pmu-window=*|--pmu-scalar-nops|--pmu-scalar-nops=*|\ - --pmu-icache-trials|--pmu-icache-trials=*|--pmu-json|--pmu-json=*|\ - --winner-workload|--winner-workload=*|--real-compute-count|--real-compute-count=*|\ - --real-compute-counts|--real-compute-counts=*) + --pmu-icache-trials|--pmu-icache-trials=*|--pmu-json|--pmu-json=*) echo "CCEC-only option $argument is not supported by backend '$backend'." >&2 exit 1 ;; @@ -173,7 +173,7 @@ else fi # 后端约束必须早于 build/run/smoke/swimlane 的任何文件创建、构建或设备动作。 -reject_ccec_only_options_for_non_ccec "$BACKEND" "$@" +reject_ccec_pmu_options_for_non_ccec "$BACKEND" "$@" case "$ACTION" in build) @@ -203,7 +203,7 @@ case "$ACTION" in --nop-counts|--nop-counts=*|--winner-workload|--winner-workload=*|\ --real-compute-count|--real-compute-count=*|--real-compute-counts|\ --real-compute-counts=*) - echo "The smoke action fixes b1/r1/scalar-nop=0; use 'run ccec' for real-compute." >&2 + echo "The smoke action fixes b1/r1/scalar-nop=0; use the run action for real-compute." >&2 exit 1 ;; esac From 0c9cebc35172f6ccf5557e076a2d30b74cb90bd3 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 13:04:00 +0000 Subject: [PATCH 019/214] =?UTF-8?q?=E6=B5=8B=E8=AF=95(a5):=20=E5=A2=9E?= =?UTF-8?q?=E5=8A=A0=E7=9C=9F=E5=AE=9E=E5=BC=95=E6=93=8E=E9=9D=9E=E5=9D=87?= =?UTF-8?q?=E5=8C=80=E5=B8=83=E5=B1=80=E8=AF=8A=E6=96=AD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 为三后端增加constant与layout-diagnostic输入模式\n- 以带权对角A和非对称B逐元素验证转置、ND/NZ与输出重排\n- 将输入模式写入泳道及CCEC PMU sidecar并校验非法元数据\n- 固化CCEC、AscendC、CPU分层上板结果与真实负载使用说明\n- 保持constant性能默认及scalar-nop回归路径不变 --- ...05\345\206\265\345\210\206\346\236\220.md" | 139 +++++++++++- ...77\347\224\250\346\214\207\345\215\227.md" | 201 ++++++++++++++---- .../pa_scheduler/ascendc/pa_scheduler.asc | 10 +- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 17 +- .../pa_scheduler/common/host_support.h | 16 +- .../common/winner_workload_host.h | 115 +++++++++- tests/atomic_probe/pa_scheduler/cpu/main.cpp | 12 +- tests/atomic_probe/pa_scheduler/run.sh | 5 +- .../pa_scheduler/swimlane_converter.py | 14 ++ .../pa_scheduler/test_swimlane_converter.py | 32 +++ 10 files changed, 493 insertions(+), 68 deletions(-) diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index c8c46989fc..03dfb407fa 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -338,10 +338,12 @@ span,或单独定义 `Submit.end - Build/Replay/Alloc.end` 为 action tail。 - 单 lane Case1 的 BlockWon 动态次数为零,以及真实泳道记录格式。 QK/SF/PV/UP 默认仍可由可控 scalar NOP 模拟,旧默认值按本文最好真实泳道的 -44.170/53.729/27.626/1.565 us 校准。2026-07-18 起 CCEC 另提供显式 -`real-compute`:QK/PV 运行真实 Cube matmul,SF/UP 运行真实 Vector add/mul, -每轮都含 GM load、计算、GM store 和完成等待;AscendC/CPU 尚按独立阶段迁移。 -两种模式都不会在 Claim、Register、PrepareMap 或等待路径中硬补 5 ms。 +44.170/53.729/27.626/1.565 us 校准。2026-07-18 起三后端均提供显式 +`real-compute`:CCEC/AscendC 的 QK/PV 运行真实 Cube matmul,SF/UP 运行 +真实 Vector add/mul,每轮都含 GM load、计算、GM store 和完成等待; +CPU 使用相同 workspace、角色路由和数学运算做协议回归,不代表 A5 引擎时间或 +PMU。两种模式都不会在 Claim、Register、PrepareMap 或等待路径中硬补 +5 ms。 当前严格校验覆盖 73,728 次 Claim、每 task 唯一 winner、1,024 个 kernel、 TensorMap/heap 最终状态、fanin、flag、vend、frontier、cursor、ring placement @@ -1378,6 +1380,129 @@ QK/PV=Cube、SF/UP=Vector 映射,并增加可见的全局 capture metadata 事 data event(增加一条 capture instant),再加 256 条 process/thread metadata, 最终 `traceEvents` 为 5221 条,`dropped=0`。转换器含旧 schema 兼容在内为 5/5 PASS。 -阶段边界:本节只证明 CCEC standalone 的真实引擎负载、数值、角色、PMU 与 -泳道闭环。AscendC 真实计算、CPU 等价算术必须继续按后端分步实现和验证;在它们 -完成前,不能宣称三后端 winner 负载已经对等,更不能直接迁移真实 PA。 +当时的阶段边界:提交 `e66001ff` 只证明 CCEC standalone 的真实引擎负载、 +数值、角色、PMU 与泳道闭环;当时 AscendC 真实计算和 CPU 对等算术尚未完成, +因此本阶段没有宣称三后端 winner 负载已经对等,也没有迁移真实 PA。 +后续完成情况分别记录在 7.5.10 和 7.5.11。 + +#### 7.5.10 AscendC winner 负载迁移到真实 Cube/Vector + +提交 `9aeda0dd` 在不改动 PA 调度模型的前提下,把 CCEC 已验证的 +winner workload 布局、参数解析和 host 数值校验抽到三后端共用头文件,然后 +在 AscendC mixed kernel 内分角色接入真实计算: + +1. AIC 上的 QK/PV 执行 `128x128 float` matmul,路径为 GM ND 到 + A1/B1 的 NZ 布局、A1/B1 到 A2/B2、`Mmad`、FIX 回写 GM; + `MTE2_MTE1`、`MTE1_M`、`M_FIX` 和最后的 `FIX_S` 保证 Kernel span + 包住本轮回写完成边界; +2. AIV 上的 SF/UP 执行 GM 到 UB、`Add`/`Mul`、UB 到 GM,并用 + `MTE2_V`、`V_MTE3`、`MTE3_S` 等待完整的 load/Vector/store 流水; +3. 12,713,984-byte workspace、输入 2/3、输出 768/5/6、每 worker-kind + 独占 tile 和 inactive sentinel 与 CCEC 共用同一个口径。H2D 初始化在 + launch 计时前,D2H 与数值校验在计时后,不把 host 搬运写入 Submit span。 + +首个 AIC 版本的数值闭环暴露了真实错误:A1 和 B1 虽然是不同逻辑位置, +但映射到同一块物理 L1;两个 64 KiB tile 都从地址 0 开始时,后搬入的 B +覆盖 A,因而把本应为 `2 * 3 * 128 = 768` 的结果错算成 +`3 * 3 * 128 = 1152`。将 B1 起始地址错开 64 KiB 后,整个输出 tile 恢复为 +768。右矩阵 B 在 GM 中是普通 KxN row-major,所以 L1 zN 到 L0B nZ 时设置 +分形 transpose,而不是盲目复用 A 的非转置参数。当前常量 B=3 不能单独 +证明转置差异,因此本阶段先以 CANN 布局语义修正;随后再用 7.5.12 的非均匀 +输入逐元素上板闭合,没有把常量输出冒充布局证据。 + +修正后的分层证据为: + +| 场景 | 参数 | Submit span | 数值/调度结果 | +| ---- | ---- | ----------: | ---------------- | +| b1 | count=`1,1,1,1` | 59.280 us | PASS,4 active tile + 188 sentinel tile | +| b8 | count=`1,1,1,1` | 166.426 us | PASS,32 active tile + 160 sentinel tile | +| b256 样本 1 | count=`6,28,4,1` | 3810.471 us | PASS,191 active tile + 1 sentinel tile | +| b256 样本 2 | count=`6,28,4,1` | 4828.567 us | PASS,192 active tile | +| b256 样本 3 | count=`6,28,4,1` | 3777.371 us | PASS,192 active tile | + +三个独立 b256 进程的 Submit span 中位数为 3810.471 us。相同三个样本中, +QK/SF/PV/UP 的每 task 平均 Kernel span 分别约为 41.1–41.4 / 47.3–49.7 / +27.5–28.2 / 2.5–2.7 us。这些 span 含上述引擎完成等待,但 AscendC 路径 +没有 CCEC Main AICPU PMU owner,因此不伪造 Cube/Vector busy counter,也不用 +这三轮总时间反推 scalar PMU。 + +b1 count=1 还完成了带 atomic 的泳道闭环:4652 条 raw data event 与静态 +期望精确一致,`dropped=0`,atomic 源码边界调用为 1088。converter 增加一条 +capture instant 后有 4653 条 data event,再加 256 条 process/thread metadata, +最终 `traceEvents=4909`。raw 和 merged 都记录 `real-compute`、`1,1,1,1` +及 QK/PV=Cube、SF/UP=Vector 映射,数值校验也同轮 PASS。 + +本阶段证明 AscendC standalone 已脱离 scalar NOP 执行真实 A5 Cube/Vector +流水,且调度语义、输出和泳道闭环一致;它不是真实 PA 迁移,也不替代 +CCEC 的 PMU 取证。 + +#### 7.5.11 CPU winner 负载补齐对等算术 + +提交 `1d3a374a` 让 CPU 后端使用同一组 workspace、角色路由、repeat +参数和输出校验:QK/PV 以三重浮点循环执行完整 `128x128` 矩阵乘, +SF/UP 执行逐元素 add/mul;每个 repeat 后保留 compiler memory boundary,避免 +因为下一轮覆盖同一 tile 而被 O3 删除。每轮重新初始化 2/3 输入和输出 +sentinel,所以 `runs>1` 不会沿用上一轮 winner 结果。 + +分层回归中,scalar-NOP b1、real-compute b1 count=`1,1,1,1` 均 PASS; +后者 Submit span 为 39.821 ms,并校验 4 active tile + 188 sentinel tile。b8 +count=`2,3,2,1` 连续两轮也均 PASS,两轮分别为 80.310/54.185 ms, +均校验 25 active tile + 167 sentinel tile。同时保留 CLI 互斥和边界门禁: +real-compute 与 NOP override 不能混用,count 必须与 real-compute 模式一起使用, +取值限制为 1..128,CCEC 专属 PMU 参数仍会被非 CCEC 后端拒绝。 + +CPU 阶段的“对等”只包括 PA 任务调度、AIC/AIV 角色选择、workspace 编址、 +repeat 次数和 768/5/6 数学结果。CPU pthread 调度、普通浮点循环和 x86 atomic +都不是 A5 Cube/Vector/scalar 流水;上述毫秒数据只用于证明用例实际执行完成, +不用作 A5 timing 或 PMU 性能结论。 + +至此,三后端的 winner workload 参数、workspace、角色路由、数学输出和 +泳道 metadata 口径已对齐;CCEC/AscendC 执行真实 A5 引擎负载,CPU 仅做 +调度与算术回归。这仍然是 standalone 验证,不能替代真实 PA 的计数闭环、 +正确性和无诊断性能 A/B。 + +#### 7.5.12 用非均匀输入闭合 Cube/Vector 数据布局 + +常量 A=2、B=3 能验证真实指令、角色和 GM 写回,却会同时掩盖 B 转置、 +ND/NZ stride 与分形重排错误。为把该盲区变成可重复工具,三后端公共 CLI 增加 +`--real-compute-pattern constant|layout-diagnostic`:默认仍为 `constant`,不改变 +性能基线;诊断模式只改变计时窗外的 host 输入生成和输出期望,不进入 +`SchedulerState`,也不在 CCEC/AscendC device 热路径增加分支。 + +诊断输入定义为: + +```text +A[r,c] = (r == c) ? (r + 1) : 0 +B[r,c] = 1 + ((131*r + 17*c + 7*r*c) mod 251) +``` + +QK/PV 的期望为 `(r+1)*B[r,c]`,SF 为 `A[r,c]+B[r,c]`,UP 为 +`A[r,c]*B[r,c]`。B 是非对称稠密矩阵,A 是带权对角矩阵,因此 B 转置、A 行映射、 +stride 或输出重排都会在确定元素上产生不同整数;最大结果不超过 32128,FP32 +可做逐元素精确比较而无需容差。 + +严格按 CCEC → AscendC → CPU 顺序,以 b1、count=`1,1,1,1` 上板/运行: + +| 后端 | Submit span | QK/SF/PV/UP Kernel span | 输出门禁 | +| ---- | ----------: | ----------------------- | -------- | +| CCEC | 37.682 us | 9.172 / 3.819 / 7.415 / 2.512 us | 4 active + 188 sentinel,PASS | +| AscendC | 55.041 us | 8.478 / 4.211 / 20.801 / 3.008 us | 4 active + 188 sentinel,PASS | +| CPU | 51.958 ms | 3.457 ms / 26.118 us / 3.390 ms / 17.349 us | 4 active + 188 sentinel,PASS | + +CCEC 先证明公共期望与 PTO `A*B` 语义正确;随后 AscendC 使用同一输入通过,直接 +闭合了 A1/B1 错位、普通 KxN B 的 zN→nZ 分形 transpose、ND/NZ stride 和 +FIXPIPE NZ→ND 输出。CPU 只证明 host 公式与调度路由,不把毫秒值外推到 A5。 +三后端又以 `pattern=constant` 做 b1 count1 回归,并以 `smoke all` 回归原 +scalar-NOP,均 PASS,说明诊断模式没有静默改变性能默认或旧控制路径。 + +AscendC 同模式泳道位于 +`outputs/pa_scheduler_swimlane_20260718_125904_3613100/ascendc/`,raw 为 +4584 条 data event、`dropped=0`,merged 增加一条 capture instant;raw/merged +的 `metadata.winner_workload.input_pattern` 都是 `layout-diagnostic`。CCEC 的 +`submit-all` PMU sidecar 同样记录该字段,且 `accepted=true`、 +`semantic_passed=true`。converter 对新字段保留旧 schema-v2 兼容,并新增非法 +pattern 拒绝回归,当前两种 unittest 入口均为 6/6 PASS。 + +该诊断证明一次完整 engine pipeline 的数学和布局,不单独证明同一 task 的 N 次 +repeat 都执行;repeat 完整性仍使用 CCEC count1→count2 engine PMU 精确倍增, +不把最终覆盖同一 tile 的结果夸大成次数证明。 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 7b3f9f355a..f9ea89d465 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -28,10 +28,11 @@ frontier 和 worker 状态,任一不符都会返回失败。 有意保留的替代只有两类: -1. 默认 `scalar-nop` 模式仍以可控 NOP 标定 QK/SF/PV/UP,供三后端沿用旧基线。 - CCEC 另有显式 `real-compute` 模式:QK/PV 运行完整 Cube matmul,SF/UP - 运行完整 Vector add/mul,并执行 GM load、引擎计算、GM store 和完成等待。 - AscendC 与 CPU 的对等真计算仍按阶段迁移,当前不能把 CCEC 结果泛化到三后端。 +1. 三后端默认的 `scalar-nop` 模式仍以可控 NOP 标定 QK/SF/PV/UP,保留旧基线。 + 三后端也都已支持显式 `real-compute`:CCEC 和 AscendC 在 A5 上让 QK/PV + 执行完整 Cube matmul,让 SF/UP 执行完整 Vector add/mul,并覆盖 GM load、 + 引擎计算、GM store 和完成等待;CPU 使用同一 `128x128 float` 输入、输出布局 + 执行普通浮点 matmul/add/mul,只用于回归算术、角色路由和输出闭环。 2. simpler 的 AICPU/runtime 装载链路由本目录 host runner 代替;测试关注的 首个 Submit 到最后一个 Submit 区间不含 AICPU 初始化和最终回收。 @@ -49,14 +50,15 @@ Register、PrepareMap 或等待路径中插入虚假延时。 | 后端 | 启动形式 | atomic load | Claim fetch-max | 当前 winner 负载 | | --- | --- | --- | --- | --- | -| CCEC | 1:2 mixed AIC/AIV ELF | `atomicAdd(addr, 0)` | `atomicMax` | `scalar-nop` 或真实 Cube/Vector | -| AscendC | `__mix__(1, 2)` | `AtomicAdd(addr, 0)` | `AtomicMax` | 当前为 `scalar-nop` | -| CPU | 96 个 pthread | `fetch_add(0)` | C++17 CAS loop | 当前为 `scalar-nop` | +| CCEC | 1:2 mixed AIC/AIV ELF | `atomicAdd(addr, 0)` | `atomicMax` | `scalar-nop` 或 A5 Cube/Vector 真计算 | +| AscendC | `__mix__(1, 2)` | `AtomicAdd(addr, 0)` | `AtomicMax` | `scalar-nop` 或 A5 Cube/Vector 真计算 | +| CPU | 96 个 pthread | `fetch_add(0)` | C++17 CAS loop | `scalar-nop` 或 CPU 对等浮点算术 | AscendC 对 64 位 vend 使用 signed `AtomicAdd(addr, 0)`。CANN 9.1 虽提供 unsigned overload,但它在本 mixed kernel 的 64 MiB heap wrap 位置发生过 稳定停滞;PA vend 小于 `INT64_MAX`,因此 signed add-zero 返回的位模式与比较 -语义不变。CPU 版本用于协议和边界检查,host 线程调度耗时不能与 A5 比较。 +语义不变。CPU 版本用于协议、边界、算术和输出检查;它没有 A5 +Cube/Vector 指令、流水线、GM 搬运和 PMU 语义,host pthread 耗时不能与 A5 比较或外推。 ## 3. Winner 计算负载 @@ -82,43 +84,78 @@ Alloc 没有模拟 kernel body。NOP 数是 A5 实测校准量,不应解释为 `--nop-count N` 同时设置四类 kernel;`--nop-counts` 的顺序固定为 QK、SF、PV、UP,允许范围为 0 到 10,000,000。 -### 3.2 CCEC 真实 Cube/Vector 模式 +### 3.2 三后端 `real-compute` 模式 -CCEC 可显式切换到真实计算: +三后端共用同一组参数,可通过 `all` 按 CCEC、AscendC、CPU 的顺序统一运行: ```bash -./run.sh run ccec --winner-workload real-compute --batches 256 --runs 1 --no-swimlane +./run.sh run all \ + --winner-workload real-compute --batches 8 --runs 1 \ + --real-compute-count 1 --no-swimlane ``` -未传 count 时,QK/SF/PV/UP 默认使用 `6,28,4,1` 次完整迭代。每次迭代均为 -`128x128 float` 的完整流水,不是 scalar NOP: +也可单独选择后端: -- AIC 的 QK/PV:GM load → MTE2/MTE1 → Cube matmul → FIX → GM store → 完成等待; -- AIV 的 SF:GM load → Vector add → GM store → 完成等待; -- AIV 的 UP:GM load → Vector mul → GM store → 完成等待。 +```bash +./run.sh run ccec --winner-workload real-compute --batches 256 --runs 1 --no-swimlane +./run.sh run ascendc --winner-workload real-compute --batches 256 --runs 1 --no-swimlane +./run.sh run cpu --winner-workload real-compute --batches 1 --runs 1 \ + --real-compute-count 1 --no-swimlane +``` + +未传 count 时,QK/SF/PV/UP 默认使用 `6,28,4,1` 次完整迭代。这组默认值来自 +CCEC 的 A5 标定;AscendC 需用自身实测解读,CPU 只复用参数含义而不复用 +性能结论。每次迭代的输入、输出形状均为 `128x128 float`,不是 scalar NOP: + +- CCEC/AscendC AIC 的 QK/PV:GM load → MTE2/MTE1 → Cube matmul → FIX + → GM store → 完成等待; +- CCEC/AscendC AIV 的 SF/UP:GM load → Vector add/mul → GM store + → 完成等待; +- CPU 的 QK/PV 执行三重循环 float matmul,SF/UP 执行 elementwise add/mul; + 这是数学与工作区布局对等,不是设备引擎对等。 可以统一或分类型覆盖,范围为 1 到 128: ```bash -./run.sh run ccec --winner-workload real-compute --real-compute-count 1 -./run.sh run ccec --winner-workload real-compute --real-compute-counts 6,28,4,1 +./run.sh run all --winner-workload real-compute --real-compute-count 1 --no-swimlane +./run.sh run ascendc --winner-workload real-compute --real-compute-counts 6,28,4,1 ``` -`--real-compute-count*` 与 `--nop-count*` 不能混用;当前这些选项只允许 CCEC。 +`--real-compute-count*` 与 `--nop-count*` 不能混用;三后端全部使用相同的 +范围 1 至 128、QK/SF/PV/UP 顺序和互斥规则。 +真计算默认使用 `constant` 输入模式做性能测量;需要核验矩阵布局时使用: + +```bash +./run.sh run all \ + --winner-workload real-compute --real-compute-count 1 \ + --real-compute-pattern layout-diagnostic --batches 1 --runs 1 --no-swimlane +``` + +`layout-diagnostic` 使用带权对角矩阵 +`A[r,c]=(r==c ? r+1 : 0)` 和非对称稠密矩阵 +`B[r,c]=1+((131r+17c+7rc) mod 251)`。QK/PV 逐元素校验 +`(r+1)*B[r,c]`,SF/UP 分别校验 `A+B` 和 `A*B`,可直接发现 B 转置、 +ND/NZ stride、分形重排或 FIXPIPE 输出重排错误。输入生成与校验都在 Submit +计时窗外,设备执行路径和 workspace ABI 不增加 pattern 分支。 + workspace 包含两个只读输入 tile,并为每个 worker 按其 role 对应的两个 task kind 各保留一个独占输出 tile,共 -12,713,984 bytes。host 在计时前初始化并 H2D,计时后 D2H;所有 active +12,713,984 bytes。CCEC/AscendC host 在计时前初始化并 H2D,计时后 D2H; +CPU 直接访问同布局的 host workspace。所有 active worker-kind 的最终 tile 必须分别等于 QK/PV 的 768、SF 的 5、UP 的 6,未获胜 输出必须保留 sentinel。同一 worker-kind 的 repeat 会覆盖同一 tile,因此最终常量 -结果只证明至少完成一次;全部 repeat 的完整性另由受控 PMU count1→2 倍增取证。 -输入使用常量 2 和 3,足以验证计算、角色路由和写回,但不单独证明转置或 stride -布局;这是一项有意保留的性能替身边界。 +结果只证明至少完成一次。CCEC 的 repeat 完整性另由受控 PMU count1→2 +精确倍增取证;AscendC 不伪造 PMU,只将官方引擎循环、输出闭环与 count1/默认 +`[KERNEL]` span 缩放合并作为证据;CPU 则在每次算术迭代后保留编译器物化边界。 +性能默认输入使用常量 2 和 3,便于与既有 768/5/6 和 PMU 标定比较;它本身 +不证明转置或 stride。正式布局闭环由上述 opt-in 诊断完成,实测见 6.4 节。 -默认次数来自三个独立 b256 进程的标定:QK、SF、PV 中位数分别约 +默认次数来自三个独立 CCEC b256 进程的标定:QK、SF、PV 中位数分别约 41.336、54.039、27.971 us,接近真实 PA 的 44.170、53.729、27.626 us。 UP 的一次完整 `128x128` 流水约 2.5 us,已经是正整数迭代下限;若后续要贴近 1.565 us,应缩小 UP tile,不能用 0 次掩盖执行。真实计算下 Cube/Vector 会在 -不同物理子核并行,因此整体 Submit 约 3.6~3.8 ms,不应增加无关工作量硬凑 5.1 ms。 +不同物理子核并行,因此 CCEC 整体 Submit 约 3.6~3.8 ms,不应增加无关工作量 +硬凑 5.1 ms。AscendC 的独立实测见 6.2 节;CPU 数值不进入 A5 性能对比。 ## 4. 本机依赖和构建 @@ -158,8 +195,10 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" `ccec|ascendc|cpu|all` 用于选择后端;`all` 始终按 CCEC、AscendC、CPU 的顺序执行。 -CCEC `run/swimlane` 可使用 `--winner-workload scalar-nop|real-compute`、 -`--real-compute-count N` 或 `--real-compute-counts QK,SF,PV,UP`;真计算 count +CCEC、AscendC 和 CPU 的 `run/swimlane` 都可使用 +`--winner-workload scalar-nop|real-compute`、 +`--real-compute-count N`、`--real-compute-counts QK,SF,PV,UP` 或 +`--real-compute-pattern constant|layout-diagnostic`;真计算 count/pattern 与 `--nop-count*` 互斥。`smoke` 有意固定 scalar-nop,不接受这些覆盖项。 ### 5.1 首次回归 @@ -193,8 +232,9 @@ semantic_status=PASS postprocess_status=PASS --profile-phases --analyze-swimlane ``` -这两条未传 `--winner-workload`,都使用默认 scalar-NOP;需要 CCEC 真计算时按 -3.2 节显式传入 `--winner-workload real-compute`。 +这两条未传 `--winner-workload`,都使用默认 scalar-NOP;需要真计算时按 +3.2 节显式传入 `--winner-workload real-compute`。CPU 使用同一选项; +需同配置串行回归三后端时,直接把 backend 换成 `all`。 此模式输出指标和泳道统计,但不会自动落盘 JSON。若只关注性能且不需要完整 泳道逐事件分析,可去掉 `--analyze-swimlane`;若也不需要记录泳道,可使用 @@ -210,6 +250,10 @@ semantic_status=PASS postprocess_status=PASS ./run.sh swimlane ccec \ --device 0 --batches 1 --winner-workload real-compute --trace-atomics + +./run.sh swimlane ascendc \ + --device 0 --batches 1 --winner-workload real-compute \ + --real-compute-count 1 ``` `swimlane` action 会管理 `--runs 1` 和输出路径,因此不要再传 @@ -223,7 +267,8 @@ export PYTHON=/path/to/venv/bin/python 转换器只使用 Python 标准库,不需要 PyTorch,也不需要安装 simpler Python 包。 该 action 固定执行一轮,产物全部位于本目录的 -`outputs/pa_scheduler_swimlane__/ccec/`: +`outputs/pa_scheduler_swimlane__//`。选择 `all` 时, +同一 output root 下会按顺序建立 `ccec/`、`ascendc/` 和 `cpu/` 三个子目录: - `l2_swimlane_records.json`:与真实 PA 相同的十列 `fdwic_events` 原始格式; - `merged_swimlane.json`:Chrome Trace Event 格式,拖入 @@ -235,7 +280,12 @@ runner 结束时会打印准确目录: [SWIMLANE] output_root=.../outputs/pa_scheduler_swimlane__ ``` -真计算泳道必须同时检查 raw/merged 顶层的 `winner_workload` metadata;逻辑 +真计算泳道必须同时检查 raw/merged 顶层 +`metadata.winner_workload`;其 `mode`、`counts`、`unit`、`input_pattern` +和 `engine_mapping` +分别应为 `real-compute`、实际 QK/SF/PV/UP 次数、 +`complete_128x128_engine_pipeline_iteration`、实际输入模式以及 +`qk/pv=cube_matmul、sf=vector_add、up=vector_mul`。逻辑 `·kernel` span 只说明 winner 执行区间,单凭轨道名称不能证明使用了硬件引擎。 查看步骤: @@ -666,7 +716,8 @@ tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ ## 6. 当前 A5 结果与真实 PA 的差异 -2026-07-17 当时版本的一轮代表性结果如下。所有严格校验均为 PASS,kernel +2026-07-17 `scalar-nop` 阶段的一轮代表性结果如下。这是保留的历史 +基线,不是后续 `real-compute` 数据;当时所有严格校验均为 PASS,kernel 时间使用上表附近的校准 NOP: | 实现 | 首轮 `submit_span_us` | EfDrain/RingBp/FinalDrain | @@ -728,10 +779,85 @@ CCEC `real-compute` 已按“构建门禁 → b1 数值 → b256 标定 → PMU process/thread metadata,共 5221 条,且无 dropped record。 这组结果证明 standalone CCEC 的显式 `real-compute` 模式已不再用 scalar NOP -冒充 winner 计算,并能从 -数值、角色、PMU 和泳道四个方向闭环。它不证明 AscendC/CPU 已迁移,也不证明 -standalone 的 3.7 ms 应等于真实 PA 的 5.1 ms;真实引擎并行正是两者时序口径 -发生变化的原因之一。 +冒充 winner 计算,并能从数值、角色、PMU 和泳道四个方向闭环。这是 +CCEC 后端当时的阶段结论;AscendC 和 CPU 后续已分步补齐,见 6.2 和 6.3 节。 +三后端闭环仍不证明 standalone 的 3~5 ms 应等于真实 PA 的 5.1 ms; +真实引擎并行和代码生成差异都会改变调度时序。 + +### 6.2 2026-07-18 AscendC 真计算验证 + +AscendC 按 CCEC 之后独立接入同一 workspace 和参数规则。AIC 路径使用 +`DataCopy/LoadData/Mmad/FIX` 完成 `128x128` matmul,AIV 路径使用 +`DataCopy/Add/Mul` 完成 elementwise 计算;两条路径都在 task 发布前等待 GM +写回完成。分层上板结果为: + +| 场景 | `submit_span_us` | QK/SF/PV/UP `[KERNEL] mean_us` | 数值输出 | +| --- | ---: | --- | --- | +| b1,count=`1,1,1,1` | 59.280 | 8.837 / 14.267 / 8.342 / 11.307 | 4 active + 188 sentinel,PASS | +| b8,count=`1,1,1,1` | 166.426 | 7.713 / 2.559 / 7.694 / 2.852 | 32 active + 160 sentinel,PASS | +| b256,count=`6,28,4,1` | 3810.471 | 41.232 / 48.047 / 27.940 / 2.528 | 191 active + 1 sentinel,PASS | + +b256 三个独立进程的 Submit span 为 3810.471、4828.567 和 3777.371 us, +中位数为 **3810.471 us**。三轮都通过全部 PA 协议、角色路由和真计算输出 +断言;数据离散也说明独立进程首轮需报告中位数,不能只挑最快值。 + +`[KERNEL] mean_us` 是公共调度器对每个 winner `ExecuteKernel` 前后的 +1 GHz `SYS_CNT` span 求均值;b256 每类均为 256 个样本。它覆盖本 task 的 +GM load、Cube/Vector 计算、GM store、完成等待及少量调用边界,因此是 +**完整 winner 计算 span 均值**,不是纯 Cube/Vector busy counter,也不是 CCEC +PMU 计数。AscendC 当前不伪造 CCEC-only PMU sidecar;本阶段的取证是官方指令 +接口、完整数值 tile、角色路由和上述 span 共同闭环。 + +已生成的 AscendC b1 count=1 泳道位于: + +```text +tests/atomic_probe/pa_scheduler/outputs/ + pa_scheduler_swimlane_20260718_124326_3574886/ascendc/ + l2_swimlane_records.json + merged_swimlane.json +``` + +该轮 raw 有 4652 条 data event,merged 增加一条 capture instant 后为 4653 条 +data event,再加 256 条 process/thread metadata,`traceEvents` 合计 4909 条。 +raw 和 merged 的 `metadata.winner_workload` 均为 `mode=real-compute`、 +`counts=1,1,1,1`、`unit=complete_128x128_engine_pipeline_iteration`,且 +QK/PV、SF、UP 分别映射到 `cube_matmul`、`vector_add`、`vector_mul`。 +泳道轮次的协议、输出、record 数和 `dropped=0` 闭环全部 PASS。 + +### 6.3 2026-07-18 CPU 对等算术回归 + +CPU 在 AscendC 之后补齐了同一 `real-compute` CLI、workspace 编址、输入 +2/3、active tile 结果 768/5/6 和 inactive sentinel 校验。b1 count=1 的 +4 active + 188 sentinel 通过;b8 count=`2,3,2,1` 连续运行两轮,两轮的 +25 active + 167 sentinel 均通过,同时验证了 runs 间输出会重置为 sentinel。 + +CPU 上的 matmul/add/mul 是普通 x86 浮点循环,`[KERNEL]`、`submit_span_us` +和泳道 `·kernel` 都会被 pthread 调度、CPU cache 和编译器影响。它们只用于 +回归公共 PA 控制流与算术闭环,不得当作 A5 Cube/Vector 时间、PMU 或端到端 +性能的估计。CPU 泳道里的 `engine_mapping` 是三后端共用的逻辑 workload +映射标签,不表示 x86 上存在对应物理引擎。 + +### 6.4 2026-07-18 非均匀布局诊断 + +常量 2/3 会掩盖 `B` 转置和分形重排错误,因此在性能默认不变的前提下增加了 +`--real-compute-pattern layout-diagnostic`。按 CCEC → AscendC → CPU 的顺序, +三后端均以 b1、count=`1,1,1,1` 运行同一带权对角 A 和非对称 B,并逐元素扫描 +4 个 active tile 与 188 个 inactive sentinel tile,全部 PASS: + +| 后端 | Submit span | QK/SF/PV/UP `[KERNEL] mean_us` | 结论 | +| --- | ---: | --- | --- | +| CCEC | 37.682 us | 9.172 / 3.819 / 7.415 / 2.512 | PTO Cube/Vector 数学与布局基准 PASS | +| AscendC | 55.041 us | 8.478 / 4.211 / 20.801 / 3.008 | L1 错位、B 分形转置、ND/NZ 与 FIXPIPE 闭环 PASS | +| CPU | 51.958 ms | 3.457 ms / 26.118 us / 3.390 ms / 17.349 us | host 期望公式与路由回归 PASS;不作 A5 性能数据 | + +AscendC 诊断泳道位于 +`outputs/pa_scheduler_swimlane_20260718_125904_3613100/ascendc/`:raw 为 +4584 条 data event,merged 增加一条 capture instant,`dropped=0`;raw/merged +的 `metadata.winner_workload.input_pattern` 都是 `layout-diagnostic`。CCEC 同模式的 +`submit-all` PMU sidecar 也记录该字段且 `accepted/semantic_passed=true`。 + +诊断只证明单次完整迭代的数学与数据布局闭环;默认常量性能负载仍用于稳定比较, +CCEC 的 repeat 次数证明仍以 count1→count2 的 engine PMU 精确倍增为准。 ## 7. 内存占用和脱仓复制 @@ -739,9 +865,10 @@ standalone 的 3.7 ms 应等于真实 PA 的 5.1 ms;真实引擎并行正是 和 TensorMap,当前 `WorkerResult` 为 832 bytes,用当前头文件实际编译得到的 `SchedulerState` 为 1,007,104,896 bytes。新增的 64 bytes 是独立、对齐的 winner workload 配置 cache line,生产 DistGlobal/DistCore 关键偏移保持不变。 -默认泳道缓冲区另占 402,660,160 bytes;CCEC `real-compute` 还分配 +默认泳道缓冲区另占 402,660,160 bytes;CCEC/AscendC `real-compute` 还在 device 分配 12,713,984 bytes workspace。因此 scalar-nop+trace 的 A5 device 占用约 1.313 GiB,real-compute+trace 约 1.325 GiB,host 侧也需分配相近内存。 +CPU 后端只在 host 侧分配相同 workspace,不存在 device 内存口径。 `smoke` 不缩小 State;只有 `--no-swimlane` 能省去泳道缓冲区。 256 batch 的正常采集约有 86 万条事件;原始 JSON 和 merged JSON 都可能达到 数十至数百 MiB。writer 与 converter 都使用临时文件后原子替换,失败时不会把 diff --git a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc index c857f18d53..be198b00ef 100644 --- a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc +++ b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc @@ -396,7 +396,8 @@ int32_t main(int32_t argc, char **argv) { stderr, "AscendC winner workload options: " "[--winner-workload scalar-nop|real-compute] " - "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP]\n" + "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP] " + "[--real-compute-pattern constant|layout-diagnostic]\n" ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; @@ -410,7 +411,7 @@ int32_t main(int32_t argc, char **argv) { std::vector workload_outputs; if (real_compute) { pa_scheduler::host::InitializeWinnerWorkloadBuffers( - &workload_image, &workload_outputs + workload_options, &workload_image, &workload_outputs ); } pa_scheduler::host::PrintBanner("AscendC", options); @@ -604,7 +605,7 @@ int32_t main(int32_t argc, char **argv) { ); const bool workload_passed = !real_compute || pa_scheduler::host::ValidateRealComputeOutputs( - *state, workload_outputs, run + *state, workload_options, workload_outputs, run ); all_passed &= metrics.passed && workload_passed; spans.push_back(metrics.submit_span_us); @@ -630,6 +631,9 @@ int32_t main(int32_t argc, char **argv) { : pa_scheduler::WorkloadCounts{ options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up }, + real_compute + ? pa_scheduler::host::RealComputePatternName(workload_options.pattern) + : "none", read_trace_records )) { postprocess_ok = false; diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index 9b9ac83a64..04449d1a25 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -896,6 +896,11 @@ bool ExportPmuJson( options.profile_phases ? "true" : "false" ); WriteJsonString(output, WinnerWorkloadModeName(workload.mode)); + std::fputs(",\"input_pattern\":", output); + WriteJsonString( + output, + real_compute ? pa_scheduler::host::RealComputePatternName(workload.pattern) : "none" + ); std::fprintf( output, ",\"config_version\":%u,\"counts\":{\"qk\":%u,\"sf\":%u,\"pv\":%u,\"up\":%u}," @@ -1194,7 +1199,8 @@ int main(int argc, char **argv) { std::fprintf( stderr, "CCEC winner workload options: [--winner-workload scalar-nop|real-compute] " - "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP]\n" + "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP] " + "[--real-compute-pattern constant|layout-diagnostic]\n" ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; @@ -1240,7 +1246,7 @@ int main(int argc, char **argv) { std::vector workload_outputs; if (real_compute) { pa_scheduler::host::InitializeWinnerWorkloadBuffers( - &workload_image, &workload_outputs + workload_options, &workload_image, &workload_outputs ); } pa_scheduler::host::PrintBanner("CCEC", options); @@ -1504,7 +1510,9 @@ int main(int argc, char **argv) { ); all_passed &= metrics.passed; const bool workload_passed = - !real_compute || ValidateRealComputeOutputs(*state, workload_outputs, run); + !real_compute || ValidateRealComputeOutputs( + *state, workload_options, workload_outputs, run + ); all_passed &= workload_passed; PmuValidation pmu_validation; const bool pmu_passed = ValidatePmu( @@ -1549,6 +1557,9 @@ int main(int argc, char **argv) { : pa_scheduler::WorkloadCounts{ options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up }, + real_compute + ? pa_scheduler::host::RealComputePatternName(workload_options.pattern) + : "none", read_trace_records )) { postprocess_ok = false; diff --git a/tests/atomic_probe/pa_scheduler/common/host_support.h b/tests/atomic_probe/pa_scheduler/common/host_support.h index a82f7fc929..f68adeaf51 100644 --- a/tests/atomic_probe/pa_scheduler/common/host_support.h +++ b/tests/atomic_probe/pa_scheduler/common/host_support.h @@ -407,7 +407,7 @@ template inline bool ExportSwimlaneRecords( const TraceHeader &header, const std::string &output_path, WinnerWorkloadMode workload_mode, const WorkloadCounts &workload_counts, - ReadRecords read_records + const char *workload_pattern, ReadRecords read_records ) { if (!ValidateTraceHeader(header, "swimlane export")) return false; if (workload_mode != WinnerWorkloadMode::ScalarNop && @@ -415,6 +415,16 @@ inline bool ExportSwimlaneRecords( std::fprintf(stderr, "swimlane export rejected invalid winner workload mode.\n"); return false; } + const bool real_compute = workload_mode == WinnerWorkloadMode::RealCompute; + const bool pattern_valid = workload_pattern != nullptr && + ((real_compute && + (std::strcmp(workload_pattern, "constant") == 0 || + std::strcmp(workload_pattern, "layout-diagnostic") == 0)) || + (!real_compute && std::strcmp(workload_pattern, "none") == 0)); + if (!pattern_valid) { + std::fprintf(stderr, "swimlane export rejected invalid winner workload input pattern.\n"); + return false; + } // 先写同目录临时文件,全部记录写完并关闭后再 rename 替换,避免把半截 JSON // 当成有效采集;这里没有 fsync 文件和目录,不承诺掉电后的持久化原子性。 @@ -438,13 +448,15 @@ inline bool ExportSwimlaneRecords( "\"trace_schema_version\":2," "\"winner_workload\":{\"mode\":\"%s\"," "\"counts\":{\"qk\":%u,\"sf\":%u,\"pv\":%u,\"up\":%u}," - "\"unit\":\"%s\",\"engine_mapping\":%s},\"core_types\":[", + "\"unit\":\"%s\",\"input_pattern\":\"%s\"," + "\"engine_mapping\":%s},\"core_types\":[", static_cast(header.frequency_hz), kWorkers, workload_mode == WinnerWorkloadMode::RealCompute ? "real-compute" : "scalar-nop", workload_counts.qk, workload_counts.sf, workload_counts.pv, workload_counts.up, workload_mode == WinnerWorkloadMode::RealCompute ? "complete_128x128_engine_pipeline_iteration" : "scalar_nop_instruction", + workload_pattern, workload_mode == WinnerWorkloadMode::RealCompute ? "{\"qk\":\"cube_matmul\",\"sf\":\"vector_add\"," "\"pv\":\"cube_matmul\",\"up\":\"vector_mul\"}" diff --git a/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h b/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h index a23dce598e..da4725d0e8 100644 --- a/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h +++ b/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h @@ -21,13 +21,23 @@ namespace pa_scheduler::host { +// 常量模式用于稳定的性能负载;布局诊断模式使用带权对角 A 和非对称稠密 B, +// 专门暴露 B 转置、ND/NZ stride 与输出重排错误。该选择只影响计时窗外的 +// host 输入生成和结果校验,不进入 SchedulerState,也不增加 device 热路径分支。 +enum class RealComputePattern : uint32_t { + Constant = 0, + LayoutDiagnostic = 1, +}; + // winner 负载参数在通用 benchmark parser 前单独剥离,CCEC 可在其后继续剥离 // PMU 参数;AscendC/CPU 则直接把剩余 argv 交给 ParseOptions。这样不把后端 // 私有功能塞入公共 PA 参数结构,也不会复制三套互斥规则。 struct WinnerWorkloadOptions { WinnerWorkloadMode mode = WinnerWorkloadMode::ScalarNop; WorkloadCounts repeats = winner_workload::kDefaultRealComputeCounts; + RealComputePattern pattern = RealComputePattern::Constant; bool counts_explicit = false; + bool pattern_explicit = false; bool nop_override_explicit = false; }; @@ -41,6 +51,16 @@ inline const char *WinnerWorkloadModeName(WinnerWorkloadMode mode) { return "invalid"; } +inline const char *RealComputePatternName(RealComputePattern pattern) { + switch (pattern) { + case RealComputePattern::Constant: + return "constant"; + case RealComputePattern::LayoutDiagnostic: + return "layout-diagnostic"; + } + return "invalid"; +} + inline bool ParseWorkloadCounts(const char *raw, WorkloadCounts *counts) { unsigned int qk = 0; unsigned int sf = 0; @@ -62,6 +82,7 @@ inline bool ParseWinnerWorkloadOptions( ) { bool mode_seen = false; bool count_seen = false; + bool pattern_seen = false; remaining_argv->clear(); remaining_argv->push_back(argv[0]); for (int index = 1; index < argc; ++index) { @@ -70,7 +91,7 @@ inline bool ParseWinnerWorkloadOptions( workload->nop_override_explicit = true; } if (argument != "--winner-workload" && argument != "--real-compute-count" && - argument != "--real-compute-counts") { + argument != "--real-compute-counts" && argument != "--real-compute-pattern") { remaining_argv->push_back(argv[index]); continue; } @@ -100,6 +121,29 @@ inline bool ParseWinnerWorkloadOptions( mode_seen = true; continue; } + if (argument == "--real-compute-pattern") { + if (pattern_seen) { + std::fprintf(stderr, "Specify --real-compute-pattern only once.\n"); + return false; + } + const std::string name = value; + if (name == "constant") { + workload->pattern = RealComputePattern::Constant; + } else if (name == "layout-diagnostic") { + workload->pattern = RealComputePattern::LayoutDiagnostic; + } else { + std::fprintf( + stderr, + "Invalid --real-compute-pattern value: %s " + "(expected constant|layout-diagnostic)\n", + value + ); + return false; + } + pattern_seen = true; + workload->pattern_explicit = true; + continue; + } if (count_seen) { std::fprintf(stderr, "Specify only one real-compute count override.\n"); return false; @@ -123,6 +167,11 @@ inline bool ParseWinnerWorkloadOptions( inline bool ValidateWinnerWorkloadOptions(const WinnerWorkloadOptions &workload) { if (workload.mode == WinnerWorkloadMode::RealCompute) { + if (workload.pattern != RealComputePattern::Constant && + workload.pattern != RealComputePattern::LayoutDiagnostic) { + std::fprintf(stderr, "Invalid real-compute input pattern.\n"); + return false; + } if (workload.nop_override_explicit) { std::fprintf( stderr, @@ -132,10 +181,10 @@ inline bool ValidateWinnerWorkloadOptions(const WinnerWorkloadOptions &workload) } return true; } - if (workload.counts_explicit) { + if (workload.counts_explicit || workload.pattern_explicit) { std::fprintf( stderr, - "--real-compute-count(s) requires --winner-workload real-compute.\n" + "--real-compute-count(s)/pattern requires --winner-workload real-compute.\n" ); return false; } @@ -153,13 +202,35 @@ inline void ConfigureWinnerWorkload( workload.mode == WinnerWorkloadMode::RealCompute ? winner_workload::kWorkspaceBytes : 0; } +inline float LayoutDiagnosticInputA(uint32_t row, uint32_t column) { + return row == column ? static_cast(row + 1U) : 0.0F; +} + +inline float LayoutDiagnosticInputB(uint32_t row, uint32_t column) { + const uint32_t value = + (131U * row + 17U * column + 7U * row * column) % 251U; + return static_cast(value + 1U); +} + inline void InitializeWinnerWorkloadBuffers( - std::vector *workspace_image, std::vector *workspace_outputs + const WinnerWorkloadOptions &workload, std::vector *workspace_image, + std::vector *workspace_outputs ) { using namespace winner_workload; workspace_image->assign(kWorkspaceTiles * kTileElements, kOutputSentinel); - std::fill_n(workspace_image->begin(), kTileElements, kInputAValue); - std::fill_n(workspace_image->begin() + kTileElements, kTileElements, kInputBValue); + if (workload.pattern == RealComputePattern::Constant) { + std::fill_n(workspace_image->begin(), kTileElements, kInputAValue); + std::fill_n(workspace_image->begin() + kTileElements, kTileElements, kInputBValue); + } else { + for (uint32_t row = 0; row < kTileRows; ++row) { + for (uint32_t column = 0; column < kTileCols; ++column) { + const size_t element = static_cast(row) * kTileCols + column; + (*workspace_image)[element] = LayoutDiagnosticInputA(row, column); + (*workspace_image)[kTileElements + element] = + LayoutDiagnosticInputB(row, column); + } + } + } workspace_outputs->resize(static_cast(kOutputTiles) * kTileElements); } @@ -178,8 +249,27 @@ inline const char *TaskKindName(TaskKind kind) { } } +inline float ExpectedRealComputeValue( + RealComputePattern pattern, TaskKind kind, uint32_t row, uint32_t column +) { + using namespace winner_workload; + if (pattern == RealComputePattern::Constant) { + return kind == TaskKind::Qk || kind == TaskKind::Pv + ? kExpectedAicValue + : (kind == TaskKind::Sf ? kExpectedSfValue : kExpectedUpValue); + } + const float input_a = LayoutDiagnosticInputA(row, column); + const float input_b = LayoutDiagnosticInputB(row, column); + if (kind == TaskKind::Qk || kind == TaskKind::Pv) { + // A 是带权对角矩阵,因此 A*B 的 (row,column) 只有一个非零乘积。 + return static_cast(row + 1U) * input_b; + } + return kind == TaskKind::Sf ? input_a + input_b : input_a * input_b; +} + inline bool ValidateRealComputeOutputs( - const SchedulerState &state, const std::vector &outputs, uint32_t run + const SchedulerState &state, const WinnerWorkloadOptions &workload, + const std::vector &outputs, uint32_t run ) { using namespace winner_workload; const size_t expected_elements = static_cast(kOutputTiles) * kTileElements; @@ -203,13 +293,15 @@ inline bool ValidateRealComputeOutputs( const TaskKind kind = kinds[kind_slot]; const uint32_t kernel_index = static_cast(kind) - 1; const bool active = result.kernel_counts[kernel_index] != 0; - const float expected = active - ? (aic ? kExpectedAicValue : (kind == TaskKind::Sf ? kExpectedSfValue : kExpectedUpValue)) - : kOutputSentinel; const size_t tile_index = static_cast(worker) * kOutputTilesPerWorker + kind_slot; const size_t begin = tile_index * kTileElements; for (size_t element = 0; element < kTileElements; ++element) { + const uint32_t row = static_cast(element / kTileCols); + const uint32_t column = static_cast(element % kTileCols); + const float expected = active + ? ExpectedRealComputeValue(workload.pattern, kind, row, column) + : kOutputSentinel; if (outputs[begin + element] == expected) continue; std::fprintf( stderr, @@ -242,8 +334,9 @@ inline void PrintWinnerWorkloadConfig( ) { if (workload.mode == WinnerWorkloadMode::RealCompute) { std::printf( - "[WINNER-WORKLOAD] mode=real-compute counts=%u,%u,%u,%u " + "[WINNER-WORKLOAD] mode=real-compute pattern=%s counts=%u,%u,%u,%u " "unit=complete_128x128_engine_pipeline_iteration workspace_bytes=%zu\n", + RealComputePatternName(workload.pattern), workload.repeats.qk, workload.repeats.sf, workload.repeats.pv, workload.repeats.up, winner_workload::kWorkspaceBytes ); diff --git a/tests/atomic_probe/pa_scheduler/cpu/main.cpp b/tests/atomic_probe/pa_scheduler/cpu/main.cpp index c351b75892..7a188326a0 100644 --- a/tests/atomic_probe/pa_scheduler/cpu/main.cpp +++ b/tests/atomic_probe/pa_scheduler/cpu/main.cpp @@ -259,7 +259,8 @@ int main(int argc, char **argv) { std::fprintf( stderr, "CPU winner workload options: [--winner-workload scalar-nop|real-compute] " - "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP]\n" + "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP] " + "[--real-compute-pattern constant|layout-diagnostic]\n" ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; @@ -300,9 +301,9 @@ int main(int argc, char **argv) { pa_scheduler::host::ConfigureTrace(state.get(), options, trace_memory); if (real_compute) { // runs>1 必须恢复所有输出 sentinel,避免上一轮 winner 的 tile 被误认 - // 为本轮结果;输入也由公共 helper 恢复成与设备后端相同的 2/3。 + // 为本轮结果;输入也由公共 helper 恢复成与设备后端相同的选定 pattern。 pa_scheduler::host::InitializeWinnerWorkloadBuffers( - &workload_image, &workload_outputs + workload_options, &workload_image, &workload_outputs ); } pa_scheduler::host::ConfigureWinnerWorkload( @@ -358,7 +359,7 @@ int main(int argc, char **argv) { ); const bool workload_passed = !real_compute || pa_scheduler::host::ValidateRealComputeOutputs( - *state, workload_outputs, run + *state, workload_options, workload_outputs, run ); all_passed &= metrics.passed && workload_passed; spans.push_back(metrics.submit_span_us); @@ -386,6 +387,9 @@ int main(int argc, char **argv) { : pa_scheduler::WorkloadCounts{ options.nops.qk, options.nops.sf, options.nops.pv, options.nops.up }, + real_compute + ? pa_scheduler::host::RealComputePatternName(workload_options.pattern) + : "none", read_trace_records )) { postprocess_ok = false; diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index 0c0261d1e2..328f024229 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -44,12 +44,15 @@ Standalone winner workload options (CCEC, AscendC, and CPU): --winner-workload scalar-nop|real-compute --real-compute-count N --real-compute-counts QK,SF,PV,UP + --real-compute-pattern constant|layout-diagnostic real-compute is opt-in. Its CCEC-calibrated A5 defaults are QK/SF/PV/UP=6/28/4/1; one count is one complete 128x128 load/engine/store/completion-wait pipeline per winner task, not a scalar NOP count. Explicit count options override those four defaults. AscendC must be calibrated independently; CPU only preserves the arithmetic and routing semantics and is not an A5 timing reference. +The constant pattern is the performance default. layout-diagnostic uses a +weighted diagonal A and asymmetric B to detect transpose/stride/reorder bugs. The swimlane action enables atomic tracing by default. For the lower-level run action, --trace-atomics still requires swimlane tracing; add @@ -202,7 +205,7 @@ case "$ACTION" in --batches|--batches=*|--runs|--runs=*|--nop-count|--nop-count=*|\ --nop-counts|--nop-counts=*|--winner-workload|--winner-workload=*|\ --real-compute-count|--real-compute-count=*|--real-compute-counts|\ - --real-compute-counts=*) + --real-compute-counts=*|--real-compute-pattern|--real-compute-pattern=*) echo "The smoke action fixes b1/r1/scalar-nop=0; use the run action for real-compute." >&2 exit 1 ;; diff --git a/tests/atomic_probe/pa_scheduler/swimlane_converter.py b/tests/atomic_probe/pa_scheduler/swimlane_converter.py index 7323cf64f0..7b258b9b99 100755 --- a/tests/atomic_probe/pa_scheduler/swimlane_converter.py +++ b/tests/atomic_probe/pa_scheduler/swimlane_converter.py @@ -146,6 +146,20 @@ def _load_and_validate( raise ValueError( f"metadata.winner_workload.unit must be {expected_unit!r} for {workload_mode}" ) + # input_pattern 是 real-compute 布局诊断新增的可选元数据。旧 schema-v2 + # 文件没有该字段,仍保持可读;新采集若给出则必须与 workload 模式一致。 + input_pattern = winner_workload.get("input_pattern") + if input_pattern is not None: + valid_patterns = ( + {"constant", "layout-diagnostic"} + if workload_mode == "real-compute" + else {"none"} + ) + if input_pattern not in valid_patterns: + raise ValueError( + "metadata.winner_workload.input_pattern is invalid for " + f"{workload_mode}" + ) engine_mapping = winner_workload.get("engine_mapping") if workload_mode == "real-compute": expected_mapping = { diff --git a/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py b/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py index 2e2184403d..60ae046fc6 100644 --- a/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py +++ b/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py @@ -33,6 +33,7 @@ def test_real_compute_metadata_is_preserved_and_visible(self) -> None: "mode": "real-compute", "counts": {"qk": 6, "sf": 28, "pv": 4, "up": 1}, "unit": "complete_128x128_engine_pipeline_iteration", + "input_pattern": "layout-diagnostic", "engine_mapping": { "qk": "cube_matmul", "sf": "vector_add", @@ -67,6 +68,37 @@ def test_real_compute_metadata_is_preserved_and_visible(self) -> None: ) self.assertEqual(capture_event["args"]["winner_workload"], workload) + def test_invalid_real_compute_input_pattern_is_rejected(self) -> None: + capture = { + "l2_swimlane_level": 1, + "metadata": { + "clock_freq_hz": 1_000_000_000, + "num_cores": 1, + "trace_schema_version": 2, + "winner_workload": { + "mode": "real-compute", + "counts": {"qk": 1, "sf": 1, "pv": 1, "up": 1}, + "unit": "complete_128x128_engine_pipeline_iteration", + "input_pattern": "unknown-layout", + "engine_mapping": { + "qk": "cube_matmul", + "sf": "vector_add", + "pv": "cube_matmul", + "up": "vector_mul", + }, + }, + "core_types": ["AIC"], + }, + "fdwic_events": [[0, 0, 0, 1, 0, "Kernel", 100, 200, 0, 0]], + } + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "input_pattern is invalid"): + convert(input_path, output_path) + self.assertFalse(output_path.exists()) + def test_atomic_and_clock_share_the_scalar_lane(self) -> None: # 同一 mixed block 放一条 AIC 和一条 AIV0;Atomic 是 AIC scalar # 上 Claim 的子区间,ClockBaseline 也是 AIV0 scalar 指令,而 From 7bb118a8ebe1b7ce31a567f09240b1435ebba515 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 13:56:30 +0000 Subject: [PATCH 020/214] =?UTF-8?q?=E6=B5=8B=E8=AF=95(a5):=20=E9=BB=98?= =?UTF-8?q?=E8=AE=A4=E5=90=AF=E7=94=A8standalone=E7=9C=9F=E5=AE=9E?= =?UTF-8?q?=E5=BC=95=E6=93=8E=E8=B4=9F=E8=BD=BD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 将三后端无参数 winner 负载切换为 real-compute/6,28,4,1。 保留 NOP override 兼容入口并让 smoke 显式使用零 NOP;同时记录 Case1 fanin 拓扑、数值数据流边界和同泳道口径性能。 --- ...05\345\206\265\345\210\206\346\236\220.md" | 96 ++++++++++++---- ...77\347\224\250\346\214\207\345\215\227.md" | 106 ++++++++++++++---- .../pa_scheduler/ascendc/pa_scheduler.asc | 5 + tests/atomic_probe/pa_scheduler/ccec/host.cpp | 5 + .../pa_scheduler/common/pa_model.h | 9 +- .../common/winner_workload_host.h | 11 +- tests/atomic_probe/pa_scheduler/cpu/main.cpp | 5 + tests/atomic_probe/pa_scheduler/run.sh | 7 +- 8 files changed, 195 insertions(+), 49 deletions(-) diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 03dfb407fa..bf8a33318f 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -337,13 +337,32 @@ span,或单独定义 `Submit.end - Build/Replay/Alloc.end` 为 action tail。 - EfDrain、Replay、WaitForSlot、HeapGuard、flag/vend/frontier 和最终 drain; - 单 lane Case1 的 BlockWon 动态次数为零,以及真实泳道记录格式。 -QK/SF/PV/UP 默认仍可由可控 scalar NOP 模拟,旧默认值按本文最好真实泳道的 -44.170/53.729/27.626/1.565 us 校准。2026-07-18 起三后端均提供显式 -`real-compute`:CCEC/AscendC 的 QK/PV 运行真实 Cube matmul,SF/UP 运行 -真实 Vector add/mul,每轮都含 GM load、计算、GM store 和完成等待; -CPU 使用相同 workspace、角色路由和数学运算做协议回归,不代表 A5 引擎时间或 -PMU。两种模式都不会在 Claim、Register、PrepareMap 或等待路径中硬补 -5 ms。 +Case1 的 standalone 依赖图由 tensor owner 与每 worker TensorMap 的 lookup 共同 +收集,不是按 task 名字直接跳过依赖:Alloc 和 QK 的 fanin 均为 0;SF 依赖 QK; +PV 依赖 SF;UP 的多个输入/原地输出按 producer 去重后依赖 Alloc、SF、PV。 +所以每 batch 的 fanin 数是 `0+0+1+1+3=5`,默认 256 batch 的严格终态断言 +要求 `fanin_edges=1280`。EfDrain 只有在这些 producer completion flag 全部 +ready 后才执行 winner 负载。 + +该图只对等 Case1 的调度依赖。真计算 workspace 用统一的受控 A/B 输入分别验证 +QK/PV matmul、SF add 和 UP mul,并按 worker-kind 写独占输出 tile;这些数值输出 +没有按 QK→SF→PV→UP 串接成 PA 的真实 tensor 数据流。因此它能验证 fanin、完成 +发布、角色路由和各类引擎算术,不能验证后继 task 消费前驱真实数值的语义。 +当前也只覆盖 Case1 的单 block group、`q_loop=1` 和全单-lane 图;通用多 group、 +多 q-loop、跨迭代更新及 joint/mixed task 的依赖数量与竞争形态均未模拟。 + +当前三后端无参数默认使用 `real-compute`:CCEC/AscendC 的 QK/PV 运行真实 Cube +matmul,SF/UP 运行真实 Vector add/mul,每轮都含 GM load、计算、GM store 和完成 +等待;CPU 使用相同 workspace、角色路由和数学运算做协议回归,不代表 A5 引擎 +时间或 PMU。可控 `scalar-nop` 只作为显式兼容/校准模式保留,其历史默认值按 +本文最好真实泳道的 44.170/53.729/27.626/1.565 us 校准。两种模式都不会在 +Claim、Register、PrepareMap 或等待路径中硬补 5 ms。 + +当前真计算默认次数为 QK/SF/PV/UP=`6,28,4,1`。依据是其每 task 完整 +load/compute/store span 已接近真实 PA 的 44.170/53.729/27.626/1.565 us;它 +优先保持 per-task core work 口径,不通过增加无关 repeat 把 standalone 总时间 +硬凑到 5.1 ms。standalone 未覆盖的真实控制流、代码布局和资源竞争仍会形成 +端到端差值,不能把这部分差值反推成缺少的 kernel repeat。 当前严格校验覆盖 73,728 次 Claim、每 task 唯一 winner、1,024 个 kernel、 TensorMap/heap 最终状态、fanin、flag、vend、frontier、cursor、ring placement @@ -699,7 +718,7 @@ worker 到达时序;真实动态调度不能由上述固定状态推导出必 #### 7.2.2 standalone 正确性与完整运行 F1 修改后重建 CCEC、AscendC 和 CPU 三后端,并分别运行 smoke、256 batch 零 -NOP、256 batch 默认 NOP。全部结果的语义断言和后处理都为 PASS;完整用例 +NOP、256 batch 当时默认 NOP。全部结果的语义断言和后处理都为 PASS;完整用例 仍为 73,728 次 Claim、1,280 个唯一 winner、1,024 个 kernel,placement 总数为 1,024。关键单轮结果如下: @@ -707,7 +726,7 @@ NOP、256 batch 默认 NOP。全部结果的语义断言和后处理都为 PASS | ---- | -----------------------: | ------: | --: | | smoke | `6 / 34.912` | `5 / 40.142` | `6 / 156383.064` | | 256 batch、零 NOP | `42546 / 3415.839` | `25515 / 3498.038` | `1983 / 163840.941` | -| 256 batch、默认 NOP | `40958 / 4019.835` | `47692 / 4283.306` | `1051412 / 185456.323` | +| 256 batch、当时默认 NOP | `40958 / 4019.835` | `47692 / 4283.306` | `1051412 / 185456.323` | CPU 只用于语义对照,不作为 A5 性能依据。单轮 fanin load 对 worker 调度非常 敏感,不能用上表三个数值直接归因,因此又执行了独立进程的交错 A/B。 @@ -715,7 +734,7 @@ CPU 只用于语义对照,不作为 A5 性能依据。单轮 fanin load 对 wo #### 7.2.3 CCEC 十对交错 A/B 基线固定在 H1 提交 `2c3dd1e2`,使用 detached worktree -`/tmp/simpler-f1-baseline`。两端都用 256 batch、默认 NOP、关闭泳道和 phase profile; +`/tmp/simpler-f1-baseline`。两端都用 256 batch、当时默认 NOP、关闭泳道和 phase profile; 每个样本是独立进程首轮,统一 60 s timeout。前 5 对为 baseline -> F1, 后 5 对为 F1 -> baseline。双方均 10/10 PASS、0 timeout、Claim=73,728、CAS retry=0。 @@ -819,7 +838,7 @@ request/miss 求和为 210,399/30,283,按总和计算的 miss rate 约 14.3931 `submit_span_us` 约 47.770。该样本只证明观察闭环可运行,不能替代 256 batch 约 5 ms 基线,也不能作为性能优化收益。 -同一源码和默认 PA NOP 随后完成了 3 个独立进程的 +同一源码和当时默认 PA NOP 随后完成了 3 个独立进程的 `batches=256,submit-all,PMU-only`。三轮均为 96/96 trusted,owner/slot/role/ triplet/start/stop 门禁全部 PASS,Restore PASS,且原始 96 核记录重算 ALL/AIC/AIV 的 `sum/mean/median/p95/max` 与 JSON summary 完全一致: @@ -929,7 +948,7 @@ D1 已在 standalone 公共调度器中增加 worker-local 软件计数,不新 `WorkerResult` 在 D1 从 704 B 扩展到 768 B;合入 atomic 泳道计数和 I-cache cold/warm 配对字段后,standalone 诊断 sidecar 按完整 cache line 扩展到 832 B。 原 PMU 字段 offset、生产 DistGlobal/DistCore offset 和 `LocalSlot` ABI 都不变。 -三后端完整重建后,smoke 和 256 batch 默认 NOP 均通过全部语义断言。 +三后端完整重建后,smoke 和 256 batch 当时默认 NOP 均通过全部语义断言。 对任一 worker,若其完成数为 `Cw`、fanin 边数为 `Ew`、失败 load 为 `Fw`,则 逐核检查: @@ -956,7 +975,7 @@ atomicMax;CPU 的 FetchMax 是 load/CAS 实现,`A` 只能解释为逻辑调 #### 7.4.2 CCEC 十轮动态基线 -在 256 batch、默认 NOP、关闭泳道和 phase profile 的同一进程十轮基线中,全部 +在 256 batch、当时默认 NOP、关闭泳道和 phase profile 的同一进程十轮基线中,全部 语义断言和上述计数恒等式均 PASS: | 指标 | 中位数 | 均值 | nearest-rank p90 | 范围 | @@ -1147,8 +1166,9 @@ trace 容量不足时必须明确报 overflow 并判该轮观察无效,不能 CCEC 只保留一个正式逐 worker Submit 窗口,CPU/AscendC 对应 hook 是空实现,不伪造 PMU 数据:`submit-all` 在本 worker 通过启动屏障、完成 `ResetTraceLap` 后,于 orchestration 初始化前打开 gate,在该 worker 最后一次 Submit 返回后关闭。窗口包含 -参数构造、全部 Submit 调度,以及本 worker 在窗口内执行的 winner 计算体:默认模式 -为 scalar NOP,CCEC `real-compute` 模式为真实 Cube/Vector 与对应搬运流水。 +参数构造、全部 Submit 调度,以及本 worker 在窗口内执行的 winner 计算体:当前 +无参数默认的 `real-compute` 是真实 Cube/Vector 与对应搬运流水;显式 +`scalar-nop` 校准样本的 NOP 则在 scalar 上执行并计入窗口。 它是“每核从 orchestration 到最后一次 Submit 返回”的累计窗口,不是全局最早 `Submit.start` 到最晚 `Submit.end` 的墙钟窗口,也不是逐次 Submit 窗口。 @@ -1334,8 +1354,10 @@ counter 仍不能把 miss 定位到 materialize/register 或某条 atomic。开 为避免 winner 执行期的 scalar NOP 污染 `scalar_busy`,2026-07-18 先在 standalone CCEC 增加显式 `--winner-workload real-compute`,没有迁移真实 PA。 -整体无参数默认仍为 `scalar-nop`,保证旧三后端基线不静默变化;选择真计算而未 -指定次数时,QK/SF/PV/UP 使用 `6,28,4,1`。 +在提交 `e66001ff` 对应的这个历史阶段,无参数默认当时仍为 `scalar-nop`,用于 +保证旧三后端基线不静默变化;当时选择真计算而未指定次数时,QK/SF/PV/UP +使用 `6,28,4,1`。三后端闭环后当前默认已经切换为 `real-compute`,但本节后续 +数据仍按当时的显式模式和参数解读,不能追改成新默认口径。 实现与门禁如下: @@ -1447,9 +1469,12 @@ sentinel,所以 `runs>1` 不会沿用上一轮 winner 结果。 分层回归中,scalar-NOP b1、real-compute b1 count=`1,1,1,1` 均 PASS; 后者 Submit span 为 39.821 ms,并校验 4 active tile + 188 sentinel tile。b8 count=`2,3,2,1` 连续两轮也均 PASS,两轮分别为 80.310/54.185 ms, -均校验 25 active tile + 167 sentinel tile。同时保留 CLI 互斥和边界门禁: -real-compute 与 NOP override 不能混用,count 必须与 real-compute 模式一起使用, -取值限制为 1..128,CCEC 专属 PMU 参数仍会被非 CCEC 后端拒绝。 +均校验 25 active tile + 167 sentinel tile。同时保留 CLI 互斥和边界门禁。在该 +历史阶段,real-compute 与 NOP override 不能混用,count 需要与 real-compute +模式一起使用,取值限制为 1..128,CCEC 专属 PMU 参数仍会被非 CCEC 后端拒绝。 +当前无参数默认已是 real-compute,因此当前 CLI 允许 count/pattern 直接覆盖默认 +真计算;显式 NOP override 会选择 scalar-nop,显式 real-compute 与 NOP override +仍然互斥。 CPU 阶段的“对等”只包括 PA 任务调度、AIC/AIV 角色选择、workspace 编址、 repeat 次数和 768/5/6 数学结果。CPU pthread 调度、普通浮点循环和 x86 atomic @@ -1493,7 +1518,7 @@ CCEC 先证明公共期望与 PTO `A*B` 语义正确;随后 AscendC 使用同 闭合了 A1/B1 错位、普通 KxN B 的 zN→nZ 分形 transpose、ND/NZ stride 和 FIXPIPE NZ→ND 输出。CPU 只证明 host 公式与调度路由,不把毫秒值外推到 A5。 三后端又以 `pattern=constant` 做 b1 count1 回归,并以 `smoke all` 回归原 -scalar-NOP,均 PASS,说明诊断模式没有静默改变性能默认或旧控制路径。 +scalar-NOP,均 PASS,说明诊断模式没有静默改变当时的性能默认或旧控制路径。 AscendC 同模式泳道位于 `outputs/pa_scheduler_swimlane_20260718_125904_3613100/ascendc/`,raw 为 @@ -1506,3 +1531,32 @@ pattern 拒绝回归,当前两种 unittest 入口均为 6/6 PASS。 该诊断证明一次完整 engine pipeline 的数学和布局,不单独证明同一 task 的 N 次 repeat 都执行;repeat 完整性仍使用 CCEC count1→count2 engine PMU 精确倍增, 不把最终覆盖同一 tile 的结果夸大成次数证明。 + +#### 7.5.13 默认切换为真负载及同口径性能验收 + +完成 CCEC→AscendC→CPU 分阶段闭环后,共享 `WinnerWorkloadOptions` +的无参数默认从 `scalar-nop` 切换为 +`real-compute/constant/6,28,4,1`。`smoke` 仍显式固定 b1/r1/scalar-nop=0; +旧命令未指定 mode 但显式给出 `--nop-count*` 时自动选择 scalar-nop。 +显式 real-compute 与 NOP override、显式 scalar-nop 与 real count/pattern 仍互斥。 + +性能验收先纠正了观察口径:早先 3.7~4.4 ms 是 `--no-swimlane`, +真实 PA 5.1 ms 是标准 L2 泳道;泳道记录不仅增加指令,还会改变 worker +到达、fanin 失败重试和 RingBp,所以不能把两者直接相减成“缺失的调度时间”。 +当前 CCEC b256 真负载、标准泳道、不开逐 atomic 的 5 个独立进程为: + +```text +5002.413 / 4875.193 / 4968.894 / 4992.477 / 4876.282 us +``` + +中位数 4,968.894 us。真实 PA 最终三轮中位数 5,115.620 us,同口径 +差 146.726 us,约 2.87%。五轮 standalone 的 QK/SF/PV/UP 每 task 均值 +中位数为 41.461/54.007/28.053/2.649 us,总 core work 已贴近真实 PA, +不通过增加 repeat 继续硬凑总时间。 + +保留的一轮 raw 为 +`outputs/performance_gap_20260718/standalone_ccec_real_b256_raw.json`;863,237 条 +记录全部有效,与真实 PA 863,232 条的基本阶段数完全相同,只额外有 5 条 +RingBp。这说明 standalone 已达到“独立复现约 5 ms 调度”的目标;仍然保留 +本文第 6 节的边界:它不依赖 simpler 生产代码,也不复刻真实 PA 数值数据流和 +通用多 group/joint 拓扑。 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index f9ea89d465..aff3238f68 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -26,13 +26,37 @@ 执行。每次运行都会校验这些数量以及最终 TensorMap、heap、cursor、flag、vend、 frontier 和 worker 状态,任一不符都会返回失败。 +Case1 的 task 不是五个彼此独立的占位符。standalone 会从 Tensor descriptor 的 +owner 和本 worker 的 TensorMap 收集 producer,去重后构造下列 fanin 图: + +| task | 直接 producer | fanin 数 | +| --- | --- | ---: | +| Alloc | 无 | 0 | +| QK | 无;query/key/block-table 是外部输入 | 0 | +| SF | QK | 1 | +| PV | SF | 1 | +| UP | Alloc、SF、PV;同一 producer 的多个 tensor 会去重 | 3 | + +因此每个 batch 恰有 5 条 fanin 边,默认 256 batch 恰有 1,280 条。worker 在 +EfDrain 中逐条读取 producer completion flag,只有全部 ready 才能执行 winner +负载;descriptor materialize、TensorMap lookup/register、fanin 去重、ring slot +拷贝和 completion 发布都走实际的 standalone 调度路径。 + +这里的“依赖对等”是 **Case1 调度依赖图对等**,不是 PA 数值数据流复刻。 +`real-compute` workspace 的 QK/SF/PV/UP 都读取同一组受控输入,并写入各 +worker-kind 的独占输出 tile;QK 的数值输出没有作为 SF 的真计算输入,SF/PV +的数值输出也没有继续串到后继真计算中。fanin 会真实约束执行次序,但 workspace +只校验每类 Cube/Vector 算术和角色路由。Case1 只有一个 block group 且 +`q_loop=1`;通用 PA 的多 group、多 q-loop、跨迭代数据更新以及 joint/mixed +task 依赖当前均未覆盖,不能从本用例外推其 fanin 数量或时序。 + 有意保留的替代只有两类: -1. 三后端默认的 `scalar-nop` 模式仍以可控 NOP 标定 QK/SF/PV/UP,保留旧基线。 - 三后端也都已支持显式 `real-compute`:CCEC 和 AscendC 在 A5 上让 QK/PV +1. 三后端当前无参数默认使用 `real-compute`:CCEC 和 AscendC 在 A5 上让 QK/PV 执行完整 Cube matmul,让 SF/UP 执行完整 Vector add/mul,并覆盖 GM load、 引擎计算、GM store 和完成等待;CPU 使用同一 `128x128 float` 输入、输出布局 执行普通浮点 matmul/add/mul,只用于回归算术、角色路由和输出闭环。 + `scalar-nop` 仍作为显式兼容/校准模式保留;历史 NOP 数据继续按当时口径解释。 2. simpler 的 AICPU/runtime 装载链路由本目录 host runner 代替;测试关注的 首个 Submit 到最后一个 Submit 区间不含 AICPU 初始化和最终回收。 @@ -62,11 +86,11 @@ Cube/Vector 指令、流水线、GM 搬运和 PMU 语义,host pthread 耗时 ## 3. Winner 计算负载 -### 3.1 三后端默认 `scalar-nop` +### 3.1 三后端 `scalar-nop` 兼容/校准模式 真实 PA 最好泳道中四类 kernel 的 1 GHz counter 均值和当前 A5 NOP 校准值为: -| Kernel | 目标时间 | 默认 NOP 数 | +| Kernel | 目标时间 | `scalar-nop` 标定数 | | --- | ---: | ---: | | QK | 44.170 us | 129,600 | | SF | 53.729 us | 157,900 | @@ -77,16 +101,20 @@ Alloc 没有模拟 kernel body。NOP 数是 A5 实测校准量,不应解释为 可以在运行时覆盖: ```bash -./run.sh run ccec --nop-count 100000 -./run.sh run ccec --nop-counts 129600,157900,79950,2400 +./run.sh run ccec --winner-workload scalar-nop --nop-count 100000 +./run.sh run ccec --winner-workload scalar-nop \ + --nop-counts 129600,157900,79950,2400 ``` `--nop-count N` 同时设置四类 kernel;`--nop-counts` 的顺序固定为 -QK、SF、PV、UP,允许范围为 0 到 10,000,000。 +QK、SF、PV、UP,允许范围为 0 到 10,000,000。兼容旧命令时,显式提供 +`--nop-count*` 而不写 `--winner-workload` 也会自动选择 `scalar-nop`;新脚本 +建议像上面一样把模式写明,避免把校准样本误认成当前默认真计算。 -### 3.2 三后端 `real-compute` 模式 +### 3.2 三后端默认 `real-compute` 模式 -三后端共用同一组参数,可通过 `all` 按 CCEC、AscendC、CPU 的顺序统一运行: +三后端当前无参数运行即选择 `real-compute`。三后端共用同一组参数,可通过 +`all` 按 CCEC、AscendC、CPU 的顺序统一运行;命令中显式写出模式仍然有效: ```bash ./run.sh run all \ @@ -154,8 +182,10 @@ worker-kind 的最终 tile 必须分别等于 QK/PV 的 768、SF 的 5、UP 的 41.336、54.039、27.971 us,接近真实 PA 的 44.170、53.729、27.626 us。 UP 的一次完整 `128x128` 流水约 2.5 us,已经是正整数迭代下限;若后续要贴近 1.565 us,应缩小 UP tile,不能用 0 次掩盖执行。真实计算下 Cube/Vector 会在 -不同物理子核并行,因此 CCEC 整体 Submit 约 3.6~3.8 ms,不应增加无关工作量 -硬凑 5.1 ms。AscendC 的独立实测见 6.2 节;CPU 数值不进入 A5 性能对比。 +不同物理子核并行;关闭泳道和开启标准泳道还会改变 worker 到达、fanin 重试与 +RingBp,不能把两种观察布局的绝对时间直接相减。当前 `6,28,4,1` 优先贴近 +真实 PA 的 **per-task core work**,不通过增加无关 repeat 硬凑 5.1 ms。 +同观察口径的完整验收见 6.5 节;CPU 数值不进入 A5 性能对比。 ## 4. 本机依赖和构建 @@ -232,9 +262,10 @@ semantic_status=PASS postprocess_status=PASS --profile-phases --analyze-swimlane ``` -这两条未传 `--winner-workload`,都使用默认 scalar-NOP;需要真计算时按 -3.2 节显式传入 `--winner-workload real-compute`。CPU 使用同一选项; -需同配置串行回归三后端时,直接把 backend 换成 `all`。 +这两条未传 `--winner-workload`,使用当前默认 `real-compute` 和 +`6,28,4,1`。需要复现历史 NOP 基线时按 3.1 节显式传入 +`--winner-workload scalar-nop`;CPU 使用同一选项。需同配置串行回归三后端 +时,直接把 backend 换成 `all`,但 CPU 真计算只作协议/算术回归,不作 A5 性能值。 此模式输出指标和泳道统计,但不会自动落盘 JSON。若只关注性能且不需要完整 泳道逐事件分析,可去掉 `--analyze-swimlane`;若也不需要记录泳道,可使用 @@ -597,8 +628,9 @@ mkdir -p "$OUT" --pmu-json "$OUT/pmu_submit_all.json" ``` -上述命令没有选择 winner 模式,采集的是默认 `scalar-nop`。验证真实引擎计数时 -必须显式写出模式和次数,例如已用于 count 倍增取证的 b8 命令: +上述命令没有选择 winner 模式,采集的是当前默认 `real-compute` 和 +`6,28,4,1`。为了让 PMU 取证参数自描述,正式样本仍建议显式写出模式和次数; +例如已用于 count 倍增取证的 b8 命令: ```bash ./run.sh run ccec \ @@ -661,8 +693,9 @@ selector、owner/slot/role/triplet 与 Restore 门禁:all/AIC/AIV 的 total 证明 `submit-all` 观察闭环可运行;`batches=1`、零模拟计算体和单次运行都不足以 支持 256 batch 性能归因或真实 PA 绝对结论。 -按上述正式命令和默认 PA NOP 还完成了 3 个独立进程的 256 batch -PMU-only 验收。Submit span 为 3,688.236/4,089.057/4,673.237 us,中位数 +按当时的正式命令和当时默认 PA NOP,还完成了 3 个独立进程的 256 batch +PMU-only 验收。该段保留的是切换默认模式之前的历史样本。Submit span 为 +3,688.236/4,089.057/4,673.237 us,中位数 4,089.057 us;I-cache request 总和为 69,812,583/69,451,706/70,065,443,miss 总和为 5,854,421/5,847,256/5,830,645,miss rate 为 8.3859%/8.4192%/8.3217%。 三轮均通过 96 核、owner/slot/role/triplet、start/stop、counter 风险门槛和 Restore, @@ -839,7 +872,8 @@ CPU 上的 matmul/add/mul 是普通 x86 浮点循环,`[KERNEL]`、`submit_span ### 6.4 2026-07-18 非均匀布局诊断 -常量 2/3 会掩盖 `B` 转置和分形重排错误,因此在性能默认不变的前提下增加了 +常量 2/3 会掩盖 `B` 转置和分形重排错误,因此在不改变 `constant` 性能输入默认的 +前提下增加了 `--real-compute-pattern layout-diagnostic`。按 CCEC → AscendC → CPU 的顺序, 三后端均以 b1、count=`1,1,1,1` 运行同一带权对角 A 和非对称 B,并逐元素扫描 4 个 active tile 与 188 个 inactive sentinel tile,全部 PASS: @@ -859,6 +893,40 @@ AscendC 诊断泳道位于 诊断只证明单次完整迭代的数学与数据布局闭环;默认常量性能负载仍用于稳定比较, CCEC 的 repeat 次数证明仍以 count1→count2 的 engine PMU 精确倍增为准。 +### 6.5 2026-07-18 默认真负载与 5 ms 口径验收 + +三后端重编后,无 workload 参数的 b1 均打印 +`mode=real-compute pattern=constant counts=6,28,4,1`,并通过 96 核、5 个 +fanin/batch、唯一 winner、角色路由、TensorMap/heap/completion 与数值输出断言。 +旧命令只给 `--nop-count 0` 时会显式打印 `mode=scalar-nop`,证明兼容 +分支没有让 NOP override 静默失效。 + +CCEC b256 关闭泳道的 3 个独立进程为 4,411.760/4,297.704/4,677.634 us, +中位数 4,411.760 us;它只用于无观察热路对比。与真实 PA 5.1 ms 比较时, +必须同样开启标准泳道且不开逐 atomic;standalone 5 个独立进程为: + +```text +5002.413 / 4875.193 / 4968.894 / 4992.477 / 4876.282 us +``` + +中位数为 **4,968.894 us**。真实 PA 最终三轮为 +5,115.620/5,145.057/5,096.685 us,中位数 **5,115.620 us**;同口径差值 +146.726 us,约 2.87%,已满足独立调度复现目标。五轮 standalone 的 +QK/SF/PV/UP 每 task 均值中位数为 41.461/54.007/28.053/2.649 us, +与真实 44.170/53.729/27.626/1.565 us 的总 core work 接近,不再调整 +repeat 追求逐微秒一致。 + +只保留一轮标准泳道原始证据: + +```text +outputs/performance_gap_20260718/standalone_ccec_real_b256_raw.json +``` + +该轮有 863,237 条记录、`dropped=0`,比真实 PA 的 863,232 条只多 5 条 +RingBp;两端的 122,880 个 Submit 与各前端阶段、1,024 个 Kernel/Fanin/Build +数量一致。这证明总体性能已接近,不等于真实 PA 数值数据流、代码生成与通用 +多 group/joint 调度已完全相同。 + ## 7. 内存占用和脱仓复制 为保持真实 DistGlobal/DistCore 偏移、65,536 个 task cell、每 worker payload diff --git a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc index be198b00ef..bf6ded0a57 100644 --- a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc +++ b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc @@ -399,6 +399,11 @@ int32_t main(int32_t argc, char **argv) { "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP] " "[--real-compute-pattern constant|layout-diagnostic]\n" ); + std::fprintf( + stderr, + "Default: real-compute, constant, counts=6,28,4,1; " + "scalar-nop is the calibration compatibility mode.\n" + ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index 04449d1a25..de6d54f7fe 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -1202,6 +1202,11 @@ int main(int argc, char **argv) { "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP] " "[--real-compute-pattern constant|layout-diagnostic]\n" ); + std::fprintf( + stderr, + "Default: real-compute, constant, counts=6,28,4,1; " + "scalar-nop is the calibration compatibility mode.\n" + ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index 8f3f436433..a9a8676607 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -80,10 +80,11 @@ static_assert((kPayloadSlots & kPayloadMask) == 0, "payload slots must be a powe static_assert(kMaxTasks < kTaskCellCapacity, "every frontier scan must terminate on an in-range not-ready flag"); // These are the measured means from the best PA A5 trace, in 1 GHz ticks. -// The scalar-NOP baseline calibrates its counts against these targets. -// 默认模式只用 NOP 代替四个计算 kernel;CCEC 显式 real-compute 模式改用 -// 完整 Cube/Vector 流水。两种模式的 Submit、依赖、heap 与 completion 路径 -// 均不靠 NOP 补时。target 是真实泳道均值,不是调度阶段预算。 +// The scalar-NOP compatibility baseline calibrates its counts against these targets. +// 无参数默认使用 real-compute:CCEC/AscendC 执行完整 Cube/Vector 流水, +// CPU 执行对等算术;下列 NOP 常量只供显式 scalar-nop 校准。两种模式的 +// Submit、依赖、heap 与 completion 路径都不靠补时修改。target 是真实泳道 +// 均值,不是调度阶段预算。 constexpr uint32_t kTargetQkTicks = 44170; constexpr uint32_t kTargetSfTicks = 53729; constexpr uint32_t kTargetPvTicks = 27626; diff --git a/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h b/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h index da4725d0e8..07972cc96b 100644 --- a/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h +++ b/tests/atomic_probe/pa_scheduler/common/winner_workload_host.h @@ -33,7 +33,7 @@ enum class RealComputePattern : uint32_t { // PMU 参数;AscendC/CPU 则直接把剩余 argv 交给 ParseOptions。这样不把后端 // 私有功能塞入公共 PA 参数结构,也不会复制三套互斥规则。 struct WinnerWorkloadOptions { - WinnerWorkloadMode mode = WinnerWorkloadMode::ScalarNop; + WinnerWorkloadMode mode = WinnerWorkloadMode::RealCompute; WorkloadCounts repeats = winner_workload::kDefaultRealComputeCounts; RealComputePattern pattern = RealComputePattern::Constant; bool counts_explicit = false; @@ -162,6 +162,13 @@ inline bool ParseWinnerWorkloadOptions( count_seen = true; workload->counts_explicit = true; } + // 无参数运行以真实 Cube/Vector 为默认。旧命令若显式给出 NOP 次数但没有 + // 指定 workload mode,则把 NOP override 本身视为选择 scalar-nop;这样 + // 既不让 --nop-count 悄悄失效,也不破坏既有标定脚本。显式指定 + // real-compute 再叠加 NOP 仍由下方互斥校验拒绝。 + if (!mode_seen && workload->nop_override_explicit) { + workload->mode = WinnerWorkloadMode::ScalarNop; + } return true; } @@ -184,7 +191,7 @@ inline bool ValidateWinnerWorkloadOptions(const WinnerWorkloadOptions &workload) if (workload.counts_explicit || workload.pattern_explicit) { std::fprintf( stderr, - "--real-compute-count(s)/pattern requires --winner-workload real-compute.\n" + "--real-compute-count(s)/pattern requires real-compute workload mode.\n" ); return false; } diff --git a/tests/atomic_probe/pa_scheduler/cpu/main.cpp b/tests/atomic_probe/pa_scheduler/cpu/main.cpp index 7a188326a0..9814d70a32 100644 --- a/tests/atomic_probe/pa_scheduler/cpu/main.cpp +++ b/tests/atomic_probe/pa_scheduler/cpu/main.cpp @@ -262,6 +262,11 @@ int main(int argc, char **argv) { "[--real-compute-count N | --real-compute-counts QK,SF,PV,UP] " "[--real-compute-pattern constant|layout-diagnostic]\n" ); + std::fprintf( + stderr, + "Default: real-compute, constant, counts=6,28,4,1; " + "scalar-nop is the calibration compatibility mode.\n" + ); } return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index 328f024229..f3a2d802b8 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -46,7 +46,7 @@ Standalone winner workload options (CCEC, AscendC, and CPU): --real-compute-counts QK,SF,PV,UP --real-compute-pattern constant|layout-diagnostic -real-compute is opt-in. Its CCEC-calibrated A5 defaults are QK/SF/PV/UP=6/28/4/1; +real-compute is the default. Its CCEC-calibrated A5 counts are QK/SF/PV/UP=6/28/4/1; one count is one complete 128x128 load/engine/store/completion-wait pipeline per winner task, not a scalar NOP count. Explicit count options override those four defaults. AscendC must be calibrated independently; CPU only preserves @@ -197,7 +197,7 @@ case "$ACTION" in done ;; smoke) - # smoke 仍启动全部 96 个 worker,并默认注入 1 batch、1 run、0 NOP; + # smoke 仍启动全部 96 个 worker,并显式注入 1 batch、1 run、scalar-nop=0; # 后置用户参数仍由共享 parser 处理。它用于快速检查原子协议、拓扑和 # 最终状态,不作为性能数据。 for argument in "$@"; do @@ -212,7 +212,8 @@ case "$ACTION" in esac done for backend in "${BACKENDS[@]}"; do - run_backend "$backend" --batches 1 --runs 1 --nop-count 0 "$@" + run_backend "$backend" --batches 1 --runs 1 \ + --winner-workload scalar-nop --nop-count 0 "$@" done ;; swimlane) From cbaf7c60a25cae7a74eed87b81303af402d2fb8e Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 16:37:51 +0000 Subject: [PATCH 021/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E6=8E=A5?= =?UTF-8?q?=E9=80=9A=E7=9C=9F=E5=AE=9EPA=20atomic=E6=B3=B3=E9=81=93?= =?UTF-8?q?=E4=B8=8E=E7=B2=BE=E7=A1=AE=E8=BD=AE=E8=AF=A2=E8=81=9A=E5=90=88?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 将 standalone PA scheduler 中已经验证的 atomic 观察语义迁入 fully_distributed_within_core,使真实 A5 PA 在 level 4 下能够直接生成可分析的 scalar atomic 泳道,同时避免逐次记录热轮询导致设备内存和 JSON 规模失控。 设备端: - 将 FDWIC raw ABI 升级为 schema v3,使用 32B 紧凑记录,并把不变的 core/block/lane 拓扑收敛到每核 64B 状态;继续复用 65536 records/core 分区。 - 为真实 PA 热路径接入 28 个稳定 atomic site 和 load/exchange/fetch_add/fetch_max/fetch_sub 五类操作;直接调用保持 call_count=1。 - direct span 按 atomic -> end/return_ready -> count -> record 排序,确保本地计数更新、PollBatch 落盘和记录写入不混入单次 atomic 的观测区间。 - 仅在显式 scheduler 等待区聚合九类 observation load;唯一的 RMW 例外是 won_lane_claim_exchange 的 old=1、desired=1 幂等失败重试,成功的 old=0 -> 1 claim 和其他 RMW 仍逐次记录。 - PollBatch 通过 flags 高 24 bit 保存精确 call_count,到 0xFFFFFF 时立即落盘并从 1 重开;保留 region、phase、lap 和最终 flush 闭合。 Host 与转换器: - A5Sim/onboard runner 透传 level 4 配置并管理 trace 生命周期;host 按核只搬运 count*32B 有效记录,校验 header、拓扑、site/op、时钟基线、容量和 dropped 状态。 - 同时维护逻辑调用数与物理记录数,强制满足 atomic_records = atomic_calls - batched_poll_calls + poll_batch_records,并逐核、全局复算 raw 闭环。 - 共享 converter 识别 schema v3 direct Atomic、PollBatch 和 ClockBaseline,将事件放回真实 scalar lane,标注 batch 语义与逻辑等待包络,禁止把 batch duration 当成单次 atomic latency。 - converter 镜像校验全量 site/op、result_used、return_ready、value_zero、FetchMax payload、每核双 ClockBaseline、cycle 范围和 producer summary,拒绝格式或计数不闭合的本地 raw。 - 输出精确 call_count,支持以 sum(call_count * calibrated_atomic_cost) 估算 scalar core-work;统一按 160ns 标尺时可直接使用 atomic_calls * 160ns。 测试与实测: - 新增 converter 正反向用例,覆盖十类合法 PollBatch、24 bit 最大值、A5/A5Sim return_ready 差异、坏 direct/clock/timing/summary 拒绝路径;相关 Python 测试 57 项通过。 - 新增直接调用 production accumulator 的 C++ GTest,验证第 0xFFFFFF 次落首批、第 0x1000000 次以 count=1 重开,flush 后状态清零且 dropped=0。 - 修订后的 AIC/AIV 和 A5Sim kernel 均已重编;真实 A5 device 0 上 PA Case1 level 4 通过:115200 次逻辑 atomic、110006 条物理 Atomic、340 条 PollBatch、dropped=0。 - 额外用真实 A5 Benchmark BGEMM/Bgemm64 验证非 PA 算子:1072 次逻辑 atomic、939 条物理 Atomic、27 条 PollBatch、dropped=0。 - 更新中文复现文档,记录十类聚合规则、schema v3、容量口径、计数公式、160ns 归因边界及最终真机产物。 --- simpler_setup/tools/swimlane_converter.py | 466 +++++++++++-- .../platform/onboard/host/device_runner.cpp | 47 +- src/a5/platform/sim/host/device_runner.cpp | 39 +- .../host/fdwic_swimlane.cpp | 657 ++++++++++++++++-- .../runtime/dist_engine/aicore/api_glue.h | 2 +- .../runtime/dist_engine/aicore/core_main.h | 14 +- .../runtime/dist_engine/aicore/submit_core.h | 96 ++- .../dist_engine/aicore/submit_helpers.h | 67 +- .../dist_engine/aicore/submit_runtime.h | 89 ++- .../dist_engine/aicore/tensor_data_access.h | 9 +- .../runtime/dist_engine/common/state.h | 2 + .../runtime/dist_engine/common/swimlane.h | 387 ++++++++++- .../dist_engine/common/swimlane_types.h | 243 ++++++- .../runtime/dist_engine/common/trace.h | 11 +- .../runtime/dist_engine/common/worker_state.h | 21 +- .../runtime/runtime.h | 5 +- .../runtime/shared/runtime.cpp | 3 +- ...RO.md => a5_fdwic_atomic_swimlane_repo.md} | 332 ++++++++- ...05\345\206\265\345\210\206\346\236\220.md" | 2 +- tests/ut/cpp/CMakeLists.txt | 12 + .../cpp/a5/test_fdwic_swimlane_poll_batch.cpp | 102 +++ tests/ut/py/test_fdwic_swimlane_converter.py | 411 +++++++++++ 22 files changed, 2742 insertions(+), 275 deletions(-) rename tests/atomic_probe/{A5_FDWIC_PAGED_ATTENTION_REPRO.md => a5_fdwic_atomic_swimlane_repo.md} (55%) create mode 100644 tests/ut/cpp/a5/test_fdwic_swimlane_poll_batch.cpp create mode 100644 tests/ut/py/test_fdwic_swimlane_converter.py diff --git a/simpler_setup/tools/swimlane_converter.py b/simpler_setup/tools/swimlane_converter.py index e25d7f2df8..f003cfc130 100644 --- a/simpler_setup/tools/swimlane_converter.py +++ b/simpler_setup/tools/swimlane_converter.py @@ -31,6 +31,105 @@ from pathlib import Path from typing import Any +_FDWIC_PHASE_NAMES = { + "Kernel": "kernel", + "Alloc": "alloc", + "Build": "build", + "DrainWon": "drain_won", + "Replay": "replay", + "RingBp": "ringbp", + "EfDrain": "efdrain", + "Commit": "commit", + "Submit": "submit", + "Materialize": "materialize", + "PrepareMap": "prepare_map", + "Claim": "claim", + "Fanin": "fanin", + "Register": "register", + "Atomic": "atomic", + "ClockBaseline": "clock_baseline", +} + +# IDs 0..14 are the standalone PA ABI. Real PA only appends IDs so archived +# captures and the standalone calibration keep the same names. +_FDWIC_ATOMIC_SITE_NAMES = { + 0: "startup_increment", + 1: "startup_poll", + 2: "fatal_poll", + 3: "fatal_set", + 4: "claim_max", + 5: "fanin_flag_load", + 6: "completion_vend_exchange", + 7: "completion_flag_exchange", + 8: "frontier_initial_load", + 9: "frontier_flag_load", + 10: "frontier_max", + 11: "heap_frontier_load", + 12: "heap_vend_load", + 13: "replay_done_increment", + 14: "replay_done_poll", + 15: "won_slot_claim_max", + 16: "won_remaining_exchange", + 17: "won_lane_reset_exchange", + 18: "won_lane_deposit_exchange", + 19: "won_state_publish_exchange", + 20: "won_any_publish_exchange", + 21: "won_any_load", + 22: "won_state_load", + 23: "won_lane_claim_exchange", + 24: "won_lane_release_exchange", + 25: "won_remaining_fetch_sub", + 26: "won_state_clear_exchange", + 27: "won_drained_load", +} + +_FDWIC_ATOMIC_OP_NAMES = { + 0: "load", + 1: "exchange", + 2: "fetch_add", + 3: "fetch_max", + 4: "fetch_sub", +} + +_FDWIC_ATOMIC_SITE_OP_IDS = { + 0: 2, + 1: 0, + 2: 0, + 3: 1, + 4: 3, + 5: 0, + 6: 1, + 7: 1, + 8: 0, + 9: 0, + 10: 3, + 11: 0, + 12: 0, + 13: 2, + 14: 0, + 15: 3, + 16: 1, + 17: 1, + 18: 1, + 19: 1, + 20: 1, + 21: 0, + 22: 0, + 23: 1, + 24: 1, + 25: 4, + 26: 1, + 27: 0, +} + +# These sites issue the atomic for its side effect and do not consume the old +# value. All other production sites consume the result. A v3 converter must +# mirror this ABI instead of trusting a self-consistent summary around a +# malformed direct Atomic row. +_FDWIC_ATOMIC_RESULT_UNUSED_SITE_IDS = {0, 3, 6, 7, 13, 16, 17, 18, 19, 20, 24, 26} + +_FDWIC_POLL_BATCH_SITE_OP_IDS = {1: 0, 2: 0, 5: 0, 11: 0, 12: 0, 14: 0, 21: 0, 22: 0, 23: 1, 27: 0} + def _func_id_to_letter(func_id): """Map a non-negative integer func_id to a numeric+letter label. @@ -70,7 +169,13 @@ def _task_display_name(func_id, func_id_to_name, tdisp): return f"func_{_func_id_to_letter(func_id)}({tdisp})" -def _append_fdwic_dist_engine_events(events, fdwic_events, func_id_to_name=None): +def _append_fdwic_dist_engine_events( # noqa: PLR0912 + events, + fdwic_events, + func_id_to_name=None, + trace_schema_version=1, + clock_freq_hz=0, +): """Append fully_distributed_within_core AICore-runtime spans. The visual shape follows the a2a3 dist_engine swimlane where it applies on @@ -84,22 +189,7 @@ def lane_name(lane): return {0: "AIC", 1: "AIV0", 2: "AIV1"}.get(int(lane), "?") def phase_name(phase): - return { - "Kernel": "kernel", - "Alloc": "alloc", - "Build": "build", - "DrainWon": "drain_won", - "Replay": "replay", - "RingBp": "ringbp", - "EfDrain": "efdrain", - "Commit": "commit", - "Submit": "submit", - "Materialize": "materialize", - "PrepareMap": "prepare_map", - "Claim": "claim", - "Fanin": "fanin", - "Register": "register", - }.get(str(phase), str(phase).lower()) + return _FDWIC_PHASE_NAMES.get(str(phase), str(phase).lower()) def kernel_name(func_id): if func_id_to_name: @@ -139,12 +229,66 @@ def kernel_name(func_id): } ) + legacy_claim_max_spans = defaultdict(list) + has_atomic_trace = False + if trace_schema_version == 1: + for event in fdwic_events: + if phase_name(event["phase"]) != "atomic": + continue + has_atomic_trace = True + if int(event["aux"]) == 4: + key = (int(event["core_id"]), int(event["block_id"]), int(event["lane"]), int(event["task_id"])) + legacy_claim_max_spans[key].append((int(event["start_cycles"]), int(event["end_cycles"]))) + for e in fdwic_events: phase = phase_name(e["phase"]) func_id = int(e["func_id"]) task_id = int(e["task_id"]) lane = int(e["lane"]) - if phase == "kernel" and func_id >= 0: + flags = int(e["flags"]) + aux = int(e["aux"]) + if phase == "claim": + claim_won = bool(flags & 0x1) + if trace_schema_version >= 2: + claim_attempted = bool(flags & 0x2) + claim_attempted_source = "raw_flag" + elif has_atomic_trace: + key = (int(e["core_id"]), int(e["block_id"]), lane, task_id) + matched_claim_max = any( + atomic_start >= int(e["start_cycles"]) and atomic_end <= int(e["end_cycles"]) + for atomic_start, atomic_end in legacy_claim_max_spans.get(key, []) + ) + claim_attempted = True if matched_claim_max else None + claim_attempted_source = ( + "contained_claim_max" if matched_claim_max else "unknown_v1_without_matching_claim_max" + ) + else: + claim_attempted = None + claim_attempted_source = "unknown_v1_without_atomic_trace" + if claim_attempted is False: + name = f"claim.not_attempted#{task_id}" + elif claim_attempted is True: + name = f"claim.{'won' if claim_won else 'lost'}#{task_id}" + else: + name = f"claim#{task_id}" + tid = lane + elif phase == "atomic": + atomic_site_id = aux + atomic_op_id = flags & 0xF + atomic_site = _FDWIC_ATOMIC_SITE_NAMES.get(atomic_site_id, f"site_{atomic_site_id}") + atomic_op = _FDWIC_ATOMIC_OP_NAMES.get(atomic_op_id, f"op_{atomic_op_id}") + atomic_poll_batch = trace_schema_version >= 3 and bool(flags & (1 << 7)) + if atomic_poll_batch: + atomic_call_count = (flags >> 8) & 0xFFFFFF + name = f"atomic.poll_batch.{atomic_site}.{atomic_op}×{atomic_call_count}" + else: + atomic_boundary_tag = "return_ready" if flags & (1 << 6) else "source_issue" + name = f"atomic.{atomic_boundary_tag}.{atomic_site}.{atomic_op}#{task_id}" + tid = lane + elif phase == "clock_baseline": + name = "clock.atomic_return_dependency_hook" if flags & 1 else "clock.consecutive_sys_cnt_reads" + tid = lane + elif phase == "kernel" and func_id >= 0: name = f"{kernel_name(func_id)}#{task_id}" tid = lane + 3 elif phase == "commit": @@ -153,23 +297,97 @@ def kernel_name(func_id): else: name = f"{phase}#{task_id}" tid = lane - events.append( - { - "ph": "X", - "name": name, - "pid": int(e["block_id"]), - "tid": tid, - "ts": round(float(e["start_time_us"]), 3), - "dur": round(float(e["duration_us"]), 3), - "args": { + event = { + "ph": "X", + "name": name, + "pid": int(e["block_id"]), + "tid": tid, + "ts": round(float(e["start_time_us"]), 3), + "dur": round(float(e["duration_us"]), 3), + "args": { + "phase": phase, + "task_id": task_id, + "func_id": func_id, + "core": int(e["core_id"]), + "mc": flags & 1, + }, + } + if phase == "atomic": + if atomic_poll_batch: + event["args"] = { + "phase": "atomic_poll_batch", + "task_id": task_id, + "func_id": func_id, + "core": int(e["core_id"]), + "site": atomic_site, + "site_id": atomic_site_id, + "op": atomic_op, + "op_id": atomic_op_id, + "call_count": atomic_call_count, + "poll_window_cycles": int(e["end_cycles"]) - int(e["start_cycles"]), + "estimate_formula": "call_count * calibrated_atomic_cost", + "is_poll_batch": True, + "batch_semantics": ( + "idempotent_failed_exchange_retries" + if atomic_site_id == 23 + else "observation_load_calls" + ), + "duration_semantics": "logical_poll_episode_envelope_not_single_atomic_latency", + "may_contain_interleaved_direct_atomics": True, + "flags": flags, + "execution_unit": "scalar", + } + event["cat"] = "atomic.poll_batch" + else: + event["args"] = { "phase": phase, "task_id": task_id, "func_id": func_id, "core": int(e["core_id"]), - "mc": int(e["flags"]) & 1, - }, + "site": atomic_site, + "site_id": atomic_site_id, + "op": atomic_op, + "op_id": atomic_op_id, + "call_count": 1, + "cycles": int(e["end_cycles"]) - int(e["start_cycles"]), + "result_used": bool(flags & (1 << 4)), + "return_ready_observed": bool(flags & (1 << 6)), + "completion_boundary": "return_value_ready" if flags & (1 << 6) else "source_issue_bracket", + "flags": flags, + "execution_unit": "scalar", + } + event["cat"] = f"atomic.{atomic_boundary_tag}" + if atomic_op_id == 0: + event["args"]["value_zero"] = bool(flags & (1 << 5)) + if atomic_op_id == 3: + event["args"]["retries"] = (flags >> 8) & 0xFFFFFF + elif phase == "claim": + event["args"] = { + "phase": phase, + "task_id": task_id, + "func_id": func_id, + "core": int(e["core_id"]), + "claim_attempted": claim_attempted, + "claim_won": claim_won, + "claim_attempted_source": claim_attempted_source, + "claim_path": "alloc" if aux == 1 else "kernel", + "execution_unit": "scalar", + "flags": flags, } - ) + event["cat"] = "scalar_scheduler" + elif phase == "clock_baseline": + dependency_hook = bool(flags & 1) + event["args"] = { + "phase": phase, + "core": int(e["core_id"]), + "ticks": int(e["end_cycles"]) - int(e["start_cycles"]), + "clock_freq_hz": int(clock_freq_hz), + "definition": ("atomic-return-dependency-hook" if dependency_hook else "consecutive-sys-cnt-reads"), + "dependency_applied": bool(flags & 2) if dependency_hook else False, + "execution_unit": "scalar", + } + event["cat"] = "scalar_clock" + events.append(event) def normalize_pto2_task_id_int(v): @@ -217,6 +435,7 @@ def read_perf_data(filepath): # noqa: PLR0912, PLR0915 "l2_swimlane_level": <1..4>, "metadata": { "clock_freq_hz": , + "trace_schema_version": <1|2|3>, # optional; default 1 "num_cores": , "core_types": ["aic"|"aiv", ...], # indexed by core_id "core_to_thread": [, ...] # optional (level >= 3) @@ -274,6 +493,10 @@ def read_perf_data(filepath): # noqa: PLR0912, PLR0915 clock_freq_hz = int(metadata.get("clock_freq_hz") or 0) if clock_freq_hz <= 0: raise ValueError(f"metadata missing/zero clock_freq_hz: {clock_freq_hz}") + trace_schema_version = int(metadata.get("trace_schema_version", 1)) + if trace_schema_version not in (1, 2, 3): + raise ValueError(f"Unsupported metadata.trace_schema_version: {trace_schema_version} (expected 1, 2, or 3)") + num_cores = int(metadata.get("num_cores") or 0) core_types = list(metadata.get("core_types") or []) core_to_thread = list(metadata.get("core_to_thread") or []) @@ -282,6 +505,14 @@ def read_perf_data(filepath): # noqa: PLR0912, PLR0915 sched_phases_raw = data.get("aicpu_scheduler_phases") or [] orch_phases_raw = data.get("aicpu_orchestrator_phases") or [] fdwic_rows = data.get("fdwic_events") or [] + fdwic_summary = metadata.get("fdwic_summary") + if trace_schema_version == 3 and level != 4: + raise ValueError("metadata.trace_schema_version=3 requires l2_swimlane_level=4") + if trace_schema_version == 3 and (num_cores <= 0 or len(core_types) != num_cores): + raise ValueError( + f"metadata.trace_schema_version=3 requires num_cores matching core_types: " + f"num_cores={num_cores} core_types={len(core_types)}" + ) # AICore lookup keyed by (core_id, reg_task_id). Two dispatches of the # same PTO2 task_token_raw to the same core (SPMD over-subscription, MIX @@ -465,28 +696,164 @@ def _phase_us(pr): aicpu_orchestrator_phases.append(converted) fdwic_events = [] - for row in fdwic_rows: + observed_summary = { + "records": len(fdwic_rows), + "atomic_records": 0, + "clock_baseline_records": 0, + "atomic_calls": 0, + "batched_poll_calls": 0, + "poll_batch_records": 0, + "dropped_records": 0, + } + v3_clock_rows = defaultdict(lambda: {"plain": 0, "dependency": 0, "return_ready": None}) + v3_result_used_direct_rows = [] + for row_index, row in enumerate(fdwic_rows): + if not isinstance(row, (list, tuple)) or len(row) != 10: + raise ValueError(f"fdwic_events[{row_index}] must contain exactly 10 columns") core_id, block_id, lane, task_id, func_id, phase, start_cycles, end_cycles, flags, aux = row - start_us = _to_us(int(start_cycles)) - end_us = _to_us(int(end_cycles)) + core_id = int(core_id) + block_id = int(block_id) + lane = int(lane) + task_id = int(task_id) + func_id = int(func_id) + start_cycles = int(start_cycles) + end_cycles = int(end_cycles) + flags = int(flags) + aux = int(aux) + phase = str(phase) + if trace_schema_version == 3: + if not 0 <= core_id < num_cores: + raise ValueError(f"fdwic_events[{row_index}] has invalid core_id {core_id}") + if task_id < -1 or func_id < -1 or aux < 0 or phase not in _FDWIC_PHASE_NAMES: + raise ValueError( + f"fdwic_events[{row_index}] has invalid base fields: " + f"task={task_id} func={func_id} phase={phase!r} aux={aux}" + ) + if not (0 <= start_cycles <= end_cycles <= 0xFFFFFFFFFFFFFFFF): + raise ValueError( + f"fdwic_events[{row_index}] has invalid cycle range {start_cycles}..{end_cycles}" + ) + if not 0 <= flags <= 0xFFFFFFFF: + raise ValueError(f"fdwic_events[{row_index}] has invalid uint32 flags {flags}") + if phase == "Claim" and trace_schema_version >= 2: + if flags & ~0x3 or (flags & 0x1 and not flags & 0x2): + raise ValueError(f"fdwic_events[{row_index}] has invalid Claim flags 0x{flags:x}") + if phase == "Atomic" and flags & (1 << 7): + call_count = (flags >> 8) & 0xFFFFFF + if ( + trace_schema_version < 3 + or call_count == 0 + or not flags & (1 << 4) + or _FDWIC_POLL_BATCH_SITE_OP_IDS.get(aux) != (flags & 0xF) + or flags & ((1 << 5) | (1 << 6)) + or task_id != -1 + or func_id != -1 + ): + raise ValueError(f"fdwic_events[{row_index}] has invalid Atomic PollBatch flags 0x{flags:x}") + elif phase == "Atomic" and trace_schema_version == 3: + op = flags & 0xF + result_used = bool(flags & (1 << 4)) + value_zero = bool(flags & (1 << 5)) + return_ready = bool(flags & (1 << 6)) + payload = flags >> 8 + expected_result_used = aux in _FDWIC_ATOMIC_SITE_OP_IDS and aux not in _FDWIC_ATOMIC_RESULT_UNUSED_SITE_IDS + if ( + _FDWIC_ATOMIC_SITE_OP_IDS.get(aux) != op + or result_used != expected_result_used + or (return_ready and not result_used) + or (value_zero and op != 0) + or (payload and op != 3) + or func_id != -1 + ): + raise ValueError( + f"fdwic_events[{row_index}] has invalid direct Atomic site={aux} flags=0x{flags:x}" + ) + if result_used: + v3_result_used_direct_rows.append((row_index, core_id, return_ready)) + if phase == "ClockBaseline" and trace_schema_version == 3: + dependency = bool(flags & 0x1) + dependency_applied = bool(flags & 0x2) + if flags & ~0x3 or (dependency_applied and not dependency) or task_id != -1 or func_id != -1 or aux != 0: + raise ValueError(f"fdwic_events[{row_index}] has invalid ClockBaseline flags=0x{flags:x} aux={aux}") + clock_state = v3_clock_rows[core_id] + if dependency: + clock_state["dependency"] += 1 + clock_state["return_ready"] = dependency_applied + else: + clock_state["plain"] += 1 + if phase == "Claim" and trace_schema_version == 3 and aux > 1: + raise ValueError(f"fdwic_events[{row_index}] has invalid Claim aux {aux}") + if phase == "Atomic": + observed_summary["atomic_records"] += 1 + if flags & (1 << 7): + call_count = (flags >> 8) & 0xFFFFFF + observed_summary["atomic_calls"] += call_count + observed_summary["batched_poll_calls"] += call_count + observed_summary["poll_batch_records"] += 1 + else: + observed_summary["atomic_calls"] += 1 + elif phase == "ClockBaseline": + observed_summary["clock_baseline_records"] += 1 + start_us = _to_us(start_cycles) + end_us = _to_us(end_cycles) fdwic_events.append( { - "core_id": int(core_id), - "block_id": int(block_id), - "lane": int(lane), - "task_id": int(task_id), - "func_id": int(func_id), - "phase": str(phase), + "core_id": core_id, + "block_id": block_id, + "lane": lane, + "task_id": task_id, + "func_id": func_id, + "phase": phase, + "start_cycles": start_cycles, + "end_cycles": end_cycles, "start_time_us": start_us, "end_time_us": end_us, "duration_us": end_us - start_us, - "flags": int(flags), - "aux": int(aux), + "flags": flags, + "aux": aux, } ) + if trace_schema_version == 3: + for core_id in range(num_cores): + clock_state = v3_clock_rows[core_id] + if clock_state["plain"] != 1 or clock_state["dependency"] != 1: + raise ValueError( + f"core {core_id} requires exactly one plain and one dependency ClockBaseline: " + f"plain={clock_state['plain']} dependency={clock_state['dependency']}" + ) + for row_index, core_id, return_ready in v3_result_used_direct_rows: + expected_return_ready = bool(v3_clock_rows[core_id]["return_ready"]) + if return_ready != expected_return_ready: + raise ValueError( + f"fdwic_events[{row_index}] direct Atomic return_ready={return_ready} does not match " + f"core {core_id} ClockBaseline dependency_applied={expected_return_ready}" + ) + if not isinstance(fdwic_summary, dict): + raise ValueError("metadata.fdwic_summary is required for trace_schema_version=3") + required_summary = { + "records": observed_summary["records"], + "atomic_records": observed_summary["atomic_records"], + "clock_baseline_records": observed_summary["clock_baseline_records"], + "atomic_calls": observed_summary["atomic_calls"], + "batched_poll_calls": observed_summary["batched_poll_calls"], + "poll_batch_records": observed_summary["poll_batch_records"], + "dropped_records": 0, + } + for key, observed_value in required_summary.items(): + try: + producer_value = int(fdwic_summary[key]) + except (KeyError, TypeError, ValueError) as exc: + raise ValueError(f"metadata.fdwic_summary.{key} is missing or invalid") from exc + if producer_value != observed_value: + raise ValueError( + f"metadata.fdwic_summary.{key}={producer_value} does not match raw value {observed_value}" + ) + out = { "l2_swimlane_level": level, + "clock_freq_hz": clock_freq_hz, + "trace_schema_version": trace_schema_version, "tasks": tasks, } if aicpu_scheduler_phases: @@ -497,6 +864,8 @@ def _phase_us(pr): out["core_to_thread"] = core_to_thread if fdwic_events: out["fdwic_events"] = fdwic_events + if trace_schema_version == 3: + out["fdwic_summary"] = dict(fdwic_summary) return out @@ -998,6 +1367,8 @@ def generate_chrome_trace_json( # noqa: PLR0912, PLR0913, PLR0915 deps_kernel_map=None, emit_overhead=False, fdwic_events=None, + trace_schema_version=1, + clock_freq_hz=0, ): """Generate Chrome Trace Event Format JSON from task data. @@ -1013,6 +1384,9 @@ def generate_chrome_trace_json( # noqa: PLR0912, PLR0913, PLR0915 scheduler_phases: Optional list of per-thread phase record lists (l2_swimlane_level >= 3) orchestrator_phases: Optional list of per-task orchestrator phase records (l2_swimlane_level >= 4) core_to_thread: Optional list mapping core_id (index) to scheduler thread index (-1 = unassigned) + trace_schema_version: FDWIC raw record schema (1 legacy, 2 explicit Claim flags, + 3 exact-count Atomic PollBatch rows) + clock_freq_hz: Raw FDWIC cycle-counter frequency used by ClockBaseline event arguments Generates processes in the trace: - pid=4 "Worker View": start_time_us to end_time_us (kernel execution) @@ -1057,7 +1431,13 @@ def generate_chrome_trace_json( # noqa: PLR0912, PLR0913, PLR0915 events = [] if fdwic_events and not tasks: - _append_fdwic_dist_engine_events(events, fdwic_events, func_id_to_name) + _append_fdwic_dist_engine_events( + events, + fdwic_events, + func_id_to_name, + trace_schema_version=trace_schema_version, + clock_freq_hz=clock_freq_hz, + ) with open(output_path, "w") as f: json.dump({"displayTimeUnit": "ns", "traceEvents": events}, f, indent=2) if verbose: @@ -2485,6 +2865,8 @@ def main(): deps_kernel_map=deps_kernel_map, emit_overhead=args.overhead, fdwic_events=data.get("fdwic_events"), + trace_schema_version=data.get("trace_schema_version", 1), + clock_freq_hz=data.get("clock_freq_hz", 0), ) if args.overhead and deps_edges is None: print( diff --git a/src/a5/platform/onboard/host/device_runner.cpp b/src/a5/platform/onboard/host/device_runner.cpp index 13ee9de0cc..d6fc5cdfca 100644 --- a/src/a5/platform/onboard/host/device_runner.cpp +++ b/src/a5/platform/onboard/host/device_runner.cpp @@ -56,7 +56,7 @@ extern "C" __attribute__((weak, visibility("hidden"))) int dep_gen_replay_emit_d } extern "C" __attribute__((weak)) int -fdwic_swimlane_host_init(Runtime *runtime, int num_cores, int enabled, const char *output_prefix); +fdwic_swimlane_host_init(Runtime *runtime, int num_cores, int level, const char *output_prefix); extern "C" __attribute__((weak)) int fdwic_swimlane_host_export(Runtime *runtime); extern "C" __attribute__((weak)) void fdwic_swimlane_host_finalize(Runtime *runtime); @@ -158,11 +158,9 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { // failure cascade into a single fast, self-explanatory error; the runner is // then recovered at finalize. if (device_unusable_) { - LOG_ERROR( - "DeviceRunner marked unusable by a prior AICore failure; refusing to run. " - "A soft reset does not clear the poison on a5; finalize() will force-reset " - "the card so the next Worker on it inits clean." - ); + LOG_ERROR("DeviceRunner marked unusable by a prior AICore failure; refusing to run. " + "A soft reset does not clear the poison on a5; finalize() will force-reset " + "the card so the next Worker on it inits clean."); return -1; } if (validate_launch_aicpu_num(launch_aicpu_num) != 0) return -1; @@ -270,8 +268,16 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { enable_l2_swimlane_ = original_enable_l2_swimlane; }); bool fdwic_swimlane_active = false; + auto fdwic_swimlane_cleanup = RAIIScopeGuard([&runtime, &fdwic_swimlane_active]() { + if (fdwic_swimlane_active && fdwic_swimlane_host_finalize != nullptr) { + fdwic_swimlane_host_finalize(&runtime); + fdwic_swimlane_active = false; + } + }); if (enable_l2_swimlane_ && fdwic_swimlane_host_init != nullptr) { - rc = fdwic_swimlane_host_init(&runtime, num_aicore, 1, output_prefix_.c_str()); + rc = fdwic_swimlane_host_init( + &runtime, num_aicore, static_cast(l2_swimlane_level_), output_prefix_.c_str() + ); if (rc < 0) { LOG_ERROR("fdwic swimlane init failed: %d", rc); return rc; @@ -279,6 +285,14 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { if (rc > 0) { fdwic_swimlane_active = true; enable_l2_swimlane_ = false; + // The FDWIC runtime owns its trace buffer and does not use the + // platform-generic L2 collector. Keep the launch bit and generic + // pointers consistent with that routing decision; these fields + // may otherwise retain a prior run's collector addresses. + CLEAR_PROFILING_FLAG(enable_profiling_flag, PROFILING_FLAG_L2_SWIMLANE); + kernel_args_.args.enable_profiling_flag = enable_profiling_flag; + kernel_args_.args.l2_swimlane_data_base = 0; + kernel_args_.args.l2_swimlane_aicore_rotation_table = 0; } } if (enable_l2_swimlane_) { @@ -325,12 +339,8 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { // Cleanup guard for early returns: stops all started collectors so // their mgmt + poll threads exit cleanly. stop() is idempotent and a // no-op on collectors that never started. - auto perf_cleanup = RAIIScopeGuard([this, &runtime, &fdwic_swimlane_active]() { + auto perf_cleanup = RAIIScopeGuard([this]() { finalize_collectors(); - if (fdwic_swimlane_active && fdwic_swimlane_host_finalize != nullptr) { - fdwic_swimlane_host_finalize(&runtime); - fdwic_swimlane_active = false; - } }); LOG_INFO_V0("=== Initialize runtime args ==="); @@ -433,7 +443,10 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { // still exports exactly once below. teardown_shared_collectors_after_run(); if (fdwic_swimlane_active && fdwic_swimlane_host_export != nullptr) { - fdwic_swimlane_host_export(&runtime); + const int export_rc = fdwic_swimlane_host_export(&runtime); + if (export_rc != 0) { + LOG_ERROR("fdwic swimlane export failed after runtime error: %d", export_rc); + } } return rc; } @@ -441,8 +454,12 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { read_device_wall_ns(); teardown_shared_collectors_after_run(); + int fdwic_swimlane_export_rc = 0; if (fdwic_swimlane_active && fdwic_swimlane_host_export != nullptr) { - fdwic_swimlane_host_export(&runtime); + fdwic_swimlane_export_rc = fdwic_swimlane_host_export(&runtime); + if (fdwic_swimlane_export_rc != 0) { + LOG_ERROR("fdwic swimlane export failed: %d", fdwic_swimlane_export_rc); + } } // a5-specific dep_gen teardown: stop + reconcile + replay emit. @@ -461,7 +478,7 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { // Print handshake results (reads from device memory, must be before free) print_handshake_results(); - return 0; + return fdwic_swimlane_export_rc; } void DeviceRunner::recover_device_or_mark_unusable(int aicore_rc) { diff --git a/src/a5/platform/sim/host/device_runner.cpp b/src/a5/platform/sim/host/device_runner.cpp index d377772f1f..a2c2d633d6 100644 --- a/src/a5/platform/sim/host/device_runner.cpp +++ b/src/a5/platform/sim/host/device_runner.cpp @@ -56,7 +56,7 @@ extern "C" __attribute__((weak, visibility("hidden"))) int dep_gen_replay_emit_d } extern "C" __attribute__((weak)) int -fdwic_swimlane_host_init(Runtime *runtime, int num_cores, int enabled, const char *output_prefix); +fdwic_swimlane_host_init(Runtime *runtime, int num_cores, int level, const char *output_prefix); extern "C" __attribute__((weak)) int fdwic_swimlane_host_export(Runtime *runtime); extern "C" __attribute__((weak)) void fdwic_swimlane_host_finalize(Runtime *runtime); @@ -262,8 +262,16 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { enable_l2_swimlane_ = original_enable_l2_swimlane; }); bool fdwic_swimlane_active = false; + auto fdwic_swimlane_cleanup = RAIIScopeGuard([&runtime, &fdwic_swimlane_active]() { + if (fdwic_swimlane_active && fdwic_swimlane_host_finalize != nullptr) { + fdwic_swimlane_host_finalize(&runtime); + fdwic_swimlane_active = false; + } + }); if (enable_l2_swimlane_ && fdwic_swimlane_host_init != nullptr) { - rc = fdwic_swimlane_host_init(&runtime, num_aicore, 1, output_prefix_.c_str()); + rc = fdwic_swimlane_host_init( + &runtime, num_aicore, static_cast(l2_swimlane_level_), output_prefix_.c_str() + ); if (rc < 0) { LOG_ERROR("fdwic swimlane init failed: %d", rc); return rc; @@ -271,6 +279,14 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { if (rc > 0) { fdwic_swimlane_active = true; enable_l2_swimlane_ = false; + // The FDWIC runtime owns its trace buffer and does not use the + // platform-generic L2 collector. Keep the launch bit and generic + // pointers consistent with that routing decision; these fields + // may otherwise retain a prior run's collector addresses. + CLEAR_PROFILING_FLAG(enable_profiling_flag, PROFILING_FLAG_L2_SWIMLANE); + kernel_args_.enable_profiling_flag = enable_profiling_flag; + kernel_args_.l2_swimlane_data_base = 0; + kernel_args_.l2_swimlane_aicore_rotation_table = 0; } } @@ -325,12 +341,8 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { // Cleanup guard for early returns: stops all started collectors so their // mgmt + poll threads exit cleanly. stop() is idempotent and a no-op on // collectors that never started. - auto perf_cleanup = RAIIScopeGuard([this, &runtime, &fdwic_swimlane_active]() { + auto perf_cleanup = RAIIScopeGuard([this]() { stop_collectors(); - if (fdwic_swimlane_active && fdwic_swimlane_host_finalize != nullptr) { - fdwic_swimlane_host_finalize(&runtime); - fdwic_swimlane_active = false; - } }); // Allocate simulated register blocks for all AICore cores. Uses sparse @@ -472,7 +484,10 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { if (runtime_rc != 0) { LOG_ERROR("AICPU execution failed with rc=%d", runtime_rc); if (fdwic_swimlane_active && fdwic_swimlane_host_export != nullptr) { - fdwic_swimlane_host_export(&runtime); + const int export_rc = fdwic_swimlane_host_export(&runtime); + if (export_rc != 0) { + LOG_ERROR("fdwic swimlane export failed after runtime error: %d", export_rc); + } } return runtime_rc; } @@ -485,8 +500,12 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { l2_swimlane_collector_.reconcile_counters(); l2_swimlane_collector_.export_swimlane_json(); } + int fdwic_swimlane_export_rc = 0; if (fdwic_swimlane_active && fdwic_swimlane_host_export != nullptr) { - fdwic_swimlane_host_export(&runtime); + fdwic_swimlane_export_rc = fdwic_swimlane_host_export(&runtime); + if (fdwic_swimlane_export_rc != 0) { + LOG_ERROR("fdwic swimlane export failed: %d", fdwic_swimlane_export_rc); + } } if (enable_dump_tensor_) { @@ -530,7 +549,7 @@ int DeviceRunner::run(Runtime &runtime, int block_dim, int launch_aicpu_num) { aicore_so_path_.clear(); } - return 0; + return fdwic_swimlane_export_rc; } void DeviceRunner::unload_executor_binaries() { diff --git a/src/a5/runtime/fully_distributed_within_core/host/fdwic_swimlane.cpp b/src/a5/runtime/fully_distributed_within_core/host/fdwic_swimlane.cpp index ea720fb1de..725031058e 100644 --- a/src/a5/runtime/fully_distributed_within_core/host/fdwic_swimlane.cpp +++ b/src/a5/runtime/fully_distributed_within_core/host/fdwic_swimlane.cpp @@ -11,6 +11,7 @@ #include "runtime.h" +#include #include #include @@ -18,6 +19,7 @@ #include #include #include +#include #include #include "common/platform_config.h" @@ -26,6 +28,27 @@ namespace { +constexpr uint32_t kFdwicSwimlaneMaxLevel = kFdwicAtomicSwimlaneLevel; +constexpr int32_t kFdwicSwimlanePhaseCount = static_cast(FdwicSwimlanePhase::ClockBaseline) + 1; + +struct TraceSummary { + uint64_t records = 0; + uint64_t atomic_records = 0; + uint64_t clock_baseline_records = 0; + uint64_t atomic_calls = 0; + uint64_t poll_calls = 0; + uint64_t poll_batch_records = 0; + uint64_t dropped_records = 0; +}; + +bool is_cpu_sim_trace() { +#if defined(SIMPLER_PLATFORM_NAME) + return std::strcmp(SIMPLER_PLATFORM_NAME, "a5sim") == 0; +#else + return false; +#endif +} + const char *phase_name(int32_t phase) { switch (static_cast(phase)) { case FdwicSwimlanePhase::Kernel: @@ -56,17 +79,396 @@ const char *phase_name(int32_t phase) { return "Fanin"; case FdwicSwimlanePhase::Register: return "Register"; + case FdwicSwimlanePhase::Atomic: + return "Atomic"; + case FdwicSwimlanePhase::ClockBaseline: + return "ClockBaseline"; } return "Unknown"; } -const char *core_type_name(uint32_t core_idx) { return (core_idx % 3 == 0) ? "aic" : "aiv"; } +const char *atomic_site_name(uint32_t site) { + static constexpr const char *names[] = { + "StartupIncrement", + "StartupPoll", + "FatalPoll", + "FatalSet", + "ClaimMax", + "FaninFlagLoad", + "CompletionVendExchange", + "CompletionFlagExchange", + "FrontierInitialLoad", + "FrontierFlagLoad", + "FrontierMax", + "HeapFrontierLoad", + "HeapVendLoad", + "ReplayDoneIncrement", + "ReplayDonePoll", + "WonSlotClaimMax", + "WonRemainingExchange", + "WonLaneResetExchange", + "WonLaneDepositExchange", + "WonStatePublishExchange", + "WonAnyPublishExchange", + "WonAnyLoad", + "WonStateLoad", + "WonLaneClaimExchange", + "WonLaneReleaseExchange", + "WonRemainingFetchSub", + "WonStateClearExchange", + "WonDrainedLoad", + }; + static_assert( + sizeof(names) / sizeof(names[0]) == static_cast(FdwicAtomicSite::Count), + "atomic site name table must match the raw ABI" + ); + return site < sizeof(names) / sizeof(names[0]) ? names[site] : "Unknown"; +} + +const char *atomic_op_name(uint32_t op) { + static constexpr const char *names[] = {"Load", "Exchange", "FetchAdd", "FetchMax", "FetchSub"}; + return op < sizeof(names) / sizeof(names[0]) ? names[op] : "Unknown"; +} + +const char *core_type_name(CoreType core_type) { + switch (core_type) { + case CoreType::AIC: + return "aic"; + case CoreType::AIV: + return "aiv"; + } + return "unknown"; +} + +bool build_expected_core_layout( + const Runtime *runtime, uint32_t num_cores, int32_t expected_blocks[RUNTIME_MAX_WORKER], + int32_t expected_lanes[RUNTIME_MAX_WORKER] +) { + uint32_t aic_count = 0; + for (uint32_t core = 0; core < num_cores; ++core) { + const CoreType core_type = runtime->workers[core].core_type; + if (core_type == CoreType::AIC) { + expected_blocks[core] = static_cast(aic_count++); + expected_lanes[core] = 0; + } else if (core_type != CoreType::AIV) { + LOG_ERROR("fdwic swimlane core %u has invalid core type %d", core, static_cast(core_type)); + return false; + } + } + if (aic_count == 0) { + LOG_ERROR("fdwic swimlane topology has no AIC workers"); + return false; + } + + uint32_t aiv_ordinal = 0; + for (uint32_t core = 0; core < num_cores; ++core) { + if (runtime->workers[core].core_type != CoreType::AIV) continue; + const uint32_t block = aiv_ordinal / 2; + if (block >= aic_count) { + LOG_ERROR( + "fdwic swimlane AIV worker %u cannot map to an AIC block: aic=%u aiv_ordinal=%u", core, aic_count, + aiv_ordinal + ); + return false; + } + expected_blocks[core] = static_cast(block); + expected_lanes[core] = 1 + static_cast(aiv_ordinal % 2); + ++aiv_ordinal; + } + if (aiv_ordinal != 2 * aic_count) { + LOG_ERROR( + "fdwic swimlane topology must contain two AIV workers per AIC: aic=%u aiv=%u", aic_count, aiv_ordinal + ); + return false; + } + return true; +} + +bool atomic_record_schema_valid(const FdwicSwimlaneRecord &record) { + if (record.aux >= static_cast(FdwicAtomicSite::Count)) return false; + const auto site = static_cast(record.aux); + const uint32_t op = record.flags & kFdwicAtomicOpMask; + if (op != static_cast(fdwic_atomic_site_op(site))) return false; + + const bool result_used = (record.flags & kFdwicAtomicResultUsed) != 0; + const bool return_ready = (record.flags & kFdwicAtomicReturnReady) != 0; + const bool value_zero = (record.flags & kFdwicAtomicValueZero) != 0; + const bool poll_batch = (record.flags & kFdwicAtomicPollBatch) != 0; + const uint32_t payload = record.flags >> kFdwicAtomicRetriesShift; + if (poll_batch) { + return fdwic_atomic_site_is_poll_batchable(site) && result_used && !return_ready && !value_zero && + payload > 0 && record.task_id == -1 && record.func_id == -1; + } + const bool expected_return_ready = result_used && !is_cpu_sim_trace(); + if (result_used != fdwic_atomic_site_result_used(site) || return_ready != expected_return_ready) return false; + if (value_zero && op != static_cast(FdwicAtomicOp::Load)) return false; + if (payload != 0 && op != static_cast(FdwicAtomicOp::FetchMax)) return false; + return record.func_id == -1; +} + +uint32_t atomic_record_call_count(const FdwicSwimlaneRecord &record) { + return (record.flags & kFdwicAtomicPollBatch) != 0 ? record.flags >> kFdwicAtomicPollCountShift : 1U; +} + +bool claim_record_schema_valid(const FdwicSwimlaneRecord &record, uint32_t level) { + if (level < kFdwicAtomicSwimlaneLevel) return record.flags <= 1 && record.aux <= 1; + if ((record.flags & ~(kFdwicClaimWon | kFdwicClaimAttempted)) != 0) return false; + if ((record.flags & kFdwicClaimWon) != 0 && (record.flags & kFdwicClaimAttempted) == 0) return false; + return record.aux <= 1; +} + +bool clock_record_schema_valid(const FdwicSwimlaneRecord &record) { + if ((record.flags & ~(kFdwicClockAtomicDependency | kFdwicClockAtomicDependencyApplied)) != 0) return false; + if ((record.flags & kFdwicClockAtomicDependencyApplied) != 0 && (record.flags & kFdwicClockAtomicDependency) == 0) { + return false; + } + const bool dependency = (record.flags & kFdwicClockAtomicDependency) != 0; + const bool dependency_applied = (record.flags & kFdwicClockAtomicDependencyApplied) != 0; + if (dependency_applied != (dependency && !is_cpu_sim_trace())) return false; + return record.task_id == -1 && record.func_id == -1 && record.aux == 0; +} + +bool ordinary_record_schema_valid(const FdwicSwimlaneRecord &record) { + switch (static_cast(record.phase)) { + case FdwicSwimlanePhase::Kernel: + case FdwicSwimlanePhase::Commit: + return record.flags <= 1 && record.aux == 0; + case FdwicSwimlanePhase::DrainWon: + return record.flags == 1 && record.aux < 4; + case FdwicSwimlanePhase::RingBp: + return record.flags == 0 && record.aux <= 1; + case FdwicSwimlanePhase::Submit: + return record.flags <= 1 && record.aux <= 1; + case FdwicSwimlanePhase::Materialize: + case FdwicSwimlanePhase::PrepareMap: + case FdwicSwimlanePhase::Register: + return record.flags == 0 && record.aux <= 1; + case FdwicSwimlanePhase::Fanin: + return record.flags == 0 && record.aux <= 16; + case FdwicSwimlanePhase::Alloc: + case FdwicSwimlanePhase::Build: + case FdwicSwimlanePhase::Replay: + case FdwicSwimlanePhase::EfDrain: + return record.flags == 0 && record.aux == 0; + case FdwicSwimlanePhase::Claim: + case FdwicSwimlanePhase::Atomic: + case FdwicSwimlanePhase::ClockBaseline: + return true; + } + return false; +} + +bool validate_header_and_counts( + const Runtime *runtime, const FdwicSwimlaneHeader *header, uint32_t level, TraceSummary &summary, + uint32_t &max_core_records +) { + const uint32_t expected_cores = runtime->fdwic_swimlane_num_cores_; + const uint32_t expected_capacity = runtime->fdwic_swimlane_records_per_core_; + const uint64_t expected_bytes = sizeof(FdwicSwimlaneHeader) + static_cast(expected_cores) * + expected_capacity * sizeof(FdwicSwimlaneRecord); + const bool header_valid = + header->magic == kFdwicSwimlaneMagic && header->version == kFdwicSwimlaneVersion && expected_cores > 0 && + expected_cores <= RUNTIME_MAX_WORKER && header->num_cores == expected_cores && + runtime->worker_count == static_cast(expected_cores) && expected_capacity > 0 && + header->records_per_core == expected_capacity && header->freq_hz == PLATFORM_PROF_SYS_CNT_FREQ && + runtime->fdwic_swimlane_bytes_ == expected_bytes && runtime->dist.swimlane_level == level && + runtime->dist.swimlane_base == runtime->fdwic_swimlane_dev_base_ && + runtime->dist.swimlane_records_per_core == expected_capacity; + if (!header_valid) { + LOG_ERROR( + "fdwic swimlane invalid header/state: magic=0x%08x version=%u cores=%u/%u worker_count=%d " + "capacity=%u/%u freq=%llu bytes=%llu/%llu", + header->magic, header->version, header->num_cores, expected_cores, runtime->worker_count, + header->records_per_core, expected_capacity, static_cast(header->freq_hz), + static_cast(runtime->fdwic_swimlane_bytes_), + static_cast(expected_bytes) + ); + return false; + } + + for (uint32_t core = 0; core < expected_cores; ++core) { + const FdwicSwimlaneCoreState &core_state = header->cores[core]; + summary.records += core_state.count; + summary.atomic_calls += core_state.atomic_calls; + summary.poll_calls += core_state.poll_calls; + summary.poll_batch_records += core_state.poll_batch_records; + summary.dropped_records += core_state.dropped; + if (core_state.count > max_core_records) max_core_records = core_state.count; + if (core_state.count > expected_capacity || core_state.dropped != 0) { + LOG_ERROR( + "fdwic swimlane core %u is incomplete: count=%u capacity=%u dropped=%u atomic_calls=%u", core, + core_state.count, expected_capacity, core_state.dropped, core_state.atomic_calls + ); + return false; + } + if (level < kFdwicAtomicSwimlaneLevel && + (core_state.atomic_calls != 0 || core_state.poll_calls != 0 || core_state.poll_batch_records != 0)) { + LOG_ERROR( + "fdwic swimlane level-%u core %u unexpectedly reports atomic counters: calls=%u poll_calls=%u " + "poll_batches=%u", + level, core, core_state.atomic_calls, core_state.poll_calls, core_state.poll_batch_records + ); + return false; + } + if (level >= kFdwicAtomicSwimlaneLevel) { + if (core_state.poll_calls > core_state.atomic_calls || + (core_state.poll_calls == 0) != (core_state.poll_batch_records == 0)) { + LOG_ERROR( + "fdwic swimlane level-4 core %u has invalid poll counters: calls=%u poll_calls=%u " + "poll_batches=%u", + core, core_state.atomic_calls, core_state.poll_calls, core_state.poll_batch_records + ); + return false; + } + const uint64_t atomic_records = + static_cast(core_state.atomic_calls) - core_state.poll_calls + core_state.poll_batch_records; + if (core_state.count < 2 || atomic_records > core_state.count - 2) { + LOG_ERROR( + "fdwic swimlane level-4 core %u cannot close physical rows: count=%u atomic_records=%llu " + "atomic_calls=%u poll_calls=%u poll_batches=%u", + core, core_state.count, static_cast(atomic_records), core_state.atomic_calls, + core_state.poll_calls, core_state.poll_batch_records + ); + return false; + } + summary.atomic_records += atomic_records; + } + } + if (level >= kFdwicAtomicSwimlaneLevel) { + summary.clock_baseline_records = 2 * static_cast(expected_cores); + } + return true; +} + +bool validate_and_write_core( + const FdwicSwimlaneHeader *header, const FdwicSwimlaneRecord *records, uint32_t core, int32_t expected_block, + int32_t expected_lane, uint32_t level, std::ofstream &out, bool &first, TraceSummary &observed +) { + const FdwicSwimlaneCoreState &core_state = header->cores[core]; + if (core_state.core_idx != static_cast(core) || core_state.block_id != expected_block || + core_state.lane != expected_lane) { + LOG_ERROR( + "fdwic swimlane invalid worker identity: worker=%u core=%d block=%d/%d lane=%d/%d", core, + core_state.core_idx, core_state.block_id, expected_block, core_state.lane, expected_lane + ); + return false; + } + uint32_t core_atomic_records = 0; + uint64_t core_atomic_calls = 0; + uint64_t core_poll_calls = 0; + uint32_t core_poll_batch_records = 0; + uint32_t core_clock_records = 0; + uint32_t core_plain_clock_records = 0; + uint32_t core_dependency_clock_records = 0; + for (uint32_t index = 0; index < core_state.count; ++index) { + const FdwicSwimlaneRecord &record = records[index]; + const bool base_valid = record.end_cycle >= record.start_cycle && record.phase < kFdwicSwimlanePhaseCount && + record.task_id >= -1 && record.func_id >= -1; + bool schema_valid = base_valid; + if (record.phase == static_cast(FdwicSwimlanePhase::Atomic)) { + schema_valid = schema_valid && atomic_record_schema_valid(record); + ++core_atomic_records; + const uint32_t call_count = atomic_record_call_count(record); + core_atomic_calls += call_count; + if ((record.flags & kFdwicAtomicPollBatch) != 0) { + core_poll_calls += call_count; + ++core_poll_batch_records; + } + } else if (record.phase == static_cast(FdwicSwimlanePhase::Claim)) { + schema_valid = schema_valid && claim_record_schema_valid(record, level); + } else if (record.phase == static_cast(FdwicSwimlanePhase::ClockBaseline)) { + schema_valid = schema_valid && clock_record_schema_valid(record); + ++core_clock_records; + if ((record.flags & kFdwicClockAtomicDependency) != 0) { + ++core_dependency_clock_records; + } else { + ++core_plain_clock_records; + } + } else { + schema_valid = schema_valid && ordinary_record_schema_valid(record); + } + if (!schema_valid) { + const uint32_t op = record.flags & kFdwicAtomicOpMask; + LOG_ERROR( + "fdwic swimlane invalid record: worker=%u index=%u core=%d block=%d lane=%d phase=%d(%s) " + "task=%d func=%d start=%llu end=%llu flags=0x%08x aux=%u site=%s op=%s", + core, index, core_state.core_idx, core_state.block_id, core_state.lane, record.phase, + phase_name(record.phase), record.task_id, record.func_id, + static_cast(record.start_cycle), static_cast(record.end_cycle), + record.flags, record.aux, atomic_site_name(record.aux), atomic_op_name(op) + ); + return false; + } + if (!first) out << ","; + out << "\n [" << core_state.core_idx << ", " << core_state.block_id << ", " << core_state.lane << ", " + << record.task_id << ", " << record.func_id << ", \"" << phase_name(record.phase) << "\", " + << record.start_cycle << ", " << record.end_cycle << ", " << record.flags << ", " << record.aux << "]"; + first = false; + } + + if (level >= kFdwicAtomicSwimlaneLevel) { + if (core_atomic_records != static_cast(core_state.atomic_calls) - core_state.poll_calls + + core_state.poll_batch_records || + core_atomic_calls != core_state.atomic_calls || core_poll_calls != core_state.poll_calls || + core_poll_batch_records != core_state.poll_batch_records || core_clock_records != 2 || + core_plain_clock_records != 1 || core_dependency_clock_records != 1) { + LOG_ERROR( + "fdwic swimlane level-4 closure failed on core %u: atomic_records=%u atomic_calls=%llu/%u " + "poll_calls=%llu/%u poll_batches=%u/%u clock=%u plain_clock=%u dependency_clock=%u", + core, core_atomic_records, static_cast(core_atomic_calls), core_state.atomic_calls, + static_cast(core_poll_calls), core_state.poll_calls, core_poll_batch_records, + core_state.poll_batch_records, core_clock_records, core_plain_clock_records, + core_dependency_clock_records + ); + return false; + } + } else if (core_atomic_records != 0 || core_state.atomic_calls != 0 || core_state.poll_calls != 0 || + core_state.poll_batch_records != 0 || core_clock_records != 0) { + LOG_ERROR( + "fdwic swimlane level-%u contains level-4 records on core %u: atomic_records=%u atomic_calls=%u " + "poll_calls=%u poll_batches=%u clock=%u", + level, core, core_atomic_records, core_state.atomic_calls, core_state.poll_calls, + core_state.poll_batch_records, core_clock_records + ); + return false; + } + observed.records += core_state.count; + observed.atomic_records += core_atomic_records; + observed.clock_baseline_records += core_clock_records; + observed.atomic_calls += core_atomic_calls; + observed.poll_calls += core_poll_calls; + observed.poll_batch_records += core_poll_batch_records; + observed.dropped_records += core_state.dropped; + return true; +} + +std::string output_path_from_prefix(const std::string &prefix) { + if (!prefix.empty() && prefix.back() == '/') return prefix + "l2_swimlane_records.json"; + return prefix + "/l2_swimlane_records.json"; +} std::string output_path(const Runtime *runtime) { - std::string base = runtime->fdwic_swimlane_output_prefix_; - if (base.empty()) base = "."; - mkdir(base.c_str(), 0755); - return base + "/l2_swimlane_records.json"; + return output_path_from_prefix(runtime->fdwic_swimlane_output_prefix_); +} + +bool prepare_output_directory(const std::string &prefix) { + struct stat info {}; + if (stat(prefix.c_str(), &info) == 0) { + if (S_ISDIR(info.st_mode)) return true; + LOG_ERROR("fdwic swimlane output prefix is not a directory: %s", prefix.c_str()); + return false; + } + if (errno != ENOENT || mkdir(prefix.c_str(), 0755) != 0) { + LOG_ERROR("cannot create fdwic swimlane output directory %s: %s", prefix.c_str(), std::strerror(errno)); + return false; + } + return true; +} + +bool remove_output_if_present(const std::string &path) { + if (std::remove(path.c_str()) == 0 || errno == ENOENT) return true; + LOG_ERROR("cannot remove stale fdwic swimlane output %s: %s", path.c_str(), std::strerror(errno)); + return false; } bool should_print_trace_export() { @@ -79,30 +481,51 @@ bool should_print_trace_export() { } // namespace -extern "C" int fdwic_swimlane_host_init(Runtime *runtime, int num_cores, int enabled, const char *output_prefix) { +extern "C" int fdwic_swimlane_host_init(Runtime *runtime, int num_cores, int level, const char *output_prefix) { if (runtime == nullptr) return -1; - runtime->dist.swimlane_enabled = 0; + runtime->dist.swimlane_level = 0; runtime->dist.swimlane_base = 0; runtime->dist.swimlane_records_per_core = 0; runtime->fdwic_swimlane_host_shadow_ = nullptr; + runtime->fdwic_swimlane_dev_allocation_ = 0; runtime->fdwic_swimlane_dev_base_ = 0; runtime->fdwic_swimlane_bytes_ = 0; runtime->fdwic_swimlane_num_cores_ = 0; runtime->fdwic_swimlane_records_per_core_ = 0; runtime->fdwic_swimlane_output_prefix_[0] = '\0'; - if (!enabled) return 0; - if (num_cores <= 0 || num_cores > 108) return -1; + if (level < 0 || level > static_cast(kFdwicSwimlaneMaxLevel)) { + LOG_ERROR("fdwic swimlane level %d is outside [0, %u]", level, kFdwicSwimlaneMaxLevel); + return -1; + } + if (level == 0) return 0; + if (num_cores <= 0 || num_cores > RUNTIME_MAX_WORKER) return -1; if (runtime->host_api.device_malloc == nullptr || runtime->host_api.device_free == nullptr || runtime->host_api.copy_to_device == nullptr || runtime->host_api.copy_from_device == nullptr) { return -1; } - const uint32_t records_per_core = kFdwicSwimlaneDefaultRecordsPerCore; + const std::string exact_output_prefix = output_prefix == nullptr || output_prefix[0] == '\0' ? "." : output_prefix; + if (exact_output_prefix.size() >= sizeof(runtime->fdwic_swimlane_output_prefix_)) { + LOG_ERROR("fdwic swimlane output prefix is too long: %zu bytes", exact_output_prefix.size()); + return -1; + } + if (!prepare_output_directory(exact_output_prefix)) return -1; + const std::string path = output_path_from_prefix(exact_output_prefix); + if (!remove_output_if_present(path) || !remove_output_if_present(path + ".tmp")) return -1; + + const uint32_t records_per_core = level >= static_cast(kFdwicAtomicSwimlaneLevel) ? + kFdwicAtomicSwimlaneRecordsPerCore : + kFdwicSwimlaneDefaultRecordsPerCore; const uint64_t bytes = sizeof(FdwicSwimlaneHeader) + static_cast(num_cores) * records_per_core * sizeof(FdwicSwimlaneRecord); - void *host_shadow = std::malloc(static_cast(bytes)); + constexpr uint64_t kDeviceAlignment = 64; + if (bytes > std::numeric_limits::max() - (kDeviceAlignment - 1)) { + LOG_ERROR("fdwic swimlane allocation is too large: %llu bytes", static_cast(bytes)); + return -1; + } + void *host_shadow = std::aligned_alloc(64, sizeof(FdwicSwimlaneHeader)); if (host_shadow == nullptr) return -1; - std::memset(host_shadow, 0, static_cast(bytes)); + std::memset(host_shadow, 0, sizeof(FdwicSwimlaneHeader)); FdwicSwimlaneHeader *header = reinterpret_cast(host_shadow); header->magic = kFdwicSwimlaneMagic; header->version = kFdwicSwimlaneVersion; @@ -110,31 +533,30 @@ extern "C" int fdwic_swimlane_host_init(Runtime *runtime, int num_cores, int ena header->records_per_core = records_per_core; header->freq_hz = PLATFORM_PROF_SYS_CNT_FREQ; - void *dev = runtime->host_api.device_malloc(static_cast(bytes)); - if (dev == nullptr) { + void *dev_allocation = runtime->host_api.device_malloc(static_cast(bytes + (kDeviceAlignment - 1))); + if (dev_allocation == nullptr) { std::free(host_shadow); return -1; } - if (runtime->host_api.copy_to_device(dev, host_shadow, static_cast(bytes)) != 0) { - runtime->host_api.device_free(dev); + const uintptr_t dev_base = + (reinterpret_cast(dev_allocation) + (kDeviceAlignment - 1)) & ~(kDeviceAlignment - 1); + void *dev = reinterpret_cast(dev_base); + if (runtime->host_api.copy_to_device(dev, host_shadow, sizeof(FdwicSwimlaneHeader)) != 0) { + runtime->host_api.device_free(dev_allocation); std::free(host_shadow); return -1; } runtime->fdwic_swimlane_host_shadow_ = host_shadow; - runtime->fdwic_swimlane_dev_base_ = reinterpret_cast(dev); + runtime->fdwic_swimlane_dev_allocation_ = reinterpret_cast(dev_allocation); + runtime->fdwic_swimlane_dev_base_ = dev_base; runtime->fdwic_swimlane_bytes_ = bytes; runtime->fdwic_swimlane_num_cores_ = static_cast(num_cores); runtime->fdwic_swimlane_records_per_core_ = records_per_core; - if (output_prefix != nullptr) { - std::strncpy( - runtime->fdwic_swimlane_output_prefix_, output_prefix, sizeof(runtime->fdwic_swimlane_output_prefix_) - 1 - ); - runtime->fdwic_swimlane_output_prefix_[sizeof(runtime->fdwic_swimlane_output_prefix_) - 1] = '\0'; - } + std::memcpy(runtime->fdwic_swimlane_output_prefix_, exact_output_prefix.c_str(), exact_output_prefix.size() + 1); runtime->dist.swimlane_base = runtime->fdwic_swimlane_dev_base_; runtime->dist.swimlane_records_per_core = records_per_core; - runtime->dist.swimlane_enabled = 1; + runtime->dist.swimlane_level = static_cast(level); return 1; } @@ -143,35 +565,103 @@ extern "C" int fdwic_swimlane_host_export(Runtime *runtime) { runtime->fdwic_swimlane_dev_base_ == 0) { return 0; } + const std::string path = output_path(runtime); + const std::string temporary_path = path + ".tmp"; + if (!remove_output_if_present(temporary_path)) return -1; + void *dev = reinterpret_cast(runtime->fdwic_swimlane_dev_base_); - if (runtime->host_api.copy_from_device( - runtime->fdwic_swimlane_host_shadow_, dev, static_cast(runtime->fdwic_swimlane_bytes_) - ) != 0) { - LOG_ERROR("fdwic swimlane D2H copy failed"); + if (runtime->host_api.copy_from_device(runtime->fdwic_swimlane_host_shadow_, dev, sizeof(FdwicSwimlaneHeader)) != + 0) { + LOG_ERROR("fdwic swimlane header D2H copy failed"); + std::remove(temporary_path.c_str()); return -1; } FdwicSwimlaneHeader *header = reinterpret_cast(runtime->fdwic_swimlane_host_shadow_); - if (header->magic != kFdwicSwimlaneMagic || header->version != kFdwicSwimlaneVersion) { - LOG_ERROR("fdwic swimlane header mismatch"); + const uint32_t level = runtime->dist.swimlane_level; + if (level == 0 || level > kFdwicSwimlaneMaxLevel) { + LOG_ERROR("fdwic swimlane export has invalid level %u", level); + std::remove(temporary_path.c_str()); return -1; } - auto *records = reinterpret_cast( - static_cast(runtime->fdwic_swimlane_host_shadow_) + sizeof(FdwicSwimlaneHeader) - ); - std::ofstream out(output_path(runtime)); - if (!out.is_open()) return -1; + TraceSummary summary; + uint32_t max_core_records = 0; + if (!validate_header_and_counts(runtime, header, level, summary, max_core_records)) { + std::remove(temporary_path.c_str()); + return -1; + } + + int32_t expected_blocks[RUNTIME_MAX_WORKER] = {}; + int32_t expected_lanes[RUNTIME_MAX_WORKER] = {}; + if (!build_expected_core_layout(runtime, header->num_cores, expected_blocks, expected_lanes)) { + std::remove(temporary_path.c_str()); + return -1; + } + + FdwicSwimlaneRecord *scratch = nullptr; + if (max_core_records != 0) { + const size_t scratch_bytes = static_cast(max_core_records) * sizeof(FdwicSwimlaneRecord); + scratch = static_cast(std::aligned_alloc(alignof(FdwicSwimlaneRecord), scratch_bytes)); + if (scratch == nullptr) { + LOG_ERROR("cannot allocate fdwic swimlane per-core scratch: %zu bytes", scratch_bytes); + std::remove(temporary_path.c_str()); + return -1; + } + } + + std::ofstream out(temporary_path, std::ios::out | std::ios::trunc); + if (!out.is_open()) { + LOG_ERROR("cannot open fdwic swimlane temporary output %s: %s", temporary_path.c_str(), std::strerror(errno)); + std::free(scratch); + std::remove(temporary_path.c_str()); + return -1; + } + auto fail_export = [&out, &scratch, &temporary_path]() { + out.close(); + std::free(scratch); + scratch = nullptr; + std::remove(temporary_path.c_str()); + return -1; + }; + out << "{\n"; - out << " \"l2_swimlane_level\": 1,\n"; + out << " \"l2_swimlane_level\": " << level << ",\n"; out << " \"metadata\": {\n"; out << " \"clock_freq_hz\": " << header->freq_hz << ",\n"; out << " \"num_cores\": " << header->num_cores << ",\n"; + const uint32_t trace_schema_version = level >= kFdwicAtomicSwimlaneLevel ? kFdwicSwimlaneTraceSchemaVersion : 1; + out << " \"trace_schema_version\": " << trace_schema_version << ",\n"; + out << " \"raw_trace_version\": " << header->version << ",\n"; + out << " \"records_per_core\": " << header->records_per_core << ",\n"; + out << " \"record_size_bytes\": " << sizeof(FdwicSwimlaneRecord) << ",\n"; + out << " \"device_trace_bytes\": " << runtime->fdwic_swimlane_bytes_ << ",\n"; out << " \"core_types\": ["; for (uint32_t c = 0; c < header->num_cores; c++) { if (c > 0) out << ", "; - out << "\"" << core_type_name(c) << "\""; + out << "\"" << core_type_name(runtime->workers[c].core_type) << "\""; + } + out << "],\n"; + out << " \"atomic_site_names\": ["; + for (uint32_t site = 0; site < static_cast(FdwicAtomicSite::Count); ++site) { + if (site > 0) out << ", "; + out << "\"" << atomic_site_name(site) << "\""; } - out << "]\n"; + out << "],\n"; + out << " \"atomic_op_names\": ["; + for (uint32_t op = 0; op <= static_cast(FdwicAtomicOp::FetchSub); ++op) { + if (op > 0) out << ", "; + out << "\"" << atomic_op_name(op) << "\""; + } + out << "],\n"; + out << " \"fdwic_summary\": {\n"; + out << " \"records\": " << summary.records << ",\n"; + out << " \"atomic_records\": " << summary.atomic_records << ",\n"; + out << " \"clock_baseline_records\": " << summary.clock_baseline_records << ",\n"; + out << " \"atomic_calls\": " << summary.atomic_calls << ",\n"; + out << " \"batched_poll_calls\": " << summary.poll_calls << ",\n"; + out << " \"poll_batch_records\": " << summary.poll_batch_records << ",\n"; + out << " \"dropped_records\": " << summary.dropped_records << "\n"; + out << " }\n"; out << " },\n"; out << " \"aicore_tasks\": [],\n"; out << " \"aicpu_tasks\": [],\n"; @@ -179,39 +669,100 @@ extern "C" int fdwic_swimlane_host_export(Runtime *runtime) { out << " \"aicpu_orchestrator_phases\": [],\n"; out << " \"fdwic_events\": ["; bool first = true; + TraceSummary observed; + const uint64_t records_base = runtime->fdwic_swimlane_dev_base_ + sizeof(FdwicSwimlaneHeader); for (uint32_t c = 0; c < header->num_cores; c++) { - const uint32_t count = - header->cores[c].count < header->records_per_core ? header->cores[c].count : header->records_per_core; - for (uint32_t i = 0; i < count; i++) { - const FdwicSwimlaneRecord &r = records[static_cast(c) * header->records_per_core + i]; - if (r.end_cycle < r.start_cycle) continue; - if (!first) out << ","; - out << "\n [" << r.core_idx << ", " << r.block_id << ", " << r.lane << ", " << r.task_id << ", " - << r.func_id << ", \"" << phase_name(r.phase) << "\", " << r.start_cycle << ", " << r.end_cycle << ", " - << r.flags << ", " << r.aux << "]"; - first = false; + const uint32_t count = header->cores[c].count; + if (count != 0) { + const uint64_t core_offset = + static_cast(c) * header->records_per_core * sizeof(FdwicSwimlaneRecord); + const void *core_records_dev = reinterpret_cast(records_base + core_offset); + const size_t core_bytes = static_cast(count) * sizeof(FdwicSwimlaneRecord); + if (runtime->host_api.copy_from_device(scratch, core_records_dev, core_bytes) != 0) { + LOG_ERROR("fdwic swimlane core %u D2H copy failed: records=%u bytes=%zu", c, count, core_bytes); + return fail_export(); + } + } + if (!validate_and_write_core( + header, scratch, c, expected_blocks[c], expected_lanes[c], level, out, first, observed + )) { + return fail_export(); } + if (!out) { + LOG_ERROR("failed while writing fdwic swimlane core %u to %s", c, temporary_path.c_str()); + return fail_export(); + } + } + const bool summary_closed = + observed.records == summary.records && observed.atomic_records == summary.atomic_records && + observed.clock_baseline_records == summary.clock_baseline_records && + observed.atomic_calls == summary.atomic_calls && observed.poll_calls == summary.poll_calls && + observed.poll_batch_records == summary.poll_batch_records && + observed.dropped_records == summary.dropped_records; + if (!summary_closed) { + LOG_ERROR( + "fdwic swimlane summary closure failed: records=%llu/%llu atomic=%llu/%llu clock=%llu/%llu " + "calls=%llu/%llu poll_calls=%llu/%llu poll_batches=%llu/%llu dropped=%llu/%llu", + static_cast(observed.records), static_cast(summary.records), + static_cast(observed.atomic_records), + static_cast(summary.atomic_records), + static_cast(observed.clock_baseline_records), + static_cast(summary.clock_baseline_records), + static_cast(observed.atomic_calls), + static_cast(summary.atomic_calls), static_cast(observed.poll_calls), + static_cast(summary.poll_calls), + static_cast(observed.poll_batch_records), + static_cast(summary.poll_batch_records), + static_cast(observed.dropped_records), + static_cast(summary.dropped_records) + ); + return fail_export(); } if (!first) out << "\n "; out << "]\n}\n"; + out.close(); + std::free(scratch); + scratch = nullptr; + if (!out) { + LOG_ERROR("failed while writing fdwic swimlane output %s", temporary_path.c_str()); + std::remove(temporary_path.c_str()); + return -1; + } + if (std::rename(temporary_path.c_str(), path.c_str()) != 0) { + LOG_ERROR("cannot finalize fdwic swimlane output %s: %s", path.c_str(), std::strerror(errno)); + std::remove(temporary_path.c_str()); + return -1; + } if (should_print_trace_export()) { - const std::string path = output_path(runtime); - LOG_INFO_V0("fdwic swimlane trace written to %s", path.c_str()); + LOG_INFO_V0( + "fdwic swimlane trace written to %s: records=%llu atomic=%llu clock=%llu calls=%llu poll_calls=%llu " + "poll_batches=%llu dropped=%llu", + path.c_str(), static_cast(summary.records), + static_cast(summary.atomic_records), + static_cast(summary.clock_baseline_records), + static_cast(summary.atomic_calls), static_cast(summary.poll_calls), + static_cast(summary.poll_batch_records), + static_cast(summary.dropped_records) + ); } return 0; } extern "C" void fdwic_swimlane_host_finalize(Runtime *runtime) { if (runtime == nullptr) return; - if (runtime->fdwic_swimlane_dev_base_ != 0 && runtime->host_api.device_free != nullptr) { - runtime->host_api.device_free(reinterpret_cast(runtime->fdwic_swimlane_dev_base_)); + if (runtime->fdwic_swimlane_dev_allocation_ != 0 && runtime->host_api.device_free != nullptr) { + runtime->host_api.device_free(reinterpret_cast(runtime->fdwic_swimlane_dev_allocation_)); } if (runtime->fdwic_swimlane_host_shadow_ != nullptr) { std::free(runtime->fdwic_swimlane_host_shadow_); } runtime->fdwic_swimlane_host_shadow_ = nullptr; + runtime->fdwic_swimlane_dev_allocation_ = 0; runtime->fdwic_swimlane_dev_base_ = 0; runtime->fdwic_swimlane_bytes_ = 0; - runtime->dist.swimlane_enabled = 0; + runtime->fdwic_swimlane_num_cores_ = 0; + runtime->fdwic_swimlane_records_per_core_ = 0; + runtime->dist.swimlane_level = 0; runtime->dist.swimlane_base = 0; + runtime->dist.swimlane_records_per_core = 0; } diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/api_glue.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/api_glue.h index e9605aa704..df64ef05d7 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/api_glue.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/api_glue.h @@ -18,7 +18,7 @@ DIST_API_ATTR PTO_DEVICE_FUNC bool dist_is_fatal_query() { #if defined(__CCE_AICORE__) return false; #else - return fatal_set(); + return fdwic_trace_is_fatal(); #endif } diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/core_main.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/core_main.h index eb0af23a47..1f06e09221 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/core_main.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/core_main.h @@ -19,19 +19,27 @@ DIST_API_ATTR PTO_DEVICE_FUNC void dist_core_main(__gm__ Runtime *runtime, int c fdwic_swimlane_attach(runtime); trace_reset_core(self); - if (!fatal_set()) { - atomic_fetch_add(g_dist.started_count, 1); + if (!fdwic_trace_is_fatal()) { + (void)fdwic_trace_atomic_fetch_add( + -1, FdwicAtomicSite::StartupIncrement, g_dist.started_count, 1, /*result_used=*/false + ); uint64_t wd_start = 0; - while (atomic_load(g_dist.started_count) < g_dist.num_workers && !fatal_set()) { + const uint32_t startup_poll_region = fdwic_atomic_poll_region_begin( + fdwic_atomic_site_mask(FdwicAtomicSite::StartupPoll) | fdwic_atomic_site_mask(FdwicAtomicSite::FatalPoll) + ); + while (fdwic_trace_atomic_load(-1, FdwicAtomicSite::StartupPoll, g_dist.started_count) < g_dist.num_workers && + !fdwic_trace_is_fatal()) { SPIN_WAIT_HINT(); watchdog(wd_start); } + fdwic_atomic_poll_region_end(startup_poll_region); } TRACE_LAP_RESET(self); // origin for the first lap span (post-barrier, pre-replay) dist_submit_replay_orch(runtime); dist_submit_drain_to_completion(self); + fdwic_swimlane_record_clock_baselines(self, core_idx); TRACE_FLUSH_CORE(self); dist_aicore_finish_worker(runtime); } diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_core.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_core.h index fa14f2bc49..0ab55676e9 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_core.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_core.h @@ -35,46 +35,76 @@ namespace { PTO_DEVICE_FUNC void publish_task_flag(int32_t task_id) { if (task_id < 0 || task_id >= kFlagCap) return; __gm__ DistTaskCell &cell = task_cell(task_id); - atomic_exchange(cell.flag, int64_t{1}, __ATOMIC_RELEASE); + (void)fdwic_trace_atomic_exchange( + task_id, FdwicAtomicSite::CompletionFlagExchange, cell.flag, int64_t{1}, /*result_used=*/false, __ATOMIC_RELEASE + ); } -PTO_DEVICE_FUNC bool task_flag_ready(int32_t task_id, int memorder) { +PTO_DEVICE_FUNC bool task_flag_ready(int32_t task_id, int memorder, FdwicAtomicSite site) { if (task_id < 0 || task_id >= kFlagCap) return false; __gm__ DistTaskCell &cell = task_cell(task_id); - return atomic_load(cell.flag, memorder) != 0; + return fdwic_trace_atomic_load(task_id, site, cell.flag, /*result_used=*/true, memorder) != 0; } PTO_DEVICE_FUNC void store_task_vend(int32_t task_id, uint64_t vend) { if (task_id < 0 || task_id >= kFlagCap) return; __gm__ DistTaskCell &cell = task_cell(task_id); - atomic_exchange(cell.vend, vend, __ATOMIC_RELAXED); + (void)fdwic_trace_atomic_exchange( + task_id, FdwicAtomicSite::CompletionVendExchange, cell.vend, vend, /*result_used=*/false, __ATOMIC_RELAXED + ); } -PTO_DEVICE_FUNC void store_won_remaining(__gm__ WonSlot &w, int32_t count) { - atomic_exchange(w.remaining.v, static_cast(count), __ATOMIC_RELAXED); +PTO_DEVICE_FUNC void store_won_remaining(__gm__ WonSlot &w, int32_t count, int32_t task_id) { + (void)fdwic_trace_atomic_exchange( + task_id, FdwicAtomicSite::WonRemainingExchange, w.remaining.v, static_cast(count), + /*result_used=*/false, __ATOMIC_RELAXED + ); } -PTO_DEVICE_FUNC void reset_won_lane(__gm__ WonSlot &w, int32_t lane) { - atomic_exchange(w.drained[lane].v, kDrainedClaimed); +PTO_DEVICE_FUNC void reset_won_lane(__gm__ WonSlot &w, int32_t lane, int32_t task_id) { + (void)fdwic_trace_atomic_exchange( + task_id, FdwicAtomicSite::WonLaneResetExchange, w.drained[lane].v, kDrainedClaimed, + /*result_used=*/false + ); w.lane[lane].present = false; } PTO_DEVICE_FUNC bool claim_won_lane(__gm__ WonSlot &w, int32_t lane) { - return atomic_exchange(w.drained[lane].v, kDrainedClaimed) == kDrainedFree; + return fdwic_trace_atomic_exchange( + -1, FdwicAtomicSite::WonLaneClaimExchange, w.drained[lane].v, kDrainedClaimed, + /*result_used=*/true + ) == kDrainedFree; } -PTO_DEVICE_FUNC void publish_won_slot(__gm__ WonSlot &w) { atomic_exchange(w.state.v, kWonStatePublished); } +PTO_DEVICE_FUNC void publish_won_slot(__gm__ WonSlot &w, int32_t task_id) { + (void)fdwic_trace_atomic_exchange( + task_id, FdwicAtomicSite::WonStatePublishExchange, w.state.v, kWonStatePublished, + /*result_used=*/false + ); +} -PTO_DEVICE_FUNC bool decrement_won_remaining_is_last(__gm__ WonSlot &w) { - return atomic_fetch_sub(w.remaining.v, 1) == 1; +PTO_DEVICE_FUNC bool decrement_won_remaining_is_last(__gm__ WonSlot &w, int32_t task_id) { + return fdwic_trace_atomic_fetch_sub( + task_id, FdwicAtomicSite::WonRemainingFetchSub, w.remaining.v, 1, /*result_used=*/true + ) == 1; } -PTO_DEVICE_FUNC void clear_won_slot_state(__gm__ WonSlot &w) { atomic_exchange(w.state.v, kWonStateFree); } +PTO_DEVICE_FUNC void clear_won_slot_state(__gm__ WonSlot &w, int32_t task_id) { + (void)fdwic_trace_atomic_exchange( + task_id, FdwicAtomicSite::WonStateClearExchange, w.state.v, kWonStateFree, /*result_used=*/false + ); +} -PTO_DEVICE_FUNC int64_t load_frontier_for_advance() { return atomic_load(g_dist.frontier); } +PTO_DEVICE_FUNC int64_t load_frontier_for_advance() { + // The first load reads global scan state, not a particular task cell. + return fdwic_trace_atomic_load(-1, FdwicAtomicSite::FrontierInitialLoad, g_dist.frontier); +} PTO_DEVICE_FUNC bool try_advance_frontier_to(int64_t &frontier, int64_t next) { - const int64_t old = atomic_fetch_max(g_dist.frontier, next); + // Pair this update with the immediately preceding next-task flag load. + const int64_t old = fdwic_trace_atomic_fetch_max( + static_cast(next), FdwicAtomicSite::FrontierMax, g_dist.frontier, next, /*result_used=*/true + ); frontier = old > next ? old : next; return next > old; } @@ -87,7 +117,7 @@ PTO_DEVICE_FUNC void advance_frontier() { while (true) { const int64_t next = f + 1; if (next >= kFlagCap) break; - if (!task_flag_ready(static_cast(next), __ATOMIC_ACQUIRE)) break; + if (!task_flag_ready(static_cast(next), __ATOMIC_ACQUIRE, FdwicAtomicSite::FrontierFlagLoad)) break; try_advance_frontier_to(f, next); } } @@ -138,8 +168,8 @@ PTO_DEVICE_FUNC void execute_slot([[maybe_unused]] __gm__ DistCore *self, __gm__ store_barrier(); if (s.is_multicore) { __gm__ WonSlot &w = g_dist.blocks[s.won_block].slots[s.won_slot]; - if (decrement_won_remaining_is_last(w)) { - clear_won_slot_state(w); + if (decrement_won_remaining_is_last(w, s.task_id)) { + clear_won_slot_state(w, s.task_id); complete_executed_task(self, s.task_id); } } else { @@ -158,7 +188,7 @@ PTO_DEVICE_FUNC int32_t drain_phase_b(__gm__ DistCore *self) { if (!s.occupied || !s.built) continue; bool ready = true; for (int32_t f = 0; f < s.fanin_count; f++) { - if (!task_flag_ready(s.fanin[f], __ATOMIC_ACQUIRE)) { + if (!task_flag_ready(s.fanin[f], __ATOMIC_ACQUIRE, FdwicAtomicSite::FaninFlagLoad)) { ready = false; break; } @@ -240,11 +270,11 @@ PTO_DEVICE_FUNC bool drain_block_won(__gm__ DistCore *self) { if (!g_fdwic_joint_submit_seen) return false; if (self == nullptr || self->lane == LANE_AIC || self->lane == LANE_NONE) return false; __gm__ BlockWon &bw = g_dist.blocks[self->block_id]; - if (atomic_load(bw.any_pub) == 0) return false; + if (fdwic_trace_atomic_load(-1, FdwicAtomicSite::WonAnyLoad, bw.any_pub) == 0) return false; bool drained = false; for (int32_t i = 0; i < kPrivateSlots; i++) { __gm__ WonSlot &w = bw.slots[i]; - if (atomic_load(w.state.v) != kWonStatePublished) continue; + if (fdwic_trace_atomic_load(-1, FdwicAtomicSite::WonStateLoad, w.state.v) != kWonStatePublished) continue; #if defined(__CCE_AICORE__) dist_aicore_invalidate_region(&w.lane[self->lane].present, sizeof(w.lane[self->lane].present)); #endif @@ -252,7 +282,10 @@ PTO_DEVICE_FUNC bool drain_block_won(__gm__ DistCore *self) { if (!claim_won_lane(w, self->lane)) continue; int32_t si = alloc_ring_slot(self); if (si < 0) { - atomic_exchange(w.drained[self->lane].v, kDrainedFree); + (void)fdwic_trace_atomic_exchange( + -1, FdwicAtomicSite::WonLaneReleaseExchange, w.drained[self->lane].v, kDrainedFree, + /*result_used=*/false + ); return drained; } #if defined(__CCE_AICORE__) @@ -280,22 +313,23 @@ PTO_DEVICE_FUNC bool has_pending_won(__gm__ DistCore *self) { if (!g_fdwic_joint_submit_seen) return false; if (self == nullptr || self->lane == LANE_AIC || self->lane == LANE_NONE) return false; __gm__ BlockWon &bw = g_dist.blocks[self->block_id]; - if (atomic_load(bw.any_pub) == 0) return false; + if (fdwic_trace_atomic_load(-1, FdwicAtomicSite::WonAnyLoad, bw.any_pub) == 0) return false; for (int32_t i = 0; i < kPrivateSlots; i++) { __gm__ WonSlot &w = bw.slots[i]; - if (atomic_load(w.state.v) != kWonStatePublished) continue; + if (fdwic_trace_atomic_load(-1, FdwicAtomicSite::WonStateLoad, w.state.v) != kWonStatePublished) continue; #if defined(__CCE_AICORE__) dist_aicore_invalidate_region(&w.lane[self->lane].present, sizeof(w.lane[self->lane].present)); #endif if (!w.lane[self->lane].present) continue; - if (atomic_load(w.drained[self->lane].v) == kDrainedFree) return true; + if (fdwic_trace_atomic_load(-1, FdwicAtomicSite::WonDrainedLoad, w.drained[self->lane].v) == kDrainedFree) + return true; } return false; } PTO_DEVICE_FUNC void dist_submit_execute_first(__gm__ DistCore *self) { TRACE_LAP_RESET(self); - if (!fatal_set()) { + if (!fdwic_trace_is_fatal()) { drain_block_won(self); drain_phase_b(self); } @@ -325,6 +359,7 @@ struct DistSubmitCtx { int32_t joint_block; int32_t joint_slot; int32_t joint_count; + bool claim_attempted; }; PTO_DEVICE_FUNC void dist_submit_begin(__gm__ DistCore *self, const L0TaskArgs &args, DistSubmitCtx &ctx) { @@ -349,11 +384,12 @@ PTO_DEVICE_FUNC void dist_submit_begin(__gm__ DistCore *self, const L0TaskArgs & ctx.joint_block = -1; ctx.joint_slot = -1; ctx.joint_count = 0; + ctx.claim_attempted = false; } PTO_DEVICE_FUNC bool dist_submit_check_task_cap(const DistSubmitCtx &ctx, DistSubmitKind kind) { if (ctx.task_id < kFlagCap) return true; - set_fatal(); + fdwic_trace_set_fatal(ctx.task_id); if (kind == DistSubmitKind::Alloc) { DIST_ERRF("[dist_engine] alloc task id %d exceeds kFlagCap %d\n", ctx.task_id, kFlagCap); } else { @@ -393,7 +429,7 @@ PTO_DEVICE_FUNC bool dist_submit_materialize_args(const L0TaskArgs &args, DistSu uint64_t task_base = PTO2_ALIGN_UP(ctx.self->heap_next, PTO2_PACKED_OUTPUT_ALIGN); if (total > 0 && g_dist.heap_base != nullptr) { if (total > ring) { - set_fatal(); + fdwic_trace_set_fatal(ctx.task_id); if (kind == DistSubmitKind::Alloc) { DIST_ERRF( "[dist_engine] alloc task %d outputs %llu B exceed heap ring %zu B\n", ctx.task_id, @@ -417,7 +453,7 @@ PTO_DEVICE_FUNC bool dist_submit_materialize_args(const L0TaskArgs &args, DistSu if ((output_mask & 1u) == 0) continue; const auto &ci = args.tensor(i).create_info(); if (g_dist.heap_base == nullptr) { - set_fatal(); + fdwic_trace_set_fatal(ctx.task_id); if (kind == DistSubmitKind::Alloc) { DIST_ERRF("[dist_engine] GM output heap not allocated at alloc %d\n", ctx.task_id); } else { @@ -573,7 +609,7 @@ PTO_DEVICE_FUNC bool dist_submit_materialize_and_prepare_map( if (!dist_submit_materialize_args(args, ctx, kind)) return false; TRACE_SPAN_END(materialize_trace, self, ctx.task_id, -1, TracePhase::Materialize, 0, static_cast(kind)); #if !defined(__CCE_AICORE__) - if (fatal_set()) return false; + if (fdwic_trace_is_fatal(ctx.task_id)) return false; #endif TRACE_SPAN_BEGIN(prepare_map_trace); dist_submit_prepare_map(self, ctx.task_id); diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_helpers.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_helpers.h index b454b377fd..6f7f24b9b6 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_helpers.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_helpers.h @@ -17,14 +17,18 @@ namespace { PTO_DEVICE_FUNC bool claim(__gm__ volatile int64_t &cursor, int32_t N) { if (N < 0 || N >= kFlagCap) return false; - const int64_t old = atomic_fetch_max(cursor, static_cast(N)); + const int64_t old = fdwic_trace_atomic_fetch_max( + N, FdwicAtomicSite::ClaimMax, cursor, static_cast(N), /*result_used=*/true + ); return N > old; } -PTO_DEVICE_FUNC uint64_t load_task_vend(int32_t task_id) { - if (task_id < 0) return 0; - __gm__ DistTaskCell &cell = task_cell(task_id); - return atomic_load(cell.vend, __ATOMIC_RELAXED); +PTO_DEVICE_FUNC uint64_t load_task_vend(int32_t current_task_id, int32_t vend_task_id) { + if (vend_task_id < 0) return 0; + __gm__ DistTaskCell &cell = task_cell(vend_task_id); + return fdwic_trace_atomic_load( + current_task_id, FdwicAtomicSite::HeapVendLoad, cell.vend, /*result_used=*/true, __ATOMIC_RELAXED + ); } // Resolve a kernel id to its executable address (CoreCallable::resolved_addr()). @@ -47,38 +51,45 @@ PTO_DEVICE_FUNC void populate_won_slot_from_submit( Runtime *runtime, const L0TaskArgs &args, const DistSubmitCtx &ctx, FaninArrPtr fanin, int32_t fc ) { w.meta.task_id = task_id; -#define POPULATE_WON_LANE_FROM_SUBMIT(L) \ - do { \ - if ((L) == own_lane || !lane_active(M, (L))) break; \ - __gm__ BuiltSubtask &b = w.lane[(L)]; \ - b.present = true; \ - b.func_id = kernel_id_for_lane(mixed, (L)); \ - b.function_bin_addr = runtime != nullptr ? resolve_kernel_addr(runtime, b.func_id) : 0; \ - b.tensor_count = ctx.tensor_count; \ - b.scalar_count = ctx.scalar_count; \ - for (int32_t i = 0; i < ctx.tensor_count; i++) \ - dist_submit_copy_arg_tensor(b.tensors[i], args, ctx, i); \ - for (int32_t j = 0; j < ctx.scalar_count; j++) \ - b.scalars[j] = args.scalar(j); \ - b.fanin_count = fc; \ - for (int32_t k = 0; k < fc; k++) \ - b.fanin[k] = fanin[k]; \ - b.sub_block_id = ((L) == LANE_AIV1) ? 1 : 0; \ - atomic_exchange(w.drained[(L)].v, kDrainedFree); \ +#define POPULATE_WON_LANE_FROM_SUBMIT(L) \ + do { \ + if ((L) == own_lane || !lane_active(M, (L))) break; \ + __gm__ BuiltSubtask &b = w.lane[(L)]; \ + b.present = true; \ + b.func_id = kernel_id_for_lane(mixed, (L)); \ + b.function_bin_addr = runtime != nullptr ? resolve_kernel_addr(runtime, b.func_id) : 0; \ + b.tensor_count = ctx.tensor_count; \ + b.scalar_count = ctx.scalar_count; \ + for (int32_t i = 0; i < ctx.tensor_count; i++) \ + dist_submit_copy_arg_tensor(b.tensors[i], args, ctx, i); \ + for (int32_t j = 0; j < ctx.scalar_count; j++) \ + b.scalars[j] = args.scalar(j); \ + b.fanin_count = fc; \ + for (int32_t k = 0; k < fc; k++) \ + b.fanin[k] = fanin[k]; \ + b.sub_block_id = ((L) == LANE_AIV1) ? 1 : 0; \ + (void)fdwic_trace_atomic_exchange( \ + task_id, FdwicAtomicSite::WonLaneDepositExchange, w.drained[(L)].v, kDrainedFree, /*result_used=*/false \ + ); \ } while (0) - reset_won_lane(w, LANE_AIC); - reset_won_lane(w, LANE_AIV0); - reset_won_lane(w, LANE_AIV1); + reset_won_lane(w, LANE_AIC, task_id); + reset_won_lane(w, LANE_AIV0, task_id); + reset_won_lane(w, LANE_AIV1, task_id); POPULATE_WON_LANE_FROM_SUBMIT(LANE_AIC); POPULATE_WON_LANE_FROM_SUBMIT(LANE_AIV0); POPULATE_WON_LANE_FROM_SUBMIT(LANE_AIV1); #undef POPULATE_WON_LANE_FROM_SUBMIT } -PTO_DEVICE_FUNC int32_t alloc_won_slot(int32_t block) { +PTO_DEVICE_FUNC int32_t alloc_won_slot(int32_t block, int32_t task_id) { __gm__ BlockWon &bw = g_dist.blocks[block]; for (int32_t i = 0; i < kPrivateSlots; i++) { - if (atomic_fetch_max(bw.slots[i].state.v, kWonStateClaimed) == kWonStateFree) return i; + if (fdwic_trace_atomic_fetch_max( + task_id, FdwicAtomicSite::WonSlotClaimMax, bw.slots[i].state.v, kWonStateClaimed, + /*result_used=*/true + ) == kWonStateFree) { + return i; + } } return -1; } diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_runtime.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_runtime.h index 56e46aae7b..37ae6432d0 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_runtime.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/submit_runtime.h @@ -42,6 +42,7 @@ PTO_DEVICE_FUNC bool dist_submit_claim_kernel(const MixedKernels &mixed, DistSub const int32_t anchor_lane = anchor_lane_for_mask(M); if (!dist_submit_self_is_lane(ctx.self, block, anchor_lane)) return false; __gm__ PaddedCursor *cursors = anchor_lane == LANE_AIC ? g_dist.cube_cursor : g_dist.vector_cursor; + ctx.claim_attempted = true; ctx.won = claim(cursors[ctx.task_id % kCursorShards].v, ctx.task_id); if (!ctx.won) return false; ctx.kernel_id = kernel_id_for_lane(mixed, anchor_lane); @@ -50,6 +51,7 @@ PTO_DEVICE_FUNC bool dist_submit_claim_kernel(const MixedKernels &mixed, DistSub } if (lane_active(M, LANE_AIC)) { if (ctx.self->role != CoreType::AIC) return false; + ctx.claim_attempted = true; ctx.won = claim(g_dist.cube_cursor[ctx.task_id % kCursorShards].v, ctx.task_id); if (!ctx.won) return false; ctx.kernel_id = mixed.aic_kernel_id; @@ -57,6 +59,7 @@ PTO_DEVICE_FUNC bool dist_submit_claim_kernel(const MixedKernels &mixed, DistSub } if (lane_active(M, LANE_AIV0) || lane_active(M, LANE_AIV1)) { if (ctx.self->role != CoreType::AIV) return false; + ctx.claim_attempted = true; ctx.won = claim(g_dist.vector_cursor[ctx.task_id % kCursorShards].v, ctx.task_id); if (!ctx.won) return false; const int32_t own_lane = lane_active(M, LANE_AIV0) ? LANE_AIV0 : LANE_AIV1; @@ -69,6 +72,7 @@ PTO_DEVICE_FUNC bool dist_submit_claim_kernel(const MixedKernels &mixed, DistSub PTO_DEVICE_FUNC bool dist_submit_claim_alloc(DistSubmitCtx &ctx) { ctx.kernel_id = INVALID_KERNEL_ID; if (ctx.self == nullptr || ctx.task_id < 0 || ctx.task_id >= kFlagCap) return false; + ctx.claim_attempted = true; ctx.won = claim(g_dist.alloc_cursor[ctx.task_id % kCursorShards].v, ctx.task_id); return ctx.won; } @@ -94,11 +98,15 @@ PTO_DEVICE_FUNC void dist_submit_wait_slot_capacity(__gm__ DistCore *self, int32 if (self == nullptr) return; bool waited = false; TRACE_SPAN_BEGIN(ring_bp_trace); + const uint32_t slot_poll_region = fdwic_atomic_poll_region_begin( + fdwic_atomic_site_mask(FdwicAtomicSite::FaninFlagLoad) | fdwic_atomic_block_won_poll_mask() + ); while (self->occupied_count >= kPrivateSlots - kWonReserve) { waited = true; drain_block_won(self); if (drain_phase_b(self) == 0) SPIN_WAIT_HINT(); } + fdwic_atomic_poll_region_end(slot_poll_region); if (waited) { TRACE_SPAN_END(ring_bp_trace, self, task_id, -1, TracePhase::RingBp, 0, 0); } @@ -109,21 +117,38 @@ PTO_DEVICE_FUNC bool dist_submit_wait_heap_capacity(DistSubmitCtx &ctx, DistSubm const size_t ring = g_dist.heap_size; bool waited = false; TRACE_SPAN_BEGIN(heap_bp_trace); - while (!fatal_set()) { + bool heap_poll_region_active = false; + uint32_t heap_poll_region = 0; + while (!fdwic_trace_is_fatal(ctx.task_id)) { // 逻辑 heap 尚未走完第一圈时,物理地址还没有发生环形复用;保留 fatal // 原子检查后,可直接跳过 frontier/vend 原子读取。 - if (ctx.self->heap_next <= ring) return true; - const int32_t f = static_cast(atomic_load(g_dist.frontier)); + if (ctx.self->heap_next <= ring) { + if (heap_poll_region_active) fdwic_atomic_poll_region_end(heap_poll_region); + return true; + } + if (!heap_poll_region_active) { + heap_poll_region = fdwic_atomic_poll_region_begin( + fdwic_atomic_site_mask(FdwicAtomicSite::FatalPoll) | + fdwic_atomic_site_mask(FdwicAtomicSite::HeapFrontierLoad) | + fdwic_atomic_site_mask(FdwicAtomicSite::HeapVendLoad) | + fdwic_atomic_site_mask(FdwicAtomicSite::FaninFlagLoad) | fdwic_atomic_block_won_poll_mask() + ); + heap_poll_region_active = true; + } + const int32_t f = static_cast(fdwic_trace_atomic_load( + ctx.task_id, FdwicAtomicSite::HeapFrontierLoad, g_dist.frontier, /*result_used=*/true + )); const int32_t R = f - g_dist.H; - const uint64_t vstart_live = load_task_vend(R); + const uint64_t vstart_live = load_task_vend(ctx.task_id, R); if (ctx.self->heap_next - vstart_live <= ring) { + fdwic_atomic_poll_region_end(heap_poll_region); if (waited) { TRACE_SPAN_END(heap_bp_trace, ctx.self, ctx.task_id, -1, TracePhase::RingBp, 0, 1); } return true; } if (f >= ctx.task_id - 1) { - set_fatal(); + fdwic_trace_set_fatal(ctx.task_id); if (kind == DistSubmitKind::Alloc) { DIST_ERRF( "[dist_engine] heap ring %zu B too small for H=%d window at alloc %d (live=%llu B)\n", ring, @@ -136,12 +161,14 @@ PTO_DEVICE_FUNC bool dist_submit_wait_heap_capacity(DistSubmitCtx &ctx, DistSubm ring, g_dist.H, ctx.task_id, (unsigned long long)(ctx.self->heap_next - vstart_live) ); } + fdwic_atomic_poll_region_end(heap_poll_region); return false; } waited = true; drain_block_won(ctx.self); if (drain_phase_b(ctx.self) == 0) SPIN_WAIT_HINT(); } + if (heap_poll_region_active) fdwic_atomic_poll_region_end(heap_poll_region); if (waited) { TRACE_SPAN_END(heap_bp_trace, ctx.self, ctx.task_id, -1, TracePhase::RingBp, 0, 1); } @@ -165,18 +192,26 @@ PTO_DEVICE_FUNC void publish_joint_deposits(DistSubmitCtx &ctx, const MixedKerne dist_aicore_flush_region(&w.meta, sizeof(w.meta)); dist_aicore_flush_region(w.lane, sizeof(w.lane)); #endif - store_won_remaining(w, ctx.joint_count); - publish_won_slot(w); - atomic_exchange(g_dist.blocks[ctx.joint_block].any_pub, 1); + store_won_remaining(w, ctx.joint_count, ctx.task_id); + publish_won_slot(w, ctx.task_id); + (void)fdwic_trace_atomic_exchange( + ctx.task_id, FdwicAtomicSite::WonAnyPublishExchange, g_dist.blocks[ctx.joint_block].any_pub, int32_t{1}, + /*result_used=*/false + ); } -PTO_DEVICE_FUNC int32_t wait_alloc_won_slot(__gm__ DistCore *self, int32_t block) { - int32_t won_slot = alloc_won_slot(block); - while (won_slot < 0 && !fatal_set()) { +PTO_DEVICE_FUNC int32_t wait_alloc_won_slot(__gm__ DistCore *self, int32_t block, int32_t task_id) { + int32_t won_slot = alloc_won_slot(block, task_id); + const uint32_t won_slot_poll_region = fdwic_atomic_poll_region_begin( + fdwic_atomic_site_mask(FdwicAtomicSite::FatalPoll) | fdwic_atomic_site_mask(FdwicAtomicSite::FaninFlagLoad) | + fdwic_atomic_block_won_poll_mask() + ); + while (won_slot < 0 && !fdwic_trace_is_fatal(task_id)) { drain_block_won(self); if (drain_phase_b(self) == 0) SPIN_WAIT_HINT(); - won_slot = alloc_won_slot(block); + won_slot = alloc_won_slot(block, task_id); } + fdwic_atomic_poll_region_end(won_slot_poll_region); return won_slot; } @@ -197,7 +232,7 @@ dist_submit_build_winner_task(DistSubmitCtx &ctx, const MixedKernels &mixed, con dist_submit_wait_slot_capacity(ctx.self, ctx.task_id); if (!dist_submit_wait_heap_capacity(ctx, DistSubmitKind::Kernel)) return; if (ctx.joint && ctx.joint_slot < 0) { - ctx.joint_slot = wait_alloc_won_slot(ctx.self, ctx.joint_block); + ctx.joint_slot = wait_alloc_won_slot(ctx.self, ctx.joint_block, ctx.task_id); if (ctx.joint_slot < 0) return; } __gm__ RingSlot *slot = dist_submit_alloc_slot(ctx.self); @@ -218,16 +253,24 @@ PTO_DEVICE_FUNC void dist_submit_complete_alloc(DistSubmitCtx &ctx) { PTO_DEVICE_FUNC void dist_submit_drain_to_completion(__gm__ DistCore *self) { if (self == nullptr) return; - atomic_fetch_add(g_dist.replay_done, 1); + (void)fdwic_trace_atomic_fetch_add( + -1, FdwicAtomicSite::ReplayDoneIncrement, g_dist.replay_done, 1, /*result_used=*/false + ); + const uint32_t final_poll_region = fdwic_atomic_poll_region_begin( + fdwic_atomic_site_mask(FdwicAtomicSite::ReplayDonePoll) | + fdwic_atomic_site_mask(FdwicAtomicSite::FaninFlagLoad) | fdwic_atomic_block_won_poll_mask() + ); while (true) { drain_block_won(self); const int32_t freed = drain_phase_b(self); - const bool all_replayed = atomic_load(g_dist.replay_done) >= g_dist.num_workers; + const bool all_replayed = + fdwic_trace_atomic_load(-1, FdwicAtomicSite::ReplayDonePoll, g_dist.replay_done) >= g_dist.num_workers; const bool ring_empty = self->occupied_count == 0; const bool pending = has_pending_won(self); if (all_replayed && ring_empty && !pending) break; if (freed == 0) SPIN_WAIT_HINT(); } + fdwic_atomic_poll_region_end(final_poll_region); } PTO_DEVICE_FUNC void dist_submit_replay_orch(__gm__ Runtime *runtime) { @@ -255,7 +298,7 @@ PTO_DEVICE_FUNC void dist_submit_replay_orch(__gm__ Runtime *runtime) { aicpu_orchestration_entry(local_args); #else (void)runtime; - if (g_dist.orch_args != nullptr && !fatal_set()) { + if (g_dist.orch_args != nullptr && !fdwic_trace_is_fatal()) { aicpu_orchestration_entry(*g_dist.orch_args); } #endif @@ -279,9 +322,11 @@ dist_submit_impl(PTO2Runtime *, const MixedKernels &mixed, const L0TaskArgs &arg if (!dist_submit_materialize_and_prepare_map(ctx.self, args, ctx, DistSubmitKind::Kernel)) return ctx.result; TRACE_SPAN_BEGIN(claim_trace); const bool is_winner = dist_submit_claim(DistSubmitKind::Kernel, &mixed, ctx); - TRACE_SPAN_END( - claim_trace, ctx.self, ctx.task_id, ctx.kernel_id, TracePhase::Claim, static_cast(is_winner), 0 - ); + const uint32_t claim_flags = + fdwic_atomic_swimlane_enabled() ? + (is_winner ? kFdwicClaimWon : 0U) | (ctx.claim_attempted ? kFdwicClaimAttempted : 0U) : + static_cast(is_winner); + TRACE_SPAN_END(claim_trace, ctx.self, ctx.task_id, ctx.kernel_id, TracePhase::Claim, claim_flags, 0); if (is_winner) { TRACE_SPAN_BEGIN(fanin_trace); ctx.fanin_count = dist_submit_collect_fanin(args, ctx, ctx.fanin); @@ -320,7 +365,11 @@ DIST_API_ATTR PTO_DEVICE_FUNC TaskOutputTensors dist_alloc_tensors(PTO2Runtime * TRACE_SPAN_END(register_trace, ctx.self, ctx.task_id, -1, TracePhase::Register, 0, 0); TRACE_SPAN_BEGIN(claim_trace); const bool is_winner = dist_submit_claim(DistSubmitKind::Alloc, nullptr, ctx); - TRACE_SPAN_END(claim_trace, ctx.self, ctx.task_id, -1, TracePhase::Claim, static_cast(is_winner), 1); + const uint32_t claim_flags = + fdwic_atomic_swimlane_enabled() ? + (is_winner ? kFdwicClaimWon : 0U) | (ctx.claim_attempted ? kFdwicClaimAttempted : 0U) : + static_cast(is_winner); + TRACE_SPAN_END(claim_trace, ctx.self, ctx.task_id, -1, TracePhase::Claim, claim_flags, 1); if (is_winner) { dist_submit_complete_alloc(ctx); TRACE_LAP(ctx.self, ctx.task_id, -1, TracePhase::Alloc); diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/tensor_data_access.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/tensor_data_access.h index 403bb43d8b..a04505cca6 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/tensor_data_access.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/aicore/tensor_data_access.h @@ -22,14 +22,19 @@ PTO_DEVICE_FUNC void wait_producer_ready(DistCore *self, const Tensor &t) { const int32_t p = dist_tensor_map_lookup(self->map, t); if (p < 0) return; uint64_t wd = 0; - while (!fatal_set()) { - if (task_flag_ready(p, __ATOMIC_ACQUIRE)) break; + const uint32_t producer_poll_region = fdwic_atomic_poll_region_begin( + fdwic_atomic_site_mask(FdwicAtomicSite::FatalPoll) | fdwic_atomic_site_mask(FdwicAtomicSite::FaninFlagLoad) | + fdwic_atomic_block_won_poll_mask() + ); + while (!fdwic_trace_is_fatal(p)) { + if (task_flag_ready(p, __ATOMIC_ACQUIRE, FdwicAtomicSite::FaninFlagLoad)) break; drain_block_won(self); if (drain_phase_b(self) == 0) { SPIN_WAIT_HINT(); watchdog(wd); } } + fdwic_atomic_poll_region_end(producer_poll_region); } #endif diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/state.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/state.h index b6734ef7b7..05a942acee 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/state.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/state.h @@ -97,6 +97,8 @@ enum class TracePhase : int32_t { Claim = 11, Fanin = 12, Register = 13, + Atomic = 14, + ClockBaseline = 15, }; struct RingSlot { diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/swimlane.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/swimlane.h index 2a8ba2266d..94667957a2 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/swimlane.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/swimlane.h @@ -31,24 +31,68 @@ PTO_DEVICE_FUNC inline uint64_t fdwic_swimlane_detail_now() { #endif } -PTO_DEVICE_FUNC inline bool fdwic_swimlane_enabled() { return g_fdwic_swimlane_enabled; } +PTO_DEVICE_FUNC inline bool fdwic_swimlane_enabled() { return g_fdwic_swimlane_level != 0; } + +PTO_DEVICE_FUNC inline bool fdwic_atomic_swimlane_enabled() { + return g_fdwic_swimlane_level >= kFdwicAtomicSwimlaneLevel; +} + +PTO_DEVICE_FUNC inline bool fdwic_atomic_return_ready_observed() { +#if defined(__CCE_AICORE__) + return true; +#else + // CPU simulation validates the scheduler and raw schema, not A5 atomic + // completion timing. Do not claim a hardware return-ready boundary there. + return false; +#endif +} + +template +PTO_DEVICE_FUNC inline uint64_t fdwic_swimlane_detail_now_after_atomic_result(T value) { +#if defined(__CCE_AICORE__) + static_assert(sizeof(T) == 4 || sizeof(T) == 8, "atomic dependency expects a scalar result"); + uint64_t cycle = 0; + // Consume the returned scalar in the same asm block as SYS_CNT. This is a + // local return-value-ready boundary, not a cross-core visibility fence. + asm volatile("MOV %0, %0\n" + "MOV %1, SYS_CNT\n" + : "+l"(value), "=&l"(cycle)); + return cycle; +#else + (void)value; + return fdwic_swimlane_detail_now(); +#endif +} PTO_DEVICE_FUNC inline void fdwic_swimlane_attach(__gm__ Runtime *runtime) { - g_fdwic_swimlane_enabled = false; + g_fdwic_swimlane_level = 0; g_fdwic_swimlane_header = nullptr; g_fdwic_swimlane_core = nullptr; g_fdwic_swimlane_records = nullptr; g_fdwic_swimlane_records_per_core = 0; + g_fdwic_atomic_poll_burst.active_mask = 0; + g_fdwic_atomic_poll_burst.enabled_mask = 0; + g_fdwic_atomic_calls = 0; + g_fdwic_poll_calls = 0; + g_fdwic_poll_batch_records = 0; + g_fdwic_atomic_counter_overflow = false; if (runtime == nullptr) return; #if defined(__CCE_AICORE__) dist_aicore_invalidate_region(const_cast<__gm__ uint64_t *>(&runtime->dist.swimlane_base), 64); #endif const uint64_t base = runtime->dist.swimlane_base; + const uint32_t level = runtime->dist.swimlane_level; const uint32_t records_per_core = runtime->dist.swimlane_records_per_core; - if (runtime->dist.swimlane_enabled == 0 || base == 0 || records_per_core == 0) return; + if (level == 0 || base == 0 || records_per_core == 0) return; g_fdwic_swimlane_header = reinterpret_cast<__gm__ FdwicSwimlaneHeader *>(base); +#if defined(__CCE_AICORE__) + // The host initializes this cache line before launching the kernel. Drop a + // possibly stale line left by a previous allocation at the same GM address. + // Do not invalidate the whole header: other cores update their own states. + dist_aicore_invalidate_region(g_fdwic_swimlane_header, 64); +#endif g_fdwic_swimlane_records_per_core = records_per_core; - g_fdwic_swimlane_enabled = true; + g_fdwic_swimlane_level = level; } PTO_DEVICE_FUNC inline __gm__ FdwicSwimlaneRecord *fdwic_swimlane_detail_records(__gm__ FdwicSwimlaneHeader *header) { @@ -63,62 +107,341 @@ PTO_DEVICE_FUNC inline void fdwic_swimlane_reset_core(__gm__ DistCore *self) { if (header == nullptr) return; if (self->core_idx < 0 || self->core_idx >= static_cast(header->num_cores)) return; g_fdwic_swimlane_core = &header->cores[self->core_idx]; - g_fdwic_swimlane_records = &fdwic_swimlane_detail_records( - header - )[static_cast(self->core_idx) * g_fdwic_swimlane_records_per_core]; + __gm__ FdwicSwimlaneRecord *records = fdwic_swimlane_detail_records(header); + const uint64_t record_offset = static_cast(self->core_idx) * g_fdwic_swimlane_records_per_core; + g_fdwic_swimlane_records = records + record_offset; g_fdwic_swimlane_core->count = 0; g_fdwic_swimlane_core->dropped = 0; + g_fdwic_swimlane_core->atomic_calls = 0; + g_fdwic_swimlane_core->poll_calls = 0; + g_fdwic_swimlane_core->poll_batch_records = 0; + g_fdwic_swimlane_core->core_idx = self->core_idx; + g_fdwic_swimlane_core->block_id = self->block_id; + g_fdwic_swimlane_core->lane = self->lane; + g_fdwic_atomic_poll_burst.active_mask = 0; + g_fdwic_atomic_poll_burst.enabled_mask = 0; + g_fdwic_atomic_calls = 0; + g_fdwic_poll_calls = 0; + g_fdwic_poll_batch_records = 0; + g_fdwic_atomic_counter_overflow = false; } -PTO_DEVICE_FUNC inline void fdwic_swimlane_flush_core(__gm__ DistCore *self) { - if (!fdwic_swimlane_enabled() || self == nullptr) return; +PTO_DEVICE_FUNC inline bool fdwic_swimlane_detail_write_record( + __gm__ DistCore *self, int32_t task_id, int32_t func_id, FdwicSwimlanePhase phase, uint64_t start_cycle, + uint64_t end_cycle, uint32_t flags, uint32_t aux +) { + if (!fdwic_swimlane_enabled() || self == nullptr) return false; const uint32_t records_per_core = g_fdwic_swimlane_records_per_core; __gm__ FdwicSwimlaneCoreState *core = g_fdwic_swimlane_core; - if (core == nullptr || g_fdwic_swimlane_records == nullptr || records_per_core == 0) return; - const uint32_t count = core->count < records_per_core ? core->count : records_per_core; - if (count > 0) { - dist_aicore_flush_region(g_fdwic_swimlane_records, static_cast(count) * sizeof(FdwicSwimlaneRecord)); + if (core == nullptr || g_fdwic_swimlane_records == nullptr || records_per_core == 0) return false; + const uint32_t slot = core->count; + if (slot >= records_per_core) { + if (core->dropped != UINT32_MAX) core->dropped = core->dropped + 1; + return false; } - dist_aicore_flush_region(core, sizeof(FdwicSwimlaneCoreState)); + __gm__ FdwicSwimlaneRecord *record = &g_fdwic_swimlane_records[slot]; + record->start_cycle = start_cycle; + record->end_cycle = end_cycle; + record->task_id = task_id; + record->func_id = func_id; + record->flags = flags; + record->phase = static_cast(phase); + record->aux = static_cast(aux); + core->count = slot + 1; + return true; } PTO_DEVICE_FUNC inline void fdwic_swimlane_detail_record( __gm__ DistCore *self, int32_t task_id, int32_t func_id, FdwicSwimlanePhase phase, uint64_t start_cycle, uint64_t end_cycle, uint32_t flags = 0, uint32_t aux = 0 ) { + (void)fdwic_swimlane_detail_write_record(self, task_id, func_id, phase, start_cycle, end_cycle, flags, aux); +} + +PTO_DEVICE_FUNC inline uint32_t fdwic_atomic_trace_flags( + FdwicAtomicOp op, bool result_used, bool return_ready, bool value_zero = false, uint64_t retries = 0 +) { + constexpr uint64_t kMaxRetries = (1ULL << (32 - kFdwicAtomicRetriesShift)) - 1; + const uint32_t encoded_retries = static_cast(retries > kMaxRetries ? kMaxRetries : retries); + return static_cast(op) | (result_used ? kFdwicAtomicResultUsed : 0U) | + (value_zero ? kFdwicAtomicValueZero : 0U) | (return_ready ? kFdwicAtomicReturnReady : 0U) | + (encoded_retries << kFdwicAtomicRetriesShift); +} + +PTO_DEVICE_FUNC inline uint32_t fdwic_atomic_poll_trace_flags(FdwicAtomicSite site, uint32_t call_count) { + return static_cast(fdwic_atomic_site_op(site)) | kFdwicAtomicResultUsed | kFdwicAtomicPollBatch | + (call_count << kFdwicAtomicPollCountShift); +} + +PTO_DEVICE_FUNC inline uint32_t fdwic_atomic_site_mask(FdwicAtomicSite site) { + return 1U << static_cast(site); +} + +PTO_DEVICE_FUNC inline uint32_t fdwic_atomic_block_won_poll_mask() { + return fdwic_atomic_site_mask(FdwicAtomicSite::WonAnyLoad) | fdwic_atomic_site_mask(FdwicAtomicSite::WonStateLoad) | + fdwic_atomic_site_mask(FdwicAtomicSite::WonLaneClaimExchange) | + fdwic_atomic_site_mask(FdwicAtomicSite::WonDrainedLoad); +} + +PTO_DEVICE_FUNC inline void fdwic_swimlane_count_atomic_call(bool poll_batch) { + if (g_fdwic_atomic_calls == UINT32_MAX) { + g_fdwic_atomic_counter_overflow = true; + return; + } + g_fdwic_atomic_calls++; + if (!poll_batch) return; + if (g_fdwic_poll_calls == UINT32_MAX) { + g_fdwic_atomic_counter_overflow = true; + return; + } + g_fdwic_poll_calls++; +} + +PTO_DEVICE_FUNC inline void fdwic_atomic_poll_boundary_at(uint64_t end_cycle) { + if (!fdwic_atomic_swimlane_enabled() || g_fdwic_atomic_poll_burst.active_mask == 0) return; + __gm__ DistCore *self = g_self; + if (self == nullptr || g_fdwic_swimlane_core == nullptr) return; + const uint32_t active_mask = g_fdwic_atomic_poll_burst.active_mask; + for (uint32_t batch_index = 0; batch_index < kFdwicAtomicPollBatchSiteCount; ++batch_index) { + const uint32_t bit = 1U << batch_index; + if ((active_mask & bit) == 0) continue; + const uint32_t call_count = g_fdwic_atomic_poll_burst.call_count[batch_index]; + if (call_count == 0 || call_count > kFdwicAtomicPollCountMax) { + g_fdwic_atomic_counter_overflow = true; + continue; + } + const FdwicAtomicSite site = fdwic_atomic_poll_batch_site(batch_index); + const uint32_t site_index = static_cast(site); + const bool written = fdwic_swimlane_detail_write_record( + self, -1, -1, FdwicSwimlanePhase::Atomic, g_fdwic_atomic_poll_burst.start_cycle[batch_index], end_cycle, + fdwic_atomic_poll_trace_flags(site, call_count), site_index + ); + if (written) { + if (g_fdwic_poll_batch_records == UINT32_MAX) { + g_fdwic_atomic_counter_overflow = true; + } else { + g_fdwic_poll_batch_records++; + } + } + g_fdwic_atomic_poll_burst.call_count[batch_index] = 0; + } + g_fdwic_atomic_poll_burst.active_mask = 0; +} + +PTO_DEVICE_FUNC inline void fdwic_atomic_poll_boundary() { + if (g_fdwic_atomic_poll_burst.active_mask == 0) return; + fdwic_atomic_poll_boundary_at(fdwic_swimlane_detail_now()); +} + +PTO_DEVICE_FUNC inline uint32_t fdwic_atomic_poll_region_begin(uint32_t site_mask) { + const uint32_t previous_mask = g_fdwic_atomic_poll_burst.enabled_mask; + if (!fdwic_atomic_swimlane_enabled()) return previous_mask; + fdwic_atomic_poll_boundary(); + g_fdwic_atomic_poll_burst.enabled_mask = previous_mask | site_mask; + return previous_mask; +} + +PTO_DEVICE_FUNC inline void fdwic_atomic_poll_region_end(uint32_t previous_mask) { + if (!fdwic_atomic_swimlane_enabled()) return; + fdwic_atomic_poll_boundary(); + g_fdwic_atomic_poll_burst.enabled_mask = previous_mask; +} + +PTO_DEVICE_FUNC inline bool fdwic_atomic_poll_batch_enabled(FdwicAtomicSite site, FdwicAtomicOp actual_op) { + if (!fdwic_atomic_site_is_poll_batchable(site) || fdwic_atomic_site_op(site) != actual_op) return false; + return (g_fdwic_atomic_poll_burst.enabled_mask & fdwic_atomic_site_mask(site)) != 0; +} + +PTO_DEVICE_FUNC inline void fdwic_swimlane_accumulate_poll_call(FdwicAtomicSite site, uint64_t start_cycle) { + const int32_t batch_index = fdwic_atomic_poll_batch_index(site); + if (batch_index < 0) { + g_fdwic_atomic_counter_overflow = true; + return; + } + const uint32_t bit = 1U << static_cast(batch_index); + if ((g_fdwic_atomic_poll_burst.active_mask & bit) == 0) { + g_fdwic_atomic_poll_burst.start_cycle[batch_index] = start_cycle; + g_fdwic_atomic_poll_burst.call_count[batch_index] = 0; + g_fdwic_atomic_poll_burst.active_mask |= bit; + } + uint32_t &call_count = g_fdwic_atomic_poll_burst.call_count[batch_index]; + call_count++; + if (call_count == kFdwicAtomicPollCountMax) fdwic_atomic_poll_boundary(); +} + +PTO_DEVICE_FUNC inline void fdwic_swimlane_flush_core(__gm__ DistCore *self) { if (!fdwic_swimlane_enabled() || self == nullptr) return; - const int32_t core_idx = self->core_idx; + fdwic_atomic_poll_boundary(); const uint32_t records_per_core = g_fdwic_swimlane_records_per_core; __gm__ FdwicSwimlaneCoreState *core = g_fdwic_swimlane_core; if (core == nullptr || g_fdwic_swimlane_records == nullptr || records_per_core == 0) return; - uint32_t slot = core->count; - if (slot >= records_per_core) { - core->dropped = core->dropped + 1; - return; + core->atomic_calls = g_fdwic_atomic_calls; + core->poll_calls = g_fdwic_poll_calls; + core->poll_batch_records = g_fdwic_poll_batch_records; + if (g_fdwic_atomic_counter_overflow && core->dropped != UINT32_MAX) core->dropped = core->dropped + 1; + const uint32_t count = core->count < records_per_core ? core->count : records_per_core; + if (count > 0) { + dist_aicore_flush_region(g_fdwic_swimlane_records, static_cast(count) * sizeof(FdwicSwimlaneRecord)); } - __gm__ FdwicSwimlaneRecord *record = &g_fdwic_swimlane_records[slot]; - record->start_cycle = start_cycle; - record->end_cycle = end_cycle; - record->task_id = task_id; - record->func_id = func_id; - record->phase = static_cast(phase); - record->lane = self->lane; - record->block_id = self->block_id; - record->core_idx = core_idx; - record->flags = flags; - record->aux = aux; - core->count = slot + 1; + dist_aicore_flush_region(core, sizeof(FdwicSwimlaneCoreState)); +} + +PTO_DEVICE_FUNC inline void fdwic_swimlane_detail_record_atomic( + int32_t task_id, FdwicAtomicSite site, FdwicAtomicOp op, uint64_t start_cycle, uint64_t end_cycle, bool result_used, + bool return_ready, bool value_zero = false, uint64_t retries = 0 +) { + __gm__ DistCore *self = g_self; + if (!fdwic_atomic_swimlane_enabled() || self == nullptr || g_fdwic_swimlane_core == nullptr) return; + fdwic_swimlane_detail_record( + self, task_id, -1, FdwicSwimlanePhase::Atomic, start_cycle, end_cycle, + fdwic_atomic_trace_flags(op, result_used, return_ready, value_zero, retries), static_cast(site) + ); +} + +// Direct atomics remain one row per source call but do not split an active +// PollBatch. A batch is a logical wait-region window and may contain these +// interleaved rows; only its call_count, not its duration, represents atomic +// work. Region/phase/lap/final boundaries still close every active batch. + +template +PTO_DEVICE_FUNC inline T fdwic_trace_atomic_load( + int32_t task_id, FdwicAtomicSite site, __gm__ volatile T &value, bool result_used = true, + int memorder = __ATOMIC_ACQUIRE +) { + if (!fdwic_atomic_swimlane_enabled()) return atomic_load(value, memorder); + const bool poll_batch = result_used && fdwic_atomic_poll_batch_enabled(site, FdwicAtomicOp::Load); + const int32_t batch_index = poll_batch ? fdwic_atomic_poll_batch_index(site) : -1; + const bool first_in_batch = + poll_batch && (g_fdwic_atomic_poll_burst.active_mask & (1U << static_cast(batch_index))) == 0; + const uint64_t begin = !poll_batch || first_in_batch ? fdwic_swimlane_detail_now() : 0; + const T old = atomic_load(value, memorder); + if (poll_batch) { + fdwic_swimlane_count_atomic_call(true); + fdwic_swimlane_accumulate_poll_call(site, begin); + return old; + } + const bool return_ready = result_used && fdwic_atomic_return_ready_observed(); + const uint64_t end = result_used ? fdwic_swimlane_detail_now_after_atomic_result(old) : fdwic_swimlane_detail_now(); + // Keep tracing bookkeeping outside the measured direct-atomic boundary. + fdwic_swimlane_count_atomic_call(false); + fdwic_swimlane_detail_record_atomic( + task_id, site, FdwicAtomicOp::Load, begin, end, result_used, return_ready, old == static_cast(0) + ); + return old; +} + +template +PTO_DEVICE_FUNC inline T fdwic_trace_atomic_exchange( + int32_t task_id, FdwicAtomicSite site, __gm__ volatile T &value, V desired, bool result_used = false, + int memorder = __ATOMIC_ACQ_REL +) { + if (!fdwic_atomic_swimlane_enabled()) return atomic_exchange(value, desired, memorder); + const T desired_value = static_cast(desired); + const bool failed_claim_batch_enabled = result_used && site == FdwicAtomicSite::WonLaneClaimExchange && + desired_value == static_cast(kDrainedClaimed) && + fdwic_atomic_poll_batch_enabled(site, FdwicAtomicOp::Exchange); + const uint64_t begin = fdwic_swimlane_detail_now(); + const T old = atomic_exchange(value, desired, memorder); + const bool failed_claim_batch = failed_claim_batch_enabled && old == desired_value; + if (failed_claim_batch) { + fdwic_swimlane_count_atomic_call(true); + fdwic_swimlane_accumulate_poll_call(site, begin); + return old; + } + const bool return_ready = result_used && fdwic_atomic_return_ready_observed(); + const uint64_t end = result_used ? fdwic_swimlane_detail_now_after_atomic_result(old) : fdwic_swimlane_detail_now(); + fdwic_swimlane_count_atomic_call(false); + // Close failed retries at the successful transition's issue boundary. The + // successful claim itself remains an exact direct row. Capture its end + // first so batch-record writes are not charged to the direct span. + if (failed_claim_batch_enabled) fdwic_atomic_poll_boundary_at(begin); + fdwic_swimlane_detail_record_atomic(task_id, site, FdwicAtomicOp::Exchange, begin, end, result_used, return_ready); + return old; +} + +template +PTO_DEVICE_FUNC inline T fdwic_trace_atomic_fetch_add( + int32_t task_id, FdwicAtomicSite site, __gm__ volatile T &value, T delta, bool result_used = false, + int memorder = __ATOMIC_ACQ_REL +) { + if (!fdwic_atomic_swimlane_enabled()) return atomic_fetch_add(value, delta, memorder); + const uint64_t begin = fdwic_swimlane_detail_now(); + const T old = atomic_fetch_add(value, delta, memorder); + const bool return_ready = result_used && fdwic_atomic_return_ready_observed(); + const uint64_t end = result_used ? fdwic_swimlane_detail_now_after_atomic_result(old) : fdwic_swimlane_detail_now(); + fdwic_swimlane_count_atomic_call(false); + fdwic_swimlane_detail_record_atomic(task_id, site, FdwicAtomicOp::FetchAdd, begin, end, result_used, return_ready); + return old; +} + +template +PTO_DEVICE_FUNC inline T fdwic_trace_atomic_fetch_sub( + int32_t task_id, FdwicAtomicSite site, __gm__ volatile T &value, T delta, bool result_used = false, + int memorder = __ATOMIC_ACQ_REL +) { + if (!fdwic_atomic_swimlane_enabled()) return atomic_fetch_sub(value, delta, memorder); + const uint64_t begin = fdwic_swimlane_detail_now(); + const T old = atomic_fetch_sub(value, delta, memorder); + const bool return_ready = result_used && fdwic_atomic_return_ready_observed(); + const uint64_t end = result_used ? fdwic_swimlane_detail_now_after_atomic_result(old) : fdwic_swimlane_detail_now(); + fdwic_swimlane_count_atomic_call(false); + fdwic_swimlane_detail_record_atomic(task_id, site, FdwicAtomicOp::FetchSub, begin, end, result_used, return_ready); + return old; +} + +template +PTO_DEVICE_FUNC inline T fdwic_trace_atomic_fetch_max( + int32_t task_id, FdwicAtomicSite site, __gm__ volatile T &value, T desired, bool result_used = true, + int memorder = __ATOMIC_ACQ_REL +) { + if (!fdwic_atomic_swimlane_enabled()) return atomic_fetch_max(value, desired, memorder); + const uint64_t begin = fdwic_swimlane_detail_now(); + const T old = atomic_fetch_max(value, desired, memorder); + const bool return_ready = result_used && fdwic_atomic_return_ready_observed(); + const uint64_t end = result_used ? fdwic_swimlane_detail_now_after_atomic_result(old) : fdwic_swimlane_detail_now(); + fdwic_swimlane_count_atomic_call(false); + fdwic_swimlane_detail_record_atomic(task_id, site, FdwicAtomicOp::FetchMax, begin, end, result_used, return_ready); + return old; +} + +PTO_DEVICE_FUNC inline bool fdwic_trace_is_fatal(int32_t task_id = -1) { + return fdwic_trace_atomic_load(task_id, FdwicAtomicSite::FatalPoll, g_dist.fatal) != 0; +} + +PTO_DEVICE_FUNC inline void fdwic_trace_set_fatal(int32_t task_id = -1) { + const int32_t previous = fdwic_trace_atomic_exchange( + task_id, FdwicAtomicSite::FatalSet, g_dist.fatal, int32_t{1}, /*result_used=*/false + ); + (void)previous; +} + +PTO_DEVICE_FUNC inline void fdwic_swimlane_record_clock_baselines(__gm__ DistCore *self, int32_t dependency_value) { + if (!fdwic_atomic_swimlane_enabled() || self == nullptr) return; + fdwic_atomic_poll_boundary(); + const uint64_t clock_begin = fdwic_swimlane_detail_now(); + const uint64_t clock_end = fdwic_swimlane_detail_now(); + fdwic_swimlane_detail_record(self, -1, -1, FdwicSwimlanePhase::ClockBaseline, clock_begin, clock_end); + const uint64_t dependency_begin = fdwic_swimlane_detail_now(); + const uint64_t dependency_end = fdwic_swimlane_detail_now_after_atomic_result(dependency_value); + const uint32_t flags = + kFdwicClockAtomicDependency | (fdwic_atomic_return_ready_observed() ? kFdwicClockAtomicDependencyApplied : 0U); + fdwic_swimlane_detail_record( + self, -1, -1, FdwicSwimlanePhase::ClockBaseline, dependency_begin, dependency_end, flags + ); } PTO_DEVICE_FUNC inline void fdwic_swimlane_lap_reset(__gm__ DistCore *self) { if (self == nullptr) return; - self->swimlane_last_cycle = fdwic_swimlane_detail_now(); + const uint64_t cycle = fdwic_swimlane_detail_now(); + fdwic_atomic_poll_boundary_at(cycle); + self->swimlane_last_cycle = cycle; } PTO_DEVICE_FUNC inline void fdwic_swimlane_lap(__gm__ DistCore *self, int32_t task_id, int32_t func_id, FdwicSwimlanePhase phase) { if (self == nullptr) return; const uint64_t end_cycle = fdwic_swimlane_detail_now(); + fdwic_atomic_poll_boundary_at(end_cycle); const uint64_t start_cycle = self->swimlane_last_cycle; fdwic_swimlane_detail_record(self, task_id, func_id, phase, start_cycle, end_cycle); self->swimlane_last_cycle = end_cycle; diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/swimlane_types.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/swimlane_types.h index dddfc3779a..0e1a8c3d73 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/swimlane_types.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/swimlane_types.h @@ -13,9 +13,20 @@ #include +#include "data_type.h" + constexpr uint32_t kFdwicSwimlaneMagic = 0x4653574Cu; // FSWL -constexpr uint32_t kFdwicSwimlaneVersion = 1; +constexpr uint32_t kFdwicSwimlaneVersion = 3; +constexpr uint32_t kFdwicSwimlaneTraceSchemaVersion = 3; constexpr uint32_t kFdwicSwimlaneDefaultRecordsPerCore = 1u << 16; +// Eligible wait-region atomic calls are aggregated into exact-count batches at +// level 4. Reuse the existing 64K partition instead of reserving hundreds of +// thousands of rows per worker for individual spin iterations. +constexpr uint32_t kFdwicAtomicSwimlaneRecordsPerCore = kFdwicSwimlaneDefaultRecordsPerCore; +constexpr uint32_t kFdwicAtomicSwimlaneLevel = 4; +static_assert( + kFdwicAtomicSwimlaneRecordsPerCore % 2 == 0, "32B record partitions must keep every worker base on a 64B boundary" +); enum class FdwicSwimlanePhase : int32_t { Kernel = 0, @@ -32,12 +43,228 @@ enum class FdwicSwimlanePhase : int32_t { Claim = 11, Fanin = 12, Register = 13, + Atomic = 14, + ClockBaseline = 15, +}; + +// Atomic/ClockBaseline extend the existing ten-column FDWIC raw record ABI. +// The first fifteen sites intentionally keep the standalone PA probe's stable +// numbering; production-only BlockWon sites are appended and must not reorder +// those existing values. +enum class FdwicAtomicSite : uint32_t { + StartupIncrement = 0, + StartupPoll = 1, + FatalPoll = 2, + FatalSet = 3, + ClaimMax = 4, + FaninFlagLoad = 5, + CompletionVendExchange = 6, + CompletionFlagExchange = 7, + FrontierInitialLoad = 8, + FrontierFlagLoad = 9, + FrontierMax = 10, + HeapFrontierLoad = 11, + HeapVendLoad = 12, + ReplayDoneIncrement = 13, + ReplayDonePoll = 14, + WonSlotClaimMax = 15, + WonRemainingExchange = 16, + WonLaneResetExchange = 17, + WonLaneDepositExchange = 18, + WonStatePublishExchange = 19, + WonAnyPublishExchange = 20, + WonAnyLoad = 21, + WonStateLoad = 22, + WonLaneClaimExchange = 23, + WonLaneReleaseExchange = 24, + WonRemainingFetchSub = 25, + WonStateClearExchange = 26, + WonDrainedLoad = 27, + Count = 28, +}; + +enum class FdwicAtomicOp : uint32_t { + Load = 0, + Exchange = 1, + FetchAdd = 2, + FetchMax = 3, + FetchSub = 4, +}; + +static_assert( + static_cast(FdwicAtomicSite::Count) <= (1U << 16), "atomic site id must fit the compact record" +); +static_assert( + static_cast(FdwicAtomicSite::Count) <= 32, "atomic site id must fit the 32-bit poll-region site mask" +); + +constexpr uint32_t kFdwicAtomicOpMask = 0x0fU; +constexpr uint32_t kFdwicAtomicResultUsed = 1U << 4; +constexpr uint32_t kFdwicAtomicValueZero = 1U << 5; +constexpr uint32_t kFdwicAtomicReturnReady = 1U << 6; +constexpr uint32_t kFdwicAtomicPollBatch = 1U << 7; +constexpr uint32_t kFdwicAtomicRetriesShift = 8; +constexpr uint32_t kFdwicAtomicPollCountShift = 8; +constexpr uint32_t kFdwicAtomicPollCountMax = (1U << (32 - kFdwicAtomicPollCountShift)) - 1; + +constexpr uint32_t kFdwicClockAtomicDependency = 1U << 0; +constexpr uint32_t kFdwicClockAtomicDependencyApplied = 1U << 1; + +constexpr uint32_t kFdwicClaimWon = 1U << 0; +constexpr uint32_t kFdwicClaimAttempted = 1U << 1; + +PTO_DEVICE_FUNC constexpr FdwicAtomicOp fdwic_atomic_site_op(FdwicAtomicSite site) { + switch (site) { + case FdwicAtomicSite::StartupIncrement: + case FdwicAtomicSite::ReplayDoneIncrement: + return FdwicAtomicOp::FetchAdd; + case FdwicAtomicSite::FatalSet: + case FdwicAtomicSite::CompletionVendExchange: + case FdwicAtomicSite::CompletionFlagExchange: + case FdwicAtomicSite::WonRemainingExchange: + case FdwicAtomicSite::WonLaneResetExchange: + case FdwicAtomicSite::WonLaneDepositExchange: + case FdwicAtomicSite::WonStatePublishExchange: + case FdwicAtomicSite::WonAnyPublishExchange: + case FdwicAtomicSite::WonLaneClaimExchange: + case FdwicAtomicSite::WonLaneReleaseExchange: + case FdwicAtomicSite::WonStateClearExchange: + return FdwicAtomicOp::Exchange; + case FdwicAtomicSite::ClaimMax: + case FdwicAtomicSite::FrontierMax: + case FdwicAtomicSite::WonSlotClaimMax: + return FdwicAtomicOp::FetchMax; + case FdwicAtomicSite::WonRemainingFetchSub: + return FdwicAtomicOp::FetchSub; + default: + return FdwicAtomicOp::Load; + } +} + +PTO_DEVICE_FUNC constexpr bool fdwic_atomic_site_result_used(FdwicAtomicSite site) { + switch (site) { + case FdwicAtomicSite::StartupIncrement: + case FdwicAtomicSite::FatalSet: + case FdwicAtomicSite::CompletionVendExchange: + case FdwicAtomicSite::CompletionFlagExchange: + case FdwicAtomicSite::ReplayDoneIncrement: + case FdwicAtomicSite::WonRemainingExchange: + case FdwicAtomicSite::WonLaneResetExchange: + case FdwicAtomicSite::WonLaneDepositExchange: + case FdwicAtomicSite::WonStatePublishExchange: + case FdwicAtomicSite::WonAnyPublishExchange: + case FdwicAtomicSite::WonLaneReleaseExchange: + case FdwicAtomicSite::WonStateClearExchange: + return false; + default: + return true; + } +} + +// Observation loads used by explicit scheduler wait regions are batchable. +// WonLaneClaimExchange is the sole RMW exception: only its idempotent failed +// retries (old value already kDrainedClaimed) are batched, while the successful +// state transition remains a one-call record. +PTO_DEVICE_FUNC constexpr bool fdwic_atomic_site_is_poll_batchable(FdwicAtomicSite site) { + switch (site) { + case FdwicAtomicSite::StartupPoll: + case FdwicAtomicSite::FatalPoll: + case FdwicAtomicSite::FaninFlagLoad: + case FdwicAtomicSite::HeapFrontierLoad: + case FdwicAtomicSite::HeapVendLoad: + case FdwicAtomicSite::ReplayDonePoll: + case FdwicAtomicSite::WonAnyLoad: + case FdwicAtomicSite::WonStateLoad: + case FdwicAtomicSite::WonLaneClaimExchange: + case FdwicAtomicSite::WonDrainedLoad: + return true; + default: + return false; + } +} + +constexpr uint32_t kFdwicAtomicPollBatchSiteCount = 10; +static_assert(kFdwicAtomicPollBatchSiteCount <= 32, "poll-batch sites must fit the 32-bit active mask"); + +PTO_DEVICE_FUNC constexpr int32_t fdwic_atomic_poll_batch_index(FdwicAtomicSite site) { + switch (site) { + case FdwicAtomicSite::StartupPoll: + return 0; + case FdwicAtomicSite::FatalPoll: + return 1; + case FdwicAtomicSite::FaninFlagLoad: + return 2; + case FdwicAtomicSite::HeapFrontierLoad: + return 3; + case FdwicAtomicSite::HeapVendLoad: + return 4; + case FdwicAtomicSite::ReplayDonePoll: + return 5; + case FdwicAtomicSite::WonAnyLoad: + return 6; + case FdwicAtomicSite::WonStateLoad: + return 7; + case FdwicAtomicSite::WonDrainedLoad: + return 8; + case FdwicAtomicSite::WonLaneClaimExchange: + return 9; + default: + return -1; + } +} + +PTO_DEVICE_FUNC constexpr FdwicAtomicSite fdwic_atomic_poll_batch_site(uint32_t index) { + switch (index) { + case 0: + return FdwicAtomicSite::StartupPoll; + case 1: + return FdwicAtomicSite::FatalPoll; + case 2: + return FdwicAtomicSite::FaninFlagLoad; + case 3: + return FdwicAtomicSite::HeapFrontierLoad; + case 4: + return FdwicAtomicSite::HeapVendLoad; + case 5: + return FdwicAtomicSite::ReplayDonePoll; + case 6: + return FdwicAtomicSite::WonAnyLoad; + case 7: + return FdwicAtomicSite::WonStateLoad; + case 8: + return FdwicAtomicSite::WonDrainedLoad; + case 9: + return FdwicAtomicSite::WonLaneClaimExchange; + default: + return FdwicAtomicSite::Count; + } +} + +struct FdwicAtomicPollBurst { + uint64_t start_cycle[kFdwicAtomicPollBatchSiteCount]; + uint32_t call_count[kFdwicAtomicPollBatchSiteCount]; + uint32_t active_mask; + uint32_t enabled_mask; }; struct FdwicSwimlaneCoreState { volatile uint32_t count; volatile uint32_t dropped; - uint32_t pad[14]; + // Exact number of source-level atomic wrapper calls made by this worker + // while level-4 tracing was active. Poll batches contribute their encoded + // call_count rather than one call per Atomic record. + volatile uint32_t atomic_calls; + // Calls represented by PollBatch rows and the physical number of those + // rows. The host derives Atomic rows as + // atomic_calls - poll_calls + poll_batch_records, then verifies the raw. + volatile uint32_t poll_calls; + volatile uint32_t poll_batch_records; + // Topology is invariant within a worker partition. Store it once here + // instead of repeating the same 12 bytes in every record. + volatile int32_t core_idx; + volatile int32_t block_id; + volatile int32_t lane; + uint32_t pad[8]; } __attribute__((aligned(64))); static_assert(sizeof(FdwicSwimlaneCoreState) == 64, "FdwicSwimlaneCoreState must occupy one cacheline"); @@ -58,12 +285,10 @@ struct FdwicSwimlaneRecord { uint64_t end_cycle; int32_t task_id; int32_t func_id; - int32_t phase; - int32_t lane; - int32_t block_id; - int32_t core_idx; uint32_t flags; - uint32_t aux; -} __attribute__((aligned(64))); + uint16_t phase; + uint16_t aux; +} __attribute__((aligned(32))); -static_assert(sizeof(FdwicSwimlaneRecord) == 64, "FdwicSwimlaneRecord must occupy one cacheline"); +static_assert(sizeof(FdwicSwimlaneRecord) == 32, "FdwicSwimlaneRecord must occupy half a cacheline"); +static_assert(alignof(FdwicSwimlaneRecord) == 32, "FdwicSwimlaneRecord alignment changed"); diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/trace.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/trace.h index 893f01c9bd..628bfd5238 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/trace.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/trace.h @@ -50,6 +50,10 @@ PTO_DEVICE_FUNC inline FdwicSwimlanePhase trace_phase_to_swimlane_phase(TracePha return FdwicSwimlanePhase::Fanin; case TracePhase::Register: return FdwicSwimlanePhase::Register; + case TracePhase::Atomic: + return FdwicSwimlanePhase::Atomic; + case TracePhase::ClockBaseline: + return FdwicSwimlanePhase::ClockBaseline; } return FdwicSwimlanePhase::Kernel; } @@ -85,10 +89,15 @@ PTO_DEVICE_FUNC inline void trace_span_impl( PTO_DEVICE_FUNC inline void trace_instant_impl(__gm__ DistCore *self, int32_t task_id, int32_t func_id, TracePhase phase, uint32_t flags = 0) { const uint64_t cycle = fdwic_swimlane_detail_now(); + fdwic_atomic_poll_boundary_at(cycle); trace_span_impl(self, task_id, func_id, phase, cycle, cycle, flags, 0); } -PTO_DEVICE_FUNC inline uint64_t trace_span_begin_impl() { return fdwic_swimlane_detail_now(); } +PTO_DEVICE_FUNC inline uint64_t trace_span_begin_impl() { + const uint64_t cycle = fdwic_swimlane_detail_now(); + fdwic_atomic_poll_boundary_at(cycle); + return cycle; +} #define TRACE_LAP(self, task_id, func_id, phase) trace_lap_impl((self), (task_id), (func_id), (phase)) #define TRACE_LAP_RESET(self) trace_lap_reset_impl((self)) diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/worker_state.h b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/worker_state.h index c32657e575..024ec91ec5 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/worker_state.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/dist_engine/common/worker_state.h @@ -30,32 +30,47 @@ [[block_local]] static int32_t g_ccec_ordinal; [[block_local]] static bool g_ccec_valid_worker; [[block_local]] static bool g_fdwic_joint_submit_seen; -[[block_local]] static bool g_fdwic_swimlane_enabled; +[[block_local]] static uint32_t g_fdwic_swimlane_level; [[block_local]] static __gm__ FdwicSwimlaneHeader *g_fdwic_swimlane_header; [[block_local]] static __gm__ FdwicSwimlaneCoreState *g_fdwic_swimlane_core; [[block_local]] static __gm__ FdwicSwimlaneRecord *g_fdwic_swimlane_records; [[block_local]] static uint32_t g_fdwic_swimlane_records_per_core; +[[block_local]] static FdwicAtomicPollBurst g_fdwic_atomic_poll_burst; +[[block_local]] static uint32_t g_fdwic_atomic_calls; +[[block_local]] static uint32_t g_fdwic_poll_calls; +[[block_local]] static uint32_t g_fdwic_poll_batch_records; +[[block_local]] static bool g_fdwic_atomic_counter_overflow; #define g_dist (*g_dist_ptr) #elif defined(__CPU_SIM) static DistGlobal g_dist_fallback; static DistGlobal *g_dist_ptr = nullptr; thread_local DistCore *g_self = nullptr; thread_local bool g_fdwic_joint_submit_seen = false; -thread_local bool g_fdwic_swimlane_enabled = false; +thread_local uint32_t g_fdwic_swimlane_level = 0; thread_local FdwicSwimlaneHeader *g_fdwic_swimlane_header = nullptr; thread_local FdwicSwimlaneCoreState *g_fdwic_swimlane_core = nullptr; thread_local FdwicSwimlaneRecord *g_fdwic_swimlane_records = nullptr; thread_local uint32_t g_fdwic_swimlane_records_per_core = 0; +thread_local FdwicAtomicPollBurst g_fdwic_atomic_poll_burst = {}; +thread_local uint32_t g_fdwic_atomic_calls = 0; +thread_local uint32_t g_fdwic_poll_calls = 0; +thread_local uint32_t g_fdwic_poll_batch_records = 0; +thread_local bool g_fdwic_atomic_counter_overflow = false; #define g_dist (*g_dist_ptr) #else static DistGlobal g_dist_fallback; static DistGlobal *g_dist_ptr = &g_dist_fallback; thread_local DistCore *g_self = nullptr; thread_local bool g_fdwic_joint_submit_seen = false; -thread_local bool g_fdwic_swimlane_enabled = false; +thread_local uint32_t g_fdwic_swimlane_level = 0; thread_local FdwicSwimlaneHeader *g_fdwic_swimlane_header = nullptr; thread_local FdwicSwimlaneCoreState *g_fdwic_swimlane_core = nullptr; thread_local FdwicSwimlaneRecord *g_fdwic_swimlane_records = nullptr; thread_local uint32_t g_fdwic_swimlane_records_per_core = 0; +thread_local FdwicAtomicPollBurst g_fdwic_atomic_poll_burst = {}; +thread_local uint32_t g_fdwic_atomic_calls = 0; +thread_local uint32_t g_fdwic_poll_calls = 0; +thread_local uint32_t g_fdwic_poll_batch_records = 0; +thread_local bool g_fdwic_atomic_counter_overflow = false; #define g_dist (*g_dist_ptr) #endif diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/runtime.h b/src/a5/runtime/fully_distributed_within_core/runtime/runtime.h index 5a35156947..839d2dd524 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/runtime.h +++ b/src/a5/runtime/fully_distributed_within_core/runtime/runtime.h @@ -254,7 +254,9 @@ class Runtime { volatile uint64_t shared_addr; volatile int32_t num_workers; // number of AICore workers participating volatile uint64_t swimlane_base; - volatile uint32_t swimlane_enabled; + // Existing L2 swimlane perf level (0..4). FDWIC keeps phase spans at + // levels 1..3 and adds source-level atomic spans at level 4. + volatile uint32_t swimlane_level; volatile uint32_t swimlane_records_per_core; Tensor ccec_orch_tensors[CHIP_MAX_TENSOR_ARGS]; uint64_t ccec_orch_scalars[CHIP_MAX_SCALAR_ARGS]; @@ -265,6 +267,7 @@ class Runtime { } dist; void *fdwic_swimlane_host_shadow_; + uint64_t fdwic_swimlane_dev_allocation_; uint64_t fdwic_swimlane_dev_base_; uint64_t fdwic_swimlane_bytes_; uint32_t fdwic_swimlane_num_cores_; diff --git a/src/a5/runtime/fully_distributed_within_core/runtime/shared/runtime.cpp b/src/a5/runtime/fully_distributed_within_core/runtime/shared/runtime.cpp index e27554f89b..111103a2ec 100644 --- a/src/a5/runtime/fully_distributed_within_core/runtime/shared/runtime.cpp +++ b/src/a5/runtime/fully_distributed_within_core/runtime/shared/runtime.cpp @@ -40,13 +40,14 @@ Runtime::Runtime() { dist.shared_addr = 0; dist.num_workers = 0; dist.swimlane_base = 0; - dist.swimlane_enabled = 0; + dist.swimlane_level = 0; dist.swimlane_records_per_core = 0; dist.orch_args.reset(); dist.ccec_orch_tensor_count = 0; dist.ccec_orch_scalar_count = 0; dist.done_count = 0; fdwic_swimlane_host_shadow_ = nullptr; + fdwic_swimlane_dev_allocation_ = 0; fdwic_swimlane_dev_base_ = 0; fdwic_swimlane_bytes_ = 0; fdwic_swimlane_num_cores_ = 0; diff --git a/tests/atomic_probe/A5_FDWIC_PAGED_ATTENTION_REPRO.md b/tests/atomic_probe/a5_fdwic_atomic_swimlane_repo.md similarity index 55% rename from tests/atomic_probe/A5_FDWIC_PAGED_ATTENTION_REPRO.md rename to tests/atomic_probe/a5_fdwic_atomic_swimlane_repo.md index 278c3b8275..26a9951544 100644 --- a/tests/atomic_probe/A5_FDWIC_PAGED_ATTENTION_REPRO.md +++ b/tests/atomic_probe/a5_fdwic_atomic_swimlane_repo.md @@ -20,13 +20,15 @@ test_paged_attention_unroll.py 本文不覆盖其他测试目录、其他 runtime、A2/A3、L3 或整段 device wall time。 A5Sim 用于功能和调度流程验证;5.6 ms 基线只从真实 A5 生成的 -l2_swimlane_records.json 中读取。 +l2_swimlane_records.json 中读取。直接 atomic 逐调用记录和等待区 poll 精确计数只在 +`--enable-l2-swimlane 4` 开启;它们用于定位真实 PA 的 scalar atomic 分布, +不能替代 level 1 到 3 的 phase-only 性能基线。 ### 已验证环境 | 项目 | 本次验证值 | | --- | --- | -| 验证日期 | 2026-07-17 | +| 验证日期 | 2026-07-17(phase 基线)、2026-07-18(level-4 atomic) | | 芯片 | Ascend950PR_958b | | 设备 | /dev/davinci0 | | Driver | 7.0.t9.0.B798,ascendhal 7.35.23 | @@ -40,8 +42,9 @@ l2_swimlane_records.json 中读取。 | pytest | 7.4.4 | | GCC 15 | 15.0.1,Ubuntu 15-20250404-0ubuntu1 | | PTO-ISA | ddafa8da9c760ecd13fe9fe2833d6ee55fb20bd8 | -| simpler 分支 | fdwic-swimlane-deps | -| simpler 实测基准 HEAD | 52ca4f5eba343c2f7b7a3a743e575cb9308d128f | +| simpler 分支 | real-pa-atomic-swimlane | +| 历史 phase 实测 HEAD | 52ca4f5eba343c2f7b7a3a743e575cb9308d128f | +| schema-v3 迁移基线 HEAD | 5274945b(迁移改动基于此展开) | 系统的 /etc/os-release 标签为 Ubuntu 20.04.6,但实际 getconf GNU_LIBC_VERSION 输出 glibc 2.39。判断 GCC 15 二进制兼容性时, @@ -53,7 +56,8 @@ getconf GNU_LIBC_VERSION 输出 glibc 2.39。判断 GCC 15 二进制兼容性时 | --- | --- | | A5Sim Case1 | PASSED,约 71.62 s | | A5 Case1 正确性 | PASSED | -| A5 swimlane Case1 | PASSED,pytest 约 85.48 s | +| A5 level-1 phase swimlane Case1 | PASSED,pytest 约 85.48 s | +| A5 level-4 atomic swimlane Case1 | PASSED,pytest 81.90 s,96 核 schema v3 闭合 | | 每核 Submit | 1280 个,task id 为 0 到 1279 | | 全局首个至末个 Submit | 5.642245 ms | | 排除 task 0 分配后的 kernel Submit | 5.635263 ms | @@ -580,7 +584,7 @@ test "$(git -C "$PTO_ISA_ROOT" rev-parse HEAD)" = \ python -m pytest "$TEST_FILE" \ --platform a5sim \ --case Case1 \ - --enable-l2-swimlane \ + --enable-l2-swimlane 4 \ --use-example-exec-time \ --clone-protocol https \ --pto-isa-commit "$PTO_ISA_COMMIT" \ @@ -590,7 +594,10 @@ python -m pytest "$TEST_FILE" \ ~~~ --use-example-exec-time 仅适用于 fully_distributed_within_core 的 sim。 -它不能用于真实 A5 命令。 +它不能用于真实 A5 命令。这里显式写出的 level 4 与裸参数 +`--enable-l2-swimlane` 等价;A5Sim 用于检查 schema v3、记录结构、加权计数闭合和 +转换结果,不提供真实 A5 atomic 完成时间。A5Sim 的直接 Atomic 只标记模拟执行的 +源码包围边界,PollBatch 只标记模拟调度中的 poll 窗口;两者都不能当作真实硬件时延。 ### 运行 A5 正确性 smoke @@ -611,7 +618,32 @@ python -m pytest "$TEST_FILE" \ -s -v ~~~ -### 运行 A5 swimlane 性能复现 +### 运行 A5 phase 泳道性能复现 + +~~~bash +python -m pytest "$TEST_FILE" \ + --platform a5 \ + --device 0 \ + --case Case1 \ + --enable-l2-swimlane 1 \ + --clone-protocol https \ + --pto-isa-commit "$PTO_ISA_COMMIT" \ + --pto-session-timeout 1200 \ + --require-pto-isa \ + -s -v +~~~ + +level 1 到 3 保留已有 FDWIC phase 记录,不增加 Atomic 和 +`ClockBaseline`。上面的 level 1 与历史 trace 的有效 level 一致,适合继续复现本文 +约 5.6 ms 的 Submit 性能口径。它们继续导出 schema 1 的 legacy Claim flags; +level 4 导出 schema v3,其中 Claim 使用 `attempted/won` 三态,Atomic 同时支持 +逐调用直接记录和精确计数 PollBatch。源码 atomic 调用点现在统一经过 level 判断, +因此不能声称新旧二进制指令级完全相同;做前后性能比较时,应使用同一版二进制分别采 +level 1 phase 基线和 level 4 诊断样本。 + +### 运行 A5 level-4 atomic 泳道 + +需要观察真实 PA 的直接 atomic 调用并统计等待区 poll 调用时,单独运行 level 4: ~~~bash python -m pytest "$TEST_FILE" \ @@ -626,40 +658,294 @@ python -m pytest "$TEST_FILE" \ -s -v ~~~ -本次实测生成: +level 4 在原有 phase 上增加真实 FDWIC PA 的 Atomic 记录:直接 atomic 保持一次 +源码调用一条记录;显式等待区内允许合并的 observation load,以及唯一一种已确认的 +幂等失败 exchange 重试,则用带精确调用次数的 PollBatch 表示。最终 drain 之后,每个 +AIC/AIV 还会写两条 `ClockBaseline`。该模式会增加记录写、改变代码布局,也可能改变 +多核到达和轮询次数,所以它是诊断样本,不是本文 5.6 ms 无 atomic 插桩基线。 + +### raw、merged 产物与重新转换 + +A5Sim 和 A5 的 Case1 都使用 SceneTest 的同一输出规则: + +~~~text +outputs/TestPagedAttentionUnroll_Case1_/ +├── l2_swimlane_records.json +└── merged_swimlane.json +~~~ + +其中 `l2_swimlane_records.json` 是保留原始 cycle 的 raw 文件;pytest 在 case +结束时调用仓内共享转换器,生成可直接载入 Perfetto 的 +`merged_swimlane.json`。历史 phase 基线 raw 与本次真实 A5 level-4 raw 分别为: ~~~text -outputs/TestPagedAttentionUnroll_Case1_20260717_023809/ -l2_swimlane_records.json +outputs/TestPagedAttentionUnroll_Case1_20260717_023809/l2_swimlane_records.json +outputs/TestPagedAttentionUnroll_Case1_20260718_161520/l2_swimlane_records.json +~~~ + +新的复现会生成不同时间戳目录。若需要重新转换已有 raw,执行: + +~~~bash +python -m simpler_setup.tools.swimlane_converter \ + outputs/TestPagedAttentionUnroll_Case1_YYYYMMDD_HHMMSS/l2_swimlane_records.json \ + -v ~~~ -新的复现会生成不同时间戳目录。trace 约几十 MiB,pytest 结束后再读取, +未指定 `-o` 时,转换器仍写到 raw 同目录的 `merged_swimlane.json`。上面的历史 +phase-only raw 约几十 MiB;level 4 的 JSON 大小取决于直接 Atomic、PollBatch 和 +phase 的实际记录条数,不再随每一次连续 poll 线性增长。pytest 结束后再读取, 不要用 pytest wall time 代替 Submit 指标。 +### level-4 atomic 数据契约 + +当前生产端导出的 level 4 raw,其 `metadata.trace_schema_version` 必须为 3;共享 +converter 仍保留对历史 schema 的读取兼容。转换后的 `Atomic` 和 +`ClockBaseline` 都画在对应 AIC、AIV0 或 AIV1 的原 scalar lane;它们不是与 +scalar 并行的伪子轨。Kernel 仍画在独立的 `AIC/AIV·kernel` 轨。 + +直接 Atomic 的名称明确给出终点语义: + +~~~text +atomic.return_ready..# +atomic.source_issue..# +~~~ + +真实 A5 上,返回值会被后续逻辑消费的直接调用使用 `return_ready`,表示返回旧值已 +可被本核 scalar 使用;返回值被丢弃的发布型直接调用使用 `source_issue`,只表示源码 +发射包围区间。两者都不表示跨核可见时刻,也不能直接称为 atomic retire 延迟。每条 +直接 Atomic 的 `args.call_count` 固定为 1。结束 cycle 在 atomic 返回后立即采样, +本地调用计数更新、PollBatch 落盘和 direct 记录写入都发生在结束 cycle 之后,不能 +混入该条 direct span。 + +以下九类 observation load 只有在对应的显式 scheduler 等待区内才允许聚合: + +- 通用等待:`startup_poll`、`fatal_poll`、`fanin_flag_load`、 + `heap_frontier_load`、`heap_vend_load`、`replay_done_poll`; +- BlockWon 等待:`won_any_load`、`won_state_load`、`won_drained_load`。 + +BlockWon 三类 load 只在 slot-capacity、heap slow path、won-slot、sim producer-ready、 +final-drain 这些已有外层等待区中聚合;普通 Submit 中的一次性或 opportunistic +BlockWon 扫描仍是直接 Atomic。 + +唯一允许聚合的 RMW 是 `won_lane_claim_exchange`,并且必须同时满足:位于上述显式 +等待区、写入 `claimed(1)`、返回旧值也是 `claimed(1)`。这表示一次 1→1、没有改变 +协议状态的失败 claim 重试。返回 `free(0)` 的成功 claim 始终逐条记录; +`won_slot_claim_max`、release、`fetch_sub`、state clear 以及其他 RMW 也全部逐条记录。 + +每个聚合记录转换为: + +~~~text +atomic.poll_batch..× +~~~ + +其 `args.call_count` 是该等待区内实际执行的源码 atomic wrapper 调用次数,不是采样值; +`task_id=-1`、`func_id=-1` 表示它归属于 scheduler 等待区而非某个任务。一个等待区可 +同时累积多个 site,因此不同 site 的 PollBatch 时间窗可以重叠;等待区内的直接 +Atomic 也可能与该窗口交错。merged 用 `batch_semantics=observation_load_calls` 或 +`idempotent_failed_exchange_retries` 区分两种计数,并显式写出 +`may_contain_interleaved_direct_atomics=true`。 + +PollBatch 的 `duration`/`poll_window_cycles` 只是 logical poll episode 的包络:它既 +不是纯 poll 时间或独占 scalar 时间,也不是其中任一次 atomic 的延迟,更不是 +`call_count` 次 atomic 串行延迟之和。raw 中的物理相邻顺序也不等于严格时间顺序; +分析应以 cycle 字段为准,不能用 PollBatch duration 计算单次 atomic 的 median 或 p95。 + +schema v3 用 Atomic flags 的 bit 7 标识 PollBatch,bits 8..31 保存无符号 24 bit +`call_count`;低 4 bit 必须是该 site 的实际 op:九类 observation 是 `load(0)`, +`won_lane_claim_exchange` 是 `exchange(1)`。bit 4 表示返回值被消费,bit 5/6 在 +PollBatch 中必须为 0。bit 7 为 0 的直接 Atomic 保留原有 flags 语义,不能把其高位 +按 poll 次数解析。 + +site 0 到 14 与 standalone PA 的稳定编号完全一致;真实 FDWIC PA 只在末尾追加 +15 到 27,覆盖生产实现的 BlockWon 路径和 `FetchSub`,没有重排已有编号: + +| `site_id` | Perfetto `site` | `op` | 真实 PA 路径 | +| --------: | --------------- | ---- | ------------ | +| 0 | `startup_increment` | `fetch_add` | 启动屏障到达计数 | +| 1 | `startup_poll` | `load` | 启动屏障轮询 | +| 2 | `fatal_poll` | `load` | fatal 状态检查 | +| 3 | `fatal_set` | `exchange` | fatal 状态发布 | +| 4 | `claim_max` | `fetch_max` | Submit lane Claim | +| 5 | `fanin_flag_load` | `load` | fanin 依赖 flag | +| 6 | `completion_vend_exchange` | `exchange` | completion vend 发布 | +| 7 | `completion_flag_exchange` | `exchange` | completion flag 发布 | +| 8 | `frontier_initial_load` | `load` | completion frontier 首次读取 | +| 9 | `frontier_flag_load` | `load` | frontier 扫描 flag | +| 10 | `frontier_max` | `fetch_max` | frontier 推进 | +| 11 | `heap_frontier_load` | `load` | HeapGuard frontier | +| 12 | `heap_vend_load` | `load` | HeapGuard vend | +| 13 | `replay_done_increment` | `fetch_add` | 回放完成屏障到达计数 | +| 14 | `replay_done_poll` | `load` | 最终 drain 轮询回放完成 | +| 15 | `won_slot_claim_max` | `fetch_max` | BlockWon slot 认领 | +| 16 | `won_remaining_exchange` | `exchange` | BlockWon remaining 初始化 | +| 17 | `won_lane_reset_exchange` | `exchange` | BlockWon lane 重置 | +| 18 | `won_lane_deposit_exchange` | `exchange` | BlockWon lane 完成发布 | +| 19 | `won_state_publish_exchange` | `exchange` | BlockWon state 发布 | +| 20 | `won_any_publish_exchange` | `exchange` | BlockWon any 发布 | +| 21 | `won_any_load` | `load` | BlockWon any 读取 | +| 22 | `won_state_load` | `load` | BlockWon state 读取 | +| 23 | `won_lane_claim_exchange` | `exchange` | BlockWon lane claim | +| 24 | `won_lane_release_exchange` | `exchange` | BlockWon lane release | +| 25 | `won_remaining_fetch_sub` | `fetch_sub` | BlockWon remaining 递减并判断最后一个 lane | +| 26 | `won_state_clear_exchange` | `exchange` | BlockWon state 清理 | +| 27 | `won_drained_load` | `load` | BlockWon drained 检查 | + +上表定义的是本文覆盖的真实 FDWIC PA / A5 hot path 可记录调用点集合,不包含 +CPU sim watchdog/debug 诊断原子,也不表示 Case1 每轮一定出现全部 28 类事件。 +Case1 的单 lane 图通常不进入 BlockWon 动态路径;某个 BlockWon site +计数为零不能单独判定为漏插桩。`fetch_sub` 的 op id 为 4,不能按 standalone +旧版只有 Load、Exchange、FetchAdd、FetchMax 四类 op 的假设解析。 + +frontier 扫描沿用 standalone 的任务归因:读取全局扫描起点的 +`frontier_initial_load` 记为 `task_id=-1`;随后读取 `next` 完成 flag 的 +`frontier_flag_load` 与推进同一个 `next` 的 `frontier_max` 都记为 +`task_id=next`。因此两条扫描事件可以按 core、task_id 配对,不归到触发本轮 +completion 的另一个任务上。 + +raw 的 `metadata.trace_schema_version=3` 中,Claim 的 flags 明确记录 +`attempted` 和 `won`,merged 中对应三种互斥状态: + +| Claim 状态 | `attempted` | `won` | Perfetto 名称 | Case1 预期数量(`N=metadata.num_cores`) | +| ---------- | ----------: | ----: | ------------- | -------------: | +| 未参与该 lane 的 Claim | 0 | 0 | `claim.not_attempted` | `N*512` | +| 已尝试但失败 | 1 | 0 | `claim.lost` | `N*768-1280` | +| 已尝试且获胜 | 1 | 1 | `claim.won` | `1280` | + +三者合计 `N*1280` 条 Claim,其中实际执行 `claim_max.fetch_max` 的数量为 +`N*768`。历史 96 核样本对应 49,152 / 72,448 / 1,280;108 核 A5Sim 对应 +55,296 / 81,664 / 1,280。`attempted=0, won=1` 是非法组合,不应出现在有效导出中。 + +host 在发布 raw 前按核执行闭合校验: + +- `count` 不得超过该核分区容量,禁止截断后继续导出; +- `dropped` 必须为 0; +- 每条直接 Atomic 计为一次调用;每条 PollBatch 按 flags 高 24 bit 编码的 + `call_count` 加权,且 `call_count` 必须大于 0; +- `atomic_calls = atomic_records - poll_batch_records + batched_poll_calls`; +- 所有 PollBatch 的 `call_count` 之和必须等于 `batched_poll_calls`,物理 PollBatch + 条数必须等于 `poll_batch_records`; +- level 4 每核必须恰有两条 `ClockBaseline`,总数必须为 `2*N`; +- level 1 到 3 不应出现 `Atomic` 或 `ClockBaseline`。 + +level 1 到 4 都使用每核 65,536 条(64K)分区。设备二进制 record version 3 不再 +逐条重复 `core_idx/block_id/lane`,而是在每核 state 中保存并校验一次,导出十列 raw +JSON 时再补回;每条物理记录为 32 byte,JSON 列格式不变。仅计 record 分区时,96 个 +worker 约占 192 MiB,108 个 worker 约占 216 MiB,另有很小的 header;raw metadata +中的 `records_per_core`、`record_size_bytes` 和 `device_trace_bytes` 给出本次运行的精确 +配置。host 只初始化 header,导出时先读取 header/core 计数,再按核搬运实际 +`count*32` byte 的有效记录,不常驻完整设备镜像。 + +PollBatch 让大量连续轮询按等待区和 site 合并,同时保留精确调用次数,因此不再需要 +为每一次 poll 预留物理记录。若单个 batch 达到高 24 bit 可表示的最大次数,实现会 +先落盘并开启下一条 batch,不会饱和后丢失计数。物理记录容量仍不是理论无界;任何 +容量溢出都必须明确失败,不能截断后发布,也不能只放宽 `dropped` 校验。 +设备侧边界 GTest 直接从 `0xFFFFFE` 累加到 `0xFFFFFF`,确认第一条立即落盘;随后 +第 `0x1000000` 次调用以 `call_count=1` 重开第二条,并验证两条之和精确等于 +`0x1000000`。 + +`batched_poll_calls` 包含上述九类 observation load 和幂等失败 exchange 重试,是 +本次启用 schema v3 插桩后真实执行的精确调用数。插桩本身会改变代码布局、核间到达 +时序和轮询节奏,因此不同插桩方案下的调用次数不能当作固定 workload 常量直接比较; +计数换算和前后对比应使用同一观察模式。 + +2026-07-18 最终 A5Sim 结构验证得到以下闭合结果;它们用于证明记录规模和计数契约, +不表示真实 A5 atomic 时延,也不参与后文按 160 ns 对真机计数所做的归因估算。 +ReuseStress 只是十类规则与 BlockWon 路径的结构压力验收,不扩大本文只复现 PA Case1 +性能的范围: + +| 样本 | raw 大小 | 总记录 | 逻辑 `atomic_calls` | 物理 Atomic | PollBatch | 单核记录峰值 | dropped | +| ---- | -------: | -----: | ---------------------: | ----------: | --------: | ------------: | ------: | +| Case1 `20260718_152435` | 124,547,744 B(约 118.8 MiB) | 1,464,594 | 187,860,395 | 493,301 | 959 | 16,357 | 0 | +| BlockWon ReuseStress `20260718_154229` | 2,055,624 B(约 1.96 MiB) | 24,442 | 243,357,709 | 10,505 | 572 | 519 | 0 | + +Case1 中 `batched_poll_calls=187,368,053`,满足 +`187,860,395 - 187,368,053 + 959 = 493,301`;108 核共有 216 条 +`ClockBaseline`。ReuseStress 中 `batched_poll_calls=243,347,776`,满足 +`243,357,709 - 243,347,776 + 572 = 10,505`;其中 22 条 site 23 PollBatch +精确表示 625,394 次幂等失败 exchange 重试,另有 348 条 site 23 直接记录。这里的 +直接记录包含成功 claim、等待区外调用或其他非聚合情形,不能全部等同为成功次数。 + +同一版 level-4 代码随后在真实 A5 device 0 上执行 Case1,得到以下实际 PA 记录: + +| 样本 | raw 大小 | merged 大小 | 核拓扑 | 总记录 | 逻辑 `atomic_calls` | 物理 Atomic | PollBatch | 单核记录峰值 | dropped | +| ---- | -------: | ----------: | ------ | -----: | ---------------------: | ----------: | --------: | ------------: | ------: | +| A5 Case1 `20260718_161520` | 77,128,944 B(约 73.6 MiB) | 333,581,552 B(约 318.1 MiB) | 32 AIC + 64 AIV | 973,430 | 115,200 | 110,006 | 340 | 10,751 | 0 | + +该真机样本中 `batched_poll_calls=5,534`,满足 +`115,200 - 5,534 + 340 = 110,006`;96 核共有 192 条 `ClockBaseline`。340 条 +PollBatch 分布在 `StartupPoll`、`FatalPoll`、`FaninFlagLoad` 和 +`ReplayDonePoll`,精确表示 5,534 次等待区调用。Case1 没有动态进入 BlockWon, +因此没有出现 21/22/23/27 类 batch;这不影响十类 allowlist 的实现和独立 +BlockWon ReuseStress 覆盖。直接事件中 106,914 条使用真机 `return_ready` 边界, +与 `result_used` 数量一致,可与 A5Sim 的 `source_issue` 边界明确区分。 + +若只为形成直观的 scalar 归因依据,暂统一使用 160 ns/次,则这份真实 A5 Case1 的 +全核累计估计为 `115,200 × 160 ns = 18.432 ms core-work`。它是跨 96 核求和后的 +工作量,不是 PA 墙钟耗时;不能把 18.432 ms 与约 5.6 ms Submit 包络直接相加。 + +任一条件不满足,export 返回非零;若 runtime 本身成功,该错误继续传播为本次运行 +失败,不能把旧文件或不完整文件当作有效样本。共享 converter 会按 raw 行重新计算 +`records`、`atomic_records`、`clock_baseline_records`、`atomic_calls`、 +`batched_poll_calls` 和 `poll_batch_records`,逐项核对 `metadata.fdwic_summary`; +`dropped_records` 无法从已导出的有效行反推,因此 converter 要求 producer summary +明确给出 0。正式分析还应确认 raw 与 merged 的物理 Atomic 条数相等,并以 +`atomic_calls` 而不是物理 Atomic 条数表示源码调用总数。 + +两条 `ClockBaseline` 分别是连续两次 `SYS_CNT` 读取,以及 atomic 返回值依赖 hook +的固定路径。它们用于观察计时分辨率和 hook 本身的分布,不是可以从每条 Atomic +机械相减的校正常数。 + +### atomic 性能解释边界 + +单条直接 Atomic 是某一 AIC/AIV scalar lane 上的本地 span。可以按 +`core_type/site/op` 查看直接事件数、中位数、p95、最大值,也可以比较同一核上某个 +直接 site 的累计分布;这些数据适合回答“哪类直接 atomic 常见、哪类本核返回等待 +长”。PollBatch 只适合统计对应 site/op 的调用次数和等待 episode 分布;其窗口可能 +包含交错的直接 atomic,duration 不能混入单次 atomic 延迟的 median/p95。 + +若已有独立 atomic probe 给出的标定开销,可把 schema v3 的精确调用数换算为直观的 +scalar 工作量估计: + +~~~text +estimated_atomic_core_work_ns = Σ(event.call_count × calibrated_atomic_cost_ns(site, op)) +~~~ + +直接 Atomic 的 `call_count=1`,PollBatch 使用其精确计数。若暂时对所有 site/op 统一 +采用约 160 ns 的单次标定值,公式简化为 +`estimated_atomic_core_work_ns ≈ atomic_calls × 160 ns`;若只归因某个 scalar +阶段,则只对属于该阶段的事件求和。该数值是所有核累计的 scalar core-work 估计, +不是 Submit wall time,也不是从 PollBatch duration 反推的单次硬件时延。 + +不能把所有核的 Atomic span 简单求和后称为 Submit 墙钟开销:不同核并行执行, +大量 span 相互重叠;Atomic 还嵌套在 Claim、Replay、Submit 等外层 phase 中,外层 +和内层也不能再次相加。全核求和只能解释为带观察的 aggregate core-work。若要判断 +对墙钟时间的影响,应结合关键 scalar lane、全局最早 Submit 到最晚 Submit 的包络, +并对优化前后使用相同观察模式;最终端到端收益仍用关闭 atomic 诊断的独立运行确认。 + ### 提取首个到末个 Submit -以下脚本自动选择最新 Case1 trace,校验 96 个 core、每核 1280 个 Submit -以及完整 task id,并输出用户关注的全局 span: +先把 `TRACE` 指向 level 1 到 3 的 phase-only Case1 raw。以下脚本拒绝 level 4, +避免误把 Atomic 插桩后的诊断时间当成约 5.6 ms 基线;随后校验 96 个 core、 +每核 1280 个 Submit 以及完整 task id,并输出用户关注的全局 span: ~~~bash +export TRACE=outputs/TestPagedAttentionUnroll_Case1_YYYYMMDD_HHMMSS/l2_swimlane_records.json + python - <<'PY' import json +import os import statistics from collections import defaultdict from pathlib import Path -traces = list( - Path("outputs").glob( - "TestPagedAttentionUnroll_Case1_*/l2_swimlane_records.json" - ) -) -if not traces: - raise SystemExit("no Case1 l2_swimlane_records.json found") - -trace = max(traces, key=lambda path: path.stat().st_mtime) +trace = Path(os.environ["TRACE"]) +if not trace.is_file(): + raise SystemExit(f"trace does not exist: {trace}") with trace.open() as stream: data = json.load(stream) +level = int(data["l2_swimlane_level"]) +assert level in (1, 2, 3), f"phase baseline requires level 1..3, got {level}" hz = int(data["metadata"]["clock_freq_hz"]) submits = [row for row in data["fdwic_events"] if row[5] == "Submit"] if not submits: diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index bf8a33318f..47649e5bc6 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -33,7 +33,7 @@ Submit 路径,供后续继续优化。快照日期更新至 2026-07-18;当 fanin 成功/失败 load 与 frontier 重复前推,再进行单变量消减。 环境安装、编译和基线复现过程见 -[A5 FDWIC Paged Attention 安装与复现指南](../A5_FDWIC_PAGED_ATTENTION_REPRO.md)。 +[A5 FDWIC Paged Attention 安装与复现指南](../a5_fdwic_atomic_swimlane_repo.md)。 ## 2. Case1 工作量与 atomic 语义 diff --git a/tests/ut/cpp/CMakeLists.txt b/tests/ut/cpp/CMakeLists.txt index f750857702..69d93a4d96 100644 --- a/tests/ut/cpp/CMakeLists.txt +++ b/tests/ut/cpp/CMakeLists.txt @@ -434,6 +434,18 @@ add_a2a3_runtime_test(test_a2a3_scope_stats_collector a2a3/test_scope_stats_coll # --------------------------------------------------------------------------- add_a5_test(test_a5_fatal a5/test_a5_fatal.cpp) +# FDWIC's device-side PollBatch accumulator is header-only. Compile the exact +# production implementation through the A5Sim target macros so the 24-bit +# split boundary is exercised without introducing a second model. +add_a5_test(test_fdwic_swimlane_poll_batch a5/test_fdwic_swimlane_poll_batch.cpp) +target_compile_definitions(test_fdwic_swimlane_poll_batch PRIVATE __CPU_SIM=1) +target_include_directories(test_fdwic_swimlane_poll_batch BEFORE PRIVATE + ${CMAKE_SOURCE_DIR}/../../../src/a5/platform/sim/aicore + ${CMAKE_SOURCE_DIR}/../../../src/a5/runtime/fully_distributed_within_core/runtime + ${CMAKE_SOURCE_DIR}/../../../src/a5/runtime/fully_distributed_within_core/common + ${CMAKE_SOURCE_DIR}/../../../src/a5/runtime +) + # A5 trb runtime UTs — mirror of a2a3 trb runtime UTs, link against a5_rt_objs. # Target names carry the a5_ prefix because hierarchical/test_tensormap (and # the unprefixed a2a3 runtime targets test_scheduler_state / test_ready_queue diff --git a/tests/ut/cpp/a5/test_fdwic_swimlane_poll_batch.cpp b/tests/ut/cpp/a5/test_fdwic_swimlane_poll_batch.cpp new file mode 100644 index 0000000000..4d88d6cdbf --- /dev/null +++ b/tests/ut/cpp/a5/test_fdwic_swimlane_poll_batch.cpp @@ -0,0 +1,102 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the LICENSE file. + * ----------------------------------------------------------------------------------------------------------- + */ + +#include + +#include + +#include "inner_kernel.h" +#include "runtime.h" +#include "dist_engine/common/swimlane.h" + +namespace { + +class FdwicPollBatchTest : public ::testing::Test { +protected: + void SetUp() override { + core_ = {}; + records_[0] = {}; + records_[1] = {}; + g_self = reinterpret_cast(uintptr_t{1}); + g_fdwic_swimlane_level = kFdwicAtomicSwimlaneLevel; + g_fdwic_swimlane_core = &core_; + g_fdwic_swimlane_records = records_; + g_fdwic_swimlane_records_per_core = 2; + g_fdwic_atomic_poll_burst = {}; + g_fdwic_atomic_calls = 0; + g_fdwic_poll_calls = 0; + g_fdwic_poll_batch_records = 0; + g_fdwic_atomic_counter_overflow = false; + } + + void TearDown() override { + g_self = nullptr; + g_fdwic_swimlane_level = 0; + g_fdwic_swimlane_core = nullptr; + g_fdwic_swimlane_records = nullptr; + g_fdwic_swimlane_records_per_core = 0; + g_fdwic_atomic_poll_burst = {}; + } + + FdwicSwimlaneCoreState core_{}; + FdwicSwimlaneRecord records_[2]{}; +}; + +TEST_F(FdwicPollBatchTest, SplitsAtMaximum24BitCountAndReopensExactly) { + constexpr FdwicAtomicSite kSite = FdwicAtomicSite::StartupPoll; + constexpr uint32_t kBatchIndex = 0; + constexpr uint32_t kBatchBit = 1U << kBatchIndex; + constexpr uint64_t kFirstStart = 111; + constexpr uint64_t kSecondStart = 222; + constexpr uint64_t kSecondEnd = 333; + + ASSERT_EQ(fdwic_atomic_poll_batch_index(kSite), static_cast(kBatchIndex)); + g_fdwic_atomic_poll_burst.active_mask = kBatchBit; + g_fdwic_atomic_poll_burst.start_cycle[kBatchIndex] = kFirstStart; + g_fdwic_atomic_poll_burst.call_count[kBatchIndex] = kFdwicAtomicPollCountMax - 1; + + // The maximum-th call belongs to the first row and triggers an immediate + // flush. It must not be dropped or carried into the next batch. + fdwic_swimlane_accumulate_poll_call(kSite, kFirstStart); + + ASSERT_EQ(core_.count, 1U); + EXPECT_EQ(g_fdwic_poll_batch_records, 1U); + EXPECT_EQ(g_fdwic_atomic_poll_burst.active_mask, 0U); + EXPECT_EQ(g_fdwic_atomic_poll_burst.call_count[kBatchIndex], 0U); + EXPECT_EQ(records_[0].phase, static_cast(FdwicSwimlanePhase::Atomic)); + EXPECT_EQ(records_[0].aux, static_cast(kSite)); + EXPECT_EQ(records_[0].start_cycle, kFirstStart); + EXPECT_GE(records_[0].end_cycle, records_[0].start_cycle); + EXPECT_NE(records_[0].flags & kFdwicAtomicPollBatch, 0U); + EXPECT_EQ(records_[0].flags >> kFdwicAtomicPollCountShift, kFdwicAtomicPollCountMax); + + // The next call starts a fresh row at count one. Closing that row must + // preserve max+1 calls exactly across the two encoded records. + fdwic_swimlane_accumulate_poll_call(kSite, kSecondStart); + ASSERT_EQ(g_fdwic_atomic_poll_burst.active_mask, kBatchBit); + ASSERT_EQ(g_fdwic_atomic_poll_burst.call_count[kBatchIndex], 1U); + fdwic_atomic_poll_boundary_at(kSecondEnd); + + ASSERT_EQ(core_.count, 2U); + EXPECT_EQ(g_fdwic_poll_batch_records, 2U); + EXPECT_EQ(g_fdwic_atomic_poll_burst.active_mask, 0U); + EXPECT_EQ(g_fdwic_atomic_poll_burst.call_count[kBatchIndex], 0U); + EXPECT_EQ(records_[1].start_cycle, kSecondStart); + EXPECT_EQ(records_[1].end_cycle, kSecondEnd); + EXPECT_EQ(records_[1].flags >> kFdwicAtomicPollCountShift, 1U); + const uint64_t represented_calls = + (records_[0].flags >> kFdwicAtomicPollCountShift) + (records_[1].flags >> kFdwicAtomicPollCountShift); + EXPECT_EQ(represented_calls, static_cast(kFdwicAtomicPollCountMax) + 1); + EXPECT_EQ(core_.dropped, 0U); + EXPECT_FALSE(g_fdwic_atomic_counter_overflow); +} + +} // namespace diff --git a/tests/ut/py/test_fdwic_swimlane_converter.py b/tests/ut/py/test_fdwic_swimlane_converter.py new file mode 100644 index 0000000000..76aed58536 --- /dev/null +++ b/tests/ut/py/test_fdwic_swimlane_converter.py @@ -0,0 +1,411 @@ +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- +"""Atomic-event coverage for the shared FDWIC swimlane converter.""" + +import json + +import pytest + +from simpler_setup.tools.swimlane_converter import generate_chrome_trace_json, read_perf_data + + +def _capture( + rows, + *, + trace_schema_version=None, + num_cores=1, + add_clock_baselines=True, + clock_dependency_applied=True, +): + rows = list(rows) + if trace_schema_version == 3 and add_clock_baselines: + dependency_flags = 0x3 if clock_dependency_applied else 0x1 + for core_id in range(num_cores): + block_id, lane = divmod(core_id, 3) + start = 10 + 4 * core_id + rows.extend( + [ + [core_id, block_id, lane, -1, -1, "ClockBaseline", start, start + 1, 0, 0], + [core_id, block_id, lane, -1, -1, "ClockBaseline", start + 2, start + 3, dependency_flags, 0], + ] + ) + metadata = { + "clock_freq_hz": 1_000_000_000, + "num_cores": num_cores, + "core_types": ["aic" if core_id % 3 == 0 else "aiv" for core_id in range(num_cores)], + } + if trace_schema_version is not None: + metadata["trace_schema_version"] = trace_schema_version + if trace_schema_version == 3: + atomic_rows = [row for row in rows if row[5] == "Atomic"] + batch_rows = [row for row in atomic_rows if int(row[8]) & (1 << 7)] + batch_calls = sum((int(row[8]) >> 8) & 0xFFFFFF for row in batch_rows) + metadata["fdwic_summary"] = { + "records": len(rows), + "atomic_records": len(atomic_rows), + "clock_baseline_records": sum(row[5] == "ClockBaseline" for row in rows), + "atomic_calls": len(atomic_rows) - len(batch_rows) + batch_calls, + "batched_poll_calls": batch_calls, + "poll_batch_records": len(batch_rows), + "dropped_records": 0, + } + return { + "l2_swimlane_level": 4 if trace_schema_version == 3 else 1, + "metadata": metadata, + "aicore_tasks": [], + "aicpu_tasks": [], + "aicpu_scheduler_phases": [], + "aicpu_orchestrator_phases": [], + "fdwic_events": rows, + } + + +def _convert(tmp_path, capture, *, pass_metadata=True): + raw_path = tmp_path / "l2_swimlane_records.json" + merged_path = tmp_path / "merged_swimlane.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + data = read_perf_data(raw_path) + kwargs = {"fdwic_events": data.get("fdwic_events")} + if pass_metadata: + kwargs.update( + trace_schema_version=data["trace_schema_version"], + clock_freq_hz=data["clock_freq_hz"], + ) + generate_chrome_trace_json(data["tasks"], merged_path, **kwargs) + return data, json.loads(merged_path.read_text(encoding="utf-8"))["traceEvents"] + + +def test_atomic_and_clock_stay_on_scalar_lane_and_preserve_atomic_count(tmp_path): + rows = [ + [0, 0, 0, 7, -1, "Claim", 100, 200, 0x2, 0], + [0, 0, 0, 7, -1, "Atomic", 120, 160, 0x53, 15], + [0, 0, 0, 7, -1, "Atomic", 161, 170, 0x54, 25], + [1, 0, 1, -1, -1, "ClockBaseline", 101, 102, 0x3, 0], + [1, 0, 1, 7, 0, "Kernel", 140, 180, 0, 0], + ] + _, events = _convert(tmp_path, _capture(rows, trace_schema_version=2, num_cores=2)) + + thread_names = { + event["tid"]: event["args"]["name"] + for event in events + if event.get("ph") == "M" and event.get("name") == "thread_name" + } + assert thread_names[0] == "AIC (core0)" + assert thread_names[1] == "AIV0 (core1)" + assert thread_names[4] == "AIV0·kernel (core1)" + assert not any("·atomic" in name for name in thread_names.values()) + + atomic_events = [event for event in events if event.get("args", {}).get("phase") == "atomic"] + clock = next(event for event in events if event.get("cat") == "scalar_clock") + kernel = next(event for event in events if event.get("name") == "f0#7") + assert len(atomic_events) == sum(row[5] == "Atomic" for row in rows) + assert all(event["tid"] == 0 for event in atomic_events) + assert clock["tid"] == 1 + assert clock["args"]["ticks"] == 1 + assert clock["args"]["clock_freq_hz"] == 1_000_000_000 + assert kernel["tid"] == 4 + + fetch_sub = next(event for event in atomic_events if event["args"]["op"] == "fetch_sub") + assert fetch_sub["name"] == "atomic.return_ready.won_remaining_fetch_sub.fetch_sub#7" + assert fetch_sub["cat"] == "atomic.return_ready" + assert fetch_sub["args"]["site_id"] == 25 + assert fetch_sub["args"]["op_id"] == 4 + assert fetch_sub["args"]["cycles"] == 9 + assert fetch_sub["args"]["call_count"] == 1 + assert isinstance(fetch_sub["args"]["cycles"], int) + assert fetch_sub["args"]["execution_unit"] == "scalar" + + +@pytest.mark.parametrize( + ("site_id", "site_name", "op_id", "op_name"), + [ + (1, "startup_poll", 0, "load"), + (2, "fatal_poll", 0, "load"), + (5, "fanin_flag_load", 0, "load"), + (11, "heap_frontier_load", 0, "load"), + (12, "heap_vend_load", 0, "load"), + (14, "replay_done_poll", 0, "load"), + (21, "won_any_load", 0, "load"), + (22, "won_state_load", 0, "load"), + (23, "won_lane_claim_exchange", 1, "exchange"), + (27, "won_drained_load", 0, "load"), + ], +) +def test_v3_poll_batch_preserves_exact_call_count_without_fake_atomic_latency( + tmp_path, site_id, site_name, op_id, op_name +): + poll_count = 12345 + flags = (poll_count << 8) | 0x80 | 0x10 | op_id + rows = [[0, 0, 0, -1, -1, "Atomic", 100, 900, flags, site_id]] + + data, events = _convert(tmp_path, _capture(rows, trace_schema_version=3)) + + assert data["trace_schema_version"] == 3 + batch = next(event for event in events if event.get("cat") == "atomic.poll_batch") + assert batch["name"] == f"atomic.poll_batch.{site_name}.{op_name}×{poll_count}" + assert batch["tid"] == 0 + assert batch["args"]["call_count"] == poll_count + assert batch["args"]["phase"] == "atomic_poll_batch" + assert batch["args"]["is_poll_batch"] is True + expected_semantics = "idempotent_failed_exchange_retries" if site_id == 23 else "observation_load_calls" + assert batch["args"]["batch_semantics"] == expected_semantics + assert batch["args"]["duration_semantics"] == "logical_poll_episode_envelope_not_single_atomic_latency" + assert batch["args"]["may_contain_interleaved_direct_atomics"] is True + assert batch["args"]["poll_window_cycles"] == 800 + assert batch["args"]["estimate_formula"] == "call_count * calibrated_atomic_cost" + assert "cycles" not in batch["args"] + assert "completion_boundary" not in batch["args"] + assert "return_ready_observed" not in batch["args"] + + +def test_v3_poll_batch_accepts_maximum_24_bit_call_count(tmp_path): + poll_count = 0xFFFFFF + flags = (poll_count << 8) | 0x90 + data, events = _convert( + tmp_path, + _capture([[0, 0, 0, -1, -1, "Atomic", 100, 900, flags, 1]], trace_schema_version=3), + ) + + batch = next(event for event in events if event.get("cat") == "atomic.poll_batch") + assert batch["args"]["call_count"] == poll_count + assert data["fdwic_summary"]["atomic_calls"] == poll_count + + +@pytest.mark.parametrize( + ("trace_schema_version", "flags", "site"), + [ + (2, (7 << 8) | 0x90, 5), # schema v2 reserves bit 7 + (3, 0x90, 5), # zero call_count + (3, (7 << 8) | 0x91, 5), # Exchange is not a batchable observation load + (3, (7 << 8) | 0x90, 9), # frontier scans are not explicit wait-region polling + (3, (7 << 8) | 0x90, 23), # failed lane-claim retries must retain their Exchange op + (3, (7 << 8) | 0x93, 15), # WonSlot FetchMax is protocol-changing, not a retry batch + (3, (7 << 8) | 0xB0, 5), # batch has no single-load value_zero meaning + (3, (7 << 8) | 0xD0, 5), # batch has no return-ready boundary + ], +) +def test_poll_batch_rejects_invalid_schema_or_flags(tmp_path, trace_schema_version, flags, site): + capture = _capture( + [[0, 0, 0, -1, -1, "Atomic", 100, 110, flags, site]], + trace_schema_version=trace_schema_version, + ) + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match="invalid Atomic PollBatch"): + read_perf_data(raw_path) + + +@pytest.mark.parametrize( + ("flags", "site", "func_id"), + [ + (0x51, 4, -1), # ClaimMax is FetchMax, not Exchange. + (0x12, 0, -1), # StartupIncrement does not consume its FetchAdd result. + (0x42, 0, -1), # return_ready cannot exist without a consumed result. + (0x73, 4, -1), # value_zero is defined only for Load. + ((1 << 8) | 0x51, 23, -1), # retry payload is defined only for FetchMax. + (0x50, 28, -1), # Unknown site. + (0x53, 4, 0), # Atomic records never carry a func id. + ], +) +def test_v3_rejects_invalid_direct_atomic_schema(tmp_path, flags, site, func_id): + capture = _capture( + [[0, 0, 0, 7, func_id, "Atomic", 100, 110, flags, site]], + trace_schema_version=3, + ) + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match="invalid direct Atomic"): + read_perf_data(raw_path) + + +@pytest.mark.parametrize( + ("task_id", "func_id", "flags", "aux"), + [ + (-1, -1, 0x2, 0), # applied requires the dependency bit. + (-1, -1, 0x4, 0), # Unknown flag bit. + (0, -1, 0, 0), + (-1, 0, 0, 0), + (-1, -1, 0, 99), + ], +) +def test_v3_rejects_invalid_clock_baseline_schema(tmp_path, task_id, func_id, flags, aux): + capture = _capture( + [[0, 0, 0, task_id, func_id, "ClockBaseline", 100, 110, flags, aux]], + trace_schema_version=3, + add_clock_baselines=False, + ) + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match="invalid ClockBaseline"): + read_perf_data(raw_path) + + +@pytest.mark.parametrize(("start", "end"), [(-1, 10), (11, 10), (0, 1 << 64)]) +def test_v3_rejects_invalid_cycle_range(tmp_path, start, end): + capture = _capture( + [[0, 0, 0, 7, -1, "Atomic", start, end, 0x53, 4]], + trace_schema_version=3, + ) + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match="invalid cycle range"): + read_perf_data(raw_path) + + +@pytest.mark.parametrize("flags", [-1, 1 << 32]) +def test_v3_rejects_non_uint32_flags(tmp_path, flags): + capture = _capture( + [[0, 0, 0, 7, -1, "Atomic", 100, 110, flags, 4]], + trace_schema_version=3, + ) + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match="invalid uint32 flags"): + read_perf_data(raw_path) + + +@pytest.mark.parametrize( + ("clock_dependency_applied", "direct_flags"), + [ + (True, 0x53), # Real A5: consumed result has a return-ready boundary. + (False, 0x13), # A5Sim: source bracket only. + ], +) +def test_v3_direct_return_boundary_matches_per_core_clock_baseline( + tmp_path, clock_dependency_applied, direct_flags +): + capture = _capture( + [[0, 0, 0, 7, -1, "Atomic", 100, 110, direct_flags, 4]], + trace_schema_version=3, + clock_dependency_applied=clock_dependency_applied, + ) + data, _ = _convert(tmp_path, capture) + assert data["fdwic_summary"]["clock_baseline_records"] == 2 + + +def test_v3_rejects_direct_return_boundary_that_disagrees_with_clock_baseline(tmp_path): + capture = _capture( + [[0, 0, 0, 7, -1, "Atomic", 100, 110, 0x13, 4]], + trace_schema_version=3, + clock_dependency_applied=True, + ) + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match="does not match.*ClockBaseline"): + read_perf_data(raw_path) + + +def test_v3_requires_two_clock_baselines_per_core(tmp_path): + capture = _capture([], trace_schema_version=3, num_cores=2, add_clock_baselines=False) + capture["fdwic_events"] = [[0, 0, 0, -1, -1, "ClockBaseline", 10, 11, 0, 0]] + capture["metadata"]["fdwic_summary"]["records"] = 1 + capture["metadata"]["fdwic_summary"]["clock_baseline_records"] = 1 + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match="requires exactly one plain and one dependency ClockBaseline"): + read_perf_data(raw_path) + + +@pytest.mark.parametrize( + "summary_key", + [ + "records", + "atomic_records", + "clock_baseline_records", + "atomic_calls", + "batched_poll_calls", + "poll_batch_records", + "dropped_records", + ], +) +def test_v3_rejects_any_broken_weighted_summary_field(tmp_path, summary_key): + rows = [ + [0, 0, 0, -1, -1, "Atomic", 100, 200, (17 << 8) | 0x90, 1], + [0, 0, 0, 4, -1, "Atomic", 210, 220, 0x53, 4], + ] + capture = _capture(rows, trace_schema_version=3) + capture["metadata"]["fdwic_summary"][summary_key] += 1 + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match=rf"fdwic_summary\.{summary_key}"): + read_perf_data(raw_path) + + +def test_schema_v3_requires_level4_and_weighted_summary(tmp_path): + capture = _capture([[0, 0, 0, -1, -1, "Atomic", 100, 110, (3 << 8) | 0x90, 14]], trace_schema_version=3) + capture["l2_swimlane_level"] = 1 + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + with pytest.raises(ValueError, match="requires l2_swimlane_level=4"): + read_perf_data(raw_path) + + capture["l2_swimlane_level"] = 4 + del capture["metadata"]["fdwic_summary"] + raw_path.write_text(json.dumps(capture), encoding="utf-8") + with pytest.raises(ValueError, match="fdwic_summary is required"): + read_perf_data(raw_path) + + +def test_v2_claim_flags_encode_all_three_states(tmp_path): + rows = [ + [0, 0, 0, 1, -1, "Claim", 100, 110, 0x0, 0], + [0, 0, 0, 2, -1, "Claim", 120, 140, 0x2, 0], + [0, 0, 0, 3, 0, "Claim", 150, 180, 0x3, 1], + ] + data, events = _convert(tmp_path, _capture(rows, trace_schema_version=2)) + + assert data["trace_schema_version"] == 2 + by_name = {event.get("name"): event for event in events} + assert by_name["claim.not_attempted#1"]["args"]["claim_attempted"] is False + assert by_name["claim.lost#2"]["args"]["claim_attempted"] is True + assert by_name["claim.won#3"]["args"]["claim_won"] is True + assert all( + by_name[name]["args"]["claim_attempted_source"] == "raw_flag" + for name in ("claim.not_attempted#1", "claim.lost#2", "claim.won#3") + ) + + +@pytest.mark.parametrize("flags", [0x1, 0x4]) +def test_v2_rejects_invalid_claim_flags(tmp_path, flags): + capture = _capture( + [[0, 0, 0, 1, -1, "Claim", 100, 110, flags, 0]], + trace_schema_version=2, + ) + raw_path = tmp_path / "l2_swimlane_records.json" + raw_path.write_text(json.dumps(capture), encoding="utf-8") + + with pytest.raises(ValueError, match="invalid Claim flags"): + read_perf_data(raw_path) + + +def test_v1_claim_attempt_requires_contained_claim_max_evidence(tmp_path): + rows = [ + [0, 0, 0, 1, -1, "Claim", 100, 200, 0, 0], + [0, 0, 0, 1, -1, "Atomic", 120, 160, 0x53, 4], + [0, 0, 0, 2, -1, "Claim", 210, 230, 0, 0], + [0, 0, 0, 2, -1, "Atomic", 231, 240, 0x53, 4], + ] + data, events = _convert(tmp_path, _capture(rows), pass_metadata=False) + + assert data["trace_schema_version"] == 1 + by_name = {event.get("name"): event for event in events} + claim_max = next(event for event in events if event.get("args", {}).get("site_id") == 4) + assert claim_max["name"] == "atomic.return_ready.claim_max.fetch_max#1" + assert by_name["claim.lost#1"]["args"]["claim_attempted"] is True + assert by_name["claim.lost#1"]["args"]["claim_attempted_source"] == "contained_claim_max" + assert by_name["claim#2"]["args"]["claim_attempted"] is None + assert by_name["claim#2"]["args"]["claim_attempted_source"] == "unknown_v1_without_matching_claim_max" From 44199a54ae6709296495eef55183e0abdcc7512b Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 14:30:46 +0000 Subject: [PATCH 022/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E5=9B=BA?= =?UTF-8?q?=E5=8C=96Submit=E5=85=A8=E7=AA=97I-cache=E9=80=90=E6=A0=B8?= =?UTF-8?q?=E5=88=86=E6=9E=90=E5=8F=A3=E5=BE=84?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 从每轮96核原始PMU记录重算并校验ALL/AIC/AIV统计,聚合独立进程后优先报告AIV平均每核request、miss与p95。 明确区分90ns串行等效标尺和实际暴露损失,并记录同语义配对A/B才可用ΔSubmit与Δmiss完成性能归因。 --- ...05\345\206\265\345\210\206\346\236\220.md" | 71 +++ ...77\347\224\250\346\214\207\345\215\227.md" | 34 ++ .../pa_scheduler/pmu_sidecar_analyzer.py | 472 ++++++++++++++++++ .../pa_scheduler/test_pmu_sidecar_analyzer.py | 221 ++++++++ 4 files changed, 798 insertions(+) create mode 100644 tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py create mode 100644 tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 47649e5bc6..5d596771f7 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -1560,3 +1560,74 @@ repeat 都执行;repeat 完整性仍使用 CCEC count1→count2 engine PMU 精 RingBp。这说明 standalone 已达到“独立复现约 5 ms 调度”的目标;仍然保留 本文第 6 节的边界:它不依赖 simpler 生产代码,也不复刻真实 PA 数值数据流和 通用多 group/joint 拓扑。 + +#### 7.5.14 默认真负载的 Submit I-cache 基线 + +本节只回答一个目标:完整 Submit 控制流窗口内,平均单核、尤其 AIV 上发生多少 +I-cache miss,以及其中多少性能损失可以被当前证据支持。正式主指标按每轮角色 +`sum / cores` 计算后再取五轮中位数;逐核百分比不做算术平均。 + +默认负载切换完成后,以当前同一 CCEC ELF 显式指定 +`real-compute/constant/6,28,4,1`,关闭泳道和逐 atomic,采集 5 个独立 +`submit-all` PMU-only 进程。每轮均为 schema v3、96/96 trusted、32 AIC + +64 AIV、32 个完整 triplet;协议、真计算数值、Submit 内 placement/engine、 +start/stop、counter 门槛和 Restore 全部 PASS。新增的 +`pmu_sidecar_analyzer.py` 从 96 条 raw 独立重算 ALL/AIC/AIV 全字段,与 host +summary 五轮完全一致: + +| 轮次 | Submit span | request 总和 | miss 总和 | 总 miss rate | AIC / AIV miss rate | +| ---- | ----------: | -----------: | --------: | -------------: | --------------------: | +| 1 | 4,027.976 us | 45,857,671 | 5,908,019 | 12.8834% | 8.5382% / 15.0323% | +| 2 | 3,719.597 us | 45,498,660 | 5,896,760 | 12.9603% | 8.5318% / 15.1902% | +| 3 | 3,732.543 us | 45,520,846 | 5,909,025 | 12.9809% | 8.5716% / 15.2014% | +| 4 | 3,621.080 us | 45,603,314 | 5,897,686 | 12.9326% | 8.4889% / 15.1748% | +| 5 | 3,581.265 us | 45,551,060 | 5,894,904 | 12.9413% | 8.5307% / 15.1632% | + +五轮中位数为 Submit 3,719.597 us、request 45,551,060、miss 5,897,686, +总/AIC/AIV miss rate 为 12.9413%/8.5318%/15.1748%。request 全范围跨度约 +0.79%,miss 约 0.24%;相比 Submit span 和 scalar busy,I-cache 总量在这 +五轮更稳定,但这不取消正式 A/B 对多个独立进程和交错顺序的要求。 + +直接面向目标的逐核结果为:ALL 平均 61,434.229 miss/核,AIC 平均 +40,626.219 miss/核,AIV 平均 **71,865.516 miss/核**;AIV 的五轮范围为 +71,768.250~72,068.734 miss/核,逐核分布 p95 的五轮中位数为 73,035。 +AIV 平均 request 为 473,306/核,按组内 `sum(miss)/sum(request)` 得到上述 +15.1748%。由于 AIC/AIV 是明显双峰,ALL 的逐核 median 不用于替代分角色均值。 + +同一 ELF 又显式运行 3 个历史标定强度的 scalar-NOP 样本 +`129600,157900,79950,2400`。raw 重算中位数为 request 70,236,792、miss +5,942,635;与真计算相比,request 多约 35.1%,绝对 miss 只多约 0.76%。AIC/AIV +每核 miss 中位数分别约 40.2K/72.8K,真计算为 40.6K/71.9K。这个对照只支持: +当前约 5.9M 的 CNT7 总量对 winner 负载模式不敏感,更可能主要来自两种模式共用的 +调度控制流和代码布局;不能仅看真计算 miss rate 较高,就错误归因为真引擎造成更多 +miss。两组负载和调度到达不同,本轮也不是交错配对性能 A/B,故不使用其 Submit +差值宣称性能收益。 + +同一 ELF 的 3 个 `empty` 进程只在 RunScheduler 后打开空 gate。每轮 96 核 +owner/selector/start/stop/Restore 均闭环,total 每核中位数为 173~179;全核 +request/miss 中位数仅 958/444,分别约为正式 Submit 中位数的 0.0021%/0.0075%。 +empty 的 46% 左右 miss rate 来自极小分母,没有性能含义,也不从 Submit 中机械 +相减。 + +按 90 ns/miss 的受控 cold/warm 标尺,AIC/AIV 每核的串行等效量约为 +3.66/6.47 ms;AIV 结果甚至超过本轮约 3.72 ms 的 PMU-only Submit span,也超过 +熟悉的约 5 ms 泳道基线。这正面证明 miss 会在不同核间并行,在单核内也可能重叠, +并且受控 cold miss 与真实热路径 miss 不能假定为同一个可加常数。因此 6.47 ms +**不是实际损失**,当前 A5 PIPE_UTIL 事件表也没有已经核实的 I-cache stall-cycle +counter。实际暴露损失必须由同语义代码布局 A/B 同时给出 `ΔAIV miss/核` 和关闭 +PMU/泳道后的 `ΔSubmit span`;在该 A/B 完成前,本节把实际损失明确记为“尚未测得”, +不从总 miss 机械换算。 + +这里的 3.719597 ms 是本组 `--no-swimlane`、PMU gate 打开的五轮中位数;约 +5 ms 是标准泳道配置的熟悉基线,两者不是同一观察配置,不能直接相减。PMU +`submit-all` 是每 worker 从 orchestration 初始化前到本核最后一次 Submit 返回; +`submit_span_us` 则是 96 核最早的首个 `Submit.begin` 到最晚的末个 +`Submit.end`。二者都覆盖完整 Submit 控制流,但边界不完全相同。 + +本机未入库的原始 sidecar 位于: + +~~~text +outputs/pmu_submit_all_real_compute_b256_20260718T140539Z/run1.json ... run5.json +outputs/pmu_empty_real_compute_b256_20260718T140908Z/run1.json ... run3.json +outputs/pmu_submit_all_scalar_nop_b256_same_elf_20260718T141455Z/run1.json ... run3.json +~~~ diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index aff3238f68..628a2c9180 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -747,6 +747,40 @@ tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ 模式扩展成 Claim、EfDrain 等多个正式局部窗口。性能 A/B 仍要保持观察布局一致, 最终端到端收益以关闭 PMU 和泳道的独立进程结果为准。 +#### 校验并聚合多轮 PMU sidecar + +`pmu_sidecar_analyzer.py` 只读消费当前 schema-v3 JSON。它不信任单轮 host 已写好的 +summary,而是从每份文件的 worker raw 记录重新计算 ALL/AIC/AIV 的 +`sum/mean/median/p95/max` 与 `sum(miss)/sum(request)`;同时检查 accepted、 +96 核 start/stop、物理核唯一性、owner membership、角色、counter 门槛和 Restore。 +任一字段不一致即拒绝整组输入。 + +同一次构建、同一观察参数的多个独立进程可直接聚合: + +```bash +source /home/q00473782/.venv/bin/activate +python pmu_sidecar_analyzer.py "$OUT"/run*.json +python pmu_sidecar_analyzer.py --json "$OUT"/run*.json +``` + +工具把 device、batch、AIC/AIV 数、PMU window、selector、trace 配置和完整 +winner workload 纳入配置指纹。`submit-all` 与 `empty`、scalar-NOP 与 +real-compute 等不同口径不能混合聚合。当前 JSON 没有记录 ELF 内容哈希,所以 +调用者仍必须用独立输出目录隔离不同构建,不能只因 kernel 路径字符串相同就认为 +是同一二进制。 + +默认 `--icache-miss-ns 90` 只打印受控 cold/warm 标尺下的一阶 core-work 等效量。 +输出明确标记 `not_wall_or_additive_stall`:96 核总和不是 Submit 墙钟,逐核估算也 +可能因 miss 重叠、事件来源和真实层级差异超过窗口时间,不能据此做绝对减法。 +文本输出的 `[PRIMARY]` 以 AIV 平均 request/miss、AIV 逐核 p95 和组内 miss rate +为主;`[ACTUAL-EXPOSED-LOSS]` 在没有同语义配对 A/B 前固定报告 `UNMEASURED`, +避免把 90 ns 标尺误写成约 5 ms Submit 中已经暴露的损失。 +分析器回归可独立执行: + +```bash +python -m unittest -v test_pmu_sidecar_analyzer.py +``` + ## 6. 当前 A5 结果与真实 PA 的差异 2026-07-17 `scalar-nop` 阶段的一轮代表性结果如下。这是保留的历史 diff --git a/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py b/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py new file mode 100644 index 0000000000..1684250bbf --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py @@ -0,0 +1,472 @@ +#!/usr/bin/env python3 +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- +"""校验并聚合 standalone PA 调度器的多轮 PMU JSON sidecar。 + +Host 已为单轮输出 ALL/AIC/AIV summary;本工具从每轮 96 条 worker raw 记录重新 +计算同一组统计量,再聚合多个独立进程。它不修改采集文件,也不把 PMU raw total、 +scalar busy 或 I-cache miss 的一阶估算冒充 Submit 墙钟时间。 +""" + +from __future__ import annotations + +import argparse +import json +import math +import statistics +import sys +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Iterable, Sequence + + +SCHEMA_NAME = "pa_scheduler_pmu_phase_windows" +SCHEMA_VERSION = 3 +GROUP_NAMES = ("all", "aic", "aiv") +METRIC_NAMES = ( + "total_cycles", + "vector_busy", + "cube_busy", + "scalar_busy", + "mte1_busy", + "mte2_busy", + "mte3_busy", + "icache_requests", + "icache_misses", + "fix_busy", +) +SUMMARY_FIELDS = ("sum", "mean", "median", "p95", "max") + +# 这些字段决定两份 sidecar 是否属于同一观察配置。动态时间、capture id 和 +# placement 分布不在其中;它们正是多轮运行允许自然变化的结果。 +CONFIG_FINGERPRINT_FIELDS = ( + "device", + "batches", + "workers", + "aic_workers", + "aiv_workers", + "pmu_window", + "selectors", + "counter_width_bits", + "phase_timestamp_calls_present", + "phase_record_writes", + "profile_accumulation", + "trace_enabled", + "atomic_trace", + "gate_start_stop_have_pipe_all_barriers", + "winner_workload", +) + + +@dataclass(frozen=True) +class Capture: + """一份已通过 raw→summary 和采集门禁的 sidecar。""" + + path: Path + data: dict[str, Any] + groups: dict[str, list[dict[str, Any]]] + fingerprint: str + + +def _require(condition: bool, message: str) -> None: + if not condition: + raise ValueError(message) + + +def _integer(value: Any, label: str) -> int: + # bool 是 int 的子类,但 JSON true/false 不能静默成为 PMU counter。 + if isinstance(value, bool) or not isinstance(value, int): + raise ValueError(f"{label} must be an integer") + if value < 0: + raise ValueError(f"{label} must be non-negative") + return value + + +def _number(value: Any, label: str) -> float: + if isinstance(value, bool) or not isinstance(value, (int, float)): + raise ValueError(f"{label} must be numeric") + result = float(value) + if not math.isfinite(result): + raise ValueError(f"{label} must be finite") + return result + + +def _nearest_rank_p95(values: Sequence[int]) -> int: + # 与 CCEC host 使用相同的 nearest-rank 定义:ceil(0.95*N) 对应的顺序统计量。 + ordered = sorted(values) + return ordered[math.ceil(0.95 * len(ordered)) - 1] + + +def _metric_summary(values: Sequence[int]) -> dict[str, int | float]: + _require(bool(values), "cannot summarize an empty metric") + return { + "sum": sum(values), + "mean": sum(values) / len(values), + "median": statistics.median(values), + "p95": _nearest_rank_p95(values), + "max": max(values), + } + + +def _same_number(lhs: int | float, rhs: Any) -> bool: + try: + rhs_number = _number(rhs, "summary value") + except ValueError: + return False + return math.isclose(float(lhs), rhs_number, rel_tol=1e-12, abs_tol=1e-9) + + +def _configuration_fingerprint(configuration: dict[str, Any]) -> str: + selected = {field: configuration.get(field) for field in CONFIG_FINGERPRINT_FIELDS} + return json.dumps(selected, ensure_ascii=False, sort_keys=True, separators=(",", ":")) + + +def _validate_group_summary( + path: Path, + group_name: str, + records: Sequence[dict[str, Any]], + expected: Any, +) -> None: + _require(isinstance(expected, dict), f"{path}: summary.{group_name} must be an object") + _require( + expected.get("cores") == len(records), + f"{path}: summary.{group_name}.cores does not match raw records", + ) + _require( + expected.get("trusted_cores") == len(records), + f"{path}: summary.{group_name}.trusted_cores is incomplete", + ) + + for metric in METRIC_NAMES: + values = [ + _integer(record.get(metric), f"{path}: records[{index}].{metric}") + for index, record in enumerate(records) + ] + actual = _metric_summary(values) + reported = expected.get(metric) + _require( + isinstance(reported, dict), + f"{path}: summary.{group_name}.{metric} must be an object", + ) + for field in SUMMARY_FIELDS: + _require( + _same_number(actual[field], reported.get(field)), + f"{path}: raw summary mismatch at {group_name}.{metric}.{field}: " + f"raw={actual[field]!r} json={reported.get(field)!r}", + ) + + requests = sum(_integer(record["icache_requests"], "icache_requests") for record in records) + misses = sum(_integer(record["icache_misses"], "icache_misses") for record in records) + _require(requests > 0, f"{path}: summary.{group_name} has zero I-cache requests") + actual_rate = misses / requests + _require( + _same_number(actual_rate, expected.get("icache_miss_rate")), + f"{path}: raw summary mismatch at {group_name}.icache_miss_rate: " + f"raw={actual_rate!r} json={expected.get('icache_miss_rate')!r}", + ) + + +def load_capture(path: Path) -> Capture: + """读取并完整校验一份 schema-v3 PMU sidecar。""" + + with path.open("r", encoding="utf-8") as input_file: + data = json.load(input_file) + _require(isinstance(data, dict), f"{path}: capture root must be an object") + schema = data.get("schema") + _require( + schema == {"name": SCHEMA_NAME, "version": SCHEMA_VERSION}, + f"{path}: expected {SCHEMA_NAME} schema v{SCHEMA_VERSION}", + ) + + capture = data.get("capture") + configuration = data.get("configuration") + validation = data.get("validation") + records = data.get("records") + summary = data.get("summary") + _require(isinstance(capture, dict), f"{path}: capture must be an object") + _require(isinstance(configuration, dict), f"{path}: configuration must be an object") + _require(isinstance(validation, dict), f"{path}: validation must be an object") + _require(isinstance(records, list), f"{path}: records must be an array") + _require(isinstance(summary, dict), f"{path}: summary must be an object") + + workers = _integer(configuration.get("workers"), f"{path}: configuration.workers") + aic_workers = _integer(configuration.get("aic_workers"), f"{path}: configuration.aic_workers") + aiv_workers = _integer(configuration.get("aiv_workers"), f"{path}: configuration.aiv_workers") + _require(workers == aic_workers + aiv_workers, f"{path}: worker role counts do not add up") + _require(len(records) == workers, f"{path}: record count does not match configuration.workers") + + # JSON 只有在运行、PMU、owner Restore 和 runtime cleanup 全部成功后才应发布。 + # 分析器仍逐项重验,防止手工复制或未来 schema 退化绕过发布门禁。 + required_true = ( + (capture, "accepted"), + (capture, "published_after_runtime_cleanup"), + (capture, "runtime_cleanup_passed"), + (capture, "owner_restore_passed"), + (validation, "semantic_passed"), + (validation, "pmu_passed"), + (validation, "icache_measurement_valid"), + (validation, "icache_miss_le_request"), + (validation, "counter_below_risk_threshold"), + ) + for owner, field in required_true: + _require(owner.get(field) is True, f"{path}: {field} is not true") + + expected_records = ( + "trusted_records", + "unique_physical_core_ids", + "owner_bitmap_member_records", + "exact_worker_slot_records", + "physical_role_match_records", + "window_started_records", + "window_stopped_records", + ) + for field in expected_records: + _require( + validation.get(field) == workers, + f"{path}: validation.{field} is incomplete", + ) + + groups: dict[str, list[dict[str, Any]]] = { + "all": records, + "aic": [record for record in records if record.get("role") == "aic"], + "aiv": [record for record in records if record.get("role") == "aiv"], + } + _require(len(groups["aic"]) == aic_workers, f"{path}: AIC raw record count mismatch") + _require(len(groups["aiv"]) == aiv_workers, f"{path}: AIV raw record count mismatch") + + worker_ids: set[int] = set() + physical_core_ids: set[int] = set() + for index, record in enumerate(records): + _require(isinstance(record, dict), f"{path}: records[{index}] must be an object") + worker_id = _integer(record.get("worker_id"), f"{path}: records[{index}].worker_id") + physical_id = _integer( + record.get("physical_core_id"), f"{path}: records[{index}].physical_core_id" + ) + _require(worker_id not in worker_ids, f"{path}: duplicate worker_id {worker_id}") + _require( + physical_id not in physical_core_ids, + f"{path}: duplicate physical_core_id {physical_id}", + ) + worker_ids.add(worker_id) + physical_core_ids.add(physical_id) + for field in ( + "trusted", + "selectors_match", + "owner_bitmap_member", + "worker_slot_exact", + "physical_role_matches", + "window_started", + "window_stopped", + ): + _require(record.get(field) is True, f"{path}: records[{index}].{field} is not true") + requests = _integer( + record.get("icache_requests"), f"{path}: records[{index}].icache_requests" + ) + misses = _integer( + record.get("icache_misses"), f"{path}: records[{index}].icache_misses" + ) + _require(misses <= requests, f"{path}: records[{index}] has miss > request") + + for group_name in GROUP_NAMES: + _validate_group_summary(path, group_name, groups[group_name], summary.get(group_name)) + + return Capture(path, data, groups, _configuration_fingerprint(configuration)) + + +def _median(values: Iterable[int | float]) -> int | float: + return statistics.median(list(values)) + + +def analyze(paths: Sequence[Path], miss_penalty_ns: float = 90.0) -> dict[str, Any]: + """校验同配置多轮 sidecar,并返回可序列化的跨轮汇总。""" + + _require(bool(paths), "at least one PMU JSON path is required") + _require(math.isfinite(miss_penalty_ns) and miss_penalty_ns > 0, "miss penalty must be positive") + captures = [load_capture(path) for path in paths] + fingerprint = captures[0].fingerprint + for capture in captures[1:]: + _require( + capture.fingerprint == fingerprint, + f"{capture.path}: observation configuration differs from {captures[0].path}", + ) + + per_run: list[dict[str, Any]] = [] + for capture in captures: + configuration = capture.data["configuration"] + summary = capture.data["summary"] + row: dict[str, Any] = { + "path": str(capture.path), + "capture_id": capture.data["capture"].get("capture_id"), + "submit_span_us": _number( + configuration.get("submit_span_us"), + f"{capture.path}: configuration.submit_span_us", + ), + "groups": {}, + } + for group_name in GROUP_NAMES: + group = summary[group_name] + cores = _integer(group.get("cores"), f"summary.{group_name}.cores") + requests = _integer(group["icache_requests"].get("sum"), "icache request sum") + misses = _integer(group["icache_misses"].get("sum"), "icache miss sum") + row["groups"][group_name] = { + "cores": cores, + "icache_requests_sum": requests, + "icache_misses_sum": misses, + "icache_miss_rate": misses / requests, + "icache_requests_per_core": requests / cores, + "icache_misses_per_core": misses / cores, + "icache_misses_per_core_median": group["icache_misses"]["median"], + "icache_misses_per_core_p95": group["icache_misses"]["p95"], + "first_order_miss_core_equivalent_us": misses * miss_penalty_ns / 1000.0, + "first_order_miss_per_core_us": misses * miss_penalty_ns / cores / 1000.0, + "scalar_busy_sum": group["scalar_busy"]["sum"], + "total_cycles_sum": group["total_cycles"]["sum"], + } + per_run.append(row) + + aggregate: dict[str, Any] = { + "runs": len(per_run), + "submit_span_us": { + "median": _median(row["submit_span_us"] for row in per_run), + "min": min(row["submit_span_us"] for row in per_run), + "max": max(row["submit_span_us"] for row in per_run), + }, + "groups": {}, + } + aggregate_fields = ( + "icache_requests_sum", + "icache_misses_sum", + "icache_miss_rate", + "icache_requests_per_core", + "icache_misses_per_core", + "icache_misses_per_core_median", + "icache_misses_per_core_p95", + "first_order_miss_core_equivalent_us", + "first_order_miss_per_core_us", + "scalar_busy_sum", + "total_cycles_sum", + ) + for group_name in GROUP_NAMES: + aggregate["groups"][group_name] = { + field: { + "median": _median(row["groups"][group_name][field] for row in per_run), + "min": min(row["groups"][group_name][field] for row in per_run), + "max": max(row["groups"][group_name][field] for row in per_run), + } + for field in aggregate_fields + } + + configuration = captures[0].data["configuration"] + return { + "schema": {"name": "pa_scheduler_pmu_multi_run_summary", "version": 1}, + "input_schema": {"name": SCHEMA_NAME, "version": SCHEMA_VERSION}, + "configuration": { + field: configuration.get(field) for field in CONFIG_FINGERPRINT_FIELDS + }, + "estimation": { + "icache_miss_penalty_ns": miss_penalty_ns, + "meaning": "controlled_cold_warm_first_order_core_equivalent", + "not_wall_time": True, + "not_additive_stall_time": True, + }, + "actual_exposed_loss": { + "status": "requires_same_semantics_paired_ab", + "reason": "A5 submit-all has no verified I-cache stall-cycle counter", + "required_observations": ("delta_submit_span_us", "delta_icache_misses_per_core"), + }, + "validation": { + "raw_to_summary_all_fields_match": True, + "all_inputs_accepted_and_restored": True, + "same_observation_configuration": True, + }, + "per_run": per_run, + "aggregate": aggregate, + } + + +def _print_text(result: dict[str, Any]) -> None: + configuration = result["configuration"] + workload = configuration.get("winner_workload") or {} + counts = workload.get("counts") or {} + print( + "[CONFIG] " + f"window={configuration.get('pmu_window')} batches={configuration.get('batches')} " + f"workers={configuration.get('workers')} workload={workload.get('mode')} " + f"counts={counts.get('qk')},{counts.get('sf')},{counts.get('pv')},{counts.get('up')}" + ) + print("[VALIDATION] raw_to_summary=PASS accepted_restore=PASS same_configuration=PASS") + print( + "run submit_us all_miss/core all_rate " + "aic_miss/core aic_rate aiv_miss/core aiv_rate" + ) + for index, row in enumerate(result["per_run"], start=1): + all_group = row["groups"]["all"] + aic = row["groups"]["aic"] + aiv = row["groups"]["aiv"] + print( + f"{index:>3} {row['submit_span_us']:>9.3f} " + f"{all_group['icache_misses_per_core']:>13.3f} " + f"{all_group['icache_miss_rate'] * 100:>7.4f}% " + f"{aic['icache_misses_per_core']:>13.3f} {aic['icache_miss_rate'] * 100:>7.4f}% " + f"{aiv['icache_misses_per_core']:>14.3f} {aiv['icache_miss_rate'] * 100:>7.4f}%" + ) + + aggregate = result["aggregate"] + all_group = aggregate["groups"]["all"] + aic = aggregate["groups"]["aic"] + aiv = aggregate["groups"]["aiv"] + print( + "[PRIMARY] " + f"submit_us={aggregate['submit_span_us']['median']:.3f} " + f"all_miss_per_core={all_group['icache_misses_per_core']['median']:.3f} " + f"AIC_miss_per_core={aic['icache_misses_per_core']['median']:.3f} " + f"AIV_request_per_core={aiv['icache_requests_per_core']['median']:.3f} " + f"AIV_miss_per_core={aiv['icache_misses_per_core']['median']:.3f} " + f"AIV_core_miss_p95={aiv['icache_misses_per_core_p95']['median']:.3f} " + f"AIV_miss_rate={aiv['icache_miss_rate']['median'] * 100:.4f}%" + ) + print( + "[SERIAL-EQUIVALENT] " + f"penalty={result['estimation']['icache_miss_penalty_ns']:.3f}ns/miss " + f"AIC_per_core_us={aic['first_order_miss_per_core_us']['median']:.3f} " + f"AIV_per_core_us={aiv['first_order_miss_per_core_us']['median']:.3f} " + "meaning=core_equivalent_not_wall_or_additive_stall" + ) + print( + "[ACTUAL-EXPOSED-LOSS] status=UNMEASURED " + "method=requires_same_semantics_paired_AB_delta_submit_and_delta_miss" + ) + + +def main(argv: Sequence[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("inputs", nargs="+", type=Path, help="schema-v3 PMU JSON sidecars") + parser.add_argument( + "--icache-miss-ns", + type=float, + default=90.0, + help="受控 cold/warm 标尺;只用于一阶 core-work 等效估算(默认 90)", + ) + parser.add_argument("--json", action="store_true", help="输出机器可读的聚合 JSON") + arguments = parser.parse_args(argv) + try: + result = analyze(arguments.inputs, arguments.icache_miss_ns) + except (OSError, ValueError, json.JSONDecodeError) as error: + print(f"PMU sidecar analysis failed: {error}", file=sys.stderr) + return 1 + if arguments.json: + json.dump(result, sys.stdout, ensure_ascii=False, indent=2) + print() + else: + _print_text(result) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py b/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py new file mode 100644 index 0000000000..94d699522e --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py @@ -0,0 +1,221 @@ +#!/usr/bin/env python3 +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +"""standalone PMU sidecar 多轮分析器的 raw 门禁与聚合回归。""" + +from __future__ import annotations + +import copy +import json +import math +import tempfile +import unittest +from pathlib import Path +from typing import Any, Sequence + +try: + from .pmu_sidecar_analyzer import METRIC_NAMES, analyze, load_capture +except ImportError: + from pmu_sidecar_analyzer import METRIC_NAMES, analyze, load_capture + + +def _p95(values: Sequence[int]) -> int: + ordered = sorted(values) + return ordered[math.ceil(0.95 * len(ordered)) - 1] + + +def _summary(records: list[dict[str, Any]]) -> dict[str, Any]: + result: dict[str, Any] = { + "cores": len(records), + "active_cores": len(records), + "trusted_cores": len(records), + } + for metric in METRIC_NAMES: + values = [record[metric] for record in records] + result[metric] = { + "sum": sum(values), + "mean": sum(values) / len(values), + "median": sorted(values)[len(values) // 2] + if len(values) % 2 + else (sorted(values)[len(values) // 2 - 1] + sorted(values)[len(values) // 2]) / 2, + "p95": _p95(values), + "max": max(values), + } + result["icache_miss_rate"] = ( + result["icache_misses"]["sum"] / result["icache_requests"]["sum"] + ) + return result + + +def _capture(offset: int = 0, window: str = "submit-all") -> dict[str, Any]: + records: list[dict[str, Any]] = [] + for worker_id, role in enumerate(("aic", "aiv", "aiv")): + base = 100 + worker_id * 10 + offset + record: dict[str, Any] = { + "worker_id": worker_id, + "physical_core_id": 10 + worker_id, + "role": role, + "trusted": True, + "selectors_match": True, + "owner_bitmap_member": True, + "worker_slot_exact": True, + "physical_role_matches": True, + "window_started": True, + "window_stopped": True, + } + for metric_index, metric in enumerate(METRIC_NAMES): + record[metric] = base + metric_index + # miss/request 取独立、直观的数值,便于断言聚合公式。 + record["icache_requests"] = 1000 + base + record["icache_misses"] = 100 + base // 10 + records.append(record) + + groups = { + "all": records, + "aic": [record for record in records if record["role"] == "aic"], + "aiv": [record for record in records if record["role"] == "aiv"], + } + return { + "schema": {"name": "pa_scheduler_pmu_phase_windows", "version": 3}, + "capture": { + "capture_id": f"capture-{offset}", + "accepted": True, + "published_after_runtime_cleanup": True, + "runtime_cleanup_passed": True, + "owner_restore_passed": True, + }, + "configuration": { + "device": 0, + "batches": 256, + "workers": 3, + "aic_workers": 1, + "aiv_workers": 2, + "pmu_window": window, + "submit_span_us": 5000.0 + offset, + "selectors": {"cnt6_icache_request": 0x034, "cnt7_icache_miss": 0x035}, + "counter_width_bits": {"total": 64, "programmable": 32}, + "phase_timestamp_calls_present": True, + "phase_record_writes": False, + "profile_accumulation": False, + "trace_enabled": False, + "atomic_trace": False, + "gate_start_stop_have_pipe_all_barriers": True, + "winner_workload": { + "mode": "real-compute", + "counts": {"qk": 6, "sf": 28, "pv": 4, "up": 1}, + "unit": "complete_128x128_engine_pipeline_iteration", + }, + }, + "validation": { + "semantic_passed": True, + "pmu_passed": True, + "icache_measurement_valid": True, + "icache_miss_le_request": True, + "counter_below_risk_threshold": True, + "trusted_records": 3, + "unique_physical_core_ids": 3, + "owner_bitmap_member_records": 3, + "exact_worker_slot_records": 3, + "physical_role_match_records": 3, + "window_started_records": 3, + "window_stopped_records": 3, + }, + "records": records, + "summary": {name: _summary(group) for name, group in groups.items()}, + } + + +class PmuSidecarAnalyzerTest(unittest.TestCase): + def _write(self, directory: str, name: str, capture: dict[str, Any]) -> Path: + path = Path(directory) / name + path.write_text(json.dumps(capture), encoding="utf-8") + return path + + def test_valid_raw_is_recomputed_and_multiple_runs_are_aggregated(self) -> None: + with tempfile.TemporaryDirectory() as directory: + first = self._write(directory, "first.json", _capture(0)) + second = self._write(directory, "second.json", _capture(20)) + result = analyze([first, second], miss_penalty_ns=90.0) + + self.assertTrue(result["validation"]["raw_to_summary_all_fields_match"]) + self.assertEqual(result["aggregate"]["runs"], 2) + self.assertEqual(result["aggregate"]["submit_span_us"]["median"], 5010.0) + expected_misses = [ + _capture(offset)["summary"]["all"]["icache_misses"]["sum"] + for offset in (0, 20) + ] + self.assertEqual( + result["aggregate"]["groups"]["all"]["icache_misses_sum"]["median"], + sum(expected_misses) / 2, + ) + self.assertTrue(result["estimation"]["not_wall_time"]) + self.assertTrue(result["estimation"]["not_additive_stall_time"]) + self.assertEqual( + result["actual_exposed_loss"]["status"], + "requires_same_semantics_paired_ab", + ) + expected_aiv_misses_per_core = [ + _capture(offset)["summary"]["aiv"]["icache_misses"]["sum"] / 2 + for offset in (0, 20) + ] + self.assertEqual( + result["aggregate"]["groups"]["aiv"]["icache_misses_per_core"]["median"], + sum(expected_aiv_misses_per_core) / 2, + ) + + def test_tampered_host_summary_is_rejected(self) -> None: + capture = _capture() + capture["summary"]["aiv"]["icache_misses"]["sum"] += 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "tampered.json", capture) + with self.assertRaisesRegex(ValueError, "raw summary mismatch"): + load_capture(path) + + def test_different_observation_configurations_cannot_be_merged(self) -> None: + with tempfile.TemporaryDirectory() as directory: + first = self._write(directory, "submit.json", _capture(window="submit-all")) + second = self._write(directory, "empty.json", _capture(window="empty")) + with self.assertRaisesRegex(ValueError, "observation configuration differs"): + analyze([first, second]) + + def test_failed_restore_is_rejected(self) -> None: + capture = _capture() + capture["capture"]["owner_restore_passed"] = False + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "restore_failed.json", capture) + with self.assertRaisesRegex(ValueError, "owner_restore_passed is not true"): + load_capture(path) + + def test_duplicate_physical_core_is_rejected(self) -> None: + capture = _capture() + capture["records"][1]["physical_core_id"] = capture["records"][0]["physical_core_id"] + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "duplicate.json", capture) + with self.assertRaisesRegex(ValueError, "duplicate physical_core_id"): + load_capture(path) + + def test_miss_greater_than_request_is_rejected_before_summary_use(self) -> None: + capture = _capture() + capture["records"][0]["icache_misses"] = capture["records"][0]["icache_requests"] + 1 + # 同步重算 summary,证明失败来自逐核物理约束,而不是 summary 不一致。 + records = capture["records"] + capture["summary"] = { + "all": _summary(records), + "aic": _summary([record for record in records if record["role"] == "aic"]), + "aiv": _summary([record for record in records if record["role"] == "aiv"]), + } + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "invalid_miss.json", capture) + with self.assertRaisesRegex(ValueError, "miss > request"): + load_capture(path) + + +if __name__ == "__main__": + unittest.main() From 187e54bcdbc77fba242d26ce78a03f4ccfcc3957 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 15:57:00 +0000 Subject: [PATCH 023/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E5=AE=8C?= =?UTF-8?q?=E5=96=84standalone=20atomic=E5=90=88=E5=B9=B6=E6=B3=B3?= =?UTF-8?q?=E9=81=93?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 对齐真实PA的15个atomic site及return-ready/source-issue语义 - 将六类显式等待轮询聚合为携精确次数的PollBatch - 在phase、lap和Kernel边界关闭等待包并补齐96核计数闭环 - 使用仅GM指针与标量的非内联落盘函数,避免CCEC栈对象跨调用损坏 - 增加24位拆批、嵌套region、allowlist、拓扑与raw转换回归 - A5 b1/b256语义、计数与真实计算校验通过,b256 Submit为5.774 ms --- ...05\345\206\265\345\210\206\346\236\220.md" | 240 +++++++--- ...77\347\224\250\346\214\207\345\215\227.md" | 212 +++++++-- .../pa_scheduler/ascendc/pa_scheduler.asc | 2 + tests/atomic_probe/pa_scheduler/ccec/build.sh | 67 ++- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 20 + .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 26 +- .../pa_scheduler/common/host_support.h | 388 ++++++++++++++--- .../pa_scheduler/common/pa_model.h | 121 +++++- .../pa_scheduler/common/pa_scheduler_core.h | 129 ++++-- .../pa_scheduler/common/pa_trace.h | 290 ++++++++++++- .../common/test_atomic_poll_batch.cpp | 280 ++++++++++++ tests/atomic_probe/pa_scheduler/cpu/build.sh | 12 + tests/atomic_probe/pa_scheduler/cpu/main.cpp | 9 +- tests/atomic_probe/pa_scheduler/run.sh | 6 +- .../pa_scheduler/swimlane_converter.py | 321 ++++++++++++-- .../pa_scheduler/test_swimlane_converter.py | 409 ++++++++++++++++++ 16 files changed, 2244 insertions(+), 288 deletions(-) create mode 100644 tests/atomic_probe/pa_scheduler/common/test_atomic_poll_batch.cpp diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 5d596771f7..2ee1ca2673 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -1012,26 +1012,28 @@ frontier helping 的额外 FetchMax 中位数为 `A-T=14085` 次。ready-prefix 结果并撤回,不迁移真实 FDWIC。standalone 第一版只验证当前单-lane PA Case1, 不能拿它的 PASS 代替 joint/BlockWon 覆盖。 -### 7.5 阶段 O2:逐 atomic 泳道与 Submit scalar PMU +### 7.5 阶段 O2:atomic schema-v3 泳道与 Submit scalar PMU #### 7.5.1 观察目标与证据拆分 O2 先完成观察链路,不在同一阶段继续消减 atomic。它回答两个不同问题: -1. 逐 atomic 泳道回答“哪一个源码 atomic 调用发生在何时、属于哪个 site/op、 - 该调用按源码语义是 source-issue 还是本核 return-ready 边界、bracket 多宽”; +1. atomic 泳道回答“每个 site/op 执行了多少次逻辑调用”:direct Atomic 逐条给出 + source-issue 或本核 return-ready bracket;显式等待区内六类 observation load + 用带精确 `call_count` 的 PollBatch 给出逻辑轮询 episode; 2. PMU sidecar 回答“指定 Submit 窗口内,每个物理子核累计了多少 scalar/vector/cube/MTE/fix busy、I-cache request/miss 和 raw total”。 -两类数据默认分开采集。逐 atomic trace 会增加两次 `get_sys_cnt()`、分支和一条 -64 B 私有 record 写入,进而改变代码布局、I-cache、worker 到达顺序、共享地址竞争 -和轮询次数。PMU-only 不写完整泳道记录,观察机制与扰动来源不同;由于 +两类数据默认分开采集。direct trace 会增加两次 `get_sys_cnt()`、分支和一条 +64 B 私有 record 写入,PollBatch 还会增加等待区内的私有累计与边界落盘;二者都会 +改变代码布局、I-cache、worker 到达顺序、共享地址竞争和轮询次数。PMU-only 不写 +完整泳道记录,观察机制与扰动来源不同;由于 `submit-all` gate 自带 `PIPE_ALL`,当前没有同配置 A/B 证明其总扰动一定更小。 它用于取 AIC/AIV 平均和 PMU event A/B。正式 PMU JSON 强制 `--no-swimlane`,不与 atomic trace 或 phase profile 合并; 两套结果只按同一源码版本和各自明确的 scope 交叉解释,不能声称逐 tick 精确对齐。 -#### 7.5.2 十五个 site 与四种 op +#### 7.5.2 十五个 site、四种 op 与六类 PollBatch `AtomicSite` 是 raw/merged trace 的稳定编号,当前覆盖 standalone PA 公共调度器中 所有显式共享 atomic 源码调用点。`AtomicOp` 只有 `Load/Exchange/FetchAdd/FetchMax` @@ -1055,27 +1057,54 @@ O2 先完成观察链路,不在同一阶段继续消减 atomic。它回答两 | 13 | `ReplayDoneIncrement` | `FetchAdd` | worker 回放结束计数发布 | | 14 | `ReplayDonePoll` | `Load` | 最终 drain 的 replay_done 轮询 | -每次源码调用只生成一条同时包含 `start_cycle/end_cycle` 的 `Atomic` span,避免用 -start/end 两条记录把容量再放大一倍。raw schema 使用: - -- `auxiliary`:上表 site id; -- `flags[3:0]`:op id;bit 4:返回的旧值是否参与后续判断; -- 对 `Load`,bit 5 表示观察值是否为零; -- bit 6 表示本条结束时刻是否有“返回值本核可消费”依赖证据; -- 对 `FetchMax`,bits 8..31 保存饱和后的软件 retry 数;A5 单条硬件 - `atomicMax` 当前为 0,CPU CAS 回归可非零; -- `task_id`:能归属任务时写真实 task id,生命周期 atomic 写 `-1`。 - -converter 把这些事件放在对应 AIC/AIV 的原 scalar lane,atomic 不再 -伪装成与 scalar 并行的独立执行单元。名称显式带边界: +schema-v3 要求 `metadata.trace_schema_version=3` 且 `l2_swimlane_level=4`,并将 +**逻辑调用数**与**物理记录数**分开。direct Atomic 仍是一条源码调用 +对应一条同时包含 `start_cycle/end_cycle` 的记录;只有下列六类 observation load +在匹配的显式等待区内才允许聚合为 PollBatch: + +| `site_id` | site | op | +| ---: | --- | --- | +| 1 | `startup_poll` | `Load` | +| 2 | `fatal_poll` | `Load` | +| 5 | `fanin_flag_load` | `Load` | +| 11 | `heap_frontier_load` | `Load` | +| 12 | `heap_vend_load` | `Load` | +| 14 | `replay_done_poll` | `Load` | + +standalone 没有真实 PA 追加的 BlockWon site,也没有允许聚合的幂等失败 exchange; +不能照搬真实 PA 的“九类 observation load 加一类 exchange”allowlist。同一 site 在 +显式等待区外的一次性或 opportunistic 读取仍逐条记录。 + +raw 的 `auxiliary` 保存 site id。direct Atomic 的 `flags` 约定为:低 4 bit 是 op id, +bit 4 表示返回旧值是否参与后续判断,bit 5 仅对 Load 表示观察值是否为零,bit 6 +表示是否有“返回值本核可消费”依赖证据,bit 7 必须为 0;bits 8..31 仅对 direct +FetchMax 保存软件 retry 数,不能按调用次数解析。能归属任务时 `task_id` 写真实 +task id,生命周期 atomic 写 `-1`。 + +PollBatch 的 `flags` 则必须同时满足:低 4 bit 为 `Load(0)`、bit 4 为 1、bit 5/6 +为 0、bit 7 为 1,bits 8..31 保存 `1..0xFFFFFF` 的精确无符号 24 bit +`call_count`;`task_id=-1`、`function_id=-1`。达到 `0xFFFFFF` 时先落盘,再从 +1 开启下一条 batch,不能饱和后丢计数。 + +converter 把 direct Atomic、PollBatch 和 ClockBaseline 放在对应 AIC/AIV 的原 +scalar lane,atomic 不再伪装成与 scalar 并行的独立执行单元。direct 名称显式带边界: `atomic.return_ready..#` 或 `atomic.source_issue..#`;category 也分别为 -`atomic.return_ready`/`atomic.source_issue`,无需点开 span 即可过滤区分。args 中 -保留整数 `cycles`、site/op、 +`atomic.return_ready`/`atomic.source_issue`,无需点开 span 即可过滤区分。direct +args 中保留 `call_count=1`、整数 `cycles`、site/op、 `result_used`、`return_ready_observed`、`completion_boundary`、Load 的 `value_zero` 和 FetchMax 的 `retries`。Perfetto 的浮点微秒显示不用于 替代 raw 整数 tick。 +PollBatch 名称为 `atomic.poll_batch..load×`,category 为 +`atomic.poll_batch`;args 还明确给出 `poll_window_cycles`、 +`batch_semantics=observation_load_calls`、 +`duration_semantics=logical_poll_episode_envelope_not_single_atomic_latency` 和 +`may_contain_interleaved_direct_atomics=true`。一个等待区可以同时累计多个 site, +所以不同 PollBatch 窗口可以重叠,窗口内也可能交错 direct Atomic。 +host 文字分析也保持两套口径:`[TRACE_ATOMIC]` 只统计 direct bracket, +`[TRACE_ATOMIC_POLL]` 只统计 episode 数、精确逻辑调用数和等待包络分布。 + #### 7.5.3 按调用点语义区分的两种结束边界 不能按 `Exchange/FetchAdd/FetchMax` 指令名称一概选边界,必须看该源码 @@ -1127,11 +1156,29 @@ PA 的端到端 GAP 是“发布发射 -> 自然消费者首次观察到新值 复用现有 load 记录做离线派生,但其包含互连可见性、消费者调度与轮询 间隔,且在对外报告跨核时间差前还需单独验证各核 `SYS_CNT` 的对齐性。 -两种边界的 `end` 都在本条 64 B trace record 写入之前取得,所以本条 -duration 不直接包含自己的 record 写入;但这次写入、附加指令和代码布局 -会影响后续 atomic 到达、竞争与 I-cache,整轮仍是插桩运行。 - -开启逐 atomic trace 时,每个 worker 在最终 drain 之后额外记录两条 +两种 direct 边界的 `end` 都在本条 64 B trace record 写入之前取得,所以本条 +duration 不直接包含自己的 record 写入;但这次写入、附加指令和代码布局会影响 +后续 atomic 到达、竞争与 I-cache,整轮仍是插桩运行。direct record 写入不会隐式 +关闭活跃 PollBatch,否则等待区内自然交错的发布/推进 atomic 会把一个逻辑等待 +episode 人为切碎。 + +PollBatch 使用另一种时间语义:`duration`/`poll_window_cycles` 是从该 site 在显式 +等待区内首次累计调用到边界关闭的逻辑等待 episode 包络。它不是独占 scalar 时间, +不是 `call_count` 次 atomic 延迟之和,也不是其中某次 load 的 return-ready 延迟; +不同 site 的窗口可以重叠,窗口内还可能包含 direct Atomic,不能把 PollBatch 混入 +direct 单次延迟的 median/p95。 + +边界实现复用真实 PA 已验证的规则,并让 phase/lap 与 batch 使用同一次 cycle 采样: + +1. 显式等待区退出时关闭与该 region 匹配的 PollBatch; +2. `TraceTimestamp` 先采 cycle,再以该 cycle 关闭全部活跃 batch,然后写 phase + begin/end; +3. `ResetTraceLap` 在推进 lap 起点前关闭,`WriteTraceLap` 在写 lap 前用同一 + `end_cycle` 关闭; +4. Kernel begin/end 都经过 `TraceTimestamp`,所以 PollBatch 不能跨入或跨出 Kernel; +5. `FlushTraceCore` 的最终关闭只作防御性兜底,不能代替上述语义边界。 + +开启 atomic trace 时,每个 worker 在最终 drain 之后额外记录两条 `ClockBaseline`:一条是连续两次 `get_sys_cnt()`,另一条是纯寄存器依赖 hook 后读 `SYS_CNT`。它们只给出同一二进制、同一物理核上两类边界的 计时分辨率和固定底噪分布,使用边界是: @@ -1143,17 +1190,37 @@ hook 后读 `SYS_CNT`。它们只给出同一二进制、同一物理核上两 #### 7.5.4 计数闭环与容量门禁 -逐 atomic 结果只有同时满足以下闭环才可进入正式分析: - -1. 每个 worker 的 raw `Atomic` 记录数等于其独占 - `WorkerResult.atomic_trace_calls`; -2. 全局 raw `Atomic` 记录数等于 96 个 worker 的软件计数之和; -3. 每个 worker 恰有 2 条 `ClockBaseline`(连续时钟与返回依赖 hook 各一条), - 因此全局固定为 192 条; -4. 原阶段记录、动态 wait 记录、atomic 记录和 192 条 ClockBaseline 的总和与 - trace header count 精确相等; -5. 每核 `dropped==0`,且 raw 到 merged 后逐 atomic 事件数不变; -6. 原有 Claim、winner、kernel、fanin、frontier、cursor、heap 和每 worker +schema-v3 结果只有同时满足逻辑调用与物理记录闭环才可进入正式分析。设 +`direct_atomic_records` 是 bit 7 为 0 的物理 Atomic 条数,则逐核和全局都必须满足: + +~~~text +logical_atomic_calls = direct_atomic_records + Σ(PollBatch.call_count) +physical_atomic_records = direct_atomic_records + poll_batch_records +physical_atomic_records + = logical_atomic_calls - batched_poll_calls + poll_batch_records +~~~ + +这里 `batched_poll_calls` 是所有 PollBatch `call_count` 之和,不是 batch 条数。 +producer 的逐核 state、host 扫描 raw 行、导出的 metadata 和 converter 重算必须同时 +闭合以下七项: + +~~~text +records +atomic_records +clock_baseline_records +atomic_calls +batched_poll_calls +poll_batch_records +dropped_records +~~~ + +此外还必须满足: + +1. 每条 PollBatch 都属于六类 allowlist,flags 合法且 `call_count>0`; +2. 每个 worker 恰有 2 条 ClockBaseline,因此 96 核全局固定为 192 条; +3. 原 phase、动态 wait、物理 Atomic 和 ClockBaseline 总数与 trace header 精确相等; +4. 每核和全局 `dropped_records==0`,raw 到 merged 后物理 Atomic 条数不变; +5. 原有 Claim、winner、kernel、fanin、frontier、cursor、heap 和每 worker 1280 Submit 语义断言仍全部 PASS。 trace 容量不足时必须明确报 overflow 并判该轮观察无效,不能只分析前缀,也不能 @@ -1232,25 +1299,60 @@ JSON/同名 `.tmp`,并只在协议、PMU/owner 门禁、Restore 和 runtime `SYS_CNT` 调用点仍存在;sidecar 会明确记录“有 timestamp call、无 record write、 无 atomic trace、无 profile accumulation”,不能把该模式描述成编译期零插桩。 -#### 7.5.6 当前证据状态与正式重采矩阵 +#### 7.5.6 当前实现状态、历史证据与正式重采矩阵 + +当前源码已经实现 schema-v3 的六类 PollBatch、精确 `call_count`、七项 summary +闭环、两条 ClockBaseline、scalar lane converter,以及 phase/lap/Kernel 共 cycle +边界关闭。2026-07-18 已用边界修复版 CCEC 完成 b1 与 b256 真机重测,当前证据为: + +| 样本 | winner 负载 | 总 `records` | 逻辑 `atomic_calls` | direct | 物理 Atomic | `batched_poll_calls` | PollBatch | ClockBaseline | dropped | 首末 Submit | +| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| b1 | `scalar-nop=0` | 4,414 | 1,031 | 613 | 850 | 418 | 237 | 192 | 0 | 54.056 us | +| b256 | `real-compute/6,28,4,1` | 967,307 | 105,580 | 103,618 | 103,883 | 1,962 | 265 | 192 | 0 | 5,774.295 us | + +两轮的 producer、raw 行与 converter 七项 summary 均闭合,96 核逐核异常数为 0: + +~~~text +b1: 613 + 418 = 1,031; 613 + 237 = 850 = 1,031 - 418 + 237 +b256: 103,618 + 1,962 = 105,580; 103,618 + 265 = 103,883 = 105,580 - 1,962 + 265 +~~~ + +b1/b256 每核记录峰值分别为 57/10,252,均低于 65,536 条固定容量。PollBatch +实际分布在 `startup_poll/fatal_poll/fanin_flag_load/replay_done_poll` 四类 site: +b1 的物理 episode/逻辑调用依次为 `96/143、42/47、2/6、97/222`,b256 为 +`96/143、42/47、16/467、111/1305`;HeapGuard 两类 allowlist 本轮为 0,不是漏插桩。 + +同核区间复核中,b1 的 237 个 PollBatch 与 4 个 Kernel、b256 的 265 个 PollBatch +与 1,024 个 Kernel 均为严格 overlap 0;分别有 3 和 31 处仅在 `end==begin` 端点 +相接,符合“边界先以同一 cycle 关闭 PollBatch,再进入/退出 Kernel”的设计。b256 +真实 Cube/Vector 计算、调度终态和全部语义断言均 PASS。产物为: + +~~~text +outputs/pa_scheduler_swimlane_20260718_182649_4060527/ccec/l2_swimlane_records.json +outputs/pa_scheduler_swimlane_20260718_182649_4060527/ccec/merged_swimlane.json +outputs/pa_scheduler_swimlane_20260718_182725_4061524/ccec/l2_swimlane_records.json +outputs/pa_scheduler_swimlane_20260718_182725_4061524/ccec/merged_swimlane.json +~~~ + +b1 是零 winner 负载的快速验收,b256 是开启 atomic 泳道的诊断运行;5.774295 ms +只证明观察构建保持目标量级,不是关闭 trace 的正式性能基线,也不能与下列历史样本 +做单轮减法归因观察开销。 -O2 的 schema、逐 atomic wrapper、两条 ClockBaseline、scalar lane converter、Submit -PMU hook 和 CNT0..8 sidecar 已实现。2026-07-18 已用干净重编的 CCEC AIC/AIV -二进制完成 b1 atomic-trace-only 上板:全部协议断言 PASS,raw 共 4959 条、 -`expected=4959`、`dropped=0`,其中 Atomic 1395 条、ClockBaseline 192 条。 -Claim raw schema v2 也已直接闭环为 `won=5/lost=283/not_attempted=192`,不再 -借助 Atomic 记录反推 role-filtered Claim。当时 converter 原四项在直接脚本与 -`python -m unittest` 两种入口均为 4/4 PASS;增加 winner workload metadata -回归后,当前完整集合为 5/5 PASS。b1 的 Atomic 按边界闭环为 -`return_ready=1193/source_issue=202`。本轮未入库产物为: +以下是 2026-07-18 的**历史 schema-v2、逐调用、边界修复前**证据,保留用于追溯, +不作为当前 schema-v3 的物理记录规模、逻辑调用数或边界闭合验收。历史 b1 +atomic-trace-only 上板中,全部协议断言 PASS,raw 共 4959 条、 +`expected=4959`、`dropped=0`,其中逐条 Atomic 1395 条、ClockBaseline 192 条。 +Claim 当时已按 schema-v2 闭合为 `won=5/lost=283/not_attempted=192`。当时 converter +回归最终为 5/5 PASS,Atomic direct bracket 为 +`return_ready=1193/source_issue=202`。历史未入库产物为: ~~~text outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b1/raw.json outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b1/merged_swimlane.json ~~~ -b1 用于确认边界、schema 和泳道布局,不用它的插桩后 Submit span 替代 -256 batch 约 5 ms 的无诊断基线。同一二进制的 256 batch 随后也完成一轮: +该历史 b1 只用于说明旧边界、旧 schema 和泳道布局,不用它的插桩后 Submit span +替代 256 batch 约 5 ms 的无诊断基线。同一历史二进制的 256 batch 随后也完成一轮: 全部断言 PASS,raw/merged 均为 963368 个 span,`expected=963368`、 `dropped=0`,Atomic 99944 条、ClockBaseline 192 条;Claim 为 `won=1280/lost=72448/not_attempted=49152`,恰好闭环 `96*1280`。合并泳道 @@ -1264,9 +1366,9 @@ outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b256/raw.json outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b256/merged_swimlane.json ~~~ -同一源码随后关闭 swimlane/atomic-trace/PMU 做了三个独立进程,Submit 为 +同一历史源码随后关闭 swimlane/atomic-trace/PMU 做了三个独立进程,Submit 为 3.729925/4.904346/5.563417 ms,中位数 4.904346 ms,三轮语义全部 PASS。 -这组数据说明当前 standalone 仍复现约 5 ms 量级,也显示了多核轮询、 +这组数据说明该历史 standalone 版本复现了约 5 ms 量级,也显示了多核轮询、 frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 ms 插桩轮 不能与三轮中任一轮做单样本减法后归因 trace 成本。 @@ -1274,9 +1376,11 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 | 结果 | 当前状态 | | ---- | -------- | -| 256 batch 逐 atomic raw/merged 事件数与 dropped | **已闭环** | -| 15 site 按 AIC/AIV 的 events/median/p95/max | **b1/256 batch 已采,未执行 site 明确为 0** | -| 192 条 ClockBaseline 的 AIC/AIV 分布 | **b1/256 batch 已闭环** | +| schema-v3 六类 PollBatch、flags、logical/physical 公式与拓扑的 converter 静态回归 | **19/19 PASS;不替代真机验收** | +| schema-v3 b1 的七项 summary、逐核/全局公式与 dropped | **96/96 闭合,dropped=0** | +| schema-v3 256 batch 的 direct/PollBatch raw→merged 与容量 | **已闭合,单核峰值 10,252/65,536** | +| 15-site schema 与六类 PollBatch allowlist | **两轮 flags 全合法;实际四类有事件、HeapGuard 两类为 0** | +| 当前版本 192 条 ClockBaseline 与 Kernel 边界 | **两轮均闭合;严格 overlap=0** | | `submit-all` 的 owner/gate/CNT0..8/total 闭环 | **b1、零 NOP 单次上板已通过** | | 256 batch `submit-all` 的 I-cache 与 pipe 分组分布 | **3 个 PMU-only 独立进程已采并完成 raw→summary 重算** | | real-compute 引擎 PMU 与 placement | **b8 count1/count2 精确倍增并逐 worker 闭合** | @@ -1284,7 +1388,9 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 此前开发过程中的探索性输出不在这里引用为正式结论。最终按同一源码依次执行: 1. 三后端全量重建及 no-trace/no-PMU 语义回归; -2. CCEC atomic-trace-only,检查上述计数闭环、ClockBaseline 和 raw→merged; +2. CCEC schema-v3 atomic-trace-only 已按 b1、b256 顺序完成,六类 allowlist、flags、 + 七项 summary、逐核/全局三条公式、ClockBaseline、Kernel overlap 和 raw→merged + 均已检查;后续改动仍按同一门禁复测; 3. CCEC PMU-only 的 `submit-all` 已完成 3 个独立进程 A/A;后续每轮仍使用 唯一 JSON 路径,并检查 96 核、owner bitmap/triplet、start/stop、25% 风险门槛和 Restore; 4. real-compute b8 count1/count2 已完成 Cube/Vector busy 精确倍增与 @@ -1300,7 +1406,10 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 - 单条 source-issue 或 return-ready bracket 就是硬件 atomic retire 延迟、跨核 可见延迟或一致性完成时间; - 所有 atomic duration 相加就是 Submit 中“atomic 占用时间”,或删除这些 atomic - 一定能等量缩短墙钟时间;不同核 bracket 会重叠,poll 数也会随插桩改变; + 一定能等量缩短墙钟时间;不同核 direct bracket 会重叠,PollBatch 之间也可重叠, + poll 数还会随插桩改变; +- PollBatch duration 是单次 load 延迟、`call_count` 次延迟之和或独占 scalar 时间; + 它只表示逻辑等待 episode 包络; - `ClockBaseline` 可以逐事件相减并得到无偏的 atomic 硬件净耗时; - trace 开启后的 Submit span 可以直接与无 trace 基线比较并宣布性能收益; - `scalar_busy/total_cycles` 是墙钟时间,或 I-cache miss 数可以直接换算成精确 @@ -1313,9 +1422,12 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 - standalone 的 PMU/atomic 分布可以直接替代真实 FDWIC PA。迁移真实用例时仍需 复用已验证的最小观察代码,并重新完成计数闭环、正确性和无诊断性能 A/B。 -#### 7.5.8 256 batch loser ClaimMax 的定量归因 +#### 7.5.8 历史 schema-v2:256 batch loser ClaimMax 的定量归因 -2026-07-18 的 `atomic_inlineasm_ccec_b256` 中,Claim 三态为 1280 个 winner、 +本节只保留 2026-07-18 边界修复前 `atomic_inlineasm_ccec_b256` 的历史定量结果。 +ClaimMax 在 schema-v3 中仍是 direct FetchMax,不会被 PollBatch 聚合;但下列数值来自 +旧二进制,只能作为后续重测的优先级假设,不能充当当前版本验收数据。该历史样本中, +Claim 三态为 1280 个 winner、 72448 个 attempted loser 和 49152 个 role-filtered not-attempted。每个 attempted Claim 恰好包含一条 `claim_max.fetch_max`,与 73728 条 ClaimMax 精确闭环。 @@ -1343,7 +1455,7 @@ AIV core93 上,对应数字为 0.261731/5.191702 ms,也约 5.0%。这只是 可见的本核返回等待占比;真正改动 Claim 协议后还会改变 winner 到达、 frontier/fanin 时序和竞争形态,不能简单从 5.209261 ms 中减去 0.258 ms。 -阶段性结论是:ClaimMax loser 是后续 atomic 消减的第一优先级,但尚未证明 +历史阶段性结论是:ClaimMax loser 是后续 atomic 消减的第一优先级,但尚未证明 它是整个调度的主瓶颈。256 batch PMU-only 三轮 A/A 已补齐 Submit 全窗的 I-cache 和 pipe 分组基线;它显示 AIV miss rate 在本三轮持续高于 AIC,但现有 counter 仍不能把 miss 定位到 materialize/register 或某条 atomic。开始改 Claim 后, @@ -1543,7 +1655,7 @@ repeat 都执行;repeat 完整性仍使用 CCEC count1→count2 engine PMU 精 性能验收先纠正了观察口径:早先 3.7~4.4 ms 是 `--no-swimlane`, 真实 PA 5.1 ms 是标准 L2 泳道;泳道记录不仅增加指令,还会改变 worker 到达、fanin 失败重试和 RingBp,所以不能把两者直接相减成“缺失的调度时间”。 -当前 CCEC b256 真负载、标准泳道、不开逐 atomic 的 5 个独立进程为: +当前保留的 CCEC b256 真负载历史 phase-only 泳道、不开逐 atomic 的 5 个独立进程为: ```text 5002.413 / 4875.193 / 4968.894 / 4992.477 / 4876.282 us @@ -1554,12 +1666,14 @@ repeat 都执行;repeat 完整性仍使用 CCEC count1→count2 engine PMU 精 中位数为 41.461/54.007/28.053/2.649 us,总 core work 已贴近真实 PA, 不通过增加 repeat 继续硬凑总时间。 -保留的一轮 raw 为 +保留的一轮历史 phase-only raw 为 `outputs/performance_gap_20260718/standalone_ccec_real_b256_raw.json`;863,237 条 记录全部有效,与真实 PA 863,232 条的基本阶段数完全相同,只额外有 5 条 RingBp。这说明 standalone 已达到“独立复现约 5 ms 调度”的目标;仍然保留 本文第 6 节的边界:它不依赖 simpler 生产代码,也不复刻真实 PA 数值数据流和 -通用多 group/joint 拓扑。 +通用多 group/joint 拓扑。当前正式 `swimlane` action 已固定合并普通阶段和 +逐 atomic 记录,因此该 863,237 条历史文件只用于同口径性能参照,不能作为当前 +合并泳道的记录数量或容量证据。 #### 7.5.14 默认真负载的 Submit I-cache 基线 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 628a2c9180..28328a4886 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -280,14 +280,16 @@ semantic_status=PASS postprocess_status=PASS --device 0 --batches 256 --profile-phases --analyze-swimlane ./run.sh swimlane ccec \ - --device 0 --batches 1 --winner-workload real-compute --trace-atomics + --device 0 --batches 1 --winner-workload real-compute ./run.sh swimlane ascendc \ --device 0 --batches 1 --winner-workload real-compute \ --real-compute-count 1 ``` -`swimlane` action 会管理 `--runs 1` 和输出路径,因此不要再传 +`swimlane` 是唯一的正式泳道构建口径,固定合并普通阶段与 schema-v3 atomic +记录(direct Atomic 加 PollBatch); +无需再显式传 `--trace-atomics`。该 action 会管理 `--runs 1` 和输出路径,因此不要再传 `--runs`、`--swimlane-json` 或 `--no-swimlane`。转换器默认使用 `python3`;如需 固定到用户自己的 Python,可在命令前设置: @@ -327,9 +329,10 @@ runner 结束时会打印准确目录: 3. 每个 `block0` 至 `block31` 是一个物理 1AIC+2AIV block; 4. `AIC`、`AIV0`、`AIV1` 轨展示 Submit、Claim、EfDrain、Replay、RingBp 等 runtime 阶段,带 `·kernel` 的轨展示 QK、SF、PV、UP; -5. 开启 `--trace-atomics` 时,Atomic 及 ClockBaseline 直接画在对应 - `AIC/AIV` scalar lane,名称与 category 显式区分 `return_ready` 和 - `source_issue`;不生成带 `·atomic` 的伪并行子轨; +5. direct Atomic、PollBatch 及 ClockBaseline 固定画在对应 + `AIC/AIV` scalar lane;direct 名称与 category 显式区分 `return_ready` 和 + `source_issue`,PollBatch 单独标识逻辑轮询 episode;不生成带 `·atomic` 的 + 伪并行子轨; 6. 普通事件可查看 `task_id`、`func_id`、`core`、`mc` 和 `aux`;Atomic 的 字段和解读边界见 5.6 节。 @@ -378,8 +381,9 @@ CPU 完整协议回归建议关闭大泳道缓冲区: - `--profile-phases`:分别统计 Claim、EfDrain、WaitForSlot、HeapGuard; - `--analyze-swimlane`:读取完整记录,输出各阶段的 per-worker 累计分布以及 EfDrain/Materialize/Claim/Register 的 per-role、per-task-kind 单事件分布; -- `--trace-atomics`:在已开启的泳道中记录每次源码 atomic 调用括号;建议同时 - 传 `--analyze-swimlane` 输出按 AIC/AIV、调用点分组的分布。不能与 +- `--trace-atomics`:在已开启的泳道中记录 atomic 逻辑调用;direct 调用逐条记录, + 显式等待区内六类 observation load 用带精确 `call_count` 的 PollBatch 聚合。 + 建议同时传 `--analyze-swimlane` 输出按 AIC/AIV、调用点分组的分布。不能与 `--no-swimlane` 同用; - `--swimlane-json FILE`:流式导出原始 `fdwic_events` JSON,要求单轮运行; - `--no-swimlane`:关闭泳道记录,不能与 `--analyze-swimlane` 或 @@ -431,20 +435,27 @@ frontier_initial=... frontier_flag=... frontier_ready_fetch_max=... frontier_ter 结果区,不为诊断新增共享 atomic。它们仍会增加少量 scalar 指令,因此优化 A/B 必须使用相同的计数布局;不能把启用分类后的绝对时间直接与旧二进制比较。 -### 5.6 逐 atomic 语义边界泳道 +### 5.6 合并泳道中的 atomic schema-v3 语义边界 -`--trace-atomics` 记录 standalone 调度器中每一次动态 atomic 调用,不只记录 -winner 或慢样本。生成带文字分析的 CCEC 泳道可直接执行: +正式 `swimlane` action 固定记录 standalone 调度器中的 atomic **逻辑调用**, +不只记录 winner 或慢样本。普通 direct Atomic 仍是一条源码调用对应一条物理记录; +显式等待区内允许聚合的 observation load 则用一条带精确调用次数的 PollBatch 表示。 +生成带文字分析的 CCEC 合并泳道可直接执行: ```bash ./run.sh swimlane ccec \ --device 0 --batches 256 \ - --trace-atomics --analyze-swimlane + --analyze-swimlane ``` -转换后 Atomic 和 ClockBaseline 都放在对应 AIC/AIV 的原 scalar lane; -它们本来就是 scalar 指令,不再伪装成与 scalar 并行的独立子轨。 -Kernel 仍放在独立计算单元轨。Atomic 事件名显式区分两种边界: +只有使用低层 `run` action 手工导出 raw 时,才需要显式传入 +`--trace-atomics`;该兼容入口不代表存在第二种 atomic-swimlane 构建。 + +schema-v3 raw 的 `metadata.trace_schema_version` 必须为 3,且 +`metadata.l2_swimlane_level` 必须为 4。转换后 direct Atomic、PollBatch 和 +ClockBaseline 都放在对应 AIC/AIV 的原 scalar lane;它们属于 scalar +调度观察,不再伪装成与 scalar 并行的独立子轨。Kernel 仍放在独立计算单元轨。 +direct Atomic 事件名显式区分两种边界: ```text atomic.return_ready..# @@ -454,6 +465,15 @@ atomic.source_issue..# category 也分别为 `atomic.return_ready` 与 `atomic.source_issue`,可在 Perfetto 中直接过滤,无需逐条点开 args 才能区分。 +PollBatch 转换为: + +```text +atomic.poll_batch..load× +``` + +其 category 为 `atomic.poll_batch`;`args.call_count` 是实际执行的源码 wrapper +调用次数,不是采样或估算值。 + 当前固定 schema 共有 15 个调用点: | `site_id` | Perfetto `site` | `op` | 所属路径 | @@ -475,19 +495,45 @@ Perfetto 中直接过滤,无需逐条点开 args 才能区分。 | 14 | `replay_done_poll` | `load` | 最终 drain 中轮询回放完成 | 上表是源码调用点集合,不代表每轮都会出现 15 类事件;例如正常成功路径不应 -执行 `fatal_set`。轮询点则会为每一次实际 load 生成独立事件。 - -原始 `fdwic_events` 仍是十列格式。对 `phase="Atomic"` 的记录, -`task_id` 是所属 PA task(启动/最终屏障等生命周期 atomic 为 -1), -`function_id` 固定为 -1,`auxiliary` 是上表 `site_id`,`flags` 使用独立 ABI:低 4 bit 是 -`op_id`(Load/Exchange/FetchAdd/FetchMax 依次为 0/1/2/3),bit 4 表示 -返回的旧值参与后续逻辑,bit 5 仅对 Load 表示本次读到零,bit 6 -表示是否取得了“返回值本核可消费”边界,bits 8..31 -仅对 FetchMax 保存饱和后的软件重试次数。CCEC/AscendC 的硬件 -`atomicMax` 当前报告重试数 0;CPU CAS loop 才有可观察的软件重试。 -merged 事件的 `args` 会显式导出 `site/site_id`、`op/op_id`、原始整数 -`cycles`、`result_used`、`return_ready_observed`、`completion_boundary`,以及操作 -适用时的 `value_zero` 或 `retries`。 +执行 `fatal_set`。standalone 也没有真实 PA 后续追加的 BlockWon site,不能把真实 +PA 的九类 load 加一类 exchange allowlist 照搬到这里。 + +standalone 只允许以下六类 observation load 在**匹配的显式等待区内**进入 +PollBatch;同一 site 在等待区外的一次性或 opportunistic 读取仍是 direct Atomic: + +| `site_id` | `site` | `op` | +| ---: | --- | --- | +| 1 | `startup_poll` | `load` | +| 2 | `fatal_poll` | `load` | +| 5 | `fanin_flag_load` | `load` | +| 11 | `heap_frontier_load` | `load` | +| 12 | `heap_vend_load` | `load` | +| 14 | `replay_done_poll` | `load` | + +一个等待区可以同时累积多个 site,所以不同 site 的 PollBatch 时间窗可以重叠; +等待区内也可以交错 direct Atomic。direct record 写入不能隐式关闭 PollBatch, +否则这些自然交错会把同一个等待 episode 人为切碎。 + +原始 `fdwic_events` 仍是十列格式。对 `phase="Atomic"` 的记录,`auxiliary` 是 +`site_id`,`flags` 使用以下 ABI: + +- direct Atomic:bit 7 为 0;低 4 bit 是 `op_id` + (Load/Exchange/FetchAdd/FetchMax 依次为 0/1/2/3),bit 4 表示返回旧值参与 + 后续逻辑,bit 5 仅对 Load 表示本次读到零,bit 6 表示是否取得 + “返回值本核可消费”边界;bits 8..31 只对 direct FetchMax 表示软件 retry, + 不能解释为调用次数; +- PollBatch:低 4 bit 必须为 `load(0)`,bit 4 必须为 1,bit 5/6 必须为 0, + bit 7 必须为 1;bits 8..31 保存无符号 24 bit `call_count`,有效范围为 + `1..0xFFFFFF`。达到上限时先落盘,再从 1 开启下一条 batch,不允许饱和后 + 丢失调用数;`task_id=-1`、`function_id=-1`。 + +merged direct 事件的 `args` 显式导出 `call_count=1`、`site/site_id`、 +`op/op_id`、原始整数 `cycles`、`result_used`、`return_ready_observed`、 +`completion_boundary`,以及适用时的 `value_zero` 或 `retries`。PollBatch 则导出 +精确 `call_count`、`poll_window_cycles`、 +`batch_semantics=observation_load_calls`、 +`duration_semantics=logical_poll_episode_envelope_not_single_atomic_latency` 和 +`may_contain_interleaved_direct_atomics=true`。 边界必须按源码调用点语义解读: @@ -503,19 +549,107 @@ merged 事件的 `args` 会显式导出 `site/site_id`、`op/op_id`、原始整 加 DSB/ISB/额外 GM load;这些操作要么后端不支持,要么会明显改写 被测路径。两种 bracket 都不能直接称为跨核可见或 atomic retire 延迟。 +PollBatch 的 `duration`/`poll_window_cycles` 是从该 site 在显式等待区内首次累计 +调用到边界关闭的**逻辑等待 episode 包络**。它不是独占 scalar 时间,不是 +`call_count` 次 atomic 延迟之和,也不是其中任意一次 load 的单次延迟;因此不能把 +它放进 direct atomic 的 median/p95,或用 `duration/call_count` 推导单次成本。 + +边界关闭规则与 phase/lap 共用同一次 cycle 采样: + +- 显式等待区退出时关闭匹配的 PollBatch; +- `TraceTimestamp` 在写 phase begin/end 前关闭全部活跃 batch; +- `ResetTraceLap` 在推进 lap 起点前关闭,`WriteTraceLap` 在写 lap 前以同一结束 + cycle 关闭; +- Kernel begin/end 也通过 `TraceTimestamp`,所以 PollBatch 不能跨入或跨出 Kernel; +- 最终 flush 只作防御性兜底,不能替代上述语义边界。 + 开启该诊断时,每个 worker 在最终 drain 之后还会写两条 `ClockBaseline`:`clock.consecutive_sys_cnt_reads` 和 `clock.atomic_return_dependency_hook`。前者量连续时钟读,后者量纯寄存器依赖 hook 的固定底噪。全局因此恰有 `96*2=192` 条,都只是分辨率参考,不是 -可以从每条 Atomic 机械相减的校正常数。`[TRACE_ATOMIC]` -按 AIC/AIV、site 和 op 输出事件数、源码括号原始累计、中位数、p95 和最大值。 +可以从每条 Atomic 机械相减的校正常数。`[TRACE_ATOMIC]` 只按 AIC/AIV、site 和 op +输出 direct 事件数、源码 bracket 原始累计、中位数、p95 和最大值; +`[TRACE_ATOMIC_POLL]` 单独输出 PollBatch 的 episode 数、精确逻辑调用数和等待包络 +分布,二者不会混算。 + +schema-v3 必须按逻辑调用与物理记录两套口径闭合。设 direct 物理记录数为 +`direct_atomic_records`,则逐核和全局都必须满足: + +```text +logical_atomic_calls = direct_atomic_records + Σ(PollBatch.call_count) +physical_atomic_records = direct_atomic_records + poll_batch_records +physical_atomic_records + = logical_atomic_calls - batched_poll_calls + poll_batch_records +``` + +raw metadata 的 `fdwic_summary` 七项 +`records/atomic_records/clock_baseline_records/atomic_calls/` +`batched_poll_calls/poll_batch_records/dropped_records` 必须与 producer、逐核 state、 +raw 行重算和 converter 重算逐项一致;`dropped_records` 必须为 0,每个 worker +必须恰有 2 条 ClockBaseline。raw 到 merged 后物理 Atomic 条数保持不变;正式报告 +源码调用总数必须使用 `atomic_calls`,不能使用压缩后的 `atomic_records`。 + +记录写和 PollBatch 维护本身仍会改变后续指令布局、cache、多核到达顺序、atomic +争用与轮询次数。所以不应将 bracket 或 PollBatch duration 与未插桩 Submit 时间相减, +也不能用它们计算 atomic 对 golden 的绝对占比。每核分区固定容纳 65,536 条记录; +任何容量溢出或闭合失败都使该轮 trace 无效,不能截断后继续分析。 + +#### schema-v3 边界修复版 A5 验收 + +2026-07-18 已用边界修复后的同一版 standalone CCEC 依次完成 b1 与 b256 真机重测: + +```text +outputs/pa_scheduler_swimlane_20260718_182649_4060527/ccec/ +outputs/pa_scheduler_swimlane_20260718_182725_4061524/ccec/ +``` + +每个目录都包含 `l2_swimlane_records.json` 与 `merged_swimlane.json`。raw metadata +与事件行复算结果如下;`records` 是包含普通 phase、Atomic 和 ClockBaseline 的总物理 +记录数,不能与 `atomic_records` 混用: + +| 样本 | winner 负载 | `records` | 逻辑 `atomic_calls` | direct Atomic | 物理 `atomic_records` | `batched_poll_calls` | PollBatch | 单核记录峰值 | ClockBaseline | dropped | 首末 Submit | +| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| b1 | `scalar-nop=0` | 4,414 | 1,031 | 613 | 850 | 418 | 237 | 57/65,536 | 192 | 0 | 54.056 us | +| b256 | `real-compute/6,28,4,1` | 967,307 | 105,580 | 103,618 | 103,883 | 1,962 | 265 | 10,252/65,536 | 192 | 0 | 5,774.295 us | + +两轮逐核 producer state 与 raw 扫描均为 96/96 闭合、异常核为 0;全局公式展开为: + +```text +b1: +logical = 613 + 418 = 1,031 +physical = 613 + 237 = 850 = 1,031 - 418 + 237 + +b256: +logical = 103,618 + 1,962 = 105,580 +physical = 103,618 + 265 = 103,883 = 105,580 - 1,962 + 265 +``` + +PollBatch 只出现在本轮实际进入的四类 allowlist site,下面每项依次为 +`物理 episode/逻辑调用`: + +- b1:`startup_poll=96/143`、`fatal_poll=42/47`、 + `fanin_flag_load=2/6`、`replay_done_poll=97/222`; +- b256:`startup_poll=96/143`、`fatal_poll=42/47`、 + `fanin_flag_load=16/467`、`replay_done_poll=111/1305`。 + +`heap_frontier_load/heap_vend_load` 在这两轮均未进入相应 slow path,计数为 0, +不表示 allowlist 漏实现。按同一物理核区间严格检查,b1 的 237 个 PollBatch 与 +4 个 Kernel、b256 的 265 个 PollBatch 与 1,024 个 Kernel 都是严格 overlap 0; +分别有 3 和 31 处仅 `end==begin` 的端点相接。这直接验证 PollBatch 没有跨入或跨出 +Kernel。b256 的真实 Cube/Vector 计算、调度终态和全部语义断言均 PASS。 + +converter 的 schema-v3 静态回归同时为 19/19 PASS。b1 是零 winner 负载的快速验收, +b256 是开启 atomic 泳道的诊断运行;上表 Submit 只能证明当前观察构建的运行量级, +不能替代关闭 trace 的性能基线或与历史样本做单轮减法。 + +#### 历史 schema-v2 样本(仅保留旧口径) -记录写本身不在它自己的 span 内,但会改变后续指令布局、cache、多核到达顺序和 -atomic 争用;轮询次数也可能随之变化。所以不应将 `source_bracket_cycles_total` -与未插桩 Submit 时间相减,或用它计算 atomic 对 golden 的绝对占比。每核分区固定 -容纳 65,536 条记录;通过必须同时满足 `dropped=0`、总记录数闭合,以及每 worker -新增诊断记录数精确等于 `atomic_trace_calls + 2` 闭合。CPU 线程调度可能放大启动轮询并撑满分区;这种 -情况应视为本次 trace 无效,不能截断后继续分析。 +2026-07-18 的旧 CCEC b256 文件 +`outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b256/raw.json` +记录了 963,368 条物理记录,其中逐条 Atomic 99,944 条、ClockBaseline 192 条, +逐核峰值 10,308/65,536,且当轮 `dropped=0`。这些数字来自引入 PollBatch 与上述 +phase/lap/Kernel 边界修复之前的 schema-v2 逐调用模型,只能用于追溯旧版观察结果; +不能拿 99,944 当作当前 schema-v3 的物理容量、逻辑调用数或闭合证据。 ### 5.7 CCEC 每核 PMU 与 I-cache sidecar @@ -937,7 +1071,7 @@ fanin/batch、唯一 winner、角色路由、TensorMap/heap/completion 与数值 CCEC b256 关闭泳道的 3 个独立进程为 4,411.760/4,297.704/4,677.634 us, 中位数 4,411.760 us;它只用于无观察热路对比。与真实 PA 5.1 ms 比较时, -必须同样开启标准泳道且不开逐 atomic;standalone 5 个独立进程为: +当时必须同样开启 phase-only 泳道且不开逐 atomic;standalone 5 个独立进程为: ```text 5002.413 / 4875.193 / 4968.894 / 4992.477 / 4876.282 us @@ -950,7 +1084,7 @@ QK/SF/PV/UP 每 task 均值中位数为 41.461/54.007/28.053/2.649 us, 与真实 44.170/53.729/27.626/1.565 us 的总 core work 接近,不再调整 repeat 追求逐微秒一致。 -只保留一轮标准泳道原始证据: +只保留一轮历史 phase-only 泳道原始证据: ```text outputs/performance_gap_20260718/standalone_ccec_real_b256_raw.json @@ -959,7 +1093,9 @@ outputs/performance_gap_20260718/standalone_ccec_real_b256_raw.json 该轮有 863,237 条记录、`dropped=0`,比真实 PA 的 863,232 条只多 5 条 RingBp;两端的 122,880 个 Submit 与各前端阶段、1,024 个 Kernel/Fanin/Build 数量一致。这证明总体性能已接近,不等于真实 PA 数值数据流、代码生成与通用 -多 group/joint 调度已完全相同。 +多 group/joint 调度已完全相同。该文件用于保留历史 5 ms 同口径性能证据, +不能冒充当前 `swimlane` action 的合并记录;当前 action 还会同时加入 Atomic 与 +ClockBaseline,并继续以逐核容量、调用数、总记录数和 `dropped=0` 闭环。 ## 7. 内存占用和脱仓复制 diff --git a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc index bf6ded0a57..29a6a5eeae 100644 --- a/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc +++ b/tests/atomic_probe/pa_scheduler/ascendc/pa_scheduler.asc @@ -24,6 +24,7 @@ using namespace AscendC; #define PA_DEVICE __aicore__ inline +#define PA_DEVICE_NOINLINE static __aicore__ __attribute__((noinline)) #define PA_GM __gm__ // 通过两个宏把公共调度器实例化为 AscendC 设备代码;公共头本身不依赖 // kernel_operator.h,也不会为 AscendC 复制一套调度协议。 @@ -639,6 +640,7 @@ int32_t main(int32_t argc, char **argv) { real_compute ? pa_scheduler::host::RealComputePatternName(workload_options.pattern) : "none", + options.trace_atomics, read_trace_records )) { postprocess_ok = false; diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh index 1428f6568d..13779f46d1 100755 --- a/tests/atomic_probe/pa_scheduler/ccec/build.sh +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -14,7 +14,16 @@ set -euo pipefail # 所有输入和产物都从脚本自身位置解析,调用者无需位于仓库根目录。 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" + +# CCEC 不再生成同时夹带泳道与 PMU 的统一 ELF。无参数保持兼容并明确等价于 +# swimlane;该产物只保留普通阶段和 atomic 的合并泳道能力。 +BUILD_VARIANT="${1:-swimlane}" +if [[ $# -gt 1 || "$BUILD_VARIANT" != "swimlane" ]]; then + echo "Usage: $0 [swimlane]" >&2 + exit 1 +fi BUILD_DIR="$ROOT_DIR/build/ccec" +VARIANT_DEFINES=(-DPA_BUILD_SWIMLANE=1 -DPA_BUILD_SUBMIT_PMU=0) # 编译只依赖本目录源码与用户安装的 CANN/PTO 头,不引用 pa_scheduler 目录外的 simpler 构建产物。 if [[ -z "${ASCEND_HOME_PATH:-}" ]]; then @@ -30,8 +39,8 @@ READELF_BIN="${READELF:-readelf}" PTO_INCLUDE_ROOT="${PTO_ISA_ROOT:-$ASCEND_HOME_PATH/x86_64-linux}" # ccec/ld.lld 必须来自当前已 source 的 CANN;host 编译器和 readelf 允许用户通过环境变量替换。 -if [[ ! -x "$CCEC" || ! -x "$LD" || ! -x "$HCC" ]]; then - echo "CCEC, ld.lld, or the AICPU HCC compiler is missing under ASCEND_HOME_PATH=$ASCEND_HOME_PATH" >&2 +if [[ ! -x "$CCEC" || ! -x "$LD" ]]; then + echo "CCEC or ld.lld is missing under ASCEND_HOME_PATH=$ASCEND_HOME_PATH" >&2 exit 1 fi if ! command -v "$READELF_BIN" >/dev/null 2>&1; then @@ -50,6 +59,11 @@ for header in pto/pto-inst.hpp pto/common/constants.hpp pto/common/pto_tile.hpp; done mkdir -p "$BUILD_DIR" +# 旧统一构建可能在根目录残留 PMU owner;swimlane 构建主动移除这两个 +# 不属于本变体的产物,避免 direct host 调用误加载上一版诊断 SO。 +rm -f \ + "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so" \ + "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so" # 关闭编译器自动插入的 scalar DCCI,由 kernel.cpp 中与 PA 对齐的显式失效/回写协议负责 cache 可见性。 # 两种架构共用这些 ABI、栈和优化参数,避免 AIC/AIV 对共享 SchedulerState 产生不同解释。 @@ -64,6 +78,7 @@ COMMON_FLAGS=( -mllvm -cce-aicore-dcci-before-kernel-end=false -I"$ROOT_DIR/common" -I"$PTO_INCLUDE_ROOT/include" + "${VARIANT_DEFINES[@]}" ) # 同一入口源码分别面向 cube 与 vector ISA 编译,宏只选择各自的全局入口和 mixed metadata。 @@ -185,54 +200,24 @@ check_icache_probe_layout() { "thrash=0x$thrash_hex/$thrash_size" } -# 编译器不能把目标扩到两个 16B fetch block,也不能折叠 64 KiB 冲刷体;否则 -# cold/warm 虽然仍可能产生数字,却不再代表可解释的单次 I-cache miss。 -check_icache_probe_layout aic -check_icache_probe_layout aiv +# swimlane ELF 明确不含 PMU 校准目标;I-cache 布局检查只属于后续独立的 +# submit-pmu 诊断构建,不能反过来要求泳道产物携带 64 KiB 冲刷体。 +if [[ "$BUILD_VARIANT" != "swimlane" ]]; then + check_icache_probe_layout aic + check_icache_probe_layout aiv +fi # PMU selector/CTRL 的所有权必须由主 aicpu_scheduler 配置并在退出前恢复。 # standalone 目录内自带 Path-A dispatcher 与 owner:前者负责把 owner SO # 落到设备预安装目录,后者由 mode=0 JSON 注册并通过统一入口执行命令。 -echo "[BUILD] self-contained AICPU PMU dispatcher" -"$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra -Werror \ - -Wl,--build-id \ - "$SCRIPT_DIR/pmu_owner_dispatcher.cpp" \ - -o "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so" - -echo "[BUILD] self-contained AICPU PMU owner" -"$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra -Werror \ - -Wl,--build-id \ - -I"$SCRIPT_DIR" \ - "$SCRIPT_DIR/pmu_owner_aicpu.cpp" \ - -o "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so" - -OWNER_HEADER="$("$READELF_BIN" --file-header "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so")" -OWNER_SYMBOLS="$("$READELF_BIN" --dyn-syms --wide "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so")" -DISPATCHER_HEADER="$("$READELF_BIN" --file-header "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so")" -DISPATCHER_SYMBOLS="$("$READELF_BIN" --dyn-syms --wide "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so")" -if [[ "$OWNER_HEADER" != *"Type: DYN"* || - "$OWNER_HEADER" != *"Machine: AArch64"* || - "$DISPATCHER_HEADER" != *"Type: DYN"* || - "$DISPATCHER_HEADER" != *"Machine: AArch64"* ]]; then - echo "PMU dispatcher and owner must both be AArch64 shared objects." >&2 - exit 1 -fi -if [[ "$OWNER_SYMBOLS" != *" simpler_aicpu_exec"* ]]; then - echo "Missing main AICPU PMU owner entry: simpler_aicpu_exec" >&2 - exit 1 -fi -for entry in StaticTileFwkBackendKernelServer DynTileFwkBackendKernelServerInit DynTileFwkBackendKernelServer; do - if [[ "$DISPATCHER_SYMBOLS" != *" $entry"* ]]; then - echo "Missing AICPU PMU dispatcher entry: $entry" >&2 - exit 1 - fi -done -echo "[CHECK] Path-A dispatcher and main AICPU PMU owner exports are present" +# swimlane 构建不生成 PMU owner/dispatcher;二者必须与 submit-pmu kernel、 +# host 使用同一编译期 phase 配置并保存在对应诊断目录中。 # host runner 只链接用户 CANN 9.1 的 ACL/runtime,并写入同一安装目录的 rpath,运行时不需要 simpler 动态库。 # `-Werror` 让 host API 签名或尺寸类型变化在构建期暴露,避免到上板阶段才出现参数截断。 echo "[BUILD] CCEC host runner" "$CXX_BIN" -O2 -std=c++17 -Wall -Wextra -Werror -Wno-deprecated-declarations \ + "${VARIANT_DEFINES[@]}" \ -I"$ROOT_DIR/common" \ -I"$ASCEND_HOME_PATH/include" \ -I"$ASCEND_HOME_PATH/pkg_inc" \ diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index de6d54f7fe..a169031efb 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -1211,6 +1211,25 @@ int main(int argc, char **argv) { return parse_status == pa_scheduler::host::ParseStatus::Help ? EXIT_SUCCESS : EXIT_FAILURE; } if (!ValidateWinnerWorkloadOptions(workload_options)) return EXIT_FAILURE; +#if PA_BUILD_SWIMLANE + // swimlane host 与同目录 kernel 是成套产物;它不允许借旧参数重新开启 + // 已从 device ELF 编译掉的 PMU/phase-profile 路径。 + if (pmu_options.mode != pa_scheduler::ccec_pmu::WindowMode::Off || + !pmu_options.json_path.empty()) { + std::fprintf( + stderr, + "This is a swimlane build; PMU collection requires the separate submit-pmu build.\n" + ); + return EXIT_FAILURE; + } + if (options.profile_phases) { + std::fprintf( + stderr, + "--profile-phases is not part of the swimlane build; use submit-pmu phase attribution.\n" + ); + return EXIT_FAILURE; + } +#endif if (!pmu_options.json_path.empty() && pmu_options.mode == pa_scheduler::ccec_pmu::WindowMode::Off) { std::fprintf(stderr, "--pmu-json requires a non-off --pmu-window.\n"); @@ -1565,6 +1584,7 @@ int main(int argc, char **argv) { real_compute ? pa_scheduler::host::RealComputePatternName(workload_options.pattern) : "none", + options.trace_atomics, read_trace_records )) { postprocess_ok = false; diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index ef3751cdbd..a59a387c05 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -19,6 +19,8 @@ #include "../common/winner_workload.h" #define PA_DEVICE __aicore__ inline +#define PA_DEVICE_NOINLINE static __aicore__ __attribute__((noinline)) +#define PA_LOOP_NOUNROLL _Pragma("clang loop unroll(disable)") #define PA_GM __gm__ #include "../common/pa_scheduler_core.h" @@ -212,16 +214,17 @@ static __aicore__ __attribute__((noinline, used)) void pa_execute_real_winner_wo #endif } -#if defined(PA_BUILD_AIC) +#if PA_BUILD_SUBMIT_PMU && defined(PA_BUILD_AIC) #define PA_ICACHE_TARGET_NAME pa_icache_target_aic #define PA_ICACHE_MEASURE_NAME pa_icache_measure_aic #define PA_ICACHE_THRASH_NAME pa_icache_thrash_aic -#elif defined(PA_BUILD_AIV) +#elif PA_BUILD_SUBMIT_PMU && defined(PA_BUILD_AIV) #define PA_ICACHE_TARGET_NAME pa_icache_target_aiv #define PA_ICACHE_MEASURE_NAME pa_icache_measure_aiv #define PA_ICACHE_THRASH_NAME pa_icache_thrash_aiv #endif +#if PA_BUILD_SUBMIT_PMU // cold/warm 两条路径调用同一个 8B 目标函数。入口按 128B I-cache line // 对齐,构建脚本还会核对符号尺寸、对齐和 target -> harness -> thrash 布局。 __aicore__ static __attribute__((noinline, used, aligned(128), section(".text.pa_icache_target"))) @@ -235,6 +238,7 @@ void PA_ICACHE_TARGET_NAME() { __aicore__ static __attribute__((noinline, used, aligned(128), section(".text.pa_icache_thrash"))) void PA_ICACHE_THRASH_NAME(); +#endif struct CcecOps { static constexpr bool kAtomicReturnReadyObserved = true; @@ -316,6 +320,7 @@ struct CcecOps { pa_execute_real_winner_workload(state, worker, kind); } +#if PA_BUILD_SUBMIT_PMU __aicore__ static inline bool PmuWindowStart( __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id ); @@ -323,6 +328,17 @@ struct CcecOps { __aicore__ static inline void PmuWindowStop( __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id, bool started ); +#else + // swimlane 产物不携带 PMU 读寄存器或门控代码;公共调度器保留同一 hook + // 形状,编译器会把这两个空实现完整消去。 + __aicore__ static inline bool PmuWindowStart( + __gm__ pa_scheduler::SchedulerState *, uint32_t + ) { return false; } + + __aicore__ static inline void PmuWindowStop( + __gm__ pa_scheduler::SchedulerState *, uint32_t, bool + ) {} +#endif // SPIN_WAIT_HINT is also a no-op in the real A5 inner-kernel contract. // 同理不额外插入 nop,让等待循环保留真实 PA 内核“不主动退避”的指令成本。 @@ -361,6 +377,7 @@ struct CcecOps { } }; +#if PA_BUILD_SUBMIT_PMU struct PmuSnapshot { uint64_t total_cycles = 0; uint32_t vector_busy = 0; @@ -656,6 +673,7 @@ __aicore__ inline void RunPmuProbe(__gm__ pa_scheduler::SchedulerState *state, u } PublishPmuSnapshot(result, sample); } +#endif // PA_BUILD_SUBMIT_PMU } // namespace @@ -667,7 +685,9 @@ extern "C" __global__ __aicore__ void pa_scheduler_0_mix_aic(__gm__ pa_scheduler // 32 个物理 block 的 AIC 直接使用 block_idx,形成连续 worker 0..31。 const uint32_t worker_id = static_cast(get_block_idx()); pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aic); +#if PA_BUILD_SUBMIT_PMU RunPmuProbe(state, worker_id); +#endif } #elif defined(PA_BUILD_AIV) PTO_SYNCALL_MIX_AIC_KERNEL_META(pa_scheduler_0_mix_aiv, 1, 2); @@ -677,7 +697,9 @@ extern "C" __global__ __aicore__ void pa_scheduler_0_mix_aiv(__gm__ pa_scheduler const uint32_t vector_id = static_cast(get_block_idx() * get_subblockdim() + get_subblockid()); const uint32_t worker_id = pa_scheduler::kAicWorkers + vector_id; pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aiv); +#if PA_BUILD_SUBMIT_PMU RunPmuProbe(state, worker_id); +#endif } #else #error "Compile with PA_BUILD_AIC or PA_BUILD_AIV" diff --git a/tests/atomic_probe/pa_scheduler/common/host_support.h b/tests/atomic_probe/pa_scheduler/common/host_support.h index f68adeaf51..d205b39ffc 100644 --- a/tests/atomic_probe/pa_scheduler/common/host_support.h +++ b/tests/atomic_probe/pa_scheduler/common/host_support.h @@ -224,10 +224,11 @@ inline void ConfigureTrace(SchedulerState *state, const Options &options, const } inline void InitializeTraceHeader(TraceHeader *header) { - // magic、版本、108 槽 header 布局和 1 GHz 频率均与真实 FDWIC 泳道 ABI 对齐。 + // version=3 表示 core state 已携带 weighted atomic/PollBatch 计数和权威拓扑; + // JSON 的 trace_schema_version 则仍按是否启用 atomic 分别导出 v2/v3。 std::memset(header, 0, sizeof(*header)); header->magic = 0x4653574cU; - header->version = 1; + header->version = 3; header->num_cores = kWorkers; header->records_per_core = kTraceRecordsPerCore; header->frequency_hz = kSystemCounterHz; @@ -364,32 +365,25 @@ inline const char *AtomicOpName(uint32_t op) { } inline AtomicOp AtomicSiteOp(AtomicSite site) { - switch (site) { - case AtomicSite::StartupIncrement: - case AtomicSite::ReplayDoneIncrement: - return AtomicOp::FetchAdd; - case AtomicSite::FatalSet: - case AtomicSite::CompletionVendExchange: - case AtomicSite::CompletionFlagExchange: - return AtomicOp::Exchange; - case AtomicSite::ClaimMax: - case AtomicSite::FrontierMax: - return AtomicOp::FetchMax; - default: - return AtomicOp::Load; - } + return AtomicSiteExpectedOp(site); } inline bool ValidateTraceHeader(const TraceHeader &header, const char *operation) { // 在任何 D2H record 搬运前先验证容量和 dropped,防止损坏 header 导致 scratch 越界或导出残缺泳道。 // 频率也要求精确为 1 GHz,否则后续 ns/us 换算即使 JSON 合法也没有性能意义。 - const bool valid = header.magic == 0x4653574cU && header.version == 1 && + const bool valid = header.magic == 0x4653574cU && header.version == 3 && header.num_cores == kWorkers && header.records_per_core == kTraceRecordsPerCore && header.frequency_hz == kSystemCounterHz; bool core_states_valid = true; for (uint32_t worker = 0; worker < kWorkers; ++worker) { - core_states_valid &= header.cores[worker].count <= kTraceRecordsPerCore; - core_states_valid &= header.cores[worker].dropped == 0; + const TraceCoreState &core = header.cores[worker]; + core_states_valid &= core.count <= kTraceRecordsPerCore; + core_states_valid &= core.dropped == 0; + core_states_valid &= core.poll_calls <= core.atomic_calls; + core_states_valid &= (core.poll_calls == 0) == (core.poll_batch_records == 0); + const uint64_t physical_atomic = + static_cast(core.atomic_calls) - core.poll_calls + core.poll_batch_records; + core_states_valid &= physical_atomic <= core.count; } if (!valid || !core_states_valid) { std::fprintf( @@ -403,11 +397,77 @@ inline bool ValidateTraceHeader(const TraceHeader &header, const char *operation return valid && core_states_valid; } +struct TraceExportSummary { + uint64_t records = 0; + uint64_t atomic_records = 0; + uint64_t clock_baseline_records = 0; + uint64_t atomic_calls = 0; + uint64_t poll_calls = 0; + uint64_t poll_batch_records = 0; + uint64_t dropped_records = 0; +}; + +inline bool SameTraceSummary(const TraceExportSummary &left, const TraceExportSummary &right) { + return left.records == right.records && left.atomic_records == right.atomic_records && + left.clock_baseline_records == right.clock_baseline_records && + left.atomic_calls == right.atomic_calls && left.poll_calls == right.poll_calls && + left.poll_batch_records == right.poll_batch_records && + left.dropped_records == right.dropped_records; +} + +inline uint32_t AtomicRecordCallCount(const TraceRecord &record) { + return (record.flags & kAtomicPollBatch) != 0 + ? record.flags >> kAtomicPollCountShift + : 1U; +} + +inline bool AtomicRecordSchemaValid(const TraceRecord &record, bool atomic_trace_enabled) { + if (!atomic_trace_enabled || record.auxiliary >= static_cast(AtomicSite::Count)) { + return false; + } + const AtomicSite site = static_cast(record.auxiliary); + const uint32_t op = record.flags & kAtomicOpMask; + if (op != static_cast(AtomicSiteExpectedOp(site))) return false; + + const bool result_used = (record.flags & kAtomicResultUsed) != 0; + const bool value_zero = (record.flags & kAtomicValueZero) != 0; + const bool return_ready = (record.flags & kAtomicReturnReady) != 0; + const bool poll_batch = (record.flags & kAtomicPollBatch) != 0; + const uint32_t payload = record.flags >> kAtomicRetriesShift; + if (poll_batch) { + return AtomicSiteIsPollBatchable(site) && result_used && !value_zero && !return_ready && + payload > 0 && record.task_id == -1 && record.function_id == -1; + } + if (result_used != AtomicSiteResultUsed(site) || (return_ready && !result_used)) return false; + if (value_zero && op != static_cast(AtomicOp::Load)) return false; + if (payload != 0 && op != static_cast(AtomicOp::FetchMax)) return false; + return record.function_id == -1; +} + +inline bool ClockRecordSchemaValid(const TraceRecord &record) { + const bool dependency = (record.flags & kClockAtomicDependency) != 0; + const bool dependency_applied = (record.flags & kClockAtomicDependencyApplied) != 0; + return (record.flags & ~(kClockAtomicDependency | kClockAtomicDependencyApplied)) == 0 && + (!dependency_applied || dependency) && record.task_id == -1 && + record.function_id == -1 && record.auxiliary == 0; +} + +inline void ExpectedTraceTopology(uint32_t worker, int32_t *block_id, int32_t *lane) { + if (worker < kAicWorkers) { + *block_id = static_cast(worker); + *lane = 0; + return; + } + const uint32_t vector_id = worker - kAicWorkers; + *block_id = static_cast(vector_id / 2); + *lane = static_cast(1 + vector_id % 2); +} + template inline bool ExportSwimlaneRecords( const TraceHeader &header, const std::string &output_path, WinnerWorkloadMode workload_mode, const WorkloadCounts &workload_counts, - const char *workload_pattern, ReadRecords read_records + const char *workload_pattern, bool atomic_trace_enabled, ReadRecords read_records ) { if (!ValidateTraceHeader(header, "swimlane export")) return false; if (workload_mode != WinnerWorkloadMode::ScalarNop && @@ -426,6 +486,51 @@ inline bool ExportSwimlaneRecords( return false; } + TraceExportSummary producer_summary; + for (uint32_t worker = 0; worker < kWorkers; ++worker) { + const TraceCoreState &core = header.cores[worker]; + int32_t expected_block = -1; + int32_t expected_lane = -1; + ExpectedTraceTopology(worker, &expected_block, &expected_lane); + if (core.core_idx != static_cast(worker) || core.block_id != expected_block || + core.lane != expected_lane) { + std::fprintf( + stderr, + "swimlane export rejected worker topology: worker=%u core=%d block=%d/%d lane=%d/%d\n", + worker, core.core_idx, core.block_id, expected_block, core.lane, expected_lane + ); + return false; + } + producer_summary.records += core.count; + producer_summary.atomic_calls += core.atomic_calls; + producer_summary.poll_calls += core.poll_calls; + producer_summary.poll_batch_records += core.poll_batch_records; + producer_summary.dropped_records += core.dropped; + if (!atomic_trace_enabled) { + if (core.atomic_calls != 0 || core.poll_calls != 0 || core.poll_batch_records != 0) { + std::fprintf( + stderr, + "phase-only swimlane worker %u unexpectedly reports atomic counters: calls=%u polls=%u batches=%u\n", + worker, core.atomic_calls, core.poll_calls, core.poll_batch_records + ); + return false; + } + continue; + } + if (core.poll_calls > core.atomic_calls || + (core.poll_calls == 0) != (core.poll_batch_records == 0)) { + std::fprintf( + stderr, + "atomic swimlane worker %u has invalid counters: calls=%u polls=%u batches=%u\n", + worker, core.atomic_calls, core.poll_calls, core.poll_batch_records + ); + return false; + } + producer_summary.atomic_records += + static_cast(core.atomic_calls) - core.poll_calls + core.poll_batch_records; + } + producer_summary.clock_baseline_records = atomic_trace_enabled ? 2ULL * kWorkers : 0; + // 先写同目录临时文件,全部记录写完并关闭后再 rename 替换,避免把半截 JSON // 当成有效采集;这里没有 fsync 文件和目录,不承诺掉电后的持久化原子性。 const std::string temporary_path = output_path + ".tmp"; @@ -443,14 +548,16 @@ inline bool ExportSwimlaneRecords( std::setvbuf(output, output_buffer.data(), _IOFBF, output_buffer.size()); std::fprintf( output, - "{\n\"l2_swimlane_level\":1,\n" + "{\n\"l2_swimlane_level\":%u,\n" "\"metadata\":{\"clock_freq_hz\":%llu,\"num_cores\":%u," - "\"trace_schema_version\":2," + "\"trace_schema_version\":%u," "\"winner_workload\":{\"mode\":\"%s\"," "\"counts\":{\"qk\":%u,\"sf\":%u,\"pv\":%u,\"up\":%u}," "\"unit\":\"%s\",\"input_pattern\":\"%s\"," "\"engine_mapping\":%s},\"core_types\":[", + atomic_trace_enabled ? 4U : 1U, static_cast(header.frequency_hz), kWorkers, + atomic_trace_enabled ? 3U : 2U, workload_mode == WinnerWorkloadMode::RealCompute ? "real-compute" : "scalar-nop", workload_counts.qk, workload_counts.sf, workload_counts.pv, workload_counts.up, workload_mode == WinnerWorkloadMode::RealCompute @@ -465,9 +572,27 @@ inline bool ExportSwimlaneRecords( for (uint32_t worker = 0; worker < kWorkers; ++worker) { std::fprintf(output, "%s\"%s\"", worker == 0 ? "" : ",", worker < kAicWorkers ? "aic" : "aiv"); } + if (atomic_trace_enabled) { + std::fprintf( + output, + "],\"fdwic_summary\":{\"records\":%llu,\"atomic_records\":%llu," + "\"clock_baseline_records\":%llu,\"atomic_calls\":%llu," + "\"batched_poll_calls\":%llu,\"poll_batch_records\":%llu," + "\"dropped_records\":%llu}", + static_cast(producer_summary.records), + static_cast(producer_summary.atomic_records), + static_cast(producer_summary.clock_baseline_records), + static_cast(producer_summary.atomic_calls), + static_cast(producer_summary.poll_calls), + static_cast(producer_summary.poll_batch_records), + static_cast(producer_summary.dropped_records) + ); + } else { + std::fprintf(output, "]"); + } std::fprintf( output, - "]},\n\"aicore_tasks\":[],\n\"aicpu_tasks\":[],\n" + "},\n\"aicore_tasks\":[],\n\"aicpu_tasks\":[],\n" "\"aicpu_scheduler_phases\":[],\n\"aicpu_orchestrator_phases\":[],\n\"fdwic_events\":[\n" ); // fdwic_events 每行固定十列:core、block、lane、task、function、phase、起止周期、flags、aux。 @@ -475,6 +600,7 @@ inline bool ExportSwimlaneRecords( bool success = true; bool first_record = true; uint64_t exported_records = 0; + TraceExportSummary observed_summary; std::vector scratch(kTraceRecordsPerCore); constexpr int32_t kTracePhaseCount = static_cast(TracePhase::ClockBaseline) + 1; for (uint32_t worker = 0; worker < kWorkers && success; ++worker) { @@ -492,46 +618,71 @@ inline bool ExportSwimlaneRecords( success = false; break; } + const TraceCoreState &core = header.cores[worker]; + uint64_t core_atomic_calls = 0; + uint64_t core_poll_calls = 0; + uint32_t core_atomic_records = 0; + uint32_t core_poll_batch_records = 0; + uint32_t core_clock_records = 0; + uint32_t core_plain_clock_records = 0; + uint32_t core_dependency_clock_records = 0; + bool dependency_applied = false; + bool direct_result_used_return_ready = false; + bool direct_result_used_source_issue = false; for (uint32_t index = 0; index < available; ++index) { const TraceRecord &record = scratch[index]; - // 这里只检查 lane/block 的合法范围以及 core_idx 是否落在所属 worker 槽, - // 不把生产者应遵守的 worker↔block/lane 精确映射误说成 exporter 已完成的校验。 const bool atomic_record = record.phase == static_cast(TracePhase::Atomic); const bool claim_record = record.phase == static_cast(TracePhase::Claim); - const uint32_t atomic_op = record.flags & kAtomicOpMask; - const bool atomic_result_used = (record.flags & kAtomicResultUsed) != 0; - const bool atomic_return_ready = (record.flags & kAtomicReturnReady) != 0; - const bool atomic_return_ready_valid = !atomic_return_ready || - (atomic_result_used && - (atomic_op == static_cast(AtomicOp::Load) || - atomic_op == static_cast(AtomicOp::FetchMax))); + const bool clock_record = record.phase == static_cast(TracePhase::ClockBaseline); const bool atomic_schema_valid = !atomic_record || - (record.auxiliary < static_cast(AtomicSite::Count) && - atomic_op <= static_cast(AtomicOp::FetchMax) && - atomic_return_ready_valid && - atomic_op == static_cast( - AtomicSiteOp(static_cast(record.auxiliary)) - )); + AtomicRecordSchemaValid(record, atomic_trace_enabled); const bool claim_schema_valid = !claim_record || ((record.flags & ~(kClaimWon | kClaimAttempted)) == 0 && - ((record.flags & kClaimWon) == 0 || (record.flags & kClaimAttempted) != 0)); + ((record.flags & kClaimWon) == 0 || (record.flags & kClaimAttempted) != 0) && + record.auxiliary <= 1); + const bool clock_schema_valid = !clock_record || + (atomic_trace_enabled && ClockRecordSchemaValid(record)); const bool record_valid = record.end_cycle >= record.start_cycle && record.phase >= 0 && - record.phase < kTracePhaseCount && record.lane >= 0 && record.lane <= 2 && - record.block_id >= 0 && record.block_id < static_cast(kAicWorkers) && - record.core_idx == static_cast(worker) && atomic_schema_valid && - claim_schema_valid; + record.phase < kTracePhaseCount && record.task_id >= -1 && + record.function_id >= -1 && record.lane == core.lane && + record.block_id == core.block_id && + record.core_idx == core.core_idx && atomic_schema_valid && + claim_schema_valid && clock_schema_valid; if (!record_valid) { std::fprintf( stderr, "Invalid trace record at worker=%u index=%u: phase=%d lane=%d block=%d core=%d " - "start=%llu end=%llu\n", + "start=%llu end=%llu flags=0x%08x aux=%u\n", worker, index, record.phase, record.lane, record.block_id, record.core_idx, static_cast(record.start_cycle), - static_cast(record.end_cycle) + static_cast(record.end_cycle), record.flags, record.auxiliary ); success = false; break; } + if (atomic_record) { + ++core_atomic_records; + const uint32_t call_count = AtomicRecordCallCount(record); + core_atomic_calls += call_count; + if ((record.flags & kAtomicPollBatch) != 0) { + core_poll_calls += call_count; + ++core_poll_batch_records; + } else if ((record.flags & kAtomicResultUsed) != 0) { + if ((record.flags & kAtomicReturnReady) != 0) { + direct_result_used_return_ready = true; + } else { + direct_result_used_source_issue = true; + } + } + } else if (clock_record) { + ++core_clock_records; + if ((record.flags & kClockAtomicDependency) != 0) { + ++core_dependency_clock_records; + dependency_applied = (record.flags & kClockAtomicDependencyApplied) != 0; + } else { + ++core_plain_clock_records; + } + } std::fprintf( output, "%s[%d,%d,%d,%d,%d,\"%s\",%llu,%llu,%u,%u]", @@ -543,6 +694,64 @@ inline bool ExportSwimlaneRecords( first_record = false; ++exported_records; } + if (!success) break; + bool core_closed = true; + if (atomic_trace_enabled) { + const uint64_t expected_atomic_records = + static_cast(core.atomic_calls) - core.poll_calls + core.poll_batch_records; + core_closed = core_atomic_records == expected_atomic_records && + core_atomic_calls == core.atomic_calls && core_poll_calls == core.poll_calls && + core_poll_batch_records == core.poll_batch_records && core_clock_records == 2 && + core_plain_clock_records == 1 && core_dependency_clock_records == 1 && + (!dependency_applied || !direct_result_used_source_issue) && + (dependency_applied || !direct_result_used_return_ready); + } else { + core_closed = core_atomic_records == 0 && core_atomic_calls == 0 && core_poll_calls == 0 && + core_poll_batch_records == 0 && core_clock_records == 0; + } + if (!core_closed) { + std::fprintf( + stderr, + "swimlane closure failed on worker=%u: physical_atomic=%u logical_atomic=%llu/%u " + "poll_calls=%llu/%u poll_batches=%u/%u clock=%u plain=%u dependency=%u " + "dependency_applied=%s direct_ready=%s direct_issue=%s\n", + worker, core_atomic_records, static_cast(core_atomic_calls), + core.atomic_calls, static_cast(core_poll_calls), core.poll_calls, + core_poll_batch_records, core.poll_batch_records, core_clock_records, + core_plain_clock_records, core_dependency_clock_records, + dependency_applied ? "yes" : "no", direct_result_used_return_ready ? "yes" : "no", + direct_result_used_source_issue ? "yes" : "no" + ); + success = false; + break; + } + observed_summary.records += available; + observed_summary.atomic_records += core_atomic_records; + observed_summary.clock_baseline_records += core_clock_records; + observed_summary.atomic_calls += core_atomic_calls; + observed_summary.poll_calls += core_poll_calls; + observed_summary.poll_batch_records += core_poll_batch_records; + observed_summary.dropped_records += core.dropped; + } + if (success && !SameTraceSummary(producer_summary, observed_summary)) { + std::fprintf( + stderr, + "swimlane producer/raw summary mismatch: records=%llu/%llu atomic_records=%llu/%llu " + "atomic_calls=%llu/%llu poll_calls=%llu/%llu poll_batches=%llu/%llu clock=%llu/%llu\n", + static_cast(observed_summary.records), + static_cast(producer_summary.records), + static_cast(observed_summary.atomic_records), + static_cast(producer_summary.atomic_records), + static_cast(observed_summary.atomic_calls), + static_cast(producer_summary.atomic_calls), + static_cast(observed_summary.poll_calls), + static_cast(producer_summary.poll_calls), + static_cast(observed_summary.poll_batch_records), + static_cast(producer_summary.poll_batch_records), + static_cast(observed_summary.clock_baseline_records), + static_cast(producer_summary.clock_baseline_records) + ); + success = false; } if (success) std::fprintf(output, "\n]}\n"); if (std::ferror(output) != 0) { @@ -586,6 +795,8 @@ inline bool AnalyzeSwimlaneRecords( std::vector task_durations[2][kTasksPerBatch][sizeof(kDetailedPhases) / sizeof(kDetailedPhases[0])]; std::vector atomic_durations[2][static_cast(AtomicSite::Count)]; uint64_t atomic_return_ready_counts[2][static_cast(AtomicSite::Count)] = {}; + std::vector atomic_poll_windows[2][static_cast(AtomicSite::Count)]; + uint64_t atomic_poll_calls[2][static_cast(AtomicSite::Count)] = {}; std::vector clock_baselines[2]; std::vector clock_dependency_baselines[2]; uint64_t clock_dependency_applied[2] = {}; @@ -605,15 +816,27 @@ inline bool AnalyzeSwimlaneRecords( } const uint32_t phase = static_cast(record.phase); const uint64_t duration = record.end_cycle - record.start_cycle; - cycles[worker][phase] += duration; - ++counts[worker][phase]; + const bool atomic_poll_batch = + record.phase == static_cast(TracePhase::Atomic) && + (record.flags & kAtomicPollBatch) != 0; + // PollBatch 的 duration 是一次等待 episode 的包络,允许夹着其他直接 + // atomic/调度代码;不能混入“Atomic 单次括号”的累计时间或分位数。 + if (!atomic_poll_batch) { + cycles[worker][phase] += duration; + ++counts[worker][phase]; + } if (record.phase == static_cast(TracePhase::Atomic) && record.auxiliary < static_cast(AtomicSite::Count)) { const uint32_t role_index = state.results[worker].role == static_cast(CoreRole::Aic) ? 0U : 1U; - atomic_durations[role_index][record.auxiliary].push_back(duration); - atomic_return_ready_counts[role_index][record.auxiliary] += - (record.flags & kAtomicReturnReady) != 0; + if (atomic_poll_batch) { + atomic_poll_windows[role_index][record.auxiliary].push_back(duration); + atomic_poll_calls[role_index][record.auxiliary] += AtomicRecordCallCount(record); + } else { + atomic_durations[role_index][record.auxiliary].push_back(duration); + atomic_return_ready_counts[role_index][record.auxiliary] += + (record.flags & kAtomicReturnReady) != 0; + } } if (record.phase == static_cast(TracePhase::ClockBaseline)) { const uint32_t role_index = @@ -712,6 +935,24 @@ inline bool AnalyzeSwimlaneRecords( ); } } + // 等待聚合只报告 episode 数、精确逻辑调用数与包络分布。window 不能除以 + // calls 当作单次 atomic latency,也不能与 Submit 墙钟直接相加。 + for (uint32_t role_index = 0; role_index < 2; ++role_index) { + for (uint32_t site = 0; site < static_cast(AtomicSite::Count); ++site) { + const std::vector &windows = atomic_poll_windows[role_index][site]; + if (windows.empty()) continue; + const Uint64Distribution summary = SummarizeUint64(windows); + std::printf( + "[TRACE_ATOMIC_POLL] role=%s site=%s op=%s episodes=%zu logical_calls=%llu " + "window_definition=wait-episode-envelope median_ns=%.1f p95_ns=%llu max_ns=%llu\n", + role_names[role_index], AtomicSiteName(site), + AtomicOpName(static_cast(AtomicSiteOp(static_cast(site)))), + windows.size(), static_cast(atomic_poll_calls[role_index][site]), + summary.median, static_cast(summary.p95), + static_cast(summary.maximum) + ); + } + } const char *kind_names[] = {"Alloc", "QK", "SF", "PV", "UP"}; // 单事件统计按 role 与 task kind 展开,可区分“该 role 真实参与”与“只回放前端”的成本。 for (uint32_t role_index = 0; role_index < 2; ++role_index) { @@ -1047,34 +1288,58 @@ inline Metrics Validate( bool trace_shape_ok = trace_header != nullptr; uint64_t trace_records = 0; uint64_t trace_dropped = 0; + uint64_t physical_atomic_records = 0; + uint64_t batched_poll_calls = 0; + uint64_t poll_batch_records = 0; bool per_worker_trace_counts_ok = true; if (trace_header != nullptr) { trace_shape_ok &= trace_header->magic == 0x4653574cU; - trace_shape_ok &= trace_header->version == 1; + trace_shape_ok &= trace_header->version == 3; trace_shape_ok &= trace_header->num_cores == kWorkers; trace_shape_ok &= trace_header->records_per_core == kTraceRecordsPerCore; trace_shape_ok &= trace_header->frequency_hz == kSystemCounterHz; for (uint32_t worker = 0; worker < kWorkers; ++worker) { - trace_records += trace_header->cores[worker].count; - trace_dropped += trace_header->cores[worker].dropped; - trace_shape_ok &= trace_header->cores[worker].count <= kTraceRecordsPerCore; + const TraceCoreState &core = trace_header->cores[worker]; + trace_records += core.count; + trace_dropped += core.dropped; + trace_shape_ok &= core.count <= kTraceRecordsPerCore; + int32_t expected_block = -1; + int32_t expected_lane = -1; + ExpectedTraceTopology(worker, &expected_block, &expected_lane); + trace_shape_ok &= core.core_idx == static_cast(worker); + trace_shape_ok &= core.block_id == expected_block; + trace_shape_ok &= core.lane == expected_lane; const WorkerResult &result = state.results[worker]; const uint64_t worker_kernels = result.kernel_counts[0] + result.kernel_counts[1] + result.kernel_counts[2] + result.kernel_counts[3]; + uint64_t worker_physical_atomic = 0; + if ((state.config.trace_enabled & kTraceAtomicsEnabled) != 0) { + trace_shape_ok &= core.atomic_calls == result.atomic_trace_calls; + trace_shape_ok &= core.poll_calls <= core.atomic_calls; + trace_shape_ok &= (core.poll_calls == 0) == (core.poll_batch_records == 0); + worker_physical_atomic = + static_cast(core.atomic_calls) - core.poll_calls + core.poll_batch_records; + physical_atomic_records += worker_physical_atomic; + batched_poll_calls += core.poll_calls; + poll_batch_records += core.poll_batch_records; + } else { + trace_shape_ok &= core.atomic_calls == 0 && core.poll_calls == 0 && + core.poll_batch_records == 0; + } const uint64_t worker_expected = 7 * result.submits + result.claim_wins - result.wins[0] + 2 * worker_kernels + result.wait_events[0] + result.wait_events[1] + (((state.config.trace_enabled & kTraceAtomicsEnabled) != 0) - ? result.atomic_trace_calls + 2 + ? worker_physical_atomic + 2 : 0); - per_worker_trace_counts_ok &= trace_header->cores[worker].count == worker_expected; + per_worker_trace_counts_ok &= core.count == worker_expected; } } const uint64_t expected_trace_records = static_cast(batches) * (static_cast(kWorkers) * 35 + 12) + trace_wait_records + (((state.config.trace_enabled & kTraceAtomicsEnabled) != 0) - ? atomic_trace_calls + 2 * kWorkers + ? physical_atomic_records + 2 * kWorkers : 0); // 每 batch 固定记录为 96*35+12;RingBp 等真实等待按运行时次数额外加入。 Expect(trace_shape_ok, "swimlane header and per-worker capacities are valid", &metrics); @@ -1093,9 +1358,14 @@ inline Metrics Validate( static_cast(trace_dropped), kTraceBytes ); std::printf( - "[ATOMIC_TRACE] enabled=%s calls=%llu definition=per-record-boundary-flags\n", + "[ATOMIC_TRACE] enabled=%s logical_calls=%llu physical_records=%llu " + "batched_poll_calls=%llu poll_batch_records=%llu " + "closure=physical=logical-batched+batch_records\n", (state.config.trace_enabled & kTraceAtomicsEnabled) != 0 ? "yes" : "no", - static_cast(atomic_trace_calls) + static_cast(atomic_trace_calls), + static_cast(physical_atomic_records), + static_cast(batched_poll_calls), + static_cast(poll_batch_records) ); } diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index a9a8676607..5c084f4eea 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -15,6 +15,21 @@ #include #include +// CCEC 的正式产物只允许二选一:swimlane 保存普通阶段与 atomic 记录, +// submit-pmu 则编译掉泳道观察代码。CPU/AscendC 未传这些宏时继续使用原有 +// 通用实现,避免公共模型反向依赖某个后端的构建脚本。 +#ifndef PA_BUILD_SWIMLANE +#define PA_BUILD_SWIMLANE 0 +#endif + +#ifndef PA_BUILD_SUBMIT_PMU +#define PA_BUILD_SUBMIT_PMU 0 +#endif + +#if PA_BUILD_SWIMLANE && PA_BUILD_SUBMIT_PMU +#error "PA_BUILD_SWIMLANE and PA_BUILD_SUBMIT_PMU are mutually exclusive" +#endif + namespace pa_scheduler { // 这里固定的是 PA Case1 的调度拓扑,而不是为了缩小 standalone 人为选择的规模: @@ -267,7 +282,8 @@ enum class AtomicSite : uint32_t { // Atomic 记录 flags 的低四位保存操作种类;bit4 表示返回值参与后续判断, // bit5 表示 Load 观察到零,bit6 表示结束时间已由返回值依赖推进到 -// return-ready 边界,bits[31:8] 保存 FetchMax 的软件重试数(饱和)。 +// return-ready 边界。schema-v3 中 bit7 区分等待区 PollBatch:此时 +// bits[31:8] 是精确调用次数;直接 FetchMax 中同一区域仍表示软件重试数。 enum class AtomicOp : uint32_t { Load = 0, Exchange = 1, @@ -278,7 +294,98 @@ constexpr uint32_t kAtomicOpMask = 0x0fU; constexpr uint32_t kAtomicResultUsed = 1U << 4; constexpr uint32_t kAtomicValueZero = 1U << 5; constexpr uint32_t kAtomicReturnReady = 1U << 6; +constexpr uint32_t kAtomicPollBatch = 1U << 7; constexpr uint32_t kAtomicRetriesShift = 8; +constexpr uint32_t kAtomicPollCountShift = 8; +constexpr uint32_t kAtomicPollCountMax = 0x00ffffffU; +constexpr uint32_t kAtomicPollBatchSiteCount = 6; + +// 这些映射是 raw ABI 的一部分,同时被 device 聚合器与 host 闭环校验使用。 +// 0..14 与真实 PA 保持稳定;BlockWon 尚未在 standalone 中实现,不能只为 +// 编号齐全而追加没有真实调用路径的 site。 +#ifdef PA_DEVICE +#define PA_MODEL_INLINE PA_DEVICE +#else +#define PA_MODEL_INLINE inline +#endif + +PA_MODEL_INLINE constexpr AtomicOp AtomicSiteExpectedOp(AtomicSite site) { + switch (site) { + case AtomicSite::StartupIncrement: + case AtomicSite::ReplayDoneIncrement: + return AtomicOp::FetchAdd; + case AtomicSite::FatalSet: + case AtomicSite::CompletionVendExchange: + case AtomicSite::CompletionFlagExchange: + return AtomicOp::Exchange; + case AtomicSite::ClaimMax: + case AtomicSite::FrontierMax: + return AtomicOp::FetchMax; + default: + return AtomicOp::Load; + } +} + +PA_MODEL_INLINE constexpr bool AtomicSiteResultUsed(AtomicSite site) { + switch (site) { + case AtomicSite::StartupIncrement: + case AtomicSite::FatalSet: + case AtomicSite::CompletionVendExchange: + case AtomicSite::CompletionFlagExchange: + case AtomicSite::ReplayDoneIncrement: + return false; + default: + return true; + } +} + +PA_MODEL_INLINE constexpr int32_t AtomicPollBatchIndex(AtomicSite site) { + switch (site) { + case AtomicSite::StartupPoll: + return 0; + case AtomicSite::FatalPoll: + return 1; + case AtomicSite::FaninFlagLoad: + return 2; + case AtomicSite::HeapFrontierLoad: + return 3; + case AtomicSite::HeapVendLoad: + return 4; + case AtomicSite::ReplayDonePoll: + return 5; + default: + return -1; + } +} + +PA_MODEL_INLINE constexpr AtomicSite AtomicPollBatchSite(uint32_t index) { + switch (index) { + case 0: + return AtomicSite::StartupPoll; + case 1: + return AtomicSite::FatalPoll; + case 2: + return AtomicSite::FaninFlagLoad; + case 3: + return AtomicSite::HeapFrontierLoad; + case 4: + return AtomicSite::HeapVendLoad; + case 5: + return AtomicSite::ReplayDonePoll; + default: + return AtomicSite::Count; + } +} + +PA_MODEL_INLINE constexpr bool AtomicSiteIsPollBatchable(AtomicSite site) { + return AtomicPollBatchIndex(site) >= 0; +} + +PA_MODEL_INLINE constexpr uint32_t AtomicSiteMask(AtomicSite site) { + return 1U << static_cast(site); +} + +#undef PA_MODEL_INLINE // ClockBaseline 的 bit0 区分普通连续 SYS_CNT 与后端的 atomic 返回依赖 // 计时钩子;后者用于量化那一条依赖 MOV 自身带来的固定底噪。 @@ -288,7 +395,17 @@ constexpr uint32_t kClockAtomicDependencyApplied = 1U << 1; struct alignas(64) TraceCoreState { volatile uint32_t count; volatile uint32_t dropped; - uint32_t padding[14]; + // logical atomic 调用数与物理记录数分开闭合:PollBatch 的一条记录可以 + // 表示多次只读轮询,physical = atomic_calls - poll_calls + batch_records。 + volatile uint32_t atomic_calls; + volatile uint32_t poll_calls; + volatile uint32_t poll_batch_records; + // 拓扑在一个 worker 分区内恒定;当前仍保留 64B TraceRecord 兼容布局, + // 但在 core state 再保存一份权威身份,host 会验证每条记录与之相符。 + volatile int32_t core_idx; + volatile int32_t block_id; + volatile int32_t lane; + uint32_t padding[8]; }; // 每个 worker 独占一个计数 cache line 和一段定长 records,不需要为了写 trace // 再引入跨核 atomic;满容量后只增加本 worker 的 dropped。 diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index 12df67f5ca..e6c1ff2eee 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -31,6 +31,17 @@ struct LocalStats { TraceContext trace; }; +template +PA_DEVICE uint64_t TraceTimestamp(TraceContext &trace, WorkerResult &result) { + (void)result; + // 对齐真实 FDWIC 的 TRACE_SPAN_BEGIN/END:取一次时间后立即以同一 + // cycle 关闭活跃 PollBatch。不能在 WriteTrace 中统一关闭,否则直接 + // Atomic 记录也会错误切断等待 episode。 + const uint64_t cycle = Ops::Now(); + AtomicPollBoundaryAt(trace, cycle); + return cycle; +} + PA_DEVICE uint32_t KindIndex(TaskKind kind) { return static_cast(kind); } PA_DEVICE TaskKind GetTaskKind(uint32_t task_id) { return static_cast(task_id % kTasksPerBatch); } @@ -216,16 +227,16 @@ PA_DEVICE uint32_t DrainReady( continue; } const TaskKind kind = static_cast(slot.kind + 1); - const uint64_t kernel_begin = Ops::Now(); + const uint64_t kernel_begin = TraceTimestamp(stats.trace, stats.result); Ops::ExecuteKernel(state, worker, kind, NopCountForKind(state->config.nops, kind)); - const uint64_t kernel_end = Ops::Now(); + const uint64_t kernel_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(slot.task_id), static_cast(slot.kind), TracePhase::Kernel, ProfilePhase::ReplayTail, kernel_begin, kernel_end ); RecordKernelCycles(stats, kind, kernel_end - kernel_begin); CompleteTask(state, worker, slot.task_id, stats); - const uint64_t commit_cycle = Ops::Now(); + const uint64_t commit_cycle = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(slot.task_id), static_cast(slot.kind), TracePhase::Commit, ProfilePhase::ReplayTail, commit_cycle, commit_cycle @@ -253,8 +264,13 @@ PA_DEVICE void WaitForSlot( PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_id, LocalStats &stats ) { // 四个物理 slot 中预留两个 won slot 语义位,仅有 kUsableSlots 个可供本图使用;满时靠 drain 取得进展。 - const uint64_t wait_begin = Ops::Now(); + const uint64_t wait_begin = TraceTimestamp(stats.trace, stats.result); bool waited = false; + // 只聚合这个显式背压等待区中的 fanin 观察;每次 Submit 开头的 + // opportunistic EfDrain 仍保留逐条 Atomic,不能仅凭 site 名称全局聚合。 + const uint32_t poll_region = AtomicPollRegionBegin( + stats.trace, stats.result, TraceAtomicSiteMask(AtomicSite::FaninFlagLoad) + ); // 退出条件只有 occupied_count 重新低于可用容量;依赖尚未 ready 时 SpinHint 后继续重试。 while (worker.occupied_count >= kUsableSlots) { waited = true; @@ -265,15 +281,19 @@ PA_DEVICE void WaitForSlot( Ops::SpinHint(); } } + AtomicPollRegionEnd(stats.trace, stats.result, poll_region); if (waited) { ++stats.result.wait_events[0]; - const uint64_t wait_end = Ops::Now(); + const uint64_t wait_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::RingBp, ProfilePhase::WaitForSlot, wait_begin, wait_end, 0, 0 ); } else if constexpr (Profile) { - AccumulatePhase(stats.result, ProfilePhase::WaitForSlot, wait_begin, Ops::Now()); + AccumulatePhase( + stats.result, ProfilePhase::WaitForSlot, wait_begin, + TraceTimestamp(stats.trace, stats.result) + ); } } @@ -289,18 +309,38 @@ PA_DEVICE bool HeapGuard( } const uint64_t ring = state->heap_size; ++stats.result.heap_guards; - const uint64_t wait_begin = Ops::Now(); + const uint64_t wait_begin = TraceTimestamp(stats.trace, stats.result); bool waited = false; + bool poll_region_active = false; + uint32_t poll_region = 0; // 正常出口是 heap_next-vend 落入一个 ring;检测到不可能释放的覆盖或其他核 fatal 时返回失败。 while (!IsFatal(state, stats, static_cast(task_id))) { // 逻辑 heap 尚未走完第一圈时,所有物理输出区间都位于 [0, heap_next), // 不可能覆盖此前分配;保留上面的 fatal 原子检查后,可直接跳过 frontier/vend 读取。 if (worker.heap_next <= ring) { + if (poll_region_active) { + AtomicPollRegionEnd(stats.trace, stats.result, poll_region); + } if constexpr (Profile) { - AccumulatePhase(stats.result, ProfilePhase::HeapGuard, wait_begin, Ops::Now()); + AccumulatePhase( + stats.result, ProfilePhase::HeapGuard, wait_begin, + TraceTimestamp(stats.trace, stats.result) + ); } return true; } + // 与真实 PA 一样,首圈 fast path 上方的 FatalPoll 仍是直接记录;只有 + // 确认进入 heap wrap 慢路径后,才开启本等待 episode 的四类观察聚合。 + if (!poll_region_active) { + poll_region = AtomicPollRegionBegin( + stats.trace, stats.result, + TraceAtomicSiteMask(AtomicSite::FatalPoll) | + TraceAtomicSiteMask(AtomicSite::HeapFrontierLoad) | + TraceAtomicSiteMask(AtomicSite::HeapVendLoad) | + TraceAtomicSiteMask(AtomicSite::FaninFlagLoad) + ); + poll_region_active = true; + } const int64_t frontier = LoadLine( state->frontier, stats, AtomicSite::HeapFrontierLoad, static_cast(task_id) ); @@ -312,20 +352,25 @@ PA_DEVICE bool HeapGuard( &state->tasks[retire].vend ); if (worker.heap_next - vend <= ring) { + AtomicPollRegionEnd(stats.trace, stats.result, poll_region); if (waited) { ++stats.result.wait_events[1]; - const uint64_t wait_end = Ops::Now(); + const uint64_t wait_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::RingBp, ProfilePhase::HeapGuard, wait_begin, wait_end, 0, 1 ); } else if constexpr (Profile) { - AccumulatePhase(stats.result, ProfilePhase::HeapGuard, wait_begin, Ops::Now()); + AccumulatePhase( + stats.result, ProfilePhase::HeapGuard, wait_begin, + TraceTimestamp(stats.trace, stats.result) + ); } return true; } if (frontier >= static_cast(task_id) - 1) { SetFatal(state, stats, static_cast(task_id)); + AtomicPollRegionEnd(stats.trace, stats.result, poll_region); return false; } waited = true; @@ -336,9 +381,12 @@ PA_DEVICE bool HeapGuard( Ops::SpinHint(); } } + if (poll_region_active) { + AtomicPollRegionEnd(stats.trace, stats.result, poll_region); + } if (waited) { ++stats.result.wait_events[1]; - const uint64_t wait_end = Ops::Now(); + const uint64_t wait_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::RingBp, ProfilePhase::HeapGuard, wait_begin, wait_end, 0, 1 @@ -470,12 +518,12 @@ PA_DEVICE bool SubmitTask( // 且 winner 不入 kernel slot,而是在 heap guard 后直接发布完成。 BeginSubmit(worker, args, context); const uint32_t task_id = static_cast(context.task_id); - const uint64_t submit_begin = Ops::Now(); + const uint64_t submit_begin = TraceTimestamp(stats.trace, stats.result); if (task_id == 0) { stats.result.submit_begin = submit_begin; } - ResetTraceLap(worker); + ResetTraceLap(stats.trace, stats.result, worker); // EfDrain 在当前 Submit 的参数物化前执行上一批已就绪 slot,是绝大多数 kernel 的正常落点。 DrainReady(state, worker, DrainPlace::EfDrain, stats); WriteTraceLap( @@ -487,13 +535,13 @@ PA_DEVICE bool SubmitTask( // two independently traced spans. Build/Replay later consumes this origin. // lap 起点在 materialize 前重置;Materialize/PrepareMap 各自取独立绝对区间,而后续 // Build/Replay 会从这个起点形成覆盖式 span。因此泳道上的这些阶段不能直接相加。 - ResetTraceLap(worker); - const uint64_t materialize_begin = Ops::Now(); + ResetTraceLap(stats.trace, stats.result, worker); + const uint64_t materialize_begin = TraceTimestamp(stats.trace, stats.result); if (!MaterializeTask(worker, task_id, args, context, state->heap_base, state->heap_size)) { SetFatal(state, stats, static_cast(task_id)); return false; } - const uint64_t materialize_end = Ops::Now(); + const uint64_t materialize_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Materialize, ProfilePhase::Materialize, materialize_begin, materialize_end, 0, @@ -501,9 +549,9 @@ PA_DEVICE bool SubmitTask( ); stats.result.materialized_outputs += context.result.count; - const uint64_t prepare_begin = Ops::Now(); + const uint64_t prepare_begin = TraceTimestamp(stats.trace, stats.result); AdvanceTensorMap(worker.map, task_id, static_cast(state->heap_window)); - const uint64_t prepare_end = Ops::Now(); + const uint64_t prepare_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::PrepareMap, ProfilePhase::PrepareMap, prepare_begin, prepare_end, 0, kind == TaskKind::Alloc ? 1U : 0U @@ -514,20 +562,20 @@ PA_DEVICE bool SubmitTask( if (kind == TaskKind::Alloc) { // Alloc 没有 kernel lane,96 个 worker 都维护本地物化/heap 状态,但只有 Claim winner 发布全局完成。 - const uint64_t register_begin = Ops::Now(); + const uint64_t register_begin = TraceTimestamp(stats.trace, stats.result); RegisterOutputs(context, args, false); - const uint64_t register_end = Ops::Now(); + const uint64_t register_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Register, ProfilePhase::Register, register_begin, register_end, 0, 0 ); - const uint64_t claim_begin = Ops::Now(); + const uint64_t claim_begin = TraceTimestamp(stats.trace, stats.result); const ClaimOutcome claim = Claim(state, worker, task_id, kind, stats); winner = claim.won; context.won = winner; context.kernel_id = claim.function_id; - const uint64_t claim_end = Ops::Now(); + const uint64_t claim_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Claim, ProfilePhase::Claim, claim_begin, claim_end, @@ -553,13 +601,13 @@ PA_DEVICE bool SubmitTask( ); } } else { - const uint64_t claim_begin = Ops::Now(); + const uint64_t claim_begin = TraceTimestamp(stats.trace, stats.result); const ClaimOutcome claim = Claim(state, worker, task_id, kind, stats); winner = claim.won; function_id = claim.function_id; context.won = winner; context.kernel_id = function_id; - const uint64_t claim_end = Ops::Now(); + const uint64_t claim_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Claim, ProfilePhase::Claim, claim_begin, claim_end, @@ -568,9 +616,9 @@ PA_DEVICE bool SubmitTask( RecordClaimOutcome(stats, kind, claim); if (winner) { - const uint64_t fanin_begin = Ops::Now(); + const uint64_t fanin_begin = TraceTimestamp(stats.trace, stats.result); context.fanin_count = static_cast(CollectFanin(worker.map, args, context.fanin)); - const uint64_t fanin_end = Ops::Now(); + const uint64_t fanin_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Fanin, ProfilePhase::Fanin, fanin_begin, fanin_end, 0, static_cast(context.fanin_count) @@ -578,9 +626,9 @@ PA_DEVICE bool SubmitTask( stats.result.map_lookups += static_cast(args.tensor_count) - context.result.count; } - const uint64_t register_begin = Ops::Now(); + const uint64_t register_begin = TraceTimestamp(stats.trace, stats.result); RegisterOutputs(context, args, true); - const uint64_t register_end = Ops::Now(); + const uint64_t register_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Register, ProfilePhase::Register, register_begin, register_end, 0, 1 @@ -610,7 +658,7 @@ PA_DEVICE bool SubmitTask( } ++stats.result.submits; - const uint64_t submit_end = Ops::Now(); + const uint64_t submit_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Submit, ProfilePhase::Submit, submit_begin, submit_end, winner ? 1U : 0U, kind == TaskKind::Alloc ? 1U : 0U @@ -741,6 +789,10 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, ); const uint64_t start_wait = Ops::Now(); uint32_t start_polls = 0; + const uint32_t startup_poll_region = AtomicPollRegionBegin( + stats.trace, stats.result, + TraceAtomicSiteMask(AtomicSite::StartupPoll) | TraceAtomicSiteMask(AtomicSite::FatalPoll) + ); // 全员到齐或任一核发布 fatal 即退出启动等待;watchdog 防止缺失参与者造成永久挂死。 while (LoadLine(state->started_count, stats, AtomicSite::StartupPoll) < static_cast(state->config.workers) && @@ -750,6 +802,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, break; } } + AtomicPollRegionEnd(stats.trace, stats.result, startup_poll_region); const uint32_t batches = state->config.batches; const uint32_t task_count = batches * kTasksPerBatch; @@ -761,9 +814,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, // CCEC 可在这里开启本 worker 私有 PMU 窗口;CPU/AscendC 适配层是空实现。 // 窗口覆盖从首个参数构造到末次 Submit 返回,与全局“首 Submit.begin~末 Submit.end” // 口径接近但不相同,host sidecar 必须按 per-worker 累计解释。 - ResetTraceLap(worker); - // lap 重置属于泳道观察自身,不应污染 PMU-only 的 Submit 取数;窗口从 - // orchestration 初始化(即首批参数构造)前一条边界开始。 + ResetTraceLap(stats.trace, stats.result, worker); const bool pmu_window_started = Ops::PmuWindowStart(state, worker_id); InitPaOrchestration(orchestration, batches, &state->context_lens[0]); for (uint32_t batch = 0; batch < batches; ++batch) { @@ -831,6 +882,10 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, stats.trace, stats.result, -1, AtomicSite::ReplayDoneIncrement, &state->replay_done.value, 1 ); + const uint32_t final_poll_region = AtomicPollRegionBegin( + stats.trace, stats.result, + TraceAtomicSiteMask(AtomicSite::ReplayDonePoll) | TraceAtomicSiteMask(AtomicSite::FaninFlagLoad) + ); while (true) { const uint32_t freed = DrainReady(state, worker, DrainPlace::FinalDrain, stats); @@ -845,6 +900,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, Ops::SpinHint(); } } + AtomicPollRegionEnd(stats.trace, stats.result, final_poll_region); if (stats.trace.atomics_enabled) { // 两条基线都放在最终 drain 之后。第一条量连续 @@ -870,7 +926,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, // PA writes swimlane records through the ordinary GM cache and explicitly // cleans each worker's record range before the kernel finishes. - FlushTraceCore(stats.trace); + FlushTraceCore(stats.trace, stats.result); stats.result.finish_cycle = Ops::Now(); stats.result.max_occupied = stats.max_occupied; stats.result.final_occupied = worker.occupied_count; @@ -884,12 +940,19 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, template PA_DEVICE void RunScheduler(PA_GM SchedulerState *state, uint32_t worker_id, CoreRole role) { + // 两个正式 CCEC 构建都不再携带旧 phase-profile 模板副本:swimlane 用 + // records 表达阶段,submit-pmu 使用独立 PMU 边界。其他后端暂时保留原 + // 运行时入口,保证公共 standalone 的 CPU/AscendC 回归不被 CCEC 构建切分影响。 +#if PA_BUILD_SWIMLANE || PA_BUILD_SUBMIT_PMU + RunSchedulerImpl(state, worker_id, role); +#else // Profile 作为编译期模板参数,只在显式开启时保留阶段累计代码,关闭时不在热路径增加运行时分支。 if (state->config.profile_phases != 0) { RunSchedulerImpl(state, worker_id, role); } else { RunSchedulerImpl(state, worker_id, role); } +#endif } } // namespace pa_scheduler diff --git a/tests/atomic_probe/pa_scheduler/common/pa_trace.h b/tests/atomic_probe/pa_scheduler/common/pa_trace.h index f6f88111fb..511ec8eacd 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_trace.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_trace.h @@ -14,6 +14,14 @@ #include "pa_model.h" +#ifndef PA_DEVICE_NOINLINE +#define PA_DEVICE_NOINLINE PA_DEVICE +#endif + +#ifndef PA_LOOP_NOUNROLL +#define PA_LOOP_NOUNROLL +#endif + namespace pa_scheduler { // 记录区与真实 PA 一样直接拼在定长 Header 后面;worker_id 只选择自己的 @@ -24,6 +32,13 @@ PA_DEVICE PA_GM TraceRecord *GetTraceRecords(PA_GM TraceHeader *header) { return reinterpret_cast(reinterpret_cast(header) + sizeof(TraceHeader)); } +struct AtomicPollBurst { + uint64_t start_cycle[kAtomicPollBatchSiteCount]; + uint32_t call_count[kAtomicPollBatchSiteCount]; + uint32_t active_mask; + uint32_t enabled_mask; +}; + struct TraceContext { PA_GM TraceCoreState *core; PA_GM TraceRecord *records; @@ -32,15 +47,90 @@ struct TraceContext { int32_t lane; int32_t block_id; int32_t core_idx; + // 轮询调用数留在 worker 私有上下文,最终一次性发布到 core state; + // 等待热路不为计数再写共享/GM 状态。 + uint64_t poll_calls; + uint64_t poll_batch_records; + bool atomic_counter_overflow; + AtomicPollBurst poll_burst; }; +// pa_model.h 也向 host 暴露同一 raw ABI 映射,但 CCEC/AscendC 的单个 TU +// 会先以 host 语境包含该头,再实例化 device 调度器。这里保留明确的 device +// 版本,避免设备函数误调用先前已实例化的 __host__ helper。 +PA_DEVICE AtomicOp TraceAtomicSiteExpectedOp(AtomicSite site) { + switch (site) { + case AtomicSite::StartupIncrement: + case AtomicSite::ReplayDoneIncrement: + return AtomicOp::FetchAdd; + case AtomicSite::FatalSet: + case AtomicSite::CompletionVendExchange: + case AtomicSite::CompletionFlagExchange: + return AtomicOp::Exchange; + case AtomicSite::ClaimMax: + case AtomicSite::FrontierMax: + return AtomicOp::FetchMax; + default: + return AtomicOp::Load; + } +} + +PA_DEVICE int32_t TraceAtomicPollBatchIndex(AtomicSite site) { + switch (site) { + case AtomicSite::StartupPoll: + return 0; + case AtomicSite::FatalPoll: + return 1; + case AtomicSite::FaninFlagLoad: + return 2; + case AtomicSite::HeapFrontierLoad: + return 3; + case AtomicSite::HeapVendLoad: + return 4; + case AtomicSite::ReplayDonePoll: + return 5; + default: + return -1; + } +} + +PA_DEVICE AtomicSite TraceAtomicPollBatchSite(uint32_t index) { + switch (index) { + case 0: + return AtomicSite::StartupPoll; + case 1: + return AtomicSite::FatalPoll; + case 2: + return AtomicSite::FaninFlagLoad; + case 3: + return AtomicSite::HeapFrontierLoad; + case 4: + return AtomicSite::HeapVendLoad; + case 5: + return AtomicSite::ReplayDonePoll; + default: + return AtomicSite::Count; + } +} + +PA_DEVICE bool TraceAtomicSiteIsPollBatchable(AtomicSite site) { + return TraceAtomicPollBatchIndex(site) >= 0; +} + +PA_DEVICE uint32_t TraceAtomicSiteMask(AtomicSite site) { + return 1U << static_cast(site); +} + // Attach 只缓存本 worker 的 header 状态、分区首址和物理 lane 信息。配置先做 // cache invalidate,确保 A5 worker 看到 host 在 launch 前写入的 trace 开关、地址与 winner 负载配置。 template PA_DEVICE TraceContext AttachTrace( PA_GM SchedulerState *state, PA_GM const WorkerState &worker, uint32_t worker_id ) { - TraceContext trace{nullptr, nullptr, 0, false, worker.lane, worker.block_id, static_cast(worker_id)}; + TraceContext trace{}; + trace.lane = worker.lane; + trace.block_id = worker.block_id; + trace.core_idx = static_cast(worker_id); Ops::InvalidateRegion( &state->config, sizeof(state->config) + sizeof(state->winner_workload) ); @@ -62,6 +152,12 @@ PA_DEVICE TraceContext AttachTrace( trace.atomics_enabled = (state->config.trace_enabled & kTraceAtomicsEnabled) != 0; trace.core->count = 0; trace.core->dropped = 0; + trace.core->atomic_calls = 0; + trace.core->poll_calls = 0; + trace.core->poll_batch_records = 0; + trace.core->core_idx = trace.core_idx; + trace.core->block_id = trace.block_id; + trace.core->lane = trace.lane; return trace; } @@ -72,6 +168,40 @@ PA_DEVICE void WriteTrace( uint32_t flags = 0, uint32_t auxiliary = 0 ); +// CCEC 不让栈上的 TraceContext/WorkerResult 引用跨非内联调用。这里仅把 +// PollBatch 固定形状的 64-byte GM 写入抽成共享函数,以抑制各 phase 边界 +// 内联后的代码膨胀;参数只有 GM 指针与标量,局部 batch 状态仍由调用者维护。 +PA_DEVICE_NOINLINE bool WritePollBatchRecordRaw( + PA_GM TraceCoreState *core, PA_GM TraceRecord *records, uint32_t capacity, + uint64_t start_cycle, uint64_t end_cycle, uint32_t call_count, uint32_t site_id +) { + if (core == nullptr || records == nullptr || capacity == 0) { + return false; + } + const uint32_t slot = core->count; + if (slot >= capacity) { + core->dropped = core->dropped + 1; + return false; + } + PA_GM TraceRecord &record = records[slot]; + record.start_cycle = start_cycle; + record.end_cycle = end_cycle; + record.task_id = -1; + record.function_id = -1; + record.phase = static_cast(TracePhase::Atomic); + record.lane = core->lane; + record.block_id = core->block_id; + record.core_idx = core->core_idx; + const AtomicSite site = static_cast(site_id); + record.flags = static_cast(TraceAtomicSiteExpectedOp(site)) | + kAtomicResultUsed | kAtomicPollBatch | + (call_count << kAtomicPollCountShift); + record.auxiliary = site_id; + // count 最后更新,使其始终指向下一空槽;单写者条件下无需 reserve/commit 两阶段。 + core->count = slot + 1; + return true; +} + PA_DEVICE uint32_t AtomicTraceFlags( AtomicOp op, bool result_used, bool return_ready, bool value_zero = false, uint64_t retries = 0 @@ -85,6 +215,115 @@ PA_DEVICE uint32_t AtomicTraceFlags( (encoded_retries << kAtomicRetriesShift); } +PA_DEVICE void CountAtomicCall( + TraceContext &trace, WorkerResult &result, bool poll_batch +) { + if (result.atomic_trace_calls == UINT64_MAX) { + trace.atomic_counter_overflow = true; + return; + } + ++result.atomic_trace_calls; + if (!poll_batch) return; + if (trace.poll_calls == UINT64_MAX) { + trace.atomic_counter_overflow = true; + return; + } + ++trace.poll_calls; +} + +template +PA_DEVICE void AtomicPollBoundaryAt( + TraceContext &trace, uint64_t end_cycle +) { + if (!trace.atomics_enabled || trace.poll_burst.active_mask == 0) return; + const uint32_t active_mask = trace.poll_burst.active_mask; + // CCEC 默认会把固定 6-site 循环完整展开,再随几十个 phase 边界复制。 + // 禁止展开只控制代码体积;循环次数、site 顺序和同 cycle 关闭语义不变。 + PA_LOOP_NOUNROLL + for (uint32_t index = 0; index < kAtomicPollBatchSiteCount; ++index) { + const uint32_t bit = 1U << index; + if ((active_mask & bit) == 0) continue; + const uint32_t call_count = trace.poll_burst.call_count[index]; + if (call_count == 0 || call_count > kAtomicPollCountMax) { + trace.atomic_counter_overflow = true; + continue; + } + const AtomicSite site = TraceAtomicPollBatchSite(index); + const bool written = WritePollBatchRecordRaw( + trace.core, trace.records, trace.capacity, + trace.poll_burst.start_cycle[index], end_cycle, + call_count, static_cast(site) + ); + if (written) { + if (trace.poll_batch_records == UINT64_MAX) { + trace.atomic_counter_overflow = true; + } else { + ++trace.poll_batch_records; + } + } + trace.poll_burst.call_count[index] = 0; + } + trace.poll_burst.active_mask = 0; +} + +template +PA_DEVICE void AtomicPollBoundary(TraceContext &trace, WorkerResult &result) { + (void)result; + if (trace.poll_burst.active_mask == 0) return; + AtomicPollBoundaryAt(trace, Ops::Now()); +} + +template +PA_DEVICE uint32_t AtomicPollRegionBegin( + TraceContext &trace, WorkerResult &result, uint32_t site_mask +) { + const uint32_t previous_mask = trace.poll_burst.enabled_mask; + if (!trace.atomics_enabled) return previous_mask; + AtomicPollBoundary(trace, result); + trace.poll_burst.enabled_mask = previous_mask | site_mask; + return previous_mask; +} + +template +PA_DEVICE void AtomicPollRegionEnd( + TraceContext &trace, WorkerResult &result, uint32_t previous_mask +) { + if (!trace.atomics_enabled) return; + AtomicPollBoundary(trace, result); + trace.poll_burst.enabled_mask = previous_mask; +} + +PA_DEVICE bool AtomicPollBatchEnabled( + TraceContext &trace, AtomicSite site, AtomicOp actual_op +) { + return trace.atomics_enabled && TraceAtomicSiteIsPollBatchable(site) && + TraceAtomicSiteExpectedOp(site) == actual_op && + (trace.poll_burst.enabled_mask & TraceAtomicSiteMask(site)) != 0; +} + +template +PA_DEVICE void AccumulateAtomicPollCall( + TraceContext &trace, WorkerResult &result, AtomicSite site, uint64_t start_cycle +) { + const int32_t signed_index = TraceAtomicPollBatchIndex(site); + if (signed_index < 0) { + trace.atomic_counter_overflow = true; + return; + } + const uint32_t index = static_cast(signed_index); + const uint32_t bit = 1U << index; + if ((trace.poll_burst.active_mask & bit) == 0) { + trace.poll_burst.start_cycle[index] = start_cycle; + trace.poll_burst.call_count[index] = 0; + trace.poll_burst.active_mask |= bit; + } + uint32_t &call_count = trace.poll_burst.call_count[index]; + ++call_count; + if (call_count == kAtomicPollCountMax) { + AtomicPollBoundary(trace, result); + } +} + template PA_DEVICE void WriteAtomicTrace( TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, AtomicOp op, @@ -93,7 +332,7 @@ PA_DEVICE void WriteAtomicTrace( ) { // 一次源码 atomic 只写一条同时含 start/end 的 span;结束时间先于 64B record // 写入,因此本条区间不直接包含自己的记录写开销,但下一次竞争到达会受它影响。 - ++result.atomic_trace_calls; + CountAtomicCall(trace, result, false); WriteTrace( trace, result, task_id, -1, TracePhase::Atomic, ProfilePhase::ReplayTail, start_cycle, end_cycle, @@ -108,8 +347,17 @@ PA_DEVICE T TraceAtomicLoad( PA_GM volatile T *address, bool result_used = true ) { if (!trace.atomics_enabled) return Ops::Load(address); - const uint64_t begin = Ops::Now(); + const bool poll_batch = result_used && AtomicPollBatchEnabled(trace, site, AtomicOp::Load); + const int32_t poll_index = poll_batch ? TraceAtomicPollBatchIndex(site) : -1; + const bool first_in_batch = poll_batch && + (trace.poll_burst.active_mask & (1U << static_cast(poll_index))) == 0; + const uint64_t begin = !poll_batch || first_in_batch ? Ops::Now() : 0; const T old = Ops::Load(address); + if (poll_batch) { + CountAtomicCall(trace, result, true); + AccumulateAtomicPollCall(trace, result, site, begin); + return old; + } // CCEC 只在返回值本来就参与协议判断时插入一条依赖 MOV,再读 SYS_CNT。 // 这样不会把未消费返回值的 RED/no-return 路径强制改成返回型 ATOM。 const bool return_ready = result_used && Ops::kAtomicReturnReadyObserved; @@ -129,9 +377,10 @@ PA_DEVICE T TraceAtomicExchange( if (!trace.atomics_enabled) return Ops::Exchange(address, value); const uint64_t begin = Ops::Now(); const T old = Ops::Exchange(address, value); - const uint64_t end = Ops::Now(); + const bool return_ready = result_used && Ops::kAtomicReturnReadyObserved; + const uint64_t end = result_used ? Ops::NowAfterAtomicResult(old) : Ops::Now(); WriteAtomicTrace( - trace, result, task_id, site, AtomicOp::Exchange, begin, end, result_used, false + trace, result, task_id, site, AtomicOp::Exchange, begin, end, result_used, return_ready ); return old; } @@ -144,9 +393,10 @@ PA_DEVICE int64_t TraceAtomicFetchAdd( if (!trace.atomics_enabled) return Ops::FetchAdd(address, value); const uint64_t begin = Ops::Now(); const int64_t old = Ops::FetchAdd(address, value); - const uint64_t end = Ops::Now(); + const bool return_ready = result_used && Ops::kAtomicReturnReadyObserved; + const uint64_t end = result_used ? Ops::NowAfterAtomicResult(old) : Ops::Now(); WriteAtomicTrace( - trace, result, task_id, site, AtomicOp::FetchAdd, begin, end, result_used, false + trace, result, task_id, site, AtomicOp::FetchAdd, begin, end, result_used, return_ready ); return old; } @@ -223,18 +473,35 @@ PA_DEVICE void WriteTrace( } template -PA_DEVICE void ResetTraceLap(PA_GM WorkerState &worker) { +PA_DEVICE void ResetTraceLap( + TraceContext &trace, WorkerResult &result, PA_GM WorkerState &worker +) { // lap 是后续 Build/Replay/Alloc 等覆盖式阶段的共同起点,不代表新增嵌套 span。 // 因此分析时不能把 lap 时长再与其中的 Materialize/Claim/Register 直接相加。 - worker.swimlane_last_cycle = Ops::Now(); + (void)result; + const uint64_t cycle = Ops::Now(); + // 与真实 FDWIC 的 TRACE_LAP_RESET 保持同一边界:等待区 PollBatch + // 只能覆盖本次逻辑轮询 episode,不能跨进下一段 lap 或计算单元执行。 + AtomicPollBoundaryAt(trace, cycle); + worker.swimlane_last_cycle = cycle; } template -PA_DEVICE void FlushTraceCore(TraceContext &trace) { +PA_DEVICE void FlushTraceCore(TraceContext &trace, WorkerResult &result) { if (trace.core == nullptr || trace.records == nullptr || trace.capacity == 0) { return; } + // 防御性关闭任何尚未由显式 region end 关闭的等待包;正常路径上 active_mask + // 应为 0,这里仍保证异常早退不会留下“有逻辑调用、无物理 batch”的半截采集。 + AtomicPollBoundary(trace, result); PA_GM TraceCoreState &core = *trace.core; + if (trace.atomic_counter_overflow || result.atomic_trace_calls > UINT32_MAX || + trace.poll_calls > UINT32_MAX || trace.poll_batch_records > UINT32_MAX) { + if (core.dropped != UINT32_MAX) core.dropped = core.dropped + 1; + } + core.atomic_calls = static_cast(result.atomic_trace_calls); + core.poll_calls = static_cast(trace.poll_calls); + core.poll_batch_records = static_cast(trace.poll_batch_records); const uint32_t count = core.count < trace.capacity ? core.count : trace.capacity; // A5 侧记录经普通 GM cache 写入,kernel 结束前必须把有效 records 与最后的 // count/dropped cache line 显式 clean,host 的 D2H 才能得到完整且自洽的快照。 @@ -253,6 +520,9 @@ PA_DEVICE uint64_t WriteTraceLap( // lap 记录区间 [上一次 Reset/WriteTraceLap, 当前时刻],写完立即推进起点。 // 显式 WriteTrace span 不会修改该起点,这正是生产泳道中阶段可重叠的原因。 const uint64_t end_cycle = Ops::Now(); + // 真实 FDWIC 在 TRACE_LAP 取到结束时间后先关闭 PollBatch,再写 lap。 + // 复用同一个 end_cycle,避免额外 SYS_CNT 造成可见缝隙。 + AtomicPollBoundaryAt(trace, end_cycle); WriteTrace( trace, result, task_id, function_id, trace_phase, profile_phase, worker.swimlane_last_cycle, end_cycle, flags, auxiliary diff --git a/tests/atomic_probe/pa_scheduler/common/test_atomic_poll_batch.cpp b/tests/atomic_probe/pa_scheduler/common/test_atomic_poll_batch.cpp new file mode 100644 index 0000000000..5795765788 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/common/test_atomic_poll_batch.cpp @@ -0,0 +1,280 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +#include +#include +#include + +// 该自测在普通 CPU 编译器上直接实例化 device 公共模板:只消去地址空间 +// 修饰符,不复制 PollBatch 实现,避免测试与被测代码各维护一套逻辑。 +#define PA_DEVICE inline +#define PA_GM +#include "pa_trace.h" + +namespace { + +using pa_scheduler::AtomicOp; +using pa_scheduler::AtomicPollBatchEnabled; +using pa_scheduler::AtomicPollBoundaryAt; +using pa_scheduler::AtomicPollRegionBegin; +using pa_scheduler::AtomicPollRegionEnd; +using pa_scheduler::AtomicSite; +using pa_scheduler::AccumulateAtomicPollCall; +using pa_scheduler::TraceAtomicLoad; +using pa_scheduler::TraceAtomicPollBatchIndex; +using pa_scheduler::TraceAtomicSiteMask; +using pa_scheduler::TraceContext; +using pa_scheduler::TraceCoreState; +using pa_scheduler::TracePhase; +using pa_scheduler::TraceRecord; +using pa_scheduler::WorkerResult; +using pa_scheduler::kAtomicPollBatch; +using pa_scheduler::kAtomicPollCountMax; +using pa_scheduler::kAtomicPollCountShift; + +int g_failures = 0; + +void Expect(bool condition, const char *message) { + if (condition) return; + std::fprintf(stderr, "[FAIL] atomic PollBatch: %s\n", message); + ++g_failures; +} + +// 可控时钟让边界断言不依赖 host 调度;Load 只为验证 trace 包装器的分流, +// 不尝试在这个单线程单元测试中模拟 A5 atomicAdd(0) 的硬件时延。 +struct TestOps { + static constexpr bool kAtomicReturnReadyObserved = false; + static uint64_t now; + + static uint64_t Now() { return now++; } + + template + static uint64_t NowAfterAtomicResult(T value) { + (void)value; + return Now(); + } + + template + static T Load(volatile T *address) { + return *address; + } +}; + +uint64_t TestOps::now = 0; + +struct Fixture { + TraceCoreState core{}; + TraceRecord records[8]{}; + WorkerResult result{}; + TraceContext trace{}; + + Fixture() { + trace.core = &core; + trace.records = records; + trace.capacity = static_cast(sizeof(records) / sizeof(records[0])); + trace.atomics_enabled = true; + } +}; + +void TestSplitAtMaximumCount() { + Fixture fixture; + constexpr AtomicSite kSite = AtomicSite::StartupPoll; + constexpr uint32_t kBatchIndex = 0; + constexpr uint32_t kBatchBit = 1U << kBatchIndex; + constexpr uint64_t kFirstStart = 111; + constexpr uint64_t kSecondStart = 222; + constexpr uint64_t kSecondEnd = 333; + + Expect( + TraceAtomicPollBatchIndex(kSite) == static_cast(kBatchIndex), + "StartupPoll 的 batch index 与稳定映射不一致" + ); + fixture.trace.poll_burst.active_mask = kBatchBit; + fixture.trace.poll_burst.start_cycle[kBatchIndex] = kFirstStart; + fixture.trace.poll_burst.call_count[kBatchIndex] = kAtomicPollCountMax - 1; + TestOps::now = 200; + + // 第 0xFFFFFF 次调用属于第一条记录,并在达到 24-bit 上限时立即落盘。 + AccumulateAtomicPollCall(fixture.trace, fixture.result, kSite, kFirstStart); + + Expect(fixture.core.count == 1U, "达到最大计数时应立即写出第一条记录"); + Expect(fixture.trace.poll_batch_records == 1U, "第一条 PollBatch 物理记录计数错误"); + Expect(fixture.trace.poll_burst.active_mask == 0U, "达到上限后 active mask 应清零"); + Expect( + fixture.trace.poll_burst.call_count[kBatchIndex] == 0U, + "达到上限后站点调用计数应清零" + ); + Expect( + fixture.records[0].phase == static_cast(TracePhase::Atomic), + "第一条记录 phase 不是 Atomic" + ); + Expect( + fixture.records[0].auxiliary == static_cast(kSite), + "第一条记录 site 不正确" + ); + Expect(fixture.records[0].start_cycle == kFirstStart, "第一条记录起始时钟不正确"); + Expect( + fixture.records[0].end_cycle >= fixture.records[0].start_cycle, + "第一条记录的结束时钟早于起始时钟" + ); + Expect( + (fixture.records[0].flags & kAtomicPollBatch) != 0U, + "第一条记录缺少 PollBatch 标志" + ); + Expect( + fixture.records[0].flags >> kAtomicPollCountShift == kAtomicPollCountMax, + "第一条记录没有编码最大 24-bit 调用数" + ); + + // 第 0x1000000 次调用必须重新开启 count=1 的新 batch,不能饱和或丢失。 + AccumulateAtomicPollCall(fixture.trace, fixture.result, kSite, kSecondStart); + Expect(fixture.trace.poll_burst.active_mask == kBatchBit, "上限后的下一次调用没有重开 batch"); + Expect( + fixture.trace.poll_burst.call_count[kBatchIndex] == 1U, + "重开 batch 的初始调用数不是 1" + ); + AtomicPollBoundaryAt(fixture.trace, kSecondEnd); + + Expect(fixture.core.count == 2U, "max+1 次调用应写出两条记录"); + Expect(fixture.trace.poll_batch_records == 2U, "max+1 次调用的物理 batch 数错误"); + Expect(fixture.trace.poll_burst.active_mask == 0U, "第二条记录关闭后 active mask 未清零"); + Expect( + fixture.trace.poll_burst.call_count[kBatchIndex] == 0U, + "第二条记录关闭后站点调用计数未清零" + ); + Expect(fixture.records[1].start_cycle == kSecondStart, "第二条记录起始时钟不正确"); + Expect(fixture.records[1].end_cycle == kSecondEnd, "第二条记录结束时钟不正确"); + Expect( + (fixture.records[1].flags & kAtomicPollBatch) != 0U, + "第二条记录缺少 PollBatch 标志" + ); + Expect( + fixture.records[1].flags >> kAtomicPollCountShift == 1U, + "第二条记录没有精确编码一次调用" + ); + const uint64_t represented_calls = + static_cast(fixture.records[0].flags >> kAtomicPollCountShift) + + static_cast(fixture.records[1].flags >> kAtomicPollCountShift); + Expect( + represented_calls == static_cast(kAtomicPollCountMax) + 1U, + "两条记录的加权调用数没有闭合到 max+1" + ); + Expect(fixture.core.dropped == 0U, "边界拆批不应丢记录"); + Expect(!fixture.trace.atomic_counter_overflow, "边界拆批不应报告计数溢出"); +} + +void TestNestedRegionRestoresMask() { + Fixture fixture; + volatile int64_t startup_value = 96; + volatile int64_t fanin_value = 1; + const uint32_t startup_mask = TraceAtomicSiteMask(AtomicSite::StartupPoll); + const uint32_t fanin_mask = TraceAtomicSiteMask(AtomicSite::FaninFlagLoad); + TestOps::now = 1000; + + const uint32_t outer_previous = + AtomicPollRegionBegin(fixture.trace, fixture.result, startup_mask); + Expect(outer_previous == 0U, "最外层 region 的 previous mask 应为零"); + Expect(fixture.trace.poll_burst.enabled_mask == startup_mask, "最外层 region 未启用 startup site"); + Expect( + AtomicPollBatchEnabled(fixture.trace, AtomicSite::StartupPoll, AtomicOp::Load), + "最外层 startup site 应允许聚合" + ); + (void)TraceAtomicLoad( + fixture.trace, fixture.result, -1, AtomicSite::StartupPoll, &startup_value + ); + + // 嵌套 begin 会先关闭外层已有 batch,再把 inner mask 与外层 mask 合并。 + const uint32_t inner_previous = + AtomicPollRegionBegin(fixture.trace, fixture.result, fanin_mask); + Expect(inner_previous == startup_mask, "内层 region 没有保存外层 mask"); + Expect( + fixture.trace.poll_burst.enabled_mask == (startup_mask | fanin_mask), + "内层 region 没有合并两层 mask" + ); + Expect(fixture.core.count == 1U, "内层 begin 没有关闭外层 active batch"); + (void)TraceAtomicLoad( + fixture.trace, fixture.result, -1, AtomicSite::FaninFlagLoad, &fanin_value + ); + + AtomicPollRegionEnd(fixture.trace, fixture.result, inner_previous); + Expect(fixture.trace.poll_burst.enabled_mask == startup_mask, "内层 end 没有还原外层 mask"); + Expect(fixture.core.count == 2U, "内层 end 没有关闭内层 active batch"); + (void)TraceAtomicLoad( + fixture.trace, fixture.result, -1, AtomicSite::StartupPoll, &startup_value + ); + + AtomicPollRegionEnd(fixture.trace, fixture.result, outer_previous); + Expect(fixture.trace.poll_burst.enabled_mask == 0U, "最外层 end 没有还原初始 mask"); + Expect(fixture.trace.poll_burst.active_mask == 0U, "嵌套 region 结束后仍有 active batch"); + Expect(fixture.core.count == 3U, "嵌套 region 应按三个边界写出三条 batch"); + Expect(fixture.trace.poll_batch_records == 3U, "嵌套 region 的物理 batch 计数错误"); + Expect(fixture.trace.poll_calls == 3U, "嵌套 region 的逻辑 poll 调用数错误"); + Expect(fixture.result.atomic_trace_calls == 3U, "嵌套 region 的逻辑 atomic 调用数错误"); + Expect( + fixture.records[0].auxiliary == static_cast(AtomicSite::StartupPoll) && + fixture.records[1].auxiliary == static_cast(AtomicSite::FaninFlagLoad) && + fixture.records[2].auxiliary == static_cast(AtomicSite::StartupPoll), + "嵌套 region 的 batch site 顺序错误" + ); + Expect(fixture.core.dropped == 0U, "嵌套 region 不应丢记录"); + Expect(!fixture.trace.atomic_counter_overflow, "嵌套 region 不应报告计数溢出"); +} + +void TestNonAllowlistedSiteStaysDirect() { + Fixture fixture; + volatile int64_t frontier_flag = 7; + constexpr AtomicSite kSite = AtomicSite::FrontierFlagLoad; + const uint32_t site_mask = TraceAtomicSiteMask(kSite); + TestOps::now = 2000; + + const uint32_t previous = + AtomicPollRegionBegin(fixture.trace, fixture.result, site_mask); + Expect( + !AtomicPollBatchEnabled(fixture.trace, kSite, AtomicOp::Load), + "非 allowlist 的 frontier scan 不得因 region mask 被聚合" + ); + const int64_t observed = TraceAtomicLoad( + fixture.trace, fixture.result, 37, kSite, &frontier_flag + ); + AtomicPollRegionEnd(fixture.trace, fixture.result, previous); + + Expect(observed == frontier_flag, "非 allowlist direct load 返回值错误"); + Expect(fixture.core.count == 1U, "非 allowlist load 应写一条 direct 记录"); + Expect(fixture.trace.poll_calls == 0U, "非 allowlist load 不得增加 batched poll 调用数"); + Expect(fixture.trace.poll_batch_records == 0U, "非 allowlist load 不得写 PollBatch 记录"); + Expect(fixture.result.atomic_trace_calls == 1U, "非 allowlist direct load 的逻辑调用计数错误"); + Expect( + (fixture.records[0].flags & kAtomicPollBatch) == 0U, + "非 allowlist load 被错误标成 PollBatch" + ); + Expect(fixture.records[0].task_id == 37, "非 allowlist direct load 丢失 task 归因"); + Expect( + fixture.records[0].auxiliary == static_cast(kSite), + "非 allowlist direct load 的 site 错误" + ); + Expect(fixture.trace.poll_burst.active_mask == 0U, "非 allowlist load 不应留下 active batch"); + Expect(fixture.core.dropped == 0U, "非 allowlist direct load 不应丢记录"); + Expect(!fixture.trace.atomic_counter_overflow, "非 allowlist direct load 不应报告溢出"); +} + +} // namespace + +int main() { + TestSplitAtMaximumCount(); + TestNestedRegionRestoresMask(); + TestNonAllowlistedSiteStaysDirect(); + if (g_failures != 0) { + std::fprintf(stderr, "[FAIL] atomic PollBatch self-test failures=%d\n", g_failures); + return EXIT_FAILURE; + } + std::printf("[PASS] atomic PollBatch split/region/allowlist self-test\n"); + return EXIT_SUCCESS; +} diff --git a/tests/atomic_probe/pa_scheduler/cpu/build.sh b/tests/atomic_probe/pa_scheduler/cpu/build.sh index d3fd5f0f75..32907158fc 100755 --- a/tests/atomic_probe/pa_scheduler/cpu/build.sh +++ b/tests/atomic_probe/pa_scheduler/cpu/build.sh @@ -30,6 +30,18 @@ echo "[BUILD] CPU scheduler executable" "$SCRIPT_DIR/main.cpp" \ -o "$BUILD_DIR/pa_scheduler_cpu" +# PollBatch 是 common/ 中的设备/CPU 共用模板。这里用普通 C++17 编译器 +# 直接实例化并执行边界自测;任一断言失败都会借助 set -e 阻止构建成功。 +echo "[BUILD] atomic PollBatch boundary self-test" +"$CXX_BIN" -O2 -std=c++17 -Wall -Wextra -Werror \ + -DPA_BUILD_SWIMLANE=1 \ + -I"$ROOT_DIR/common" \ + "$ROOT_DIR/common/test_atomic_poll_batch.cpp" \ + -o "$BUILD_DIR/test_atomic_poll_batch" + +echo "[TEST] atomic PollBatch boundary self-test" +"$BUILD_DIR/test_atomic_poll_batch" + # set -e 保证编译或链接失败时不会打印 complete,也不会在组合构建中继续 # 后续步骤;只有成功退出的构建才被本脚本声明为可运行产物。 echo "[BUILD] complete: $BUILD_DIR/pa_scheduler_cpu" diff --git a/tests/atomic_probe/pa_scheduler/cpu/main.cpp b/tests/atomic_probe/pa_scheduler/cpu/main.cpp index 9814d70a32..17a19e78a7 100644 --- a/tests/atomic_probe/pa_scheduler/cpu/main.cpp +++ b/tests/atomic_probe/pa_scheduler/cpu/main.cpp @@ -13,6 +13,7 @@ #include "../common/winner_workload_host.h" #define PA_DEVICE inline +#define PA_DEVICE_NOINLINE static __attribute__((noinline)) #define PA_GM // CPU 后端直接实例化与设备端相同的公共调度器;这里只消去 AICore 地址空间 // 修饰符,不另写一套简化状态机,因此它可以承担协议和边界回归。 @@ -135,9 +136,10 @@ __attribute__((noinline)) void ExecuteRealWinnerWorkload( } struct CpuOps { - // CPU 原子 built-in 在函数返回前已产生旧值;该后端只做协议回归,不把 - // x86 时间分布外推到 A5。 - static constexpr bool kAtomicReturnReadyObserved = true; + // CPU 后端只验证调度协议与 raw schema,没有建立与 A5 CCEC + // 同构的“atomic 返回值依赖 + SYS_CNT”硬件边界;因此必须标记为 + // source_issue,不能让 x86 built-in 的函数返回冒充 A5 return_ready。 + static constexpr bool kAtomicReturnReadyObserved = false; // 用 fetch_add(0) 模拟 A5 atomicAdd(addr, 0) 原子读,而不是退化为普通 // CPU load。Acquire/AcqRel 只建立本 CPU 协议回归需要的发布/观察关系, @@ -395,6 +397,7 @@ int main(int argc, char **argv) { real_compute ? pa_scheduler::host::RealComputePatternName(workload_options.pattern) : "none", + options.trace_atomics, read_trace_records )) { postprocess_ok = false; diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index f3a2d802b8..238e2aeb81 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -84,7 +84,11 @@ build_backend() { # 固定为 CCEC、AscendC、CPU,便于按用户要求分阶段复现。 case "$1" in ccec|ascendc|cpu) - "$SCRIPT_DIR/$1/build.sh" + if [[ "$1" == "ccec" ]]; then + "$SCRIPT_DIR/ccec/build.sh" swimlane + else + "$SCRIPT_DIR/$1/build.sh" + fi ;; *) echo "Unknown backend: $1" >&2 diff --git a/tests/atomic_probe/pa_scheduler/swimlane_converter.py b/tests/atomic_probe/pa_scheduler/swimlane_converter.py index 7b258b9b99..57fc4eb8cb 100755 --- a/tests/atomic_probe/pa_scheduler/swimlane_converter.py +++ b/tests/atomic_probe/pa_scheduler/swimlane_converter.py @@ -74,6 +74,50 @@ 3: "fetch_max", } +# schema-v3 的校验表必须与 standalone C++ 的稳定 AtomicSite 编号一致。 +# 这里只描述本独立调度器实际实现的 0..14;真实 PA 追加的 BlockWon site +# 不属于本用例,不能为了兼容生产 converter 在这里凭空放宽输入。 +ATOMIC_SITE_OP_IDS = { + 0: 2, + 1: 0, + 2: 0, + 3: 1, + 4: 3, + 5: 0, + 6: 1, + 7: 1, + 8: 0, + 9: 0, + 10: 3, + 11: 0, + 12: 0, + 13: 2, + 14: 0, +} +# 这些发布型调用不消费 atomic 返回的旧值;其余 standalone site 的 +# 返回值都参与协议判断。v3 输入必须与源码语义完全一致。 +ATOMIC_RESULT_UNUSED_SITE_IDS = {0, 3, 6, 7, 13} +# 只有显式 scheduler 等待区中的六类 observation load 可以合并。 +# frontier 扫描和 Claim 即使调用很多次也必须继续保留逐调用记录。 +POLL_BATCH_SITE_OP_IDS = {1: 0, 2: 0, 5: 0, 11: 0, 12: 0, 14: 0} + +ATOMIC_RESULT_USED = 1 << 4 +ATOMIC_VALUE_ZERO = 1 << 5 +ATOMIC_RETURN_READY = 1 << 6 +ATOMIC_POLL_BATCH = 1 << 7 +ATOMIC_PAYLOAD_SHIFT = 8 +ATOMIC_PAYLOAD_MASK = 0xFFFFFF + + +# schema-v3 只由本目录的 standalone producer 生成;其 worker 编号与 +# 32 AIC + 64 AIV 的 mixed-block 映射是 raw ABI 的一部分,converter 不再 +# 只检查“同一 block/lane 不重复”这个弱条件。 +def _standalone_topology(core_id: int) -> tuple[int, int, str]: + if core_id < 32: + return core_id, 0, "aic" + vector_id = core_id - 32 + return vector_id // 2, 1 + vector_id % 2, "aiv" + # 把可转为整数的 raw 标量归一为 int,并在错误中保留精确字段路径。 def _integer(value: Any, label: str) -> int: @@ -106,17 +150,30 @@ def _load_and_validate( frequency_hz = _integer(metadata.get("clock_freq_hz"), "metadata.clock_freq_hz") if frequency_hz <= 0: raise ValueError("metadata.clock_freq_hz must be positive") - # v1 是旧 raw,Claim flags 只有 winner bit;v2 追加 attempted bit。 + # v1 是旧 raw,Claim flags 只有 winner bit;v2 追加 attempted bit; + # v3 再加入精确计数 PollBatch 与 producer summary 闭合。 # 不认识的新版本直接拒绝,避免把新 flags 按旧语义误读。 trace_schema_version = _integer(metadata.get("trace_schema_version", 1), "metadata.trace_schema_version") - if trace_schema_version not in (1, 2): + if trace_schema_version not in (1, 2, 3): raise ValueError(f"unsupported metadata.trace_schema_version: {trace_schema_version}") + if trace_schema_version == 3 and level != 4: + raise ValueError("metadata.trace_schema_version=3 requires l2_swimlane_level=4") num_cores = _integer(metadata.get("num_cores"), "metadata.num_cores") if num_cores <= 0: raise ValueError("metadata.num_cores must be positive") core_types = metadata.get("core_types") if not isinstance(core_types, list) or len(core_types) != num_cores: raise ValueError("metadata.core_types length must equal metadata.num_cores") + if trace_schema_version == 3: + if num_cores > 96: + raise ValueError("schema-v3 standalone metadata.num_cores must not exceed 96") + for core_id, core_type in enumerate(core_types): + expected_type = _standalone_topology(core_id)[2] + if core_type != expected_type: + raise ValueError( + f"metadata.core_types[{core_id}]={core_type!r} does not match " + f"standalone topology {expected_type!r}" + ) winner_workload = metadata.get("winner_workload") if winner_workload is not None: if not isinstance(winner_workload, dict): @@ -183,6 +240,22 @@ def _load_and_validate( # 若在 raw 中映射到两个 core,说明采集已损坏,不能继续生成误导性泳道。 core_by_block_lane: dict[tuple[int, int], int] = {} base_cycle: int | None = None + observed_summary = { + "records": len(rows), + "atomic_records": 0, + "clock_baseline_records": 0, + "atomic_calls": 0, + "batched_poll_calls": 0, + "poll_batch_records": 0, + # dropped 无法从已经导出的有效行反推;v3 必须由 producer summary + # 明确承诺为零,下面再逐字段核对。 + "dropped_records": 0, + } + v3_clock_rows: dict[int, dict[str, int | bool | None]] = { + core_id: {"plain": 0, "dependency": 0, "return_ready": None} + for core_id in range(num_cores) + } + v3_result_used_direct_rows: list[tuple[int, int, bool]] = [] # 逐行在写输出前检查列数、范围和可转整数的字段。任一行不满足这些约束 # 都会整体拒绝输入,不生成缺少关键阶段的“部分可看”泳道。 for index, row in enumerate(rows): @@ -206,11 +279,101 @@ def _load_and_validate( raise ValueError(f"fdwic_events[{index}] has invalid lane {lane}") if phase not in PHASE_NAMES: raise ValueError(f"fdwic_events[{index}] has unknown phase {phase!r}") + if trace_schema_version == 3: + if task_id < -1 or function_id < -1 or auxiliary < 0: + raise ValueError( + f"fdwic_events[{index}] has invalid v3 base fields: " + f"task={task_id} func={function_id} aux={auxiliary}" + ) + if not 0 <= flags <= 0xFFFFFFFF: + raise ValueError( + f"fdwic_events[{index}] has invalid uint32 flags {flags}" + ) + expected_block, expected_lane, _ = _standalone_topology(core_id) + if block_id != expected_block or lane != expected_lane: + raise ValueError( + f"fdwic_events[{index}] block/lane={block_id}/{lane} does not match " + f"standalone topology {expected_block}/{expected_lane} for core {core_id}" + ) if phase == "Claim" and trace_schema_version >= 2: if flags & ~0x3 or (flags & 0x1 and not flags & 0x2): raise ValueError( f"fdwic_events[{index}] has invalid Claim flags 0x{flags:x}" ) + if trace_schema_version == 3 and auxiliary > 1: + raise ValueError( + f"fdwic_events[{index}] has invalid Claim auxiliary {auxiliary}" + ) + if phase == "Atomic": + poll_batch = bool(flags & ATOMIC_POLL_BATCH) + atomic_op = flags & 0xF + result_used = bool(flags & ATOMIC_RESULT_USED) + value_zero = bool(flags & ATOMIC_VALUE_ZERO) + return_ready = bool(flags & ATOMIC_RETURN_READY) + payload = (flags >> ATOMIC_PAYLOAD_SHIFT) & ATOMIC_PAYLOAD_MASK + if poll_batch: + if ( + trace_schema_version != 3 + or payload == 0 + or POLL_BATCH_SITE_OP_IDS.get(auxiliary) != atomic_op + or not result_used + or value_zero + or return_ready + or task_id != -1 + or function_id != -1 + ): + raise ValueError( + f"fdwic_events[{index}] has invalid Atomic PollBatch " + f"site={auxiliary} flags=0x{flags:x}" + ) + elif trace_schema_version == 3: + expected_result_used = ( + auxiliary in ATOMIC_SITE_OP_IDS + and auxiliary not in ATOMIC_RESULT_UNUSED_SITE_IDS + ) + if ( + ATOMIC_SITE_OP_IDS.get(auxiliary) != atomic_op + or result_used != expected_result_used + or (return_ready and not result_used) + or (value_zero and atomic_op != 0) + or (payload and atomic_op != 3) + or function_id != -1 + ): + raise ValueError( + f"fdwic_events[{index}] has invalid direct Atomic " + f"site={auxiliary} flags=0x{flags:x}" + ) + if result_used: + v3_result_used_direct_rows.append((index, core_id, return_ready)) + observed_summary["atomic_records"] += 1 + if poll_batch: + observed_summary["atomic_calls"] += payload + observed_summary["batched_poll_calls"] += payload + observed_summary["poll_batch_records"] += 1 + else: + observed_summary["atomic_calls"] += 1 + elif phase == "ClockBaseline": + observed_summary["clock_baseline_records"] += 1 + if trace_schema_version == 3: + dependency = bool(flags & 0x1) + dependency_applied = bool(flags & 0x2) + if ( + flags & ~0x3 + or (dependency_applied and not dependency) + or task_id != -1 + or function_id != -1 + or auxiliary != 0 + ): + raise ValueError( + f"fdwic_events[{index}] has invalid ClockBaseline " + f"flags=0x{flags:x} auxiliary={auxiliary}" + ) + clock_state = v3_clock_rows[core_id] + if dependency: + clock_state["dependency"] = int(clock_state["dependency"]) + 1 + clock_state["return_ready"] = dependency_applied + else: + clock_state["plain"] = int(clock_state["plain"]) + 1 if start_cycle <= 0 or end_cycle < start_cycle: raise ValueError( f"fdwic_events[{index}] has invalid cycles start={start_cycle} end={end_cycle}" @@ -238,6 +401,39 @@ def _load_and_validate( ) assert base_cycle is not None + if trace_schema_version == 3: + # 每核两条基线同时证明采集完整性和该后端是否真正应用了 + # atomic 返回值依赖;所有消费返回值的直接记录必须与本核证据一致。 + for core_id, clock_state in v3_clock_rows.items(): + if clock_state["plain"] != 1 or clock_state["dependency"] != 1: + raise ValueError( + f"core {core_id} requires exactly one plain and one dependency " + f"ClockBaseline: plain={clock_state['plain']} " + f"dependency={clock_state['dependency']}" + ) + for row_index, core_id, return_ready in v3_result_used_direct_rows: + expected_return_ready = bool(v3_clock_rows[core_id]["return_ready"]) + if return_ready != expected_return_ready: + raise ValueError( + f"fdwic_events[{row_index}] direct Atomic return_ready={return_ready} " + f"does not match core {core_id} ClockBaseline " + f"dependency_applied={expected_return_ready}" + ) + + producer_summary = metadata.get("fdwic_summary") + if not isinstance(producer_summary, dict): + raise ValueError( + "metadata.fdwic_summary is required for trace_schema_version=3" + ) + for key, observed_value in observed_summary.items(): + producer_value = _integer( + producer_summary.get(key), f"metadata.fdwic_summary.{key}" + ) + if producer_value != observed_value: + raise ValueError( + f"metadata.fdwic_summary.{key}={producer_value} " + f"does not match raw value {observed_value}" + ) return frequency_hz, trace_schema_version, rows, core_by_block_lane, base_cycle, metadata @@ -392,10 +588,29 @@ def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: atomic_op_id = flags & 0xF atomic_site = ATOMIC_SITE_NAMES.get(atomic_site_id, f"site_{atomic_site_id}") atomic_op = ATOMIC_OP_NAMES.get(atomic_op_id, f"op_{atomic_op_id}") - # 边界直接写入 span 名称,打开泳道后无需点开 args - # 就能区分“本核返回值可消费”和“只包围源码发射”。 - atomic_boundary_tag = "return_ready" if flags & (1 << 6) else "source_issue" - name = f"atomic.{atomic_boundary_tag}.{atomic_site}.{atomic_op}#{task_id}" + atomic_poll_batch = ( + trace_schema_version >= 3 and bool(flags & ATOMIC_POLL_BATCH) + ) + if atomic_poll_batch: + atomic_call_count = ( + flags >> ATOMIC_PAYLOAD_SHIFT + ) & ATOMIC_PAYLOAD_MASK + name = ( + f"atomic.poll_batch.{atomic_site}.{atomic_op}" + f"×{atomic_call_count}" + ) + else: + # 边界直接写入 span 名称,打开泳道后无需点开 args + # 就能区分“本核返回值可消费”和“只包围源码发射”。 + atomic_boundary_tag = ( + "return_ready" + if flags & ATOMIC_RETURN_READY + else "source_issue" + ) + name = ( + f"atomic.{atomic_boundary_tag}.{atomic_site}." + f"{atomic_op}#{task_id}" + ) thread_id = lane elif phase == "clock_baseline": name = ( @@ -433,36 +648,70 @@ def convert(input_path: Path, output_path: Path) -> tuple[int, int, int]: }, } if phase == "atomic": - # Atomic 的 flags/aux 有独立 ABI,不沿用普通 phase 的 mc 语义。 - # cycles 保留原始整数,避免 Perfetto dur 的微秒浮点换算丢失短 atomic 精度。 - event["args"] = { - "phase": phase, - "task_id": task_id, - "func_id": function_id, - "core": core_id, - "site": atomic_site, - "site_id": atomic_site_id, - "op": atomic_op, - "op_id": atomic_op_id, - "cycles": end - start, - "result_used": bool(flags & (1 << 4)), - "return_ready_observed": bool(flags & (1 << 6)), - "completion_boundary": ( - "return_value_ready" - if flags & (1 << 6) - else "source_issue_bracket" - ), - "flags": flags, - "execution_unit": "scalar", - } - # 分类同样带边界,便于 Perfetto 过滤和分组;二者 - # 仍在同一 AIC/AIV scalar lane,不伪造并行执行单元。 - event["cat"] = f"atomic.{atomic_boundary_tag}" - # bit5 只对 Load 有意义;bits8..31 只对 FetchMax 表示饱和后的 retry 数。 - if atomic_op_id == 0: - event["args"]["value_zero"] = bool(flags & (1 << 5)) - if atomic_op_id == 3: - event["args"]["retries"] = (flags >> 8) & 0xFFFFFF + # PollBatch 表示显式等待区内的逻辑调用次数;它的 span + # 只是 episode 包络,可能与其他 site 或直接 Atomic 交错, + # 因而绝不能伪装成一次 atomic 的 completion boundary。 + if atomic_poll_batch: + event["args"] = { + "phase": "atomic_poll_batch", + "task_id": task_id, + "func_id": function_id, + "core": core_id, + "site": atomic_site, + "site_id": atomic_site_id, + "op": atomic_op, + "op_id": atomic_op_id, + "call_count": atomic_call_count, + "poll_window_cycles": end - start, + "estimate_formula": "call_count * calibrated_atomic_cost", + "is_poll_batch": True, + "batch_semantics": "observation_load_calls", + "duration_semantics": ( + "logical_poll_episode_envelope_not_single_atomic_latency" + ), + "may_contain_interleaved_direct_atomics": True, + "flags": flags, + "execution_unit": "scalar", + } + event["cat"] = "atomic.poll_batch" + else: + # 直接 Atomic 的 flags/aux 有独立 ABI,不沿用普通 + # phase 的 mc 语义。cycles 保留原始整数,避免短 + # atomic 经微秒浮点换算后丢失 tick 精度。 + event["args"] = { + "phase": phase, + "task_id": task_id, + "func_id": function_id, + "core": core_id, + "site": atomic_site, + "site_id": atomic_site_id, + "op": atomic_op, + "op_id": atomic_op_id, + "call_count": 1, + "cycles": end - start, + "result_used": bool(flags & ATOMIC_RESULT_USED), + "return_ready_observed": bool(flags & ATOMIC_RETURN_READY), + "completion_boundary": ( + "return_value_ready" + if flags & ATOMIC_RETURN_READY + else "source_issue_bracket" + ), + "flags": flags, + "execution_unit": "scalar", + } + # 分类同样带边界,便于 Perfetto 过滤和分组;二者 + # 仍在同一 AIC/AIV scalar lane,不伪造并行执行单元。 + event["cat"] = f"atomic.{atomic_boundary_tag}" + # bit5 只对 Load 有意义;bits8..31 只对 FetchMax + # 表示饱和后的 retry 数。 + if atomic_op_id == 0: + event["args"]["value_zero"] = bool( + flags & ATOMIC_VALUE_ZERO + ) + if atomic_op_id == 3: + event["args"]["retries"] = ( + flags >> ATOMIC_PAYLOAD_SHIFT + ) & ATOMIC_PAYLOAD_MASK elif phase == "claim": event["args"] = { "phase": phase, diff --git a/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py b/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py index 60ae046fc6..51fcd52e81 100644 --- a/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py +++ b/tests/atomic_probe/pa_scheduler/test_swimlane_converter.py @@ -25,6 +25,84 @@ from swimlane_converter import convert +def _standalone_topology(core_id: int) -> tuple[int, int, str]: + """返回 standalone 固定 32 AIC + 64 AIV 拓扑中的 block/lane/type。""" + if core_id < 32: + return core_id, 0, "aic" + vector_id = core_id - 32 + return vector_id // 2, 1 + vector_id % 2, "aiv" + + +def _v3_capture( + rows: list[list[object]], + *, + num_cores: int = 1, + add_clock_baselines: bool = True, + dependency_applied: bool = True, +) -> dict[str, object]: + """构造带 producer weighted summary 的最小 schema-v3 raw。""" + all_rows = [list(row) for row in rows] + if add_clock_baselines: + dependency_flags = 0x3 if dependency_applied else 0x1 + for core_id in range(num_cores): + block_id, lane, _ = _standalone_topology(core_id) + start = 10 + core_id * 4 + all_rows.extend( + [ + [ + core_id, + block_id, + lane, + -1, + -1, + "ClockBaseline", + start, + start + 1, + 0, + 0, + ], + [ + core_id, + block_id, + lane, + -1, + -1, + "ClockBaseline", + start + 2, + start + 3, + dependency_flags, + 0, + ], + ] + ) + atomic_rows = [row for row in all_rows if row[5] == "Atomic"] + batch_rows = [row for row in atomic_rows if int(row[8]) & 0x80] + batch_calls = sum((int(row[8]) >> 8) & 0xFFFFFF for row in batch_rows) + core_types = [_standalone_topology(core_id)[2] for core_id in range(num_cores)] + summary = { + "records": len(all_rows), + "atomic_records": len(atomic_rows), + "clock_baseline_records": sum( + row[5] == "ClockBaseline" for row in all_rows + ), + "atomic_calls": len(atomic_rows) - len(batch_rows) + batch_calls, + "batched_poll_calls": batch_calls, + "poll_batch_records": len(batch_rows), + "dropped_records": 0, + } + return { + "l2_swimlane_level": 4, + "metadata": { + "clock_freq_hz": 1_000_000_000, + "num_cores": num_cores, + "trace_schema_version": 3, + "core_types": core_types, + "fdwic_summary": summary, + }, + "fdwic_events": all_rows, + } + + class SwimlaneConverterLayoutTest(unittest.TestCase): def test_real_compute_metadata_is_preserved_and_visible(self) -> None: # raw 与 merged 都必须自描述真实 engine 负载;否则同名 QK/SF/PV/UP @@ -250,6 +328,337 @@ def test_v2_rejects_winner_without_attempt(self) -> None: with self.assertRaisesRegex(ValueError, "invalid Claim flags"): convert(input_path, output_path) + def test_v3_poll_batches_preserve_exact_calls_on_scalar_lane(self) -> None: + # standalone 只允许六类显式等待区 observation load 聚合;每个 + # PollBatch 都必须保留精确 call_count,但不能伪装成单次延迟。 + sites = { + 1: "startup_poll", + 2: "fatal_poll", + 5: "fanin_flag_load", + 11: "heap_frontier_load", + 12: "heap_vend_load", + 14: "replay_done_poll", + } + call_count = 12_345 + flags = (call_count << 8) | 0x90 + for site_id, site_name in sites.items(): + with self.subTest(site=site_name), tempfile.TemporaryDirectory() as directory: + capture = _v3_capture( + [[0, 0, 0, -1, -1, "Atomic", 100, 900, flags, site_id]] + ) + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + emitted, blocks, base_cycle = convert(input_path, output_path) + merged = json.loads(output_path.read_text(encoding="utf-8")) + + self.assertEqual((emitted, blocks, base_cycle), (3, 1, 10)) + batch = next( + event + for event in merged["traceEvents"] + if event.get("cat") == "atomic.poll_batch" + ) + self.assertEqual( + batch["name"], f"atomic.poll_batch.{site_name}.load×{call_count}" + ) + self.assertEqual((batch["pid"], batch["tid"]), (0, 0)) + self.assertEqual(batch["args"]["call_count"], call_count) + self.assertEqual(batch["args"]["poll_window_cycles"], 800) + self.assertEqual( + batch["args"]["duration_semantics"], + "logical_poll_episode_envelope_not_single_atomic_latency", + ) + self.assertEqual( + batch["args"]["batch_semantics"], "observation_load_calls" + ) + self.assertTrue( + batch["args"]["may_contain_interleaved_direct_atomics"] + ) + self.assertNotIn("cycles", batch["args"]) + self.assertNotIn("completion_boundary", batch["args"]) + self.assertNotIn("return_ready_observed", batch["args"]) + self.assertEqual( + merged["metadata"]["fdwic_summary"]["atomic_calls"], + call_count, + ) + + def test_v3_poll_batch_accepts_maximum_24_bit_count(self) -> None: + call_count = 0xFFFFFF + capture = _v3_capture( + [[ + 0, + 0, + 0, + -1, + -1, + "Atomic", + 100, + 900, + (call_count << 8) | 0x90, + 1, + ]] + ) + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + convert(input_path, output_path) + events = json.loads(output_path.read_text(encoding="utf-8"))["traceEvents"] + + batch = next(event for event in events if event.get("cat") == "atomic.poll_batch") + self.assertEqual(batch["args"]["call_count"], call_count) + + def test_v3_rejects_invalid_poll_batch_schema(self) -> None: + valid = (7 << 8) | 0x90 + cases = ( + (0x90, 5, -1, -1), # call_count=0 + ((7 << 8) | 0x91, 5, -1, -1), # observation 只能是 Load + (valid, 9, -1, -1), # frontier scan 不是显式等待区 + ((7 << 8) | 0xB0, 5, -1, -1), # batch 没有 value_zero + ((7 << 8) | 0xD0, 5, -1, -1), # batch 没有 return-ready + (valid, 5, 0, -1), # batch 不归属单个 task + (valid, 5, -1, 0), # batch 不归属 kernel function + ) + for flags, site, task_id, func_id in cases: + with self.subTest(flags=flags, site=site), tempfile.TemporaryDirectory() as directory: + capture = _v3_capture( + [[ + 0, + 0, + 0, + task_id, + func_id, + "Atomic", + 100, + 110, + flags, + site, + ]] + ) + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "invalid Atomic PollBatch"): + convert(input_path, output_path) + self.assertFalse(output_path.exists()) + + def test_v2_reserves_poll_batch_flag(self) -> None: + capture = { + "l2_swimlane_level": 4, + "metadata": { + "clock_freq_hz": 1_000_000_000, + "num_cores": 1, + "trace_schema_version": 2, + "core_types": ["aic"], + }, + "fdwic_events": [ + [0, 0, 0, -1, -1, "Atomic", 100, 110, (7 << 8) | 0x90, 5] + ], + } + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "invalid Atomic PollBatch"): + convert(input_path, output_path) + + def test_v3_rejects_invalid_direct_atomic_schema(self) -> None: + cases = ( + (0x51, 4, -1), # ClaimMax 的 op 必须是 FetchMax + (0x12, 0, -1), # StartupIncrement 不消费返回值 + (0x42, 0, -1), # 未消费返回值不能声明 return-ready + (0x73, 4, -1), # value_zero 只属于 Load + ((1 << 8) | 0x50, 1, -1), # retry payload 只属于 FetchMax + (0x50, 15, -1), # standalone 未定义生产 BlockWon site + (0x53, 4, 0), # Atomic 不携带 function id + ) + for flags, site, func_id in cases: + with self.subTest(flags=flags, site=site), tempfile.TemporaryDirectory() as directory: + capture = _v3_capture( + [[0, 0, 0, 7, func_id, "Atomic", 100, 110, flags, site]] + ) + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "invalid direct Atomic"): + convert(input_path, output_path) + + def test_v3_direct_boundary_must_match_core_clock_baseline(self) -> None: + # baseline 声明该后端应用依赖钩子,消费返回值的直接 Atomic 却没有 + # return-ready bit;converter 必须拒绝这种自相矛盾的 raw。 + capture = _v3_capture( + [[0, 0, 0, 7, -1, "Atomic", 100, 110, 0x13, 4]], + dependency_applied=True, + ) + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "does not match.*ClockBaseline"): + convert(input_path, output_path) + + def test_v3_source_issue_direct_boundary_matches_cpu_baseline(self) -> None: + # CPU/A5Sim 的依赖基线明确声明 dependency_applied=0;消费返回值的 + # direct span 因而保留 source-issue,不能被 converter 擅自升级。 + capture = _v3_capture( + [[0, 0, 0, 7, -1, "Atomic", 100, 110, 0x13, 4]], + dependency_applied=False, + ) + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + convert(input_path, output_path) + events = json.loads(output_path.read_text(encoding="utf-8"))["traceEvents"] + + direct = next( + event for event in events if event.get("cat") == "atomic.source_issue" + ) + self.assertEqual(direct["args"]["call_count"], 1) + self.assertTrue(direct["args"]["result_used"]) + self.assertFalse(direct["args"]["return_ready_observed"]) + + def test_v3_rejects_invalid_clock_baseline_schema(self) -> None: + cases = ( + (0x2, -1, -1, 0), # applied 不能脱离 dependency bit + (0x4, -1, -1, 0), # 未定义 flag + (0x0, 0, -1, 0), # baseline 不归属 task + (0x0, -1, 0, 0), # baseline 不归属 function + (0x0, -1, -1, 1), # aux 必须为零 + ) + for flags, task_id, func_id, auxiliary in cases: + with self.subTest(flags=flags), tempfile.TemporaryDirectory() as directory: + capture = _v3_capture([], add_clock_baselines=False) + capture["fdwic_events"] = [ + [ + 0, + 0, + 0, + task_id, + func_id, + "ClockBaseline", + 10, + 11, + flags, + auxiliary, + ] + ] + summary = capture["metadata"]["fdwic_summary"] + summary["records"] = 1 + summary["clock_baseline_records"] = 1 + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "invalid ClockBaseline"): + convert(input_path, output_path) + + def test_v3_requires_two_clock_baselines_per_core(self) -> None: + capture = _v3_capture([], add_clock_baselines=False) + capture["fdwic_events"] = [ + [0, 0, 0, -1, -1, "ClockBaseline", 10, 11, 0, 0] + ] + summary = capture["metadata"]["fdwic_summary"] + summary["records"] = 1 + summary["clock_baseline_records"] = 1 + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "requires exactly one plain"): + convert(input_path, output_path) + + def test_v3_rejects_each_broken_weighted_summary_field(self) -> None: + rows = [ + [0, 0, 0, -1, -1, "Atomic", 100, 200, (17 << 8) | 0x90, 1], + [0, 0, 0, 4, -1, "Atomic", 210, 220, 0x53, 4], + ] + keys = ( + "records", + "atomic_records", + "clock_baseline_records", + "atomic_calls", + "batched_poll_calls", + "poll_batch_records", + "dropped_records", + ) + for key in keys: + with self.subTest(key=key), tempfile.TemporaryDirectory() as directory: + capture = _v3_capture(rows) + capture["metadata"]["fdwic_summary"][key] += 1 + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, rf"fdwic_summary\.{key}"): + convert(input_path, output_path) + + def test_v3_weighted_summary_closes_mixed_direct_and_batches(self) -> None: + rows = [ + [0, 0, 0, -1, -1, "Atomic", 100, 200, (17 << 8) | 0x90, 1], + [0, 0, 0, -1, -1, "Atomic", 201, 250, (9 << 8) | 0x90, 14], + [0, 0, 0, 4, -1, "Atomic", 251, 260, 0x53, 4], + ] + capture = _v3_capture(rows) + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + emitted, _, _ = convert(input_path, output_path) + merged = json.loads(output_path.read_text(encoding="utf-8")) + + summary = merged["metadata"]["fdwic_summary"] + self.assertEqual(summary["records"], 5) + self.assertEqual(summary["atomic_records"], 3) + self.assertEqual(summary["clock_baseline_records"], 2) + self.assertEqual(summary["atomic_calls"], 27) + self.assertEqual(summary["batched_poll_calls"], 26) + self.assertEqual(summary["poll_batch_records"], 2) + self.assertEqual(summary["dropped_records"], 0) + self.assertEqual(emitted, 5) + atomic_events = [ + event + for event in merged["traceEvents"] + if str(event.get("cat", "")).startswith("atomic.") + ] + self.assertEqual(len(atomic_events), summary["atomic_records"]) + + def test_v3_requires_level4_and_producer_summary(self) -> None: + capture = _v3_capture( + [[0, 0, 0, -1, -1, "Atomic", 100, 110, (3 << 8) | 0x90, 14]] + ) + with tempfile.TemporaryDirectory() as directory: + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + + capture["l2_swimlane_level"] = 1 + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "requires l2_swimlane_level=4"): + convert(input_path, output_path) + + capture["l2_swimlane_level"] = 4 + del capture["metadata"]["fdwic_summary"] + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "fdwic_summary is required"): + convert(input_path, output_path) + + def test_v3_rejects_non_standalone_topology(self) -> None: + cases = { + "core_type": lambda capture: capture["metadata"]["core_types"].__setitem__( + 0, "aiv" + ), + "block": lambda capture: capture["fdwic_events"][0].__setitem__(1, 1), + "lane": lambda capture: capture["fdwic_events"][0].__setitem__(2, 1), + } + for name, mutate in cases.items(): + with self.subTest(field=name), tempfile.TemporaryDirectory() as directory: + capture = _v3_capture([]) + mutate(capture) + input_path = Path(directory) / "raw.json" + output_path = Path(directory) / "merged.json" + input_path.write_text(json.dumps(capture), encoding="utf-8") + with self.assertRaisesRegex(ValueError, "does not match standalone topology"): + convert(input_path, output_path) + self.assertFalse(output_path.exists()) + if __name__ == "__main__": unittest.main() From 5c2d39b8237a87bda253c6c9ef934f770c567731 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 19:41:37 +0000 Subject: [PATCH 024/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E5=9B=BA?= =?UTF-8?q?=E5=8C=96standalone=20Submit=20PMU=E8=A7=82=E6=B5=8B=E5=8F=A3?= =?UTF-8?q?=E5=BE=84?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 拆分无泳道的 submit-pmu 构建,使用四件套 manifest 固化 phase 身份与产物校验。 以 CNT6/7 保留完整 Submit 权威计数;none 要求逐核精确闭合,running phase 显式报告 read-clear 下界、loss 与保守上界,并补齐逐核调用和 A5 拓扑门禁。 增加 raw 独立复算、负向回归与中文 I-cache 使用文档,明确 90ns 仅为 core-equivalent 标尺。 --- tests/atomic_probe/icache_miss_usage_guide.md | 434 ++++++++ ...05\345\206\265\345\210\206\346\236\220.md" | 162 ++- ...77\347\224\250\346\214\207\345\215\227.md" | 163 ++- tests/atomic_probe/pa_scheduler/ccec/build.sh | 159 ++- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 447 ++++++-- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 351 +++---- .../pa_scheduler/ccec/pmu_owner_aicpu.cpp | 4 +- .../pa_scheduler/ccec/pmu_owner_control.h | 18 +- .../pa_scheduler/ccec/pmu_probe.h | 37 + .../pa_scheduler/common/pa_model.h | 50 +- .../pa_scheduler/common/pa_scheduler_core.h | 71 +- .../pa_scheduler/common/pa_trace.h | 129 ++- .../pa_scheduler/pmu_sidecar_analyzer.py | 954 +++++++++++++++++- tests/atomic_probe/pa_scheduler/run.sh | 140 +++ .../pa_scheduler/test_pmu_sidecar_analyzer.py | 723 ++++++++++++- 15 files changed, 3469 insertions(+), 373 deletions(-) create mode 100644 tests/atomic_probe/icache_miss_usage_guide.md diff --git a/tests/atomic_probe/icache_miss_usage_guide.md b/tests/atomic_probe/icache_miss_usage_guide.md new file mode 100644 index 0000000000..7d97d4b49c --- /dev/null +++ b/tests/atomic_probe/icache_miss_usage_guide.md @@ -0,0 +1,434 @@ +# A5 PA Scheduler I-cache Miss 采集与分析指南 + +## 1. 目标与最终构建口径 + +本指南面向 +`tests/atomic_probe/pa_scheduler` +的 standalone CCEC 分支,目标是回答:在完整 Submit 期间,32 个 AIC +和 64 个 AIV 每核发生了多少 I-cache request/miss,其中哪些 miss +值得继续优化。它不依赖 simpler 生产代码,也不将 standalone 结果 +冒充为真实 PA 的绝对 profile。 + +当前最终只保留两类正式观察构建: + +| 构建 | 内容 | 是否包含 PMU | +| --- | --- | --- | +| `swimlane` | 普通阶段泳道 + 逐 atomic 泳道,在同一 AIC/AIV scalar lane 合并采集 | 否 | +| `submit-pmu` | 每物理子核的完整 Submit PMU,并可编译一个局部 phase | 是,仅 CCEC | + +`run` 、`smoke` 和 phase 名是运行或编译选择,不是额外的第三类 +构建。 + +## 2. 为什么 I-cache miss 必须独立重编译 + +I-cache 数据对代码布局极其敏感。泳道和逐 atomic 观察会增加: + +- 阶段和 atomic 的 `SYS_CNT` 读取; +- atomic wrapper、ClockBaseline 与 record 发布分支; +- 更大的 scalar `.text` 和不同的函数/对齐布局; +- 由此引起的 worker 到达、轮询和跨核竞争时序变化。 + +因此,“代码仍在 ELF 中,只在运行时关闭 record”不足以得到干净的 +I-cache 观察。`submit-pmu` 会独立重编译,编译掉泳道 record、逐 +atomic wrapper、ClockBaseline、runtime phase-profile 和旧 cold/warm 冲刷体。 +`swimlane` 则不构建 PMU owner。 + +两种数据不在同一进程采集: + +- 看事件时序和 atomic bracket,使用 `swimlane`; +- 看完整 Submit 每核 I-cache request/miss,使用 `submit-pmu`。 + +两份证据可以按同一源码版本交叉理解,不能做逐 tick 对齐,也不能 +把 PMU 的平均 miss 回填为某一条 atomic span 的属性。 + +## 3. `none` 与 `claim` 如何选择 + +当前只有两个编译期 phase: + +| phase | 边界 | 优先用途 | +| --- | --- | --- | +| `none` | 完整 Submit 中不读局部 shadow counter | 回答完整 Submit 的 AIC/AIV 每核 request/miss;这是默认选择 | +| `claim` | 每次 `Claim()` 调用前后读局部 shadow counter | 当 `none` 已证明 miss 值得追踪时,试验 Claim 的 running read-clear 下界/上界归因链路 | + +`claim` 的 begin/end 读取本身会执行 scalar 指令、占用取指并改变多核 +时序。因此: + +- `claim` 的 phase request/miss 是带局部边界扰动的观察值; +- `claim` 的 raw 观察值是下界,上界为该核下界加 primary-shadow loss; +- `none` 和 `claim` 是不同 ELF、不同进程,不能以两者相减声称得到 + 了零扰动 Claim 净值; +- 未来不同 phase ELF 的局部 request/miss 不可相加成“完整 Submit”; + 完整 Submit 始终以每个 ELF 自己的 primary whole 为准。 + +该区间只描述同一插桩 ELF、当前边界定义下的局部事件,不是无插桩 Claim +的真实区间。完整 Submit `none` 没有运行中 read-clear,仍执行 96/96 +逐核严格闭合。 + +## 4. 环境、构建与产物 + +命令在 `tests/atomic_probe/pa_scheduler` 目录下执行。非交互 shell +建议显式 source CANN 并选择本用户 GCC 15: + +```bash +source /home/q00473782/Ascend/cann-9.1.0-weekly-20260708/cann-9.1.0/set_env.sh +export GCC15_ROOT=/home/q00473782/.local/gcc-15/root +export PATH="$GCC15_ROOT/usr/bin:$PATH" +export LD_LIBRARY_PATH="$GCC15_ROOT/usr/lib/x86_64-linux-gnu:$GCC15_ROOT/usr/lib/gcc/x86_64-linux-gnu/15${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" +export CXX="$GCC15_ROOT/usr/bin/g++-15" +``` + +构建完整 Submit 基准: + +```bash +./run.sh build-submit-pmu ccec none +``` + +需要 Claim 局部归因时另行构建: + +```bash +./run.sh build-submit-pmu ccec claim +``` + +产物分别位于: + +```text +pa_scheduler/build/ccec/submit-pmu/none/ +pa_scheduler/build/ccec/submit-pmu/claim/ +``` + +每个 phase 目录中的 `pa_scheduler_host`、`pa_scheduler_kernel.o`、 +`libpa_scheduler_pmu_owner_aicpu.so` 和 +`libpa_scheduler_pmu_owner_dispatcher.so` 是一个不可拆分的构建集。host +会按 kernel 所在目录加载两个 SO;不得从另一个 phase 目录复制或 +拼接产物。构建只在四件套全部完成后原子发布 +`submit_pmu_artifacts.manifest`;`run.sh` 在启动 host 前核对 schema、phase、 +固定文件列表和四个 SHA256。 + +`swimlane` 的 CCEC 产物仍在 `pa_scheduler/build/ccec/`,不是 PMU +产物。 + +## 5. 采集完整 Submit 和 Claim + +### 5.1 完整 Submit `none` + +```bash +OUT="./outputs/submit_pmu_none_$(date -u +%Y%m%dT%H%M%SZ)" +mkdir -p "$OUT" + +./run.sh submit-pmu ccec none \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT/run1.json" +``` + +多轮比较必须使用多个独立进程和唯一文件名: + +```bash +./run.sh submit-pmu ccec none --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT/run2.json" + +./run.sh submit-pmu ccec none --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT/run3.json" +``` + +### 5.2 Claim 局部归因 + +```bash +OUT_CLAIM="./outputs/submit_pmu_claim_$(date -u +%Y%m%dT%H%M%SZ)" +mkdir -p "$OUT_CLAIM" + +./run.sh submit-pmu ccec claim \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT_CLAIM/run1.json" +``` + +`submit-pmu` action 已固定: + +```text +--runs 1 --no-swimlane --pmu-window submit-all +``` + +调用者不要重复传入这三项,也不能添加 +`--profile-phases`、`--trace-atomics`、`--analyze-swimlane` 或 +`--swimlane-json`。`--pmu-json` 可选;但要做 raw 复算和多轮汇总时必须 +使用它。host 拒绝覆盖已有 JSON 或同名 `.tmp`。 + +## 6. Primary/shadow 计数和可信门禁 + +### 6.1 计数器分工 + +| 计数 | selector | 用途 | +| --- | --- | --- | +| CNT6 | `0x34` | 完整 Submit primary I-cache request;局部边界从不读它 | +| CNT7 | `0x35` | 完整 Submit primary I-cache miss;局部边界从不读它 | +| CNT8 | `0x34` | read-to-clear shadow request | +| CNT5 | `0x35` | read-to-clear shadow miss;诊断 ELF 因此不再提供 MTE3 busy | +| CNT9 | `0x0` | 未使用 | + +A5 b1 实测已证明将 `0x35` 配置到 CNT9 时计数始终为 0,所以 +CNT9 不能作 shadow miss。这个变化只影响 `submit-pmu` 诊断构建, +不影响 `swimlane` 构建。 + +shadow PMU counter 是 read-to-clear。`claim` 在阶段 begin 读取 CNT8/CNT5,将 +之前的片段加入 shadow whole;在 end 再读一次,同时加入 shadow +whole 和 Claim phase;完整 Submit stop 后读 tail。`none` 不做中途读取, +只在 stop 后取 tail。 + +`none` 对每个物理子核必须精确满足: + +```text +shadow_whole_icache_requests == icache_requests +shadow_whole_icache_misses == icache_misses +``` + +即 stop 后读取的 CNT8/CNT5 分别等于同 selector、同 gate 的 CNT6/CNT7。 +这个 96/96 精确相等门禁验证完整 Submit 观察闭合;它不把 PMU 进程的 +Submit span 变成无诊断墙钟基线,也不把 standalone 数据冒充真实 PA profile。 + +运行中切片的 `claim` 已在 A5 b1/b256 上证明可能发生单向少计,接受规则为: + +```text +shadow_request <= primary_request +shadow_miss <= primary_miss + +request_loss = primary_request - shadow_request +miss_loss = primary_miss - shadow_miss + +phase_request_lower = phase_request_observed +phase_request_upper = phase_request_observed + request_loss +phase_miss_lower = phase_miss_observed +phase_miss_upper = phase_miss_observed + miss_loss +``` + +上下界必须先逐核计算,再分别聚合;不能拿聚合后的 median 相减拼区间。 +CNT8/CNT5 是顺序 `ld_dev`,不是同一时刻的原子配对快照,因此局部 +`phase_miss <= phase_request` 不是硬门禁。二者分别不超过对应 shadow, +各自上界不超过对应 primary。 + +结合 `none` exact、running phase bounded 和调用次数门禁,可分别验证: + +- 复制 selector 在本机 A5 上确实计数; +- 边界调用覆盖预定代码片段,且 begin/end/tail 次数闭合;边界竞态少计 + 由 loss 和 lower/upper 区间显式保留; +- primary whole 没有被局部归因读取破坏。 + +### 6.2 正式 JSON 必须通过的门禁 + +只有下列条件全部成立,host 才发布最终 JSON: + +- 语义、winner 真计算输出和 Submit placement/engine 闭合通过; +- 96 条记录可信,32 AIC + 64 AIV,物理子核 id 唯一; +- owner bitmap membership、worker slot、物理 role 和 32 个 1:2 triplet 全部匹配; +- 96 个核都真实执行完整 Submit PMU start/stop,owner Restore 成功; +- build variant 和编译 phase id 在 96 条记录中全部匹配; +- `none` 的 shadow whole 与 primary whole 逐核精确相等; +- running phase 的 shadow whole 逐核不大于 primary,loss 与 upper-bound + 公式逐核闭合;exact 核数只作诊断; +- `none` 的 phase calls/begin/end/request/miss 全部为 0; +- `claim` 的 begin/end/calls 逐核平衡,每核 calls 为 `batches * 5`, + 全局 calls 为 `batches * 5 * 96`; +- phase request/miss 分别不超过对应 shadow/primary,且可编程 counter + 低于当前 25% 保守风险阈值。 + +`metrics_prof_start/stop()` 在完整 Submit 前后各执行一次,其 +`PIPE_ALL` 边界会改变流水和多核时序。PMU 结果只与相同构建、 +相同 phase、相同负载的独立进程比较,不把 PMU 进程的 Submit +span 当作无诊断性能基线。 + +## 7. JSON 字段与 AIC/AIV 分析口径 + +当前 `submit-pmu` 输出 schema v4。`records` 保留 96 个 worker 的 raw, +`summary.all/aic/aiv` 分别对 96/32/64 个核统计: + +```text +sum / mean / median / p95 / max +``` + +完整 Submit 优先查看: + +- `icache_requests` / `icache_misses`:CNT6/CNT7 primary whole; +- `shadow_whole_icache_requests` / `shadow_whole_icache_misses`:闭合或分段 + loss 用 shadow whole; +- `shadow_request_loss` / `shadow_miss_loss`:本核 primary-shadow residual; +- `phase_calls` / `phase_icache_requests` / `phase_icache_misses`:选定 phase + 的 running read-clear lower; +- `phase_icache_requests_upper_bound` / `phase_icache_misses_upper_bound`: + lower 加本核对应 loss; +- `configuration.compiled_phase` 和 `validation.phase_measurement_valid`:确认文件口径。 +- `validation.shadow_primary_match_records` / `shadow_primary_bounded_records`: + 区分逐值 exact 与单向 bounded 核数; +- `configuration.phase_values_are_running_read_clear_lower_bounds`:确认局部字段 + 是否采用下界语义。 + +`phase_observed_read_clear_ratio` 只是 lower miss/lower request 的观测比值; +分子和分母各有独立区间,因此它不是实际 phase miss rate 的数学下界。 + +每核平均值按角色求: + +```text +AIC request/core = summary.aic.icache_requests.sum / 32 +AIC miss/core = summary.aic.icache_misses.sum / 32 +AIV request/core = summary.aiv.icache_requests.sum / 64 +AIV miss/core = summary.aiv.icache_misses.sum / 64 +``` + +`median` 和 `p95` 直接来自同角色逐核 raw 分布,用于观察典型核和高 +尾核。I-cache miss rate 只按组内加权口径计算: + +```text +AIC miss rate = Σ(AIC miss) / Σ(AIC request) +AIV miss rate = Σ(AIV miss) / Σ(AIV request) +``` + +不平均 32 或 64 个逐核百分比。AIC/AIV 核数不同,比较每核强度时 +使用 mean/median/p95 或 miss rate,不直接比较两组 sum。 + +## 8. 分析命令与结果文件 + +使用本用户 Python 环境从 raw 重算 host summary,并聚合相同配置的 +多个独立进程: + +```bash +PYTHON=/home/q00473782/.venv/bin/python + +"$PYTHON" ./pmu_sidecar_analyzer.py \ + "$OUT/run1.json" "$OUT/run2.json" "$OUT/run3.json" +``` + +需要机器可读汇总时: + +```bash +"$PYTHON" ./pmu_sidecar_analyzer.py --json \ + "$OUT/run1.json" "$OUT/run2.json" "$OUT/run3.json" \ + > "$OUT/summary.json" +``` + +分析器会先逐份复算 96 条 raw 与 host summary,然后拒绝聚合下列混用: + +- `none` 与 `claim`; +- 不同 schema/build variant; +- 不同 batches、winner mode/count/pattern、selector 或观察开关。 + +建议将每次采集的 JSON 和分析器生成的 summary 放在同一唯一目录; +`outputs/` 为本机证据目录,不作为源码提交的一部分。 + +## 9. 如何使用约 90 ns/miss 标尺 + +当前约 `90 ns/miss` 来自已有隔离 cold/warm 微基准,只用于建立 +数量级感性。对某一角色,可以计算: + +```text +角色总 core-work 串行等效量(us) = Σmiss * 0.09 +该角色平均每核等效量(us/core) = (Σmiss / core_count) * 0.09 +``` + +例如 AIV 平均 70,000 miss/核,感性等效量是约 6,300 us/核。这不表示 +Submit 墙钟真的损失了 6.3 ms,原因包括: + +- 64 个 AIV 之间并行; +- 同一核的 miss、预取、其他流水和等待可能重叠; +- 隔离 cold miss 与真实热路 capacity/conflict/compulsory miss 不一定同价; +- 当前已核实的 A5 事件中没有可直接换算墙钟损失的 I-cache + stall-cycle counter。 + +把 AIC 和 AIV 的 `Σmiss * 90 ns` 相加,只能得到全部核的感性 +core-work 等效总量,不是端到端 Submit 总损失。要测真正暴露的性能 +损失,必须对同语义代码做交错 A/B: + +1. 用相同 `submit-pmu none` 口径确认 `ΔAIC/AIV miss/core`; +2. 另用不开 PMU/泳道的性能构建测 `ΔSubmit span`; +3. 只有第 2 项是实际暴露的墙钟收益;第 1 项用于证明收益与 I-cache + 变化同时出现,`90 ns` 仅提供一阶数量级解释。 + +## 10. 新增局部 phase 的修改清单 + +新 phase 不能只增加一个 CLI 字符串。最小完整修改包括: + +1. `pa_scheduler/common/pa_model.h`:在 `SubmitPmuPhase` 尾部追加稳定 + id,不重排已有 `None=0/Claim=1`。 +2. `pa_scheduler/ccec/pmu_probe.h`:为 `SubmitPmuPhaseName()` 增加名称映射, + 并核对 phase status/边界闭合定义。 +3. `pa_scheduler/ccec/build.sh`:在白名单中将 phase 名映射到稳定 + `PA_SUBMIT_PMU_PHASE_ID`;先校验名称,再用于输出目录。 +4. `pa_scheduler/run.sh`:同步 `build-submit-pmu/submit-pmu` 的 phase + 白名单和 usage。 +5. `pa_scheduler/common/pa_scheduler_core.h`:在真实目标代码段前后放置 + `BeginSubmitPmuPhase<...>()` / `EndSubmitPmuPhase<...>()`。必须检查所有 + 早退、winner/loser 和 Alloc/非 Alloc 分支,不得留下只 begin 不 end + 的路径。 +6. `pa_scheduler/ccec/host.cpp`:增加该 phase 的预期 calls/begin/end + 形状。如果它不是每次 Submit 都调用,不能复用 + `batches * 5 * 96`。 +7. `pa_scheduler/pmu_sidecar_analyzer.py`:同步 phase 名/id 和配置指纹, + 保证不同 phase 输入不会被聚合。 +8. 补充 host/analyzer 回归:`none` 验证 96/96 primary-shadow 精确相等; + running phase 验证逐核 bounded、loss/upper 公式、begin/end/calls 和语义, + 任一 shadow 反向大于 primary 都必须拒绝。先跑 A5 b1,再进入 b256。 + +每个 phase 必须是独立 ELF 和独立进程。不为了一次运行得到多个 +phase,而在热路加运行时 phase switch 或多组 begin/end。 + +## 11. 常见问题与排错 + +### 提示缺少 submit-pmu 产物 + +确认 phase 名与构建命令一致: + +```bash +./run.sh build-submit-pmu ccec none +./run.sh submit-pmu ccec none --device 0 --batches 1 +``` + +不要用 `./run.sh build ccec` 代替;后者生成的是 `swimlane` 产物。 + +### host 提示 swimlane 构建不能采 PMU + +这表示运行了 `build/ccec/pa_scheduler_host`。应通过 +`./run.sh submit-pmu ...` 启动 phase 目录内的 host/kernel/SO 整套产物, +不要手工指向根目录 kernel。 + +### shadow miss 始终为 0 + +先检查 selector 是否错把 `0x35` 放到 CNT9。本机 A5 b1 已经反证 +CNT9 路径;正式配置应为 CNT5 shadow miss、CNT8 shadow request、 +CNT9 unused。 + +### `none` 不相等,或任意 phase 出现 `shadow > primary` + +这两种情况都表示观察链路门禁失败,最终 JSON 不应发布。按下列顺序排查: + +1. host/kernel/owner/dispatcher 是否来自同一 phase 目录; +2. owner 读回的 CNT5/CNT6/CNT7/CNT8 selector 是否与期望一致; +3. begin/end 数是否精确相等,是否有早退路径留下 armed phase; +4. 先缩到 b1;`none` 确认 96/96 exact,running phase 确认 96/96 bounded; +5. 检查可编程 counter 是否超过风险门槛。 + +running phase 出现小幅 `shadow < primary` 时,代码显式发布 loss 和局部 +lower/upper;这不是 standalone 调度正确性异常。只有协议、数值输出、 +placement/engine 也失败时,才应转向调度代码排查。 + +### owner 或 Restore 失败 + +不要在同一设备上并发运行另一个 standalone PMU owner、`msprof` PMU +会话或其他会改 selector 的进程。检查 CANN 环境、两个 AArch64 SO +是否在 kernel 同目录,以及 96 个可用 slot/32 个完整 triplet 是否闭合。 + +### JSON 拒绝覆盖 + +每个独立进程使用新文件名,并处理上次失败留下的同名 `.tmp`。 +host 不会静默覆盖旧证据。 + +### 分析器报配置不一致 + +不要强行合并。逐项比较 phase、batches、winner workload/count/pattern、 +selector、schema 和 build variant。重采相同配置的独立进程。 + +### miss rate 高,但 Submit 没有同比例变慢 + +这不构成矛盾。miss rate 是事件比例,不是 stall 时间比例;多核、 +预取、流水重叠和资源等待都会改变实际暴露量。优先看 AIC/AIV +每核 miss、median/p95 和优化前后的 `Δmiss`,实际性能收益仍以无 +PMU/泳道的交错 A/B 为准。 diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 2ee1ca2673..af14851df0 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -31,9 +31,15 @@ Submit 路径,供后续继续优化。快照日期更新至 2026-07-18;当 - 优化没有改变通用 atomic 语义,也没有把任务推迟到最终 drain 来制造 表面收益。F1 的 fanin 顺序重排已经证明性能回退并撤销;下一步先精确区分 fanin 成功/失败 load 与 frontier 重复前推,再进行单变量消减。 +- standalone 观察产物现已固定为两类:`swimlane` 合并普通阶段与 schema-v3 + atomic(direct Atomic 加 PollBatch)泳道;`submit-pmu` 独立重编译完整 Submit PMU,当前只支持 + `none|claim`。两者不在同一进程采集;`none` 提供完整 Submit 的严格 + 闭合计数,局部 phase 只提供 running read-clear 的下界/保守上界。 环境安装、编译和基线复现过程见 [A5 FDWIC Paged Attention 安装与复现指南](../a5_fdwic_atomic_swimlane_repo.md)。 +standalone I-cache 的当前构建、采集和解读见 +[`../icache_miss_usage_guide.md`](../icache_miss_usage_guide.md)。 ## 2. Case1 工作量与 atomic 语义 @@ -1337,7 +1343,6 @@ outputs/pa_scheduler_swimlane_20260718_182725_4061524/ccec/merged_swimlane.json b1 是零 winner 负载的快速验收,b256 是开启 atomic 泳道的诊断运行;5.774295 ms 只证明观察构建保持目标量级,不是关闭 trace 的正式性能基线,也不能与下列历史样本 做单轮减法归因观察开销。 - 以下是 2026-07-18 的**历史 schema-v2、逐调用、边界修复前**证据,保留用于追溯, 不作为当前 schema-v3 的物理记录规模、逻辑调用数或边界闭合验收。历史 b1 atomic-trace-only 上板中,全部协议断言 PASS,raw 共 4959 条、 @@ -1745,3 +1750,158 @@ outputs/pmu_submit_all_real_compute_b256_20260718T140539Z/run1.json ... run5.jso outputs/pmu_empty_real_compute_b256_20260718T140908Z/run1.json ... run3.json outputs/pmu_submit_all_scalar_nop_b256_same_elf_20260718T141455Z/run1.json ... run3.json ~~~ + +#### 7.5.15 观察构建收敛为 `swimlane` 与 `submit-pmu` + +2026-07-18 在旧统一 ELF 完成 atomic 泳道、PMU owner 和 I-cache 取数 +可行性取证后,standalone 的最终观察产物收敛为两类: + +| 构建 | 正式内容 | 隔离要求 | +| --- | --- | --- | +| `swimlane` | 普通阶段和逐 atomic 记录合并在对应 AIC/AIV scalar lane | 不配置 PMU,不导出 PMU JSON | +| `submit-pmu` | 每核完整 Submit PMU,可带一个编译期局部 phase | 编译掉泳道、逐 atomic、ClockBaseline、runtime phase-profile 和旧 I-cache 冲刷体 | + +旧 O1/O2 文字中的 `empty/scalar/scalar-double/icache-single`、CNT8 fix-busy、 +schema-v3 以及“`--no-swimlane` 仍保留 phase timestamp”都是观察链路建设过程。 +它们保留为历史证据,不再定义当前命令和当前 ELF 口径。 + +当前 `submit-pmu` 仅白名单支持: + +- `none`:不执行局部 counter 边界读取,用于回答完整 Submit 的 + AIC/AIV 每核 request/miss; +- `claim`:在每次 `Claim()` 前后读取 read-to-clear shadow,累计每核 + 1,280 次 Claim 的观测下界,并用本核 primary-shadow residual 给出保守 + 上界;b256 全局期望 calls 为 `256 * 5 * 96 = 122880`。 + +对应命令和产物为: + +~~~bash +./run.sh build-submit-pmu ccec none +./run.sh build-submit-pmu ccec claim + +./run.sh submit-pmu ccec none \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json ./outputs//run1.json + +./run.sh submit-pmu ccec claim \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json ./outputs//run1.json +~~~ + +~~~text +build/ccec/submit-pmu/none/ +build/ccec/submit-pmu/claim/ +~~~ + +每个 phase 目录中的 host、mixed kernel、owner 与 dispatcher 是同一构建 +集,不能跨 phase 复用。`submit-pmu` action 固定单轮、关闭泳道且只打开 +完整 Submit 窗口;不接受 atomic trace、phase profile、泳道分析或泳道 JSON。 + +完整 Submit 的权威 I-cache 计数为不在局部边界读取的 +`CNT6=0x34 primary request` 和 `CNT7=0x35 primary miss`。局部归因需要 +一对可中途 read-to-clear 的重复计数槽。首版将 miss 放在 +`CNT9=0x35`,但 A5 b1 上板中 CNT9 始终为 0,已反证该槽可用性。 +因此正式配置调整为: + +| 槽位 | 配置 | 用途 | +| --- | --- | --- | +| CNT5 | `0x35` | shadow miss;诊断 ELF 不再保留 MTE3 busy | +| CNT6 | `0x34` | primary whole request | +| CNT7 | `0x35` | primary whole miss | +| CNT8 | `0x34` | shadow request | +| CNT9 | `0x0` | 未使用 | + +这个取舍只影响 `submit-pmu` 诊断 ELF,不影响标准 `swimlane`。 +`claim` 在 begin/end 读 CNT8/CNT5,stop 后再读 tail;所有片段的软件 +累加构成 shadow whole。两种构建的接受条件不同: + +~~~text +none(没有运行中 read-clear): + shadow request == primary request + shadow miss == primary miss + +claim(运行中反复 read-clear): + shadow request <= primary request + shadow miss <= primary miss + + request loss = primary request - shadow request + miss loss = primary miss - shadow miss + + phase request ∈ [observed request, observed request + request loss] + phase miss ∈ [observed miss, observed miss + miss loss] +~~~ + +上下界先按每核 raw 计算,再分别按 AIC/AIV 聚合,不能拿聚合后的 median +相减拼区间。CNT8/CNT5 是两条顺序 `ld_dev`,不是原子配对快照,所以 +`phase miss <= phase request` 不是硬门禁;二者分别不得超过对应 shadow, +上界分别不得超过对应 primary。 + +除此之外,还必须闭合 build variant/phase id、每核 begin/end/calls、完整 +Submit 的 `miss <= request`、96 个唯一物理子核、owner bitmap/role/triplet、 +真计算输出、Submit placement/engine、counter 风险门槛和 Restore。`none` +的 phase calls/begin/end/request/miss 必须全为 0,并要求 96/96 shadow 精确 +等于 primary;`claim` 要求 96/96 shadow 不大于 primary,exact 核数只作 +诊断,不再伪装成逐事件精确切片。 + +局部 begin/end 读本身会增加 scalar 取指和改变多核时序,所以 +`claim` 是带观察边界扰动的归因 ELF。不同 phase 的局部 request/miss +不可相加,也不能用 `claim - none` 宣称得到零扰动 Claim 净值。 +每个 ELF 的完整 Submit 始终以它自己的 CNT6/CNT7 primary whole 为准。 +当协议、数值输出和 placement/engine 门禁全部通过时,运行中 shadow 的 +单向负差只能描述为局部 PMU 分段误差,不能描述成 standalone 调度异常。 + +schema-v4 JSON 保留 96 条 raw,并分别给出 ALL/AIC/AIV 的 authoritative +whole、shadow loss 以及 phase lower/upper。raw 中显式保存 +`shadow_request_loss`、`shadow_miss_loss`、 +`phase_icache_requests_upper_bound` 和 `phase_icache_misses_upper_bound`。 +完整 Submit 的 miss rate 只按 `Σmiss/Σrequest` 计算,不平均逐核百分比; +局部 lower miss/lower request 之比只叫 observed read-clear ratio,不是实际 +miss rate 的数学下界。已有隔离微基准的约 +90 ns/miss 只用于 `Σmiss * 0.09 us` 的 core-work 数量级感性估算; +它不是可相加的 Submit stall 常数。真正暴露的墙钟收益必须通过同语义 +代码的交错 A/B,同时观察 `Δmiss/core` 和无 PMU/泳道的 +`ΔSubmit span`。完整操作和排错见 +[`../icache_miss_usage_guide.md`](../icache_miss_usage_guide.md)。 + +#### 7.5.16 完整 Submit 精确闭合与局部 read-clear 边界取证 + +2026-07-18 的四个独立 b256 `submit-pmu none` 进程均使用默认真负载 +`real-compute/6,28,4,1`,且 96/96 核 shadow 与 primary 逐值相等: + +| 轮次 | Submit span/us | request 总和 | miss 总和 | AIC miss/core | AIV miss/core | +| --- | ---: | ---: | ---: | ---: | ---: | +| 1 | 3825.420 | 40,020,837 | 4,748,592 | 38,702.656 | 54,845.422 | +| 2 | 3600.091 | 40,035,347 | 4,726,896 | 38,466.438 | 54,624.531 | +| 3 | 3610.648 | 39,977,052 | 4,728,552 | 38,481.000 | 54,643.125 | +| 4 | 3731.247 | 39,787,176 | 4,715,096 | 38,583.875 | 54,381.438 | + +四轮中位数为:Submit span 3670.948 us、AIC miss/core 38,532.438、 +AIV miss/core 54,633.828。`none` 的严格精确仅指同 selector、同 gate 的完整 +Submit counter 逐核闭合;它不等于真实 PA 的绝对 profile,也不把 PMU +进程的 Submit span 当成无诊断墙钟基线。 + +补充 interval-schema 的单轮 b256 A5 复核: + +| 构建 | Submit span/us | exact/bounded 核 | shadow request loss | shadow miss loss | 语义与真计算 | +| --- | ---: | ---: | ---: | ---: | --- | +| `none` | 4584.835 | 96/96,96/96 | 0 | 0 | 全部 PASS | +| `claim` | 4382.161 | 40/96,96/96 | 253 | 580 | 全部 PASS | + +`claim` 共执行 122,880 次 phase 调用;loss 随运行中边界读取次数放大,但 +untouched primary、PA 任务拓扑、atomic 协议、winner、输出和 engine 观察 +仍全部通过。这组证据把问题定位在运行中 read-to-clear 的局部分段能力, +而不是 standalone scheduler。 + +四轮 `none` 的约 90 ns/miss 感性标尺对应 AIC 约 3.468 ms/core-equivalent、 +AIV 约 4.917 ms/core-equivalent。它们不能相加,也不能叫 Submit 墙钟损失; +实际暴露损失仍为 `UNMEASURED`,必须另做同语义优化前后交错 A/B。 + +本机原始证据位于: + +~~~text +outputs/submit_pmu_none_validation_20260718_b256_real/run1.json ... run4.json +outputs/submit_pmu_final_gate_20260718/none_b256/run1.json +outputs/submit_pmu_final_gate_20260718/claim_b256/run1.json +~~~ diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 28328a4886..91ce316a2b 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -195,7 +195,7 @@ CCEC 与 AscendC 使用本用户安装的 CANN 9.1。非交互 shell 不保证 ```bash cd /path/to/pa_scheduler -source /home/q00473782/Ascend/cann-9.1.0-weekly-20260708/cann/set_env.sh +source /home/q00473782/Ascend/cann-9.1.0-weekly-20260708/cann-9.1.0/set_env.sh export GCC15_ROOT=/home/q00473782/.local/gcc-15/root export PATH="$GCC15_ROOT/usr/bin:$PATH" @@ -211,10 +211,22 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" `include/pto/common/kernel_meta.hpp` 的目录;同一 include tree 还必须具有 `pto/pto-inst.hpp`、`pto/common/constants.hpp` 和 `pto/common/pto_tile.hpp`。 +当前最终只保留两类正式观察构建,不再生成同时夹带泳道与 PMU +诊断代码的统一 CCEC ELF: + +| 构建 | 后端 | 内容 | 构建命令 | 产物目录 | +| --- | --- | --- | --- | --- | +| `swimlane` | CCEC/AscendC/CPU | 普通阶段与 schema-v3 atomic(direct + PollBatch)合并采集;不配置 PMU | `./run.sh build ccec` 或 `./run.sh build all` | CCEC 为 `build/ccec/` | +| `submit-pmu` | 仅 CCEC | 每核完整 Submit PMU,并在编译期可选一个局部阶段;当前只有 `none|claim` | `./run.sh build-submit-pmu ccec none|claim` | `build/ccec/submit-pmu//` | + +`./run.sh build all` 只构建三后端的 `swimlane` 产物;`submit-pmu` +必须按 phase 另行构建。`none` 是不做局部边界读取的完整 Submit +基准,不是第三类构建。 + ## 5. 使用说明:运行、测量与泳道查看 以下命令均在 `pa_scheduler` 目录执行。首次使用应先按第 4 节 source CANN -环境并完成构建。四个 action 的用途如下: +环境并完成构建。主要 action 的用途如下: | action | 用途 | 是否生成泳道文件 | | --- | --- | --- | @@ -222,6 +234,8 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" | `smoke` | 固定 b1/r1/`scalar-nop=0` 的快速语义回归 | 否,只做内存记录校验 | | `run` | 自行控制 batch、run、winner 负载和诊断参数 | 仅显式传入 `--swimlane-json` 时生成 raw | | `swimlane` | 单轮运行并自动生成 raw 和 Perfetto merged JSON | 是 | +| `build-submit-pmu` | 构建指定 `none|claim` 的 CCEC PMU-only ELF | 否 | +| `submit-pmu` | 单轮采集完整 Submit PMU,可选导出 JSON | 否,与泳道隔离 | `ccec|ascendc|cpu|all` 用于选择后端;`all` 始终按 CCEC、AscendC、CPU 的顺序执行。 @@ -255,7 +269,7 @@ semantic_status=PASS postprocess_status=PASS ```bash ./run.sh run ccec \ --device 0 --batches 256 --runs 5 \ - --profile-phases --analyze-swimlane + --analyze-swimlane ./run.sh run ascendc \ --device 0 --batches 256 --runs 5 \ @@ -277,7 +291,7 @@ semantic_status=PASS postprocess_status=PASS ```bash ./run.sh swimlane ccec \ - --device 0 --batches 256 --profile-phases --analyze-swimlane + --device 0 --batches 256 --analyze-swimlane ./run.sh swimlane ccec \ --device 0 --batches 1 --winner-workload real-compute @@ -337,8 +351,9 @@ runner 结束时会打印准确目录: 字段和解读边界见 5.6 节。 WaitForSlot 和 HeapGuard 没有可伪造的逐事件起止时间,因此不单独生成 Perfetto -事件;它们由 `--profile-phases` 的 `[PHASE]` 累计统计呈现。实际发生等待时, -泳道中会出现 RingBp 事件。 +事件;实际发生等待时,泳道中会出现 RingBp 事件。CCEC 的局部 PMU +归因不复用运行时 `--profile-phases`,而是使用第 5.7 节的独立 +`submit-pmu` phase ELF。 `outputs/` 已被 Git 忽略,生成的几十至数百 MiB 泳道文件不会被普通 `git add` 意外纳入提交。 @@ -378,7 +393,9 @@ CPU 完整协议回归建议关闭大泳道缓冲区: 主要选项: -- `--profile-phases`:分别统计 Claim、EfDrain、WaitForSlot、HeapGuard; +- `--profile-phases`:CPU/AscendC 兼容诊断中分别统计 Claim、EfDrain、 + WaitForSlot、HeapGuard;最终 CCEC `swimlane` 构建不接受该选项, + CCEC 局部归因使用独立 `submit-pmu` phase; - `--analyze-swimlane`:读取完整记录,输出各阶段的 per-worker 累计分布以及 EfDrain/Materialize/Claim/Register 的 per-role、per-task-kind 单事件分布; - `--trace-atomics`:在已开启的泳道中记录 atomic 逻辑调用;direct 调用逐条记录, @@ -399,10 +416,11 @@ CPU 完整协议回归建议关闭大泳道缓冲区: ``` 它排除启动屏障和最终 drain。`host_launch_us` 另外包含 host launch、最终 drain -和 stream/thread 同步。不同版本比较时,必须同时开启或同时关闭 -`--profile-phases` 和泳道,因为计时与记录本身会影响竞争时序。 +和 stream/thread 同步。不同版本比较时,必须使用相同构建、相同 phase +和相同泳道/PMU 开关,因为计时、记录和 PMU 边界本身都会影响竞争时序。 -`[PHASE]` 的每个阶段都是每 worker 在 1,280 次 Submit 中的累计时间: +CPU/AscendC 兼容诊断中,`[PHASE]` 的每个阶段都是每 worker 在 +1,280 次 Submit 中的累计时间: - Claim:当前 worker 实际参与或跳过对应 lane Claim 的完整 span; - EfDrain:每次 Submit 开头执行已就绪私有 slot 的时间; @@ -643,7 +661,6 @@ b256 是开启 atomic 泳道的诊断运行;上表 Submit 只能证明当前 不能替代关闭 trace 的性能基线或与历史样本做单轮减法。 #### 历史 schema-v2 样本(仅保留旧口径) - 2026-07-18 的旧 CCEC b256 文件 `outputs/scalar_observation_final_20260718/atomic_inlineasm_ccec_b256/raw.json` 记录了 963,368 条物理记录,其中逐条 Atomic 99,944 条、ClockBaseline 192 条, @@ -651,7 +668,119 @@ b256 是开启 atomic 泳道的诊断运行;上表 Submit 只能证明当前 phase/lap/Kernel 边界修复之前的 schema-v2 逐调用模型,只能用于追溯旧版观察结果; 不能拿 99,944 当作当前 schema-v3 的物理容量、逻辑调用数或闭合证据。 -### 5.7 CCEC 每核 PMU 与 I-cache sidecar +### 5.7 两类正式构建与 CCEC Submit PMU + +`swimlane` 和 `submit-pmu` 是两个独立重编译的观察产物: + +- `swimlane` 编译普通阶段和逐 atomic record,把 atomic 画在对应 + AIC/AIV scalar lane;不生成 PMU owner,也不输出 PMU JSON。 +- `submit-pmu` 编译掉泳道 record、逐 atomic wrapper、ClockBaseline、 + runtime phase-profile 和旧 cold/warm 冲刷体,只保留完整 Submit PMU + 与一个编译期 phase。 + +两者不能在同一进程同时采集。这不只是 CLI 限制:泳道/逐 atomic +代码会改变 scalar 指令布局和 I-cache 本身,将其保留在 PMU ELF 里即使 +运行时关闭 record,也会污染要观察的取指环境。 + +当前 `submit-pmu` 只支持: + +| phase | 局部边界 | 用途 | +| --- | --- | --- | +| `none` | 不做任何中途 shadow counter 读取 | 完整 Submit 主基准,优先用于回答 AIC/AIV 每核 request/miss | +| `claim` | 每次 `Claim()` 调用前后读取 shadow counter | 验证局部归因链路,输出带观察扰动的 running read-clear 下界和保守上界 | + +分别构建: + +```bash +./run.sh build-submit-pmu ccec none +./run.sh build-submit-pmu ccec claim +``` + +产物完全分开: + +```text +build/ccec/submit-pmu/none/ +build/ccec/submit-pmu/claim/ +``` + +每个目录都自包含同 phase 的 host、mixed kernel、PMU owner 和 dispatcher, +不得跨 phase 拼装。构建完成后才会原子发布包含 phase 身份和四个 SHA256 的 +manifest;`submit-pmu` action 在启动 host 前逐项复核,缺件、串 phase 或内容 +变化都会直接拒绝。一次正式采集示例: + +```bash +OUT="./outputs/submit_pmu_none_$(date -u +%Y%m%dT%H%M%SZ)" +mkdir -p "$OUT" +./run.sh submit-pmu ccec none \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT/run1.json" +``` + +`submit-pmu` action 自己固定 `--runs 1 --no-swimlane --pmu-window submit-all`, +不要重复传入这三项,也不能传入 `--profile-phases`、 +`--trace-atomics`、`--analyze-swimlane` 或 `--swimlane-json`。 + +完整 Submit 的权威 I-cache 主计数是从不在局部边界读取的 +`CNT6=request` 和 `CNT7=miss`。A5 b1 实测已反证 `CNT9=0x35` +可作有效计数槽:它始终为 0。因此正式 `submit-pmu` 用 +`CNT8=0x34` 作 shadow request、`CNT5=0x35` 作 shadow miss,`CNT9` +保持未使用;这会牺牲 PMU 诊断版的 `mte3_busy`,不影响标准 +`swimlane` 构建。 + +shadow 计数器是 read-to-clear:`claim` 在 begin/end 切分片段,stop 时再加 +tail,从而软件重建完整 Submit shadow whole。`none` 没有运行中读取,必须 +在每个物理子核精确满足: + +```text +CNT8 shadow whole request == CNT6 primary whole request +CNT5 shadow whole miss == CNT7 primary whole miss +``` + +`claim` 在 A5 上运行中反复 read-clear 时,shadow 可能在边界处单向少计, +因此接受条件改为逐核: + +```text +shadow request <= primary request +shadow miss <= primary miss + +request loss = primary request - shadow request +miss loss = primary miss - shadow miss + +phase request ∈ [observed request, observed request + request loss] +phase miss ∈ [observed miss, observed miss + miss loss] +``` + +区间必须逐核构造后再聚合。CNT8/CNT5 是顺序 `ld_dev` 而非原子配对快照, +不要求局部 `phase miss <= phase request`;只要求二者分别不超过对应 shadow, +上界分别不超过对应 primary。`none` 中 phase calls/begin/end/request/miss 必须 +全为 0;`claim` 中每核 begin/end/calls 必须配对,且每核 calls 必须等于 +`batches * 5`,全局为 `batches * 5 * 96`。 + +局部边界读取本身会增加 scalar 指令、改变 I-cache 布局和多核时序, +因此 `claim` 是带边界扰动的归因结果。`none` 与 `claim` 是不同 ELF/ +不同进程;今后新增的不同 phase 也必须各自单独采集。不同 phase +ELF 的局部 request/miss **不可相加**,也不能与 `none` 相减后宣称 +得到了零扰动的阶段净值。这个区间只约束同一插桩 ELF、当前边界定义下的 +局部事件;它不是无插桩 Claim 的真实区间。调度语义、真计算输出和 +placement/engine 门禁都通过时,running shadow 的负差属于观测边界行为, +不得描述为 standalone scheduler 异常。 + +JSON 保留 96 条 raw record,并按 ALL/AIC/AIV 输出 authoritative whole、 +shadow loss 和 phase lower/upper。raw 中包含 `shadow_request_loss`、 +`shadow_miss_loss`、`phase_icache_requests_upper_bound` 与 +`phase_icache_misses_upper_bound`;host 还分别报告 exact/bounded 核数。 +完整 Submit 的组内 miss rate 才按 `sum(misses)/sum(requests)` 计算,不平均 +逐核百分比;局部 lower miss/lower request 之比只是 observed read-clear +ratio,不是实际 miss rate 的数学下界。更完整的 I-cache 采集、分析、估算与排错见 +[`../icache_miss_usage_guide.md`](../icache_miss_usage_guide.md)。 + +#### 历史 PMU 校准资料(不属于当前两类正式构建) + +以下 `empty/scalar/scalar-double/icache-single`、CNT8 fix-busy 和 schema-v3 +文字保留为 2026-07-18 观察链路的建设过程与历史数据。当前 +`swimlane` 构建不提供 PMU,当前 `submit-pmu` 也只接受完整 +Submit 的 `none|claim`;不应继续照抄下文的历史校准命令作为当前用法。 CCEC 后端提供与泳道分离的 PMU sidecar。正式取数由本目录自带的 Main AICPU Path-A owner 配置 selector、保存并恢复 PMU 状态;kernel 在每个物理子核内门控并 @@ -682,7 +811,7 @@ Path-A owner 配置 selector、保存并恢复 PMU 状态;kernel 在每个物 | `pmu_warm_icache_requests` | `CNT6 = 0x034` | 同核 warm 对照 request | | `pmu_warm_icache_misses` | `CNT7 = 0x035` | 同核 warm 对照 miss | -#### Main AICPU Path-A owner +#### 历史:Main AICPU Path-A owner owner 已自包含在 `ccec/`:构建会同时产出 dispatcher 和 owner AArch64 SO。host 通过 CANN 9.1 已验证的 Main AICPU Path-A 完成 bootstrap 和 mode-0 注册,运行时 @@ -747,7 +876,7 @@ icache_pairs=96/96 calibrated_cores=96/96 时间差为正。也就是说,最终系数的分母不是推测的循环次数,而是严格闭合的 `CNT7` miss 差值。 -#### 生成正式 PMU-only JSON +#### 历史:生成旧 PMU-only JSON 正式 sidecar 使用单轮、独立进程和唯一输出路径: @@ -812,7 +941,7 @@ JSON 包含: owner 配置, 并用多个独立进程交错 A/B。 -#### 2026-07-18 上板验收样本 +#### 历史:2026-07-18 上板验收样本 自包含 owner 的本次验收中,`empty`、`scalar 100,000` 和 `scalar-double 2×100,000` 的 96 核 total 中位数分别约为 214、56,568 和 @@ -837,7 +966,7 @@ PMU-only 验收。该段保留的是切换默认模式之前的历史样本。Su 同配置 PMU 取数可重复;由于 gate 包含 `PIPE_ALL` 且未与无 PMU 样本交错配对, 不应将它们与约 5 ms 无诊断基线直接相减。 -#### 单次 CNT7 I-cache miss 的 scalar 一阶估算标尺 +#### 历史:单次 CNT7 I-cache miss 的 scalar 一阶估算标尺 2026-07-18 的 96 核并发 cold/warm 配对中,15/15 轮均精确满足 `cold CNT7 == trials`、`warm CNT7 == 0`和 `calibrated_cores=96/96`。按每轮 @@ -881,7 +1010,7 @@ tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ 模式扩展成 Claim、EfDrain 等多个正式局部窗口。性能 A/B 仍要保持观察布局一致, 最终端到端收益以关闭 PMU 和泳道的独立进程结果为准。 -#### 校验并聚合多轮 PMU sidecar +#### 历史:校验并聚合多轮 PMU sidecar `pmu_sidecar_analyzer.py` 只读消费当前 schema-v3 JSON。它不信任单轮 host 已写好的 summary,而是从每份文件的 worker raw 记录重新计算 ALL/AIC/AIV 的 diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh index 13779f46d1..673788ad2f 100755 --- a/tests/atomic_probe/pa_scheduler/ccec/build.sh +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -14,16 +14,45 @@ set -euo pipefail # 所有输入和产物都从脚本自身位置解析,调用者无需位于仓库根目录。 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" +SUBMIT_PMU_MANIFEST_NAME="submit_pmu_artifacts.manifest" # CCEC 不再生成同时夹带泳道与 PMU 的统一 ELF。无参数保持兼容并明确等价于 -# swimlane;该产物只保留普通阶段和 atomic 的合并泳道能力。 +# swimlane;submit-pmu 的 phase 必须先由白名单映射为稳定数值,不能把任意 +# 字符串直接用于目录或三套编译器的宏。 BUILD_VARIANT="${1:-swimlane}" -if [[ $# -gt 1 || "$BUILD_VARIANT" != "swimlane" ]]; then - echo "Usage: $0 [swimlane]" >&2 - exit 1 -fi -BUILD_DIR="$ROOT_DIR/build/ccec" -VARIANT_DEFINES=(-DPA_BUILD_SWIMLANE=1 -DPA_BUILD_SUBMIT_PMU=0) +case "$BUILD_VARIANT" in + swimlane) + if [[ $# -gt 1 ]]; then + echo "Usage: $0 [swimlane]" >&2 + exit 1 + fi + PHASE_NAME="none" + PHASE_ID=0 + BUILD_DIR="$ROOT_DIR/build/ccec" + VARIANT_DEFINES=(-DPA_BUILD_SWIMLANE=1 -DPA_BUILD_SUBMIT_PMU=0 -DPA_SUBMIT_PMU_PHASE_ID=0) + ;; + submit-pmu) + if [[ $# -ne 2 ]]; then + echo "Usage: $0 submit-pmu none|claim" >&2 + exit 1 + fi + PHASE_NAME="$2" + case "$PHASE_NAME" in + none) PHASE_ID=0 ;; + claim) PHASE_ID=1 ;; + *) + echo "Unknown submit-pmu phase: $PHASE_NAME (expected none|claim)" >&2 + exit 1 + ;; + esac + BUILD_DIR="$ROOT_DIR/build/ccec/submit-pmu/$PHASE_NAME" + VARIANT_DEFINES=(-DPA_BUILD_SWIMLANE=0 -DPA_BUILD_SUBMIT_PMU=1 "-DPA_SUBMIT_PMU_PHASE_ID=$PHASE_ID") + ;; + *) + echo "Usage: $0 [swimlane] | $0 submit-pmu none|claim" >&2 + exit 1 + ;; +esac # 编译只依赖本目录源码与用户安装的 CANN/PTO 头,不引用 pa_scheduler 目录外的 simpler 构建产物。 if [[ -z "${ASCEND_HOME_PATH:-}" ]]; then @@ -43,10 +72,18 @@ if [[ ! -x "$CCEC" || ! -x "$LD" ]]; then echo "CCEC or ld.lld is missing under ASCEND_HOME_PATH=$ASCEND_HOME_PATH" >&2 exit 1 fi +if [[ "$BUILD_VARIANT" == "submit-pmu" && ! -x "$HCC" ]]; then + echo "The AICPU HCC compiler is missing under ASCEND_HOME_PATH=$ASCEND_HOME_PATH" >&2 + exit 1 +fi if ! command -v "$READELF_BIN" >/dev/null 2>&1; then echo "readelf is required to verify the mixed AICore ELF." >&2 exit 1 fi +if [[ "$BUILD_VARIANT" == "submit-pmu" ]] && ! command -v sha256sum >/dev/null 2>&1; then + echo "sha256sum is required to publish the submit-pmu artifact manifest." >&2 + exit 1 +fi if [[ ! -f "$PTO_INCLUDE_ROOT/include/pto/common/kernel_meta.hpp" ]]; then echo "PTO kernel metadata header is missing under $PTO_INCLUDE_ROOT/include" >&2 exit 1 @@ -59,11 +96,17 @@ for header in pto/pto-inst.hpp pto/common/constants.hpp pto/common/pto_tile.hpp; done mkdir -p "$BUILD_DIR" -# 旧统一构建可能在根目录残留 PMU owner;swimlane 构建主动移除这两个 -# 不属于本变体的产物,避免 direct host 调用误加载上一版诊断 SO。 -rm -f \ - "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so" \ - "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so" +if [[ "$BUILD_VARIANT" == "swimlane" ]]; then + # 旧统一构建可能在根目录残留 PMU owner;swimlane 构建主动移除这两个 + # 不属于本变体的产物,避免 direct host 调用误加载上一版诊断 SO。 + rm -f \ + "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so" \ + "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so" +else + # manifest 是同一 phase 四件套唯一的“可运行”标记。重建一开始先使旧 + # manifest 失效;即使后续编译中断,run.sh 也不会消费目录里的半成品。 + rm -f -- "$BUILD_DIR/$SUBMIT_PMU_MANIFEST_NAME" +fi # 关闭编译器自动插入的 scalar DCCI,由 kernel.cpp 中与 PA 对齐的显式失效/回写协议负责 cache 可见性。 # 两种架构共用这些 ABI、栈和优化参数,避免 AIC/AIV 对共享 SchedulerState 产生不同解释。 @@ -200,18 +243,52 @@ check_icache_probe_layout() { "thrash=0x$thrash_hex/$thrash_size" } -# swimlane ELF 明确不含 PMU 校准目标;I-cache 布局检查只属于后续独立的 -# submit-pmu 诊断构建,不能反过来要求泳道产物携带 64 KiB 冲刷体。 -if [[ "$BUILD_VARIANT" != "swimlane" ]]; then - check_icache_probe_layout aic - check_icache_probe_layout aiv -fi +# 两个正式 ELF 都不携带旧 cold/warm 校准冲刷体;submit-pmu 只观察真实 +# Submit。保留上面的检查函数供历史布局取证时复核,但正式构建不调用它。 # PMU selector/CTRL 的所有权必须由主 aicpu_scheduler 配置并在退出前恢复。 # standalone 目录内自带 Path-A dispatcher 与 owner:前者负责把 owner SO # 落到设备预安装目录,后者由 mode=0 JSON 注册并通过统一入口执行命令。 -# swimlane 构建不生成 PMU owner/dispatcher;二者必须与 submit-pmu kernel、 -# host 使用同一编译期 phase 配置并保存在对应诊断目录中。 +# swimlane 构建不生成 PMU owner/dispatcher;submit-pmu 则把 kernel、host、 +# owner 与 dispatcher 全部放在同一个 phase 目录,禁止跨 phase 复用。 +if [[ "$BUILD_VARIANT" == "submit-pmu" ]]; then + echo "[BUILD] self-contained AICPU PMU dispatcher" + "$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra -Werror \ + -Wl,--build-id \ + "$SCRIPT_DIR/pmu_owner_dispatcher.cpp" \ + -o "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so" + + echo "[BUILD] self-contained AICPU PMU owner" + "$HCC" -shared -fPIC -O3 -g -std=gnu++17 -Wall -Wextra -Werror \ + -Wl,--build-id \ + "${VARIANT_DEFINES[@]}" \ + -I"$SCRIPT_DIR" \ + "$SCRIPT_DIR/pmu_owner_aicpu.cpp" \ + -o "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so" + + OWNER_HEADER="$("$READELF_BIN" --file-header "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so")" + OWNER_SYMBOLS="$("$READELF_BIN" --dyn-syms --wide "$BUILD_DIR/libpa_scheduler_pmu_owner_aicpu.so")" + DISPATCHER_HEADER="$("$READELF_BIN" --file-header "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so")" + DISPATCHER_SYMBOLS="$("$READELF_BIN" --dyn-syms --wide "$BUILD_DIR/libpa_scheduler_pmu_owner_dispatcher.so")" + if [[ "$OWNER_HEADER" != *"Type: DYN"* || + "$OWNER_HEADER" != *"Machine: AArch64"* || + "$DISPATCHER_HEADER" != *"Type: DYN"* || + "$DISPATCHER_HEADER" != *"Machine: AArch64"* ]]; then + echo "PMU dispatcher and owner must both be AArch64 shared objects." >&2 + exit 1 + fi + if [[ "$OWNER_SYMBOLS" != *" simpler_aicpu_exec"* ]]; then + echo "Missing main AICPU PMU owner entry: simpler_aicpu_exec" >&2 + exit 1 + fi + for entry in StaticTileFwkBackendKernelServer DynTileFwkBackendKernelServerInit DynTileFwkBackendKernelServer; do + if [[ "$DISPATCHER_SYMBOLS" != *" $entry"* ]]; then + echo "Missing AICPU PMU dispatcher entry: $entry" >&2 + exit 1 + fi + done + echo "[CHECK] Path-A dispatcher and main AICPU PMU owner exports are present" +fi # host runner 只链接用户 CANN 9.1 的 ACL/runtime,并写入同一安装目录的 rpath,运行时不需要 simpler 动态库。 # `-Werror` 让 host API 签名或尺寸类型变化在构建期暴露,避免到上板阶段才出现参数截断。 @@ -230,4 +307,46 @@ echo "[BUILD] CCEC host runner" -ldl \ -o "$BUILD_DIR/pa_scheduler_host" +if [[ "$BUILD_VARIANT" == "submit-pmu" ]]; then + # host、kernel、owner、dispatcher 全部成功后才生成 manifest;校验和使用 + # 相对文件名,目录复制后仍可在 run 前原样复核。临时文件与最终文件位于 + # 同一目录,mv 只承担单文件原子发布,不会暴露半写 manifest。 + SUBMIT_PMU_ARTIFACTS=( + pa_scheduler_host + pa_scheduler_kernel.o + libpa_scheduler_pmu_owner_aicpu.so + libpa_scheduler_pmu_owner_dispatcher.so + ) + for artifact in "${SUBMIT_PMU_ARTIFACTS[@]}"; do + if [[ ! -s "$BUILD_DIR/$artifact" ]]; then + echo "Cannot publish submit-pmu manifest; artifact is missing or empty: $artifact" >&2 + exit 1 + fi + done + if [[ ! -x "$BUILD_DIR/pa_scheduler_host" ]]; then + echo "Cannot publish submit-pmu manifest; host runner is not executable." >&2 + exit 1 + fi + + MANIFEST_PATH="$BUILD_DIR/$SUBMIT_PMU_MANIFEST_NAME" + MANIFEST_TMP="$(mktemp "$BUILD_DIR/.${SUBMIT_PMU_MANIFEST_NAME}.tmp.XXXXXX")" + cleanup_manifest_tmp() { + if [[ -n "${MANIFEST_TMP:-}" ]]; then + rm -f -- "$MANIFEST_TMP" + fi + } + trap cleanup_manifest_tmp EXIT + { + printf '# schema=pa_scheduler_submit_pmu_artifacts/v1\n' + printf '# variant=submit-pmu\n' + printf '# phase=%s\n' "$PHASE_NAME" + printf '# phase_id=%u\n' "$PHASE_ID" + (cd "$BUILD_DIR" && sha256sum "${SUBMIT_PMU_ARTIFACTS[@]}") + } > "$MANIFEST_TMP" + mv -f -- "$MANIFEST_TMP" "$MANIFEST_PATH" + MANIFEST_TMP="" + trap - EXIT + echo "[CHECK] submit-pmu artifact manifest published: $MANIFEST_PATH" +fi + echo "[BUILD] complete: $BUILD_DIR" diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index a169031efb..c3775a4294 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -284,12 +284,16 @@ struct PmuAggregate { std::vector scalar_busy; std::vector mte1_busy; std::vector mte2_busy; - std::vector mte3_busy; std::vector icache_requests; std::vector icache_misses; std::vector warm_icache_requests; std::vector warm_icache_misses; std::vector fix_busy; + std::vector phase_calls; + std::vector phase_icache_requests; + std::vector phase_icache_misses; + std::vector shadow_icache_requests; + std::vector shadow_icache_misses; uint32_t trusted = 0; }; @@ -303,14 +307,21 @@ void AddPmuSample(const pa_scheduler::WorkerResult &result, PmuAggregate *aggreg aggregate->scalar_busy.push_back(result.pmu_scalar_busy); aggregate->mte1_busy.push_back(result.pmu_mte1_busy); aggregate->mte2_busy.push_back(result.pmu_mte2_busy); - aggregate->mte3_busy.push_back(result.pmu_mte3_busy); aggregate->icache_requests.push_back(result.pmu_icache_requests); aggregate->icache_misses.push_back(result.pmu_icache_misses); aggregate->warm_icache_requests.push_back(result.pmu_warm_icache_requests); aggregate->warm_icache_misses.push_back(result.pmu_warm_icache_misses); aggregate->fix_busy.push_back(result.pmu_fix_busy); + aggregate->phase_calls.push_back(result.pmu_phase_calls); + aggregate->phase_icache_requests.push_back(result.pmu_phase_icache_requests); + aggregate->phase_icache_misses.push_back(result.pmu_phase_icache_misses); + aggregate->shadow_icache_requests.push_back(result.pmu_shadow_icache_requests); + aggregate->shadow_icache_misses.push_back(result.pmu_shadow_icache_misses); aggregate->trusted += - (result.pmu_status & pa_scheduler::ccec_pmu::kStatusRequired) == pa_scheduler::ccec_pmu::kStatusRequired; + (result.pmu_status & pa_scheduler::ccec_pmu::kStatusRequired) == + pa_scheduler::ccec_pmu::kStatusRequired && + (result.pmu_phase_status & pa_scheduler::ccec_pmu::kPhaseStatusRequired) == + pa_scheduler::ccec_pmu::kPhaseStatusRequired; } bool PrintSingleIcacheAggregate( @@ -379,8 +390,6 @@ void PrintPmuAggregate(const char *name, const PmuAggregate &aggregate) { pa_scheduler::host::SummarizeUint64(aggregate.mte1_busy); const pa_scheduler::host::Uint64Distribution mte2 = pa_scheduler::host::SummarizeUint64(aggregate.mte2_busy); - const pa_scheduler::host::Uint64Distribution mte3 = - pa_scheduler::host::SummarizeUint64(aggregate.mte3_busy); const pa_scheduler::host::Uint64Distribution requests = pa_scheduler::host::SummarizeUint64(aggregate.icache_requests); const pa_scheduler::host::Uint64Distribution misses = @@ -389,16 +398,55 @@ void PrintPmuAggregate(const char *name, const PmuAggregate &aggregate) { std::printf( "[PMU-%s] cores=%zu total_sum=%llu total_median=%.1f total_p95=%llu " "scalar_busy=%llu vector_busy=%llu cube_busy=%llu mte1_busy=%llu mte2_busy=%llu " - "mte3_busy=%llu icache_req=%llu icache_miss=%llu miss_rate=%.4f%%\n", + "icache_req=%llu icache_miss=%llu miss_rate=%.4f%%\n", name, aggregate.total_cycles.size(), static_cast(total.total), total.median, static_cast(total.p95), static_cast(scalar.total), static_cast(vector.total), static_cast(cube.total), static_cast(mte1.total), static_cast(mte2.total), - static_cast(mte3.total), static_cast(requests.total), static_cast(misses.total), miss_rate ); } +void PrintSubmitPmuPhaseAggregate(const char *name, const PmuAggregate &aggregate) { + const pa_scheduler::host::Uint64Distribution calls = + pa_scheduler::host::SummarizeUint64(aggregate.phase_calls); + const pa_scheduler::host::Uint64Distribution requests = + pa_scheduler::host::SummarizeUint64(aggregate.phase_icache_requests); + const pa_scheduler::host::Uint64Distribution misses = + pa_scheduler::host::SummarizeUint64(aggregate.phase_icache_misses); + const pa_scheduler::host::Uint64Distribution primary_requests = + pa_scheduler::host::SummarizeUint64(aggregate.icache_requests); + const pa_scheduler::host::Uint64Distribution primary_misses = + pa_scheduler::host::SummarizeUint64(aggregate.icache_misses); + const pa_scheduler::host::Uint64Distribution shadow_requests = + pa_scheduler::host::SummarizeUint64(aggregate.shadow_icache_requests); + const pa_scheduler::host::Uint64Distribution shadow_misses = + pa_scheduler::host::SummarizeUint64(aggregate.shadow_icache_misses); + const uint64_t request_loss = primary_requests.total >= shadow_requests.total + ? primary_requests.total - shadow_requests.total + : 0U; + const uint64_t miss_loss = primary_misses.total >= shadow_misses.total + ? primary_misses.total - shadow_misses.total + : 0U; + const double miss_rate = requests.total == 0U ? 0.0 : 100.0 * misses.total / requests.total; + std::printf( + "[PMU-PHASE-%s] phase=%s semantics=%s cores=%zu calls=%llu " + "icache_req=[%llu,%llu] icache_miss=[%llu,%llu] " + "observed_read_clear_ratio=%.4f%% shadow_loss=%llu/%llu\n", + name, pa_scheduler::ccec_pmu::SubmitPmuPhaseName(pa_scheduler::kCompiledSubmitPmuPhase), + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? "disabled" + : "running_read_clear_lower_bound", + aggregate.phase_calls.size(), static_cast(calls.total), + static_cast(requests.total), + static_cast(requests.total + request_loss), + static_cast(misses.total), + static_cast(misses.total + miss_loss), miss_rate, + static_cast(request_loss), + static_cast(miss_loss) + ); +} + struct PmuValidation { uint32_t trusted = 0; uint32_t unique_physical_core_ids = 0; @@ -414,10 +462,25 @@ struct PmuValidation { uint32_t submit_engine_workers_expected = 0; uint32_t submit_engine_workers_matched = 0; uint32_t maximum_programmable_counter = 0; + uint32_t build_variant_matches = 0; + uint32_t phase_id_matches = 0; + uint32_t phase_status_trusted = 0; + uint32_t shadow_primary_matches = 0; + uint32_t shadow_primary_bounded = 0; + uint32_t phase_boundary_matches = 0; + uint32_t phase_call_shape_matches = 0; + uint64_t phase_calls = 0; + uint64_t shadow_request_abs_delta_sum = 0; + uint64_t shadow_miss_abs_delta_sum = 0; + int64_t shadow_request_signed_delta_sum = 0; + int64_t shadow_miss_signed_delta_sum = 0; + uint32_t shadow_request_abs_delta_max = 0; + uint32_t shadow_miss_abs_delta_max = 0; bool icache_order_valid = true; bool icache_measurement_valid = true; bool submit_engine_observation_valid = true; bool counter_below_risk_threshold = true; + bool phase_measurement_valid = false; bool passed = true; }; @@ -450,19 +513,81 @@ bool ValidatePmu( uint32_t submit_engine_workers_expected = 0; uint32_t submit_engine_workers_matched = 0; uint32_t maximum_programmable_counter = 0; + uint32_t build_variant_matches = 0; + uint32_t phase_id_matches = 0; + uint32_t phase_status_trusted = 0; + uint32_t shadow_primary_matches = 0; + uint32_t shadow_primary_bounded = 0; + uint32_t phase_boundary_matches = 0; + uint32_t phase_call_shape_matches = 0; + uint64_t phase_calls = 0; + uint64_t shadow_request_abs_delta_sum = 0; + uint64_t shadow_miss_abs_delta_sum = 0; + int64_t shadow_request_signed_delta_sum = 0; + int64_t shadow_miss_signed_delta_sum = 0; + uint32_t shadow_request_abs_delta_max = 0; + uint32_t shadow_miss_abs_delta_max = 0; bool icache_order_valid = true; uint32_t bad_printed = 0; PmuAggregate all; PmuAggregate aic; PmuAggregate aiv; + const uint32_t expected_phase_calls_per_worker = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? 0U + : state.config.batches * pa_scheduler::kTasksPerBatch; for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { const pa_scheduler::WorkerResult &result = state.results[worker]; const uint32_t status = result.pmu_status; const uint32_t core_id = StatusCoreId(status); const bool record_trusted = (status & kStatusRequired) == kStatusRequired; + const bool variant_matches = result.pmu_build_variant == pa_scheduler::kBuildVariantSubmitPmu; + const bool phase_id_matches_record = + result.pmu_phase_id == static_cast(pa_scheduler::kCompiledSubmitPmuPhase); + const bool phase_status_ok = + (result.pmu_phase_status & kPhaseStatusRequired) == kPhaseStatusRequired; + const bool shadow_matches = + result.pmu_shadow_icache_requests == result.pmu_icache_requests && + result.pmu_shadow_icache_misses == result.pmu_icache_misses; + const bool shadow_bounded = + result.pmu_shadow_icache_requests <= result.pmu_icache_requests && + result.pmu_shadow_icache_misses <= result.pmu_icache_misses; + const bool shadow_acceptable = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? shadow_matches + : shadow_bounded; + const uint32_t request_abs_delta = + result.pmu_shadow_icache_requests >= result.pmu_icache_requests + ? result.pmu_shadow_icache_requests - result.pmu_icache_requests + : result.pmu_icache_requests - result.pmu_shadow_icache_requests; + const uint32_t miss_abs_delta = + result.pmu_shadow_icache_misses >= result.pmu_icache_misses + ? result.pmu_shadow_icache_misses - result.pmu_icache_misses + : result.pmu_icache_misses - result.pmu_shadow_icache_misses; + const bool boundaries_match = + result.pmu_phase_begin_reads == result.pmu_phase_calls && + result.pmu_phase_end_reads == result.pmu_phase_calls; + const bool phase_call_shape_matches_record = + result.pmu_phase_calls == expected_phase_calls_per_worker; const bool logical_aic = worker < pa_scheduler::kAicWorkers; const bool physical_aic = pa_scheduler::pmu_owner::IsAicPhysicalSlot(core_id); trusted += record_trusted; + build_variant_matches += variant_matches; + phase_id_matches += phase_id_matches_record; + phase_status_trusted += phase_status_ok; + shadow_primary_matches += shadow_matches; + shadow_primary_bounded += shadow_bounded; + phase_boundary_matches += boundaries_match; + phase_call_shape_matches += phase_call_shape_matches_record; + phase_calls += result.pmu_phase_calls; + shadow_request_abs_delta_sum += request_abs_delta; + shadow_miss_abs_delta_sum += miss_abs_delta; + shadow_request_signed_delta_sum += + static_cast(result.pmu_shadow_icache_requests) - result.pmu_icache_requests; + shadow_miss_signed_delta_sum += + static_cast(result.pmu_shadow_icache_misses) - result.pmu_icache_misses; + shadow_request_abs_delta_max = std::max(shadow_request_abs_delta_max, request_abs_delta); + shadow_miss_abs_delta_max = std::max(shadow_miss_abs_delta_max, miss_abs_delta); owner_members += owner != nullptr && pa_scheduler::pmu_owner::IsConfigured(*owner, core_id); exact_worker_slots += result.worker_id == worker; physical_role_matches += logical_aic == physical_aic; @@ -481,8 +606,15 @@ bool ValidatePmu( submit_engine_workers_expected += submit_engine_tasks != 0U; submit_engine_workers_matched += engine_observation_matches; } + // phase 的 request/miss 由两条顺序 ld_dev 划界,局部窗口边界并不 + // 完全重合;只要求它们各自不超过完整窗口,不把 phase miss<=request + // 误设成硬门槛。完整 Submit 的 miss<=request 仍必须成立。 icache_order_valid &= result.pmu_icache_misses <= result.pmu_icache_requests && - result.pmu_warm_icache_misses <= result.pmu_warm_icache_requests; + result.pmu_shadow_icache_misses <= result.pmu_shadow_icache_requests && + result.pmu_shadow_icache_requests <= result.pmu_icache_requests && + result.pmu_shadow_icache_misses <= result.pmu_icache_misses && + result.pmu_phase_icache_requests <= result.pmu_shadow_icache_requests && + result.pmu_phase_icache_misses <= result.pmu_shadow_icache_misses; if (pmu.mode == WindowMode::IcacheSingle) { const int64_t worker_cycle_delta = static_cast(result.pmu_total_cycles) - static_cast(result.pmu_warm_total_cycles); @@ -497,9 +629,10 @@ bool ValidatePmu( } const uint32_t programmable[] = { result.pmu_vector_busy, result.pmu_cube_busy, result.pmu_scalar_busy, - result.pmu_mte1_busy, result.pmu_mte2_busy, result.pmu_mte3_busy, - result.pmu_icache_requests, result.pmu_icache_misses, result.pmu_fix_busy, - result.pmu_warm_icache_requests, result.pmu_warm_icache_misses, + result.pmu_mte1_busy, result.pmu_mte2_busy, + result.pmu_icache_requests, result.pmu_icache_misses, + result.pmu_phase_icache_requests, result.pmu_phase_icache_misses, + result.pmu_shadow_icache_requests, result.pmu_shadow_icache_misses, }; for (uint32_t value : programmable) { maximum_programmable_counter = std::max(maximum_programmable_counter, value); @@ -508,15 +641,20 @@ bool ValidatePmu( seen[core_id] = true; ++unique; } - if (!record_trusted && bad_printed < 8) { + if ((!record_trusted || !variant_matches || !phase_id_matches_record || + !phase_status_ok || !shadow_acceptable || !boundaries_match || + !phase_call_shape_matches_record) && bad_printed < 8) { std::printf( "[PMU-BAD] worker=%u role=%llu coreid=%u status=0x%08x total=%llu scalar=%u " - "req=%u miss=%u warm_total=%llu warm_req=%u warm_miss=%u\n", + "req=%u miss=%u phase_status=0x%08x phase=%u/%u calls=%u boundaries=%u/%u " + "shadow=%u/%u\n", worker, static_cast(result.role), core_id, status, static_cast(result.pmu_total_cycles), result.pmu_scalar_busy, result.pmu_icache_requests, result.pmu_icache_misses, - static_cast(result.pmu_warm_total_cycles), - result.pmu_warm_icache_requests, result.pmu_warm_icache_misses + result.pmu_phase_status, result.pmu_phase_id, + static_cast(pa_scheduler::kCompiledSubmitPmuPhase), result.pmu_phase_calls, + result.pmu_phase_begin_reads, result.pmu_phase_end_reads, + result.pmu_shadow_icache_requests, result.pmu_shadow_icache_misses ); ++bad_printed; } @@ -545,6 +683,9 @@ bool ValidatePmu( PrintPmuAggregate("ALL", all); PrintPmuAggregate("AIC", aic); PrintPmuAggregate("AIV", aiv); + PrintSubmitPmuPhaseAggregate("ALL", all); + PrintSubmitPmuPhaseAggregate("AIC", aic); + PrintSubmitPmuPhaseAggregate("AIV", aiv); bool icache_measurement_ok = true; if (pmu.mode == WindowMode::IcacheSingle) { const bool all_ok = PrintSingleIcacheAggregate("ALL", all, pmu.icache_trials); @@ -562,6 +703,23 @@ bool ValidatePmu( const bool mixed_triplets_ok = mixed_triplet_matches == pa_scheduler::kAicWorkers; const bool windows_started_ok = window_started == pa_scheduler::kWorkers; const bool windows_stopped_ok = window_stopped == pa_scheduler::kWorkers; + const bool build_variant_ok = build_variant_matches == pa_scheduler::kWorkers; + const bool phase_id_ok = phase_id_matches == pa_scheduler::kWorkers; + const bool phase_status_ok = phase_status_trusted == pa_scheduler::kWorkers; + const bool shadow_primary_exact_ok = shadow_primary_matches == pa_scheduler::kWorkers; + const bool shadow_primary_bounded_ok = shadow_primary_bounded == pa_scheduler::kWorkers; + const bool shadow_partition_ok = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? shadow_primary_exact_ok + : shadow_primary_bounded_ok; + const bool phase_boundaries_ok = phase_boundary_matches == pa_scheduler::kWorkers; + const bool phase_call_shape_ok = phase_call_shape_matches == pa_scheduler::kWorkers; + const uint64_t expected_phase_calls = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? 0ULL + : static_cast(state.config.batches) * pa_scheduler::kTasksPerBatch * + pa_scheduler::kWorkers; + const bool phase_calls_ok = phase_calls == expected_phase_calls; const bool submit_engine_observation_ok = pmu.mode != WindowMode::SubmitAll || workload.mode != pa_scheduler::WinnerWorkloadMode::RealCompute || @@ -578,6 +736,16 @@ bool ValidatePmu( maximum_programmable_counter, UINT32_MAX - maximum_programmable_counter ); + std::printf( + "[PMU-SHADOW-DELTA] exact=%u/%u bounded=%u/%u request_abs_sum=%llu request_abs_max=%u " + "request_signed_sum=%lld miss_abs_sum=%llu miss_abs_max=%u miss_signed_sum=%lld\n", + shadow_primary_matches, pa_scheduler::kWorkers, + shadow_primary_bounded, pa_scheduler::kWorkers, + static_cast(shadow_request_abs_delta_sum), + shadow_request_abs_delta_max, static_cast(shadow_request_signed_delta_sum), + static_cast(shadow_miss_abs_delta_sum), + shadow_miss_abs_delta_max, static_cast(shadow_miss_signed_delta_sum) + ); std::printf("[ASSERT] %-48s %s\n", "all PMU records have configured selectors and data", records_ok ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "all 96 PMU physical subcore ids are unique", @@ -594,6 +762,18 @@ bool ValidatePmu( windows_started_ok ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "all 96 started PMU windows executed stop", windows_stopped_ok ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "all records match submit-pmu build and phase ids", + build_variant_ok && phase_id_ok ? "PASS" : "FAIL"); + std::printf( + "[ASSERT] %-48s %s\n", + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? "disabled shadow counters exactly match primary" + : "running shadow partitions do not exceed primary", + shadow_partition_ok ? "PASS" : "FAIL" + ); + std::printf("[ASSERT] %-48s %s\n", "all phase boundaries and per-worker calls are exact", + phase_status_ok && phase_boundaries_ok && phase_call_shape_ok && phase_calls_ok + ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "I-cache misses do not exceed requests", icache_order_valid ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "programmable counters stay below 25% risk threshold", @@ -625,13 +805,31 @@ bool ValidatePmu( validation->submit_engine_workers_expected = submit_engine_workers_expected; validation->submit_engine_workers_matched = submit_engine_workers_matched; validation->maximum_programmable_counter = maximum_programmable_counter; + validation->build_variant_matches = build_variant_matches; + validation->phase_id_matches = phase_id_matches; + validation->phase_status_trusted = phase_status_trusted; + validation->shadow_primary_matches = shadow_primary_matches; + validation->shadow_primary_bounded = shadow_primary_bounded; + validation->phase_boundary_matches = phase_boundary_matches; + validation->phase_call_shape_matches = phase_call_shape_matches; + validation->phase_calls = phase_calls; + validation->shadow_request_abs_delta_sum = shadow_request_abs_delta_sum; + validation->shadow_miss_abs_delta_sum = shadow_miss_abs_delta_sum; + validation->shadow_request_signed_delta_sum = shadow_request_signed_delta_sum; + validation->shadow_miss_signed_delta_sum = shadow_miss_signed_delta_sum; + validation->shadow_request_abs_delta_max = shadow_request_abs_delta_max; + validation->shadow_miss_abs_delta_max = shadow_miss_abs_delta_max; validation->icache_order_valid = icache_order_valid; validation->icache_measurement_valid = icache_measurement_ok; validation->submit_engine_observation_valid = submit_engine_observation_ok; validation->counter_below_risk_threshold = counter_below_risk_threshold; + validation->phase_measurement_valid = + build_variant_ok && phase_id_ok && phase_status_ok && shadow_partition_ok && + phase_boundaries_ok && phase_call_shape_ok && phase_calls_ok; validation->passed = records_ok && core_ids_ok && owner_members_ok && worker_slots_ok && physical_roles_ok && mixed_triplets_ok && windows_started_ok && windows_stopped_ok && icache_order_valid && icache_measurement_ok && submit_engine_observation_ok && + validation->phase_measurement_valid && counter_below_risk_threshold; return validation->passed; } @@ -685,6 +883,7 @@ void WriteMetricDistribution(std::FILE *output, const std::vector &val void WritePmuAggregateJson( std::FILE *output, const PmuAggregate &aggregate, bool icache_single ) { + (void)icache_single; const pa_scheduler::host::Uint64Distribution requests = pa_scheduler::host::SummarizeUint64(aggregate.icache_requests); const pa_scheduler::host::Uint64Distribution misses = @@ -706,14 +905,20 @@ void WritePmuAggregateJson( WriteMetricDistribution(output, aggregate.mte1_busy); std::fputs(",\"mte2_busy\":", output); WriteMetricDistribution(output, aggregate.mte2_busy); - std::fputs(",\"mte3_busy\":", output); - WriteMetricDistribution(output, aggregate.mte3_busy); std::fputs(",\"icache_requests\":", output); WriteMetricDistribution(output, aggregate.icache_requests); std::fputs(",\"icache_misses\":", output); WriteMetricDistribution(output, aggregate.icache_misses); - std::fputs(",\"fix_busy\":", output); - WriteMetricDistribution(output, aggregate.fix_busy); + std::fputs(",\"shadow_whole_icache_requests\":", output); + WriteMetricDistribution(output, aggregate.shadow_icache_requests); + std::fputs(",\"shadow_whole_icache_misses\":", output); + WriteMetricDistribution(output, aggregate.shadow_icache_misses); + std::fputs(",\"phase_calls\":", output); + WriteMetricDistribution(output, aggregate.phase_calls); + std::fputs(",\"phase_icache_requests\":", output); + WriteMetricDistribution(output, aggregate.phase_icache_requests); + std::fputs(",\"phase_icache_misses\":", output); + WriteMetricDistribution(output, aggregate.phase_icache_misses); std::fputs(",\"icache_miss_rate\":", output); if (requests.total == 0) { std::fputs("null", output); @@ -721,60 +926,21 @@ void WritePmuAggregateJson( // 全局 miss rate 必须以总 miss/总 request 计算,不能平均逐核百分比。 std::fprintf(output, "%.17g", static_cast(misses.total) / requests.total); } - if (icache_single) { - const pa_scheduler::host::Uint64Distribution cold_cycles = - pa_scheduler::host::SummarizeUint64(aggregate.total_cycles); - const pa_scheduler::host::Uint64Distribution warm_cycles = - pa_scheduler::host::SummarizeUint64(aggregate.warm_total_cycles); - const pa_scheduler::host::Uint64Distribution cold_ticks = - pa_scheduler::host::SummarizeUint64(aggregate.window_ticks); - const pa_scheduler::host::Uint64Distribution warm_ticks = - pa_scheduler::host::SummarizeUint64(aggregate.warm_window_ticks); - const pa_scheduler::host::Uint64Distribution warm_requests = - pa_scheduler::host::SummarizeUint64(aggregate.warm_icache_requests); - const pa_scheduler::host::Uint64Distribution warm_misses = - pa_scheduler::host::SummarizeUint64(aggregate.warm_icache_misses); - const int64_t cycle_delta = static_cast(cold_cycles.total) - - static_cast(warm_cycles.total); - const int64_t tick_delta = static_cast(cold_ticks.total) - - static_cast(warm_ticks.total); - const int64_t miss_delta = static_cast(misses.total) - - static_cast(warm_misses.total); - std::fputs(",\"icache_single_pair\":{\"cold_window_ticks\":", output); - WriteMetricDistribution(output, aggregate.window_ticks); - std::fputs(",\"warm_total_cycles\":", output); - WriteMetricDistribution(output, aggregate.warm_total_cycles); - std::fputs(",\"warm_window_ticks\":", output); - WriteMetricDistribution(output, aggregate.warm_window_ticks); - std::fputs(",\"warm_icache_requests\":", output); - WriteMetricDistribution(output, aggregate.warm_icache_requests); - std::fputs(",\"warm_icache_misses\":", output); - WriteMetricDistribution(output, aggregate.warm_icache_misses); - std::fprintf( - output, - ",\"cycle_delta\":%lld,\"tick_delta\":%lld,\"miss_delta\":%lld," - "\"cold_request_sum\":%llu,\"warm_request_sum\":%llu,\"ns_per_miss\":", - static_cast(cycle_delta), static_cast(tick_delta), - static_cast(miss_delta), static_cast(requests.total), - static_cast(warm_requests.total) - ); - if (miss_delta <= 0) { - std::fputs("null", output); - } else { - std::fprintf(output, "%.17g", static_cast(tick_delta) / miss_delta); - } - std::fputc('}', output); + const pa_scheduler::host::Uint64Distribution phase_requests = + pa_scheduler::host::SummarizeUint64(aggregate.phase_icache_requests); + const pa_scheduler::host::Uint64Distribution phase_misses = + pa_scheduler::host::SummarizeUint64(aggregate.phase_icache_misses); + // 两个 phase counter 是顺序 read-to-clear,下界之比不是实际 miss rate + // 的数学下界;字段名只陈述它是本次 read-clear 观察值之比。 + std::fputs(",\"phase_observed_read_clear_ratio\":", output); + if (phase_requests.total == 0U) { + std::fputs("null", output); + } else { + std::fprintf(output, "%.17g", static_cast(phase_misses.total) / phase_requests.total); } std::fputc('}', output); } -uint32_t PmuWindowSegments(const PmuOptions &pmu) { - const pa_scheduler::ccec_pmu::WindowMode mode = pmu.mode; - if (mode == pa_scheduler::ccec_pmu::WindowMode::ScalarDouble) return 2U; - if (mode == pa_scheduler::ccec_pmu::WindowMode::IcacheSingle) return pmu.icache_trials * 2U; - return 1U; -} - uint32_t CountConfiguredMixedTriplets(const pa_scheduler::pmu_owner::PmuOwnerControl &owner) { uint32_t complete = 0U; for (uint32_t die_base = 0U; @@ -863,7 +1029,7 @@ bool ExportPmuJson( }; const uint32_t owner_bitmap_count = pa_scheduler::pmu_owner::CountConfigured(owner); const uint32_t owner_complete_triplets = CountConfiguredMixedTriplets(owner); - std::fputs("{\n\"schema\":{\"name\":\"pa_scheduler_pmu_phase_windows\",\"version\":3},\n", output); + std::fputs("{\n\"schema\":{\"name\":\"pa_scheduler_pmu_phase_windows\",\"version\":4},\n", output); std::fputs("\"capture\":{\"capture_id\":", output); WriteJsonString(output, capture_id); std::fprintf( @@ -885,6 +1051,12 @@ bool ExportPmuJson( ); std::fputs("\"configuration\":{\"kernel_path\":", output); WriteJsonString(output, options.kernel_path); + std::fputs(",\"build_variant\":\"submit-pmu\",\"build_variant_id\":2,\"compiled_phase\":", output); + WriteJsonString(output, SubmitPmuPhaseName(pa_scheduler::kCompiledSubmitPmuPhase)); + std::fprintf( + output, ",\"compiled_phase_id\":%u", + static_cast(pa_scheduler::kCompiledSubmitPmuPhase) + ); std::fprintf( output, ",\"device\":%u,\"batches\":%u,\"workers\":%u,\"aic_workers\":%u,\"aiv_workers\":%u," @@ -959,26 +1131,43 @@ bool ExportPmuJson( } std::fprintf( output, - ",\"window_segments_are_mode_contract_per_record\":true," + ",\"primary_window_segments_per_record\":1," "\"icache_single_discarded_training_samples_per_core\":%u," "\"icache_single_sys_counter_tick_ns\":%s," "\"host_launch_to_sync_us\":%.17g,\"submit_span_us\":%.17g," "\"selectors\":{\"cnt0_vector_busy\":%u,\"cnt1_cube_busy\":%u," "\"cnt2_scalar_busy\":%u,\"cnt3_mte1_busy\":%u,\"cnt4_mte2_busy\":%u," - "\"cnt5_mte3_busy\":%u,\"cnt6_icache_request\":%u,\"cnt7_icache_miss\":%u," - "\"cnt8_fix_busy\":%u},\"counter_width_bits\":{\"total\":64,\"programmable\":32}," + "\"cnt5_shadow_icache_miss\":%u,\"cnt6_primary_icache_request\":%u," + "\"cnt7_primary_icache_miss\":%u,\"cnt8_shadow_icache_request\":%u," + "\"cnt9_unused\":0},\"unavailable_metrics\":[\"mte3_busy\"]," + "\"counter_width_bits\":{\"total\":64,\"programmable\":32}," "\"counter_wrap_not_directly_detectable\":true,\"counter_wrap_absence_proven\":false," "\"programmable_counter_risk_threshold\":%u," "\"gate_start_stop_have_pipe_all_barriers\":true," - "\"phase_timestamp_calls_present\":true,\"phase_record_writes\":false," + "\"phase_timestamp_calls_present\":false,\"phase_record_writes\":false," "\"atomic_trace\":false,\"profile_accumulation\":false," + "\"phase_boundary_observation_included\":%s," + "\"phase_counter_pair_snapshot_atomic\":false," + "\"primary_counters_read_at_phase_boundaries\":false," + "\"phase_shadow_partition_exact_required\":%s," + "\"phase_values_are_running_read_clear_lower_bounds\":%s," + "\"cross_phase_elf_sums_valid\":false," "\"simulated_task_nop_mechanism_executes_on_scalar\":%s," "\"simulated_task_nops_nonzero\":%s," "\"icache_miss_rate_definition\":\"sum(icache_misses)/sum(icache_requests)\"},\n", icache_single ? 2U : 0U, icache_single ? "1" : "null", host_us, submit_span_us, kVectorBusyEvent, kCubeBusyEvent, kScalarBusyEvent, - kMte1BusyEvent, kMte2BusyEvent, kMte3BusyEvent, kIcacheRequestEvent, kIcacheMissEvent, - kFixBusyEvent, kProgrammableCounterRiskThreshold, + kMte1BusyEvent, kMte2BusyEvent, kIcacheMissEvent, kIcacheRequestEvent, kIcacheMissEvent, + kIcacheRequestEvent, kProgrammableCounterRiskThreshold, + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? "false" + : "true", + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? "true" + : "false", + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? "false" + : "true", real_compute ? "false" : "true", simulated_task_nops_nonzero ? "true" : "false" ); @@ -1006,7 +1195,17 @@ bool ExportPmuJson( "\"icache_measurement_valid\":%s," "\"icache_miss_le_request\":%s,\"counter_below_risk_threshold\":%s," "\"maximum_programmable_counter\":%u,\"programmable_counter_risk_threshold\":%u," - "\"programmable_counter_headroom\":%u},\n", + "\"programmable_counter_headroom\":%u," + "\"build_variant_match_records\":%u,\"phase_id_match_records\":%u," + "\"phase_status_trusted_records\":%u,\"shadow_primary_match_records\":%u," + "\"shadow_primary_bounded_records\":%u," + "\"shadow_request_abs_delta_sum\":%llu,\"shadow_request_abs_delta_max\":%u," + "\"shadow_request_signed_delta_sum\":%lld," + "\"shadow_miss_abs_delta_sum\":%llu,\"shadow_miss_abs_delta_max\":%u," + "\"shadow_miss_signed_delta_sum\":%lld," + "\"phase_boundary_match_records\":%u,\"phase_call_shape_match_records\":%u," + "\"phase_calls\":%llu," + "\"phase_measurement_valid\":%s},\n", semantic_passed ? "true" : "false", validation.passed ? "true" : "false", real_compute ? "true" : "false", real_compute ? (workload_output_passed ? "true" : "false") : "null", @@ -1032,7 +1231,20 @@ bool ExportPmuJson( validation.icache_order_valid ? "true" : "false", validation.counter_below_risk_threshold ? "true" : "false", validation.maximum_programmable_counter, kProgrammableCounterRiskThreshold, - UINT32_MAX - validation.maximum_programmable_counter + UINT32_MAX - validation.maximum_programmable_counter, + validation.build_variant_matches, validation.phase_id_matches, + validation.phase_status_trusted, validation.shadow_primary_matches, + validation.shadow_primary_bounded, + static_cast(validation.shadow_request_abs_delta_sum), + validation.shadow_request_abs_delta_max, + static_cast(validation.shadow_request_signed_delta_sum), + static_cast(validation.shadow_miss_abs_delta_sum), + validation.shadow_miss_abs_delta_max, + static_cast(validation.shadow_miss_signed_delta_sum), + validation.phase_boundary_matches, + validation.phase_call_shape_matches, + static_cast(validation.phase_calls), + validation.phase_measurement_valid ? "true" : "false" ); std::fprintf( output, @@ -1063,42 +1275,72 @@ bool ExportPmuJson( const pa_scheduler::WorkerResult &result = state.results[worker]; const uint32_t status = result.pmu_status; const uint32_t physical_core_id = StatusCoreId(status); - const bool trusted = (status & kStatusRequired) == kStatusRequired; + const bool primary_trusted = (status & kStatusRequired) == kStatusRequired; + const bool phase_trusted = + (result.pmu_phase_status & kPhaseStatusRequired) == kPhaseStatusRequired; + const bool trusted = primary_trusted && phase_trusted; const bool is_aic = result.role == static_cast(pa_scheduler::CoreRole::Aic); const uint32_t vector_id = is_aic ? 0U : worker - pa_scheduler::kAicWorkers; const uint32_t block_id = is_aic ? worker : vector_id / 2U; const uint32_t lane = is_aic ? 0U : 1U + vector_id % 2U; - const uint32_t segments = PmuWindowSegments(pmu); + const uint32_t shadow_read_segments = result.pmu_phase_calls * 2U + 1U; const bool owner_bitmap_member = pa_scheduler::pmu_owner::IsConfigured(owner, physical_core_id); const bool worker_slot_exact = result.worker_id == worker; const bool physical_role_matches = is_aic == pa_scheduler::pmu_owner::IsAicPhysicalSlot(physical_core_id); const bool window_started = (status & kStatusWindowStarted) != 0U; const bool window_stopped = (status & kStatusWindowStopped) != 0U; - const bool icache_pair_observed = (status & kStatusIcachePairObserved) != 0U; + const bool shadow_matches = + result.pmu_shadow_icache_requests == result.pmu_icache_requests && + result.pmu_shadow_icache_misses == result.pmu_icache_misses; + const bool shadow_bounded = + result.pmu_shadow_icache_requests <= result.pmu_icache_requests && + result.pmu_shadow_icache_misses <= result.pmu_icache_misses; + const uint32_t shadow_request_loss = + result.pmu_icache_requests - result.pmu_shadow_icache_requests; + const uint32_t shadow_miss_loss = + result.pmu_icache_misses - result.pmu_shadow_icache_misses; + const uint32_t phase_request_upper = + result.pmu_phase_icache_requests + shadow_request_loss; + const uint32_t phase_miss_upper = + result.pmu_phase_icache_misses + shadow_miss_loss; + const bool boundaries_balanced = + result.pmu_phase_begin_reads == result.pmu_phase_calls && + result.pmu_phase_end_reads == result.pmu_phase_calls; std::fprintf( output, "%s{\"worker_id\":%u,\"physical_core_id\":%u,\"role\":\"%s\",\"block_id\":%u," - "\"lane\":%u,\"window_segments\":%u,\"window_segment_count_source\":\"mode_contract\"," + "\"lane\":%u,\"primary_window_segments\":1,\"shadow_read_segments\":%u," "\"window_started\":%s,\"window_stopped\":%s,\"total_cycles\":%llu,\"vector_busy\":%u," "\"cube_busy\":%u,\"scalar_busy\":%u,\"mte1_busy\":%u,\"mte2_busy\":%u," - "\"mte3_busy\":%u,\"icache_requests\":%u,\"icache_misses\":%u,\"fix_busy\":%u," - "\"window_ticks\":%llu,\"warm_total_cycles\":%llu,\"warm_window_ticks\":%llu," - "\"warm_icache_requests\":%u,\"warm_icache_misses\":%u," + "\"icache_requests\":%u,\"icache_misses\":%u," + "\"build_variant_id\":%u,\"compiled_phase_id\":%u,\"phase_calls\":%u," + "\"phase_begin_reads\":%u,\"phase_end_reads\":%u," + "\"phase_icache_requests\":%u,\"phase_icache_misses\":%u," + "\"phase_icache_requests_upper_bound\":%u," + "\"phase_icache_misses_upper_bound\":%u," + "\"shadow_whole_icache_requests\":%u,\"shadow_whole_icache_misses\":%u," + "\"shadow_matches_primary\":%s,\"shadow_not_greater_than_primary\":%s," + "\"shadow_request_loss\":%u,\"shadow_miss_loss\":%u," + "\"phase_boundaries_balanced\":%s," + "\"phase_status\":%u,\"phase_status_hex\":\"0x%08x\"," "\"status\":%u,\"status_hex\":" "\"0x%08x\",\"trusted\":%s,\"physical_core_id_valid\":%s,\"selectors_match\":%s," "\"owner_bitmap_member\":%s,\"worker_slot_exact\":%s," - "\"physical_role_matches\":%s,\"icache_pair_observed\":%s," - "\"prior_snapshot_larger\":%s}", + "\"physical_role_matches\":%s}", worker == 0 ? "" : ",\n", worker, physical_core_id, is_aic ? "aic" : "aiv", block_id, - lane, segments, window_started ? "true" : "false", window_stopped ? "true" : "false", + lane, shadow_read_segments, window_started ? "true" : "false", window_stopped ? "true" : "false", static_cast(result.pmu_total_cycles), result.pmu_vector_busy, result.pmu_cube_busy, result.pmu_scalar_busy, result.pmu_mte1_busy, result.pmu_mte2_busy, - result.pmu_mte3_busy, result.pmu_icache_requests, result.pmu_icache_misses, - result.pmu_fix_busy, static_cast(result.pmu_window_ticks), - static_cast(result.pmu_warm_total_cycles), - static_cast(result.pmu_warm_window_ticks), - result.pmu_warm_icache_requests, result.pmu_warm_icache_misses, + result.pmu_icache_requests, result.pmu_icache_misses, + result.pmu_build_variant, result.pmu_phase_id, result.pmu_phase_calls, + result.pmu_phase_begin_reads, result.pmu_phase_end_reads, + result.pmu_phase_icache_requests, result.pmu_phase_icache_misses, + phase_request_upper, phase_miss_upper, + result.pmu_shadow_icache_requests, result.pmu_shadow_icache_misses, + shadow_matches ? "true" : "false", shadow_bounded ? "true" : "false", + shadow_request_loss, shadow_miss_loss, boundaries_balanced ? "true" : "false", + result.pmu_phase_status, result.pmu_phase_status, status, status, trusted ? "true" : "false", (status & kStatusCoreIdValid) != 0 ? "true" : "false", (status & (kStatusCnt0Selector | kStatusCnt1Selector | kStatusCnt2Selector | @@ -1110,9 +1352,7 @@ bool ExportPmuJson( ? "true" : "false", owner_bitmap_member ? "true" : "false", worker_slot_exact ? "true" : "false", - physical_role_matches ? "true" : "false", - icache_pair_observed ? "true" : "false", - (status & kStatusPriorSnapshotLarger) != 0 ? "true" : "false" + physical_role_matches ? "true" : "false" ); } std::fputs("\n],\n\"summary\":{\"all\":", output); @@ -1229,6 +1469,21 @@ int main(int argc, char **argv) { ); return EXIT_FAILURE; } +#elif PA_BUILD_SUBMIT_PMU + // submit-pmu 是编译期固定 phase 的单轮诊断产物;host、kernel 与 owner + // 必须共同拒绝旧校准窗口和任何泳道/phase-profile 观察代码。 + if (pmu_options.mode != pa_scheduler::ccec_pmu::WindowMode::SubmitAll) { + std::fprintf(stderr, "The submit-pmu build requires --pmu-window submit-all.\n"); + return EXIT_FAILURE; + } + if (options.runs != 1 || options.trace_enabled || options.trace_atomics || + options.profile_phases || options.analyze_swimlane || !options.swimlane_json.empty()) { + std::fprintf( + stderr, + "submit-pmu requires one PMU-only run: --runs 1 --no-swimlane and no trace/profile options.\n" + ); + return EXIT_FAILURE; + } #endif if (!pmu_options.json_path.empty() && pmu_options.mode == pa_scheduler::ccec_pmu::WindowMode::Off) { diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index a59a387c05..929f77c7de 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -214,30 +214,8 @@ static __aicore__ __attribute__((noinline, used)) void pa_execute_real_winner_wo #endif } -#if PA_BUILD_SUBMIT_PMU && defined(PA_BUILD_AIC) -#define PA_ICACHE_TARGET_NAME pa_icache_target_aic -#define PA_ICACHE_MEASURE_NAME pa_icache_measure_aic -#define PA_ICACHE_THRASH_NAME pa_icache_thrash_aic -#elif PA_BUILD_SUBMIT_PMU && defined(PA_BUILD_AIV) -#define PA_ICACHE_TARGET_NAME pa_icache_target_aiv -#define PA_ICACHE_MEASURE_NAME pa_icache_measure_aiv -#define PA_ICACHE_THRASH_NAME pa_icache_thrash_aiv -#endif - #if PA_BUILD_SUBMIT_PMU -// cold/warm 两条路径调用同一个 8B 目标函数。入口按 128B I-cache line -// 对齐,构建脚本还会核对符号尺寸、对齐和 target -> harness -> thrash 布局。 -__aicore__ static __attribute__((noinline, used, aligned(128), section(".text.pa_icache_target"))) -void PA_ICACHE_TARGET_NAME() { - asm volatile( - ".rept 1\n" - "nop\n" - ".endr\n" - ); -} - -__aicore__ static __attribute__((noinline, used, aligned(128), section(".text.pa_icache_thrash"))) -void PA_ICACHE_THRASH_NAME(); +struct SubmitPmuContext; #endif struct CcecOps { @@ -321,12 +299,18 @@ struct CcecOps { } #if PA_BUILD_SUBMIT_PMU - __aicore__ static inline bool PmuWindowStart( + using PmuContext = SubmitPmuContext; + + __aicore__ static inline PmuContext PmuWindowStart( __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id ); + __aicore__ static inline void PmuPhaseBegin(PmuContext &context); + + __aicore__ static inline void PmuPhaseEnd(PmuContext &context); + __aicore__ static inline void PmuWindowStop( - __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id, bool started + __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id, PmuContext &context ); #else // swimlane 产物不携带 PMU 读寄存器或门控代码;公共调度器保留同一 hook @@ -392,11 +376,25 @@ struct PmuSnapshot { uint32_t status = 0; }; -struct IcachePairSnapshot { - PmuSnapshot cold; - PmuSnapshot warm; - uint64_t cold_window_ticks = 0; - uint64_t warm_window_ticks = 0; +struct IcacheShadowSnapshot { + uint32_t requests = 0; + uint32_t misses = 0; +}; + +struct SubmitPmuContext { + uint64_t reg_base = 0; + uint64_t shadow_requests = 0; + uint64_t shadow_misses = 0; + uint64_t phase_requests = 0; + uint64_t phase_misses = 0; + uint32_t selector_status = 0; + uint32_t phase_status = pa_scheduler::ccec_pmu::kPhaseStatusRequested; + uint32_t phase_calls = 0; + uint32_t begin_reads = 0; + uint32_t end_reads = 0; + bool started = false; + bool phase_armed = false; + bool boundary_error = false; }; template @@ -418,14 +416,13 @@ __aicore__ inline PmuSnapshot ReadObservedCounters(uint64_t reg_base) { pa_scheduler::ccec_pmu::kCnt3Offset>(reg_base); sample.mte2_busy = ReadPmuRegister(reg_base); - sample.mte3_busy = ReadPmuRegister(reg_base); + // submit-pmu 将 CNT5 留给 shadow I-cache miss。这里不能提前读取,否则 + // read-to-clear 会让随后的 shadow tail 漏计;MTE3 busy 在该诊断构建不可用。 + sample.mte3_busy = 0; sample.icache_requests = ReadPmuRegister(reg_base); sample.icache_misses = ReadPmuRegister(reg_base); - sample.fix_busy = ReadPmuRegister(reg_base); const uint64_t low = ReadPmuRegister(reg_base); const uint64_t high = ReadPmuRegistertotal_cycles += sample.total_cycles; - total->vector_busy += sample.vector_busy; - total->cube_busy += sample.cube_busy; - total->scalar_busy += sample.scalar_busy; - total->mte1_busy += sample.mte1_busy; - total->mte2_busy += sample.mte2_busy; - total->mte3_busy += sample.mte3_busy; - total->icache_requests += sample.icache_requests; - total->icache_misses += sample.icache_misses; - total->fix_busy += sample.fix_busy; -} - -// 禁止内联,确保 cold/warm 经过同一函数体、同一 PMU gate 和同一目标 -// callsite;否则编译器复制 harness 后,两条路径的取指状态不再是单变量实验。 -__aicore__ static __attribute__((noinline, used, aligned(128), section(".text.pa_icache_harness"))) -void PA_ICACHE_MEASURE_NAME( - uint64_t reg_base, bool warm, PmuSnapshot *total, uint64_t *window_ticks -) { - // 两条路径都先在窗口外执行 64 KiB capacity sweep。warm 只多一次窗外 - // target 预取,并且发生在 read-clear 之前,避免把预热 miss 计入 warm 窗口。 - bisheng::cce::metrics_prof_stop(); - PA_ICACHE_THRASH_NAME(); - if (warm) PA_ICACHE_TARGET_NAME(); - (void)ReadObservedCounters(reg_base); - - bisheng::cce::metrics_prof_start(); - const uint64_t begin = static_cast(get_sys_cnt()); - PA_ICACHE_TARGET_NAME(); - const uint64_t end = static_cast(get_sys_cnt()); - bisheng::cce::metrics_prof_stop(); - - AccumulateObserved(ReadObservedCounters(reg_base), total); - *window_ticks += end - begin; -} - -// DAV_3510 scalar I-cache 最大为 32 KiB。thrash 放在 target/harness 之后, -// 顺序执行 64 KiB 指令覆盖所有 set 多轮;返回低地址 harness 时不向前预取 target。 -__aicore__ static void PA_ICACHE_THRASH_NAME() { - asm volatile( - ".rept 16384\n" - "nop\n" - ".endr\n" - ); -} - -__aicore__ inline void RunIcachePhase( - uint64_t reg_base, bool warm, uint32_t trials, PmuSnapshot *total, uint64_t *window_ticks -) { - // 每个 phase 先丢弃一次同分支训练样本,避免逐 trial 交替分支历史把 - // false 路径的目标行提前取回。 - PmuSnapshot discarded; - uint64_t discarded_ticks = 0; - PA_ICACHE_MEASURE_NAME(reg_base, warm, &discarded, &discarded_ticks); - for (uint32_t trial = 0; trial < trials; ++trial) { - PA_ICACHE_MEASURE_NAME(reg_base, warm, total, window_ticks); - } -} - -__aicore__ inline IcachePairSnapshot RunSingleIcacheProbe( - uint64_t reg_base, uint32_t trials, uint32_t worker_id -) { - IcachePairSnapshot pair; - // 每个 role 内各一半 worker 使用 cold-first/warm-first,抵消两个 phase - // 的固定时间顺序;phase 内保持相同分支历史。 - if ((worker_id & 1U) == 0U) { - RunIcachePhase(reg_base, false, trials, &pair.cold, &pair.cold_window_ticks); - RunIcachePhase(reg_base, true, trials, &pair.warm, &pair.warm_window_ticks); - } else { - RunIcachePhase(reg_base, true, trials, &pair.warm, &pair.warm_window_ticks); - RunIcachePhase(reg_base, false, trials, &pair.cold, &pair.cold_window_ticks); - } - return pair; +__aicore__ inline IcacheShadowSnapshot ReadShadowCounters(uint64_t reg_base) { + IcacheShadowSnapshot sample; + sample.requests = ReadPmuRegister(reg_base); + sample.misses = ReadPmuRegister(reg_base); + return sample; } struct PmuRegisterContext { uint64_t reg_base = 0; uint32_t status = 0; + bool shadow_selectors = false; }; __aicore__ inline PmuRegisterContext ResolvePmuRegisters(__gm__ pa_scheduler::SchedulerState *state) { @@ -532,8 +464,8 @@ __aicore__ inline PmuRegisterContext ResolvePmuRegisters(__gm__ pa_scheduler::Sc if (context.reg_base == 0) return context; context.status |= kStatusRegMapped; - // selector 与 standalone owner 的 A5 PIPE_UTILIZATION 事件表逐项核对,避免 - // 配置或 ABI 错位时仍把 CNT0..8 的数值按 vector/scalar/I-cache 名称导出。 + // selector 与同 phase 目录内的 owner 逐项核对。CNT5/CNT8 分别重复 + // CNT7/CNT6;CNT9 保持正式 PIPE_UTIL 的 unused(0) 口径。 if (ReadPmuRegister(context.reg_base) == kVectorBusyEvent) context.status |= kStatusCnt0Selector; if (ReadPmuRegister(context.reg_base) == kCubeBusyEvent) @@ -544,14 +476,21 @@ __aicore__ inline PmuRegisterContext ResolvePmuRegisters(__gm__ pa_scheduler::Sc context.status |= kStatusCnt3Selector; if (ReadPmuRegister(context.reg_base) == kMte2BusyEvent) context.status |= kStatusCnt4Selector; - if (ReadPmuRegister(context.reg_base) == kMte3BusyEvent) + const bool cnt5_ok = + ReadPmuRegister(context.reg_base) == kIcacheMissEvent; + if (cnt5_ok) context.status |= kStatusCnt5Selector; if (ReadPmuRegister(context.reg_base) == kIcacheRequestEvent) context.status |= kStatusCnt6Selector; if (ReadPmuRegister(context.reg_base) == kIcacheMissEvent) context.status |= kStatusCnt7Selector; - if (ReadPmuRegister(context.reg_base) == kFixBusyEvent) + const bool cnt8_ok = + ReadPmuRegister(context.reg_base) == kIcacheRequestEvent; + const bool cnt9_unused = + ReadPmuRegister(context.reg_base) == 0U; + if (cnt8_ok) context.status |= kStatusCnt8Selector; + context.shadow_selectors = cnt5_ok && cnt8_ok && cnt9_unused; return context; } @@ -570,108 +509,146 @@ __aicore__ inline void PublishPmuSnapshot( CcecOps::Publish(&result.pmu_mte1_busy, sample.mte1_busy); CcecOps::Publish(&result.pmu_mte2_busy, sample.mte2_busy); CcecOps::Publish(&result.pmu_mte3_busy, sample.mte3_busy); - CcecOps::Publish(&result.pmu_fix_busy, sample.fix_busy); - // 非 icache-single 模式也显式清零配对 sidecar,避免同一 device allocation - // 被后续 run 复用时把陈旧 warm 数据误当成本轮结果。 + // CNT8 已改作 shadow request,submit-pmu 不再发布 fix-busy。 + CcecOps::Publish(&result.pmu_fix_busy, static_cast(0)); CcecOps::Publish(&result.pmu_window_ticks, static_cast(0)); CcecOps::Publish(&result.pmu_warm_total_cycles, static_cast(0)); CcecOps::Publish(&result.pmu_warm_window_ticks, static_cast(0)); - CcecOps::Publish(&result.pmu_warm_icache_requests, static_cast(0)); - CcecOps::Publish(&result.pmu_warm_icache_misses, static_cast(0)); } -__aicore__ inline void PublishIcachePair( - __gm__ pa_scheduler::WorkerResult &result, const IcachePairSnapshot &pair +__aicore__ inline bool FitsUint32(uint64_t value) { + return value <= 0xffffffffULL; +} + +__aicore__ inline void PublishSubmitPmuContext( + __gm__ pa_scheduler::WorkerResult &result, const SubmitPmuContext &context ) { - PublishPmuSnapshot(result, pair.cold); - CcecOps::Publish(&result.pmu_window_ticks, pair.cold_window_ticks); - CcecOps::Publish(&result.pmu_warm_total_cycles, pair.warm.total_cycles); - CcecOps::Publish(&result.pmu_warm_window_ticks, pair.warm_window_ticks); - CcecOps::Publish(&result.pmu_warm_icache_requests, pair.warm.icache_requests); - CcecOps::Publish(&result.pmu_warm_icache_misses, pair.warm.icache_misses); + CcecOps::Publish(&result.pmu_build_variant, pa_scheduler::kBuildVariantSubmitPmu); + CcecOps::Publish( + &result.pmu_phase_id, + static_cast(pa_scheduler::kCompiledSubmitPmuPhase) + ); + CcecOps::Publish(&result.pmu_phase_calls, context.phase_calls); + CcecOps::Publish(&result.pmu_phase_status, context.phase_status); + CcecOps::Publish(&result.pmu_phase_begin_reads, context.begin_reads); + CcecOps::Publish(&result.pmu_phase_end_reads, context.end_reads); + CcecOps::Publish(&result.pmu_phase_icache_requests, static_cast(context.phase_requests)); + CcecOps::Publish(&result.pmu_phase_icache_misses, static_cast(context.phase_misses)); + CcecOps::Publish(&result.pmu_shadow_icache_requests, static_cast(context.shadow_requests)); + CcecOps::Publish(&result.pmu_shadow_icache_misses, static_cast(context.shadow_misses)); } -__aicore__ inline bool CcecOps::PmuWindowStart( +__aicore__ inline CcecOps::PmuContext CcecOps::PmuWindowStart( __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id ) { using namespace pa_scheduler::ccec_pmu; (void)worker_id; + SubmitPmuContext context; const WindowMode mode = static_cast(state->config.reserved[kConfigMode]); - if (mode != WindowMode::SubmitAll) return false; - const PmuRegisterContext context = ResolvePmuRegisters(state); - if (context.reg_base == 0) return false; + if (mode != WindowMode::SubmitAll) return context; // Main AICPU owner 已在 launch 前配置并开启计数;先 stop + snapshot/read-clear, - // 再从本 worker 的首个 orchestration 动作开始独立累计。 + // 再解析 selector,避免这些 ld_dev 污染完整 Submit 窗口。 bisheng::cce::metrics_prof_stop(); + const PmuRegisterContext registers = ResolvePmuRegisters(state); + context.reg_base = registers.reg_base; + context.selector_status = registers.status; + if (registers.shadow_selectors) { + context.phase_status |= kPhaseStatusShadowSelectors; + } + if (context.reg_base == 0) return context; (void)ReadObservedCounters(context.reg_base); + (void)ReadShadowCounters(context.reg_base); bisheng::cce::metrics_prof_start(); - return true; + context.started = true; + context.phase_status |= kPhaseStatusWindowStarted; + return context; +} + +__aicore__ inline void CcecOps::PmuPhaseBegin(PmuContext &context) { + if (!context.started || context.reg_base == 0 || context.phase_armed) { + context.boundary_error = true; + return; + } + // counter 在运行中读取即清零;begin 之前的片段只进入 shadow whole。 + const IcacheShadowSnapshot sample = ReadShadowCounters(context.reg_base); + context.shadow_requests += sample.requests; + context.shadow_misses += sample.misses; + ++context.begin_reads; + context.phase_armed = true; +} + +__aicore__ inline void CcecOps::PmuPhaseEnd(PmuContext &context) { + if (!context.started || context.reg_base == 0 || !context.phase_armed) { + context.boundary_error = true; + return; + } + // end 读出的片段同时属于完整 shadow 重建与被选中的局部阶段。 + const IcacheShadowSnapshot sample = ReadShadowCounters(context.reg_base); + context.shadow_requests += sample.requests; + context.shadow_misses += sample.misses; + context.phase_requests += sample.requests; + context.phase_misses += sample.misses; + ++context.end_reads; + ++context.phase_calls; + context.phase_armed = false; } __aicore__ inline void CcecOps::PmuWindowStop( - __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id, bool started + __gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id, PmuContext &context ) { using namespace pa_scheduler::ccec_pmu; - if (!started) return; - // 先冻结 gate,再读取 selector 和 counter。若先 ResolvePmuRegisters,九次 - // selector ld_dev 会被误计入本 worker 的 Submit 窗口。 - bisheng::cce::metrics_prof_stop(); - const PmuRegisterContext context = ResolvePmuRegisters(state); + __gm__ pa_scheduler::WorkerResult &result = state->results[worker_id]; PmuSnapshot sample; - sample.status = context.status; - if (context.reg_base != 0) { + sample.status = context.selector_status; + if (context.started && context.reg_base != 0) { + // gate 只在整个 Submit 前后各操作一次。停止后先读从未中途清零的 + // primary counter(不含 shadow CNT5)之后,再读取 CNT8/CNT5 tail + // 完成软件重建。 + bisheng::cce::metrics_prof_stop(); sample = ReadObservedCounters(context.reg_base); - sample.status = context.status | kStatusWindowStarted | kStatusWindowStopped; + const IcacheShadowSnapshot tail = ReadShadowCounters(context.reg_base); + context.shadow_requests += tail.requests; + context.shadow_misses += tail.misses; + sample.status = context.selector_status | kStatusWindowStarted | kStatusWindowStopped; + context.phase_status |= kPhaseStatusWindowStopped; if (sample.total_cycles != 0) sample.status |= kStatusTotalNonzero; } - PublishPmuSnapshot(state->results[worker_id], sample); -} -__aicore__ inline void RunPmuProbe(__gm__ pa_scheduler::SchedulerState *state, uint32_t worker_id) { - using namespace pa_scheduler::ccec_pmu; - __gm__ pa_scheduler::WorkerResult &result = state->results[worker_id]; - const WindowMode mode = static_cast(state->config.reserved[kConfigMode]); - // SubmitAll 已在公共调度器 hook 内完成 start/stop/read/publish;此处保留 - // empty/scalar/scalar-double/icache-single 校准,和正式窗口可独立复验。 - if (mode == WindowMode::SubmitAll) return; + if (context.shadow_requests == sample.icache_requests) + context.phase_status |= kPhaseStatusShadowRequestsMatch; + if (context.shadow_misses == sample.icache_misses) + context.phase_status |= kPhaseStatusShadowMissesMatch; + if (!context.boundary_error && !context.phase_armed && + context.begin_reads == context.end_reads && context.end_reads == context.phase_calls) + context.phase_status |= kPhaseStatusBoundariesBalanced; + // 两个 shadow counter 是顺序 ld_dev,并非同一时刻的原子快照;局部 + // phase 的 miss/request 边界会错开数条指令,故不能硬性要求局部 + // miss<=request。A5 上运行中 read-to-clear 还会与同周期事件递增竞争, + // shadow 允许小于未中途读取的 primary,但绝不能反向超过它。primary- + // shadow 是该次采集可直接给出的局部分段误差包络,而不是要静默吞掉的差值。 + if (context.phase_requests <= context.shadow_requests && + context.phase_misses <= context.shadow_misses && + context.shadow_misses <= context.shadow_requests && + context.shadow_requests <= sample.icache_requests && + context.shadow_misses <= sample.icache_misses) + context.phase_status |= kPhaseStatusValuesOrdered; + if (FitsUint32(context.shadow_requests) && FitsUint32(context.shadow_misses) && + FitsUint32(context.phase_requests) && FitsUint32(context.phase_misses)) + context.phase_status |= kPhaseStatusUint32Fit; + + const bool none_shape = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None && + context.phase_calls == 0 && context.begin_reads == 0 && context.end_reads == 0 && + context.phase_requests == 0 && context.phase_misses == 0; + const bool claim_shape = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::Claim && + context.phase_calls == state->config.batches * pa_scheduler::kTasksPerBatch && + context.begin_reads == context.phase_calls && + context.end_reads == context.phase_calls; + if (none_shape || claim_shape) context.phase_status |= kPhaseStatusPhaseShape; - PmuSnapshot sample; - if (mode != WindowMode::Off) { - const PmuRegisterContext context = ResolvePmuRegisters(state); - sample.status = context.status; - if (context.reg_base != 0) { - bisheng::cce::metrics_prof_stop(); - const PmuSnapshot prior = ReadObservedCounters(context.reg_base); - if (mode == WindowMode::IcacheSingle) { - IcachePairSnapshot pair = RunSingleIcacheProbe( - context.reg_base, state->config.reserved[kConfigIcacheTrials], worker_id - ); - pair.cold.status = context.status | kStatusWindowStarted | kStatusWindowStopped | - kStatusIcachePairObserved; - if (pair.cold.total_cycles != 0U) pair.cold.status |= kStatusTotalNonzero; - if (pair.cold.total_cycles < prior.total_cycles) { - pair.cold.status |= kStatusPriorSnapshotLarger; - } - PublishIcachePair(result, pair); - return; - } - bisheng::cce::metrics_prof_start(); - if (mode == WindowMode::Scalar || mode == WindowMode::ScalarDouble) { - RuntimeNop(state->config.reserved[kConfigScalarNops]); - } - if (mode == WindowMode::ScalarDouble) { - bisheng::cce::metrics_prof_stop(); - bisheng::cce::metrics_prof_start(); - RuntimeNop(state->config.reserved[kConfigScalarNops]); - } - bisheng::cce::metrics_prof_stop(); - sample = ReadObservedCounters(context.reg_base); - sample.status = context.status | kStatusWindowStarted | kStatusWindowStopped; - if (sample.total_cycles != 0) sample.status |= kStatusTotalNonzero; - if (sample.total_cycles < prior.total_cycles) sample.status |= kStatusPriorSnapshotLarger; - } - } PublishPmuSnapshot(result, sample); + PublishSubmitPmuContext(result, context); } #endif // PA_BUILD_SUBMIT_PMU @@ -685,9 +662,6 @@ extern "C" __global__ __aicore__ void pa_scheduler_0_mix_aic(__gm__ pa_scheduler // 32 个物理 block 的 AIC 直接使用 block_idx,形成连续 worker 0..31。 const uint32_t worker_id = static_cast(get_block_idx()); pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aic); -#if PA_BUILD_SUBMIT_PMU - RunPmuProbe(state, worker_id); -#endif } #elif defined(PA_BUILD_AIV) PTO_SYNCALL_MIX_AIC_KERNEL_META(pa_scheduler_0_mix_aiv, 1, 2); @@ -697,9 +671,6 @@ extern "C" __global__ __aicore__ void pa_scheduler_0_mix_aiv(__gm__ pa_scheduler const uint32_t vector_id = static_cast(get_block_idx() * get_subblockdim() + get_subblockid()); const uint32_t worker_id = pa_scheduler::kAicWorkers + vector_id; pa_scheduler::RunScheduler(state, worker_id, pa_scheduler::CoreRole::Aiv); -#if PA_BUILD_SUBMIT_PMU - RunPmuProbe(state, worker_id); -#endif } #else #error "Compile with PA_BUILD_AIC or PA_BUILD_AIV" diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_aicpu.cpp b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_aicpu.cpp index 31237bfc12..137dc71d71 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_aicpu.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_aicpu.cpp @@ -185,8 +185,8 @@ void SaveOne(PmuOwnerControl *control, uint64_t base, uint32_t index) void ConfigureOne(uint64_t base) { - // 先冻结 PMU,再写完整 10 槽 selector。CNT9=0 是正式配置中的未用策略, - // 不是遗漏;它和另外九槽一样会在 Restore 时恢复旧值。 + // 先冻结 PMU,再写完整 10 槽 selector。submit-pmu 的 CNT8/CNT5 是 + // request/miss shadow;包括 unused CNT9 在内的所有槽都会在 Restore 恢复。 WriteMmio(base, kCtrl0Offset, 0U); WriteMmio(base, kCtrl1Offset, 0U); for (uint32_t counter = 0U; counter < pa_scheduler::pmu_owner::kPmuCounterCount; ++counter) { diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_control.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_control.h index f8a7c0d371..eb17c4c1b0 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_control.h +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_owner_control.h @@ -38,8 +38,14 @@ static_assert(kExpectedAicCount + kExpectedAivCount == kExpectedSubcoreCount, "a static_assert(kAicPerDie * 2U == 36U, "physical AIC topology changed"); static_assert(kSubcoresPerDie * 2U == kPhysicalSubcoreCount, "physical subcore topology changed"); -// A5 PIPE_UTILIZATION 的正式 counter 槽位布局。CNT9 的事件号为 0,明确 -// 表示本轮不消费该槽;仍保存并恢复其旧 selector,也仍读一次 counter 清零。 +// A5 PIPE_UTILIZATION 的正式 counter 槽位布局。submit-pmu 用 CNT8/CNT5 +// 重复配置 I-cache request/miss,作为允许中途 read-to-clear 的 shadow; +// CNT6/7 始终不在阶段边界读取,保留为完整 Submit 的权威对照。 +// +// 不能把 miss 放进 CNT9:A5 b1 实测表明 CNT9 selector 虽能回读 0x35, +// 但计数恒为 0;正式 PIPE_UTIL 表也把 CNT9 标成 unused。0x35 在独立 +// I-cache 微基准的低位 counter 已验证可计数,因此诊断构建让 CNT5 承担 +// shadow miss,并明确放弃该构建中的 MTE3 busy。 constexpr uint32_t kPmuCounterCount = 10U; constexpr uint32_t kConfiguredSelectors[kPmuCounterCount] = { 0x501U, // CNT0: vector busy @@ -47,11 +53,19 @@ constexpr uint32_t kConfiguredSelectors[kPmuCounterCount] = { 0x001U, // CNT2: scalar busy 0x701U, // CNT3: MTE1 busy 0x202U, // CNT4: MTE2 busy +#if PA_BUILD_SUBMIT_PMU + 0x035U, // CNT5: shadow I-cache miss + 0x034U, // CNT6: I-cache request(完整 Submit) + 0x035U, // CNT7: I-cache miss(完整 Submit) + 0x034U, // CNT8: shadow I-cache request + 0x000U, // CNT9: A5 PIPE_UTIL 正式未使用 +#else 0x203U, // CNT5: MTE3 busy 0x034U, // CNT6: I-cache request 0x035U, // CNT7: I-cache miss 0x714U, // CNT8: fix-pipe busy 0x000U, // CNT9: 未使用 +#endif }; constexpr int32_t kStatusPending = 0x7fffffff; diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h index 85cbe3cc6b..ba1435ec58 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h @@ -14,6 +14,8 @@ #include +#include "../common/pa_model.h" + namespace pa_scheduler::ccec_pmu { // Empty/Scalar/ScalarDouble 在调度结束后校准门控底噪和 scalar 正向响应; @@ -78,6 +80,7 @@ constexpr uint32_t kCnt5Offset = 0x4238U; constexpr uint32_t kCnt6Offset = 0x4240U; constexpr uint32_t kCnt7Offset = 0x4248U; constexpr uint32_t kCnt8Offset = 0x4250U; +constexpr uint32_t kCnt9Offset = 0x4254U; constexpr uint32_t kTotalLowOffset = 0x4260U; constexpr uint32_t kTotalHighOffset = 0x4264U; constexpr uint32_t kCnt2SelectorOffset = 0x2508U; @@ -89,6 +92,7 @@ constexpr uint32_t kCnt5SelectorOffset = 0x2514U; constexpr uint32_t kCnt6SelectorOffset = 0x2518U; constexpr uint32_t kCnt7SelectorOffset = 0x251cU; constexpr uint32_t kCnt8SelectorOffset = 0x2520U; +constexpr uint32_t kCnt9SelectorOffset = 0x2524U; // pmu_status 的 bits16..27 保存 get_coreid();bits28..31 留给不参与 core id // 解码的模式诊断。其余低位描述本条记录是否可信。 @@ -119,6 +123,39 @@ constexpr uint32_t kStatusRequired = kStatusRequested | kStatusRegMapped | kStat constexpr uint32_t kStatusCoreIdShift = 16; constexpr uint32_t kStatusCoreIdMask = 0x0fffU; +// pmu_phase_status 独立于旧 pmu_status,避免与其中的物理 core-id 位域 +// 冲突。bits4/5 只记录 shadow 是否恰好等于 primary:phase=none 没有 +// 运行中 read-to-clear,host 会要求两位都成立;局部 phase 会在计数仍开启时 +// 读取 shadow,A5 实测存在同周期递增与读清竞争,因此不能把“逐次严格相等” +// 作为可信记录的共同必选位。局部 phase 的方向和误差包络由 host/raw 独立校验。 +constexpr uint32_t kPhaseStatusRequested = 1U << 0; +constexpr uint32_t kPhaseStatusShadowSelectors = 1U << 1; +constexpr uint32_t kPhaseStatusWindowStarted = 1U << 2; +constexpr uint32_t kPhaseStatusWindowStopped = 1U << 3; +constexpr uint32_t kPhaseStatusShadowRequestsMatch = 1U << 4; +constexpr uint32_t kPhaseStatusShadowMissesMatch = 1U << 5; +constexpr uint32_t kPhaseStatusBoundariesBalanced = 1U << 6; +constexpr uint32_t kPhaseStatusValuesOrdered = 1U << 7; +constexpr uint32_t kPhaseStatusUint32Fit = 1U << 8; +constexpr uint32_t kPhaseStatusPhaseShape = 1U << 9; +constexpr uint32_t kPhaseStatusRequired = + kPhaseStatusRequested | kPhaseStatusShadowSelectors | + kPhaseStatusWindowStarted | kPhaseStatusWindowStopped | + kPhaseStatusBoundariesBalanced | kPhaseStatusValuesOrdered | + kPhaseStatusUint32Fit | kPhaseStatusPhaseShape; + +inline const char *SubmitPmuPhaseName(SubmitPmuPhase phase) { + switch (phase) { + case SubmitPmuPhase::None: + return "none"; + case SubmitPmuPhase::Claim: + return "claim"; + case SubmitPmuPhase::Count: + break; + } + return "invalid"; +} + inline uint64_t PackPointer(const uint32_t *words) { return static_cast(words[kConfigRegTableLow]) | (static_cast(words[kConfigRegTableHigh]) << 32); diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index 5c084f4eea..45320aad65 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -182,6 +182,29 @@ enum class ProfilePhase : uint32_t { Count = 12, }; +// submit-pmu 每个 ELF 只编译一个局部归因阶段。none 不做中途 counter +// 读取,是完整 Submit 的正式基线;claim 是首个连续、无提前返回的验证阶段。 +// 后续阶段只能在各自边界和闭环经过 A5 验证后向枚举尾部追加。 +enum class SubmitPmuPhase : uint32_t { + None = 0, + Claim = 1, + Count = 2, +}; + +#ifndef PA_SUBMIT_PMU_PHASE_ID +#define PA_SUBMIT_PMU_PHASE_ID 0 +#endif + +constexpr SubmitPmuPhase kCompiledSubmitPmuPhase = + static_cast(PA_SUBMIT_PMU_PHASE_ID); +constexpr uint32_t kBuildVariantSwimlane = 1U; +constexpr uint32_t kBuildVariantSubmitPmu = 2U; +static_assert( + PA_SUBMIT_PMU_PHASE_ID >= 0 && + PA_SUBMIT_PMU_PHASE_ID < static_cast(SubmitPmuPhase::Count), + "invalid compiled submit-pmu phase" +); + struct NopCounts { uint32_t qk; uint32_t sf; @@ -727,8 +750,14 @@ struct alignas(64) WorkerResult { uint64_t pmu_window_ticks; uint64_t pmu_warm_total_cycles; uint64_t pmu_warm_window_ticks; - uint32_t pmu_warm_icache_requests; - uint32_t pmu_warm_icache_misses; + union { + uint32_t pmu_warm_icache_requests; + uint32_t pmu_phase_begin_reads; + }; + union { + uint32_t pmu_warm_icache_misses; + uint32_t pmu_phase_end_reads; + }; // PIPE_UTILIZATION 已同时配置 CNT0/1/3/4/5/8;与上面的 scalar/I-cache // 一样只保存每核原始累计值,AIC/AIV 汇总与比率统一在 host sidecar 中计算。 @@ -737,8 +766,23 @@ struct alignas(64) WorkerResult { uint32_t pmu_cube_busy; uint32_t pmu_mte1_busy; uint32_t pmu_mte2_busy; + // swimlane ABI 保留该槽;submit-pmu 将物理 CNT5 改作 shadow miss, + // 因而显式发布 0,并在 schema v4 标记 mte3_busy 不可用。 uint32_t pmu_mte3_busy; uint32_t pmu_fix_busy; + + // 复用 WorkerResult 原有的 32B cache-line 尾洞,不扩大 832B stride。 + // CNT6/7 是从不中途读取的权威整窗,CNT8/CNT5 是 read-to-clear shadow; + // none 在 stop 后要求逐核精确相等;运行中切片的 phase 只允许 shadow + // 单向小于 primary,并显式导出差值形成局部观测区间。 + uint32_t pmu_build_variant; + uint32_t pmu_phase_id; + uint32_t pmu_phase_calls; + uint32_t pmu_phase_status; + uint32_t pmu_phase_icache_requests; + uint32_t pmu_phase_icache_misses; + uint32_t pmu_shadow_icache_requests; + uint32_t pmu_shadow_icache_misses; }; // WorkerResult 是 standalone 尾部的诊断 sidecar,不属于真实 DistCore ABI;按 // cache line 隔离后,各 worker 发布统计不会相互覆盖或污染被测共享状态。 @@ -749,6 +793,8 @@ static_assert(offsetof(WorkerResult, fanin_not_ready_loads) == 704, "WorkerResul static_assert(offsetof(WorkerResult, atomic_trace_calls) == 736, "WorkerResult atomic trace offset mismatch"); static_assert(offsetof(WorkerResult, pmu_window_ticks) == 744, "WorkerResult PMU timing offset mismatch"); static_assert(offsetof(WorkerResult, pmu_vector_busy) == 776, "WorkerResult extended PMU offset mismatch"); +static_assert(offsetof(WorkerResult, pmu_build_variant) == 800, "WorkerResult submit-PMU offset mismatch"); +static_assert(offsetof(WorkerResult, pmu_shadow_icache_misses) == 828, "WorkerResult submit-PMU tail mismatch"); // 从 cube_cursor 到 workers 结束保留关键字段 offset、DistCore ABI 和生产总字节跨度, // 并非字段级完整镜像。RunConfig、输入 context_lens 与校验结果追加在该跨度之后, diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index e6c1ff2eee..f0ddc1d7b5 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -31,8 +31,37 @@ struct LocalStats { TraceContext trace; }; +// submit-pmu 的 phase 在编译期固定;非诊断构建完全不引用 Ops 的 phase +// 接口。这样公共调度代码保持一份,swimlane/CPU/AscendC 也不会多出运行时分支。 +template +PA_DEVICE void BeginSubmitPmuPhase(PmuContext &context) { +#if PA_BUILD_SUBMIT_PMU + if constexpr (kCompiledSubmitPmuPhase == Phase) { + Ops::PmuPhaseBegin(context); + } +#else + (void)context; +#endif +} + +template +PA_DEVICE void EndSubmitPmuPhase(PmuContext &context) { +#if PA_BUILD_SUBMIT_PMU + if constexpr (kCompiledSubmitPmuPhase == Phase) { + Ops::PmuPhaseEnd(context); + } +#else + (void)context; +#endif +} + template PA_DEVICE uint64_t TraceTimestamp(TraceContext &trace, WorkerResult &result) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + return 0; +#else (void)result; // 对齐真实 FDWIC 的 TRACE_SPAN_BEGIN/END:取一次时间后立即以同一 // cycle 关闭活跃 PollBatch。不能在 WriteTrace 中统一关闭,否则直接 @@ -40,6 +69,7 @@ PA_DEVICE uint64_t TraceTimestamp(TraceContext &trace, WorkerResult &result) { const uint64_t cycle = Ops::Now(); AtomicPollBoundaryAt(trace, cycle); return cycle; +#endif } PA_DEVICE uint32_t KindIndex(TaskKind kind) { return static_cast(kind); } @@ -508,17 +538,23 @@ PA_DEVICE bool BuildWinner( return true; } -template +template PA_DEVICE bool SubmitTask( PA_GM SchedulerState *state, PA_GM WorkerState &worker, uint32_t task_count, TaskKind kind, - const TaskArgs &args, SubmitContext &context, LocalStats &stats + const TaskArgs &args, SubmitContext &context, LocalStats &stats, PmuContext &pmu_context ) { // 每个 worker 都完整回放相同 task stream。主流程为:EfDrain -> materialize -> TensorMap retire // -> Claim -> winner 收集 fanin -> 全员 register -> winner Build / loser Replay。Alloc 在 Claim 前 register, // 且 winner 不入 kernel slot,而是在 heap guard 后直接发布完成。 BeginSubmit(worker, args, context); const uint32_t task_id = static_cast(context.task_id); +#if PA_BUILD_SUBMIT_PMU + // PMU-only ELF 只保留首/末 Submit 的全局时间边界,不再为 1280 次调用 + // 各执行两条 trace-only SYS_CNT。 + const uint64_t submit_begin = task_id == 0 ? Ops::Now() : 0; +#else const uint64_t submit_begin = TraceTimestamp(stats.trace, stats.result); +#endif if (task_id == 0) { stats.result.submit_begin = submit_begin; } @@ -571,7 +607,9 @@ PA_DEVICE bool SubmitTask( ); const uint64_t claim_begin = TraceTimestamp(stats.trace, stats.result); + BeginSubmitPmuPhase(pmu_context); const ClaimOutcome claim = Claim(state, worker, task_id, kind, stats); + EndSubmitPmuPhase(pmu_context); winner = claim.won; context.won = winner; context.kernel_id = claim.function_id; @@ -602,7 +640,9 @@ PA_DEVICE bool SubmitTask( } } else { const uint64_t claim_begin = TraceTimestamp(stats.trace, stats.result); + BeginSubmitPmuPhase(pmu_context); const ClaimOutcome claim = Claim(state, worker, task_id, kind, stats); + EndSubmitPmuPhase(pmu_context); winner = claim.won; function_id = claim.function_id; context.won = winner; @@ -658,7 +698,11 @@ PA_DEVICE bool SubmitTask( } ++stats.result.submits; +#if PA_BUILD_SUBMIT_PMU + const uint64_t submit_end = task_id + 1 == task_count ? Ops::Now() : 0; +#else const uint64_t submit_end = TraceTimestamp(stats.trace, stats.result); +#endif WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Submit, ProfilePhase::Submit, submit_begin, submit_end, winner ? 1U : 0U, kind == TaskKind::Alloc ? 1U : 0U @@ -779,6 +823,11 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, stats.result.worker_id = worker_id; stats.result.role = static_cast(role); stats.result.checksum = 0xcbf29ce484222325ULL ^ worker_id; + // 该 invalidate 原先藏在 AttachTrace 中;它同时保护 PMU mode/register + // table 与 winner workload,必须在两个构建中都执行。 + Ops::InvalidateRegion( + &state->config, sizeof(state->config) + sizeof(state->winner_workload) + ); stats.trace = AttachTrace(state, worker, worker_id); // 96 个参与者全部完成本地状态初始化后再进入 task 0,主要用于压低启动偏斜对 @@ -815,7 +864,9 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, // 窗口覆盖从首个参数构造到末次 Submit 返回,与全局“首 Submit.begin~末 Submit.end” // 口径接近但不相同,host sidecar 必须按 per-worker 累计解释。 ResetTraceLap(stats.trace, stats.result, worker); - const bool pmu_window_started = Ops::PmuWindowStart(state, worker_id); + // lap 重置属于泳道观察自身,不应污染 PMU-only 的 Submit 取数;窗口从 + // orchestration 初始化(即首批参数构造)前一条边界开始。 + auto pmu_context = Ops::PmuWindowStart(state, worker_id); InitPaOrchestration(orchestration, batches, &state->context_lens[0]); for (uint32_t batch = 0; batch < batches; ++batch) { BuildAllocArgs(orchestration, args, batch); @@ -823,7 +874,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, stats.result.views_created += 2; stats.result.tensor_args_added += 3; if (!SubmitTask( - state, worker, task_count, TaskKind::Alloc, args, context, stats + state, worker, task_count, TaskKind::Alloc, args, context, stats, pmu_context )) { break; } @@ -835,7 +886,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, stats.result.tensor_args_added += 4; stats.result.scalar_args_added += 2; if (!SubmitTask( - state, worker, task_count, TaskKind::Qk, args, context, stats + state, worker, task_count, TaskKind::Qk, args, context, stats, pmu_context )) { break; } @@ -847,7 +898,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, stats.result.tensor_args_added += 4; stats.result.scalar_args_added += 3; if (!SubmitTask( - state, worker, task_count, TaskKind::Sf, args, context, stats + state, worker, task_count, TaskKind::Sf, args, context, stats, pmu_context )) { break; } @@ -858,7 +909,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, stats.result.tensor_args_added += 4; stats.result.scalar_args_added += 2; if (!SubmitTask( - state, worker, task_count, TaskKind::Pv, args, context, stats + state, worker, task_count, TaskKind::Pv, args, context, stats, pmu_context )) { break; } @@ -869,12 +920,12 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, stats.result.tensor_args_added += 7; stats.result.scalar_args_added += 2; if (!SubmitTask( - state, worker, task_count, TaskKind::Up, args, context, stats + state, worker, task_count, TaskKind::Up, args, context, stats, pmu_context )) { break; } } - Ops::PmuWindowStop(state, worker_id, pmu_window_started); + Ops::PmuWindowStop(state, worker_id, pmu_context); } // replay_done 表示所有 worker 已退出回放循环(成功路径即完整提交);之后仍需 drain 到本核 slot 为空。 @@ -902,6 +953,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, } AtomicPollRegionEnd(stats.trace, stats.result, final_poll_region); +#if !PA_BUILD_SUBMIT_PMU if (stats.trace.atomics_enabled) { // 两条基线都放在最终 drain 之后。第一条量连续 // SYS_CNT,第二条量返回依赖钩子的固定成本;它们只描述计时底噪,不能 @@ -923,6 +975,7 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, (Ops::kAtomicReturnReadyObserved ? kClockAtomicDependencyApplied : 0U) ); } +#endif // PA writes swimlane records through the ordinary GM cache and explicitly // cleans each worker's record range before the kernel finishes. diff --git a/tests/atomic_probe/pa_scheduler/common/pa_trace.h b/tests/atomic_probe/pa_scheduler/common/pa_trace.h index 511ec8eacd..c70a4e681b 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_trace.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_trace.h @@ -121,8 +121,8 @@ PA_DEVICE uint32_t TraceAtomicSiteMask(AtomicSite site) { return 1U << static_cast(site); } -// Attach 只缓存本 worker 的 header 状态、分区首址和物理 lane 信息。配置先做 -// cache invalidate,确保 A5 worker 看到 host 在 launch 前写入的 trace 开关、地址与 winner 负载配置。 +// Attach 只缓存本 worker 的 header 状态、分区首址和物理 lane 信息。控制区 +// cache invalidate 在公共调度入口完成,不能随 submit-pmu 编译掉泳道而消失。 template PA_DEVICE TraceContext AttachTrace( PA_GM SchedulerState *state, PA_GM const WorkerState &worker, uint32_t worker_id @@ -131,9 +131,12 @@ PA_DEVICE TraceContext AttachTrace( trace.lane = worker.lane; trace.block_id = worker.block_id; trace.core_idx = static_cast(worker_id); - Ops::InvalidateRegion( - &state->config, sizeof(state->config) + sizeof(state->winner_workload) - ); +#if PA_BUILD_SUBMIT_PMU + // 诊断 ELF 不含 records 写入、atomic span 或 trace-only SYS_CNT;保留同一 + // TraceContext 形状只是为了复用调度协议源码。 + (void)state; + return trace; +#else const uint64_t base = state->config.trace_base; const uint32_t capacity = state->config.trace_records_per_core; if ((state->config.trace_enabled & kTracePhasesEnabled) == 0 || base == 0 || capacity == 0 || @@ -159,6 +162,7 @@ PA_DEVICE TraceContext AttachTrace( trace.core->block_id = trace.block_id; trace.core->lane = trace.lane; return trace; +#endif } template @@ -175,6 +179,16 @@ PA_DEVICE_NOINLINE bool WritePollBatchRecordRaw( PA_GM TraceCoreState *core, PA_GM TraceRecord *records, uint32_t capacity, uint64_t start_cycle, uint64_t end_cycle, uint32_t call_count, uint32_t site_id ) { +#if PA_BUILD_SUBMIT_PMU + (void)core; + (void)records; + (void)capacity; + (void)start_cycle; + (void)end_cycle; + (void)call_count; + (void)site_id; + return false; +#else if (core == nullptr || records == nullptr || capacity == 0) { return false; } @@ -200,6 +214,7 @@ PA_DEVICE_NOINLINE bool WritePollBatchRecordRaw( // count 最后更新,使其始终指向下一空槽;单写者条件下无需 reserve/commit 两阶段。 core->count = slot + 1; return true; +#endif } PA_DEVICE uint32_t AtomicTraceFlags( @@ -235,6 +250,10 @@ template PA_DEVICE void AtomicPollBoundaryAt( TraceContext &trace, uint64_t end_cycle ) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)end_cycle; +#else if (!trace.atomics_enabled || trace.poll_burst.active_mask == 0) return; const uint32_t active_mask = trace.poll_burst.active_mask; // CCEC 默认会把固定 6-site 循环完整展开,再随几十个 phase 边界复制。 @@ -264,13 +283,19 @@ PA_DEVICE void AtomicPollBoundaryAt( trace.poll_burst.call_count[index] = 0; } trace.poll_burst.active_mask = 0; +#endif } template PA_DEVICE void AtomicPollBoundary(TraceContext &trace, WorkerResult &result) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; +#else (void)result; if (trace.poll_burst.active_mask == 0) return; AtomicPollBoundaryAt(trace, Ops::Now()); +#endif } template @@ -278,9 +303,14 @@ PA_DEVICE uint32_t AtomicPollRegionBegin( TraceContext &trace, WorkerResult &result, uint32_t site_mask ) { const uint32_t previous_mask = trace.poll_burst.enabled_mask; +#if PA_BUILD_SUBMIT_PMU + (void)result; + (void)site_mask; +#else if (!trace.atomics_enabled) return previous_mask; AtomicPollBoundary(trace, result); trace.poll_burst.enabled_mask = previous_mask | site_mask; +#endif return previous_mask; } @@ -288,9 +318,15 @@ template PA_DEVICE void AtomicPollRegionEnd( TraceContext &trace, WorkerResult &result, uint32_t previous_mask ) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + (void)previous_mask; +#else if (!trace.atomics_enabled) return; AtomicPollBoundary(trace, result); trace.poll_burst.enabled_mask = previous_mask; +#endif } PA_DEVICE bool AtomicPollBatchEnabled( @@ -346,6 +382,14 @@ PA_DEVICE T TraceAtomicLoad( TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, PA_GM volatile T *address, bool result_used = true ) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + (void)task_id; + (void)site; + (void)result_used; + return Ops::Load(address); +#else if (!trace.atomics_enabled) return Ops::Load(address); const bool poll_batch = result_used && AtomicPollBatchEnabled(trace, site, AtomicOp::Load); const int32_t poll_index = poll_batch ? TraceAtomicPollBatchIndex(site) : -1; @@ -367,6 +411,7 @@ PA_DEVICE T TraceAtomicLoad( old == static_cast(0) ); return old; +#endif } template @@ -374,6 +419,14 @@ PA_DEVICE T TraceAtomicExchange( TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, PA_GM volatile T *address, T value, bool result_used = false ) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + (void)task_id; + (void)site; + (void)result_used; + return Ops::Exchange(address, value); +#else if (!trace.atomics_enabled) return Ops::Exchange(address, value); const uint64_t begin = Ops::Now(); const T old = Ops::Exchange(address, value); @@ -383,6 +436,7 @@ PA_DEVICE T TraceAtomicExchange( trace, result, task_id, site, AtomicOp::Exchange, begin, end, result_used, return_ready ); return old; +#endif } template @@ -390,6 +444,14 @@ PA_DEVICE int64_t TraceAtomicFetchAdd( TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, PA_GM volatile int64_t *address, int64_t value, bool result_used = false ) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + (void)task_id; + (void)site; + (void)result_used; + return Ops::FetchAdd(address, value); +#else if (!trace.atomics_enabled) return Ops::FetchAdd(address, value); const uint64_t begin = Ops::Now(); const int64_t old = Ops::FetchAdd(address, value); @@ -399,6 +461,7 @@ PA_DEVICE int64_t TraceAtomicFetchAdd( trace, result, task_id, site, AtomicOp::FetchAdd, begin, end, result_used, return_ready ); return old; +#endif } template @@ -406,6 +469,14 @@ PA_DEVICE int64_t TraceAtomicFetchMax( TraceContext &trace, WorkerResult &result, int32_t task_id, AtomicSite site, PA_GM volatile int64_t *address, int64_t value, uint64_t &retries, bool result_used = true ) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + (void)task_id; + (void)site; + (void)result_used; + return Ops::FetchMax(address, value, retries); +#else if (!trace.atomics_enabled) return Ops::FetchMax(address, value, retries); const uint64_t begin = Ops::Now(); const int64_t old = Ops::FetchMax(address, value, retries); @@ -416,12 +487,20 @@ PA_DEVICE int64_t TraceAtomicFetchMax( return_ready, false, retries ); return old; +#endif } template PA_DEVICE void AccumulatePhase( WorkerResult &result, ProfilePhase phase, uint64_t start_cycle, uint64_t end_cycle ) { +#if PA_BUILD_SUBMIT_PMU + (void)result; + (void)phase; + (void)start_cycle; + (void)end_cycle; + return; +#else // phase profile 与完整泳道是两套正交机制:即使关闭 records,Profile=true // 仍会累计用户当前关注的 Claim/EfDrain/WaitForSlot/HeapGuard 四段。 if constexpr (Profile) { @@ -436,6 +515,7 @@ PA_DEVICE void AccumulatePhase( result.phase_cycles[index] += duration; ++result.phase_calls[index]; } +#endif } template @@ -444,6 +524,19 @@ PA_DEVICE void WriteTrace( ProfilePhase profile_phase, uint64_t start_cycle, uint64_t end_cycle, uint32_t flags, uint32_t auxiliary ) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + (void)task_id; + (void)function_id; + (void)trace_phase; + (void)profile_phase; + (void)start_cycle; + (void)end_cycle; + (void)flags; + (void)auxiliary; + return; +#else // 每段先更新轻量 phase 统计,再按需写 64-byte 原始记录。一个分区只有对应 // worker 写入,因此 count/dropped 保持普通单写者更新,不额外引入 atomic。 AccumulatePhase(result, profile_phase, start_cycle, end_cycle); @@ -470,6 +563,7 @@ PA_DEVICE void WriteTrace( // count 最后更新,使其始终指向下一空槽;单写者条件下无需 reserve/commit 两阶段。 // 最终 FlushTraceCore 会把记录体先于该计数一并导出。 core.count = slot + 1; +#endif } template @@ -478,16 +572,27 @@ PA_DEVICE void ResetTraceLap( ) { // lap 是后续 Build/Replay/Alloc 等覆盖式阶段的共同起点,不代表新增嵌套 span。 // 因此分析时不能把 lap 时长再与其中的 Materialize/Claim/Register 直接相加。 +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + (void)worker; +#else (void)result; const uint64_t cycle = Ops::Now(); // 与真实 FDWIC 的 TRACE_LAP_RESET 保持同一边界:等待区 PollBatch // 只能覆盖本次逻辑轮询 episode,不能跨进下一段 lap 或计算单元执行。 AtomicPollBoundaryAt(trace, cycle); worker.swimlane_last_cycle = cycle; +#endif } template PA_DEVICE void FlushTraceCore(TraceContext &trace, WorkerResult &result) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)result; + return; +#else if (trace.core == nullptr || trace.records == nullptr || trace.capacity == 0) { return; } @@ -509,6 +614,7 @@ PA_DEVICE void FlushTraceCore(TraceContext &trace, WorkerResult &result) { Ops::FlushRegion(trace.records, static_cast(count) * sizeof(TraceRecord)); } Ops::FlushRegion(&core, sizeof(core)); +#endif } template @@ -517,6 +623,18 @@ PA_DEVICE uint64_t WriteTraceLap( int32_t function_id, TracePhase trace_phase, ProfilePhase profile_phase, uint32_t flags = 0, uint32_t auxiliary = 0 ) { +#if PA_BUILD_SUBMIT_PMU + (void)trace; + (void)worker; + (void)result; + (void)task_id; + (void)function_id; + (void)trace_phase; + (void)profile_phase; + (void)flags; + (void)auxiliary; + return 0; +#else // lap 记录区间 [上一次 Reset/WriteTraceLap, 当前时刻],写完立即推进起点。 // 显式 WriteTrace span 不会修改该起点,这正是生产泳道中阶段可重叠的原因。 const uint64_t end_cycle = Ops::Now(); @@ -529,6 +647,7 @@ PA_DEVICE uint64_t WriteTraceLap( ); worker.swimlane_last_cycle = end_cycle; return end_cycle; +#endif } } // namespace pa_scheduler diff --git a/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py b/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py index 1684250bbf..8b8f7f915c 100644 --- a/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py +++ b/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py @@ -27,7 +27,7 @@ SCHEMA_NAME = "pa_scheduler_pmu_phase_windows" -SCHEMA_VERSION = 3 +SCHEMA_VERSIONS = (3, 4) GROUP_NAMES = ("all", "aic", "aiv") METRIC_NAMES = ( "total_cycles", @@ -41,7 +41,40 @@ "icache_misses", "fix_busy", ) +SUBMIT_PMU_METRIC_NAMES = ( + "total_cycles", + "vector_busy", + "cube_busy", + "scalar_busy", + "mte1_busy", + "mte2_busy", + "icache_requests", + "icache_misses", + "shadow_whole_icache_requests", + "shadow_whole_icache_misses", + "phase_calls", + "phase_icache_requests", + "phase_icache_misses", +) SUMMARY_FIELDS = ("sum", "mean", "median", "p95", "max") +SUBMIT_PMU_BUILD_VARIANT = "submit-pmu" +SUBMIT_PMU_BUILD_VARIANT_ID = 2 +SUBMIT_PMU_PHASE_IDS = {"none": 0, "claim": 1} +TASKS_PER_BATCH = 5 +PHASE_STATUS_REQUIRED_MASK = 0x3CF + +# schema-v4 只描述 A5 standalone submit-pmu 正式采集,不接受由 JSON 自报的 +# 任意缩小拓扑。物理槽按每 die 18 AIC + 36 AIV 排列;当前 runtime 实际开放 +# 32 个 AIC 与 64 个 AIV,共组成 32 组 1:2 mixed triplet。 +A5_WORKERS = 96 +A5_AIC_WORKERS = 32 +A5_AIV_WORKERS = 64 +A5_PHYSICAL_SUBCORES = 108 +A5_AIC_PER_DIE = 18 +A5_SUBCORES_PER_DIE = 54 +A5_OWNER_BITMAP_WORDS = 4 +A5_OWNER_MAGIC = 0x504D554F +A5_OWNER_VERSION = 1 # 这些字段决定两份 sidecar 是否属于同一观察配置。动态时间、capture id 和 # placement 分布不在其中;它们正是多轮运行允许自然变化的结果。 @@ -62,6 +95,20 @@ "gate_start_stop_have_pipe_all_barriers", "winner_workload", ) +SUBMIT_PMU_FINGERPRINT_FIELDS = CONFIG_FINGERPRINT_FIELDS + ( + "build_variant", + "build_variant_id", + "compiled_phase", + "compiled_phase_id", + "primary_window_segments_per_record", + "unavailable_metrics", + "phase_boundary_observation_included", + "phase_counter_pair_snapshot_atomic", + "primary_counters_read_at_phase_boundaries", + "phase_shadow_partition_exact_required", + "phase_values_are_running_read_clear_lower_bounds", + "cross_phase_elf_sums_valid", +) @dataclass(frozen=True) @@ -72,6 +119,19 @@ class Capture: data: dict[str, Any] groups: dict[str, list[dict[str, Any]]] fingerprint: str + schema_version: int + + +@dataclass(frozen=True) +class PhasePartitionEvidence: + """一条 raw 记录独立重算出的 shadow 分区证据。""" + + shadow_exact: bool + shadow_bounded: bool + request_abs_delta: int + request_signed_delta: int + miss_abs_delta: int + miss_signed_delta: int def _require(condition: bool, message: str) -> None: @@ -88,6 +148,12 @@ def _integer(value: Any, label: str) -> int: return value +def _signed_integer(value: Any, label: str) -> int: + if isinstance(value, bool) or not isinstance(value, int): + raise ValueError(f"{label} must be an integer") + return value + + def _number(value: Any, label: str) -> float: if isinstance(value, bool) or not isinstance(value, (int, float)): raise ValueError(f"{label} must be numeric") @@ -122,8 +188,14 @@ def _same_number(lhs: int | float, rhs: Any) -> bool: return math.isclose(float(lhs), rhs_number, rel_tol=1e-12, abs_tol=1e-9) -def _configuration_fingerprint(configuration: dict[str, Any]) -> str: - selected = {field: configuration.get(field) for field in CONFIG_FINGERPRINT_FIELDS} +def _configuration_fingerprint(configuration: dict[str, Any], schema_version: int) -> str: + fields = ( + SUBMIT_PMU_FINGERPRINT_FIELDS if schema_version == 4 else CONFIG_FINGERPRINT_FIELDS + ) + selected = {field: configuration.get(field) for field in fields} + # schema version 不是 configuration 字段,但必须进入指纹,防止 v3 历史文件与 + # v4 submit-pmu 恰好具有相同运行参数时被静默聚合。 + selected["schema_version"] = schema_version return json.dumps(selected, ensure_ascii=False, sort_keys=True, separators=(",", ":")) @@ -132,6 +204,8 @@ def _validate_group_summary( group_name: str, records: Sequence[dict[str, Any]], expected: Any, + metric_names: Sequence[str], + schema_version: int, ) -> None: _require(isinstance(expected, dict), f"{path}: summary.{group_name} must be an object") _require( @@ -143,7 +217,7 @@ def _validate_group_summary( f"{path}: summary.{group_name}.trusted_cores is incomplete", ) - for metric in METRIC_NAMES: + for metric in metric_names: values = [ _integer(record.get(metric), f"{path}: records[{index}].{metric}") for index, record in enumerate(records) @@ -171,17 +245,439 @@ def _validate_group_summary( f"raw={actual_rate!r} json={expected.get('icache_miss_rate')!r}", ) + if schema_version == 4: + phase_requests = sum( + _integer(record["phase_icache_requests"], "phase_icache_requests") + for record in records + ) + phase_misses = sum( + _integer(record["phase_icache_misses"], "phase_icache_misses") + for record in records + ) + reported_rate = expected.get("phase_observed_read_clear_ratio") + if phase_requests == 0: + _require( + reported_rate is None, + f"{path}: summary.{group_name}.phase_observed_read_clear_ratio " + "must be null for zero requests", + ) + else: + actual_phase_rate = phase_misses / phase_requests + _require( + _same_number(actual_phase_rate, reported_rate), + f"{path}: raw summary mismatch at " + f"{group_name}.phase_observed_read_clear_ratio: " + f"raw={actual_phase_rate!r} json={reported_rate!r}", + ) + + +def _validate_submit_pmu_configuration(path: Path, configuration: dict[str, Any]) -> tuple[str, int]: + """校验 v4 的编译期构建身份、局部阶段和重复 selector 契约。""" + + _require( + configuration.get("build_variant") == SUBMIT_PMU_BUILD_VARIANT, + f"{path}: configuration.build_variant must be {SUBMIT_PMU_BUILD_VARIANT!r}", + ) + variant_id = _integer( + configuration.get("build_variant_id"), f"{path}: configuration.build_variant_id" + ) + _require( + variant_id == SUBMIT_PMU_BUILD_VARIANT_ID, + f"{path}: unexpected submit-pmu build_variant_id {variant_id}", + ) + phase_name = configuration.get("compiled_phase") + _require( + isinstance(phase_name, str) and phase_name in SUBMIT_PMU_PHASE_IDS, + f"{path}: unsupported configuration.compiled_phase {phase_name!r}", + ) + phase_id = _integer( + configuration.get("compiled_phase_id"), f"{path}: configuration.compiled_phase_id" + ) + _require( + phase_id == SUBMIT_PMU_PHASE_IDS[phase_name], + f"{path}: compiled phase name/id mismatch", + ) + _require( + configuration.get("pmu_window") == "submit-all", + f"{path}: schema-v4 submit-pmu requires pmu_window='submit-all'", + ) + _require( + _integer( + configuration.get("primary_window_segments_per_record"), + f"{path}: configuration.primary_window_segments_per_record", + ) + == 1, + f"{path}: configuration.primary_window_segments_per_record must be one", + ) + _require( + configuration.get("unavailable_metrics") == ["mte3_busy"], + f"{path}: configuration.unavailable_metrics must identify mte3_busy", + ) + + for field in ( + "trace_enabled", + "trace_atomics", + "profile_phases", + "phase_timestamp_calls_present", + "phase_record_writes", + "atomic_trace", + "profile_accumulation", + "primary_counters_read_at_phase_boundaries", + "cross_phase_elf_sums_valid", + ): + _require(configuration.get(field) is False, f"{path}: configuration.{field} is not false") + expected_boundary_observation = phase_name != "none" + _require( + configuration.get("phase_boundary_observation_included") + is expected_boundary_observation, + f"{path}: configuration.phase_boundary_observation_included does not match the phase", + ) + _require( + configuration.get("phase_counter_pair_snapshot_atomic") is False, + f"{path}: configuration.phase_counter_pair_snapshot_atomic is not false", + ) + exact_partition_required = phase_name == "none" + _require( + configuration.get("phase_shadow_partition_exact_required") + is exact_partition_required, + f"{path}: configuration.phase_shadow_partition_exact_required does not match the phase", + ) + running_lower_bounds = phase_name != "none" + _require( + configuration.get("phase_values_are_running_read_clear_lower_bounds") + is running_lower_bounds, + f"{path}: configuration.phase_values_are_running_read_clear_lower_bounds does not match the phase", + ) + + selectors = configuration.get("selectors") + _require(isinstance(selectors, dict), f"{path}: configuration.selectors must be an object") + expected_selectors = { + "cnt6_primary_icache_request": 0x034, + "cnt7_primary_icache_miss": 0x035, + "cnt5_shadow_icache_miss": 0x035, + "cnt8_shadow_icache_request": 0x034, + "cnt9_unused": 0x000, + } + for field, expected in expected_selectors.items(): + _require( + _integer(selectors.get(field), f"{path}: configuration.selectors.{field}") == expected, + f"{path}: configuration.selectors.{field} is not 0x{expected:03x}", + ) + return phase_name, phase_id + + +def _is_aic_physical_slot(physical_id: int) -> bool: + return ( + physical_id < A5_PHYSICAL_SUBCORES + and physical_id % A5_SUBCORES_PER_DIE < A5_AIC_PER_DIE + ) + + +def _expected_logical_triplet(worker_id: int) -> tuple[str, int, int]: + """按 mixed launch ABI 返回 worker 的 role/block/lane。""" + + if worker_id < A5_AIC_WORKERS: + return "aic", worker_id, 0 + vector_id = worker_id - A5_AIC_WORKERS + return "aiv", vector_id // 2, 1 + vector_id % 2 + + +def _physical_aiv_pair(aic_physical_id: int) -> tuple[int, int]: + """返回一个物理 AIC 槽对应的两个 AIV 槽。""" + + die_base = (aic_physical_id // A5_SUBCORES_PER_DIE) * A5_SUBCORES_PER_DIE + local_aic = aic_physical_id % A5_SUBCORES_PER_DIE + first_aiv = die_base + A5_AIC_PER_DIE + local_aic * 2 + return first_aiv, first_aiv + 1 + + +def _require_owner_role_counts( + path: Path, label: str, value: Any, expected: tuple[int, int, int] +) -> None: + _require(isinstance(value, dict), f"{path}: owner.{label} must be an object") + for field, expected_value in zip(("total", "aic", "aiv"), expected): + actual = _integer(value.get(field), f"{path}: owner.{label}.{field}") + _require( + actual == expected_value, + f"{path}: owner.{label}.{field} must be {expected_value}", + ) + + +def _validate_submit_pmu_owner( + path: Path, owner: Any, capture: dict[str, Any] +) -> set[int]: + """独立重验 configure 后、restore 前保存的 PMU owner 快照。""" + + _require(isinstance(owner, dict), f"{path}: schema-v4 owner must be an object") + _require(owner.get("mode") == "main_aicpu_path_a", f"{path}: unexpected owner.mode") + _require( + owner.get("snapshot_phase") == "after_configure_before_restore", + f"{path}: unexpected owner.snapshot_phase", + ) + _require( + _integer(owner.get("control_magic"), f"{path}: owner.control_magic") + == A5_OWNER_MAGIC, + f"{path}: owner.control_magic mismatch", + ) + _require( + _integer(owner.get("control_version"), f"{path}: owner.control_version") + == A5_OWNER_VERSION, + f"{path}: owner.control_version mismatch", + ) + _require( + _integer(owner.get("configure_status"), f"{path}: owner.configure_status") == 0, + f"{path}: owner.configure_status is not success", + ) + _require( + _integer(owner.get("configured_flag"), f"{path}: owner.configured_flag") == 1, + f"{path}: owner.configured_flag is not one", + ) + + topology = (A5_WORKERS, A5_AIC_WORKERS, A5_AIV_WORKERS) + for label in ("expected", "active", "discovered"): + _require_owner_role_counts(path, label, owner.get(label), topology) + _require( + _integer(owner.get("physical_slots_scanned"), f"{path}: owner.physical_slots_scanned") + == A5_PHYSICAL_SUBCORES, + f"{path}: owner.physical_slots_scanned mismatch", + ) + _require( + _integer( + owner.get("skipped_physical_slots"), f"{path}: owner.skipped_physical_slots" + ) + == A5_PHYSICAL_SUBCORES - A5_WORKERS, + f"{path}: owner.skipped_physical_slots mismatch", + ) + _require( + owner.get("configured_bitmap_word_order") + == "least_significant_physical_ids_first", + f"{path}: unexpected owner.configured_bitmap_word_order", + ) + + words = owner.get("configured_bitmap_words") + _require( + isinstance(words, list) and len(words) == A5_OWNER_BITMAP_WORDS, + f"{path}: owner.configured_bitmap_words must contain four words", + ) + bitmap_words: list[int] = [] + for index, value in enumerate(words): + word = _integer(value, f"{path}: owner.configured_bitmap_words[{index}]") + _require(word <= 0xFFFFFFFF, f"{path}: owner bitmap word exceeds 32 bits") + bitmap_words.append(word) + + configured_ids = { + physical_id + for physical_id in range(A5_OWNER_BITMAP_WORDS * 32) + if bitmap_words[physical_id // 32] & (1 << (physical_id % 32)) + } + _require( + all(physical_id < A5_PHYSICAL_SUBCORES for physical_id in configured_ids), + f"{path}: owner bitmap sets a bit outside the 108 physical slots", + ) + configured_count = _integer( + owner.get("configured_bitmap_count"), f"{path}: owner.configured_bitmap_count" + ) + _require( + configured_count == len(configured_ids) == A5_WORKERS, + f"{path}: owner bitmap count is not exactly 96", + ) + configured_aic = {physical_id for physical_id in configured_ids if _is_aic_physical_slot(physical_id)} + configured_aiv = configured_ids - configured_aic + _require( + len(configured_aic) == A5_AIC_WORKERS and len(configured_aiv) == A5_AIV_WORKERS, + f"{path}: owner bitmap is not a 32 AIC / 64 AIV set", + ) + + complete_triplets = sum( + all(aiv_id in configured_aiv for aiv_id in _physical_aiv_pair(aic_id)) + for aic_id in configured_aic + ) + broken_triplets = len(configured_aic) - complete_triplets + _require( + _integer( + owner.get("configured_complete_mixed_triplets"), + f"{path}: owner.configured_complete_mixed_triplets", + ) + == complete_triplets + == A5_AIC_WORKERS, + f"{path}: owner bitmap does not contain 32 complete mixed triplets", + ) + _require( + _integer( + owner.get("expected_complete_mixed_triplets"), + f"{path}: owner.expected_complete_mixed_triplets", + ) + == A5_AIC_WORKERS, + f"{path}: owner.expected_complete_mixed_triplets mismatch", + ) + _require( + _integer( + owner.get("configured_broken_mixed_triplets"), + f"{path}: owner.configured_broken_mixed_triplets", + ) + == broken_triplets + == 0, + f"{path}: owner bitmap contains a broken mixed triplet", + ) + _require(owner.get("restore_passed") is True, f"{path}: owner.restore_passed is not true") + _require( + owner.get("restore_passed") is capture.get("owner_restore_passed"), + f"{path}: owner and capture restore results disagree", + ) + return configured_ids + + +def _validate_submit_pmu_record( + path: Path, + index: int, + record: dict[str, Any], + phase_name: str, + phase_id: int, + batches: int, +) -> PhasePartitionEvidence: + """重验 raw phase 分区;claim 只形成 read-to-clear 的上下界。""" + + prefix = f"{path}: records[{index}]" + _require( + _integer(record.get("build_variant_id"), f"{prefix}.build_variant_id") + == SUBMIT_PMU_BUILD_VARIANT_ID, + f"{prefix} build_variant_id mismatch", + ) + _require( + _integer(record.get("compiled_phase_id"), f"{prefix}.compiled_phase_id") == phase_id, + f"{prefix} compiled_phase_id mismatch", + ) + phase_status = _integer(record.get("phase_status"), f"{prefix}.phase_status") + _require( + (phase_status & PHASE_STATUS_REQUIRED_MASK) == PHASE_STATUS_REQUIRED_MASK, + f"{prefix} phase_status is incomplete", + ) + + primary_requests = _integer(record.get("icache_requests"), f"{prefix}.icache_requests") + primary_misses = _integer(record.get("icache_misses"), f"{prefix}.icache_misses") + shadow_requests = _integer( + record.get("shadow_whole_icache_requests"), f"{prefix}.shadow_whole_icache_requests" + ) + shadow_misses = _integer( + record.get("shadow_whole_icache_misses"), f"{prefix}.shadow_whole_icache_misses" + ) + shadow_exact = shadow_requests == primary_requests and shadow_misses == primary_misses + shadow_bounded = shadow_requests <= primary_requests and shadow_misses <= primary_misses + _require( + record.get("shadow_matches_primary") is shadow_exact, + f"{prefix}.shadow_matches_primary disagrees with raw counters", + ) + _require( + record.get("shadow_not_greater_than_primary") is shadow_bounded, + f"{prefix}.shadow_not_greater_than_primary disagrees with raw counters", + ) + _require(shadow_bounded, f"{prefix} shadow whole exceeds the authoritative primary whole") + if phase_name == "none": + _require(shadow_exact, f"{prefix} phase=none requires shadow whole to equal primary") + + request_loss = primary_requests - shadow_requests + miss_loss = primary_misses - shadow_misses + _require( + _integer(record.get("shadow_request_loss"), f"{prefix}.shadow_request_loss") + == request_loss, + f"{prefix}.shadow_request_loss disagrees with raw counters", + ) + _require( + _integer(record.get("shadow_miss_loss"), f"{prefix}.shadow_miss_loss") == miss_loss, + f"{prefix}.shadow_miss_loss disagrees with raw counters", + ) + + calls = _integer(record.get("phase_calls"), f"{prefix}.phase_calls") + begin_reads = _integer(record.get("phase_begin_reads"), f"{prefix}.phase_begin_reads") + end_reads = _integer(record.get("phase_end_reads"), f"{prefix}.phase_end_reads") + _require( + record.get("phase_boundaries_balanced") is True, + f"{prefix}.phase_boundaries_balanced is not true", + ) + _require( + begin_reads == calls and end_reads == calls, + f"{prefix} phase begin/end boundaries do not match calls", + ) + primary_segments = _integer( + record.get("primary_window_segments"), f"{prefix}.primary_window_segments" + ) + shadow_segments = _integer( + record.get("shadow_read_segments"), f"{prefix}.shadow_read_segments" + ) + _require(primary_segments == 1, f"{prefix} primary_window_segments must be one") + _require( + shadow_segments == 2 * calls + 1, + f"{prefix} shadow_read_segments does not match phase boundaries plus tail", + ) + + phase_requests = _integer( + record.get("phase_icache_requests"), f"{prefix}.phase_icache_requests" + ) + phase_misses = _integer( + record.get("phase_icache_misses"), f"{prefix}.phase_icache_misses" + ) + phase_request_upper = _integer( + record.get("phase_icache_requests_upper_bound"), + f"{prefix}.phase_icache_requests_upper_bound", + ) + phase_miss_upper = _integer( + record.get("phase_icache_misses_upper_bound"), + f"{prefix}.phase_icache_misses_upper_bound", + ) + _require( + phase_requests <= shadow_requests and phase_misses <= shadow_misses, + f"{prefix} phase counters exceed the Submit shadow whole", + ) + _require( + phase_request_upper == phase_requests + request_loss, + f"{prefix}.phase_icache_requests_upper_bound is not lower plus shadow loss", + ) + _require( + phase_miss_upper == phase_misses + miss_loss, + f"{prefix}.phase_icache_misses_upper_bound is not lower plus shadow loss", + ) + _require( + phase_request_upper <= primary_requests and phase_miss_upper <= primary_misses, + f"{prefix} phase upper bound exceeds the authoritative primary whole", + ) + + expected_calls = 0 if phase_name == "none" else batches * TASKS_PER_BATCH + _require(calls == expected_calls, f"{prefix} phase_calls does not match the phase contract") + if phase_name == "none": + _require( + phase_requests == 0 and phase_misses == 0, + f"{prefix} phase=none must have zero phase counters", + ) + _require( + phase_request_upper == phase_requests and phase_miss_upper == phase_misses, + f"{prefix} phase=none lower and upper bounds must be equal", + ) + + return PhasePartitionEvidence( + shadow_exact=shadow_exact, + shadow_bounded=shadow_bounded, + request_abs_delta=abs(shadow_requests - primary_requests), + request_signed_delta=shadow_requests - primary_requests, + miss_abs_delta=abs(shadow_misses - primary_misses), + miss_signed_delta=shadow_misses - primary_misses, + ) + def load_capture(path: Path) -> Capture: - """读取并完整校验一份 schema-v3 PMU sidecar。""" + """读取并完整校验一份历史 v3 或 submit-pmu v4 sidecar。""" with path.open("r", encoding="utf-8") as input_file: data = json.load(input_file) _require(isinstance(data, dict), f"{path}: capture root must be an object") schema = data.get("schema") _require( - schema == {"name": SCHEMA_NAME, "version": SCHEMA_VERSION}, - f"{path}: expected {SCHEMA_NAME} schema v{SCHEMA_VERSION}", + isinstance(schema, dict) and schema.get("name") == SCHEMA_NAME, + f"{path}: expected schema name {SCHEMA_NAME}", + ) + schema_version = _integer(schema.get("version"), f"{path}: schema.version") + _require( + schema_version in SCHEMA_VERSIONS, + f"{path}: expected {SCHEMA_NAME} schema v3 or v4", ) capture = data.get("capture") @@ -198,8 +694,18 @@ def load_capture(path: Path) -> Capture: workers = _integer(configuration.get("workers"), f"{path}: configuration.workers") aic_workers = _integer(configuration.get("aic_workers"), f"{path}: configuration.aic_workers") aiv_workers = _integer(configuration.get("aiv_workers"), f"{path}: configuration.aiv_workers") + batches = _integer(configuration.get("batches"), f"{path}: configuration.batches") _require(workers == aic_workers + aiv_workers, f"{path}: worker role counts do not add up") _require(len(records) == workers, f"{path}: record count does not match configuration.workers") + phase_name: str | None = None + phase_id: int | None = None + if schema_version == 4: + _require( + (workers, aic_workers, aiv_workers) + == (A5_WORKERS, A5_AIC_WORKERS, A5_AIV_WORKERS), + f"{path}: schema-v4 requires the fixed 96/32/64 A5 topology", + ) + phase_name, phase_id = _validate_submit_pmu_configuration(path, configuration) # JSON 只有在运行、PMU、owner Restore 和 runtime cleanup 全部成功后才应发布。 # 分析器仍逐项重验,防止手工复制或未来 schema 退化绕过发布门禁。 @@ -216,6 +722,13 @@ def load_capture(path: Path) -> Capture: ) for owner, field in required_true: _require(owner.get(field) is True, f"{path}: {field} is not true") + configured_physical_ids: set[int] | None = None + if schema_version == 4: + _require( + validation.get("phase_measurement_valid") is True, + f"{path}: phase_measurement_valid is not true", + ) + configured_physical_ids = _validate_submit_pmu_owner(path, data.get("owner"), capture) expected_records = ( "trusted_records", @@ -231,17 +744,51 @@ def load_capture(path: Path) -> Capture: validation.get(field) == workers, f"{path}: validation.{field} is incomplete", ) + if schema_version == 4: + expected_host_counts = { + "expected_records": A5_WORKERS, + "expected_unique_core_ids": A5_WORKERS, + "expected_owner_bitmap_member_records": A5_WORKERS, + "expected_exact_worker_slot_records": A5_WORKERS, + "expected_physical_role_match_records": A5_WORKERS, + "mixed_triplet_matches": A5_AIC_WORKERS, + "expected_mixed_triplet_matches": A5_AIC_WORKERS, + "expected_window_records": A5_WORKERS, + } + for field, expected in expected_host_counts.items(): + _require( + _integer(validation.get(field), f"{path}: validation.{field}") == expected, + f"{path}: validation.{field} does not match the A5 topology", + ) + for field in ( + "build_variant_match_records", + "phase_id_match_records", + "phase_status_trusted_records", + "phase_boundary_match_records", + "phase_call_shape_match_records", + ): + _require( + validation.get(field) == workers, + f"{path}: validation.{field} is incomplete", + ) + expected_phase_calls = ( + 0 if phase_name == "none" else workers * batches * TASKS_PER_BATCH + ) + _require( + validation.get("phase_calls") == expected_phase_calls, + f"{path}: validation.phase_calls does not match the compiled phase contract", + ) groups: dict[str, list[dict[str, Any]]] = { "all": records, "aic": [record for record in records if record.get("role") == "aic"], "aiv": [record for record in records if record.get("role") == "aiv"], } - _require(len(groups["aic"]) == aic_workers, f"{path}: AIC raw record count mismatch") - _require(len(groups["aiv"]) == aiv_workers, f"{path}: AIV raw record count mismatch") worker_ids: set[int] = set() physical_core_ids: set[int] = set() + ordered_physical_ids: list[int] = [] + phase_partition_evidence: list[PhasePartitionEvidence] = [] for index, record in enumerate(records): _require(isinstance(record, dict), f"{path}: records[{index}] must be an object") worker_id = _integer(record.get("worker_id"), f"{path}: records[{index}].worker_id") @@ -255,6 +802,43 @@ def load_capture(path: Path) -> Capture: ) worker_ids.add(worker_id) physical_core_ids.add(physical_id) + ordered_physical_ids.append(physical_id) + if schema_version == 4: + _require( + worker_id == index, + f"{path}: records[{index}].worker_id does not match its exact worker slot", + ) + expected_role, expected_block, expected_lane = _expected_logical_triplet(index) + _require( + record.get("role") == expected_role, + f"{path}: records[{index}].role does not match the logical worker topology", + ) + _require( + _integer(record.get("block_id"), f"{path}: records[{index}].block_id") + == expected_block, + f"{path}: records[{index}].block_id does not match its mixed block", + ) + _require( + _integer(record.get("lane"), f"{path}: records[{index}].lane") == expected_lane, + f"{path}: records[{index}].lane does not match its mixed lane", + ) + _require( + physical_id < A5_PHYSICAL_SUBCORES, + f"{path}: records[{index}].physical_core_id is outside the 108-slot topology", + ) + _require( + _is_aic_physical_slot(physical_id) == (expected_role == "aic"), + f"{path}: records[{index}] logical role does not match its physical slot", + ) + _require( + record.get("physical_core_id_valid") is True, + f"{path}: records[{index}].physical_core_id_valid is not true", + ) + assert configured_physical_ids is not None + _require( + physical_id in configured_physical_ids, + f"{path}: records[{index}].physical_core_id is absent from the owner bitmap", + ) for field in ( "trusted", "selectors_match", @@ -272,11 +856,113 @@ def load_capture(path: Path) -> Capture: record.get("icache_misses"), f"{path}: records[{index}].icache_misses" ) _require(misses <= requests, f"{path}: records[{index}] has miss > request") + if schema_version == 4: + # 上面的 configuration 校验已保证二者不是 None;显式 assert 只帮助 + # 类型收窄,不替代任何 JSON 运行时门禁。 + assert phase_name is not None and phase_id is not None + phase_partition_evidence.append( + _validate_submit_pmu_record(path, index, record, phase_name, phase_id, batches) + ) + + if schema_version == 4: + assert configured_physical_ids is not None + _require( + physical_core_ids == configured_physical_ids, + f"{path}: worker physical-core set does not exactly equal the owner bitmap", + ) + for block in range(A5_AIC_WORKERS): + aic_id = ordered_physical_ids[block] + expected_aiv0, expected_aiv1 = _physical_aiv_pair(aic_id) + actual_aiv0 = ordered_physical_ids[A5_AIC_WORKERS + block * 2] + actual_aiv1 = ordered_physical_ids[A5_AIC_WORKERS + block * 2 + 1] + _require( + (actual_aiv0, actual_aiv1) == (expected_aiv0, expected_aiv1), + f"{path}: mixed block {block} does not map to one physical AIC and its two AIVs", + ) + + shadow_exact_records = sum(item.shadow_exact for item in phase_partition_evidence) + shadow_bounded_records = sum(item.shadow_bounded for item in phase_partition_evidence) + _require( + shadow_bounded_records == A5_WORKERS, + f"{path}: not all shadow partitions are bounded by primary", + ) + assert phase_name is not None + if phase_name == "none": + _require( + shadow_exact_records == A5_WORKERS, + f"{path}: phase=none requires all shadow partitions to be exact", + ) + _require( + _integer( + validation.get("shadow_primary_match_records"), + f"{path}: validation.shadow_primary_match_records", + ) + == shadow_exact_records, + f"{path}: validation.shadow_primary_match_records disagrees with raw records", + ) + _require( + _integer( + validation.get("shadow_primary_bounded_records"), + f"{path}: validation.shadow_primary_bounded_records", + ) + == shadow_bounded_records, + f"{path}: validation.shadow_primary_bounded_records disagrees with raw records", + ) + unsigned_deltas = { + "shadow_request_abs_delta_sum": sum( + item.request_abs_delta for item in phase_partition_evidence + ), + "shadow_request_abs_delta_max": max( + item.request_abs_delta for item in phase_partition_evidence + ), + "shadow_miss_abs_delta_sum": sum( + item.miss_abs_delta for item in phase_partition_evidence + ), + "shadow_miss_abs_delta_max": max( + item.miss_abs_delta for item in phase_partition_evidence + ), + } + signed_deltas = { + "shadow_request_signed_delta_sum": sum( + item.request_signed_delta for item in phase_partition_evidence + ), + "shadow_miss_signed_delta_sum": sum( + item.miss_signed_delta for item in phase_partition_evidence + ), + } + for field, expected in unsigned_deltas.items(): + _require( + _integer(validation.get(field), f"{path}: validation.{field}") == expected, + f"{path}: validation.{field} disagrees with raw records", + ) + for field, expected in signed_deltas.items(): + _require( + _signed_integer(validation.get(field), f"{path}: validation.{field}") + == expected, + f"{path}: validation.{field} disagrees with raw records", + ) + _require(len(groups["aic"]) == aic_workers, f"{path}: AIC raw record count mismatch") + _require(len(groups["aiv"]) == aiv_workers, f"{path}: AIV raw record count mismatch") + + metric_names = SUBMIT_PMU_METRIC_NAMES if schema_version == 4 else METRIC_NAMES for group_name in GROUP_NAMES: - _validate_group_summary(path, group_name, groups[group_name], summary.get(group_name)) + _validate_group_summary( + path, + group_name, + groups[group_name], + summary.get(group_name), + metric_names, + schema_version, + ) - return Capture(path, data, groups, _configuration_fingerprint(configuration)) + return Capture( + path, + data, + groups, + _configuration_fingerprint(configuration, schema_version), + schema_version, + ) def _median(values: Iterable[int | float]) -> int | float: @@ -289,8 +975,13 @@ def analyze(paths: Sequence[Path], miss_penalty_ns: float = 90.0) -> dict[str, A _require(bool(paths), "at least one PMU JSON path is required") _require(math.isfinite(miss_penalty_ns) and miss_penalty_ns > 0, "miss penalty must be positive") captures = [load_capture(path) for path in paths] + schema_version = captures[0].schema_version fingerprint = captures[0].fingerprint for capture in captures[1:]: + _require( + capture.schema_version == schema_version, + f"{capture.path}: input schema differs from {captures[0].path}", + ) _require( capture.fingerprint == fingerprint, f"{capture.path}: observation configuration differs from {captures[0].path}", @@ -328,6 +1019,105 @@ def analyze(paths: Sequence[Path], miss_penalty_ns: float = 90.0) -> dict[str, A "scalar_busy_sum": group["scalar_busy"]["sum"], "total_cycles_sum": group["total_cycles"]["sum"], } + if schema_version == 4: + raw_group = capture.groups[group_name] + phase_requests = _integer( + group["phase_icache_requests"].get("sum"), "phase I-cache request sum" + ) + phase_misses = _integer( + group["phase_icache_misses"].get("sum"), "phase I-cache miss sum" + ) + phase_calls = _integer(group["phase_calls"].get("sum"), "phase call sum") + phase_request_uppers = [ + _integer( + record.get("phase_icache_requests_upper_bound"), + "phase I-cache request upper bound", + ) + for record in raw_group + ] + phase_miss_uppers = [ + _integer( + record.get("phase_icache_misses_upper_bound"), + "phase I-cache miss upper bound", + ) + for record in raw_group + ] + request_losses = [ + _integer(record.get("shadow_request_loss"), "shadow request loss") + for record in raw_group + ] + miss_losses = [ + _integer(record.get("shadow_miss_loss"), "shadow miss loss") + for record in raw_group + ] + phase_request_upper_summary = _metric_summary(phase_request_uppers) + phase_miss_upper_summary = _metric_summary(phase_miss_uppers) + phase_request_lower_summary = _metric_summary( + [ + _integer( + record.get("phase_icache_requests"), + "phase I-cache request lower bound", + ) + for record in raw_group + ] + ) + phase_miss_lower_summary = _metric_summary( + [ + _integer( + record.get("phase_icache_misses"), + "phase I-cache miss lower bound", + ) + for record in raw_group + ] + ) + request_loss_summary = _metric_summary(request_losses) + miss_loss_summary = _metric_summary(miss_losses) + row["groups"][group_name].update( + { + "phase_calls_sum": phase_calls, + "phase_calls_per_core": phase_calls / cores, + "phase_icache_requests_lower_bound_sum": phase_requests, + "phase_icache_requests_upper_bound_sum": sum(phase_request_uppers), + "phase_icache_misses_lower_bound_sum": phase_misses, + "phase_icache_misses_upper_bound_sum": sum(phase_miss_uppers), + "phase_icache_requests_lower_bound_per_core": phase_requests / cores, + "phase_icache_requests_upper_bound_per_core": sum(phase_request_uppers) + / cores, + "phase_icache_misses_lower_bound_per_core": phase_misses / cores, + "phase_icache_misses_upper_bound_per_core": sum(phase_miss_uppers) / cores, + "phase_icache_requests_lower_bound_per_core_median": + phase_request_lower_summary["median"], + "phase_icache_requests_lower_bound_per_core_p95": + phase_request_lower_summary["p95"], + "phase_icache_requests_upper_bound_per_core_median": + phase_request_upper_summary["median"], + "phase_icache_requests_upper_bound_per_core_p95": + phase_request_upper_summary["p95"], + "phase_icache_misses_lower_bound_per_core_median": + phase_miss_lower_summary["median"], + "phase_icache_misses_lower_bound_per_core_p95": + phase_miss_lower_summary["p95"], + "phase_icache_misses_upper_bound_per_core_median": + phase_miss_upper_summary["median"], + "phase_icache_misses_upper_bound_per_core_p95": + phase_miss_upper_summary["p95"], + "shadow_request_loss_sum": sum(request_losses), + "shadow_request_loss_per_core": sum(request_losses) / cores, + "shadow_request_loss_per_core_median": request_loss_summary["median"], + "shadow_request_loss_per_core_p95": request_loss_summary["p95"], + "shadow_miss_loss_sum": sum(miss_losses), + "shadow_miss_loss_per_core": sum(miss_losses) / cores, + "shadow_miss_loss_per_core_median": miss_loss_summary["median"], + "shadow_miss_loss_per_core_p95": miss_loss_summary["p95"], + "phase_observed_read_clear_ratio": ( + None if phase_requests == 0 else phase_misses / phase_requests + ), + "phase_icache_request_lower_bound_share_of_submit": + phase_requests / requests, + "phase_icache_miss_lower_bound_share_of_submit": + phase_misses / misses if misses != 0 else None, + } + ) per_run.append(row) aggregate: dict[str, Any] = { @@ -352,22 +1142,68 @@ def analyze(paths: Sequence[Path], miss_penalty_ns: float = 90.0) -> dict[str, A "scalar_busy_sum", "total_cycles_sum", ) + if schema_version == 4: + aggregate_fields += ( + "phase_calls_sum", + "phase_calls_per_core", + "phase_icache_requests_lower_bound_sum", + "phase_icache_requests_upper_bound_sum", + "phase_icache_misses_lower_bound_sum", + "phase_icache_misses_upper_bound_sum", + "phase_icache_requests_lower_bound_per_core", + "phase_icache_requests_upper_bound_per_core", + "phase_icache_misses_lower_bound_per_core", + "phase_icache_misses_upper_bound_per_core", + "phase_icache_requests_lower_bound_per_core_median", + "phase_icache_requests_lower_bound_per_core_p95", + "phase_icache_requests_upper_bound_per_core_median", + "phase_icache_requests_upper_bound_per_core_p95", + "phase_icache_misses_lower_bound_per_core_median", + "phase_icache_misses_lower_bound_per_core_p95", + "phase_icache_misses_upper_bound_per_core_median", + "phase_icache_misses_upper_bound_per_core_p95", + "shadow_request_loss_sum", + "shadow_request_loss_per_core", + "shadow_request_loss_per_core_median", + "shadow_request_loss_per_core_p95", + "shadow_miss_loss_sum", + "shadow_miss_loss_per_core", + "shadow_miss_loss_per_core_median", + "shadow_miss_loss_per_core_p95", + "phase_observed_read_clear_ratio", + "phase_icache_request_lower_bound_share_of_submit", + "phase_icache_miss_lower_bound_share_of_submit", + ) for group_name in GROUP_NAMES: - aggregate["groups"][group_name] = { - field: { - "median": _median(row["groups"][group_name][field] for row in per_run), - "min": min(row["groups"][group_name][field] for row in per_run), - "max": max(row["groups"][group_name][field] for row in per_run), - } - for field in aggregate_fields - } + aggregate["groups"][group_name] = {} + for field in aggregate_fields: + values = [ + row["groups"][group_name][field] + for row in per_run + if row["groups"][group_name][field] is not None + ] + aggregate["groups"][group_name][field] = ( + { + "median": _median(values), + "min": min(values), + "max": max(values), + } + if values + else {"median": None, "min": None, "max": None} + ) configuration = captures[0].data["configuration"] - return { - "schema": {"name": "pa_scheduler_pmu_multi_run_summary", "version": 1}, - "input_schema": {"name": SCHEMA_NAME, "version": SCHEMA_VERSION}, + fingerprint_fields = ( + SUBMIT_PMU_FINGERPRINT_FIELDS if schema_version == 4 else CONFIG_FINGERPRINT_FIELDS + ) + result: dict[str, Any] = { + "schema": { + "name": "pa_scheduler_pmu_multi_run_summary", + "version": 2 if schema_version == 4 else 1, + }, + "input_schema": {"name": SCHEMA_NAME, "version": schema_version}, "configuration": { - field: configuration.get(field) for field in CONFIG_FINGERPRINT_FIELDS + field: configuration.get(field) for field in fingerprint_fields }, "estimation": { "icache_miss_penalty_ns": miss_penalty_ns, @@ -388,17 +1224,48 @@ def analyze(paths: Sequence[Path], miss_penalty_ns: float = 90.0) -> dict[str, A "per_run": per_run, "aggregate": aggregate, } + if schema_version == 4: + phase_enabled = configuration["compiled_phase"] != "none" + result["phase_observation"] = { + "compiled_phase": configuration["compiled_phase"], + "compiled_phase_id": configuration["compiled_phase_id"], + "enabled": phase_enabled, + "primary_whole_authoritative": True, + "phase_share_is_same_elf_submit_only": True, + "phase_boundary_observer_perturbed": phase_enabled, + "phase_counter_pair_snapshot_atomic": False, + "phase_shadow_partition_exact_required": configuration[ + "phase_shadow_partition_exact_required" + ], + "phase_values_are_running_read_clear_lower_bounds": configuration[ + "phase_values_are_running_read_clear_lower_bounds" + ], + "phase_value_semantics": ( + "disabled_zero" + if not phase_enabled + else "running_read_clear_lower_to_loss_adjusted_upper_bound" + ), + "cross_phase_elf_sums_valid": False, + } + return result def _print_text(result: dict[str, Any]) -> None: configuration = result["configuration"] workload = configuration.get("winner_workload") or {} counts = workload.get("counts") or {} + build_phase = "" + if result["input_schema"]["version"] == 4: + build_phase = ( + f" build={configuration.get('build_variant')}" + f" phase={configuration.get('compiled_phase')}" + ) print( "[CONFIG] " f"window={configuration.get('pmu_window')} batches={configuration.get('batches')} " f"workers={configuration.get('workers')} workload={workload.get('mode')} " f"counts={counts.get('qk')},{counts.get('sf')},{counts.get('pv')},{counts.get('up')}" + f"{build_phase}" ) print("[VALIDATION] raw_to_summary=PASS accepted_restore=PASS same_configuration=PASS") print( @@ -431,6 +1298,43 @@ def _print_text(result: dict[str, Any]) -> None: f"AIV_core_miss_p95={aiv['icache_misses_per_core_p95']['median']:.3f} " f"AIV_miss_rate={aiv['icache_miss_rate']['median'] * 100:.4f}%" ) + if result["input_schema"]["version"] == 4: + phase = result["phase_observation"] + if not phase["enabled"]: + print( + "[PHASE-BOUNDS] selected=none status=DISABLED " + "request_per_core=0..0 miss_per_core=0..0 shadow_loss_per_core=request:0,miss:0 " + "shadow_partition=EXACT_DISABLED" + ) + else: + print( + "[PHASE-BOUNDS] " + f"selected={phase['compiled_phase']} semantics=RUNNING_READ_CLEAR_BOUNDS " + f"AIC_request_per_core=" + f"{aic['phase_icache_requests_lower_bound_per_core']['median']:.3f}.." + f"{aic['phase_icache_requests_upper_bound_per_core']['median']:.3f} " + f"AIC_miss_per_core=" + f"{aic['phase_icache_misses_lower_bound_per_core']['median']:.3f}.." + f"{aic['phase_icache_misses_upper_bound_per_core']['median']:.3f} " + f"AIC_shadow_loss_per_core=request:{aic['shadow_request_loss_per_core']['median']:.3f}," + f"miss:{aic['shadow_miss_loss_per_core']['median']:.3f} " + f"AIV_request_per_core=" + f"{aiv['phase_icache_requests_lower_bound_per_core']['median']:.3f}.." + f"{aiv['phase_icache_requests_upper_bound_per_core']['median']:.3f} " + f"AIV_miss_per_core=" + f"{aiv['phase_icache_misses_lower_bound_per_core']['median']:.3f}.." + f"{aiv['phase_icache_misses_upper_bound_per_core']['median']:.3f} " + f"AIV_shadow_loss_per_core=request:{aiv['shadow_request_loss_per_core']['median']:.3f}," + f"miss:{aiv['shadow_miss_loss_per_core']['median']:.3f} " + "shadow_partition=BOUNDED_NOT_EXACT_REQUIRED" + ) + boundary_state = "PERTURBED" if phase["enabled"] else "DISABLED" + pair_state = "FALSE" if phase["enabled"] else "NOT_APPLICABLE" + print( + "[PERTURBATION] primary_submit_whole=AUTHORITATIVE " + f"phase_boundary_observer={boundary_state} phase_counter_pair_atomic={pair_state} " + "cross_phase_elf_sum=INVALID" + ) print( "[SERIAL-EQUIVALENT] " f"penalty={result['estimation']['icache_miss_penalty_ns']:.3f}ns/miss " @@ -446,7 +1350,9 @@ def _print_text(result: dict[str, Any]) -> None: def main(argv: Sequence[str] | None = None) -> int: parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("inputs", nargs="+", type=Path, help="schema-v3 PMU JSON sidecars") + parser.add_argument( + "inputs", nargs="+", type=Path, help="历史 schema-v3 或 submit-pmu schema-v4 JSON sidecars" + ) parser.add_argument( "--icache-miss-ns", type=float, diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index 238e2aeb81..fba024cc65 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -21,6 +21,8 @@ Usage: ./run.sh run ccec|ascendc|cpu|all [benchmark options] ./run.sh smoke ccec|ascendc|cpu|all [--device N] ./run.sh swimlane ccec|ascendc|cpu|all [benchmark options] + ./run.sh build-submit-pmu ccec none|claim + ./run.sh submit-pmu ccec none|claim [benchmark options] Benchmark options: --device N @@ -60,6 +62,11 @@ action, --trace-atomics still requires swimlane tracing; add --pmu-json requires --runs 1 and a non-off PMU window. PMU probe options are CCEC-only and cannot target all. +The submit-pmu action is a separate CCEC-only build. It fixes one PMU-only run +covering the complete Submit window. phase=none performs no internal snapshots; +phase=claim reports running read-clear lower/loss-adjusted upper bounds for one +compile-time phase while CNT6/7 retain the authoritative whole-window counters. + The swimlane action performs exactly one run and writes both the raw capture and merged Perfetto JSON below this directory's outputs/ folder. It rejects --runs, --swimlane-json, and --no-swimlane because those are managed by the action. @@ -127,6 +134,119 @@ run_backend() { esac } +validate_submit_pmu_phase() { + case "$1" in + none|claim) ;; + *) + echo "Unknown submit-pmu phase: $1 (expected none|claim)" >&2 + exit 1 + ;; + esac +} + +submit_pmu_artifact_failure() { + local phase="$1" + local reason="$2" + echo "Invalid submit-pmu artifact set for phase '$phase': $reason" >&2 + echo "Run: $0 build-submit-pmu ccec $phase" >&2 + return 1 +} + +validate_submit_pmu_artifacts() { + local phase="$1" + local build_dir="$2" + local phase_id + case "$phase" in + none) phase_id=0 ;; + claim) phase_id=1 ;; + *) submit_pmu_artifact_failure "$phase" "unsupported phase"; return 1 ;; + esac + + local manifest_name="submit_pmu_artifacts.manifest" + local manifest="$build_dir/$manifest_name" + local artifacts=( + pa_scheduler_host + pa_scheduler_kernel.o + libpa_scheduler_pmu_owner_aicpu.so + libpa_scheduler_pmu_owner_dispatcher.so + ) + if [[ ! -x "$build_dir/${artifacts[0]}" ]]; then + submit_pmu_artifact_failure "$phase" "host runner is missing, empty, or not executable" + return 1 + fi + local artifact + for artifact in "${artifacts[@]:1}"; do + if [[ ! -s "$build_dir/$artifact" ]]; then + submit_pmu_artifact_failure "$phase" "artifact is missing or empty: $artifact" + return 1 + fi + done + if [[ ! -s "$manifest" ]]; then + submit_pmu_artifact_failure "$phase" "ready manifest is missing or empty" + return 1 + fi + if ! command -v sha256sum >/dev/null 2>&1; then + submit_pmu_artifact_failure "$phase" "sha256sum is unavailable" + return 1 + fi + + # manifest 固定为四行身份头和四行校验和;既检查 phase/variant,也拒绝 + # 漏项、增项、绝对路径或重复文件,避免 sha256sum 只校验到一个子集。 + local manifest_lines=() + mapfile -t manifest_lines < "$manifest" + if [[ ${#manifest_lines[@]} -ne 8 || + "${manifest_lines[0]}" != "# schema=pa_scheduler_submit_pmu_artifacts/v1" || + "${manifest_lines[1]}" != "# variant=submit-pmu" || + "${manifest_lines[2]}" != "# phase=$phase" || + "${manifest_lines[3]}" != "# phase_id=$phase_id" ]]; then + submit_pmu_artifact_failure "$phase" "manifest schema, variant, or phase metadata does not match" + return 1 + fi + local index digest filename extra + for index in "${!artifacts[@]}"; do + digest="" + filename="" + extra="" + read -r digest filename extra <<< "${manifest_lines[index + 4]}" + if [[ ! "$digest" =~ ^[[:xdigit:]]{64}$ || + "$filename" != "${artifacts[index]}" || -n "$extra" ]]; then + submit_pmu_artifact_failure "$phase" "manifest checksum entry $((index + 1)) is malformed or out of order" + return 1 + fi + done + if ! (cd "$build_dir" && sha256sum --check --strict --status "$manifest_name"); then + submit_pmu_artifact_failure "$phase" "one or more artifact SHA256 values do not match" + return 1 + fi + echo "[CHECK] submit-pmu artifact manifest verified: $manifest" +} + +reject_managed_submit_pmu_options() { + # 这些参数定义诊断 ELF 与窗口边界,必须由 action 独占;允许用户只传 + # device/batches/workload 和可选的 --pmu-json。 + for argument in "$@"; do + case "$argument" in + --kernel|--kernel=*|--runs|--runs=*|--pmu-window|--pmu-window=*|\ + --no-swimlane|--profile-phases|--trace-atomics|--analyze-swimlane|\ + --swimlane-json|--swimlane-json=*|--pmu-scalar-nops|--pmu-scalar-nops=*|\ + --pmu-icache-trials|--pmu-icache-trials=*) + echo "The submit-pmu action manages or forbids $argument." >&2 + exit 1 + ;; + esac + done +} + +run_submit_pmu() { + local phase="$1" + shift + local build_dir="$SCRIPT_DIR/build/ccec/submit-pmu/$phase" + local host="$build_dir/pa_scheduler_host" + local kernel="$build_dir/pa_scheduler_kernel.o" + validate_submit_pmu_artifacts "$phase" "$build_dir" + "$host" --kernel "$kernel" --runs 1 --no-swimlane --pmu-window submit-all "$@" +} + reject_managed_swimlane_options() { # swimlane action 必须独占轮数、raw 路径和 trace 开关,才能保证每个 # backend 恰好对应一对 raw/merged 文件且不会发生多轮覆盖。 @@ -255,6 +375,26 @@ case "$ACTION" in done echo "[SWIMLANE] output_root=$OUTPUT_ROOT" ;; + build-submit-pmu) + if [[ "$BACKEND" != "ccec" || $# -ne 1 ]]; then + echo "Usage: $0 build-submit-pmu ccec none|claim" >&2 + exit 1 + fi + PHASE="$1" + validate_submit_pmu_phase "$PHASE" + "$SCRIPT_DIR/ccec/build.sh" submit-pmu "$PHASE" + ;; + submit-pmu) + if [[ "$BACKEND" != "ccec" || $# -lt 1 ]]; then + echo "Usage: $0 submit-pmu ccec none|claim [benchmark options]" >&2 + exit 1 + fi + PHASE="$1" + shift + validate_submit_pmu_phase "$PHASE" + reject_managed_submit_pmu_options "$@" + run_submit_pmu "$PHASE" "$@" + ;; *) # 未知 action 不尝试推断用户意图,也不会触发任何构建或设备操作。 echo "Unknown action: $ACTION" >&2 diff --git a/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py b/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py index 94d699522e..ea0d813706 100644 --- a/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py +++ b/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py @@ -12,7 +12,6 @@ from __future__ import annotations -import copy import json import math import tempfile @@ -21,9 +20,39 @@ from typing import Any, Sequence try: - from .pmu_sidecar_analyzer import METRIC_NAMES, analyze, load_capture + from .pmu_sidecar_analyzer import ( + A5_AIC_PER_DIE, + A5_AIC_WORKERS, + A5_AIV_WORKERS, + A5_OWNER_MAGIC, + A5_OWNER_VERSION, + A5_PHYSICAL_SUBCORES, + A5_SUBCORES_PER_DIE, + A5_WORKERS, + METRIC_NAMES, + PHASE_STATUS_REQUIRED_MASK, + SUBMIT_PMU_METRIC_NAMES, + TASKS_PER_BATCH, + analyze, + load_capture, + ) except ImportError: - from pmu_sidecar_analyzer import METRIC_NAMES, analyze, load_capture + from pmu_sidecar_analyzer import ( + A5_AIC_PER_DIE, + A5_AIC_WORKERS, + A5_AIV_WORKERS, + A5_OWNER_MAGIC, + A5_OWNER_VERSION, + A5_PHYSICAL_SUBCORES, + A5_SUBCORES_PER_DIE, + A5_WORKERS, + METRIC_NAMES, + PHASE_STATUS_REQUIRED_MASK, + SUBMIT_PMU_METRIC_NAMES, + TASKS_PER_BATCH, + analyze, + load_capture, + ) def _p95(values: Sequence[int]) -> int: @@ -31,13 +60,15 @@ def _p95(values: Sequence[int]) -> int: return ordered[math.ceil(0.95 * len(ordered)) - 1] -def _summary(records: list[dict[str, Any]]) -> dict[str, Any]: +def _summary_for_metrics( + records: list[dict[str, Any]], metric_names: Sequence[str] +) -> dict[str, Any]: result: dict[str, Any] = { "cores": len(records), "active_cores": len(records), "trusted_cores": len(records), } - for metric in METRIC_NAMES: + for metric in metric_names: values = [record[metric] for record in records] result[metric] = { "sum": sum(values), @@ -54,6 +85,43 @@ def _summary(records: list[dict[str, Any]]) -> dict[str, Any]: return result +def _summary(records: list[dict[str, Any]]) -> dict[str, Any]: + return _summary_for_metrics(records, METRIC_NAMES) + + +def _submit_pmu_summary(records: list[dict[str, Any]]) -> dict[str, Any]: + result = _summary_for_metrics(records, SUBMIT_PMU_METRIC_NAMES) + phase_requests = result["phase_icache_requests"]["sum"] + phase_misses = result["phase_icache_misses"]["sum"] + result["phase_observed_read_clear_ratio"] = ( + None if phase_requests == 0 else phase_misses / phase_requests + ) + return result + + +def _submit_pmu_physical_id(worker_id: int) -> int: + """构造与 host mixed launch 相同的 32 组物理 1:2 triplet。""" + + if worker_id < A5_AIC_WORKERS: + block = worker_id + die_base = 0 if block < A5_AIC_WORKERS // 2 else A5_SUBCORES_PER_DIE + return die_base + block % (A5_AIC_WORKERS // 2) + vector_id = worker_id - A5_AIC_WORKERS + block = vector_id // 2 + aic_id = _submit_pmu_physical_id(block) + die_base = (aic_id // A5_SUBCORES_PER_DIE) * A5_SUBCORES_PER_DIE + local_aic = aic_id % A5_SUBCORES_PER_DIE + return die_base + A5_AIC_PER_DIE + local_aic * 2 + vector_id % 2 + + +def _submit_pmu_bitmap_words() -> list[int]: + words = [0, 0, 0, 0] + for worker_id in range(A5_WORKERS): + physical_id = _submit_pmu_physical_id(worker_id) + words[physical_id // 32] |= 1 << (physical_id % 32) + return words + + def _capture(offset: int = 0, window: str = "submit-all") -> dict[str, Any]: records: list[dict[str, Any]] = [] for worker_id, role in enumerate(("aic", "aiv", "aiv")): @@ -132,6 +200,207 @@ def _capture(offset: int = 0, window: str = "submit-all") -> dict[str, Any]: } +def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any]: + phase_ids = {"none": 0, "claim": 1} + phase_id = phase_ids[phase] + batches = 2 + calls_per_worker = 0 if phase == "none" else batches * TASKS_PER_BATCH + records: list[dict[str, Any]] = [] + for worker_id in range(A5_WORKERS): + role = "aic" if worker_id < A5_AIC_WORKERS else "aiv" + vector_id = 0 if role == "aic" else worker_id - A5_AIC_WORKERS + block_id = worker_id if role == "aic" else vector_id // 2 + lane = 0 if role == "aic" else 1 + vector_id % 2 + base = 100 + worker_id * 10 + offset + primary_requests = 1000 + base + primary_misses = 100 + base // 10 + phase_requests = 0 if phase == "none" else 100 + worker_id * 10 + offset + phase_misses = 0 if phase == "none" else 10 + worker_id + offset // 10 + request_loss = 0 if phase == "none" else 1 + worker_id % 3 + miss_loss = 0 if phase == "none" else 1 + worker_id % 2 + shadow_requests = primary_requests - request_loss + shadow_misses = primary_misses - miss_loss + record: dict[str, Any] = { + "worker_id": worker_id, + "physical_core_id": _submit_pmu_physical_id(worker_id), + "role": role, + "block_id": block_id, + "lane": lane, + "trusted": True, + "physical_core_id_valid": True, + "selectors_match": True, + "owner_bitmap_member": True, + "worker_slot_exact": True, + "physical_role_matches": True, + "window_started": True, + "window_stopped": True, + "build_variant_id": 2, + "compiled_phase_id": phase_id, + "phase_status": PHASE_STATUS_REQUIRED_MASK | (0x30 if phase == "none" else 0), + "phase_calls": calls_per_worker, + "phase_begin_reads": calls_per_worker, + "phase_end_reads": calls_per_worker, + "primary_window_segments": 1, + "shadow_read_segments": 2 * calls_per_worker + 1, + "phase_icache_requests": phase_requests, + "phase_icache_misses": phase_misses, + "phase_icache_requests_upper_bound": phase_requests + request_loss, + "phase_icache_misses_upper_bound": phase_misses + miss_loss, + "shadow_whole_icache_requests": shadow_requests, + "shadow_whole_icache_misses": shadow_misses, + "shadow_matches_primary": request_loss == 0 and miss_loss == 0, + "shadow_not_greater_than_primary": True, + "shadow_request_loss": request_loss, + "shadow_miss_loss": miss_loss, + "phase_boundaries_balanced": True, + } + for metric_index, metric in enumerate(SUBMIT_PMU_METRIC_NAMES): + record.setdefault(metric, base + metric_index) + record["icache_requests"] = primary_requests + record["icache_misses"] = primary_misses + # setdefault 不覆盖上面按 phase 契约填写的五个扩展字段。 + records.append(record) + + groups = { + "all": records, + "aic": [record for record in records if record["role"] == "aic"], + "aiv": [record for record in records if record["role"] == "aiv"], + } + workers = len(records) + exact_records = sum(record["shadow_matches_primary"] for record in records) + bounded_records = sum(record["shadow_not_greater_than_primary"] for record in records) + request_losses = [record["shadow_request_loss"] for record in records] + miss_losses = [record["shadow_miss_loss"] for record in records] + return { + "schema": {"name": "pa_scheduler_pmu_phase_windows", "version": 4}, + "capture": { + "capture_id": f"submit-pmu-{phase}-{offset}", + "accepted": True, + "published_after_runtime_cleanup": True, + "runtime_cleanup_passed": True, + "owner_restore_passed": True, + }, + "configuration": { + "build_variant": "submit-pmu", + "build_variant_id": 2, + "compiled_phase": phase, + "compiled_phase_id": phase_id, + "device": 0, + "batches": batches, + "workers": workers, + "aic_workers": A5_AIC_WORKERS, + "aiv_workers": A5_AIV_WORKERS, + "pmu_window": "submit-all", + "primary_window_segments_per_record": 1, + "unavailable_metrics": ["mte3_busy"], + "submit_span_us": 5000.0 + offset, + "selectors": { + "cnt0_vector_busy": 0x501, + "cnt1_cube_busy": 0x301, + "cnt2_scalar_busy": 0x001, + "cnt3_mte1_busy": 0x701, + "cnt4_mte2_busy": 0x202, + "cnt5_shadow_icache_miss": 0x035, + "cnt6_primary_icache_request": 0x034, + "cnt7_primary_icache_miss": 0x035, + "cnt8_shadow_icache_request": 0x034, + "cnt9_unused": 0x000, + }, + "counter_width_bits": {"total": 64, "programmable": 32}, + "phase_timestamp_calls_present": False, + "phase_record_writes": False, + "profile_accumulation": False, + "trace_enabled": False, + "trace_atomics": False, + "atomic_trace": False, + "profile_phases": False, + "gate_start_stop_have_pipe_all_barriers": True, + "phase_boundary_observation_included": phase != "none", + "phase_counter_pair_snapshot_atomic": False, + "primary_counters_read_at_phase_boundaries": False, + "phase_shadow_partition_exact_required": phase == "none", + "phase_values_are_running_read_clear_lower_bounds": phase != "none", + "cross_phase_elf_sums_valid": False, + "winner_workload": { + "mode": "real-compute", + "counts": {"qk": 6, "sf": 28, "pv": 4, "up": 1}, + "unit": "complete_128x128_engine_pipeline_iteration", + }, + }, + "validation": { + "semantic_passed": True, + "pmu_passed": True, + "icache_measurement_valid": True, + "icache_miss_le_request": True, + "counter_below_risk_threshold": True, + "phase_measurement_valid": True, + "trusted_records": workers, + "unique_physical_core_ids": workers, + "owner_bitmap_member_records": workers, + "exact_worker_slot_records": workers, + "physical_role_match_records": workers, + "window_started_records": workers, + "window_stopped_records": workers, + "build_variant_match_records": workers, + "phase_id_match_records": workers, + "phase_status_trusted_records": workers, + "shadow_primary_match_records": exact_records, + "shadow_primary_bounded_records": bounded_records, + "shadow_request_abs_delta_sum": sum(request_losses), + "shadow_request_abs_delta_max": max(request_losses), + "shadow_request_signed_delta_sum": -sum(request_losses), + "shadow_miss_abs_delta_sum": sum(miss_losses), + "shadow_miss_abs_delta_max": max(miss_losses), + "shadow_miss_signed_delta_sum": -sum(miss_losses), + "phase_boundary_match_records": workers, + "phase_call_shape_match_records": workers, + "phase_calls": workers * calls_per_worker, + "expected_records": A5_WORKERS, + "expected_unique_core_ids": A5_WORKERS, + "expected_owner_bitmap_member_records": A5_WORKERS, + "expected_exact_worker_slot_records": A5_WORKERS, + "expected_physical_role_match_records": A5_WORKERS, + "mixed_triplet_matches": A5_AIC_WORKERS, + "expected_mixed_triplet_matches": A5_AIC_WORKERS, + "expected_window_records": A5_WORKERS, + }, + "owner": { + "mode": "main_aicpu_path_a", + "snapshot_phase": "after_configure_before_restore", + "control_magic": A5_OWNER_MAGIC, + "control_version": A5_OWNER_VERSION, + "configure_status": 0, + "configured_flag": 1, + "configured_bitmap_count": A5_WORKERS, + "expected": { + "total": A5_WORKERS, + "aic": A5_AIC_WORKERS, + "aiv": A5_AIV_WORKERS, + }, + "active": { + "total": A5_WORKERS, + "aic": A5_AIC_WORKERS, + "aiv": A5_AIV_WORKERS, + }, + "discovered": { + "total": A5_WORKERS, + "aic": A5_AIC_WORKERS, + "aiv": A5_AIV_WORKERS, + }, + "physical_slots_scanned": A5_PHYSICAL_SUBCORES, + "skipped_physical_slots": A5_PHYSICAL_SUBCORES - A5_WORKERS, + "configured_bitmap_word_order": "least_significant_physical_ids_first", + "configured_bitmap_words": _submit_pmu_bitmap_words(), + "configured_complete_mixed_triplets": A5_AIC_WORKERS, + "expected_complete_mixed_triplets": A5_AIC_WORKERS, + "configured_broken_mixed_triplets": 0, + "restore_passed": True, + }, + "records": records, + "summary": {name: _submit_pmu_summary(group) for name, group in groups.items()}, + } + + class PmuSidecarAnalyzerTest(unittest.TestCase): def _write(self, directory: str, name: str, capture: dict[str, Any]) -> Path: path = Path(directory) / name @@ -216,6 +485,450 @@ def test_miss_greater_than_request_is_rejected_before_summary_use(self) -> None: with self.assertRaisesRegex(ValueError, "miss > request"): load_capture(path) + def test_submit_pmu_v4_claim_is_recomputed_and_aggregated(self) -> None: + with tempfile.TemporaryDirectory() as directory: + first = self._write(directory, "claim-first.json", _submit_pmu_capture(0, "claim")) + second = self._write(directory, "claim-second.json", _submit_pmu_capture(10, "claim")) + result = analyze([first, second]) + + self.assertEqual(result["input_schema"]["version"], 4) + self.assertEqual(result["schema"]["version"], 2) + self.assertEqual(result["phase_observation"]["compiled_phase"], "claim") + self.assertTrue(result["phase_observation"]["enabled"]) + self.assertFalse(result["phase_observation"]["cross_phase_elf_sums_valid"]) + self.assertEqual( + result["phase_observation"]["phase_value_semantics"], + "running_read_clear_lower_to_loss_adjusted_upper_bound", + ) + expected_aiv_phase_misses = [ + _submit_pmu_capture(offset, "claim")["summary"]["aiv"][ + "phase_icache_misses" + ]["sum"] + / A5_AIV_WORKERS + for offset in (0, 10) + ] + self.assertEqual( + result["aggregate"]["groups"]["aiv"][ + "phase_icache_misses_lower_bound_per_core" + ]["median"], + sum(expected_aiv_phase_misses) / 2, + ) + self.assertGreater( + result["aggregate"]["groups"]["aiv"][ + "phase_icache_miss_lower_bound_share_of_submit" + ]["median"], + 0, + ) + claim_aiv = result["aggregate"]["groups"]["aiv"] + self.assertGreater( + claim_aiv["phase_icache_requests_upper_bound_per_core"]["median"], + claim_aiv["phase_icache_requests_lower_bound_per_core"]["median"], + ) + self.assertGreater(claim_aiv["shadow_request_loss_per_core"]["median"], 0) + for row in result["per_run"]: + for group_name in ("all", "aic", "aiv"): + group = row["groups"][group_name] + self.assertEqual( + group["phase_icache_requests_upper_bound_sum"] + - group["phase_icache_requests_lower_bound_sum"], + group["shadow_request_loss_sum"], + ) + self.assertEqual( + group["phase_icache_misses_upper_bound_sum"] + - group["phase_icache_misses_lower_bound_sum"], + group["shadow_miss_loss_sum"], + ) + self.assertLessEqual( + group["phase_icache_requests_lower_bound_per_core_median"], + group["phase_icache_requests_upper_bound_per_core_median"], + ) + self.assertLessEqual( + group["phase_icache_misses_lower_bound_per_core_p95"], + group["phase_icache_misses_upper_bound_per_core_p95"], + ) + self.assertLess( + _submit_pmu_capture()["validation"]["shadow_request_signed_delta_sum"], 0 + ) + + def test_submit_pmu_v4_none_has_zero_disabled_phase(self) -> None: + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "none.json", _submit_pmu_capture(0, "none")) + result = analyze([path]) + + self.assertFalse(result["phase_observation"]["enabled"]) + phase = result["aggregate"]["groups"]["aiv"] + self.assertEqual(phase["phase_calls_sum"]["median"], 0) + self.assertEqual(phase["phase_icache_misses_lower_bound_per_core"]["median"], 0) + self.assertIsNone(phase["phase_observed_read_clear_ratio"]["median"]) + self.assertEqual( + phase["phase_icache_miss_lower_bound_share_of_submit"]["median"], 0 + ) + self.assertEqual( + phase["phase_icache_requests_lower_bound_per_core"]["median"], + phase["phase_icache_requests_upper_bound_per_core"]["median"], + ) + self.assertEqual(phase["shadow_request_loss_sum"]["median"], 0) + + def test_submit_pmu_v4_requires_fixed_a5_topology(self) -> None: + capture = _submit_pmu_capture() + # 仍保持 workers=aic+aiv,证明拒绝原因是 schema-v4 的固定 A5 拓扑, + # 不是原有的自报计数加和检查。 + capture["configuration"]["aic_workers"] = A5_AIC_WORKERS - 1 + capture["configuration"]["aiv_workers"] = A5_AIV_WORKERS + 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "wrong-topology.json", capture) + with self.assertRaisesRegex(ValueError, "fixed 96/32/64 A5 topology"): + load_capture(path) + + def test_submit_pmu_v4_logical_worker_triplet_is_recomputed(self) -> None: + mutations = ( + (0, "worker_id", A5_WORKERS, "exact worker slot"), + (0, "role", "aiv", "logical worker topology"), + (0, "block_id", 1, "mixed block"), + (A5_AIC_WORKERS, "lane", 2, "mixed lane"), + (0, "physical_core_id_valid", False, "physical_core_id_valid"), + ) + for record_index, field, value, message in mutations: + with self.subTest(field=field): + capture = _submit_pmu_capture() + capture["records"][record_index][field] = value + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, f"wrong-{field}.json", capture) + with self.assertRaisesRegex(ValueError, message): + load_capture(path) + + def test_submit_pmu_v4_physical_range_and_triplets_are_recomputed(self) -> None: + capture = _submit_pmu_capture() + capture["records"][0]["physical_core_id"] = A5_PHYSICAL_SUBCORES + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "out-of-range-core.json", capture) + with self.assertRaisesRegex(ValueError, "outside the 108-slot topology"): + load_capture(path) + + capture = _submit_pmu_capture() + # 交换两个 block 的 AIV0:ID 仍唯一、仍属于 owner、物理角色仍是 AIV, + # 只有逐 block 的 1:2 关系被破坏。 + first = A5_AIC_WORKERS + second = A5_AIC_WORKERS + 2 + capture["records"][first]["physical_core_id"], capture["records"][second][ + "physical_core_id" + ] = ( + capture["records"][second]["physical_core_id"], + capture["records"][first]["physical_core_id"], + ) + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "crossed-aiv-triplets.json", capture) + with self.assertRaisesRegex(ValueError, "mixed block 0"): + load_capture(path) + + def test_submit_pmu_v4_owner_control_fields_are_rechecked(self) -> None: + mutations = ( + (("control_magic",), 0, "control_magic mismatch"), + (("control_version",), A5_OWNER_VERSION + 1, "control_version mismatch"), + (("configure_status",), 1, "configure_status is not success"), + (("configured_flag",), 0, "configured_flag is not one"), + (("expected", "total"), A5_WORKERS - 1, "owner.expected.total"), + (("active", "aic"), A5_AIC_WORKERS - 1, "owner.active.aic"), + (("discovered", "aiv"), A5_AIV_WORKERS - 1, "owner.discovered.aiv"), + (("physical_slots_scanned",), A5_PHYSICAL_SUBCORES - 1, "physical_slots_scanned"), + (("skipped_physical_slots",), 11, "skipped_physical_slots"), + (("configured_bitmap_count",), A5_WORKERS - 1, "bitmap count"), + (("configured_complete_mixed_triplets",), A5_AIC_WORKERS - 1, "complete mixed triplets"), + (("configured_broken_mixed_triplets",), 1, "broken mixed triplet"), + (("restore_passed",), False, "owner.restore_passed"), + ) + for keys, value, message in mutations: + with self.subTest(field=".".join(keys)): + capture = _submit_pmu_capture() + target = capture["owner"] + for key in keys[:-1]: + target = target[key] + target[keys[-1]] = value + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, f"bad-owner-{keys[-1]}.json", capture) + with self.assertRaisesRegex(ValueError, message): + load_capture(path) + + def test_submit_pmu_v4_owner_object_and_bitmap_are_rechecked(self) -> None: + capture = _submit_pmu_capture() + del capture["owner"] + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "missing-owner.json", capture) + with self.assertRaisesRegex(ValueError, "schema-v4 owner must be an object"): + load_capture(path) + + capture = _submit_pmu_capture() + capture["owner"]["configured_bitmap_words"][3] |= 1 << 31 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "owner-high-bit.json", capture) + with self.assertRaisesRegex(ValueError, "outside the 108 physical slots"): + load_capture(path) + + capture = _submit_pmu_capture() + words = capture["owner"]["configured_bitmap_words"] + # 用另一个完整 triplet 替换 block15 对应的三个位。bitmap 本身仍是 + # 96/32/64 且 32 组完整 triplet,但不再等于 worker raw 的物理 ID 集合。 + for physical_id in (15, 48, 49): + words[physical_id // 32] &= ~(1 << (physical_id % 32)) + for physical_id in (16, 50, 51): + words[physical_id // 32] |= 1 << (physical_id % 32) + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "owner-record-set-mismatch.json", capture) + with self.assertRaisesRegex(ValueError, "absent from the owner bitmap"): + load_capture(path) + + def test_submit_pmu_v4_host_triplet_count_is_not_blindly_trusted(self) -> None: + capture = _submit_pmu_capture() + capture["validation"]["mixed_triplet_matches"] -= 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "host-triplet-count.json", capture) + with self.assertRaisesRegex(ValueError, "mixed_triplet_matches"): + load_capture(path) + + def test_submit_pmu_claim_calls_are_exact_per_worker(self) -> None: + capture = _submit_pmu_capture() + first = capture["records"][0] + second = capture["records"][1] + first["phase_calls"] -= 1 + first["phase_begin_reads"] -= 1 + first["phase_end_reads"] -= 1 + first["shadow_read_segments"] -= 2 + second["phase_calls"] += 1 + second["phase_begin_reads"] += 1 + second["phase_end_reads"] += 1 + second["shadow_read_segments"] += 2 + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), + "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), + } + # 全局 calls、每条 begin/end 与 shadow segment 都保持闭合,只有逐核 + # claim 调用契约被破坏。 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "redistributed-claim-calls.json", capture) + with self.assertRaisesRegex(ValueError, "phase_calls does not match"): + load_capture(path) + + def test_submit_pmu_none_rejects_nonzero_phase_counters(self) -> None: + capture = _submit_pmu_capture(phase="none") + capture["records"][0]["phase_icache_requests"] = 1 + capture["records"][0]["phase_icache_requests_upper_bound"] = 1 + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), + "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), + } + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "none-nonzero-phase.json", capture) + with self.assertRaisesRegex(ValueError, "phase=none must have zero"): + load_capture(path) + + def test_submit_pmu_shadow_greater_than_primary_is_rejected_from_raw(self) -> None: + capture = _submit_pmu_capture() + record = capture["records"][0] + record["shadow_whole_icache_requests"] = record["icache_requests"] + 1 + record["shadow_not_greater_than_primary"] = False + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "shadow-greater-than-primary.json", capture) + with self.assertRaisesRegex(ValueError, "shadow whole exceeds"): + load_capture(path) + + def test_submit_pmu_shadow_booleans_loss_and_upper_are_recomputed(self) -> None: + mutations = ( + ("shadow_matches_primary", True, "shadow_matches_primary disagrees"), + ("shadow_not_greater_than_primary", False, "shadow_not_greater_than_primary disagrees"), + ("shadow_request_loss", 999, "shadow_request_loss disagrees"), + ( + "phase_icache_requests_upper_bound", + 999, + "upper_bound is not lower plus shadow loss", + ), + ("shadow_miss_loss", 999, "shadow_miss_loss disagrees"), + ( + "phase_icache_misses_upper_bound", + 999, + "upper_bound is not lower plus shadow loss", + ), + ) + for field, value, message in mutations: + with self.subTest(field=field): + capture = _submit_pmu_capture() + capture["records"][0][field] = value + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, f"fake-{field}.json", capture) + with self.assertRaisesRegex(ValueError, message): + load_capture(path) + + def test_submit_pmu_host_shadow_counts_and_deltas_are_recomputed(self) -> None: + fields = ( + "shadow_primary_match_records", + "shadow_primary_bounded_records", + "shadow_request_abs_delta_sum", + "shadow_request_abs_delta_max", + "shadow_request_signed_delta_sum", + "shadow_miss_abs_delta_sum", + "shadow_miss_abs_delta_max", + "shadow_miss_signed_delta_sum", + ) + for field in fields: + with self.subTest(field=field): + capture = _submit_pmu_capture() + capture["validation"][field] += 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, f"tampered-{field}.json", capture) + with self.assertRaisesRegex(ValueError, field): + load_capture(path) + + def test_submit_pmu_unbalanced_phase_boundary_is_rejected_from_raw(self) -> None: + capture = _submit_pmu_capture() + capture["records"][0]["phase_end_reads"] -= 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "boundary-mismatch.json", capture) + with self.assertRaisesRegex(ValueError, "boundaries do not match calls"): + load_capture(path) + + capture = _submit_pmu_capture() + capture["records"][0]["shadow_read_segments"] -= 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "segment-mismatch.json", capture) + with self.assertRaisesRegex(ValueError, "shadow_read_segments does not match"): + load_capture(path) + + def test_submit_pmu_record_build_and_phase_ids_are_rechecked(self) -> None: + for field, value, message in ( + ("build_variant_id", 1, "build_variant_id mismatch"), + ("compiled_phase_id", 0, "compiled_phase_id mismatch"), + ): + with self.subTest(field=field): + capture = _submit_pmu_capture() + capture["records"][0][field] = value + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, f"bad-{field}.json", capture) + with self.assertRaisesRegex(ValueError, message): + load_capture(path) + + def test_submit_pmu_configuration_and_host_gate_are_rechecked(self) -> None: + cases = ( + ("build_variant", "swimlane", "configuration.build_variant"), + ("compiled_phase_id", 0, "compiled phase name/id mismatch"), + ( + "phase_boundary_observation_included", + False, + "phase_boundary_observation_included does not match", + ), + ( + "phase_counter_pair_snapshot_atomic", + True, + "phase_counter_pair_snapshot_atomic is not false", + ), + ( + "phase_shadow_partition_exact_required", + True, + "phase_shadow_partition_exact_required does not match", + ), + ( + "phase_values_are_running_read_clear_lower_bounds", + False, + "phase_values_are_running_read_clear_lower_bounds does not match", + ), + ) + for field, value, message in cases: + with self.subTest(field=field): + capture = _submit_pmu_capture() + capture["configuration"][field] = value + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, f"bad-config-{field}.json", capture) + with self.assertRaisesRegex(ValueError, message): + load_capture(path) + + capture = _submit_pmu_capture() + capture["configuration"]["selectors"]["cnt5_shadow_icache_miss"] = 0x203 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "bad-shadow-selector.json", capture) + with self.assertRaisesRegex(ValueError, "cnt5_shadow_icache_miss"): + load_capture(path) + + capture = _submit_pmu_capture() + capture["validation"]["phase_boundary_match_records"] -= 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "bad-host-phase-gate.json", capture) + with self.assertRaisesRegex(ValueError, "phase_boundary_match_records is incomplete"): + load_capture(path) + + capture = _submit_pmu_capture() + capture["validation"]["phase_call_shape_match_records"] -= 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "bad-host-phase-call-shape.json", capture) + with self.assertRaisesRegex( + ValueError, "phase_call_shape_match_records is incomplete" + ): + load_capture(path) + + def test_submit_pmu_phase_counter_order_is_rechecked(self) -> None: + # 两个 ld_dev 不是原子配对快照;边界漂移可使局部 miss 略大于 request, + # 只要两者分别不超过各自的 Submit whole 就仍是合法 raw 观察。 + capture = _submit_pmu_capture() + capture["records"][0]["phase_icache_misses"] = ( + capture["records"][0]["phase_icache_requests"] + 5 + ) + capture["records"][0]["phase_icache_misses_upper_bound"] = ( + capture["records"][0]["phase_icache_misses"] + + capture["records"][0]["shadow_miss_loss"] + ) + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary( + [record for record in records if record["role"] == "aic"] + ), + "aiv": _submit_pmu_summary( + [record for record in records if record["role"] == "aiv"] + ), + } + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "phase-miss-above-request.json", capture) + load_capture(path) + + mutations = ( + ("phase_icache_misses", 200, "phase counters exceed"), + ("phase_icache_requests", 2000, "phase counters exceed"), + ) + for field, value, message in mutations: + with self.subTest(field=field): + capture = _submit_pmu_capture() + capture["records"][0][field] = value + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, f"bad-{field}.json", capture) + with self.assertRaisesRegex(ValueError, message): + load_capture(path) + + def test_submit_pmu_phase_summary_tampering_is_rejected(self) -> None: + capture = _submit_pmu_capture() + capture["summary"]["aiv"]["phase_icache_misses"]["sum"] += 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "phase-summary-tampered.json", capture) + with self.assertRaisesRegex(ValueError, "raw summary mismatch"): + load_capture(path) + + capture = _submit_pmu_capture() + capture["summary"]["aiv"]["phase_observed_read_clear_ratio"] += 0.01 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "phase-ratio-tampered.json", capture) + with self.assertRaisesRegex(ValueError, "phase_observed_read_clear_ratio"): + load_capture(path) + + def test_different_schema_or_submit_pmu_phase_cannot_be_merged(self) -> None: + with tempfile.TemporaryDirectory() as directory: + legacy = self._write(directory, "legacy.json", _capture()) + claim = self._write(directory, "claim.json", _submit_pmu_capture(0, "claim")) + none = self._write(directory, "none.json", _submit_pmu_capture(0, "none")) + with self.assertRaisesRegex(ValueError, "input schema differs"): + analyze([legacy, claim]) + with self.assertRaisesRegex(ValueError, "observation configuration differs"): + analyze([claim, none]) + if __name__ == "__main__": unittest.main() From 8430f4184561d9b613b988ce57a24e6f185837e7 Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sat, 18 Jul 2026 19:55:34 +0000 Subject: [PATCH 025/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E5=A2=9E?= =?UTF-8?q?=E5=8A=A0EfDrain=20Submit=20PMU=E7=8B=AC=E7=AB=8B=E5=BD=92?= =?UTF-8?q?=E5=9B=A0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 在Submit开头唯一的EfDrain调用点划定编译期PMU边界,按每核5B次调用闭合host、device与离线分析器门禁。 完成b1/b256真实计算负载A5验证:调度语义、真实计算输出、96核拓扑及观测边界全部通过;局部read-clear仅按同一ELF下界与误差包络解释。同步完善构建入口、负例测试和中文使用文档。 --- tests/atomic_probe/icache_miss_usage_guide.md | 30 ++++++++- ...05\345\206\265\345\210\206\346\236\220.md" | 36 ++++++++++- ...77\347\224\250\346\214\207\345\215\227.md" | 15 +++-- tests/atomic_probe/pa_scheduler/ccec/build.sh | 7 ++- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 34 +++++++--- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 8 ++- .../pa_scheduler/ccec/pmu_probe.h | 2 + .../pa_scheduler/common/pa_model.h | 6 +- .../pa_scheduler/common/pa_scheduler_core.h | 4 ++ .../pa_scheduler/pmu_sidecar_analyzer.py | 8 ++- tests/atomic_probe/pa_scheduler/run.sh | 17 ++--- .../pa_scheduler/test_pmu_sidecar_analyzer.py | 63 ++++++++++++------- 12 files changed, 172 insertions(+), 58 deletions(-) diff --git a/tests/atomic_probe/icache_miss_usage_guide.md b/tests/atomic_probe/icache_miss_usage_guide.md index 7d97d4b49c..f67bb7c04f 100644 --- a/tests/atomic_probe/icache_miss_usage_guide.md +++ b/tests/atomic_probe/icache_miss_usage_guide.md @@ -43,12 +43,13 @@ atomic wrapper、ClockBaseline、runtime phase-profile 和旧 cold/warm 冲刷 ## 3. `none` 与 `claim` 如何选择 -当前只有两个编译期 phase: +当前已验证三个编译期 phase: | phase | 边界 | 优先用途 | | --- | --- | --- | | `none` | 完整 Submit 中不读局部 shadow counter | 回答完整 Submit 的 AIC/AIV 每核 request/miss;这是默认选择 | | `claim` | 每次 `Claim()` 调用前后读局部 shadow counter | 当 `none` 已证明 miss 值得追踪时,试验 Claim 的 running read-clear 下界/上界归因链路 | +| `efdrain` | 每次 Submit 开头唯一的 `DrainReady(...EfDrain...)` 前后 | 观察 opportunistic drain;不混入 RingBackpressure 或 FinalDrain | `claim` 的 begin/end 读取本身会执行 scalar 指令、占用取指并改变多核 时序。因此: @@ -89,11 +90,18 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" ./run.sh build-submit-pmu ccec claim ``` +需要 EfDrain 局部归因时独立构建: + +```bash +./run.sh build-submit-pmu ccec efdrain +``` + 产物分别位于: ```text pa_scheduler/build/ccec/submit-pmu/none/ pa_scheduler/build/ccec/submit-pmu/claim/ +pa_scheduler/build/ccec/submit-pmu/efdrain/ ``` 每个 phase 目录中的 `pa_scheduler_host`、`pa_scheduler_kernel.o`、 @@ -145,6 +153,22 @@ mkdir -p "$OUT_CLAIM" --pmu-json "$OUT_CLAIM/run1.json" ``` +### 5.3 EfDrain 局部归因 + +```bash +OUT_EFDRAIN="./outputs/submit_pmu_efdrain_$(date -u +%Y%m%dT%H%M%SZ)" +mkdir -p "$OUT_EFDRAIN" + +./run.sh submit-pmu ccec efdrain \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT_EFDRAIN/run1.json" +``` + +`efdrain` 每核固定调用 `batches * 5` 次;b256 的 AIC/AIV/global calls +分别为 40,960/81,920/122,880。插点只位于 Submit 开头的 EfDrain 专属 +call-site;复用的 `DrainReady()` 函数体不插桩。 + `submit-pmu` action 已固定: ```text @@ -230,6 +254,8 @@ CNT8/CNT5 是顺序 `ld_dev`,不是同一时刻的原子配对快照,因此 - `none` 的 phase calls/begin/end/request/miss 全部为 0; - `claim` 的 begin/end/calls 逐核平衡,每核 calls 为 `batches * 5`, 全局 calls 为 `batches * 5 * 96`; +- `efdrain` 与 `claim` 具有相同的每核固定 calls 形状,但边界必须只覆盖 + Submit 开头的 EfDrain call-site; - phase request/miss 分别不超过对应 shadow/primary,且可编程 counter 低于当前 25% 保守风险阈值。 @@ -348,7 +374,7 @@ core-work 等效总量,不是端到端 Submit 总损失。要测真正暴露 新 phase 不能只增加一个 CLI 字符串。最小完整修改包括: 1. `pa_scheduler/common/pa_model.h`:在 `SubmitPmuPhase` 尾部追加稳定 - id,不重排已有 `None=0/Claim=1`。 + id,不重排已有 `None=0/Claim=1/EfDrain=2`。 2. `pa_scheduler/ccec/pmu_probe.h`:为 `SubmitPmuPhaseName()` 增加名称映射, 并核对 phase status/边界闭合定义。 3. `pa_scheduler/ccec/build.sh`:在白名单中将 phase 名映射到稳定 diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index af14851df0..88513ee1e7 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -33,7 +33,7 @@ Submit 路径,供后续继续优化。快照日期更新至 2026-07-18;当 fanin 成功/失败 load 与 frontier 重复前推,再进行单变量消减。 - standalone 观察产物现已固定为两类:`swimlane` 合并普通阶段与 schema-v3 atomic(direct Atomic 加 PollBatch)泳道;`submit-pmu` 独立重编译完整 Submit PMU,当前只支持 - `none|claim`。两者不在同一进程采集;`none` 提供完整 Submit 的严格 + `none|claim|efdrain`。两者不在同一进程采集;`none` 提供完整 Submit 的严格 闭合计数,局部 phase 只提供 running read-clear 的下界/保守上界。 环境安装、编译和基线复现过程见 @@ -1772,12 +1772,16 @@ schema-v3 以及“`--no-swimlane` 仍保留 phase timestamp”都是观察链 - `claim`:在每次 `Claim()` 前后读取 read-to-clear shadow,累计每核 1,280 次 Claim 的观测下界,并用本核 primary-shadow residual 给出保守 上界;b256 全局期望 calls 为 `256 * 5 * 96 = 122880`。 +- `efdrain`:只包围每次 Submit 开头唯一的 `DrainReady(...EfDrain...)`, + 不把复用该函数的 RingBackpressure/FinalDrain 混入;calls 与 Claim 同为 + 每核 `5 * batches`。 对应命令和产物为: ~~~bash ./run.sh build-submit-pmu ccec none ./run.sh build-submit-pmu ccec claim +./run.sh build-submit-pmu ccec efdrain ./run.sh submit-pmu ccec none \ --device 0 --batches 256 \ @@ -1788,11 +1792,17 @@ schema-v3 以及“`--no-swimlane` 仍保留 phase timestamp”都是观察链 --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json ./outputs//run1.json + +./run.sh submit-pmu ccec efdrain \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json ./outputs//run1.json ~~~ ~~~text build/ccec/submit-pmu/none/ build/ccec/submit-pmu/claim/ +build/ccec/submit-pmu/efdrain/ ~~~ 每个 phase 目录中的 host、mixed kernel、owner 与 dispatcher 是同一构建 @@ -1905,3 +1915,27 @@ outputs/submit_pmu_none_validation_20260718_b256_real/run1.json ... run4.json outputs/submit_pmu_final_gate_20260718/none_b256/run1.json outputs/submit_pmu_final_gate_20260718/claim_b256/run1.json ~~~ + +#### 7.5.17 EfDrain 独立 phase 的 A5 闭环 + +EfDrain 只在 `SubmitTask` 开头的专属 call-site 划界;`DrainReady()` 仍由 +RingBackpressure、HeapGuard 和 FinalDrain 复用,函数体本身没有 phase 分支。 +因此成功流每核固定 calls 为 `5 * batches`: + +| 规模 | 每核 calls | AIC/AIV calls | global calls | exact/bounded | request loss | miss loss | Submit span | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| b1 | 5 | 160 / 320 | 480 | 87/96,96/96 | 9 | 0 | 62.402 us | +| b256 | 1,280 | 40,960 / 81,920 | 122,880 | 60/96,96/96 | 1,113 | 0 | 4,844.973 us | + +b256 的 phase request 为 `3,258,436..3,259,549`,phase miss 为 +`482,396..482,396`;调度语义、任务拓扑、真计算输出、placement/engine、 +owner/Restore 和分析器 raw 复算全部 PASS。该窗口既可能走空 ring fast path, +也可能执行 ready slot 的真实 Cube/Vector workload、scalar wait、completion +发布和 slot 回收。不同 phase ELF 的 Submit span 与局部计数仍不可相减。 + +本机证据: + +~~~text +outputs/submit_pmu_phases_20260718/efdrain_b1/run1.json +outputs/submit_pmu_phases_20260718/efdrain_b256/run1.json +~~~ diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 91ce316a2b..d8d857ec0b 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -217,7 +217,7 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" | 构建 | 后端 | 内容 | 构建命令 | 产物目录 | | --- | --- | --- | --- | --- | | `swimlane` | CCEC/AscendC/CPU | 普通阶段与 schema-v3 atomic(direct + PollBatch)合并采集;不配置 PMU | `./run.sh build ccec` 或 `./run.sh build all` | CCEC 为 `build/ccec/` | -| `submit-pmu` | 仅 CCEC | 每核完整 Submit PMU,并在编译期可选一个局部阶段;当前只有 `none|claim` | `./run.sh build-submit-pmu ccec none|claim` | `build/ccec/submit-pmu//` | +| `submit-pmu` | 仅 CCEC | 每核完整 Submit PMU,并在编译期可选一个局部阶段;当前有 `none|claim|efdrain` | `./run.sh build-submit-pmu ccec ` | `build/ccec/submit-pmu//` | `./run.sh build all` 只构建三后端的 `swimlane` 产物;`submit-pmu` 必须按 phase 另行构建。`none` 是不做局部边界读取的完整 Submit @@ -234,7 +234,7 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" | `smoke` | 固定 b1/r1/`scalar-nop=0` 的快速语义回归 | 否,只做内存记录校验 | | `run` | 自行控制 batch、run、winner 负载和诊断参数 | 仅显式传入 `--swimlane-json` 时生成 raw | | `swimlane` | 单轮运行并自动生成 raw 和 Perfetto merged JSON | 是 | -| `build-submit-pmu` | 构建指定 `none|claim` 的 CCEC PMU-only ELF | 否 | +| `build-submit-pmu` | 构建指定 `none|claim|efdrain` 的 CCEC PMU-only ELF | 否 | | `submit-pmu` | 单轮采集完整 Submit PMU,可选导出 JSON | 否,与泳道隔离 | `ccec|ascendc|cpu|all` 用于选择后端;`all` 始终按 CCEC、AscendC、CPU @@ -688,12 +688,14 @@ phase/lap/Kernel 边界修复之前的 schema-v2 逐调用模型,只能用于 | --- | --- | --- | | `none` | 不做任何中途 shadow counter 读取 | 完整 Submit 主基准,优先用于回答 AIC/AIV 每核 request/miss | | `claim` | 每次 `Claim()` 调用前后读取 shadow counter | 验证局部归因链路,输出带观察扰动的 running read-clear 下界和保守上界 | +| `efdrain` | Submit 开头唯一的 EfDrain call-site 前后 | 归因 opportunistic drain,不包含 RingBackpressure/FinalDrain | 分别构建: ```bash ./run.sh build-submit-pmu ccec none ./run.sh build-submit-pmu ccec claim +./run.sh build-submit-pmu ccec efdrain ``` 产物完全分开: @@ -701,6 +703,7 @@ phase/lap/Kernel 边界修复之前的 schema-v2 逐调用模型,只能用于 ```text build/ccec/submit-pmu/none/ build/ccec/submit-pmu/claim/ +build/ccec/submit-pmu/efdrain/ ``` 每个目录都自包含同 phase 的 host、mixed kernel、PMU owner 和 dispatcher, @@ -737,7 +740,7 @@ CNT8 shadow whole request == CNT6 primary whole request CNT5 shadow whole miss == CNT7 primary whole miss ``` -`claim` 在 A5 上运行中反复 read-clear 时,shadow 可能在边界处单向少计, +`claim/efdrain` 在 A5 上运行中反复 read-clear 时,shadow 可能在边界处单向少计, 因此接受条件改为逐核: ```text @@ -755,7 +758,9 @@ phase miss ∈ [observed miss, observed miss + miss loss] 不要求局部 `phase miss <= phase request`;只要求二者分别不超过对应 shadow, 上界分别不超过对应 primary。`none` 中 phase calls/begin/end/request/miss 必须 全为 0;`claim` 中每核 begin/end/calls 必须配对,且每核 calls 必须等于 -`batches * 5`,全局为 `batches * 5 * 96`。 +`batches * 5`,全局为 `batches * 5 * 96`。`efdrain` 的 calls 形状相同; +但插点只允许包围 Submit 开头的 EfDrain 专属调用,不能插入复用的 +`DrainReady()` 函数体。 局部边界读取本身会增加 scalar 指令、改变 I-cache 布局和多核时序, 因此 `claim` 是带边界扰动的归因结果。`none` 与 `claim` 是不同 ELF/ @@ -780,7 +785,7 @@ ratio,不是实际 miss rate 的数学下界。更完整的 I-cache 采集、 以下 `empty/scalar/scalar-double/icache-single`、CNT8 fix-busy 和 schema-v3 文字保留为 2026-07-18 观察链路的建设过程与历史数据。当前 `swimlane` 构建不提供 PMU,当前 `submit-pmu` 也只接受完整 -Submit 的 `none|claim`;不应继续照抄下文的历史校准命令作为当前用法。 +Submit 的 `none|claim|efdrain`;不应继续照抄下文的历史校准命令作为当前用法。 CCEC 后端提供与泳道分离的 PMU sidecar。正式取数由本目录自带的 Main AICPU Path-A owner 配置 selector、保存并恢复 PMU 状态;kernel 在每个物理子核内门控并 diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh index 673788ad2f..e14c15a82a 100755 --- a/tests/atomic_probe/pa_scheduler/ccec/build.sh +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -33,15 +33,16 @@ case "$BUILD_VARIANT" in ;; submit-pmu) if [[ $# -ne 2 ]]; then - echo "Usage: $0 submit-pmu none|claim" >&2 + echo "Usage: $0 submit-pmu none|claim|efdrain" >&2 exit 1 fi PHASE_NAME="$2" case "$PHASE_NAME" in none) PHASE_ID=0 ;; claim) PHASE_ID=1 ;; + efdrain) PHASE_ID=2 ;; *) - echo "Unknown submit-pmu phase: $PHASE_NAME (expected none|claim)" >&2 + echo "Unknown submit-pmu phase: $PHASE_NAME (expected none|claim|efdrain)" >&2 exit 1 ;; esac @@ -49,7 +50,7 @@ case "$BUILD_VARIANT" in VARIANT_DEFINES=(-DPA_BUILD_SWIMLANE=0 -DPA_BUILD_SUBMIT_PMU=1 "-DPA_SUBMIT_PMU_PHASE_ID=$PHASE_ID") ;; *) - echo "Usage: $0 [swimlane] | $0 submit-pmu none|claim" >&2 + echo "Usage: $0 [swimlane] | $0 submit-pmu none|claim|efdrain" >&2 exit 1 ;; esac diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index c3775a4294..6de9eb3e6c 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -488,6 +488,24 @@ struct PmuValidation { // 高水位作为保守拒绝阈值;它只降低风险,不把“未越线”表述成回卷证明。 constexpr uint32_t kProgrammableCounterRiskThreshold = UINT32_MAX / 4U; +uint32_t ExpectedSubmitPmuPhaseCallsPerWorker( + const pa_scheduler::WorkerResult &result, uint32_t batches +) { + // 固定流阶段按每核完整回放次数闭合;保留 result 入参,后续 winner-only + // 阶段必须从本核真实 wins/heap_guards 推导,不能继续套用固定 5B。 + (void)result; + switch (pa_scheduler::kCompiledSubmitPmuPhase) { + case pa_scheduler::SubmitPmuPhase::None: + return 0U; + case pa_scheduler::SubmitPmuPhase::Claim: + case pa_scheduler::SubmitPmuPhase::EfDrain: + return batches * pa_scheduler::kTasksPerBatch; + case pa_scheduler::SubmitPmuPhase::Count: + break; + } + return UINT32_MAX; +} + bool ValidatePmu( const pa_scheduler::SchedulerState &state, uint32_t run, const PmuOptions &pmu, const WinnerWorkloadOptions &workload, @@ -521,6 +539,7 @@ bool ValidatePmu( uint32_t phase_boundary_matches = 0; uint32_t phase_call_shape_matches = 0; uint64_t phase_calls = 0; + uint64_t expected_phase_calls = 0; uint64_t shadow_request_abs_delta_sum = 0; uint64_t shadow_miss_abs_delta_sum = 0; int64_t shadow_request_signed_delta_sum = 0; @@ -532,12 +551,10 @@ bool ValidatePmu( PmuAggregate all; PmuAggregate aic; PmuAggregate aiv; - const uint32_t expected_phase_calls_per_worker = - pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None - ? 0U - : state.config.batches * pa_scheduler::kTasksPerBatch; for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { const pa_scheduler::WorkerResult &result = state.results[worker]; + const uint32_t expected_phase_calls_per_worker = + ExpectedSubmitPmuPhaseCallsPerWorker(result, state.config.batches); const uint32_t status = result.pmu_status; const uint32_t core_id = StatusCoreId(status); const bool record_trusted = (status & kStatusRequired) == kStatusRequired; @@ -580,6 +597,7 @@ bool ValidatePmu( phase_boundary_matches += boundaries_match; phase_call_shape_matches += phase_call_shape_matches_record; phase_calls += result.pmu_phase_calls; + expected_phase_calls += expected_phase_calls_per_worker; shadow_request_abs_delta_sum += request_abs_delta; shadow_miss_abs_delta_sum += miss_abs_delta; shadow_request_signed_delta_sum += @@ -646,13 +664,14 @@ bool ValidatePmu( !phase_call_shape_matches_record) && bad_printed < 8) { std::printf( "[PMU-BAD] worker=%u role=%llu coreid=%u status=0x%08x total=%llu scalar=%u " - "req=%u miss=%u phase_status=0x%08x phase=%u/%u calls=%u boundaries=%u/%u " + "req=%u miss=%u phase_status=0x%08x phase=%u/%u calls=%u/%u boundaries=%u/%u " "shadow=%u/%u\n", worker, static_cast(result.role), core_id, status, static_cast(result.pmu_total_cycles), result.pmu_scalar_busy, result.pmu_icache_requests, result.pmu_icache_misses, result.pmu_phase_status, result.pmu_phase_id, static_cast(pa_scheduler::kCompiledSubmitPmuPhase), result.pmu_phase_calls, + expected_phase_calls_per_worker, result.pmu_phase_begin_reads, result.pmu_phase_end_reads, result.pmu_shadow_icache_requests, result.pmu_shadow_icache_misses ); @@ -714,11 +733,6 @@ bool ValidatePmu( : shadow_primary_bounded_ok; const bool phase_boundaries_ok = phase_boundary_matches == pa_scheduler::kWorkers; const bool phase_call_shape_ok = phase_call_shape_matches == pa_scheduler::kWorkers; - const uint64_t expected_phase_calls = - pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None - ? 0ULL - : static_cast(state.config.batches) * pa_scheduler::kTasksPerBatch * - pa_scheduler::kWorkers; const bool phase_calls_ok = phase_calls == expected_phase_calls; const bool submit_engine_observation_ok = pmu.mode != WindowMode::SubmitAll || diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index 929f77c7de..a18e09b3a9 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -645,7 +645,13 @@ __aicore__ inline void CcecOps::PmuWindowStop( context.phase_calls == state->config.batches * pa_scheduler::kTasksPerBatch && context.begin_reads == context.phase_calls && context.end_reads == context.phase_calls; - if (none_shape || claim_shape) context.phase_status |= kPhaseStatusPhaseShape; + const bool efdrain_shape = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::EfDrain && + context.phase_calls == state->config.batches * pa_scheduler::kTasksPerBatch && + context.begin_reads == context.phase_calls && + context.end_reads == context.phase_calls; + if (none_shape || claim_shape || efdrain_shape) + context.phase_status |= kPhaseStatusPhaseShape; PublishPmuSnapshot(result, sample); PublishSubmitPmuContext(result, context); diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h index ba1435ec58..efa9d6ca32 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h @@ -150,6 +150,8 @@ inline const char *SubmitPmuPhaseName(SubmitPmuPhase phase) { return "none"; case SubmitPmuPhase::Claim: return "claim"; + case SubmitPmuPhase::EfDrain: + return "efdrain"; case SubmitPmuPhase::Count: break; } diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index 45320aad65..eff3b709ad 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -183,12 +183,14 @@ enum class ProfilePhase : uint32_t { }; // submit-pmu 每个 ELF 只编译一个局部归因阶段。none 不做中途 counter -// 读取,是完整 Submit 的正式基线;claim 是首个连续、无提前返回的验证阶段。 +// 读取,是完整 Submit 的正式基线;claim 是首个连续、无提前返回的验证阶段, +// EfDrain 只包围每次 Submit 开头唯一的 opportunistic drain call-site。 // 后续阶段只能在各自边界和闭环经过 A5 验证后向枚举尾部追加。 enum class SubmitPmuPhase : uint32_t { None = 0, Claim = 1, - Count = 2, + EfDrain = 2, + Count = 3, }; #ifndef PA_SUBMIT_PMU_PHASE_ID diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index f0ddc1d7b5..268b218475 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -561,7 +561,11 @@ PA_DEVICE bool SubmitTask( ResetTraceLap(stats.trace, stats.result, worker); // EfDrain 在当前 Submit 的参数物化前执行上一批已就绪 slot,是绝大多数 kernel 的正常落点。 + // 只在这个唯一 call-site 划 PMU 边界;DrainReady 还被 ring 背压和最终 drain + // 复用,不能把 phase 插入函数体后按 place 混合累计。 + BeginSubmitPmuPhase(pmu_context); DrainReady(state, worker, DrainPlace::EfDrain, stats); + EndSubmitPmuPhase(pmu_context); WriteTraceLap( stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::EfDrain, ProfilePhase::EfDrain diff --git a/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py b/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py index 8b8f7f915c..280d31148c 100644 --- a/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py +++ b/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py @@ -59,7 +59,7 @@ SUMMARY_FIELDS = ("sum", "mean", "median", "p95", "max") SUBMIT_PMU_BUILD_VARIANT = "submit-pmu" SUBMIT_PMU_BUILD_VARIANT_ID = 2 -SUBMIT_PMU_PHASE_IDS = {"none": 0, "claim": 1} +SUBMIT_PMU_PHASE_IDS = {"none": 0, "claim": 1, "efdrain": 2} TASKS_PER_BATCH = 5 PHASE_STATUS_REQUIRED_MASK = 0x3CF @@ -641,7 +641,11 @@ def _validate_submit_pmu_record( f"{prefix} phase upper bound exceeds the authoritative primary whole", ) - expected_calls = 0 if phase_name == "none" else batches * TASKS_PER_BATCH + expected_calls = ( + 0 if phase_name == "none" + else batches * TASKS_PER_BATCH if phase_name in ("claim", "efdrain") + else -1 + ) _require(calls == expected_calls, f"{prefix} phase_calls does not match the phase contract") if phase_name == "none": _require( diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index fba024cc65..730a3ca9be 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -21,8 +21,8 @@ Usage: ./run.sh run ccec|ascendc|cpu|all [benchmark options] ./run.sh smoke ccec|ascendc|cpu|all [--device N] ./run.sh swimlane ccec|ascendc|cpu|all [benchmark options] - ./run.sh build-submit-pmu ccec none|claim - ./run.sh submit-pmu ccec none|claim [benchmark options] + ./run.sh build-submit-pmu ccec none|claim|efdrain + ./run.sh submit-pmu ccec none|claim|efdrain [benchmark options] Benchmark options: --device N @@ -64,8 +64,8 @@ CCEC-only and cannot target all. The submit-pmu action is a separate CCEC-only build. It fixes one PMU-only run covering the complete Submit window. phase=none performs no internal snapshots; -phase=claim reports running read-clear lower/loss-adjusted upper bounds for one -compile-time phase while CNT6/7 retain the authoritative whole-window counters. +phase=claim/efdrain reports running read-clear lower/loss-adjusted upper bounds +for one compile-time phase while CNT6/7 retain the authoritative whole-window counters. The swimlane action performs exactly one run and writes both the raw capture and merged Perfetto JSON below this directory's outputs/ folder. It rejects @@ -136,9 +136,9 @@ run_backend() { validate_submit_pmu_phase() { case "$1" in - none|claim) ;; + none|claim|efdrain) ;; *) - echo "Unknown submit-pmu phase: $1 (expected none|claim)" >&2 + echo "Unknown submit-pmu phase: $1 (expected none|claim|efdrain)" >&2 exit 1 ;; esac @@ -159,6 +159,7 @@ validate_submit_pmu_artifacts() { case "$phase" in none) phase_id=0 ;; claim) phase_id=1 ;; + efdrain) phase_id=2 ;; *) submit_pmu_artifact_failure "$phase" "unsupported phase"; return 1 ;; esac @@ -377,7 +378,7 @@ case "$ACTION" in ;; build-submit-pmu) if [[ "$BACKEND" != "ccec" || $# -ne 1 ]]; then - echo "Usage: $0 build-submit-pmu ccec none|claim" >&2 + echo "Usage: $0 build-submit-pmu ccec none|claim|efdrain" >&2 exit 1 fi PHASE="$1" @@ -386,7 +387,7 @@ case "$ACTION" in ;; submit-pmu) if [[ "$BACKEND" != "ccec" || $# -lt 1 ]]; then - echo "Usage: $0 submit-pmu ccec none|claim [benchmark options]" >&2 + echo "Usage: $0 submit-pmu ccec none|claim|efdrain [benchmark options]" >&2 exit 1 fi PHASE="$1" diff --git a/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py b/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py index ea0d813706..4d33b4ba4d 100644 --- a/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py +++ b/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py @@ -201,7 +201,7 @@ def _capture(offset: int = 0, window: str = "submit-all") -> dict[str, Any]: def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any]: - phase_ids = {"none": 0, "claim": 1} + phase_ids = {"none": 0, "claim": 1, "efdrain": 2} phase_id = phase_ids[phase] batches = 2 calls_per_worker = 0 if phase == "none" else batches * TASKS_PER_BATCH @@ -685,30 +685,45 @@ def test_submit_pmu_v4_host_triplet_count_is_not_blindly_trusted(self) -> None: with self.assertRaisesRegex(ValueError, "mixed_triplet_matches"): load_capture(path) - def test_submit_pmu_claim_calls_are_exact_per_worker(self) -> None: - capture = _submit_pmu_capture() - first = capture["records"][0] - second = capture["records"][1] - first["phase_calls"] -= 1 - first["phase_begin_reads"] -= 1 - first["phase_end_reads"] -= 1 - first["shadow_read_segments"] -= 2 - second["phase_calls"] += 1 - second["phase_begin_reads"] += 1 - second["phase_end_reads"] += 1 - second["shadow_read_segments"] += 2 - records = capture["records"] - capture["summary"] = { - "all": _submit_pmu_summary(records), - "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), - "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), - } - # 全局 calls、每条 begin/end 与 shadow segment 都保持闭合,只有逐核 - # claim 调用契约被破坏。 + def test_submit_pmu_fixed_phase_calls_are_exact_per_worker(self) -> None: + for phase in ("claim", "efdrain"): + with self.subTest(phase=phase): + capture = _submit_pmu_capture(phase=phase) + first = capture["records"][0] + second = capture["records"][1] + first["phase_calls"] -= 1 + first["phase_begin_reads"] -= 1 + first["phase_end_reads"] -= 1 + first["shadow_read_segments"] -= 2 + second["phase_calls"] += 1 + second["phase_begin_reads"] += 1 + second["phase_end_reads"] += 1 + second["shadow_read_segments"] += 2 + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), + "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), + } + # 全局 calls、每条 begin/end 与 shadow segment 都保持闭合, + # 只有逐核固定流 phase 的调用契约被破坏。 + with tempfile.TemporaryDirectory() as directory: + path = self._write( + directory, f"redistributed-{phase}-calls.json", capture + ) + with self.assertRaisesRegex(ValueError, "phase_calls does not match"): + load_capture(path) + + def test_submit_pmu_efdrain_is_an_independent_phase(self) -> None: with tempfile.TemporaryDirectory() as directory: - path = self._write(directory, "redistributed-claim-calls.json", capture) - with self.assertRaisesRegex(ValueError, "phase_calls does not match"): - load_capture(path) + path = self._write(directory, "efdrain.json", _submit_pmu_capture(phase="efdrain")) + result = analyze([path]) + + self.assertEqual(result["phase_observation"]["compiled_phase"], "efdrain") + self.assertEqual( + result["aggregate"]["groups"]["all"]["phase_calls_per_core"]["median"], + 2 * TASKS_PER_BATCH, + ) def test_submit_pmu_none_rejects_nonzero_phase_counters(self) -> None: capture = _submit_pmu_capture(phase="none") From 7466e6f574aa6a42362cc8689ad12e7fc694b83e Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sun, 19 Jul 2026 01:06:25 +0000 Subject: [PATCH 026/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E5=AE=8C?= =?UTF-8?q?=E5=96=84Submit=20PMU=E5=B1=80=E9=83=A8=E5=BD=92=E5=9B=A0?= =?UTF-8?q?=E4=B8=8EI-cache=E5=8F=AF=E8=A7=86=E6=8A=A5=E5=91=8A?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 退役WaitForSlot局部PMU及wins兼容字段,分析器明确拒绝旧phase ID 3 - 新增Materialize与Register编译期观测窗口,按每核5×batches校验调用闭环 - 保留完整Submit primary计数,输出局部read-clear下界与保守上界 - 增强selector、counter位宽、风险阈值、total/scalar及shadow关系的独立复算 - 自动生成自包含HTML,增加ALL/AIC/AIV局部占总体比例图 - 将宽数字表和逐核图限制在独立滚动区域,修复页面内容越界 - 补充I-cache使用指南、scalar wait指标支持边界及A5 b1/b256实测记录 - 完成60项单元测试以及A5语义、PMU拓扑和owner恢复验证 --- tests/atomic_probe/icache_miss_usage_guide.md | 281 ++++++- ...05\345\206\265\345\210\206\346\236\220.md" | 204 ++++- ...77\347\224\250\346\214\207\345\215\227.md" | 104 ++- tests/atomic_probe/pa_scheduler/ccec/build.sh | 8 +- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 43 +- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 11 +- .../pa_scheduler/ccec/pmu_probe.h | 4 + .../pa_scheduler/common/pa_model.h | 17 +- .../pa_scheduler/common/pa_scheduler_core.h | 10 +- .../pa_scheduler/pmu_html_report.py | 750 ++++++++++++++++++ .../pa_scheduler/pmu_sidecar_analyzer.py | 197 ++++- tests/atomic_probe/pa_scheduler/run.sh | 48 +- .../pa_scheduler/test_pmu_html_report.py | 229 ++++++ .../pa_scheduler/test_pmu_sidecar_analyzer.py | 158 +++- 14 files changed, 1903 insertions(+), 161 deletions(-) create mode 100644 tests/atomic_probe/pa_scheduler/pmu_html_report.py create mode 100644 tests/atomic_probe/pa_scheduler/test_pmu_html_report.py diff --git a/tests/atomic_probe/icache_miss_usage_guide.md b/tests/atomic_probe/icache_miss_usage_guide.md index f67bb7c04f..2871c1d97f 100644 --- a/tests/atomic_probe/icache_miss_usage_guide.md +++ b/tests/atomic_probe/icache_miss_usage_guide.md @@ -41,27 +41,29 @@ atomic wrapper、ClockBaseline、runtime phase-profile 和旧 cold/warm 冲刷 两份证据可以按同一源码版本交叉理解,不能做逐 tick 对齐,也不能 把 PMU 的平均 miss 回填为某一条 atomic span 的属性。 -## 3. `none` 与 `claim` 如何选择 +## 3. `none` 与局部 phase 如何选择 -当前已验证三个编译期 phase: +当前正式支持五个编译期 phase: | phase | 边界 | 优先用途 | | --- | --- | --- | | `none` | 完整 Submit 中不读局部 shadow counter | 回答完整 Submit 的 AIC/AIV 每核 request/miss;这是默认选择 | | `claim` | 每次 `Claim()` 调用前后读局部 shadow counter | 当 `none` 已证明 miss 值得追踪时,试验 Claim 的 running read-clear 下界/上界归因链路 | | `efdrain` | 每次 Submit 开头唯一的 `DrainReady(...EfDrain...)` 前后 | 观察 opportunistic drain;不混入 RingBackpressure 或 FinalDrain | +| `materialize` | 每次 `MaterializeTask()` 调用前后 | 观察输出 descriptor/layout、本地 register mask、输出字节数和 heap 游标等 scalar 工作;不包含后续 slot payload 拷贝 | +| `register` | Alloc/non-Alloc 两个互斥的 `RegisterOutputs()` 调用点前后 | 观察输出注册;边界与普通泳道的 Register span 对齐 | -`claim` 的 begin/end 读取本身会执行 scalar 指令、占用取指并改变多核 +running phase 的 begin/end 读取本身会执行 scalar 指令、占用取指并改变多核 时序。因此: -- `claim` 的 phase request/miss 是带局部边界扰动的观察值; -- `claim` 的 raw 观察值是下界,上界为该核下界加 primary-shadow loss; -- `none` 和 `claim` 是不同 ELF、不同进程,不能以两者相减声称得到 - 了零扰动 Claim 净值; +- running phase 的 phase request/miss 是带局部边界扰动的观察值; +- raw 观察值是下界,上界为该核下界加 primary-shadow loss; +- `none` 和任一 running phase 是不同 ELF、不同进程,不能以两者相减声称 + 得到了零扰动的局部净值; - 未来不同 phase ELF 的局部 request/miss 不可相加成“完整 Submit”; 完整 Submit 始终以每个 ELF 自己的 primary whole 为准。 -该区间只描述同一插桩 ELF、当前边界定义下的局部事件,不是无插桩 Claim +该区间只描述同一插桩 ELF、当前边界定义下的局部事件,不是无插桩局部阶段 的真实区间。完整 Submit `none` 没有运行中 read-clear,仍执行 96/96 逐核严格闭合。 @@ -76,6 +78,7 @@ export GCC15_ROOT=/home/q00473782/.local/gcc-15/root export PATH="$GCC15_ROOT/usr/bin:$PATH" export LD_LIBRARY_PATH="$GCC15_ROOT/usr/lib/x86_64-linux-gnu:$GCC15_ROOT/usr/lib/gcc/x86_64-linux-gnu/15${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" export CXX="$GCC15_ROOT/usr/bin/g++-15" +export PYTHON=/home/q00473782/.venv/bin/python ``` 构建完整 Submit 基准: @@ -96,12 +99,26 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" ./run.sh build-submit-pmu ccec efdrain ``` +需要 Materialize 局部归因时独立构建: + +```bash +./run.sh build-submit-pmu ccec materialize +``` + +需要 Register 局部归因时独立构建: + +```bash +./run.sh build-submit-pmu ccec register +``` + 产物分别位于: ```text -pa_scheduler/build/ccec/submit-pmu/none/ -pa_scheduler/build/ccec/submit-pmu/claim/ -pa_scheduler/build/ccec/submit-pmu/efdrain/ +build/ccec/submit-pmu/none/ +build/ccec/submit-pmu/claim/ +build/ccec/submit-pmu/efdrain/ +build/ccec/submit-pmu/materialize/ +build/ccec/submit-pmu/register/ ``` 每个 phase 目录中的 `pa_scheduler_host`、`pa_scheduler_kernel.o`、 @@ -112,10 +129,10 @@ pa_scheduler/build/ccec/submit-pmu/efdrain/ `submit_pmu_artifacts.manifest`;`run.sh` 在启动 host 前核对 schema、phase、 固定文件列表和四个 SHA256。 -`swimlane` 的 CCEC 产物仍在 `pa_scheduler/build/ccec/`,不是 PMU +`swimlane` 的 CCEC 产物仍在 `build/ccec/`,不是 PMU 产物。 -## 5. 采集完整 Submit 和 Claim +## 5. 采集完整 Submit 与局部 phase ### 5.1 完整 Submit `none` @@ -126,19 +143,22 @@ mkdir -p "$OUT" ./run.sh submit-pmu ccec none \ --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json "$OUT/run1.json" + --pmu-json "$OUT/submit_icache_raw.json" ``` -多轮比较必须使用多个独立进程和唯一文件名: +多轮比较必须使用多个独立进程和独立子目录;每个采集目录内部都保持 +同一组描述性文件名: ```bash +mkdir -p "$OUT/capture_02" "$OUT/capture_03" + ./run.sh submit-pmu ccec none --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json "$OUT/run2.json" + --pmu-json "$OUT/capture_02/submit_icache_raw.json" ./run.sh submit-pmu ccec none --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json "$OUT/run3.json" + --pmu-json "$OUT/capture_03/submit_icache_raw.json" ``` ### 5.2 Claim 局部归因 @@ -150,7 +170,7 @@ mkdir -p "$OUT_CLAIM" ./run.sh submit-pmu ccec claim \ --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json "$OUT_CLAIM/run1.json" + --pmu-json "$OUT_CLAIM/submit_icache_raw.json" ``` ### 5.3 EfDrain 局部归因 @@ -162,13 +182,80 @@ mkdir -p "$OUT_EFDRAIN" ./run.sh submit-pmu ccec efdrain \ --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json "$OUT_EFDRAIN/run1.json" + --pmu-json "$OUT_EFDRAIN/submit_icache_raw.json" ``` `efdrain` 每核固定调用 `batches * 5` 次;b256 的 AIC/AIV/global calls 分别为 40,960/81,920/122,880。插点只位于 Submit 开头的 EfDrain 专属 call-site;复用的 `DrainReady()` 函数体不插桩。 +### 5.4 Materialize 局部归因 + +先用 b1 验证边界和 96 核闭合,再跑 b256 正式负载: + +```bash +OUT_MAT_B1="./outputs/submit_pmu_materialize_$(date -u +%Y%m%dT%H%M%SZ)_b1" +OUT_MAT_B256="./outputs/submit_pmu_materialize_$(date -u +%Y%m%dT%H%M%SZ)_b256" +mkdir -p "$OUT_MAT_B1" "$OUT_MAT_B256" + +./run.sh submit-pmu ccec materialize \ + --device 0 --batches 1 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT_MAT_B1/submit_icache_raw.json" + +./run.sh submit-pmu ccec materialize \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT_MAT_B256/submit_icache_raw.json" +``` + +边界位于 `MaterializeTask()` 唯一调用点前后。实现必须先保存返回值、关闭 +phase,再处理失败返回,避免失败路径留下 begin/end 不平衡。每核固定 +`5 * batches` 次;b1 的 AIC/AIV/global calls 为 160/320/480,b256 为 +40,960/81,920/122,880。 + +2026-07-19 A5 实测闭环如下;四轮均满足 capture accepted、语义、PMU 和 +phase measurement PASS: + +| phase | batches | calls/expected | begin/end 与 call shape | primary=shadow | shadow≤primary | request/miss loss | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| `materialize` | 1 | 480/480 | 96/96 | 94/96 | 96/96 | 2/0 | +| `materialize` | 256 | 122,880/122,880 | 96/96 | 74/96 | 96/96 | 160/8 | +| `register` | 1 | 480/480 | 96/96 | 93/96 | 96/96 | 2/1 | +| `register` | 256 | 122,880/122,880 | 96/96 | 36/96 | 96/96 | 2,834/654 | + +对应 raw/HTML 位于 +`outputs/submit_pmu_{materialize,register}_20260719_b{1,256}/`;raw 是权威 +取数件,HTML 是同目录的加工展示件。 + +running read-clear 的硬门禁是 96/96 `shadow≤primary`,不是要求 96/96 +逐值相等;表中的 loss 已进入每核局部 lower/upper 区间,不能被静默忽略。 + +### 5.5 Register 局部归因 + +```bash +OUT_REG_B1="./outputs/submit_pmu_register_$(date -u +%Y%m%dT%H%M%SZ)_b1" +OUT_REG_B256="./outputs/submit_pmu_register_$(date -u +%Y%m%dT%H%M%SZ)_b256" +mkdir -p "$OUT_REG_B1" "$OUT_REG_B256" + +./run.sh submit-pmu ccec register \ + --device 0 --batches 1 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT_REG_B1/submit_icache_raw.json" + +./run.sh submit-pmu ccec register \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT_REG_B256/submit_icache_raw.json" +``` + +Register 有 Alloc 和 non-Alloc 两个互斥调用点,二者都使用同一 phase +边界;winner/loser 均从其中一条路径通过,因此仍是每核固定 +`5 * batches`,调用规模与 Materialize 相同。该窗口与普通泳道 Register +span 对齐;其中较短或未实际插入 map 的调用仍会放大 PMU begin/end +边界扰动,解释结果时必须使用 lower/upper,而不能把 observed lower +当成无扰动净开销。 + `submit-pmu` action 已固定: ```text @@ -177,8 +264,31 @@ call-site;复用的 `DrainReady()` 函数体不插桩。 调用者不要重复传入这三项,也不能添加 `--profile-phases`、`--trace-atomics`、`--analyze-swimlane` 或 -`--swimlane-json`。`--pmu-json` 可选;但要做 raw 复算和多轮汇总时必须 -使用它。host 拒绝覆盖已有 JSON 或同名 `.tmp`。 +`--swimlane-json`。`--pmu-json` 可选;但要做 raw 复算、HTML 可视报告和 +多轮汇总时必须使用它。host 拒绝覆盖已有 JSON 或同名 `.tmp`。 + +raw 成功发布后,`run.sh` 会调用本目录的独立分析器并在同一目录生成: + +```text +submit_icache_raw.json # 96 核权威原始件及 host summary +submit_icache_report.html # 可离线浏览的加工件 +``` + +HTML 使用内联 CSS/SVG,不访问网络,也不依赖外部前端库;浏览器直接打开即可。 +它包含完整 Submit 的 AIC/AIV 对比、逐物理核 request/miss/rate 分布、96 核 +明细和 90 ns core-equivalent 提示。报告同时展示 AIC/AIV/ALL 的 PMU raw +`total_cycles`、CNT2 `scalar_busy`、`Σscalar/Σtotal` 以及 +“非 Scalar-busy 残余”。其中 total 是每个物理子核在 Submit gate 内的 PMU +累计周期,96 核求和是 core-work,不是 Submit 墙钟;“非 Scalar-busy 残余” +严格等于 `total−scalar_busy`,既不是 Scalar 空闲时间,也不是 I-cache stall, +其中还包含同步等待、vector/cube engine 等待以及其他未归因周期。受控微基准中,依赖返回的 atomic 等待大部分进入 +scalar busy,而 I-cache refill 的额外周期大部分只进入 total;这个现象 +不能把二者之差提升为 I-cache 专属计数器。 + +对于局部 phase,HTML 还会单列 calls、 +request/miss 下界—上界、shadow loss,以及局部事件占同一 ELF 完整 Submit +primary 的比例区间,不把局部数据冒充可相加的精确分解。 +报告生成失败时 action 返回非零,但已成功发布的 raw 会保留用于排查。 ## 6. Primary/shadow 计数和可信门禁 @@ -186,6 +296,8 @@ call-site;复用的 `DrainReady()` 函数体不插桩。 | 计数 | selector | 用途 | | --- | --- | --- | +| PMU raw total | 固定 64-bit total low/high | Submit gate 内每个物理子核的累计周期;不是 96 核求和后的墙钟 | +| CNT2 | `0x001` | scalar instruction busy cycle;不包含全部等待周期 | | CNT6 | `0x34` | 完整 Submit primary I-cache request;局部边界从不读它 | | CNT7 | `0x35` | 完整 Submit primary I-cache miss;局部边界从不读它 | | CNT8 | `0x34` | read-to-clear shadow request | @@ -196,9 +308,34 @@ A5 b1 实测已证明将 `0x35` 配置到 CNT9 时计数始终为 0,所以 CNT9 不能作 shadow miss。这个变化只影响 `submit-pmu` 诊断构建, 不影响 `swimlane` 构建。 -shadow PMU counter 是 read-to-clear。`claim` 在阶段 begin 读取 CNT8/CNT5,将 +#### A5 `scalar_wait_ib_time` 的支持边界 + +2026-07-19 使用本机 CANN 9.1 在同一 A5/DAV3510 上对 standalone b1 依次验证了 +`PipeUtilization`、`PipeUtilization,MemoryDetail` 和 `Default` 三种正式 +`msopprof` 采集入口。三次均能生成 `PipeUtilization.csv`,但表头都不包含 +`aic/aiv_scalar_wait_ib_time`,也不包含 `aic/aiv_scalar_wait_time`。本机 CANN +9.1 `CHIP_V6_MAP` 与本仓 DAV3510 正式事件表同样只给出已使用的 +`scalar_busy(0x001)`、I-cache request `0x34`、I-cache miss `0x35` 等事件, +没有 wait-IB/wait 的 selector 或派生公式。 + +三次原始证据分别保存在: + +```text +outputs/wait_ib_official_msopprof_20260719_b1_probe2/ +outputs/wait_ib_official_msopprof_20260719_b1_probe3_memory_detail/ +outputs/wait_ib_official_msopprof_20260719_b1_probe4_default/ +``` + +因此当前 A5 正式可编程路径的结论是:**不能采集这两个指标**。CANN 共享 +`msopprof` 二进制包含相应字段字符串、官方文档也在 A2/A3 产品章节解释其 +含义,但这些证据不能推出 DAV3510 selector。不得把旧架构或其他产品的事件号 +套到 A5。若后续 CANN/A5 正式事件表新增这两项,必须重新用 scalar NOP、 +I-cache warm/cold、真实 Vector/Cube `PIPE_* -> PIPE_S` wait 和依赖 atomic +四组对照校准后再纳入报告。 + +shadow PMU counter 是 read-to-clear。任一 running phase 在阶段 begin 读取 CNT8/CNT5,将 之前的片段加入 shadow whole;在 end 再读一次,同时加入 shadow -whole 和 Claim phase;完整 Submit stop 后读 tail。`none` 不做中途读取, +whole 和所选 phase;完整 Submit stop 后读 tail。`none` 不做中途读取, 只在 stop 后取 tail。 `none` 对每个物理子核必须精确满足: @@ -212,7 +349,7 @@ shadow_whole_icache_misses == icache_misses 这个 96/96 精确相等门禁验证完整 Submit 观察闭合;它不把 PMU 进程的 Submit span 变成无诊断墙钟基线,也不把 standalone 数据冒充真实 PA profile。 -运行中切片的 `claim` 已在 A5 b1/b256 上证明可能发生单向少计,接受规则为: +运行中切片已在 A5 b1/b256 上证明可能发生单向少计,接受规则为: ```text shadow_request <= primary_request @@ -252,10 +389,12 @@ CNT8/CNT5 是顺序 `ld_dev`,不是同一时刻的原子配对快照,因此 - running phase 的 shadow whole 逐核不大于 primary,loss 与 upper-bound 公式逐核闭合;exact 核数只作诊断; - `none` 的 phase calls/begin/end/request/miss 全部为 0; -- `claim` 的 begin/end/calls 逐核平衡,每核 calls 为 `batches * 5`, - 全局 calls 为 `batches * 5 * 96`; -- `efdrain` 与 `claim` 具有相同的每核固定 calls 形状,但边界必须只覆盖 - Submit 开头的 EfDrain call-site; +- `claim`、`efdrain`、`materialize`、`register` 的 begin/end/calls 逐核 + 平衡,每核 calls 为 `batches * 5`,全局 calls 为 + `batches * 5 * 96`; +- 四个 running phase 必须分别命中自己的边界:Claim 调用、Submit 开头 + EfDrain 专属 call-site、Materialize 唯一调用点,以及 Alloc/non-Alloc + 两个互斥 Register 调用点; - phase request/miss 分别不超过对应 shadow/primary,且可编程 counter 低于当前 25% 保守风险阈值。 @@ -275,6 +414,13 @@ sum / mean / median / p95 / max 完整 Submit 优先查看: +- `configuration.submit_span_us`:本轮从第一个 Submit 进入到最后一个 + Submit 返回的整体 span;HTML 顶部换算成毫秒展示; +- `total_cycles`:每核 Submit gate 内的 64-bit PMU raw total;按角色的 + sum 是 core-work,mean/median/p95 才适合比较典型单核,均不等于 host + 看到的 Submit 墙钟; +- `scalar_busy`:CNT2 `scalar_instr_busy(0x001)`,表示 scalar instruction + busy cycle;依赖返回的 atomic 等待可进入此项,但它不是“纯算术指令数”; - `icache_requests` / `icache_misses`:CNT6/CNT7 primary whole; - `shadow_whole_icache_requests` / `shadow_whole_icache_misses`:闭合或分段 loss 用 shadow whole; @@ -289,12 +435,29 @@ sum / mean / median / p95 / max - `configuration.phase_values_are_running_read_clear_lower_bounds`:确认局部字段 是否采用下界语义。 +HTML 中展示的“非 Scalar-busy 残余/core”严格等于 +`(total−scalar_busy)/core`,只用于观察未被 scalar busy 覆盖周期的数量级。 +它可能同时包含 I-cache refill、同步等待、vector/cube engine 等待和其他 +流水空隙,不能命名为 Scalar 空闲或 I-cache stall,也不能用它反推单次 miss +代价。 + +离线分析器与 HTML 会从这些 raw 字段继续派生 +`phase_icache_request_lower_bound_share_of_submit`、 +`phase_icache_request_upper_bound_share_of_submit`、 +`phase_icache_miss_lower_bound_share_of_submit` 和 +`phase_icache_miss_upper_bound_share_of_submit`:局部 lower/upper 分别除以 +同一角色、同一次采集的完整 Submit primary 总数。 + `phase_observed_read_clear_ratio` 只是 lower miss/lower request 的观测比值; 分子和分母各有独立区间,因此它不是实际 phase miss rate 的数学下界。 每核平均值按角色求: ```text +AIC PMU total/core = summary.aic.total_cycles.sum / 32 +AIC scalar/core = summary.aic.scalar_busy.sum / 32 +AIV PMU total/core = summary.aiv.total_cycles.sum / 64 +AIV scalar/core = summary.aiv.scalar_busy.sum / 64 AIC request/core = summary.aic.icache_requests.sum / 32 AIC miss/core = summary.aic.icache_misses.sum / 32 AIV request/core = summary.aiv.icache_requests.sum / 64 @@ -312,7 +475,33 @@ AIV miss rate = Σ(AIV miss) / Σ(AIV request) 不平均 32 或 64 个逐核百分比。AIC/AIV 核数不同,比较每核强度时 使用 mean/median/p95 或 miss rate,不直接比较两组 sum。 -## 8. 分析命令与结果文件 +局部 phase 占完整 Submit 的比例同样使用组内总量: + +```text +request share lower = Σphase_request_lower / Σprimary_request +request share upper = Σphase_request_upper / Σprimary_request +miss share lower = Σphase_miss_lower / Σprimary_miss +miss share upper = Σphase_miss_upper / Σprimary_miss +``` + +分子与分母必须来自同一个 phase ELF、同一轮采集和同一角色。该比例回答“当前 +插桩 ELF 中局部窗口占自身完整 Submit 事件的多少”,不能拿 `claim` 分子除以 +另一份 `none` 的分母。 + +## 8. HTML 报告与多轮分析命令 + +单份 raw 的 HTML 已由 `submit-pmu` action 自动生成。需要手工重建时: + +```bash +PYTHON=/home/q00473782/.venv/bin/python + +"$PYTHON" ./pmu_html_report.py \ + "$OUT/submit_icache_raw.json" +``` + +默认输出为同目录的 `submit_icache_report.html`;也可用 `-o` 指定路径。 +生成器先调用 `pmu_sidecar_analyzer.py` 的完整 raw 门禁,只有 96 核拓扑、 +raw→summary、primary/shadow、采集接受状态和 owner Restore 全部通过才发布 HTML。 使用本用户 Python 环境从 raw 重算 host summary,并聚合相同配置的 多个独立进程: @@ -321,20 +510,20 @@ AIV miss rate = Σ(AIV miss) / Σ(AIV request) PYTHON=/home/q00473782/.venv/bin/python "$PYTHON" ./pmu_sidecar_analyzer.py \ - "$OUT/run1.json" "$OUT/run2.json" "$OUT/run3.json" + "$OUT/submit_icache_raw.json" "$OUT"/capture_*/submit_icache_raw.json ``` 需要机器可读汇总时: ```bash "$PYTHON" ./pmu_sidecar_analyzer.py --json \ - "$OUT/run1.json" "$OUT/run2.json" "$OUT/run3.json" \ + "$OUT/submit_icache_raw.json" "$OUT"/capture_*/submit_icache_raw.json \ > "$OUT/summary.json" ``` 分析器会先逐份复算 96 条 raw 与 host summary,然后拒绝聚合下列混用: -- `none` 与 `claim`; +- 任意不同 phase,例如 `none` 与 `claim`,或 `materialize` 与 `register`; - 不同 schema/build variant; - 不同 batches、winner mode/count/pattern、selector 或观察开关。 @@ -369,12 +558,38 @@ core-work 等效总量,不是端到端 Submit 总损失。要测真正暴露 3. 只有第 2 项是实际暴露的墙钟收益;第 1 项用于证明收益与 I-cache 变化同时出现,`90 ns` 仅提供一阶数量级解释。 +### 9.1 当前 b256 `none` 参考数据 + +2026-07-19 用最终 `submit-pmu none` ELF、`real-compute/6,28,4,1` +在 A5 上采集一轮,完整 Submit span 为 `4.750810 ms`。96 核 raw、owner +Restore、selector、counter 阈值和离线复算全部通过。 + +| 指标 | AIC(32 核) | AIV(64 核) | +| --- | ---: | ---: | +| request/core | 408,317.344 | 422,480.609 | +| miss/core | 38,664.344 | 55,098.625 | +| `Σmiss/Σrequest` | 9.4692% | 13.0417% | +| total/core | 7,471,385.531 | 7,085,178.734 | +| scalar busy/core | 5,944,751.250 | 5,603,587.469 | +| `Σscalar/Σtotal` | 79.5669% | 79.0889% | +| 非 Scalar-busy 残余/core | 1,526,634.281 | 1,481,591.266 | +| `miss/core × 90 ns` | 3,479.791 us | 4,958.876 us | + +最后一行只是单核串行等效标尺,不能与 `4.750810 ms` 相减或解释成 +端到端损失;非 Scalar-busy 残余也不是空闲或 I-cache stall。原始件和 +自包含报告位于: + +```text +outputs/submit_pmu_none_20260719_b256_final/submit_icache_raw.json +outputs/submit_pmu_none_20260719_b256_final/submit_icache_report.html +``` + ## 10. 新增局部 phase 的修改清单 新 phase 不能只增加一个 CLI 字符串。最小完整修改包括: 1. `pa_scheduler/common/pa_model.h`:在 `SubmitPmuPhase` 尾部追加稳定 - id,不重排已有 `None=0/Claim=1/EfDrain=2`。 + id,不重排已有 `None=0/Claim=1/EfDrain=2/Materialize=4/Register=5`。 2. `pa_scheduler/ccec/pmu_probe.h`:为 `SubmitPmuPhaseName()` 增加名称映射, 并核对 phase status/边界闭合定义。 3. `pa_scheduler/ccec/build.sh`:在白名单中将 phase 名映射到稳定 diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index 88513ee1e7..d12d54ace4 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -4,7 +4,8 @@ 本文记录 `TestPagedAttentionUnroll::Case1` 在真实 A5 上的 FDWIC AICore Submit 路径,供后续继续优化。快照日期更新至 2026-07-18;当前保留的生产 -优化基线为 `2c3dd1e2`,F1 负结果记录提交为 `c93c3666`。 +优化基线为 `2c3dd1e2`,F1 负结果记录提交为 `c93c3666`。文档中 +standalone 观察链路的最新状态更新至 2026-07-19。 范围限定为: @@ -32,9 +33,10 @@ Submit 路径,供后续继续优化。快照日期更新至 2026-07-18;当 表面收益。F1 的 fanin 顺序重排已经证明性能回退并撤销;下一步先精确区分 fanin 成功/失败 load 与 frontier 重复前推,再进行单变量消减。 - standalone 观察产物现已固定为两类:`swimlane` 合并普通阶段与 schema-v3 - atomic(direct Atomic 加 PollBatch)泳道;`submit-pmu` 独立重编译完整 Submit PMU,当前只支持 - `none|claim|efdrain`。两者不在同一进程采集;`none` 提供完整 Submit 的严格 - 闭合计数,局部 phase 只提供 running read-clear 的下界/保守上界。 + atomic(direct Atomic 加 PollBatch)泳道;`submit-pmu` 独立重编译完整 Submit PMU,现行 + 白名单为 `none|claim|efdrain|materialize|register`。两者不在同一进程采集; + `none` 提供完整 Submit 的严格闭合计数,局部 phase 只提供 running + read-clear 的下界/保守上界。 环境安装、编译和基线复现过程见 [A5 FDWIC Paged Attention 安装与复现指南](../a5_fdwic_atomic_swimlane_repo.md)。 @@ -380,7 +382,9 @@ TensorMap/heap 最终状态、fanin、flag、vend、frontier、cursor、ring pla 拆设备目标和全局 memory clobber 实验曾分别触发状态破坏、device exception 或 明显 RingBp,均已回退。 -四阶段诊断通过 `--profile-phases` 输出:Claim 和 EfDrain 每 worker 调用 +下列四阶段是普通 runtime `--profile-phases` 的历史诊断口径,不是第 7.5 节的 +`submit-pmu` 局部白名单;调度器中的 WaitForSlot 协议和普通泳道仍保留。 +Claim 和 EfDrain 每 worker 调用 1280 次;WaitForSlot 由 1024 个 kernel winner 调用;HeapGuard 由每 batch 的 Alloc/QK/SF/PV winner 调用,共 1024 次。当前代表性 CCEC 轮次的累计中位数为: @@ -778,7 +782,7 @@ standalone 的 Submit 中位数、均值和配对中心都没有改善。因此 #### 7.3.1 直接 PMU owner 是唯一正式取数链路 整任务级 raw counter 无法由 kernel 内局部 gate 缩成 Submit 子窗口,因此不能作为 -Claim、EfDrain、WaitForSlot、HeapGuard 或 Submit 局部取数依据。当前正式链路不消费 +Claim、EfDrain、Materialize、Register 的局部取数依据。当前正式链路不消费 这类整任务汇总:CCEC host 使用本目录自带的 Main AICPU Path-A owner 保存、配置、 读回并最终恢复 PMU 状态;kernel 在同一 runtime TU 内完成门控、读取和发布。 @@ -1258,11 +1262,16 @@ PMU JSON sidecar 按 worker 保留 raw 记录,并分别汇总 32 AIC、64 AIV - `total_cycles`:gate 活跃期间的每核 64 bit PMU raw total;除非平台时钟/事件语义 另有正式证明,不直接按 1 GHz 换算成微秒;96 核求和是 core-work raw count, - 不是 Submit 墙钟时间; + 不是 Submit 墙钟时间。墙钟只看 host 记录的 `submit_span_us`; - `scalar_busy`:`CNT2 scalar_instr_busy` 的事件累计,不等于窗口内全部时间, - 也不等于“纯调度耗时”; + 也不等于“纯调度耗时”。atomic 由 scalar 发射,其发射、返回值依赖或资源 + 阻塞可能在该事件中有所体现,但不能据此认定某条 atomic 的全部执行完成延迟 + 都等量算入 `scalar_busy`; - `icache_requests/misses`:`CNT6/CNT7` 的事件累计。整体 miss rate 必须用 - `sum(misses)/sum(requests)`,AIC/AIV 分开计算,不能平均 96 个逐核百分比; + `sum(misses)/sum(requests)`,AIC/AIV 分开计算,不能平均 96 个逐核百分比。 + request/miss 是“次数”事件,不是 busy 或 stall 时长;I-cache miss 引起的取指空泡 + 可能体现为 scalar 无法发射而不增加 `scalar_busy`,因此不存在“每个 miss 自动加到 + scalar busy”的恒等式; - `vector/cube/MTE1/MTE2/MTE3/fix busy`:同一 PipeUtilization 配置的辅助证据, 用来检查窗口内实际活跃单元,不能从事件名称反推出一条指令造成的精确 stall; - `window_started/window_stopped`:来自每个 worker 实际执行 gate 的状态位,不用模式 @@ -1270,6 +1279,14 @@ PMU JSON sidecar 按 worker 保留 raw 记录,并分别汇总 32 AIC、64 AIV owner bitmap membership、worker slot/物理 role/triplet、miss 不大于 request 和 counter 风险门槛等门禁。 +raw JSON 与自包含 HTML 都必须同时保留完整 Submit 的 `submit_span_us`、 +`total_cycles` 和 `scalar_busy`,并按 ALL/AIC/AIV 显示每核分布;不得只展示 +I-cache request/miss。当前局部 phase 边界仅中途 read-clear shadow request/miss, +没有局部 `total_cycles` 或 `scalar_busy`。文档和 HTML 中的 total/scalar 因此都是 +该 phase ELF 的完整 Submit 窗口,不能当成 Materialize、Register 等局部阶段的独占时间。 +90 ns/miss 仍只是受控 cold/warm 探针的一阶 core-equivalent 标尺,既不加入 +`scalar_busy`,也不从 `total_cycles` 中直接扣除。 + ##### 7.5.5.1 自包含 Main AICPU Path-A owner 闭环 owner 已收入 `pa_scheduler/ccec`,所有构建和运行文件均位于 standalone 目录内。 @@ -1767,14 +1784,20 @@ schema-v3 以及“`--no-swimlane` 仍保留 phase timestamp”都是观察链 当前 `submit-pmu` 仅白名单支持: -- `none`:不执行局部 counter 边界读取,用于回答完整 Submit 的 - AIC/AIV 每核 request/miss; -- `claim`:在每次 `Claim()` 前后读取 read-to-clear shadow,累计每核 - 1,280 次 Claim 的观测下界,并用本核 primary-shadow residual 给出保守 - 上界;b256 全局期望 calls 为 `256 * 5 * 96 = 122880`。 -- `efdrain`:只包围每次 Submit 开头唯一的 `DrainReady(...EfDrain...)`, - 不把复用该函数的 RingBackpressure/FinalDrain 混入;calls 与 Claim 同为 - 每核 `5 * batches`。 +| phase | id | 现行边界 | 成功流 calls | +| --- | ---: | --- | ---: | +| `none` | 0 | 不执行局部 counter 读取,只取完整 Submit | 0 | +| `claim` | 1 | 每次 `Claim()` 前后 | 每核 `5*batches` | +| `efdrain` | 2 | 每次 Submit 开头唯一的 `DrainReady(...EfDrain...)` 前后 | 每核 `5*batches` | +| `materialize` | 4 | 每次 `MaterializeTask()` 前后,包含成功与失败返回边界 | 每核 `5*batches` | +| `register` | 5 | Alloc 和非 Alloc 两个互斥 `RegisterOutputs()` call-site 前后 | 每核 `5*batches` | + +`efdrain` 不把复用 `DrainReady()` 的 RingBackpressure/FinalDrain 混入。 +`register` 在 Alloc 路径传入 `include_existing=false`,在非 Alloc 路径传入 +`true`;两个 call-site 互斥且所有 worker 每个 task 只进入一次,所以与 +Claim/EfDrain/Materialize 一样可以用固定 `5*batches` 闭合。这个窗口匹配 +现有 Register 泳道 span,但不意味五类 task 每次都实际执行 TensorMap insert; +短路径中局部 PMU 边界本身的扰动占比会更明显。 对应命令和产物为: @@ -1782,27 +1805,45 @@ schema-v3 以及“`--no-swimlane` 仍保留 phase timestamp”都是观察链 ./run.sh build-submit-pmu ccec none ./run.sh build-submit-pmu ccec claim ./run.sh build-submit-pmu ccec efdrain +./run.sh build-submit-pmu ccec materialize +./run.sh build-submit-pmu ccec register ./run.sh submit-pmu ccec none \ --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json ./outputs//run1.json + --pmu-json ./outputs//submit_icache_raw.json ./run.sh submit-pmu ccec claim \ --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json ./outputs//run1.json + --pmu-json ./outputs//submit_icache_raw.json ./run.sh submit-pmu ccec efdrain \ --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json ./outputs//run1.json + --pmu-json ./outputs//submit_icache_raw.json + +./run.sh submit-pmu ccec materialize \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json ./outputs//submit_icache_raw.json + +./run.sh submit-pmu ccec register \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json ./outputs//submit_icache_raw.json ~~~ +成功采集后同目录自动生成 `submit_icache_report.html`。raw 继续作为权威证据, +HTML 只提供 AIC/AIV 汇总、逐核分布和 running phase lower/upper 的离线可视化, +不改变原始统计和可信门禁。 + ~~~text build/ccec/submit-pmu/none/ build/ccec/submit-pmu/claim/ build/ccec/submit-pmu/efdrain/ +build/ccec/submit-pmu/materialize/ +build/ccec/submit-pmu/register/ ~~~ 每个 phase 目录中的 host、mixed kernel、owner 与 dispatcher 是同一构建 @@ -1824,7 +1865,7 @@ build/ccec/submit-pmu/efdrain/ | CNT9 | `0x0` | 未使用 | 这个取舍只影响 `submit-pmu` 诊断 ELF,不影响标准 `swimlane`。 -`claim` 在 begin/end 读 CNT8/CNT5,stop 后再读 tail;所有片段的软件 +任一现行局部 phase 在 begin/end 读 CNT8/CNT5,stop 后再读 tail;所有片段的软件 累加构成 shadow whole。两种构建的接受条件不同: ~~~text @@ -1832,7 +1873,7 @@ none(没有运行中 read-clear): shadow request == primary request shadow miss == primary miss -claim(运行中反复 read-clear): +局部 phase(运行中反复 read-clear): shadow request <= primary request shadow miss <= primary miss @@ -1852,12 +1893,13 @@ claim(运行中反复 read-clear): Submit 的 `miss <= request`、96 个唯一物理子核、owner bitmap/role/triplet、 真计算输出、Submit placement/engine、counter 风险门槛和 Restore。`none` 的 phase calls/begin/end/request/miss 必须全为 0,并要求 96/96 shadow 精确 -等于 primary;`claim` 要求 96/96 shadow 不大于 primary,exact 核数只作 +等于 primary;其余现行局部 phase 要求 96/96 shadow 不大于 primary, +exact 核数只作 诊断,不再伪装成逐事件精确切片。 局部 begin/end 读本身会增加 scalar 取指和改变多核时序,所以 -`claim` 是带观察边界扰动的归因 ELF。不同 phase 的局部 request/miss -不可相加,也不能用 `claim - none` 宣称得到零扰动 Claim 净值。 +所有局部 phase 都是带观察边界扰动的归因 ELF。不同 phase 的局部 +request/miss 不可相加,也不能用 `phase - none` 宣称得到零扰动净值。 每个 ELF 的完整 Submit 始终以它自己的 CNT6/CNT7 primary whole 为准。 当协议、数值输出和 placement/engine 门禁全部通过时,运行中 shadow 的 单向负差只能描述为局部 PMU 分段误差,不能描述成 standalone 调度异常。 @@ -1939,3 +1981,115 @@ owner/Restore 和分析器 raw 复算全部 PASS。该窗口既可能走空 ring outputs/submit_pmu_phases_20260718/efdrain_b1/run1.json outputs/submit_pmu_phases_20260718/efdrain_b256/run1.json ~~~ + +#### 7.5.18 描述性 I-cache 产物与现行五阶段 b256 复核 + +2026-07-19 使用相同的 `real-compute/6,28,4,1` 负载,分别为 `none`、 +`claim`、`efdrain`、`materialize` 和 `register` 启动一个独立 b256 进程。 +所有采集均通过 +语义、96 核拓扑、owner/Restore、raw 重算和 phase calls 门禁。产物统一命名为: + +```text +submit_icache_raw.json # 96 核权威原始件 +submit_icache_report.html # 自包含 HTML 加工件 +``` + +完整 Submit 的 primary whole 统计如下。每一行来自不同 phase ELF;边界读取会 +改变代码布局与运行时序,因此这些行用于检查各自采集是否合理,不能相减成 phase +净开销。 + +| phase | Submit/us | AIC request/core | AIC miss/core | AIC miss rate | AIV request/core | AIV miss/core | AIV miss rate | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| `none` | 4750.810 | 408317.344 | 38664.344 | 9.4692% | 422480.609 | 55098.625 | 13.0417% | +| `claim` | 5271.392 | 443636.531 | 25863.438 | 5.8299% | 445036.422 | 61462.859 | 13.8107% | +| `efdrain` | 4807.369 | 442476.094 | 15960.688 | 3.6071% | 442261.094 | 57427.188 | 12.9849% | +| `materialize` | 4195.642 | 434706.750 | 20468.656 | 4.7086% | 445711.047 | 56744.156 | 12.7312% | +| `register` | 4960.087 | 431990.906 | 26459.562 | 6.1250% | 438417.688 | 50043.422 | 11.4146% | + +同一批 raw 中的完整 Submit PMU total/scalar busy 如下。这些是每核 raw event +的均值,不是微秒;`scalar/total` 仅是两个同窗口事件求和的描述性比值, +不得称为 scalar 利用率或局部 phase 耗时占比。 + +| phase | ALL total/core | ALL scalar/core | scalar/total | +| --- | ---: | ---: | ---: | +| `none` | 7,213,914.333 | 5,717,308.729 | 79.2539% | +| `claim` | 6,878,355.458 | 5,371,850.677 | 78.0979% | +| `efdrain` | 6,910,059.479 | 5,499,963.656 | 79.5936% | +| `materialize` | 6,577,155.385 | 5,119,585.177 | 77.8389% | +| `register` | 6,913,673.615 | 5,524,170.406 | 79.9021% | + +局部 running read-clear 结果为: + +| phase | AIC/AIV/global calls | phase request lower..upper | request/Submit primary | phase miss lower..upper | miss/Submit primary | exact/bounded 核 | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| `claim` | 40960 / 81920 / 122880 | 2114523..2114676 | 4.9545%..4.9549% | 322606..322696 | 6.7757%..6.7775% | 36/96,96/96 | +| `efdrain` | 40960 / 81920 / 122880 | 3293444..3294284 | 7.7559%..7.7578% | 481284..481284 | 11.4972%..11.4972% | 73/96,96/96 | +| `materialize` | 40960 / 81920 / 122880 | 15285878..15286038 | 36.0209%..36.0213% | 1317810..1317818 | 30.7424%..30.7426% | 74/96,96/96 | +| `register` | 40960 / 81920 / 122880 | 5486034..5488868 | 13.0986%..13.1054% | 394420..395074 | 9.7400%..9.7562% | 36/96,96/96 | + +这里 `Claim()` 为每个 worker 的五类任务选择唯一 winner;EfDrain 是每次 +Submit 开头对已 ready 私有 ring slot 的机会式回收;Materialize 构造每个 worker +的任务 tensor 结果和本地 heap 状态;Register 包围两个互斥输出登记 call-site。 +四个局部 phase 都每核固定 `5*batches` 次。 +表中的占比只在每一行自己的 phase ELF 内计算:局部 lower/upper 分别除以 +该 ELF 的完整 Submit primary 总量;不能用另一份 `none` 作分母。 + +本机证据: + +~~~text +outputs/submit_pmu_none_20260719_b256_final/{submit_icache_raw.json,submit_icache_report.html} +outputs/submit_pmu_claim_20260719_b256/{submit_icache_raw.json,submit_icache_report.html} +outputs/submit_pmu_efdrain_20260719_b256/{submit_icache_raw.json,submit_icache_report.html} +outputs/submit_pmu_materialize_20260719_b256/{submit_icache_raw.json,submit_icache_report.html} +outputs/submit_pmu_register_20260719_b256/{submit_icache_raw.json,submit_icache_report.html} +~~~ + +#### 7.5.19 Materialize/Register b1+b256 真机闭环 + +2026-07-19 对新增的 Materialize 和 Register 分别完成 b1 与 b256 真实 A5 +独立进程采集。每个 worker 对 Alloc/QK/SF/PV/UP 五类 Submit 各执行一次 +该边界,所以 b1 固定 +5 calls/core,b256 固定 1,280 calls/core。AIC/AIV/global 闭合分别为: + +| phase | 规模 | AIC/AIV/global calls | exact/bounded | request loss | miss loss | Submit span | +| --- | --- | ---: | ---: | ---: | ---: | ---: | +| `materialize` | b1 | 160 / 320 / 480 | 94/96,96/96 | 2 | 0 | 64.391 us | +| `materialize` | b256 | 40,960 / 81,920 / 122,880 | 74/96,96/96 | 160 | 8 | 4,195.642 us | +| `register` | b1 | 160 / 320 / 480 | 93/96,96/96 | 2 | 1 | 134.418 us | +| `register` | b256 | 40,960 / 81,920 / 122,880 | 36/96,96/96 | 2,834 | 654 | 4,960.087 us | + +局部 request/miss 及它们占**同一 phase ELF**完整 Submit primary 的比例为: + +| phase | 规模 | phase request lower..upper | request/whole | phase miss lower..upper | miss/whole | +| --- | --- | ---: | ---: | ---: | ---: | +| `materialize` | b1 | 95,704..95,706 | 42.5425%..42.5434% | 4,920..4,920 | 28.8462%..28.8462% | +| `materialize` | b256 | 15,285,878..15,286,038 | 36.0209%..36.0213% | 1,317,810..1,317,818 | 30.7424%..30.7426% | +| `register` | b1 | 34,883..34,885 | 16.7122%..16.7131% | 1,910..1,911 | 11.0853%..11.0911% | +| `register` | b256 | 5,486,034..5,488,868 | 13.0986%..13.1054% | 394,420..395,074 | 9.7400%..9.7562% | + +完整 Submit 的 PMU total/scalar busy 仍由 primary whole gate 取得,与上表的局部 +request/miss 不是同一种切片口径: + +| phase | 规模 | PMU total sum | scalar busy sum | +| --- | --- | ---: | ---: | +| `materialize` | b1 | 2,596,263 | 2,074,125 | +| `materialize` | b256 | 631,406,917 | 491,480,177 | +| `register` | b1 | 2,670,764 | 2,075,880 | +| `register` | b256 | 663,712,667 | 530,320,359 | + +四轮的 `capture.accepted`、语义、真计算输出、placement/engine、96 核拓扑、 +phase call shape、primary/shadow bounded、counter 风险门槛、owner Restore 和离线 raw +复算均为 **PASS**。Materialize 在上层先保存 `MaterializeTask()` 布尔返回值、 +关闭 phase 后再处理 fatal,因此失败返回也不会留下未闭合边界。Register 的 +Alloc/非 Alloc call-site 互斥,因此没有重复计数。这些 PASS 证明当前工具与 +standalone 协议闭合,不把单轮 Submit span 当成无观察基线,也不把局部 +request/miss 比例解释成局部耗时比例。 + +权威 raw 和对应 HTML 加工件位于: + +~~~text +outputs/submit_pmu_materialize_20260719_b1/{submit_icache_raw.json,submit_icache_report.html} +outputs/submit_pmu_materialize_20260719_b256/{submit_icache_raw.json,submit_icache_report.html} +outputs/submit_pmu_register_20260719_b1/{submit_icache_raw.json,submit_icache_report.html} +outputs/submit_pmu_register_20260719_b256/{submit_icache_raw.json,submit_icache_report.html} +~~~ diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index d8d857ec0b..9c7fea80bb 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -217,7 +217,7 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" | 构建 | 后端 | 内容 | 构建命令 | 产物目录 | | --- | --- | --- | --- | --- | | `swimlane` | CCEC/AscendC/CPU | 普通阶段与 schema-v3 atomic(direct + PollBatch)合并采集;不配置 PMU | `./run.sh build ccec` 或 `./run.sh build all` | CCEC 为 `build/ccec/` | -| `submit-pmu` | 仅 CCEC | 每核完整 Submit PMU,并在编译期可选一个局部阶段;当前有 `none|claim|efdrain` | `./run.sh build-submit-pmu ccec ` | `build/ccec/submit-pmu//` | +| `submit-pmu` | 仅 CCEC | 每核完整 Submit PMU,并在编译期可选一个局部阶段;当前有 `none|claim|efdrain|materialize|register` | `./run.sh build-submit-pmu ccec ` | `build/ccec/submit-pmu//` | `./run.sh build all` 只构建三后端的 `swimlane` 产物;`submit-pmu` 必须按 phase 另行构建。`none` 是不做局部边界读取的完整 Submit @@ -234,7 +234,7 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" | `smoke` | 固定 b1/r1/`scalar-nop=0` 的快速语义回归 | 否,只做内存记录校验 | | `run` | 自行控制 batch、run、winner 负载和诊断参数 | 仅显式传入 `--swimlane-json` 时生成 raw | | `swimlane` | 单轮运行并自动生成 raw 和 Perfetto merged JSON | 是 | -| `build-submit-pmu` | 构建指定 `none|claim|efdrain` 的 CCEC PMU-only ELF | 否 | +| `build-submit-pmu` | 构建指定 `none|claim|efdrain|materialize|register` 的 CCEC PMU-only ELF | 否 | | `submit-pmu` | 单轮采集完整 Submit PMU,可选导出 JSON | 否,与泳道隔离 | `ccec|ascendc|cpu|all` 用于选择后端;`all` 始终按 CCEC、AscendC、CPU @@ -351,8 +351,8 @@ runner 结束时会打印准确目录: 字段和解读边界见 5.6 节。 WaitForSlot 和 HeapGuard 没有可伪造的逐事件起止时间,因此不单独生成 Perfetto -事件;实际发生等待时,泳道中会出现 RingBp 事件。CCEC 的局部 PMU -归因不复用运行时 `--profile-phases`,而是使用第 5.7 节的独立 +事件;实际发生等待时,泳道中会出现 RingBp 事件。现行 CCEC 局部 PMU +只支持 Claim、EfDrain、Materialize 和 Register,使用第 5.7 节的独立 `submit-pmu` phase ELF。 `outputs/` 已被 Git 忽略,生成的几十至数百 MiB 泳道文件不会被普通 @@ -395,7 +395,7 @@ CPU 完整协议回归建议关闭大泳道缓冲区: - `--profile-phases`:CPU/AscendC 兼容诊断中分别统计 Claim、EfDrain、 WaitForSlot、HeapGuard;最终 CCEC `swimlane` 构建不接受该选项, - CCEC 局部归因使用独立 `submit-pmu` phase; + CCEC 的独立 `submit-pmu` 局部归因只支持 Claim、EfDrain、Materialize、Register; - `--analyze-swimlane`:读取完整记录,输出各阶段的 per-worker 累计分布以及 EfDrain/Materialize/Claim/Register 的 per-role、per-task-kind 单事件分布; - `--trace-atomics`:在已开启的泳道中记录 atomic 逻辑调用;direct 调用逐条记录, @@ -684,11 +684,13 @@ phase/lap/Kernel 边界修复之前的 schema-v2 逐调用模型,只能用于 当前 `submit-pmu` 只支持: -| phase | 局部边界 | 用途 | -| --- | --- | --- | -| `none` | 不做任何中途 shadow counter 读取 | 完整 Submit 主基准,优先用于回答 AIC/AIV 每核 request/miss | -| `claim` | 每次 `Claim()` 调用前后读取 shadow counter | 验证局部归因链路,输出带观察扰动的 running read-clear 下界和保守上界 | -| `efdrain` | Submit 开头唯一的 EfDrain call-site 前后 | 归因 opportunistic drain,不包含 RingBackpressure/FinalDrain | +| phase | 编译期 ID | 局部边界 | 用途 | +| --- | ---: | --- | --- | +| `none` | 0 | 不做任何中途 shadow counter 读取 | 完整 Submit 主基准,优先用于回答 AIC/AIV 每核 request/miss | +| `claim` | 1 | 每次 `Claim()` 调用前后读取 shadow counter | 验证局部归因链路,输出带观察扰动的 running read-clear 下界和保守上界 | +| `efdrain` | 2 | Submit 开头唯一的 EfDrain call-site 前后 | 归因 opportunistic drain,不包含 RingBackpressure/FinalDrain | +| `materialize` | 4 | 每次 `MaterializeTask()` 调用前后 | 归因 descriptor materialize;成功和失败出口都由同一闭合边界覆盖 | +| `register` | 5 | 每次 `RegisterOutputs()` 调用前后 | 归因输出注册;Alloc 与非 Alloc 两个互斥调用点合起来仍是每次 Submit 一次 | 分别构建: @@ -696,6 +698,8 @@ phase/lap/Kernel 边界修复之前的 schema-v2 逐调用模型,只能用于 ./run.sh build-submit-pmu ccec none ./run.sh build-submit-pmu ccec claim ./run.sh build-submit-pmu ccec efdrain +./run.sh build-submit-pmu ccec materialize +./run.sh build-submit-pmu ccec register ``` 产物完全分开: @@ -704,6 +708,8 @@ phase/lap/Kernel 边界修复之前的 schema-v2 逐调用模型,只能用于 build/ccec/submit-pmu/none/ build/ccec/submit-pmu/claim/ build/ccec/submit-pmu/efdrain/ +build/ccec/submit-pmu/materialize/ +build/ccec/submit-pmu/register/ ``` 每个目录都自包含同 phase 的 host、mixed kernel、PMU owner 和 dispatcher, @@ -712,12 +718,56 @@ manifest;`submit-pmu` action 在启动 host 前逐项复核,缺件、串 pha 变化都会直接拒绝。一次正式采集示例: ```bash +export PYTHON=/home/q00473782/.venv/bin/python OUT="./outputs/submit_pmu_none_$(date -u +%Y%m%dT%H%M%SZ)" mkdir -p "$OUT" ./run.sh submit-pmu ccec none \ --device 0 --batches 256 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json "$OUT/run1.json" + --pmu-json "$OUT/submit_icache_raw.json" +``` + +raw 成功发布后,`run.sh` 会在同目录自动生成一份自包含的加工件: + +```text +submit_icache_raw.json # 96 核权威原始数据 +submit_icache_report.html # 浏览器直接打开的离线图表和汇总 +``` + +HTML 中包含 AIC/AIV 的每核 request、miss、miss rate、p95、96 核散点和 +局部 phase 的 lower/upper 区间,以及局部 request/miss 占同一 ELF 完整 +Submit primary 的比例区间。报告也展示 ALL/AIC/AIV 的逐核 PMU `total_cycles` +与 `scalar_busy` 的 mean/median/p95、scalar/total 比例和逐核散点。报告将 +`total_cycles-scalar_busy` 明确标为“非 Scalar-busy 残余”。`total_cycles` +是每个物理子核在 Submit gate 内的 64-bit PMU raw total,96 核求和是 core-work, +不是约 5 ms 的 Submit 墙钟;`scalar_busy` 是 CNT2 的 +`scalar_instr_busy(0x001)`。依赖返回值的 atomic 等待可以落入 scalar busy, +而 I-cache refill 的额外周期可能主要只增加 total,但 +**`total_cycles - scalar_busy` 既不能解释为 Scalar 空闲,也不能解释为 +I-cache stall**:差值还混有同步等待、Cube/Vector/MTE 等 engine 等待 +及其他非 scalar-busy 周期。2026-07-19 用本机 CANN 9.1 在 A5/DAV3510 +上依次实测 `PipeUtilization`、`PipeUtilization,MemoryDetail` 和 `Default`, +三份 `PipeUtilization.csv` 均没有 `scalar_wait_ib_time` 或 +`scalar_wait_time`;DAV3510 正式事件表也没有对应 selector/公式。因此 +当前 A5 正式可编程路径不采这两项,不套用其他产品的事件号。 +原始证据位于 `outputs/wait_ib_official_msopprof_20260719_b1_probe2/`、 +`outputs/wait_ib_official_msopprof_20260719_b1_probe3_memory_detail/` 和 +`outputs/wait_ib_official_msopprof_20260719_b1_probe4_default/`。 +报告只复用 `pmu_sidecar_analyzer.py` 已校验的统计口径;生成失败不会删除已经发布 +的 raw,但本次 action 会返回非零。 + +两类新增局部 phase 可按与 `none` 相同的参数分别运行;输出文件名应体现 phase, +避免误把不同 ELF 的结果放进同一组: + +```bash +for phase in materialize register; do + OUT="./outputs/submit_pmu_${phase}_$(date -u +%Y%m%dT%H%M%SZ)" + mkdir -p "$OUT" + ./run.sh submit-pmu ccec "$phase" \ + --device 0 --batches 256 \ + --winner-workload real-compute --real-compute-counts 6,28,4,1 \ + --pmu-json "$OUT/submit_icache_raw.json" +done ``` `submit-pmu` action 自己固定 `--runs 1 --no-swimlane --pmu-window submit-all`, @@ -731,8 +781,8 @@ mkdir -p "$OUT" 保持未使用;这会牺牲 PMU 诊断版的 `mte3_busy`,不影响标准 `swimlane` 构建。 -shadow 计数器是 read-to-clear:`claim` 在 begin/end 切分片段,stop 时再加 -tail,从而软件重建完整 Submit shadow whole。`none` 没有运行中读取,必须 +shadow 计数器是 read-to-clear:选中的局部 phase 在 begin/end 切分片段,stop 时 +再加 tail,从而软件重建完整 Submit shadow whole。`none` 没有运行中读取,必须 在每个物理子核精确满足: ```text @@ -740,7 +790,8 @@ CNT8 shadow whole request == CNT6 primary whole request CNT5 shadow whole miss == CNT7 primary whole miss ``` -`claim/efdrain` 在 A5 上运行中反复 read-clear 时,shadow 可能在边界处单向少计, +`claim/efdrain/materialize/register` 在 A5 上运行中反复 read-clear 时,shadow +可能在边界处单向少计, 因此接受条件改为逐核: ```text @@ -757,17 +808,22 @@ phase miss ∈ [observed miss, observed miss + miss loss] 区间必须逐核构造后再聚合。CNT8/CNT5 是顺序 `ld_dev` 而非原子配对快照, 不要求局部 `phase miss <= phase request`;只要求二者分别不超过对应 shadow, 上界分别不超过对应 primary。`none` 中 phase calls/begin/end/request/miss 必须 -全为 0;`claim` 中每核 begin/end/calls 必须配对,且每核 calls 必须等于 -`batches * 5`,全局为 `batches * 5 * 96`。`efdrain` 的 calls 形状相同; -但插点只允许包围 Submit 开头的 EfDrain 专属调用,不能插入复用的 -`DrainReady()` 函数体。 +全为 0;其余四个 phase 的每核 begin/end/calls 都必须配对,且每核 calls 固定为 +`batches * 5`,全局为 `batches * 5 * 96`。原因是每个 batch 固定提交 +Alloc/QK/SF/PV/UP 五个 task,每次 Submit 都恰好执行一次 Claim、开头 EfDrain、 +Materialize 和 Register 边界。`efdrain` 插点只允许包围 Submit 开头的专属调用, +不能插入复用的 `DrainReady()` 函数体;`materialize` 必须先保存真实返回值再关闭 +边界,保证失败出口也闭合;`register` 的 Alloc 与非 Alloc 两个源码调用点互斥, +不能误算成每次 Submit 两次。当前 Case1 中真实 TensorMap insert 工作主要发生在 +UP 的输出注册,其他 task 的 Register 可能很短或没有 insert;因此该 phase 的 +固定调用数只证明边界覆盖完整,不能解释为五类 task 拥有等量注册工作。 局部边界读取本身会增加 scalar 指令、改变 I-cache 布局和多核时序, -因此 `claim` 是带边界扰动的归因结果。`none` 与 `claim` 是不同 ELF/ -不同进程;今后新增的不同 phase 也必须各自单独采集。不同 phase +因此所有非 `none` phase 都是带边界扰动的归因结果。`none` 与每个局部 phase +是不同 ELF/不同进程;不同 phase 必须各自单独采集。不同 phase ELF 的局部 request/miss **不可相加**,也不能与 `none` 相减后宣称 得到了零扰动的阶段净值。这个区间只约束同一插桩 ELF、当前边界定义下的 -局部事件;它不是无插桩 Claim 的真实区间。调度语义、真计算输出和 +局部事件;它不是对应阶段在无插桩构建中的真实区间。调度语义、真计算输出和 placement/engine 门禁都通过时,running shadow 的负差属于观测边界行为, 不得描述为 standalone scheduler 异常。 @@ -785,7 +841,7 @@ ratio,不是实际 miss rate 的数学下界。更完整的 I-cache 采集、 以下 `empty/scalar/scalar-double/icache-single`、CNT8 fix-busy 和 schema-v3 文字保留为 2026-07-18 观察链路的建设过程与历史数据。当前 `swimlane` 构建不提供 PMU,当前 `submit-pmu` 也只接受完整 -Submit 的 `none|claim|efdrain`;不应继续照抄下文的历史校准命令作为当前用法。 +Submit 的 `none|claim|efdrain|materialize|register`;不应继续照抄下文的历史校准命令作为当前用法。 CCEC 后端提供与泳道分离的 PMU sidecar。正式取数由本目录自带的 Main AICPU Path-A owner 配置 selector、保存并恢复 PMU 状态;kernel 在每个物理子核内门控并 @@ -1046,7 +1102,9 @@ real-compute 等不同口径不能混合聚合。当前 JSON 没有记录 ELF 分析器回归可独立执行: ```bash -python -m unittest -v test_pmu_sidecar_analyzer.py +python -m unittest -v \ + test_pmu_sidecar_analyzer.py \ + test_pmu_html_report.py ``` ## 6. 当前 A5 结果与真实 PA 的差异 diff --git a/tests/atomic_probe/pa_scheduler/ccec/build.sh b/tests/atomic_probe/pa_scheduler/ccec/build.sh index e14c15a82a..a91682f5cc 100755 --- a/tests/atomic_probe/pa_scheduler/ccec/build.sh +++ b/tests/atomic_probe/pa_scheduler/ccec/build.sh @@ -33,7 +33,7 @@ case "$BUILD_VARIANT" in ;; submit-pmu) if [[ $# -ne 2 ]]; then - echo "Usage: $0 submit-pmu none|claim|efdrain" >&2 + echo "Usage: $0 submit-pmu none|claim|efdrain|materialize|register" >&2 exit 1 fi PHASE_NAME="$2" @@ -41,8 +41,10 @@ case "$BUILD_VARIANT" in none) PHASE_ID=0 ;; claim) PHASE_ID=1 ;; efdrain) PHASE_ID=2 ;; + materialize) PHASE_ID=4 ;; + register) PHASE_ID=5 ;; *) - echo "Unknown submit-pmu phase: $PHASE_NAME (expected none|claim|efdrain)" >&2 + echo "Unknown submit-pmu phase: $PHASE_NAME (expected none|claim|efdrain|materialize|register)" >&2 exit 1 ;; esac @@ -50,7 +52,7 @@ case "$BUILD_VARIANT" in VARIANT_DEFINES=(-DPA_BUILD_SWIMLANE=0 -DPA_BUILD_SUBMIT_PMU=1 "-DPA_SUBMIT_PMU_PHASE_ID=$PHASE_ID") ;; *) - echo "Usage: $0 [swimlane] | $0 submit-pmu none|claim|efdrain" >&2 + echo "Usage: $0 [swimlane] | $0 submit-pmu none|claim|efdrain|materialize|register" >&2 exit 1 ;; esac diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index 6de9eb3e6c..d4c80dbf44 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -467,9 +467,11 @@ struct PmuValidation { uint32_t phase_status_trusted = 0; uint32_t shadow_primary_matches = 0; uint32_t shadow_primary_bounded = 0; + uint32_t phase_shadow_acceptable = 0; uint32_t phase_boundary_matches = 0; uint32_t phase_call_shape_matches = 0; uint64_t phase_calls = 0; + uint64_t expected_phase_calls = 0; uint64_t shadow_request_abs_delta_sum = 0; uint64_t shadow_miss_abs_delta_sum = 0; int64_t shadow_request_signed_delta_sum = 0; @@ -488,17 +490,15 @@ struct PmuValidation { // 高水位作为保守拒绝阈值;它只降低风险,不把“未越线”表述成回卷证明。 constexpr uint32_t kProgrammableCounterRiskThreshold = UINT32_MAX / 4U; -uint32_t ExpectedSubmitPmuPhaseCallsPerWorker( - const pa_scheduler::WorkerResult &result, uint32_t batches -) { - // 固定流阶段按每核完整回放次数闭合;保留 result 入参,后续 winner-only - // 阶段必须从本核真实 wins/heap_guards 推导,不能继续套用固定 5B。 - (void)result; +uint32_t ExpectedSubmitPmuPhaseCallsPerWorker(uint32_t batches) { + // 当前所有 running phase 都覆盖每个 worker 的每次 Submit,固定为 5B。 switch (pa_scheduler::kCompiledSubmitPmuPhase) { case pa_scheduler::SubmitPmuPhase::None: return 0U; case pa_scheduler::SubmitPmuPhase::Claim: case pa_scheduler::SubmitPmuPhase::EfDrain: + case pa_scheduler::SubmitPmuPhase::Materialize: + case pa_scheduler::SubmitPmuPhase::Register: return batches * pa_scheduler::kTasksPerBatch; case pa_scheduler::SubmitPmuPhase::Count: break; @@ -536,6 +536,7 @@ bool ValidatePmu( uint32_t phase_status_trusted = 0; uint32_t shadow_primary_matches = 0; uint32_t shadow_primary_bounded = 0; + uint32_t phase_shadow_acceptable = 0; uint32_t phase_boundary_matches = 0; uint32_t phase_call_shape_matches = 0; uint64_t phase_calls = 0; @@ -554,7 +555,7 @@ bool ValidatePmu( for (uint32_t worker = 0; worker < pa_scheduler::kWorkers; ++worker) { const pa_scheduler::WorkerResult &result = state.results[worker]; const uint32_t expected_phase_calls_per_worker = - ExpectedSubmitPmuPhaseCallsPerWorker(result, state.config.batches); + ExpectedSubmitPmuPhaseCallsPerWorker(state.config.batches); const uint32_t status = result.pmu_status; const uint32_t core_id = StatusCoreId(status); const bool record_trusted = (status & kStatusRequired) == kStatusRequired; @@ -569,8 +570,10 @@ bool ValidatePmu( const bool shadow_bounded = result.pmu_shadow_icache_requests <= result.pmu_icache_requests && result.pmu_shadow_icache_misses <= result.pmu_icache_misses; + const bool phase_requires_exact_shadow = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None; const bool shadow_acceptable = - pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + phase_requires_exact_shadow ? shadow_matches : shadow_bounded; const uint32_t request_abs_delta = @@ -594,6 +597,7 @@ bool ValidatePmu( phase_status_trusted += phase_status_ok; shadow_primary_matches += shadow_matches; shadow_primary_bounded += shadow_bounded; + phase_shadow_acceptable += shadow_acceptable; phase_boundary_matches += boundaries_match; phase_call_shape_matches += phase_call_shape_matches_record; phase_calls += result.pmu_phase_calls; @@ -725,12 +729,7 @@ bool ValidatePmu( const bool build_variant_ok = build_variant_matches == pa_scheduler::kWorkers; const bool phase_id_ok = phase_id_matches == pa_scheduler::kWorkers; const bool phase_status_ok = phase_status_trusted == pa_scheduler::kWorkers; - const bool shadow_primary_exact_ok = shadow_primary_matches == pa_scheduler::kWorkers; - const bool shadow_primary_bounded_ok = shadow_primary_bounded == pa_scheduler::kWorkers; - const bool shadow_partition_ok = - pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None - ? shadow_primary_exact_ok - : shadow_primary_bounded_ok; + const bool shadow_partition_ok = phase_shadow_acceptable == pa_scheduler::kWorkers; const bool phase_boundaries_ok = phase_boundary_matches == pa_scheduler::kWorkers; const bool phase_call_shape_ok = phase_call_shape_matches == pa_scheduler::kWorkers; const bool phase_calls_ok = phase_calls == expected_phase_calls; @@ -780,9 +779,7 @@ bool ValidatePmu( build_variant_ok && phase_id_ok ? "PASS" : "FAIL"); std::printf( "[ASSERT] %-48s %s\n", - pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None - ? "disabled shadow counters exactly match primary" - : "running shadow partitions do not exceed primary", + "phase shadow partitions satisfy exact-or-bounded contract", shadow_partition_ok ? "PASS" : "FAIL" ); std::printf("[ASSERT] %-48s %s\n", "all phase boundaries and per-worker calls are exact", @@ -824,9 +821,11 @@ bool ValidatePmu( validation->phase_status_trusted = phase_status_trusted; validation->shadow_primary_matches = shadow_primary_matches; validation->shadow_primary_bounded = shadow_primary_bounded; + validation->phase_shadow_acceptable = phase_shadow_acceptable; validation->phase_boundary_matches = phase_boundary_matches; validation->phase_call_shape_matches = phase_call_shape_matches; validation->phase_calls = phase_calls; + validation->expected_phase_calls = expected_phase_calls; validation->shadow_request_abs_delta_sum = shadow_request_abs_delta_sum; validation->shadow_miss_abs_delta_sum = shadow_miss_abs_delta_sum; validation->shadow_request_signed_delta_sum = shadow_request_signed_delta_sum; @@ -1213,12 +1212,13 @@ bool ExportPmuJson( "\"build_variant_match_records\":%u,\"phase_id_match_records\":%u," "\"phase_status_trusted_records\":%u,\"shadow_primary_match_records\":%u," "\"shadow_primary_bounded_records\":%u," + "\"phase_shadow_acceptable_records\":%u," "\"shadow_request_abs_delta_sum\":%llu,\"shadow_request_abs_delta_max\":%u," "\"shadow_request_signed_delta_sum\":%lld," "\"shadow_miss_abs_delta_sum\":%llu,\"shadow_miss_abs_delta_max\":%u," "\"shadow_miss_signed_delta_sum\":%lld," "\"phase_boundary_match_records\":%u,\"phase_call_shape_match_records\":%u," - "\"phase_calls\":%llu," + "\"phase_calls\":%llu,\"phase_expected_calls\":%llu," "\"phase_measurement_valid\":%s},\n", semantic_passed ? "true" : "false", validation.passed ? "true" : "false", real_compute ? "true" : "false", @@ -1248,7 +1248,7 @@ bool ExportPmuJson( UINT32_MAX - validation.maximum_programmable_counter, validation.build_variant_matches, validation.phase_id_matches, validation.phase_status_trusted, validation.shadow_primary_matches, - validation.shadow_primary_bounded, + validation.shadow_primary_bounded, validation.phase_shadow_acceptable, static_cast(validation.shadow_request_abs_delta_sum), validation.shadow_request_abs_delta_max, static_cast(validation.shadow_request_signed_delta_sum), @@ -1258,6 +1258,7 @@ bool ExportPmuJson( validation.phase_boundary_matches, validation.phase_call_shape_matches, static_cast(validation.phase_calls), + static_cast(validation.expected_phase_calls), validation.phase_measurement_valid ? "true" : "false" ); std::fprintf( @@ -1321,6 +1322,8 @@ bool ExportPmuJson( const bool boundaries_balanced = result.pmu_phase_begin_reads == result.pmu_phase_calls && result.pmu_phase_end_reads == result.pmu_phase_calls; + const uint32_t expected_phase_calls = + ExpectedSubmitPmuPhaseCallsPerWorker(state.config.batches); std::fprintf( output, "%s{\"worker_id\":%u,\"physical_core_id\":%u,\"role\":\"%s\",\"block_id\":%u," @@ -1329,6 +1332,7 @@ bool ExportPmuJson( "\"cube_busy\":%u,\"scalar_busy\":%u,\"mte1_busy\":%u,\"mte2_busy\":%u," "\"icache_requests\":%u,\"icache_misses\":%u," "\"build_variant_id\":%u,\"compiled_phase_id\":%u,\"phase_calls\":%u," + "\"phase_expected_calls\":%u," "\"phase_begin_reads\":%u,\"phase_end_reads\":%u," "\"phase_icache_requests\":%u,\"phase_icache_misses\":%u," "\"phase_icache_requests_upper_bound\":%u," @@ -1348,6 +1352,7 @@ bool ExportPmuJson( result.pmu_cube_busy, result.pmu_scalar_busy, result.pmu_mte1_busy, result.pmu_mte2_busy, result.pmu_icache_requests, result.pmu_icache_misses, result.pmu_build_variant, result.pmu_phase_id, result.pmu_phase_calls, + expected_phase_calls, result.pmu_phase_begin_reads, result.pmu_phase_end_reads, result.pmu_phase_icache_requests, result.pmu_phase_icache_misses, phase_request_upper, phase_miss_upper, diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index a18e09b3a9..319dfe67e2 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -640,17 +640,12 @@ __aicore__ inline void CcecOps::PmuWindowStop( pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None && context.phase_calls == 0 && context.begin_reads == 0 && context.end_reads == 0 && context.phase_requests == 0 && context.phase_misses == 0; - const bool claim_shape = - pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::Claim && + const bool running_shape = + pa_scheduler::kCompiledSubmitPmuPhase != pa_scheduler::SubmitPmuPhase::None && context.phase_calls == state->config.batches * pa_scheduler::kTasksPerBatch && context.begin_reads == context.phase_calls && context.end_reads == context.phase_calls; - const bool efdrain_shape = - pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::EfDrain && - context.phase_calls == state->config.batches * pa_scheduler::kTasksPerBatch && - context.begin_reads == context.phase_calls && - context.end_reads == context.phase_calls; - if (none_shape || claim_shape || efdrain_shape) + if (none_shape || running_shape) context.phase_status |= kPhaseStatusPhaseShape; PublishPmuSnapshot(result, sample); diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h index efa9d6ca32..16db1e4f1b 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h @@ -152,6 +152,10 @@ inline const char *SubmitPmuPhaseName(SubmitPmuPhase phase) { return "claim"; case SubmitPmuPhase::EfDrain: return "efdrain"; + case SubmitPmuPhase::Materialize: + return "materialize"; + case SubmitPmuPhase::Register: + return "register"; case SubmitPmuPhase::Count: break; } diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index eff3b709ad..14613cf4cf 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -183,14 +183,16 @@ enum class ProfilePhase : uint32_t { }; // submit-pmu 每个 ELF 只编译一个局部归因阶段。none 不做中途 counter -// 读取,是完整 Submit 的正式基线;claim 是首个连续、无提前返回的验证阶段, -// EfDrain 只包围每次 Submit 开头唯一的 opportunistic drain call-site。 -// 后续阶段只能在各自边界和闭环经过 A5 验证后向枚举尾部追加。 +// 读取,是完整 Submit 的正式基线;其余阶段都在每个 worker 的五次 Submit +// 上各执行一次,因此统一按固定 5*batches 次数闭合。历史 ID=3 曾用于 +// winner-only WaitForSlot,现已退役且不复用,避免旧 raw 被误认成新阶段。 enum class SubmitPmuPhase : uint32_t { None = 0, Claim = 1, EfDrain = 2, - Count = 3, + Materialize = 4, + Register = 5, + Count = 6, }; #ifndef PA_SUBMIT_PMU_PHASE_ID @@ -202,8 +204,11 @@ constexpr SubmitPmuPhase kCompiledSubmitPmuPhase = constexpr uint32_t kBuildVariantSwimlane = 1U; constexpr uint32_t kBuildVariantSubmitPmu = 2U; static_assert( - PA_SUBMIT_PMU_PHASE_ID >= 0 && - PA_SUBMIT_PMU_PHASE_ID < static_cast(SubmitPmuPhase::Count), + PA_SUBMIT_PMU_PHASE_ID == static_cast(SubmitPmuPhase::None) || + PA_SUBMIT_PMU_PHASE_ID == static_cast(SubmitPmuPhase::Claim) || + PA_SUBMIT_PMU_PHASE_ID == static_cast(SubmitPmuPhase::EfDrain) || + PA_SUBMIT_PMU_PHASE_ID == static_cast(SubmitPmuPhase::Materialize) || + PA_SUBMIT_PMU_PHASE_ID == static_cast(SubmitPmuPhase::Register), "invalid compiled submit-pmu phase" ); diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index 268b218475..d4237df070 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -577,7 +577,11 @@ PA_DEVICE bool SubmitTask( // Build/Replay 会从这个起点形成覆盖式 span。因此泳道上的这些阶段不能直接相加。 ResetTraceLap(stats.trace, stats.result, worker); const uint64_t materialize_begin = TraceTimestamp(stats.trace, stats.result); - if (!MaterializeTask(worker, task_id, args, context, state->heap_base, state->heap_size)) { + BeginSubmitPmuPhase(pmu_context); + const bool materialized = + MaterializeTask(worker, task_id, args, context, state->heap_base, state->heap_size); + EndSubmitPmuPhase(pmu_context); + if (!materialized) { SetFatal(state, stats, static_cast(task_id)); return false; } @@ -603,7 +607,9 @@ PA_DEVICE bool SubmitTask( if (kind == TaskKind::Alloc) { // Alloc 没有 kernel lane,96 个 worker 都维护本地物化/heap 状态,但只有 Claim winner 发布全局完成。 const uint64_t register_begin = TraceTimestamp(stats.trace, stats.result); + BeginSubmitPmuPhase(pmu_context); RegisterOutputs(context, args, false); + EndSubmitPmuPhase(pmu_context); const uint64_t register_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Register, @@ -671,7 +677,9 @@ PA_DEVICE bool SubmitTask( } const uint64_t register_begin = TraceTimestamp(stats.trace, stats.result); + BeginSubmitPmuPhase(pmu_context); RegisterOutputs(context, args, true); + EndSubmitPmuPhase(pmu_context); const uint64_t register_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Register, diff --git a/tests/atomic_probe/pa_scheduler/pmu_html_report.py b/tests/atomic_probe/pa_scheduler/pmu_html_report.py new file mode 100644 index 0000000000..020477f5c2 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/pmu_html_report.py @@ -0,0 +1,750 @@ +#!/usr/bin/env python3 +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- +"""将 standalone submit-pmu raw JSON 转成可离线浏览的自包含 HTML 报告。""" + +from __future__ import annotations + +import argparse +import hashlib +import html +import math +import os +import sys +import tempfile +from datetime import datetime, timezone +from pathlib import Path +from typing import Any, Sequence +from urllib.parse import quote + +try: + from .pmu_sidecar_analyzer import analyze, load_capture +except ImportError: + from pmu_sidecar_analyzer import analyze, load_capture + + +REPORT_VERSION = 2 +AIC_COLOR = "#2563eb" +AIV_COLOR = "#ea580c" +GRID_COLOR = "#cbd5e1" +TEXT_COLOR = "#334155" + + +def default_output_path(input_path: Path) -> Path: + """由描述性 raw 名称稳定推导报告名称。""" + + name = input_path.name + if name.endswith("_raw.json"): + return input_path.with_name(f"{name[:-len('_raw.json')]}_report.html") + if name.endswith(".json"): + return input_path.with_name(f"{name[:-len('.json')]}_report.html") + return input_path.with_name(f"{name}_report.html") + + +def _format_count(value: int | float) -> str: + return f"{value:,.0f}" + + +def _format_per_core(value: int | float) -> str: + return f"{value:,.2f}" + + +def _format_rate(value: int | float) -> str: + return f"{value * 100:.4f}%" + + +def _escape(value: object) -> str: + return html.escape(str(value), quote=True) + + +def _raw_href(input_path: Path, output_path: Path) -> str: + relative = os.path.relpath(input_path, output_path.parent) + return quote(relative.replace(os.sep, "/"), safe="/") + + +def _group_metrics( + analysis_group: dict[str, Any], raw_group: dict[str, Any] +) -> dict[str, int | float]: + cores = int(analysis_group["cores"]) + total_sum = int(analysis_group["total_cycles_sum"]) + scalar_sum = int(analysis_group["scalar_busy_sum"]) + return { + "cores": cores, + "total_sum": total_sum, + "total_per_core": total_sum / cores, + "total_median": raw_group["total_cycles"]["median"], + "total_p95": raw_group["total_cycles"]["p95"], + "scalar_sum": scalar_sum, + "scalar_per_core": scalar_sum / cores, + "scalar_median": raw_group["scalar_busy"]["median"], + "scalar_p95": raw_group["scalar_busy"]["p95"], + "scalar_share": 0.0 if total_sum == 0 else scalar_sum / total_sum, + "non_scalar_busy_per_core": (total_sum - scalar_sum) / cores, + "requests_per_core": analysis_group["icache_requests_per_core"], + "requests_p95": raw_group["icache_requests"]["p95"], + "misses_per_core": analysis_group["icache_misses_per_core"], + "misses_median": analysis_group["icache_misses_per_core_median"], + "misses_p95": analysis_group["icache_misses_per_core_p95"], + "misses_max": raw_group["icache_misses"]["max"], + "miss_rate": analysis_group["icache_miss_rate"], + "serial_equivalent_us": analysis_group["first_order_miss_per_core_us"], + } + + +def _plot_value(record: dict[str, Any], metric: str) -> float: + if metric == "icache_miss_rate": + requests = int(record["icache_requests"]) + return 0.0 if requests == 0 else int(record["icache_misses"]) / requests + return float(record[metric]) + + +def _plot_label(metric: str, value: float) -> str: + if metric == "icache_miss_rate": + return f"{value * 100:.2f}%" + return f"{value:,.0f}" + + +def _distribution_svg( + records: Sequence[dict[str, Any]], + metric: str, + title: str, + description: str, + p95_by_role: dict[str, float] | None = None, +) -> str: + """按 physical_core_id 绘制离散点;核编号只是位置,不连接成时间线。""" + + width = 1080 + height = 310 + left, right, top, bottom = 78, 28, 34, 54 + plot_width = width - left - right + plot_height = height - top - bottom + points = [(record, _plot_value(record, metric)) for record in records] + maximum = max(value for _, value in points) + maximum = maximum * 1.08 if maximum > 0 else 1.0 + + def x_position(physical_id: int) -> float: + return left + plot_width * physical_id / 107.0 + + def y_position(value: float) -> float: + return top + plot_height * (1.0 - value / maximum) + + title_id = f"plot-{metric}-title" + desc_id = f"plot-{metric}-desc" + fragments = [ + f'', + f'{_escape(title)}', + f'{_escape(description)}', + ] + for index in range(5): + ratio = index / 4 + value = maximum * (1.0 - ratio) + y = top + plot_height * ratio + fragments.append( + f'' + ) + fragments.append( + f'' + f'{_escape(_plot_label(metric, value))}' + ) + + for tick in (0, 18, 36, 54, 72, 90, 107): + x = x_position(tick) + fragments.append( + f'' + ) + fragments.append( + f'{tick}' + ) + fragments.append( + f'physical_core_id(0–107,未连接)' + ) + + if p95_by_role is not None: + for role, color in (("aic", AIC_COLOR), ("aiv", AIV_COLOR)): + value = p95_by_role[role] + y = y_position(value) + fragments.append( + f'' + ) + fragments.append( + f'{role.upper()} p95 ' + f'{_escape(_plot_label(metric, value))}' + ) + + for record, value in sorted(points, key=lambda item: int(item[0]["physical_core_id"])): + role = str(record["role"]) + physical_id = int(record["physical_core_id"]) + x = x_position(physical_id) + y = y_position(value) + requests = int(record["icache_requests"]) + misses = int(record["icache_misses"]) + rate = 0.0 if requests == 0 else misses / requests + tooltip = ( + f"{role.upper()} worker={record['worker_id']} physical={physical_id} " + f"block={record['block_id']} lane={record['lane']} " + f"{metric}={_plot_label(metric, value)} whole_request={requests:,} " + f"whole_miss={misses:,} whole_rate={rate * 100:.4f}%" + ) + if role == "aic": + fragments.append( + f'' + f'{_escape(tooltip)}' + ) + else: + fragments.append( + f'{_escape(tooltip)}' + ) + fragments.append("") + return "".join(fragments) + + +def _per_core_rows(records: Sequence[dict[str, Any]]) -> str: + rows: list[str] = [] + for record in sorted(records, key=lambda item: int(item["physical_core_id"])): + requests = int(record["icache_requests"]) + misses = int(record["icache_misses"]) + total = int(record["total_cycles"]) + scalar = int(record["scalar_busy"]) + rate = 0.0 if requests == 0 else misses / requests + scalar_share = 0.0 if total == 0 else scalar / total + exact = bool(record.get("shadow_matches_primary")) + rows.append( + f'' + f"{int(record['worker_id'])}" + f"{int(record['physical_core_id'])}" + f"{_escape(str(record['role']).upper())}" + f"{int(record['block_id'])}" + f"{int(record['lane'])}" + f"{total:,}" + f"{scalar:,}" + f"{scalar_share * 100:.4f}%" + f"{requests:,}" + f"{misses:,}" + f"{rate * 100:.4f}%" + f"{'是' if exact else '否'}" + f"{'PASS' if record.get('trusted') is True else 'FAIL'}" + "" + ) + return "".join(rows) + + +def _phase_group_row(label: str, group: dict[str, Any]) -> str: + ratio = group["phase_observed_read_clear_ratio"] + ratio_text = "—" if ratio is None else f"{ratio * 100:.4f}%" + request_share = _format_share_bounds( + group["phase_icache_request_lower_bound_share_of_submit"], + group["phase_icache_request_upper_bound_share_of_submit"], + ) + miss_share = _format_share_bounds( + group["phase_icache_miss_lower_bound_share_of_submit"], + group["phase_icache_miss_upper_bound_share_of_submit"], + ) + return ( + "" + f"{_escape(label)}" + f"{int(group['phase_calls_sum']):,}" + f"{group['phase_icache_requests_lower_bound_sum']:,}..{group['phase_icache_requests_upper_bound_sum']:,}" + f"{group['phase_icache_requests_lower_bound_per_core']:,.3f}..{group['phase_icache_requests_upper_bound_per_core']:,.3f}" + f"{request_share}" + f"{group['phase_icache_misses_lower_bound_sum']:,}..{group['phase_icache_misses_upper_bound_sum']:,}" + f"{group['phase_icache_misses_lower_bound_per_core']:,.3f}..{group['phase_icache_misses_upper_bound_per_core']:,.3f}" + f"{miss_share}" + f"{int(group['shadow_request_loss_sum']):,} / {int(group['shadow_miss_loss_sum']):,}" + f"{ratio_text}" + "" + ) + + +def _format_share_bounds(lower: float | None, upper: float | None) -> str: + """将局部事件占同组完整 Submit primary 的比例格式化为区间。""" + + if lower is None or upper is None: + return "—" + return f"{lower * 100:.4f}%..{upper * 100:.4f}%" + + +def _phase_share_metric( + group_label: str, + label: str, + metric_class: str, + lower_share: float | None, + upper_share: float | None, + lower_count: int, + upper_count: int, +) -> str: + """生成一个以完整 Submit primary 为 100% 的局部占比区间条。""" + + if lower_share is None or upper_share is None: + return ( + f'
' + f'
{_escape(label)}
' + '
完整窗口分母为 0,比例不可计算
' + "
" + ) + lower_percent = lower_share * 100.0 + upper_percent = upper_share * 100.0 + bounds_text = f"{lower_percent:.4f}%..{upper_percent:.4f}%" + aria_label = ( + f"{label} 局部占完整 Submit primary," + f"下界 {lower_percent:.4f}%,上界 {upper_percent:.4f}%" + ) + return f""" +
+
{_escape(label)}{bounds_text}
+ +
0%50%100%
+
事件数 {lower_count:,}..{upper_count:,}
+
+""" + + +def _phase_share_card(label: str, group: dict[str, Any]) -> str: + """把一个角色组的 request/miss 局部占比放在同一张响应式卡片中。""" + + request_metric = _phase_share_metric( + label, + "I-cache request", + "request", + group["phase_icache_request_lower_bound_share_of_submit"], + group["phase_icache_request_upper_bound_share_of_submit"], + int(group["phase_icache_requests_lower_bound_sum"]), + int(group["phase_icache_requests_upper_bound_sum"]), + ) + miss_metric = _phase_share_metric( + label, + "I-cache miss", + "miss", + group["phase_icache_miss_lower_bound_share_of_submit"], + group["phase_icache_miss_upper_bound_share_of_submit"], + int(group["phase_icache_misses_lower_bound_sum"]), + int(group["phase_icache_misses_upper_bound_sum"]), + ) + role_class = "" if label == "ALL" else f" role-{label.lower()}" + return f""" +
+
{_escape(label)}{int(group['phase_calls_sum']):,} calls
+ {request_metric} + {miss_metric} +
+""" + + +def render_report( + input_path: Path, + output_path: Path | None = None, + miss_penalty_ns: float = 90.0, +) -> str: + """先通过独立 analyzer 门禁,再生成一个无需网络的完整 HTML 字符串。""" + + input_path = Path(input_path) + output_path = default_output_path(input_path) if output_path is None else Path(output_path) + # 原始件只读取一次;analyzer 与页面元数据都基于同一份私有快照,避免生成期间 + # raw 被替换后,统计值、元数据和页面 SHA256 分属不同版本。 + raw_bytes = input_path.read_bytes() + raw_digest = hashlib.sha256(raw_bytes).hexdigest() + with tempfile.TemporaryDirectory(prefix="pa-submit-pmu-report-") as snapshot_directory: + snapshot_path = Path(snapshot_directory) / input_path.name + snapshot_path.write_bytes(raw_bytes) + analysis = analyze([snapshot_path], miss_penalty_ns) + capture = load_capture(snapshot_path) + if capture.schema_version != 4: + raise ValueError("HTML report requires submit-pmu schema-v4 input") + + data = capture.data + configuration = data["configuration"] + validation = data["validation"] + summary = data["summary"] + records = data["records"] + per_run = analysis["per_run"][0] + all_metrics = _group_metrics(per_run["groups"]["all"], summary["all"]) + aic = _group_metrics(per_run["groups"]["aic"], summary["aic"]) + aiv = _group_metrics(per_run["groups"]["aiv"], summary["aiv"]) + submit_us = float(per_run["submit_span_us"]) + aic_request_per_core = float(aic["requests_per_core"]) + aiv_request_per_core = float(aiv["requests_per_core"]) + request_delta_percent = ( + None + if aic_request_per_core == 0.0 + else (aiv_request_per_core / aic_request_per_core - 1.0) * 100.0 + ) + aic_miss_per_core = float(aic["misses_per_core"]) + aiv_miss_per_core = float(aiv["misses_per_core"]) + delta_misses = aiv_miss_per_core - aic_miss_per_core + delta_percent = ( + None if aic_miss_per_core == 0.0 else delta_misses / aic_miss_per_core * 100.0 + ) + delta_rate_pp = (float(aiv["miss_rate"]) - float(aic["miss_rate"])) * 100.0 + request_comparison = ( + "AIC request/core 为 0,AIV 相对变化不可计算" + if request_delta_percent is None + else f"AIV 的 request/core 相比 AIC {'高' if request_delta_percent >= 0 else '低'} " + f"{abs(request_delta_percent):.2f}%" + ) + miss_comparison = ( + "AIC miss/core 为 0,AIV 相对变化不可计算" + if delta_percent is None + else f"miss/core {'高' if delta_percent >= 0 else '低'} {abs(delta_percent):.2f}%" + f"({delta_misses:+,.2f}/core)" + ) + rate_comparison = ( + f"miss rate {'高' if delta_rate_pp >= 0 else '低'} {abs(delta_rate_pp):.3f} 个百分点" + ) + total_requests = int(per_run["groups"]["all"]["icache_requests_sum"]) + total_misses = int(per_run["groups"]["all"]["icache_misses_sum"]) + phase = str(configuration["compiled_phase"]) + workload = configuration.get("winner_workload") or {} + workload_counts = workload.get("counts") or {} + exact_records = int(validation["shadow_primary_match_records"]) + bounded_records = int(validation["shadow_primary_bounded_records"]) + workers = int(configuration["workers"]) + generated_at = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC") + raw_link = _raw_href(input_path, output_path) + + total_plot = _distribution_svg( + records, + "total_cycles", + "逐物理核 PMU total cycle", + "每核 Submit gate 内的 PMU raw total;96 核求和是 core-work,不是墙钟时间。", + { + "aic": float(summary["aic"]["total_cycles"]["p95"]), + "aiv": float(summary["aiv"]["total_cycles"]["p95"]), + }, + ) + scalar_plot = _distribution_svg( + records, + "scalar_busy", + "逐物理核 scalar busy cycle", + "CNT2 scalar_instr_busy(0x001) 的每核累计;它不包含全部等待周期。", + { + "aic": float(summary["aic"]["scalar_busy"]["p95"]), + "aiv": float(summary["aiv"]["scalar_busy"]["p95"]), + }, + ) + request_plot = _distribution_svg( + records, + "icache_requests", + "逐物理核 I-cache request", + "96 个实测物理子核的 request 离散分布;AIC 为圆点,AIV 为方点。", + { + "aic": float(summary["aic"]["icache_requests"]["p95"]), + "aiv": float(summary["aiv"]["icache_requests"]["p95"]), + }, + ) + miss_plot = _distribution_svg( + records, + "icache_misses", + "逐物理核 I-cache miss", + "96 个实测物理子核的 miss 离散分布;虚线是各角色 nearest-rank p95。", + {"aic": float(aic["misses_p95"]), "aiv": float(aiv["misses_p95"])}, + ) + rate_plot = _distribution_svg( + records, + "icache_miss_rate", + "逐物理核 I-cache miss rate", + "每核 miss/request,仅用于观察离散分布;汇总 rate 仍按总 miss 除以总 request。", + ) + per_core_rows = _per_core_rows(records) + if phase == "none": + shadow_badge = f"PRIMARY ↔ SHADOW EXACT {exact_records}/{workers}" + phase_section = """ +
+

局部 phase:none

+

完整 Submit 中未执行任何 running read-clear 边界;局部 calls、request 和 miss 全部为 0。此构建是完整 Submit I-cache 的主观察口径。

+
+""" + else: + shadow_badge = f"SHADOW BOUNDED {bounded_records}/{workers}(exact {exact_records}/{workers})" + phase_groups = per_run["groups"] + phase_share_cards = "".join( + ( + _phase_share_card("ALL", phase_groups["all"]), + _phase_share_card("AIC", phase_groups["aic"]), + _phase_share_card("AIV", phase_groups["aiv"]), + ) + ) + phase_plot = _distribution_svg( + records, + "phase_icache_misses", + f"{phase} 局部 I-cache miss 观测下界", + "running read-clear 的逐核观测下界;上界还需加该核 primary-shadow residual。", + { + "aic": float(summary["aic"]["phase_icache_misses"]["p95"]), + "aiv": float(summary["aiv"]["phase_icache_misses"]["p95"]), + }, + ) + phase_rows = "".join( + ( + _phase_group_row("ALL", phase_groups["all"]), + _phase_group_row("AIC", phase_groups["aic"]), + _phase_group_row("AIV", phase_groups["aiv"]), + ) + ) + phase_section = f""" +

局部 phase:{_escape(phase)}

+
+
带边界扰动的 running read-clear 区间。下界是直接观测值,上界是下界加本核 primary-shadow residual;不同 phase ELF 的局部值不能相加,也不能从 none 相减得到无扰动净值。
+

局部占同一 ELF 完整 Submit primary

+

每条灰色底轨代表该组完整 Submit 的 100%;深色为直接观测下界,浅色延伸到保守上界,空白部分属于局部窗口之外。request 与 miss 使用各自的完整窗口事件数作分母。

+
{phase_share_cards}
+
+ 展开 ALL / AIC / AIV 完整数字表 +
+ + + {phase_rows} +
分组callsrequest sum 下界..上界request/core 下界..上界局部 request / Submit primarymiss sum 下界..上界miss/core 下界..上界局部 miss / Submit primaryshadow loss req/missobserved miss/request
+
+
+

逐核 phase miss 下界

+
{phase_plot}
+
+""" + + return f""" + + + + + Standalone PA Submit I-cache 报告 + + + +
+
+

Standalone PA Submit I-cache 报告

+
submit-pmu / {_escape(phase)} / batch {_escape(configuration['batches'])} / {_escape(workload.get('mode'))} / QK,SF,PV,UP={_escape(workload_counts.get('qk'))},{_escape(workload_counts.get('sf'))},{_escape(workload_counts.get('pv'))},{_escape(workload_counts.get('up'))}
+
+ RAW → SUMMARY PASS + 语义与 PMU PASS + OWNER RESTORE PASS + {_escape(shadow_badge)} +
+
+ +
+
完整 Submit
{submit_us / 1000:.6f} ms
+
ALL PMU total/core mean
{_format_per_core(all_metrics['total_per_core'])}
raw cycle;Σ={_format_count(all_metrics['total_sum'])}
+
ALL scalar busy/core mean
{_format_per_core(all_metrics['scalar_per_core'])}
Σscalar/Σtotal={_format_rate(all_metrics['scalar_share'])}
+
完整 Submit primary I-cache request
{total_requests:,}
+
完整 Submit primary I-cache miss
{total_misses:,}
+
聚合 miss rate
{_format_rate(float(all_metrics['miss_rate']))}
+
实测物理子核
{workers}(32 AIC + 64 AIV)
+
Primary/Shadow
exact {exact_records}/{workers}
bounded {bounded_records}/{workers}
+
+ +

PMU total 与 scalar busy

+
+ + + + + + + +
角色核数total/core meantotal mediantotal p95scalar/core meanscalar medianscalar p95Σscalar/Σtotal非 Scalar-busy 残余/core
ALL{_format_count(all_metrics['cores'])}{_format_per_core(all_metrics['total_per_core'])}{_format_count(all_metrics['total_median'])}{_format_count(all_metrics['total_p95'])}{_format_per_core(all_metrics['scalar_per_core'])}{_format_count(all_metrics['scalar_median'])}{_format_count(all_metrics['scalar_p95'])}{_format_rate(all_metrics['scalar_share'])}{_format_per_core(all_metrics['non_scalar_busy_per_core'])}
AIC{_format_count(aic['cores'])}{_format_per_core(aic['total_per_core'])}{_format_count(aic['total_median'])}{_format_count(aic['total_p95'])}{_format_per_core(aic['scalar_per_core'])}{_format_count(aic['scalar_median'])}{_format_count(aic['scalar_p95'])}{_format_rate(aic['scalar_share'])}{_format_per_core(aic['non_scalar_busy_per_core'])}
AIV{_format_count(aiv['cores'])}{_format_per_core(aiv['total_per_core'])}{_format_count(aiv['total_median'])}{_format_count(aiv['total_p95'])}{_format_per_core(aiv['scalar_per_core'])}{_format_count(aiv['scalar_median'])}{_format_count(aiv['scalar_p95'])}{_format_rate(aiv['scalar_share'])}{_format_per_core(aiv['non_scalar_busy_per_core'])}
+
口径:total 是每个物理子核在 Submit gate 内的 PMU raw total cycle;求和代表 96 核 core-work。scalar busy 是 CNT2 scalar_instr_busy(0x001)。表中的“非 Scalar-busy 残余”严格等于 total−scalar busy,它不是空闲时间,也不是 I-cache stall,其中还混有同步等待、engine 等待及其他未归因周期。受控微基准已观察到依赖返回的 atomic 等待大部分进入 scalar busy,而 I-cache refill 的额外周期大部分只进入 total。当前 A5/DAV3510 正式事件表和 CANN 9.1 上板输出均不提供 scalar_wait_ib_time/scalar_wait_time,报告不会用其他产品的 selector 猜测这两项。
+

PMU total cycle

{total_plot} +

Scalar busy cycle

{scalar_plot} +
+ +

AIC 与 AIV 的 I-cache 对比

+
+ + + + + + +
角色核数request/corerequest p95miss/core meanmiss/core medianmiss p95miss maxΣmiss/Σrequest
AIC{_format_count(aic['cores'])}{_format_per_core(aic['requests_per_core'])}{_format_count(aic['requests_p95'])}{_format_per_core(aic['misses_per_core'])}{_format_count(aic['misses_median'])}{_format_count(aic['misses_p95'])}{_format_count(aic['misses_max'])}{_format_rate(aic['miss_rate'])}
AIV{_format_count(aiv['cores'])}{_format_per_core(aiv['requests_per_core'])}{_format_count(aiv['requests_p95'])}{_format_per_core(aiv['misses_per_core'])}{_format_count(aiv['misses_median'])}{_format_count(aiv['misses_p95'])}{_format_count(aiv['misses_max'])}{_format_rate(aiv['miss_rate'])}
+
{_escape(request_comparison)};{_escape(miss_comparison)};聚合 {_escape(rate_comparison)}。
+
+ +

逐物理核分布

+
+
AIC 圆点 AIV 方点
+

I-cache request

{request_plot} +

I-cache miss

{miss_plot} +

每核 miss rate

{rate_plot} +
+ + {phase_section} + +
+

假设性 core-equivalent,不是 Submit 墙钟损失

+

AIC:{_format_per_core(aic['misses_per_core'])} miss/core × {miss_penalty_ns:.3f} ns = {float(aic['serial_equivalent_us']):,.3f} µs/core-equivalent

+

AIV:{_format_per_core(aiv['misses_per_core'])} miss/core × {miss_penalty_ns:.3f} ns = {float(aiv['serial_equivalent_us']):,.3f} µs/core-equivalent

+

该标尺不可跨 96 核相加,也不可直接从 {submit_us / 1000:.6f} ms Submit 中扣除。各核并行,miss 可能重叠或被流水/等待隐藏;实际收益必须由相同语义优化前后的 ΔSubmit 与 Δmiss/core 成对实验确认。

+
+ +

96 核精确数据

+
+
+ 展开逐核表格 + + + {per_core_rows} +
workerphysicalroleblocklanePMU totalscalar busyscalar/totalrequestsmissesper-core rateprimary=shadowtrusted
+
+

聚合 miss rate 使用 Σmiss/Σrequest,不平均逐核百分比。p95 使用 nearest-rank:ceil(0.95 × N)。

+
+ +
+ +
capture_id:{_escape(data['capture'].get('capture_id'))}
+
SHA-256:{raw_digest}
+
生成时间:{generated_at};生成器:pmu_html_report schema v{REPORT_VERSION}
+
+
+ + +""" + + +def write_report( + input_path: Path, + output_path: Path | None = None, + miss_penalty_ns: float = 90.0, +) -> Path: + """在完整 HTML 构造成功后原子发布,失败时不留下半截报告。""" + + input_path = Path(input_path) + output_path = default_output_path(input_path) if output_path is None else Path(output_path) + if input_path.resolve() == output_path.resolve(): + raise ValueError("HTML output path must differ from raw JSON input") + document = render_report(input_path, output_path, miss_penalty_ns) + output_path.parent.mkdir(parents=True, exist_ok=True) + temporary_name: str | None = None + try: + with tempfile.NamedTemporaryFile( + mode="w", + encoding="utf-8", + prefix=f".{output_path.name}.", + suffix=".tmp", + dir=output_path.parent, + delete=False, + ) as temporary: + temporary.write(document) + temporary.flush() + os.fsync(temporary.fileno()) + os.fchmod(temporary.fileno(), 0o644) + temporary_name = temporary.name + os.replace(temporary_name, output_path) + finally: + if temporary_name is not None: + Path(temporary_name).unlink(missing_ok=True) + return output_path + + +def main(argv: Sequence[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("input", type=Path, help="一份已完成的 submit-pmu raw JSON") + parser.add_argument("-o", "--output", type=Path, help="HTML 输出路径;默认由 *_raw.json 推导") + parser.add_argument( + "--icache-miss-ns", + type=float, + default=90.0, + help="受控 cold/warm 串行标尺,仅用于 core-equivalent(默认 90)", + ) + arguments = parser.parse_args(argv) + if not math.isfinite(arguments.icache_miss_ns) or arguments.icache_miss_ns <= 0: + parser.error("--icache-miss-ns must be finite and positive") + try: + output = write_report(arguments.input, arguments.output, arguments.icache_miss_ns) + except (OSError, ValueError) as error: + print(f"PMU HTML report failed: {error}", file=sys.stderr) + return 1 + print(f"[PMU-HTML] raw={arguments.input} report={output}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py b/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py index 280d31148c..18ebe998ff 100644 --- a/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py +++ b/tests/atomic_probe/pa_scheduler/pmu_sidecar_analyzer.py @@ -59,7 +59,15 @@ SUMMARY_FIELDS = ("sum", "mean", "median", "p95", "max") SUBMIT_PMU_BUILD_VARIANT = "submit-pmu" SUBMIT_PMU_BUILD_VARIANT_ID = 2 -SUBMIT_PMU_PHASE_IDS = {"none": 0, "claim": 1, "efdrain": 2} +PROGRAMMABLE_COUNTER_BITS = 32 +PROGRAMMABLE_COUNTER_RISK_THRESHOLD = (1 << PROGRAMMABLE_COUNTER_BITS) // 4 +SUBMIT_PMU_PHASE_IDS = { + "none": 0, + "claim": 1, + "efdrain": 2, + "materialize": 4, + "register": 5, +} TASKS_PER_BATCH = 5 PHASE_STATUS_REQUIRED_MASK = 0x3CF @@ -132,6 +140,7 @@ class PhasePartitionEvidence: request_signed_delta: int miss_abs_delta: int miss_signed_delta: int + expected_calls: int def _require(condition: bool, message: str) -> None: @@ -216,6 +225,10 @@ def _validate_group_summary( expected.get("trusted_cores") == len(records), f"{path}: summary.{group_name}.trusted_cores is incomplete", ) + _require( + expected.get("active_cores") == len(records), + f"{path}: summary.{group_name}.active_cores is incomplete", + ) for metric in metric_names: values = [ @@ -229,8 +242,15 @@ def _validate_group_summary( f"{path}: summary.{group_name}.{metric} must be an object", ) for field in SUMMARY_FIELDS: + if field in ("sum", "p95", "max"): + reported_value = _integer( + reported.get(field), f"{path}: summary.{group_name}.{metric}.{field}" + ) + matches = reported_value == actual[field] + else: + matches = _same_number(actual[field], reported.get(field)) _require( - _same_number(actual[field], reported.get(field)), + matches, f"{path}: raw summary mismatch at {group_name}.{metric}.{field}: " f"raw={actual[field]!r} json={reported.get(field)!r}", ) @@ -314,6 +334,25 @@ def _validate_submit_pmu_configuration(path: Path, configuration: dict[str, Any] f"{path}: configuration.unavailable_metrics must identify mte3_busy", ) + counter_widths = configuration.get("counter_width_bits") + _require( + isinstance(counter_widths, dict), + f"{path}: configuration.counter_width_bits must be an object", + ) + _require( + _integer(counter_widths.get("total"), f"{path}: configuration.counter_width_bits.total") + == 64, + f"{path}: configuration.counter_width_bits.total must be 64", + ) + _require( + _integer( + counter_widths.get("programmable"), + f"{path}: configuration.counter_width_bits.programmable", + ) + == PROGRAMMABLE_COUNTER_BITS, + f"{path}: configuration.counter_width_bits.programmable must be 32", + ) + for field in ( "trace_enabled", "trace_atomics", @@ -352,6 +391,11 @@ def _validate_submit_pmu_configuration(path: Path, configuration: dict[str, Any] selectors = configuration.get("selectors") _require(isinstance(selectors, dict), f"{path}: configuration.selectors must be an object") expected_selectors = { + "cnt0_vector_busy": 0x501, + "cnt1_cube_busy": 0x301, + "cnt2_scalar_busy": 0x001, + "cnt3_mte1_busy": 0x701, + "cnt4_mte2_busy": 0x202, "cnt6_primary_icache_request": 0x034, "cnt7_primary_icache_miss": 0x035, "cnt5_shadow_icache_miss": 0x035, @@ -527,6 +571,12 @@ def _validate_submit_pmu_owner( return configured_ids +def _expected_submit_pmu_phase_calls(phase_name: str, batches: int) -> int: + """当前 running phase 都覆盖每核每次 Submit,调用次数固定为 5B。""" + + return 0 if phase_name == "none" else batches * TASKS_PER_BATCH + + def _validate_submit_pmu_record( path: Path, index: int, @@ -535,7 +585,7 @@ def _validate_submit_pmu_record( phase_id: int, batches: int, ) -> PhasePartitionEvidence: - """重验 raw phase 分区;claim 只形成 read-to-clear 的上下界。""" + """重验 raw phase 分区;运行中 read-clear 只形成上下界。""" prefix = f"{path}: records[{index}]" _require( @@ -552,6 +602,13 @@ def _validate_submit_pmu_record( (phase_status & PHASE_STATUS_REQUIRED_MASK) == PHASE_STATUS_REQUIRED_MASK, f"{prefix} phase_status is incomplete", ) + expected_calls = _expected_submit_pmu_phase_calls(phase_name, batches) + if "phase_expected_calls" in record: + _require( + _integer(record.get("phase_expected_calls"), f"{prefix}.phase_expected_calls") + == expected_calls, + f"{prefix}.phase_expected_calls disagrees with the fixed phase contract", + ) primary_requests = _integer(record.get("icache_requests"), f"{prefix}.icache_requests") primary_misses = _integer(record.get("icache_misses"), f"{prefix}.icache_misses") @@ -563,6 +620,10 @@ def _validate_submit_pmu_record( ) shadow_exact = shadow_requests == primary_requests and shadow_misses == primary_misses shadow_bounded = shadow_requests <= primary_requests and shadow_misses <= primary_misses + _require( + shadow_misses <= shadow_requests, + f"{prefix} has shadow I-cache miss > request", + ) _require( record.get("shadow_matches_primary") is shadow_exact, f"{prefix}.shadow_matches_primary disagrees with raw counters", @@ -573,7 +634,10 @@ def _validate_submit_pmu_record( ) _require(shadow_bounded, f"{prefix} shadow whole exceeds the authoritative primary whole") if phase_name == "none": - _require(shadow_exact, f"{prefix} phase=none requires shadow whole to equal primary") + _require( + shadow_exact, + f"{prefix} a phase with zero local calls requires shadow whole to equal primary", + ) request_loss = primary_requests - shadow_requests miss_loss = primary_misses - shadow_misses @@ -641,20 +705,15 @@ def _validate_submit_pmu_record( f"{prefix} phase upper bound exceeds the authoritative primary whole", ) - expected_calls = ( - 0 if phase_name == "none" - else batches * TASKS_PER_BATCH if phase_name in ("claim", "efdrain") - else -1 - ) _require(calls == expected_calls, f"{prefix} phase_calls does not match the phase contract") - if phase_name == "none": + if expected_calls == 0: _require( phase_requests == 0 and phase_misses == 0, - f"{prefix} phase=none must have zero phase counters", + f"{prefix} a phase with zero calls must have zero phase counters", ) _require( phase_request_upper == phase_requests and phase_miss_upper == phase_misses, - f"{prefix} phase=none lower and upper bounds must be equal", + f"{prefix} a phase with zero calls must have zero-width bounds", ) return PhasePartitionEvidence( @@ -664,6 +723,7 @@ def _validate_submit_pmu_record( request_signed_delta=shadow_requests - primary_requests, miss_abs_delta=abs(shadow_misses - primary_misses), miss_signed_delta=shadow_misses - primary_misses, + expected_calls=expected_calls, ) @@ -710,6 +770,27 @@ def load_capture(path: Path) -> Capture: f"{path}: schema-v4 requires the fixed 96/32/64 A5 topology", ) phase_name, phase_id = _validate_submit_pmu_configuration(path, configuration) + else: + counter_widths = configuration.get("counter_width_bits") + _require( + isinstance(counter_widths, dict), + f"{path}: configuration.counter_width_bits must be an object", + ) + _require( + _integer( + counter_widths.get("total"), f"{path}: configuration.counter_width_bits.total" + ) + == 64, + f"{path}: configuration.counter_width_bits.total must be 64", + ) + _require( + _integer( + counter_widths.get("programmable"), + f"{path}: configuration.counter_width_bits.programmable", + ) + == PROGRAMMABLE_COUNTER_BITS, + f"{path}: configuration.counter_width_bits.programmable must be 32", + ) # JSON 只有在运行、PMU、owner Restore 和 runtime cleanup 全部成功后才应发布。 # 分析器仍逐项重验,防止手工复制或未来 schema 退化绕过发布门禁。 @@ -775,14 +856,6 @@ def load_capture(path: Path) -> Capture: validation.get(field) == workers, f"{path}: validation.{field} is incomplete", ) - expected_phase_calls = ( - 0 if phase_name == "none" else workers * batches * TASKS_PER_BATCH - ) - _require( - validation.get("phase_calls") == expected_phase_calls, - f"{path}: validation.phase_calls does not match the compiled phase contract", - ) - groups: dict[str, list[dict[str, Any]]] = { "all": records, "aic": [record for record in records if record.get("role") == "aic"], @@ -859,7 +932,41 @@ def load_capture(path: Path) -> Capture: misses = _integer( record.get("icache_misses"), f"{path}: records[{index}].icache_misses" ) + total_cycles = _integer( + record.get("total_cycles"), f"{path}: records[{index}].total_cycles" + ) + scalar_busy = _integer( + record.get("scalar_busy"), f"{path}: records[{index}].scalar_busy" + ) _require(misses <= requests, f"{path}: records[{index}] has miss > request") + _require(total_cycles > 0, f"{path}: records[{index}] has zero total_cycles") + _require( + scalar_busy <= total_cycles, + f"{path}: records[{index}] has scalar_busy > total_cycles", + ) + programmable_fields = ( + ( + "vector_busy", + "cube_busy", + "scalar_busy", + "mte1_busy", + "mte2_busy", + "icache_requests", + "icache_misses", + "phase_icache_requests", + "phase_icache_misses", + "shadow_whole_icache_requests", + "shadow_whole_icache_misses", + ) + if schema_version == 4 + else tuple(metric for metric in METRIC_NAMES if metric != "total_cycles") + ) + for field in programmable_fields: + value = _integer(record.get(field), f"{path}: records[{index}].{field}") + _require( + value < PROGRAMMABLE_COUNTER_RISK_THRESHOLD, + f"{path}: records[{index}].{field} reaches the 32-bit counter risk threshold", + ) if schema_version == 4: # 上面的 configuration 校验已保证二者不是 None;显式 assert 只帮助 # 类型收窄,不替代任何 JSON 运行时门禁。 @@ -886,16 +993,48 @@ def load_capture(path: Path) -> Capture: shadow_exact_records = sum(item.shadow_exact for item in phase_partition_evidence) shadow_bounded_records = sum(item.shadow_bounded for item in phase_partition_evidence) + phase_shadow_acceptable_records = sum( + item.shadow_exact if item.expected_calls == 0 else item.shadow_bounded + for item in phase_partition_evidence + ) + expected_phase_calls = sum(item.expected_calls for item in phase_partition_evidence) _require( shadow_bounded_records == A5_WORKERS, f"{path}: not all shadow partitions are bounded by primary", ) + _require( + phase_shadow_acceptable_records == A5_WORKERS, + f"{path}: zero-call phase records are not exact or active records are unbounded", + ) assert phase_name is not None if phase_name == "none": _require( shadow_exact_records == A5_WORKERS, f"{path}: phase=none requires all shadow partitions to be exact", ) + _require( + _integer(validation.get("phase_calls"), f"{path}: validation.phase_calls") + == expected_phase_calls, + f"{path}: validation.phase_calls does not match raw per-worker contracts", + ) + if "phase_expected_calls" in validation: + _require( + _integer( + validation.get("phase_expected_calls"), + f"{path}: validation.phase_expected_calls", + ) + == expected_phase_calls, + f"{path}: validation.phase_expected_calls disagrees with raw evidence", + ) + if "phase_shadow_acceptable_records" in validation: + _require( + _integer( + validation.get("phase_shadow_acceptable_records"), + f"{path}: validation.phase_shadow_acceptable_records", + ) + == phase_shadow_acceptable_records, + f"{path}: validation.phase_shadow_acceptable_records disagrees with raw records", + ) _require( _integer( validation.get("shadow_primary_match_records"), @@ -1076,19 +1215,21 @@ def analyze(paths: Sequence[Path], miss_penalty_ns: float = 90.0) -> dict[str, A ) request_loss_summary = _metric_summary(request_losses) miss_loss_summary = _metric_summary(miss_losses) + phase_request_upper_sum = sum(phase_request_uppers) + phase_miss_upper_sum = sum(phase_miss_uppers) row["groups"][group_name].update( { "phase_calls_sum": phase_calls, "phase_calls_per_core": phase_calls / cores, "phase_icache_requests_lower_bound_sum": phase_requests, - "phase_icache_requests_upper_bound_sum": sum(phase_request_uppers), + "phase_icache_requests_upper_bound_sum": phase_request_upper_sum, "phase_icache_misses_lower_bound_sum": phase_misses, - "phase_icache_misses_upper_bound_sum": sum(phase_miss_uppers), + "phase_icache_misses_upper_bound_sum": phase_miss_upper_sum, "phase_icache_requests_lower_bound_per_core": phase_requests / cores, - "phase_icache_requests_upper_bound_per_core": sum(phase_request_uppers) - / cores, + "phase_icache_requests_upper_bound_per_core": + phase_request_upper_sum / cores, "phase_icache_misses_lower_bound_per_core": phase_misses / cores, - "phase_icache_misses_upper_bound_per_core": sum(phase_miss_uppers) / cores, + "phase_icache_misses_upper_bound_per_core": phase_miss_upper_sum / cores, "phase_icache_requests_lower_bound_per_core_median": phase_request_lower_summary["median"], "phase_icache_requests_lower_bound_per_core_p95": @@ -1118,8 +1259,12 @@ def analyze(paths: Sequence[Path], miss_penalty_ns: float = 90.0) -> dict[str, A ), "phase_icache_request_lower_bound_share_of_submit": phase_requests / requests, + "phase_icache_request_upper_bound_share_of_submit": + phase_request_upper_sum / requests, "phase_icache_miss_lower_bound_share_of_submit": phase_misses / misses if misses != 0 else None, + "phase_icache_miss_upper_bound_share_of_submit": + phase_miss_upper_sum / misses if misses != 0 else None, } ) per_run.append(row) @@ -1176,7 +1321,9 @@ def analyze(paths: Sequence[Path], miss_penalty_ns: float = 90.0) -> dict[str, A "shadow_miss_loss_per_core_p95", "phase_observed_read_clear_ratio", "phase_icache_request_lower_bound_share_of_submit", + "phase_icache_request_upper_bound_share_of_submit", "phase_icache_miss_lower_bound_share_of_submit", + "phase_icache_miss_upper_bound_share_of_submit", ) for group_name in GROUP_NAMES: aggregate["groups"][group_name] = {} diff --git a/tests/atomic_probe/pa_scheduler/run.sh b/tests/atomic_probe/pa_scheduler/run.sh index 730a3ca9be..8b6cb69a28 100755 --- a/tests/atomic_probe/pa_scheduler/run.sh +++ b/tests/atomic_probe/pa_scheduler/run.sh @@ -21,8 +21,8 @@ Usage: ./run.sh run ccec|ascendc|cpu|all [benchmark options] ./run.sh smoke ccec|ascendc|cpu|all [--device N] ./run.sh swimlane ccec|ascendc|cpu|all [benchmark options] - ./run.sh build-submit-pmu ccec none|claim|efdrain - ./run.sh submit-pmu ccec none|claim|efdrain [benchmark options] + ./run.sh build-submit-pmu ccec none|claim|efdrain|materialize|register + ./run.sh submit-pmu ccec none|claim|efdrain|materialize|register [benchmark options] Benchmark options: --device N @@ -59,12 +59,14 @@ weighted diagonal A and asymmetric B to detect transpose/stride/reorder bugs. The swimlane action enables atomic tracing by default. For the lower-level run action, --trace-atomics still requires swimlane tracing; add --analyze-swimlane to print the per-role/per-site timing distributions. ---pmu-json requires --runs 1 and a non-off PMU window. PMU probe options are +--pmu-json requires --runs 1 and a non-off PMU window. For submit-pmu, a +successful raw capture also generates a self-contained HTML report beside it; +submit_icache_raw.json maps to submit_icache_report.html. PMU probe options are CCEC-only and cannot target all. The submit-pmu action is a separate CCEC-only build. It fixes one PMU-only run covering the complete Submit window. phase=none performs no internal snapshots; -phase=claim/efdrain reports running read-clear lower/loss-adjusted upper bounds +phase=claim/efdrain/materialize/register reports running read-clear lower/loss-adjusted upper bounds for one compile-time phase while CNT6/7 retain the authoritative whole-window counters. The swimlane action performs exactly one run and writes both the raw capture @@ -136,9 +138,9 @@ run_backend() { validate_submit_pmu_phase() { case "$1" in - none|claim|efdrain) ;; + none|claim|efdrain|materialize|register) ;; *) - echo "Unknown submit-pmu phase: $1 (expected none|claim|efdrain)" >&2 + echo "Unknown submit-pmu phase: $1 (expected none|claim|efdrain|materialize|register)" >&2 exit 1 ;; esac @@ -160,6 +162,8 @@ validate_submit_pmu_artifacts() { none) phase_id=0 ;; claim) phase_id=1 ;; efdrain) phase_id=2 ;; + materialize) phase_id=4 ;; + register) phase_id=5 ;; *) submit_pmu_artifact_failure "$phase" "unsupported phase"; return 1 ;; esac @@ -244,8 +248,36 @@ run_submit_pmu() { local build_dir="$SCRIPT_DIR/build/ccec/submit-pmu/$phase" local host="$build_dir/pa_scheduler_host" local kernel="$build_dir/pa_scheduler_kernel.o" + local pmu_json="" + local expect_pmu_json_value=false + local argument + # host 仍是 raw 文件的唯一写入者;这里只提取同一个路径,在 host 成功且 + # raw 已原子发布后调用独立分析器生成可视 HTML。原参数保持原样透传。 + for argument in "$@"; do + if [[ "$expect_pmu_json_value" == true ]]; then + pmu_json="$argument" + expect_pmu_json_value=false + continue + fi + case "$argument" in + --pmu-json) expect_pmu_json_value=true ;; + --pmu-json=*) pmu_json="${argument#--pmu-json=}" ;; + esac + done validate_submit_pmu_artifacts "$phase" "$build_dir" "$host" --kernel "$kernel" --runs 1 --no-swimlane --pmu-window submit-all "$@" + if [[ -n "$pmu_json" ]]; then + local python_bin="${PYTHON:-python3}" + if ! command -v "$python_bin" >/dev/null 2>&1; then + echo "Python executable not found for PMU HTML report: $python_bin" >&2 + return 1 + fi + if [[ ! -f "$SCRIPT_DIR/pmu_html_report.py" ]]; then + echo "Missing local PMU HTML report generator: $SCRIPT_DIR/pmu_html_report.py" >&2 + return 1 + fi + "$python_bin" "$SCRIPT_DIR/pmu_html_report.py" "$pmu_json" + fi } reject_managed_swimlane_options() { @@ -378,7 +410,7 @@ case "$ACTION" in ;; build-submit-pmu) if [[ "$BACKEND" != "ccec" || $# -ne 1 ]]; then - echo "Usage: $0 build-submit-pmu ccec none|claim|efdrain" >&2 + echo "Usage: $0 build-submit-pmu ccec none|claim|efdrain|materialize|register" >&2 exit 1 fi PHASE="$1" @@ -387,7 +419,7 @@ case "$ACTION" in ;; submit-pmu) if [[ "$BACKEND" != "ccec" || $# -lt 1 ]]; then - echo "Usage: $0 submit-pmu ccec none|claim|efdrain [benchmark options]" >&2 + echo "Usage: $0 submit-pmu ccec none|claim|efdrain|materialize|register [benchmark options]" >&2 exit 1 fi PHASE="$1" diff --git a/tests/atomic_probe/pa_scheduler/test_pmu_html_report.py b/tests/atomic_probe/pa_scheduler/test_pmu_html_report.py new file mode 100644 index 0000000000..f9a1002b10 --- /dev/null +++ b/tests/atomic_probe/pa_scheduler/test_pmu_html_report.py @@ -0,0 +1,229 @@ +#!/usr/bin/env python3 +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- + +"""standalone submit-pmu HTML 可视报告的生成与失败原子性回归。""" + +from __future__ import annotations + +import json +import re +import stat +import tempfile +import unittest +from pathlib import Path + +try: + from .pmu_html_report import _phase_share_metric, default_output_path, render_report, write_report + from .pmu_sidecar_analyzer import analyze + from .test_pmu_sidecar_analyzer import _submit_pmu_capture, _submit_pmu_summary +except ImportError: + from pmu_html_report import _phase_share_metric, default_output_path, render_report, write_report + from pmu_sidecar_analyzer import analyze + from test_pmu_sidecar_analyzer import _submit_pmu_capture, _submit_pmu_summary + + +class PmuHtmlReportTest(unittest.TestCase): + def _write_capture( + self, directory: str, capture: dict, name: str = "submit_icache_raw.json" + ) -> Path: + path = Path(directory) / name + path.write_text(json.dumps(capture, ensure_ascii=False), encoding="utf-8") + return path + + def test_default_output_uses_descriptive_report_name(self) -> None: + self.assertEqual( + default_output_path(Path("result/submit_icache_raw.json")), + Path("result/submit_icache_report.html"), + ) + self.assertEqual( + default_output_path(Path("result/custom.json")), + Path("result/custom_report.html"), + ) + + def test_none_report_reuses_analyzer_values_and_has_96_rows(self) -> None: + capture = _submit_pmu_capture(phase="none") + with tempfile.TemporaryDirectory() as directory: + path = self._write_capture(directory, capture) + analysis = analyze([path]) + document = render_report(path) + + aiv = analysis["per_run"][0]["groups"]["aiv"] + self.assertIn("Standalone PA Submit I-cache 报告", document) + self.assertIn(f"{aiv['icache_misses_per_core']:,.2f}", document) + self.assertIn(f"{aiv['icache_miss_rate'] * 100:.4f}%", document) + self.assertIn("PRIMARY ↔ SHADOW EXACT 96/96", document) + self.assertIn("局部 phase:none", document) + self.assertIn("PMU total 与 scalar busy", document) + self.assertIn(f"{aiv['total_cycles_sum'] / 64:,.2f}", document) + self.assertIn(f"{aiv['scalar_busy_sum'] / 64:,.2f}", document) + self.assertIn("非 Scalar-busy 残余", document) + self.assertIn("它不是空闲时间,也不是 I-cache stall", document) + self.assertIn("不提供 scalar_wait_ib_time/scalar_wait_time", document) + self.assertIn("PMU totalscalar busy", document) + self.assertIn("total_cycles=", document) + self.assertIn("不是 Submit 墙钟损失", document) + self.assertEqual(document.count('data-worker-id="'), 96) + self.assertNotIn("http://", document) + self.assertNotIn("https://", document) + self.assertNotIn(" None: + capture = _submit_pmu_capture(phase="claim") + with tempfile.TemporaryDirectory() as directory: + path = self._write_capture(directory, capture) + analysis = analyze([path]) + document = render_report(path) + + group = analysis["per_run"][0]["groups"]["all"] + self.assertIn("局部 phase:claim", document) + self.assertIn("带边界扰动的 running read-clear 区间", document) + self.assertIn(f"{group['phase_calls_sum']:,}", document) + self.assertIn( + f"{group['phase_icache_misses_lower_bound_sum']:,}.." + f"{group['phase_icache_misses_upper_bound_sum']:,}", + document, + ) + self.assertIn("局部占同一 ELF 完整 Submit primary", document) + self.assertEqual(document.count('class="phase-share-card"'), 3) + self.assertEqual(document.count('class="phase-share-metric"'), 6) + self.assertEqual(document.count('class="phase-share-track '), 6) + for group_name in ("all", "aic", "aiv"): + phase_group = analysis["per_run"][0]["groups"][group_name] + for metric, lower_key, upper_key in ( + ( + "request", + "phase_icache_request_lower_bound_share_of_submit", + "phase_icache_request_upper_bound_share_of_submit", + ), + ( + "miss", + "phase_icache_miss_lower_bound_share_of_submit", + "phase_icache_miss_upper_bound_share_of_submit", + ), + ): + self.assertIn( + f'data-phase-group="{group_name}" data-metric="{metric}" ' + f'data-lower-share="{phase_group[lower_key]:.12f}" ' + f'data-upper-share="{phase_group[upper_key]:.12f}"', + document, + ) + lower_percent = phase_group[lower_key] * 100.0 + upper_percent = phase_group[upper_key] * 100.0 + metric_pattern = re.compile( + rf'data-phase-group="{group_name}" data-metric="{metric}" ' + rf'data-lower-share="{phase_group[lower_key]:.12f}" ' + rf'data-upper-share="{phase_group[upper_key]:.12f}".*?' + rf'下界 {lower_percent:.4f}%,上界 {upper_percent:.4f}%.*?' + rf'class="phase-share-upper" style="width:{upper_percent:.6f}%".*?' + rf'class="phase-share-lower" style="width:{lower_percent:.6f}%".*?' + rf'class="phase-share-upper-marker" style="left:{upper_percent:.6f}%"', + re.DOTALL, + ) + self.assertRegex(document, metric_pattern) + self.assertEqual( + document.count( + '
0%50%100%
' + ), + 6, + ) + self.assertIn('class="phase-table-scroll"', document) + self.assertIn('class="phase-table"', document) + self.assertIn('
', document) + self.assertNotIn('
', document) + self.assertIn( + '
\n ', + document, + ) + self.assertIn("展开 ALL / AIC / AIV 完整数字表", document) + self.assertIn(".phase-panel { overflow:hidden; }", document) + self.assertIn( + ".phase-table-scroll { width:100%; max-width:100%; overflow-x:auto;", + document, + ) + self.assertIn('class="phase-plot-scroll"', document) + self.assertIn( + ".phase-plot-scroll { width:100%; max-width:100%; overflow-x:auto;", + document, + ) + self.assertIn("不同 phase ELF 的局部值不能相加", document) + self.assertIn("phase_icache_misses=", document) + + def test_phase_share_equal_bounds_keep_visible_value_and_marker(self) -> None: + fragment = _phase_share_metric("AIC", "I-cache miss", "miss", 0.25, 0.25, 10, 10) + self.assertIn("25.0000%..25.0000%", fragment) + self.assertIn('class="phase-share-upper" style="width:25.000000%"', fragment) + self.assertIn('class="phase-share-lower" style="width:25.000000%"', fragment) + self.assertIn('class="phase-share-upper-marker" style="left:25.000000%"', fragment) + + def test_phase_share_zero_denominator_is_explicitly_unavailable(self) -> None: + fragment = _phase_share_metric("AIC", "I-cache miss", "miss", None, None, 0, 0) + self.assertIn("比例不可计算", fragment) + self.assertNotIn("None%", fragment) + self.assertNotIn("nan%", fragment.lower()) + + def test_zero_aic_miss_has_an_explicit_non_dividing_comparison(self) -> None: + capture = _submit_pmu_capture(phase="none") + for record in capture["records"]: + if record["role"] == "aic": + record["icache_misses"] = 0 + record["shadow_whole_icache_misses"] = 0 + groups = { + "all": capture["records"], + "aic": [record for record in capture["records"] if record["role"] == "aic"], + "aiv": [record for record in capture["records"] if record["role"] == "aiv"], + } + capture["summary"] = { + name: _submit_pmu_summary(records) for name, records in groups.items() + } + with tempfile.TemporaryDirectory() as directory: + path = self._write_capture(directory, capture) + document = render_report(path) + + self.assertIn("AIC miss/core 为 0,AIV 相对变化不可计算", document) + + def test_dynamic_strings_and_raw_link_are_escaped(self) -> None: + capture = _submit_pmu_capture(phase="none") + capture["capture"]["capture_id"] = '' + with tempfile.TemporaryDirectory() as directory: + path = self._write_capture(directory, capture, "submit__raw.json") + document = render_report(path) + + self.assertNotIn('', document) + self.assertIn("<capture & "probe">", document) + self.assertIn("submit_%3Cicache%3E_raw.json", document) + self.assertIn("submit_<icache>_raw.json", document) + + def test_write_report_is_complete_readable_and_leaves_no_temp(self) -> None: + capture = _submit_pmu_capture(phase="none") + with tempfile.TemporaryDirectory() as directory: + path = self._write_capture(directory, capture) + output = write_report(path) + mode = stat.S_IMODE(output.stat().st_mode) + temporary_files = list(Path(directory).glob(".*.tmp")) + + self.assertEqual(output.name, "submit_icache_report.html") + self.assertEqual(mode, 0o644) + self.assertTrue(output.read_text(encoding="utf-8").endswith("\n")) + self.assertEqual(temporary_files, []) + + def test_invalid_raw_does_not_publish_html(self) -> None: + capture = _submit_pmu_capture(phase="none") + capture["records"][0]["icache_misses"] = capture["records"][0]["icache_requests"] + 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write_capture(directory, capture) + output = default_output_path(path) + with self.assertRaisesRegex(ValueError, "miss > request|raw summary mismatch"): + write_report(path) + self.assertFalse(output.exists()) + self.assertEqual(list(Path(directory).glob(".*.tmp")), []) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py b/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py index 4d33b4ba4d..0352118666 100644 --- a/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py +++ b/tests/atomic_probe/pa_scheduler/test_pmu_sidecar_analyzer.py @@ -31,6 +31,7 @@ A5_WORKERS, METRIC_NAMES, PHASE_STATUS_REQUIRED_MASK, + PROGRAMMABLE_COUNTER_RISK_THRESHOLD, SUBMIT_PMU_METRIC_NAMES, TASKS_PER_BATCH, analyze, @@ -48,6 +49,7 @@ A5_WORKERS, METRIC_NAMES, PHASE_STATUS_REQUIRED_MASK, + PROGRAMMABLE_COUNTER_RISK_THRESHOLD, SUBMIT_PMU_METRIC_NAMES, TASKS_PER_BATCH, analyze, @@ -140,6 +142,7 @@ def _capture(offset: int = 0, window: str = "submit-all") -> dict[str, Any]: } for metric_index, metric in enumerate(METRIC_NAMES): record[metric] = base + metric_index + record["total_cycles"] = base + len(METRIC_NAMES) # miss/request 取独立、直观的数值,便于断言聚合公式。 record["icache_requests"] = 1000 + base record["icache_misses"] = 100 + base // 10 @@ -201,10 +204,15 @@ def _capture(offset: int = 0, window: str = "submit-all") -> dict[str, Any]: def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any]: - phase_ids = {"none": 0, "claim": 1, "efdrain": 2} + phase_ids = { + "none": 0, + "claim": 1, + "efdrain": 2, + "materialize": 4, + "register": 5, + } phase_id = phase_ids[phase] batches = 2 - calls_per_worker = 0 if phase == "none" else batches * TASKS_PER_BATCH records: list[dict[str, Any]] = [] for worker_id in range(A5_WORKERS): role = "aic" if worker_id < A5_AIC_WORKERS else "aiv" @@ -212,12 +220,13 @@ def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any] block_id = worker_id if role == "aic" else vector_id // 2 lane = 0 if role == "aic" else 1 + vector_id % 2 base = 100 + worker_id * 10 + offset + calls_per_worker = 0 if phase == "none" else batches * TASKS_PER_BATCH primary_requests = 1000 + base primary_misses = 100 + base // 10 - phase_requests = 0 if phase == "none" else 100 + worker_id * 10 + offset - phase_misses = 0 if phase == "none" else 10 + worker_id + offset // 10 - request_loss = 0 if phase == "none" else 1 + worker_id % 3 - miss_loss = 0 if phase == "none" else 1 + worker_id % 2 + phase_requests = 0 if calls_per_worker == 0 else 100 + worker_id * 10 + offset + phase_misses = 0 if calls_per_worker == 0 else 10 + worker_id + offset // 10 + request_loss = 0 if calls_per_worker == 0 else 1 + worker_id % 3 + miss_loss = 0 if calls_per_worker == 0 else 1 + worker_id % 2 shadow_requests = primary_requests - request_loss shadow_misses = primary_misses - miss_loss record: dict[str, Any] = { @@ -236,8 +245,9 @@ def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any] "window_stopped": True, "build_variant_id": 2, "compiled_phase_id": phase_id, - "phase_status": PHASE_STATUS_REQUIRED_MASK | (0x30 if phase == "none" else 0), + "phase_status": PHASE_STATUS_REQUIRED_MASK | (0x30 if calls_per_worker == 0 else 0), "phase_calls": calls_per_worker, + "phase_expected_calls": calls_per_worker, "phase_begin_reads": calls_per_worker, "phase_end_reads": calls_per_worker, "primary_window_segments": 1, @@ -256,6 +266,8 @@ def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any] } for metric_index, metric in enumerate(SUBMIT_PMU_METRIC_NAMES): record.setdefault(metric, base + metric_index) + # total 是同一窗口的包络,fixture 也必须满足 scalar_busy <= total_cycles。 + record["total_cycles"] = base + len(SUBMIT_PMU_METRIC_NAMES) record["icache_requests"] = primary_requests record["icache_misses"] = primary_misses # setdefault 不覆盖上面按 phase 契约填写的五个扩展字段。 @@ -269,6 +281,12 @@ def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any] workers = len(records) exact_records = sum(record["shadow_matches_primary"] for record in records) bounded_records = sum(record["shadow_not_greater_than_primary"] for record in records) + acceptable_records = sum( + record["shadow_matches_primary"] + if record["phase_expected_calls"] == 0 + else record["shadow_not_greater_than_primary"] + for record in records + ) request_losses = [record["shadow_request_loss"] for record in records] miss_losses = [record["shadow_miss_loss"] for record in records] return { @@ -346,6 +364,7 @@ def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any] "phase_status_trusted_records": workers, "shadow_primary_match_records": exact_records, "shadow_primary_bounded_records": bounded_records, + "phase_shadow_acceptable_records": acceptable_records, "shadow_request_abs_delta_sum": sum(request_losses), "shadow_request_abs_delta_max": max(request_losses), "shadow_request_signed_delta_sum": -sum(request_losses), @@ -354,7 +373,8 @@ def _submit_pmu_capture(offset: int = 0, phase: str = "claim") -> dict[str, Any] "shadow_miss_signed_delta_sum": -sum(miss_losses), "phase_boundary_match_records": workers, "phase_call_shape_match_records": workers, - "phase_calls": workers * calls_per_worker, + "phase_calls": sum(record["phase_calls"] for record in records), + "phase_expected_calls": sum(record["phase_expected_calls"] for record in records), "expected_records": A5_WORKERS, "expected_unique_core_ids": A5_WORKERS, "expected_owner_bitmap_member_records": A5_WORKERS, @@ -546,6 +566,24 @@ def test_submit_pmu_v4_claim_is_recomputed_and_aggregated(self) -> None: group["phase_icache_misses_lower_bound_per_core_p95"], group["phase_icache_misses_upper_bound_per_core_p95"], ) + self.assertEqual( + group["phase_icache_request_lower_bound_share_of_submit"], + group["phase_icache_requests_lower_bound_sum"] + / group["icache_requests_sum"], + ) + self.assertEqual( + group["phase_icache_request_upper_bound_share_of_submit"], + group["phase_icache_requests_upper_bound_sum"] + / group["icache_requests_sum"], + ) + self.assertEqual( + group["phase_icache_miss_lower_bound_share_of_submit"], + group["phase_icache_misses_lower_bound_sum"] / group["icache_misses_sum"], + ) + self.assertEqual( + group["phase_icache_miss_upper_bound_share_of_submit"], + group["phase_icache_misses_upper_bound_sum"] / group["icache_misses_sum"], + ) self.assertLess( _submit_pmu_capture()["validation"]["shadow_request_signed_delta_sum"], 0 ) @@ -563,6 +601,9 @@ def test_submit_pmu_v4_none_has_zero_disabled_phase(self) -> None: self.assertEqual( phase["phase_icache_miss_lower_bound_share_of_submit"]["median"], 0 ) + self.assertEqual( + phase["phase_icache_miss_upper_bound_share_of_submit"]["median"], 0 + ) self.assertEqual( phase["phase_icache_requests_lower_bound_per_core"]["median"], phase["phase_icache_requests_upper_bound_per_core"]["median"], @@ -686,7 +727,7 @@ def test_submit_pmu_v4_host_triplet_count_is_not_blindly_trusted(self) -> None: load_capture(path) def test_submit_pmu_fixed_phase_calls_are_exact_per_worker(self) -> None: - for phase in ("claim", "efdrain"): + for phase in ("claim", "efdrain", "materialize", "register"): with self.subTest(phase=phase): capture = _submit_pmu_capture(phase=phase) first = capture["records"][0] @@ -737,7 +778,7 @@ def test_submit_pmu_none_rejects_nonzero_phase_counters(self) -> None: } with tempfile.TemporaryDirectory() as directory: path = self._write(directory, "none-nonzero-phase.json", capture) - with self.assertRaisesRegex(ValueError, "phase=none must have zero"): + with self.assertRaisesRegex(ValueError, "zero calls must have zero"): load_capture(path) def test_submit_pmu_shadow_greater_than_primary_is_rejected_from_raw(self) -> None: @@ -750,6 +791,21 @@ def test_submit_pmu_shadow_greater_than_primary_is_rejected_from_raw(self) -> No with self.assertRaisesRegex(ValueError, "shadow whole exceeds"): load_capture(path) + def test_submit_pmu_shadow_miss_greater_than_request_is_rejected(self) -> None: + capture = _submit_pmu_capture() + record = capture["records"][0] + record["shadow_whole_icache_requests"] = record["shadow_whole_icache_misses"] - 1 + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), + "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), + } + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "shadow-miss-greater-than-request.json", capture) + with self.assertRaisesRegex(ValueError, "shadow I-cache miss > request"): + load_capture(path) + def test_submit_pmu_shadow_booleans_loss_and_upper_are_recomputed(self) -> None: mutations = ( ("shadow_matches_primary", True, "shadow_matches_primary disagrees"), @@ -865,6 +921,20 @@ def test_submit_pmu_configuration_and_host_gate_are_rechecked(self) -> None: with self.assertRaisesRegex(ValueError, "cnt5_shadow_icache_miss"): load_capture(path) + capture = _submit_pmu_capture() + capture["configuration"]["selectors"]["cnt2_scalar_busy"] = 0xDEAD + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "bad-scalar-selector.json", capture) + with self.assertRaisesRegex(ValueError, "cnt2_scalar_busy"): + load_capture(path) + + capture = _submit_pmu_capture() + capture["configuration"]["counter_width_bits"]["programmable"] = 64 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "bad-programmable-counter-width.json", capture) + with self.assertRaisesRegex(ValueError, "programmable must be 32"): + load_capture(path) + capture = _submit_pmu_capture() capture["validation"]["phase_boundary_match_records"] -= 1 with tempfile.TemporaryDirectory() as directory: @@ -881,6 +951,74 @@ def test_submit_pmu_configuration_and_host_gate_are_rechecked(self) -> None: ): load_capture(path) + def test_submit_pmu_rejects_retired_phase_id_three(self) -> None: + capture = _submit_pmu_capture() + capture["configuration"]["compiled_phase"] = "wait-for-slot" + capture["configuration"]["compiled_phase_id"] = 3 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "retired-phase-id-three.json", capture) + with self.assertRaisesRegex(ValueError, "unsupported configuration.compiled_phase"): + load_capture(path) + + def test_submit_pmu_rejects_scalar_busy_above_total(self) -> None: + capture = _submit_pmu_capture() + capture["records"][0]["scalar_busy"] = capture["records"][0]["total_cycles"] + 1 + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), + "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), + } + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "scalar-busy-above-total.json", capture) + with self.assertRaisesRegex(ValueError, "scalar_busy > total_cycles"): + load_capture(path) + + def test_submit_pmu_rejects_zero_total_cycles(self) -> None: + capture = _submit_pmu_capture() + capture["records"][0]["total_cycles"] = 0 + capture["records"][0]["scalar_busy"] = 0 + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), + "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), + } + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "zero-total-cycles.json", capture) + with self.assertRaisesRegex(ValueError, "zero total_cycles"): + load_capture(path) + + def test_submit_pmu_recomputes_programmable_counter_risk_threshold(self) -> None: + capture = _submit_pmu_capture() + capture["records"][0]["vector_busy"] = PROGRAMMABLE_COUNTER_RISK_THRESHOLD + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), + "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), + } + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "counter-risk-threshold.json", capture) + with self.assertRaisesRegex(ValueError, "32-bit counter risk threshold"): + load_capture(path) + + def test_integer_summary_fields_require_exact_equality(self) -> None: + capture = _submit_pmu_capture() + for worker_id, record in enumerate(capture["records"]): + record["total_cycles"] = 10**12 + worker_id + records = capture["records"] + capture["summary"] = { + "all": _submit_pmu_summary(records), + "aic": _submit_pmu_summary(records[:A5_AIC_WORKERS]), + "aiv": _submit_pmu_summary(records[A5_AIC_WORKERS:]), + } + capture["summary"]["all"]["total_cycles"]["sum"] += 1 + with tempfile.TemporaryDirectory() as directory: + path = self._write(directory, "inexact-large-integer-summary.json", capture) + with self.assertRaisesRegex(ValueError, "all.total_cycles.sum"): + load_capture(path) + def test_submit_pmu_phase_counter_order_is_rechecked(self) -> None: # 两个 ld_dev 不是原子配对快照;边界漂移可使局部 miss 略大于 request, # 只要两者分别不超过各自的 Submit whole 就仍是合法 raw 观察。 From 6caa269caab3b97a759b5f90385c0762c1f82c0b Mon Sep 17 00:00:00 2001 From: qinchuanyu Date: Sun, 19 Jul 2026 11:56:25 +0000 Subject: [PATCH 027/214] =?UTF-8?q?=E5=B7=A5=E5=85=B7(a5):=20=E6=94=B6?= =?UTF-8?q?=E6=95=9BSubmit=E8=A7=82=E6=B5=8B=E8=BE=B9=E7=95=8C=E4=B8=8E?= =?UTF-8?q?=E6=8E=92=E4=BB=96=E6=B3=B3=E9=81=93=E5=88=86=E6=9E=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 完善独立 Submit PMU 的逐核 I-cache、scalar 与阶段时间校验及 HTML 汇总\n- 将 schema-v4 Submit、编排和最终排空划分为严格排他区间,离线拆分内部、尾部与 Submit 间 residual\n- 删除无真实计算的 loser 伪阶段,瘦身 Perfetto 事件并保持 raw 记录规模不增长\n- 接入独立闭合分析器,补齐单元测试、A5 b1/b256 实测证据和中文使用文档 --- tests/atomic_probe/icache_miss_usage_guide.md | 278 +++- ...05\345\206\265\345\210\206\346\236\220.md" | 235 +++- ...77\347\224\250\346\214\207\345\215\227.md" | 258 +++- tests/atomic_probe/pa_scheduler/ccec/host.cpp | 84 +- .../atomic_probe/pa_scheduler/ccec/kernel.cpp | 26 +- .../pa_scheduler/ccec/pmu_probe.h | 5 +- .../pa_scheduler/common/host_support.h | 70 +- .../pa_scheduler/common/pa_model.h | 19 +- .../pa_scheduler/common/pa_scheduler_core.h | 122 +- .../pa_scheduler/pmu_html_report.py | 394 ++++-- .../pa_scheduler/pmu_sidecar_analyzer.py | 224 ++- tests/atomic_probe/pa_scheduler/run.sh | 25 +- .../pa_scheduler/swimlane_converter.py | 286 +++- .../swimlane_exclusive_analyzer.py | 1199 +++++++++++++++++ .../pa_scheduler/swimlane_opt_anal.md | 660 +++++++++ .../pa_scheduler/test_pmu_html_report.py | 263 +++- .../pa_scheduler/test_pmu_sidecar_analyzer.py | 170 ++- .../pa_scheduler/test_swimlane_converter.py | 219 +++ .../test_swimlane_exclusive_analyzer.py | 777 +++++++++++ 19 files changed, 4880 insertions(+), 434 deletions(-) create mode 100755 tests/atomic_probe/pa_scheduler/swimlane_exclusive_analyzer.py create mode 100644 tests/atomic_probe/pa_scheduler/swimlane_opt_anal.md create mode 100644 tests/atomic_probe/pa_scheduler/test_swimlane_exclusive_analyzer.py diff --git a/tests/atomic_probe/icache_miss_usage_guide.md b/tests/atomic_probe/icache_miss_usage_guide.md index 2871c1d97f..eac49ee484 100644 --- a/tests/atomic_probe/icache_miss_usage_guide.md +++ b/tests/atomic_probe/icache_miss_usage_guide.md @@ -4,7 +4,7 @@ 本指南面向 `tests/atomic_probe/pa_scheduler` -的 standalone CCEC 分支,目标是回答:在完整 Submit 期间,32 个 AIC +的 standalone CCEC 分支,目标是回答:在 Submit-all 整个调度回放期,32 个 AIC 和 64 个 AIV 每核发生了多少 I-cache request/miss,其中哪些 miss 值得继续优化。它不依赖 simpler 生产代码,也不将 standalone 结果 冒充为真实 PA 的绝对 profile。 @@ -14,7 +14,7 @@ | 构建 | 内容 | 是否包含 PMU | | --- | --- | --- | | `swimlane` | 普通阶段泳道 + 逐 atomic 泳道,在同一 AIC/AIV scalar lane 合并采集 | 否 | -| `submit-pmu` | 每物理子核的完整 Submit PMU,并可编译一个局部 phase | 是,仅 CCEC | +| `submit-pmu` | 每物理子核的 Submit-all PMU 整窗,并可编译一个局部 phase | 是,仅 CCEC | `run` 、`smoke` 和 phase 名是运行或编译选择,不是额外的第三类 构建。 @@ -36,7 +36,7 @@ atomic wrapper、ClockBaseline、runtime phase-profile 和旧 cold/warm 冲刷 两种数据不在同一进程采集: - 看事件时序和 atomic bracket,使用 `swimlane`; -- 看完整 Submit 每核 I-cache request/miss,使用 `submit-pmu`。 +- 看 Submit-all 整窗每核 I-cache request/miss,使用 `submit-pmu`。 两份证据可以按同一源码版本交叉理解,不能做逐 tick 对齐,也不能 把 PMU 的平均 miss 回填为某一条 atomic span 的属性。 @@ -47,26 +47,49 @@ atomic wrapper、ClockBaseline、runtime phase-profile 和旧 cold/warm 冲刷 | phase | 边界 | 优先用途 | | --- | --- | --- | -| `none` | 完整 Submit 中不读局部 shadow counter | 回答完整 Submit 的 AIC/AIV 每核 request/miss;这是默认选择 | -| `claim` | 每次 `Claim()` 调用前后读局部 shadow counter | 当 `none` 已证明 miss 值得追踪时,试验 Claim 的 running read-clear 下界/上界归因链路 | +| `none` | Submit-all 整窗中不读局部 shadow counter | 回答整个调度回放期的 AIC/AIV 每核 request/miss;这是默认选择 | +| `claim` | `Claim()`、结果写回 context 及 claim 本地统计前后读局部 shadow counter | 当 `none` 已证明 miss 值得追踪时,试验 Claim 的 running read-clear 下界/上界归因链路 | | `efdrain` | 每次 Submit 开头唯一的 `DrainReady(...EfDrain...)` 前后 | 观察 opportunistic drain;不混入 RingBackpressure 或 FinalDrain | -| `materialize` | 每次 `MaterializeTask()` 调用前后 | 观察输出 descriptor/layout、本地 register mask、输出字节数和 heap 游标等 scalar 工作;不包含后续 slot payload 拷贝 | -| `register` | Alloc/non-Alloc 两个互斥的 `RegisterOutputs()` 调用点前后 | 观察输出注册;边界与普通泳道的 Register span 对齐 | +| `materialize` | `MaterializeTask()` 及成功路径 `materialized_outputs` 本地统计前后 | 观察输出 descriptor/layout、本地 register mask、输出字节数和 heap 游标等 scalar 工作;不包含后续 slot payload 拷贝 | +| `register` | Alloc/non-Alloc 两个互斥的 `RegisterOutputs()`,non-Alloc 还包含 `map_inserts` 本地统计 | 观察输出注册语义体 | + +普通泳道为了减少观察扰动,会让相邻阶段复用同一个 +`SYS_CNT` 边界。PMU-only ELF 不生成这些泳道时间戳;局部 PMU bracket +只包围同一语义体,使用自己的 shadow read-clear 和 `SYS_CNT`,不做 +跨 ELF 的逐 tick 对齐。 running phase 的 begin/end 读取本身会执行 scalar 指令、占用取指并改变多核 -时序。因此: +时序;schema-v5 还在每次 begin/end 各读取一次 1 ns/tick 的 SYS_CNT。因此: - running phase 的 phase request/miss 是带局部边界扰动的观察值; -- raw 观察值是下界,上界为该核下界加 primary-shadow loss; +- running phase 的累计时间也是带边界扰动的直接观察值;起点在 begin 的 + shadow read-clear 之后,终点在 end 的 shadow read-clear 之前,所以不包含 + 两侧 `ld_dev`,但包含每次调用两次 SYS_CNT 的观察扰动; +- request/miss raw 观察值是下界,上界为该核下界加 + primary-shadow loss;阶段时间是单点观察值,没有伪造的上下界; - `none` 和任一 running phase 是不同 ELF、不同进程,不能以两者相减声称 得到了零扰动的局部净值; -- 未来不同 phase ELF 的局部 request/miss 不可相加成“完整 Submit”; - 完整 Submit 始终以每个 ELF 自己的 primary whole 为准。 +- 未来不同 phase ELF 的局部 request/miss 不可相加成 whole gate; + Submit-all 整窗始终以每个 ELF 自己的 primary whole 为准。 该区间只描述同一插桩 ELF、当前边界定义下的局部事件,不是无插桩局部阶段 -的真实区间。完整 Submit `none` 没有运行中 read-clear,仍执行 96/96 +的真实区间。Submit-all `none` 没有运行中 read-clear,仍执行 96/96 逐核严格闭合。 +报告中有三个相关但不相同的时间窗,不能都简称为“完整 +Submit”: + +1. **PMU whole gate**:每核在 `InitPaOrchestration()` 和首次构参之前 + `metrics_prof_start()`,在末次 UP `SubmitTask()` 返回后立即 stop。它包含 + orchestration 初始化、`Build*Args()`、`AcceptTaskOutputs()` 和 Submit 间衔接, + 排除 FinalDrain;与泳道 `OrchestrationReplay` 父区间接近但不做逐 tick + 对齐。CNT6/CNT7 primary 和 PMU total/scalar-busy 都使用这个窗。 +2. **`submit_elapsed_ticks`**:每核从首个 `BeginSubmit()` 之后到最后一个 + Submit 的 `submits++` 之后,包含两端之间的 Submit 间衔接,但不包含 + whole gate 首尾的 orchestration 外围。局部 phase 时间占比以它为分母。 +3. **`configuration.submit_span_us`**:96 核中最早 `submit_begin` 到最晚 + `submit_end` 的全局墙钟范围,不是逐核 PMU 时间的平均值。 + ## 4. 环境、构建与产物 命令在 `tests/atomic_probe/pa_scheduler` 目录下执行。非交互 shell @@ -81,7 +104,7 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" export PYTHON=/home/q00473782/.venv/bin/python ``` -构建完整 Submit 基准: +构建 Submit-all 整窗基准: ```bash ./run.sh build-submit-pmu ccec none @@ -132,16 +155,16 @@ build/ccec/submit-pmu/register/ `swimlane` 的 CCEC 产物仍在 `build/ccec/`,不是 PMU 产物。 -## 5. 采集完整 Submit 与局部 phase +## 5. 采集 Submit-all 整窗与局部 phase -### 5.1 完整 Submit `none` +### 5.1 Submit-all 整窗 `none` ```bash OUT="./outputs/submit_pmu_none_$(date -u +%Y%m%dT%H%M%SZ)" mkdir -p "$OUT" ./run.sh submit-pmu ccec none \ - --device 0 --batches 256 \ + --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT/submit_icache_raw.json" ``` @@ -152,11 +175,11 @@ mkdir -p "$OUT" ```bash mkdir -p "$OUT/capture_02" "$OUT/capture_03" -./run.sh submit-pmu ccec none --device 0 --batches 256 \ +./run.sh submit-pmu ccec none --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT/capture_02/submit_icache_raw.json" -./run.sh submit-pmu ccec none --device 0 --batches 256 \ +./run.sh submit-pmu ccec none --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT/capture_03/submit_icache_raw.json" ``` @@ -168,7 +191,7 @@ OUT_CLAIM="./outputs/submit_pmu_claim_$(date -u +%Y%m%dT%H%M%SZ)" mkdir -p "$OUT_CLAIM" ./run.sh submit-pmu ccec claim \ - --device 0 --batches 256 \ + --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT_CLAIM/submit_icache_raw.json" ``` @@ -180,42 +203,36 @@ OUT_EFDRAIN="./outputs/submit_pmu_efdrain_$(date -u +%Y%m%dT%H%M%SZ)" mkdir -p "$OUT_EFDRAIN" ./run.sh submit-pmu ccec efdrain \ - --device 0 --batches 256 \ + --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT_EFDRAIN/submit_icache_raw.json" ``` -`efdrain` 每核固定调用 `batches * 5` 次;b256 的 AIC/AIV/global calls -分别为 40,960/81,920/122,880。插点只位于 Submit 开头的 EfDrain 专属 +`efdrain` 每核固定调用 `batches * 5` 次;b1 的 AIC/AIV/global calls +分别为 160/320/480。插点只位于 Submit 开头的 EfDrain 专属 call-site;复用的 `DrainReady()` 函数体不插桩。 ### 5.4 Materialize 局部归因 -先用 b1 验证边界和 96 核闭合,再跑 b256 正式负载: +后续边界和 96 核闭合验证固定使用 b1: ```bash OUT_MAT_B1="./outputs/submit_pmu_materialize_$(date -u +%Y%m%dT%H%M%SZ)_b1" -OUT_MAT_B256="./outputs/submit_pmu_materialize_$(date -u +%Y%m%dT%H%M%SZ)_b256" -mkdir -p "$OUT_MAT_B1" "$OUT_MAT_B256" +mkdir -p "$OUT_MAT_B1" ./run.sh submit-pmu ccec materialize \ --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT_MAT_B1/submit_icache_raw.json" - -./run.sh submit-pmu ccec materialize \ - --device 0 --batches 256 \ - --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json "$OUT_MAT_B256/submit_icache_raw.json" ``` 边界位于 `MaterializeTask()` 唯一调用点前后。实现必须先保存返回值、关闭 phase,再处理失败返回,避免失败路径留下 begin/end 不平衡。每核固定 -`5 * batches` 次;b1 的 AIC/AIV/global calls 为 160/320/480,b256 为 -40,960/81,920/122,880。 +`5 * batches` 次;b1 的 AIC/AIV/global calls 为 160/320/480。 -2026-07-19 A5 实测闭环如下;四轮均满足 capture accepted、语义、PMU 和 -phase measurement PASS: +2026-07-19 A5 历史实测闭环如下;四轮均满足 capture accepted、 +语义、PMU 和 phase measurement PASS。b256 数据只作归档证据,不是后续迭代的 +重跑要求: | phase | batches | calls/expected | begin/end 与 call shape | primary=shadow | shadow≤primary | request/miss loss | | --- | ---: | ---: | ---: | ---: | ---: | ---: | @@ -235,24 +252,19 @@ running read-clear 的硬门禁是 96/96 `shadow≤primary`,不是要求 96/96 ```bash OUT_REG_B1="./outputs/submit_pmu_register_$(date -u +%Y%m%dT%H%M%SZ)_b1" -OUT_REG_B256="./outputs/submit_pmu_register_$(date -u +%Y%m%dT%H%M%SZ)_b256" -mkdir -p "$OUT_REG_B1" "$OUT_REG_B256" +mkdir -p "$OUT_REG_B1" ./run.sh submit-pmu ccec register \ --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT_REG_B1/submit_icache_raw.json" - -./run.sh submit-pmu ccec register \ - --device 0 --batches 256 \ - --winner-workload real-compute --real-compute-counts 6,28,4,1 \ - --pmu-json "$OUT_REG_B256/submit_icache_raw.json" ``` Register 有 Alloc 和 non-Alloc 两个互斥调用点,二者都使用同一 phase 边界;winner/loser 均从其中一条路径通过,因此仍是每核固定 -`5 * batches`,调用规模与 Materialize 相同。该窗口与普通泳道 Register -span 对齐;其中较短或未实际插入 map 的调用仍会放大 PMU begin/end +`5 * batches`,调用规模与 Materialize 相同。它与普通泳道包围 +同一 Register 语义体,但两个 ELF 各自取时,不做逐 tick 对齐;其中 +较短或未实际插入 map 的调用仍会放大 PMU begin/end 边界扰动,解释结果时必须使用 lower/upper,而不能把 observed lower 当成无扰动净开销。 @@ -274,20 +286,48 @@ submit_icache_raw.json # 96 核权威原始件及 host summary submit_icache_report.html # 可离线浏览的加工件 ``` -HTML 使用内联 CSS/SVG,不访问网络,也不依赖外部前端库;浏览器直接打开即可。 -它包含完整 Submit 的 AIC/AIV 对比、逐物理核 request/miss/rate 分布、96 核 +HTML 使用内联 CSS/SVG,不依赖外部前端库;浏览器直接打开即可。 +它包含 Submit-all PMU 整窗的 AIC/AIV 对比、逐物理核 request/miss/rate 分布、96 核 明细和 90 ns core-equivalent 提示。报告同时展示 AIC/AIV/ALL 的 PMU raw `total_cycles`、CNT2 `scalar_busy`、`Σscalar/Σtotal` 以及 -“非 Scalar-busy 残余”。其中 total 是每个物理子核在 Submit gate 内的 PMU +“非 Scalar-busy 残余”,并在保留 raw cycle 的同时给出校准后的每核等效时间。 +顶部的“完整 Submit”固定表示**最早一次 Submit 进入到最晚一次 Submit 返回**; +它是 96 核共同形成的整体墙钟范围,不等于逐核 PMU whole gate +的平均值;对应到每个 worker,首末 Submit 边界也窄于该核的 PMU +whole gate。 +ALL/AIC/AIV 分别用响应式卡片展示逐核 `min/mean/max`:`mean` 作为唯一的 +典型值,`min/max` 只用于呈现核间范围。raw summary 没有 `min` 字段,HTML +从通过 96 核门禁的 `records` 对称推导 `min/max`;PMU total 与 scalar busy +各自独立取极值,不保证来自同一个物理核。较宽的 I-cache 对比表和 96 核明细 +只在表格内部横向滚动,不再撑宽整页。 + +本机 A5 受控 cold/warm 同窗校准得到: + +```text +PMU cycle_delta = 1,817,457 +SYS_CNT tick_delta = 1,101,593 ns +ALL = 1.649844 cycles/ns +AIC = 1.650062 cycles/ns +AIV = 1.649731 cycles/ns +time_us = PMU cycles / (cycles_per_ns * 1000) +``` + +ALL/AIC/AIV 汇总分别使用对应频率,逐核明细按该核角色使用 AIC 或 AIV +频率。其中 total 是每个物理子核在 PMU whole gate 内的 累计周期,96 核求和是 core-work,不是 Submit 墙钟;“非 Scalar-busy 残余” 严格等于 `total−scalar_busy`,既不是 Scalar 空闲时间,也不是 I-cache stall, 其中还包含同步等待、vector/cube engine 等待以及其他未归因周期。受控微基准中,依赖返回的 atomic 等待大部分进入 scalar busy,而 I-cache refill 的额外周期大部分只进入 total;这个现象 不能把二者之差提升为 I-cache 专属计数器。 -对于局部 phase,HTML 还会单列 calls、 -request/miss 下界—上界、shadow loss,以及局部事件占同一 ELF 完整 Submit -primary 的比例区间,不把局部数据冒充可相加的精确分解。 +对于局部 phase,HTML 最前面先按 ALL/AIC/AIV 展示 calls、阶段时间占比、 +request/miss 占比;后面再列阶段时间/core、阶段时间/call、request/miss +下界—上界和 shadow loss。阶段时间占比是同一角色内 +`Σphase_elapsed_ticks / Σsubmit_elapsed_ticks`,request/miss 则仍是占同一 +ELF PMU whole-gate primary 的比例区间。两类占比的分母边界不同,只能 +分别用于时间和 I-cache 归因,不能把它们当成同一精确分区。`none` +明确显示“不适用”,历史 schema-v4 +因没有阶段时间 raw 字段而显示“不可用”,不会伪造 0%。 报告生成失败时 action 返回非零,但已成功发布的 raw 会保留用于排查。 ## 6. Primary/shadow 计数和可信门禁 @@ -296,10 +336,10 @@ primary 的比例区间,不把局部数据冒充可相加的精确分解。 | 计数 | selector | 用途 | | --- | --- | --- | -| PMU raw total | 固定 64-bit total low/high | Submit gate 内每个物理子核的累计周期;不是 96 核求和后的墙钟 | -| CNT2 | `0x001` | scalar instruction busy cycle;不包含全部等待周期 | -| CNT6 | `0x34` | 完整 Submit primary I-cache request;局部边界从不读它 | -| CNT7 | `0x35` | 完整 Submit primary I-cache miss;局部边界从不读它 | +| PMU raw total | 固定 64-bit total low/high | PMU whole gate 内每个物理子核的累计周期;不是 96 核求和后的墙钟;HTML 另按实测频率显示等效时间 | +| CNT2 | `0x001` | scalar instruction busy cycle;不包含全部等待周期;HTML 另按实测频率显示等效时间 | +| CNT6 | `0x34` | PMU whole-gate primary I-cache request;局部边界从不读它 | +| CNT7 | `0x35` | PMU whole-gate primary I-cache miss;局部边界从不读它 | | CNT8 | `0x34` | read-to-clear shadow request | | CNT5 | `0x35` | read-to-clear shadow miss;诊断 ELF 因此不再提供 MTE3 busy | | CNT9 | `0x0` | 未使用 | @@ -335,7 +375,7 @@ I-cache warm/cold、真实 Vector/Cube `PIPE_* -> PIPE_S` wait 和依赖 atomic shadow PMU counter 是 read-to-clear。任一 running phase 在阶段 begin 读取 CNT8/CNT5,将 之前的片段加入 shadow whole;在 end 再读一次,同时加入 shadow -whole 和所选 phase;完整 Submit stop 后读 tail。`none` 不做中途读取, +whole 和所选 phase;PMU whole gate stop 后读 tail。`none` 不做中途读取, 只在 stop 后取 tail。 `none` 对每个物理子核必须精确满足: @@ -346,10 +386,10 @@ shadow_whole_icache_misses == icache_misses ``` 即 stop 后读取的 CNT8/CNT5 分别等于同 selector、同 gate 的 CNT6/CNT7。 -这个 96/96 精确相等门禁验证完整 Submit 观察闭合;它不把 PMU 进程的 +这个 96/96 精确相等门禁验证 whole-gate 观察闭合;它不把 PMU 进程的 Submit span 变成无诊断墙钟基线,也不把 standalone 数据冒充真实 PA profile。 -运行中切片已在 A5 b1/b256 上证明可能发生单向少计,接受规则为: +历史 A5 b1/b256 取证已证明运行中切片可能发生单向少计,接受规则为: ```text shadow_request <= primary_request @@ -383,7 +423,7 @@ CNT8/CNT5 是顺序 `ld_dev`,不是同一时刻的原子配对快照,因此 - 语义、winner 真计算输出和 Submit placement/engine 闭合通过; - 96 条记录可信,32 AIC + 64 AIV,物理子核 id 唯一; - owner bitmap membership、worker slot、物理 role 和 32 个 1:2 triplet 全部匹配; -- 96 个核都真实执行完整 Submit PMU start/stop,owner Restore 成功; +- 96 个核都真实执行 Submit-all PMU whole-gate start/stop,owner Restore 成功; - build variant 和编译 phase id 在 96 条记录中全部匹配; - `none` 的 shadow whole 与 primary whole 逐核精确相等; - running phase 的 shadow whole 逐核不大于 primary,loss 与 upper-bound @@ -397,36 +437,49 @@ CNT8/CNT5 是顺序 `ld_dev`,不是同一时刻的原子配对快照,因此 两个互斥 Register 调用点; - phase request/miss 分别不超过对应 shadow/primary,且可编程 counter 低于当前 25% 保守风险阈值。 +- 96 个核的 `submit_elapsed_ticks` 都大于 0;running phase 的 + `phase_elapsed_ticks` 大于 0 且不超过本核 `submit_elapsed_ticks`,`none` + 则必须精确为 0;phase time 状态位和 host 复核都必须通过。 -`metrics_prof_start/stop()` 在完整 Submit 前后各执行一次,其 +`metrics_prof_start/stop()` 在 PMU whole gate 前后各执行一次,其 `PIPE_ALL` 边界会改变流水和多核时序。PMU 结果只与相同构建、 相同 phase、相同负载的独立进程比较,不把 PMU 进程的 Submit span 当作无诊断性能基线。 ## 7. JSON 字段与 AIC/AIV 分析口径 -当前 `submit-pmu` 输出 schema v4。`records` 保留 96 个 worker 的 raw, +当前 `submit-pmu` 输出 schema v5;分析器继续只读兼容历史 schema-v4。 +`records` 保留 96 个 worker 的 raw, `summary.all/aic/aiv` 分别对 96/32/64 个核统计: ```text sum / mean / median / p95 / max ``` -完整 Submit 优先查看: +Submit-all 整窗优先查看: - `configuration.submit_span_us`:本轮从第一个 Submit 进入到最后一个 Submit 返回的整体 span;HTML 顶部换算成毫秒展示; -- `total_cycles`:每核 Submit gate 内的 64-bit PMU raw total;按角色的 - sum 是 core-work,mean/median/p95 才适合比较典型单核,均不等于 host - 看到的 Submit 墙钟; +- `total_cycles`:每核 PMU whole gate 内的 64-bit PMU raw total;按角色的 + sum 是 core-work,raw 仍保留 mean/median/p95;HTML 只选 mean 表示典型 + 单核,并辅以从逐核记录得到的 min/max,三者均不等于 host 看到的 Submit + 墙钟。HTML 按 ALL/AIC/AIV 的实测频率将 raw cycle 换算为单核 + cycle-equivalent; - `scalar_busy`:CNT2 `scalar_instr_busy(0x001)`,表示 scalar instruction busy cycle;依赖返回的 atomic 等待可进入此项,但它不是“纯算术指令数”; -- `icache_requests` / `icache_misses`:CNT6/CNT7 primary whole; + 换算后的时间也只是 scalar-busy cycle-equivalent; +- `icache_requests` / `icache_misses`:CNT6/CNT7 PMU whole-gate primary; - `shadow_whole_icache_requests` / `shadow_whole_icache_misses`:闭合或分段 loss 用 shadow whole; - `shadow_request_loss` / `shadow_miss_loss`:本核 primary-shadow residual; - `phase_calls` / `phase_icache_requests` / `phase_icache_misses`:选定 phase 的 running read-clear lower; +- `submit_elapsed_ticks`:本 worker 从首个 `submit_begin` 计时点到末个 + `submit_end` 计时点的 SYS_CNT 差值;起点位于首个 + `BeginSubmit()` 上下文初始化之后,终点位于末个 Submit 返回之前; + 1 tick = 1 ns; +- `phase_elapsed_ticks`:所选 phase 所有调用的 SYS_CNT 差值累计;running + phase 必须非零且不超过同核 `submit_elapsed_ticks`,`none` 必须为 0; - `phase_icache_requests_upper_bound` / `phase_icache_misses_upper_bound`: lower 加本核对应 loss; - `configuration.compiled_phase` 和 `validation.phase_measurement_valid`:确认文件口径。 @@ -446,7 +499,7 @@ HTML 中展示的“非 Scalar-busy 残余/core”严格等于 `phase_icache_request_upper_bound_share_of_submit`、 `phase_icache_miss_lower_bound_share_of_submit` 和 `phase_icache_miss_upper_bound_share_of_submit`:局部 lower/upper 分别除以 -同一角色、同一次采集的完整 Submit primary 总数。 +同一角色、同一次采集的 PMU whole-gate primary 总数。 `phase_observed_read_clear_ratio` 只是 lower miss/lower request 的观测比值; 分子和分母各有独立区间,因此它不是实际 phase miss rate 的数学下界。 @@ -464,6 +517,19 @@ AIV request/core = summary.aiv.icache_requests.sum / 64 AIV miss/core = summary.aiv.icache_misses.sum / 64 ``` +PMU cycle 的时间换算通式和默认校准值为: + +```text +time_us = cycles / cycles_per_ns / 1000 +ALL cycles_per_ns = 1.649844 +AIC cycles_per_ns = 1.650062 +AIV cycles_per_ns = 1.649731 +``` + +这里的 `time_us` 是每核 cycle-equivalent。`total−scalar_busy` 即使换算为时间, +仍不是 Scalar 空闲时间或 I-cache stall;96 核 sum 换算后也仍是 core-work, +不能冒充 Submit 墙钟。 + `median` 和 `p95` 直接来自同角色逐核 raw 分布,用于观察典型核和高 尾核。I-cache miss rate 只按组内加权口径计算: @@ -475,17 +541,23 @@ AIV miss rate = Σ(AIV miss) / Σ(AIV request) 不平均 32 或 64 个逐核百分比。AIC/AIV 核数不同,比较每核强度时 使用 mean/median/p95 或 miss rate,不直接比较两组 sum。 -局部 phase 占完整 Submit 的比例同样使用组内总量: +局部 phase 的时间和 I-cache 占比都使用组内总量,但分母边界不同: ```text +time share = Σphase_elapsed_ticks / Σsubmit_elapsed_ticks request share lower = Σphase_request_lower / Σprimary_request request share upper = Σphase_request_upper / Σprimary_request miss share lower = Σphase_miss_lower / Σprimary_miss miss share upper = Σphase_miss_upper / Σprimary_miss ``` +时间分子、分母来自同一个 1 ns SYS_CNT,是逐核累计 core-time 构成;不能用 +`Σphase_elapsed_ticks` 除以 96 核共同形成的 `submit_span_us`。时间是直接观察 +单值,不仿造 request/miss 那样的 lower/upper。多个独立进程聚合时,分析器展示 +每轮上述 `Σ/Σ` 比值的分布,不把不同轮的 raw 重新拼成一次虚构运行。 + 分子与分母必须来自同一个 phase ELF、同一轮采集和同一角色。该比例回答“当前 -插桩 ELF 中局部窗口占自身完整 Submit 事件的多少”,不能拿 `claim` 分子除以 +插桩 ELF 中局部窗口占自身对应分母的多少”,不能拿 `claim` 分子除以 另一份 `none` 的分母。 ## 8. HTML 报告与多轮分析命令 @@ -540,6 +612,9 @@ PYTHON=/home/q00473782/.venv/bin/python 该角色平均每核等效量(us/core) = (Σmiss / core_count) * 0.09 ``` +约 1.65 GHz 的频率只用于换算 PMU cycle 事件;它不改变由 1 ns SYS_CNT +cold/warm delta 得到的 `90 ns/miss` 标尺。 + 例如 AIV 平均 70,000 miss/核,感性等效量是约 6,300 us/核。这不表示 Submit 墙钟真的损失了 6.3 ms,原因包括: @@ -558,11 +633,12 @@ core-work 等效总量,不是端到端 Submit 总损失。要测真正暴露 3. 只有第 2 项是实际暴露的墙钟收益;第 1 项用于证明收益与 I-cache 变化同时出现,`90 ns` 仅提供一阶数量级解释。 -### 9.1 当前 b256 `none` 参考数据 +### 9.1 历史 b256 `none` 参考数据 -2026-07-19 用最终 `submit-pmu none` ELF、`real-compute/6,28,4,1` -在 A5 上采集一轮,完整 Submit span 为 `4.750810 ms`。96 核 raw、owner +2026-07-19 用当时的 `submit-pmu none` ELF、`real-compute/6,28,4,1` +在 A5 上采集一轮,全局首末 Submit span 为 `4.750810 ms`。96 核 raw、owner Restore、selector、counter 阈值和离线复算全部通过。 +该 b256 只作归档数据,不是后续重跑要求。 | 指标 | AIC(32 核) | AIV(64 核) | | --- | ---: | ---: | @@ -570,9 +646,12 @@ Restore、selector、counter 阈值和离线复算全部通过。 | miss/core | 38,664.344 | 55,098.625 | | `Σmiss/Σrequest` | 9.4692% | 13.0417% | | total/core | 7,471,385.531 | 7,085,178.734 | +| total/core 校准等效时间 | 4,527.942 us | 4,294.748 us | | scalar busy/core | 5,944,751.250 | 5,603,587.469 | +| scalar busy/core 校准等效时间 | 3,602.744 us | 3,396.667 us | | `Σscalar/Σtotal` | 79.5669% | 79.0889% | | 非 Scalar-busy 残余/core | 1,526,634.281 | 1,481,591.266 | +| 非 Scalar-busy 残余/core 校准等效时间 | 925.198 us | 898.081 us | | `miss/core × 90 ns` | 3,479.791 us | 4,958.876 us | 最后一行只是单核串行等效标尺,不能与 `4.750810 ms` 相减或解释成 @@ -584,6 +663,61 @@ outputs/submit_pmu_none_20260719_b256_final/submit_icache_raw.json outputs/submit_pmu_none_20260719_b256_final/submit_icache_report.html ``` +### 9.2 历史 schema-v5 b256 分段时间参考数据 + +2026-07-19 在同一 A5、`real-compute/6,28,4,1`、b256 配置下重新独立采集 +`none|claim|efdrain|materialize|register`。五轮均为 96/96 有效记录;四个 +running phase 都是 1,280 calls/core,时间均满足 `0 < phase <= 同核 Submit`。 +这些是历史归档件;后续边界迭代和重采默认只使用 b1。 + +| phase | Submit span | ALL 时间占比 | AIC 时间占比 | AIV 时间占比 | +| --- | ---: | ---: | ---: | ---: | +| `none` | 3.711584 ms | 不适用 | 不适用 | 不适用 | +| `claim` | 4.401747 ms | 12.0845% | 7.7096% | 14.2972% | +| `efdrain` | 3.592376 ms | 15.5068% | 20.2974% | 13.1974% | +| `materialize` | 6.770266 ms | 16.7186% | 15.4860% | 17.3556% | +| `register` | 4.086936 ms | 4.3089% | 3.6256% | 4.6568% | + +每行都只解释自己的诊断 ELF。尤其 Materialize 的运行中边界读取显著改变了 +该轮 Submit 时序;这些时间占比不能跨行相加,Submit span 也不能与 `none` +相减成局部净开销。对应权威 raw 和加工 HTML 为: + +```text +pa_scheduler/outputs/submit_pmu_phase_time_v5_20260719/none_b256/ +pa_scheduler/outputs/submit_pmu_phase_time_v5_20260719/claim_b256/ +pa_scheduler/outputs/submit_pmu_phase_time_v5_20260719/efdrain_b256/ +pa_scheduler/outputs/submit_pmu_phase_time_v5_20260719/materialize_b256/ +pa_scheduler/outputs/submit_pmu_phase_time_v5_20260719/register_b256/ +``` + +### 9.3 当前边界联动版 b1 门禁 + +2026-07-19 在普通泳道相邻边界收敛后,重新构建五个 +`submit-pmu` ELF,并只跑 A5 b1。五轮均使用 +`real-compute/6,28,4,1`,都通过 96/96 物理核、真计算输出、mixed 引擎观察、 +PMU start/stop、owner Restore、phase call/time 和 primary/shadow 门禁: + +| phase | 全局首末 Submit | calls | exact/bounded 核 | request/miss loss | phase 时间占比 | +| --- | ---: | ---: | ---: | ---: | ---: | +| `none` | 113.821 us | 0/0 | 96/96 | 0/0 | 不适用 | +| `claim` | 61.689 us | 480/480 | 80/96 | 7/9 | 22.1744% | +| `efdrain` | 127.969 us | 480/480 | 92/96 | 4/0 | 9.7505% | +| `materialize` | 65.023 us | 480/480 | 96/96 | 0/0 | 41.7821% | +| `register` | 134.714 us | 480/480 | 90/96 | 6/0 | 8.5663% | + +`exact/bounded` 中 exact 表示 shadow 与 primary 逐值相等,bounded 表示满足 +`shadow <= primary`。五轮 bounded 都是 96/96,小幅 loss 已进入局部 +lower/upper,没有被忽略。 + +该表只证明当前业务语义体与局部 PMU bracket 同步修正后仍严格 +闭合。b1 的全局 Submit 易受冷启动、多核到达和局部边界扰动影响, +不用五行之间的时间差声称 phase 净成本或性能改善。权威 raw 和各自 +HTML 位于: + +```text +pa_scheduler/outputs/submit_pmu_boundary_sync_b1_20260719/{none,claim,efdrain,materialize,register}/ +``` + ## 10. 新增局部 phase 的修改清单 新 phase 不能只增加一个 CLI 字符串。最小完整修改包括: @@ -607,7 +741,9 @@ outputs/submit_pmu_none_20260719_b256_final/submit_icache_report.html 保证不同 phase 输入不会被聚合。 8. 补充 host/analyzer 回归:`none` 验证 96/96 primary-shadow 精确相等; running phase 验证逐核 bounded、loss/upper 公式、begin/end/calls 和语义, - 任一 shadow 反向大于 primary 都必须拒绝。先跑 A5 b1,再进入 b256。 + 任一 shadow 反向大于 primary 都必须拒绝。新增 phase 的构建、门禁和迭代 + 默认只跑 A5 b1。普通泳道仍复用相邻既有 end,不为了对齐 PMU + 而额外增加泳道 `SYS_CNT`。 每个 phase 必须是独立 ELF 和独立进程。不为了一次运行得到多个 phase,而在热路加运行时 phase switch 或多组 begin/end。 diff --git "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" index d12d54ace4..91883f18ae 100644 --- "a/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" +++ "b/tests/atomic_probe/pa_scheduler/PA-atomic\346\203\205\345\206\265\345\210\206\346\236\220.md" @@ -32,8 +32,10 @@ standalone 观察链路的最新状态更新至 2026-07-19。 - 优化没有改变通用 atomic 语义,也没有把任务推迟到最终 drain 来制造 表面收益。F1 的 fanin 顺序重排已经证明性能回退并撤销;下一步先精确区分 fanin 成功/失败 load 与 frontier 重复前推,再进行单变量消减。 -- standalone 观察产物现已固定为两类:`swimlane` 合并普通阶段与 schema-v3 - atomic(direct Atomic 加 PollBatch)泳道;`submit-pmu` 独立重编译完整 Submit PMU,现行 +- standalone 观察产物现已固定为两类:`swimlane` 使用 schema-v4 合并 + 排他业务阶段与 atomic(direct Atomic 加 PollBatch)泳道;atomic flags、 + weighted call 和 PollBatch ABI 沿用已验证的 schema-v3 语义。`submit-pmu` + 独立重编译完整 Submit PMU,现行 白名单为 `none|claim|efdrain|materialize|register`。两者不在同一进程采集; `none` 提供完整 Submit 的严格闭合计数,局部 phase 只提供 running read-clear 的下界/保守上界。 @@ -327,10 +329,12 @@ Submit 调度而非 kernel 计算变快。 - `DrainWon`、`RingBp`、kernel placement 和最终 drain 数量没有异常漂移; - 同时报告中位数、离散度、最好/最差值,并记录设备是否独占。 -泳道统计时,`Build` 和 `Replay` 是 lap marker,会覆盖前面的阶段,不能与 -`Materialize/PrepareMap/Claim/Fanin/Register` 相加。可加总的是显式互斥 -span,或单独定义 `Submit.end - Build/Replay/Alloc.end` 为 action tail。 -后续文档和脚本都应沿用这一口径。 +真实 PA 与历史 standalone schema-v3 泳道中的 `Build` 和 `Replay` 是 lap +marker,会覆盖前面的阶段,不能与 +`Materialize/PrepareMap/Claim/Fanin/Register` 相加。当前 standalone +schema-v4 已改为显式互斥的 `WinnerBuild/LoserReplay/AllocComplete` 尾 span; +后续文档和脚本必须先按 capture schema 选择口径,不能把历史 lap 与 v4 尾 span +混为一类。 ## 6. 独立 PA 调度复现的对应关系 @@ -1024,6 +1028,11 @@ frontier helping 的额外 FetchMax 中位数为 `A-T=14085` 次。ready-prefix ### 7.5 阶段 O2:atomic schema-v3 泳道与 Submit scalar PMU +本节保留 atomic ABI 在 schema-v3 中建立和验收的完整过程。当前 raw 已 +升为 schema-v4,atomic site/op/flags、PollBatch 与 weighted summary 语义不变; +atomic ABI 没有重定义。phase schema 则新增排他父区间,以真实 Submit 尾 span +替换旧 lap,将 EfDrain 改为显式 span,并禁止未使用的 `DrainWon`。 + #### 7.5.1 观察目标与证据拆分 O2 先完成观察链路,不在同一阶段继续消减 atomic。它回答两个不同问题: @@ -1178,7 +1187,9 @@ PollBatch 使用另一种时间语义:`duration`/`poll_window_cycles` 是从 不同 site 的窗口可以重叠,窗口内还可能包含 direct Atomic,不能把 PollBatch 混入 direct 单次延迟的 median/p95。 -边界实现复用真实 PA 已验证的规则,并让 phase/lap 与 batch 使用同一次 cycle 采样: +以下是历史 schema-v3 边界实现复用真实 PA 规则时的约束;其中 lap helper 仍保留 +历史兼容能力,但当前 standalone schema-v4 producer 已不再调用 +`ResetTraceLap/WriteTraceLap`: 1. 显式等待区退出时关闭与该 region 匹配的 PollBatch; 2. `TraceTimestamp` 先采 cycle,再以该 cycle 关闭全部活跃 batch,然后写 phase @@ -1241,8 +1252,8 @@ trace 容量不足时必须明确报 overflow 并判该轮观察无效,不能 #### 7.5.5 Submit PMU 窗口与平均口径 CCEC 只保留一个正式逐 worker Submit 窗口,CPU/AscendC 对应 hook 是空实现,不伪造 -PMU 数据:`submit-all` 在本 worker 通过启动屏障、完成 `ResetTraceLap` 后,于 -orchestration 初始化前打开 gate,在该 worker 最后一次 Submit 返回后关闭。窗口包含 +PMU 数据:`submit-all` 在本 worker 通过启动屏障后,于 orchestration 初始化前 +直接调用 `PmuWindowStart`,在该 worker 最后一次 Submit 返回后关闭。窗口包含 参数构造、全部 Submit 调度,以及本 worker 在窗口内执行的 winner 计算体:当前 无参数默认的 `real-compute` 是真实 Cube/Vector 与对应搬运流水;显式 `scalar-nop` 校准样本的 NOP 则在 scalar 上执行并计入窗口。 @@ -1260,9 +1271,12 @@ worker 到达与争用时序。这个开销属于观察配置本身;只能比 PMU JSON sidecar 按 worker 保留 raw 记录,并分别汇总 32 AIC、64 AIV 和全部 96 核。 各字段严格按下面口径解释: -- `total_cycles`:gate 活跃期间的每核 64 bit PMU raw total;除非平台时钟/事件语义 - 另有正式证明,不直接按 1 GHz 换算成微秒;96 核求和是 core-work raw count, - 不是 Submit 墙钟时间。墙钟只看 host 记录的 `submit_span_us`; +- `total_cycles`:gate 活跃期间的每核 64 bit PMU raw total;本机受控同窗校准为 + `1,817,457 PMU cycles / 1,101,593 ns = 1.649844 cycles/ns`,AIC/AIV + 分别约为 `1.650062/1.649731 cycles/ns`。因此可按 + `time_us = cycles / cycles_per_ns / 1000` 换算每核 cycle-equivalent;96 核 + 求和仍是 core-work,不是 Submit 墙钟时间。墙钟只看 host 记录的 + `submit_span_us`; - `scalar_busy`:`CNT2 scalar_instr_busy` 的事件累计,不等于窗口内全部时间, 也不等于“纯调度耗时”。atomic 由 scalar 发射,其发射、返回值依赖或资源 阻塞可能在该事件中有所体现,但不能据此认定某条 atomic 的全部执行完成延迟 @@ -1281,7 +1295,12 @@ PMU JSON sidecar 按 worker 保留 raw 记录,并分别汇总 32 AIC、64 AIV raw JSON 与自包含 HTML 都必须同时保留完整 Submit 的 `submit_span_us`、 `total_cycles` 和 `scalar_busy`,并按 ALL/AIC/AIV 显示每核分布;不得只展示 -I-cache request/miss。当前局部 phase 边界仅中途 read-clear shadow request/miss, +I-cache request/miss。HTML 在保留 raw cycle 的同时按上述角色频率显示等效 µs; +顶部“完整 Submit(最早开始 → 最晚结束)”单列 96 核整体耗时,ALL/AIC/AIV +角色卡片则只用逐核 mean 表示典型值,并补充 min/max 显示核间范围,二者不是 +同一个统计量;total 与 scalar 的极值也不保证来自同一物理核。 +SYS_CNT 的 1 ns tick 和 `90 ns/miss` 一阶标尺不受这一 PMU 频率换算影响。当前局部 +phase 边界仅中途 read-clear shadow request/miss, 没有局部 `total_cycles` 或 `scalar_busy`。文档和 HTML 中的 total/scalar 因此都是 该 phase ELF 的完整 Submit 窗口,不能当成 Materialize、Register 等局部阶段的独占时间。 90 ns/miss 仍只是受控 cold/warm 探针的一阶 core-equivalent 标尺,既不加入 @@ -1324,9 +1343,9 @@ JSON/同名 `.tmp`,并只在协议、PMU/owner 门禁、Restore 和 runtime #### 7.5.6 当前实现状态、历史证据与正式重采矩阵 -当前源码已经实现 schema-v3 的六类 PollBatch、精确 `call_count`、七项 summary -闭环、两条 ClockBaseline、scalar lane converter,以及 phase/lap/Kernel 共 cycle -边界关闭。2026-07-18 已用边界修复版 CCEC 完成 b1 与 b256 真机重测,当前证据为: +当前 schema-v4 源码完整沿用 schema-v3 建立的六类 PollBatch、精确 +`call_count`、七项 summary 闭环、两条 ClockBaseline 和 scalar lane converter。 +以下先保留 2026-07-18 schema-v3 边界修复版 CCEC b1/b256 的历史证据: | 样本 | winner 负载 | 总 `records` | 逻辑 `atomic_calls` | direct | 物理 Atomic | `batched_poll_calls` | PollBatch | ClockBaseline | dropped | 首末 Submit | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | @@ -1360,8 +1379,39 @@ outputs/pa_scheduler_swimlane_20260718_182725_4061524/ccec/merged_swimlane.json b1 是零 winner 负载的快速验收,b256 是开启 atomic 泳道的诊断运行;5.774295 ms 只证明观察构建保持目标量级,不是关闭 trace 的正式性能基线,也不能与下列历史样本 做单轮减法归因观察开销。 + +schema-v4 于 2026-07-19 用同一 atomic ABI 完成 CCEC b1、AscendC b1 与 +CCEC b256 真机验收。v4 新增 `OrchestrationReplay/FinalDrain` 父 span,并用 +`WinnerBuild/LoserReplay/AllocComplete` 替代旧 lap;Atomic 仍由原 direct 与 +PollBatch 规则产生。三轮均为 96 核、192 条 ClockBaseline、 +`dropped=0`,Atomic weighted summary 和 raw 行逐项闭合: + +| 样本 | 总 `records` | 逻辑 `atomic_calls` | 物理 Atomic | `batched_poll_calls` | PollBatch | 首末 Submit | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| CCEC b1 | 4,602 | 1,403 | 846 | 790 | 233 | 90.741 us | +| AscendC b1 | 4,597 | 1,338 | 841 | 725 | 228 | 86.037 us | +| CCEC b256 | 964,724 | 102,324 | 101,108 | 1,471 | 255 | 5,680.749 us | + +CCEC b256 还验证了每核 1,280 个 Submit、1,024 个 Kernel 全部落入 +EfDrain 或 FinalDrain:前者 1,011 个、后者 13 个,孤儿和越界均为 0; +Submit/EfDrain/Orchestration/FinalDrain/WorkerCompletion 排他闭合。最终分析器 +还逐条验证 exclusive child 的 task 身份,以及 Alloc loser=`Claim.end`、 +非 Alloc loser=`Register.end` 的零时长锚点。完整产物为: + +~~~text +outputs/pa_scheduler_swimlane_20260719_055449_334552/ccec/ +outputs/pa_scheduler_swimlane_20260719_060409_345364/ascendc/ +outputs/pa_scheduler_swimlane_20260719_060634_347455/ccec/ +~~~ + +最终源码和相同 v4 level-4 配置连续三轮 CCEC b256 为 +5.785939/5.503109/5.381844 ms,中位数 5.503109 ms,极差 +0.404095 ms;完整导出轮比上文 v3 导出轮低 1.62%,仍小于当前自身 +轮间波动。因此当前只记录为“未观察到明显回退”,不把这个单轮差值 +当成优化收益。 + 以下是 2026-07-18 的**历史 schema-v2、逐调用、边界修复前**证据,保留用于追溯, -不作为当前 schema-v3 的物理记录规模、逻辑调用数或边界闭合验收。历史 b1 +不作为当前 schema-v4 的物理记录规模、逻辑调用数或边界闭合验收。历史 b1 atomic-trace-only 上板中,全部协议断言 PASS,raw 共 4959 条、 `expected=4959`、`dropped=0`,其中逐条 Atomic 1395 条、ClockBaseline 192 条。 Claim 当时已按 schema-v2 闭合为 `won=5/lost=283/not_attempted=192`。当时 converter @@ -1401,8 +1451,10 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 | schema-v3 六类 PollBatch、flags、logical/physical 公式与拓扑的 converter 静态回归 | **19/19 PASS;不替代真机验收** | | schema-v3 b1 的七项 summary、逐核/全局公式与 dropped | **96/96 闭合,dropped=0** | | schema-v3 256 batch 的 direct/PollBatch raw→merged 与容量 | **已闭合,单核峰值 10,252/65,536** | -| 15-site schema 与六类 PollBatch allowlist | **两轮 flags 全合法;实际四类有事件、HeapGuard 两类为 0** | -| 当前版本 192 条 ClockBaseline 与 Kernel 边界 | **两轮均闭合;严格 overlap=0** | +| schema-v4 排他父区间、真实 Submit 尾动作与 atomic ABI | **CCEC/AscendC b1 均通过;旧 lap=0** | +| schema-v4 CCEC b256 raw→merged→exclusive report | **964,724 条,dropped=0,六组整数 cycle 闭合** | +| 15-site schema 与六类 PollBatch allowlist | **历史 schema-v3 b1/b256 两轮 flags 全合法;实际四类有事件、HeapGuard 两类为 0** | +| 192 条 ClockBaseline 与 Kernel 边界 | **历史 schema-v3 b1/b256 两轮均闭合;严格 overlap=0** | | `submit-all` 的 owner/gate/CNT0..8/total 闭环 | **b1、零 NOP 单次上板已通过** | | 256 batch `submit-all` 的 I-cache 与 pipe 分组分布 | **3 个 PMU-only 独立进程已采并完成 raw→summary 重算** | | real-compute 引擎 PMU 与 placement | **b8 count1/count2 精确倍增并逐 worker 闭合** | @@ -1410,9 +1462,10 @@ frontier helping 和 winner 分布会带来明显轮间波动;一次 5.209261 此前开发过程中的探索性输出不在这里引用为正式结论。最终按同一源码依次执行: 1. 三后端全量重建及 no-trace/no-PMU 语义回归; -2. CCEC schema-v3 atomic-trace-only 已按 b1、b256 顺序完成,六类 allowlist、flags、 - 七项 summary、逐核/全局三条公式、ClockBaseline、Kernel overlap 和 raw→merged - 均已检查;后续改动仍按同一门禁复测; +2. 历史 schema-v3 atomic-trace-only 已按 b1、b256 完成;当前 schema-v4 + 又按 CCEC b1、AscendC b1、CCEC b256 完成六类 allowlist、flags、七项 + summary、ClockBaseline、Kernel 包含、raw→merged 与排他报告闭合; + 后续改动仍按同一门禁复测; 3. CCEC PMU-only 的 `submit-all` 已完成 3 个独立进程 A/A;后续每轮仍使用 唯一 JSON 路径,并检查 96 核、owner bitmap/triplet、start/stop、25% 风险门槛和 Restore; 4. real-compute b8 count1/count2 已完成 Cube/Vector busy 精确倍增与 @@ -1835,8 +1888,9 @@ Claim/EfDrain/Materialize 一样可以用固定 `5*batches` 闭合。这个窗 ~~~ 成功采集后同目录自动生成 `submit_icache_report.html`。raw 继续作为权威证据, -HTML 只提供 AIC/AIV 汇总、逐核分布和 running phase lower/upper 的离线可视化, -不改变原始统计和可信门禁。 +HTML 只提供 ALL/AIC/AIV 汇总、逐核分布和 running phase lower/upper 的离线 +可视化;顶部单列完整 Submit(最早开始 → 最晚结束)的整体耗时,PMU +total/scalar 角色卡片只显示逐核 min/mean/max。HTML 不改变原始统计和可信门禁。 ~~~text build/ccec/submit-pmu/none/ @@ -1904,7 +1958,7 @@ request/miss 不可相加,也不能用 `phase - none` 宣称得到零扰动净 当协议、数值输出和 placement/engine 门禁全部通过时,运行中 shadow 的 单向负差只能描述为局部 PMU 分段误差,不能描述成 standalone 调度异常。 -schema-v4 JSON 保留 96 条 raw,并分别给出 ALL/AIC/AIV 的 authoritative +submit-pmu schema-v5 JSON 保留 96 条 raw,并分别给出 ALL/AIC/AIV 的 authoritative whole、shadow loss 以及 phase lower/upper。raw 中显式保存 `shadow_request_loss`、`shadow_miss_loss`、 `phase_icache_requests_upper_bound` 和 `phase_icache_misses_upper_bound`。 @@ -2006,17 +2060,19 @@ submit_icache_report.html # 自包含 HTML 加工件 | `materialize` | 4195.642 | 434706.750 | 20468.656 | 4.7086% | 445711.047 | 56744.156 | 12.7312% | | `register` | 4960.087 | 431990.906 | 26459.562 | 6.1250% | 438417.688 | 50043.422 | 11.4146% | -同一批 raw 中的完整 Submit PMU total/scalar busy 如下。这些是每核 raw event -的均值,不是微秒;`scalar/total` 仅是两个同窗口事件求和的描述性比值, -不得称为 scalar 利用率或局部 phase 耗时占比。 +同一批 raw 中的完整 Submit PMU total/scalar busy 如下。上表 `Submit/us` 是 +96 核完整 Submit 的整体耗时;下表 raw 列是每核 event 均值,二者不是同一 +统计量。等效时间列按 ALL 的 `1.649844 cycles/ns` 换算,只表示每核 +cycle-equivalent。`scalar/total` 仅是两个同窗口事件求和的描述性比值, +不得称为 scalar 利用率、局部 phase 耗时占比或 96 核墙钟。 -| phase | ALL total/core | ALL scalar/core | scalar/total | -| --- | ---: | ---: | ---: | -| `none` | 7,213,914.333 | 5,717,308.729 | 79.2539% | -| `claim` | 6,878,355.458 | 5,371,850.677 | 78.0979% | -| `efdrain` | 6,910,059.479 | 5,499,963.656 | 79.5936% | -| `materialize` | 6,577,155.385 | 5,119,585.177 | 77.8389% | -| `register` | 6,913,673.615 | 5,524,170.406 | 79.9021% | +| phase | ALL total/core raw | total 等效 us/core | ALL scalar/core raw | scalar 等效 us/core | scalar/total | +| --- | ---: | ---: | ---: | ---: | ---: | +| `none` | 7,213,914.333 | 4,372.483 | 5,717,308.729 | 3,465.363 | 79.2539% | +| `claim` | 6,878,355.458 | 4,169.094 | 5,371,850.677 | 3,255.975 | 78.0979% | +| `efdrain` | 6,910,059.479 | 4,188.311 | 5,499,963.656 | 3,333.626 | 79.5936% | +| `materialize` | 6,577,155.385 | 3,986.532 | 5,119,585.177 | 3,103.072 | 77.8389% | +| `register` | 6,913,673.615 | 4,190.501 | 5,524,170.406 | 3,348.299 | 79.9021% | 局部 running read-clear 结果为: @@ -2093,3 +2149,112 @@ outputs/submit_pmu_materialize_20260719_b256/{submit_icache_raw.json,submit_icac outputs/submit_pmu_register_20260719_b1/{submit_icache_raw.json,submit_icache_report.html} outputs/submit_pmu_register_20260719_b256/{submit_icache_raw.json,submit_icache_report.html} ~~~ + +#### 7.5.20 schema-v5 局部阶段时间取证 + +2026-07-19 在不增加新 phase、不改业务调用点的前提下, +`submit-pmu` 从 schema-v4 升到 schema-v5,给既有 +`claim|efdrain|materialize|register` 边界补充 SYS_CNT 时间累计。设备侧复用 +`WorkerResult` offset 744 的 64-bit 诊断槽,结构仍为 832B,没有改变 +worker stride 或相邻字段 offset。 + +时间边界严格定义为: + +~~~text +begin: shadow request/miss read-clear -> phase_begin = SYS_CNT +end: phase_end = SYS_CNT -> shadow request/miss read-clear + +phase_elapsed += phase_end - phase_begin +~~~ + +因此阶段时间不包含两侧顺序 `ld_dev`,但包含每次调用两次 +SYS_CNT 和相关边界 bookkeeping 对被观察 ELF 的扰动。没有加 DSB/ISB。 +当前 CANN `llvm-objdump` 对 `elf64-hiipu` 只能列符号而不能解码指令, +所以这里只声明已由源码顺序、device status 和真机逐核时间门禁闭环, +不冒充已完成 ISA 反汇编证明。 + +raw 对 96 个 worker 新增 `submit_elapsed_ticks`、`phase_elapsed_ticks`和 +`phase_time_valid`;host 和离线分析器都要求 running phase 满足 +`0 < phase_elapsed_ticks <= submit_elapsed_ticks`,`none` 则必须为 0。当前 v5 +required status mask 为 `0x7cf`;历史 v4 仍以 `0x3cf` 只读兼容,且明确 +标记阶段时间不可用,不从 PMU total 或 request/miss 反推。 + +ALL/AIC/AIV 的时间占比统一按同一 SYS_CNT 口径计算: + +~~~text +时间占比 = Σ本组 phase_elapsed_ticks / Σ本组 submit_elapsed_ticks +~~~ + +这是逐核累计 core-time 构成,不是局部阶段占全局约 5 ms 墙钟的时间片; +不能把分子改除 `submit_span_us`,也不能平均 96 个逐核百分比。 +HTML 已把 ALL/AIC/AIV 的时间、request、miss 三类占比移到页面最前; +时间是直接观察单值,request/miss 继续显示 lower..upper。 + +同一 A5、`real-compute/6,28,4,1`、b256 下的首轮 v5 取证如下。五个 +独立进程均通过 96/96 记录、phase shape、`phase<=submit`、owner Restore、 +真计算输出与 placement/engine 门禁: + +| phase | Submit span | ALL 时间/core | ALL 占比 | AIC 占比 | AIV 占比 | +| --- | ---: | ---: | ---: | ---: | ---: | +| `none` | 3.711584 ms | 不适用 | 不适用 | 不适用 | 不适用 | +| `claim` | 4.401747 ms | 470.641 us | 12.0845% | 7.7096% | 14.2972% | +| `efdrain` | 3.592376 ms | 536.279 us | 15.5068% | 20.2974% | 13.1974% | +| `materialize` | 6.770266 ms | 1,026.859 us | 16.7186% | 15.4860% | 17.3556% | +| `register` | 4.086936 ms | 158.728 us | 4.3089% | 3.6256% | 4.6568% | + +`ALL 时间/core` 等于本轮 `Σphase_elapsed_ticks / 96`,不是全局墙钟。 +Materialize 诊断 ELF 的 Submit span 增至 6.77 ms,直接说明高频边界观察本身 +会改变取指和多核争用。所以表中比例只解释同一行的诊断 ELF;四行 +不能相加,也不能与 `none` 相减得到无扰动阶段净时间。 + +本机权威 raw 和页面最前已带时间占比的 HTML 位于: + +~~~text +outputs/submit_pmu_phase_time_v5_20260719/none_b256/ +outputs/submit_pmu_phase_time_v5_20260719/claim_b256/ +outputs/submit_pmu_phase_time_v5_20260719/efdrain_b256/ +outputs/submit_pmu_phase_time_v5_20260719/materialize_b256/ +outputs/submit_pmu_phase_time_v5_20260719/register_b256/ +~~~ + +#### 7.5.21 排他泳道边界收敛对 atomic 口径的影响 + +2026-07-19 的后续改动只收敛普通 phase 边界和离线 residual, +没有增删 Atomic site,也没有改变 direct/PollBatch、`return_ready`/ +`source_issue` 或 logical/physical weighted 计数公式。 + +standalone loser 没有真实 Replay 计算,因此已删除每个 loser 一条的 +`LoserReplay` 过程态 phase 记录。这不是 atomic 消减:相关 Claim +FetchMax、fanin/completion/frontier 等 atomic 仍按实际调用采集;只是不再 +用一条零工作 phase 伪装 loser 尾动作。最后一个真实 child 到 +`SubmitEnd` 的时间在 Perfetto 中由离线 `submit_tail_gap` 展示,在排他 +报告中汇总为 `submit_tail_residual`;它不被命名为 loser 业务阶段。 + +schema-v4 merged 中 direct atomic 名仍编码 +`boundary/site/op/task_id`,PollBatch 名仍编码 `site/op/call_count`; +为控制数百 MiB 产物,duration 事件不再逐条复制 raw 的 `args/cat`。 +精确 flags、cycle、retry、value-zero 和 weighted 计数仍以同目录 +`l2_swimlane_records.json` 为准,不因 merged 瘦身丢失。 + +当前最新 CCEC A5 b1 为: + +~~~text +outputs/pa_scheduler_swimlane_20260719_110756_584549/ccec/ +~~~ + +该轮 4,118 条 raw 事件、`dropped=0`,全部 atomic 闭合与六类排他 +时间闭合均通过;当前 converter 生成的 merged 为 428,455 bytes。 +后续 atomic/边界迭代默认只跑 A5 b1;b256 仅用于阶段性规模/容量 +收口或明确要求的长负载结论。 + +按明确要求完成的当前生产者 CCEC b256 规模复核位于: + +~~~text +outputs/pa_scheduler_swimlane_20260719_114815_617346/ccec/ +~~~ + +该轮 839,526 条 raw、97,510 条物理 Atomic、`dropped=0`,全局 Submit +为 5,360.061 us;全部 atomic 与排他闭合通过。Submit 内部阶段间 residual +为 0,尾部 residual 为 41,008,786/433,383,588 cycle(9.4625%),Submit +间 residual 为 67,065,321/500,448,909 cycle(13.4010%)。这些数值是 +相同观测口径下的归因基线,不把边界重分类宣称为 atomic 或调度性能收益。 diff --git "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" index 9c7fea80bb..313ce259d3 100644 --- "a/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/tests/atomic_probe/pa_scheduler/PA\350\260\203\345\272\246\345\231\250\347\213\254\347\253\213\345\244\215\347\216\260\344\270\216\346\263\263\351\201\223\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -216,7 +216,7 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" | 构建 | 后端 | 内容 | 构建命令 | 产物目录 | | --- | --- | --- | --- | --- | -| `swimlane` | CCEC/AscendC/CPU | 普通阶段与 schema-v3 atomic(direct + PollBatch)合并采集;不配置 PMU | `./run.sh build ccec` 或 `./run.sh build all` | CCEC 为 `build/ccec/` | +| `swimlane` | CCEC/AscendC/CPU | schema-v4 普通阶段、业务父区间、真实 Submit 尾动作与 atomic(direct + PollBatch)合并采集;不配置 PMU | `./run.sh build ccec` 或 `./run.sh build all` | CCEC 为 `build/ccec/` | | `submit-pmu` | 仅 CCEC | 每核完整 Submit PMU,并在编译期可选一个局部阶段;当前有 `none|claim|efdrain|materialize|register` | `./run.sh build-submit-pmu ccec ` | `build/ccec/submit-pmu//` | `./run.sh build all` 只构建三后端的 `swimlane` 产物;`submit-pmu` @@ -233,7 +233,7 @@ export CXX="$GCC15_ROOT/usr/bin/g++-15" | `build` | 构建指定后端 | 否 | | `smoke` | 固定 b1/r1/`scalar-nop=0` 的快速语义回归 | 否,只做内存记录校验 | | `run` | 自行控制 batch、run、winner 负载和诊断参数 | 仅显式传入 `--swimlane-json` 时生成 raw | -| `swimlane` | 单轮运行并自动生成 raw 和 Perfetto merged JSON | 是 | +| `swimlane` | 单轮运行并自动生成 raw、Perfetto merged JSON 和排他闭合分析 JSON | 是 | | `build-submit-pmu` | 构建指定 `none|claim|efdrain|materialize|register` 的 CCEC PMU-only ELF | 否 | | `submit-pmu` | 单轮采集完整 Submit PMU,可选导出 JSON | 否,与泳道隔离 | @@ -290,9 +290,6 @@ semantic_status=PASS postprocess_status=PASS 生成可直接载入 Perfetto 的泳道文件时使用独立的 `swimlane` action: ```bash -./run.sh swimlane ccec \ - --device 0 --batches 256 --analyze-swimlane - ./run.sh swimlane ccec \ --device 0 --batches 1 --winner-workload real-compute @@ -301,8 +298,9 @@ semantic_status=PASS postprocess_status=PASS --real-compute-count 1 ``` -`swimlane` 是唯一的正式泳道构建口径,固定合并普通阶段与 schema-v3 atomic -记录(direct Atomic 加 PollBatch); +`swimlane` 是唯一的正式泳道构建口径,固定合并 schema-v4 +普通阶段、业务父区间、真实 Submit 尾动作与 atomic 记录(direct +Atomic 加 PollBatch); 无需再显式传 `--trace-atomics`。该 action 会管理 `--runs 1` 和输出路径,因此不要再传 `--runs`、`--swimlane-json` 或 `--no-swimlane`。转换器默认使用 `python3`;如需 固定到用户自己的 Python,可在命令前设置: @@ -311,15 +309,37 @@ semantic_status=PASS postprocess_status=PASS export PYTHON=/path/to/venv/bin/python ``` -转换器只使用 Python 标准库,不需要 PyTorch,也不需要安装 simpler Python 包。 +转换器和排他分析器都只使用 Python 标准库,不需要 PyTorch, +也不需要安装 simpler Python 包。`--analyze-swimlane` 仍只控制 runner +终端中的传统分组文字统计;无论是否传它,`swimlane` action 都会生成 +排他闭合报告。 + +修改 C++ 头文件或 kernel 后必须先执行 `./run.sh build ccec`, +`swimlane` action 只消费已有构建件,不会隐式重编译。日常边界迭代默认只跑 +A5 b1;b256 只用于阶段性规模/容量收口或明确指定的长负载结论。 该 action 固定执行一轮,产物全部位于本目录的 `outputs/pa_scheduler_swimlane__//`。选择 `all` 时, 同一 output root 下会按顺序建立 `ccec/`、`ascendc/` 和 `cpu/` 三个子目录: -- `l2_swimlane_records.json`:与真实 PA 相同的十列 `fdwic_events` 原始格式; -- `merged_swimlane.json`:Chrome Trace Event 格式,拖入 - 即可查看泳道。 +- `l2_swimlane_records.json`:与真实 PA 相同的十列 `fdwic_events` + 权威原始件,所有字段复算以它为准; +- `merged_swimlane.json`:只用于 Perfetto 可视化。schema-v4 的 duration + 事件只保留 `ph/name/pid/tid/ts/dur` 六个必需字段,不再逐事件 + 复制 raw 中的 `args/cat`;拖入 即可查看; +- `swimlane_exclusive_analysis.json`:以原始整数 cycle 校验并汇总 + Submit、EfDrain、OrchestrationReplay、FinalDrain 和 WorkerCompletion + 排他闭合关系,并将 Submit 内和 Submit 间的 residual 按相邻边界小表 + 聚合。完整父子层级、排他角色和数值统计以该报告为准,不再逐事件 + 塞入 merged。 + +schema-v4 禁止产生历史 `Alloc/Build/Replay` lap 与未使用的 +`DrainWon`,只用 `AllocComplete/WinnerBuild` 表达真实 Submit winner +尾动作。loser 没有可单独计时的业务动作,不生成 `LoserReplay` +伪阶段;其未归因尾段只由离线 residual 展示。每个 Kernel 还必须唯一归入 EfDrain、WinnerBuild、AllocComplete 或 +FinalDrain;孤儿、越界或多重归属都会使排他分析失败。 +除 Kernel 可以执行前序 task 外,所有 Submit 前端和尾动作的 `task_id` 必须与 +包含它的 Submit 一致。 runner 结束时会打印准确目录: @@ -341,14 +361,16 @@ runner 结束时会打印准确目录: 2. 将 `merged_swimlane.json` 拖入页面,不要拖原始的 `l2_swimlane_records.json`; 3. 每个 `block0` 至 `block31` 是一个物理 1AIC+2AIV block; -4. `AIC`、`AIV0`、`AIV1` 轨展示 Submit、Claim、EfDrain、Replay、RingBp 等 - runtime 阶段,带 `·kernel` 的轨展示 QK、SF、PV、UP; +4. `AIC`、`AIV0`、`AIV1` 轨展示 OrchestrationReplay、Submit、Claim、 + EfDrain、WinnerBuild、AllocComplete、FinalDrain、Residual 和 RingBp + 等 runtime 阶段,带 `·kernel` 的轨展示 QK、SF、PV、UP; 5. direct Atomic、PollBatch 及 ClockBaseline 固定画在对应 - `AIC/AIV` scalar lane;direct 名称与 category 显式区分 `return_ready` 和 + `AIC/AIV` scalar lane;direct 名称显式区分 `return_ready` 和 `source_issue`,PollBatch 单独标识逻辑轮询 episode;不生成带 `·atomic` 的 伪并行子轨; -6. 普通事件可查看 `task_id`、`func_id`、`core`、`mc` 和 `aux`;Atomic 的 - 字段和解读边界见 5.6 节。 +6. merged 事件名保留 phase/task,atomic 名还保留 + `site/op/boundary/call_count`;需要 `func_id/core/flags/aux` 等精确字段时 + 查同目录 raw,Atomic 的解读边界见 5.6 节。 WaitForSlot 和 HeapGuard 没有可伪造的逐事件起止时间,因此不单独生成 Perfetto 事件;实际发生等待时,泳道中会出现 RingBp 事件。现行 CCEC 局部 PMU @@ -367,6 +389,10 @@ WaitForSlot 和 HeapGuard 没有可伪造的逐事件起止时间,因此不单 python3 ./swimlane_converter.py \ ./outputs//ccec/l2_swimlane_records.json \ -o ./outputs//ccec/merged_swimlane.json + +python3 ./swimlane_exclusive_analyzer.py \ + ./outputs//ccec/l2_swimlane_records.json \ + -o ./outputs//ccec/swimlane_exclusive_analysis.json ``` 若只需要原始记录,可通过通用 `run` action 显式指定文件;导出为避免多轮覆盖 @@ -378,8 +404,12 @@ mkdir -p ./outputs/manual --swimlane-json ./outputs/manual/l2_swimlane_records.json ``` -手工 `--swimlane-json` 只生成 raw,不会自动生成 merged;需要随后调用上面的 -`swimlane_converter.py`。该参数强制要求 `--runs 1`,避免多轮静默覆盖同一文件。 +手工 `--swimlane-json` 只生成 raw,不会自动生成 merged 或排他报告; +需要随后调用上面两个脚本。该参数强制要求 `--runs 1`,避免多轮 +静默覆盖同一文件。runner、converter 和 analyzer 都只在单件写完后 +原子替换各自目标;这是逐文件发布而不是三件整体事务:runner 失败时 +不启动后处理,converter 失败时保留已完成的 raw,analyzer 失败时保留 +已完成的 raw 与 merged,任何阶段都不会把半截 JSON 冒充完整产物。 ### 5.5 CPU 回归、参数与测量口径 @@ -453,7 +483,7 @@ frontier_initial=... frontier_flag=... frontier_ready_fetch_max=... frontier_ter 结果区,不为诊断新增共享 atomic。它们仍会增加少量 scalar 指令,因此优化 A/B 必须使用相同的计数布局;不能把启用分类后的绝对时间直接与旧二进制比较。 -### 5.6 合并泳道中的 atomic schema-v3 语义边界 +### 5.6 合并泳道中的 atomic schema-v4 语义边界 正式 `swimlane` action 固定记录 standalone 调度器中的 atomic **逻辑调用**, 不只记录 winner 或慢样本。普通 direct Atomic 仍是一条源码调用对应一条物理记录; @@ -462,15 +492,15 @@ frontier_initial=... frontier_flag=... frontier_ready_fetch_max=... frontier_ter ```bash ./run.sh swimlane ccec \ - --device 0 --batches 256 \ + --device 0 --batches 1 \ --analyze-swimlane ``` 只有使用低层 `run` action 手工导出 raw 时,才需要显式传入 `--trace-atomics`;该兼容入口不代表存在第二种 atomic-swimlane 构建。 -schema-v3 raw 的 `metadata.trace_schema_version` 必须为 3,且 -`metadata.l2_swimlane_level` 必须为 4。转换后 direct Atomic、PollBatch 和 +schema-v4 raw 的 `metadata.trace_schema_version` 必须为 4,且顶层 +`l2_swimlane_level` 必须为 4。转换后 direct Atomic、PollBatch 和 ClockBaseline 都放在对应 AIC/AIV 的原 scalar lane;它们属于 scalar 调度观察,不再伪装成与 scalar 并行的独立子轨。Kernel 仍放在独立计算单元轨。 direct Atomic 事件名显式区分两种边界: @@ -480,8 +510,8 @@ atomic.return_ready..# atomic.source_issue..# ``` -category 也分别为 `atomic.return_ready` 与 `atomic.source_issue`,可在 -Perfetto 中直接过滤,无需逐条点开 args 才能区分。 +边界已编码在事件名中,可在 Perfetto 中按名称搜索或过滤, +无需为每条事件保留 category/args。 PollBatch 转换为: @@ -489,8 +519,8 @@ PollBatch 转换为: atomic.poll_batch..load× ``` -其 category 为 `atomic.poll_batch`;`args.call_count` 是实际执行的源码 wrapper -调用次数,不是采样或估算值。 +名称中的 `call_count` 是实际执行的源码 wrapper 调用次数, +不是采样或估算值。 当前固定 schema 共有 15 个调用点: @@ -545,13 +575,11 @@ PollBatch;同一 site 在等待区外的一次性或 opportunistic 读取仍 `1..0xFFFFFF`。达到上限时先落盘,再从 1 开启下一条 batch,不允许饱和后 丢失调用数;`task_id=-1`、`function_id=-1`。 -merged direct 事件的 `args` 显式导出 `call_count=1`、`site/site_id`、 -`op/op_id`、原始整数 `cycles`、`result_used`、`return_ready_observed`、 -`completion_boundary`,以及适用时的 `value_zero` 或 `retries`。PollBatch 则导出 -精确 `call_count`、`poll_window_cycles`、 -`batch_semantics=observation_load_calls`、 -`duration_semantics=logical_poll_episode_envelope_not_single_atomic_latency` 和 -`may_contain_interleaved_direct_atomics=true`。 +schema-v4 merged 只保留可视化必需的六个 duration 字段。direct 的 +`site/op/boundary/task_id` 和 PollBatch 的 `site/op/call_count` 均在名称中; +`site_id/op_id`、原始整数 cycle、flags、retry 和 value-zero 等精确值从同目录 +raw 十列记录复算。不把这些重复复制到 merged,是为了控制数百万 +事件时的文件大小和观察工具内存。 边界必须按源码调用点语义解读: @@ -576,8 +604,8 @@ PollBatch 的 `duration`/`poll_window_cycles` 是从该 site 在显式等待区 - 显式等待区退出时关闭匹配的 PollBatch; - `TraceTimestamp` 在写 phase begin/end 前关闭全部活跃 batch; -- `ResetTraceLap` 在推进 lap 起点前关闭,`WriteTraceLap` 在写 lap 前以同一结束 - cycle 关闭; +- schema-v4 producer 不再生成旧 `Alloc/Build/Replay` lap;历史 helper + 仍有自身关闭规则,但不得出现在当前 raw 中; - Kernel begin/end 也通过 `TraceTimestamp`,所以 PollBatch 不能跨入或跨出 Kernel; - 最终 flush 只作防御性兜底,不能替代上述语义边界。 @@ -590,7 +618,7 @@ PollBatch 的 `duration`/`poll_window_cycles` 是从该 site 在显式等待区 `[TRACE_ATOMIC_POLL]` 单独输出 PollBatch 的 episode 数、精确逻辑调用数和等待包络 分布,二者不会混算。 -schema-v3 必须按逻辑调用与物理记录两套口径闭合。设 direct 物理记录数为 +schema-v4 level-4 raw 必须按逻辑调用与物理记录两套口径闭合。设 direct 物理记录数为 `direct_atomic_records`,则逐核和全局都必须满足: ```text @@ -612,9 +640,67 @@ raw 行重算和 converter 重算逐项一致;`dropped_records` 必须为 0, 也不能用它们计算 atomic 对 golden 的绝对占比。每核分区固定容纳 65,536 条记录; 任何容量溢出或闭合失败都使该轮 trace 无效,不能截断后继续分析。 -#### schema-v3 边界修复版 A5 验收 +#### schema-v4 排他闭合版 A5 验收 -2026-07-18 已用边界修复后的同一版 standalone CCEC 依次完成 b1 与 b256 真机重测: +2026-07-19 早期曾完成 CPU/CCEC/AscendC b1 语义门禁,但该过程态 +raw 仍含现已删除的 `LoserReplay`,当前 converter 会拒绝它们。下面 +只列当前无 loser 记录的 CCEC 证据。删除无业务实体的 loser 标记后, +b256 规模样本位于: + +```text +outputs/pa_scheduler_swimlane_20260719_103435_542368/ccec/l2_swimlane_records.json +outputs/pa_scheduler_swimlane_20260719_103435_542368/ccec/merged_swimlane_thin.json +outputs/pa_scheduler_swimlane_20260719_103435_542368/ccec/swimlane_exclusive_analysis.json +``` + +该轮使用 `real-compute/6,28,4,1`,每核 1,280 个 Submit,raw +845,813 条事件、`dropped=0`,首末 Submit 为 5,326.055 us,六类整数 +cycle 闭合全部精确相等。raw 为 56,212,672 bytes;旧格式 merged 为 +248,767,986 bytes,同一 raw 经当前六字段 converter 生成的 +`merged_swimlane_thin.json` 为 138,349,686 bytes,减少 44.4%。这是离线 +可视化瘦身,不改变该轮设备采集。同目录的 +`merged_swimlane.json` 是旧胖版,查看该规模样本时应打开上述 thin 文件。 +该轮早于最终相邻边界复用,只作规模/容量门禁,不代表当前 residual +构成。 + +边界收敛后的最新日常验证只跑 CCEC b1: + +```text +outputs/pa_scheduler_swimlane_20260719_110756_584549/ccec/ +``` + +该轮 raw 4,118 条事件、`dropped=0`,全局 Submit 89.313 us,merged +428,455 bytes,六类闭合全部通过。Submit 内所有 child-to-child +gap 已为零,即 `submit_internal_residual=0`;最后一个真实 child 到 +`SubmitEnd` 的 `submit_tail_residual` 为 184,788/2,241,892 cycle +(8.2425%)。Submit 间 residual 为 155,679/2,397,571 cycle(首末 +Submit 包络的 6.493%)。Perfetto 分别用 `submit_tail_gap` 与 +`between_submit_residual` 展示两类补集,不新增 raw 记录或字段。 + +按明确要求完成的当前 CCEC b256 规模复核位于: + +```text +outputs/pa_scheduler_swimlane_20260719_114815_617346/ccec/ +``` + +该轮使用 `real-compute/6,28,4,1`,全局 Submit 为 5,360.061 us;raw +839,526 条、`dropped=0`,merged 1,085,191 条事件。raw/merged 分别为 +55,791,947/88,775,668 bytes,全部语义断言与整数闭合通过。 +Submit aggregate core-work 为 433,383,588 cycle:内部 residual 为 0, +尾部 residual 为 41,008,786 cycle(9.4625%);逐核首末 Submit 包络为 +500,448,909 cycle,Submit 间 residual 为 67,065,321 cycle(13.4010%)。 +122,880 条 `submit_tail_gap` 与 122,784 条 `between_submit_residual` +分别精确对应 `96*1280` 和 `96*(1280-1)`,没有增加设备事件。 + +后续边界迭代默认只跑 A5 b1;b256 仅在阶段性规模/容量收口或 +明确要求时重跑。历史 level-4 b256 多轮波动证据仍保留在本文后续 +章节,不将 b1 单轮时间宣称为性能改善。 + +#### 历史:schema-v3 边界修复版 A5 验收 + +2026-07-18 已用边界修复后的同一版 standalone CCEC 依次完成 b1 与 b256 +真机重测。下列是升级到当前 schema-v4 之前的历史样本,不应修改其 +metadata 或用当前父区间口径强行解释: ```text outputs/pa_scheduler_swimlane_20260718_182649_4060527/ccec/ @@ -666,7 +752,7 @@ b256 是开启 atomic 泳道的诊断运行;上表 Submit 只能证明当前 记录了 963,368 条物理记录,其中逐条 Atomic 99,944 条、ClockBaseline 192 条, 逐核峰值 10,308/65,536,且当轮 `dropped=0`。这些数字来自引入 PollBatch 与上述 phase/lap/Kernel 边界修复之前的 schema-v2 逐调用模型,只能用于追溯旧版观察结果; -不能拿 99,944 当作当前 schema-v3 的物理容量、逻辑调用数或闭合证据。 +不能拿 99,944 当作当前 schema-v4 的物理容量、逻辑调用数或闭合证据。 ### 5.7 两类正式构建与 CCEC Submit PMU @@ -722,7 +808,7 @@ export PYTHON=/home/q00473782/.venv/bin/python OUT="./outputs/submit_pmu_none_$(date -u +%Y%m%dT%H%M%SZ)" mkdir -p "$OUT" ./run.sh submit-pmu ccec none \ - --device 0 --batches 256 \ + --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT/submit_icache_raw.json" ``` @@ -735,11 +821,24 @@ submit_icache_report.html # 浏览器直接打开的离线图表和汇总 ``` HTML 中包含 AIC/AIV 的每核 request、miss、miss rate、p95、96 核散点和 -局部 phase 的 lower/upper 区间,以及局部 request/miss 占同一 ELF 完整 -Submit primary 的比例区间。报告也展示 ALL/AIC/AIV 的逐核 PMU `total_cycles` -与 `scalar_busy` 的 mean/median/p95、scalar/total 比例和逐核散点。报告将 +局部 phase 的 lower/upper 区间。schema-v5 的 running phase 还在页面 +最前面按 ALL/AIC/AIV 并列阶段时间、request 和 miss 占比;阶段时间 +是 `Σphase_elapsed_ticks / Σsubmit_elapsed_ticks`,request/miss 是占同一 +ELF PMU whole-gate primary 的比例区间。两者分母边界不同,不是同一 +精确分区。`none` 显示“不适用”,历史 +schema-v4 因没有阶段时间 raw 字段而显示“不可用”。报告也展示 +ALL/AIC/AIV 的逐核 PMU `total_cycles` +与 `scalar_busy`:三个响应式角色卡只选 mean 作为典型值,并补充逐核 +min/max 显示核间范围;PMU total 与 scalar busy 的极值分别独立计算,不保证 +来自同一个物理核。顶部“完整 Submit(最早开始 → 最晚结束)”是 96 核整体 +墙钟范围,不能与逐核 PMU mean 混为一个统计量。报告还展示 scalar/total 比例 +和逐核散点,并同时保留 raw cycle 与本机校准后的每核等效时间;宽表只在表格 +内部横向滚动。受控 cold/warm 同窗实测 +`1,817,457 PMU cycles / 1,101,593 ns = 1.649844 cycles/ns`;AIC/AIV +分别使用 `1.650062/1.649731 cycles/ns`,换算式为 +`time_us = cycles / cycles_per_ns / 1000`。报告将 `total_cycles-scalar_busy` 明确标为“非 Scalar-busy 残余”。`total_cycles` -是每个物理子核在 Submit gate 内的 64-bit PMU raw total,96 核求和是 core-work, +是每个物理子核在 PMU whole gate 内的 64-bit raw total,96 核求和是 core-work, 不是约 5 ms 的 Submit 墙钟;`scalar_busy` 是 CNT2 的 `scalar_instr_busy(0x001)`。依赖返回值的 atomic 等待可以落入 scalar busy, 而 I-cache refill 的额外周期可能主要只增加 total,但 @@ -756,6 +855,23 @@ I-cache stall**:差值还混有同步等待、Cube/Vector/MTE 等 engine 等 报告只复用 `pmu_sidecar_analyzer.py` 已校验的统计口径;生成失败不会删除已经发布 的 raw,但本次 action 会返回非零。 +这里的 PMU whole gate 从 orchestration 初始化/首次构参前开始,到末次 +Submit 返回后停止,包含 Submit 间构参和 `AcceptTaskOutputs()`,排除 +FinalDrain。`submit_elapsed_ticks` 是每核首末 Submit 时间;顶部 +`submit_span_us` 则是 96 核共同墙钟范围。三者不能混用,详细定义见 +`../icache_miss_usage_guide.md`。 + +当前边界联动版已对 `none|claim|efdrain|materialize|register` 五个独立 +ELF 完成 A5 b1 门禁,五轮均为 96/96 有效记录并通过语义、真计算、 +phase call/time、primary/shadow 和 owner Restore。产物位于: + +```text +outputs/submit_pmu_boundary_sync_b1_20260719/{none,claim,efdrain,materialize,register}/ +``` + +该 b1 只作源码边界与工具闭合证据,不用不同 phase ELF 的单轮时间差 +宣称性能改善。 + 两类新增局部 phase 可按与 `none` 相同的参数分别运行;输出文件名应体现 phase, 避免误把不同 ELF 的结果放进同一组: @@ -764,7 +880,7 @@ for phase in materialize register; do OUT="./outputs/submit_pmu_${phase}_$(date -u +%Y%m%dT%H%M%SZ)" mkdir -p "$OUT" ./run.sh submit-pmu ccec "$phase" \ - --device 0 --batches 256 \ + --device 0 --batches 1 \ --winner-workload real-compute --real-compute-counts 6,28,4,1 \ --pmu-json "$OUT/submit_icache_raw.json" done @@ -774,7 +890,10 @@ done 不要重复传入这三项,也不能传入 `--profile-phases`、 `--trace-atomics`、`--analyze-swimlane` 或 `--swimlane-json`。 -完整 Submit 的权威 I-cache 主计数是从不在局部边界读取的 +后续 submit-pmu 构建、门禁和边界迭代默认只跑 A5 b1;b256 只用于 +阶段性规模/容量收口或明确要求的长负载结论。 + +Submit-all PMU 整窗的权威 I-cache 主计数是从不在局部边界读取的 `CNT6=request` 和 `CNT7=miss`。A5 b1 实测已反证 `CNT9=0x35` 可作有效计数槽:它始终为 0。因此正式 `submit-pmu` 用 `CNT8=0x34` 作 shadow request、`CNT5=0x35` 作 shadow miss,`CNT9` @@ -782,7 +901,10 @@ done `swimlane` 构建。 shadow 计数器是 read-to-clear:选中的局部 phase 在 begin/end 切分片段,stop 时 -再加 tail,从而软件重建完整 Submit shadow whole。`none` 没有运行中读取,必须 +再加 tail,从而软件重建 PMU whole-gate shadow whole。schema-v5 同时在 +begin read-clear 之后取阶段起点,在 end read-clear 之前先取终点;因此 +阶段时间不包含两侧 `ld_dev`,但包含每次调用两次 SYS_CNT 的观察 +扰动。`none` 没有运行中读取,必须 在每个物理子核精确满足: ```text @@ -814,7 +936,12 @@ Alloc/QK/SF/PV/UP 五个 task,每次 Submit 都恰好执行一次 Claim、开 Materialize 和 Register 边界。`efdrain` 插点只允许包围 Submit 开头的专属调用, 不能插入复用的 `DrainReady()` 函数体;`materialize` 必须先保存真实返回值再关闭 边界,保证失败出口也闭合;`register` 的 Alloc 与非 Alloc 两个源码调用点互斥, -不能误算成每次 Submit 两次。当前 Case1 中真实 TensorMap insert 工作主要发生在 +不能误算成每次 Submit 两次。每个 running phase 还要求每核 +`phase_elapsed_ticks > 0`且不超过同核从首个 `submit_begin` 计时点 +到末个 `submit_end` 计时点的 `submit_elapsed_ticks`;前者位于首个 +`BeginSubmit()` 上下文初始化之后,后者位于末个 Submit 返回之前。 +`none` 的 phase elapsed 必须精确为 0。 +当前 Case1 中真实 TensorMap insert 工作主要发生在 UP 的输出注册,其他 task 的 Register 可能很短或没有 insert;因此该 phase 的 固定调用数只证明边界覆盖完整,不能解释为五类 task 拥有等量注册工作。 @@ -830,7 +957,16 @@ placement/engine 门禁都通过时,running shadow 的负差属于观测边界 JSON 保留 96 条 raw record,并按 ALL/AIC/AIV 输出 authoritative whole、 shadow loss 和 phase lower/upper。raw 中包含 `shadow_request_loss`、 `shadow_miss_loss`、`phase_icache_requests_upper_bound` 与 -`phase_icache_misses_upper_bound`;host 还分别报告 exact/bounded 核数。 +`phase_icache_misses_upper_bound`;schema-v5 还包含逐核 +`submit_elapsed_ticks`、`phase_elapsed_ticks`和 `phase_time_valid`。host 还分别 +报告 exact/bounded 核数。时间占比必须在 ALL/AIC/AIV 各自范围内按 + +```text +Σphase_elapsed_ticks / Σsubmit_elapsed_ticks +``` + +计算;分子、分母都是 1 ns/tick 的逐核 SYS_CNT core-time,不能改用 +96 核整体 `submit_span_us`,也不能平均 96 个逐核百分比。 完整 Submit 的组内 miss rate 才按 `sum(misses)/sum(requests)` 计算,不平均 逐核百分比;局部 lower miss/lower request 之比只是 observed read-clear ratio,不是实际 miss rate 的数学下界。更完整的 I-cache 采集、分析、估算与排错见 @@ -990,7 +1126,9 @@ JSON 包含: `sum(icache_misses) / sum(icache_requests)` 计算,不平均逐核百分比。AIC 与 AIV 核数不同,比较每核强度时应看 mean/median 或 miss rate,不能直接比较两组 sum。 `total_cycles` 是 PMU total 的原始值;96 核求和是 core-work,不是 Submit 墙钟 -时间,在没有额外核实其时钟/事件语义前不应直接换算成微秒。CNT0..CNT8 是 +时间。本机现已通过 PMU/SYS_CNT 同窗校准核实其频率,HTML 可按 +ALL/AIC/AIV 的 `1.649844/1.650062/1.649731 cycles/ns` 显示每核 +cycle-equivalent;该换算仍不能把 96 核 core-work 冒充墙钟。CNT0..CNT8 是 32 bit,total 是 64 bit。正式门禁要求本轮最大可编程 counter 小于 `UINT32_MAX/4` (25% 高水位),这只是缩短窗口后采用的保守风险阈值;最终 32-bit 值无法证明 计数器没有恰好回卷一圈或多圈,因此通过该门禁也不能声称“已证明无回卷”。 @@ -1073,7 +1211,7 @@ tests/atomic_probe/pa_scheduler/outputs/pmu_validation/ #### 历史:校验并聚合多轮 PMU sidecar -`pmu_sidecar_analyzer.py` 只读消费当前 schema-v3 JSON。它不信任单轮 host 已写好的 +`pmu_sidecar_analyzer.py` 在该历史流程中只读消费当时的 schema-v3 JSON。它不信任单轮 host 已写好的 summary,而是从每份文件的 worker raw 记录重新计算 ALL/AIC/AIV 的 `sum/mean/median/p95/max` 与 `sum(miss)/sum(request)`;同时检查 accepted、 96 核 start/stop、物理核唯一性、owner membership、角色、counter 门槛和 Restore。 @@ -1300,9 +1438,15 @@ winner workload 配置 cache line,生产 DistGlobal/DistCore 关键偏移保 1.313 GiB,real-compute+trace 约 1.325 GiB,host 侧也需分配相近内存。 CPU 后端只在 host 侧分配相同 workspace,不存在 device 内存口径。 `smoke` 不缩小 State;只有 `--no-swimlane` 能省去泳道缓冲区。 -256 batch 的正常采集约有 86 万条事件;原始 JSON 和 merged JSON 都可能达到 -数十至数百 MiB。writer 与 converter 都使用临时文件后原子替换,失败时不会把 -半截文件冒充完整产物。 +256 batch 的历史 phase-only 采集约有 86.3 万条事件。删除 +`LoserReplay` 过程态后,当前 schema-v4 level-4 b256 规模门禁为 +845,813 条 raw 事件、56,212,672 bytes;同一 raw 使用旧 merged +字段布局为 248,767,986 bytes,使用当前六字段 duration 布局为 +138,349,686 bytes,减少 44.4%。按 96 核聚合而不复制每个 gap +属性的排他报告约 117 KiB。文件瘦身不改变 raw 记录写入数, +也不改变固定 trace buffer 分配;三者是独立口径。 +runner、converter 与 analyzer 都使用临时文件后原子替换自己的目标,失败时 +不会把半截文件冒充完整产物。 脱离 simpler 时必须复制整个目录,因为三个后端共用 `common/`: @@ -1314,6 +1458,6 @@ cd /tmp/pa_scheduler ``` CCEC/AscendC 只需再 source CANN 环境。本目录的构建脚本不会搜索 Git 根目录, -也不会引用 `simpler/src`、`simpler/examples` 或其他仓内文件。泳道转换只需 -Python 3 标准库;复制后的 `./run.sh swimlane ...` 仍使用当前目录内的 -`swimlane_converter.py`。 +也不会引用 `simpler/src`、`simpler/examples` 或其他仓内文件。泳道转换与排他 +分析都只需 Python 3 标准库;复制后的 `./run.sh swimlane ...` 仍使用当前目录内的 +`swimlane_converter.py` 和 `swimlane_exclusive_analyzer.py`。 diff --git a/tests/atomic_probe/pa_scheduler/ccec/host.cpp b/tests/atomic_probe/pa_scheduler/ccec/host.cpp index d4c80dbf44..1ee342645f 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/host.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/host.cpp @@ -277,6 +277,8 @@ void ConfigurePmu(pa_scheduler::SchedulerState *state, const PmuOptions &pmu, co struct PmuAggregate { std::vector total_cycles; std::vector window_ticks; + std::vector submit_elapsed_ticks; + std::vector phase_elapsed_ticks; std::vector warm_total_cycles; std::vector warm_window_ticks; std::vector vector_busy; @@ -300,6 +302,10 @@ struct PmuAggregate { void AddPmuSample(const pa_scheduler::WorkerResult &result, PmuAggregate *aggregate) { aggregate->total_cycles.push_back(result.pmu_total_cycles); aggregate->window_ticks.push_back(result.pmu_window_ticks); + aggregate->submit_elapsed_ticks.push_back( + result.submit_end >= result.submit_begin ? result.submit_end - result.submit_begin : 0U + ); + aggregate->phase_elapsed_ticks.push_back(result.pmu_phase_elapsed_ticks); aggregate->warm_total_cycles.push_back(result.pmu_warm_total_cycles); aggregate->warm_window_ticks.push_back(result.pmu_warm_window_ticks); aggregate->vector_busy.push_back(result.pmu_vector_busy); @@ -422,6 +428,10 @@ void PrintSubmitPmuPhaseAggregate(const char *name, const PmuAggregate &aggregat pa_scheduler::host::SummarizeUint64(aggregate.shadow_icache_requests); const pa_scheduler::host::Uint64Distribution shadow_misses = pa_scheduler::host::SummarizeUint64(aggregate.shadow_icache_misses); + const pa_scheduler::host::Uint64Distribution submit_ticks = + pa_scheduler::host::SummarizeUint64(aggregate.submit_elapsed_ticks); + const pa_scheduler::host::Uint64Distribution phase_ticks = + pa_scheduler::host::SummarizeUint64(aggregate.phase_elapsed_ticks); const uint64_t request_loss = primary_requests.total >= shadow_requests.total ? primary_requests.total - shadow_requests.total : 0U; @@ -429,10 +439,14 @@ void PrintSubmitPmuPhaseAggregate(const char *name, const PmuAggregate &aggregat ? primary_misses.total - shadow_misses.total : 0U; const double miss_rate = requests.total == 0U ? 0.0 : 100.0 * misses.total / requests.total; + const double phase_time_share = submit_ticks.total == 0U + ? 0.0 + : 100.0 * phase_ticks.total / submit_ticks.total; std::printf( "[PMU-PHASE-%s] phase=%s semantics=%s cores=%zu calls=%llu " "icache_req=[%llu,%llu] icache_miss=[%llu,%llu] " - "observed_read_clear_ratio=%.4f%% shadow_loss=%llu/%llu\n", + "observed_read_clear_ratio=%.4f%% phase_ticks=%llu submit_ticks=%llu " + "phase_time_share=%.4f%% shadow_loss=%llu/%llu\n", name, pa_scheduler::ccec_pmu::SubmitPmuPhaseName(pa_scheduler::kCompiledSubmitPmuPhase), pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None ? "disabled" @@ -442,6 +456,8 @@ void PrintSubmitPmuPhaseAggregate(const char *name, const PmuAggregate &aggregat static_cast(requests.total + request_loss), static_cast(misses.total), static_cast(misses.total + miss_loss), miss_rate, + static_cast(phase_ticks.total), + static_cast(submit_ticks.total), phase_time_share, static_cast(request_loss), static_cast(miss_loss) ); @@ -470,6 +486,7 @@ struct PmuValidation { uint32_t phase_shadow_acceptable = 0; uint32_t phase_boundary_matches = 0; uint32_t phase_call_shape_matches = 0; + uint32_t phase_time_valid_records = 0; uint64_t phase_calls = 0; uint64_t expected_phase_calls = 0; uint64_t shadow_request_abs_delta_sum = 0; @@ -539,6 +556,7 @@ bool ValidatePmu( uint32_t phase_shadow_acceptable = 0; uint32_t phase_boundary_matches = 0; uint32_t phase_call_shape_matches = 0; + uint32_t phase_time_valid_records = 0; uint64_t phase_calls = 0; uint64_t expected_phase_calls = 0; uint64_t shadow_request_abs_delta_sum = 0; @@ -589,6 +607,15 @@ bool ValidatePmu( result.pmu_phase_end_reads == result.pmu_phase_calls; const bool phase_call_shape_matches_record = result.pmu_phase_calls == expected_phase_calls_per_worker; + const uint64_t submit_elapsed_ticks = result.submit_end >= result.submit_begin + ? result.submit_end - result.submit_begin + : 0U; + const bool phase_time_valid_record = + submit_elapsed_ticks != 0U && + result.pmu_phase_elapsed_ticks <= submit_elapsed_ticks && + (pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? result.pmu_phase_elapsed_ticks == 0U + : result.pmu_phase_elapsed_ticks != 0U); const bool logical_aic = worker < pa_scheduler::kAicWorkers; const bool physical_aic = pa_scheduler::pmu_owner::IsAicPhysicalSlot(core_id); trusted += record_trusted; @@ -600,6 +627,7 @@ bool ValidatePmu( phase_shadow_acceptable += shadow_acceptable; phase_boundary_matches += boundaries_match; phase_call_shape_matches += phase_call_shape_matches_record; + phase_time_valid_records += phase_time_valid_record; phase_calls += result.pmu_phase_calls; expected_phase_calls += expected_phase_calls_per_worker; shadow_request_abs_delta_sum += request_abs_delta; @@ -665,11 +693,11 @@ bool ValidatePmu( } if ((!record_trusted || !variant_matches || !phase_id_matches_record || !phase_status_ok || !shadow_acceptable || !boundaries_match || - !phase_call_shape_matches_record) && bad_printed < 8) { + !phase_call_shape_matches_record || !phase_time_valid_record) && bad_printed < 8) { std::printf( "[PMU-BAD] worker=%u role=%llu coreid=%u status=0x%08x total=%llu scalar=%u " "req=%u miss=%u phase_status=0x%08x phase=%u/%u calls=%u/%u boundaries=%u/%u " - "shadow=%u/%u\n", + "phase_ticks=%llu submit_ticks=%llu shadow=%u/%u\n", worker, static_cast(result.role), core_id, status, static_cast(result.pmu_total_cycles), result.pmu_scalar_busy, result.pmu_icache_requests, result.pmu_icache_misses, @@ -677,6 +705,8 @@ bool ValidatePmu( static_cast(pa_scheduler::kCompiledSubmitPmuPhase), result.pmu_phase_calls, expected_phase_calls_per_worker, result.pmu_phase_begin_reads, result.pmu_phase_end_reads, + static_cast(result.pmu_phase_elapsed_ticks), + static_cast(submit_elapsed_ticks), result.pmu_shadow_icache_requests, result.pmu_shadow_icache_misses ); ++bad_printed; @@ -732,6 +762,7 @@ bool ValidatePmu( const bool shadow_partition_ok = phase_shadow_acceptable == pa_scheduler::kWorkers; const bool phase_boundaries_ok = phase_boundary_matches == pa_scheduler::kWorkers; const bool phase_call_shape_ok = phase_call_shape_matches == pa_scheduler::kWorkers; + const bool phase_time_ok = phase_time_valid_records == pa_scheduler::kWorkers; const bool phase_calls_ok = phase_calls == expected_phase_calls; const bool submit_engine_observation_ok = pmu.mode != WindowMode::SubmitAll || @@ -785,6 +816,8 @@ bool ValidatePmu( std::printf("[ASSERT] %-48s %s\n", "all phase boundaries and per-worker calls are exact", phase_status_ok && phase_boundaries_ok && phase_call_shape_ok && phase_calls_ok ? "PASS" : "FAIL"); + std::printf("[ASSERT] %-48s %s\n", "all phase times fit their per-worker Submit windows", + phase_time_ok ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "I-cache misses do not exceed requests", icache_order_valid ? "PASS" : "FAIL"); std::printf("[ASSERT] %-48s %s\n", "programmable counters stay below 25% risk threshold", @@ -824,6 +857,7 @@ bool ValidatePmu( validation->phase_shadow_acceptable = phase_shadow_acceptable; validation->phase_boundary_matches = phase_boundary_matches; validation->phase_call_shape_matches = phase_call_shape_matches; + validation->phase_time_valid_records = phase_time_valid_records; validation->phase_calls = phase_calls; validation->expected_phase_calls = expected_phase_calls; validation->shadow_request_abs_delta_sum = shadow_request_abs_delta_sum; @@ -838,7 +872,7 @@ bool ValidatePmu( validation->counter_below_risk_threshold = counter_below_risk_threshold; validation->phase_measurement_valid = build_variant_ok && phase_id_ok && phase_status_ok && shadow_partition_ok && - phase_boundaries_ok && phase_call_shape_ok && phase_calls_ok; + phase_boundaries_ok && phase_call_shape_ok && phase_calls_ok && phase_time_ok; validation->passed = records_ok && core_ids_ok && owner_members_ok && worker_slots_ok && physical_roles_ok && mixed_triplets_ok && windows_started_ok && windows_stopped_ok && icache_order_valid && icache_measurement_ok && submit_engine_observation_ok && @@ -928,6 +962,10 @@ void WritePmuAggregateJson( WriteMetricDistribution(output, aggregate.shadow_icache_misses); std::fputs(",\"phase_calls\":", output); WriteMetricDistribution(output, aggregate.phase_calls); + std::fputs(",\"submit_elapsed_ticks\":", output); + WriteMetricDistribution(output, aggregate.submit_elapsed_ticks); + std::fputs(",\"phase_elapsed_ticks\":", output); + WriteMetricDistribution(output, aggregate.phase_elapsed_ticks); std::fputs(",\"phase_icache_requests\":", output); WriteMetricDistribution(output, aggregate.phase_icache_requests); std::fputs(",\"phase_icache_misses\":", output); @@ -1042,7 +1080,7 @@ bool ExportPmuJson( }; const uint32_t owner_bitmap_count = pa_scheduler::pmu_owner::CountConfigured(owner); const uint32_t owner_complete_triplets = CountConfiguredMixedTriplets(owner); - std::fputs("{\n\"schema\":{\"name\":\"pa_scheduler_pmu_phase_windows\",\"version\":4},\n", output); + std::fputs("{\n\"schema\":{\"name\":\"pa_scheduler_pmu_phase_windows\",\"version\":5},\n", output); std::fputs("\"capture\":{\"capture_id\":", output); WriteJsonString(output, capture_id); std::fprintf( @@ -1157,9 +1195,18 @@ bool ExportPmuJson( "\"counter_wrap_not_directly_detectable\":true,\"counter_wrap_absence_proven\":false," "\"programmable_counter_risk_threshold\":%u," "\"gate_start_stop_have_pipe_all_barriers\":true," - "\"phase_timestamp_calls_present\":false,\"phase_record_writes\":false," + "\"phase_timestamp_calls_present\":%s,\"phase_record_writes\":false," "\"atomic_trace\":false,\"profile_accumulation\":false," "\"phase_boundary_observation_included\":%s," + "\"phase_time_observation_included\":%s," + "\"phase_time_sys_counter_tick_ns\":1," + "\"phase_time_boundary\":\"after_begin_read_clear_to_before_end_read_clear\"," + "\"phase_time_excludes_shadow_read_overhead\":true," + "\"phase_time_includes_timestamp_overhead\":true," + "\"phase_time_share_definition\":" + "\"sum(phase_elapsed_ticks)/sum(submit_elapsed_ticks)\"," + "\"phase_time_denominator_scope\":" + "\"per_worker_first_submit_begin_to_last_submit_end\"," "\"phase_counter_pair_snapshot_atomic\":false," "\"primary_counters_read_at_phase_boundaries\":false," "\"phase_shadow_partition_exact_required\":%s," @@ -1172,6 +1219,12 @@ bool ExportPmuJson( host_us, submit_span_us, kVectorBusyEvent, kCubeBusyEvent, kScalarBusyEvent, kMte1BusyEvent, kMte2BusyEvent, kIcacheMissEvent, kIcacheRequestEvent, kIcacheMissEvent, kIcacheRequestEvent, kProgrammableCounterRiskThreshold, + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? "false" + : "true", + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? "false" + : "true", pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None ? "false" : "true", @@ -1218,6 +1271,7 @@ bool ExportPmuJson( "\"shadow_miss_abs_delta_sum\":%llu,\"shadow_miss_abs_delta_max\":%u," "\"shadow_miss_signed_delta_sum\":%lld," "\"phase_boundary_match_records\":%u,\"phase_call_shape_match_records\":%u," + "\"phase_time_valid_records\":%u,\"phase_time_measurement_valid\":%s," "\"phase_calls\":%llu,\"phase_expected_calls\":%llu," "\"phase_measurement_valid\":%s},\n", semantic_passed ? "true" : "false", validation.passed ? "true" : "false", @@ -1257,6 +1311,8 @@ bool ExportPmuJson( static_cast(validation.shadow_miss_signed_delta_sum), validation.phase_boundary_matches, validation.phase_call_shape_matches, + validation.phase_time_valid_records, + validation.phase_time_valid_records == pa_scheduler::kWorkers ? "true" : "false", static_cast(validation.phase_calls), static_cast(validation.expected_phase_calls), validation.phase_measurement_valid ? "true" : "false" @@ -1293,7 +1349,16 @@ bool ExportPmuJson( const bool primary_trusted = (status & kStatusRequired) == kStatusRequired; const bool phase_trusted = (result.pmu_phase_status & kPhaseStatusRequired) == kPhaseStatusRequired; - const bool trusted = primary_trusted && phase_trusted; + const uint64_t submit_elapsed_ticks = result.submit_end >= result.submit_begin + ? result.submit_end - result.submit_begin + : 0U; + const bool phase_time_valid = + submit_elapsed_ticks != 0U && + result.pmu_phase_elapsed_ticks <= submit_elapsed_ticks && + (pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? result.pmu_phase_elapsed_ticks == 0U + : result.pmu_phase_elapsed_ticks != 0U); + const bool trusted = primary_trusted && phase_trusted && phase_time_valid; const bool is_aic = result.role == static_cast(pa_scheduler::CoreRole::Aic); const uint32_t vector_id = is_aic ? 0U : worker - pa_scheduler::kAicWorkers; const uint32_t block_id = is_aic ? worker : vector_id / 2U; @@ -1334,6 +1399,8 @@ bool ExportPmuJson( "\"build_variant_id\":%u,\"compiled_phase_id\":%u,\"phase_calls\":%u," "\"phase_expected_calls\":%u," "\"phase_begin_reads\":%u,\"phase_end_reads\":%u," + "\"submit_elapsed_ticks\":%llu,\"phase_elapsed_ticks\":%llu," + "\"phase_time_valid\":%s," "\"phase_icache_requests\":%u,\"phase_icache_misses\":%u," "\"phase_icache_requests_upper_bound\":%u," "\"phase_icache_misses_upper_bound\":%u," @@ -1354,6 +1421,9 @@ bool ExportPmuJson( result.pmu_build_variant, result.pmu_phase_id, result.pmu_phase_calls, expected_phase_calls, result.pmu_phase_begin_reads, result.pmu_phase_end_reads, + static_cast(submit_elapsed_ticks), + static_cast(result.pmu_phase_elapsed_ticks), + phase_time_valid ? "true" : "false", result.pmu_phase_icache_requests, result.pmu_phase_icache_misses, phase_request_upper, phase_miss_upper, result.pmu_shadow_icache_requests, result.pmu_shadow_icache_misses, diff --git a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp index 319dfe67e2..c63f601e81 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp +++ b/tests/atomic_probe/pa_scheduler/ccec/kernel.cpp @@ -387,6 +387,10 @@ struct SubmitPmuContext { uint64_t shadow_misses = 0; uint64_t phase_requests = 0; uint64_t phase_misses = 0; + // begin 的 shadow read-clear 完成后取起点,end 的 shadow read-clear 之前 + // 取终点;累计值因此不包含两次 PMU 寄存器读取本身。 + uint64_t phase_elapsed_ticks = 0; + uint64_t phase_begin_tick = 0; uint32_t selector_status = 0; uint32_t phase_status = pa_scheduler::ccec_pmu::kPhaseStatusRequested; uint32_t phase_calls = 0; @@ -511,7 +515,6 @@ __aicore__ inline void PublishPmuSnapshot( CcecOps::Publish(&result.pmu_mte3_busy, sample.mte3_busy); // CNT8 已改作 shadow request,submit-pmu 不再发布 fix-busy。 CcecOps::Publish(&result.pmu_fix_busy, static_cast(0)); - CcecOps::Publish(&result.pmu_window_ticks, static_cast(0)); CcecOps::Publish(&result.pmu_warm_total_cycles, static_cast(0)); CcecOps::Publish(&result.pmu_warm_window_ticks, static_cast(0)); } @@ -532,6 +535,7 @@ __aicore__ inline void PublishSubmitPmuContext( CcecOps::Publish(&result.pmu_phase_status, context.phase_status); CcecOps::Publish(&result.pmu_phase_begin_reads, context.begin_reads); CcecOps::Publish(&result.pmu_phase_end_reads, context.end_reads); + CcecOps::Publish(&result.pmu_phase_elapsed_ticks, context.phase_elapsed_ticks); CcecOps::Publish(&result.pmu_phase_icache_requests, static_cast(context.phase_requests)); CcecOps::Publish(&result.pmu_phase_icache_misses, static_cast(context.phase_misses)); CcecOps::Publish(&result.pmu_shadow_icache_requests, static_cast(context.shadow_requests)); @@ -575,13 +579,23 @@ __aicore__ inline void CcecOps::PmuPhaseBegin(PmuContext &context) { context.shadow_misses += sample.misses; ++context.begin_reads; context.phase_armed = true; + // get_sys_cnt() 是本机已校准为 1 ns/tick 的 A5 系统计数器。该读取位于 + // begin 的两条 ld_dev 之后,因此不会把 read-clear 成本算进阶段时间。 + context.phase_begin_tick = CcecOps::Now(); } __aicore__ inline void CcecOps::PmuPhaseEnd(PmuContext &context) { + // 先取终点再读取 shadow counter,使 end 的两条 ld_dev 同样位于阶段之外。 + const uint64_t phase_end_tick = CcecOps::Now(); if (!context.started || context.reg_base == 0 || !context.phase_armed) { context.boundary_error = true; return; } + if (phase_end_tick < context.phase_begin_tick) { + context.boundary_error = true; + } else { + context.phase_elapsed_ticks += phase_end_tick - context.phase_begin_tick; + } // end 读出的片段同时属于完整 shadow 重建与被选中的局部阶段。 const IcacheShadowSnapshot sample = ReadShadowCounters(context.reg_base); context.shadow_requests += sample.requests; @@ -591,6 +605,7 @@ __aicore__ inline void CcecOps::PmuPhaseEnd(PmuContext &context) { ++context.end_reads; ++context.phase_calls; context.phase_armed = false; + context.phase_begin_tick = 0; } __aicore__ inline void CcecOps::PmuWindowStop( @@ -639,7 +654,8 @@ __aicore__ inline void CcecOps::PmuWindowStop( const bool none_shape = pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None && context.phase_calls == 0 && context.begin_reads == 0 && context.end_reads == 0 && - context.phase_requests == 0 && context.phase_misses == 0; + context.phase_requests == 0 && context.phase_misses == 0 && + context.phase_elapsed_ticks == 0; const bool running_shape = pa_scheduler::kCompiledSubmitPmuPhase != pa_scheduler::SubmitPmuPhase::None && context.phase_calls == state->config.batches * pa_scheduler::kTasksPerBatch && @@ -647,6 +663,12 @@ __aicore__ inline void CcecOps::PmuWindowStop( context.end_reads == context.phase_calls; if (none_shape || running_shape) context.phase_status |= kPhaseStatusPhaseShape; + const bool phase_time_valid = + pa_scheduler::kCompiledSubmitPmuPhase == pa_scheduler::SubmitPmuPhase::None + ? context.phase_elapsed_ticks == 0 + : context.phase_calls != 0 && context.phase_elapsed_ticks != 0; + if (phase_time_valid) + context.phase_status |= kPhaseStatusTimeValid; PublishPmuSnapshot(result, sample); PublishSubmitPmuContext(result, context); diff --git a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h index 16db1e4f1b..a2e8cab6a0 100644 --- a/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h +++ b/tests/atomic_probe/pa_scheduler/ccec/pmu_probe.h @@ -138,11 +138,14 @@ constexpr uint32_t kPhaseStatusBoundariesBalanced = 1U << 6; constexpr uint32_t kPhaseStatusValuesOrdered = 1U << 7; constexpr uint32_t kPhaseStatusUint32Fit = 1U << 8; constexpr uint32_t kPhaseStatusPhaseShape = 1U << 9; +// none 必须保持 0 tick;运行阶段则必须确实累计到非零 SYS_CNT。阶段时间是否 +// 不超过同核首 Submit 到末 Submit 的完整区间,由拿到两端结果的 host 再校验。 +constexpr uint32_t kPhaseStatusTimeValid = 1U << 10; constexpr uint32_t kPhaseStatusRequired = kPhaseStatusRequested | kPhaseStatusShadowSelectors | kPhaseStatusWindowStarted | kPhaseStatusWindowStopped | kPhaseStatusBoundariesBalanced | kPhaseStatusValuesOrdered | - kPhaseStatusUint32Fit | kPhaseStatusPhaseShape; + kPhaseStatusUint32Fit | kPhaseStatusPhaseShape | kPhaseStatusTimeValid; inline const char *SubmitPmuPhaseName(SubmitPmuPhase phase) { switch (phase) { diff --git a/tests/atomic_probe/pa_scheduler/common/host_support.h b/tests/atomic_probe/pa_scheduler/common/host_support.h index d205b39ffc..9c4c81aa03 100644 --- a/tests/atomic_probe/pa_scheduler/common/host_support.h +++ b/tests/atomic_probe/pa_scheduler/common/host_support.h @@ -224,11 +224,11 @@ inline void ConfigureTrace(SchedulerState *state, const Options &options, const } inline void InitializeTraceHeader(TraceHeader *header) { - // version=3 表示 core state 已携带 weighted atomic/PollBatch 计数和权威拓扑; - // JSON 的 trace_schema_version 则仍按是否启用 atomic 分别导出 v2/v3。 + // version=4 表示 phase ABI 已追加父区间和真实 Submit 尾动作;core state + // 继续携带 weighted atomic/PollBatch 计数和权威拓扑。 std::memset(header, 0, sizeof(*header)); header->magic = 0x4653574cU; - header->version = 3; + header->version = 4; header->num_cores = kWorkers; header->records_per_core = kTraceRecordsPerCore; header->frequency_hz = kSystemCounterHz; @@ -343,8 +343,13 @@ inline const char *TracePhaseName(uint32_t phase) { const char *names[] = { "Kernel", "Alloc", "Build", "DrainWon", "Replay", "RingBp", "EfDrain", "Commit", "Submit", "Materialize", "PrepareMap", "Claim", "Fanin", "Register", "Atomic", - "ClockBaseline", + "ClockBaseline", "OrchestrationReplay", "FinalDrain", "WinnerBuild", + "AllocComplete", }; + static_assert( + sizeof(names) / sizeof(names[0]) == static_cast(TracePhase::Count), + "TracePhaseName must cover every trace phase" + ); return phase < sizeof(names) / sizeof(names[0]) ? names[phase] : "Unknown"; } @@ -371,7 +376,7 @@ inline AtomicOp AtomicSiteOp(AtomicSite site) { inline bool ValidateTraceHeader(const TraceHeader &header, const char *operation) { // 在任何 D2H record 搬运前先验证容量和 dropped,防止损坏 header 导致 scratch 越界或导出残缺泳道。 // 频率也要求精确为 1 GHz,否则后续 ns/us 换算即使 JSON 合法也没有性能意义。 - const bool valid = header.magic == 0x4653574cU && header.version == 3 && + const bool valid = header.magic == 0x4653574cU && header.version == 4 && header.num_cores == kWorkers && header.records_per_core == kTraceRecordsPerCore && header.frequency_hz == kSystemCounterHz; bool core_states_valid = true; @@ -557,7 +562,7 @@ inline bool ExportSwimlaneRecords( "\"engine_mapping\":%s},\"core_types\":[", atomic_trace_enabled ? 4U : 1U, static_cast(header.frequency_hz), kWorkers, - atomic_trace_enabled ? 3U : 2U, + 4U, workload_mode == WinnerWorkloadMode::RealCompute ? "real-compute" : "scalar-nop", workload_counts.qk, workload_counts.sf, workload_counts.pv, workload_counts.up, workload_mode == WinnerWorkloadMode::RealCompute @@ -572,24 +577,22 @@ inline bool ExportSwimlaneRecords( for (uint32_t worker = 0; worker < kWorkers; ++worker) { std::fprintf(output, "%s\"%s\"", worker == 0 ? "" : ",", worker < kAicWorkers ? "aic" : "aiv"); } - if (atomic_trace_enabled) { - std::fprintf( - output, - "],\"fdwic_summary\":{\"records\":%llu,\"atomic_records\":%llu," - "\"clock_baseline_records\":%llu,\"atomic_calls\":%llu," - "\"batched_poll_calls\":%llu,\"poll_batch_records\":%llu," - "\"dropped_records\":%llu}", - static_cast(producer_summary.records), - static_cast(producer_summary.atomic_records), - static_cast(producer_summary.clock_baseline_records), - static_cast(producer_summary.atomic_calls), - static_cast(producer_summary.poll_calls), - static_cast(producer_summary.poll_batch_records), - static_cast(producer_summary.dropped_records) - ); - } else { - std::fprintf(output, "]"); - } + // schema-v4 无论是否开启 atomic 都导出 producer summary;phase-only 的 + // atomic/clock 字段为零,离线分析仍可独立证明 records 与 dropped 闭合。 + std::fprintf( + output, + "],\"fdwic_summary\":{\"records\":%llu,\"atomic_records\":%llu," + "\"clock_baseline_records\":%llu,\"atomic_calls\":%llu," + "\"batched_poll_calls\":%llu,\"poll_batch_records\":%llu," + "\"dropped_records\":%llu}", + static_cast(producer_summary.records), + static_cast(producer_summary.atomic_records), + static_cast(producer_summary.clock_baseline_records), + static_cast(producer_summary.atomic_calls), + static_cast(producer_summary.poll_calls), + static_cast(producer_summary.poll_batch_records), + static_cast(producer_summary.dropped_records) + ); std::fprintf( output, "},\n\"aicore_tasks\":[],\n\"aicpu_tasks\":[],\n" @@ -602,7 +605,7 @@ inline bool ExportSwimlaneRecords( uint64_t exported_records = 0; TraceExportSummary observed_summary; std::vector scratch(kTraceRecordsPerCore); - constexpr int32_t kTracePhaseCount = static_cast(TracePhase::ClockBaseline) + 1; + constexpr int32_t kTracePhaseCount = static_cast(TracePhase::Count); for (uint32_t worker = 0; worker < kWorkers && success; ++worker) { // 每次只读取一个 worker 的有效区间;完整 384 MiB trace 缓冲无需整体回拷。 const uint32_t available = header.cores[worker].count; @@ -786,7 +789,7 @@ inline bool AnalyzeSwimlaneRecords( if (!ValidateTraceHeader(header, "swimlane analysis")) return false; // 第一组数组统计“每个 worker 在某阶段的累计时间”;task_durations 则保留重点阶段的单事件分布。 - constexpr uint32_t kTracePhaseCount = static_cast(TracePhase::ClockBaseline) + 1; + constexpr uint32_t kTracePhaseCount = static_cast(TracePhase::Count); constexpr TracePhase kDetailedPhases[] = { TracePhase::EfDrain, TracePhase::Materialize, TracePhase::Claim, TracePhase::Register, }; @@ -1294,7 +1297,7 @@ inline Metrics Validate( bool per_worker_trace_counts_ok = true; if (trace_header != nullptr) { trace_shape_ok &= trace_header->magic == 0x4653574cU; - trace_shape_ok &= trace_header->version == 3; + trace_shape_ok &= trace_header->version == 4; trace_shape_ok &= trace_header->num_cores == kWorkers; trace_shape_ok &= trace_header->records_per_core == kTraceRecordsPerCore; trace_shape_ok &= trace_header->frequency_hz == kSystemCounterHz; @@ -1327,8 +1330,8 @@ inline Metrics Validate( core.poll_batch_records == 0; } const uint64_t worker_expected = - 7 * result.submits + result.claim_wins - result.wins[0] + - 2 * worker_kernels + result.wait_events[0] + result.wait_events[1] + + 6 * result.submits + 2 * result.claim_wins - result.wins[0] + + 2 * worker_kernels + result.wait_events[0] + result.wait_events[1] + 2 + (((state.config.trace_enabled & kTraceAtomicsEnabled) != 0) ? worker_physical_atomic + 2 : 0); @@ -1336,12 +1339,15 @@ inline Metrics Validate( } } const uint64_t expected_trace_records = - static_cast(batches) * (static_cast(kWorkers) * 35 + 12) + - trace_wait_records + + static_cast(batches) * (static_cast(kWorkers) * 30 + 17) + + trace_wait_records + 2 * kWorkers + (((state.config.trace_enabled & kTraceAtomicsEnabled) != 0) ? physical_atomic_records + 2 * kWorkers : 0); - // 每 batch 固定记录为 96*35+12;RingBp 等真实等待按运行时次数额外加入。 + // 每 batch 的 96*30 是六条每 Submit 固定记录;17 条是 + // 5 条 winner tail、4 条 Fanin 和 8 条 Kernel/Commit。loser 不再写 + // 零时长 marker。两个父 span 再各核固定增加 2 条; + // RingBp 等真实等待按运行时次数额外加入。 Expect(trace_shape_ok, "swimlane header and per-worker capacities are valid", &metrics); Expect(trace_dropped == 0, "swimlane records fit without drops", &metrics); Expect(trace_records == expected_trace_records, "swimlane record count matches PA phase flow", &metrics); diff --git a/tests/atomic_probe/pa_scheduler/common/pa_model.h b/tests/atomic_probe/pa_scheduler/common/pa_model.h index 14613cf4cf..0f0a2954c2 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_model.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_model.h @@ -287,6 +287,13 @@ enum class TracePhase : int32_t { // 逐 atomic 诊断构建中,每个 worker 只记录一次连续两次 SYS_CNT 的 // 空括号,用来给出同一二进制、同一物理核上的计时分辨率下限。 ClockBaseline = 15, + // schema-v4 追加的父区间与真实动作区间。loser 没有可单列的真实动作, + // 其时间直接归入离线计算的 Submit residual,不占用 raw 记录。 + OrchestrationReplay = 16, + FinalDrain = 17, + WinnerBuild = 18, + AllocComplete = 19, + Count = 20, }; // AtomicSite 按 standalone PA 中真实出现的源码调用点分类。编号写入 TraceRecord::auxiliary, @@ -752,9 +759,13 @@ struct alignas(64) WorkerResult { // host 用它与 Atomic span 数逐 worker 闭合,禁止把丢记录的泳道当成完整结果。 uint64_t atomic_trace_calls; - // I-cache 单 miss 探针把 cold 样本放在既有 PMU 字段中,并在扩展的诊断尾部 - // 保存同核配对的 warm 样本与 1 GHz 系统计数器窗口;非该模式均写零。 - uint64_t pmu_window_ticks; + // I-cache 单 miss 探针用该槽保存 cold 窗口的 1 GHz SYS_CNT;submit-pmu + // 复用同一 64-bit 槽保存所选局部阶段的逐调用累计时间。两种构建互斥, + // 因而无需扩大 832B WorkerResult,也不会改变相邻 worker 的 cache-line 布局。 + union { + uint64_t pmu_window_ticks; + uint64_t pmu_phase_elapsed_ticks; + }; uint64_t pmu_warm_total_cycles; uint64_t pmu_warm_window_ticks; union { @@ -774,7 +785,7 @@ struct alignas(64) WorkerResult { uint32_t pmu_mte1_busy; uint32_t pmu_mte2_busy; // swimlane ABI 保留该槽;submit-pmu 将物理 CNT5 改作 shadow miss, - // 因而显式发布 0,并在 schema v4 标记 mte3_busy 不可用。 + // 因而显式发布 0,并在当前 submit-pmu schema-v5 标记 mte3_busy 不可用。 uint32_t pmu_mte3_busy; uint32_t pmu_fix_busy; diff --git a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h index d4237df070..f867dba26e 100644 --- a/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h +++ b/tests/atomic_probe/pa_scheduler/common/pa_scheduler_core.h @@ -559,41 +559,43 @@ PA_DEVICE bool SubmitTask( stats.result.submit_begin = submit_begin; } - ResetTraceLap(stats.trace, stats.result, worker); // EfDrain 在当前 Submit 的参数物化前执行上一批已就绪 slot,是绝大多数 kernel 的正常落点。 // 只在这个唯一 call-site 划 PMU 边界;DrainReady 还被 ring 背压和最终 drain // 复用,不能把 phase 插入函数体后按 place 混合累计。 + // EfDrain 是 Submit 的第一个真实阶段,直接复用父区间起点;这样每次 + // Submit 少一次 trace-only SYS_CNT,也不会留下人为的 prefix residual。 + const uint64_t efdrain_begin = submit_begin; BeginSubmitPmuPhase(pmu_context); DrainReady(state, worker, DrainPlace::EfDrain, stats); EndSubmitPmuPhase(pmu_context); - WriteTraceLap( - stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::EfDrain, - ProfilePhase::EfDrain + const uint64_t efdrain_end = TraceTimestamp(stats.trace, stats.result); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, TracePhase::EfDrain, + ProfilePhase::EfDrain, efdrain_begin, efdrain_end ); - // dist_submit_materialize_and_prepare_map resets the lap origin before its - // two independently traced spans. Build/Replay later consumes this origin. - // lap 起点在 materialize 前重置;Materialize/PrepareMap 各自取独立绝对区间,而后续 - // Build/Replay 会从这个起点形成覆盖式 span。因此泳道上的这些阶段不能直接相加。 - ResetTraceLap(stats.trace, stats.result, worker); - const uint64_t materialize_begin = TraceTimestamp(stats.trace, stats.result); + // schema-v4 的所有 Submit 子阶段都使用显式 start/end;不再通过共同 lap + // 起点生成相互覆盖的 Build/Replay/Alloc 区间。 + // 后继 segment 复用前一阶段 end:既少一次 SYS_CNT,也把前一条 trace + // 发布和阶段间胶水明确归入 Materialize,而不是留成无名 residual。 + const uint64_t materialize_begin = efdrain_end; BeginSubmitPmuPhase(pmu_context); const bool materialized = MaterializeTask(worker, task_id, args, context, state->heap_base, state->heap_size); - EndSubmitPmuPhase(pmu_context); if (!materialized) { + EndSubmitPmuPhase(pmu_context); SetFatal(state, stats, static_cast(task_id)); return false; } + stats.result.materialized_outputs += context.result.count; + EndSubmitPmuPhase(pmu_context); const uint64_t materialize_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Materialize, ProfilePhase::Materialize, materialize_begin, materialize_end, 0, kind == TaskKind::Alloc ? 1U : 0U ); - stats.result.materialized_outputs += context.result.count; - - const uint64_t prepare_begin = TraceTimestamp(stats.trace, stats.result); + const uint64_t prepare_begin = materialize_end; AdvanceTensorMap(worker.map, task_id, static_cast(state->heap_window)); const uint64_t prepare_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( @@ -606,7 +608,7 @@ PA_DEVICE bool SubmitTask( if (kind == TaskKind::Alloc) { // Alloc 没有 kernel lane,96 个 worker 都维护本地物化/heap 状态,但只有 Claim winner 发布全局完成。 - const uint64_t register_begin = TraceTimestamp(stats.trace, stats.result); + const uint64_t register_begin = prepare_end; BeginSubmitPmuPhase(pmu_context); RegisterOutputs(context, args, false); EndSubmitPmuPhase(pmu_context); @@ -616,96 +618,101 @@ PA_DEVICE bool SubmitTask( ProfilePhase::Register, register_begin, register_end, 0, 0 ); - const uint64_t claim_begin = TraceTimestamp(stats.trace, stats.result); + const uint64_t claim_begin = register_end; BeginSubmitPmuPhase(pmu_context); const ClaimOutcome claim = Claim(state, worker, task_id, kind, stats); - EndSubmitPmuPhase(pmu_context); winner = claim.won; context.won = winner; context.kernel_id = claim.function_id; + // Claim 的本地结果归档属于同一阶段;放在共同 end 边界内,避免把 + // winner/context/stat bookkeeping 留成无法归因的 Submit residual。 + RecordClaimOutcome(stats, kind, claim); + EndSubmitPmuPhase(pmu_context); const uint64_t claim_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), -1, TracePhase::Claim, ProfilePhase::Claim, claim_begin, claim_end, (winner ? kClaimWon : 0U) | (claim.attempted ? kClaimAttempted : 0U), 1 ); - RecordClaimOutcome(stats, kind, claim); if (winner) { + const uint64_t alloc_complete_begin = claim_end; if (!HeapGuard( state, worker, task_id, context.output_bytes, stats )) { return false; } CompleteTask(state, worker, task_id, stats); - WriteTraceLap( - stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::Alloc, - ProfilePhase::ReplayTail + const uint64_t alloc_complete_end = TraceTimestamp(stats.trace, stats.result); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), -1, + TracePhase::AllocComplete, ProfilePhase::ReplayTail, + alloc_complete_begin, alloc_complete_end ); } else { - // Replay 表示该 worker 输掉 Claim;前面的物化、TensorMap 和 register 仍已执行,以保持本地状态同步。 - WriteTraceLap( - stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::Replay, - ProfilePhase::ReplayTail - ); + // standalone 的 Alloc loser 没有真实 GM/Replay 动作,不再为业务 + // 路径名字写一条零时长记录。Claim 后到 Submit.end 的真实 + // scalar 时间由离线 submit_tail_gap 补集展示;排他报告将其汇总为 + // submit_tail_residual,避免伪装成 Alloc loser 业务阶段。 } } else { - const uint64_t claim_begin = TraceTimestamp(stats.trace, stats.result); + const uint64_t claim_begin = prepare_end; BeginSubmitPmuPhase(pmu_context); const ClaimOutcome claim = Claim(state, worker, task_id, kind, stats); - EndSubmitPmuPhase(pmu_context); winner = claim.won; function_id = claim.function_id; context.won = winner; context.kernel_id = function_id; + RecordClaimOutcome(stats, kind, claim); + EndSubmitPmuPhase(pmu_context); const uint64_t claim_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Claim, ProfilePhase::Claim, claim_begin, claim_end, (winner ? kClaimWon : 0U) | (claim.attempted ? kClaimAttempted : 0U), 0 ); - RecordClaimOutcome(stats, kind, claim); - + uint64_t register_begin = claim_end; if (winner) { - const uint64_t fanin_begin = TraceTimestamp(stats.trace, stats.result); + const uint64_t fanin_begin = claim_end; context.fanin_count = static_cast(CollectFanin(worker.map, args, context.fanin)); + stats.result.map_lookups += static_cast(args.tensor_count) - context.result.count; const uint64_t fanin_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Fanin, ProfilePhase::Fanin, fanin_begin, fanin_end, 0, static_cast(context.fanin_count) ); - stats.result.map_lookups += static_cast(args.tensor_count) - context.result.count; + register_begin = fanin_end; } - const uint64_t register_begin = TraceTimestamp(stats.trace, stats.result); + // loser 直接承接 Claim.end;winner 则承接 Fanin.end。两条路径都 + // 复用已有边界,不再为 Register 单独读取 SYS_CNT。 BeginSubmitPmuPhase(pmu_context); RegisterOutputs(context, args, true); + stats.result.map_inserts += CountBits(context.register_mask); EndSubmitPmuPhase(pmu_context); const uint64_t register_end = TraceTimestamp(stats.trace, stats.result); WriteTrace( stats.trace, stats.result, static_cast(task_id), function_id, TracePhase::Register, ProfilePhase::Register, register_begin, register_end, 0, 1 ); - stats.result.map_inserts += CountBits(context.register_mask); - if (winner) { - WriteTraceLap( - stats.trace, worker, stats.result, static_cast(task_id), function_id, TracePhase::Build, - ProfilePhase::ReplayTail - ); + const uint64_t winner_build_begin = register_end; if (!BuildWinner( state, worker, task_id, kind, args, context, context.fanin, static_cast(context.fanin_count), stats )) { return false; } - } else { - // 非 winner 不占用私有 ring slot,也不执行 kernel;Replay marker 覆盖本次前端回放的尾段。 - WriteTraceLap( - stats.trace, worker, stats.result, static_cast(task_id), -1, TracePhase::Replay, - ProfilePhase::ReplayTail + const uint64_t winner_build_end = TraceTimestamp(stats.trace, stats.result); + WriteTrace( + stats.trace, stats.result, static_cast(task_id), function_id, + TracePhase::WinnerBuild, ProfilePhase::ReplayTail, + winner_build_begin, winner_build_end ); - // drain_block_won() is a local boolean early-return for this - // single-lane PA graph, so it intentionally performs no GM access. + } else { + // 非 winner 不占用私有 ring slot,也没有可单列的 Replay 计算。 + // Register 后到 Submit.end 由离线 submit_tail_gap 补集展示;排他 + // 报告汇总为 submit_tail_residual,避免零时长 marker 增加 raw + // 体积和 trace-buffer 写开销。 } } @@ -870,15 +877,17 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, PaOrchestrationState orchestration; TaskArgs args; SubmitContext context; + uint64_t orchestration_begin = 0; + uint64_t orchestration_end = 0; if (!IsFatal(state, stats)) { // Case1 每个 batch 固定回放 Alloc/QK/SF/PV/UP 五个 task;所有 worker 顺序相同,执行 lane 由 Claim 筛选。 // CCEC 可在这里开启本 worker 私有 PMU 窗口;CPU/AscendC 适配层是空实现。 // 窗口覆盖从首个参数构造到末次 Submit 返回,与全局“首 Submit.begin~末 Submit.end” // 口径接近但不相同,host sidecar 必须按 per-worker 累计解释。 - ResetTraceLap(stats.trace, stats.result, worker); - // lap 重置属于泳道观察自身,不应污染 PMU-only 的 Submit 取数;窗口从 + // 泳道父边界在 PMU-only 构建中会被编译为空,不应污染 Submit 取数;窗口从 // orchestration 初始化(即首批参数构造)前一条边界开始。 auto pmu_context = Ops::PmuWindowStart(state, worker_id); + orchestration_begin = TraceTimestamp(stats.trace, stats.result); InitPaOrchestration(orchestration, batches, &state->context_lens[0]); for (uint32_t batch = 0; batch < batches; ++batch) { BuildAllocArgs(orchestration, args, batch); @@ -938,9 +947,15 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, } } Ops::PmuWindowStop(state, worker_id, pmu_context); + orchestration_end = TraceTimestamp(stats.trace, stats.result); } // replay_done 表示所有 worker 已退出回放循环(成功路径即完整提交);之后仍需 drain 到本核 slot 为空。 + // 成功路径复用 orchestration end 作为 final drain start,使两个业务父区间 + // 首尾相接;父记录延后到 final drain 结束再写,避免记录自身落进任一业务 span。 + const uint64_t final_drain_begin = orchestration_end != 0 + ? orchestration_end + : TraceTimestamp(stats.trace, stats.result); TraceAtomicFetchAdd( stats.trace, stats.result, -1, AtomicSite::ReplayDoneIncrement, &state->replay_done.value, 1 @@ -964,6 +979,17 @@ PA_DEVICE void RunSchedulerImpl(PA_GM SchedulerState *state, uint32_t worker_id, } } AtomicPollRegionEnd(stats.trace, stats.result, final_poll_region); + const uint64_t final_drain_end = TraceTimestamp(stats.trace, stats.result); + if (orchestration_begin != 0 && orchestration_end >= orchestration_begin) { + WriteTrace( + stats.trace, stats.result, -1, -1, TracePhase::OrchestrationReplay, + ProfilePhase::Orchestration, orchestration_begin, orchestration_end + ); + } + WriteTrace( + stats.trace, stats.result, -1, -1, TracePhase::FinalDrain, + ProfilePhase::ReplayTail, final_drain_begin, final_drain_end + ); #if !PA_BUILD_SUBMIT_PMU if (stats.trace.atomics_enabled) { diff --git a/tests/atomic_probe/pa_scheduler/pmu_html_report.py b/tests/atomic_probe/pa_scheduler/pmu_html_report.py index 020477f5c2..5933d8e081 100644 --- a/tests/atomic_probe/pa_scheduler/pmu_html_report.py +++ b/tests/atomic_probe/pa_scheduler/pmu_html_report.py @@ -29,12 +29,20 @@ from pmu_sidecar_analyzer import analyze, load_capture -REPORT_VERSION = 2 +REPORT_VERSION = 5 AIC_COLOR = "#2563eb" AIV_COLOR = "#ea580c" GRID_COLOR = "#cbd5e1" TEXT_COLOR = "#334155" +# 本机 A5 受控 cold/warm 校准:PMU cycle 与 1 ns SYS_CNT 同窗读取。 +# ALL 使用整组实测比值;AIC/AIV 使用各自分组的实测比值,避免用名义频率替代证据。 +PMU_CALIBRATION_CYCLE_DELTA = 1_817_457 +PMU_CALIBRATION_SYS_TICK_NS = 1_101_593 +DEFAULT_PMU_CYCLES_PER_NS = 1.649844 +DEFAULT_AIC_PMU_CYCLES_PER_NS = 1.650062 +DEFAULT_AIV_PMU_CYCLES_PER_NS = 1.649731 + def default_output_path(input_path: Path) -> Path: """由描述性 raw 名称稳定推导报告名称。""" @@ -59,6 +67,22 @@ def _format_rate(value: int | float) -> str: return f"{value * 100:.4f}%" +def _cycles_to_us(cycles: int | float, cycles_per_ns: float) -> float: + """按受控校准频率把 PMU cycle 换算为单核等效微秒。""" + + return float(cycles) / cycles_per_ns / 1000.0 + + +def _cycle_time_cell(cycles: int | float, cycles_per_ns: float, decimals: int = 0) -> str: + """同时保留原始 cycle 与校准后的等效时间,避免丢失原始证据。""" + + cycle_text = f"{float(cycles):,.{decimals}f}" + return ( + f'{cycle_text} cycle' + f'≈{_cycles_to_us(cycles, cycles_per_ns):,.3f} µs' + ) + + def _escape(value: object) -> str: return html.escape(str(value), quote=True) @@ -69,29 +93,40 @@ def _raw_href(input_path: Path, output_path: Path) -> str: def _group_metrics( - analysis_group: dict[str, Any], raw_group: dict[str, Any] + analysis_group: dict[str, Any], + group_records: Sequence[dict[str, Any]], + cycles_per_ns: float, ) -> dict[str, int | float]: cores = int(analysis_group["cores"]) total_sum = int(analysis_group["total_cycles_sum"]) scalar_sum = int(analysis_group["scalar_busy_sum"]) + if len(group_records) != cores: + raise ValueError("PMU report group record count does not match analyzer core count") + total_values = [int(record["total_cycles"]) for record in group_records] + scalar_values = [int(record["scalar_busy"]) for record in group_records] + request_values = [int(record["icache_requests"]) for record in group_records] + miss_values = [int(record["icache_misses"]) for record in group_records] return { "cores": cores, "total_sum": total_sum, + "total_min": min(total_values), "total_per_core": total_sum / cores, - "total_median": raw_group["total_cycles"]["median"], - "total_p95": raw_group["total_cycles"]["p95"], + "total_max": max(total_values), + "total_per_core_us": _cycles_to_us(total_sum / cores, cycles_per_ns), "scalar_sum": scalar_sum, + "scalar_min": min(scalar_values), "scalar_per_core": scalar_sum / cores, - "scalar_median": raw_group["scalar_busy"]["median"], - "scalar_p95": raw_group["scalar_busy"]["p95"], + "scalar_max": max(scalar_values), + "scalar_per_core_us": _cycles_to_us(scalar_sum / cores, cycles_per_ns), "scalar_share": 0.0 if total_sum == 0 else scalar_sum / total_sum, "non_scalar_busy_per_core": (total_sum - scalar_sum) / cores, + "cycles_per_ns": cycles_per_ns, + "requests_min": min(request_values), "requests_per_core": analysis_group["icache_requests_per_core"], - "requests_p95": raw_group["icache_requests"]["p95"], + "requests_max": max(request_values), + "misses_min": min(miss_values), "misses_per_core": analysis_group["icache_misses_per_core"], - "misses_median": analysis_group["icache_misses_per_core_median"], - "misses_p95": analysis_group["icache_misses_per_core_p95"], - "misses_max": raw_group["icache_misses"]["max"], + "misses_max": max(miss_values), "miss_rate": analysis_group["icache_miss_rate"], "serial_equivalent_us": analysis_group["first_order_miss_per_core_us"], } @@ -115,7 +150,7 @@ def _distribution_svg( metric: str, title: str, description: str, - p95_by_role: dict[str, float] | None = None, + cycles_per_ns_by_role: dict[str, float] | None = None, ) -> str: """按 physical_core_id 绘制离散点;核编号只是位置,不连接成时间线。""" @@ -169,20 +204,6 @@ def y_position(value: float) -> float: 'text-anchor="middle">physical_core_id(0–107,未连接)' ) - if p95_by_role is not None: - for role, color in (("aic", AIC_COLOR), ("aiv", AIV_COLOR)): - value = p95_by_role[role] - y = y_position(value) - fragments.append( - f'' - ) - fragments.append( - f'{role.upper()} p95 ' - f'{_escape(_plot_label(metric, value))}' - ) - for record, value in sorted(points, key=lambda item: int(item[0]["physical_core_id"])): role = str(record["role"]) physical_id = int(record["physical_core_id"]) @@ -197,6 +218,10 @@ def y_position(value: float) -> float: f"{metric}={_plot_label(metric, value)} whole_request={requests:,} " f"whole_miss={misses:,} whole_rate={rate * 100:.4f}%" ) + if metric in ("total_cycles", "scalar_busy") and cycles_per_ns_by_role is not None: + tooltip += ( + f" calibrated_time={_cycles_to_us(value, cycles_per_ns_by_role[role]):,.3f}us" + ) if role == "aic": fragments.append( f'' @@ -211,13 +236,16 @@ def y_position(value: float) -> float: return "".join(fragments) -def _per_core_rows(records: Sequence[dict[str, Any]]) -> str: +def _per_core_rows( + records: Sequence[dict[str, Any]], cycles_per_ns_by_role: dict[str, float] +) -> str: rows: list[str] = [] for record in sorted(records, key=lambda item: int(item["physical_core_id"])): requests = int(record["icache_requests"]) misses = int(record["icache_misses"]) total = int(record["total_cycles"]) scalar = int(record["scalar_busy"]) + cycles_per_ns = cycles_per_ns_by_role[str(record["role"])] rate = 0.0 if requests == 0 else misses / requests scalar_share = 0.0 if total == 0 else scalar / total exact = bool(record.get("shadow_matches_primary")) @@ -228,8 +256,8 @@ def _per_core_rows(records: Sequence[dict[str, Any]]) -> str: f"" f"" f"" - f"" - f"" + f"" + f"" f"" f"" f"" @@ -252,10 +280,25 @@ def _phase_group_row(label: str, group: dict[str, Any]) -> str: group["phase_icache_miss_lower_bound_share_of_submit"], group["phase_icache_miss_upper_bound_share_of_submit"], ) + time_share = group.get("phase_time_share_of_submit") + time_share_text = "—" if time_share is None else f"{time_share * 100:.4f}%" + phase_time_text = ( + "—" + if group.get("phase_elapsed_per_core_us") is None + else f"{group['phase_elapsed_per_core_us']:,.3f} µs" + ) + per_call_text = ( + "—" + if group.get("phase_elapsed_per_call_ns") is None + else f"{group['phase_elapsed_per_call_ns']:,.3f} ns" + ) return ( "" f"" f"" + f"" + f"" + f"" f"" f"" f"" @@ -316,9 +359,39 @@ def _phase_share_metric( """ +def _phase_time_metric(group_label: str, group: dict[str, Any]) -> str: + """生成所选阶段的逐核累计时间占比;该值是单点观察,不伪造上下界。""" + + share = group.get("phase_time_share_of_submit") + if share is None: + return ( + f'
' + '
阶段时间
' + '
本次 raw 未采集阶段 SYS_CNT,不能由 I-cache 或 PMU total 反推
' + "
" + ) + percent = float(share) * 100.0 + phase_per_core_us = float(group["phase_elapsed_per_core_us"]) + per_call_ns = group.get("phase_elapsed_per_call_ns") + per_call_text = "—" if per_call_ns is None else f"{float(per_call_ns):,.3f} ns/call" + aria_label = f"{group_label} 阶段逐核累计时间占同核完整 Submit {percent:.4f}%" + return f""" +
+
阶段时间{percent:.4f}%
+ +
0%50%100%
+
平均 {phase_per_core_us:,.3f} µs/核 · {per_call_text}
+
+""" + + def _phase_share_card(label: str, group: dict[str, Any]) -> str: - """把一个角色组的 request/miss 局部占比放在同一张响应式卡片中。""" + """把一个角色组的时间/request/miss 局部占比放在同一张响应式卡片中。""" + time_metric = _phase_time_metric(label, group) request_metric = _phase_share_metric( label, "I-cache request", @@ -340,20 +413,83 @@ def _phase_share_card(label: str, group: dict[str, Any]) -> str: role_class = "" if label == "ALL" else f" role-{label.lower()}" return f"""
-
{_escape(label)}{int(group['phase_calls_sum']):,} calls
+
{_escape(label)}{int(group['phase_calls_sum']):,} calls · {group['phase_calls_per_core']:,.0f}/核
+ {time_metric} {request_metric} {miss_metric}
""" +def _pmu_stat_row( + label: str, + stat: str, + total_cycles: int | float, + scalar_cycles: int | float, + cycles_per_ns: float, + decimals: int = 0, +) -> str: + """生成一行紧凑的 PMU 分布统计,同时保留 raw cycle 与校准时间。""" + + return ( + f'' + f"" + f"" + f"" + "" + ) + + +def _pmu_role_card(label: str, group: dict[str, Any]) -> str: + """按角色生成纵向分布卡片,避免横向堆叠十余列。""" + + role_name = label.lower() + role_class = "" if label == "ALL" else f" role-{role_name}" + cycles_per_ns = float(group["cycles_per_ns"]) + rows = "".join( + ( + _pmu_stat_row( + "最小值", "min", group["total_min"], group["scalar_min"], cycles_per_ns + ), + _pmu_stat_row( + "平均值", "mean", group["total_per_core"], group["scalar_per_core"], + cycles_per_ns, 2 + ), + _pmu_stat_row( + "最大值", "max", group["total_max"], group["scalar_max"], cycles_per_ns + ), + ) + ) + return f""" +
+
{_escape(label)}{_format_count(group['cores'])} 核 · {cycles_per_ns:.6f} cycles/ns
+
{_escape(str(record['role']).upper())}{int(record['block_id'])}{int(record['lane'])}{total:,}{scalar:,}{_cycle_time_cell(total, cycles_per_ns)}{_cycle_time_cell(scalar, cycles_per_ns)}{scalar_share * 100:.4f}%{requests:,}{misses:,}
{_escape(label)}{int(group['phase_calls_sum']):,}{phase_time_text}{per_call_text}{time_share_text}{group['phase_icache_requests_lower_bound_sum']:,}..{group['phase_icache_requests_upper_bound_sum']:,}{group['phase_icache_requests_lower_bound_per_core']:,.3f}..{group['phase_icache_requests_upper_bound_per_core']:,.3f}{request_share}
{_escape(label)}{_cycle_time_cell(total_cycles, cycles_per_ns, decimals)}{_cycle_time_cell(scalar_cycles, cycles_per_ns, decimals)}
+ + {rows} +
统计PMU totalscalar busy
+
Σscalar/Σtotal:{_format_rate(group['scalar_share'])}非 Scalar-busy 残余/核:{_cycle_time_cell(group['non_scalar_busy_per_core'], cycles_per_ns, 2)}
+ +""" + + def render_report( input_path: Path, output_path: Path | None = None, miss_penalty_ns: float = 90.0, + pmu_cycles_per_ns: float = DEFAULT_PMU_CYCLES_PER_NS, + aic_pmu_cycles_per_ns: float = DEFAULT_AIC_PMU_CYCLES_PER_NS, + aiv_pmu_cycles_per_ns: float = DEFAULT_AIV_PMU_CYCLES_PER_NS, ) -> str: """先通过独立 analyzer 门禁,再生成一个无需网络的完整 HTML 字符串。""" + for name, value in ( + ("pmu_cycles_per_ns", pmu_cycles_per_ns), + ("aic_pmu_cycles_per_ns", aic_pmu_cycles_per_ns), + ("aiv_pmu_cycles_per_ns", aiv_pmu_cycles_per_ns), + ): + if not math.isfinite(value) or value <= 0: + raise ValueError(f"{name} must be finite and positive") + input_path = Path(input_path) output_path = default_output_path(input_path) if output_path is None else Path(output_path) # 原始件只读取一次;analyzer 与页面元数据都基于同一份私有快照,避免生成期间 @@ -365,18 +501,23 @@ def render_report( snapshot_path.write_bytes(raw_bytes) analysis = analyze([snapshot_path], miss_penalty_ns) capture = load_capture(snapshot_path) - if capture.schema_version != 4: - raise ValueError("HTML report requires submit-pmu schema-v4 input") + if capture.schema_version not in (4, 5): + raise ValueError("HTML report requires submit-pmu schema-v4/v5 input") data = capture.data configuration = data["configuration"] validation = data["validation"] - summary = data["summary"] records = data["records"] + aic_records = [record for record in records if record["role"] == "aic"] + aiv_records = [record for record in records if record["role"] == "aiv"] per_run = analysis["per_run"][0] - all_metrics = _group_metrics(per_run["groups"]["all"], summary["all"]) - aic = _group_metrics(per_run["groups"]["aic"], summary["aic"]) - aiv = _group_metrics(per_run["groups"]["aiv"], summary["aiv"]) + all_metrics = _group_metrics(per_run["groups"]["all"], records, pmu_cycles_per_ns) + aic = _group_metrics(per_run["groups"]["aic"], aic_records, aic_pmu_cycles_per_ns) + aiv = _group_metrics(per_run["groups"]["aiv"], aiv_records, aiv_pmu_cycles_per_ns) + cycles_per_ns_by_role = { + "aic": aic_pmu_cycles_per_ns, + "aiv": aiv_pmu_cycles_per_ns, + } submit_us = float(per_run["submit_span_us"]) aic_request_per_core = float(aic["requests_per_core"]) aiv_request_per_core = float(aiv["requests_per_core"]) @@ -423,37 +564,26 @@ def render_report( "total_cycles", "逐物理核 PMU total cycle", "每核 Submit gate 内的 PMU raw total;96 核求和是 core-work,不是墙钟时间。", - { - "aic": float(summary["aic"]["total_cycles"]["p95"]), - "aiv": float(summary["aiv"]["total_cycles"]["p95"]), - }, + cycles_per_ns_by_role=cycles_per_ns_by_role, ) scalar_plot = _distribution_svg( records, "scalar_busy", "逐物理核 scalar busy cycle", "CNT2 scalar_instr_busy(0x001) 的每核累计;它不包含全部等待周期。", - { - "aic": float(summary["aic"]["scalar_busy"]["p95"]), - "aiv": float(summary["aiv"]["scalar_busy"]["p95"]), - }, + cycles_per_ns_by_role=cycles_per_ns_by_role, ) request_plot = _distribution_svg( records, "icache_requests", "逐物理核 I-cache request", "96 个实测物理子核的 request 离散分布;AIC 为圆点,AIV 为方点。", - { - "aic": float(summary["aic"]["icache_requests"]["p95"]), - "aiv": float(summary["aiv"]["icache_requests"]["p95"]), - }, ) miss_plot = _distribution_svg( records, "icache_misses", "逐物理核 I-cache miss", - "96 个实测物理子核的 miss 离散分布;虚线是各角色 nearest-rank p95。", - {"aic": float(aic["misses_p95"]), "aiv": float(aiv["misses_p95"])}, + "96 个实测物理子核的 miss 离散分布;AIC 为圆点,AIV 为方点。", ) rate_plot = _distribution_svg( records, @@ -461,13 +591,26 @@ def render_report( "逐物理核 I-cache miss rate", "每核 miss/request,仅用于观察离散分布;汇总 rate 仍按总 miss 除以总 request。", ) - per_core_rows = _per_core_rows(records) + per_core_rows = _per_core_rows(records, cycles_per_ns_by_role) + pmu_role_cards = "".join( + ( + _pmu_role_card("ALL", all_metrics), + _pmu_role_card("AIC", aic), + _pmu_role_card("AIV", aiv), + ) + ) if phase == "none": shadow_badge = f"PRIMARY ↔ SHADOW EXACT {exact_records}/{workers}" + phase_front_section = """ +
+

局部阶段总览:none

+

不适用:该 ELF 未编译局部阶段。request、miss 和阶段时间都不能作为某个局部阶段的 0% 结果;本报告只提供完整 Submit 基准。

+
+""" phase_section = """

局部 phase:none

-

完整 Submit 中未执行任何 running read-clear 边界;局部 calls、request 和 miss 全部为 0。此构建是完整 Submit I-cache 的主观察口径。

+

未执行 running read-clear 或阶段 SYS_CNT 边界;raw 中局部字段按契约为 0,但语义是“未选择阶段”,不是某个阶段实测为 0。

""" else: @@ -480,15 +623,18 @@ def render_report( _phase_share_card("AIV", phase_groups["aiv"]), ) ) + phase_front_section = f""" +
+

局部阶段总览:{_escape(phase)}

+

时间占比是 Σ阶段 SYS_CNT / Σ同核首个 submit_begin 计时点到末个 submit_end 计时点 SYS_CNT,分别在 ALL/AIC/AIV 内先求和再相除。首个 submit_begin 位于 BeginSubmit 上下文初始化之后,末个 submit_end 位于返回之前。它是逐核累计 core-time 构成,不是该阶段占全局约 5 ms 墙钟的切片。request/miss 仍显示同一 ELF primary-shadow 形成的下界..上界;阶段时间是单点观察值。

+
{phase_share_cards}
+
+""" phase_plot = _distribution_svg( records, "phase_icache_misses", f"{phase} 局部 I-cache miss 观测下界", "running read-clear 的逐核观测下界;上界还需加该核 primary-shadow residual。", - { - "aic": float(summary["aic"]["phase_icache_misses"]["p95"]), - "aiv": float(summary["aiv"]["phase_icache_misses"]["p95"]), - }, ) phase_rows = "".join( ( @@ -498,17 +644,14 @@ def render_report( ) ) phase_section = f""" -

局部 phase:{_escape(phase)}

+

局部阶段详细数据:{_escape(phase)}

-
带边界扰动的 running read-clear 区间。下界是直接观测值,上界是下界加本核 primary-shadow residual;不同 phase ELF 的局部值不能相加,也不能从 none 相减得到无扰动净值。
-

局部占同一 ELF 完整 Submit primary

-

每条灰色底轨代表该组完整 Submit 的 100%;深色为直接观测下界,浅色延伸到保守上界,空白部分属于局部窗口之外。request 与 miss 使用各自的完整窗口事件数作分母。

-
{phase_share_cards}
+
带边界扰动的诊断区间。每次调用新增两次 SYS_CNT;时间起点位于 begin shadow read-clear 之后,终点位于 end shadow read-clear 之前,因此不包含两侧 ld_dev,但包含时间戳边界本身的扰动。request/miss 下界是直接观测值,上界是下界加本核 primary-shadow residual;不同 phase ELF 的局部值不能相加,也不能从 none 相减得到无扰动净值。
展开 ALL / AIC / AIV 完整数字表
- + {phase_rows}
分组callsrequest sum 下界..上界request/core 下界..上界局部 request / Submit primarymiss sum 下界..上界miss/core 下界..上界局部 miss / Submit primaryshadow loss req/missobserved miss/request
分组calls阶段时间/core阶段时间/call阶段 core-time / Submitrequest sum 下界..上界request/core 下界..上界局部 request / Submit primarymiss sum 下界..上界miss/core 下界..上界局部 miss / Submit primaryshadow loss req/missobserved miss/request
@@ -536,10 +679,12 @@ def render_report( .badges {{ display:flex; flex-wrap:wrap; gap:8px; margin:16px 0; }} .badge {{ border:1px solid #86efac; background:#f0fdf4; color:var(--pass); border-radius:999px; padding:4px 10px; font-weight:700; font-size:.86rem; }} .cards {{ display:grid; grid-template-columns:repeat(auto-fit,minmax(190px,1fr)); gap:12px; }} - .card,.panel {{ background:white; border:1px solid var(--line); border-radius:12px; box-shadow:0 4px 18px #0f172a0d; }} + .card,.panel {{ min-width:0; background:white; border:1px solid var(--line); border-radius:12px; box-shadow:0 4px 18px #0f172a0d; }} .card {{ padding:16px; }} .card .label {{ color:var(--muted); font-size:.86rem; }} .card .value {{ margin-top:4px; font-size:1.55rem; font-weight:750; font-variant-numeric:tabular-nums; }} + .cycle-value,.cycle-time {{ display:block; }} + .cycle-time {{ color:var(--muted); font-size:.82rem; }} .panel {{ padding:18px; overflow-x:auto; }} table {{ width:100%; border-collapse:collapse; font-variant-numeric:tabular-nums; }} th,td {{ padding:9px 11px; text-align:right; border-bottom:1px solid #e2e8f0; white-space:nowrap; }} @@ -547,9 +692,26 @@ def render_report( thead th {{ background:#f8fafc; color:#475569; font-size:.82rem; text-transform:none; }} .role {{ display:inline-block; min-width:42px; text-align:center; border-radius:999px; color:white; padding:2px 7px; font-size:.78rem; font-weight:700; }} .role-aic {{ background:var(--aic); }} .role-aiv {{ background:var(--aiv); }} + .pmu-role-grid {{ display:grid; grid-template-columns:repeat(auto-fit,minmax(min(330px,100%),1fr)); gap:12px; width:100%; }} + .pmu-role-card {{ min-width:0; padding:14px; border:1px solid #dbe3ee; border-radius:10px; background:#f8fafc; }} + .pmu-role-title {{ display:flex; flex-wrap:wrap; align-items:center; justify-content:space-between; gap:8px; margin-bottom:10px; color:var(--muted); font-size:.82rem; font-variant-numeric:tabular-nums; }} + .pmu-role-label {{ display:inline-block; min-width:44px; padding:3px 9px; border-radius:999px; background:#334155; color:white; text-align:center; font-weight:800; }} + .pmu-role-label.role-aic {{ background:var(--aic); }} .pmu-role-label.role-aiv {{ background:var(--aiv); }} + .pmu-compact-table {{ table-layout:fixed; font-size:.86rem; }} + .pmu-compact-table th,.pmu-compact-table td {{ padding:7px 6px; white-space:normal; vertical-align:top; }} + .pmu-compact-table th:first-child {{ width:23%; }} + .pmu-compact-table .cycle-value,.pmu-compact-table .cycle-time {{ white-space:nowrap; }} + .pmu-role-foot {{ display:grid; gap:5px; margin-top:10px; color:#475569; font-size:.82rem; }} + .pmu-role-foot .cycle-value,.pmu-role-foot .cycle-time {{ display:inline; white-space:nowrap; }} + .pmu-role-foot .cycle-time {{ margin-left:5px; }} + .table-scroll {{ width:100%; max-width:100%; overflow-x:auto; overscroll-behavior-x:contain; }} + .icache-table {{ table-layout:fixed; min-width:760px; }} + .icache-table th,.icache-table td {{ padding:7px 6px; white-space:normal; overflow-wrap:anywhere; }} .insight {{ margin-top:12px; padding:12px 14px; border-left:4px solid var(--aiv); background:#fff7ed; }} .phase-warning {{ margin-bottom:14px; padding:12px 14px; border-left:4px solid #7c3aed; background:#f5f3ff; }} .phase-panel {{ overflow:hidden; }} + .phase-overview-front {{ margin:0 0 20px; overflow:hidden; }} + .phase-overview-front h2 {{ margin-top:0; }} .phase-share-title {{ margin:20px 0 5px; }} .phase-share-note {{ max-width:900px; margin:0 0 14px; color:var(--muted); }} .phase-share-grid {{ display:grid; grid-template-columns:repeat(auto-fit,minmax(min(280px,100%),1fr)); gap:12px; width:100%; }} @@ -564,6 +726,7 @@ def render_report( .phase-share-upper,.phase-share-lower {{ position:absolute; inset:0 auto 0 0; }} .share-request .phase-share-upper {{ background:#93c5fd; }} .share-request .phase-share-lower {{ background:#2563eb; }} .share-miss .phase-share-upper {{ background:#c4b5fd; }} .share-miss .phase-share-lower {{ background:#7c3aed; }} + .phase-time-value {{ position:absolute; inset:0 auto 0 0; background:#0f766e; }} .phase-share-upper-marker {{ position:absolute; top:-1px; bottom:-1px; width:2px; transform:translateX(-1px); background:#0f172a; opacity:.75; }} .phase-share-axis {{ display:grid; grid-template-columns:repeat(3,1fr); margin-top:2px; color:#64748b; font-size:.68rem; }} .phase-share-axis span:nth-child(2) {{ text-align:center; }} .phase-share-axis span:last-child {{ text-align:right; }} @@ -572,18 +735,17 @@ def render_report( .phase-table-details {{ margin-top:18px; }} .phase-table-details summary {{ padding-bottom:8px; }} .phase-table-scroll {{ width:100%; max-width:100%; overflow-x:auto; border:1px solid #dbe3ee; border-radius:10px; overscroll-behavior-x:contain; }} - .phase-table {{ min-width:1240px; }} + .phase-table {{ min-width:1540px; }} .phase-table th,.phase-table td {{ padding:8px 10px; }} .phase-plot-scroll {{ width:100%; max-width:100%; overflow-x:auto; overscroll-behavior-x:contain; }} - .phase-disabled {{ margin-top:34px; }} .phase-disabled h2 {{ margin-top:0; }} + .phase-disabled {{ margin-top:34px; }} .phase-overview-front.phase-disabled {{ margin-top:0; }} .phase-disabled h2 {{ margin-top:0; }} .legend {{ display:flex; gap:18px; align-items:center; color:#475569; font-size:.88rem; margin-bottom:8px; }} .circle-key {{ width:10px; height:10px; border-radius:50%; background:var(--aic); }} .square-key {{ width:10px; height:10px; background:var(--aiv); }} .distribution {{ width:100%; min-width:760px; height:auto; display:block; }} .grid {{ stroke:{GRID_COLOR}; stroke-width:1; }} .tick {{ stroke:#64748b; }} - .axis-label,.axis-title,.p95-label {{ fill:{TEXT_COLOR}; font-size:12px; }} - .axis-title {{ font-size:13px; font-weight:650; }} .p95-label {{ font-weight:700; }} - .p95-line {{ stroke-width:1.5; stroke-dasharray:7 5; opacity:.75; }} + .axis-label,.axis-title {{ fill:{TEXT_COLOR}; font-size:12px; }} + .axis-title {{ font-size:13px; font-weight:650; }} .aic-point {{ fill:var(--aic); }} .aiv-point {{ fill:var(--aiv); }} .point {{ opacity:.82; outline:none; }} .point:hover,.point:focus {{ opacity:1; stroke:#020617; stroke-width:2; }} .warning {{ border:2px dashed #94a3b8; background:#f8fafc; border-radius:12px; padding:18px; }} @@ -592,8 +754,8 @@ def render_report( a {{ color:#1d4ed8; }} code {{ overflow-wrap:anywhere; }} footer {{ margin-top:32px; color:#475569; font-size:.86rem; }} :focus-visible {{ outline:3px solid #0ea5e9; outline-offset:2px; }} - @media (max-width:640px) {{ main {{ width:min(100% - 16px,1180px); margin-top:12px; }} .panel {{ padding:13px; }} .phase-share-grid {{ grid-template-columns:1fr; }} }} - @media print {{ body {{ background:white; }} main {{ width:100%; margin:0; }} .card,.panel {{ box-shadow:none; break-inside:avoid; }} details {{ display:block; }} .phase-share-upper,.phase-share-lower,.phase-share-upper-marker {{ print-color-adjust:exact; -webkit-print-color-adjust:exact; }} }} + @media (max-width:640px) {{ main {{ width:min(100% - 16px,1180px); margin-top:12px; }} .panel {{ padding:13px; }} .phase-share-grid,.pmu-role-grid {{ grid-template-columns:1fr; }} }} + @media print {{ body {{ background:white; }} main {{ width:100%; margin:0; }} .card,.panel {{ box-shadow:none; break-inside:avoid; }} details {{ display:block; }} .phase-share-upper,.phase-share-lower,.phase-share-upper-marker,.phase-time-value {{ print-color-adjust:exact; -webkit-print-color-adjust:exact; }} }} @media (prefers-reduced-motion:reduce) {{ * {{ scroll-behavior:auto !important; }} }} @@ -610,41 +772,45 @@ def render_report(
+ {phase_front_section} +
-
完整 Submit
{submit_us / 1000:.6f} ms
-
ALL PMU total/core mean
{_format_per_core(all_metrics['total_per_core'])}
raw cycle;Σ={_format_count(all_metrics['total_sum'])}
-
ALL scalar busy/core mean
{_format_per_core(all_metrics['scalar_per_core'])}
Σscalar/Σtotal={_format_rate(all_metrics['scalar_share'])}
-
完整 Submit primary I-cache request
{total_requests:,}
-
完整 Submit primary I-cache miss
{total_misses:,}
+
完整 Submit(最早开始 → 最晚结束)
{submit_us / 1000:.6f} ms
96 核整体 Submit 耗时
+
96 核逐核 PMU total 平均值(校准)
{float(all_metrics['total_per_core_us']):,.3f} µs
最小 {_cycles_to_us(all_metrics['total_min'], pmu_cycles_per_ns):,.3f} µs · 最大 {_cycles_to_us(all_metrics['total_max'], pmu_cycles_per_ns):,.3f} µs
{_format_per_core(all_metrics['total_per_core'])} raw cycle/核
+
96 核逐核 scalar busy 平均值(校准)
{float(all_metrics['scalar_per_core_us']):,.3f} µs
最小 {_cycles_to_us(all_metrics['scalar_min'], pmu_cycles_per_ns):,.3f} µs · 最大 {_cycles_to_us(all_metrics['scalar_max'], pmu_cycles_per_ns):,.3f} µs
Σscalar/Σtotal={_format_rate(all_metrics['scalar_share'])}
+
完整 Submit primary I-cache request(96 核总和)
{total_requests:,}
逐核平均 {_format_per_core(all_metrics['requests_per_core'])}
逐核最小 {_format_count(all_metrics['requests_min'])} · 逐核最大 {_format_count(all_metrics['requests_max'])}
+
完整 Submit primary I-cache miss(96 核总和)
{total_misses:,}
逐核平均 {_format_per_core(all_metrics['misses_per_core'])}
逐核最小 {_format_count(all_metrics['misses_min'])} · 逐核最大 {_format_count(all_metrics['misses_max'])}
聚合 miss rate
{_format_rate(float(all_metrics['miss_rate']))}
实测物理子核
{workers}(32 AIC + 64 AIV)
Primary/Shadow
exact {exact_records}/{workers}
bounded {bounded_records}/{workers}
+

PMU cycle 频率校准

+
+

本机受控 cold/warm 同窗证据:PMU cycle_delta = {PMU_CALIBRATION_CYCLE_DELTA:,},1 ns SYS_CNT tick_delta = {PMU_CALIBRATION_SYS_TICK_NS:,} ns;二者相除为 {PMU_CALIBRATION_CYCLE_DELTA / PMU_CALIBRATION_SYS_TICK_NS:.6f} cycles/ns(约 1.65 GHz)。

+

当前报告换算频率:ALL = {pmu_cycles_per_ns:.6f}、AIC = {aic_pmu_cycles_per_ns:.6f}、AIV = {aiv_pmu_cycles_per_ns:.6f} cycles/ns。公式:等效 µs = PMU cycles / (cycles/ns) / 1000

+

SYS_CNT 仍按 1 ns/tick 解释;表中的 PMU 时间是每个物理子核 gate 内 cycle 的校准等效时间。它不是 96 核 cycle 求和后的墙钟时间,也不替代独立记录的完整 Submit 墙钟。

+
+

PMU total 与 scalar busy

- - - - - - - -
角色核数total/core meantotal mediantotal p95scalar/core meanscalar medianscalar p95Σscalar/Σtotal非 Scalar-busy 残余/core
ALL{_format_count(all_metrics['cores'])}{_format_per_core(all_metrics['total_per_core'])}{_format_count(all_metrics['total_median'])}{_format_count(all_metrics['total_p95'])}{_format_per_core(all_metrics['scalar_per_core'])}{_format_count(all_metrics['scalar_median'])}{_format_count(all_metrics['scalar_p95'])}{_format_rate(all_metrics['scalar_share'])}{_format_per_core(all_metrics['non_scalar_busy_per_core'])}
AIC{_format_count(aic['cores'])}{_format_per_core(aic['total_per_core'])}{_format_count(aic['total_median'])}{_format_count(aic['total_p95'])}{_format_per_core(aic['scalar_per_core'])}{_format_count(aic['scalar_median'])}{_format_count(aic['scalar_p95'])}{_format_rate(aic['scalar_share'])}{_format_per_core(aic['non_scalar_busy_per_core'])}
AIV{_format_count(aiv['cores'])}{_format_per_core(aiv['total_per_core'])}{_format_count(aiv['total_median'])}{_format_count(aiv['total_p95'])}{_format_per_core(aiv['scalar_per_core'])}{_format_count(aiv['scalar_median'])}{_format_count(aiv['scalar_p95'])}{_format_rate(aiv['scalar_share'])}{_format_per_core(aiv['non_scalar_busy_per_core'])}
-
口径:total 是每个物理子核在 Submit gate 内的 PMU raw total cycle;求和代表 96 核 core-work。scalar busy 是 CNT2 scalar_instr_busy(0x001)。表中的“非 Scalar-busy 残余”严格等于 total−scalar busy,它不是空闲时间,也不是 I-cache stall,其中还混有同步等待、engine 等待及其他未归因周期。受控微基准已观察到依赖返回的 atomic 等待大部分进入 scalar busy,而 I-cache refill 的额外周期大部分只进入 total。当前 A5/DAV3510 正式事件表和 CANN 9.1 上板输出均不提供 scalar_wait_ib_time/scalar_wait_time,报告不会用其他产品的 selector 猜测这两项。
+
{pmu_role_cards}
+
口径:每张卡只保留最小值、平均值和最大值。total 与 scalar busy 的最小/最大值分别从各自逐核分布独立取得,不保证来自同一个物理核,也不能相减成配对差值。total 是每个物理子核在 Submit gate 内的 PMU raw total cycle;求和代表 96 核 core-work。页面始终保留 raw cycle,旁边的 µs 只按本机实测频率换算。逐核最大值用于观察慢核,但仍不等同于“最早开始至最晚结束”的完整 Submit。scalar busy 是 CNT2 scalar_instr_busy(0x001)。卡片中的“非 Scalar-busy 残余”严格等于 total−scalar busy,它不是空闲时间,也不是 I-cache stall,其中还混有同步等待、engine 等待及其他未归因周期。受控微基准已观察到依赖返回的 atomic 等待大部分进入 scalar busy,而 I-cache refill 的额外周期大部分只进入 total。当前 A5/DAV3510 正式事件表和 CANN 9.1 上板输出均不提供 scalar_wait_ib_time/scalar_wait_time,报告不会用其他产品的 selector 猜测这两项。

PMU total cycle

{total_plot}

Scalar busy cycle

{scalar_plot}

AIC 与 AIV 的 I-cache 对比

- - +
+
角色核数request/corerequest p95miss/core meanmiss/core medianmiss p95miss maxΣmiss/Σrequest
+ - - + +
角色核数request min/corerequest mean/corerequest max/coremiss min/coremiss mean/coremiss max/coreΣmiss/Σrequest
AIC{_format_count(aic['cores'])}{_format_per_core(aic['requests_per_core'])}{_format_count(aic['requests_p95'])}{_format_per_core(aic['misses_per_core'])}{_format_count(aic['misses_median'])}{_format_count(aic['misses_p95'])}{_format_count(aic['misses_max'])}{_format_rate(aic['miss_rate'])}
AIV{_format_count(aiv['cores'])}{_format_per_core(aiv['requests_per_core'])}{_format_count(aiv['requests_p95'])}{_format_per_core(aiv['misses_per_core'])}{_format_count(aiv['misses_median'])}{_format_count(aiv['misses_p95'])}{_format_count(aiv['misses_max'])}{_format_rate(aiv['miss_rate'])}
AIC{_format_count(aic['cores'])}{_format_count(aic['requests_min'])}{_format_per_core(aic['requests_per_core'])}{_format_count(aic['requests_max'])}{_format_count(aic['misses_min'])}{_format_per_core(aic['misses_per_core'])}{_format_count(aic['misses_max'])}{_format_rate(aic['miss_rate'])}
AIV{_format_count(aiv['cores'])}{_format_count(aiv['requests_min'])}{_format_per_core(aiv['requests_per_core'])}{_format_count(aiv['requests_max'])}{_format_count(aiv['misses_min'])}{_format_per_core(aiv['misses_per_core'])}{_format_count(aiv['misses_max'])}{_format_rate(aiv['miss_rate'])}
+
{_escape(request_comparison)};{_escape(miss_comparison)};聚合 {_escape(rate_comparison)}。
@@ -669,12 +835,14 @@ def render_report(
展开逐核表格 +
- + {per_core_rows}
workerphysicalroleblocklanePMU totalscalar busyscalar/totalrequestsmissesper-core rateprimary=shadowtrusted
workerphysicalroleblocklanePMU total
cycle / 等效 µs
scalar busy
cycle / 等效 µs
scalar/totalrequestsmissesper-core rateprimary=shadowtrusted
+
-

聚合 miss rate 使用 Σmiss/Σrequest,不平均逐核百分比。p95 使用 nearest-rank:ceil(0.95 × N)。

+

聚合 miss rate 使用 Σmiss/Σrequest,不平均逐核百分比。request/miss 的 min、mean、max 都从同组逐核 raw 计算;request 与 miss 的极值不保证来自同一个物理核。