Skip to content

[NPU] add A5/ascend950 support for A2E and E2A ops - #295

Open
wangyibo1005 wants to merge 1 commit into
vllm-project:mainfrom
wangyibo1005:main
Open

[NPU] add A5/ascend950 support for A2E and E2A ops#295
wangyibo1005 wants to merge 1 commit into
vllm-project:mainfrom
wangyibo1005:main

Conversation

@wangyibo1005

@wangyibo1005 wangyibo1005 commented Sep 1, 2026

Copy link
Copy Markdown

Register ascend950, gate SetCommEngine behind the 950 host build, and read A5 HCCL windows from the flat windowsIn array so 910C stays on the existing remoteRes path.

PLEASE FILL IN THE PR DESCRIPTION AND MAKE SURE THE CHECKLIST ITEMS HAVE BEEN CONSIDERED.

Purpose

Issue

  • Related issue(s): #
  • Closing keyword, only if fully resolved: Closes #

Scope

  • In scope:
  • Out of scope:

Implementation Notes

Test Plan

Test Result

Docs Impact

  • Files updated:
  • If none, reason:
Essential PR Checklist
  • Purpose is clear and linked to public context when possible.
  • Scope is bounded.
  • Compatibility with vLLM v0.26.0 is considered.
  • No changes are made to the vLLM source checkout.
  • Plugin-owned classes or explicit dotted class paths are preferred over monkey patches.
  • Any compat shim or monkey patch is isolated, idempotent, version-guarded, documented, and tested.
  • Imports remain CPU-safe; CUDA-heavy work is delayed or GPU-gated.
  • Validation evidence is included, including skipped GPU tests when applicable.
  • Documentation impact is stated.

@wangyibo1005
[NPU] add A5/ascend950 support for A2E and E2A ops
364d358 
Register ascend950, gate SetCommEngine behind the 950 host build, and
read A5 HCCL windows from the flat windowsIn array so 910C stays on the
existing remoteRes path.

Signed-off-by: wangyibo1005 <2633333316@qq.com>
ksiyuan added a commit to ksiyuan/afd-plugin that referenced this pull request Sep 1, 2026
@ksiyuan
debug(A5): surface latched device error and dump HCCL window addresses
22ebf96 
TEMP diagnostics for the A5 device error (507035) seen right after the
a2e/e2a ops in profile_run:

- camp2p.py: add torch.npu.synchronize() after each a2e/e2a call so a
  latched illegal-memory-access from the A5 window ABI surfaces at the
  call site instead of at the next unrelated device op. Marked TEMP and
  to be removed after debugging (breaks ACL graph capture).
- a2e.h/e2a.h: guarded (AFD_A5_DUMP_WINDOWS, off by default) device-side
  print of HcclCombinOpParam rankId/rankDim/winSize/workSpace and the
  windowsIn/windowsOut arrays, to validate whether windowsIn[rank] needs
  the +1MB state offset (PR vllm-project#276) or not (PR vllm-project#295).
ksiyuan added a commit to ksiyuan/afd-plugin that referenced this pull request Sep 1, 2026
@ksiyuan
revert(A5): restore sendX flag write to remote window (undo 784a58e)
c11bd6d 
The 784a58e experiment redirected the a2e sendX flag to the LOCAL window,
which made the attention side pass (no 507035) but the FFN side hang
waiting for the flag. Restore the original remote-window write so the
kernel matches PR vllm-project#295; keep the debug dumps/traces.
ksiyuan added a commit to ksiyuan/afd-plugin that referenced this pull request Sep 1, 2026
@ksiyuan
feat(npu): A5 (Ascend950) support for CAMP2p via HCCL p2p (Route B2)
3770266 
a2e/e2a custom ops and native MC2 dispatch/combine both fail on the AFD mixed group on A5 (507035 MTE out-of-range; remote HCCL windows unreachable). Replace the four a2e/e2a call sites on A5 with plain dist.send/recv over the per-ubatch 'afd' HCCL groups. Validated by tools/npu_a5_p2p_probe.py (2-rank round trip MATCH=True) and a 2-rank eager 1A+1F DeepSeek-V2-Lite completion smoke on Ascend 950PR.

- camp2p.py: A5 branches in send_attn_output/recv_ffn_output/recv_attn_output/send_ffn_output; init_afd_connector skips the custom-op loader on A5 (a2e/e2a register only ascend910_93).
- camp2p_a5.py (new): is_a5(), contiguous rank mapping (attention i -> ffn i//(attn//ffn), matching _num_tokens_for_ffn_rank), per-peer DP token counts, p2p primitives; unit tests added.
- Gate stays on FFN (only hidden_states crosses the wire). 910C custom-op path unchanged.
- docs/npu/A5_ADAPTATION.md: decision + troubleshooting record (custom-op 507035 root cause, native MC2 mixed-group exclusion, B2 p2p).
- Known gaps: ACL graph capture, DBO, multi-rank validation, 910C regression.

csrc/ is back at the 910C-only upstream state (PR vllm-project#295 ascend950 support reverted), so this commit is pure Python/docs.
@ksiyuan ksiyuan mentioned this pull request Sep 1, 2026
Open
ksiyuan added a commit to ksiyuan/afd-plugin that referenced this pull request Sep 3, 2026
@ksiyuan
docs(npu): restore A5 investigation record and p2p probe
cbf0562 
Recovered from tag backup/a5-full-record so the a5-custom-op-research
branch carries the prior 507035 investigation (driver-log evidence,
PR vllm-project#276/vllm-project#295 timeline) and the Route B2 HCCL p2p carrier probe.

Signed-off-by: ksiyuan <ksiyuan@umich.edu>
ksiyuan added a commit to ksiyuan/afd-plugin that referenced this pull request Sep 3, 2026
@ksiyuan
feat(npu): default A5 a2e/e2a to the A3 remoteRes-tree window ABI
4f26513 
PR vllm-project#276/vllm-project#295 read A5 remote IPC windows from a flat HcclCombinOpParam
windowsIn[] array. That faults with 507035 (MTE out-of-range) on every
remote-window access while local access works - the fingerprint of
reading windowsIn[k] out of a struct that is actually the A3
HcclOpResParam tree (windowsIn[0] aliases localWindowsIn, windowsIn[>=1]
aliases localWindowsOut / hcomId bytes). vLLM-Ascend's own MC2 op
dispatch_ffn_combine reads peer windows from the remoteRes tree on every
arch including arch35, so switch A5 to the same tree path while keeping
PR vllm-project#295's host-side SetCommEngine(3) + AddConfig(ascend950) tiling.

- a2e.h/e2a.h/comm_args.h: A5 default is the tree path; define
  AFD_A5_FLAT_WINDOW_ABI to restore the flat interpretation.
- AFD_A5_DUMP_WINDOWS prints both interpretations of the HCCL context on
  block 0 so the ABI can be settled from one run.
- op_host/CMakeLists.txt forwards both toggles from the environment to
  the AscendC kernel compile.
- 910C path is untouched (AFD_ARCH_A5 undefined -> toggles inert).
- docs/npu/A5_custom_op_investigation.md: hypothesis, the untested
  tree-ABI + working-host-config cell, and the A5-node test sequence.

Needs an A5-node build + 2-rank eager smoke to verify.

Signed-off-by: ksiyuan <ksiyuan@umich.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hsliuustc0106 hsliuustc0106 Awaiting requested review from hsliuustc0106 hsliuustc0106 is a code owner
@jiangkuaixue123 jiangkuaixue123 Awaiting requested review from jiangkuaixue123 jiangkuaixue123 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@wangyibo1005