[RFC]: Add Ascend NPU ModelRunnerV2 support for AFD

[jiangkuaixue123]

Motivation

AFD currently supports Ascend NPU execution through vLLM-Ascend ModelRunner V1. The synchronous NPU path also carries substantial plugin-owned uBatch/DBO logic for batch splitting, Attention metadata, forward contexts, streams, threads, ACL graph capture/replay, DP coordination, and output merging. This works, but it creates a large upgrade and maintenance surface.

The upstream direction has changed since the earlier investigation in #171:

• vLLM PR #50945 proposes eager DBO support for ModelRunner V2 through a dedicated UBatchRunner, persistent per-uBatch buffers, pre-sliced InputBatch objects, per-uBatch Attention metadata builders, and num_ubatches as part of BatchExecutionDescriptor.
• vLLM PR #51700 follows with FULL CUDA graph capture/replay for microbatched steps.
• [RFC]: Add GPU ModelRunnerV2 Support for AFD #223 separately tracks GPU ModelRunner V2 support for AFD.

These changes make it possible to pursue an MRV2-native Ascend design without porting the V1 execution loop or continuing to grow AFD-owned copies of vLLM-Ascend internals.

This RFC proposes staged Ascend NPU ModelRunner V2 support. The synchronous CAMP2pAFDConnector path comes first, followed by CAMAsyncAFDConnector. Each phase must preserve native vLLM/vLLM-Ascend state ownership and pass its own architecture, correctness, E2E, and accuracy gates before the next phase starts.

Related work:

• [RFC]: Evaluate ModelRunnerV2 adoption for AFD Attention runners and DBO #171 — earlier MRV2 adoption investigation
• [RFC]: Add GPU ModelRunnerV2 Support for AFD #223 — GPU ModelRunner V2 support
• [DeepSeek V4][NPU] Support CAMAsyncAFDConnector #227 — DeepSeek V4 with CAMAsyncAFDConnector
• [DeepSeek V4][NPU] Support CAMP2pAFDConnector #228 — DeepSeek V4 with CAMP2pAFDConnector

Goals

• Add a maintainable Ascend NPU ModelRunner V2 path for AFD.
• Reuse native MRV2 request state, InputBatch, Attention preparation, sampling, and graph ownership.
• Support CAMP2pAFDConnector first with DeepSeek-V2-Lite eager and FULL_DECODE_ONLY execution.
• Add an Ascend-native AscendUBatchRunner following the architecture introduced by vLLM #50945 instead of copying the V1 NPU runner.
• Add CAMAsyncAFDConnector support after the synchronous path is stable.
• Keep native DBO and CAMAsync's plugin-managed MoE-only ubatching as separate execution policies.
• Retain ModelRunner V1 until the selected MRV2 feature matrix is fully qualified.

Non-goals

• This RFC does not immediately remove ModelRunner V1.
• It does not claim support for every MRV2 model, graph mode, speculative method, parallelism mode, or connector.
• It does not modify vLLM or vLLM-Ascend source trees from the AFD repository.
• It does not accept a permanent implementation that copies MRV2 execute_model, input preparation, Attention metadata construction, sampling, or graph-manager lifecycle.
• It does not treat successful import, model construction, or server startup as feature validation.
• It does not merge CAMAsync MoE-only ubatching with native DBO or enable it through native DBO flags.

Proposed architecture

vLLM ModelRunner V2 uBatch contract
  BatchExecutionDescriptor / InputBatch / ModelState / UBatchRunner
                              |
                              v
vLLM-Ascend AscendUBatchRunner
  NPU streams/events, Ascend forward context, ACL graph, NPU resource control
                              |
                              v
AFD composition/hooks
  connector lifecycle, AFD metadata, control payloads, stage identity and cleanup

The intended ownership is:

vLLM-owned

• persistent request state and top-level scheduler updates;
• InputBatch and model inputs;
• BatchExecutionDescriptor.num_ubatches;
• generic split decisions and DP agreement where the upstream contract applies;
• per-uBatch ModelState.prepare_attn(..., ubatch_idx=...);
• top-level output merging and one sampling/output step.

vLLM-Ascend-owned

• AscendUBatchRunner;
• NPU stream/event/context switching;
• Ascend Attention metadata and per-uBatch metadata builders;
• ACL graph capture/replay integration;
• FlashComm/SP and NPU padding requirements;
• NPU compute/communication resource control;
• device-specific workspace ownership.

AFD-owned

• CAMP2pAFDConnector and CAMAsyncAFDConnector lifecycle;
• AFD transaction and stage metadata;
• control-plane payload publication outside graph bodies;
• Attention/FFN role coordination and connector transfer ordering;
• connector-specific DP metadata when native DP topology does not describe the disaggregated roles;
• CAMAsync's separate MoE-only asynchronous stage policy;
• AFD-specific validation, profiling, failure cleanup, recipes, and E2E coverage.

A small upstream/vLLM-Ascend extension should be preferred over an AFD copy. In particular, an overridable uBatch-runner factory and hooks for stream/context/resource-control construction would keep the Ascend implementation narrow.

Staged delivery

Each phase is a separate reviewable change. A later phase must not begin until the previous phase passes its required native control and AFD validation.

Phase 0 — Freeze runtime contracts and feature matrix

Before production changes:

• resolve the exact target vLLM and vLLM-Ascend refs to immutable SHAs;
• verify that they are a supported runtime pair;
• record Python, PyTorch, torch_npu, CANN, Triton Ascend, and operator-package requirements;
• validate native vLLM-Ascend MRV2 with the selected DeepSeek-V2-Lite checkpoint;
• map current NPU V1 overrides to MRV2 owners or missing hooks;
• define the initial topology, quantization, eager/graph modes, and unsupported combinations;
• keep V1 and V2 selection explicit and fail closed.

Phase 1 — CAMP2P + DeepSeek-V2-Lite eager

Add the first no-DBO MRV2 path:

• DeepSeek-V2-Lite;
• CAMP2pAFDConnector;
• eager execution;
• Attention and FFN role construction through the native Ascend MRV2 model lifecycle;
• prefill and decode round trips;
• AFD control metadata derived from MRV2's actual and padded execution shapes;
• repeated requests, profiling/dummy execution, failure cleanup, and shutdown;
• no DBO and no CAMAsync in this phase.

Acceptance criteria:

• native non-AFD MRV2 control passes on the same runtime/model/topology;
• AFD performs a real Attention-to-FFN-to-Attention round trip;
• output and accuracy match the approved native/V1 reference;
• no full copy of MRV2 execution or Attention preparation is introduced;
• ModelRunner V1 remains functional.

Phase 2 — CAMP2P + DeepSeek-V2-Lite FULL_DECODE_ONLY

Extend the Phase 1 path with ACL graph execution:

• FULL_DECODE_ONLY capture and replay;
• graph miss/fallback and changing batch shapes;
• real versus padded token metadata;
• control-plane communication outside graph capture and replay bodies;
• Attention and FFN graph-key compatibility;
• warmup, capture, replay, repeated requests, and cleanup after failure.

Acceptance criteria:

• eager/graph output and accuracy parity;
• graph capture and replay are observed on both required roles;
• no stale graph, forward-context, transaction, or connector state;
• unsupported graph combinations fail before communication resources are created.

Phase 3 — AscendUBatchRunner and native DBO

Implement native MRV2 DBO for the synchronous CAMP2P path, following the design direction of vLLM #50945:

• make num_ubatches part of the execution descriptor/graph key;
• split the MRV2 InputBatch before building Attention metadata;
• use persistent per-uBatch buffers;
• allocate/select one Attention metadata builder per live uBatch;
• build one Ascend/AFD forward context per uBatch;
• keep DP split decisions rank-consistent;
• run and merge uBatch outputs through AscendUBatchRunner;
• use NPU streams/events and ACL graph ownership rather than CUDA-specific helpers;
• initially support exactly two uBatches;
• start with eager DBO, then qualify FULL_DECODE_ONLY DBO after eager correctness is stable.

The preferred home for AscendUBatchRunner is vLLM-Ascend. AFD should extend it through composition or narrow hooks rather than carrying a fork of the GPU UBatchRunner.

Acceptance criteria:

• eager two-uBatch execution is correct for decode, prefill, and mixed batches where supported;
• uneven DP token counts, padding-only tails, and collective ordering are validated;
• connector groups and payloads use the correct uBatch/stage identity;
• graph DBO passes capture/replay and shape-change coverage before being claimed;
• exceptions do not leave sibling threads, streams, forward contexts, or connector transactions alive;
• the complete non-accuracy NPU E2E gate passes before full accuracy.

Phase 4 — CAMAsync + DeepSeek-V2-Lite

Add MRV2 support for CAMAsyncAFDConnector with DeepSeek-V2-Lite:

• eager execution first;
• preserve the existing asynchronous Attention/MoE/FFN work-item lifecycle;
• keep CAMAsync MoE-only ubatching separate from native DBO;
• do not use --enable-dbo or native DBO thresholds for this path;
• validate stage-local Attention metadata, routing/gate inputs, PCP/SP state when required, stream/context isolation, uneven token counts, and cleanup.

Acceptance criteria:

• native DBO and CAMAsync capability flags remain mutually clear and independently validated;
• real multi-stage asynchronous execution is observed;
• correctness and full accuracy pass on the approved topology;
• existing synchronous CAMP2P MRV2 and V1 coverage remain green.

Phase 5 — CAMAsync + DeepSeek-V4-Flash

Extend the Phase 4 architecture to DeepSeek-V4-Flash:

• reuse the native vLLM/vLLM-Ascend DeepSeek-V4-Flash model, MoE, Attention, cache, routing, and quantization lifecycle;
• add only model-specific AFD boundary logic that cannot be expressed through the DeepSeek-V2-Lite path;
• align with the model-support work tracked by [RFC]: Add DeepSeek V4 support on GPU and NPU for vLLM 0.26.0 #169 and [DeepSeek V4][NPU] Support CAMAsyncAFDConnector #227;
• validate the required topology, quantization, async stage protocol, PCP/SP behavior, and failure cleanup;
• add a reproducible NPU recipe and complete support-matrix evidence.

Acceptance criteria:

• native non-AFD DeepSeek-V4-Flash MRV2 control passes first;
• AFD correctness and accuracy match the approved native reference;
• real CAMAsync multi-stage execution is validated;
• no DeepSeek-V2-specific private state is assumed silently;
• unsupported combinations fail early and are documented.

Validation gates

For every phase, record:

• exact AFD, vLLM, and vLLM-Ascend SHAs and the complete NPU software stack;
• model checkpoint, quantization, topology, physical devices, connector, execution mode, and uBatch policy;
• focused unit/contract tests;
• native vLLM-Ascend MRV2 real-request control;
• AFD prefill/decode/mixed request E2E as applicable;
• pass/fail/error/skip counts and reasons;
• process, port, stream, thread, graph, and device cleanup;
• full non-accuracy NPU E2E before full accuracy;
• eager/graph and V1/V2 correctness/performance comparisons;
• explicit unsupported or unvalidated cells.

A phase is not complete if it depends on bypassing an upstream validator without an approved upstream/removal plan.

Risks and alternatives

Risks

• vLLM #50945 and #51700 are still under review and their final contracts may change.
• Ascend MRV2 may lack a stable uBatch-runner factory or backend hooks.
• CUDA assumptions in the GPU UBatchRunner cannot be copied directly to NPU.
• ACL graph ownership, FlashComm/SP, DP padding, and disaggregated connector topology may require Ascend-specific behavior.
• Maintaining V1 and V2 temporarily increases the test matrix.
• CAMAsync and native DBO can be confused if they share flags or metadata without an explicit policy boundary.
• DeepSeek-V4-Flash native support may move independently of this RFC.

Alternatives considered

• Continue using ModelRunner V1: lower immediate risk but retains the current maintenance burden and misses the MRV2 direction.
• Copy the GPU MRV2 UBatchRunner into AFD: faster initially but creates another backend fork and is not acceptable as the long-term design.
• Port the current V1 NPU runner wholesale to MRV2: preserves implementation shape rather than architecture and should be rejected.
• Implement every connector/model at once: increases the debugging surface and prevents evidence-based phase gates.

Decisions requested

1. Is the CAMP2P-first phase ordering acceptable?
2. Should AscendUBatchRunner be implemented in vLLM-Ascend, with only AFD hooks kept in this repository?
3. Which immutable vLLM/vLLM-Ascend pair and NPU software stack should be the first implementation target?
4. Which DeepSeek-V2-Lite quantization and TP/DP/EP topology must gate Phases 1–3?
5. Should Phase 3 require FULL_DECODE_ONLY DBO parity before CAMAsync work begins, or may graph DBO follow as a separately gated sub-phase?
6. What exact checkpoint/name should be used for the DeepSeek-V4-Flash Phase 5 support claim?

Feedback period

Proposed feedback period: one week, through 2026-08-20.

CC

@specture724 @jiaran-king