[WIP]fix(npu): trim vllm-ascend colocate patch and pin native IPC engine#310
[WIP]fix(npu): trim vllm-ascend colocate patch and pin native IPC engine#310CalvinXKY wants to merge 2 commits into
Conversation
Drop the redundant npu_ipc_engine.py hunk now that vLLM Ascend 5ca762a provides the NPU IPC init fix upstream. Pin VLLM_ASCEND_COMMIT in Dockerfile.npu and document colocate validation notes. Signed-off-by: kaiyuan <kyxiezju@163.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request pins the vllm-ascend dependency to a specific commit (5ca762a) to leverage the upstreamed native NPU IPC weight transfer engine, which simplifies the vllm-ascend.patch to only include the colocate worker patch. It also updates the Dockerfile, README, and NPU documentation to reflect these changes, including a new guide and example for running Qwen3-4B in colocate mode. The review feedback correctly identifies a potential syntax error in the documentation's bash example due to a trailing backslash followed by a placeholder, and suggests a clearer command structure.
| --tensor-model-parallel-size 8 \ | ||
| ... # same GRPO / optimizer flags as the decoupled example |
There was a problem hiding this comment.
The trailing backslash on line 222 followed by the ... placeholder on line 223 will cause a syntax error if the user copies and runs the command directly. Additionally, users should be cautioned not to copy the 4-GPU parallel size flags (such as --tensor-model-parallel-size 4) from the decoupled example, as they conflict with the 8-GPU colocate configuration.
| --tensor-model-parallel-size 8 \ | |
| ... # same GRPO / optimizer flags as the decoupled example | |
| --tensor-model-parallel-size 8 | |
| # [Add the same GRPO / optimizer flags as the decoupled example, but do not copy the 4-GPU parallel size flags] |
Document dev snapshot version strings next to commit hashes in the component table for easier cross-reference with validation reports. Signed-off-by: kaiyuan <kyxiezju@163.com>
CalvinXKY
changed the title
Documentation build overview
42 files changed ·
|
1 participant
Summary
Follow-up to #285 per reviewer feedback (@Meihan-chen): drop the redundant
npu_ipc_engine.pyhunk fromvllm-ascend.patchnow that upstream providesthe NPU IPC engine fix, pin a verified vLLM Ascend commit in
Dockerfile.npu,and document NPU colocate validation notes.
vllm-ascend.patch: keep only colocateworker.pyhunks (skip free-memorychecks when training and inference share NPUs).
Dockerfile.npu: pinVLLM_ASCEND_COMMIT=5ca762a704f2a7acbd1bd839c3f3b421e5c0dcaf;vLLM commit is still taken from
.github/vllm-main-verified.commitat that pin.docs/en/get_started/NPU.md: add colocate caveats (no--vllm-enforce-eager,pre-run HBM cleanup, upstream IPC requirement).
Pinned versions
0.3.0(ascendbranch, post #285)0.1.dev1+g5ca762a704f2(dev snapshot)5ca762a704f2a7acbd1bd839c3f3b421e5c0dcaf— vllm-ascend #10996 onmain(nativeNPUIPCWeightTransferEngine); rc backport: vllm-ascend #11030 @f206fbf31onv0.22.1rc0.22.1rc1.dev263+g967c5c3bc967c5c3bc38891f4465d3f4e99917ed837bb3833(from.github/vllm-main-verified.commitat the vLLM Ascend pin above)2.10.0+cpu3.12.139.0.0Validation
Qwen3-4B, TP=8, colocate +
--megatron-to-hf-mode bridge, native IPC weight sync,direct
python train.py(no Ray job submit), no--vllm-enforce-eager.Container image:
registry-cbu.huawei.com/vime/vime-ascend-npu:vime_0.3.0-vllm_0.22.1-pytorch_2.10.0-cann_9.0.0-py_3.12-snt9btrain_rollout_logprob_abs_diffFollow-up (out of scope)
Migrate colocate weight sync to fully reuse native
NPUIPCWeightTransferEngine(replace custom
vLLMColocateWorkerExtensionpath) — tracked separately.Test plan
Dockerfile.npu(CI / manual)