[NPU] Port DSV4 CAM fixes to main - #311
Merged
Merged
Conversation
…-project#301) ## Purpose Fix DeepSeek V4 Hash routing on NPU Attention ranks when FlashComm v1 sequence parallelism is enabled. After FlashComm splits hidden states across tensor-parallel ranks, router logits contain only rank-local tokens, while forward_context.input_ids can still contain the unsplit global token list. The previous alignment check therefore raised: DSV4 Hash routing input_ids/token count mismatch on Attention This change pads input_ids with the existing forward-context pad size, splits them with the same contiguous tensor-parallel layout, and selects the current TP rank before invoking the CANN Hash routing operator. ## Issue - Related issue(s): None - Closing keyword, only if fully resolved: N/A ## Scope - In scope: - DSV4 NPU Async CAM Hash routing on Attention ranks with FlashComm v1 enabled. - Validation that the Hash-routing token IDs follow the local TP token layout. - Out of scope: - Non-Hash routing. - Native vLLM-Ascend MoE communication. - GPU behavior and unrelated DSV4 execution paths. ## Implementation Notes - The adjustment is only applied when FlashComm v1 is enabled and the input ID count differs from the local router-logit token count. - Existing forward_context.pad_size is applied before splitting so the token layout is divisible across TP ranks. - vLLM-Ascend split_tensor_along_first_dim is used with contiguous chunks, and the current TP rank selects its local input IDs. - The existing post-split count validation remains in place, so unexpected layouts still fail explicitly. - No vLLM source checkout or monkey patch is modified. ## Test Plan - Run the focused CPU-safe unit test. - Run Ruff lint and formatting checks on both changed files. - NPU multi-rank validation is hardware-gated and was not available locally. ## Test Result - python3 -m pytest -q tests/unit/model_executor/models/test_deepseek_v4_attention_gate.py - 2 passed - python3 -m ruff check afd_plugin/model_executor/models/npu/deepseek_v4_attention_gate.py tests/unit/model_executor/models/test_deepseek_v4_attention_gate.py - All checks passed - python3 -m ruff format --check afd_plugin/model_executor/models/npu/deepseek_v4_attention_gate.py tests/unit/model_executor/models/test_deepseek_v4_attention_gate.py - 2 files already formatted - git diff --check - Passed - NPU multi-rank E2E: not run because no NPU environment was available locally. ## Docs Impact - Files updated: None - If none, reason: This is an internal runtime correctness fix with no user-facing API or configuration change. --- <details> <summary>Essential PR Checklist</summary> - [x] Purpose is clear and linked to public context when possible. - [x] Scope is bounded. - [x] Compatibility with vLLM v0.26.0 is considered. - [x] No changes are made to the vLLM source checkout. - [x] Plugin-owned classes or explicit dotted class paths are preferred over monkey patches. - [x] Any compat shim or monkey patch is isolated, idempotent, version-guarded, documented, and tested. Not applicable; no shim or monkey patch is changed. - [x] Imports remain CPU-safe; NPU-specific imports stay deferred inside the runtime-only branch. - [x] Validation evidence is included, including skipped NPU tests. - [x] Documentation impact is stated. </details> Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Keeps CAM dynamicQuant shared inputs in the INT8 path through W13, fused dequant-SwiGLU-quant, and W2. Adds contract coverage for the fused scale handoff.\n\nValidation: python3 -m compileall and git diff --check. Pytest could not run because the local .venv pytest launcher points to a missing interpreter. --------- Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Port the following merged changes from
v0.26.0_camasync_dsv4tomain:Changes
Validation
python3 -m compileall -qon the four changed Python filesgit diff --check upstream/main..HEADpytestinstalled (No module named pytest).Signed-off-by: jiangkuaixue123 jiangxiaozhou111@163.com