Skip to content

docs(recipe): add DeepSeek-V4-Flash Async CAM recipe - #312

Open
ShwStone wants to merge 1 commit into
vllm-project:mainfrom
ShwStone:agent/dsv4-flash-recipe
Open

docs(recipe): add DeepSeek-V4-Flash Async CAM recipe#312
ShwStone wants to merge 1 commit into
vllm-project:mainfrom
ShwStone:agent/dsv4-flash-recipe

Conversation

@ShwStone

@ShwStone ShwStone commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Purpose

Add a user-facing DeepSeek-V4-Flash W8A8 deployment recipe for the
CAMAsyncAFDConnector performance work tracked by #227.

The recipe describes a single-node, 16-NPU deployment:

  • Attention: DP4 × TP2 on devices 0-7, with FlashComm1/SP enabled;
  • FFN: DP8 × TP1 / EP8 on devices 8-15, with FlashComm1 disabled;
  • AFD-managed two-stage token split;
  • MBT 65536, shared compressor workspace, and a 4096 MB CAM HCCL buffer.

Issue

Scope

In scope

  • Add a portable launch.sh <attention|ffn> entry point.
  • Document runtime dependencies, topology, configuration, launch commands,
    limitations, and readiness checks.
  • Record the corresponding issue [DeepSeek V4][NPU] Support CAMAsyncAFDConnector #227 performance summary.
  • Link the recipe from the recipe index and CAM Async user guide.

Out of scope

  • Runtime, connector, model, scheduler, or kernel changes.
  • Multi-node orchestration or environment-specific job tooling.
  • New NPU qualification runs.

Runtime dependency

The recorded performance cell used:

  • vLLM 568afb3a1;
  • vLLM-Ascend 80d8c194f;
  • commit 2e01d4c5 from the still-open
    vLLM-Ascend PR #15452;
  • follow-up slot-mapping fix e19e14da;
  • CANN 9.0.1 and CAM 209.x operators.

PR #15452 provides the DeepSeek-V4 compressor-tail workspace reuse required
for the MBT 65536 configuration. The recipe explicitly documents that this
support is not yet available from the vLLM-Ascend base commit alone.

Implementation notes

The launcher:

  • accepts attention or ffn as a positional role;
  • requires only the model path, local communication IP, and network interface;
  • contains no cluster-specific paths, IP addresses, or orchestration commands;
  • configures FlashComm1 only for Attention;
  • verifies the model configuration and CAM op-api library;
  • handles CAM 209.x installations that expose only libcust_opapi.so;
  • does not impose a startup delay or role ordering;
  • does not enable vLLM native DBO.

Recorded performance context

For the issue #227 single-node DP4TP2/EP8 token-split matrix, all offered-rate
cells completed 512/512 requests without failure.

  • 1.0x: 32,830 effective input tokens/s, TTFT p99 15.6s;
  • 1.5x: 40,096 effective input tokens/s, TTFT p99 34.3s;
  • versus the best baseline point, effective throughput improved by 4.8% at
    1.0x, 6.6% at 1.25x, and 8.6% at 1.5x;
  • at 1.0x, TTFT p50 improved by 37.4%, TTFT p99 by 22.1%, and the 10-second
    TTFT SLO rate increased from 65.2% to 85.2%.

These are historical results for the recorded software and workload cell, not
a performance guarantee or a new qualification result for this PR.

Test result

  • bash -n recipe/npu/CAMAsyncAFDConnector/deepseek_v4_flash/launch.sh — passed
  • git diff --check — passed
  • NPU execution — not run
  • ShellCheck — not run because shellcheck is unavailable in the local environment

Docs impact

Updated:

  • recipe/README.md
  • docs/npu/CAM_ASYNC_CONNECTOR_USER_GUIDE.md

Added:

  • recipe/npu/CAMAsyncAFDConnector/deepseek_v4_flash/README.md
  • recipe/npu/CAMAsyncAFDConnector/deepseek_v4_flash/launch.sh

Signed-off-by: ShwStone <HaowenShi@outlook.com>
Copilot AI lite review requested due to automatic review settings September 4, 2026 10:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants