Skip to content

[NPU] Clean up DSV4 async CAM follow-ups from #264 #283

Description

@jiangkuaixue123

Background

PR #264 added the DSV4 NPU async CAM/ubatching path. A few tactical workarounds and duplicated flows were intentionally left for follow-up so that their runtime contracts can be validated separately.

Related: #264 and #150.

Follow-up tasks

  • Remove the device-wide torch.npu.synchronize() from the connector-driven FFN path. Replace it with the narrow completion/credit mechanism required to keep CAM buffers alive and prevent premature reuse. Coordinate this with the shared stream-pipeline work in [Feature]: Add CAMAsync multi-stream execution on a shared NPU stream pipeline #150.
  • Remove the direct register_afd() call from get_afd_model_config(). Model registration should be established through the appropriate plugin or worker initialization path rather than as a side effect of model-config conversion.
  • Investigate why the DSV4 async deployment requires spawn. Provide a minimal reproducer and identify which component depends on it. If it is not required, remove AFD_FORCE_SPAWN_MULTIPROCESSING and the process-wide multiprocessing override. If it is required, isolate it to the affected NPU runtime/launcher path and avoid modifying Python's global/private multiprocessing context from package import.
  • Low priority: compare the DSV4 async flow with the existing DSV2 async CAM implementation and extract genuinely shared scheduling, metadata, and connector logic while keeping model-specific behavior explicit.

Validation

  • Cover buffer lifetime and ordering after removing the synchronization, including multiple layers/stages and repeated in-flight work.
  • Verify model resolution in spawned workers after removing the direct register_afd() call.
  • Test the supported multiprocessing methods on the affected Python/CANN/CAM environment and document the result.
  • Keep existing DSV2 and DSV4 async CAM unit/E2E behavior unchanged after refactoring.

Acceptance criteria

  • No device-wide synchronization remains in the DSV4 connector-driven FFN steady-state path unless a demonstrated external CAM contract requires it.
  • Model-config conversion no longer invokes register_afd() directly.
  • The spawn requirement is either removed or backed by a reproducible, narrowly scoped runtime contract without a package-import-time global multiprocessing monkey patch.
  • Any shared DSV2/DSV4 abstraction reduces real duplication and preserves explicit model-specific differences.

Before submitting

  • I searched existing issues and RFCs.
  • I identified this as plugin-owned NPU model/runner initialization and cleanup work; no vLLM source-tree modification is expected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    AscendAscend NPU platform and related changesenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions