Skip to content

Adapt DeepSeek persistent worker to offset cache copies - #118

Draft
high-cloud wants to merge 1 commit into
hw-native-sys:mainfrom
high-cloud:agent/deepseek-persistent-window-reset
Draft

Adapt DeepSeek persistent worker to offset cache copies#118
high-cloud wants to merge 1 commit into
hw-native-sys:mainfrom
high-cloud:agent/deepseek-persistent-window-reset

Conversation

@high-cloud

@high-cloud high-cloud commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Upload dirty DeepSeek decode work-cache pages through a base allocation plus byte offset instead of passing an interior device pointer to copy_to.
  • Keep the DeepSeek shared L3 worker persistent and rely on the corresponding PyPTO default of reset_persistent_windows=False, so serving does not perform a host-side comm-window reset between requests.

Root cause

The latest simpler device-pointer provenance guard requires copy_to to receive the base address of a live allocation. DeepSeek work-cache synchronization previously passed shard.data_ptr + offset, which is an interior pointer and is rejected before prefill starts.

Dependencies

This PR depends on:

  • PyPTO #2164, which adds DistributedWorker.copy_to_offset(...);
  • simpler #1538, which adds Orchestrator.copy_to_offset(...) and validates the live base allocation before applying the byte offset;
  • PyPTO #2163, which makes reset_persistent_windows=False the persistent-worker default.

Validation

  • python -m pytest -q tests/test_deepseek_v4.py — 32 passed.
  • ruff check --config ruff.toml pypto_serving/model/deepseek/npu_runner.py — passed.
  • python tests/lint/check_headers.py — passed.
  • python tests/lint/check_english_only.py — 53 passed.
  • python -m compileall -q pypto_serving/model/deepseek/npu_runner.py — passed.
  • 8-card DeepSeek V4 MTP accuracy: task_20260727_032719_85546414704 — 4 passed, serving shut down normally, and profiling trace was written to artifacts/forward-end-clear-mtp-v2/trace.json.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 20421ad0-81e8-4016-9f64-0c720394e904

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant