[vLLM][XPU] 修复 torch.compile 跟踪时 LoRA 算子被裁剪的问题 - #635
Open
sicnuyudidi wants to merge 2 commits into
Open
Conversation
Signed-off-by: yudidi <1270896784@qq.com>
Signed-off-by: yudidi <1270896784@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
修复 vLLM 在
torch.compilecompilation tracing 阶段,XPU Punica 的no-adapter fast path 错误地从 compiled graph 中 prune LoRA operations 的问题。
Fixes #634.
根因
PunicaWrapperXPU.no_lora是 mutable host-side Python state。常规 profilingbatch 没有 active adapter,因此 compilation tracing 时
self.no_lora=True。现有的 Python early return 会在 tracing 阶段被固化到 compiled graph 中,导致
LoRA operations 被直接 prune。之后即使 adapter 请求的 request-time mapping
正确,compiled graph 也不会再执行 LoRA delta,最终表现为请求成功但 LoRA
effect 被静默忽略。
改动
_should_skip_lora():在 eager execution path 下保留 no-adapter fastpath,但在 compilation tracing 阶段禁止使用该 shortcut。
实现说明
torch.compiler.is_compiling()是“当前 Python frame 正在被 TorchDynamo捕获”的上下文谓词,不是服务是否启用
torch.compile的全局开关。helper采用显式 compile-first 分支:
这样 tracing 期间不会读取 mutable
self.no_lora,LoRA operators 必须保留在compiled graph 中;非 tracing 的 eager path 仍可根据当前 batch 的
self.no_lora使用 fast path。验证
--enforce-eager后 LoRA effect 恢复。effect。
git apply将更新后的 llm-scaler patch 成功应用到干净的上游vLLM v0.21.0 源码。
git diff --check,均通过。compiling × no_lora四态真值表,并验证 tracing 分支不会读取 mutable host state。已在 Intel XPU 环境完成 end-to-end validation。