fix(trace): 用户取消后将 trace run 收尾为 CANCELLED - #80
Open
qiyu-lu wants to merge 1 commit into
Open
Conversation
用户点击「停止生成」后 t_rag_trace_run 永远停在 RUNNING,end_time 与 duration_ms 为 NULL,且无补偿任务回收;同一条 trace 内子节点已标记 CANCELLED,父 run 却仍是 RUNNING。而 Dashboard 的成功率分母只含 SUCCESS + ERROR,被中断的会话被系统性剔除,指标呈现幸存者偏差。 取消信号在 provider client 层就被 ForwardingStreamCallback 拦截了 (对外部取消不透传 delegate,否则流式 failover 切换候选时会终止用户 SSE),run 级终态无法由回调链驱动。StreamTaskManager.cancelLocal 的 CAS 成功分支是全链路唯一能区分「用户主动取消」与「failover 内部取消」 的位置,因此在该处上报。 - RagTraceRecordService 新增 cancelRunByTaskId:按 taskId 定位仍在 RUNNING 的 run,以带 status 的条件更新收尾,与正常终态竞争时只有 一方生效;duration_ms 由 start_time 实算 - StreamTaskManager 在取消动作完成后上报,异常仅告警,不影响取消本身 - Dashboard 成功率与错误率的分母纳入 CANCELLED Closes nageoffer#79 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
该 PR 修复用户主动取消流式生成后,t_rag_trace_run 仍停留在 RUNNING 的问题,并将 CANCELLED 会话纳入管理端 Dashboard 成功率/错误率的分母统计,使看板指标口径更真实一致。
Changes:
- 在 trace 记录服务新增按
taskId将仍为RUNNING的 run 收尾为CANCELLED的能力,并计算真实duration_ms - 在
StreamTaskManager的用户取消 CAS 成功分支上报 run 级取消终态,避免回调链路被 provider 层“外部收尾”拦截导致 run 悬挂 - Dashboard 成功率/错误率分母纳入
CANCELLED,并补充对应单元测试覆盖
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| bootstrap/src/main/java/com/nageoffer/ai/ragent/rag/service/RagTraceRecordService.java | 新增 cancelRunByTaskId(taskId, endTime) 接口用于 run 级取消终态上报 |
| bootstrap/src/main/java/com/nageoffer/ai/ragent/rag/service/impl/RagTraceRecordServiceImpl.java | 实现按 taskId 条件更新 RUNNING→CANCELLED,并计算 duration |
| bootstrap/src/main/java/com/nageoffer/ai/ragent/rag/service/handler/StreamTaskManager.java | 在用户取消 CAS 成功分支完成取消后,上报 run 级 CANCELLED 终态 |
| bootstrap/src/main/java/com/nageoffer/ai/ragent/admin/service/impl/DashboardServiceImpl.java | 成功率/错误率分母新增 CANCELLED 计数 |
| bootstrap/src/test/java/com/nageoffer/ai/ragent/rag/service/impl/RagTraceRecordServiceImplTest.java | 覆盖取消收尾的条件更新、duration 计算与竞争结果 |
| bootstrap/src/test/java/com/nageoffer/ai/ragent/rag/service/handler/StreamTaskManagerCancelTraceTest.java | 覆盖取消上报仅一次、未知任务不报、trace 上报失败不影响取消 |
| bootstrap/src/test/java/com/nageoffer/ai/ragent/admin/service/impl/DashboardPerformanceCancelledTest.java | 覆盖 Dashboard 成功率/错误率分母纳入 CANCELLED 的新口径 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+38
to
+40
| private static final String STATUS_RUNNING = "RUNNING"; | ||
| private static final String STATUS_CANCELLED = "CANCELLED"; | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
关联 Issue
Closes #79
改动内容
修复「用户点停止生成后
t_rag_trace_run永远停在 RUNNING」的问题,并让 Dashboard 成功率分母纳入CANCELLED。RagTraceRecordService(+Impl)cancelRunByTaskId(taskId, endTime)StreamTaskManagercancelLocal的 CAS 成功分支上报取消终态DashboardServiceImplCANCELLED为什么修在
StreamTaskManager而不是回调链上这是本 PR 唯一需要讨论的设计点。
取消信号在 provider client 层就被
ForwardingStreamCallback.finishExternally拦截了(注释写明「外部路径(如 cancel)触发收尾,不再透传 delegate」),因此外层StreamChatTraceRunner的traceAwareCallback收不到任何终态事件,finishRun永不触发。这个「不透传」不能改:
RoutingLLMService:163与:189在候选失败切换时同样调用handle.cancel(),一旦透传onError,failover 时第一个候选失败就会终止用户 SSE,「首包探测 + 候选切换对前端无感」的设计会被破坏。而且即使透传,
finishExternally(false, null)传的是success=false,finishRun会写成ERROR而不是CANCELLED,只是把「成功率虚高」换成「错误率虚高」。所以 run 级终态只能由能区分取消来源的那一层上报,而全链路只有
cancelLocal的 CAS 成功分支能确定「这是用户主动取消」。关于分层的说明(请 Review 时特别看一下)
这样做的代价是:
StreamTaskManager原本只依赖 Redisson 与 SSE,现在多了一个RagTraceRecordService依赖,即在流控/取消组件里引入了观测组件。考虑过的替代方案是发布一个 Spring 事件、由 trace 侧订阅,可以保持
StreamTaskManager对观测无感知。没有采用是为了让本 PR 的改动面尽量小。如果维护者认为分层更重要,我可以改成事件方式,请指示。
实现细节
cancelRunByTaskId的更新条件是trace_id = ? AND status = 'RUNNING'。当取消上报与正常终态(onComplete/onError)并发时,只有一方能生效,返回值表明本次是否真的翻转了状态。log.warn。trace 是旁路观测,其故障不应连累用户取消。duration_ms实算:从查出的start_time计算,而非置 0 或 NULL,使被取消的会话也具备可统计的真实耗时。rag.trace.enabled=false时没有 run 行,条件更新匹配 0 行,直接返回 false。测试
新增 3 个测试类共 9 个用例,全部为纯单元测试,不依赖外部环境:
StreamTaskManagerCancelTraceTestRagTraceRecordServiceImplTestDashboardPerformanceCancelledTest兼容性
t_rag_trace_run.status复用已有的CANCELLED取值(RagStreamTraceSupportImpl已在 node 级使用)DashboardPerformanceVO字段不变