feat(sight): two-track detour experience + lossy SSE decompression - #2338
feat(sight): two-track detour experience + lossy SSE decompression#2338chengshuyi wants to merge 5 commits into
Conversation
Replace the half-dead 试错型/返工型 shape split in WasteExperience with the 成功经验/失败教训 two-track structure from the offline distill skill's criteria. The 返工型 half (rule/good_example/bad_example/scope) was wired from the storage layer up through the frontend renderer but had no producer: neither the prompt contract nor DetourFix carried the fields, so cost/llm.rs always left them at Default. Rather than add the missing producer, adopt the two-track shape that the offline run already validated. Keeps the turns anchor: savings stay summed from the ledger in Rust, so the per-finding granularity is preserved rather than moving to criteria.md's trajectory-level lists. Co-Authored-By: Claude <noreply@anthropic.com>
Replace WasteExperience's 试错型/返工型 shape split with the 成功经验/失败教训 two-track structure. The 返工型 half was wired from the storage layer through the frontend renderer but had no producer — neither waste_detour.md nor DetourFix carried rule/good_example/bad_example/scope, so cost/llm.rs left them at Default forever and the frontend rendered four permanently-empty lines. - lesson (失败教训) is required: a finding whose fix carries no lesson is dropped, since a failure nobody can say how to avoid is a dead record. - playbook (成功经验) is optional: a detour caused by环境抖动 has no reusable path, and padding it would only dilute signal. - 偶发故障 strips both tracks but keeps the row — the detour is real, only the experience is unfalsifiable. This ordering is load-bearing: the strip must run before the lesson gate or transient detours would be swallowed whole. - Adds the one admission criterion the product lacked: 非显然 (exclude model common sense). 会再遇到 and 能改下次行为 are already covered by the required `when` and `instead`/`how` fields. The turns anchor is untouched: savings are still summed from the ledger in Rust, never estimated by the model. Legacy cost_waste payloads deserialize to both tracks None rather than erroring; re-running an analysis restores them. No DB schema change, no new API, no extra LLM call. Co-Authored-By: Claude <noreply@anthropic.com>
The two-track output was structurally present but rhetorically buried: the model's role line said only "弯路诊断器", the tracks appeared in one mid-document subsection framed as an appendage of `fix`, and the JSON keys were `lesson` / `playbook` — neither of which states which track it is. A model reading the prompt top-down would take detour detection as the whole job. - Role line now names both jobs and states which one is the product: 弯路本身只是 证据，经验才是产物. - 归因决定产出 → 归因决定能不能沉淀出经验; both tracks reverse-engineered from the turn that worked. - The two tracks get their own section with a heading each, plus the four categories worth distilling on the success side (有效路径 / 决策依据 / 捷径与 工具用法 / 环境稳定事实) taken from the offline skill's criteria. - 会再遇到 is now stated as an admission criterion alongside 非显然 rather than trailing as a note about `when`. - JSON keys renamed lesson → failure_lesson, playbook → success_playbook, synced through Rust, TS types and the renderer. Also renames prompts/waste_detour.md → prompts/detour.md: every identifier in the code says detour (is_detour, DETOUR_PROMPT, DetourFix, MIN_DETOUR_TURNS, candidate id "detour"), and cost/prompts/cost_identification.rs ↔ prompts/cost_identification.md establishes that the prompt file matches the .rs that includes it. The waste_ prefix was the only one of its kind in the dir. Co-Authored-By: Claude <noreply@anthropic.com>
- Add decompress_body_lossy for incomplete SSE streams (mid-frame EOF no longer discards the entire decoded prefix) - Aggregator fallback uses lossy decoding; incremental zstd path unchanged - Dashboard: extract atifSource utility, source badges link to trajectory viewer pinned by store - AtifViewerPage: source-aware loading with session/trace/conversation - Config/parser: SSE event metadata enhancements - Tests: pipeline integration + atifSource regression
github-actions
Bot
added
component:sight
![qoderai[bot]](https://avatars.githubusercontent.com/in/1396158?s=60&v=4){kind=small}
There was a problem hiding this comment.
[P1] SSE 压缩流重路由条件过于粗粒度，在同一 SSL 读产生多条真实 SSE 事件时，只有首条被保留，后续事件会被整体吞掉，导致响应侧事件缺失。
[P1] lossy 解压路径新增流式 zstd/gzip/brotli 解码但仍依赖旧的 MAX_DECOMPRESSED_LEN 常量，上限是否与 runtime_limits.max_connection_body_mb 一致未在代码中声明，极端大 SSE 响应可能突破预期内存预算。
[P2] ATIF 源自动选择在双源数据几乎为空时仍给出默认源，虽然 badge 文案提示缺数据，但未显式标记“严重缺失”，可能弱化对捕获链路异常的诊断信号。
🤖 Generated by Qoder • View workflow run
| return messages; | ||
| } | ||
|
|
||
| let mut out = Vec::with_capacity(messages.len()); |
There was a problem hiding this comment.
[P1] SSE 压缩流重路由可能遗漏事件源
压缩 SSE 流在 unified 聚合器中通过 reroute_compressed_sse 将同一 source_event_rc 的后续 ParsedSseEvent 全部改写为 RawData（src/agentsight/src/aggregator/unified.rs:121-138）。如果解析器对同一 SSL 读产生了多条有效 SSE 事件（部分是真正的 SSE，部分误判），后面的真实事件也会被丢弃，只保留第一条。这在多 event 同帧的场景下会导致响应侧事件缺失。
建议：在判断是否重路由时进一步约束条件，仅在事件 body 明显非文本或 metrics 完全为空的情况下才改写为 RawData，或按字节偏移做更精细的区分，避免真实 SSE 事件被整体吞掉。
🤖 Generated by Qoder • Fix in Qoder
There was a problem hiding this comment.
Pre-existing logic from #2297, not introduced by this PR. The reroute fires only when compressed_buffer is Some (i.e. the connection is a compressed SSE stream) — in that mode the parser does NOT produce multiple independent ParsedSseEvents from one SSL read; it yields one SseEvent + RawData continuation by design. Acknowledge the theoretical edge case; will track as a follow-up hardening item if we see real-world event loss.
| let mut out = Vec::new(); | ||
| { | ||
| let sink = CappedSink { | ||
| out: &mut out, |
There was a problem hiding this comment.
[P1] lossy 解压路径缺少 MAX_DECOMPRESSED_LEN 上限校验回归
decompress_body_lossy 使用 CappedSink 绑定输出长度到 MAX_DECOMPRESSED_LEN，但上限值仍沿用 decompress_body 的常量（src/agentsight/src/utils/decompress.rs:331-333）。在 lossy 路径中新增 zstd/gzip/brotli 流式解码后，这个上限需要重新核对是否与 runtime_limits.max_connection_body_mb 一致，否则极端响应体（如大日志 SSE）可能在解压侧突破预期内存预算。
建议：显式将 MAX_DECOMPRESSED_LEN 计算改为基于 max_connection_body_mb 的统一函数，确保严格/损失性两条解压路径共享同一内存上限，并补充针对大体积 SSE 响应的回归用例。
🤖 Generated by Qoder • Fix in Qoder
There was a problem hiding this comment.
Both strict and lossy paths share the same CappedWriter / CappedSink bounded by MAX_DECOMPRESSED_LEN (200 MB). This constant is intentionally a hard backstop independent of max_connection_body_mb (which bounds the compressed input, not the decoded output). The two limits serve different purposes: input cap prevents buffering abuse, output cap prevents decompression bombs. Adding a comment to clarify the relationship; no functional change needed.
| throw new Error(`未找到该 Session：${sessionId}（既无 eBPF 捕获记录，也无采集轨迹）`); | ||
| } | ||
|
|
||
| // Compare scores directly rather than the identity of pickRicherAtifDoc's |
There was a problem hiding this comment.
[P2] ATIF source 自动选择在双缺数据时仍可能误导
loadSessionAtifSources 在 export 与 collected 都存在但两边 agent 步 payload 几乎为 0 的情况下仍按 content score 选默认源，并将另一个经 backfill 后保留（src/agentsight/dashboard/src/utils/atifSource.ts:187-196）。但在捕获几乎完全失败的 session（withPayload≈0,total≫0）中，两边得分都接近 0，用户看到的默认源 badge 文案虽然提示缺数据，仍可能误读为“该源数据可信，只是少量缺口”。
建议：当两个源的 coverage 比例都低于某个阈值（例如 withPayload/total < 0.05）时，将 defaultSource 置为 eBPF 并在 SourceSwitcher 旁增加显式“此会话响应侧数据严重缺失，需检查探针配置 / BoringSSL 静态链接”等提示，以强化这类诊断信号。
🤖 Generated by Qoder • Fix in Qoder
There was a problem hiding this comment.
Valid UX observation. The current design intentionally avoids blocking the UI on partial data — the badge already shows a warning icon when coverage is low. Agreed that a stronger "severely degraded" signal would help diagnostics; will address in a follow-up UI polish pass (threshold-based banner).
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 146d31069e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| window.open( | ||
| `#/atif?type=session&id=${encodeURIComponent(sessionId)}&source=${s}`, |
There was a problem hiding this comment.
When a Codex rollout row is merged via the trailing-UUID fallback, sessionId here is the eBPF UUID, not the collected trajectory key (rollout-...-uuid). The trajectory endpoint does an exact lookup by collected session_id, and the collector stores flat Codex sessions under the rollout file stem, so clicking the new log badge (or opening source=log) for these merged rows still fetches /api/trajectories/<bare uuid> and 404s instead of showing the log trajectory. The merged row needs to retain/use the per-source collected id for log navigation.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed: added log_session_id field to MergedSession; set during merge when the trailing-UUID fallback matches a different collected key. SourceBadge now uses log_session_id for source=log navigation.
|
|
||
| // 失败教训是报一段弯路的必要条件：说不清那个坑，这条经验就是死记录。 | ||
| // 偶发故障豁免 —— 它按上面的规则本就不带经验。 | ||
| if !transient && fix.as_ref().is_none_or(|fx| fx.failure_lesson.is_none()) { |
There was a problem hiding this comment.
For a non-transient finding where the model returns failure_lesson: {} (or leaves instead/when blank), serde turns that into Some(ExperienceLesson::default()), so this presence-only gate accepts the row. The frontend only renders a lesson when instead is populated, which means the report can keep exactly the "no reusable lesson" findings this change says should be dropped. Please validate the required lesson content, not just that the option exists.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed: the gate now validates lesson content, not just presence. failure_lesson.as_ref().is_none_or(|l| l.instead.is_empty()) — an empty object from serde Default is dropped.
| /// LRU capacity for concurrently tracked HTTP connections. Shared across | ||
| /// every traced process, so a host running several agents at once needs | ||
| /// considerably more than one agent's worth. | ||
| pub connection_capacity: Option<usize>, |
There was a problem hiding this comment.
This adds runtime_limits.connection_capacity, but the user-facing runtime limit docs and sample config still omit it; I checked src/agentsight/README*.md and src/agentsight/agentsight.json, and rg -n "connection_capacity" src/agentsight docs specs only finds code/tests. Users therefore cannot discover the override/default from the documented config surface, despite the repository’s mandatory documentation standard for config changes.
AGENTS.md reference: AGENTS.md:L319-L325
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed: added connection_capacity: 256 to agentsight.json runtime_limits block and a row in the AGENTS.md runtime_limits table.
- cost/llm.rs: validate lesson content (instead non-empty), not just Option presence — empty serde Default no longer passes the gate - AgentSessionsPage: preserve collected log_session_id for Codex rollout rows so source=log badge navigates to the correct key - agentsight.json + AGENTS.md: document connection_capacity (256) in runtime_limits table and sample config
1 participant
Description
成本弯路检测的经验沉淀使用扁平三字段（applicability/pitfall/effective_path），无法区分"要绕开的坑"与"可照搬的路"，
且缺少失败教训的 finding 仍然被报出——说不清怎么避免的失败是死记录，报了也无法复用。
本 PR 将经验拆为两路独立结构：
failure_lesson（失败教训，必填）和success_playbook（成功经验，可缺），并在
expand_detour_items中加入教训门控：缺 failure_lesson 的弯路 finding 整条不报（偶发故障豁免）。同时迁移 frontier 分支的 lossy 解压与 ATIF source-aware dashboard 改动。
Related Issue
no-issue: 经验模型重构 + frontier 迁移，无单独 issue
Type of Change
Scope
sight(agentsight)Key Changes
Opt crate — 两路经验模型:
crates/agentsight-opt/src/types.rs: 新增ExperienceLesson/ExperiencePlaybook结构体，WasteExperience和DetourFix改用两路 Optioncrates/agentsight-opt/src/cost/llm.rs:expand_detour_items加入 failure_lesson 门控 + 偶发故障豁免逻辑crates/agentsight-opt/prompts/detour.md: 重写 prompt，明确两路经验的判据和必填/可缺语义crates/agentsight-opt/src/cost/prompts/detour.rs: prompt 文件路径更新Sight — lossy 解压 + dashboard:
src/utils/decompress.rs: 新增decompress_body_lossy— 中帧 EOF 不再丢弃已解码前缀src/aggregator/http/aggregator.rs: fallback 路径改用 lossy 解码src/aggregator/unified.rs/src/config.rs/src/parser/sse/event.rs: SSE 元数据增强dashboard/src/utils/atifSource.ts: 新增 ATIF source 工具模块dashboard/src/pages/AtifViewerPage.tsx: source-aware 加载dashboard/src/pages/AgentSessionsPage.tsx: source badge 点击跳转轨迹查看器设计文档:
docs/design/2026-08-05-detour-experience-two-track-design.md: 两路经验设计 specChecklist
cargo fmt --checkpasscargo clippy --all-targets -- -D warningspasscargo llvm-cov测试 + 增量覆盖率diff-cover --fail-under=80passCargo.lock)Testing
expand_detour_items单元测试验证：缺 failure_lesson 的 finding 被过滤、偶发故障豁免、两路经验正确映射decompress_body_lossy单元测试验证：截断 zstd 帧解码出前缀事件（decompress.rs 内已有）atifSource-regression.test.cjs）