Skip to content

feat(gateway): auto-compact long sessions and retry once on context overflow - #424

Open
veenyi wants to merge 1 commit into
TencentCloud:developfrom
veenyi:contrib-auto-context
Open

feat(gateway): auto-compact long sessions and retry once on context overflow#424
veenyi wants to merge 1 commit into
TencentCloud:developfrom
veenyi:contrib-auto-context

Conversation

@veenyi

@veenyi veenyi commented Aug 25, 2026

Copy link
Copy Markdown

问题

会话(IM 群/私聊、工作台、CLI)无限增长。一旦对话超过模型上下文窗口,LLM 返回 400 "Requested token count exceeds the model's maximum context length",消息被静默丢弃——通道直接哑火,直到用户手动删除会话才恢复。deepagents 自带的 SummarizationMiddleware(默认 85% 触发)在该栈中未能可靠触发(实测会话涨到 518K token、模型窗口 512K 仍未压缩)。

修复

在消息处理器(octop/infra/gateway/process/processor.py)增加双层兜底:

  1. turn 前自动压缩:每次消息进入前,通过 agent.aget_context_usage(thread_id) 检查会话上下文占用,超过模型窗口 75% 时自动 agent.acompact_conversation(thread_id)(保留最近消息 + 摘要,内容不丢失)。
  2. 失败重试:若仍触发上下文超限错误(_is_context_overflow 匹配 "Requested token count exceeds" / "maximum context length" / "input prompt token len"),自动压缩并重试一次——即使会话已超限也能自动恢复,无需用户手动操作。

说明

  • 所有消息都经过 GlobalProcessor.__call__,因此对所有通道生效(QQ 群/私聊、工作台、CLI)。
  • 两个辅助方法均优雅降级:aget_context_usage 可能返回过期的内存快照(重试路径覆盖该情况);所有异常只记录日志、绝不向上抛出。
  • 相关:feat: session model_ref overrides agent default silently — confusing when switching models #418(会话级 model_ref 静默覆盖默认模型——建议一起阅读"会话持续可用"的完整方案)。

…verflow

IM/dashboard sessions grow unbounded; once the conversation exceeds the
model context window the LLM returns 400 ("Requested token count exceeds
the model's maximum context length") and messages are silently dropped —
users must manually delete the session to restore the channel. deepagents'
SummarizationMiddleware (default 85% trigger) does not fire reliably in
this stack, so add a belt-and-braces layer in the message processor:

- Before each turn, check thread context usage (aget_context_usage); when
  it exceeds 75% of the model window, auto-compact via acompact_conversation
  (keeps recent messages + summary, content is not lost).
- If a turn still fails with a context-overflow error, compact and retry
  once so long-running QQ/IM sessions keep working without manual resets.

Applies to all channels (QQ group/DM, dashboard, CLI).
@veenyi

veenyi commented Aug 25, 2026

Copy link
Copy Markdown
Author

Windows CI failure — pre-existing, not from this PR

The failing check is:

tests/unit/browser/test_browser_setup.py::test_temp_scope_token_uses_username_when_getuid_missing
AttributeError: <module os (frozen)> has no attribute getuid

That test (introduced in #420) simulates a missing os.getuid but on Windows the frozen os module simply has no getuid, so the test itself errors out. It is unrelated to this PR (which only touches octop/infra/gateway/process/processor.py — the gateway session processor, nothing in the browser module).

  • Linux / Python 3.12: ✅ success (full test suite passed)
  • Windows / Python 3.12: ❌ fails on the pre-existing test_browser_setup.py os.getuid compatibility issue

Could the maintainers take a look at making test_temp_scope_token_uses_username_when_getuid_missing Windows-compatible (e.g. skip when os.name == "nt")? Once that test is fixed, this PR should go green.

@veenyi

veenyi commented Aug 25, 2026

Copy link
Copy Markdown
Author

沟通语言说明

后续沟通请用中文(团队是腾讯中国团队)。以下是 Windows CI 失败的中文说明:

Windows CI 失败是本 PR 之前就存在的问题,与本 PR 无关

tests/unit/browser/test_browser_setup.py::test_temp_scope_token_uses_username_when_getuid_missing
AttributeError: <module os (frozen)> has no attribute getuid

该测试由 #420 引入,模拟 getuid 缺失场景,但 Windows 的 frozen os 模块根本没有 getuid,测试本身报错。它只涉及 browser 模块,与本 PR 改的 processor.py(网关会话处理)无关。

  • Linux / Python 3.12:✅ 全绿
  • Windows / Python 3.12:❌ 挂在官方 test_browser_setup.py 的 os.getuid 兼容问题

建议维护者将该测试在 Windows 上跳过(如 pytest.mark.skipif(os.name == "nt"))。修好后本 PR 即可全绿。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant