Skip to content

feat: add parseCompressArgs — lenient compress-arg parsing with diagnostics - #111

Open
ranxianglei wants to merge 3 commits into
masterfrom
2026-08-22_compress-arg-parser
Open

feat: add parseCompressArgs — lenient compress-arg parsing with diagnostics#111
ranxianglei wants to merge 3 commits into
masterfrom
2026-08-22_compress-arg-parser

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

背景

跟踪 issue: #108

compress 参数解析目前散落在 5 个实现里,各自 strict JSON.parse、各自静默失败:

位置 现状
kernel rebuild.ts(fork-recovery) strict parse + 只收 Array.isArray(content);vLLM stringified content 静默丢全部压缩块
billion-context proxy safeJsonParse catch → {} → 模型收到 Compression FAILED: no valid ranges parsed
billion-context-pi strict parse 失败 throw(retry 依赖此,但无 salvage)
billion-context-omp strict parse catch → null,无日志
已删的 omp auto-compress(#74 曾有 fence 剥离 + 截断 JSON salvage,随架构调整删除——能力没有先提升到拥有它的层

生产事故形状:流截断(truncated JSON prefix)、markdown 围栏、尾逗号、字符串内裸换行、gateway 把参数 / 嵌套数组 stringify(vLLM,billion-context#176)。

改动

  • src/parse-compress-input.ts(新) parseCompressArgs(input, opts?: { callId }) → { ranges, diagnostics }
    • 接受:decoded object / JSON 字符串 / 一层 double-stringification / fenced JSON(带或不带语言标签)/ 尾逗号 / 字符串内裸换行 / vLLM stringified content
    • 截断输入走 salvage:字符串/转义/深度状态机从存活前缀恢复 content 数组中完整的条目;partial 条目丢弃,绝不发明
    • 字段名变体:startRef | startId | messageIdendRef | endId
    • diagnostics 是数据不是日志:kindok | empty-input | not-object | missing-content | content-not-array | malformed-json | truncated | no-valid-ranges,附 rawPrefix(≤800 字符)/ length / keys / invalidItems;adapter 决定落点(日志 / debug event / tool result)
  • src/rebuild.ts 删除第 5 个实现(strict CompressInputEntry + extractRanges),换用 parseCompressArgs——顺带修掉 vLLM fork-recovery 静默丢块
  • src/index.ts 导出 parseCompressArgs + CompressParseDiagnostics / CompressParseKind / ParsedCompressInput
  • DESIGN.md 职责表加行:compress argument parsing (lenient) = core

边界(不在本 PR)

验证

  • npm test:424 pass / 0 fail(新增 30 用例:全部生产形状 + salvage 边界 + rebuild 回归——strict object 不变量 / vLLM stringified 恢复 / 截断 salvage / garbage 不重建)
  • npm run typecheck 干净;npm run build 成功

…ostics

Single shared parser for compress tool arguments, replacing the five
divergent strict parsers across the stack (kernel rebuild, proxy, pi,
omp). Accepts fenced / trailing-comma / raw-newline / double-stringified
/ stringified-content shapes, salvages complete entries from truncated
JSON prefixes (partial entries dropped, never guessed), and returns
structured diagnostics (kind, rawPrefix, keys, invalidItems) as data
for adapters to emit. rebuildCompressionState now recovers vLLM
stringified content and truncated tool-call text instead of silently
dropping them.

Refs #108
…ars fallbacks

Downstream validation (proxy/omp/pi) surfaced three real shapes the
parser did not cover:
- a single range at the top level with no content array (proxy drift)
- top-level topic fallback (omp/pi schemas)
- top-level summaryMaxChars (omp/pi schemas)

All are now owned by the kernel so adapters can converge on it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant