Skip to content

feat(desktop): show context compaction in transcript - #3651

Open
liuxiaocs7 wants to merge 1 commit into
apache:mainfrom
liuxiaocs7:feat/3587-context-compaction-transcript
Open

feat(desktop): show context compaction in transcript#3651
liuxiaocs7 wants to merge 1 commit into
apache:mainfrom
liuxiaocs7:feat/3587-context-compaction-transcript

Conversation

@liuxiaocs7

@liuxiaocs7 liuxiaocs7 commented Aug 23, 2026

Copy link
Copy Markdown
Member

Summary

  • Project explicit context compaction as a durable transcript Turn with running, compacted, unchanged, failed, and interrupted states.
  • Carry the compaction execution kind and typed outcome through persistence, recovery, repair, and Runtime Host wire projections.
  • Render one localized inline system row in transcript order while suppressing the duplicate legacy compaction note.
  • Keep the existing running and terminal Toast feedback; the transcript row provides the durable in-context record requested by the issue.

Fixes #3587

Verification

  • npm --workspace @maka/core run typecheck
  • npm --workspace @maka/runtime run typecheck
  • npm --workspace @maka/runtime-host run typecheck
  • npm --workspace @maka/ui run typecheck
  • Builds passed for @maka/core, @maka/runtime, @maka/runtime-host, and @maka/ui.
  • 368 targeted tests passed across UI materialization, Runtime read-model/repair/session lifecycle, and Runtime Host protocol/projection.
  • Biome lint and format checks passed for all changed files.
  • Full-repository format check was not clean because the user-owned untracked docs/contribution/.zhwork/manifest.json needs formatting; it is not part of this PR.
  • The latest main currently has an unrelated @maka/storage test compilation error: codex-session-adapter.test.ts calls decodeStoredMessage without importing it.
  • Manually verified in Desktop that the existing compaction Toast remains and the inline transcript row updates from 正在压缩上下文… to 上下文已压缩。.
image image

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Codex implemented the Runtime, wire, and UI projection changes, added tests, addressed review findings, and assisted with verification.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@liuxiaocs7
liuxiaocs7 force-pushed the feat/3587-context-compaction-transcript branch from 5548e5c to 2332d67 Compare August 23, 2026 21:05
@liuxiaocs7

Copy link
Copy Markdown
Member Author

wait #3653

@jackwener jackwener left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review of 2332d67d0dff4eb03b37761b27f62451568e4414.

GO. No P0–P2. MERGEABLE but BLOCKED. Hosted test on this head is terminal failure; I am not approving. The failing compile is packages/storage/src/__tests__/codex-session-adapter.test.ts using decodeStoredMessage without importing it — that file is not in this diff.

What this solves

Explicit context compaction was a toast-only side effect. This PR makes that dedicated compact Turn durable in the transcript: rootExecutionKind: 'context_compact' plus a typed contextCompactionOutcome, projected through persistence, repair, Host wire (epoch 44→45), and one localized system row.

That is a defined external contract, not a leak of mid-turn internal compaction. Automatic in-turn compact still uses the existing system_note (context_compacted / context_compaction_failed_open). The two paths are different executions and keep different copy, which is the right cut.

Boundary / predicate (the questions for this PR)

  • Running / terminal: label is turn.status for running and aborted; otherwise outcome.kind (compacted / unchanged / failed). Internal reason / checkpointId stay off the UI.
  • Failure: a failed compact still writes the legacy context_compaction_failed_open note, then suppresses it when the same turnId is a compact Turn, so the user sees one row.
  • Multiple: compact is refused while another Turn is running. Repeated compact is separate Turns; each turnId emits one row, in storage order.
  • History: sessions from before this field was persisted do not grow a new row (no rootExecutionKind). Old visible system_notes still render. No reorder/hole introduced by the new projector.
  • Predicate sufficiency: the UI keys off the compact-Turn identity, not “a compact-shaped event happened.” Missing outcome on a completed compact Turn fail-closes to the failed label (same as Host context-coordinator). That is conservative, not a premature unlock.

No inline findings. No merge from me.

@liuxiaocs7
liuxiaocs7 force-pushed the feat/3587-context-compaction-transcript branch from 2332d67 to d31caba Compare August 24, 2026 01:37
@liuxiaocs7

Copy link
Copy Markdown
Member Author

#3656 resolved the build error

@liuxiaocs7
liuxiaocs7 requested a review from jackwener August 24, 2026 02:21
@liuxiaocs7

Copy link
Copy Markdown
Member Author

Hi, @Astro-Han, could you help take a look when free, thanks!

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this — the core contract work is solid. The strict isContextCompactionOutcome validator, the epoch bump, and the projector/snapshot reuse all line up, and CI is green on this head. I have four things I would like resolved before this goes in, three of which are about paths where the new identity does not survive.

[P2] Repair can never enrich the history that needs it most.
This PR threads rootExecutionKind and contextCompactionOutcome all the way into #appendTurnStateIfMissing, but that method's early return is unchanged: it bails when a terminal turn_state row already exists with the same status. For a session whose compaction completed before this upgrade, exactly that row exists — terminal, same status, missing both new fields. So the enrichment is unreachable for the population it was written for. Combined with terminal transcript reads being storage-only (packages/runtime-host/src/server/session-transcript-reader.ts:52-57 returns [] from readActiveOverlay once the root turn is terminal), the canonical facts are present — the Run header already carries rootExecutionKind: context_compact, the terminal RuntimeEvent already carries the outcome — but nothing renders. Reopen a session with a completed pre-upgrade compaction and no inline row appears. I think repair should append an enriched terminal projection when the semantic fields are missing, not only when the status differs. Inline note on the guard.

[P2] A copied conversation's terminal outcome points at a checkpoint that does not exist in the target.
packages/runtime/src/conversation-copy.ts:679-700 rebuilds the checkpoint under a fresh target ID and records the mapping in checkpointIds. But RuntimeEvents are cloned earlier at :330-350, and rewriteRuntimeEventActions (:1052-1059) only rewrites toolDispatch/toolRecovery operation IDs — everything else passes through the ...actions spread untouched, including stateDelta.contextCompactionOutcome. Since { kind: 'compacted', checkpointId } is introduced by this PR, the rewriter has no way to know about it yet. The result is a target terminal Turn whose outcome names the source checkpoint, while the target's own history_compact_checkpoint_recorded event names a different one; the read model and wire then publish the stale reference. Branch through a successful explicit compaction and compare the two IDs. Rewriting the outcome once the checkpoint map is known would close this.

[P2] Branch and edit-and-resend copies lose the compaction row entirely.
packages/runtime/src/conversation-copy.ts:174-180 drops every turn_state message from a copy slice, and cloneConversationRuntimeLedger returns only that stripped set (:393-395, :452-454). Because terminal reads take no RuntimeEvent overlay, a copied conversation has no stored rootExecutionKind row for packages/ui/src/materialize.ts:171-174 to render. A state-only compaction is affected more sharply: retained Run selection is recomputed from the stripped messages at :227-233, so those Runs are not carried over at all.

To be clear about attribution: that strip predates this PR — conversation-copy.ts is untouched here, and the same line exists at the merge-base. I am raising it because this PR is what makes stored turn_state load-bearing for rendering, so the gap only becomes visible now. It is a coverage hole in the new feature rather than a regression, and if you would rather handle it in a follow-up I am fine with that as long as it is written down somewhere.

[P3] The new epoch test cannot detect removal of the bump it guards.
Inline note.

Two smaller things I did not count as findings: the compaction-row projection is duplicated between materializeChat and materializeTurns, and rootExecutionKind + contextCompactionOutcome travel as an optional pair that permits combinations the domain does not have. Both are judgment calls and I am happy to leave them.

Because these are open items rather than a green light, I am leaving this as a comment rather than an approval. This is also a feature rather than a fix, so the merge call is a human's either way.

简体中文

感谢这个 PR,核心契约部分做得很扎实:严格的 isContextCompactionOutcome 校验、epoch 提升、以及 projector/snapshot 复用同一个校验器,都对得上,这个 head 上 CI 也是绿的。合并前我有四点想解决,其中三点都是"新身份在某条路径上没能存活下来"。

[P2] repair 恰恰无法补全最需要补全的那部分历史。
这个 PR 把 rootExecutionKindcontextCompactionOutcome 一路传进了 #appendTurnStateIfMissing,但该方法的提前返回没有改:当已存在一条同状态的终态 turn_state 行时它就直接返回。而对于升级前就完成了压缩的会话,存在的正是这样一行——终态、同状态、缺少这两个新字段。于是这段补全逻辑对它本该服务的人群是不可达的。再加上终态 transcript 读取是纯存储的(packages/runtime-host/src/server/session-transcript-reader.ts:52-57,root turn 一旦终态,readActiveOverlay 就返回 []),权威事实其实都在——Run header 已经记了 rootExecutionKind: context_compact,终态 RuntimeEvent 也已经记了 outcome——但什么都渲染不出来。打开一个升级前已完成压缩的会话,行内那条记录不会出现。我认为 repair 应该在语义字段缺失时就追加一条补全后的终态投影,而不是只在状态不同时才追加。已在那个守卫处留了行内注记。

[P2] 复制出来的会话,其终态 outcome 指向一个在目标会话中并不存在的 checkpoint。
packages/runtime/src/conversation-copy.ts:679-700 会用新的目标 ID 重建 checkpoint,并把映射记进 checkpointIds。但 RuntimeEvent 在更早的 :330-350 就已经克隆完了,而 rewriteRuntimeEventActions:1052-1059)只改写 toolDispatch/toolRecovery 的 operation ID,其余内容都经由 ...actions 原样透传,其中就包括 stateDelta.contextCompactionOutcome。由于 { kind: 'compacted', checkpointId } 是本 PR 新引入的,改写器还无从知晓它。结果是:目标会话的终态 Turn,其 outcome 指向源 checkpoint,而目标自己的 history_compact_checkpoint_recorded 事件指向另一个 ID;read model 和 wire 随后发布的是过期的那个。从一次成功的显式压缩处 branch 出去,对比这两个 ID 即可复现。在拿到 checkpoint 映射之后再改写 outcome 就能解决。

[P2] branch 和 edit-and-resend 复制会完全丢失压缩记录行。
packages/runtime/src/conversation-copy.ts:174-180 会从复制切片中剔除所有 turn_state 消息,而 cloneConversationRuntimeLedger 返回的正是这个被剔除后的集合(:393-395, :452-454)。由于终态读取不叠加 RuntimeEvent,复制出的会话里没有任何带 rootExecutionKind 的存储行供 packages/ui/src/materialize.ts:171-174 渲染。仅有状态、没有普通消息的压缩 Turn 受影响更重:保留 Run 的选择在 :227-233 处是基于剔除后的消息重算的,因此那些 Run 根本不会被复制过去。

归因上我说清楚:这个剔除早于本 PR——conversation-copy.ts 在本 PR 中未被改动,merge-base 上就有同一行。我提出来,是因为本 PR 让存储的 turn_state 成为渲染的承重结构,这个缺口到现在才显现。它属于新功能的覆盖空缺,而不是回归;如果你更愿意放到后续 PR 处理,我没有意见,只要有个地方把它记下来。

[P3] 新增的 epoch 测试无法发现它所守护的那次提升被移除。
已留行内注记。

另外两点我没有计为 finding:压缩行的投影在 materializeChatmaterializeTurns 之间存在重复;rootExecutionKindcontextCompactionOutcome 作为一对可选字段一起传递,允许出现领域中并不存在的组合。这两点都是取舍问题,保持现状我也接受。

由于这些是待办事项而不是放行,我这次留的是评论而非 approve。另外这是新功能而不是修复,合并与否本来也要由人类决定。

Comment thread packages/runtime/src/runtime-ledger-repair.ts
Comment thread packages/runtime-host/src/__tests__/protocol.test.ts Outdated
@Astro-Han

Copy link
Copy Markdown
Contributor

Update on the epoch: this branch now conflicts with main, and that is my doing rather than yours.

#3573 merged a few minutes ago as d77e854f7, and it publishes epoch 47. main is therefore at 47 now, and git merge-tree origin/main 9e525b24d8 reports a conflict in packages/runtime-host/src/protocol/index.ts — both sides declare 47 with their own note.

I should be clear that I caused this. When I rebased #3573 I picked 47 without first checking whether another open PR had already claimed it, and yours had. Nothing in your branch was wrong; it just lost a race it was never told it was in.

What this needs when you next touch the branch:

I scanned all open PRs that declare an epoch: after this merge, no other one claims 48, so 48 is free. Every other open branch sits at 45 or below and will renumber whenever it rebases.

Happy to push the rebase to your branch if that is easier — just say the word. Otherwise the three findings in my earlier review are the substantive part; this is only bookkeeping.

简体中文

epoch 的进展同步一下:这个分支现在和 main 冲突了,而这是我造成的,不是你的问题。

#3573 在几分钟前以 d77e854f7 合入,它发布的是 epoch 47。因此 main 现在就是 47,git merge-tree origin/main 9e525b24d8 会在 packages/runtime-host/src/protocol/index.ts 报冲突——两边都声明了 47,各自带着自己的说明。

我要说清楚:这是我造成的。我在 rebase #3573 时选了 47,事先没有检查是否已有别的开放 PR 占用了这个号,而你的 PR 正好占了。你的分支没有任何地方写错,它只是输掉了一场从没被告知存在的竞速。

等你下次动这个分支时需要做的:

我扫过所有声明了 epoch 的开放 PR:本次合并之后没有别的 PR 占用 48,所以 48 是空的。其余开放分支都停在 45 及以下,各自 rebase 时再重新编号即可。

如果由我把 rebase 推到你的分支更省事,说一声就行。除此之外,我先前那条 review 里的三个问题才是实质部分,这一条只是记账。

@liuxiaocs7
liuxiaocs7 force-pushed the feat/3587-context-compaction-transcript branch from 9e525b2 to ade23fb Compare August 24, 2026 10:54
@Astro-Han

Copy link
Copy Markdown
Contributor

Correction to my previous comment: do not take 48. #3669 already claims it, and I did not check before suggesting a number. That is the same mistake that produced this conflict in the first place, so let me give you the rule instead of another number.

Current state: main publishes 47. Among open PRs, #3669 declares 48 and #3479 still declares 47 (stale, it will renumber when it rebases). Nothing declares 49.

So the reliable procedure, at the moment you rebase:

  1. Read what main publishes: git show origin/main:packages/runtime-host/src/protocol/index.ts | grep 'COMPATIBILITY_EPOCH = '
  2. Check which numbers above it are already claimed by open PRs:
for n in $(gh pr list --state open --limit 300 --json number -q '.[].number'); do
  H=$(gh pr view $n --json headRefOid -q .headRefOid)
  v=$(git show $H:packages/runtime-host/src/protocol/index.ts 2>/dev/null \
      | grep -oP 'COMPATIBILITY_EPOCH = \K[0-9]+')
  [ -n "$v" ] && echo "#$n = $v"
done | sort -t= -k2 -n | tail
  1. Take the lowest number that is above main and unclaimed, and set the new assertion in protocol.test.ts to > (yourNumber - 1) so it actually guards your own bump.

As of right now that gives you 49, but please re-run step 2 rather than trusting that figure — #3669 may land before you rebase, which would move main to 48 and leave 49 correct anyway, or another branch may claim it in between.

Sorry for the churn. The three findings in my earlier review are unaffected by any of this — #3573 only touched the epoch declaration, so those still stand at whatever head you land on.

简体中文

更正我上一条评论:不要用 48。 #3669 已经占了 48,而我上次建议这个数字之前没有检查——这正是最初造成这次冲突的同一个错误。所以这次我给你规则,而不是再给一个数字。

当前状态:main 发布的是 47。开放 PR 中,#3669 声明了 48,#3479 仍声明 47(已过期,它 rebase 时会重新编号)。没有任何 PR 声明 49。

因此,在你 rebase 的那一刻,可靠的做法是:

  1. main 当前发布的值:git show origin/main:packages/runtime-host/src/protocol/index.ts | grep 'COMPATIBILITY_EPOCH = '
  2. 检查比它大的号里哪些已被开放 PR 占用:
for n in $(gh pr list --state open --limit 300 --json number -q '.[].number'); do
  H=$(gh pr view $n --json headRefOid -q .headRefOid)
  v=$(git show $H:packages/runtime-host/src/protocol/index.ts 2>/dev/null \
      | grep -oP 'COMPATIBILITY_EPOCH = \K[0-9]+')
  [ -n "$v" ] && echo "#$n = $v"
done | sort -t= -k2 -n | tail
  1. 取大于 main 且未被占用的最小值,并把 protocol.test.ts 里新增的断言改成 > (你的号 - 1),这样它才真正守住你自己这次提升。

按现在的情况这会得到 49,但请你重新跑一遍第 2 步,不要直接采信这个数字——#3669 可能在你 rebase 之前合入,那会把 main 推到 48,49 仍然正确;也可能中间又有别的分支占了它。

为这些反复致歉。我先前 review 里的三个问题不受此影响——#3573 只改动了 epoch 声明,因此无论你最终落在哪个 head,那三条依然成立。

@liuxiaocs7
liuxiaocs7 force-pushed the feat/3587-context-compaction-transcript branch from c7623ff to a793027 Compare August 24, 2026 11:32
@liuxiaocs7

Copy link
Copy Markdown
Member Author

Addressed all four findings in a793027:

  • repair now enriches legacy same-status terminal turn-state rows with compaction identity/outcome;
  • conversation copies retain context-compaction turn-state rows, including state-only turns;
  • copied RuntimeEvent and turn-state compacted outcomes now reference the rebuilt target checkpoint ID and fail closed on missing mappings;
  • after rechecking open epoch claims, this branch moved to epoch 49 (PR fix(desktop): snapshot linked children for side conversations #3669 currently owns 48), with the guard pinned to > 48.

Local verification: targeted Runtime tests 18/18, Runtime Host protocol tests 49/49, full typecheck, lint, format check, and git diff --check all pass.

@Astro-Han

Copy link
Copy Markdown
Contributor

The epoch is now clean: this branch declares RUNTIME_HOST_COMPATIBILITY_EPOCH = 49 with a matching > 48 assertion, and 49 is not claimed by any other open pull request. Thanks for the renumber.

The remaining test failure is not caused by this branch. Check ASF source headers reports five files under packages/runtime/src missing the license header; those files are missing it on main as well, so the same failure reproduces without your changes. The fix has just landed on main in commit 389802684d24932ac8c9f37c9251a2ee73c2f4bb. Please rebase onto the latest main and the check should pass.

One thing to watch after the rebase: main will keep advancing, and 49 is only free as of now. If another branch lands an epoch bump before this one merges, the number will need to move again — worth re-checking at merge time rather than assuming it holds.

简体中文

epoch 现在是干净的:这个分支声明 RUNTIME_HOST_COMPATIBILITY_EPOCH = 49,断言写法 > 48 与之匹配,而且 49 没有被任何其它开着的 PR 占用。感谢重新编号。

剩下的 test 失败与这个分支无关。Check ASF source headerspackages/runtime/src 下五个文件缺少 license header,这五个文件在 main 上同样缺,不带你的改动也能复现同一个失败。修复刚刚合入 main,commit 为 389802684d24932ac8c9f37c9251a2ee73c2f4bb。请 rebase 到最新 main,这个检查即可通过。

rebase 后有一点需要留意:main 会继续前进,49 只是此刻空闲。如果在本 PR 合并之前又有分支落地了 epoch 变更,这个号还需要再改一次——建议在合并前重新确认,而不是假定它一直有效。

@liuxiaocs7
liuxiaocs7 force-pushed the feat/3587-context-compaction-transcript branch 2 times, most recently from 0654222 to 9537219 Compare August 24, 2026 13:16
@Astro-Han

Copy link
Copy Markdown
Contributor

Correction to my earlier comment: I said 49 was not claimed by any other open pull request. That is no longer true, and it was already becoming untrue when I wrote it.

#3299 also declares RUNTIME_HOST_COMPATIBILITY_EPOCH = 49. It is approved and waiting, and it was updated shortly before this branch was. main has also moved to 48 since I last checked, so 49 is now the immediate next number and both branches want it.

Whichever of the two merges first takes 49; the other will have to move to 50 and change its assertion to > 49. I am not asking you to renumber pre-emptively — the number only has to be right at the moment of merge, and moving now could just as easily collide again. What I would do is re-derive it right before merging rather than trusting any earlier answer, including mine:

git fetch origin main
git show origin/main:packages/runtime-host/src/protocol/index.ts | grep COMPATIBILITY_EPOCH
gh pr list --repo apache/maka --limit 200 --json number --jq '.[].number' | while read n; do
  gh api repos/apache/maka/pulls/$n/files --paginate \
    --jq '.[]|select(.filename=="packages/runtime-host/src/protocol/index.ts")|.patch' 2>/dev/null \
  | grep -oP '^\+export const RUNTIME_HOST_COMPATIBILITY_EPOCH = \K\d+' | sed "s/^/#$n /"
done

Note the ^\+ in that last pattern: it matches only lines the branch actually adds. Reading the file's current value instead reports whatever the branch inherited from main at its last rebase, which makes almost every branch look like it declares an epoch. That distinction is what I got wrong.

Sorry for the noise. The rest of my earlier comment stands: the 49 / > 48 pairing is the correct shape, and the ASF header failure is still not yours.

简体中文

更正我之前那条评论:我说 49 没有被任何其它开放 PR 占用。这句话现在不成立,而且在我写下它的时候就已经在失效了。

#3299 同样声明了 RUNTIME_HOST_COMPATIBILITY_EPOCH = 49 它已获批准、正在等待,并且更新时间就在本分支之前不久。另外 main 在我上次查看之后已经推进到 48,所以 49 现在是紧接着的下一个号,两个分支都想要它。

两者中先合并的那个拿到 49;另一个需要改成 50,并把断言改为 > 49我不是要你现在就改号——这个号只需要在合并那一刻是对的,现在改反而可能再撞一次。我的建议是:在合并前重新推导一次,而不是相信任何更早的答案,包括我给的。

git fetch origin main
git show origin/main:packages/runtime-host/src/protocol/index.ts | grep COMPATIBILITY_EPOCH
gh pr list --repo apache/maka --limit 200 --json number --jq '.[].number' | while read n; do
  gh api repos/apache/maka/pulls/$n/files --paginate \
    --jq '.[]|select(.filename=="packages/runtime-host/src/protocol/index.ts")|.patch' 2>/dev/null \
  | grep -oP '^\+export const RUNTIME_HOST_COMPATIBILITY_EPOCH = \K\d+' | sed "s/^/#$n /"
done

注意最后那个模式里的 ^\+:它只匹配分支真正新增的那一行。如果改为直接读文件里的当前值,读到的是该分支上次 rebase 时从 main 继承来的数字,会让几乎每个分支看起来都声明了 epoch。我之前弄错的正是这个区别。

抱歉打扰。之前那条评论的其余部分仍然成立:49> 48 的配对写法是对的,ASF header 的失败也依然不是你造成的。

Make context-compaction turns durable across runtime projection, recovery,
conversation copies, the Runtime Host protocol, and transcript UI surfaces.

Closes apache#3587

Generated-by: Codex
@liuxiaocs7
liuxiaocs7 force-pushed the feat/3587-context-compaction-transcript branch from a662967 to 30d6e1b Compare August 24, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(desktop): show context compaction in the task transcript

3 participants