Skip to content

feat(chat): mark where the prompt's history begins - #395

Merged
hydall merged 1 commit into
nightlyfrom
feat/context-limit-marker
Sep 6, 2026
Merged

feat(chat): mark where the prompt's history begins#395
hydall merged 1 commit into
nightlyfrom
feat/context-limit-marker

Conversation

@hydall

@hydall hydall commented Sep 6, 2026

Copy link
Copy Markdown
Owner

The chat gave no sign of where the context window starts: the tokenizer sheet counted the messages that did not fit, three taps away, and the chat itself rendered a cut message and a carried one identically. This draws the boundary where it is read — a CONTEXT LIMIT rule inside the oldest message the next prompt still carries.

The marker

  • Add TokenBreakdown.windowStartMessageId — the oldest message the trim kept. Read from trimmedHistory rather than from historyAnchorId, which only HistoryTrimMode.stepped fills in, so the same boundary comes out of either trim mode.
  • Add contextWindowStartProvider (lib/features/chat/state/context_window_marker.dart): the id the chat marks, or null when the chat fits whole (cutoffIndex == 0), when no prompt has been calculated yet, or when the cached breakdown belongs to a session that is no longer open.
  • Push it over the bridge as MessageBridgeCommands.setContextWindowStart. The page patches only the two rows involved, reaching them through itemMap, so the rule lands whether or not the boundary is currently mounted — and the renderer keeps the id, so a full re-render (a preset switch, a scrollback batch) draws it again without another call from Flutter.
  • Draw the rule as the first child of the in-prompt message's section: same shape as a date separator, amber like the cutoff notice in the tokenizer sheet that counts the same messages.
  • Seed it from ChatWebViewInitializer before the first paint. The WebView is kept alive across chats, so the renderer would otherwise carry the previous chat's boundary into this one — the same reason the search state is pushed early.

Keeping the numbers honest

  • Key the drawer's TokenBreakdownCache on the connection's contextBudgetSignature, as tokenizer_sheet.dart already does. Without it a breakdown taken under one trim mode could be served after a switch to the other, and both the drawer's numbers and the new rule come off that cache.
  • Carry historyAnchorId and visibleMessageIds through MagicDrawerStatsService's vector-lore rebuild of a breakdown instead of dropping them.

Verification

  • test/context_window_marker_test.dart (11 cases): windowStartMessageId under both trim modes — the sliding cut moves every turn, the stepped boundary holds still while its anchor does — plus the provider end to end against a real in-memory chat, asserting the rule appears in both modes and is withheld for a chat that fits, a stale cross-session breakdown, and a chat with no calculated prompt.
  • test/webview_js/specs/context_limit.spec.js (5 cases): the rule renders inside the right message as its first child, moving the boundary leaves exactly one, an empty id retires it, a re-render redraws it without another push, and an id this chat has no message for draws nothing.
  • flutter analyze: no errors (9 pre-existing warnings/infos on the base branch). npx playwright test: 86 passed. flutter test: 3851 passing; the 3 failures in test/chat_input_bar_test.dart (clipboard attachment thumbnails) also fail on nightly unchanged and are unrelated to this PR.

🤖 Generated with Claude Code

The chat gave no sign of where the context window starts: the tokenizer
sheet counted the messages that did not fit, three taps away, and the chat
itself rendered a cut and an uncut message identically. Draw the boundary
where it is read.

- Add `TokenBreakdown.windowStartMessageId` — the oldest message the trim
  kept. Taken from `trimmedHistory` rather than from `historyAnchorId`, which
  only stepped mode fills in, so the same boundary comes out of either trim
  mode.
- Add `contextWindowStartProvider`: the id the chat marks, or null when the
  chat fits whole, when no prompt has been calculated yet, or when the cached
  breakdown belongs to a session that is no longer open.
- Push it over the bridge as `setContextWindowStart`. The page patches only
  the two rows involved, through `itemMap`, so the rule lands whether or not
  the boundary is currently mounted; the renderer keeps the id, so a full
  re-render draws it again without another call.
- Draw the rule inside the first in-prompt message, styled like a date
  separator and amber like the cutoff notice that counts the same messages.
- Seed it from the initializer: the WebView is kept alive across chats, so
  the renderer would otherwise carry the previous chat's boundary into this
  one.

## Keeping the numbers honest

- Key the drawer's `TokenBreakdownCache` on the connection's context signature
  as the tokenizer sheet already does, so a breakdown taken under one trim
  mode is never served under another.
- Carry `historyAnchorId` and `visibleMessageIds` through the drawer's
  vector-lore rebuild of a breakdown instead of dropping them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@hydall
hydall merged commit 13beedb into nightly Sep 6, 2026
3 checks passed
@hydall
hydall deleted the feat/context-limit-marker branch September 6, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant