Rework the Prompt Inspector and give history trimming a cache-friendly mode - #392
Merged
Conversation
The Prompt Inspector mixed three looks — GlassSurface tiles, Material rows at 3 % onSurface and hand-styled Colors.white containers — and answered "what goes into the prompt?" twice: once as the next request, once as a coverage row beside it. Rebuild it on the card the chat header already wears. - Add InspectorPlaque, the context card's shell (surface @ 94 %, primary hairline, soft shadow) with no BackdropFilter, so it is safe per row of a 200-message list where GlassSurface was not. - Collapse the timeline's two top rows into one: coverage of the next request is now a block inside its preview, between the parameters and the messages, exactly where a captured request carries its own. - Give a captured request the preview's layout via a shared RequestBodyView — budget bar, parameters, coverage, filtered messages — so past and next requests stop looking like two features. - Drop the bare Material the surface hid behind: Card, LinearProgressIndicator, TextButton, and every Colors.white/red/green/orange literal. - Delete the now-unreferenced CoveragePanel sheet and update the widget-file list in the UI→DB characterization test. Verified: flutter analyze clean; flutter test (3821 passing); installed on an emulator and walked the timeline, next request, a captured request and the coverage block. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…pped mode Trimming history one message at a time moves the start of the prompt on almost every turn, so markStablePrefixCacheControl finds nothing in common with the last request and a provider's prefix cache misses every single turn — exactly for the long chats that need it most. Give the connection a choice. ## Trim modes - Add ApiConfig.historyTrimMode: `sliding` (the existing cut, still the default) and `stepped`, which holds the start of the history still and jumps it forward a block at a time. - Anchor the stepped cut on the oldest message it kept and honour it for as long as the window it opens still fits the budget; a raised max output tokens, a grown lorebook or a deleted anchor moves it, nothing else does. - Store the anchor as a reserved `__historyAnchor` session var through the existing atomic updateSessionVarsJson, so it needs no new column and no read-modify-write. Only the generation path writes it — a preview shows what would be sent without moving it. - Expose the two knobs the mode spends as connection fields, shown under the mode picker only when it is on `stepped`: step threshold (85 %) and step size (30 %). ## Where the settings live - Add a Context section to the connection: trim mode, then max output tokens and context size, which used to sit among the sampling knobs — the two numbers that decide what the model *sees* did not belong next to the ones that decide how it writes. - Put every sampling parameter back into Generation Parameters; they were a card of their own behind "Advanced", which hid the controls people reach for most often. - Give MenuSelectorItem the same outline, fill and metrics as MenuFieldItem's text field, so a selector stops reading as a different kind of control from the fields around it. MenuRangeItem gains the `description` both siblings already had. ## Keeping the previews honest - Key TokenBreakdownCache on the trim settings as well as the window, so a breakdown taken under one mode is never served under another. - Rebuild the Context tab and the next-request preview when the connection's context signature changes, and invalidate the shared breakdown from the save itself, so surfaces nobody has open (drawer stats, the chat header card) are not left stale. ## Cleanup - Reword the cutoff notice to "N messages didn't fit into context size" and put a gear on it that opens the connection at the Context section. - Remove the manual "hide top N" action, its two global AppSettings knobs and the near-limit banner: curating a chat is what message selection and hide are for, and the cache workaround is now a trim mode. Verified: new test/history_trim_mode_test.dart (11 cases) covers both modes, anchor hold/move, a deleted anchor, the max-output-tokens interaction and cache invalidation; flutter analyze clean; flutter test 3821 passing; migration 133 -> 135 applied on a real device database (columns present, existing connection left on `sliding`), and the Context section walked on an emulator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Selecting a long run meant tapping each message. The selection toolbar gains two buttons that extend the selection from the last tapped message to the top or the bottom of the chat; tapping again with that run already selected clears it. Authored in the working tree by hydall; carried in this PR rather than left uncommitted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prompt Inspector
InspectorPlaque— the context card's shell from the chat header (surface @ 94 %, primary hairline, soft shadow). It paints noBackdropFilter, so unlikeGlassSurfaceit is safe per row of a 200-message list.RequestBodyView, so a past and a next request stop looking like two different features.Card,LinearProgressIndicator,TextButton, and everyColors.white/red/green/orangeliteral.CoveragePanelsheet and updated the widget-file list in the UI→DB characterization test.History trimming
Cutting history one message at a time moves the start of the prompt on almost every turn, so
markStablePrefixCacheControlfinds nothing in common with the previous request and a provider's prefix cache misses every turn — precisely for the long chats that need it most.ApiConfig.historyTrimMode:sliding(the existing cut, still the default) andstepped, which holds the start of the history still and moves it a block at a time.__historyAnchorsession var through the existing atomicupdateSessionVarsJson— no new column, no read-modify-write. Only the generation path writes it, so opening a preview never moves it.steppedis selected: step threshold (85 %) and step size (30 %). Both are per connection.Connection settings
MenuSelectorItemnow has the same outline, fill and metrics asMenuFieldItem's text field, so the protocol selector stops reading as a different kind of control from the fields around it.MenuRangeItemgains thedescriptionboth siblings already had.Keeping the previews honest
TokenBreakdownCacheis keyed on the trim settings as well as the window, so a breakdown taken under one mode is never served under another.Cleanup
AppSettingsknobs and the near-limit banner. Curating a chat is what message selection and hide are for; the cache workaround is a trim mode now.Message selection
Verification
test/history_trim_mode_test.dart— 11 cases covering both modes, anchor hold and move, a deleted anchor, the max-output-tokens interaction, and cache invalidation across a mode or knob change.flutter analyzeclean;flutter test3821 passing.user_versionbumped, all three columns present, the existing connection left onslidingso behaviour is unchanged until someone opts in.Known: three
chat_input_barclipboard tests fail on a Windows host — the test helper's Windows branch handspasteboarda file path the mock never reads back. They are host-dependent and pass on CI'subuntu-latest; unrelated to this PR's changes.