feat(chat): attach pasted images, up to four per message - #389
Merged
Conversation
Ctrl/Cmd+V in the composer now attaches whatever image is on the clipboard, and a message can carry up to four of them — laid out as a grid in the bubble rather than one picture under another. ## Clipboard paste - `ClipboardImages` (`core/platform/clipboard_images.dart`) reads the system clipboard through `pasteboard`: a bitmap (browser copy, screenshot tool) or, on desktop, the image files copied in the file manager. `Clipboard` from `flutter/services.dart` only ever sees `text/plain`, so neither was reachable before. - The MIME type is sniffed from the magic bytes, not the extension — a clipboard bitmap has no filename, and a mislabelled type is what makes a provider reject an otherwise valid multimodal request. - The composer takes the paste shortcut over wholesale: an image is attached, anything else is inserted as text at the caret. Letting the field also paste natively put the source URL of a copied picture into the box next to it. - A new `paste` composer action covers touch, where there is no shortcut to press. It ships as a card in the drawer's Actions tab rather than a fifth button in the row. ## Up to four attachments - `maxMessageAttachments` (4) caps the composer; the file picker now takes several files at once, and the preview strip becomes a row of square thumbnails past the first image. - Storage keeps the pair split: `ChatMessage.imagePath` is the first attachment and `extraImagePaths` the rest, so sessions written before this still render. `ChatMessage.attachments` is the one list every reader uses and `splitAttachments` the one writer. - `PromptMessage` carries `imagePaths`, and `toApiMap()` emits one `image_url` content part per image, in order. The eye toggle still covers the whole message (INV-PS1b). - The WebView renders one `.msg-image-attachment` block holding every picture: free-size for one, a tile grid for two to four (2-up, one tall plus two stacked, 2x2). Keeping it a single element is what lets `updateMessage` lift it out of the body and put it back. ## Verified - `flutter analyze` — 9 issues, all pre-existing on `nightly`. - `flutter test` — 3811 passed, including new coverage for the paste shortcut, the four-image cap, the multi-part request and the MIME sniffing. - `test/webview_js` — 81 passed, including a new `attachments.spec.js` that measures the grid the browser actually lays out for one to four images. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ2H48SAD3gVT4wEeMRG4E
Follow-up to the multi-attach change. ## No paste button of our own - Removed the `paste` composer action, its drawer card and its label. The system already offers paste wherever a paste is possible. - The composer's `TextField` now builds its own selection toolbar with **Paste** rewired to the image-aware handler, and adds the item when Flutter left it out — which it does whenever the clipboard holds no *text*, exactly the case an image paste has to survive. That is the way in on touch; Ctrl/Cmd+V is unchanged. ## No limits - Dropped `maxMessageAttachments`: the composer, the send path and the renderer take as many images as they are handed. - Dropped the 12 MB cap in `ClipboardImages` and the `limit` on its read. - The attachment grid gains `.count-many` — a three-up layout of square tiles that grows downwards, so any count past the named 1..4 layouts still renders as a grid. - The composer's preview strip became one horizontally scrolling row of thumbnails; a wrapping grid of them would have pushed the composer off the screen once the count stopped being bounded. ## Verified - `flutter analyze` — 9 issues, all pre-existing on `nightly`. - `flutter test` — 3811 passed, including a rewritten composer test that pastes six images and sends all six, and one that drives the toolbar's Paste item and asserts it attaches rather than inserts text. - `test/webview_js` — 81 passed, including a seven-image case measuring the three-up grid the browser lays out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ2H48SAD3gVT4wEeMRG4E
… size cap Corrects the previous commit: the four-image cap is back, and it is the size limit that is gone. ## Keyboard-committed images Gboard's clipboard chip and its sticker/GIF pickers never put anything on the clipboard — the keyboard commits the content straight into the field over the input connection, so neither the paste shortcut nor the selection toolbar could see it. The composer's `TextField` now declares `contentInsertionConfiguration`, and the bytes the engine hands over are attached like any other picture. Content the engine could not read (no bytes, only a `content://` URI, which nothing in Dart can open) is logged and dropped rather than attached empty. The MIME type a keyboard declares is treated as a fallback, not as truth: `encodeImageDataUrl` still sniffs the magic bytes first, and only falls back to the declared type — normalising Android's unregistered `image/jpg` to `image/jpeg` — when the header says nothing. ## Limits - `maxMessageAttachments` (4) is back: the composer, the send path and the attachment grid all cap at four again, with the toast on the fifth. - The 12 MB cap on a clipboard image is gone for good. Size is not something to refuse over. ## Verified - `flutter analyze` — 9 issues, all pre-existing on `nightly`. - `flutter test` — 3814 passed, including new tests that drive the content-insertion callback with and without bytes, and the restored fifth-paste-refused case. - `test/webview_js` — 81 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ2H48SAD3gVT4wEeMRG4E
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A message can now carry up to four images instead of one, and they can come
from the clipboard rather than only from the file picker. The bubble lays
several of them out as a grid rather than stacking one picture under another.
Getting an image in
ClipboardImages(core/platform/clipboard_images.dart), which readsthe system clipboard through the new
pasteboarddependency: a bitmap(browser copy, screenshot tool) or, on desktop, the image files copied in the
file manager.
Clipboardfromflutter/services.dartonly ever seestext/plain, so neither was reachable before._handleComposerKey: animage is attached, anything else is inserted as text at the caret. Letting
the field also paste natively put the source URL of a copied picture into the
box next to it.
_buildContextMenurebuilds the field's own selection toolbar with Pasterewired to the same handler, and adds the item when Flutter left it out —
which it does whenever the clipboard holds no text, exactly the case an
image paste has to survive. That is the long-press route on touch.
TextFielddeclarescontentInsertionConfiguration, sopictures the on-screen keyboard commits — Gboard's clipboard chip and its
sticker/GIF pickers — are attached too. Those never reach the clipboard at
all, so no paste path could see them. Content the engine could not read (a
content://URI with no bytes, which nothing in Dart can open) is logged anddropped rather than attached empty.
possible.
encodeImageDataUrlsniffs the MIME type from the magic bytes rather thantrusting the source. A clipboard bitmap has no filename and a keyboard names
whichever of the requested types it likes; a mislabelled type is what makes a
provider reject an otherwise valid multimodal request. A declared type is
consulted only when the header says nothing, and Android's unregistered
image/jpgis normalised toimage/jpeg.Carrying several images
maxMessageAttachments(4, incore/models/chat_message.dart) caps thecomposer; the file picker takes several files at once and the preview strip
becomes a row of square thumbnails past the first image.
ChatMessage.imagePathis the firstand the new
extraImagePathsthe rest — so sessions written before thisstill render.
ChatMessage.attachmentsis the one list every reader uses andsplitAttachmentsthe one writer of the pair.PromptMessagenow carriesimagePathsinstead of a singleimagePath, andtoApiMap()emits oneimage_urlcontent part per image, in order. Everyrebuild site that used to copy
imagePath(HistoryAssembler.assemble,buildFallbackPrompt,applyPromptRegexes,StudioHistoryLimiter,StudioStreamInterceptor, the memory injector, both append-to-last builders)copies the list. The eye toggle still covers the whole message (INV-PS1b).
ChatMessageMappersendsimagePathsalongsideimagePath(still thefirst, for anything that only knows the single-image shape), and
_resolveMappedFileUrlsresolves each path, dropping any the WebView cannotreach instead of rendering a broken tile.
.msg-image-attachmentblock holding every picture:free-size for one, a tile grid for two to four (2-up, one tall plus two
stacked, 2x2). Keeping it a single element is what lets
updateMessageliftit out of the body and put it back, and what keeps one eye toggle per
message.
docs/INVARIANTS.md.Verified
Run against a local Flutter 3.44.9 (the same version CI pins):
flutter analyze --no-fatal-infos --no-fatal-warnings— 9 issues, all ofthem pre-existing on
nightlyand none in the files this PR touches.flutter test— 3814 passed. New coverage: the paste shortcut end to end(clipboard image → thumbnail → data URL in the send callback), the fall
through to a text paste when the clipboard holds no image, the fifth
attachment being refused, the toolbar's Paste item, the content-insertion
callback with and without bytes, the multi-part request, the pre-multi-attach
JSON payload, and the MIME sniffing.
cd test/webview_js && npm ci && npx playwright test— 81 passed, includinga new
attachments.spec.jsthat measures the grid a headless Chromiumactually lays out for one to four images.
Not run: a real device or desktop build. The clipboard and keyboard paths cross
a platform channel, so they are covered here by mocked channels rather than by
a real Gboard.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MJ2H48SAD3gVT4wEeMRG4E
Generated by Claude Code