Skip to content

feat(chat): attach pasted images, up to four per message - #389

Merged
hydall merged 3 commits into
nightlyfrom
claude/paste-images-clipboard-2dsdgf
Sep 5, 2026
Merged

feat(chat): attach pasted images, up to four per message#389
hydall merged 3 commits into
nightlyfrom
claude/paste-images-clipboard-2dsdgf

Conversation

@hydall

@hydall hydall commented Sep 5, 2026

Copy link
Copy Markdown
Owner

A message can now carry up to four images instead of one, and they can come
from the clipboard rather than only from the file picker. The bubble lays
several of them out as a grid rather than stacking one picture under another.

Getting an image in

  • Added ClipboardImages (core/platform/clipboard_images.dart), which reads
    the system clipboard through the new pasteboard dependency: a bitmap
    (browser copy, screenshot tool) or, on desktop, the image files copied in the
    file manager. Clipboard from flutter/services.dart only ever sees
    text/plain, so neither was reachable before.
  • The composer takes Ctrl/Cmd+V over wholesale in _handleComposerKey: an
    image is attached, anything else is inserted as text at the caret. Letting
    the field also paste natively put the source URL of a copied picture into the
    box next to it.
  • _buildContextMenu rebuilds the field's own selection toolbar with Paste
    rewired to the same handler, and adds the item when Flutter left it out —
    which it does whenever the clipboard holds no text, exactly the case an
    image paste has to survive. That is the long-press route on touch.
  • The composer's TextField declares contentInsertionConfiguration, so
    pictures the on-screen keyboard commits — Gboard's clipboard chip and its
    sticker/GIF pickers — are attached too. Those never reach the clipboard at
    all, so no paste path could see them. Content the engine could not read (a
    content:// URI with no bytes, which nothing in Dart can open) is logged and
    dropped rather than attached empty.
  • No new button anywhere: the system already offers paste wherever a paste is
    possible.
  • encodeImageDataUrl sniffs the MIME type from the magic bytes rather than
    trusting the source. A clipboard bitmap has no filename and a keyboard names
    whichever of the requested types it likes; a mislabelled type is what makes a
    provider reject an otherwise valid multimodal request. A declared type is
    consulted only when the header says nothing, and Android's unregistered
    image/jpg is normalised to image/jpeg.

Carrying several images

  • maxMessageAttachments (4, in core/models/chat_message.dart) caps the
    composer; the file picker takes several files at once and the preview strip
    becomes a row of square thumbnails past the first image.
  • Storage keeps the attachments split — ChatMessage.imagePath is the first
    and the new extraImagePaths the rest — so sessions written before this
    still render. ChatMessage.attachments is the one list every reader uses and
    splitAttachments the one writer of the pair.
  • PromptMessage now carries imagePaths instead of a single imagePath, and
    toApiMap() emits one image_url content part per image, in order. Every
    rebuild site that used to copy imagePath (HistoryAssembler.assemble,
    buildFallbackPrompt, applyPromptRegexes, StudioHistoryLimiter,
    StudioStreamInterceptor, the memory injector, both append-to-last builders)
    copies the list. The eye toggle still covers the whole message (INV-PS1b).
  • ChatMessageMapper sends imagePaths alongside imagePath (still the
    first, for anything that only knows the single-image shape), and
    _resolveMappedFileUrls resolves each path, dropping any the WebView cannot
    reach instead of rendering a broken tile.
  • The WebView renders one .msg-image-attachment block holding every picture:
    free-size for one, a tile grid for two to four (2-up, one tall plus two
    stacked, 2x2). Keeping it a single element is what lets updateMessage lift
    it out of the body and put it back, and what keeps one eye toggle per
    message.
  • Recorded as INV-PS1c in docs/INVARIANTS.md.

Verified

Run against a local Flutter 3.44.9 (the same version CI pins):

  • flutter analyze --no-fatal-infos --no-fatal-warnings — 9 issues, all of
    them pre-existing on nightly and none in the files this PR touches.
  • flutter test — 3814 passed. New coverage: the paste shortcut end to end
    (clipboard image → thumbnail → data URL in the send callback), the fall
    through to a text paste when the clipboard holds no image, the fifth
    attachment being refused, the toolbar's Paste item, the content-insertion
    callback with and without bytes, the multi-part request, the pre-multi-attach
    JSON payload, and the MIME sniffing.
  • cd test/webview_js && npm ci && npx playwright test — 81 passed, including
    a new attachments.spec.js that measures the grid a headless Chromium
    actually lays out for one to four images.

Not run: a real device or desktop build. The clipboard and keyboard paths cross
a platform channel, so they are covered here by mocked channels rather than by
a real Gboard.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MJ2H48SAD3gVT4wEeMRG4E


Generated by Claude Code

Claude and others added 3 commits September 5, 2026 10:52
Ctrl/Cmd+V in the composer now attaches whatever image is on the
clipboard, and a message can carry up to four of them — laid out as a
grid in the bubble rather than one picture under another.

## Clipboard paste

- `ClipboardImages` (`core/platform/clipboard_images.dart`) reads the
  system clipboard through `pasteboard`: a bitmap (browser copy,
  screenshot tool) or, on desktop, the image files copied in the file
  manager. `Clipboard` from `flutter/services.dart` only ever sees
  `text/plain`, so neither was reachable before.
- The MIME type is sniffed from the magic bytes, not the extension — a
  clipboard bitmap has no filename, and a mislabelled type is what makes
  a provider reject an otherwise valid multimodal request.
- The composer takes the paste shortcut over wholesale: an image is
  attached, anything else is inserted as text at the caret. Letting the
  field also paste natively put the source URL of a copied picture into
  the box next to it.
- A new `paste` composer action covers touch, where there is no shortcut
  to press. It ships as a card in the drawer's Actions tab rather than a
  fifth button in the row.

## Up to four attachments

- `maxMessageAttachments` (4) caps the composer; the file picker now
  takes several files at once, and the preview strip becomes a row of
  square thumbnails past the first image.
- Storage keeps the pair split: `ChatMessage.imagePath` is the first
  attachment and `extraImagePaths` the rest, so sessions written before
  this still render. `ChatMessage.attachments` is the one list every
  reader uses and `splitAttachments` the one writer.
- `PromptMessage` carries `imagePaths`, and `toApiMap()` emits one
  `image_url` content part per image, in order. The eye toggle still
  covers the whole message (INV-PS1b).
- The WebView renders one `.msg-image-attachment` block holding every
  picture: free-size for one, a tile grid for two to four (2-up, one tall
  plus two stacked, 2x2). Keeping it a single element is what lets
  `updateMessage` lift it out of the body and put it back.

## Verified

- `flutter analyze` — 9 issues, all pre-existing on `nightly`.
- `flutter test` — 3811 passed, including new coverage for the paste
  shortcut, the four-image cap, the multi-part request and the MIME
  sniffing.
- `test/webview_js` — 81 passed, including a new `attachments.spec.js`
  that measures the grid the browser actually lays out for one to four
  images.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ2H48SAD3gVT4wEeMRG4E
Follow-up to the multi-attach change.

## No paste button of our own

- Removed the `paste` composer action, its drawer card and its label. The
  system already offers paste wherever a paste is possible.
- The composer's `TextField` now builds its own selection toolbar with
  **Paste** rewired to the image-aware handler, and adds the item when
  Flutter left it out — which it does whenever the clipboard holds no
  *text*, exactly the case an image paste has to survive. That is the way
  in on touch; Ctrl/Cmd+V is unchanged.

## No limits

- Dropped `maxMessageAttachments`: the composer, the send path and the
  renderer take as many images as they are handed.
- Dropped the 12 MB cap in `ClipboardImages` and the `limit` on its read.
- The attachment grid gains `.count-many` — a three-up layout of square
  tiles that grows downwards, so any count past the named 1..4 layouts
  still renders as a grid.
- The composer's preview strip became one horizontally scrolling row of
  thumbnails; a wrapping grid of them would have pushed the composer off
  the screen once the count stopped being bounded.

## Verified

- `flutter analyze` — 9 issues, all pre-existing on `nightly`.
- `flutter test` — 3811 passed, including a rewritten composer test that
  pastes six images and sends all six, and one that drives the toolbar's
  Paste item and asserts it attaches rather than inserts text.
- `test/webview_js` — 81 passed, including a seven-image case measuring
  the three-up grid the browser lays out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ2H48SAD3gVT4wEeMRG4E
… size cap

Corrects the previous commit: the four-image cap is back, and it is the
size limit that is gone.

## Keyboard-committed images

Gboard's clipboard chip and its sticker/GIF pickers never put anything on
the clipboard — the keyboard commits the content straight into the field
over the input connection, so neither the paste shortcut nor the selection
toolbar could see it. The composer's `TextField` now declares
`contentInsertionConfiguration`, and the bytes the engine hands over are
attached like any other picture. Content the engine could not read (no
bytes, only a `content://` URI, which nothing in Dart can open) is logged
and dropped rather than attached empty.

The MIME type a keyboard declares is treated as a fallback, not as truth:
`encodeImageDataUrl` still sniffs the magic bytes first, and only falls
back to the declared type — normalising Android's unregistered
`image/jpg` to `image/jpeg` — when the header says nothing.

## Limits

- `maxMessageAttachments` (4) is back: the composer, the send path and the
  attachment grid all cap at four again, with the toast on the fifth.
- The 12 MB cap on a clipboard image is gone for good. Size is not
  something to refuse over.

## Verified

- `flutter analyze` — 9 issues, all pre-existing on `nightly`.
- `flutter test` — 3814 passed, including new tests that drive the
  content-insertion callback with and without bytes, and the restored
  fifth-paste-refused case.
- `test/webview_js` — 81 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ2H48SAD3gVT4wEeMRG4E
@hydall
hydall merged commit 7e998d1 into nightly Sep 5, 2026
3 checks passed
@hydall
hydall deleted the claude/paste-images-clipboard-2dsdgf branch September 6, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant