fix(ai): normalize tool-result image data URLs - #1421
Open
Altairpaca wants to merge 2 commits into
Open
Conversation
Author
|
Implementation is ready for review. The branch is current-main based and the diff is limited to the shared tool-result normalization seam plus the focused #1260 regression. All three fork workflows are currently |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1260 by restoring the
ImageContent.datacontract at the shared message-normalization boundary before provider serialization.Tool results occasionally contain an already-prefixed
data:<mime>;base64,...string even thoughImageContent.datais defined as raw base64. Responses adapters then add their own prefix and produce an invalid double-prefixedimage_url.Root cause
transformMessages()normalized unsupported media and tool-call IDs but passed supported tool-result image payloads through unchanged. Provider adapters therefore had to trust thatImageContent.datawas raw base64, and an already-prefixed value leaked into the Responses serializer.Changes
transform-messages.ts.datastarts with the exact canonical prefix for its declared MIME type (data:${mimeType};base64,), strip that prefix and keep raw base64 internally.data:payloads.convertResponsesMessages().Scope / risk
The change is limited to tool-result media normalization. User attachments are unchanged. Provider serializers retain their existing raw-base64 contract and continue adding the wire prefix themselves.
Validation
A focused converter regression is included and the branch is based directly on current upstream
main(5a23f6edfa628f5d1a86422f683742e092feebc2). Repository CI is expected to exercise the package test/typecheck and changelog gates.Closes #1260
Summary by cubic
Fixes tool-result images with already-prefixed data URLs getting double-prefixed in Responses serializers, producing invalid
image_urlvalues. Now strips the canonicaldata:<mime>;base64,prefix from tool-result image data before provider serialization so only one prefix gets added.data:payloads pass through unchanged.convertResponsesMessages().Written for commit 9b082b4. Summary will update on new commits.