fix: image generation with reference image support#52
Open
usexless wants to merge 5 commits into
Open
Conversation
- main-v2.cjs: fix image gen DOM extraction for ChatGPT reasoning UI - Added dalleImgs + bigImgs fallback for images outside assistant containers - Stop-button guard: only return image when ChatGPT fully done generating - Reference image false-positive fix: exclude BUTTON[aria-haspopup=dialog] wrappers - Added "Thought for X s" detection in diagnostic logging - postProcessImages(): download signed ChatGPT URLs via session cookies → local file - downloadImageWithSession() via electronNet with useSessionCookies - chatgptOldImageUrls pre-send snapshot to filter stale sidebar thumbnails - rest-api.cjs: add Perplexity model aliases (sonar-pro, sonar-reasoning, r1-1776, etc.) - mcp-server-v3.js: generate_image tool — remove DALL-E mention, add patience/flow instructions, prompt structure guide, reference_image param for style consistency - NOTES.md: architecture docs, fix log, critical workflow notes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Code Review by QodoSorry, something went wrongWe weren't able to complete the code review on our side. Please try again |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Image generation returning "No response captured"
When ChatGPT generates an image, it navigates to a new conversation URL and renders the result in a layout that has no
[data-message-author-role="assistant"]containers. The existing DOM extraction relied on these containers and returned nothing.Fix: When no assistant containers are found, fall back to a full-page image search — but exclude uploaded file thumbnails (reference images appear inside
BUTTON[aria-haspopup="dialog"]wrappers in the user message, so they are filtered out by walking up the parent chain).Premature image capture (stop-button guard)
The image appeared in the DOM before ChatGPT finished generating (streaming still in progress). The tool returned an intermediate or wrong result.
Fix: Check for the stop button (
button[aria-label*="Stop"]) before returning any image. If visible, return empty and keep polling. Image is only returned once the stop button disappears — i.e. generation is truly complete.Reference image returned as the generated image
When a local file was uploaded as a style reference, the uploader's thumbnail appeared in the DOM immediately. The old
bigImgsfallback picked it up before ChatGPT had generated anything.Fix:
isInsideDialogButton()traversal — any image found inside a dialog-trigger button is skipped as a reference thumbnail, not a generated result.New feature:
reference_imageparametergenerate_imagenow accepts an optionalreference_image(absolute local file path). The engine uploads it to ChatGPT before sending the prompt, allowing style-consistent multi-image generation.generate_imagetool descriptionRewrote the MCP tool description so AI clients know:
new_conversationbefore each image (prevents context bleed / stale returns)[subject], [action], [setting], [lighting], [style], [quality]Files changed
electron/main-v2.cjs— DOM image extraction, stop-button guard,isInsideDialogButton(),postProcessImages(),downloadImageWithSession()src/mcp-server-v3.js—generate_imagetool description +reference_imageparamNOTES.md— architecture docs and fix log (new file)Tested
reference_image→ newly generated image returned, not the uploaded file