Imagio: Optical Character Recognition to Image Generation

Imagio is a macOS desktop application that transforms text-bearing images into new visual content through an inspectable pipeline:

1. acquire an image from a file, screenshot, or drag-and-drop action;
2. preprocess the image and recognise its text with Tesseract OCR;
3. refine the recognised text or synthesise an image-ready prompt with a configurable large language model (LLM) endpoint; and
4. send the prompt to a selected text-to-image (T2I) backend, then preview, copy, or save the returned image.

This repository contains the application implementation developed for the M.Eng. report Optical Character Recognition to Image Generation. It focuses on integrating OCR, prompt optimisation, and image generation in one interactive desktop workflow while retaining user control at each stage.

System Overview

flowchart LR
    A["Input image<br/>file, screenshot, or drag-and-drop"] --> B["Rust OCR pipeline<br/>preprocess + Tesseract"]
    B --> C["Recognised text"]
    C --> D["LLM prompt optimisation<br/>OpenAI-compatible endpoint"]
    D --> E["Optimised prompt"]
    E --> F["T2I provider client"]
    F --> G["Generated image<br/>preview, copy, save"]
    C -. session data .-> H["Local session persistence"]
    E -. session data .-> H
    G -. local image file .-> H

Loading

The desktop shell is implemented with Tauri 2. The Rust host process performs OCR, image preprocessing, screenshot capture, file operations, and clipboard operations. A React and TypeScript WebView presents the interface, maintains sessions, invokes LLM/T2I requests, and coordinates the full workflow through typed Tauri commands.

The application supports manual inspection of intermediate results as well as an automated sequence in which OCR output is refined, converted into a prompt, sent for image generation, and saved after completion.

Implemented Subsystems

OCR And Adaptive Preprocessing

The OCR subsystem is implemented in Rust under src-tauri/src/. Recognition is performed using Tesseract through the Rust tesseract binding. The language selector supports the traineddata language packs available to the local Tesseract installation, including English, simplified and traditional Chinese, Japanese, Korean, French, German, and Spanish when those packs are installed.

Before recognition, the user may configure an ordered preprocessing pipeline:

1. border removal;
2. skew correction using a Hough-transform or projection-profile approach;
3. Gaussian or bilateral noise reduction;
4. brightness and contrast adjustment;
5. sharpening;
6. contrast-limited adaptive histogram equalisation (CLAHE);
7. morphological refinement; and
8. binarisation using Otsu, adaptive, mean, or Sauvola thresholding.

An adaptive mode calculates image-quality measurements and selects preprocessing parameters automatically. This is intended for photographs, screenshots, and degraded document captures where a fixed configuration is not appropriate for every input.

Prompt Optimisation

The prompt subsystem accepts OCR text and supports two related operations: text refinement for correcting common recognition artefacts, and prompt synthesis for converting the content into a visually descriptive prompt. The generated prompt can be reviewed or edited before image generation.

The LLM client communicates through an OpenAI-compatible Chat Completions interface. A local Ollama endpoint can therefore be used for local text processing, while a remote compatible API can be selected by changing the base URL, model name, and API key in the settings panel or local configuration file.

Image Generation And Persistence

The generation layer provides a common UI contract over multiple generation clients. Nano Banana is the default text-to-image (T2I) model in the application and provides the primary generation path for the workflow. Supported alternative models can still be selected through the settings panel when required. Responses are normalised to a local image preview regardless of whether the provider returns an image directly or requires polling for a completed job.

Generated images are persisted under the application's local data directory so that a session can be restored. The user can also explicitly save a result to a chosen path or copy an image to the clipboard.

Sessions And Workflow Control

Each input image is represented as an independent session containing its image, OCR state, prompt state, selected configuration, and generated output. The application supports multiple sessions, session switching, locally persisted history, and automation controls for OCR refinement, prompt generation, image generation, and image saving.

Repository Structure

Imagio/
|-- public/
|   `-- config.local.json.example   Local configuration template
|-- src/
|   |-- components/                 Shared interface components
|   |-- context/                    Session and automation contexts
|   |-- features/
|   |   |-- ocr/                    OCR-facing UI and state
|   |   |-- promptOptimization/     Prompt synthesis and model selection
|   |   `-- imageGeneration/        Provider clients and image handling
|   |-- hooks/                      Workflow and persistence hooks
|   `-- utils/                      LLM transport and utility functions
|-- src-tauri/
|   |-- src/
|   |   |-- ocr/                    OCR pipeline orchestration
|   |   |-- preprocessing/          Geometric and filtering operations
|   |   |-- binarization/           Thresholding and CLAHE operations
|   |   |-- morphology/             Morphological transformations
|   |   `-- quality/                Adaptive-mode quality metrics
|   |-- icons/                      Bundle icons
|   |-- resources/                  macOS bundle resources
|   |-- Cargo.toml
|   `-- tauri.conf.json
|-- package.json
`-- vite.config.ts

Requirements

Imagio has been developed and verified on macOS. Development requires:

• Node.js 20.19 or later, or Node.js 22.12 or later;
• pnpm;
• Rust 1.77.2 or later;
• Tesseract OCR with the required language packs installed; and
• API credentials only for any remote LLM or image-generation provider used.

On macOS with Homebrew, Tesseract may be installed with:

brew install tesseract

Additional traineddata language packs must be available to Tesseract before selecting the corresponding recognition language in the application.

Running The Application

Download The macOS Application

Prebuilt Apple silicon macOS application bundles are published on the GitHub Releases page. Before launching a downloaded release, install the OCR runtime and language data:

brew install tesseract tesseract-lang

Download the macOS .zip release asset, extract Imagio.app, and move it to the Applications folder. Depending on local macOS security settings, the first launch of a downloaded application may require confirmation through Finder's Open action.

Run From Source

Install JavaScript dependencies:

pnpm install

Start the desktop application in development mode:

pnpm run tauri:dev

Build the macOS application bundle:

pnpm run tauri:build

The release bundle is produced under src-tauri/target/release/bundle/macos/Imagio.app.

Local Development Configuration

The repository includes public/config.local.json.example. To define local service settings while running from source, create public/config.local.json:

{
  "llm": {
    "apiBaseUrl": "https://api.openai.com/v1",
    "apiKey": "your-openai-api-key",
    "modelName": "gpt-5.5",
    "temperature": 1
  },
  "selectedModel": "nano-banana",
  "bltcyApiKey": "your-image-generation-api-key"
}

The recommended configuration above uses gpt-5.5 through an OpenAI-compatible endpoint for text processing. A local compatible model may also be used by replacing the endpoint and model name with a loopback address such as http://127.0.0.1:11434/v1. The image-generation configuration shown above selects Nano Banana as the default model. Image generation requires network access and its corresponding credential; alternative supported models can be selected and configured in the Settings panel. Older local files that still use the imageGenModel field continue to load correctly, and informal aliases such as gemini or flux are normalized to the corresponding Settings entries.

The same configuration values may be changed in the application's Settings panel. public/config.local.json is excluded from version control and is stripped from production builds (the build fails if it remains in dist/) so that local credentials are not bundled into a published application. Users of a downloaded release should enter private credentials through the Settings panel instead of placing them in the public repository.

Development Verification

The current source tree can be checked with:

pnpm run build
pnpm test -- --run
cargo check --manifest-path src-tauri/Cargo.toml

Scope And Limitations

The repository provides the integrated desktop application and its preprocessing, workflow, and provider-client implementation. Recognition quality depends on the installed Tesseract language models and the characteristics of the input image. LLM-based refinement may alter text and should be reviewed when textual fidelity matters.

OCR and local LLM processing can be run without sending source text to a remote LLM when a local compatible endpoint is configured. Image generation uses a selected remote provider and therefore requires a network connection and appropriate credentials. The application has been developed and tested on macOS; portability to other platforms is not claimed by this repository.

Acknowledgements

Imagio builds upon Tesseract OCR, Tauri, and React.