Marquee

Snip any region of your screen — prose, tables, or math — straight to Markdown on your clipboard.

Marquee is a lightweight Windows tray app, in the spirit of PowerToys Text Extractor, but backed by Pix2Text so a single capture handles paragraphs, tables, and equations (as LaTeX) at once. Press a hotkey, drag a box, and the recognized Markdown (with inline/block LaTeX) is on your clipboard a couple of seconds later.

Platform: Windows 10/11, 64-bit. DPI / multi-monitor handling is Windows-specific.

---

Features

• One hotkey, anywhere — Win+Shift+T (configurable) drags a selection over any monitor.
• Prose + tables + math in a single pass, output as clean Markdown with LaTeX. A hybrid recogniser keeps prose readable even on wide captures that mix text and formulas (where the bare engine would garble the prose).
• Pixel-accurate capture across multi-monitor and mixed-DPI setups.
• Warm model loaded once at startup in the background — the tray stays responsive and inference never blocks the UI.
• Many input sources — screen capture, an image file, the clipboard image, drag-and-drop onto the result window, batch a whole folder, OCR a PDF (each to one Markdown file), recapture the last region without re-dragging, or auto-OCR images copied to the clipboard (e.g. from Snipping Tool).
• Editable result window with output quick-wins: Re-OCR the same snip in another format/accuracy, Reflow wrapped prose, Copy table as TSV (pastes as real cells in Excel/Sheets), Save to .md/.txt/.tex, Append to notes, Save/Copy image, plus an optional KaTeX live preview and keyboard shortcuts.
• Local searchable history — every capture is saved (opt-out) with a thumbnail; search it, re-copy, re-open, pin favourites (exempt from the retention cap), export to Markdown, or clear it. Fully local SQLite.
• Configurable output: Markdown, plain text, or LaTeX-only.
• Pause the global hotkey, capture a region straight to a PNG (no OCR), toggle tray notifications, Start with Windows, and a tray icon that shows model state (loading / ready / working).

Install

1. Download Marquee-<version>-setup.exe from the latest release.
2. Run it. It installs per-user (no admin prompt) and adds Start Menu / Desktop shortcuts. (Windows SmartScreen may warn about the unsigned installer → More info → Run anyway.)
3. Launch Marquee from the Start Menu. The tray icon is amber while it downloads the OCR model on first run (a few hundred MB to %APPDATA%, needs internet once), then turns green when ready.
4. Press Win+Shift+T, drag a box over text/math — the Markdown is on your clipboard. A toast confirms "copied".

Prefer no installer? The release also ships Marquee-<version>-win64.zip — unzip and run Marquee.exe (keep the whole folder together).

Updating

Marquee checks for a newer release on startup (and on demand via tray → Check for updates…). When one is found, tray → Install update vX.Y.Z downloads the installer and applies it in place — it keeps your settings and relaunches the app, no manual uninstall/reinstall. The zip build and "from source" runs just open the Releases page instead.

Usage

Action	How
Capture	Win+Shift+T, or double-click the tray icon
Cancel a capture	Esc or right-click during the overlay
Other inputs	Tray → OCR image file… / OCR image on clipboard / OCR a folder… / OCR a PDF… / Recapture last region, or drag an image onto the result window
History	Tray → History… (search, copy, open, pin, export, clear); Copy recent submenu for the last 10
Settings / Quit	Right-click the tray icon

Result window shortcuts: Ctrl+Enter copy & close, Ctrl+S save, Ctrl+R reflow, Ctrl+Shift+C copy table (TSV), Ctrl+W / Esc close.

Settings (right-click tray → Settings…):

• Hotkey — any combo in pynput format, e.g. <ctrl>+<shift>+t, <alt>+s. Re-binds live. An optional Recapture hotkey re-OCRs the last region.
• Auto-OCR clipboard images — OCR a picture the moment it's copied (off by default).
• Output format — Markdown (prose + LaTeX), plain text, or LaTeX-only.
• Accurate layout — recognize in reading order, keep matrices/tables structured, and preserve sub/superscripts in formulas the layout step would otherwise treat as headings (uses pix2text's layout analysis). More correct for mixed text+math and matrices, but slower on CPU (~3–4s vs ~2s; ~1s on a GPU). Off by default; only affects Markdown output.
• Show result window — toggle the editable popup.
• LaTeX preview — render a KaTeX preview in the popup (fully offline; assets bundled). Off by default.
• Reflow prose — join hard-wrapped OCR lines back into paragraphs (code, math, tables and lists preserved). Off by default.
• Notes file — target for the popup's Append to notes button.
• Save history / History limit — keep a local searchable history (on by default) capped to N captures; pinned entries are exempt. Clear history now wipes it.
• Show tray notifications — silence routine balloons (errors still show).
• Start with Windows — adds/removes a per-user Run registry entry.
• Restore defaults — reset every setting (applied when you click OK).

Settings persist to %APPDATA%\Marquee\config.json; history lives in %APPDATA%\Marquee\history\ (open it via tray → Open data folder).

Build from source

Requires Python 3.11+ (developed on 3.14) on 64-bit Windows.

# 1. Install dependencies
py -m pip install -r marquee/requirements.txt

# 2. Run from source
py marquee/app.py

# 3. Package a standalone build (PyInstaller one-folder)
py marquee/scripts/build.py            # -> dist/Marquee/Marquee.exe

# 4. Build release artifacts (zip + installer; installer needs Inno Setup 6)
py marquee/scripts/package_release.py         # version from version.py -> marquee/release/

How it works

Win+Shift+T ─► translucent full-screen overlay (all monitors, physical pixels)
            ─► drag a rectangle ─► Pillow crop ─► Pix2Text.recognize(...)
            ─► Markdown + LaTeX ─► clipboard (+ editable popup)

• Capture (capture.py) disables Qt high-DPI scaling so Qt coordinates equal physical pixels, matching mss's virtual-desktop geometry — this sidesteps the whole class of mixed-DPI mapping bugs that plague snip tools. The pure coordinate math lives in geometry.py.
• OCR (ocr.py) loads a model once and serializes inference behind a lock (it runs on a worker thread, never the UI thread). The engine sits behind an OCRBackend interface (Pix2TextBackend by default), so adding another backend — Surya, GOT-OCR, a cloud vision API — is a new subclass.
• App (app.py) is the resident tray controller. The global hotkey fires on pynput's thread and is marshalled to the Qt GUI thread via a signal; capture and clipboard run on the GUI thread, OCR on a worker thread.
• Platform (winplatform.py) holds all Windows-specific bootstrapping (DPI awareness, Qt env, autostart registry) in one place. Logging (logsetup.py) writes a rotating log to %APPDATA%\Marquee\logs\marquee.log for bug reports; major operations (model load, capture, OCR timings) are traced there.

Project layout

marquee/
  app.py            # resident tray app (entry point)
  capture.py        # multi-monitor, DPI-correct capture overlay
  geometry.py       # pure coordinate/box math (no Qt/mss)
  ocr.py            # OCRBackend interface + Pix2TextBackend + warm Recognizer
  config.py         # settings load/save
  history.py        # local SQLite capture history (FTS5 + LIKE fallback)
  batch.py          # OCR a folder of images -> one Markdown doc (pure core)
  reflow.py         # join hard-wrapped prose into paragraphs (pure)
  mdtable.py        # Markdown pipe-table -> TSV/HTML (pure)
  imaging.py        # image input: file / clipboard / QImage -> PIL
  hotkeys.py        # hotkey parse/validate/format
  winplatform.py    # Windows bootstrap: DPI, Qt env, autostart registry
  logsetup.py       # central rotating-file + console logging
  updater.py        # in-app update check + in-place install (GitHub Releases)
  version.py        # single source of truth for the version
  ui/
    result.py       # editable result popup (re-OCR, reflow, TSV, save, notes)
    history.py      # searchable history browser (pin / export / clear)
    settings.py     # settings dialog
  assets/
    marquee.ico
    vendor/         # bundled KaTeX + marked (offline preview; see fetch_vendor)
  scripts/
    build.py            # PyInstaller build
    package_release.py  # zip + Inno Setup installer
    make_icon.py        # regenerate the app icon
    fetch_vendor.py     # download KaTeX/marked into assets/vendor
    probe_api.py        # Pix2Text API sanity check
    snip_to_console.py  # capture -> recognize -> stdout
    diagnose_hotkey.py  # hotkey troubleshooting
    mqtool.py           # headless test harness (see below)
  Marquee.spec      # PyInstaller spec (size-optimized)
  installer.iss     # Inno Setup installer script
  requirements.txt
  requirements-dev.txt  # lint/type/test tooling
tests/              # pytest: config, hotkeys, geometry, OCR (+ hybrid merge),
                    #   history, reflow, mdtable, batch, imaging
pyproject.toml      # ruff / mypy / pytest config
.github/workflows/ci.yml  # lint + type-check + tests (Windows)

Testing without the GUI

Everything except the human drag-to-select is scriptable, so scripts/mqtool.py drives the whole pipeline headlessly — handy for regression checks and CI:

py marquee/scripts/mqtool.py check                       # self-test: render known
                                                          #   images, OCR, assert (exit 0/1)
py marquee/scripts/mqtool.py grab --box 100 100 700 300 -o region.png   # overlay-free screenshot
py marquee/scripts/mqtool.py e2e --monitor 1             # grab + recognize a whole monitor
py marquee/scripts/mqtool.py recognize image.png --format latex
py marquee/scripts/mqtool.py batch ./shots -o out.md         # OCR a folder of images
py marquee/scripts/mqtool.py pdf paper.pdf -o paper.md       # OCR a PDF
py marquee/scripts/mqtool.py app-e2e --box 0 0 1200 400  # drive the real tray controller
                                                          #   (faked overlay) + verify clipboard
py marquee/scripts/mqtool.py info                        # config, device, monitors

check is the regression gate: it renders deterministic text/math/mixed images and asserts a majority of expected tokens survive OCR. app-e2e exercises the production wiring (background model load → worker thread → clipboard) end to end.

For the pure logic there's a fast pytest suite (no model, GUI or screen needed) covering config persistence, hotkey parsing, DPI/geometry math, and the OCR recognizer lifecycle (singleton, locking, failure handling via a fake backend):

py -m pip install -r marquee/requirements-dev.txt
py -m ruff check marquee tests     # lint
py -m mypy                         # type-check the pure-logic modules
py -m pytest                       # tests (~2s)

These three run on every push/PR via .github/workflows/ci.yml.

Notes & limitations

• First run needs internet to download model weights (cached afterwards in %APPDATA%\pix2text, %APPDATA%\cnstd, %APPDATA%\cnocr).
• History is local but persistent: captured text + thumbnails are stored in %APPDATA%\Marquee\history\. It's on by default with a retention cap; turn it off, lower the cap, or clear it in Settings. Pinned entries are never pruned.
• Elevated windows: a non-admin app can't receive global hotkeys while an elevated (Run-as-administrator) window is focused — a Windows security boundary. Run Marquee as admin too if you need to snip over admin apps.
• CPU inference of a small region is ≈ 2–3 s; Accurate layout adds the layout model (≈ 3–4 s total on CPU). A CUDA GPU is used automatically if a compatible PyTorch build is installed (~1 s).
• Threading: Marquee sets OMP_WAIT_POLICY=PASSIVE / KMP_BLOCKTIME=0 at startup so the torch (layout) and onnxruntime (formula/text) thread pools don't busy-wait and thrash each other — this keeps Accurate-layout timing steady (~3.5 s) instead of swinging to 15 s+ on CPU.

License

MIT © SuleimanGrape