ocr-invoice

Lightweight tools for extracting invoice data via OCR + LLM or VLLM and generating GiroCode QR codes to make paying a breeze. Works on Dropbox or locally.

Paying medical bills and submitting them to health insurance is time-consuming, so I built this tool to remove the busywork. I scan invoices with the Dropbox mobile app, and a Raspberry Pi job runs ocr-invoice every few minutes. Whenever ocr-invoice finds a new bill, it renames the file and generates a PDF with a GiroCode QR code that most banking apps can scan. In my tests, the OCR + LLM pipeline succeeds about 8/10 (it struggles with rough scans), while the VLLM vision backend using qwen2.5-vl-72b-instruct scored 10/10.

Installation

pip install -e .              # core CLI
pip install -e .[dropbox]     # add Dropbox SDK
pip install -e .[vision]      # add vision + evaluation deps

Quickstart

ocr-invoice ./invoices/sample.pdf              # local file
ocr-invoice /Remote/Folder --storage dropbox   # Dropbox path

Use --output-dir to redirect artifacts and --debug to keep originals in place. Source PDFs are auto-recompressed before processing; override the 20 MiB limit via MAX_PDF_SIZE (bytes) in your shell or .env file.

Dropbox Setup

• Install the Dropbox extras: pip install -e .[dropbox].
• Create a Dropbox app in the developer console, then copy its App key and App secret into your .env as DROPBOX_KEY and DROPBOX_SECRET.
• Run the login helper to exchange an authorization code for long-lived tokens: ocr-invoice-dropboxlogin. It opens a browser, saves DROPBOX_REFRESH_TOKEN / DROPBOX_ACCESS_TOKEN into .env, and reuses them on subsequent runs.
• Point the CLI at a Dropbox folder (leading slash required) to verify access: ocr-invoice /Remote/Invoices --storage dropbox --recursive.

API Keys

• .env is loaded automatically by the CLI (via python-dotenv).
• MISTRAL_API_KEY powers the OCR step.
• The default text LLM is groq/llama-3.3-70b-versatile; set GROQ_API_KEY for it to work.
• If you swap in another LiteLLM model, export the corresponding provider key (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, MISTRAL_API_KEY) following LiteLLM's <PROVIDER>_API_KEY naming convention.

Batch Processing

Point the CLI at a directory (local or Dropbox) and it skips PDFs that already have _qr, _bill, _prescription, or _unknown siblings. Use --recursive for nested folders.

Vision Extraction

Switch to multimodal models with --extraction-backend vllm plus the --vision-* flags (model, DPI, page limit, detail). Requires the vision extras.

Form Submit Helper

Define PDF_FORM_SUBMIT_URL (and optional redirect/base vars) to embed a "Regenerate QR" button inside generated PDFs. Run the helper server locally:

python -m form_submit_server --host 0.0.0.0 --port 8745
# or
FORM_SUBMIT_SERVER_PORT=9000 docker compose -f form_submit_server/docker-compose.yaml up --build

The Flask app prints incoming payloads, exposes /tokens, and can regenerate QR PNGs via /redirect.

Configuration

Field mappings + prompts live in ocr_invoice/default_config.yaml. Pass --config custom.yaml to override without changing code.

Evaluation

ocr-invoice-eval replays the extraction flow against reviewed _qr.pdf files and writes a JSONL report:

ocr-invoice-eval --input-dir ./invoices --report evaluation/report.jsonl

Use the same flags as the main CLI (--model, --ocr, --extraction-backend, --vision-*) to benchmark alternative setups.

Disclaimer

Much of the code for this application was written with the help of Codex. This is the first project in which I have used AI to such an extent. However, I reviewed all the code and often had to change the structure, since Codex tended to produce spaghetti code. Please note that the automatically generated tests were not audited.