An AI documentation copilot for customs clearing agents and freight forwarders on the India–UAE trade lane.
A clearing agent forwards a shipment's documents — commercial invoice, packing list, bill of lading / air waybill, certificate of origin — to a WhatsApp or Telegram number, as PDFs, scans, or phone photos. Within a few minutes LaneCheck replies, in the same chat, with:
- a discrepancy report that cross-checks the documents against each other, citing exact fields, values, and the source document for every flag; and
- a Mirsal review sheet — a PDF that lays the extracted values out in the order of the Dubai Customs declaration form, so a clerk can key the declaration in ~2 minutes.
That is V1. The full product spec lives in CLAUDE.md; the domain research behind it in docs/CONTEXT.md.
The model's only job is converting each document into structured fields, each with a per-field confidence. Every cross-document comparison is deterministic Python — the model is never asked "do these documents match?". This keeps the product trustworthy:
- Every flag cites its evidence — exact field names, both values, both documents.
- Every field carries confidence. A field below the threshold routes to a human review queue instead of firing its rule — a wrong flag costs more than a miss.
- Precision over recall. Thresholds are tuned so a false alarm is rare.
- Every correction is captured as labeled training data (the
correctionstable — the data flywheel).
inbound (WhatsApp / Telegram)
│ one bundle per shipment (debounced)
▼
preprocess ─→ classify ─→ extract ─→ compare ─→ report
deskew, Haiku per Claude 12 coded discrepancy report
denoise, page vision → rules, + Mirsal sheet,
rasterize Pydantic confidence back in the chat
(per doc) gate
A single multi-document PDF (e.g. invoice + packing list in one file) is split per page into separate documents. Low-confidence fields hold the bundle for human review; resolving them in the internal /review page writes a correction and resumes the pipeline.
12 deterministic checks, each with a stable code, severity, and a plain-language agent-facing explanation — e.g. WT_PL_BOL (packing-list vs B/L gross weight, 2% tolerance), VAL_INV_SUM (invoice line items vs stated total), NAME_CONSIGNEE (consignee across all docs, transliteration-aware, "TO ORDER" bills compared against the notify party), INVREF_COO (CoO invoice reference — a mismatch forfeits the 0% CEPA preferential duty). See app/pipeline/compare.py.
| Layer | Choice |
|---|---|
| Language | Python 3.12, mypy --strict |
| API / worker | FastAPI + arq (Redis queue) |
| DB | PostgreSQL 16, SQLAlchemy 2.0 + Alembic |
| Storage | S3-compatible (MinIO locally) |
| LLM | Anthropic Claude — Haiku (classify) + Sonnet/Opus (extract), structured outputs |
| Channels | Meta WhatsApp Cloud API + Telegram Bot API |
| Docs / images | pypdf, PyMuPDF, OpenCV, Pillow |
Everything runs as one FastAPI service, one Postgres, one worker, and S3 — boring infrastructure, optimized for iteration speed.
Requires Docker, Python 3.12, and uv.
cp .env.example .env # fill in ANTHROPIC_API_KEY (and a channel token, below)
uv sync # install deps
make up # postgres + redis + minio
make migrate # apply schema
make test # 181 testsRun the service and worker:
make run # FastAPI (webhooks + /review) on :8000
make worker # arq worker: process_bundle + SLA watchdog + daily metrics- Create a bot with @BotFather; put the token in
.envasTELEGRAM_BOT_TOKENand pick anyTELEGRAM_SECRET_TOKEN. - Run the local long-poller (no public URL needed) alongside the worker:
make telegram-poll make worker # in another terminal - Message your bot, then forward a shipment's documents. You'll get an acknowledgment, a "checking now" status, and the report with inline buttons (👍 / ✏️ correct a field / 📋 Mirsal sheet). Reply
field = value(e.g.totals.total = 25,000.00) to correct an extracted value.
For production, set the Telegram webhook to POST /webhook/telegram instead of polling. WhatsApp uses POST /webhook/whatsapp (the channel is abstracted, so the pipeline is identical for both).
make eval runs the full pipeline over a golden set of real, anonymized documents and reports field accuracy, classification accuracy, latency, and per-document Claude cost. Source documents are gitignored; only the labels are committed (see evals/README.md).
Current results and the Sonnet-vs-Opus comparison are written up in docs/EVAL_RESULTS.md.
All five build phases (bootstrap → pipeline → comparison/report → loop closure → hardening) are implemented, plus a Telegram channel, interactive buttons, and chat corrections. The bot works end-to-end on Telegram today; WhatsApp is pending Meta business verification. Extraction accuracy is ~0.92–0.96 on the current golden set, with the gap to the 0.97 launch gate concentrated entirely on degraded carbon-copy scans (an image-quality floor, not a model gap).
app/ FastAPI service, channels, pipeline (preprocess/classify/extract/compare/report), models
prompts/ versioned per-doc-type extraction prompts
evals/ golden-set harness + labels
migrations/ Alembic
docs/ CONTEXT.md (domain research), EVAL_RESULTS.md, mirsal_field_order.md
CLAUDE.md the product spec — source of truth