docling-json-batch

Parallel, fault-tolerant CLI that converts a folder of PDFs into Docling JSON — the structured DoclingDocument serialization (layout, reading order, tables, bboxes, semantic labels). Round-trippable back into a DoclingDocument.

Why JSON (not markdown)

• Lossless layout. Every element carries prov with page + bbox.
• Tables stay tables. Row/col spans and header flags preserved in a grid.
• Typed labels. Filter by title, section_header, list_item, caption, footnote, picture, etc.
• Round-trips. DoclingDocument.load_from_json(path) reloads the full model; re-export to markdown/HTML/DocTags on demand.

Install

git clone https://github.com/elyas-karbouch/docling-json-batch.git
cd docling-json-batch
pip install -r requirements.txt

Docling downloads ~1 GB of models on first run.

Usage

# All PDFs in ./my_pdfs -> ./out/*.json, pretty-printed, all cores
python convert.py ./my_pdfs

# Custom output, 8 workers, compact JSON
python convert.py ./my_pdfs -o ./json -w 8 --indent 0

# Walk subfolders, redo everything
python convert.py ./my_pdfs --recursive --overwrite

Options

Flag	Default	Purpose
input_dir	(required)	Folder containing PDFs
-o, --output	./out	Where .json files are written
-w, --workers	CPU count	Parallel worker processes
--overwrite	off	Reprocess PDFs whose .json already exists
--recursive	off	Recurse into subfolders
--indent	2	JSON indent (use 0 for compact)

JSON shape (top level)

{
  "schema_name": "DoclingDocument",
  "version": "1.x.x",
  "name": "paper",
  "origin": { "mimetype": "application/pdf", "filename": "paper.pdf", ... },
  "furniture": { ... },   // headers/footers excluded from body
  "body":      { ... },   // reading-order tree root
  "groups":    [ ... ],
  "texts":     [ ... ],   // every text item with label + prov
  "pictures":  [ ... ],
  "tables":    [ ... ],   // grid cells with row/col spans
  "pages":     { "1": { "size": {...}, "page_no": 1 } }
}

Items live in flat arrays; the tree is built via $ref pointers (self_ref, parent, children). Walk from body.children.

Reloading

from pathlib import Path
from docling_core.types.doc import DoclingDocument

doc = DoclingDocument.load_from_json(Path("out/paper.json"))
print(doc.export_to_markdown())   # or export_to_html(), iterate_items(), etc.

Failure handling

• Per-file isolation. Each PDF runs in its own worker process — a crash or OOM in one file can't kill the batch.
• Never raises. Workers return ("fail", error) instead of propagating.
• Resume. Reruns skip any PDF whose <stem>.json already exists. Use --overwrite to redo.
• Failure log. <output>/failures.log — one line per failed PDF with error + short traceback.
• Exit code. 0 if all ok, 1 if any failed, 130 on Ctrl+C.

Run tests

python -m unittest test_convert.py

End-to-end test auto-skips if reportlab isn't installed.

License

MIT — see LICENSE.