diff --git a/.github/codeql/codeql-config.yml b/.github/codeql/codeql-config.yml new file mode 100644 index 0000000..f3a6144 --- /dev/null +++ b/.github/codeql/codeql-config.yml @@ -0,0 +1,18 @@ +name: ffpdf CodeQL config + +# security-extended: broader security queries than the default suite; the right +# bar for a parser that eats untrusted input. +queries: + - uses: security-extended + +query-filters: + # cpp/path-injection flags every fopen() of a path the user passed on the + # command line. For a CLI whose entire job is reading the files named in + # argv, that is the interface, not a vulnerability -- there is no trust + # boundary between "user input" and "the file the user asked to open". + # This rule fired on ~every PR that adds a file-opening path (dismissed on + # alerts 4-8, 10-15), so it is excluded here rather than dismissed each time. + # If ffpdf ever accepts a path from genuinely untrusted input (e.g. a field + # value that names an external file), revisit this exclusion. + - exclude: + id: cpp/path-injection diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 989f231..31fa4e0 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -27,9 +27,9 @@ jobs: with: languages: c-cpp build-mode: manual - # security-extended: broader security queries than the default suite; - # right bar for a parser that eats untrusted input. - queries: security-extended + # Query suite (security-extended) and the cpp/path-injection exclusion + # live in the config file. + config-file: ./.github/codeql/codeql-config.yml - name: Build run: | diff --git a/.gitignore b/.gitignore index e6ee4db..af1bd1f 100644 --- a/.gitignore +++ b/.gitignore @@ -6,5 +6,7 @@ fuzz-findings/ FUTURE_WORK.md filled.pdf answers.fdf +values.json out/ row.fdf +sbom.gen.json diff --git a/CHANGELOG.md b/CHANGELOG.md index 3b0af5f..944f6be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,13 +7,46 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.1.1] - 2026-07-10 + ### Added -- `fields` command: list every fillable field as JSON, with names, types, - current values (UTF-16 values decoded to UTF-8), choice options and +- `fields` command: list every fillable field as JSON, with names, + human-readable labels (`/TU`), types, current values (UTF-16 values decoded + to UTF-8), `required`/`readonly`/`maxlen` constraints, choice options and combo/multi-select flags, and checkbox on-state names. A machine-readable companion to `fdf-extract` for scripts and AI agents. +- `fill --json` writes a machine-readable result to stdout (the `updated` + and `not_found` field names and their counts; requires `-o` for the PDF), + and `fill --strict` exits 3 if any FDF field did not match a form field. + For scripts and AI agents that need to verify a fill by exit code or + parsed output rather than scraping stderr. +- Documented that multi-select choice fields are filled with an array, + `/V [(opt1) (opt2)]`. +- `fill` now accepts a **JSON** values object as well as an FDF, auto-detected + from the file's content, and takes the PDF and values file in **either + order** (`-` reads the values from stdin). JSON values may be a string, a + multi-select array, a number, or a boolean (to check/clear a checkbox). This + closes the discover-then-fill loop in one format for agents. The documented + argument order is now `fill `. +- Radio-button groups: `fields` now reports the group's valid option names, + and `fill` sets the selected button's appearance state (`/AS`) so the choice + renders, not just the group value. + +### Changed + +- `fill` now exits **2** (and writes no output) when no field in the input + matched the form. A no-op fill — e.g. every field name misspelled — is + treated as a failure by default rather than a silent success, so callers + need no flag to detect it. `--flatten` is exempt (it removes the form even + with no new values), and a partial match (some fields land) still exits 0 + unless `--strict` is given. + +- A value longer than a text field's `/MaxLen` is now truncated to fit (on a + UTF-8 character boundary) with a warning, instead of writing an over-length, + non-conformant value; `fill --json` lists such fields under `truncated`. + ### Fixed - ffpdf can now re-read PDFs it filled. Streams without a `/Filter` (legal diff --git a/Makefile b/Makefile index f432082..6b35848 100644 --- a/Makefile +++ b/Makefile @@ -157,7 +157,7 @@ docs/demo.gif: docs/demo.tape docs/demo-form.pdf $(TARGET) { echo "vhs not found -- install it (and ttyd + ffmpeg): https://github.com/charmbracelet/vhs"; exit 1; } vhs docs/demo.tape -docs/demo-batch.gif: docs/demo-batch.tape docs/demo-template.fdf docs/demo-customers.csv docs/demo-form.pdf $(TARGET) +docs/demo-batch.gif: docs/demo-batch.tape docs/demo-template.json docs/demo-customers.csv docs/demo-form.pdf $(TARGET) @command -v vhs >/dev/null || \ { echo "vhs not found -- install it (and ttyd + ffmpeg): https://github.com/charmbracelet/vhs"; exit 1; } vhs docs/demo-batch.tape diff --git a/README.md b/README.md index 055c02b..c69bc9a 100644 --- a/README.md +++ b/README.md @@ -4,14 +4,14 @@ [![License](https://img.shields.io/github/license/pagebrooks/ffpdf)](LICENSE) [![Latest release](https://img.shields.io/github/v/release/pagebrooks/ffpdf)](https://github.com/pagebrooks/ffpdf/releases/latest) -**ffpdf** is a small, fast command-line tool for **reading and filling PDF form fields**. It is written in plain C with no dependency beyond zlib. No Java, no pdftk, no headless browser. Point it at a PDF, hand it a list of values, and it writes a filled PDF to stdout. +**ffpdf** is a small, fast command-line tool for **reading and filling PDF form fields**, with a **JSON in / JSON out** contract that scripts and AI agents can drive. It is written in plain C with no dependency beyond zlib: no Java, no pdftk, no headless browser. Ask it what fields a form has, hand back the values as JSON, and it writes a filled PDF. ![ffpdf demo: discover fields, fill the form, get a valid PDF](docs/demo.gif) It’s built to handle real-world PDFs: government forms, bank documents, LiveCycle/XFA forms, and encrypted “secured” PDFs, not just clean textbook files. ```console -$ ./ffpdf fill answers.fdf form.pdf > filled.pdf +$ ./ffpdf fill form.pdf values.json > filled.pdf ``` --- @@ -22,6 +22,7 @@ Most "fill a PDF form" tools drag in a huge runtime or require commercial licens - **Starts instantly and stays small.** Filling streams the original file straight through and only buffers the bytes it appends, so memory stays proportional to *your changes*, not the file size. A 22 MB PDF parses in a fraction of a second. - **Preserves the original.** Fills are written as a PDF *incremental update*: the original bytes are copied verbatim and the changes are appended. Nothing is silently rewritten. +- **Scriptable and agent-friendly.** `fields` lists a form's fields as JSON (names, types, options, constraints); `fill` takes the values back as JSON and reports the outcome as JSON with a meaningful exit code. No FDF knowledge required. (FDF is still supported and auto-detected, handy if you already work with FDF or pdftk.) - **Handles the hard cases** (encryption, XFA, compressed object streams, weird cross-reference layouts). - **Made for the thousand-PDF loop.** With no runtime to start, a fill is just a process exec: ~600 real forms per second on one core. See [Filling PDFs by the thousands](#filling-pdfs-by-the-thousands). @@ -33,67 +34,58 @@ Most "fill a PDF form" tools drag in a huge runtime or require commercial licens # 1. Build it (needs gcc and zlib) make -# 2. See what fields a form has -./ffpdf fdf-extract form.pdf +# 2. See what fields a form has (as JSON) +./ffpdf fields form.pdf -# 3. Fill it -./ffpdf fill answers.fdf form.pdf > filled.pdf +# 3. Fill it from a JSON object of values +./ffpdf fill form.pdf values.json > filled.pdf ``` -That’s the whole loop: **discover → edit → fill.** +That’s the whole loop: **discover → fill → verify**, all in JSON. ### A complete example Say `form.pdf` has a text field named `applicant_name` and a checkbox named `agree`. -**1. Discover the field names** with `fdf-extract` (it prints an FDF listing every field): +**1. Discover the fields** with `fields`. The JSON gives each field's name and type, and — for checkboxes and choice fields — the values it accepts: ```console -$ ./ffpdf fdf-extract form.pdf -%FDF-1.2 -1 0 obj -<< -/FDF -<< -/Fields [ -<< -/T (applicant_name) -/V () ->> -<< -/T (agree) -/V () ->> -] -... +$ ./ffpdf fields form.pdf +{ + "fields": [ + { "name": "applicant_name", "type": "text", "value": "", "required": false, "readonly": false }, + { "name": "agree", "type": "button", "value": "Off", "required": false, "readonly": false, "on_state": "Yes" } + ], + "count": 2, + "xfa": false, + "dynamic_xfa": false +} ``` -Each field is a `<< /T (name) /V (current value) >>` block; the `/T` is the name you fill by. +The `on_state` says `agree` is checked by the value `Yes` (or simply `true`). -**2. Write your answers** into a small FDF file, `answers.fdf`: +**2. Write your answers** as a JSON object keyed by field name, `values.json`: -``` -%FDF-1.2 -1 0 obj -<< /FDF << /Fields [ - << /T (applicant_name) /V (Ada Lovelace) >> - << /T (agree) /V (Yes) >> -] >> >> -endobj -trailer -<< /Root 1 0 R >> -%%EOF +```json +{ + "applicant_name": "Ada Lovelace", + "agree": true +} ``` -**3. Fill:** +**3. Fill and verify:** ```console -$ ./ffpdf fill answers.fdf form.pdf > filled.pdf +$ ./ffpdf fill --strict -o filled.pdf form.pdf values.json +$ echo $? +0 ``` -Open `filled.pdf`: the name is typed in and the box is checked. The tool figures out the right on‑state for the checkbox, generates the visual appearance so the values show in *any* viewer, and (for dynamic XFA forms) also updates the XFA data packet so Adobe renders it too. +Open `filled.pdf`: the name is typed in and the box is checked. The tool picks the right on‑state for the checkbox, generates the visual appearance so the values show in *any* viewer, and (for dynamic XFA forms) also updates the XFA data packet so Adobe renders it too. `--strict` makes it exit non‑zero if any value failed to match a field, so a script can trust the outcome without parsing output. -> **Tip:** field names can be long and nested (e.g. `topmostSubform[0].Page1[0].f1_1[0]`). You can use the full name from `fdf-extract`, or just the last `.`-separated piece if it’s unique. +> **Tip:** field names can be long and nested (e.g. `topmostSubform[0].Page1[0].f1_1[0]`). Use the full name from `fields`, or just the last `.`-separated piece if it’s unique. +> +> Prefer PDF‑native **FDF**? `fill` takes that too; it's auto‑detected, and handy if you already work with FDF or pdftk. See [batch fills](#filling-pdfs-by-the-thousands). ### See it on a real form @@ -117,35 +109,29 @@ The `docs/` folder carries the actual files, which you can open right in your br | [`example-flattened.pdf`](docs/example-flattened.pdf) | The `--flatten` result: the same values baked into the page content, with the form removed (nothing left to edit) | ```console -$ ffpdf fill -o example-filled.pdf example-answers.fdf example-form.pdf -$ ffpdf fill --flatten -o example-flattened.pdf example-answers.fdf example-form.pdf +$ ffpdf fill -o example-filled.pdf example-form.pdf example-answers.fdf +$ ffpdf fill --flatten -o example-flattened.pdf example-form.pdf example-answers.fdf ``` -The PDFs in the repo were produced by ffpdf itself (`make examples` regenerates all of them), and the values are visible in any viewer because fill generates the visual appearance streams rather than relying on the viewer to draw them. +The values here are held in an FDF (`example-answers.fdf`); the same fill works from an equivalent JSON object — `fill` auto-detects the format. The PDFs in the repo were produced by ffpdf itself (`make examples` regenerates all of them), and the values are visible in any viewer because fill generates the visual appearance streams rather than relying on the viewer to draw them. --- ## Filling PDFs by the thousands -Where this tool really earns its keep is **batch filling**. An FDF is plain text, so the file `fdf-extract` gave you is one search-and-replace away from being a **template**. And because ffpdf is a ~100 KB binary with no runtime to warm up, invoking it once per record costs almost nothing. +Filling one form is easy; the payoff is filling **thousands**. Because ffpdf is a ~100 KB binary with no runtime to warm up, invoking it once per record costs almost nothing. The pattern is a shell loop over your data, one `fill` per row, each writing its own output file, with the values for each record supplied as JSON. -![ffpdf mail-merge demo: a CSV plus an FDF template renders one filled PDF per record](docs/demo-batch.gif) +![ffpdf bulk-fill (mail-merge) demo: a CSV plus a JSON template renders one filled PDF per record](docs/demo-batch.gif) -Turn the extracted FDF into a template by putting placeholders where the values go: +The quick path turns a JSON object into a **template** with placeholders where the values go: -``` -%FDF-1.2 -1 0 obj -<< /FDF << /Fields [ - << /T (Address) /V (${ADDRESS}) >> - << /T (City) /V (${CITY}) >> - << /T (State) /V (${STATE}) >> - << /T (Zip) /V (${ZIP}) >> -] >> >> -endobj -trailer -<< /Root 1 0 R >> -%%EOF +```json +{ + "Address": "${ADDRESS}", + "City": "${CITY}", + "State": "${STATE}", + "Zip": "${ZIP}" +} ``` then loop over your data; `envsubst` (from gettext) does the substitution: @@ -153,19 +139,22 @@ then loop over your data; `envsubst` (from gettext) does the substitution: ```bash while IFS=, read -r ADDRESS CITY STATE ZIP; do export ADDRESS CITY STATE ZIP - envsubst < template.fdf | ffpdf fill -o "out/$ZIP-$CITY.pdf" - form.pdf + envsubst < template.json | ffpdf fill -o "out/$ZIP-$CITY.pdf" form.pdf - done < customers.csv ``` -(`-` reads the FDF straight from the pipe, with no intermediate file to manage, and `-o` writes each PDF **atomically**, so a killed job never leaves partial files in `out/`.) +(`-` reads the values straight from the pipe, with no intermediate file to manage, and `-o` writes each PDF **atomically**, so a killed job never leaves partial files in `out/`.) Measured on a single ordinary core: **1,000 filled PDFs in about a second** for a small form, and **~600 fills/second on a real 74 KB IRS form**, roughly 36,000 a minute before you even reach for `xargs -P`. The per-invocation startup cost that makes JVM-based tools painful in this loop simply isn’t there. -Two practical notes: +A few practical notes: -- **Escape your data.** `(`, `)` and `\` inside a value must be backslash-escaped in FDF strings, e.g. `/V (Smith \(Jr\))`. +- **For messy data, generate the JSON instead of templating it.** `envsubst` is plain text substitution, so a value containing `"` or a backslash will break the JSON. When your data is untrusted or may contain quotes, build each object with a real serializer (`jq -n --arg city "$CITY" '{City: $city}'`, or your language's JSON library), which escapes correctly. The template above is the quick path for clean data. +- **Multi-select fields take an array.** A list box that accepts several values is filled with `["opt1", "opt2"]` (the `fields` output flags these with `"multi_select": true` and lists the valid `options`). - **Parallelize trivially.** Every fill is an independent process with no shared state: split the input and run N loops, or hand the whole thing to `xargs -P`/GNU `parallel`. +**Already have FDF?** `fill` auto-detects it, so the very same loop works with an FDF template (`envsubst < template.fdf | ...`). That's the natural choice when you interoperate with pdftk or a system that already emits FDF; just remember FDF strings need `(`, `)` and `\` backslash-escaped. + --- ## Commands @@ -174,16 +163,40 @@ Two practical notes: |---|---| | `fdf-extract ` | Extract all form fields as an **FDF** (to stdout) | | `xfdf-extract ` | Extract all form fields as an **XFDF** (XML flavour, to stdout) | -| `fields ` | List all form fields as **JSON**: names, types, current values, a choice field's options and flags, a checkbox's on-state. The machine-readable companion to `fdf-extract`, built for scripts and AI agents | -| `fill [-f\|--flatten] [-o FILE] ` | Fill `` with the values in `` (`-` reads the FDF from stdin), write the result to stdout or, with **`-o`**, atomically to `FILE`. With **`--flatten`** (**`-f`**), bake the values into the page and remove the interactive form, producing a non‑editable PDF | +| `fields ` | List all form fields as **JSON**: names, human-readable labels, types, current values, page numbers, `required`/`readonly`/`maxlen` constraints, a choice field's options and flags, a checkbox's on-state, plus document `xfa`/`dynamic_xfa` flags. The machine-readable companion to `fdf-extract`, built for scripts and AI agents | +| `fill [-f\|--flatten] [-o FILE] ` | Fill `` from a **``** file — an **FDF or a JSON object**, auto-detected; the PDF and values may be in either order, and `` may be `-` for stdin. Write the result to stdout or, with **`-o`**, atomically to `FILE`. With **`--flatten`** (**`-f`**), bake the values into the page and remove the interactive form, producing a non‑editable PDF | | `xref ` | Dump the parsed cross-reference table as JSON (a debugging aid) | | `help` (`-h`, `--help`) | Print full help to stdout and exit | | `version` (`-v`, `--version`) | Print version and license info and exit | -All commands are subcommands of `ffpdf` (e.g. `ffpdf fill data.fdf form.pdf`). +All commands are subcommands of `ffpdf` (e.g. `ffpdf fill form.pdf data.fdf`). Everything goes to **stdout**, so redirect it (`... > out.pdf`), or use `-o out.pdf`, which writes **atomically** (the file is assembled as `out.pdf.tmp` and renamed into place only on success, so a failed fill never leaves a partial output). As a safety net, `fill` refuses to write PDF bytes to a terminal. Progress and warnings go to stderr, so they won’t corrupt the output. `--` ends option parsing for file names that begin with `-`. Run `ffpdf --help` for a built-in summary, or see the `ffpdf(1)` man page. +**Checking the result programmatically.** By default `fill` already fails loudly when nothing lands: if **no** field in the input matched the form, it writes no output and exits `2` (a no-op fill is treated as a failure, not a silent success — `--flatten` is exempt). For more detail, `fill --json` writes a JSON summary to stdout (which fields were `updated`, `not_found`, or `truncated` to fit a `/MaxLen`, plus counts; pass `-o` for the PDF since stdout then carries the JSON), and `fill --strict` exits `3` if *any* field didn't match — so a caller can trust the exit code instead of parsing text: + +```console +$ ffpdf fill --json --strict -o out.pdf form.pdf answers.fdf +{ "updated": ["applicant_name"], "not_found": ["agree_typo"], + "updated_count": 1, "not_found_count": 1 } +$ echo $? +3 +``` + +Exit codes: **0** success · **1** bad usage or unreadable PDF · **2** nothing matched (no output) · **3** `--strict` and some field didn't match. + +**JSON values reference.** The `` file is a flat JSON object keyed by field name (the shape `fields` emits — see the [worked example](#a-complete-example)). The value's *type* selects the fill behaviour: + +| JSON value | Fills | +|---|---| +| **string** | a text or choice field (`"State": "NH"`) | +| **array of strings** | a multi-select list box (`["Flood", "Identity theft"]`) | +| **number** | used verbatim as text — handy for ZIPs and IDs (`3801` → `"3801"`) | +| **`true` / `false`** | checks / clears a checkbox (mapped to its real on-state) | +| **`null`** | skipped — the field is left as-is | + +`values.json` may be `-` to read from stdin. String values are UTF-8 (or `\uXXXX` escapes), so `"Zoë 😀"` round-trips correctly. A value longer than a text field's `/MaxLen` is truncated to fit, with a warning (and is listed under `truncated` in the `--json` result). + --- ## What it supports @@ -197,7 +210,7 @@ Everything goes to **stdout**, so redirect it (`... > out.pdf`), or use `-o out. | Type | Filling behaviour | |---|---| | Text | Value written; visual appearance generated (UTF‑16 for non‑ASCII) | -| Checkbox / radio | Maps `Yes`/`On`/`1` (or an explicit state) to the widget’s real on‑state | +| Checkbox / radio | Maps `Yes`/`On`/`1` (or an explicit state) to the widget’s real on‑state; a radio group's valid option names are listed by `fields`, and fill sets both the group value and the selected button | | Choice (combo / list box) | Matches your value against the options, sets the selection index, and draws it, single **or** multi‑select | | Signature | Left untouched (a signature isn’t a fillable value) | diff --git a/docs/demo-batch.gif b/docs/demo-batch.gif index 0dcc8d0..b1a9681 100644 Binary files a/docs/demo-batch.gif and b/docs/demo-batch.gif differ diff --git a/docs/demo-batch.tape b/docs/demo-batch.tape index df92c6b..fbc7005 100644 --- a/docs/demo-batch.tape +++ b/docs/demo-batch.tape @@ -1,9 +1,9 @@ -# VHS tape for the mail-merge demo GIF (https://github.com/charmbracelet/vhs). +# VHS tape for the bulk-fill (mail-merge) demo GIF (https://github.com/charmbracelet/vhs). # Regenerate from the repo root with `make demo` (needs vhs + ttyd + ffmpeg, # pdftotext, envsubst, and a built ./ffpdf). # # Shows the batch use case: one form + a CSV -> one filled PDF per record, -# using docs/demo-template.fdf and docs/demo-customers.csv. +# using docs/demo-template.json and docs/demo-customers.csv. Output docs/demo-batch.gif @@ -15,7 +15,7 @@ Set WindowBar Colorful Set Theme "Catppuccin Mocha" Set TypingSpeed 40ms -Type "# mail merge: fill one form for every row of a CSV" +Type "# bulk fill: one form for every row of a CSV" Enter Enter Sleep 500ms @@ -24,11 +24,11 @@ Enter Sleep 2.5s Enter -Type "# the FDF from fdf-extract, with placeholders where the values go" +Type "# a JSON template with placeholders where the values go" Enter Enter Sleep 500ms -Type "cat docs/demo-template.fdf" +Type "cat docs/demo-template.json" Enter Sleep 3s @@ -45,9 +45,9 @@ Type "while IFS=, read -r ADDRESS CITY STATE ZIP; do" Enter Type " export ADDRESS CITY STATE ZIP" Enter -Type " envsubst < docs/demo-template.fdf |" +Type " envsubst < docs/demo-template.json |" Enter -Type ' ./ffpdf fill --flatten -o "out/$ZIP-$CITY.pdf" - docs/demo-form.pdf 2>/dev/null' +Type ' ./ffpdf fill --flatten -o "out/$ZIP-$CITY.pdf" docs/demo-form.pdf - 2>/dev/null' Enter Type "done < docs/demo-customers.csv" Enter diff --git a/docs/demo-template.fdf b/docs/demo-template.fdf deleted file mode 100644 index 0e47ff0..0000000 --- a/docs/demo-template.fdf +++ /dev/null @@ -1,12 +0,0 @@ -%FDF-1.2 -1 0 obj -<< /FDF << /Fields [ - << /T (Address) /V (${ADDRESS}) >> - << /T (City) /V (${CITY}) >> - << /T (State) /V (${STATE}) >> - << /T (Zip) /V (${ZIP}) >> -] >> >> -endobj -trailer -<< /Root 1 0 R >> -%%EOF diff --git a/docs/demo-template.json b/docs/demo-template.json new file mode 100644 index 0000000..fa01787 --- /dev/null +++ b/docs/demo-template.json @@ -0,0 +1,6 @@ +{ + "Address": "${ADDRESS}", + "City": "${CITY}", + "State": "${STATE}", + "Zip": "${ZIP}" +} diff --git a/docs/demo.gif b/docs/demo.gif index 0a29d51..5d14680 100644 Binary files a/docs/demo.gif and b/docs/demo.gif differ diff --git a/docs/demo.tape b/docs/demo.tape index e9fed10..9c8c6fd 100644 --- a/docs/demo.tape +++ b/docs/demo.tape @@ -1,71 +1,65 @@ # VHS tape for the README demo GIF (https://github.com/charmbracelet/vhs). # Regenerate from the repo root with `make demo` (needs vhs + ttyd + ffmpeg, -# pdftotext, and a built ./ffpdf). Deterministic by design: re-run whenever +# jq, pdftotext, and a built ./ffpdf). Deterministic by design: re-run whenever # CLI output changes so the README demo never goes stale. # # Uses docs/demo-form.pdf, a tiny AcroForm with Address/City/State/Zip fields. +# Shows the JSON-first loop: discover fields -> author values as JSON -> fill. Output docs/demo.gif Set FontSize 18 -Set Width 1000 +Set Width 1100 Set Height 740 Set Padding 20 Set WindowBar Colorful Set Theme "Catppuccin Mocha" Set TypingSpeed 40ms -Type "# 1. extract the form's fields to an FDF" +Type "# 1. discover the form's fields as JSON" Enter Enter Sleep 500ms -Type "./ffpdf fdf-extract docs/demo-form.pdf > answers.fdf" +Type "./ffpdf fields docs/demo-form.pdf | jq -c '.fields[] | {name, type}'" Enter -Sleep 300ms +Sleep 2.5s Enter -Type "# 2. fill in the values with any editor" +Type "# 2. author the answers as a JSON object" Enter Enter Sleep 500ms -Type "vi answers.fdf" +Type "vi values.json" Enter -Sleep 1.5s -Type@25ms "/V ()" -Sleep 300ms +Sleep 1.2s +Type ":set paste" Enter -Sleep 300ms -Type@25ms "3li" -Type@25ms "123 Main St" -Sleep 300ms -Escape Sleep 200ms -Type@25ms "n3li" -Type@25ms "Springfield" -Sleep 300ms -Escape -Sleep 200ms -Type@25ms "n3li" -Type@25ms "IL" -Sleep 300ms +Type "i" +Type@30ms "{" +Enter +Type@30ms ' "Address": "123 Main St",' +Enter +Type@30ms ' "City": "Springfield",' +Enter +Type@30ms ' "State": "IL",' +Enter +Type@30ms ' "Zip": "62704"' +Enter +Type@30ms "}" Escape -Sleep 200ms -Type@25ms "n3li" -Type@25ms "62704" Sleep 300ms -Escape -Sleep 200ms Type ":wq" Enter Sleep 800ms Type "clear" Enter -Type "# 3. fill the form; --flatten bakes the values in and removes the form" +Type "# 3. fill it (JSON in, JSON out); --flatten bakes the values in" Enter Enter Sleep 500ms -Type "./ffpdf fill --flatten answers.fdf docs/demo-form.pdf > filled.pdf" +Type "./ffpdf fill --flatten --json -o filled.pdf docs/demo-form.pdf values.json 2>/dev/null" Enter Sleep 3.5s diff --git a/docs/example-filled.pdf b/docs/example-filled.pdf index cda556a..0199615 100644 Binary files a/docs/example-filled.pdf and b/docs/example-filled.pdf differ diff --git a/docs/example-flattened.pdf b/docs/example-flattened.pdf index dce8dc1..64da068 100644 Binary files a/docs/example-flattened.pdf and b/docs/example-flattened.pdf differ diff --git a/docs/example-form.pdf b/docs/example-form.pdf index 4491340..0bf92be 100644 Binary files a/docs/example-form.pdf and b/docs/example-form.pdf differ diff --git a/docs/gen-example-form.py b/docs/gen-example-form.py index a6d82b2..0b739e4 100644 --- a/docs/gen-example-form.py +++ b/docs/gen-example-form.py @@ -63,11 +63,24 @@ def box(x, y, w, h): c.rect(x, y, w, h, fill="1 1 1", stroke=BOX_BORDER) -def text_field(name, lab, x, y, w, h=22): +def meta(lab, flags=0): + """/TU tooltip (human-readable label) + /Ff Required for '*'-marked fields. + Machine consumers (ffpdf fields, AI agents) read these.""" + required = lab.rstrip().endswith("*") + tu = lab.rstrip().rstrip("*").rstrip() + if required: + flags |= 2 # /Ff bit 2 = Required + ff = f" /Ff {flags}" if flags else "" + return f"/TU ({tu}){ff} " + + +def text_field(name, lab, x, y, w, h=22, maxlen=None): label(x, y + h + 5, lab) box(x, y, w, h) + ml = f" /MaxLen {maxlen}" if maxlen else "" widgets.append( f"<< /Type /Annot /Subtype /Widget /FT /Tx /T ({name}) /V () " + f"{meta(lab)}{ml}" f"/Rect [{x} {y} {x + w} {y + h}] /DA (/Helv 11 Tf 0 g) " f"/F 4 /P 3 0 R >>") @@ -79,7 +92,8 @@ def combo_field(name, lab, x, y, w, opts, h=22): c.raw(f"0.45 g {ax} {y + 13} m {ax + 8} {y + 13} l {ax + 4} {y + 7} l f") opt = "".join(f"({o})" for o in opts) widgets.append( - f"<< /Type /Annot /Subtype /Widget /FT /Ch /Ff 131072 /T ({name}) " + f"<< /Type /Annot /Subtype /Widget /FT /Ch /T ({name}) " + f"{meta(lab, 0x20000)}" f"/V () /Opt [{opt}] /Rect [{x} {y} {x + w} {y + h}] " f"/DA (/Helv 11 Tf 0 g) /F 4 /P 3 0 R >>") @@ -89,7 +103,8 @@ def list_field(name, lab, x, y, w, h, opts): box(x, y, w, h) opt = "".join(f"({o})" for o in opts) widgets.append( - f"<< /Type /Annot /Subtype /Widget /FT /Ch /Ff 2097152 /T ({name}) " + f"<< /Type /Annot /Subtype /Widget /FT /Ch /T ({name}) " + f"{meta(lab, 0x200000)}" f"/V () /Opt [{opt}] /Rect [{x} {y} {x + w} {y + h}] " f"/DA (/Helv 10 Tf 0 g) /F 4 /P 3 0 R >>") @@ -122,8 +137,8 @@ def section(n, title, y): section(2, "MAILING ADDRESS", 536) text_field("StreetAddress", "Street address *", LM, 488, 504) text_field("City", "City *", LM, 440, 246) -text_field("State", "State *", 330, 440, 70) -text_field("Zip", "ZIP *", 430, 440, 128) +text_field("State", "State *", 330, 440, 70, maxlen=2) +text_field("Zip", "ZIP *", 430, 440, 128, maxlen=10) # ---- 3. policy ----------------------------------------------------------- section(3, "POLICY", 404) @@ -150,6 +165,7 @@ def section(n, title, y): color=f"{GRAY_TXT} {GRAY_TXT} {GRAY_TXT}") widgets.append( f"<< /Type /Annot /Subtype /Widget /FT /Sig /T (Signature) " + f"/TU (Signature) " f"/Rect [{LM} 234 354 258] /F 4 /P 3 0 R >>") text_field("SignatureDate", "Date", 430, 232, 128) @@ -181,6 +197,7 @@ def section(n, title, y): widgets[CHECKBOX_DICT_INDEX] = ( f"<< /Type /Annot /Subtype /Widget /FT /Btn /T (PaperlessBilling) " + f"/TU (Enroll in paperless billing) " f"/V /Off /AS /Off /Rect [{CB[0]} {CB[1]} {CB[2]} {CB[3]}] " f"/F 4 /P 3 0 R /MK << /CA (4) >> " f"/AP << /N << /On {ap_on} 0 R /Off {ap_off} 0 R >> >> >>") diff --git a/ffpdf.1 b/ffpdf.1 index 0a41d61..a942b3f 100644 --- a/ffpdf.1 +++ b/ffpdf.1 @@ -1,7 +1,7 @@ .\" Manual page for ffpdf. .\" SPDX-License-Identifier: Apache-2.0 .\" Copyright (C) 2026 Page Brooks -.TH FFPDF 1 "July 2026" "ffpdf 0.1.0" "User Commands" +.TH FFPDF 1 "July 2026" "ffpdf 0.1.1" "User Commands" .SH NAME ffpdf \- extract and fill PDF form fields .SH SYNOPSIS @@ -23,7 +23,7 @@ ffpdf \- extract and fill PDF form fields .RB [ \-o .IR file ] .RB [ \-\- ] -.I fdf pdf +.I pdf values .br .B ffpdf .B xref @@ -84,26 +84,42 @@ on standard output: each field's fully-qualified name, type .BR button , .BR choice , .BR signature ), -and current value, plus a choice field's options and combo/multi-select flags -and a checkbox's on-state name. This is the machine-readable companion to +and current value. Where the PDF provides them, each field also carries a +human-readable +.B label +(the field's tooltip / alternate text), the +.BR required ", " readonly ", and " maxlen +constraints, its 1-based +.BR page , +and, per type, a choice field's options and combo/multi-select flags and a +checkbox's on-state name. The top-level object also reports +.B xfa +and +.B dynamic_xfa +flags: for a dynamic XFA form an empty field list means the fields live in the +XFA template, not that the form is empty. This is the machine-readable companion to .BR fdf-extract , intended for scripts and AI agents that need to know what a form accepts before filling it. .TP -.BI fill " \fR[\fP\-f\fR|\fP\-\-flatten\fR] [\fP\-o \fIfile\fR] [\fP\-\-\fR]\fP fdf pdf" +.BI fill " \fR[\fP\-f\fR|\fP\-\-flatten\fR] [\fP\-o \fIfile\fR] [\fP\-\-\fR]\fP pdf values" Fill .I pdf -using the values in -.IR fdf , -writing the resulting PDF to standard output. Note the argument order: -the +using a +.I values +file, which may be an .B FDF -comes first, then the -.BR PDF . -.I fdf +or a flat +.B JSON +object (the format is auto-detected from its first byte). The +.I pdf +and +.I values +arguments may be given in either order, and +.I values may be .B \- -to read the FDF from standard input. +to read from standard input. The tool selects each field's correct type behaviour (checkbox on\-state, choice selection index, text appearance) and, for forms carrying an XFA data packet, keeps that packet in sync so viewers that render XFA show the values. @@ -145,6 +161,38 @@ ends option parsing, for file names that begin with The extraction and .B xref commands accept it too. +.IP +With +.B \-\-json +a machine-readable result is written to standard output listing the +.B updated +and +.B not_found +field names and their counts; because standard output then carries the JSON, +.B \-o +is required for the PDF. With +.B \-\-strict +the command exits 3 if any value did not match a form field (the +output is still written with the fields that matched). +.IP +A multi-select choice field is filled with an array of option strings: +.BR "/V [(opt1) (opt2)]" . +.IP +A +.B JSON +values file is an object mapping field name to a string, an array of strings +(multi-select), a number (used verbatim), or a boolean +.RB ( true / false +to check/clear a checkbox) \(em the same shape +.B fields +produces, so a caller can fill without authoring FDF. +.IP +A value longer than a text field's +.B /MaxLen +is truncated to fit, with a warning on standard error; with +.B \-\-json +the affected field names are listed under +.BR truncated . .TP .BI xref " pdf" Parse the cross\-reference table of @@ -174,7 +222,9 @@ All four AcroForm field types are handled: .BR On , or .B 1 -map to the widget's real on\-state), +map to the widget's real on\-state, and a radio group's option names are +reported by +.BR fields ), .B choice (combo box and list box, single or multi\-select), and .B signature @@ -202,6 +252,19 @@ Success. .B 1 An error occurred: invalid usage, a file that could not be opened, or a PDF whose cross\-reference table could not be parsed. +.TP +.B 2 +.B fill +only: no field in the input matched the form, so nothing was filled and no +output was written. A no-op fill (for example, every field name misspelled) is +treated as a failure rather than a silent success. +.RB ( \-\-flatten +is exempt, since it removes the form even with no new values.) +.TP +.B 3 +.B fill \-\-strict +only: the fill succeeded but some \fI(not all)\fR value did not match a form +field. .SH EXAMPLES List the fields of a form: .PP @@ -215,7 +278,7 @@ Fill a form and save the result: .PP .RS .EX -ffpdf fill answers.fdf form.pdf > filled.pdf +ffpdf fill form.pdf answers.fdf > filled.pdf .EE .RE .PP @@ -223,15 +286,40 @@ Fill and flatten to a non-editable PDF, written atomically with \-o: .PP .RS .EX -ffpdf fill \-\-flatten \-o final.pdf answers.fdf form.pdf +ffpdf fill \-\-flatten \-o final.pdf form.pdf answers.fdf +.EE +.RE +.PP +Bulk fill (mail merge): pipe a templated FDF straight in (no temp file): +.PP +.RS +.EX +envsubst < template.fdf | ffpdf fill \-o out.pdf form.pdf \- +.EE +.RE +.PP +Fill from a JSON values object (the shape +.B fields +produces): +.PP +.RS +.EX +ffpdf fill \-o out.pdf form.pdf values.json .EE .RE .PP -Mail merge: pipe a templated FDF straight in (no temp file): +where +.I values.json +is a flat object keyed by field name, e.g.: .PP .RS .EX -envsubst < template.fdf | ffpdf fill \-o out.pdf \- form.pdf +{ + "applicant_name": "Ada Lovelace", + "coverages": ["Flood", "Fire"], + "zip": 94043, + "agree": true +} .EE .RE .PP diff --git a/field_map.c b/field_map.c index 2a1e962..b364c26 100644 --- a/field_map.c +++ b/field_map.c @@ -493,7 +493,7 @@ int find_acroform_obj(FILE *f, XRefTable *xref, int root_obj) { } // Resolve the /Root (catalog) object number from the trailer / xref-stream dict. -static int find_root_obj(FILE *f) { +int find_root_obj(FILE *f) { long sx = find_startxref(f); if (sx < 0) return 0; fseek(f, 0, SEEK_END); diff --git a/field_map.h b/field_map.h index ba11e23..c319031 100644 --- a/field_map.h +++ b/field_map.h @@ -61,6 +61,9 @@ const PdfCrypt *pdf_doc_crypt(void); // Locate the AcroForm object number via the document catalog (Root). int find_acroform_obj(FILE *f, XRefTable *xref, int root_obj); +// Resolve the /Root (catalog) object number from the trailer. 0 if not found. +int find_root_obj(FILE *f); + // ---- Shared raw-object / dictionary helpers (reused by the fill writer) ---- // Raw dictionary text ("<<...>>") of an object, verbatim. Reads uncompressed diff --git a/form_extractor.c b/form_extractor.c index 4571999..55b1546 100644 --- a/form_extractor.c +++ b/form_extractor.c @@ -259,19 +259,118 @@ static void json_value(const char *v) { } } +// Advance past an indirect reference "N G R" whose N was just read. +static const char *skip_ref(const char *p) { + while (isdigit((unsigned char)*p)) p++; // N + while (isspace((unsigned char)*p)) p++; + while (isdigit((unsigned char)*p)) p++; // G + while (isspace((unsigned char)*p)) p++; + if (*p == 'R') p++; + return p; +} + +typedef struct { int obj; int page; } AnnotPage; + +// Walk the page tree from `node`, and for each /Page leaf record every entry of +// its /Annots array against that page's 1-based number. This maps a widget's +// object number to its page (fields often lack a back-pointer /P, but every +// widget appears in some page's /Annots). Bounded against cycles and huge trees. +static void map_annots_to_pages(FILE *f, XRefTable *xref, int node, int depth, + AnnotPage *m, int *n, int cap, int *pageno) { + if (depth > 64 || *n >= cap) return; + char *d = get_object_raw(f, xref, node, NULL); + if (!d) return; + const char *kids = find_key(d, "/Kids"); + if (kids && *kids == '[') { // intermediate /Pages node + const char *p = kids + 1; + while (*p && *p != ']' && *n < cap) { + while (isspace((unsigned char)*p)) p++; + if (isdigit((unsigned char)*p)) { + int kid = atoi(p); + p = skip_ref(p); + map_annots_to_pages(f, xref, kid, depth + 1, m, n, cap, pageno); + } else if (*p) p++; + } + } else { // /Page leaf + int page = ++(*pageno); + const char *an = find_key(d, "/Annots"); + char *anbuf = NULL; + if (an && isdigit((unsigned char)*an)) { // /Annots N 0 R (indirect) + anbuf = get_object_raw(f, xref, atoi(an), NULL); + an = anbuf; + while (an && *an && *an != '[') an++; + } + if (an && *an == '[') { + const char *p = an + 1; + while (*p && *p != ']' && *n < cap) { + while (isspace((unsigned char)*p)) p++; + if (isdigit((unsigned char)*p)) { + m[*n].obj = atoi(p); + m[*n].page = page; + (*n)++; + p = skip_ref(p); + } else if (*p) p++; + } + } + free(anbuf); + } + free(d); +} + void extract_form_fields_json(FILE *f, const XRefTable *xref_table) { FieldMap map = {0}; build_field_map(f, (XRefTable *)xref_table, 0, &map); + // Document-level facts an agent needs to interpret the field list: whether + // the form is XFA (AcroForm /XFA) and, if so, whether it is dynamic + // (catalog /NeedsRendering) -- for a dynamic form an empty field list means + // fields live in the XFA template, not that the form is empty. + int root = find_root_obj(f); + char *cat = root ? get_object_raw(f, (XRefTable *)xref_table, root, NULL) : NULL; + int dynamic_xfa = 0; + if (cat) { + const char *nr = find_key(cat, "/NeedsRendering"); + if (nr && strncmp(nr, "true", 4) == 0) dynamic_xfa = 1; + } + int acro = find_acroform_obj(f, (XRefTable *)xref_table, root); + char *afd = acro > 0 ? get_object_raw(f, (XRefTable *)xref_table, acro, NULL) : NULL; + int xfa = afd && find_key(afd, "/XFA") != NULL; + free(afd); + + // Widget-object -> page-number map (see map_annots_to_pages). + int pcap = 4096, nmap = 0, pageno = 0; + AnnotPage *pmap = malloc((size_t)pcap * sizeof(AnnotPage)); + const char *pk = cat ? find_key(cat, "/Pages") : NULL; + if (pmap && pk && isdigit((unsigned char)*pk)) + map_annots_to_pages(f, (XRefTable *)xref_table, atoi(pk), 0, + pmap, &nmap, pcap, &pageno); + free(cat); + printf("{\n \"fields\": ["); int emitted = 0; for (int i = 0; i < map.count; i++) { if (!map.items[i].terminal) continue; char *dict = get_object_raw(f, (XRefTable *)xref_table, map.items[i].obj_num, NULL); + // Field flags (/Ff) and /TU/MaxLen are read from the terminal field + // object (the common layout; inheritance from a parent field node is + // not resolved, matching the choice-flag handling below). + const char *ff = dict ? find_key(dict, "/Ff") : NULL; + int flags = ff ? atoi(ff) : 0; + printf("%s\n {\n \"name\": \"", emitted++ ? "," : ""); json_body((const unsigned char *)map.items[i].qname, strlen(map.items[i].qname)); + putchar('"'); + + // label: the /TU tooltip, i.e. the human-readable field name. Only + // emitted when present as a direct string (agents map data by this). + const char *tu = dict ? find_key(dict, "/TU") : NULL; + if (tu && (*tu == '(' || (*tu == '<' && tu[1] != '<'))) { + printf(",\n \"label\": "); + json_value(tu); + } + const char *type = "unknown"; switch (map.items[i].ftype) { case 'T': type = "text"; break; @@ -279,21 +378,52 @@ void extract_form_fields_json(FILE *f, const XRefTable *xref_table) { case 'C': type = "choice"; break; case 'S': type = "signature"; break; } - printf("\",\n \"type\": \"%s\",\n \"value\": ", type); + printf(",\n \"type\": \"%s\",\n \"value\": ", type); const char *v = dict ? find_key(dict, "/V") : NULL; json_value(v && !isdigit((unsigned char)*v) ? v : NULL); + // page: 1-based, resolved via the widget's presence in a page /Annots. + for (int j = 0; j < nmap; j++) + if (pmap[j].obj == map.items[i].obj_num) { + printf(",\n \"page\": %d", pmap[j].page); + break; + } + + // /Ff bit 1 = ReadOnly, bit 2 = Required (all field types). + printf(",\n \"required\": %s,\n \"readonly\": %s", + (flags & 2) ? "true" : "false", + (flags & 1) ? "true" : "false"); + + // maxlen: text fields only, when /MaxLen is present. + if (dict && map.items[i].ftype == 'T') { + const char *ml = find_key(dict, "/MaxLen"); + if (ml && isdigit((unsigned char)*ml)) + printf(",\n \"maxlen\": %d", atoi(ml)); + } + if (dict && map.items[i].ftype == 'B') { - char on[128]; - if (field_checkbox_on_state(f, (XRefTable *)xref_table, dict, - on, sizeof(on))) { - printf(",\n \"on_state\": "); - json_pdf_text((const unsigned char *)on, strlen(on)); + if (flags & 0x8000) { // radio group + char names[64][128]; + int rn = field_radio_options(f, (XRefTable *)xref_table, + dict, names, 64); + if (rn > 0) { + printf(",\n \"options\": ["); + for (int j = 0; j < rn; j++) { + if (j) fputs(", ", stdout); + json_pdf_text((const unsigned char *)names[j], strlen(names[j])); + } + putchar(']'); + } + } else { // checkbox + char on[128]; + if (field_checkbox_on_state(f, (XRefTable *)xref_table, dict, + on, sizeof(on))) { + printf(",\n \"on_state\": "); + json_pdf_text((const unsigned char *)on, strlen(on)); + } } } if (dict && map.items[i].ftype == 'C') { - const char *ff = find_key(dict, "/Ff"); - int flags = ff ? atoi(ff) : 0; printf(",\n \"combo\": %s,\n \"multi_select\": %s", (flags & 0x20000) ? "true" : "false", (flags & 0x200000) ? "true" : "false"); @@ -313,8 +443,10 @@ void extract_form_fields_json(FILE *f, const XRefTable *xref_table) { printf("\n }"); free(dict); } - printf("\n ],\n \"count\": %d\n}\n", emitted); + printf("\n ],\n \"count\": %d,\n \"xfa\": %s,\n \"dynamic_xfa\": %s\n}\n", + emitted, xfa ? "true" : "false", dynamic_xfa ? "true" : "false"); + free(pmap); field_map_free(&map); objstm_cache_reset(); } diff --git a/llms.txt b/llms.txt new file mode 100644 index 0000000..3b9fdf5 --- /dev/null +++ b/llms.txt @@ -0,0 +1,48 @@ +# ffpdf + +> A ~100 KB command-line tool in C for reading and filling PDF form fields +> (AcroForm). Only dependency: zlib. Fills text, checkbox, and choice fields, +> syncs XFA data packets, handles encrypted PDFs (empty user password), and +> writes fills as incremental updates. Built for batch use: roughly 600 form +> fills per second per core, with no runtime to start. + +Key commands: `ffpdf fields form.pdf` lists all fields as JSON (names, labels, +types, values, page, required/readonly/maxlen, choice options, checkbox +on-states, and document xfa/dynamic_xfa flags). `ffpdf fill form.pdf +values.json > out.pdf` fills a form from a values file. The values file may be +an FDF or a JSON object (the same shape `fields` emits, so agents never author +FDF), auto-detected; the PDF and values may be in either order, and the values +may be `-` for stdin. The JSON is a flat object keyed by field name; the value +type says how to fill it: string (text/choice), array of strings (multi-select), +number (verbatim), boolean (check/clear a checkbox), null (skip). Example: +`{"State": "NH", "Coverages": ["Flood","Fire"], "Paperless": true, "Zip": 3801}`. +`-o FILE` writes atomically; `--flatten` bakes values in and removes the form; +`--json` writes a JSON result (updated/not_found/truncated fields) to stdout; +`--strict` exits 3 if any value did not match a field. Exit codes: 0 success, +1 error, 2 nothing matched (no output written), 3 --strict partial mismatch. +Diagnostics go to stderr, output to stdout. + +Install: download a static binary archive from +https://github.com/pagebrooks/ffpdf/releases (Linux, macOS, Windows), or +build from source with `make` (needs gcc and zlib). + +## Documentation + +- [README](https://github.com/pagebrooks/ffpdf/blob/main/README.md): full + guide with a worked example and the bulk-fill (mail-merge) batch pattern +- [Man page](https://github.com/pagebrooks/ffpdf/blob/main/ffpdf.1): complete + command and option reference +- [Security datasheet](https://github.com/pagebrooks/ffpdf/blob/main/docs/SECURITY-PRIVACY.md): + data handling, no network access, PHI guidance + +## Examples + +- [Example form](https://github.com/pagebrooks/ffpdf/blob/main/docs/example-form.pdf): + unfilled form exercising every field type +- [Example FDF](https://github.com/pagebrooks/ffpdf/blob/main/docs/example-answers.fdf): + the values file format, including a multi-select array + +## Optional + +- [CHANGELOG](https://github.com/pagebrooks/ffpdf/blob/main/CHANGELOG.md): release history +- [CONTRIBUTING](https://github.com/pagebrooks/ffpdf/blob/main/CONTRIBUTING.md): build, test, DCO sign-off diff --git a/main.c b/main.c index c28ae90..f666db6 100644 --- a/main.c +++ b/main.c @@ -30,7 +30,7 @@ extern long find_startxref(FILE *f); #define PROG_NAME "ffpdf" -#define PROG_VERSION "0.1.0" +#define PROG_VERSION "0.1.1" // Program name as invoked (without any directory), for diagnostics. static const char *prog_basename(const char *p) { @@ -66,16 +66,25 @@ PROG_NAME " " PROG_VERSION " \xe2\x80\x94 read and fill PDF form fields.\n" "COMMANDS\n" " fdf-extract Extract form fields as an FDF\n" " xfdf-extract Extract form fields as an XFDF\n" -" fields List form fields as JSON: names, types,\n" -" current values, choice options, checkbox\n" -" on-states. Made for scripts and AI agents\n" -" fill [options] Fill with values from \n" +" fields List form fields as JSON: names, labels,\n" +" types, values, page, required/readonly/\n" +" maxlen, choice options, checkbox on-states,\n" +" plus document xfa/dynamic_xfa flags. Made\n" +" for scripts and AI agents\n" +" fill [options] Fill from a file, which may\n" +" be an FDF or a JSON object (auto-detected);\n" +" the PDF and values may be in either order,\n" +" and may be '-' for stdin.\n" " -f, --flatten bake the values in and remove\n" " the form -> a non-editable PDF\n" " -o, --output FILE write to FILE instead of\n" " stdout (atomically: FILE.tmp is\n" " renamed over FILE on success)\n" -" may be '-' to read the FDF from stdin\n" +" --json write a JSON result (updated /\n" +" not_found fields) to stdout;\n" +" requires -o for the PDF\n" +" --strict exit 3 if any value did not\n" +" match a form field\n" " xref Dump the parsed cross-reference table (debug)\n" " help Show this help and exit (also -h, --help)\n" " version Show version information (also -v, --version)\n" @@ -83,9 +92,9 @@ PROG_NAME " " PROG_VERSION " \xe2\x80\x94 read and fill PDF form fields.\n" "WORKFLOW\n" " 1. Discover field names: " PROG_NAME " fdf-extract form.pdf\n" " 2. Edit the FDF, setting each field's /V (value) beside its /T (name).\n" -" 3. Fill it: " PROG_NAME " fill answers.fdf form.pdf > filled.pdf\n" +" 3. Fill it: " PROG_NAME " fill form.pdf answers.fdf > filled.pdf\n" "\n" -" Mail merge / bulk fills: an FDF is plain text, so template it and loop --\n" +" Bulk fills / mail merge: an FDF is plain text, so template it and loop --\n" " substitute per-record values (e.g. ${NAME} + envsubst) and run fill once\n" " per record. No startup cost: thousands of filled PDFs per minute per core.\n" "\n" @@ -95,6 +104,10 @@ PROG_NAME " " PROG_VERSION " \xe2\x80\x94 read and fill PDF form fields.\n" " Progress and warnings go to stderr, so they never corrupt the output.\n" " * '--' ends option parsing (for file names that begin with '-').\n" " * fill takes the FDF first, then the PDF: fill .\n" +" * A JSON values file maps field name -> value (string), array (multi-\n" +" select), number, or boolean (true/false to check/clear a box).\n"" * A value longer than a text field's /MaxLen is truncated to fit (with a\n" +" warning; --json lists it under \"truncated\").\n" +" * A multi-select choice field in FDF takes an array: /V [(opt1) (opt2)].\n" " * Fills are incremental updates: the original bytes are preserved and the\n" " changes appended. --flatten instead bakes the values in and removes the form.\n" " * Encrypted PDFs are supported for the empty user password (RC4/AES).\n" @@ -110,6 +123,8 @@ PROG_NAME " " PROG_VERSION " \xe2\x80\x94 read and fill PDF form fields.\n" "EXIT STATUS\n" " 0 success\n" " 1 error: bad usage, or an unreadable / unparseable PDF\n" +" 2 fill: no field in the input matched the form (nothing filled; no output)\n" +" 3 fill --strict: some (but not all) fields did not match a form field\n" "\n" "See ffpdf(1) or README.md for details.\n", stdout); @@ -175,14 +190,35 @@ static int cmd_extract(int mode, const char *path) { // Run `fill`: resolve the stdin-FDF and -o conventions, then delegate to // fill_pdf_with_fdf_ex. With -o the PDF is written to .tmp and renamed // into place only on success, so a failed fill never leaves a partial output. -static int cmd_fill(const char *prog, const char *fdf_arg, const char *pdf_arg, - const char *outpath, int flatten) { +// Peek a file's first bytes: is it a PDF (starts with "%PDF" after optional +// whitespace)? Returns 1 (PDF), 0 (not), -1 (cannot open). Used to tell the +// PDF and the values file apart regardless of argument order. +static int file_is_pdf(const char *path) { + FILE *f = portable_fopen(path, "rb"); + if (!f) return -1; + char buf[16]; + size_t n = fread(buf, 1, sizeof buf, f); + fclose(f); + size_t i = 0; + while (i < n && (buf[i] == ' ' || buf[i] == '\t' || buf[i] == '\n' || buf[i] == '\r')) i++; + return (n - i >= 4 && memcmp(buf + i, "%PDF", 4) == 0) ? 1 : 0; +} + +// Run `fill`: `values_arg` is an FDF or JSON file (or "-" for stdin), auto- +// detected; with -o the PDF is written to .tmp and renamed on success. +static int cmd_fill(const char *prog, const char *pdf_arg, const char *values_arg, + const char *outpath, int flatten, int json, int strict) { + // --json writes the machine-readable result to stdout, so the PDF must go + // to a file (-o); the two structured outputs cannot share stdout. + if (json && !outpath) { + fprintf(stderr, "%s: --json requires -o FILE (stdout carries the JSON result)\n", prog); + return 1; + } int explicit_stdout = 0; if (outpath && strcmp(outpath, "-") == 0) { // -o - : stdout, explicitly outpath = NULL; explicit_stdout = 1; } - if (!outpath && !explicit_stdout && portable_isatty_stdout()) { fprintf(stderr, "%s: refusing to write a PDF to a terminal.\n" @@ -191,66 +227,57 @@ static int cmd_fill(const char *prog, const char *fdf_arg, const char *pdf_arg, return 1; } - // FDF from stdin ("-"): spool to a temp file so the parser has a path. - char fdf_tmp[4096]; - int fdf_is_tmp = 0; - if (strcmp(fdf_arg, "-") == 0) { - FILE *tf = os_temp_file(fdf_tmp, sizeof fdf_tmp); - if (!tf) { - fprintf(stderr, "%s: cannot create a temp file for the stdin FDF\n", prog); - return 1; - } - char buf[65536]; - size_t r; - while ((r = fread(buf, 1, sizeof buf, stdin)) > 0) { - if (fwrite(buf, 1, r, tf) != r) { - fprintf(stderr, "%s: cannot spool stdin FDF: write failed\n", prog); - fclose(tf); - portable_remove(fdf_tmp); - return 1; - } - } - fclose(tf); - fdf_arg = fdf_tmp; - fdf_is_tmp = 1; - } - FILE *out = stdout; char out_tmp[4096]; if (outpath) { int n = snprintf(out_tmp, sizeof out_tmp, "%s.tmp", outpath); if (n < 0 || (size_t)n >= sizeof out_tmp) { fprintf(stderr, "%s: output path too long\n", prog); - if (fdf_is_tmp) portable_remove(fdf_tmp); return 1; } out = portable_fopen(out_tmp, "wb"); if (!out) { fprintf(stderr, "%s: cannot open %s for writing\n", prog, out_tmp); - if (fdf_is_tmp) portable_remove(fdf_tmp); return 1; } } - int rc = fill_pdf_with_fdf_ex(pdf_arg, fdf_arg, out, flatten); + FillResult res = {0}; + FillResult *resptr = (json || strict) ? &res : NULL; + int rc = fill_pdf_with_values(pdf_arg, values_arg, out, flatten, resptr); if (outpath) { if (fclose(out) != 0 && rc == 0) { fprintf(stderr, "%s: write to %s failed\n", prog, out_tmp); rc = 1; } - if (rc == 0) { - if (portable_rename(out_tmp, outpath) != 0) { - fprintf(stderr, "%s: cannot rename %s to %s\n", prog, out_tmp, outpath); - rc = 1; - } + if (rc == 0 && portable_rename(out_tmp, outpath) != 0) { + fprintf(stderr, "%s: cannot rename %s to %s\n", prog, out_tmp, outpath); + rc = 1; } if (rc != 0) portable_remove(out_tmp); // never leave a partial output } - if (fdf_is_tmp) portable_remove(fdf_tmp); + + // --json result to stdout whenever the fill ran (rc 0 = ok, 2 = no match). + if (json && (rc == 0 || rc == 2)) { + printf("{\n \"updated\": ["); + for (int i = 0; i < res.n_updated; i++) + printf("%s\n \"%s\"", i ? "," : "", res.updated[i]); + printf("%s ],\n \"not_found\": [", res.n_updated ? "\n" : ""); + for (int i = 0; i < res.n_not_found; i++) + printf("%s\n \"%s\"", i ? "," : "", res.not_found[i]); + printf("%s ],\n \"truncated\": [", res.n_not_found ? "\n" : ""); + for (int i = 0; i < res.n_truncated; i++) + printf("%s\n \"%s\"", i ? "," : "", res.truncated[i]); + printf("%s ],\n \"updated_count\": %d,\n \"not_found_count\": %d\n}\n", + res.n_truncated ? "\n" : "", res.n_updated, res.n_not_found); + } + // --strict: a clean write with any unmatched field is a soft failure (3). + if (strict && rc == 0 && res.n_not_found > 0) rc = 3; + + fill_result_free(&res); return rc; } - int main(int argc, char *argv[]) { #ifdef _WIN32 // On Windows stdout defaults to text mode, which rewrites '\n' as "\r\n" and @@ -281,7 +308,7 @@ int main(int argc, char *argv[]) { // fill [--flatten] [-o FILE] [--] -> PDF on stdout (or FILE) if (strcmp(cmd, "fill") == 0) { - int flatten = 0, a = 2; + int flatten = 0, json = 0, strict = 0, a = 2; const char *outpath = NULL; for (; a < argc; a++) { if (strcmp(argv[a], "--") == 0) { a++; break; } // end of options @@ -289,6 +316,8 @@ int main(int argc, char *argv[]) { flatten = 1; continue; } + if (strcmp(argv[a], "--json") == 0) { json = 1; continue; } + if (strcmp(argv[a], "--strict") == 0) { strict = 1; continue; } if (strcmp(argv[a], "-o") == 0 || strcmp(argv[a], "--output") == 0) { if (a + 1 >= argc) { fprintf(stderr, "%s: %s requires a file name\n", prog, argv[a]); @@ -306,7 +335,26 @@ int main(int argc, char *argv[]) { break; // first positional } if (argc - a != 2) { print_usage(stderr, prog); return 1; } - return cmd_fill(prog, argv[a], argv[a + 1], outpath, flatten); + // Two positionals: a PDF and a values file (FDF or JSON), in either + // order. "-" is always the values (stdin); otherwise the "%PDF" file is + // the PDF and the other is the values. + const char *arg1 = argv[a], *arg2 = argv[a + 1]; + const char *pdf = NULL, *values = NULL; + if (strcmp(arg1, "-") == 0) { values = arg1; pdf = arg2; } + else if (strcmp(arg2, "-") == 0) { values = arg2; pdf = arg1; } + else { + int p1 = file_is_pdf(arg1), p2 = file_is_pdf(arg2); + if (p1 < 0) { fprintf(stderr, "%s: cannot open '%s'\n", prog, arg1); return 1; } + if (p2 < 0) { fprintf(stderr, "%s: cannot open '%s'\n", prog, arg2); return 1; } + if (p1 && !p2) { pdf = arg1; values = arg2; } + else if (p2 && !p1) { pdf = arg2; values = arg1; } + else { + fprintf(stderr, "%s: give one PDF and one values file (FDF or JSON); " + "could not tell them apart\n", prog); + return 1; + } + } + return cmd_fill(prog, pdf, values, outpath, flatten, json, strict); } // single-PDF commands diff --git a/pdf_filler.c b/pdf_filler.c index ffb6009..59ad1da 100644 --- a/pdf_filler.c +++ b/pdf_filler.c @@ -561,6 +561,51 @@ int field_checkbox_on_state(FILE *f, XRefTable *xref, const char *dict, char *ou return ok; } +// Iterate a field's /Kids widget object numbers, calling `visit(kid_obj, ctx)`. +static void for_each_kid(FILE *f, XRefTable *xref, const char *dict, + void (*visit)(int, void *), void *vctx) { + char *kids = field_array_text(f, xref, dict, "/Kids"); + if (!kids) return; + const char *p = kids; + if (*p == '[') p++; + while (*p && *p != ']') { + while (isspace((unsigned char)*p)) p++; + if (!isdigit((unsigned char)*p)) { if (*p) p++; continue; } + int kid = atoi(p); + while (isdigit((unsigned char)*p)) p++; // obj num + while (isspace((unsigned char)*p)) p++; + while (isdigit((unsigned char)*p)) p++; // gen + while (isspace((unsigned char)*p)) p++; + if (*p == 'R') p++; + visit(kid, vctx); + } + free(kids); +} + +// Collect a radio group's option (on-state) names from its kids' /AP /N. These +// are the values fill accepts to select a button. Returns the count (<= max), +// 0 if the field is not a radio group with named kid states. +struct radio_opts { FILE *f; XRefTable *xref; char (*names)[128]; int n, max; }; +static void radio_opt_visit(int kid, void *vctx) { + struct radio_opts *r = vctx; + if (r->n >= r->max) return; + char *kd = get_object_raw(r->f, r->xref, kid, NULL); + if (!kd) return; + char on[128]; + if (field_checkbox_on_state(r->f, r->xref, kd, on, sizeof on)) { + int dup = 0; + for (int j = 0; j < r->n; j++) if (strcmp(r->names[j], on) == 0) dup = 1; + if (!dup) snprintf(r->names[r->n++], 128, "%s", on); + } + free(kd); +} +int field_radio_options(FILE *f, XRefTable *xref, const char *dict, + char names[][128], int max) { + struct radio_opts r = { f, xref, names, 0, max }; + for_each_kid(f, xref, dict, radio_opt_visit, &r); + return r.n; +} + // Resolve the appearance-state name to write for a button field. An off-like // value maps to /Off; otherwise the widget's real on-state (so a caller value of // "Yes"/"On"/"1" checks the box even when the on-state is named differently), @@ -710,20 +755,176 @@ static void fdf_add(FdfData *d, const char *name, const char *value) { } -FdfData *parse_fdf_file(const char *fdf_filename) { - FILE *fp = portable_fopen(fdf_filename, "rb"); - if (!fp) { fprintf(stderr, "ERROR: cannot open FDF file: %s\n", fdf_filename); return NULL; } - fseek(fp, 0, SEEK_END); - long sz = ftell(fp); - fseek(fp, 0, SEEK_SET); - char *content = malloc((size_t)sz + 1); - if (!content) { fclose(fp); return NULL; } - size_t got = fread(content, 1, (size_t)sz, fp); - content[got] = '\0'; - fclose(fp); +// Read a whole file (or stdin for "-") into a malloc'd, NUL-terminated buffer. +static char *slurp_file(const char *path, size_t *len_out) { + FILE *fp; + if (strcmp(path, "-") == 0) fp = stdin; + else { fp = portable_fopen(path, "rb"); if (!fp) return NULL; } + size_t cap = 65536, n = 0; + char *buf = malloc(cap); + if (!buf) { if (fp != stdin) fclose(fp); return NULL; } + size_t r; + while ((r = fread(buf + n, 1, cap - n, fp)) > 0) { + n += r; + if (n == cap) { cap *= 2; char *g = realloc(buf, cap); if (!g) { free(buf); if (fp != stdin) fclose(fp); return NULL; } buf = g; } + } + if (fp != stdin) fclose(fp); + buf[n] = '\0'; + if (len_out) *len_out = n; + return buf; +} + +static int hex4(const char *p) { + int v = 0; + for (int i = 0; i < 4; i++) { + int c = p[i], d; + if (c >= '0' && c <= '9') d = c - '0'; + else if (c >= 'a' && c <= 'f') d = c - 'a' + 10; + else if (c >= 'A' && c <= 'F') d = c - 'A' + 10; + else return -1; + v = (v << 4) | d; + } + return v; +} + +static void utf8_append(char **s, size_t *n, size_t *cap, unsigned long cp) { + char tmp[4]; int k = 0; + if (cp < 0x80) tmp[k++] = (char)cp; + else if (cp < 0x800) { tmp[k++] = (char)(0xC0 | (cp >> 6)); tmp[k++] = (char)(0x80 | (cp & 0x3F)); } + else if (cp < 0x10000) { tmp[k++] = (char)(0xE0 | (cp >> 12)); tmp[k++] = (char)(0x80 | ((cp >> 6) & 0x3F)); tmp[k++] = (char)(0x80 | (cp & 0x3F)); } + else { tmp[k++] = (char)(0xF0 | (cp >> 18)); tmp[k++] = (char)(0x80 | ((cp >> 12) & 0x3F)); tmp[k++] = (char)(0x80 | ((cp >> 6) & 0x3F)); tmp[k++] = (char)(0x80 | (cp & 0x3F)); } + while (*n + (size_t)k + 1 > *cap) { *cap *= 2; char *g = realloc(*s, *cap); if (!g) return; *s = g; } + for (int i = 0; i < k; i++) (*s)[(*n)++] = tmp[i]; +} + +// Parse a JSON string at `p` ('"'); return the position past the closing quote, +// with the decoded UTF-8 in *out (malloc'd). NULL on malformed input. +static const char *json_str(const char *p, char **out) { + if (*p != '"') return NULL; + p++; + size_t cap = 32, n = 0; + char *s = malloc(cap); + if (!s) return NULL; + while (*p && *p != '"') { + if (*p == '\\') { + p++; + char c = 0; + switch (*p) { + case '"': c = '"'; break; case '\\': c = '\\'; break; case '/': c = '/'; break; + case 'n': c = '\n'; break; case 't': c = '\t'; break; case 'r': c = '\r'; break; + case 'b': c = '\b'; break; case 'f': c = '\f'; break; + case 'u': { + int cp = hex4(p + 1); + if (cp < 0) { free(s); return NULL; } + p += 4; + if (cp >= 0xD800 && cp <= 0xDBFF && p[1] == '\\' && p[2] == 'u') { + int lo = hex4(p + 3); + if (lo >= 0xDC00 && lo <= 0xDFFF) { cp = 0x10000 + ((cp - 0xD800) << 10) + (lo - 0xDC00); p += 6; } + } + utf8_append(&s, &n, &cap, (unsigned long)cp); + p++; + continue; + } + default: free(s); return NULL; // invalid escape + } + p++; + if (n + 2 > cap) { cap *= 2; char *g = realloc(s, cap); if (!g) { free(s); return NULL; } s = g; } + s[n++] = c; + } else { + if (n + 2 > cap) { cap *= 2; char *g = realloc(s, cap); if (!g) { free(s); return NULL; } s = g; } + s[n++] = *p++; + } + } + if (*p != '"') { free(s); return NULL; } + s[n] = '\0'; + *out = s; + return p + 1; +} + +static const char *json_ws(const char *p) { + while (*p == ' ' || *p == '\t' || *p == '\n' || *p == '\r') p++; + return p; +} + +// Parse a flat JSON object of field values into an FdfData (the same structure +// parse_fdf_file produces), so `fill` can consume the JSON an agent already got +// from `fields`. Values may be a string, an array of strings (multi-select), a +// number (used verbatim), or a boolean (true -> "Yes" to check a box, false -> +// "Off"); null skips the field. `path` may be "-" for stdin. NULL on error. +static FdfData *parse_json_mem(const char *content) { + FdfData *d = calloc(1, sizeof(FdfData)); + if (!d) return NULL; + + const char *p = json_ws(content); + if (*p != '{') { fprintf(stderr, "ERROR: JSON values must be an object\n"); goto bad; } + p = json_ws(p + 1); + while (*p && *p != '}') { + char *key = NULL; + p = json_str(p, &key); + if (!p) { fprintf(stderr, "ERROR: malformed JSON key\n"); goto bad; } + p = json_ws(p); + if (*p != ':') { free(key); fprintf(stderr, "ERROR: expected ':' after key\n"); goto bad; } + p = json_ws(p + 1); + + if (*p == '"') { // string value + char *val = NULL; + p = json_str(p, &val); + if (!p) { free(key); fprintf(stderr, "ERROR: malformed JSON string value\n"); goto bad; } + fdf_add(d, key, val ? val : ""); + free(val); + } else if (*p == '[') { // array of strings (multi-select) + char *marr[64]; int mc = 0; + p = json_ws(p + 1); + while (*p && *p != ']') { + if (*p != '"') { for (int j = 0; j < mc; j++) free(marr[j]); free(key); fprintf(stderr, "ERROR: array must hold strings\n"); goto bad; } + char *v = NULL; + p = json_str(p, &v); + if (!p) { for (int j = 0; j < mc; j++) free(marr[j]); free(key); goto bad; } + if (mc < 64) marr[mc++] = v; else free(v); + p = json_ws(p); + if (*p == ',') p = json_ws(p + 1); + } + if (*p == ']') p++; + fdf_add(d, key, mc > 0 ? marr[0] : ""); + if (mc > 1 && d->field_count > 0) { + FdfField *fl = &d->fields[d->field_count - 1]; + fl->values = malloc((size_t)mc * sizeof(char *)); + if (fl->values) { for (int j = 0; j < mc; j++) fl->values[j] = marr[j]; fl->nvalues = mc; } + } else { + for (int j = 0; j < mc; j++) free(marr[j]); + } + } else if (strncmp(p, "true", 4) == 0) { // check a box + fdf_add(d, key, "Yes"); p += 4; + } else if (strncmp(p, "false", 5) == 0) { // uncheck + fdf_add(d, key, "Off"); p += 5; + } else if (strncmp(p, "null", 4) == 0) { // skip + p += 4; + } else if (*p == '-' || (*p >= '0' && *p <= '9')) { // number: verbatim text + const char *q = p; + while (*q == '-' || *q == '+' || *q == '.' || *q == 'e' || *q == 'E' || + (*q >= '0' && *q <= '9')) q++; + char num[64]; size_t nl = (size_t)(q - p); + if (nl >= sizeof(num)) nl = sizeof(num) - 1; + memcpy(num, p, nl); num[nl] = '\0'; + fdf_add(d, key, num); + p = q; + } else { + free(key); fprintf(stderr, "ERROR: unsupported JSON value type\n"); goto bad; + } + free(key); + p = json_ws(p); + if (*p == ',') p = json_ws(p + 1); + } + return d; + +bad: + free_fdf_data(d); + return NULL; +} +static FdfData *parse_fdf_mem(const char *content) { FdfData *d = calloc(1, sizeof(FdfData)); - if (!d) { free(content); return NULL; } + if (!d) return NULL; // Walk field dictionaries: each has /T (name) and optionally /V (value). // We scan for "/T" occurrences and read the paired "/V" that follows within @@ -778,6 +979,29 @@ FdfData *parse_fdf_file(const char *fdf_filename) { p = after; } + return d; +} + +FdfData *parse_fdf_file(const char *fdf_filename) { + char *content = slurp_file(fdf_filename, NULL); + if (!content) { fprintf(stderr, "ERROR: cannot read FDF file: %s\n", fdf_filename); return NULL; } + FdfData *d = parse_fdf_mem(content); + free(content); + return d; +} + +// Read a values source (FDF or JSON, or "-" for stdin) and parse it into an +// FdfData, auto-detecting the format from the first non-whitespace byte: +// '{' is JSON, '%' is FDF. NULL on error. +FdfData *parse_values(const char *path) { + char *content = slurp_file(path, NULL); + if (!content) { fprintf(stderr, "ERROR: cannot read values file: %s\n", path); return NULL; } + const char *p = json_ws(content); + FdfData *d = NULL; + if (*p == '{') d = parse_json_mem(content); + else if (*p == '%') d = parse_fdf_mem(content); // %FDF + else fprintf(stderr, "ERROR: unrecognized values format " + "(expected an FDF or a JSON object)\n"); free(content); return d; } @@ -985,6 +1209,39 @@ static void emit_field_appearance(FillCtx *c, const char *fodict, int ap_obj, fill_record(c, ap_obj, apoff, 0); } +// Emit each kid widget of a radio group with /AS set: the kid whose on-state +// matches `state` gets /AS/, the rest /AS/Off, so the selection renders +// in viewers that draw a widget from its own /AS (not just the parent /V). +struct radio_emit { FillCtx *c; const char *state; }; +static void radio_kid_emit(int kid, void *vctx) { + struct radio_emit *r = vctx; + int gen = 0; + char *kd = get_object_raw(r->c->f, r->c->xref, kid, &gen); + if (!kd) return; + char on[128]; + int selected = field_checkbox_on_state(r->c->f, r->c->xref, kd, on, sizeof on) + && strcmp(on, r->state) == 0; + char *tmp = extract_dict_inner_alloc(kd); + if (tmp) { + remove_entry(tmp, "/AS"); + size_t ln = strlen(tmp); + while (ln > 0 && isspace((unsigned char)tmp[ln - 1])) tmp[--ln] = '\0'; + Buf kb = {0}; + buf_puts(&kb, tmp); + buf_printf(&kb, "/AS/%s", selected ? r->state : "Off"); + long off = r->c->base_len + (long)r->c->o.len; + emit_dict_object(&r->c->o, r->c->crypt, kid, gen, (const char *)kb.data); + fill_record(r->c, kid, off, gen); + free(kb.data); + free(tmp); + } + free(kd); +} +static void emit_radio_kids(FillCtx *c, const char *field_dict, const char *state) { + struct radio_emit r = { c, state }; + for_each_kid(c->f, c->xref, field_dict, radio_kid_emit, &r); +} + // Emit the updated object (and any appearance stream) for FDF field `i`, which // maps to `loc`. Skips signature fields and no-ops on unresolved dictionaries. static void emit_filled_field(FillCtx *c, FieldMap *map, FdfData *fdf, int i, FieldLoc *loc) { @@ -1059,6 +1316,9 @@ static void emit_filled_field(FillCtx *c, FieldMap *map, FdfData *fdf, int i, Fi emit_dict_object(&c->o, c->crypt, loc->obj_num, loc->gen_num, (const char *)fb.data); fill_record(c, loc->obj_num, off, loc->gen_num); c->updated_fields++; + // Radio group: also set the matching kid widget's /AS so it renders. + if (loc->ftype == 'B' && (field_flags(fodict) & 0x8000)) + emit_radio_kids(c, fodict, btn_state); if (log_field_values()) fprintf(stderr, "Filled '%s' (obj %d) = '%s'%s%s\n", loc->qname, loc->obj_num, value, (loc->ftype == 'C' && choice_idx >= 0) ? " [+/I]" : "", @@ -1423,10 +1683,77 @@ static void flatten_form(FillCtx *c, int root_obj) { fprintf(stderr, "Flattened %d page(s); removed the interactive form\n", npages); } -int fill_pdf_with_fdf_ex(const char *pdf_filename, const char *fdf_filename, FILE *out, int flatten) { - FdfData *fdf = parse_fdf_file(fdf_filename); - if (!fdf) return 1; - fprintf(stderr, "Parsed %d field value(s) from FDF\n", fdf->field_count); +// Append a strdup'd copy of `s` to a growable char* array (FillResult lists). +static void strv_add(char ***arr, int *n, const char *s) { + char **g = realloc(*arr, (size_t)(*n + 1) * sizeof(char *)); + if (!g) return; + *arr = g; + (*arr)[*n] = strdup(s); + if ((*arr)[*n]) (*n)++; +} + +void fill_result_free(FillResult *r) { + if (!r) return; + for (int i = 0; i < r->n_updated; i++) free(r->updated[i]); + for (int i = 0; i < r->n_not_found; i++) free(r->not_found[i]); + for (int i = 0; i < r->n_truncated; i++) free(r->truncated[i]); + free(r->updated); free(r->not_found); free(r->truncated); + r->updated = r->not_found = r->truncated = NULL; + r->n_updated = r->n_not_found = r->n_truncated = 0; +} + +// Byte length of the prefix of `s` holding at most `n` UTF-8 characters +// (never splitting a multibyte sequence). +static size_t utf8_prefix_bytes(const char *s, int n) { + size_t i = 0, len = strlen(s); + int chars = 0; + while (i < len && chars < n) { + unsigned char c = s[i]; + size_t step = (c < 0x80) ? 1 : (c < 0xE0) ? 2 : (c < 0xF0) ? 3 : 4; + if (i + step > len) step = len - i; // clamp a malformed trailing seq + i += step; + chars++; + } + return i; +} +static int utf8_len(const char *s) { + size_t i = 0, len = strlen(s); + int n = 0; + while (i < len) { + unsigned char c = s[i]; + size_t step = (c < 0x80) ? 1 : (c < 0xE0) ? 2 : (c < 0xF0) ? 3 : 4; + if (i + step > len) step = len - i; + i += step; + n++; + } + return n; +} + +// If `loc` is a text field with a /MaxLen shorter than its value, truncate the +// value in place (it is heap-owned), warn, and record it in the result. This +// keeps the written /V conformant, matching what a viewer would enforce. +static void enforce_maxlen(FILE *f, XRefTable *xref, FdfData *fdf, int i, + FieldLoc *loc, FillResult *res) { + if (loc->ftype != 'T') return; + char *fdict = get_object_raw(f, xref, loc->obj_num, NULL); + if (!fdict) return; + const char *ml = find_key(fdict, "/MaxLen"); + if (ml && isdigit((unsigned char)*ml)) { + int maxlen = atoi(ml); + char *v = fdf->fields[i].field_value; + if (maxlen > 0 && v && utf8_len(v) > maxlen) { + size_t cut = utf8_prefix_bytes(v, maxlen); + v[cut] = '\0'; + fprintf(stderr, "WARNING: value for '%s' exceeds MaxLen %d; truncated\n", + loc->qname, maxlen); + if (res) strv_add(&res->truncated, &res->n_truncated, loc->qname); + } + } + free(fdict); +} + +static int fill_core(const char *pdf_filename, FdfData *fdf, + FILE *out, int flatten, FillResult *res) { // All resources released together at `cleanup` (rc carries the exit code); // xref/map/ctx are safe to free in their zero-initialized state, so every @@ -1508,6 +1835,7 @@ int fill_pdf_with_fdf_ex(const char *pdf_filename, const char *fdf_filename, FIL FieldLoc *loc = match_field(&map, fdf->fields[i].field_name); if (!loc) { fprintf(stderr, "WARNING: field not found in PDF: '%s'\n", fdf->fields[i].field_name); + if (res) strv_add(&res->not_found, &res->n_not_found, fdf->fields[i].field_name); continue; } int superseded = 0; @@ -1516,7 +1844,9 @@ int fill_pdf_with_fdf_ex(const char *pdf_filename, const char *fdf_filename, FIL if (l2 && l2->obj_num == loc->obj_num) superseded = 1; } if (superseded) continue; + enforce_maxlen(f, &xref, fdf, i, loc, res); emit_filled_field(&ctx, &map, fdf, i, loc); + if (res) strv_add(&res->updated, &res->n_updated, loc->qname); } if (ctx.flatten) { @@ -1526,8 +1856,17 @@ int fill_pdf_with_fdf_ex(const char *pdf_filename, const char *fdf_filename, FIL sync_xfa_datasets(&ctx, fdf, &map, acroform); } + // Nothing in the input matched a form field: a no-op fill is a failure, not + // a silent success. Exit 2 (distinct from 1 = bad input). Flatten is exempt + // -- flattening removes the form even when no new values are supplied. + if (!ctx.flatten && ctx.updated_fields == 0) { + fprintf(stderr, "ERROR: no field in the input matched the form; " + "nothing to fill\n"); + rc = 2; + goto cleanup; + } if (ctx.nentries == 0) { - fprintf(stderr, "ERROR: no fields were updated; not writing output\n"); + fprintf(stderr, "ERROR: no changes to write\n"); goto cleanup; } @@ -1570,10 +1909,33 @@ int fill_pdf_with_fdf_ex(const char *pdf_filename, const char *fdf_filename, FIL objstm_cache_reset(); xref_free(&xref); if (f) fclose(f); + return rc; +} + +int fill_pdf_with_fdf_res(const char *pdf_filename, const char *fdf_filename, + FILE *out, int flatten, FillResult *res) { + FdfData *fdf = parse_fdf_file(fdf_filename); + if (!fdf) return 1; + fprintf(stderr, "Parsed %d field value(s) from FDF\n", fdf->field_count); + int rc = fill_core(pdf_filename, fdf, out, flatten, res); + free_fdf_data(fdf); + return rc; +} + +int fill_pdf_with_values(const char *pdf_filename, const char *values_filename, + FILE *out, int flatten, FillResult *res) { + FdfData *fdf = parse_values(values_filename); + if (!fdf) return 1; + fprintf(stderr, "Parsed %d field value(s)\n", fdf->field_count); + int rc = fill_core(pdf_filename, fdf, out, flatten, res); free_fdf_data(fdf); return rc; } +int fill_pdf_with_fdf_ex(const char *pdf_filename, const char *fdf_filename, FILE *out, int flatten) { + return fill_pdf_with_fdf_res(pdf_filename, fdf_filename, out, flatten, NULL); +} + int fill_pdf_with_fdf(const char *pdf_filename, const char *fdf_filename, FILE *out) { return fill_pdf_with_fdf_ex(pdf_filename, fdf_filename, out, 0); } diff --git a/pdf_filler.h b/pdf_filler.h index eab784d..32b0efc 100644 --- a/pdf_filler.h +++ b/pdf_filler.h @@ -47,6 +47,29 @@ int fill_pdf_with_fdf(const char *pdf_filename, const char *fdf_filename, FILE * // XFA) is removed, producing a non-editable ("flattened") PDF. int fill_pdf_with_fdf_ex(const char *pdf_filename, const char *fdf_filename, FILE *out, int flatten); +// Structured outcome of a fill, for programmatic callers (the `fill --json` +// path). `updated` holds the qualified names of the fields actually written; +// `not_found` holds the FDF field names that matched no form field. +typedef struct { + char **updated; int n_updated; + char **not_found; int n_not_found; + char **truncated; int n_truncated; // text fields cut to their /MaxLen +} FillResult; + +// As fill_pdf_with_fdf_ex, but when `res` is non-NULL it is populated with the +// matched/unmatched field names (caller frees with fill_result_free). +int fill_pdf_with_fdf_res(const char *pdf_filename, const char *fdf_filename, + FILE *out, int flatten, FillResult *res); +void fill_result_free(FillResult *res); + +// As fill_pdf_with_fdf_res, but the values source may be an FDF *or* a flat JSON +// object { "FieldName": "value", "Multi": ["a","b"], "Box": true }; the format +// is auto-detected from the file's first byte, and `values_filename` may be "-" +// for stdin. Lets an agent fill from the same JSON shape `fields` produces. +int fill_pdf_with_values(const char *pdf_filename, const char *values_filename, + FILE *out, int flatten, FillResult *res); +FdfData *parse_values(const char *path); + // Field-introspection helpers shared with the `fields` command (JSON listing). // field_choice_options fills disp[] with a choice field's option display texts // (the strings fill's /Opt matching accepts); returns the count, 0 when /Opt is @@ -58,6 +81,11 @@ int field_choice_options(FILE *f, XRefTable *xref, const char *dict, int field_checkbox_on_state(FILE *f, XRefTable *xref, const char *dict, char *out, size_t cap); +// field_radio_options fills names[] with a radio group's option (on-state) +// names -- the values fill accepts to select a button. Returns the count. +int field_radio_options(FILE *f, XRefTable *xref, const char *dict, + char names[][128], int max); + // FDF parsing (exposed for testing). FdfData *parse_fdf_file(const char *fdf_filename); void free_fdf_data(FdfData *fdf_data); diff --git a/sbom.json b/sbom.json index 31c643f..3845338 100644 --- a/sbom.json +++ b/sbom.json @@ -5,7 +5,7 @@ "component": { "type": "application", "name": "ffpdf", - "version": "0.1.0", + "version": "0.1.1", "description": "Fast PDF form-field extractor and filler.", "licenses": [{ "license": { "id": "Apache-2.0" } }] } diff --git a/test_e2e.sh b/test_e2e.sh index b3c9d27..5667ac1 100755 --- a/test_e2e.sh +++ b/test_e2e.sh @@ -130,6 +130,16 @@ assert f["CoverageType"]["options"] == ["Auto", "Home", "Life", "Umbrella"] assert f["AdditionalCoverages"]["multi_select"] assert f["PaperlessBilling"]["on_state"] == "On" assert f["Signature"]["type"] == "signature" and f["Signature"]["value"] is None +# metadata for agents: /TU labels, required (/Ff *-marked), maxlen (/MaxLen) +assert f["FullName"]["label"] == "Full name" +assert f["State"]["label"] == "State" and f["State"]["maxlen"] == 2 +assert f["Zip"]["maxlen"] == 10 +assert f["FullName"]["required"] is True # "Full name *" +assert f["Phone"]["required"] is False # "Phone" (no *) +assert all(x["readonly"] is False for x in d["fields"]) +# document-level flags and per-field page numbers +assert d["xfa"] is False and d["dynamic_xfa"] is False +assert all(x["page"] == 1 for x in d["fields"]) PY $BIN fields docs/example-filled.pdf 2>/dev/null > "$TMP/fields_filled.json" python3 - "$TMP/fields_filled.json" <<'PY' && pass "fields: filled values incl. multi-select array" || fail "fields JSON wrong (filled)" @@ -141,8 +151,138 @@ assert f["AdditionalCoverages"]["value"] == ["Flood", "Identity theft"] assert f["PaperlessBilling"]["value"] == "On" PY # Real-world file (compressed object streams): output must stay valid JSON. -$BIN fields "$PDF" 2>/dev/null | python3 -c 'import json,sys; d=json.load(sys.stdin); assert d["count"] == 45' \ - && pass "fields: valid JSON for f8821 (45 fields)" || fail "fields JSON invalid for f8821" +$BIN fields "$PDF" 2>/dev/null | python3 -c ' +import json, sys +d = json.load(sys.stdin) +assert d["count"] == 45 +assert d["xfa"] is True and d["dynamic_xfa"] is False # static XFA +assert all(x.get("page") == 1 for x in d["fields"]) # via /Annots (no /P) +' && pass "fields: f8821 JSON incl. xfa flag + page numbers" || fail "fields JSON invalid for f8821" +# readonly flag (/Ff bit 1) on a minimal fixture. +python3 - "$TMP/ro.pdf" <<'PY' +import sys +objs = { + 1: b"<>", + 2: b"<>", + 3: b"<>", + 4: b"<>", + 5: b"<>", +} +o = bytearray(b"%PDF-1.7\n"); off = {} +for n in sorted(objs): + off[n] = len(o); o += b"%d 0 obj\n" % n + objs[n] + b"\nendobj\n" +x = len(o) +o += b"xref\n0 %d\n0000000000 65535 f \n" % (len(objs)+1) +o += b"".join(b"%010d 00000 n \n" % off[n] for n in sorted(objs)) +o += b"trailer\n<>\nstartxref\n%d\n%%%%EOF\n" % (len(objs)+1, x) +open(sys.argv[1], "wb").write(o) +PY +$BIN fields "$TMP/ro.pdf" 2>/dev/null | python3 -c ' +import json, sys +f = json.load(sys.stdin)["fields"][0] +assert f["readonly"] is True and f["required"] is False and f["value"] == "preset" +' && pass "fields: readonly flag reported" || fail "fields readonly flag wrong" +# page numbers across a 2-page form (one field per page). +python3 - "$TMP/2pg.pdf" <<'PY' +import sys +objs = { + 1: b"<>", + 2: b"<>", + 3: b"<>", + 4: b"<>", + 5: b"<>", + 6: b"<>", + 7: b"<>", +} +o = bytearray(b"%PDF-1.7\n"); off = {} +for n in sorted(objs): + off[n] = len(o); o += b"%d 0 obj\n" % n + objs[n] + b"\nendobj\n" +x = len(o) +o += b"xref\n0 %d\n0000000000 65535 f \n" % (len(objs)+1) +o += b"".join(b"%010d 00000 n \n" % off[n] for n in sorted(objs)) +o += b"trailer\n<>\nstartxref\n%d\n%%%%EOF\n" % (len(objs)+1, x) +open(sys.argv[1], "wb").write(o) +PY +$BIN fields "$TMP/2pg.pdf" 2>/dev/null | python3 -c ' +import json, sys +f = {x["name"]: x["page"] for x in json.load(sys.stdin)["fields"]} +assert f["p1"] == 1 and f["p2"] == 2, f +' && pass "fields: page number differs per page" || fail "fields page numbering wrong" + +echo "== radio group: discovery options + fill sets kid /AS ==" +python3 - "$TMP/radio.pdf" <<'PY' +import sys +opts = ["Email", "Phone", "Mail"] +objs = { + 1: b"<>", + 2: b"<>", + 3: b"<>", + 4: b"<>", + 8: b"<>", + 9: b"<>\nstream\nendstream", + 10: b"<>\nstream\nendstream", +} +for i, o in enumerate(opts): + x = 10 + i * 90 + objs[5+i] = (f"<>>>/P 4 0 R>>").encode() +o = bytearray(b"%PDF-1.7\n"); off = {} +for n in sorted(objs): + off[n] = len(o); o += b"%d 0 obj\n" % n + objs[n] + b"\nendobj\n" +x = len(o) +o += b"xref\n0 %d\n0000000000 65535 f \n" % (len(objs)+1) +o += b"".join(b"%010d 00000 n \n" % off[n] for n in sorted(objs)) +o += b"trailer\n<>\nstartxref\n%d\n%%%%EOF\n" % (len(objs)+1, x) +open(sys.argv[1], "wb").write(o) +PY +$BIN fields "$TMP/radio.pdf" 2>/dev/null | python3 -c ' +import json, sys +f = json.load(sys.stdin)["fields"][0] +assert f["type"] == "button" and f["label"] == "Preferred contact" +assert f["options"] == ["Email", "Phone", "Mail"], f +' && pass "radio: discovery lists options" || fail "radio options missing" +printf "%%FDF-1.2\n1 0 obj\n<< /FDF << /Fields [ << /T (ContactMethod) /V (Phone) >> ] >> >>\nendobj\ntrailer\n<< /Root 1 0 R >>\n%%%%EOF\n" > "$TMP/radio.fdf" +$BIN fill -o "$TMP/radio-out.pdf" "$TMP/radio.fdf" "$TMP/radio.pdf" 2>/dev/null +python3 - "$TMP/radio-out.pdf" <<'PY' && pass "radio: fill sets parent /V and matching kid /AS" || fail "radio fill /AS wrong" +import sys, re +d = open(sys.argv[1], "rb").read() +tail = d[d.rfind(b"%PDF"):] # whole file +# parent obj 8 -> /V/Phone +assert re.search(rb"8 0 obj\b.*?/V\s*/Phone", tail, re.S), "parent /V not /Phone" +# the Phone kid (obj 6) -> /AS/Phone ; an Email/Mail kid -> /AS/Off +assert re.search(rb"6 0 obj\b.*?/AS\s*/Phone", tail, re.S), "phone kid /AS not set" +assert re.search(rb"5 0 obj\b.*?/AS\s*/Off", tail, re.S), "email kid not /Off" +PY + +echo "== fill --json / --strict (machine-readable outcome) ==" +# An FDF with one good field and one typo -> partial fill. +printf '%%FDF-1.2\n1 0 obj\n<< /FDF << /Fields [ << /T (%s) /V (X) >> << /T (NoSuchField) /V (y) >> ] >> >>\nendobj\ntrailer\n<< /Root 1 0 R >>\n%%%%EOF\n' "$NAME" > "$TMP/partial.fdf" +# --json: result to stdout (PDF to -o), stderr diagnostics ignored. +$BIN fill --json -o "$TMP/pj.pdf" "$TMP/partial.fdf" "$PDF" 2>/dev/null > "$TMP/result.json" +python3 - "$TMP/result.json" <<'PY' && pass "fill --json: machine-readable result" || fail "fill --json wrong" +import json, sys +d = json.load(open(sys.argv[1])) +assert d["updated_count"] == 1 and d["not_found_count"] == 1 +assert d["not_found"] == ["NoSuchField"] +assert len(d["updated"]) == 1 +PY +# --json without -o must error (stdout carries the JSON, not the PDF). +$BIN fill --json "$TMP/partial.fdf" "$PDF" >/dev/null 2>&1 +[ "$?" -eq 1 ] && pass "fill --json without -o errors" || fail "--json/-o guard missing" +# --strict: any unmatched field -> exit 3; all matched -> exit 0. +$BIN fill --strict -o "$TMP/ps.pdf" "$TMP/partial.fdf" "$PDF" >/dev/null 2>&1 +[ "$?" -eq 3 ] && pass "fill --strict: exit 3 on unmatched field" || fail "--strict exit code wrong" +$BIN fill --strict -o "$TMP/ps.pdf" "$TMP/in.fdf" "$PDF" >/dev/null 2>&1 +[ "$?" -eq 0 ] && pass "fill --strict: exit 0 when all match" || fail "--strict false positive" +# Zero fields matched is a failure by default (exit 2), no flag needed, and no +# output is written. +printf '%%FDF-1.2\n1 0 obj\n<< /FDF << /Fields [ << /T (NoSuchField) /V (x) >> ] >> >>\nendobj\ntrailer\n<< /Root 1 0 R >>\n%%%%EOF\n' > "$TMP/none.fdf" +rm -f "$TMP/none.pdf" +$BIN fill -o "$TMP/none.pdf" "$PDF" "$TMP/none.fdf" >/dev/null 2>&1 +[ "$?" -eq 2 ] && [ ! -e "$TMP/none.pdf" ] && pass "fill: zero matches exits 2, no output" || fail "zero-match exit/output wrong" +# Flatten is exempt: it removes the form even with no new values. +$BIN fill --flatten -o "$TMP/flatz.pdf" "$PDF" "$TMP/none.fdf" >/dev/null 2>&1 +[ "$?" -eq 0 ] && [ -s "$TMP/flatz.pdf" ] && pass "fill --flatten: writes even with zero matches" || fail "flatten zero-match wrong" echo "== CLI conventions (--, -o, stdin FDF) ==" # '--' ends option parsing on every command (POSIX guideline 10). @@ -170,6 +310,71 @@ $BIN fill -o "$TMP/never.pdf" "$TMP/in.fdf" "$TMP/does-not-exist.pdf" 2>/dev/nul [ "$?" -ne 0 ] && [ ! -e "$TMP/never.pdf" ] && [ ! -e "$TMP/never.pdf.tmp" ] \ && pass "-o: failed fill leaves no partial output" || fail "-o atomicity broken" +echo "== fill: /MaxLen truncation + warning + JSON report ==" +python3 - "$TMP/ml.pdf" <<'PY' +import sys +objs = { + 1: b"<>", + 2: b"<>", + 3: b"<>", + 4: b"<>>>/NeedAppearances true>>", + 5: b"<>", + 6: b"<>", +} +o = bytearray(b"%PDF-1.7\n"); off = {} +for n in sorted(objs): + off[n] = len(o); o += b"%d 0 obj\n" % n + objs[n] + b"\nendobj\n" +x = len(o) +o += b"xref\n0 %d\n0000000000 65535 f \n" % (len(objs)+1) +o += b"".join(b"%010d 00000 n \n" % off[n] for n in sorted(objs)) +o += b"trailer\n<>\nstartxref\n%d\n%%%%EOF\n" % (len(objs)+1, x) +open(sys.argv[1], "wb").write(o) +PY +# over-length value: truncated to 5, warning on stderr, listed in --json. +printf '%%FDF-1.2\n1 0 obj\n<< /FDF << /Fields [ << /T (code) /V (ABCDEFGHIJKL) >> ] >> >>\nendobj\ntrailer\n<< /Root 1 0 R >>\n%%%%EOF\n' > "$TMP/ml.fdf" +$BIN fill --json -o "$TMP/mlo.pdf" "$TMP/ml.pdf" "$TMP/ml.fdf" 2>"$TMP/mlerr" > "$TMP/mlres.json" +grep -q "exceeds MaxLen 5; truncated" "$TMP/mlerr" && pass "MaxLen: warns on over-length value" || fail "no MaxLen warning" +python3 - "$TMP/mlres.json" "$TMP/mlo.pdf" <<'PY' && pass "MaxLen: value truncated + reported in JSON" || fail "MaxLen truncation wrong" +import json, sys, re +d = json.load(open(sys.argv[1])) +assert d["truncated"] == ["code"], d +v = re.search(rb"/V\s*\(([^)]*)\)", open(sys.argv[2], "rb").read()[::-1].replace(b")", b")")) # crude +data = open(sys.argv[2], "rb").read() +m = re.findall(rb"/V\(([A-Z]+)\)", data) +assert m and m[-1] == b"ABCDE", m # cut to 5 chars +PY + +echo "== fill: JSON values + either-order / auto-detected input ==" +# The values file may be an FDF or a JSON object, in either order relative to +# the PDF, auto-detected by content. Same JSON shape `fields` produces. +cat > "$TMP/values.json" <<'JSON' +{ "topmostSubform[0].Page1[0].Pg1Header[0].f1_1[0]": "JSON_VALUE", + "topmostSubform[0].Page1[0].c1_1[0]": true } +JSON +# New order (pdf first), JSON values, with --json result. +$BIN fill --json -o "$TMP/vj.pdf" "$PDF" "$TMP/values.json" 2>/dev/null > "$TMP/vjres.json" +python3 - "$TMP/vjres.json" <<'PY' && pass "fill: JSON values (pdf-first) + result" || fail "JSON values fill wrong" +import json, sys +d = json.load(open(sys.argv[1])) +assert d["updated_count"] == 2 and d["not_found_count"] == 0, d +PY +grep -qa 'JSON_VALUE' "$TMP/vj.pdf" && pass "fill: JSON value present in output" || fail "JSON value missing" +# Old order (values first) must still work (back-compat, keeps demos valid). +$BIN fill -o "$TMP/vo.pdf" "$TMP/values.json" "$PDF" 2>/dev/null +grep -qa 'JSON_VALUE' "$TMP/vo.pdf" && pass "fill: values-first order still accepted" || fail "old order broken" +# JSON from stdin ("-"). +echo '{"topmostSubform[0].Page1[0].Pg1Header[0].f1_1[0]": "PIPED"}' \ + | $BIN fill -o "$TMP/vp.pdf" "$PDF" - 2>/dev/null +grep -qa 'PIPED' "$TMP/vp.pdf" && pass "fill: JSON from stdin (-)" || fail "stdin JSON broken" +# Malformed JSON must be rejected (exit 1). +echo '{ bad json' | $BIN fill -o "$TMP/vb.pdf" "$PDF" - >/dev/null 2>&1 +[ "$?" -eq 1 ] && pass "fill: malformed JSON rejected" || fail "bad JSON accepted" +# Two PDFs (or two non-PDFs) -> a clear error, not a wrong guess. +$BIN fill -o "$TMP/x.pdf" "$PDF" "$PDF" >/dev/null 2>&1 +[ "$?" -ne 0 ] && pass "fill: ambiguous args rejected" || fail "ambiguous args accepted" + +echo "== XFA datasets XML escaping ==" + echo "== XFA datasets XML escaping ==" # A value with XML metacharacters (& < >) must be escaped when spliced into the # XFA datasets XML, or the datasets packet becomes malformed. Fill f8821 (a @@ -237,6 +442,38 @@ assert b[:2] == b'\xfe\xff', "missing UTF-16BE BOM" assert b[2:].decode('utf-16-be') == "café señor 日本 \U0001f600", "value mismatch" PY +echo "== emoji field value (astral-plane, UTF-16 surrogate pairs) ==" +# Emoji live outside the BMP (e.g. U+1F600), so they must be written as UTF-16BE +# surrogate pairs and decoded back exactly. Exercises both fill (encode) and +# fields (decode), via the JSON input path with raw UTF-8 emoji bytes. +python3 - "$TMP/emoji.json" <<'PY' +import sys +open(sys.argv[1], "w", encoding="utf-8").write('{"myfield": "go \U0001f600 \U0001f44d"}') +PY +$BIN fill -o "$TMP/emoji.pdf" "$TMP/indirect.pdf" "$TMP/emoji.json" 2>/dev/null +python3 - "$TMP/emoji.pdf" <<'PY' && pass "emoji: written as UTF-16BE surrogate pairs" || fail "emoji encoding wrong" +import sys, re +d = open(sys.argv[1], 'rb').read() +m = [x for x in re.finditer(rb'myfield.*?/V\s*<([0-9A-Fa-f]+)>', d, re.S)] +assert m, "no /V hex string (emoji not UTF-16BE encoded)" +b = bytes.fromhex(m[-1].group(1).decode()) +assert b[:2] == b'\xfe\xff', "missing UTF-16BE BOM" +assert b[2:].decode('utf-16-be') == "go \U0001f600 \U0001f44d", "emoji value mismatch" +u = b[2:] # a lead surrogate (0xD800-0xDBFF) must be present +assert any(0xD8 <= u[i] <= 0xDB for i in range(0, len(u) - 1, 2)), "no surrogate pair" +PY +# Read the fields output from a binary file (not sys.stdin) and decode UTF-8 +# explicitly: on Windows, sys.stdin text mode uses the console codepage, not +# UTF-8, and mangles the multibyte emoji bytes. The two checks above already +# proved the PDF holds the correct UTF-16BE bytes, so this isolates decode. +$BIN fields "$TMP/emoji.pdf" 2>/dev/null > "$TMP/emoji.fields.json" +python3 - "$TMP/emoji.fields.json" <<'PY' && pass "emoji: round-trips through fields (decode)" || fail "emoji decode wrong" +import json, sys +data = json.loads(open(sys.argv[1], "rb").read().decode("utf-8")) +v = [x["value"] for x in data["fields"] if x["name"] == "myfield"][0] +assert v == "go \U0001f600 \U0001f44d", repr(v) +PY + echo "== hybrid-reference file (/XRefStm) ==" # A classic xref table (for legacy readers) plus a cross-reference stream at # /XRefStm holding the one field, which is compressed inside object stream 5.