Skip to content

feat: support template page ranges - #750

Open
pjkersten wants to merge 2 commits into
invoice-x:masterfrom
pjkersten:feat/template-page-ranges
Open

feat: support template page ranges#750
pjkersten wants to merge 2 commits into
invoice-x:masterfrom
pjkersten:feat/template-page-ranges

Conversation

@pjkersten

Copy link
Copy Markdown
Contributor

Summary

Adds an optional top-level template setting:

pages: "2-3"

It restricts template matching and extraction to an inclusive page range.
pdftotext and PDFium explicitly support it; unsupported readers are skipped
for such a template.

## Tests

19 targeted tests passed, including page-range syntax, scoped matching, text
caching and area extraction.

@pjkersten
pjkersten marked this pull request as ready for review August 21, 2026 15:20
@bosd

bosd commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Nice PR — the capability-flag pattern mirrors SUPPORTS_AREA cleanly, parse_pages is neatly isolated, cache-key extension is correct, and the pdftotext/pdfium page-range boundary math (-f/-l and range(first - 1, min(last, len(document)))) both look right.

Must-fix (blocking CI reds)

pre-commit — pydoclint on src/invoice2data/input/__init__.py:113 (extract_text):

  • DOC105pages param is typed Any in the signature but the docstring says int | str | None. Either narrow the signature or widen the docstring so they match.
  • DOC501extract_text now raises ValueError (unsupported backend + parse_pages errors) but the docstring has no Raises: section. Add it.

pre-commit — ruff-format wants one file reformatted; the long error message strings need wrapping.

mypy 3.10 + 3.13 — three test functions in tests/test_page_ranges.py need type annotations:

  • _page_reader's inner to_text(_path, _area=None, pages=None) — needs typed params (e.g. _area: dict[str, Any] | None = None, pages: tuple[int, int] | None = None)
  • test_page_syntax_is_accepted(value, expected) and test_invalid_page_syntax_is_rejected(value) — parametrize params need annotations (value: Any, expected: tuple[int, int] / value: Any)

Should-fix — real correctness concern

Field-level area: bypasses template-level pages:. template.extract() uses the scoped text I hand it — good — but a field's area: calls extract_text(input_module, invoice_file, area=area_dict) inside _handle_area without a pages= kwarg, so the area is cropped from the whole PDF instead of just the scoped range. If a template sets both pages: \"2-3\" and a field with area: {f: 1, ...}, the field silently extracts from page 1 — contradictory. Two calls:

  • (a) Document the limitation in the tutorial ("pages: scopes template-level extraction but not per-field area: — set f/l on the area itself for that").
  • (b) Thread pages into _handle_area so field-level areas honour it too.

(a) is fine as a first cut; (b) is the right long-term shape. Either is acceptable for this PR.

Test-coverage gaps

  • pages: + field-level area: together — either a regression test locking design (b) above, or a doc-only test locking design (a).
  • Cascade behaviour: a page-scoped template is silently skipped when the current backend doesn't support pages (asserts the except (OSError, ValueError): continue path).
  • parse_pages(\"2-\") (trailing-empty range) — currently bubbles up via int(\"\"); worth a test locking that it raises.

Nits

  • Log noise on unsupported-backend skip. _match_template_for_reader's logger.warning(\"Template %s cannot use pages ...\") fires once per page-scoped template per file per backend. For a cascade of 3 backends × 10 page-scoped templates = 30 warnings per file processed. Prefer logger.debug here, or warn-once per (template, backend) pair.
  • Single-page example. The cookbook uses pages: \"2-3\"parse_pages also accepts pages: 2 (single int) and pages: \"2\". Worth showing that too since "only page 2" is probably the most common case.
  • Docs should call out the keyword-scoping effect explicitly in the tutorial ("pages: also gates keyword matching — if a template's keywords: refer to text on a cover page you're excluding, the template won't match"). It's implied by the PR body but a first-time reader will trip on it.

Not concerns (looked at, all good)

  • _cached_to_text cache key includes pages → memoization sound; identical file+pages call twice = 1 backend hit.
  • Backward compatibility: to_text(path, area_details) without pages still works on both bundled backends and any third-party reader; _cached_to_text keeps the positional-area call shape when pages is None.
  • The capability check + template-skip fallback is the right shape (matches how SUPPORTS_AREA behaves).

@pjkersten

Copy link
Copy Markdown
Contributor Author

Thanks, addressed in a96765c.

  • Fixed pydoclint, formatting and test annotations.
  • Implemented option (b): field-level areas are intersected with the template-wide page range.
  • Added regression coverage for area+pages, unsupported cascade readers and pages: "2-".
  • Changed unsupported-reader logging to debug and expanded the documentation.

Side effect: some multi page invioces with crucial info on page 1 but the real invoice on later pages, can no longer parse outside their page boundary. Most notably: some MS invoices, at least here in NL.

@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/invoice2data
  api.py 282
  src/invoice2data/extract
  invoice_template.py
  src/invoice2data/input
  __init__.py 103, 128
  pdfium.py
  pdftotext.py 193
  tests
  test_page_ranges.py
Project Total  

This report was generated by python-coverage-comment-action

@bosd
bosd force-pushed the feat/template-page-ranges branch from a96765c to f2c1e8e Compare August 21, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants