Skip to content

CI: cherry-pick auto-derive addon requires_mod step from PR 820#14

Merged
eduralph merged 1 commit into
maintenance/gramps60from
fix/ci-auto-derive-requires-mod
May 8, 2026
Merged

CI: cherry-pick auto-derive addon requires_mod step from PR 820#14
eduralph merged 1 commit into
maintenance/gramps60from
fix/ci-auto-derive-requires-mod

Conversation

@eduralph

@eduralph eduralph commented May 8, 2026

Copy link
Copy Markdown
Owner

Summary

Cherry-picks c6aa10e0 from feature/ci-cd-pipeline-upstream (= upstream PR gramps-project#820) onto fork's maintenance/gramps60. Adds an "Install addon runtime deps (derived from requires_mod)" pre-test step to all three job that exercises addon code: unit-test-linux, unit-test-windows, integration-test.

This is the third of upstream PR gramps-project#820's five CI commits — the other two (28febdcd shell:bash, 715e71dd OS-split) landed via PR #12. This one wasn't picked at the time because integration-test wasn't running successfully end-to-end yet, so the missing addon deps weren't surfaced.

Now that PR #13 brings in upstream's WebSearch fix (gramps-project#833) and the integration-test step actually exercises TMGimporter's tests/test_integration.py to completion, the missing dbf package surfaces:

For TMG Importer to work please install 'dbf >= 0.99.0' (No module named 'dbf')
setUpClass (TMGimporter.tests.test_integration.TestFullPipelineRelph) ... ERROR
File ".../TMGimporter/libtmg.py", line 521, in map_dbfs_to_tables
    tmgPeople = Table(
NameError: name 'Table' is not defined

dbf is declared in TMGimporter.gpr.py's requires_mod list. Without the auto-derive step, it's never installed.

What the step does

For every */*.gpr.py in the checkout, parses out requires_mod = [...] literals via ast.literal_eval, takes the union, and pip-installs each package one at a time (per-package install so a single failing build — pygraphviz without graphviz-dev, etc. — doesn't abort the rest).

Mirrors Gramps' Addon Manager install path (gramps/gui/plug/_windows.py __on_install_clicked → req.install → gen/utils/requirements.py), keeping .gpr.py files as the single source of truth for addon deps.

Why fork-only

The same fix is already part of upstream PR gramps-project#820 as commit c6aa10e0. Once gramps-project#820 lands upstream, a future fork-resync makes this cherry-pick redundant. No upstream PR needed for this.

Verification

  • YAML parses (yaml.safe_load → 7 jobs).
  • The "Install addon runtime deps" step now appears in three jobs, each placed BEFORE that job's test-running step:
    L103: unit-test-linux  — install step before  L141: Run per-addon unit tests
    L207: unit-test-windows — install step before  L237: Run per-addon unit tests
    L277: integration-test  — install step before  L309: Run plugin registration tests
    
  • Replaying the auto-derive snippet against the local addons-source tree resolves 9 addon deps including dbf:
    boto3, dbf, life_line_chart, litellm, networkx, psycopg,
    psycopg2, pygraphviz, svgwrite
    
  • After this lands and PR Sync fork's maintenance/gramps60 with upstream (2026-05-08) #13 merges, the next CI run on maintenance/gramps60 should clear the integration-test step's NameError (dbf will be on the path) and let TestFullPipelineRelph actually run.

Stacking note

This PR is independent of #13 (the upstream-sync PR). They can land in either order; if #13 lands first, integration-test starts working as soon as this lands too. No coordination required.

🤖 Generated with Claude Code

The Dockerfile bakes in only `dbf`, but addons declare a wider set of
Python deps in their .gpr.py `requires_mod` lists (networkx, psycopg2,
pygraphviz, lxml, svgwrite, boto3, litellm, life_line_chart, psycopg).
Without these installed, per-addon unit tests and the plugin-
registration subprocess load fail with ImportError/NameError.

Add a pre-test step to unit-test-linux, unit-test-windows, and
integration-test that globs every *.gpr.py, extracts the requires_mod
union via ast.literal_eval, and pip-installs each package one at a
time. Per-package install (not batched) keeps a single build failure
(pygraphviz without graphviz-dev, psycopg2 without libpq-dev) from
aborting the rest — the affected addon's tests will skip or fail in
isolation without blocking others.

Mirrors Gramps' Addon Manager install path
(gramps/gui/plug/_windows.py __on_install_clicked → req.install →
gen/utils/requirements.py), keeping .gpr.py files as the single source
of truth for addon deps. New addon deps do not need a parallel update
to the Dockerfile or this workflow.
@eduralph eduralph merged commit 02cf1ef into maintenance/gramps60 May 8, 2026
2 of 7 checks passed
@eduralph eduralph deleted the fix/ci-auto-derive-requires-mod branch May 8, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant