[CI-test] Split OS-specific addon tests via filename convention#10
Closed
eduralph wants to merge 4 commits into
Closed
[CI-test] Split OS-specific addon tests via filename convention#10eduralph wants to merge 4 commits into
eduralph wants to merge 4 commits into
Conversation
The Dockerfile bakes in only `dbf`, but addons declare a wider set of Python deps in their .gpr.py `requires_mod` lists (networkx, psycopg2, pygraphviz, lxml, svgwrite, boto3, litellm, life_line_chart, psycopg). Without these installed, per-addon unit tests and the plugin- registration subprocess load fail with ImportError/NameError. Add a pre-test step to unit-test-linux, unit-test-windows, and integration-test that globs every *.gpr.py, extracts the requires_mod union via ast.literal_eval, and pip-installs each package one at a time. Per-package install (not batched) keeps a single build failure (pygraphviz without graphviz-dev, psycopg2 without libpq-dev) from aborting the rest — the affected addon's tests will skip or fail in isolation without blocking others. Mirrors Gramps' Addon Manager install path (gramps/gui/plug/_windows.py __on_install_clicked → req.install → gen/utils/requirements.py), keeping .gpr.py files as the single source of truth for addon deps. New addon deps do not need a parallel update to the Dockerfile or this workflow.
With ci.yml's auto-derive step in place (previous commit), dbf is installed at CI runtime from TMGimporter's .gpr.py requires_mod list. Keeping it baked into the Dockerfile and environment.yml in parallel would defeat the "single source of truth = .gpr.py" goal and drift the moment a new addon declares an additional dep. Remove dbf from both; leave the stable base (PyGObject, pycairo, Gramps, orjson, ruff) since those are not addon deps. Add a comment pointing readers at the auto-derive step so future edits do not re-bake runtime deps back in.
Root cause of "Unit Tests (Linux)" and "Integration Tests (Gramps)"
failures was not broken test modules — the steps never invoked
unittest. The container's default shell is /bin/sh (dash on
python:3.12-slim), and the inline scripts use bash-only parameter
expansions (${f%.py}, ${mod//\//.}) to build the dotted module list.
Dash fails with "Bad substitution" on the first such line; the rest
of the script never runs. continue-on-error: true masked this as a
generic job failure for two CI rounds.
Add "shell: bash" explicitly to:
- unit-test-linux / Run per-addon unit tests (bashisms)
- integration-test / Run per-addon integration tests (bashisms)
- integration-test / Run plugin registration tests (no bashisms today,
but consistent and future-proof)
Compile Check already sets shell: bash. Windows jobs inherit bash
via defaults.run at the job level. No other steps affected.
The Windows unit-test job hung on TMGimporter's DB-backed tests because
make_database("sqlite").load(":memory:", None) deadlocks under the
conda-forge GTK + pip Gramps combination. Rather than patch the hang,
introduce a filename convention so per-addon authors can declare OS
scope up front:
test_*.py general (every OS)
test_linux_*.py Linux-only
test_windows_*.py Windows-only
test_integration_*.py Linux-only, full-pipeline/DB-backed (pre-existing)
unit-test-linux skips test_windows_* and test_integration_*;
unit-test-windows skips test_linux_* and test_integration_*.
Applied to TMGimporter: the 13 DB-backed classes in tests/test_libtmg.py
move to tests/test_linux_libtmg.py (along with the _Rec/_table/_make_db/
_add_person/_MockUser helpers they use). The 7 pure-logic classes
(TestStripTmgCodes, TestTmgDateToGrampsDate, TestNumTo{Month,Date},
TestParseDate, TestRepoTypeFromName, TestUrlFromName) stay in
test_libtmg.py and will run on every OS.
Locally all 175 tests still pass via run-addon-unit.sh TMGimporter.
Owner
Author
|
CI signal captured: Windows completed in 3m43s instead of hanging. Filename convention merged upstream into PR gramps-project#820 → see commit 715e71d. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fork-internal PR to trigger CI on the filename-convention split.
Summary
Validation done locally
Test plan