ci: run Snakemake DAG dry-run against a committed fixture, cache pip, enforce markers#150
Merged
Merged
Conversation
… enforce markers Makes the CI pipeline-wiring gate real and adds two low-risk hygiene improvements. No runtime code or dependency versions change. 1. Snakemake DAG dry-run now runs in CI (.github/workflows/ci.yml) The dry-run was gated behind a gitignored dataset (immunogenicity_dataset_v4.csv) that is absent on runners, so the guard's else-branch always fired and the DAG check NEVER ran - a renamed rule output or a broken include: would pass CI. This adds a committed synthetic fixture (tests/fixtures/dag_smoke/) and runs `snakemake --snakefile pipeline.smk --configfile ... --dry-run` unconditionally against it. A dry-run only needs the DAG's root inputs to EXIST (not be valid), so a 1-antigen fixture + stub CSVs/joblib resolves the full 12-job DAG (including the standardize_outputs.smk include: run_prime / run_predig / standardize_predictor_outputs). Verified locally: clean wiring -> exit 0, 12 jobs. 2. pip caching (.github/workflows/ci.yml) Adds cache: 'pip' to the setup-python steps of the lint, compat, and test jobs. 3. Marker strictness (pytest.ini) Adds --strict-markers and testpaths = tests. The heavy/slow/smoke markers are already declared and no test uses an undeclared marker, so this is non-breaking and prevents future marker typos silently no-op'ing. Verified: --collect-only under --strict-markers succeeds. Note: the "Install Snakemake" step uses an unpinned `pip install "snakemake>=8.0.0"` (pure-Python resolver, fast). Hash-pinning it for OpenSSF Pinned-Dependencies parity is a flagged optional follow-up, not required for the wiring gate. Integrity trio green (selftest 18/18; harness 20 PASS / 1 WARN / 0 FAIL / 2 SKIP).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the CI pipeline-wiring gate real and adds two low-risk hygiene improvements. No runtime code or dependency versions change.
1. Snakemake DAG dry-run now runs in CI
The dry-run was gated behind a gitignored dataset (
immunogenicity_dataset_v4.csv) that is absent on runners, so the guard's else-branch always fired and the DAG check never ran - a renamed rule output or a brokeninclude:would pass CI. This adds a committed synthetic fixture (tests/fixtures/dag_smoke/) and runssnakemake --snakefile pipeline.smk --configfile ... --dry-rununconditionally against it.A dry-run only needs the DAG's root inputs to EXIST (not be valid), so a 1-antigen fixture + stub CSVs/joblib resolves the full 12-job DAG (including the
standardize_outputs.smkinclude:run_prime/run_predig/standardize_predictor_outputs). Verified locally: clean wiring -> exit 0, 12 jobs planned.2. pip caching
Adds
cache: 'pip'to the setup-python steps of thelint,compat, andtestjobs.3. Marker strictness (pytest.ini)
Adds
--strict-markersandtestpaths = tests. Theheavy/slow/smokemarkers are already declared and no test uses an undeclared marker, so this is non-breaking and prevents future marker typos silently no-op'ing. Verified:--collect-onlyunder--strict-markerssucceeds.Note
The "Install Snakemake" step uses an unpinned
pip install "snakemake>=8.0.0"(pure-Python resolver, fast). Hash-pinning it for OpenSSF Pinned-Dependencies parity is a flagged optional follow-up, not required for the wiring gate.Scope
This is the CI-correctness half of the staged DevOps work. The pytest-xdist parallelism piece (which regenerates the hash-pinned
requirements-ci-pytest-cov.txt) is a separate follow-up so it can be validated through a green CI run on its own.Integrity trio green (selftest 18/18; harness 20 PASS / 1 WARN / 0 FAIL / 2 SKIP).