test(ci): parallelize the test suite with pytest-xdist [WIP]

[Elarwei001]

⚠️ WIP / draft — opened to measure the real CI speedup on the runners, not for merge yet.

Motivation

CI runs ~29 min per Python version (py3.12/3.13/3.14), dominated by network-bound live tests. From a local --durations run, test_virus alone is ~64% of the runtime: each test_virus_with_*_filter downloads a broad NCBI taxon ("Zika virus", thousands of genomes) server-side-filtered, so every test re-fetches a large package (75–195 s each). These tests are I/O-bound (waiting on NCBI/Ensembl), so running them concurrently should cut wall-clock a lot without changing what is tested.

Change

• Add pytest-xdist to the test dependency group; run hatch-test with -n auto.
• Isolate the test_virus output directory per test (tempfile.mkdtemp) instead of a single shared "test_virus_output" dir — the shared dir (rmtree + recreate in setUp) would race across parallel workers. This is the only fixed shared output dir in the suite.

All tests remain live — no mocking, no semantic change; only the output-dir isolation and the parallel runner.

Local check

4 virus filter tests run concurrently (-n 4) pass with no races; ~70–90 s serial → 31 s parallel.

Open questions (why WIP)

• Real speedup on GH runners (this PR's CI run is the measurement).
• Whether unauthenticated parallel requests trip NCBI/Ensembl rate limits — may need to lower -n or add an NCBI_API_KEY secret (the virus module already supports api_key).

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.83%. Comparing base (e43a804) to head (b6b2cbe).
⚠️ Report is 2 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #262      +/-   ##
==========================================
+ Coverage   56.70%   56.83%   +0.12%     
==========================================
  Files          29       29              
  Lines        9392     9392              
==========================================
+ Hits         5326     5338      +12     
+ Misses       4066     4054      -12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Elarwei001]

Closing this experiment. The parallel run gave only a modest speedup (py3.12 ~28→23 min, py3.13 ~26→24 min) and py3.14 failed — most of the wall-clock is a handful of network-bound live tests (test_virus dominates by re-downloading a broad NCBI taxon for every filter test), and unauthenticated parallel requests appear to hit upstream rate limits rather than scale. The effective fix is to shrink those downloads (smaller taxon) instead of parallelizing; I'll pursue that separately.