[DO NOT MERGE] Probe changed Python test selection#9
Open
mataylor-nvidia wants to merge 21 commits into
Open
Conversation
pytest-testmon 2.x only records coverage inside the pytest process, so tests that exercise code by spawning child Python processes never have those dependencies tracked and are not reselected when the child-only code changes. Add tools/testmon_subprocess_coverage.py and its testmon_coverage_context coverage plugin, which install a process_startup .pth shim, tag child coverage with the active test's node id, and merge child coverage back into Testmon before each report. Wire the hooks in via a repository-root conftest.py; they are no-ops unless Testmon is collecting. Include a regression test that verifies a subprocess-only dependency edit reselects the test.
Restart coverage in a finally block so a merge failure cannot leave it permanently stopped, guard the private _started attribute with a getattr default, migrate the deprecated hookwrapper hook to wrapper, and restore PYTHONPATH on session teardown.
Wrap the subprocess-coverage hook import in contextlib.suppress so a partial checkout without tools/ leaves the hooks as unregistered no-ops instead of aborting all test collection with ModuleNotFoundError.
Signed-off-by: Matthew Taylor <mataylor@nvidia.com>
The detect step invokes .github/actions/run-package-tests/select_testmon_mode.py, but the job never checked out the repository, so the script was missing and the step failed with exit code 2. Add an actions/checkout step so the file is present.
Close two unclosed if blocks (the testmon-mode echo and the testmon-data-dir case) that produced bash parse errors, and align the run_tests local parameter order with the call site so testmon-data-dir, testmon-mode, and test-k-expr receive the correct positional arguments.
Broaden the _RELEVANT regex so any .github/workflows/*.yml change (not just build.yaml/config.yaml) forces collect mode. Otherwise a PR that only touches a workflow such as install-ci.yml would trigger its test job while testmon deselected every test, silently running nothing.
The install CI container runs as the non-root 'isaaclab' user (uid 1000), while the testmon data directory is bind-mounted from the host. When the Testmon cache is restored, SQLite opens the database in WAL mode and must write its sibling files (.testmondata-wal / .testmondata-shm). Only the main .testmondata file was made world-writable, so a restored sibling with default 0644 permissions caused SQLite to report 'attempt to write a readonly database' during pytest_configure, aborting the run. World-write every file in the testmon data directory (not just .testmondata) and tolerate chmod failures, matching the handling used for the results directory.
The test_node_ids_file/test_node_ids_key local variables and the TEST_NODE_IDS_FILE/TEST_NODE_IDS_KEY docker env wiring were left over from the node-ID selection mechanism replaced by testmon. Their guards are always false now, so the blocks are unreachable dead code.
Use getattr fallbacks for the collect_plugin.testmon.cov chain so a future testmon rename disables subprocess-coverage merging gracefully instead of raising AttributeError and breaking the test session.
Testmon has no dependency data for Python helpers under .github/actions/ because they run outside the pytest process. The mode selector's Python-only fast path incorrectly returned 'select' for changes to those scripts, so a PR touching only such a file would deselect every test and leave CI green without verifying anything. Exclude .github/ Python files from the tracked-python fast path so they trigger a full collection, add a regression test, and include the root conftest.py and subprocess-coverage plugin in the testmon cache key.
Owner
Author
|
Closing the fork-only stack copy because Docker + Tests only runs for develop/main targets. The upstream probe PR is linked from the replacement set. |
mataylor-nvidia
changed the base branch from
mataylor/testmon-subprocess-coverage
to
develop
July 7, 2026 00:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tests PR isaac-sim#6346 affected-test selection when a Python test file changes. The PR intentionally adds one passing test so CI output shows which jobs and tests run.