Document safe CLI exit tests

[cvolkcvolk]

Summary

Document safe CLI exit testing for future agents

Detailed description

After Isaac Sim starts, calling sys.exit both raises SystemExit and schedules the shared simulation application to stop. Catching the exception does not cancel the scheduled shutdown.
This allowed a CLI test to pass while later, unrelated simulation tests failed, making earlier CI failures (#929 and #933) painfully hard to debug.

[arena-review-bot]

🤖 Isaac Lab-Arena Review Bot

Summary

Docs-only change adding two bullets to AGENTS.md on safely testing CLI paths that may call sys.exit once Isaac Sim has started. The guidance is accurate, matches the existing Don't/Instead style, correctly references TestConstants.python_path (defined in isaaclab_arena/tests/utils/constants.py), and correctly clarifies that with_subprocess selects a test phase rather than spawning a child process. This is a useful correction that would have made the #929/#933 CI failures easier to diagnose.

Findings

None. No code paths change; the added guidance is verified against the repo.

Test Coverage

Not applicable — documentation only.

Verdict

Ship it

[greptile-apps]

Greptile Summary

This PR documents the correct pattern for testing CLI exit behavior in the Isaac Sim test suite, following CI failures in #929 and #933 where a misplaced sys.exit test passed while silently queuing a Kit shutdown that broke later simulation tests.

• Adds a Don't rule explaining that calling any CLI path that can invoke sys.exit (argparse --help, invalid args, parser.error()) directly inside pytest is unsafe once Isaac Sim is running, because catching SystemExit does not cancel the already-queued Kit shutdown.
• Adds an Instead rule directing contributors to isolate these checks in a child process via subprocess.run([TestConstants.python_path, ...]), and clarifies that with_subprocess is a marker for tests where the child starts Isaac Sim — it does not itself spawn a subprocess.

Confidence Score: 5/5

Documentation-only change with no executable code; safe to merge as-is.

The change is two new bullet points in AGENTS.md. The guidance accurately describes the Isaac Sim / Kit shutdown behavior and the subprocess isolation remedy. No code is modified, no logic paths are affected, and the new text is consistent in style and placement with the rest of the file.

No files require special attention.

Important Files Changed

Filename	Overview
AGENTS.md	Adds two guidance bullets clarifying that CLI exit tests (sys.exit, argparse --help) must run in a child process via subprocess.run, not directly in pytest, and that with_subprocess does not create a child process.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant PT as pytest process
    participant SIM as Isaac Sim / Kit

    Note over PT,SIM: ❌ Don't — sys.exit inside pytest after Sim starts
    PT->>SIM: SimulationApp starts
    PT->>PT: call CLI path → sys.exit / parser.error()
    PT->>PT: catch SystemExit (test "passes")
    SIM-->>PT: Kit shutdown still queued
    PT->>SIM: later simulation tests fail unexpectedly

    Note over PT,SIM: ✅ Instead — subprocess isolation
    PT->>PT: subprocess.run([python_path, ...])
    PT->>PT: child process calls sys.exit
    PT->>PT: assert returncode / stdout
    PT->>SIM: parent pytest + Sim continue cleanly

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant PT as pytest process
    participant SIM as Isaac Sim / Kit

    Note over PT,SIM: ❌ Don't — sys.exit inside pytest after Sim starts
    PT->>SIM: SimulationApp starts
    PT->>PT: call CLI path → sys.exit / parser.error()
    PT->>PT: catch SystemExit (test "passes")
    SIM-->>PT: Kit shutdown still queued
    PT->>SIM: later simulation tests fail unexpectedly

    Note over PT,SIM: ✅ Instead — subprocess isolation
    PT->>PT: subprocess.run([python_path, ...])
    PT->>PT: child process calls sys.exit
    PT->>PT: assert returncode / stdout
    PT->>SIM: parent pytest + Sim continue cleanly

Loading

_{Reviews (1): Last reviewed commit: "Document safe CLI exit tests" | Re-trigger Greptile}