Fix: take platform and devices from fixtures in notify-demo tests

[ChaoWao]

Fixes #1561

Problem

Four *_notify_demo tests were plain pytest functions that hardcoded both the platform string and the device IDs, so --platform and --device were ignored:

def test_async_notify_demo() -> None:
    assert run("a2a3", [0, 1]) == 0

They grabbed devices 0 and 1 whatever task-submit had allocated, and the a5 variants built an a5 callable on a2a3-only silicon, where the forked chip child segfaults in ChipWorker.init.

Why CI never caught it. _collect_resource_jobs (conftest.py:634) dispatches a standalone function only when it declares both @pytest.mark.device_count and @pytest.mark.runtime. Carrying neither, these four were collected but never selected — the full-CI invocation never ran them at all. They fired only on a hand-scoped --runtime/--level invocation, which enters child mode and bypasses the dispatcher. That is what makes them an efficient trap: the failure looks like a regression in whatever branch is being validated.

Change

Adopt the marker + fixture pattern the sibling demos in the same directories already use (sdma_async_completion_demo, urma_deferred_completion_demo):

@pytest.mark.platforms(["a2a3"])
@pytest.mark.runtime("tensormap_and_ringbuffer")
@pytest.mark.device_count(2)
def test_async_notify_demo(st_device_ids, st_platform) -> None:
    assert run(st_platform, [int(d) for d in st_device_ids]) == 0

This both honours the CLI options and enrols the tests in resource-phase dispatch, so they genuinely run in CI. main() and run() are untouched — python test_*.py -p ... -d ... still works.

deferred_notify_demo stays sim-only on both arches. It does pass onboard a2a3 in isolation, but it failed in the full sweep when temporarily widened, so widening it would trade one trap for another; that is logged for follow-up rather than shipped here.

New lint

tests/lint/check_test_platform_literals.py, wired into pre-commit. It AST-walks test bodies under examples/ and tests/st/ and rejects a platform name passed as a call argument. A literal compared against the fixture (if st_platform != "a2a3": pytest.skip(...)) is correct usage and is not flagged, and @pytest.mark.platforms([...]) lives in decorator_list so it is out of scope by construction. Platform names come from PLATFORM_MAP in simpler_setup/platform_info.py, parsed rather than imported so the hook runs in pre-commit's isolated env and cannot drift when an arch is added.

The four fixed here were the only violations in the tree.

Verification

Onboard a2a3 (task-submit-locked) and sim:

• The actual fix — async_notify_demo now runs on the allocated devices: [scheduler] START standalone test_async_notify_demo (dev=2) devices=[3, 5] → PASS. Previously it used 0 and 1 regardless.

• Wrong-arch trap closed — pytest examples/a2a3/.../{async,deferred}_notify_demo --platform a5 --collect-only → 2 deselected (was: collected and run).

• Sim — both deferred_notify_demo pass on --device 6-7 under a2a3sim / a5sim, i.e. no longer pinned to 0/1.

• No regression — full examples/a2a3 tests/st/a2a3 --platform a2a3 --runtime tensormap_and_ringbuffer sweep on devices 1,3,5,7:

  • this branch: 3 failed, 50 passed, 26 deselected
  • unmodified main @ 5768aac2: 3 failed, 51 passed, 25 deselected
  • identical failure set (qwen3_14b_decode, sdma_async_completion_demo, prepared_callable) — pre-existing, independent of this change. Two of the three are quarantined out of this sweep in CI (HEAVY_IGNORE / SDMA_IGNORE).

  The 51→50 / 25→26 delta is this change working: on main, deferred_notify_demo was selected during an onboard sweep and ran a a2a3sim workload there. It is now correctly deselected.

• pre-commit run — all hooks pass, including the new one; verified it fails when one file is reverted.

Note for the a5 owner

This box is a2a3-only, so the a5 markers (["a5"] for async, ["a5sim"] for deferred) preserve existing behaviour and are not onboard-validated here. Adding the markers means the a5 demos now actually run in the a5 CI job, where they previously did not — worth a look at the first a5 pipeline on this PR.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 41dce0a5-9c25-4fbf-80d0-963dfa15980a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The notify demo tests now use pytest markers and fixtures for platform and device selection. A new AST-based lint script detects hardcoded platform literals, and pre-commit runs it on relevant test files. The capability survey documents the updated routing and CI coverage.

Changes

Platform-aware test routing

Layer / File(s)	Summary
Platform literal checker tests/lint/check_test_platform_literals.py	Adds AST-based detection of known platform strings in test function call arguments, file filtering, CLI reporting, and exit-status handling.
Notify demo test routing examples/a2a3/.../test_*.py, examples/a5/.../test_*.py	Updates four notify demo tests to use platform, runtime, and device-count markers with fixture-provided execution values.
Lint enforcement and documentation .pre-commit-config.yaml, docs/capability-survey.md	Registers the lint script in pre-commit and documents marker-based routing and CI coverage.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

• Issue 1561: The changes update the four notify-demo tests and add lint protection against recurring hardcoded platform literals.

Poem

I’m a rabbit hopping through tests,
Marking platforms with tiny quests.
No more hardcoded paths to roam,
Fixtures guide each demo home.
Pre-commit guards the carrot trail,
Clean little checks ensure they sail.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: moving notify-demo tests to fixture-driven platform and device selection.
Description check	✅ Passed	The description is detailed and directly matches the changeset, including the fixture switch and new lint hook.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.pre-commit-config.yaml:
- Line 33: Update the pre-commit `files` regex to match test files directly
under `examples` and `tests/st` as well as within nested directories, while
preserving the existing test filename pattern and root scope.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 19974d66-13b6-4308-bc04-dcf7391cad77

📥 Commits

Reviewing files that changed from the base of the PR and between 8e00319 and 5c64fcc.

📒 Files selected for processing (7)

• .pre-commit-config.yaml
• docs/capability-survey.md
• examples/a2a3/tensormap_and_ringbuffer/async_notify_demo/test_async_notify_demo.py
• examples/a2a3/tensormap_and_ringbuffer/deferred_notify_demo/test_deferred_notify_demo.py
• examples/a5/tensormap_and_ringbuffer/async_notify_demo/test_async_notify_demo.py
• examples/a5/tensormap_and_ringbuffer/deferred_notify_demo/test_deferred_notify_demo.py
• tests/lint/check_test_platform_literals.py