Skip to content

RecordingCoordinatorTests.testOutputRecordingFailsWhenAccessibilityDenied flaky on self-hosted CI runner #922

Description

@malpern

testOutputRecordingFailsWhenAccessibilityDenied fails on the self-hosted CI runner's full lane while passing deterministically on dev machines. It is red on master independent of any PR (observed on the #908 master commit's full-coverage lane), so it is a pre-existing environmental flake, not a regression.

Why it's not a product bug

The test fixture is fully hermetic — it injects a StubPermissionProvider (returning a mocked .denied snapshot) and a stub capture via coordinator.configure(...). It does not read the real PermissionOracle or the runner's real Accessibility state. The product behavior it checks (output recording refused when accessibility denied) is correct; the failure is in the harness — most likely a startup-task / expectation-fulfillment timing race around the real RuntimeCoordinator() the fixture constructs, which behaves differently on the headless runner.

Interim mitigation

Gated with CI_ENVIRONMENT skip (same pattern as #896 for RemapEndToEndTests) so the full lane is honest and not perpetually red. The test still runs — and passes — locally, preserving real coverage of the permission-denied path.

Proper fix (post-1.0)

Make the fixture deterministic on the runner: drain/await the RuntimeCoordinator startup tasks explicitly rather than best-effort Task.yield() loops, or inject a stub runtime coordinator so no real async startup runs during the test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    tech-debttestingTest coverage and test infrastructure

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions