Skip to content

Flaky e2e: ~5 chat.spec.ts tests are load-sensitive (shared-server / isolation), not a #61 regression #70

Description

@gbasin

Symptom

The chat e2e suite flakes rarely in CI. Concretely, the offline-send test turned master red once (96e4cc6) and passed on a clean re-run — i.e. a non-deterministic, load-sensitive flake, not a hard break. CI already cushions it (retries: 2 + trace: 'on-first-retry'), so it's low-frequency and non-blocking.

Investigation (evidence)

The load-sensitive tests

  • surface/e2e/tests/chat.spec.ts:235 — thread: reply in thread; root shows reply count
  • surface/e2e/tests/chat.spec.ts:299 — unread badge clears
  • surface/e2e/tests/chat.spec.ts:325 — cross-device read sync
  • surface/e2e/tests/chat.spec.ts:362 — search (⌘K) jump to message
  • surface/e2e/tests/chat.spec.ts:383 — offline send survives reload (the one that reddened master)

Root-cause hypothesis

The "fail together, pass alone" signature points at shared-server saturation and/or test isolation — 45 tests share one webServer (reuseExistingServer: false, started once) and one atrium_e2e DB reset per run. Under contention, heavy tests (offline + service worker + ws + reload + cross-context sync) exceed timeouts or see leaked state.

Recommended direction (not band-aids)

  • Root-cause from an actual CI failure trace (already captured on retry) rather than an overloaded local box.
  • Investigate per-test isolation (unique channels/users, not just unique text) + whether the shared server/DB leaks state across tests.
  • Consider sharding or serializing the heavy offline/disconnect/sync tests.
  • Avoid: more retries / blind timeout bumps (they'd mask the offline-durability behavior these tests legitimately guard).

Priority

Low / non-blocking — rare in CI, well-instrumented, no regression. Worth a deliberate stabilization pass if the flake rate becomes painful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions