Skip to content

test: rebuild e2e suite on semantic selectors with real assertions - #66

Merged
nick-transition merged 1 commit into
mainfrom
e2e-hardening
Jul 3, 2026
Merged

test: rebuild e2e suite on semantic selectors with real assertions#66
nick-transition merged 1 commit into
mainfrom
e2e-hardening

Conversation

@nick-transition

Copy link
Copy Markdown
Owner

Summary

The Playwright e2e suite navigated by hardcoded screen coordinates and its only assertion was body.innerHTML.length > 100, so it could not fail on wrong content — and its coordinates were already stale (they clicked a home "Workouts" tab that no longer exists). This PR rebuilds it on Flutter web's semantics tree with real assertions. The suite now runs in ~9s locally (was minutes of fixed sleeps).

Changes

  • Semantic selectors: lib/main.dart force-enables the semantics tree behind --dart-define=ENABLE_SEMANTICS=true; specs target widgets by accessible role/name via shared helpers in e2e/helpers.ts instead of clickAt(x, y) + waitForTimeout
  • Real read-only assertions: coach views now assert absence of Edit workout / Start Session / Delete Session and presence of the Coach View chip
  • Hermetic runs: e2e/global-setup.ts clears the auth+firestore emulators and reseeds before every run
  • Config hardening: CI retries, trace on first retry, video retained on failure; README screenshot generation split into a separate Playwright project (npm run e2e:screenshots) so CI runs only the regression suite
  • CI: wait for the Firestore emulator too, cache emulator jars, upload Playwright report + traces
  • DX: run_e2e.sh kills emulators it started, gains --skip-build; new npm run e2e / e2e:quick / e2e:screenshots; unused integration_test dependency dropped; local run artifacts gitignored/untracked

Bugs found by the new assertions

  1. SessionCard had no tap semantics — assistive tech could not open session details at all. Fixed by exposing the card as a semantic button (lib/widgets/session_card.dart).
  2. Seeder wrote session entries as an array field while the app reads the entries subcollection, so seeded session details always showed "No exercises recorded". Fixed in scripts/seed_emulator.js.

README screenshots regenerated to match the current 3-tab UI.

Test plan

  • npx playwright test --project=e2e — 4/4 pass locally against emulators
  • npx playwright test --project=screenshots — passes, screenshots verified non-blank
  • flutter analyze clean for changed files; flutter test failures are pre-existing on main (tracked separately)

🤖 Generated with Claude Code

The Playwright suite navigated by hardcoded screen coordinates and its only
assertion was body.innerHTML.length > 100, so it could not fail on wrong
content — and its coordinates were already stale (the home Workouts tab no
longer exists).

- Enable Flutter web semantics behind --dart-define=ENABLE_SEMANTICS so
  Playwright targets widgets by accessible role/name instead of coordinates
- Replace fixed waitForTimeout sleeps with auto-waiting selectors: the suite
  now runs in ~9s instead of minutes
- Assert actual read-only behavior in coach views (no Edit workout / Start
  Session / Delete Session; Coach View chip present)
- Add e2e/global-setup.ts: clears auth+firestore emulators and reseeds every
  run, so tests never depend on leftover state
- Fix SessionCard missing tap semantics (assistive tech could not open
  session details at all) by exposing the card as a button
- Fix seed_emulator.js writing session entries as an array field; the app
  reads the entries subcollection, so session detail always showed
  "No exercises recorded"
- Split README screenshot generation into its own Playwright project so CI
  only runs the regression suite; regenerate screenshots for the current UI
- CI: also wait for the Firestore emulator, cache emulator jars, upload the
  Playwright report and traces; retries + trace on-first-retry in config
- run_e2e.sh: kill emulators it started on exit, --skip-build flag; add npm
  run e2e / e2e:quick / e2e:screenshots
- Drop unused integration_test dependency; gitignore local run artifacts and
  untrack stray logs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@nick-transition
nick-transition merged commit 3dafd4d into main Jul 3, 2026
2 checks passed
@nick-transition
nick-transition deleted the e2e-hardening branch July 3, 2026 01:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant