Skip to content

fix(playwright): infer snapshot platform for Accept and Undo - #1

Draft
dimazollo wants to merge 2 commits into
masterfrom
fix/playwright-snapshot-platform
Draft

fix(playwright): infer snapshot platform for Accept and Undo#1
dimazollo wants to merge 2 commits into
masterfrom
fix/playwright-snapshot-platform

Conversation

@dimazollo

@dimazollo dimazollo commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Problem

Opening a Linux Playwright screenshot report on macOS made GUI Accept resolve
the baseline using the GUI host's platform. It can create a new *-darwin.png
instead of updating the existing *-linux.png. Blob-merged reports do not
necessarily retain the original baseline filename or source platform.

Playwright 1.59 also uses a new screenshot error header. The adapter classified
it as a generic error, leaving an accepted attempt with an error status but no
error message (displayed as Unknown error).

Changes

  • Automatically persist the original platform per test result. Direct runs use
    their test process; blob merges read each source ZIP's metadata header using
    Playwright's V2 per-report merge hook. Queued results retain their own source.
  • Use recorded platform metadata for both {platform} and the default
    {snapshotSuffix} in GUI Accept and Undo. Preserve it in accepted results.
  • Refuse to guess the GUI host OS when metadata is missing or unknown. Validate
    every selected platform before starting a batch of writes.
  • Keep --snapshot-platform linux|darwin|win32 only as a legacy fallback. Reject
    it if it conflicts with a recorded platform; it is not required in normal use.
  • Recognize modern screenshot mismatch headers only with matching expected,
    actual and diff attachments. Preserve unrelated functional failures.
  • Cover CLI validation, platform resolution, Accept/Undo forwarding, modern
    mismatch classification and mixed failures with regression tests.

Verification

  • npm test on Node 20.19.6: 690 server tests and 847 frontend tests passed;
    full lint and test typechecks passed.
  • npm run build passed (existing webpack warnings).
  • Real application screenshot run in Docker Linux/amd64 with Playwright 1.59.1,
    producing a blob report. Merged on macOS using both reporters.
  • Re-merged the original Linux blob with the fixed reporter on macOS: the HTML
    report database retained html-reporter:platform=linux.
  • Moved the source blob directory out of the way and started GUI without any
    platform flag. Clicked Accept: existing Linux baseline changed to the exact
    actual image bytes; no Darwin baseline was created. The image exposed Undo.
  • Clicked Undo: original SHA-256 restored byte-for-byte, visual diff restored.
  • Verified a native macOS run separately: its result records darwin.
  • Regression tests also cover mixed source platforms with asynchronous writes,
    unreadable or malformed metadata, unknown platforms, conflicting fallback and
    batch preflight before any writes.
  • External automated review was not run because transmission permission was
    unavailable; local checks and UI verification above were completed.

Usage

npx html-reporter gui --tool playwright

Run from the source checkout containing the matching Playwright config and
downloaded report database/images. A static file server cannot write baselines.

Automatic blob detection is verified with Playwright 1.59.1 and uses internal V2
merge hooks and onBlobReportMetadata.userAgent. Unsupported formats, unknown
Linux distribution IDs, and older merge versions without the hook fail closed.
This does not recover arbitrary custom testInfo.snapshotSuffix values or migrate
metadata/errors already stored by old reporters. Re-merge their original blobs
with the fixed reporter; use the explicit fallback only when the source platform
has been independently verified and cannot be recovered. This is a draft in the
personal fork, not an upstream submission.

Downloaded blob reports may not retain the original baseline path. Allow GUI Accept and Undo to resolve platform tokens explicitly without changing the default local behavior.

Recognize modern screenshot matcher errors using their diff attachments so accepted images do not leave an error result with no message. Keep mixed functional failures non-acceptable as a whole.
Persist the test-process platform or read it from each source blob during merge. Retain provenance across queued results and synthetic Accept/Undo attempts so opening a Linux report on macOS needs no platform flag.

Use the flag only as a fallback for legacy reports. Refuse unknown or conflicting platforms before any batch writes instead of silently selecting the GUI host OS.
@dimazollo dimazollo changed the title fix(playwright): accept CI snapshots on another platform fix(playwright): infer snapshot platform for Accept and Undo Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant