Skip to content

test(e2e): automate Obsidian desktop sync flows with Playwright #115

Description

@ClaudiaFang

Context

#57 establishes real-provider/core E2E for SyncManager + production GitHub/GitLab/Gitea services. That layer should remain headless and deterministic.

This issue adds a second E2E layer for the actual Obsidian desktop experience: launch Obsidian, load the built plugin into an isolated vault, operate the plugin through its real GUI, and verify remote state against a real provider using an independent raw API verifier.

Goal

Verify that a user can complete the critical sync workflow through the real Obsidian GUI, not only through service classes.

The GUI suite must prove both sides of the boundary:

Playwright
  -> real Obsidian Desktop
  -> plugin UI / commands / settings
  -> SyncManager
  -> production provider service
  -> real Git provider
  -> independent raw API verification

A success toast alone is never sufficient evidence of a successful remote operation.

Dependency

Scope v1

Desktop harness

  • Launch an isolated Obsidian desktop instance from Playwright/Electron.
  • Create/use a disposable test vault per run.
  • Install the current plugin build (main.js, manifest.json, styles.css) into the test vault.
  • Enable the plugin automatically for the test vault.
  • Wait for Obsidian + plugin readiness using deterministic UI/state checks rather than arbitrary sleeps.
  • Capture screenshot + Playwright trace on failure.
  • Remove the disposable vault after a successful run; allow an opt-in keep-on-failure mode for debugging.

GUI smoke contract — all providers

Run the same minimal user workflow against GitHub, GitLab, and Gitea:

  • Open plugin settings.
  • Select/configure provider.
  • Configure repository/project, branch, root path, and token/credentials.
  • Execute Test Connection through the GUI and assert the visible result.
  • Create a note in Obsidian.
  • Trigger Push through the real plugin UI/command.
  • Verify the remote file through an independent raw provider API.
  • Mutate the remote file through the independent provider helper.
  • Trigger Pull through the real plugin UI/command.
  • Verify the local Vault/editor content changed to the remote content.

Canonical GUI provider — GitHub

Use GitHub as the exhaustive GUI provider after the cross-provider smoke contract is stable:

  • push new file;
  • push unchanged file and prove no extra remote commit;
  • pull remote update;
  • rename/move a note and verify old path removal + new path creation;
  • delete a note and verify remote deletion;
  • batch push path;
  • conflict protection flow;
  • plugin reload preserves settings and sync metadata;
  • relevant command palette / context menu entry remains functional;
  • error states surface actionable UI feedback.

Do not multiply every exhaustive GUI case across all three providers unless a case is genuinely provider-specific.

Provider matrix

Layer GitHub GitLab Gitea
Provider/core contract (#57) Full Full Full
SyncManager real-provider (#57) Full Full Full
GUI smoke Full Basic Basic
GUI exhaustive Canonical No No

Provider-specific API semantics continue to belong in #57. GUI tests are responsible for user-visible wiring and Obsidian integration.

High-value regressions this layer should catch

  • settings page fails to render on a supported Obsidian version;
  • provider selector/settings are not propagated into the production service;
  • command/ribbon/context-menu registration breaks;
  • modal or confirmation wiring blocks an operation;
  • Vault create/rename/delete event ordering causes an incorrect sync action;
  • plugin reload loses persisted configuration or sync metadata;
  • UI reports success although the remote repository was not changed;
  • remote update succeeds but the Obsidian editor/Vault is not refreshed;
  • Obsidian API changes break plugin initialization while unit/core E2E remain green.

Test architecture

Suggested structure:

e2e-gui/
├── playwright.config.ts
├── fixtures/
│   ├── obsidian-app.ts
│   ├── test-vault.ts
│   └── plugin-build.ts
├── pages/
│   ├── settings-page.ts
│   └── sync-ui.ts
├── suites/
│   ├── provider-smoke.spec.ts
│   └── github-full.spec.ts
└── helpers/
    └── diagnostics.ts

Prefer semantic selectors (getByRole, accessible name, stable data-* hooks where needed) over brittle CSS/layout selectors.

Credentials and sandboxing

CI strategy

Phase-in after local/manual stability:

  1. local GUI E2E against GitHub;
  2. manual workflow_dispatch for all providers;
  3. GitHub GUI smoke on internal PRs touching UI/sync/provider integration;
  4. GitLab/Gitea GUI smoke on main/manual/scheduled runs as stability permits.

Do not make the entire three-provider GUI matrix a required PR check initially. Keep #57's headless/core suite as the deterministic release gate; promote GUI checks only after flake rate is acceptable.

Acceptance criteria

  • A clean machine can launch an isolated Obsidian test vault and load the current plugin build.
  • GitHub/GitLab/Gitea GUI smoke can each complete settings -> connection -> push -> remote verify -> remote mutate -> pull -> local verify.
  • GitHub exhaustive GUI cases cover rename, delete, conflict, reload, and batch paths.
  • Remote assertions use the independent verifier from outside the plugin/service under test.
  • A visible success message cannot make a test pass if remote state is wrong.
  • Failures preserve useful screenshot/trace diagnostics without leaking credentials.
  • Existing headless/unit/provider E2E remain independent and runnable without Obsidian desktop.

Non-goals

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions