Investigate: replace @playwright/mcp with playwright-cli to eliminate MCP protocol overhead

[gurvinder-dhillon]

Background

OpenQA currently uses @playwright/mcp via createConnection() to expose Playwright browser automation as MCP tools. Claude Code connects to these tools through a TCP bridge we set up per step. This adds several layers:

• MCP protocol (JSON-RPC over stdio/TCP)
• In-process MCP server initialisation
• TCP bridge process to proxy Claude Code's stdio to our TCP server
• Per-step setup and teardown of all the above (see issue Investigate: reuse a single @playwright/mcp connection per browser context instead of one per step #12)

playwright-cli exposes the same set of browser automation capabilities (navigate, click, type, snapshot, verify, etc.) as a direct CLI interface — no MCP protocol layer involved.

Key note from the author: Both @playwright/mcp and playwright-cli live in the Playwright source code. The GitHub repos at https://github.com/microsoft/playwright-mcp and https://github.com/microsoft/playwright-cli are used for versioning, release management, and documentation only — the actual implementation is in the main Playwright repository.

What to investigate

1. Tool parity

Does playwright-cli expose the same set of actions and assertions that Claude Code currently uses via MCP?

The tools currently used include:

• browser_navigate, browser_snapshot, browser_click, browser_type
• browser_verify_text_visible, browser_verify_element_visible, browser_verify_list_visible, browser_verify_value
• browser_evaluate, browser_run_code_unsafe

Confirm whether equivalent commands exist in playwright-cli and whether their input/output format is compatible with how Claude Code would invoke them.

2. How Claude Code would interact with playwright-cli

With @playwright/mcp, Claude Code gets structured MCP tool calls with typed parameters and structured responses. With playwright-cli, Claude Code would instead use its built-in Bash tool to run CLI commands — unstructured shell invocations.

Consider:

• Would the system prompt need to be redesigned to instruct Claude to use Bash commands instead of MCP tool calls?
• Is a CLI-invocation model as reliable for an LLM as structured MCP tool calls with explicit parameter schemas?
• How does error reporting compare — MCP returns structured error objects; CLI returns exit codes and stderr.

3. Browser context sharing — the critical constraint

Our current approach passes the live in-process Playwright browser context directly to createConnection(). This means Claude Code's tools operate on the exact same page object the test fixture holds — no new browser is launched.

With playwright-cli as an external process, it cannot receive a Node.js object. It would need to connect to the running browser via a CDP endpoint or similar mechanism. Investigate:

• Can Playwright expose a running BrowserContext as a CDP endpoint that playwright-cli can connect to?
• Does this introduce any state isolation issues (e.g. separate page objects, event handling)?
• Is there a performance cost to the CDP roundtrip vs in-process calls?

Check the --cdp-endpoint and --remote-endpoint options in @playwright/mcp's config for prior art on how the existing tool handles connecting to an already-running browser.

4. Snapshot and output format

@playwright/mcp returns accessibility tree snapshots as YAML with [ref=eXX] element refs that Claude uses as action targets. Understand what playwright-cli returns for equivalent snapshot/inspect commands and whether it uses a compatible format or requires prompt changes.

5. Performance tradeoff

MCP adds protocol overhead but keeps everything in-process. CLI removes the protocol but spawns a new process per command. With many tool calls per step (navigate, snapshot, click, verify…), the per-invocation process spawn cost may exceed the MCP protocol overhead. Benchmark if possible.

Key references

• playwright-cli release repo: https://github.com/microsoft/playwright-cli
• playwright-mcp release repo (for comparison): https://github.com/microsoft/playwright-mcp
• Both implementations live in the main Playwright repository: https://github.com/microsoft/playwright
• Current createConnection() usage: src/agent/Orchestrator.js
• Related issue — reuse single MCP connection per context: Investigate: reuse a single @playwright/mcp connection per browser context instead of one per step #12

Expected outcome

A clear recommendation: switch to playwright-cli, stay with @playwright/mcp, or adopt a hybrid (e.g. use playwright-cli for actions that don't need the live context, MCP for those that do). Document the browser context sharing approach if a switch is viable.