feat: Electron support#247
Draft
latekvo wants to merge 10 commits into
Draft
Conversation
latekvo
changed the title
latekvo
force-pushed
the
worktree-electron-support
branch
from
fcb0f5f to
b44c33d
Compare
Argent already drives iOS simulators and Android emulators. This commit adds a third platform, `electron`, so any webapp wrapped in an Electron process can be controlled the same way. Mechanism: the user launches Electron with `--remote-debugging-port=<port>` (boot-device does this automatically given an app path) and Argent drives the renderer over Chrome DevTools Protocol — no native binary needed. What works for electron: - list-devices: probes 9222 + ARGENT_ELECTRON_PORTS + ports boot-device opened in this process; entries carry platform="electron". - boot-device: new electronAppPath param spawns an Electron binary, picks a free port, waits for CDP, returns electron-cdp-<port>. - gesture-tap / gesture-swipe: CDP Input.dispatchMouseEvent with normalized → CSS-pixel conversion. - keyboard: CDP Input.dispatchKeyEvent (named keys + per-char typing). - screenshot: CDP Page.captureScreenshot persisted under tmpdir. - describe: walks the renderer DOM via Runtime.evaluate, emits the same DescribeNode shape as iOS/Android; format-tree renders nested mode. - open-url: Page.navigate + viewport refresh. - launch-app: no-op (the renderer is already running) but refreshes the viewport so a resize doesn't trip the next tap. - run-sequence: routes through the electron CDP session. Mobile-only tools (button, rotate, gesture-pinch, gesture-rotate, gesture-custom) declare no electron capability and reject cleanly at the HTTP gate. Platform plumbing: - Platform union extended to "ios" | "android" | "electron"; DeviceKind adds "app". - ToolCapability.electron added; assertSupported routes through a per-platform matrix. - dispatchByPlatform accepts an optional electron branch; if a tool declares electron support but no handler, it throws NotImplementedOnPlatformError. - classifyDevice checks the "electron-cdp-" prefix first so iOS UUIDs and Android serials stay unambiguous. - CDPClient gained a sendOrigin: false option since Chromium's devtools-target rejects upgrades that carry an Origin header. Tests: +13 cases under test/electron-*.test.ts covering classification, capability, dispatch, discovery (fake CDP server), the format-tree mode switch, and a smoke test of the blueprint factory against a WebSocketServer that mocks CDP replies.
Verify-agent follow-ups on the initial Electron commit. Correctness - electron-cdp: throw when /json/list returns only devtools:// pages (driving input into the inspector silently masks the bug instead of reaching the real BrowserWindow). - electron-cdp: readViewport now throws on a non-string / unparseable / zero-dimension reply rather than masking with a fake 800x600, which would silently corrupt every tap's coordinate math. - electron-cdp: dispatchMouseEvent guards x/y against NaN / Infinity and keys the `buttons` bitmask off the resolved button so an explicit `button: "none"` no longer ships with buttons=1. - electron-cdp: evaluate() now honours its returnByValue option. - boot-electron: race waitForCdpReady against child `exit` — a crash during startup now surfaces as "exited with code N" instead of a generic 30s readiness timeout. - boot-electron: strip user-supplied --remote-debugging-port from extraArgs so an override can't drift the port we tracked. - boot-electron: escalate to SIGKILL 2s after SIGTERM if the child ignores the polite signal (Intel GPU drivers can deadlock here). - describe: walk open shadow roots and same-origin iframes — without this, VS Code-class Electron apps return empty trees. - describe: cap the walker at 5000 nodes / depth 24 with a truncated flag so a runaway SPA can't overflow CDP's evaluate-payload limit. - run-sequence: pre-flight each sub-tool's capability gate against the device before invoking — a mobile-only step on an electron udid now fails cleanly instead of descending into a blueprint factory error. - run-sequence: pre-warm the right transport (simulator-server vs electron-cdp) based on the device platform. Ripple cleanup - preview.ts: drop Electron entries from /simulators (UI streams via simulator-server WS) and reject /simulator-server/<electron-id> with a 400 so a forged URL can't spawn a sim-server for an Electron id. - stop-all-simulator-servers / stop-simulator-server: include the Electron CDP namespace so session-end cleanup tears down CDP sessions too. - ax-service / native-devtools / native-profiler-session: replace the hard-coded "classifies as Android" error wording with the actual device.platform so an Electron udid that somehow reaches these factories produces an accurate message. - run-sequence description: mark each sub-tool's platform support; udid description now mentions Electron. Tests - ios-only-blueprint-gate.test.ts: assertion regex updated for the new dynamic platform wording.
Adds a per-Electron-device `ElectronServer` that runs in-process inside
the tool-server and exposes the same conceptual API surface as the Rust
sim-server used for iOS / Android. All work is layered onto a single CDP
connection per device so consumers don't have to reason about Chromium
internals.
New package directory: packages/tool-server/src/electron-server/
types.ts — shared TouchType/Button/Rotate/Wheel/Screenshot/etc.
cdp-session.ts — connect + discover primary page + domain enable
viewport.ts — Runtime.evaluate-backed viewport read, throws on
bad replies (no fake 800x600 fallback)
input.ts — touch/key/button/wheel/rotate translation to CDP
Input.* and Emulation.setDeviceMetricsOverride;
multi-touch dispatched via Input.dispatchTouchEvent
navigation.ts — Page.navigate / reload / history back+forward
clipboard.ts — setClipboardText via navigator.clipboard fallback to
document.execCommand("copy"); clipboard-sync stub
placeholder for future native bridge
fps.ts — frame-arrival counter that emits fpsReport once/sec
when reporting is enabled
screencast.ts — refcounted Page.startScreencast manager; one CDP
session shared across all subscribers, frame events
ack'd automatically so Chromium keeps streaming
screenshot.ts — Page.captureScreenshot + optional rotation +
optional downscale (lanczos3/box/bilinear/nearest)
via dynamically-loaded sharp; graceful fallback +
one-time warning when sharp is not installed
http-api.ts — Express router mirroring sim-server's REST surface
(POST /api/screenshot, /api/clipboard/text, /api/fps,
/api/navigate, /api/reload, /api/history/back+forward,
GET /viewport) plus GET /stream.mjpeg multipart JPEG
stream; WebSocket attach for input + events bus
index.ts — ElectronServer factory wiring everything together
Tool-server integration:
- electron-cdp blueprint refactored as a thin wrapper around
createElectronServer; legacy ElectronCdpApi surface retained so
existing tools (gesture-tap, screenshot, describe, keyboard,
run-sequence) keep working with no callsite changes. `api.server`
exposes the new abstraction for callers that want it.
- screenshot tool now accepts `rotation`, `scale`, and `downscaler`
for Electron and threads them through to the new pipeline (iOS /
Android path unchanged).
- http.ts mounts a `/electron-server/:deviceId/*` namespace that
lazily resolves the registry service and forwards every request to
the corresponding per-device router. Hidden from MCP — same posture
as `/preview`.
Sharp is an optional dependency. When missing, scale/rotation are
skipped with one stderr warning per process, and the full-resolution
PNG is still produced. Adding sharp as a hard dep would bloat the
install for every consumer regardless of platform.
Tests: +31 new cases across electron-server/{input, screenshot,
screencast, navigation, fps}; sharp-missing path uses Module._resolveFilename
stubbing so it's deterministic whether or not sharp is installed.
The /electron-server/:id/ws endpoint was implemented in http-api.ts but
the upgrade handler was never attached to the live http.Server, so
clients hit a 404. Plumb it now:
- HttpAppHandle gains attachElectronWebsockets(server) — splitting WS
bootstrap from createHttpApp keeps the Express construction synchronous
(the upgrade hook needs the Node http.Server instance, not the Express
app, which only exists after listen()).
- index.ts calls attachElectronWebsockets immediately after app.listen()
so the handler is bound by the time the tool-server advertises ready.
- The resolver looks up the ElectronServer from the registry by URN
rather than calling resolveService — a CDP connect inside the upgrade
handler would stall the TCP socket. Clients should hit a REST endpoint
first (or boot-device, which auto-resolves) to warm the session.
Verified with a Node client sending touch + wheel commands; replies
arrive as {"id":..., "status":"ok"} and the renderer's counter
incremented as expected.
Four debugger-* tools now work against Electron by talking directly to the
page CDP session that boot-device opens, instead of going through Metro's
/inspect/device discovery loop:
- debugger-connect — reports session info; no port arg required
- debugger-status — connection state + loaded scripts + enabled domains
- debugger-evaluate — Runtime.evaluate on Chromium (same wire as Hermes)
- debugger-log-registry — captures Runtime.consoleAPICalled into the existing
LogFileWriter / cluster pipeline
Dispatch lives in tools/debugger/debugger-service-ref.ts: an Electron device id
routes to the new ElectronJsRuntimeDebugger blueprint (a thin adapter over
ElectronCdp), everything else stays on Metro. The blueprint exposes the same
JsRuntimeDebuggerApi shape so tools don't need conditional code.
ElectronCdp's factory now tolerates being resolved as a transitive dependency
(URN-only, no options channel — see Registry._resolve), synthesizing DeviceInfo
from the URN payload when options.device is absent. Explicit options.device is
still honored and validated against the URN to surface wiring bugs.
Tools that depend on the React Native inspector, the React DevTools backend,
the JS-fetch network interceptor, or Hermes-format trace files now declare
capability without an `electron` block, so the HTTP gate rejects them up-front
with "Tool 'X' is not supported on electron app" instead of failing deep:
- debugger-component-tree, debugger-reload-metro, debugger-inspect-element
- view-network-logs, view-network-request-details
- all react-profiler-* (start/stop/status/renders/fiber-tree/cpu-summary/analyze)
- all profiler-* query tools (cpu-query/commit-query/stack-query/load/combined-report)
E2E verified against a live Electron app: 4 ported tools return real data
(eval round-trips, console capture surfaces clustered logs); 17 locked tools
return HTTP 400 with the clear capability message.
Four-agent verification swarm (correctness/scope/edge/ripple) surfaced these on top of the 0b0112b commit. All fixed in this commit, all re-verified: Real defects in ElectronJsRuntimeDebugger: - Attach the cdp.disconnected → events.terminated bridge BEFORE the awaits in factory (was at line 208, after createConsoleLogServer + addBinding — a CDP termination during those awaits left the registry believing the service was healthy until the next CDP send). - Coerce non-finite consoleAPICalled.timestamp to Date.now() before constructing a Date — new Date(NaN).toISOString() throws RangeError, which the typed emitter swallows, silently dropping the log entry. - dispose() now off()s both listeners symmetrically (the disconnected one was leaking). Scope tightening: - Add capability: RN_ONLY_TOOL_CAPABILITY to react-profiler-component-source for parity with the rest of react-profiler-*. The HTTP gate is a no-op here (the tool takes no device_id), but the declaration is now consistent intent — an LLM agent reading the catalogue should see this is paired with the other react-profiler tools and not reach for it on Electron. Doc accuracy: - argent-metro-debugger SKILL: frontmatter description + intro updated to cover both Metro (full surface) and Electron (4-tool subset). The "requires Metro dev server" preamble was outright wrong for the ported tools. - gesture-tap description no longer recommends debugger-component-tree (an Electron-unsupported tool) for discovery on every platform — it now points Electron callers at `describe` and reserves the RN-specific tools for iOS / Android. Tests: - Table-driven test in electron-debugger-dispatch.test.ts iterates every locked tool's capability against an Electron device — was 2 tools, now all 15 exported ToolDefinitions. A spec-count assertion guards against silent additions/omissions. - Three new cases in electron-js-runtime-debugger.test.ts cover the disconnected → terminated propagation (with and without cause), the listener detachment on dispose, and the NaN-timestamp coercion path. 781 vitest cases now passing (was 761). Build + prettier clean. E2E re-verified against a live Electron app: the 4 ported tools still work end-to-end and the react-profiler-component-source declaration doesn't break its disk-only path. Out-of-scope-but-flagged: - boot-electron.ts:166 — spawn() lacks an 'error' handler; ENOENT escalates to uncaughtException. Pre-existing in commits before this PR's scope. - All package.json versions on this branch read 0.7.1 vs main's 0.8.0; the main bump landed in #245 after this branch was cut. Release-management concern, not a code regression.
`spawn()` returns synchronously, but ENOENT / EACCES / EAGAIN are delivered
on the next tick as an `'error'` event on the child process. EventEmitter
convention: an unhandled 'error' event escapes as an uncaught exception. Before
this fix, calling boot-device with electronAppPath on a host that didn't have
electron on PATH would crash the entire tool-server.
Fold the error event into the readiness race alongside the existing exit-event
handler, with a message that names the code and tells the agent how to fix it
("install electron in the app dir or globally"). Pattern lifted from the
existing boot-device-spawn-error coverage so the same regression doesn't bite
the new Electron path.
Includes a regression test that mocks spawn, emits ENOENT / EACCES, and
confirms the boot promise rejects (not hangs) with a useful message — plus a
case for the no-pid fallback.
Second verification swarm surfaced one real bug, one misleading rationale
comment, and two stale skill cross-references. All fixed here:
- boot-electron: a pid-less child + deferred 'error' event combo would have
left the spawn-error listener attached when the function threw
synchronously, then resolved reject() on a promise nobody awaits — Node's
default --unhandled-rejections=throw would have crashed the tool-server.
Detach the listener in the no-pid throw branch and null out the reject
closure so a late event no-ops cleanly. Regression test mocks the
sequence end-to-end (no listeners remain, no unhandled rejection fires).
- electron-js-runtime-debugger.ts: the rationale comment for attaching the
disconnect listener early claimed the registry depends on the in-factory
terminated emit, but the registry only subscribes to instance.events
AFTER factory returns. The actual safety net for in-factory disconnects
is the upstream ElectronCdp's own terminated event, which the registry
has already bound. Comment updated to describe the real division of
labor: upstream covers the init window, our bridge covers everything
after factory returns. The dispose-symmetry rationale is preserved.
- SKILL doc drift: argent-react-native-app-workflow's quick-reference table
described argent-metro-debugger as "Full Metro CDP debugging" — now
inaccurate since the same skill spans both Metro and the four Electron-
ported tools. Same for argent-metro-debugger's own quick-reference row
("Connect to Metro CDP" → "Connect to CDP (Metro / Electron)").
786 vitest cases pass (was 785 — one new orphan-rejection regression case).
Build + prettier clean.
…he function Swarm v3 found the symmetric leak the v2 fix missed: the spawn `'error'` and exit listeners were left attached on the success path. The child is detached + unref'd by design (so it survives beyond the boot function), so a NORMAL later action — user closing the Electron window — fires `'exit'` against the still-attached listener, which then calls reject() on an orphan promise. With Node's default --unhandled-rejections=throw, that crashes the tool-server. Refactor: lift `onExit` out of the IIFE, mirror the spawn-error null-and-detach pattern via a `detachBootListeners()` helper, and call it in both the success and failure paths after `Promise.race` resolves. Failure path detaches BEFORE killChildEscalating so the impending kill→exit doesn't chain into a stale earlyExit rejection. Regression test stands up a real http server that satisfies ensureCdpReachable + discoverPrimaryPage, boots cleanly, then emits `'exit'` and a late `'error'` to confirm no unhandled rejection arrives. Verifies both listener counts are zero after return. 787 vitest cases pass (was 786). Build + prettier clean.
Swarm v4 spotted two minor improvements on the symmetric-leak fix:
- The success-path detach has a test; the failure-path detach (catch block)
doesn't. Add one: drive bootElectronApp into a readyTimeoutMs timeout
against an unbound port, confirm both listenerCount("error") and
listenerCount("exit") drop to 0, then emit a synthetic post-kill 'exit'
and confirm no unhandled rejection arrives.
- Add an INVARIANT comment near the catch block stating that
detachBootListeners() must remain the first synchronous statement —
inserting an await before it would re-introduce the orphan-rejection
window the v3 fix closed.
788 vitest cases pass (was 787). Build + prettier clean.
latekvo
force-pushed
the
worktree-electron-support
branch
from
b44c33d to
45e903e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Electron (Chromium + CDP) as a third device platform alongside iOS and Android. Argent can boot an Electron app, list it next to simulators/emulators, drive it with the same tool surface where it makes sense — taps, swipes, keyboard, screenshots, describe, navigation — and debug it via the four ported
debugger-*tools.Mirrors the sim-server architecture in TypeScript: a per-device server with HTTP API, MJPEG screencast, refcounted CDP screencast, FPS tracker, clipboard, and a WebSocket command bus.
What works on Electron
boot-device(withelectronAppPath),launch-app,open-urlscreenshot,describe(DOM walker w/ shadow-DOM + same-origin iframes)gesture-tap,gesture-swipe,keyboard(text + named keys)run-sequence(per-step capability check)debugger-connect,debugger-status,debugger-evaluate,debugger-log-registrylist-devices(Electron entries shown alongside sims/emulators)stop-simulator-server,stop-all-simulator-servers,stop-metroupdate-argent,dismiss-update,gather-workspace-data, allflow-*recording toolsDebugger port — how it routes
Dispatch lives in
tools/debugger/debugger-service-ref.ts: an Electron device id (electron-cdp-<port>) routes to a newElectronJsRuntimeDebuggerblueprint that wraps the page CDP session already opened byboot-device. Everything else stays on the Metro-drivenJsRuntimeDebuggerblueprint. The new blueprint exposes the sameJsRuntimeDebuggerApishape so tool code is unchanged.Runtime.consoleAPICalledevents feed into the existingLogFileWriter+ cluster pipeline —debugger-log-registryreturns the same shape on Electron as it does on Metro.What is cleanly rejected with
Tool 'X' is not supported on electron appHTTP capability gate declines these up-front (HTTP 400):
gesture-pinch,gesture-rotate,gesture-custombutton,rotaterestart-app,reinstall-apppaste, allnative-*,native-profiler-*debugger-component-tree,debugger-reload-metro,debugger-inspect-elementview-network-logs,view-network-request-detailsreact-profiler-*(incl.component-source), allprofiler-*query / combined toolsVerification history
Three commits resolved findings from two parallel verification swarms:
507b295— swarm v1 fixes: early-disconnect race in new blueprint, NaN timestamp crash, disconnect listener leak, missing test coverage for 13 of 15 lockouts, missing disconnect→terminated propagation test, stale skill + tool descriptions.dd0f185— pre-existing crash:spawn()in boot-electron lacked an'error'event handler; ENOENT escalated touncaughtException. + 4 regression test cases.95161ab— swarm v2 fixes: orphan promise rejection when pid-less spawn fires deferred'error', misleading rationale comment, stale cross-skill references calling this "Metro debugging only".Branch was also cherry-picked onto v0.8.0 (
e1327b4) to pick up #245.Follow-ups left for separate PRs
Profiler.start/Profiler.stopwork fine; only the V8 sample format differs from Hermes'. A normalizer in the query layer would unlockreact-profiler-cpu-summary+profiler-cpu-queryfor arbitrary Electron renderers (without the React-DevTools commit half).debugger-inspect-elementcould be rebuilt on top ofDOM.getNodeForLocation+ CSS source maps, but it'd be a from-scratch implementation rather than a port.New sim-server-equivalent HTTP surface
Mounted at
/electron-server/:deviceId/*— not advertised to MCP, consumed by preview UIs and integration tests:GET /viewport,GET /stream.mjpeg(multipart-JPEG screencast)POST /api/screenshot,/api/clipboard/text,/api/fps,/api/navigate,/api/reload,/api/history/back,/api/history/forwardWS /ws(input commands + event bus)Sharp is loaded as an optional dependency — screenshots still work without it (no rotate/resize, just raw PNG).
Test plan
1+1, JSON-stringified globals,document.title) / log-registry (clusters + log file).Tool 'X' is not supported on electron app.