refactor: rewrite generative-ui extension — display-only widgets, cross-platform, typed protocol - #5
Merged
Merged
Conversation
added 9 commits
June 2, 2026 17:10
- protocol.ts: shared discriminated-union message types (host ↔ page) - runtime/bridge.ts: deliver hook + typed on()/send() + page→host RPC - runtime/morph.ts: morphdom-based applyHTML + runScripts (final-only) - runtime/features/svg-saver.ts: hover menu, calls svg.copy / svg.save RPC - runtime/index.ts: boot — install bridge, subscribe to content, mount features - build.mjs: esbuild bundles runtime → IIFE → inlined into shell HTML → runtime.bundle.ts exports RUNTIME_HTML/RUNTIME_VERSION (committed) - tsconfig.json, devDeps (esbuild, morphdom, vitest, typescript, @types/node) No behavior change yet — index.ts still consumes the legacy code path. Host-side rewrite lands in chunk 2.
Host-side rewrite. index.ts shrinks from 632 to ~220 lines and the
streaming / execute() handoff collapses into one owner.
- protocol/rpc.ts: typed message bridge attached per window. One
win.on('message') listener routes 'rpc-call' to handlers, wraps
user payloads (window.glimpse.send({...})) into 'user-message'.
- session.ts: WidgetSession owns one window's lifetime. onChunk()
debounces, onComplete() finalizes, awaitInteraction() races
message/closed/error/abort/timeout. Ready handshake is a promise.
- platform/: darwin (osascript+pbcopy), linux (wl-copy/xclip/xsel +
zenity/kdialog), win32 (PowerShell). Drops 'os: darwin' lock.
- features/svg-saver.ts: registers svg.copy / svg.save RPC methods,
delegates to platform shim. ~40 lines.
- runtime/bridge.ts: wraps window.glimpse.send so user payloads get
the 'user-message' envelope; protocol messages pass through.
- index.ts: tool registration only. Streaming path creates a session
on toolcall_start, feeds chunks, hands off to execute() by content
index. No escapeJS. No __glimpse_svg_action magic key. No WeakSet
dedupe. No setSvgSaverReady triple-call. No win.info guessing.
Also switches Type import to 'typebox' (the new package pi-ai/pi-coding
-agent use). The old @sinclair/typebox peer dep was incompatible — its
TSchema and typebox's TSchema are nominally distinct, which collapsed
Static<TParams> to unknown[].
- tests/fake-window.ts: in-memory glimpse window stand-in. Captures win.send() eval payloads and decodes the __glimpseUI.deliver(JSON) back into structured messages so assertions stay readable. - tests/protocol.test.ts (7): isHostToPage / isPageToHost guards - tests/rpc.test.ts (7): attach() handler routing, error wrapping, unknown method response, user-message dispatch, malformed-message drops, attach() idempotence, </script> escaping in push() - tests/session.test.ts (11): ready-handshake, debounce, MIN_CHUNK, duplicate suppression, onComplete cancels debounce + emits final, post-complete chunks ignored, awaitInteraction message/closed/ error/abort/timeout, first-terminator-wins race. - tests/platform.test.ts (7): mocked child_process. darwin pbcopy + osascript argv; linux wl-copy/xclip selection by env, missing-tool error, zenity invocation; the getPlatform() factory. - tests/integration.test.ts (2): opens a real glimpse window (skipped via describe.skip when the native binary is absent and FORCE_INTEGRATION is unset). Streams a chunk, finalizes, asserts the user-message round-trip. Second case wraps __glimpseUI.deliver to surface an RPC result back, exercising the full host↔page loop. vitest.config.ts added (node env, 10s timeout).
Architectural fixes:
- glimpse-window.ts: one structural type (GlimpseWindowLike + Opener) shared
by session, rpc, and tests. Drops 'as never' / 'as unknown as object' casts
throughout production code.
- runtime/features/svg-saver subscribes to bridge 'content' events directly
and owns its own ready bit. Kills the __glimpseUiSvgSetReady global that
was the same anti-pattern the original code shipped (runtime entry knowing
about a specific feature by name via a window global).
- runtime/index.ts no longer references any feature; just installs bridge,
morph loop, then features (each manages itself).
Correctness fixes:
- session.awaitInteraction: clears its setTimeout on win and removes its
abort listener when another terminator wins. No more leaked timers/handles.
- session.awaitInteraction throws if called twice (tested).
- Page-side rpc() now has a configurable timeout (30s default) so orphaned
calls don't hang forever in the pending Map.
- rpc.jsLiteral: dropped the bogus <\!-- escape (no-op in JS strings) and
switched </script> to \u003C/script (decodes back to original on eval).
- Host-side svg-saver is sole owner of filename sanitization; the client
just sends suggestedName as a hint.
- features/svg-saver: trigger no longer has a click handler conflict; the
toggle works whether the menu is open or closed.
UX fixes (followed user testing):
- SVG menu auto-dismisses ~900ms after a successful Copy/Save (skips
dismiss on Save-dialog cancellation — they're likely about to retry).
- Click outside menu, or Escape, dismisses immediately.
- Trigger and menu hover/transit decoupled: 120ms grace lets the cursor
cross the trigger↔menu gap without collapsing the dropdown. The earlier
bug where Download became unclickable is gone.
- Trigger visibility (host) and menu visibility are independent state
machines; SVG hover only drives the trigger, not the menu.
- Dropped the 4px blue outline on the trigger; kept the subtle background
darken when the menu is open.
Platform cleanup:
- darwin: spawn + execFile imported once.
- win32: single runPS(ps, stdin?) helper; copyText no longer duplicates
the PowerShell invocation; stale 'env to avoid quoting' comment removed.
- platform/index: 'unsupported' branch returns a Platform with name:
'unsupported' instead of an unsound cast on process.platform.
Dead code removed:
- setHTML in GlimpseWindowLike / FakeWindow.
- WidgetSession.onUserMessage public method.
Tests:
- FakeWindow implements GlimpseWindowLike directly; session.test drops
'as never'.
- Added: awaitInteraction throws on second call; timer cleanup on
first-terminator-wins; abort listener removal.
- rpc test verifies </script> survives the round-trip via eval+JSON.parse.
- Integration tests pass {hidden: true} and wrap in try/finally so failed
assertions don't leak windows; second test uses the public
__glimpseUI.rpc API instead of monkey-patching deliver.
README and CHANGELOG updated to reflect the new architecture, the
cross-platform support (macOS / Linux / Windows via Glimpse 0.8), and
the public widget RPC surface.
Reflects all the host↔page changes above plus the SVG menu UX work in runtime/features/svg-saver.ts. Hash 8a9f3b16e91a, 19.9 KB. Run `npm run build:runtime` to regenerate.
CI workflow (.github/workflows/test.yml):
- Matrix: macos-latest, ubuntu-latest, windows-latest × node 20, 22.
- Typecheck, build runtime, verify the committed bundle stays in sync
with sources (catches forgotten 'npm run build:runtime' commits), then
run the full vitest suite on every platform.
- macOS does a full 'npm ci' so glimpseui's postinstall compiles the
Swift binary and the real-window integration tests run.
- Linux + Windows use '--ignore-scripts' (avoids requiring Rust+GTK or
.NET 8 SDK in the runner). The integration tests skip themselves when
the native binary is absent, so unit + platform tests still execute.
.gitattributes:
- Force LF line endings on all text files so the runtime.bundle.ts hash
is identical across Windows / Linux / macOS — otherwise the
bundle-sync check above would false-positive on Windows.
win32 platform tests (tests/platform.test.ts, +4 tests):
- copyText: powershell.exe is invoked with -EncodedCommand, the encoded
command decodes to the expected PS, and the SVG payload reaches
proc.stdin.
- chooseSavePath: PS contains SaveFileDialog + JSON-quoted filename +
OverwritePrompt, trimmed stdout is returned as the chosen path.
- chooseSavePath returns null on empty (cancelled) stdout.
- GLIMPSE_PS_PATH env override is honored.
The execFile mock was extended to also record stdin for callers that
write to proc.stdin synchronously after the call (the win32 copyText
pattern). Existing darwin/linux tests are unaffected because they check
the right array (__execFileCalls vs __spawnCalls).
Polish:
- guidelines.ts: MODULE_SECTIONS is now 'as const satisfies …', exports
a Module union type. AVAILABLE_MODULES is 'readonly Module[]' instead
of 'string[]'. index.ts drops the 'as readonly string[]' and 'as
string[]' casts; renderCall now takes 'readonly string[]'.
- runtime/features/svg-saver: trigger cursor is 'pointer' (was 'default')
now that the trigger has a real click handler.
- platform/win32: PSExecutable env var renamed to GLIMPSE_PS_PATH
(conventional naming, namespaced).
- tests/integration: drops the last 'open as never' by importing the
Opener type and casting the dynamic glimpse import to '{ open: Opener }'.
… final polish show_widget no longer waits for or surfaces user interactions ───────────────────────────────────────────────────────────── - execute() resolves the moment the final HTML is delivered. No more 'Widget still open (timed out waiting for interaction)' two minutes after the user already moved on. - The window stays open; the user closes it when done. - WidgetSession loses ~60 lines: no awaitInteraction, no SessionResult union, no timeout, no interactionStarted guard, no terminator-race. - Host-side rpc.ts drops onUserMessage and the user-message branch. - runtime/bridge.ts drops wrapGlimpse (was envelope-wrapping user glimpse.send payloads). Widget code can still call glimpse.send(...); the payload reaches native and is then dropped on the floor by design. - PageToHost collapses to RpcCall. Tool description now tells the truth ──────────────────────────────────── The previous description and promptGuidelines promised the LLM that 'window.glimpse.send(data) sends JSON data back to the agent' — but after this commit it doesn't. Updated both to say widgets are display-only and that glimpse.send / sendPrompt patterns are no-ops. The LLM will stop emitting dead interactivity. External script loading fix ─────────────────────────── runScripts in runtime/morph.ts now AWAITS the load event for each external <script src=…> before processing the next one. Previously the naive loop inserted the CDN script and immediately the inline script that used it — running Chart.js initialization before Chart was defined → blank canvas. Sequential loading fixes Chart.js, D3, mermaid, anything that's 'CDN then init'. SVG colour styling tuned ──────────────────────── Each c-* ramp now uses the 400-stop as the stroke (mid-luminance) instead of the 200-stop (high-contrast outline) at 1px stroke-width instead of 0.5px. Result: calmer outlines, crisper edges, less 'marker pen on construction paper'. --t neutral (used for .arr / .leader) moved from #707070 to #b4b2a9 so arrows and leader lines are visible without screaming. Abort handling tightened ──────────────────────── - signal.aborted is checked at execute() entry and again after we own a session (covers the race between the two checks). - The abort listener is wired BEFORE the async session.onComplete(code) await so an abort that arrives during the streaming flush still closes the window. Static type plumbing ──────────────────── - Tool schemas (ReadMeParams, ShowWidgetParams) extracted as module constants, paired with explicit Details interfaces and passed as type params to registerTool<typeof Schema, Details>. renderCall and renderResult drop their hand-rolled arg types; TypeScript infers the proper Static<TParams> from the schema. Misc polish ─────────── - session.ts: window-error events now logged (were silently swallowed). - session.ts: MIN_CHUNK_BYTES has a doc comment explaining the threshold. - protocol.ts: PageToHost / HostToPage members extracted as named interfaces (ContentMessage, RpcOk, RpcErr, RpcCall) so the discriminated union reads as one type per member instead of a wall of inline literals. - rpc.ts: stale doc claiming we 'forward user-message payloads to registered listeners' removed. - README + CHANGELOG updated to reflect the display-only contract.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full rewrite of the generative-ui extension. Started as "the SVG export menu PR was kinda gnarly, can we do better?" and grew into a complete restructure.
What this does
For the user:
execute()returns the moment the final HTML lands — no more 2-minute "waiting for interaction" timeout."os": ["darwin"]lock.<svg>in a widget → floating menu with Copy to clipboard / Download file. Backed by a typed RPC layer that talks to per-OS shims.For the codebase:
index.ts: 632 → ~250 lines. Just tool registration + a thin streaming bridge.platform/{darwin,linux,win32}.ts) so clipboard + save-dialog are one-call interfaces.Highlights from the journey
I asked for honest critique three times mid-PR. Each pass turned up real issues:
awaitInteraction, dropped wrong<\!--escape, tightened types.Static<TParams>types.Commits
Behavior changes worth flagging
window.glimpse.send(data)from widgets is now a no-op. The LLM is explicitly told not to emit it. Existing widgets that used it will silently lose their callbacks. The new contract: widgets are display-only; in-widget interactivity (sliders updating charts, hovers, canvas animations) still works fine.@mariozechner/pi-*→@earendil-works/pi-*,@sinclair/typebox→typebox(the framework switched packages).^0.3.5→^0.8.1(the cross-platform release).npm run build:runtimeon install. The committedruntime.bundle.tsmeans end users don't need esbuild present; CI verifies the bundle stays in sync.Testing
npm testruns:child_process)34/34 green locally. CI matrix in
.github/workflows/test.ymlruns the same suite on macOS, Linux, and Windows.