diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md new file mode 100644 index 000000000..fde154325 --- /dev/null +++ b/docs/design/wire-observability-debug-host.md @@ -0,0 +1,332 @@ +# Wire Observability and Debug Host + +| | | +| ------------------ | ------------------------------------------------------------------------------- | +| **Start Date** | 2026-07-25 | +| **Authors** | Nidish Ramakrishnan | +| **Implementation** | truapi#295 (Rust tap in `truapi-server`; `@parity/truapi-debugger` engine + surfaces) | +| **Description** | A payload-blind tap in the Rust host core streams every product↔host TrUAPI frame, under one correlation id, to a debugger that groups and decodes it. Two surfaces consume the same engine: a standalone loopback inspector and an in-host embed. | + +The short version: product↔host TrUAPI traffic is opaque SCALE frames with no +Network tab. A tap in the Rust host core emits every frame as opaque +`{channelId, dir, bytes}`; the debugger — never the core — correlates them into +per-operation traces and decodes them. This is a **strictly dev-only tool**: the +tap and the decoder are compiled out of production builds, so there is nothing to +turn on in a shipped host. + +## The problem + +A product and its host exchange everything — account derivation, signing, chain +reads, scoped storage, subscriptions — as serialized SCALE `ProtocolMessage` +frames across a process boundary (`MessagePort`, iframe `postMessage`, or a native +webview channel). On the wire those frames are just bytes. There is no per-call +view, no way to see which operation a frame belongs to, no request/response +pairing, and no decoded payload. When a call misbehaves, the developer is staring +at a byte channel. + +This design gives that byte channel a Network tab: capture every frame, group the +frames of one call together, and — because it is a dev tool looking at the +developer's own session — decode them. + +## Where the tap lives + +The tap is in the **Rust host core (`truapi-server`)**, behind a sink trait — not +in the TypeScript transport, and not turned on from the product side. +`truapi-server` has exactly two frame choke points, and every host funnels through +them: + +| Direction | Choke point | Location | +|---|---|---| +| inbound (product → core) | `ProductRuntime::receive_frame()` — taps **before** dispatch | `host_core.rs` | +| outbound (core → product) | `SinkTransport::send()` — delivers via `FrameSink::emit_frame`, then taps | `host_core.rs` | + +One tap covers every platform, and **nothing in `@parity/truapi` (the product +package) changes** — it has no debug seam at all. That the product package is +genuinely untouched is the test for the tap being in the right place: a seam in +the product transport would fail it. Decoding and correlation live in the +debugger, never in the core; the core only ever hands out opaque bytes. + +The tap is **fire-and-forget**. `DebugSink::emit` must not block the frame path +and must not fail the operation that produced the event, so a slow, absent, or +crashed debugger only loses a trace, never a session. The two in-path call sites +wrap `emit` in `catch_unwind` (`emit_debug`), so even a panicking out-of-repo sink +is caught, logged, and swallowed rather than unwinding into a live dispatch. + +```rust +/// Dev-only sink for host debug events. Unset ⇒ the tap is inert. +pub trait DebugSink: Send + Sync { + /// Fire-and-forget: must not block the frame path or fail the operation. + fn emit(&self, event: DebugEvent); +} + +pub struct ChannelId(pub String); +pub enum FrameDirection { In, Out } + +#[non_exhaustive] // room for non-frame events (e.g. SSO); adding one is not breaking +pub enum DebugEvent { + /// A SCALE wire frame crossing a product channel. `bytes` are the untouched + /// `ProtocolMessage`; the debugger decodes them, the core never does. + Frame { channel_id: ChannelId, dir: FrameDirection, bytes: Vec }, +} +``` + +A sink is installed per product channel via +`ProductRuntime::set_debug_sink(channel_id, sink)`; unset by default, so a host +that never installs one pays nothing. The concrete sink is provided by the host +adapter, not the core: + +- the **web** host bridges `emit` to a JS `debugEmit(channelId, dir, frame)` + callback — the host worker owns the actual socket and its dev-only URL gate; +- the **native** host's `WsDebugSink` (`native_debug.rs`) implements the trait + directly over a loopback WebSocket. It only serializes and pushes onto a + bounded queue (count- and byte-capped); a background task owns the socket, + reconnects with capped backoff, and counts dropped frames when the queue is + full. It compiles only under the `ws-bridge` feature, out of the wasm graph. It + is a ready seam for later native hosts, not one of the two surfaces below. + +## The envelope and wire identity + +Each tapped frame becomes one envelope: + +``` + { channelId, dir, frame } +``` + +`frame` is the untouched SCALE `ProtocolMessage` bytes; `dir` is **product-vantage** +(`out` = the frame left the product, `in` = it arrived at it). The Rust tap names +directions host-vantage internally and flips them to this convention on the way +out (`FrameDirection::wire_str`), so both ends always agree on which way a frame +went. Where the envelope crosses a text transport (the WS surface below), +`frame` is base64 so the whole thing fits one JSON line. + +A producer also stamps a **wire-contract fingerprint** alongside the envelope: an +envelope version (`v`), the coarse codec version (`codec` = `TRUAPI_CODEC_VERSION`), +and — the load-bearing one — `TRUAPI_WIRE_SCHEMA_HASH` (`schema`), a hash of every +frame id and its method leg (e.g. `"c18def0e997626eb"`). Frame ids are `u8` +discriminants that get reassigned as the API evolves, so a frame from a host built +against a different contract would silently decode to the *wrong* method and the +wrong value. The hash catches that: the debugger allows the decode path only for a +channel whose `schema` affirmatively matches its own. An absent or mismatched +schema still groups (grouping is payload-blind) but is never trusted to decode — +which also closes the omit-the-identity-to-bypass hole. + +## The debugger engine + +The engine is one shared core (`@parity/truapi-debugger`) that both surfaces +drive, so the surface a developer picks never changes what is shown. + +**Ingest.** Each envelope is decoded once (`decodeWireMessage`) to recover the +correlation `requestId` and the wire discriminant (`frameId`) — everything +grouping needs. An undecodable frame is surfaced as a `malformed` sentinel, not +dropped, so a trace records the failure instead of going dark. Each frame's +lifecycle `role` (request / response / start / receive / …) is resolved **at +ingest** from the frame id's wire-table kind, so every downstream consumer sees +the real role rather than `unknown`. Retained id strings are length-clamped +(default 256 chars): anything that can reach the tap could otherwise send +200k-char ids, one copy per frame, while real ids are short (`myapp.dot`, `p:1`). + +**Grouping by `(channelId, requestId)`.** Frames are grouped into per-operation +traces keyed on the channel *and* the request id — not `requestId` alone, since +each host mints its own `p:1`, `p:2`, … and two hosts reuse the same values. The +channel keeps their operations apart. This `requestId` is also the id product-sdk +telemetry spans correlate on, so a frame trace and a product span line up under +one id with no extra plumbing. + +**Generation segmentation.** A product may recycle a `requestId` for a later, +unrelated call. When a fresh opener (`request` / `start`) arrives for an id whose +current operation already opened, the engine rotates to a new *generation* rather +than merging two unrelated calls under one id. + +**Retry-storm detection.** A burst of like operations — one host hammering +`signing.createTransaction` five times in 400ms — is a cross-operation signal no +single-trace view can see. The engine groups traces by `(channelId, opener +frameId)` and slides a window (default: 3 ops within 1000ms) over their start +times, tagging every trace in a burst with a `retry-storm` badge. + +**Bounded retention, with visible eviction.** Memory is bounded three ways, and +each bound surfaces rather than lying about the data: + +- an LRU cap on retained traces (default 256); whole-op evictions are counted + (`evictedTraces()`) so a session that overflowed does not silently under-report + its op count; +- a per-trace frame cap (default 1024), ring-buffered from index 1 so a + long-lived subscription never drops its opener (pairing and retry-storm signals + key on `frames[0]`); +- a per-trace byte cap (default 1 MiB, a true bound including the opener), which + only bites when bytes are retained for decode. + +A trace that lost frames or bytes to either cap carries a `truncated` badge, and a +producer that dropped frames (link backlog full) reports the count, surfaced in +stats — so "kept N of M" is never mistaken for "only N happened". + +## The two surfaces + +One engine, two ways to look at it. Both decode every frame; neither can surface a +value the engine would not, because both run the same code. + +### (A) Standalone inspector — loopback `:9231` + +A runnable Bun server (WS + HTTP) that a host dials **outward** to. Only the inside +can initiate the connection (nothing on a dev machine dials into a device or a +Worker), so the host is always the client. Over the socket the host sends one +`{channelId, dir, frame}` text message per frame; the server groups them and +serves a full-screen, Network-tab-style **web inspector** (aggregate summary +strip, filterable/sortable operation list, per-frame drill-down with a decoded +payload column) plus a **CLI / REPL** for headless or SSH work — one-shot +`ls` / `stats` / `show` / `tail` for scripting, and an interactive query prompt +(`filter`, `sort`, `use `, `show`). The CLI is a thin client over the same +HTTP endpoints (`/traces`, `/stats`, `/frame`) rebuilding the same view model, so +both frontends agree on operations, badges, and payloads. + +What the web inspector looks like — an aggregate strip, a filterable operation +list on the left, and a detail pane on the right where every frame of the opened +op is decoded inline (no decode toggle, no reveal step): + +``` + ┌─────────────────────────────────────────────────────────────────────────────┐ + │ TrUAPI Wire Inspector [ filter… ] [ arrival ▾ ] ( all ) ( dotli.app ) │ + ├─────────────────────────────────────────────────────────────────────────────┤ + │ 11 OPS 18 FRAMES 72 B 1 LIVE SUB 54ms AVG 5 ORPHANED 6 RETRY STORMS │ + │ account.getAccount 6 signing.signRaw 2 … │ + ├───────────────────────────────────┬───────────────────────────────────────────┤ + │ ▶ system.handshake 2f · 30ms │ p:2 signing.signRaw 2 frames · 164ms │ + │ ▶ account.getAccount 2f · 71ms │ │ + │ [RETRY STORM] │ ▶ REQUEST signing.signRaw 22B +0 │ + │ ▶ signing.signRaw 2f · 164ms ◀──┤ { "tag": "V1", "value": { "account": { │ + │ ↻ account.connectionStatus live │ "dotNsIdentifier": "alice.dot", │ + │ ▶ payment.topUp 2f · 92ms │ "derivationIndex": { … } }, … } } │ + │ ▶ signing.signRaw [ORPHANED] │ ◀ RESPONSE signing.signRaw 4B ⟳ 164ms │ + └───────────────────────────────────┴───────────────────────────────────────────┘ + operation list: filter / sort, detail pane: each frame decoded to its + health badges, live subscription SCALE value inline, refused only on + a wire-schema (codec) mismatch +``` + +And the topology it runs in — the host always dials outward to the loopback server: + +``` + dev machine (loopback only) + ┌──────────────────────────────────────────────┐ + │ web host (browser) debugger :9231 │ + │ ┌────────────────────┐ ┌──────────────┐ │ + │ │ product (iframe) │ │ Bun WS + HTTP │ │ + │ │ │ frames │ │ ├ engine │ │ + │ │ ▼ │ │ ├ web UI │ │ + │ │ host worker ─ tap ──┼──ws──▶ └ CLI/REPL │ │ + │ │ rust wasm │ ▲ └──────────────┘ │ + │ └────────────────────┘ │ │ + │ WasmDebugSink → debugEmit → host worker │ + │ owns the socket + dev-only URL gate │ + └──────────────────────────────────────────────┘ + host dials outward; loopback bind, Origin + + Host-header gated (see Confinement) +``` + +### (B) In-host embed — dotli ribbon + +The same inspector, mounted inside the host, with no server and no dial-out. dotli +runs the host in one realm (the host iframe/worker) and the inspector in the top +frame. The tap in the host realm **tees** each `{channelId, dir, frame}` to the top +frame via `window.top` `postMessage`; the top frame feeds them into +`createInAppDebugger().handleFrame(...)`. A right-edge **ribbon** mounts the full +inspector on demand. The realm-tee is payload-blind **transport** — the raw +`ProtocolMessage` bytes cross the postMessage boundary untouched — and the panel +decodes at the point of display (decode-in-panel). Because the frames never leave +the app, each browser tab is its own tenant; there is nothing to host or scope. + +``` + dotli (one browser app) + ┌──────────────────────────────────────────────┐ + │ TOP FRAME (dotli-owned) │ + │ createInAppDebugger ── engine + inspector │ + │ ▲ handleFrame(channelId, dir, frame) │ + │ │ window.top.postMessage (raw bytes) │ + │ ─────┼────────────── realm boundary ──── │ + │ HOST REALM (iframe / worker) │ + │ host core ── tap ──▶ tee to verified top │ + │ rust wasm │ + └──────────────────────────────────────────────┘ + right-edge ribbon mounts the inspector on demand + tee target pinned to a verified dotli-owned top +``` + +## Decode posture + +Stated plainly: this tool **decodes every frame by default**. There is no sensitive +redaction, no denylist, and no reveal toggle. A developer inspecting their own +session's traffic sees the real values, in both surfaces. Decoding reuses +`WIRE_DECODE_TABLE[frameId]` from `@parity/truapi/wire-decode` — the same generated, +dev-only codecs the client uses; the debugger writes none of its own — and is +confined to the per-frame drill-down. The list view (`/traces`) is byte- and +value-free in every configuration: it never serializes raw bytes or decoded +values. + +The safety here is not "decode carefully". It is that **the tap and the decoder do +not exist in a production build**: + +- The web host reads its debugger URL behind `import.meta.env.DEV`. Vite replaces + that with a boolean literal, so a production bundle returns `null` + unconditionally and the tap stays inert — a stray `localStorage` key cannot turn + it on in prod. Absent a URL, the host never installs `debugEmit`, so the Rust + core never installs its sink. +- The dotli ribbon is gated on a **build-time** `DEBUG` flag, not the runtime + `?debug` toggle — a production dotli build has no ribbon to mount. +- `@parity/truapi-debugger` is a private, dev-only package that is not part of any + product or host production bundle. + +**Why "redact the sensitive ones" was dropped.** An earlier draft kept a denylist: +decode everything *except* a hand-marked set of key-bearing methods, backed by a +content check and a gated reveal. That was a false guarantee. A denylist is only as +good as its marking — a secret-bearing method added without the annotation leaks, +and a content check is a heuristic. Worse, the whole edifice implies the tool is +safe-ish to point at real traffic, which invites exactly the use it must never +have. Replacing it with "not in prod, ever" is both simpler and more honest: there +is no production surface to redact, so the guarantee is structural rather than a +list someone has to keep correct. + +## Confinement and hardening + +Even as a dev tool, each surface is confined to the machine it runs on: + +- **Loopback bind.** The standalone server binds `127.0.0.1`, so off-box peers + cannot reach it. +- **CSWSH Origin gate.** The WebSocket upgrade is refused unless the request's + `Origin` is a loopback host. A non-browser client (CLI, curl) sends no Origin and + is allowed; a cross-origin browser page trying to dial the debugger to inject + frames or drive the decoder is rejected — something binding to loopback alone + does not prevent. +- **Host-header allowlist (DNS-rebinding guard).** A page served from `evil.com` + whose DNS has been rebound to `127.0.0.1` can issue same-origin `fetch`es to the + debugger; those still carry `Host: evil.com`. Requests are 403'd unless the + `Host` header is an exact loopback name (never a fuzzy match that would read + `127.0.0.1.evil.com` as loopback). +- **Bounded payloads and retention.** The WS transport caps `maxPayloadLength` + (1 MiB), and the engine's byte-bounded retention keeps a hostile or runaway + stream from growing memory without bound. +- **Panic-safe tap.** The `catch_unwind` at the two tap sites keeps a misbehaving + sink from unwinding into a live dispatch. +- **Loopback-only producer.** The native `WsDebugSink` accepts only a `ws://` + loopback URL, resolves it, and dials the resolved loopback address directly — + closing the "validate one string, dial another" gap. No `wss`, no LAN. +- **Pinned realm-tee.** The dotli embed's `postMessage` tee is pinned to a verified + dotli-owned top frame, so the host realm neither forwards frames to nor accepts a + mount from an untrusted window. + +## Non-goals + +- No tap in `@parity/truapi` or the TS transport — the whole point of putting it in + the core. +- No sensitive-frame redaction, denylist, or reveal path — a dev-only tool decodes + everything, and production has no tap to redact. +- No mocking or mutation — the tap is one-way today. The `DebugSink` contract + (`emit`, never wait) is also the extension point: a future sink that reads a + reply could deliver-modified (mutation) or respond (mock) with no envelope or + topology change. + +## References + +- Implementation: truapi#295 — `rust/crates/truapi-server/src/{host_core.rs, + native_debug.rs}` (tap + sinks), `rust/crates/truapi-codegen` (schema hash), + `js/packages/truapi-debugger/src` (engine + surfaces). +- Tracking: sdk-team#26 — validation status, per-host sequencing, and remaining + work live there, not in this doc.