From b31bf9e4c1981a023aef22086c0c8af17a0b8cd5 Mon Sep 17 00:00:00 2001 From: Nidish Date: Sat, 25 Jul 2026 11:25:30 +0530 Subject: [PATCH 1/8] docs(design): add wire observability and debug host design doc --- docs/design/wire-observability-debug-host.md | 301 +++++++++++++++++++ 1 file changed, 301 insertions(+) create mode 100644 docs/design/wire-observability-debug-host.md diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md new file mode 100644 index 000000000..7b25e7df8 --- /dev/null +++ b/docs/design/wire-observability-debug-host.md @@ -0,0 +1,301 @@ +--- +title: "Wire Observability and Debug Host" +owner: "@decrypto21" +--- + +# Wire Observability and Debug Host + +| | | +| --------------- | ------------------------------------------------------------------------------- | +| **Start Date** | 2026-07-25 | +| **Description** | A payload-blind observe seam on the TrUAPI transport, a wire debugger, a mock/forward debug host, and a WebSocket relay - correlated by the wire `requestId`, and enabled in a product by a single URL flag. | +| **Authors** | @decrypto21 | + +## Summary + +Every TrUAPI operation is a sequence of SCALE-encoded frames crossing the product↔host +transport, already keyed by one id the transport mints: the `requestId`. This design adds a single +**payload-blind observe seam** on that id, a **wire debugger** that groups frames into +per-`requestId` traces, a **debug host** that can answer frames with scripted mocks or forward +them verbatim to a real host, and a **relay** (a `WireProvider` over WebSocket) that lets a +product point its transport at a debug host running in another process. It also adds the +integration surface that makes this usable by a real product with no application-code change: the +`@parity/truapi/sandbox` bootstrap installs the wire debugger when the embedding URL carries +`?debug=wire`, and the TrUAPI playground renders a live per-`requestId` trace panel off it. +Everything is in `@parity/truapi` and carries frame shape and timing only - never decoded payloads +or key material - so the recorder is safe to leave on anywhere, including production. + +## Motivation + +Every TrUAPI operation crosses the wire as an opaque frame `{ requestId, payload: { id, value } +}`. When an operation misbehaves - a wrong response, a subscription that never delivers, a frame +the host drops silently - the everyday question is *"what did this operation put on the wire, and +where did it stall?"* Today, answering it means hand-decoding SCALE bytes or bolting an ad-hoc +logger onto a private copy of the transport. Each team re-invents this; there is no shared way to +see the wire, and no way to line up a product's own logs with what the host received. + +Requirements: + +1. **Observe every frame without decoding it.** Shape and timing must be visible; payloads and + key material must never be exposed at this layer, so the same recorder is safe in production. +2. **One correlation id.** A single operation must be followable end to end - the wire trace and + the host dispatch under one id - with no second correlation scheme introduced. +3. **Host-agnostic.** The seam must sit on the protocol's own transport, not on any one host's + internals, so it works against any host that speaks TrUAPI. +4. **Active as well as passive.** Beyond watching, a developer must be able to answer a frame with + a fake response (to develop against host behaviour that does not exist yet, or to reproduce an + error path) or forward it unchanged to a real host - without touching that host. + +## Detailed Design + +The model in one line: **one payload-blind seam emits an `ObservedFrame` per wire frame, keyed on +the transport-minted `requestId`; a wire debugger groups those into traces, and a debug host +answers or forwards frames on the same id.** + +### The correlation id + +The transport mints `requestId` (`p:1`, `p:2`, …) when a product starts an operation, and every +frame of that operation carries it. The same id appears on the wire envelope and in the host's +`CallContext.requestId` at dispatch, so one operation is followable from the product, across the +wire, into the host, and back - under a single id. No second correlation id is introduced. + +``` + product code transport wire debugger host dispatch + │ getAccount() │ │ │ + │────────────────▶│ mint requestId = p:1 │ │ + │ │─ observe(out, p:1) ───▶│ push → WireTrace │ + │ │─ frame{ p:1 } ───────────────────────────▶ │ ctx.requestId=p:1 + │ │◀ frame{ p:1 } ─────────────────────────────│ + │ │─ observe(in, p:1) ────▶│ push → WireTrace │ + │◀ Ok(response) ──│ │ │ +``` + +### The observe seam + +`createTransport` gains an optional emit-only observer. It is the whole surface of the seam: + +```ts +function createTransport(provider: WireProvider, options?: CreateTransportOptions): TrUApiTransport; +interface CreateTransportOptions { codecVersion?: number; observe?: TransportObserver; } +type TransportObserver = (frame: ObservedFrame) => void; + +interface ObservedFrame { + direction: "out" | "in"; // out = sent by this transport, in = received + requestId: string; // the one correlation id, e.g. "p:1" + frameId: number; // wire-table discriminant, e.g. 22 + role: FrameRole; // request|response|start|receive|interrupt|stop|handshake|malformed|unknown + byteLength: number; // encoded SCALE length - shape only, never contents + timestamp: number; // epoch ms +} +``` + +Enforced properties (all in `client.ts`): + +- **Payload-blind.** `ObservedFrame` carries shape and timing only. `byteLength` is read off the + encoded bytes without decoding them. The key-set is frozen and tested. +- **Causally ordered.** The outbound frame is observed *before* `provider.postMessage`, so a + request precedes the responses even over a synchronous in-memory provider. +- **Failure-isolated.** A throwing observer is swallowed - an observer must never break the + transport's message loop. +- **Zero-cost when unset.** The notify path short-circuits before constructing anything. +- **Corrupt frames are recorded.** A frame that fails envelope decode is surfaced as a + `malformed` observed frame (byte length only, sentinel `requestId`/`frameId`) before the + transport closes, so a decode failure is recorded rather than the trace going dark. Malformed + payload *values* are not seen here (the seam never decodes values); those errors reach the + caller. + +### The wire debugger + +```ts +function createWireDebugger(options?: WireDebuggerOptions): WireDebugger; +interface WireDebuggerOptions { + sink?: (line: string) => void; // formatted lines; defaults to console.debug + forward?: TransportObserver; // a second observer (e.g. onward to a panel) + maxTraces?: number; // LRU cap, default 256 + methodNames?: ReadonlyMap; +} +interface WireDebugger { + readonly observe: TransportObserver; // hand to createTransport({ observe }) + traces(): WireTrace[]; + trace(requestId: string): WireTrace | undefined; + clear(): void; +} +interface WireTrace { requestId: string; frames: ObservedFrame[]; startedAt: number; lastAt: number; } + +function createMethodNameMap(table: Record, services: readonly string[]): ReadonlyMap; +interface WireMethodInfo { method: string; kind: WireFrameKind; } +``` + +Frames group into a `WireTrace` by `requestId`; traces are held in an insertion-ordered map capped +at `maxTraces` (LRU - each touch re-inserts, oldest evicted past the cap). `sink` and `forward` +are each isolated in `try/catch`. `createMethodNameMap` inverts the generated wire-table into +`frameId → { method, kind }`, resolving the longest service prefix first (`LOCAL_STORAGE_READ → +localStorage.read`, not `local.storageRead`); it is the runtime source of readable names. A trace +reads: + +```text +[wire p:1] → request account.getAccount (id=22, 14B) +[wire p:1] ← response account.getAccount (id=23, 35B) +``` + +### The debug host + +A `WireProvider`-shaped man-in-the-middle. It answers frames it is scripted to claim, and forwards +the rest verbatim to a real host. + +```ts +function createDebugHost(options: CreateDebugHostOptions): DebugHost; +interface CreateDebugHostOptions { + provider: WireProvider; // the product side + entries?: readonly DebugHostEntry[]; // mock entries, each claiming its wire ids + forward?: WireProvider; // optional pipe to a real host + observe?: TransportObserver; // payload-blind, host vantage + onDecision?: (d: DebugHostDecision) => void; +} +interface DebugHost { dispose(): void; } + +interface DebugRequestEntry { + readonly kind: "request"; + readonly ids: RequestFrameIds; // { request, response } + handle(ctx: DebugCallContext, payload: Uint8Array): Uint8Array | Promise; +} +interface DebugSubscriptionEntry { + readonly kind: "subscription"; + readonly ids: SubscriptionFrameIds; // { start, receive, interrupt, stop } + start(ctx: DebugCallContext, payload: Uint8Array, port: DebugSubscriptionPort): DebugSubscriptionCleanup | Promise; +} +type DebugHostEntry = DebugRequestEntry | DebugSubscriptionEntry; +interface DebugHostDecision { tier: "mock" | "forward" | "unhandled"; method?: string; frame: ObservedFrame; } +``` + +Mechanics (`debug-host.ts`): + +- **Entries are byte-level and keyed by wire id** - a flat list, not a nested `{ service: { method + } }` handler tree, and there is no internal loopback dispatcher. A `handle` takes and returns raw + SCALE bytes. The debug host imports the generated codecs (`encodeWireMessage` / + `decodeWireMessage`), and the property "a mock cannot emit a malformed frame" holds by the + convention that the caller encodes answers with those codecs - it is not enforced by the type of + `handle`. +- **Router.** Each inbound frame is split by wire id: a claiming request/subscription entry answers + (`tier: "mock"`); a claimed `stop` is terminal (marked `mock`, stops the slot if live); + otherwise, if a `forward` pipe is set, the frame travels it **byte-verbatim** with `requestId` + untouched (`tier: "forward"`) and the answer relays back; with no forward pipe, the frame + surfaces loudly as `tier: "unhandled"` (a `console.warn` - the caller would otherwise hang). +- **Every frame is marked** via `onDecision` with its `tier`, so a scripted answer can never be + silently mistaken for real host behaviour. +- **Teardown.** `dispose()` sends `stop` frames upstream for any live forwarded subscriptions, so a + torn-down debug session does not leak subscriptions on the real host. + +The debug host is an **observability seam, not a test host.** With no forward pipe it answers +scripted bytes with no core behind it - no dispatch, no permissions, no storage - so it is a +debugging convenience, not a testing tier. The deterministic testing tier is the mock host +(truapi#294, real core + mocked platform/wallet seams); the canonical headless host is truapi#264. +This debug host sits *in front of* those and forwards *to* them; it does not replace them. + +### The relay + +`createRelayProvider` is a `WireProvider` that carries frames over a WebSocket wrapped in a routing +envelope `{ v, role, sessionId, productId, frame }`; the relay routes by `(sessionId, role)` and +never parses a frame. Because it is just a `WireProvider`, pointing a product at a debug host in +another process is a **provider swap** - the transport and product code are untouched, and the same +provider drops into a debug host's `provider` / `forward` slots. + +```ts +function createRelayProvider(opts: { url: string; sessionId: string; productId: string; role: "product" | "host" | "debugger"; optIn?: boolean }): WireProvider; +interface RelayEnvelope { v: 1; role: "product" | "host" | "debugger"; sessionId: string; productId: string; frame: Uint8Array; } +class RelayRouter { join(sessionId, peer): void; leave(sessionId, peer): void; handleEnvelope(from, bytes): void; } +``` + +The client is **dev-gated**: `createRelayProvider` throws unless built with `TRUAPI_RELAY=1` or +passed `{ optIn: true }` - no silent fallback, a session that cannot reach its relay fails loudly. +`RelayRouter` is the transport-agnostic routing core (join-order-independent: frames that arrive +before the counterpart joins are buffered and flushed on join); `createLoopbackSocketFactory` runs +a relay in-process for tests and single-tab use. A reference Bun WebSocket relay ships as +`examples/relay-server.mjs`. + +### How a product turns it on + +No product changes its call sites. The `@parity/truapi/sandbox` bootstrap - the shared path that +builds the transport for browser-embedded products (including the TrUAPI playground) - reads the +embedding URL: with `?debug=wire`, `getClientSync()` installs a `createWireDebugger` on the +transport's `observe` hook and exposes it via `getWireDebugger()` (and +`window.__truapiWireDebugger__`). The playground renders a live per-`requestId` trace panel off +that. So enabling the debugger for a sandbox-based product is a **URL flag**, not a code change. + +### Testing and verification + +`bun test` covers request/response and subscription lifecycle under one id, method-name mapping +across the full wire-table, payload-blindness key-set assertions, observer/sink/forward failure +isolation, LRU eviction at the cap, the debug host's mock/forward/unhandled routing and marking, +entry-precedence over the forward pipe, the dispose-time upstream stop (with a cross-service drift +guard), and the start-pending-vs-stop race; the relay's envelope round-trip, router routing + +join-order buffering + dev gate, and two end-to-end flows over a loopback relay; and the sandbox +`?debug=wire` opt-in. The suite is green (199 pass); `tsc -b` is clean, and the TrUAPI playground +(which mounts the trace panel) builds and lints clean. + +Validated end to end against the genuine Rust core run headless as WASM: a real `localStorage.read` +round trip observed under one `requestId` and forwarded verbatim through the debug host (the host +callback received the core's own namespaced key and the core's own response encoding, confirming it +is the real dispatcher, not a shim). The relay is also proven over a real cross-process socket: a +relay server, a debug host, and a product running as three separate OS processes, communicating +only over WebSockets, carried a call answered under one `requestId` end to end. Not yet exercised: +auth-gated methods (signing, needs a paired session) and a live in-browser playground run inside a +host. + +### Privacy + +Load-bearing and by design: `ObservedFrame` carries no decoded payload and no key material, so the +seam is safe to run in production. No component here introduces decoded payloads - the relay +carries frames as opaque bytes; mocked responses are always marked (`tier: "mock"`). + +### Compatibility and performance + +Purely additive. `observe` is a new optional field on `CreateTransportOptions`; `debug.ts`, +`debug-host.ts`, and `relay.ts` are new modules with new barrel exports. No existing interface +changes; no migration required. The `ObservedFrame` key-set is frozen (additive evolution only). +The observe hook is zero-cost when unset (a single short-circuit before any allocation); when set, +it allocates one small record per frame into an LRU-capped map, so memory is bounded regardless of +session length. + +### Out of scope + +No interactive UI beyond the playground trace panel (a host-panel bridge and a handler editor are +later); no host-side observe hook; no generated method-name constant (the runtime +`createMethodNameMap` is the source today); no decoded payloads anywhere. Relay session pairing +(minting a short `sessionId`) and a dev-preview-server mount for the reference relay are also +deferred. + +## Drawbacks + +- **`requestId` is transport-local.** Each `createTransport` mints `p:1, p:2, …` from zero, so a + wire debugger / debug host instance is 1:1 with a transport; sharing one across transports would + collapse unrelated operations under the same id. Namespacing the trace key to aggregate multiple + transports is deferred until a shared-aggregation use case appears. +- **A second thing named "host."** The debug host adds a third artifact that reads as a + "headless/mock host" next to truapi#264 and truapi#294; the naming space is crowded. The + export keeps the name `createDebugHost`, and the distinction from those hosts is documented + above. +- **Byte-level mock entries are lower-level than a typed handler tree** - authoring a mock means + encoding bytes with the generated codecs, which is less ergonomic than a typed method handler + would be (a future surface concern, not a blocker). + +## Alternatives + +- **Host-specific debugger (rejected).** Tapping one host's internal hook (prior art: a dotli-only, + read-only panel) ties the tool to one host's unstable internals and cannot mock or forward. This + design hooks the protocol's own transport instead, so it is host-agnostic and active. +- **Frame rewriting instead of a dispatcher-level mock (deferred).** A mock could hand-rewrite SCALE + frames; instead, entries answer through the generated codecs, which keeps mocked frames + well-formed. Raw frame rewriting remains possible later at the relay layer. +- **A standalone debugger application (deferred).** A separate desktop app was considered; the + playground trace panel plus the relay cover the near-term need without a new app to ship. +- **Introducing a second correlation id (rejected).** Every layer keys on the transport-minted + `requestId`, which already appears on the wire and in the host dispatch context; a second id would + add plumbing with no benefit. + +## References + +- Implementation: truapi#295 (`js/packages/truapi/src/{client,debug,debug-host,relay,sandbox}.ts`) + and the TrUAPI playground trace panel. +- Peers in the "headless / mock host" space: the mock host (truapi#294) and the headless host + (truapi#264). From b7b8e06ae2476c1a096d047e70f287ef29f6ea5b Mon Sep 17 00:00:00 2001 From: Nidish Date: Sat, 25 Jul 2026 14:32:48 +0530 Subject: [PATCH 2/8] docs(design): add the dev-gated wire-decode typed-value view --- docs/design/wire-observability-debug-host.md | 66 ++++++++++++++++---- 1 file changed, 55 insertions(+), 11 deletions(-) diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md index 7b25e7df8..f18cd2418 100644 --- a/docs/design/wire-observability-debug-host.md +++ b/docs/design/wire-observability-debug-host.md @@ -23,7 +23,10 @@ integration surface that makes this usable by a real product with no application `@parity/truapi/sandbox` bootstrap installs the wire debugger when the embedding URL carries `?debug=wire`, and the TrUAPI playground renders a live per-`requestId` trace panel off it. Everything is in `@parity/truapi` and carries frame shape and timing only - never decoded payloads -or key material - so the recorder is safe to leave on anywhere, including production. +or key material - so the recorder is safe to leave on anywhere, including production. On top of that +payload-blind default, a dev-gated `?debug=wire-decode` mode decodes frames to typed request/response +values through a codegen-emitted `WIRE_DECODE_TABLE`; it is off by default and never runs in the +payload-blind path. ## Motivation @@ -104,6 +107,29 @@ Enforced properties (all in `client.ts`): payload *values* are not seen here (the seam never decodes values); those errors reach the caller. +### The decoded view + +The payload-blind seam is the safe foundation; the decoded view is what lets a developer see the +typed request and response values, not just frame shapes. It is a second, dev-gated layer built on +top of the seam, and the core never decodes - decoding is a consumer concern. + +- **`exposeFrameBytes`.** A dev-gated `createTransport` option that attaches each frame's raw SCALE + `bytes` to its `ObservedFrame`. Without it, `ObservedFrame` stays shape-and-timing only; the byte + attachment exists solely so a consumer can decode. +- **`WIRE_DECODE_TABLE`.** A codegen-emitted table exported at `@parity/truapi/wire-decode`, a + `Record unknown>` with one entry per request/response frame id + plus subscription start/receive, produced by `truapi-codegen`. Decoding is therefore always + against the generated client - the same source of truth as the wire codecs - so a decoded value + cannot drift from the wire schema. +- **The `?debug=wire-decode` flag.** It enables `exposeFrameBytes` and turns on the consumer-side + decode. A dev consumer - the playground trace panel - looks each frame's id up in + `WIRE_DECODE_TABLE`, decodes the attached bytes, and renders the decoded request/response value + inline under the payload-blind row. + +Off by default: with no `?debug=wire-decode`, `exposeFrameBytes` is never set, no bytes are +attached, and `WIRE_DECODE_TABLE` is not imported, so the decode path is dead-code-eliminable in a +production build. + ### The wire debugger ```ts @@ -220,7 +246,10 @@ builds the transport for browser-embedded products (including the TrUAPI playgro embedding URL: with `?debug=wire`, `getClientSync()` installs a `createWireDebugger` on the transport's `observe` hook and exposes it via `getWireDebugger()` (and `window.__truapiWireDebugger__`). The playground renders a live per-`requestId` trace panel off -that. So enabling the debugger for a sandbox-based product is a **URL flag**, not a code change. +that. Adding `?debug=wire-decode` does everything `?debug=wire` does and also sets +`exposeFrameBytes`, so the playground panel decodes each frame through `WIRE_DECODE_TABLE` and +renders the typed request/response value inline. So enabling the debugger for a sandbox-based +product is a **URL flag**, not a code change. ### Testing and verification @@ -229,9 +258,12 @@ across the full wire-table, payload-blindness key-set assertions, observer/sink/ isolation, LRU eviction at the cap, the debug host's mock/forward/unhandled routing and marking, entry-precedence over the forward pipe, the dispose-time upstream stop (with a cross-service drift guard), and the start-pending-vs-stop race; the relay's envelope round-trip, router routing + -join-order buffering + dev gate, and two end-to-end flows over a loopback relay; and the sandbox -`?debug=wire` opt-in. The suite is green (199 pass); `tsc -b` is clean, and the TrUAPI playground -(which mounts the trace panel) builds and lints clean. +join-order buffering + dev gate, and two end-to-end flows over a loopback relay; the sandbox +`?debug=wire` opt-in; and a codegen test asserting `WIRE_DECODE_TABLE` is emitted with one entry +per request/response frame id plus subscription start/receive. A Playwright e2e asserts both the +payload-blind panel under `?debug=wire` and the decoded typed-value view under `?debug=wire-decode`. +The suite is green (203 pass); `tsc -b` is clean, and the TrUAPI playground (which mounts the trace +panel) builds and lints clean. Validated end to end against the genuine Rust core run headless as WASM: a real `localStorage.read` round trip observed under one `requestId` and forwarded verbatim through the debug host (the host @@ -244,9 +276,17 @@ host. ### Privacy -Load-bearing and by design: `ObservedFrame` carries no decoded payload and no key material, so the -seam is safe to run in production. No component here introduces decoded payloads - the relay -carries frames as opaque bytes; mocked responses are always marked (`tier: "mock"`). +Load-bearing and by design: the **default** observe surface carries no decoded payload and no key +material, so the seam is safe to run in production. The relay carries frames as opaque bytes, and +mocked responses are always marked (`tier: "mock"`). + +Decoded payloads exist **only** under the dev-gated `?debug=wire-decode` mode: `exposeFrameBytes` +attaches raw bytes and the consumer decodes them through `WIRE_DECODE_TABLE`. That mode is off by +default and dead-code-eliminable in a production build. This dev-gating is a hard requirement, not a +convenience: the raw wire can carry key material - the truapi#264 review found secret key material +reachable on the SSO response path - so the dev gate is the structural defense that keeps decoded +payloads out of production. The decoded mode is a developer tool and is not claimed to be safe to +run in production. ### Compatibility and performance @@ -261,9 +301,10 @@ session length. No interactive UI beyond the playground trace panel (a host-panel bridge and a handler editor are later); no host-side observe hook; no generated method-name constant (the runtime -`createMethodNameMap` is the source today); no decoded payloads anywhere. Relay session pairing -(minting a short `sessionId`) and a dev-preview-server mount for the reference relay are also -deferred. +`createMethodNameMap` is the source today). Decoded payloads are in scope, but only as the dev-gated +`?debug=wire-decode` mode - the default observe surface stays payload-blind and the core never +decodes. Relay session pairing (minting a short `sessionId`) and a dev-preview-server mount for the +reference relay are also deferred. ## Drawbacks @@ -299,3 +340,6 @@ deferred. and the TrUAPI playground trace panel. - Peers in the "headless / mock host" space: the mock host (truapi#294) and the headless host (truapi#264). +- Requirement source: the debugger tracker (sdk-team#26), which asks to see what a product sends + and what the host returns, "decoded to typed values" - the requirement the dev-gated + `?debug=wire-decode` view fulfills on top of the payload-blind seam. From 8d07440f35db1411ff63c2afaab090f883d6f962 Mon Sep 17 00:00:00 2001 From: Nidish Date: Tue, 28 Jul 2026 13:21:06 +0530 Subject: [PATCH 3/8] docs(design): wire observability, debug host, and relay spec --- docs/design/wire-observability-debug-host.md | 527 ++++++++++--------- 1 file changed, 268 insertions(+), 259 deletions(-) diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md index f18cd2418..84fb88355 100644 --- a/docs/design/wire-observability-debug-host.md +++ b/docs/design/wire-observability-debug-host.md @@ -5,62 +5,42 @@ owner: "@decrypto21" # Wire Observability and Debug Host -| | | -| --------------- | ------------------------------------------------------------------------------- | -| **Start Date** | 2026-07-25 | -| **Description** | A payload-blind observe seam on the TrUAPI transport, a wire debugger, a mock/forward debug host, and a WebSocket relay - correlated by the wire `requestId`, and enabled in a product by a single URL flag. | -| **Authors** | @decrypto21 | - -## Summary - -Every TrUAPI operation is a sequence of SCALE-encoded frames crossing the product↔host -transport, already keyed by one id the transport mints: the `requestId`. This design adds a single -**payload-blind observe seam** on that id, a **wire debugger** that groups frames into -per-`requestId` traces, a **debug host** that can answer frames with scripted mocks or forward -them verbatim to a real host, and a **relay** (a `WireProvider` over WebSocket) that lets a -product point its transport at a debug host running in another process. It also adds the -integration surface that makes this usable by a real product with no application-code change: the -`@parity/truapi/sandbox` bootstrap installs the wire debugger when the embedding URL carries -`?debug=wire`, and the TrUAPI playground renders a live per-`requestId` trace panel off it. -Everything is in `@parity/truapi` and carries frame shape and timing only - never decoded payloads -or key material - so the recorder is safe to leave on anywhere, including production. On top of that -payload-blind default, a dev-gated `?debug=wire-decode` mode decodes frames to typed request/response -values through a codegen-emitted `WIRE_DECODE_TABLE`; it is off by default and never runs in the -payload-blind path. - -## Motivation - -Every TrUAPI operation crosses the wire as an opaque frame `{ requestId, payload: { id, value } -}`. When an operation misbehaves - a wrong response, a subscription that never delivers, a frame -the host drops silently - the everyday question is *"what did this operation put on the wire, and -where did it stall?"* Today, answering it means hand-decoding SCALE bytes or bolting an ad-hoc -logger onto a private copy of the transport. Each team re-invents this; there is no shared way to -see the wire, and no way to line up a product's own logs with what the host received. - -Requirements: - -1. **Observe every frame without decoding it.** Shape and timing must be visible; payloads and - key material must never be exposed at this layer, so the same recorder is safe in production. -2. **One correlation id.** A single operation must be followable end to end - the wire trace and - the host dispatch under one id - with no second correlation scheme introduced. -3. **Host-agnostic.** The seam must sit on the protocol's own transport, not on any one host's +| | | +| ------------------ | ------------------------------------------------------------------------------- | +| **Start Date** | 2026-07-25 | +| **Authors** | @decrypto21 | +| **Implementation** | truapi#295 | +| **Description** | A payload-blind observe seam on the TrUAPI transport, a wire debugger, a mock/forward debug host, and a WebSocket relay - all correlated by the wire `requestId`. | + +This is a **specification of the contract**, not a walkthrough of the implementation: it fixes the +interfaces, the observable behavior, and the invariants each layer must uphold. Implementation +detail, tests, and file layout live in truapi#295; where the two disagree, this document is the +intent and the code is the bug. + +## Requirements + +Every TrUAPI operation crosses the wire as an opaque frame `{ requestId, payload: { id, value } }`. +When one misbehaves - a wrong response, a subscription that never delivers, a silently dropped +frame - the question is *"what did this put on the wire, and where did it stall?"* Answering it +today means hand-decoding SCALE or forking the transport to add logging. This design must: + +1. **Observe every frame without decoding it.** Shape and timing are visible by default; payloads + and key material are not, so the same recorder is safe to run in production. +2. **Use one correlation id.** An operation is followable end to end - wire trace and host dispatch + under a single id - with no second correlation scheme introduced. +3. **Be host-agnostic.** The seam sits on the protocol's own transport, not on any host's internals, so it works against any host that speaks TrUAPI. -4. **Active as well as passive.** Beyond watching, a developer must be able to answer a frame with - a fake response (to develop against host behaviour that does not exist yet, or to reproduce an - error path) or forward it unchanged to a real host - without touching that host. +4. **Be active, not only passive.** A developer can answer a frame with a scripted response (to + develop against behavior that doesn't exist yet, or reproduce an error path) or forward it + verbatim to a real host - without modifying that host. -## Detailed Design +## Model -The model in one line: **one payload-blind seam emits an `ObservedFrame` per wire frame, keyed on -the transport-minted `requestId`; a wire debugger groups those into traces, and a debug host -answers or forwards frames on the same id.** - -### The correlation id - -The transport mints `requestId` (`p:1`, `p:2`, …) when a product starts an operation, and every -frame of that operation carries it. The same id appears on the wire envelope and in the host's -`CallContext.requestId` at dispatch, so one operation is followable from the product, across the -wire, into the host, and back - under a single id. No second correlation id is introduced. +**One payload-blind seam emits an `ObservedFrame` per wire frame, keyed on the transport-minted +`requestId`; a wire debugger groups those into traces; a debug host answers or forwards frames on +the same id.** The transport mints `requestId` (`p:1`, `p:2`, …) when a product starts an operation +and stamps it on every frame; the same id appears on the wire envelope and in the host's +`CallContext.requestId`. No second correlation id exists. ``` product code transport wire debugger host dispatch @@ -73,13 +53,19 @@ wire, into the host, and back - under a single id. No second correlation id is i │◀ Ok(response) ──│ │ │ ``` -### The observe seam +## Interface + +The signatures below are the normative surface, all in `@parity/truapi`. -`createTransport` gains an optional emit-only observer. It is the whole surface of the seam: +### Transport observe seam ```ts function createTransport(provider: WireProvider, options?: CreateTransportOptions): TrUApiTransport; -interface CreateTransportOptions { codecVersion?: number; observe?: TransportObserver; } +interface CreateTransportOptions { + codecVersion?: number; + observe?: TransportObserver; // emit-only; the whole seam + exposeFrameBytes?: boolean; // dev-gated; see "Decoded view" +} type TransportObserver = (frame: ObservedFrame) => void; interface ObservedFrame { @@ -87,57 +73,36 @@ interface ObservedFrame { requestId: string; // the one correlation id, e.g. "p:1" frameId: number; // wire-table discriminant, e.g. 22 role: FrameRole; // request|response|start|receive|interrupt|stop|handshake|malformed|unknown - byteLength: number; // encoded SCALE length - shape only, never contents + byteLength: number; // encoded SCALE length - shape only timestamp: number; // epoch ms + bytes?: Uint8Array; // present only when exposeFrameBytes is set } ``` -Enforced properties (all in `client.ts`): +Guarantees (enforced in `client.ts`): -- **Payload-blind.** `ObservedFrame` carries shape and timing only. `byteLength` is read off the - encoded bytes without decoding them. The key-set is frozen and tested. +- **Payload-blind by default.** `ObservedFrame` carries shape and timing only; `byteLength` is read + off the encoded bytes without decoding them. The key-set is frozen (additive evolution only). - **Causally ordered.** The outbound frame is observed *before* `provider.postMessage`, so a - request precedes the responses even over a synchronous in-memory provider. -- **Failure-isolated.** A throwing observer is swallowed - an observer must never break the - transport's message loop. -- **Zero-cost when unset.** The notify path short-circuits before constructing anything. -- **Corrupt frames are recorded.** A frame that fails envelope decode is surfaced as a - `malformed` observed frame (byte length only, sentinel `requestId`/`frameId`) before the - transport closes, so a decode failure is recorded rather than the trace going dark. Malformed - payload *values* are not seen here (the seam never decodes values); those errors reach the - caller. - -### The decoded view - -The payload-blind seam is the safe foundation; the decoded view is what lets a developer see the -typed request and response values, not just frame shapes. It is a second, dev-gated layer built on -top of the seam, and the core never decodes - decoding is a consumer concern. - -- **`exposeFrameBytes`.** A dev-gated `createTransport` option that attaches each frame's raw SCALE - `bytes` to its `ObservedFrame`. Without it, `ObservedFrame` stays shape-and-timing only; the byte - attachment exists solely so a consumer can decode. -- **`WIRE_DECODE_TABLE`.** A codegen-emitted table exported at `@parity/truapi/wire-decode`, a - `Record unknown>` with one entry per request/response frame id - plus subscription start/receive, produced by `truapi-codegen`. Decoding is therefore always - against the generated client - the same source of truth as the wire codecs - so a decoded value - cannot drift from the wire schema. -- **The `?debug=wire-decode` flag.** It enables `exposeFrameBytes` and turns on the consumer-side - decode. A dev consumer - the playground trace panel - looks each frame's id up in - `WIRE_DECODE_TABLE`, decodes the attached bytes, and renders the decoded request/response value - inline under the payload-blind row. - -Off by default: with no `?debug=wire-decode`, `exposeFrameBytes` is never set, no bytes are -attached, and `WIRE_DECODE_TABLE` is not imported, so the decode path is dead-code-eliminable in a -production build. - -### The wire debugger + request precedes its responses even over a synchronous provider. +- **Failure-isolated.** A throwing observer is swallowed; it can never break the message loop. +- **Zero-cost when unset.** The notify path short-circuits before allocating anything. +- **Corrupt inbound frames are recorded.** An *inbound* frame that fails envelope decode surfaces as + a `malformed` observed frame (sentinel `requestId`/`frameId`; under `exposeFrameBytes` it also + carries the raw envelope `bytes`, which is what you want when diagnosing the corruption) before the + transport closes, so the trace never goes dark. Malformed payload *values* are not seen here - the + seam never decodes values. This covers the inbound path only: an *outbound* frame that fails to + encode closes the transport before the observe hook runs, so it is not recorded. + +### Wire debugger ```ts function createWireDebugger(options?: WireDebuggerOptions): WireDebugger; interface WireDebuggerOptions { - sink?: (line: string) => void; // formatted lines; defaults to console.debug + sink?: (line: string, frame: ObservedFrame) => void; // formatted line + its frame; defaults to console.debug forward?: TransportObserver; // a second observer (e.g. onward to a panel) - maxTraces?: number; // LRU cap, default 256 + maxTraces?: number; // LRU cap on trace count, default 256 + maxFramesPerTrace?: number; // ring-buffer cap on frames within one trace, default 1024 methodNames?: ReadonlyMap; } interface WireDebugger { @@ -152,79 +117,109 @@ function createMethodNameMap(table: Record, services: readonly interface WireMethodInfo { method: string; kind: WireFrameKind; } ``` -Frames group into a `WireTrace` by `requestId`; traces are held in an insertion-ordered map capped -at `maxTraces` (LRU - each touch re-inserts, oldest evicted past the cap). `sink` and `forward` -are each isolated in `try/catch`. `createMethodNameMap` inverts the generated wire-table into -`frameId → { method, kind }`, resolving the longest service prefix first (`LOCAL_STORAGE_READ → -localStorage.read`, not `local.storageRead`); it is the runtime source of readable names. A trace -reads: - -```text -[wire p:1] → request account.getAccount (id=22, 14B) -[wire p:1] ← response account.getAccount (id=23, 35B) -``` - -### The debug host - -A `WireProvider`-shaped man-in-the-middle. It answers frames it is scripted to claim, and forwards +Behavior: + +- Frames group into a `WireTrace` by `requestId`. Traces are held in an insertion-ordered map, + **LRU-capped at `maxTraces`**; each trace's `frames` are **ring-buffered at `maxFramesPerTrace`**, + so a long-lived subscription (whose frames all share one `requestId` that never LRU-evicts) cannot + grow without bound. Memory is therefore bounded on both axes regardless of session length. +- `sink` and `forward` are each isolated in `try/catch`. +- `createMethodNameMap` inverts the generated wire-table into `frameId → { method, kind }`, + resolving the longest service prefix first (`LOCAL_STORAGE_READ → localStorage.read`, not + `local.storageRead`). It is the runtime source of readable names; a generated constant is a + non-goal (below). A trace reads: + + ```text + [wire p:1] → request account.getAccount (id=22, 14B) + [wire p:1] ← response account.getAccount (id=23, 35B) + ``` + +### Decoded view (dev-gated) + +The payload-blind seam is the safe foundation; the decoded view lets a developer see typed +request/response values. It is a second layer on top of the seam, and the core never decodes - +decoding is a consumer concern. + +- **`exposeFrameBytes`** attaches each frame's raw SCALE `bytes` to its `ObservedFrame`. Without it, + frames stay shape-and-timing only. +- **`WIRE_DECODE_TABLE`** (`@parity/truapi/wire-decode`) is a codegen-emitted + `Record unknown>` with one entry per request/response id plus + subscription start/receive. Decoding is always against the generated client - the same source of + truth as the wire codecs - so a decoded value cannot drift from the wire schema. +- **`?debug=wire:decode`** (see [the grammar](#the-debug-grammar)) requests the decoded view, but is + **build-gated**: it enables `exposeFrameBytes` only when the bundle was built with + `TRUAPI_WIRE_DECODE=1` (`NEXT_PUBLIC_TRUAPI_WIRE_DECODE=1` for the Next-built playground, so webpack + inlines it), mirroring the relay's `TRUAPI_RELAY` gate. Without the build flag it degrades to a + plain payload-blind `?debug=wire` trace. The production playground deploy leaves the gate unset. + +**Bundle isolation.** The decode table is **not** statically imported. The playground trace panel +`import()`s `@parity/truapi/wire-decode` lazily, and only when a frame actually carries `bytes`. Under +`next dev` this code-splits into a separate chunk; the production static export forces a single chunk +(`splitChunks:false` + `maxChunks:1`), so there the module lands in the main bundle but its factory +never runs in payload-blind mode - the gate keeps it out of the *execution path*, not the bundle. The +table is non-sensitive generated codecs, and byte exposure is separately gated, so this is a +bundle-hygiene note, not a security boundary. + +### Debug host + +A `WireProvider`-shaped man-in-the-middle: it answers frames it is scripted to claim and forwards the rest verbatim to a real host. ```ts -function createDebugHost(options: CreateDebugHostOptions): DebugHost; -interface CreateDebugHostOptions { +function createWireDebugHost(options: CreateWireDebugHostOptions): WireDebugHost; +interface CreateWireDebugHostOptions { provider: WireProvider; // the product side - entries?: readonly DebugHostEntry[]; // mock entries, each claiming its wire ids + entries?: readonly WireDebugHostEntry[]; // mock entries, each claiming its wire ids forward?: WireProvider; // optional pipe to a real host observe?: TransportObserver; // payload-blind, host vantage - onDecision?: (d: DebugHostDecision) => void; + onDecision?: (d: WireDebugHostDecision) => void; } -interface DebugHost { dispose(): void; } +interface WireDebugHost { dispose(): void; } interface DebugRequestEntry { readonly kind: "request"; - readonly ids: RequestFrameIds; // { request, response } + readonly ids: RequestFrameIds; // { request, response } handle(ctx: DebugCallContext, payload: Uint8Array): Uint8Array | Promise; } interface DebugSubscriptionEntry { readonly kind: "subscription"; - readonly ids: SubscriptionFrameIds; // { start, receive, interrupt, stop } + readonly ids: SubscriptionFrameIds; // { start, receive, interrupt, stop } start(ctx: DebugCallContext, payload: Uint8Array, port: DebugSubscriptionPort): DebugSubscriptionCleanup | Promise; } -type DebugHostEntry = DebugRequestEntry | DebugSubscriptionEntry; -interface DebugHostDecision { tier: "mock" | "forward" | "unhandled"; method?: string; frame: ObservedFrame; } +type WireDebugHostEntry = DebugRequestEntry | DebugSubscriptionEntry; +interface WireDebugHostDecision { tier: "mock" | "forward" | "unhandled"; method?: string; frame: ObservedFrame; } ``` -Mechanics (`debug-host.ts`): - -- **Entries are byte-level and keyed by wire id** - a flat list, not a nested `{ service: { method - } }` handler tree, and there is no internal loopback dispatcher. A `handle` takes and returns raw - SCALE bytes. The debug host imports the generated codecs (`encodeWireMessage` / - `decodeWireMessage`), and the property "a mock cannot emit a malformed frame" holds by the - convention that the caller encodes answers with those codecs - it is not enforced by the type of - `handle`. -- **Router.** Each inbound frame is split by wire id: a claiming request/subscription entry answers - (`tier: "mock"`); a claimed `stop` is terminal (marked `mock`, stops the slot if live); - otherwise, if a `forward` pipe is set, the frame travels it **byte-verbatim** with `requestId` - untouched (`tier: "forward"`) and the answer relays back; with no forward pipe, the frame - surfaces loudly as `tier: "unhandled"` (a `console.warn` - the caller would otherwise hang). -- **Every frame is marked** via `onDecision` with its `tier`, so a scripted answer can never be - silently mistaken for real host behaviour. -- **Teardown.** `dispose()` sends `stop` frames upstream for any live forwarded subscriptions, so a - torn-down debug session does not leak subscriptions on the real host. - -The debug host is an **observability seam, not a test host.** With no forward pipe it answers -scripted bytes with no core behind it - no dispatch, no permissions, no storage - so it is a -debugging convenience, not a testing tier. The deterministic testing tier is the mock host -(truapi#294, real core + mocked platform/wallet seams); the canonical headless host is truapi#264. -This debug host sits *in front of* those and forwards *to* them; it does not replace them. - -### The relay - -`createRelayProvider` is a `WireProvider` that carries frames over a WebSocket wrapped in a routing -envelope `{ v, role, sessionId, productId, frame }`; the relay routes by `(sessionId, role)` and -never parses a frame. Because it is just a `WireProvider`, pointing a product at a debug host in -another process is a **provider swap** - the transport and product code are untouched, and the same -provider drops into a debug host's `provider` / `forward` slots. +Behavior: + +- **Entries are byte-level, keyed by wire id** - a flat list, not a nested handler tree, and there + is no internal loopback dispatcher. `handle` takes and returns raw SCALE bytes. "A mock cannot + emit a malformed frame" holds by the *convention* that the caller encodes answers with the + generated codecs (`encodeWireMessage`/`decodeWireMessage`), not by the type of `handle`. Two + entries claiming the same inbound (`request`/`start`) wire id is a construction error: + `createWireDebugHost` throws rather than silently letting one shadow the other. +- **Routing.** A claiming request/subscription entry answers (`tier: "mock"`); a claimed `stop` is + terminal. Otherwise, with a `forward` pipe set the frame travels it **byte-verbatim, `requestId` + untouched** (`tier: "forward"`) and the answer relays back; with no forward pipe it surfaces + loudly as `tier: "unhandled"` (via `onDecision`, or a `console.warn` when no listener is + set - so the stall is loud rather than silent; the caller still hangs, as no answer is sent). +- **Every frame is marked** via `onDecision`, so a scripted answer is never silently mistaken for + real host behavior. +- **Errors are loud, never a silent wrong answer.** A mock `handle`/`start` that throws, or a + response that fails to encode, emits a `console.warn` and sends nothing - the caller hangs loudly, + the same policy as `unhandled`. An undecodable *inbound* envelope is forwarded byte-transparent + when a `forward` pipe is set, or `console.warn`ed when headless - the host-side analogue of the + transport's `malformed` recording. (Repeated unhandled frames for one wire id warn once.) +- **Teardown.** `dispose()` sends `stop` upstream for any live forwarded subscription (so the real + host stops streaming into a detached pipe) and runs cleanup for every live *mock* subscription, so + a torn-down session leaks nothing on either side. + +This is an **observability seam, not a test host.** With no forward pipe it answers scripted bytes +with no core behind it (no dispatch, permissions, or storage) - a debugging convenience, not a +fidelity tier. The deterministic testing tier is the mock host (truapi#294); the canonical headless +host is truapi#264. The debug host sits *in front of* those and forwards *to* them. + +### Relay ```ts function createRelayProvider(opts: { url: string; sessionId: string; productId: string; role: "product" | "host" | "debugger"; optIn?: boolean }): WireProvider; @@ -232,114 +227,128 @@ interface RelayEnvelope { v: 1; role: "product" | "host" | "debugger"; sessionId class RelayRouter { join(sessionId, peer): void; leave(sessionId, peer): void; handleEnvelope(from, bytes): void; } ``` -The client is **dev-gated**: `createRelayProvider` throws unless built with `TRUAPI_RELAY=1` or -passed `{ optIn: true }` - no silent fallback, a session that cannot reach its relay fails loudly. -`RelayRouter` is the transport-agnostic routing core (join-order-independent: frames that arrive -before the counterpart joins are buffered and flushed on join); `createLoopbackSocketFactory` runs -a relay in-process for tests and single-tab use. A reference Bun WebSocket relay ships as -`examples/relay-server.mjs`. - -### How a product turns it on - -No product changes its call sites. The `@parity/truapi/sandbox` bootstrap - the shared path that -builds the transport for browser-embedded products (including the TrUAPI playground) - reads the -embedding URL: with `?debug=wire`, `getClientSync()` installs a `createWireDebugger` on the -transport's `observe` hook and exposes it via `getWireDebugger()` (and -`window.__truapiWireDebugger__`). The playground renders a live per-`requestId` trace panel off -that. Adding `?debug=wire-decode` does everything `?debug=wire` does and also sets -`exposeFrameBytes`, so the playground panel decodes each frame through `WIRE_DECODE_TABLE` and -renders the typed request/response value inline. So enabling the debugger for a sandbox-based -product is a **URL flag**, not a code change. - -### Testing and verification - -`bun test` covers request/response and subscription lifecycle under one id, method-name mapping -across the full wire-table, payload-blindness key-set assertions, observer/sink/forward failure -isolation, LRU eviction at the cap, the debug host's mock/forward/unhandled routing and marking, -entry-precedence over the forward pipe, the dispose-time upstream stop (with a cross-service drift -guard), and the start-pending-vs-stop race; the relay's envelope round-trip, router routing + -join-order buffering + dev gate, and two end-to-end flows over a loopback relay; the sandbox -`?debug=wire` opt-in; and a codegen test asserting `WIRE_DECODE_TABLE` is emitted with one entry -per request/response frame id plus subscription start/receive. A Playwright e2e asserts both the -payload-blind panel under `?debug=wire` and the decoded typed-value view under `?debug=wire-decode`. -The suite is green (203 pass); `tsc -b` is clean, and the TrUAPI playground (which mounts the trace -panel) builds and lints clean. - -Validated end to end against the genuine Rust core run headless as WASM: a real `localStorage.read` -round trip observed under one `requestId` and forwarded verbatim through the debug host (the host -callback received the core's own namespaced key and the core's own response encoding, confirming it -is the real dispatcher, not a shim). The relay is also proven over a real cross-process socket: a -relay server, a debug host, and a product running as three separate OS processes, communicating -only over WebSockets, carried a call answered under one `requestId` end to end. Not yet exercised: -auth-gated methods (signing, needs a paired session) and a live in-browser playground run inside a -host. - -### Privacy - -Load-bearing and by design: the **default** observe surface carries no decoded payload and no key -material, so the seam is safe to run in production. The relay carries frames as opaque bytes, and -mocked responses are always marked (`tier: "mock"`). - -Decoded payloads exist **only** under the dev-gated `?debug=wire-decode` mode: `exposeFrameBytes` -attaches raw bytes and the consumer decodes them through `WIRE_DECODE_TABLE`. That mode is off by -default and dead-code-eliminable in a production build. This dev-gating is a hard requirement, not a -convenience: the raw wire can carry key material - the truapi#264 review found secret key material -reachable on the SSO response path - so the dev gate is the structural defense that keeps decoded -payloads out of production. The decoded mode is a developer tool and is not claimed to be safe to -run in production. - -### Compatibility and performance - -Purely additive. `observe` is a new optional field on `CreateTransportOptions`; `debug.ts`, -`debug-host.ts`, and `relay.ts` are new modules with new barrel exports. No existing interface -changes; no migration required. The `ObservedFrame` key-set is frozen (additive evolution only). -The observe hook is zero-cost when unset (a single short-circuit before any allocation); when set, -it allocates one small record per frame into an LRU-capped map, so memory is bounded regardless of -session length. - -### Out of scope - -No interactive UI beyond the playground trace panel (a host-panel bridge and a handler editor are -later); no host-side observe hook; no generated method-name constant (the runtime -`createMethodNameMap` is the source today). Decoded payloads are in scope, but only as the dev-gated -`?debug=wire-decode` mode - the default observe surface stays payload-blind and the core never -decodes. Relay session pairing (minting a short `sessionId`) and a dev-preview-server mount for the -reference relay are also deferred. - -## Drawbacks - -- **`requestId` is transport-local.** Each `createTransport` mints `p:1, p:2, …` from zero, so a - wire debugger / debug host instance is 1:1 with a transport; sharing one across transports would - collapse unrelated operations under the same id. Namespacing the trace key to aggregate multiple - transports is deferred until a shared-aggregation use case appears. -- **A second thing named "host."** The debug host adds a third artifact that reads as a - "headless/mock host" next to truapi#264 and truapi#294; the naming space is crowded. The - export keeps the name `createDebugHost`, and the distinction from those hosts is documented - above. -- **Byte-level mock entries are lower-level than a typed handler tree** - authoring a mock means - encoding bytes with the generated codecs, which is less ergonomic than a typed method handler - would be (a future surface concern, not a blocker). - -## Alternatives - -- **Host-specific debugger (rejected).** Tapping one host's internal hook (prior art: a dotli-only, - read-only panel) ties the tool to one host's unstable internals and cannot mock or forward. This - design hooks the protocol's own transport instead, so it is host-agnostic and active. -- **Frame rewriting instead of a dispatcher-level mock (deferred).** A mock could hand-rewrite SCALE - frames; instead, entries answer through the generated codecs, which keeps mocked frames - well-formed. Raw frame rewriting remains possible later at the relay layer. -- **A standalone debugger application (deferred).** A separate desktop app was considered; the - playground trace panel plus the relay cover the near-term need without a new app to ship. -- **Introducing a second correlation id (rejected).** Every layer keys on the transport-minted - `requestId`, which already appears on the wire and in the host dispatch context; a second id would - add plumbing with no benefit. +`createRelayProvider` carries frames over a WebSocket in a routing envelope; the relay routes by +`(sessionId, role)` and never parses a frame. Because it is a plain `WireProvider`, pointing a +product at a debug host in another process is a **provider swap** - transport and product code +untouched, and the same provider drops into a debug host's `provider`/`forward` slots. It is +**dev-gated**: `createRelayProvider` throws unless built with `TRUAPI_RELAY=1` or passed +`{ optIn: true }` (no silent fallback; a session that cannot reach its relay fails loudly). +`RelayRouter` is the transport-agnostic core (join-order-independent: frames arriving before the +counterpart joins are buffered - capped at 1024 per session, excess dropped with a one-time warning - +and flushed on join); `createLoopbackSocketFactory` runs a relay in-process, with no network hop, for +tests and single-tab use. + +Only `v: 1` envelopes are accepted; any other version fails to decode and the frame is dropped (the +envelope is versioned so the wire can evolve without silently mis-routing an unknown shape). A +carrier must call `RelayRouter.leave()` on disconnect - it drops the peer and, once the session is +empty, deletes the session and its pending buffer; `createRelayProvider().dispose()` closes the +socket and clears subscribers. + +## Enablement + +No product changes its call sites. The `@parity/truapi/sandbox` bootstrap - the shared transport +builder for browser-embedded products, including the playground - reads the embedding URL once at +module load (snapshotted, so a product that rewrites its own URL can't drop the flag before the +first `getClientSync()`). + +### The `?debug=` grammar + +Debug surfaces are selected by a single, extensible query key rather than a new flag per feature. A +debugger accretes modes; the grammar is built to accrete with it: + +```text +?debug=[:[:…]][,…] + + wire payload-blind observe seam + wire debugger (safe anywhere) + wire:decode + typed-value decode of each frame (build-gated; see below) +``` + +- **Channels** name a debug surface (`wire` today; a `relay` channel and others slot in without a + new query key). **Modifiers** are additive verbosity levels on a channel: `wire` is the safe + baseline; `wire:decode` raises it to expose payload bytes. +- **Composes:** `?debug=wire:decode,relay` opts several channels in at once. +- **Forward-compatible:** unknown channels and modifiers are ignored, not errors, so a newer link + opened against an older build degrades to whatever that build understands instead of failing. +- **Legacy alias:** `?debug=wire-decode` is accepted as `wire:decode`. + +Effect: `wire` installs a `createWireDebugger` on the transport's `observe` hook and exposes it via +`getWireDebugger()` and `window.__truapiWireDebugger__` (the playground renders its trace panel off +this). `wire:decode` additionally sets `exposeFrameBytes` **iff the `TRUAPI_WIRE_DECODE` build gate +is on**, so the panel decodes each frame through `WIRE_DECODE_TABLE`; otherwise it degrades to a +payload-blind trace. So enabling the payload-blind debugger is a **URL flag**, not a code change; +byte-level decoding additionally takes a build-time opt-in. + +## Privacy and security + +- **Payload-blind default is the boundary.** The default surface carries no decoded payload and no + key material, so it is safe in production. The relay carries frames as opaque bytes; mocked + responses are always marked `tier: "mock"`. +- **Byte exposure is build-gated, deliberately.** A URL cannot turn on `exposeFrameBytes` by itself - + which matters because the playground is a deployed site and dotli forwards unknown query params + through to the product iframe, so the URL is attacker-influenceable. The raw wire can carry key + material (the truapi#264 review found secret key material reachable on the SSO response path), so + the build gate is the structural defense that keeps decoded payloads out of production. The + decoded mode is a developer tool and is not claimed to be production-safe. +- **Residual metadata exposure (accepted).** `?debug=wire` publishes the debugger on + `window.__truapiWireDebugger__`, so any script already on the page can read the traces - but they + are shape-and-timing only, the same metadata already in devtools' network view, so this is + defense-in-depth against a third-party script, not a payload leak. And frame shape plus timing is + what traffic analysis works from: "safe in production" means it leaks no application *content*, not + that the metadata is zero-knowledge. + +## Compatibility + +Purely additive. `observe`/`exposeFrameBytes` are new optional fields on `CreateTransportOptions`; +`debug.ts`, `debug-host.ts`, and `relay.ts` are new modules with new barrel exports. No existing +interface changes; no migration. The observe hook is zero-cost when unset and, when set, allocates +one small record per frame into the doubly-capped map. + +## Non-goals and deferred work + +- **No interactive UI beyond the playground trace panel.** A host-panel bridge and a mock-handler + editor are later. +- **No host-side observe hook** in this design; the seam is client-transport-side only. +- **No generated method-name constant.** `createMethodNameMap` is the runtime source today. +- **`requestId` stays transport-local.** Each `createTransport` mints from zero, so a debugger/debug + host is 1:1 with a transport; cross-transport aggregation (namespacing the trace key) is deferred + until a use case appears. +- **Relay session pairing** (minting a short `sessionId`) and a dev-preview-server mount for a + reference relay are deferred. + +## Validation status + +What is automated versus proven by hand, so no claim here reads as more validated than it is: + +- **Automated in truapi#295** (the `js/packages/truapi/test/*.test.mjs` suite): correlation, + mock/forward/unhandled routing, dispose-time upstream stop, the relay envelope round-trip and the + loopback mock/forward flows, and the debugger driven over a real WebSocket carrier (a single + process with a real loopback socket, not separate OS processes). The playground e2e + (`playground/tests/e2e/wire-debug.spec.ts`) covers the `?debug=wire` and `?debug=wire:decode` + panel paths. +- **Verified manually, not yet automated:** end-to-end decode against the genuine Rust core running + headless as WASM (a `localStorage.read` observed under one `requestId` and forwarded verbatim + through the debug host). This was reproduced by hand; there is no WASM-core test in the package. +- **Not yet validated at all:** auth-gated methods (signing needs a paired session) and a live + in-browser playground run inside a host. + +## Alternatives considered + +- **Host-specific debugger (rejected).** Tapping one host's internal hook (prior art: a dotli-only + read-only panel) ties the tool to that host's unstable internals and cannot mock or forward. + Hooking the protocol's own transport is host-agnostic and active. +- **A second correlation id (rejected).** Every layer keys on the transport-minted `requestId`, + which already appears on the wire and in host dispatch; a second id is pure plumbing. +- **Frame rewriting instead of codec-backed mock entries (deferred).** Entries answer through the + generated codecs so mocked frames stay well-formed; raw rewriting remains possible later at the + relay layer. +- **A standalone debugger app (deferred).** The playground panel plus the relay cover the near-term + need without a new app to ship. ## References -- Implementation: truapi#295 (`js/packages/truapi/src/{client,debug,debug-host,relay,sandbox}.ts`) - and the TrUAPI playground trace panel. -- Peers in the "headless / mock host" space: the mock host (truapi#294) and the headless host - (truapi#264). -- Requirement source: the debugger tracker (sdk-team#26), which asks to see what a product sends - and what the host returns, "decoded to typed values" - the requirement the dev-gated - `?debug=wire-decode` view fulfills on top of the payload-blind seam. +- Implementation: truapi#295 + (`js/packages/truapi/src/{client,debug,debug-host,relay,sandbox}.ts`, the playground trace panel). + Automated coverage: `js/packages/truapi/test/*.test.mjs` and + `playground/tests/e2e/wire-debug.spec.ts` (scope as listed under Non-goals → Automated). +- Peers in the "headless / mock host" space: mock host (truapi#294), headless host (truapi#264). +- Requirement source: the debugger tracker (sdk-team#26). From 8238c0829a64a116e5baa0f0531a33fb27891e3c Mon Sep 17 00:00:00 2001 From: Nidish Date: Tue, 28 Jul 2026 15:54:55 +0530 Subject: [PATCH 4/8] docs(design): host-agnostic enablement and relay carrier --- docs/design/wire-observability-debug-host.md | 89 +++++++++++++------- 1 file changed, 60 insertions(+), 29 deletions(-) diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md index 84fb88355..17553d4a6 100644 --- a/docs/design/wire-observability-debug-host.md +++ b/docs/design/wire-observability-debug-host.md @@ -8,7 +8,7 @@ owner: "@decrypto21" | | | | ------------------ | ------------------------------------------------------------------------------- | | **Start Date** | 2026-07-25 | -| **Authors** | @decrypto21 | +| **Authors** | Nidish Ramakrishnan | | **Implementation** | truapi#295 | | **Description** | A payload-blind observe seam on the TrUAPI transport, a wire debugger, a mock/forward debug host, and a WebSocket relay - all correlated by the wire `requestId`. | @@ -225,6 +225,7 @@ host is truapi#264. The debug host sits *in front of* those and forwards *to* th function createRelayProvider(opts: { url: string; sessionId: string; productId: string; role: "product" | "host" | "debugger"; optIn?: boolean }): WireProvider; interface RelayEnvelope { v: 1; role: "product" | "host" | "debugger"; sessionId: string; productId: string; frame: Uint8Array; } class RelayRouter { join(sessionId, peer): void; leave(sessionId, peer): void; handleEnvelope(from, bytes): void; } +function connectRelaySocket(router: RelayRouter, socket: { send(bytes: Uint8Array): void }): { message(bytes: Uint8Array): void; close(): void }; ``` `createRelayProvider` carries frames over a WebSocket in a routing envelope; the relay routes by @@ -239,44 +240,74 @@ and flushed on join); `createLoopbackSocketFactory` runs a relay in-process, wit tests and single-tab use. Only `v: 1` envelopes are accepted; any other version fails to decode and the frame is dropped (the -envelope is versioned so the wire can evolve without silently mis-routing an unknown shape). A -carrier must call `RelayRouter.leave()` on disconnect - it drops the peer and, once the session is -empty, deletes the session and its pending buffer; `createRelayProvider().dispose()` closes the -socket and clears subscribers. +envelope is versioned so the wire can evolve without silently mis-routing an unknown shape). +`connectRelaySocket(router, socket)` is the shipped, runtime-agnostic carrier core: a relay server +is a WS loop that calls `.message(bytes)` per frame and `.close()` on disconnect (which calls +`RelayRouter.leave()`, dropping the peer and deleting the session + its pending buffer once empty), +so the same routing runs under `Bun.serve`, `ws`, a dev preview server, or a `truapi-host` +subcommand; `createRelayProvider().dispose()` closes the client socket and clears subscribers. ## Enablement -No product changes its call sites. The `@parity/truapi/sandbox` bootstrap - the shared transport -builder for browser-embedded products, including the playground - reads the embedding URL once at -module load (snapshotted, so a product that rewrites its own URL can't drop the flag before the -first `getClientSync()`). +Enablement is a **host-side dev capability, not a URL** - the seam and the relay are transport-level +and URL-independent, so the debugger turns on the same way whether the product runs in a browser +iframe, a desktop webview, or a mobile webview (sdk-team#26). No product changes its call sites; the +`@parity/truapi/sandbox` bootstrap reads the config once at module load (snapshotted, so a product +rewriting its own URL can't drop it before the first `getClientSync()`) and wires the transport +accordingly. + +### Two enablement sources, one config + +The bootstrap merges two sources so no host shape is left out (`resolveDebugConfig`): + +| Host | How it's turned on | +|---|---| +| **Browser / iframe** (e.g. the playground) | the `?debug=` query on the embedding URL - dotli forwards unknown query params into the product iframe | +| **Native / webview** (no address bar) | the host's dev build injects `window.__truapiDebug__` before the product boots - the same host-injection path already used for `window.__HOST_API_PORT__` | + +```ts +// native host, dev build only - a production host must not set this +window.__truapiDebug__ = { + debug: "wire:decode", // same grammar as ?debug= + relay: { url: "ws://127.0.0.1:5199/relay", sessionId: "abc" }, // optional - see "Routing at a relay" +}; +``` ### The `?debug=` grammar -Debug surfaces are selected by a single, extensible query key rather than a new flag per feature. A -debugger accretes modes; the grammar is built to accrete with it: +This is the config value shared by both sources above (the URL query and the injected global's +`debug` field): ```text -?debug=[:[:…]][,…] - - wire payload-blind observe seam + wire debugger (safe anywhere) - wire:decode + typed-value decode of each frame (build-gated; see below) +wire payload-blind observe seam + wire debugger (safe anywhere) +wire:decode + typed-value decode of each frame (build-gated) +[:…][,…] (extensible, composable) ``` -- **Channels** name a debug surface (`wire` today; a `relay` channel and others slot in without a - new query key). **Modifiers** are additive verbosity levels on a channel: `wire` is the safe - baseline; `wire:decode` raises it to expose payload bytes. -- **Composes:** `?debug=wire:decode,relay` opts several channels in at once. -- **Forward-compatible:** unknown channels and modifiers are ignored, not errors, so a newer link - opened against an older build degrades to whatever that build understands instead of failing. -- **Legacy alias:** `?debug=wire-decode` is accepted as `wire:decode`. - -Effect: `wire` installs a `createWireDebugger` on the transport's `observe` hook and exposes it via -`getWireDebugger()` and `window.__truapiWireDebugger__` (the playground renders its trace panel off -this). `wire:decode` additionally sets `exposeFrameBytes` **iff the `TRUAPI_WIRE_DECODE` build gate -is on**, so the panel decodes each frame through `WIRE_DECODE_TABLE`; otherwise it degrades to a -payload-blind trace. So enabling the payload-blind debugger is a **URL flag**, not a code change; -byte-level decoding additionally takes a build-time opt-in. +`wire` installs `createWireDebugger` on the transport's `observe` hook (exposed via +`getWireDebugger()` / `window.__truapiWireDebugger__`); `wire:decode` additionally sets +`exposeFrameBytes` **iff the `TRUAPI_WIRE_DECODE` build gate is on**, else it degrades to a +payload-blind trace. Unknown channels/modifiers are ignored (forward-compatible), and `wire-decode` +is a legacy alias for `wire:decode`. + +### Routing at a relay (the host-agnostic path) + +This is sdk-team#26's core mechanism and the answer for native hosts. When the injected config +carries a `relay` endpoint - and the relay build gate (`TRUAPI_RELAY=1` / +`NEXT_PUBLIC_TRUAPI_RELAY=1`) is on - the bootstrap points the product's transport at +`createRelayProvider(...)` instead of the host channel. The product's frames then flow to a debugger +process (a standalone app, a CLI, or a panel) over the relay, which forwards to the real host or +mocks - all without an address bar and without touching the product. This is the **C8 "debuggable +host" contract**: a host is debuggable iff it exposes the observe hook and can point its product's +transport at a relay provider. + +**Scope, precisely:** every TrUAPI-side piece the native path needs is shipped here - the product +hook (`sandbox` reads `window.__truapiDebug__`), the transport-at-relay swap, the reusable relay +carrier (`connectRelaySocket`), and the debug host. So a native host's integration is a config swap, +not a fork: its dev build sets `window.__truapiDebug__` (a one-liner, exactly like the +`window.__HOST_API_PORT__` it already injects) and stands up a relay with `connectRelaySocket` in +whatever WS runtime it runs. That last step is the native app's own code, in its own repo - the only +piece not in this PR, because it is not TrUAPI's to write, not because it is unbuilt. ## Privacy and security From fc9de406f3daf36a74a0ddc2a697fbe2b4a2a3c0 Mon Sep 17 00:00:00 2001 From: Nidish Date: Wed, 29 Jul 2026 00:30:27 +0530 Subject: [PATCH 5/8] docs(design): wire observability + debug host spec --- docs/design/wire-observability-debug-host.md | 494 ++++++------------- 1 file changed, 149 insertions(+), 345 deletions(-) diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md index 17553d4a6..3153790ac 100644 --- a/docs/design/wire-observability-debug-host.md +++ b/docs/design/wire-observability-debug-host.md @@ -9,377 +9,181 @@ owner: "@decrypto21" | ------------------ | ------------------------------------------------------------------------------- | | **Start Date** | 2026-07-25 | | **Authors** | Nidish Ramakrishnan | -| **Implementation** | truapi#295 | -| **Description** | A payload-blind observe seam on the TrUAPI transport, a wire debugger, a mock/forward debug host, and a WebSocket relay - all correlated by the wire `requestId`. | - -This is a **specification of the contract**, not a walkthrough of the implementation: it fixes the -interfaces, the observable behavior, and the invariants each layer must uphold. Implementation -detail, tests, and file layout live in truapi#295; where the two disagree, this document is the -intent and the code is the bug. - -## Requirements - -Every TrUAPI operation crosses the wire as an opaque frame `{ requestId, payload: { id, value } }`. -When one misbehaves - a wrong response, a subscription that never delivers, a silently dropped -frame - the question is *"what did this put on the wire, and where did it stall?"* Answering it -today means hand-decoding SCALE or forking the transport to add logging. This design must: - -1. **Observe every frame without decoding it.** Shape and timing are visible by default; payloads - and key material are not, so the same recorder is safe to run in production. -2. **Use one correlation id.** An operation is followable end to end - wire trace and host dispatch - under a single id - with no second correlation scheme introduced. -3. **Be host-agnostic.** The seam sits on the protocol's own transport, not on any host's - internals, so it works against any host that speaks TrUAPI. -4. **Be active, not only passive.** A developer can answer a frame with a scripted response (to - develop against behavior that doesn't exist yet, or reproduce an error path) or forward it - verbatim to a real host - without modifying that host. - -## Model - -**One payload-blind seam emits an `ObservedFrame` per wire frame, keyed on the transport-minted -`requestId`; a wire debugger groups those into traces; a debug host answers or forwards frames on -the same id.** The transport mints `requestId` (`p:1`, `p:2`, …) when a product starts an operation -and stamps it on every frame; the same id appears on the wire envelope and in the host's -`CallContext.requestId`. No second correlation id exists. +| **Implementation** | truapi#295 (Rust tap in `truapi-server`) | +| **Description** | A payload-blind tap in the Rust host that streams every product↔host frame, under one correlation id, to a debugger app the host dials out to. | + +This is a **specification of the contract**, not a walkthrough of the implementation: +it fixes where the tap lives, the topology, the event shape, and the invariants each +layer upholds. Where this doc and the code disagree, this doc is the intent. + +## Where the tap lives + +The tap is in the **Rust host (`truapi-server`)**, behind a sink trait — **not** in the +TypeScript transport and **not** turned on from the product side. `truapi-server` has +exactly two frame choke points, and every host (web via `wasm.rs`, native via a +`ws_bridge` `FrameSink`) funnels through them: + +| Direction | Choke point | +|---|---| +| inbound (product → core) | `ProductRuntime::receive_frame()` — `host_core.rs` | +| outbound (core → product) | `FrameSink::emit_frame()` / `SinkTransport::send()` — `host_core.rs` | + +One tap implementation covers both platforms, and **nothing in `@parity/truapi` changes** +— the product is genuinely untouched. That the product package does not change at all is the +test for the tap being in the right place: a seam in the product transport would fail it. + +## Topology: the host dials the debugger + +The **debugger app is a WS server** on the dev machine; **every host dials outward** to it. +This is forced, not a preference — only the inside can initiate a connection: + +- native: `ws_bridge` binds `127.0.0.1` on the device; a debugger on a dev machine cannot + dial into it (the iOS Simulator "works" only because it shares the Mac's network stack — + an accident of the simulator, not a design); +- web: nothing outside the browser can dial into a Worker. ``` - product code transport wire debugger host dispatch - │ getAccount() │ │ │ - │────────────────▶│ mint requestId = p:1 │ │ - │ │─ observe(out, p:1) ───▶│ push → WireTrace │ - │ │─ frame{ p:1 } ───────────────────────────▶ │ ctx.requestId=p:1 - │ │◀ frame{ p:1 } ─────────────────────────────│ - │ │─ observe(in, p:1) ────▶│ push → WireTrace │ - │◀ Ok(response) ──│ │ │ + WEB NATIVE / DESKTOP + ┌─ browser ──────────────────┐ ┌─ device ───────────────────┐ + │ product (iframe) │ │ product (webview) │ + │ │ MessagePort │ │ │ ws loopback │ + │ ▼ │ │ ▼ │ + │ host worker ── tap │ │ ws_bridge ── tap │ + │ │ rust wasm │ │ │ rust core │ + └──────┼─────────────────────┘ └──────┼─────────────────────┘ + │ ws, dialed outward │ ws, dialed outward + └──────────────┬─────────────────────────┘ + ▼ + ┌──────────────────────┐ + │ debugger app │ ws server on the dev machine + └──────────────────────┘ ``` -## Interface +There is **no relay service** — the debugger app is the server, so the envelope carries no +routing metadata (no session/role/product fields, no join-order buffering). It is just: -The signatures below are the normative surface, all in `@parity/truapi`. +``` + host → debugger { channelId, dir: "out" | "in", frame: bytes } +``` -### Transport observe seam +### One call, end to end -```ts -function createTransport(provider: WireProvider, options?: CreateTransportOptions): TrUApiTransport; -interface CreateTransportOptions { - codecVersion?: number; - observe?: TransportObserver; // emit-only; the whole seam - exposeFrameBytes?: boolean; // dev-gated; see "Decoded view" -} -type TransportObserver = (frame: ObservedFrame) => void; - -interface ObservedFrame { - direction: "out" | "in"; // out = sent by this transport, in = received - requestId: string; // the one correlation id, e.g. "p:1" - frameId: number; // wire-table discriminant, e.g. 22 - role: FrameRole; // request|response|start|receive|interrupt|stop|handshake|malformed|unknown - byteLength: number; // encoded SCALE length - shape only - timestamp: number; // epoch ms - bytes?: Uint8Array; // present only when exposeFrameBytes is set -} +``` + product host edge rust core debugger app +(iframe / webview) (tap) (remote) + │ getAccount() │ │ │ + ├─ frame{ p:1, id=22 } ─▶ │ │ + │ ├┈┈ { dir:out, frame } ┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈▶│ log ▲ out p:1 + │ ├ delivered immediately ▶ ctx.requestId="p:1" │ + │ ◀─ frame{ p:1, id=23 } ─┤ │ + ◀ delivered immediately ┤ │ │ + │ Ok(response) ├┈┈ { dir:in, frame } ┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈▶│ log ▼ in p:1 ``` -Guarantees (enforced in `client.ts`): - -- **Payload-blind by default.** `ObservedFrame` carries shape and timing only; `byteLength` is read - off the encoded bytes without decoding them. The key-set is frozen (additive evolution only). -- **Causally ordered.** The outbound frame is observed *before* `provider.postMessage`, so a - request precedes its responses even over a synchronous provider. -- **Failure-isolated.** A throwing observer is swallowed; it can never break the message loop. -- **Zero-cost when unset.** The notify path short-circuits before allocating anything. -- **Corrupt inbound frames are recorded.** An *inbound* frame that fails envelope decode surfaces as - a `malformed` observed frame (sentinel `requestId`/`frameId`; under `exposeFrameBytes` it also - carries the raw envelope `bytes`, which is what you want when diagnosing the corruption) before the - transport closes, so the trace never goes dark. Malformed payload *values* are not seen here - the - seam never decodes values. This covers the inbound path only: an *outbound* frame that fails to - encode closes the transport before the observe hook runs, so it is not recorded. - -### Wire debugger - -```ts -function createWireDebugger(options?: WireDebuggerOptions): WireDebugger; -interface WireDebuggerOptions { - sink?: (line: string, frame: ObservedFrame) => void; // formatted line + its frame; defaults to console.debug - forward?: TransportObserver; // a second observer (e.g. onward to a panel) - maxTraces?: number; // LRU cap on trace count, default 256 - maxFramesPerTrace?: number; // ring-buffer cap on frames within one trace, default 1024 - methodNames?: ReadonlyMap; -} -interface WireDebugger { - readonly observe: TransportObserver; // hand to createTransport({ observe }) - traces(): WireTrace[]; - trace(requestId: string): WireTrace | undefined; - clear(): void; +`emit` is fire-and-forget in both legs. **Outbound** the tap delivers to the product +first, then emits; **inbound** it emits before dispatch, so an undecodable frame is still +observed. The order differs; the off-the-critical-path guarantee does not (see Invariants). + +## Invariants + +1. **In the path, not in the critical path.** The tap carries every frame, so it sees them + all, yet a slow, absent, or crashed debugger can never stall a session — only the trace is + lost. The ordering is **asymmetric by design**: outbound (core → product) the tap forwards + to the product first, then emits; inbound (product → core) it emits first, before + decode/dispatch, so an undecodable frame is still observed. The guarantee holds either way + because of invariant 2, not because of a single fixed ordering. +2. **Emit, never wait.** `emit` is fire-and-forget: no reply is read from the debugger socket, + and it must never block or fail the frame path. That is what keeps the tap off the critical + path wherever in each leg it fires. + +**The second invariant is also the extension point.** Mocking/mutation later needs no +topology or envelope change — the tap starts *reading a reply* on the frames it cares about: +deliver unchanged is today's behaviour, deliver modified is mutation, respond is a mock. +This is why the one-way version ships first. + +## Interface (`truapi-server`) + +The tap is a **sink trait**, not a hardcoded socket — both to keep the WS transport out of +the core, and because wire frames aren't the only thing worth observing (host-internal +events like SSO have no frame to hang off). The enum leaves room for them. + +```rust +/// Dev-only sink for host debug events. Unset ⇒ the tap is inert. +pub trait DebugSink: Send + Sync { + /// Fire-and-forget: must not block the frame path or fail the operation. + fn emit(&self, event: DebugEvent); } -interface WireTrace { requestId: string; frames: ObservedFrame[]; startedAt: number; lastAt: number; } -function createMethodNameMap(table: Record, services: readonly string[]): ReadonlyMap; -interface WireMethodInfo { method: string; kind: WireFrameKind; } -``` +pub struct ChannelId(pub String); -Behavior: - -- Frames group into a `WireTrace` by `requestId`. Traces are held in an insertion-ordered map, - **LRU-capped at `maxTraces`**; each trace's `frames` are **ring-buffered at `maxFramesPerTrace`**, - so a long-lived subscription (whose frames all share one `requestId` that never LRU-evicts) cannot - grow without bound. Memory is therefore bounded on both axes regardless of session length. -- `sink` and `forward` are each isolated in `try/catch`. -- `createMethodNameMap` inverts the generated wire-table into `frameId → { method, kind }`, - resolving the longest service prefix first (`LOCAL_STORAGE_READ → localStorage.read`, not - `local.storageRead`). It is the runtime source of readable names; a generated constant is a - non-goal (below). A trace reads: - - ```text - [wire p:1] → request account.getAccount (id=22, 14B) - [wire p:1] ← response account.getAccount (id=23, 35B) - ``` - -### Decoded view (dev-gated) - -The payload-blind seam is the safe foundation; the decoded view lets a developer see typed -request/response values. It is a second layer on top of the seam, and the core never decodes - -decoding is a consumer concern. - -- **`exposeFrameBytes`** attaches each frame's raw SCALE `bytes` to its `ObservedFrame`. Without it, - frames stay shape-and-timing only. -- **`WIRE_DECODE_TABLE`** (`@parity/truapi/wire-decode`) is a codegen-emitted - `Record unknown>` with one entry per request/response id plus - subscription start/receive. Decoding is always against the generated client - the same source of - truth as the wire codecs - so a decoded value cannot drift from the wire schema. -- **`?debug=wire:decode`** (see [the grammar](#the-debug-grammar)) requests the decoded view, but is - **build-gated**: it enables `exposeFrameBytes` only when the bundle was built with - `TRUAPI_WIRE_DECODE=1` (`NEXT_PUBLIC_TRUAPI_WIRE_DECODE=1` for the Next-built playground, so webpack - inlines it), mirroring the relay's `TRUAPI_RELAY` gate. Without the build flag it degrades to a - plain payload-blind `?debug=wire` trace. The production playground deploy leaves the gate unset. - -**Bundle isolation.** The decode table is **not** statically imported. The playground trace panel -`import()`s `@parity/truapi/wire-decode` lazily, and only when a frame actually carries `bytes`. Under -`next dev` this code-splits into a separate chunk; the production static export forces a single chunk -(`splitChunks:false` + `maxChunks:1`), so there the module lands in the main bundle but its factory -never runs in payload-blind mode - the gate keeps it out of the *execution path*, not the bundle. The -table is non-sensitive generated codecs, and byte exposure is separately gated, so this is a -bundle-hygiene note, not a security boundary. - -### Debug host - -A `WireProvider`-shaped man-in-the-middle: it answers frames it is scripted to claim and forwards -the rest verbatim to a real host. - -```ts -function createWireDebugHost(options: CreateWireDebugHostOptions): WireDebugHost; -interface CreateWireDebugHostOptions { - provider: WireProvider; // the product side - entries?: readonly WireDebugHostEntry[]; // mock entries, each claiming its wire ids - forward?: WireProvider; // optional pipe to a real host - observe?: TransportObserver; // payload-blind, host vantage - onDecision?: (d: WireDebugHostDecision) => void; -} -interface WireDebugHost { dispose(): void; } +pub enum FrameDirection { In, Out } -interface DebugRequestEntry { - readonly kind: "request"; - readonly ids: RequestFrameIds; // { request, response } - handle(ctx: DebugCallContext, payload: Uint8Array): Uint8Array | Promise; +#[non_exhaustive] // room for non-frame events (e.g. SSO); adding one is not breaking +pub enum DebugEvent { + /// A SCALE wire frame crossing a product channel. + Frame { channel_id: ChannelId, dir: FrameDirection, bytes: Vec }, } -interface DebugSubscriptionEntry { - readonly kind: "subscription"; - readonly ids: SubscriptionFrameIds; // { start, receive, interrupt, stop } - start(ctx: DebugCallContext, payload: Uint8Array, port: DebugSubscriptionPort): DebugSubscriptionCleanup | Promise; -} -type WireDebugHostEntry = DebugRequestEntry | DebugSubscriptionEntry; -interface WireDebugHostDecision { tier: "mock" | "forward" | "unhandled"; method?: string; frame: ObservedFrame; } ``` -Behavior: - -- **Entries are byte-level, keyed by wire id** - a flat list, not a nested handler tree, and there - is no internal loopback dispatcher. `handle` takes and returns raw SCALE bytes. "A mock cannot - emit a malformed frame" holds by the *convention* that the caller encodes answers with the - generated codecs (`encodeWireMessage`/`decodeWireMessage`), not by the type of `handle`. Two - entries claiming the same inbound (`request`/`start`) wire id is a construction error: - `createWireDebugHost` throws rather than silently letting one shadow the other. -- **Routing.** A claiming request/subscription entry answers (`tier: "mock"`); a claimed `stop` is - terminal. Otherwise, with a `forward` pipe set the frame travels it **byte-verbatim, `requestId` - untouched** (`tier: "forward"`) and the answer relays back; with no forward pipe it surfaces - loudly as `tier: "unhandled"` (via `onDecision`, or a `console.warn` when no listener is - set - so the stall is loud rather than silent; the caller still hangs, as no answer is sent). -- **Every frame is marked** via `onDecision`, so a scripted answer is never silently mistaken for - real host behavior. -- **Errors are loud, never a silent wrong answer.** A mock `handle`/`start` that throws, or a - response that fails to encode, emits a `console.warn` and sends nothing - the caller hangs loudly, - the same policy as `unhandled`. An undecodable *inbound* envelope is forwarded byte-transparent - when a `forward` pipe is set, or `console.warn`ed when headless - the host-side analogue of the - transport's `malformed` recording. (Repeated unhandled frames for one wire id warn once.) -- **Teardown.** `dispose()` sends `stop` upstream for any live forwarded subscription (so the real - host stops streaming into a detached pipe) and runs cleanup for every live *mock* subscription, so - a torn-down session leaks nothing on either side. - -This is an **observability seam, not a test host.** With no forward pipe it answers scripted bytes -with no core behind it (no dispatch, permissions, or storage) - a debugging convenience, not a -fidelity tier. The deterministic testing tier is the mock host (truapi#294); the canonical headless -host is truapi#264. The debug host sits *in front of* those and forwards *to* them. - -### Relay - -```ts -function createRelayProvider(opts: { url: string; sessionId: string; productId: string; role: "product" | "host" | "debugger"; optIn?: boolean }): WireProvider; -interface RelayEnvelope { v: 1; role: "product" | "host" | "debugger"; sessionId: string; productId: string; frame: Uint8Array; } -class RelayRouter { join(sessionId, peer): void; leave(sessionId, peer): void; handleEnvelope(from, bytes): void; } -function connectRelaySocket(router: RelayRouter, socket: { send(bytes: Uint8Array): void }): { message(bytes: Uint8Array): void; close(): void }; -``` +- Installed per product channel via `ProductRuntime::set_debug_sink(channel_id, sink)`; + `None` by default, so production pays nothing. +- The concrete sink (the outward WS dial) is provided by the **host adapter**, not the core: + web bridges `emit` to a JS `WebSocket`, native to a `ws_bridge` `URLSession`/OkHttp socket. +- `bytes` are the untouched `ProtocolMessage`; **the debugger app decodes, the core never does.** -`createRelayProvider` carries frames over a WebSocket in a routing envelope; the relay routes by -`(sessionId, role)` and never parses a frame. Because it is a plain `WireProvider`, pointing a -product at a debug host in another process is a **provider swap** - transport and product code -untouched, and the same provider drops into a debug host's `provider`/`forward` slots. It is -**dev-gated**: `createRelayProvider` throws unless built with `TRUAPI_RELAY=1` or passed -`{ optIn: true }` (no silent fallback; a session that cannot reach its relay fails loudly). -`RelayRouter` is the transport-agnostic core (join-order-independent: frames arriving before the -counterpart joins are buffered - capped at 1024 per session, excess dropped with a one-time warning - -and flushed on join); `createLoopbackSocketFactory` runs a relay in-process, with no network hop, for -tests and single-tab use. - -Only `v: 1` envelopes are accepted; any other version fails to decode and the frame is dropped (the -envelope is versioned so the wire can evolve without silently mis-routing an unknown shape). -`connectRelaySocket(router, socket)` is the shipped, runtime-agnostic carrier core: a relay server -is a WS loop that calls `.message(bytes)` per frame and `.close()` on disconnect (which calls -`RelayRouter.leave()`, dropping the peer and deleting the session + its pending buffer once empty), -so the same routing runs under `Bun.serve`, `ws`, a dev preview server, or a `truapi-host` -subcommand; `createRelayProvider().dispose()` closes the client socket and clears subscribers. - -## Enablement - -Enablement is a **host-side dev capability, not a URL** - the seam and the relay are transport-level -and URL-independent, so the debugger turns on the same way whether the product runs in a browser -iframe, a desktop webview, or a mobile webview (sdk-team#26). No product changes its call sites; the -`@parity/truapi/sandbox` bootstrap reads the config once at module load (snapshotted, so a product -rewriting its own URL can't drop it before the first `getClientSync()`) and wires the transport -accordingly. - -### Two enablement sources, one config - -The bootstrap merges two sources so no host shape is left out (`resolveDebugConfig`): - -| Host | How it's turned on | -|---|---| -| **Browser / iframe** (e.g. the playground) | the `?debug=` query on the embedding URL - dotli forwards unknown query params into the product iframe | -| **Native / webview** (no address bar) | the host's dev build injects `window.__truapiDebug__` before the product boots - the same host-injection path already used for `window.__HOST_API_PORT__` | - -```ts -// native host, dev build only - a production host must not set this -window.__truapiDebug__ = { - debug: "wire:decode", // same grammar as ?debug= - relay: { url: "ws://127.0.0.1:5199/relay", sessionId: "abc" }, // optional - see "Routing at a relay" -}; -``` +## Configuration and transport -### The `?debug=` grammar +- The debugger **URL is configurable**, not loopback-derived: with the app on a device and + the debugger on a Mac, `127.0.0.1` is wrong and the link is LAN. +- The LAN hop is why **`wss://`** is right: it keeps iOS `Info.plist` `NSAppTransportSecurity` + empty (TLS is already required and `wss` satisfies it, no exception needed), and the trace + doesn't cross the office network in the clear. +- The cost moves to **certificates**: iOS still evaluates trust, so a self-signed debugger cert + is rejected until its CA is installed and enabled under **Settings → General → About → + Certificate Trust Settings**. This is a named setup step — "use wss" is not free. -This is the config value shared by both sources above (the URL query and the injected global's -`debug` field): +## Privacy and security -```text -wire payload-blind observe seam + wire debugger (safe anywhere) -wire:decode + typed-value decode of each frame (build-gated) -[:…][,…] (extensible, composable) -``` +- Frames stream as **opaque bytes**; the core never decodes them, so nothing at the tap + reads application content or key material. Decoding happens only in the debugger app. +- **Dev-only**, off in production (the sink is unset). The debugger app is on a trusted dev + machine; `wss` keeps the LAN hop encrypted. -`wire` installs `createWireDebugger` on the transport's `observe` hook (exposed via -`getWireDebugger()` / `window.__truapiWireDebugger__`); `wire:decode` additionally sets -`exposeFrameBytes` **iff the `TRUAPI_WIRE_DECODE` build gate is on**, else it degrades to a -payload-blind trace. Unknown channels/modifiers are ignored (forward-compatible), and `wire-decode` -is a legacy alias for `wire:decode`. - -### Routing at a relay (the host-agnostic path) - -This is sdk-team#26's core mechanism and the answer for native hosts. When the injected config -carries a `relay` endpoint - and the relay build gate (`TRUAPI_RELAY=1` / -`NEXT_PUBLIC_TRUAPI_RELAY=1`) is on - the bootstrap points the product's transport at -`createRelayProvider(...)` instead of the host channel. The product's frames then flow to a debugger -process (a standalone app, a CLI, or a panel) over the relay, which forwards to the real host or -mocks - all without an address bar and without touching the product. This is the **C8 "debuggable -host" contract**: a host is debuggable iff it exposes the observe hook and can point its product's -transport at a relay provider. - -**Scope, precisely:** every TrUAPI-side piece the native path needs is shipped here - the product -hook (`sandbox` reads `window.__truapiDebug__`), the transport-at-relay swap, the reusable relay -carrier (`connectRelaySocket`), and the debug host. So a native host's integration is a config swap, -not a fork: its dev build sets `window.__truapiDebug__` (a one-liner, exactly like the -`window.__HOST_API_PORT__` it already injects) and stands up a relay with `connectRelaySocket` in -whatever WS runtime it runs. That last step is the native app's own code, in its own repo - the only -piece not in this PR, because it is not TrUAPI's to write, not because it is unbuilt. +## Hosts and scope -## Privacy and security +Every host taps at its `truapi-server` choke points and dials the debugger app: **dotli** +(web), **Polkadot Desktop**, **iOS**, **Android**. Today those hosts run the +`@novasamatech/host-*` stack, so each adopts this once it runs `truapi-server`; the tap and +envelope are identical across all of them. Each host supplies its own outward-dial sink for +its platform — web bridges `emit` to a JS `WebSocket` (built); native binds a `ws_bridge` +`URLSession`/OkHttp WebSocket — dev-gated and configured with the debugger URL. Pinning the +provisional host→debugger framing (base64-in-JSON) is a prerequisite before the native sinks land. + +## Non-goals -- **Payload-blind default is the boundary.** The default surface carries no decoded payload and no - key material, so it is safe in production. The relay carries frames as opaque bytes; mocked - responses are always marked `tier: "mock"`. -- **Byte exposure is build-gated, deliberately.** A URL cannot turn on `exposeFrameBytes` by itself - - which matters because the playground is a deployed site and dotli forwards unknown query params - through to the product iframe, so the URL is attacker-influenceable. The raw wire can carry key - material (the truapi#264 review found secret key material reachable on the SSO response path), so - the build gate is the structural defense that keeps decoded payloads out of production. The - decoded mode is a developer tool and is not claimed to be production-safe. -- **Residual metadata exposure (accepted).** `?debug=wire` publishes the debugger on - `window.__truapiWireDebugger__`, so any script already on the page can read the traces - but they - are shape-and-timing only, the same metadata already in devtools' network view, so this is - defense-in-depth against a third-party script, not a payload leak. And frame shape plus timing is - what traffic analysis works from: "safe in production" means it leaks no application *content*, not - that the metadata is zero-knowledge. - -## Compatibility - -Purely additive. `observe`/`exposeFrameBytes` are new optional fields on `CreateTransportOptions`; -`debug.ts`, `debug-host.ts`, and `relay.ts` are new modules with new barrel exports. No existing -interface changes; no migration. The observe hook is zero-cost when unset and, when set, allocates -one small record per frame into the doubly-capped map. - -## Non-goals and deferred work - -- **No interactive UI beyond the playground trace panel.** A host-panel bridge and a mock-handler - editor are later. -- **No host-side observe hook** in this design; the seam is client-transport-side only. -- **No generated method-name constant.** `createMethodNameMap` is the runtime source today. -- **`requestId` stays transport-local.** Each `createTransport` mints from zero, so a debugger/debug - host is 1:1 with a transport; cross-transport aggregation (namespacing the trace key) is deferred - until a use case appears. -- **Relay session pairing** (minting a short `sessionId`) and a dev-preview-server mount for a - reference relay are deferred. +- No tap in `@parity/truapi` / the TS transport (the whole point of putting it in the core). +- No relay service (the debugger app is the server). +- No mocking/mutation in v1 — but the envelope and topology already accommodate it. ## Validation status -What is automated versus proven by hand, so no claim here reads as more validated than it is: - -- **Automated in truapi#295** (the `js/packages/truapi/test/*.test.mjs` suite): correlation, - mock/forward/unhandled routing, dispose-time upstream stop, the relay envelope round-trip and the - loopback mock/forward flows, and the debugger driven over a real WebSocket carrier (a single - process with a real loopback socket, not separate OS processes). The playground e2e - (`playground/tests/e2e/wire-debug.spec.ts`) covers the `?debug=wire` and `?debug=wire:decode` - panel paths. -- **Verified manually, not yet automated:** end-to-end decode against the genuine Rust core running - headless as WASM (a `localStorage.read` observed under one `requestId` and forwarded verbatim - through the debug host). This was reproduced by hand; there is no WASM-core test in the package. -- **Not yet validated at all:** auth-gated methods (signing needs a paired session) and a live - in-browser playground run inside a host. - -## Alternatives considered - -- **Host-specific debugger (rejected).** Tapping one host's internal hook (prior art: a dotli-only - read-only panel) ties the tool to that host's unstable internals and cannot mock or forward. - Hooking the protocol's own transport is host-agnostic and active. -- **A second correlation id (rejected).** Every layer keys on the transport-minted `requestId`, - which already appears on the wire and in host dispatch; a second id is pure plumbing. -- **Frame rewriting instead of codec-backed mock entries (deferred).** Entries answer through the - generated codecs so mocked frames stay well-formed; raw rewriting remains possible later at the - relay layer. -- **A standalone debugger app (deferred).** The playground panel plus the relay cover the near-term - need without a new app to ship. +- **Automated (Rust tap):** both choke points tapped, `channel_id` per event, bytes untouched and + in order — proven by `cargo test` (`debug_sink_taps_frames_in_both_directions`); the product-vantage + direction convention (`out` = left the product) by `frame_direction_wire_str_is_product_vantage`. + Inert-when-unset is structural (the `has_debug` fast path skips the tap when no sink is installed); + off-the-critical-path follows from invariant 2 — `emit`'s result is discarded and never awaited, and + the sink contract forbids blocking or panicking. +- **Automated (debugger app):** the WS server end (`@parity/truapi-debugger`, in-repo) — a host + dials in, streams a frame, the server decodes + groups it and serves it on `/traces` — proven by + its `bun test` against a live server. The exact host→debugger framing (base64-in-JSON today) is + still provisional pending the envelope spec. +- **Built (web outward dial):** the web host's sink — `WasmDebugSink` bridging `emit` to a JS + `WebSocket`, dev-gated via a `localStorage` debugger URL that the host worker reads — is wired end + to end, and a product running under the `@parity/truapi-host` WASM host streams every frame to the + debugger, verified against a local host harness. +- **Not yet built:** the native outward-dial sink (`ws_bridge` `URLSession`/OkHttp) and the + `wss`/cert setup. These are the native-integration tracks. ## References - -- Implementation: truapi#295 - (`js/packages/truapi/src/{client,debug,debug-host,relay,sandbox}.ts`, the playground trace panel). - Automated coverage: `js/packages/truapi/test/*.test.mjs` and - `playground/tests/e2e/wire-debug.spec.ts` (scope as listed under Non-goals → Automated). -- Peers in the "headless / mock host" space: mock host (truapi#294), headless host (truapi#264). +- Implementation: truapi#295 (`rust/crates/truapi-server/src/host_core.rs` — `DebugSink`/`DebugEvent`/tap). - Requirement source: the debugger tracker (sdk-team#26). From 367a3e2325b549ea40e4ff5e0762946a81759e7a Mon Sep 17 00:00:00 2001 From: Nidish Date: Mon, 3 Aug 2026 01:17:19 +0530 Subject: [PATCH 6/8] docs(design): specify host-dial transport, native sink deployment, and tracking --- docs/design/wire-observability-debug-host.md | 39 ++++---------------- 1 file changed, 8 insertions(+), 31 deletions(-) diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md index 3153790ac..c600c6601 100644 --- a/docs/design/wire-observability-debug-host.md +++ b/docs/design/wire-observability-debug-host.md @@ -1,8 +1,3 @@ ---- -title: "Wire Observability and Debug Host" -owner: "@decrypto21" ---- - # Wire Observability and Debug Host | | | @@ -37,9 +32,7 @@ test for the tap being in the right place: a seam in the product transport would The **debugger app is a WS server** on the dev machine; **every host dials outward** to it. This is forced, not a preference — only the inside can initiate a connection: -- native: `ws_bridge` binds `127.0.0.1` on the device; a debugger on a dev machine cannot - dial into it (the iOS Simulator "works" only because it shares the Mac's network stack — - an accident of the simulator, not a design); +- native: nothing on a dev machine can dial into a host running inside a device; - web: nothing outside the browser can dial into a Worker. ``` @@ -59,8 +52,7 @@ This is forced, not a preference — only the inside can initiate a connection: └──────────────────────┘ ``` -There is **no relay service** — the debugger app is the server, so the envelope carries no -routing metadata (no session/role/product fields, no join-order buffering). It is just: +Because the host dials the debugger directly, the envelope needs no routing metadata — it is just: ``` host → debugger { channelId, dir: "out" | "in", frame: bytes } @@ -128,26 +120,15 @@ pub enum DebugEvent { - Installed per product channel via `ProductRuntime::set_debug_sink(channel_id, sink)`; `None` by default, so production pays nothing. - The concrete sink (the outward WS dial) is provided by the **host adapter**, not the core: - web bridges `emit` to a JS `WebSocket`, native to a `ws_bridge` `URLSession`/OkHttp socket. + the web host bridges `emit` to a JS `WebSocket`; native hosts do the equivalent on their platform. - `bytes` are the untouched `ProtocolMessage`; **the debugger app decodes, the core never does.** -## Configuration and transport - -- The debugger **URL is configurable**, not loopback-derived: with the app on a device and - the debugger on a Mac, `127.0.0.1` is wrong and the link is LAN. -- The LAN hop is why **`wss://`** is right: it keeps iOS `Info.plist` `NSAppTransportSecurity` - empty (TLS is already required and `wss` satisfies it, no exception needed), and the trace - doesn't cross the office network in the clear. -- The cost moves to **certificates**: iOS still evaluates trust, so a self-signed debugger cert - is rejected until its CA is installed and enabled under **Settings → General → About → - Certificate Trust Settings**. This is a named setup step — "use wss" is not free. - ## Privacy and security - Frames stream as **opaque bytes**; the core never decodes them, so nothing at the tap reads application content or key material. Decoding happens only in the debugger app. -- **Dev-only**, off in production (the sink is unset). The debugger app is on a trusted dev - machine; `wss` keeps the LAN hop encrypted. +- **Dev-only**, off in production (the sink is unset). The debugger app runs on a trusted + dev machine. ## Hosts and scope @@ -155,14 +136,12 @@ Every host taps at its `truapi-server` choke points and dials the debugger app: (web), **Polkadot Desktop**, **iOS**, **Android**. Today those hosts run the `@novasamatech/host-*` stack, so each adopts this once it runs `truapi-server`; the tap and envelope are identical across all of them. Each host supplies its own outward-dial sink for -its platform — web bridges `emit` to a JS `WebSocket` (built); native binds a `ws_bridge` -`URLSession`/OkHttp WebSocket — dev-gated and configured with the debugger URL. Pinning the -provisional host→debugger framing (base64-in-JSON) is a prerequisite before the native sinks land. +its platform — the web sink (bridging `emit` to a JS `WebSocket`) is built; the native sinks +are phase-2. ## Non-goals - No tap in `@parity/truapi` / the TS transport (the whole point of putting it in the core). -- No relay service (the debugger app is the server). - No mocking/mutation in v1 — but the envelope and topology already accommodate it. ## Validation status @@ -181,9 +160,7 @@ provisional host→debugger framing (base64-in-JSON) is a prerequisite before th `WebSocket`, dev-gated via a `localStorage` debugger URL that the host worker reads — is wired end to end, and a product running under the `@parity/truapi-host` WASM host streams every frame to the debugger, verified against a local host harness. -- **Not yet built:** the native outward-dial sink (`ws_bridge` `URLSession`/OkHttp) and the - `wss`/cert setup. These are the native-integration tracks. +- **Not yet built:** the native outward-dial sink. This is the native-integration track (phase-2). ## References - Implementation: truapi#295 (`rust/crates/truapi-server/src/host_core.rs` — `DebugSink`/`DebugEvent`/tap). -- Requirement source: the debugger tracker (sdk-team#26). From d8d16df808b700bc1f276e67f55c7665175f3e12 Mon Sep 17 00:00:00 2001 From: Nidish Date: Mon, 3 Aug 2026 01:17:19 +0530 Subject: [PATCH 7/8] docs(design): specify gated level-2 value decode and the web inspector + CLI frontends --- docs/design/wire-observability-debug-host.md | 152 +++++++++++++++---- 1 file changed, 123 insertions(+), 29 deletions(-) diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md index c600c6601..d2620baad 100644 --- a/docs/design/wire-observability-debug-host.md +++ b/docs/design/wire-observability-debug-host.md @@ -15,13 +15,13 @@ layer upholds. Where this doc and the code disagree, this doc is the intent. The tap is in the **Rust host (`truapi-server`)**, behind a sink trait — **not** in the TypeScript transport and **not** turned on from the product side. `truapi-server` has -exactly two frame choke points, and every host (web via `wasm.rs`, native via a -`ws_bridge` `FrameSink`) funnels through them: +exactly two frame choke points, and every host funnels through them — the web +build via `wasm.rs`, native builds via `native_debug.rs`: | Direction | Choke point | |---|---| -| inbound (product → core) | `ProductRuntime::receive_frame()` — `host_core.rs` | -| outbound (core → product) | `FrameSink::emit_frame()` / `SinkTransport::send()` — `host_core.rs` | +| inbound (product → core) | `ProductRuntime::receive_frame()` — taps before dispatch — `host_core.rs` | +| outbound (core → product) | `SinkTransport::send()` — delivers via `FrameSink::emit_frame`, then taps — `host_core.rs` | One tap implementation covers both platforms, and **nothing in `@parity/truapi` changes** — the product is genuinely untouched. That the product package does not change at all is the @@ -58,6 +58,13 @@ Because the host dials the debugger directly, the envelope needs no routing meta host → debugger { channelId, dir: "out" | "in", frame: bytes } ``` +`frame` is the base64 of the untouched SCALE `ProtocolMessage` bytes (JSON can't carry binary), so +the envelope is a single text WS message. + +The debugger groups frames into per-operation traces keyed on **`(channelId, requestId)`** — +not `requestId` alone, since each host mints its own `requestId` sequence and two hosts can +reuse the same id. `channelId` keeps their operations apart and drives the app's per-host view. + ### One call, end to end ``` @@ -120,7 +127,9 @@ pub enum DebugEvent { - Installed per product channel via `ProductRuntime::set_debug_sink(channel_id, sink)`; `None` by default, so production pays nothing. - The concrete sink (the outward WS dial) is provided by the **host adapter**, not the core: - the web host bridges `emit` to a JS `WebSocket`; native hosts do the equivalent on their platform. + the web build's `WasmDebugSink` bridges `emit` to a JS `debugEmit` callback (the host worker + owns the actual `WebSocket` and its dev-only URL gating); the native build's `WsDebugSink` + implements the trait directly over a loopback `WebSocket`. - `bytes` are the untouched `ProtocolMessage`; **the debugger app decodes, the core never does.** ## Privacy and security @@ -129,38 +138,123 @@ pub enum DebugEvent { reads application content or key material. Decoding happens only in the debugger app. - **Dev-only**, off in production (the sink is unset). The debugger app runs on a trusted dev machine. +- **Residual exposure — metadata, not payload.** Even fully payload-blind, the stream still + carries, per frame, the direction, the method id and message shape, the byte size, and the + timing. That is traffic-analysis metadata: whoever holds the debugger socket can see *which* + operations ran, *when*, and *how big*, with no payload at all. The loopback-only sink and the + unset-in-production default are what bound who can hold that socket — this is a confinement + property, not an anonymity guarantee. + +## Level 2: value decode (in the debugger, gated, off by default) + +Level 1 above is payload-blind end to end: the tap streams bytes, the debugger groups by +`requestId` and shows byte lengths. **Level 2** is a strictly-scoped extension *inside the +debugger only* — decode one frame's payload to a plain JS value in the drill-down detail +path. It changes nothing about the tap, the envelope, or the host: the host still emits +**bytes only**, unchanged. The contract: + +- **Off by default, behind a real gate.** Value decode is an explicit dev-only opt-in, read + once at the debugger's server entry point from `TRUAPI_DEBUGGER_DECODE_VALUES`. With it off, + the debugger retains no payload bytes and every frame reports its byte length and nothing + else. The gate is structural, not cosmetic: `@parity/truapi-debugger` is a private, dev-only + package that is not part of any product or host production bundle, so the decode path cannot + be reached in a shipped build — it is a build/env boundary, not a code comment and not a + client-side URL flag. +- **Reuse, don't reinvent.** Decoding is `WIRE_DECODE_TABLE[frameId]?.(bytes)` from + `@parity/truapi/wire-decode` — the same generated, dev-only codecs the client uses. The + debugger writes no codecs of its own. +- **Sensitive denylist — the security core.** The generated table can decode *every* frame, + including signing and login; it deliberately does **not** exclude them. The safety of the + feature is a denylist the debugger consults *before* the table. Sensitivity is a property of + the payload **type**, declared at the source: a method carrying key material is marked + `#[wire(sensitive)]` on the Rust trait, and codegen emits its frame ids into a generated + `SENSITIVE_FRAME_IDS` set (in `@parity/truapi/wire-table`) that the debugger consumes + directly. A newly annotated method is denylisted the moment the client is regenerated — no + name-matching, and a codegen rename cannot silently drop a family. The methods it covers: + + | Family (marked `#[wire(sensitive)]`) | Why redacted | + |---|---| + | `signing/*` (create-transaction, sign-raw, sign-payload, each +legacy) | payloads to be signed and the resulting signatures | + | `account` / `statement-store` create-proof (incl. authorized) | cryptographic proofs bound to a key/identity | + | `account/sign-vrf` | a VRF signature — key material (RFC 0023) | + | `entropy/derive` | key-derivation material | + | `account` request-login + `get-user-id` | login flow and the user id it resolves | + | `local-storage/read` + `write` | product-controlled storage — can hold tokens, session state, PII (`clear` carries only a key name, so it is not sensitive) | + | `payment/top-up` | can carry a raw sr25519 secret key (`PaymentTopUpSource`) | + | coin-payment `create-cheque` / `deposit` / `listen-for-payment` | carry a `CoinPaymentCheque` with redeemable `encryptedSecrets` | + | statement-store `subscribe` / `submit` | carry a `SignedStatement` with a `decryptionKey` | + + A sensitive frame is **never decoded**, even with the toggle on — it reports its byte + length under a `redacted · sensitive method` label (the full form carries the withheld + byte count; see *Redaction is visible* below). The default action for anything under + security review is **exclude** (bytes only), not decode. Chain reads/broadcasts, chat, + notifications, permissions, theme, resource-allocation, and preimage carry no key material + and stay decodable. + + A **fail-closed content check** backs the generated denylist: any decoded value carrying a + secret-named field (`sr25519SecretKey`, `encryptedSecrets`, `decryptionKey`, a mnemonic, a + derivation secret, …) is redacted even if its method was somehow not marked — so a + secret-bearing method added without the annotation is still caught. The `#[wire(sensitive)]` + set is the authoritative guarantee; the content check is defence in depth for the payload + fields whose names the type never advertises. + +- **Never over the wire, never in `/traces`.** Decode is confined to the per-frame drill-down + (`GET /frame` and its server-rendered `GET /frame-html`). `/traces` stays byte- and value-free + in every configuration; a decoded + value exists only transiently in the drill-down response and is never persisted or relayed. + Even a sensitive frame's payload transits only as opaque bytes, lives transiently in + debugger memory, and is never decoded, displayed, or serialized. + +- **Dev-only sensitive-reveal escape hatch.** A sensitive frame stays redacted even with decode + on. A second, independent gate — `TRUAPI_DEBUGGER_REVEAL_SENSITIVE` — can *arm* a reveal, but + it is meaningful only when decode is also on (the server folds it into the decode gate, so a + stray reveal env var alone can never arm it), and arming changes no default: the session still + redacts every sensitive frame until the operator explicitly reveals one. A reveal is a + per-frame action gated by an explicit confirmation, offered through a distinct danger-styled + control (never the ordinary decode button), and the value it returns is flagged `sensitive` so + the UI renders it as the danger it is. Like decode, this gate is read only at the dev server's + entry point, so a reveal is structurally impossible in any shipped build. +- **Redaction is visible, never silent.** An operation carrying a redacted-by-default method is + marked with a 🔒 on its op-row (and carries a `data-sensitive` attribute the "🔒 only" top-bar + filter keys on); each sensitive frame shows a 🔒 lock in the frame list; and a redacted detail + renders a clear `redacted · sensitive method · B withheld` label with the byte length, never the value. These + markers are payload-blind — they reveal nothing the method name doesn't. + +## Frontends + +One trace/decode/denylist engine drives two frontends, so a reviewer's choice of tool never +changes what is shown or what is decodable: + +- A **web inspector** — a full-screen, Network-tab-style app the host dials into: an aggregate + summary strip, a filterable/sortable operation list, and a drill-down with the blur-to-reveal + payload column, Decode-all/Encode-all, and the gated reveal. +- A **terminal frontend** — for headless / SSH / CI work where a browser isn't reachable: + one-shot `ls` / `stats` / `show` / `tail` for scripting, plus an interactive query REPL you + keep querying (filter, sort, `use `, `show`, `reveal`). + +The terminal frontend is a thin client over the running debugger's HTTP endpoints +(`/traces`, `/stats`, and the gated `/frame`) that rebuilds the **same** shared view model — it +forks neither the engine nor the sensitive denylist. Both frontends therefore agree on +operations, badges, sensitivity, redaction, and what may be revealed; the reveal gate is enforced +server-side, so neither frontend can surface a value the server would withhold. ## Hosts and scope -Every host taps at its `truapi-server` choke points and dials the debugger app: **dotli** -(web), **Polkadot Desktop**, **iOS**, **Android**. Today those hosts run the -`@novasamatech/host-*` stack, so each adopts this once it runs `truapi-server`; the tap and -envelope are identical across all of them. Each host supplies its own outward-dial sink for -its platform — the web sink (bridging `emit` to a JS `WebSocket`) is built; the native sinks -are phase-2. +The tap and envelope apply uniformly to any `truapi-server` host — web, desktop, or mobile — +each of which dials the debugger with its own platform sink. Two outward-dial sinks exist today: +the web `WasmDebugSink` (bridging `emit` to a JS `debugEmit` the host worker owns) and the +native `WsDebugSink` — a loopback-only, fire-and-forget, WASM-safe WebSocket client, compiled +only under the `ws-bridge` feature (the same feature a native host's transport uses), so it is +out of the wasm graph and out of default builds. The native sink ships here as a tested, +ready-to-install seam: the CLI host (truapi#264) is on `main` and is its intended first +consumer, but does not yet install it — wiring `WsDebugSink` into a host's runtime and +surfacing its traces is that host's own integration surface, done next. ## Non-goals - No tap in `@parity/truapi` / the TS transport (the whole point of putting it in the core). - No mocking/mutation in v1 — but the envelope and topology already accommodate it. -## Validation status - -- **Automated (Rust tap):** both choke points tapped, `channel_id` per event, bytes untouched and - in order — proven by `cargo test` (`debug_sink_taps_frames_in_both_directions`); the product-vantage - direction convention (`out` = left the product) by `frame_direction_wire_str_is_product_vantage`. - Inert-when-unset is structural (the `has_debug` fast path skips the tap when no sink is installed); - off-the-critical-path follows from invariant 2 — `emit`'s result is discarded and never awaited, and - the sink contract forbids blocking or panicking. -- **Automated (debugger app):** the WS server end (`@parity/truapi-debugger`, in-repo) — a host - dials in, streams a frame, the server decodes + groups it and serves it on `/traces` — proven by - its `bun test` against a live server. The exact host→debugger framing (base64-in-JSON today) is - still provisional pending the envelope spec. -- **Built (web outward dial):** the web host's sink — `WasmDebugSink` bridging `emit` to a JS - `WebSocket`, dev-gated via a `localStorage` debugger URL that the host worker reads — is wired end - to end, and a product running under the `@parity/truapi-host` WASM host streams every frame to the - debugger, verified against a local host harness. -- **Not yet built:** the native outward-dial sink. This is the native-integration track (phase-2). - ## References - Implementation: truapi#295 (`rust/crates/truapi-server/src/host_core.rs` — `DebugSink`/`DebugEvent`/tap). +- Tracking: sdk-team#26 — validation status, per-host sequencing, and remaining work live there, not in this doc. From b6905732a72aad84d07b817d0cc723b8928417b2 Mon Sep 17 00:00:00 2001 From: Nidish Date: Wed, 5 Aug 2026 13:42:18 +0530 Subject: [PATCH 8/8] docs(design): rewrite wire-observability doc to the shipped design (local-dev + dotli surfaces) --- docs/design/wire-observability-debug-host.md | 518 +++++++++++-------- 1 file changed, 295 insertions(+), 223 deletions(-) diff --git a/docs/design/wire-observability-debug-host.md b/docs/design/wire-observability-debug-host.md index d2620baad..fde154325 100644 --- a/docs/design/wire-observability-debug-host.md +++ b/docs/design/wire-observability-debug-host.md @@ -4,257 +4,329 @@ | ------------------ | ------------------------------------------------------------------------------- | | **Start Date** | 2026-07-25 | | **Authors** | Nidish Ramakrishnan | -| **Implementation** | truapi#295 (Rust tap in `truapi-server`) | -| **Description** | A payload-blind tap in the Rust host that streams every product↔host frame, under one correlation id, to a debugger app the host dials out to. | - -This is a **specification of the contract**, not a walkthrough of the implementation: -it fixes where the tap lives, the topology, the event shape, and the invariants each -layer upholds. Where this doc and the code disagree, this doc is the intent. +| **Implementation** | truapi#295 (Rust tap in `truapi-server`; `@parity/truapi-debugger` engine + surfaces) | +| **Description** | A payload-blind tap in the Rust host core streams every product↔host TrUAPI frame, under one correlation id, to a debugger that groups and decodes it. Two surfaces consume the same engine: a standalone loopback inspector and an in-host embed. | + +The short version: product↔host TrUAPI traffic is opaque SCALE frames with no +Network tab. A tap in the Rust host core emits every frame as opaque +`{channelId, dir, bytes}`; the debugger — never the core — correlates them into +per-operation traces and decodes them. This is a **strictly dev-only tool**: the +tap and the decoder are compiled out of production builds, so there is nothing to +turn on in a shipped host. + +## The problem + +A product and its host exchange everything — account derivation, signing, chain +reads, scoped storage, subscriptions — as serialized SCALE `ProtocolMessage` +frames across a process boundary (`MessagePort`, iframe `postMessage`, or a native +webview channel). On the wire those frames are just bytes. There is no per-call +view, no way to see which operation a frame belongs to, no request/response +pairing, and no decoded payload. When a call misbehaves, the developer is staring +at a byte channel. + +This design gives that byte channel a Network tab: capture every frame, group the +frames of one call together, and — because it is a dev tool looking at the +developer's own session — decode them. ## Where the tap lives -The tap is in the **Rust host (`truapi-server`)**, behind a sink trait — **not** in the -TypeScript transport and **not** turned on from the product side. `truapi-server` has -exactly two frame choke points, and every host funnels through them — the web -build via `wasm.rs`, native builds via `native_debug.rs`: +The tap is in the **Rust host core (`truapi-server`)**, behind a sink trait — not +in the TypeScript transport, and not turned on from the product side. +`truapi-server` has exactly two frame choke points, and every host funnels through +them: -| Direction | Choke point | -|---|---| -| inbound (product → core) | `ProductRuntime::receive_frame()` — taps before dispatch — `host_core.rs` | -| outbound (core → product) | `SinkTransport::send()` — delivers via `FrameSink::emit_frame`, then taps — `host_core.rs` | +| Direction | Choke point | Location | +|---|---|---| +| inbound (product → core) | `ProductRuntime::receive_frame()` — taps **before** dispatch | `host_core.rs` | +| outbound (core → product) | `SinkTransport::send()` — delivers via `FrameSink::emit_frame`, then taps | `host_core.rs` | -One tap implementation covers both platforms, and **nothing in `@parity/truapi` changes** -— the product is genuinely untouched. That the product package does not change at all is the -test for the tap being in the right place: a seam in the product transport would fail it. +One tap covers every platform, and **nothing in `@parity/truapi` (the product +package) changes** — it has no debug seam at all. That the product package is +genuinely untouched is the test for the tap being in the right place: a seam in +the product transport would fail it. Decoding and correlation live in the +debugger, never in the core; the core only ever hands out opaque bytes. -## Topology: the host dials the debugger +The tap is **fire-and-forget**. `DebugSink::emit` must not block the frame path +and must not fail the operation that produced the event, so a slow, absent, or +crashed debugger only loses a trace, never a session. The two in-path call sites +wrap `emit` in `catch_unwind` (`emit_debug`), so even a panicking out-of-repo sink +is caught, logged, and swallowed rather than unwinding into a live dispatch. -The **debugger app is a WS server** on the dev machine; **every host dials outward** to it. -This is forced, not a preference — only the inside can initiate a connection: +```rust +/// Dev-only sink for host debug events. Unset ⇒ the tap is inert. +pub trait DebugSink: Send + Sync { + /// Fire-and-forget: must not block the frame path or fail the operation. + fn emit(&self, event: DebugEvent); +} -- native: nothing on a dev machine can dial into a host running inside a device; -- web: nothing outside the browser can dial into a Worker. +pub struct ChannelId(pub String); +pub enum FrameDirection { In, Out } -``` - WEB NATIVE / DESKTOP - ┌─ browser ──────────────────┐ ┌─ device ───────────────────┐ - │ product (iframe) │ │ product (webview) │ - │ │ MessagePort │ │ │ ws loopback │ - │ ▼ │ │ ▼ │ - │ host worker ── tap │ │ ws_bridge ── tap │ - │ │ rust wasm │ │ │ rust core │ - └──────┼─────────────────────┘ └──────┼─────────────────────┘ - │ ws, dialed outward │ ws, dialed outward - └──────────────┬─────────────────────────┘ - ▼ - ┌──────────────────────┐ - │ debugger app │ ws server on the dev machine - └──────────────────────┘ +#[non_exhaustive] // room for non-frame events (e.g. SSO); adding one is not breaking +pub enum DebugEvent { + /// A SCALE wire frame crossing a product channel. `bytes` are the untouched + /// `ProtocolMessage`; the debugger decodes them, the core never does. + Frame { channel_id: ChannelId, dir: FrameDirection, bytes: Vec }, +} ``` -Because the host dials the debugger directly, the envelope needs no routing metadata — it is just: +A sink is installed per product channel via +`ProductRuntime::set_debug_sink(channel_id, sink)`; unset by default, so a host +that never installs one pays nothing. The concrete sink is provided by the host +adapter, not the core: -``` - host → debugger { channelId, dir: "out" | "in", frame: bytes } -``` - -`frame` is the base64 of the untouched SCALE `ProtocolMessage` bytes (JSON can't carry binary), so -the envelope is a single text WS message. +- the **web** host bridges `emit` to a JS `debugEmit(channelId, dir, frame)` + callback — the host worker owns the actual socket and its dev-only URL gate; +- the **native** host's `WsDebugSink` (`native_debug.rs`) implements the trait + directly over a loopback WebSocket. It only serializes and pushes onto a + bounded queue (count- and byte-capped); a background task owns the socket, + reconnects with capped backoff, and counts dropped frames when the queue is + full. It compiles only under the `ws-bridge` feature, out of the wasm graph. It + is a ready seam for later native hosts, not one of the two surfaces below. -The debugger groups frames into per-operation traces keyed on **`(channelId, requestId)`** — -not `requestId` alone, since each host mints its own `requestId` sequence and two hosts can -reuse the same id. `channelId` keeps their operations apart and drives the app's per-host view. +## The envelope and wire identity -### One call, end to end +Each tapped frame becomes one envelope: ``` - product host edge rust core debugger app -(iframe / webview) (tap) (remote) - │ getAccount() │ │ │ - ├─ frame{ p:1, id=22 } ─▶ │ │ - │ ├┈┈ { dir:out, frame } ┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈▶│ log ▲ out p:1 - │ ├ delivered immediately ▶ ctx.requestId="p:1" │ - │ ◀─ frame{ p:1, id=23 } ─┤ │ - ◀ delivered immediately ┤ │ │ - │ Ok(response) ├┈┈ { dir:in, frame } ┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈▶│ log ▼ in p:1 + { channelId, dir, frame } ``` -`emit` is fire-and-forget in both legs. **Outbound** the tap delivers to the product -first, then emits; **inbound** it emits before dispatch, so an undecodable frame is still -observed. The order differs; the off-the-critical-path guarantee does not (see Invariants). - -## Invariants - -1. **In the path, not in the critical path.** The tap carries every frame, so it sees them - all, yet a slow, absent, or crashed debugger can never stall a session — only the trace is - lost. The ordering is **asymmetric by design**: outbound (core → product) the tap forwards - to the product first, then emits; inbound (product → core) it emits first, before - decode/dispatch, so an undecodable frame is still observed. The guarantee holds either way - because of invariant 2, not because of a single fixed ordering. -2. **Emit, never wait.** `emit` is fire-and-forget: no reply is read from the debugger socket, - and it must never block or fail the frame path. That is what keeps the tap off the critical - path wherever in each leg it fires. +`frame` is the untouched SCALE `ProtocolMessage` bytes; `dir` is **product-vantage** +(`out` = the frame left the product, `in` = it arrived at it). The Rust tap names +directions host-vantage internally and flips them to this convention on the way +out (`FrameDirection::wire_str`), so both ends always agree on which way a frame +went. Where the envelope crosses a text transport (the WS surface below), +`frame` is base64 so the whole thing fits one JSON line. + +A producer also stamps a **wire-contract fingerprint** alongside the envelope: an +envelope version (`v`), the coarse codec version (`codec` = `TRUAPI_CODEC_VERSION`), +and — the load-bearing one — `TRUAPI_WIRE_SCHEMA_HASH` (`schema`), a hash of every +frame id and its method leg (e.g. `"c18def0e997626eb"`). Frame ids are `u8` +discriminants that get reassigned as the API evolves, so a frame from a host built +against a different contract would silently decode to the *wrong* method and the +wrong value. The hash catches that: the debugger allows the decode path only for a +channel whose `schema` affirmatively matches its own. An absent or mismatched +schema still groups (grouping is payload-blind) but is never trusted to decode — +which also closes the omit-the-identity-to-bypass hole. + +## The debugger engine + +The engine is one shared core (`@parity/truapi-debugger`) that both surfaces +drive, so the surface a developer picks never changes what is shown. + +**Ingest.** Each envelope is decoded once (`decodeWireMessage`) to recover the +correlation `requestId` and the wire discriminant (`frameId`) — everything +grouping needs. An undecodable frame is surfaced as a `malformed` sentinel, not +dropped, so a trace records the failure instead of going dark. Each frame's +lifecycle `role` (request / response / start / receive / …) is resolved **at +ingest** from the frame id's wire-table kind, so every downstream consumer sees +the real role rather than `unknown`. Retained id strings are length-clamped +(default 256 chars): anything that can reach the tap could otherwise send +200k-char ids, one copy per frame, while real ids are short (`myapp.dot`, `p:1`). + +**Grouping by `(channelId, requestId)`.** Frames are grouped into per-operation +traces keyed on the channel *and* the request id — not `requestId` alone, since +each host mints its own `p:1`, `p:2`, … and two hosts reuse the same values. The +channel keeps their operations apart. This `requestId` is also the id product-sdk +telemetry spans correlate on, so a frame trace and a product span line up under +one id with no extra plumbing. + +**Generation segmentation.** A product may recycle a `requestId` for a later, +unrelated call. When a fresh opener (`request` / `start`) arrives for an id whose +current operation already opened, the engine rotates to a new *generation* rather +than merging two unrelated calls under one id. + +**Retry-storm detection.** A burst of like operations — one host hammering +`signing.createTransaction` five times in 400ms — is a cross-operation signal no +single-trace view can see. The engine groups traces by `(channelId, opener +frameId)` and slides a window (default: 3 ops within 1000ms) over their start +times, tagging every trace in a burst with a `retry-storm` badge. + +**Bounded retention, with visible eviction.** Memory is bounded three ways, and +each bound surfaces rather than lying about the data: + +- an LRU cap on retained traces (default 256); whole-op evictions are counted + (`evictedTraces()`) so a session that overflowed does not silently under-report + its op count; +- a per-trace frame cap (default 1024), ring-buffered from index 1 so a + long-lived subscription never drops its opener (pairing and retry-storm signals + key on `frames[0]`); +- a per-trace byte cap (default 1 MiB, a true bound including the opener), which + only bites when bytes are retained for decode. + +A trace that lost frames or bytes to either cap carries a `truncated` badge, and a +producer that dropped frames (link backlog full) reports the count, surfaced in +stats — so "kept N of M" is never mistaken for "only N happened". + +## The two surfaces + +One engine, two ways to look at it. Both decode every frame; neither can surface a +value the engine would not, because both run the same code. + +### (A) Standalone inspector — loopback `:9231` + +A runnable Bun server (WS + HTTP) that a host dials **outward** to. Only the inside +can initiate the connection (nothing on a dev machine dials into a device or a +Worker), so the host is always the client. Over the socket the host sends one +`{channelId, dir, frame}` text message per frame; the server groups them and +serves a full-screen, Network-tab-style **web inspector** (aggregate summary +strip, filterable/sortable operation list, per-frame drill-down with a decoded +payload column) plus a **CLI / REPL** for headless or SSH work — one-shot +`ls` / `stats` / `show` / `tail` for scripting, and an interactive query prompt +(`filter`, `sort`, `use `, `show`). The CLI is a thin client over the same +HTTP endpoints (`/traces`, `/stats`, `/frame`) rebuilding the same view model, so +both frontends agree on operations, badges, and payloads. + +What the web inspector looks like — an aggregate strip, a filterable operation +list on the left, and a detail pane on the right where every frame of the opened +op is decoded inline (no decode toggle, no reveal step): -**The second invariant is also the extension point.** Mocking/mutation later needs no -topology or envelope change — the tap starts *reading a reply* on the frames it cares about: -deliver unchanged is today's behaviour, deliver modified is mutation, respond is a mock. -This is why the one-way version ships first. - -## Interface (`truapi-server`) +``` + ┌─────────────────────────────────────────────────────────────────────────────┐ + │ TrUAPI Wire Inspector [ filter… ] [ arrival ▾ ] ( all ) ( dotli.app ) │ + ├─────────────────────────────────────────────────────────────────────────────┤ + │ 11 OPS 18 FRAMES 72 B 1 LIVE SUB 54ms AVG 5 ORPHANED 6 RETRY STORMS │ + │ account.getAccount 6 signing.signRaw 2 … │ + ├───────────────────────────────────┬───────────────────────────────────────────┤ + │ ▶ system.handshake 2f · 30ms │ p:2 signing.signRaw 2 frames · 164ms │ + │ ▶ account.getAccount 2f · 71ms │ │ + │ [RETRY STORM] │ ▶ REQUEST signing.signRaw 22B +0 │ + │ ▶ signing.signRaw 2f · 164ms ◀──┤ { "tag": "V1", "value": { "account": { │ + │ ↻ account.connectionStatus live │ "dotNsIdentifier": "alice.dot", │ + │ ▶ payment.topUp 2f · 92ms │ "derivationIndex": { … } }, … } } │ + │ ▶ signing.signRaw [ORPHANED] │ ◀ RESPONSE signing.signRaw 4B ⟳ 164ms │ + └───────────────────────────────────┴───────────────────────────────────────────┘ + operation list: filter / sort, detail pane: each frame decoded to its + health badges, live subscription SCALE value inline, refused only on + a wire-schema (codec) mismatch +``` -The tap is a **sink trait**, not a hardcoded socket — both to keep the WS transport out of -the core, and because wire frames aren't the only thing worth observing (host-internal -events like SSO have no frame to hang off). The enum leaves room for them. +And the topology it runs in — the host always dials outward to the loopback server: -```rust -/// Dev-only sink for host debug events. Unset ⇒ the tap is inert. -pub trait DebugSink: Send + Sync { - /// Fire-and-forget: must not block the frame path or fail the operation. - fn emit(&self, event: DebugEvent); -} +``` + dev machine (loopback only) + ┌──────────────────────────────────────────────┐ + │ web host (browser) debugger :9231 │ + │ ┌────────────────────┐ ┌──────────────┐ │ + │ │ product (iframe) │ │ Bun WS + HTTP │ │ + │ │ │ frames │ │ ├ engine │ │ + │ │ ▼ │ │ ├ web UI │ │ + │ │ host worker ─ tap ──┼──ws──▶ └ CLI/REPL │ │ + │ │ rust wasm │ ▲ └──────────────┘ │ + │ └────────────────────┘ │ │ + │ WasmDebugSink → debugEmit → host worker │ + │ owns the socket + dev-only URL gate │ + └──────────────────────────────────────────────┘ + host dials outward; loopback bind, Origin + + Host-header gated (see Confinement) +``` -pub struct ChannelId(pub String); +### (B) In-host embed — dotli ribbon -pub enum FrameDirection { In, Out } +The same inspector, mounted inside the host, with no server and no dial-out. dotli +runs the host in one realm (the host iframe/worker) and the inspector in the top +frame. The tap in the host realm **tees** each `{channelId, dir, frame}` to the top +frame via `window.top` `postMessage`; the top frame feeds them into +`createInAppDebugger().handleFrame(...)`. A right-edge **ribbon** mounts the full +inspector on demand. The realm-tee is payload-blind **transport** — the raw +`ProtocolMessage` bytes cross the postMessage boundary untouched — and the panel +decodes at the point of display (decode-in-panel). Because the frames never leave +the app, each browser tab is its own tenant; there is nothing to host or scope. -#[non_exhaustive] // room for non-frame events (e.g. SSO); adding one is not breaking -pub enum DebugEvent { - /// A SCALE wire frame crossing a product channel. - Frame { channel_id: ChannelId, dir: FrameDirection, bytes: Vec }, -} +``` + dotli (one browser app) + ┌──────────────────────────────────────────────┐ + │ TOP FRAME (dotli-owned) │ + │ createInAppDebugger ── engine + inspector │ + │ ▲ handleFrame(channelId, dir, frame) │ + │ │ window.top.postMessage (raw bytes) │ + │ ─────┼────────────── realm boundary ──── │ + │ HOST REALM (iframe / worker) │ + │ host core ── tap ──▶ tee to verified top │ + │ rust wasm │ + └──────────────────────────────────────────────┘ + right-edge ribbon mounts the inspector on demand + tee target pinned to a verified dotli-owned top ``` -- Installed per product channel via `ProductRuntime::set_debug_sink(channel_id, sink)`; - `None` by default, so production pays nothing. -- The concrete sink (the outward WS dial) is provided by the **host adapter**, not the core: - the web build's `WasmDebugSink` bridges `emit` to a JS `debugEmit` callback (the host worker - owns the actual `WebSocket` and its dev-only URL gating); the native build's `WsDebugSink` - implements the trait directly over a loopback `WebSocket`. -- `bytes` are the untouched `ProtocolMessage`; **the debugger app decodes, the core never does.** - -## Privacy and security - -- Frames stream as **opaque bytes**; the core never decodes them, so nothing at the tap - reads application content or key material. Decoding happens only in the debugger app. -- **Dev-only**, off in production (the sink is unset). The debugger app runs on a trusted - dev machine. -- **Residual exposure — metadata, not payload.** Even fully payload-blind, the stream still - carries, per frame, the direction, the method id and message shape, the byte size, and the - timing. That is traffic-analysis metadata: whoever holds the debugger socket can see *which* - operations ran, *when*, and *how big*, with no payload at all. The loopback-only sink and the - unset-in-production default are what bound who can hold that socket — this is a confinement - property, not an anonymity guarantee. - -## Level 2: value decode (in the debugger, gated, off by default) - -Level 1 above is payload-blind end to end: the tap streams bytes, the debugger groups by -`requestId` and shows byte lengths. **Level 2** is a strictly-scoped extension *inside the -debugger only* — decode one frame's payload to a plain JS value in the drill-down detail -path. It changes nothing about the tap, the envelope, or the host: the host still emits -**bytes only**, unchanged. The contract: - -- **Off by default, behind a real gate.** Value decode is an explicit dev-only opt-in, read - once at the debugger's server entry point from `TRUAPI_DEBUGGER_DECODE_VALUES`. With it off, - the debugger retains no payload bytes and every frame reports its byte length and nothing - else. The gate is structural, not cosmetic: `@parity/truapi-debugger` is a private, dev-only - package that is not part of any product or host production bundle, so the decode path cannot - be reached in a shipped build — it is a build/env boundary, not a code comment and not a - client-side URL flag. -- **Reuse, don't reinvent.** Decoding is `WIRE_DECODE_TABLE[frameId]?.(bytes)` from - `@parity/truapi/wire-decode` — the same generated, dev-only codecs the client uses. The - debugger writes no codecs of its own. -- **Sensitive denylist — the security core.** The generated table can decode *every* frame, - including signing and login; it deliberately does **not** exclude them. The safety of the - feature is a denylist the debugger consults *before* the table. Sensitivity is a property of - the payload **type**, declared at the source: a method carrying key material is marked - `#[wire(sensitive)]` on the Rust trait, and codegen emits its frame ids into a generated - `SENSITIVE_FRAME_IDS` set (in `@parity/truapi/wire-table`) that the debugger consumes - directly. A newly annotated method is denylisted the moment the client is regenerated — no - name-matching, and a codegen rename cannot silently drop a family. The methods it covers: - - | Family (marked `#[wire(sensitive)]`) | Why redacted | - |---|---| - | `signing/*` (create-transaction, sign-raw, sign-payload, each +legacy) | payloads to be signed and the resulting signatures | - | `account` / `statement-store` create-proof (incl. authorized) | cryptographic proofs bound to a key/identity | - | `account/sign-vrf` | a VRF signature — key material (RFC 0023) | - | `entropy/derive` | key-derivation material | - | `account` request-login + `get-user-id` | login flow and the user id it resolves | - | `local-storage/read` + `write` | product-controlled storage — can hold tokens, session state, PII (`clear` carries only a key name, so it is not sensitive) | - | `payment/top-up` | can carry a raw sr25519 secret key (`PaymentTopUpSource`) | - | coin-payment `create-cheque` / `deposit` / `listen-for-payment` | carry a `CoinPaymentCheque` with redeemable `encryptedSecrets` | - | statement-store `subscribe` / `submit` | carry a `SignedStatement` with a `decryptionKey` | - - A sensitive frame is **never decoded**, even with the toggle on — it reports its byte - length under a `redacted · sensitive method` label (the full form carries the withheld - byte count; see *Redaction is visible* below). The default action for anything under - security review is **exclude** (bytes only), not decode. Chain reads/broadcasts, chat, - notifications, permissions, theme, resource-allocation, and preimage carry no key material - and stay decodable. - - A **fail-closed content check** backs the generated denylist: any decoded value carrying a - secret-named field (`sr25519SecretKey`, `encryptedSecrets`, `decryptionKey`, a mnemonic, a - derivation secret, …) is redacted even if its method was somehow not marked — so a - secret-bearing method added without the annotation is still caught. The `#[wire(sensitive)]` - set is the authoritative guarantee; the content check is defence in depth for the payload - fields whose names the type never advertises. - -- **Never over the wire, never in `/traces`.** Decode is confined to the per-frame drill-down - (`GET /frame` and its server-rendered `GET /frame-html`). `/traces` stays byte- and value-free - in every configuration; a decoded - value exists only transiently in the drill-down response and is never persisted or relayed. - Even a sensitive frame's payload transits only as opaque bytes, lives transiently in - debugger memory, and is never decoded, displayed, or serialized. - -- **Dev-only sensitive-reveal escape hatch.** A sensitive frame stays redacted even with decode - on. A second, independent gate — `TRUAPI_DEBUGGER_REVEAL_SENSITIVE` — can *arm* a reveal, but - it is meaningful only when decode is also on (the server folds it into the decode gate, so a - stray reveal env var alone can never arm it), and arming changes no default: the session still - redacts every sensitive frame until the operator explicitly reveals one. A reveal is a - per-frame action gated by an explicit confirmation, offered through a distinct danger-styled - control (never the ordinary decode button), and the value it returns is flagged `sensitive` so - the UI renders it as the danger it is. Like decode, this gate is read only at the dev server's - entry point, so a reveal is structurally impossible in any shipped build. -- **Redaction is visible, never silent.** An operation carrying a redacted-by-default method is - marked with a 🔒 on its op-row (and carries a `data-sensitive` attribute the "🔒 only" top-bar - filter keys on); each sensitive frame shows a 🔒 lock in the frame list; and a redacted detail - renders a clear `redacted · sensitive method · B withheld` label with the byte length, never the value. These - markers are payload-blind — they reveal nothing the method name doesn't. - -## Frontends - -One trace/decode/denylist engine drives two frontends, so a reviewer's choice of tool never -changes what is shown or what is decodable: - -- A **web inspector** — a full-screen, Network-tab-style app the host dials into: an aggregate - summary strip, a filterable/sortable operation list, and a drill-down with the blur-to-reveal - payload column, Decode-all/Encode-all, and the gated reveal. -- A **terminal frontend** — for headless / SSH / CI work where a browser isn't reachable: - one-shot `ls` / `stats` / `show` / `tail` for scripting, plus an interactive query REPL you - keep querying (filter, sort, `use `, `show`, `reveal`). - -The terminal frontend is a thin client over the running debugger's HTTP endpoints -(`/traces`, `/stats`, and the gated `/frame`) that rebuilds the **same** shared view model — it -forks neither the engine nor the sensitive denylist. Both frontends therefore agree on -operations, badges, sensitivity, redaction, and what may be revealed; the reveal gate is enforced -server-side, so neither frontend can surface a value the server would withhold. - -## Hosts and scope - -The tap and envelope apply uniformly to any `truapi-server` host — web, desktop, or mobile — -each of which dials the debugger with its own platform sink. Two outward-dial sinks exist today: -the web `WasmDebugSink` (bridging `emit` to a JS `debugEmit` the host worker owns) and the -native `WsDebugSink` — a loopback-only, fire-and-forget, WASM-safe WebSocket client, compiled -only under the `ws-bridge` feature (the same feature a native host's transport uses), so it is -out of the wasm graph and out of default builds. The native sink ships here as a tested, -ready-to-install seam: the CLI host (truapi#264) is on `main` and is its intended first -consumer, but does not yet install it — wiring `WsDebugSink` into a host's runtime and -surfacing its traces is that host's own integration surface, done next. +## Decode posture + +Stated plainly: this tool **decodes every frame by default**. There is no sensitive +redaction, no denylist, and no reveal toggle. A developer inspecting their own +session's traffic sees the real values, in both surfaces. Decoding reuses +`WIRE_DECODE_TABLE[frameId]` from `@parity/truapi/wire-decode` — the same generated, +dev-only codecs the client uses; the debugger writes none of its own — and is +confined to the per-frame drill-down. The list view (`/traces`) is byte- and +value-free in every configuration: it never serializes raw bytes or decoded +values. + +The safety here is not "decode carefully". It is that **the tap and the decoder do +not exist in a production build**: + +- The web host reads its debugger URL behind `import.meta.env.DEV`. Vite replaces + that with a boolean literal, so a production bundle returns `null` + unconditionally and the tap stays inert — a stray `localStorage` key cannot turn + it on in prod. Absent a URL, the host never installs `debugEmit`, so the Rust + core never installs its sink. +- The dotli ribbon is gated on a **build-time** `DEBUG` flag, not the runtime + `?debug` toggle — a production dotli build has no ribbon to mount. +- `@parity/truapi-debugger` is a private, dev-only package that is not part of any + product or host production bundle. + +**Why "redact the sensitive ones" was dropped.** An earlier draft kept a denylist: +decode everything *except* a hand-marked set of key-bearing methods, backed by a +content check and a gated reveal. That was a false guarantee. A denylist is only as +good as its marking — a secret-bearing method added without the annotation leaks, +and a content check is a heuristic. Worse, the whole edifice implies the tool is +safe-ish to point at real traffic, which invites exactly the use it must never +have. Replacing it with "not in prod, ever" is both simpler and more honest: there +is no production surface to redact, so the guarantee is structural rather than a +list someone has to keep correct. + +## Confinement and hardening + +Even as a dev tool, each surface is confined to the machine it runs on: + +- **Loopback bind.** The standalone server binds `127.0.0.1`, so off-box peers + cannot reach it. +- **CSWSH Origin gate.** The WebSocket upgrade is refused unless the request's + `Origin` is a loopback host. A non-browser client (CLI, curl) sends no Origin and + is allowed; a cross-origin browser page trying to dial the debugger to inject + frames or drive the decoder is rejected — something binding to loopback alone + does not prevent. +- **Host-header allowlist (DNS-rebinding guard).** A page served from `evil.com` + whose DNS has been rebound to `127.0.0.1` can issue same-origin `fetch`es to the + debugger; those still carry `Host: evil.com`. Requests are 403'd unless the + `Host` header is an exact loopback name (never a fuzzy match that would read + `127.0.0.1.evil.com` as loopback). +- **Bounded payloads and retention.** The WS transport caps `maxPayloadLength` + (1 MiB), and the engine's byte-bounded retention keeps a hostile or runaway + stream from growing memory without bound. +- **Panic-safe tap.** The `catch_unwind` at the two tap sites keeps a misbehaving + sink from unwinding into a live dispatch. +- **Loopback-only producer.** The native `WsDebugSink` accepts only a `ws://` + loopback URL, resolves it, and dials the resolved loopback address directly — + closing the "validate one string, dial another" gap. No `wss`, no LAN. +- **Pinned realm-tee.** The dotli embed's `postMessage` tee is pinned to a verified + dotli-owned top frame, so the host realm neither forwards frames to nor accepts a + mount from an untrusted window. ## Non-goals -- No tap in `@parity/truapi` / the TS transport (the whole point of putting it in the core). -- No mocking/mutation in v1 — but the envelope and topology already accommodate it. +- No tap in `@parity/truapi` or the TS transport — the whole point of putting it in + the core. +- No sensitive-frame redaction, denylist, or reveal path — a dev-only tool decodes + everything, and production has no tap to redact. +- No mocking or mutation — the tap is one-way today. The `DebugSink` contract + (`emit`, never wait) is also the extension point: a future sink that reads a + reply could deliver-modified (mutation) or respond (mock) with no envelope or + topology change. ## References -- Implementation: truapi#295 (`rust/crates/truapi-server/src/host_core.rs` — `DebugSink`/`DebugEvent`/tap). -- Tracking: sdk-team#26 — validation status, per-host sequencing, and remaining work live there, not in this doc. + +- Implementation: truapi#295 — `rust/crates/truapi-server/src/{host_core.rs, + native_debug.rs}` (tap + sinks), `rust/crates/truapi-codegen` (schema hash), + `js/packages/truapi-debugger/src` (engine + surfaces). +- Tracking: sdk-team#26 — validation status, per-host sequencing, and remaining + work live there, not in this doc.