From 7b195123d6e6f6da81b0a032c4515645fe0b10d1 Mon Sep 17 00:00:00 2001 From: mfw78 Date: Sat, 25 Jul 2026 03:13:09 +0000 Subject: [PATCH 1/2] docs: trim platform generalisation and linker seam to current contract Rewrite docs/08 as a terse description of the shipped layered WIT model: keep the Layer 1/2/3 world model, the verified universal interface signatures, the venue-adapter Layer 3 mechanics, and the complete WIT package layout; collapse the mobile, WebView, and super-app targets and their per-primitive implementation tables to one sentence each; cut the motivation essay, the host adapter specification, and the summary. Reduce the 0.2 status banner to one line and point the SDK and taxonomy topics at their owning docs. Rewrite docs/design/linker-extension-seam.md against the current Extension trait: drop the retired cow-api / shepherd-cow-host / CowBackend example, describe namespace, capabilities, link, service, provider, admit and event members, the HostServices and ExtState reach paths, and the videre venue platform as the live extension the shepherd binary registers. Align the reproduced WIT with wit/: chain request-batch takes rpc-request and returns rpc-result, the event variant carries custom, and identity and chain are stated at their actual reference-host behaviour. Part of #598. --- docs/08-platform-generalisation.md | 875 +++------------------------ docs/design/linker-extension-seam.md | 174 +++--- 2 files changed, 157 insertions(+), 892 deletions(-) diff --git a/docs/08-platform-generalisation.md b/docs/08-platform-generalisation.md index fab96f38..20cb3867 100755 --- a/docs/08-platform-generalisation.md +++ b/docs/08-platform-generalisation.md @@ -1,47 +1,10 @@ # Platform Generalisation -> **Status (0.2):** Nexum is **designed** to be portable to mobile and browser hosts; the 0.2 **reference runtime is server-only**. The mobile, WebView, and super-app targets in this document describe architectural direction, not shipping artifacts. They remain in the docs because they're load-bearing design - the WIT contract is shaped by the requirement that all four can implement it - but they are **planned** work, conditional on a named design partner for 0.3. See the per-target rows below for current status. +> **Status (0.2):** the WIT contract is host-portable; the reference server runtime (docs 01-07) is its sole implementation. Mobile, WebView, and super-app hosts are design direction only and ship nothing in 0.2. -## Motivation +A module compiles against `nexum:host/event-module`. Any host that implements the required interfaces runs the same module binary; platforms differ only in how they back those interfaces. This document defines the layered world model, the universal interface set, and the venue-adapter extension mechanism. [Doc 00](00-overview.md) owns the six-primitive taxonomy. -The Nexum runtime (docs 01-07) is designed as a server-side Rust binary embedding wasmtime. But the core abstractions - WIT-defined host interfaces, content-addressed module distribution, declarative manifests - are not inherently server-specific. The same module binary, the same packaging, and the same distribution mechanism are intended to serve multiple platform targets: - -1. **Server runtime** *(shipping in 0.2)* - the current design (Rust/Tokio/wasmtime). Headless automation: blockchain event monitoring, order submission, background computation. -2. **Mobile app (Flutter/Dart)** *(planned - see roadmap)* - a WASM runtime embedded in a native mobile application via FFI. Modules run on-device, backed by local state (SQLite) and RPC over HTTP. -3. **WebView** *(planned - see roadmap)* - a browser engine (V8/JSC/SpiderMonkey) executing WASM natively, with host functions injected from the native layer via a JavaScript bridge. Enables rich web-based UIs with blockchain-native capabilities. -4. **Decentralised super app** *(planned - see roadmap)* - a shell application (mobile or desktop) that dynamically loads modules discovered via ENS and fetched from Swarm. Some modules are headless (automation); others are interactive (UI). All are sandboxed, all are distributed without a central app store. - -The key insight: **the WIT contract is the universal interface**. Any host that implements the required interfaces can run the same module binary. The differences between platforms are in *how* the host implements those interfaces - not in what the module sees. - -This document defines the layered architecture that enables this generalisation and specifies the universal interface set. The 0.2 server runtime is the first host implementation; the experimental `nexum:host/query-module` WIT world (published but unhosted in 0.2) exists to give mobile/wallet embedders a stable target to implement against before 0.3. - -## Primitive Taxonomy - -Before diving into WIT definitions, the universal runtime is built on six primitive capabilities. These are the fundamental building blocks that any decentralised application needs: - -| Primitive | Interface | Backed by | Purpose | -|-----------|-----------|-----------|---------| -| **Chain** | `chain` | JSON-RPC (eth_*) | Read/write blockchain consensus state | -| **Identity** | `identity` | Keystore / KMS / device keychain / wallet extension | Cryptographic identity - key management and signing | -| **Local Store** | `local-store` | redb / SQLite / IndexedDB | Per-module private persistence on the device | -| **Remote Store** | `remote-store` | Ethereum Swarm | Decentralised content-addressed storage | -| **Messaging** | `messaging` | Waku | Decentralised pub/sub messaging | -| **Logging** | `logging` | tracing / console | Diagnostic output | - -These six primitives are orthogonal: - -- **Chain** is the source of truth - the blockchain consensus state. Modules read chain state and (indirectly) write to it via order submission or transactions. -- **Identity** is cryptographic agency - key management and signing. Modules can enumerate available accounts and request signatures (ECDSA secp256k1 by default, extensible). The `chain` host implementation depends on `identity` internally - signing RPC methods (e.g. `eth_sendTransaction`) delegate to `identity` for the actual signature. -- **Local Store** is the module's private scratchpad - fast, local, scoped to one module on one device. Does not replicate. -- **Remote Store** is shared persistent content - content-addressed, decentralised, survives independent of any device. Any module on any device can read what another module wrote. -- **Messaging** is real-time communication - ephemeral pub/sub messages between modules, devices, or users. Unlike remote store (persistent, content-addressed), messaging is transient and topic-based. -- **Logging** is diagnostics - one-way output for debugging and monitoring. Not a data channel. - -Together they cover the full spectrum: persistent truth (chain), cryptographic agency (identity), local scratch (local-store), shared content (remote-store), real-time coordination (messaging), and diagnostics (logging). - -The 0.2 `event-module` world imports all six. (In 0.1 the WIT inadvertently omitted `identity` from the world definition despite the docs claiming six primitives; 0.2 makes the contract match the taxonomy.) Two additional **additive** capabilities are declared via the manifest's `[capabilities]` section but are not part of the six-primitive core: `http` (allowlisted), serviced by the standard `wasi:http/outgoing-handler` interface rather than a `nexum:host` one, and `client`, the `videre:venue/client` intent surface (see [Layer 3](#layer-3-domain-extensions-venue-adapters)). - -## Architectural Principle: Layered WIT Worlds +## Layered WIT Worlds Universal runtime capabilities and domain-specific surfaces live in separate layers: @@ -50,17 +13,16 @@ graph TD subgraph L3["Layer 3: Domain Venues (adapter components)"] COW["cow-venue - CoW Protocol orderbook"] DEX["dex-venue - a DEX (hypothetical)"] - LEND["lend-venue - a lending market (hypothetical)"] end subgraph L2["Layer 2: Capability Extensions (optional, composable)"] VC["videre:venue/client - typed intent access to installed venues"] - UI["ui - user interface bridge (planned)"] + UI["ui - user interface bridge (design only, no WIT)"] end subgraph L1["Layer 1: Universal Runtime Interfaces"] CSN["chain - consensus access (JSON-RPC passthrough)"] - ID["identity - cryptographic identity (key management, signing)"] + ID["identity - cryptographic identity"] LS["local-store - local key-value persistence"] RS["remote-store - decentralised content-addressed storage"] MSG["messaging - decentralised pub/sub messaging"] @@ -72,236 +34,112 @@ graph TD L2 -->|"adds imports to"| L1 ``` -Layer 1 is the world every module compiles against. A Layer 2 capability adds an import to a module's manifest-derived world: a keeper module declares `client` and gains `videre:venue/client`. Layer 3 is not a world at all: a domain enters the system as a **venue adapter component** installed into the host's venue registry, and modules reach it through the Layer 2 client interface. No module compiles against a domain world - see [Layer 3](#layer-3-domain-extensions-venue-adapters). +Layer 1 is the world every module compiles against. A Layer 2 capability adds an import to a module's manifest-derived world: a keeper module declares `client` and gains `videre:venue/client`. Layer 3 is not a world: a domain enters as a venue adapter component installed into the host's venue registry, and modules reach it through the Layer 2 client interface. No module compiles against a domain world. ## Layer 1: Universal Interfaces -These six interfaces form the universal runtime contract. Any platform - server, mobile, WebView, desktop - can implement them. +Six interfaces form the universal runtime contract. The signatures below are the shipped `wit/nexum-host` package. -### `chain` - Consensus Access +### `chain` -The module's window into blockchain consensus. A single generic function that forwards JSON-RPC requests to the host's provider infrastructure, plus an additive batched variant. The host decides *how* to reach the chain - the module only specifies *what* to ask. +Forwards JSON-RPC to the host's provider infrastructure, plus a batched variant. The reference server host forwards a read-only method surface and refuses signing or mutating methods (`eth_sendTransaction`, `eth_sign`, `personal_sign`, ...) with a `denied` fault before they reach the provider. ```wit interface chain { use types.{chain-id, fault}; - /// A structured JSON-RPC error carrying the node code and revert bytes. record rpc-error { code: s32, message: string, data: option> } - - /// Either a shared host `fault` or a structured JSON-RPC error. variant chain-error { fault(fault), rpc(rpc-error) } + record rpc-request { method: string, params: string } + variant rpc-result { ok(string), err(chain-error) } - /// Execute a JSON-RPC request against the specified chain. - /// - /// The host routes to its configured provider for the given chain, - /// applying whatever middleware is appropriate for the platform - /// (timeout, retry, rate-limit, fallback on server; simple HTTP - /// on mobile; window.ethereum or injected provider in WebView). - /// - /// `method` includes the namespace prefix (e.g. "eth_call"). - /// `params` and the success value are JSON-encoded strings. + /// `method` includes the namespace prefix (e.g. "eth_call"); `params` + /// and the success value are JSON-encoded strings. request: func(chain-id: chain-id, method: string, params: string) -> result; - /// Additive 0.2 method: batched JSON-RPC. - request-batch: func(chain-id: chain-id, calls: list>) - -> result>, chain-error>; + /// Batched JSON-RPC over one chain. The result list matches `requests` + /// in length and order; hosts that cannot batch natively fall back to + /// sequential `request` calls. + request-batch: func(chain-id: chain-id, requests: list) + -> result, chain-error>; } ``` -**Platform implementations:** - -| Platform | `chain::request` backed by | -|----------|--------------------------| -| Server (Nexum) | alloy provider with tower middleware (timeout, retry, rate-limit, fallback) | -| Mobile (Flutter) | HTTP client (reqwest via FFI, or Dart `http` package) to configured RPC endpoint | -| WebView | JavaScript bridge -> `window.ethereum` (injected wallet) or native HTTP via message channel | -| Super app | Same as mobile, with per-module chain permissions | - -The Rust SDK's `HostTransport` (doc 07) works identically on all platforms - it implements alloy's `Transport` trait over `chain::request`, so module authors get the full alloy `Provider` API regardless of where the module runs. +The SDK's `HostTransport` (doc 07) implements alloy's `Transport` trait over `chain::request` / `chain::request-batch`, so module authors get the alloy `Provider` API regardless of host. -### `identity` - Cryptographic Identity +### `identity` -Provides key management and signing capabilities to modules. ECDSA secp256k1 by default (the Ethereum standard), extensible to other schemes. Modules can enumerate available accounts and request signatures over arbitrary data. - -The `chain` host implementation depends on `identity` internally - signing RPC methods such as `eth_sendTransaction` or `eth_signTypedData_v4` delegate to `identity` for the actual cryptographic signature. Modules can also import `identity` directly for raw signing operations outside of JSON-RPC (e.g. signing EIP-712 typed data for off-chain order submission). +Account enumeration and signing. ECDSA secp256k1 by default. The reference server host leaves this interface unimplemented: `accounts` returns an empty roster and `sign` / `sign-typed-data` return the `unsupported` fault. A keystore / KMS backend is not yet shipped. ```wit interface identity { use types.{fault}; - /// List available accounts (public keys or addresses). - /// Returns a list of account identifiers (e.g. 20-byte Ethereum addresses). + /// Account addresses (20-byte EVM) the host will sign for. Empty means + /// no signing capability. accounts: func() -> result>, fault>; - /// Sign arbitrary data with the specified account's private key. - /// Returns the signature bytes (e.g. 65-byte ECDSA signature with recovery id). - sign: func(account: list, data: list) -> result, fault>; + /// personal_sign semantics; returns a 65-byte signature. + sign: func(account: list, message: list) -> result, fault>; - /// Sign EIP-712 typed structured data. - /// `typed-data` is the JSON-encoded EIP-712 typed data structure. - /// Returns the signature bytes. + /// Sign EIP-712 typed data; `typed-data` is JSON-encoded. sign-typed-data: func(account: list, typed-data: string) -> result, fault>; } ``` -**Platform implementations:** - -| Platform | `identity` backed by | -|----------|---------------------| -| Server (Nexum) | Keystore file, AWS KMS, or HSM | -| Mobile (Flutter) | Device keychain (Keystore/Keychain) or wallet SDK | -| WebView | window.ethereum (wallet extension) or native bridge to keychain | -| Super app | Device keychain + per-module permission grants | - -**Relationship with `chain`:** - -The `chain` host implementation uses `identity` internally when it encounters signing methods. For example, when a module calls `chain::request` with `eth_sendTransaction`, the host: - -1. Constructs the transaction from the JSON-RPC params. -2. Calls `identity::sign` to produce the signature. -3. Sends the signed transaction via the provider. - -This means modules that only need to sign transactions via standard JSON-RPC methods do not need to import `identity` directly - `chain` handles it transparently. Modules that need raw signing (e.g. off-chain message signing for order submission, attestations, or custom protocols) import `identity` explicitly. - -### `local-store` - Local Key-Value Persistence +### `local-store` -The module's private scratchpad. **Local to the device/process** - does not replicate, sync, or share across instances. Scoped to one module: module A cannot read module B's local state. +The module's private, device-local scratchpad. Scoped to one module: module A cannot read module B's state. Does not replicate. ```wit interface local-store { use types.{fault}; - /// Get a value by key. Returns None if the key does not exist. get: func(key: string) -> result>, fault>; - - /// Set a key-value pair. Overwrites any existing value. - /// The host MAY enforce a size quota; if exceeded, returns fault.invalid-input. + /// The host may enforce a size quota; if exceeded, `set` returns err. set: func(key: string, value: list) -> result<_, fault>; - - /// Delete a key. No-op if the key does not exist. delete: func(key: string) -> result<_, fault>; - - /// List all keys matching a prefix. Empty prefix returns all keys. + /// Empty prefix returns all keys. list-keys: func(prefix: string) -> result, fault>; + contains: func(key: string) -> result; + len: func(key: string) -> result, fault>; + count: func(prefix: string) -> result; } ``` -**Platform implementations:** - -| Platform | `local-store` backed by | -|----------|-------------------------| -| Server (Nexum) | redb (per-module database file, ACID, MVCC) | -| Mobile (Flutter) | SQLite (per-module table or database, via `sqflite`) | -| WebView | IndexedDB (per-module object store) or `localStorage` | -| Super app | SQLite (shared database, per-module namespace isolation) | - -The semantics are deliberately minimal - get, set, delete, prefix scan. This is the LCD (lowest common denominator) that every platform can implement efficiently. Advanced features (transactions, MVCC, crash-safety) are host-specific and not exposed in the WIT. +The server runtime's transactional semantics (doc 04) are an implementation detail, not a cross-platform guarantee; modules that need stronger guarantees design for idempotency. -The server runtime's all-or-nothing transactional semantics (doc 04) remain an implementation detail of the Nexum host, not a guarantee modules can rely on across platforms. Modules that need stronger guarantees should design for idempotency. +### `remote-store` -### `remote-store` - Decentralised Content-Addressed Storage - -Backed by Ethereum Swarm. Provides decentralised persistence beyond the local device - content-addressed, censorship-resistant, and accessible from any host on any device. - -Swarm is both the distribution mechanism (modules are fetched from Swarm) and a runtime capability. This interface closes the loop - modules can publish to the same network they were distributed through. +Backed by Ethereum Swarm: content-addressed persistence beyond the local device, plus mutable feeds. The same network modules are distributed through (docs 02, 03). ```wit interface remote-store { use types.{fault}; - /// Upload raw data to the decentralised store. /// Returns the 32-byte content reference (Swarm address). - /// - /// The host routes to its configured Bee node. Postage batch - /// management is the host's responsibility - the module only - /// provides data and gets back a reference. upload: func(data: list) -> result, fault>; - - /// Download raw data by 32-byte content reference. - /// - /// The host fetches from its Bee node or a public gateway. - /// Returns the raw bytes. The caller is responsible for - /// interpreting the content (JSON, protobuf, WASM, etc.). download: func(reference: list) -> result, fault>; - /// Read the latest value from a mutable feed. - /// - /// Feeds are mutable pointers: (owner, topic) -> latest chunk. - /// `owner`: 20-byte Ethereum address of the feed owner. - /// `topic`: 32-byte topic hash. - /// - /// Returns None if the feed has no updates. - read-feed: func( - owner: list, - topic: list, - ) -> result>, fault>; - - /// Update a mutable feed with new data. - /// - /// The host signs the feed update with its configured identity - /// (Bee node's Ethereum key). Only the host's own feeds can be - /// updated - the owner is implicit (the host's address). - /// - /// `topic`: 32-byte topic hash. - /// `data`: the payload to publish. - /// - /// Returns the 32-byte reference of the new chunk. - write-feed: func( - topic: list, - data: list, - ) -> result, fault>; + /// Feeds are mutable pointers (owner, topic) -> latest chunk. + read-feed: func(owner: list, topic: list) -> result>, fault>; + /// The host signs the update with its configured identity; the owner + /// is implicit. Returns the 32-byte reference of the new chunk. + write-feed: func(topic: list, data: list) -> result, fault>; } ``` -**Platform implementations:** - -| Platform | `remote-store` backed by | -|----------|--------------------------| -| Server (Nexum) | Direct Bee API (`http://localhost:1633`) | -| Mobile (Flutter) | Bee API via HTTP (local light node or remote gateway) | -| WebView | JavaScript bridge -> native HTTP to Bee gateway | -| Super app | Embedded Bee light node or gateway proxy | - -**Why remote-store as a universal interface:** +### `messaging` -- **Decentralised persistence.** `local-store` is device-local. `remote-store` gives modules access to content-addressed storage that persists independent of any single device. -- **Content distribution.** Modules can publish data (feeds, references) that other modules or users can consume - without a central server. -- **Cross-device coordination.** Two instances of the same module on different devices can share data via feed topics - one writes via `write-feed`, the other reads via `read-feed`. -- **Consistency with distribution model.** Modules are already fetched from Swarm (doc 02, 03). Exposing `remote-store` at runtime means modules participate in the same content-addressed network they were distributed through. - -### `messaging` - Decentralised Messaging - -Backed by Waku. Provides real-time, privacy-preserving pub/sub messaging between modules, devices, and users. Unlike `remote-store` (persistent, content-addressed), `messaging` is transient and topic-based - fire-and-forget messages on content topics. +Backed by Waku: transient, topic-based pub/sub. Sending uses `publish`; receiving is declared as a manifest subscription and delivered through `on-event`. ```wit interface messaging { - use types.{fault}; - - record message { - content-topic: string, - payload: list, - timestamp: u64, // milliseconds since Unix epoch, UTC - /// Optional sender identity (protocol-dependent). - sender: option>, - } + use types.{fault, message}; - /// Publish a message to a content topic. - /// - /// The host routes to its configured Waku node. The message is - /// propagated to all subscribers of the content topic via the - /// Waku relay (gossipsub) or light push protocol. - /// - /// Content topics follow the format: //// - /// e.g. "/nexum/1/twap-updates/proto" + /// Content topics follow ////. publish: func(content-topic: string, payload: list) -> result<_, fault>; - - /// Query historical messages from the Waku store protocol. - /// - /// Returns messages matching the content topic within the - /// optional time range. Not all hosts support store queries - /// (depends on Waku node configuration). query: func( content-topic: string, start-time: option, @@ -311,68 +149,24 @@ interface messaging { } ``` -**Receiving messages** is handled through the event system, not the `messaging` interface. Modules declare message subscriptions in their manifest, and the host delivers them as events: - ```toml [[subscription]] kind = "message" content_topic = "/nexum/1/twap-updates/proto" ``` -The event variant in 0.2 carries `message` as a first-class variant: - -```wit -record message { - content-topic: string, - payload: list, - timestamp: u64, // milliseconds since Unix epoch, UTC - sender: option>, -} - -variant event { - block(block), - chain-logs(chain-logs), - tick(tick), - message(message), -} -``` - -This follows the same pattern as all other event sources: sending uses the import interface (`messaging::publish`), receiving uses the declarative subscription + `on-event` dispatch. - -**Platform implementations:** - -| Platform | `messaging` backed by | -|----------|-----------------| -| Server (Nexum) | Waku node (nwaku or go-waku) via JSON-RPC or REST API | -| Mobile (Flutter) | Waku light client via FFI (libwaku) or HTTP to remote Waku node | -| WebView | JavaScript bridge -> native Waku client, or js-waku in-browser | -| Super app | Embedded Waku light node | - -**Why messaging as a universal interface:** - -- **Module-to-module communication.** Two modules on different devices can exchange real-time messages via shared content topics. The TWAP monitor on a server can notify a mobile dashboard module that a new part was posted. -- **User notifications.** A headless server module can publish an alert to a content topic; the user's mobile app module subscribes and displays a notification. -- **Decentralised coordination.** Multiple instances of the same module (e.g. running on different operator nodes) can coordinate via messaging - leader election, work distribution, heartbeats. -- **Privacy.** Waku supports encrypted messaging and ephemeral relay. Modules can communicate without exposing data to the public chain. -- **Complementary to remote-store.** `remote-store` is for persistent content (data that should survive). `messaging` is for ephemeral signals (notifications, coordination, real-time feeds). Together they cover the full persistence spectrum. - -### `logging` - Structured Logging - -Unchanged from the current design: +### `logging` ```wit interface logging { enum level { trace, debug, info, warn, error } - - /// Emit a structured log message. - /// The host decides how to handle it (stdout, file, discard). log: func(level: level, message: string); } ``` -Every platform implements this trivially. On server: `tracing` crate. On mobile: platform logger (`android.util.Log`, `os_log`). In WebView: `console.log`. The SDK's `info!`, `debug!`, etc. macros compile to this. +### Universal world definition -### Universal World Definition +The shared types and the `event-module` world: ```wit package nexum:host@0.1.0; @@ -380,70 +174,39 @@ package nexum:host@0.1.0; interface types { type chain-id = u64; - record block { - chain-id: chain-id, - number: u64, - hash: list, - timestamp: u64, // ms since Unix epoch, UTC - } + record block { chain-id: chain-id, number: u64, hash: list, timestamp: u64 } record chain-log { - address: list, - topics: list>, - data: list, - block-hash: option>, // block-scoped fields absent on a pending log - block-number: option, - block-timestamp: option, - transaction-hash: option>, - transaction-index: option, - log-index: option, - removed: bool, + address: list, topics: list>, data: list, + block-hash: option>, block-number: option, + block-timestamp: option, transaction-hash: option>, + transaction-index: option, log-index: option, removed: bool, } - - // A batch of logs from one subscription; the alloy log carries no chain - // id, so it sits here once and every log shares the subscription's chain. - record chain-logs { - chain-id: chain-id, - logs: list, - } - - record tick { - fired-at: u64, // ms since Unix epoch, UTC - } - + record chain-logs { chain-id: chain-id, logs: list } + record tick { fired-at: u64 } record message { - content-topic: string, - payload: list, - timestamp: u64, // ms since Unix epoch, UTC + content-topic: string, payload: list, timestamp: u64, sender: option>, } + /// A domain extension's own event kind and opaque payload; the core + /// routes by `kind` and never reads `payload`. + record custom-event { kind: string, payload: list } variant event { - block(block), - chain-logs(chain-logs), - tick(tick), - message(message), + block(block), chain-logs(chain-logs), tick(tick), + message(message), custom(custom-event), } - /// Opaque config (typed variant deferred to 0.3). type config = list>; - /// Shared cross-domain failure vocabulary. Richer interfaces embed it - /// as a case; interfaces with nothing to add report it directly. variant fault { unsupported(string), unavailable(string), denied(string), rate-limited(rate-limit), timeout, invalid-input(string), internal(string), } - - record rate-limit { - retry-after-ms: option, - } + record rate-limit { retry-after-ms: option } } -// ... chain, identity, local-store, remote-store, messaging, logging interfaces as above ... - -/// Event-driven module - automation, background processing. -/// No UI capabilities. Runs on any conforming host. Six imports in 0.2. +/// Event-driven module. No UI capabilities. Runs on any conforming host. world event-module { import chain; import identity; @@ -457,129 +220,22 @@ world event-module { } ``` -A module compiled against `nexum:host/event-module` is the **maximally portable** artifact. In 0.2 it runs on the server reference runtime; mobile and WebView hosts are planned (see the status banner at the top of this doc). - -## Layer 2: UI Interface - -Interactive modules - those with a user-facing presence in a super app or WebView container - import the `ui` interface in addition to the Layer 1 universals. - -### Design Approach - -The `ui` interface is a **bridge**, not a rendering engine. It does not define a widget tree, layout system, or styling language. Instead, it provides the communication channel between the module's logic (running in WASM) and the host's UI surface (a WebView, native view, or terminal). - -For WebView-based hosts (the primary target for interactive modules), the module's UI is a web application (HTML/CSS/JS) served into a WebView by the host. The `ui` interface gives the module control over this surface and access to native capabilities that a normal web page cannot reach. - -```wit -interface ui { - record ui-error { - code: u16, - message: string, - } +Time, randomness, and outbound HTTP are WASI concerns, not `nexum:host` interfaces: `wasi:clocks` and `wasi:random` are linked into every module store; `wasi:http/outgoing-handler` is linked gated per-module by the `[capabilities.http].allow` allowlist. - /// Emit a UI update. - /// - /// For WebView hosts: `content` is an HTML fragment or a JSON - /// message that the WebView's JavaScript layer interprets. - /// For native hosts: `content` is a declarative description - /// (format negotiated via host-info). - /// - /// The `target` identifies which UI surface to update - /// (e.g. "main", "overlay", "notification-badge"). - render: func(target: string, content: string) -> result<_, ui-error>; - - /// Request navigation to a different view or module. - /// - /// `target`: a route string (e.g. "/settings", "module:price-alert"). - /// `params`: key-value parameters for the target. - navigate: func( - target: string, - params: list>, - ) -> result<_, ui-error>; - - /// Show a native notification (outside the WebView). - notify: func(title: string, body: string) -> result<_, ui-error>; - - /// Prompt the user for a yes/no decision via native dialog. - confirm: func(title: string, body: string) -> result; - - /// Query the host's UI capabilities. - record host-capabilities { - /// "android" | "ios" | "desktop" | "web" | "terminal" - platform: string, - /// Content format the host expects for render(). - /// "html" | "json-widget" | "markdown" - render-format: string, - supports-notifications: bool, - supports-biometric: bool, - } - - host-info: func() -> host-capabilities; -} -``` +The experimental `nexum:host/query-module` world (a pure `evaluate` entry point over `local-store` and `logging`) is published but has no host implementation in 0.2. -### Module Exports for Interactive Modules +## Layer 2: Capability Extensions -Interactive modules export additional lifecycle hooks beyond `init` and `on-event`: +A capability adds imports to a module's manifest-derived world beyond the Layer 1 core. Two are additive, declared in the manifest's `[capabilities]` section and not part of the six-primitive core: -```wit -/// Interactive module - has a UI presence. -world app-module { - include event-module; - import ui; - - /// Called when the module's UI surface is first displayed. - /// Returns initial content to render. - export on-render: func() -> result; - - /// Called when the user interacts with a UI element. - /// - /// `element-id`: identifier of the element (set by the module in its render output). - /// `action`: interaction type ("click", "submit", "change", etc.). - /// `data`: optional payload (form data, input value, etc.). - export on-interact: func( - element-id: string, - action: string, - data: option, - ) -> result<_, string>; -} -``` +- `client`: the `videre:venue/client` intent surface (see [Layer 3](#layer-3-domain-extensions-venue-adapters)). +- `http`: allowlisted `wasi:http/outgoing-handler`. -This creates a bidirectional loop: - -```mermaid -flowchart TD - A["Host calls on-render"] --> B["Module returns initial UI content"] - B --> C["Host displays in WebView/native surface"] - C --> D["User interacts"] - D --> E["Host calls on-interact(element, action, data)"] - E --> F["Module processes interaction"] - F --> G["module calls ui::render(target, new-content) to update UI"] - F --> H["module calls chain::request to read chain state"] - F --> I["module calls local-store::set to persist"] - G --> C -``` - -The module's logic runs in the WASM sandbox. The UI runs in the WebView (or native surface). The `ui` interface is the bridge between them. This is analogous to Elm's update loop or React's message-passing model, but across the WASM-host boundary. - -### WebView Module Packaging - -A WebView-based interactive module bundles its web assets alongside the WASM component: - -``` -price-dashboard/ -├── module.toml # manifest (declares world: app-module) -├── module.wasm # compiled WASM component -└── ui/ - ├── index.html # entry point (loaded into WebView) - ├── app.js # UI logic (receives on-interact, calls render) - └── style.css # styling -``` - -The host loads `index.html` into a WebView and injects the bridge JavaScript that connects DOM events to `on-interact` and `ui::render` calls to DOM updates. +A `ui` interface for interactive modules (a bridge between module logic and a host UI surface) is designed but ships no WIT; there is no `app-module` world in 0.2. ## Layer 3: Domain Extensions (Venue Adapters) -A domain (CoW Protocol, a DEX, a lending market) extends the platform as a **venue adapter**: a component authored with `#[videre_sdk::venue]` that exports the `videre:venue/adapter` interface and imports scoped transport only (`chain`, `messaging`, allowlisted `wasi:http`). This is the domain-extension mechanism - the domain's wire protocol, body codec, and error projection live inside the adapter component, and nothing domain-specific enters the host or any module world. +A domain (CoW Protocol, a DEX, a lending market) extends the platform as a **venue adapter**: a component authored with `#[videre_sdk::venue]` that exports the `videre:venue/adapter` interface and imports scoped transport only (`chain`, `messaging`, allowlisted `wasi:http`). The domain's wire protocol, body codec, and error projection live inside the adapter component; nothing domain-specific enters the host or any module world. ```wit package videre:venue@0.1.0; @@ -610,406 +266,47 @@ interface adapter { } ``` -The two faces meet in the host. The venue platform (`crates/videre-host`, one `nexum-runtime` extension registered at the composition root) holds the `VenueRegistry`, links `videre:venue/client` into keeper worlds, and routes each call: resolve the venue id to its installed adapter, run the advisory egress guard over the adapter's pure `derive-header` projection, then invoke the adapter face. Accepted submits go under a status watch; the platform polls the adapter's `status` and fans transitions back to subscribed modules as `intent-status` events. Intent bodies are opaque on this whole path - typing is a guest-side agreement between keeper and adapter over the venue's published `IntentBody` schema ([doc 05](05-sdk-design.md#bodies-the-intentbody-derive)). +The two faces meet in the host. The venue platform (`crates/videre-host`, one `nexum-runtime` extension registered at the composition root) holds the `VenueRegistry`, links `videre:venue/client` into keeper worlds, and routes each call: resolve the venue id to its installed adapter, run the advisory egress guard over the adapter's pure `derive-header` projection, then invoke the adapter face. Accepted submits go under a status watch; the platform polls the adapter's `status` and fans transitions back to subscribed modules as `intent-status` events. Intent bodies are opaque on this path; typing is a guest-side agreement between keeper and adapter over the venue's published `IntentBody` schema ([doc 05](05-sdk-design.md#bodies-the-intentbody-derive)). -A new domain therefore adds a component and (usually) a body crate, never a WIT package or a host change: write the adapter with `#[videre_sdk::venue]`, publish its codec vectors and header goldens, and install it via the engine's `[[adapters]]` table. The `shepherd` binary is exactly this composition: the core lattice plus the videre platform, with CoW entering only as the bundled `cow-venue` adapter - the engine itself stays venue- and cow-free. +A new domain adds a component and (usually) a body crate, never a WIT package or a host change: write the adapter with `#[videre_sdk::venue]`, publish its codec vectors and header goldens, and install it via the engine's `[[adapters]]` table. The `shepherd` binary is exactly this composition: the core lattice plus the videre platform, with CoW entering only as the bundled `cow-venue` adapter. ### The legacy read path: `shepherd:cow` -The retired predecessor to venue adapters was a Layer-3 *world* extension: the `shepherd:cow/cow-api` interface (orderbook passthrough plus `submit-order`), a `shepherd` world including `event-module` and importing it, and a host-side extension cone implementing the interface over a cached orderbook client. That model put the domain in the module's own world and a backend in every host that ran it; each new domain would have needed its own world, host cone, and SDK glue. - -The path is deleted: the `cow-api` interface, the `shepherd`/`cow-ext` worlds, and the host cone are gone, and orderbook I/O lives in the `cow-venue` adapter behind `videre:venue/client`. The `shepherd:cow` package remains only as `cow-events`, the package of record for the CoW on-chain event ABIs (signatures and topic-0 hashes) that keeper manifests and decoders are parity-tested against. [ADR-0005](adr/0005-cow-api-via-cached-orderbookapi.md) and [ADR-0006](adr/0006-cow-twap-ethflow-host-helpers.md) are superseded accordingly. +The retired predecessor to venue adapters was a Layer-3 *world* extension: a `shepherd:cow/cow-api` interface (orderbook passthrough plus `submit-order`), a `shepherd` world importing it, and a host-side extension cone implementing it over a cached orderbook client. That path is deleted. The `cow-api` interface, the `shepherd` / `cow-ext` worlds, and the host cone are gone; orderbook I/O lives in the `cow-venue` adapter behind `videre:venue/client`. The `shepherd:cow` package remains only as `cow-events`, the package of record for the CoW on-chain event ABIs (signatures and topic-0 hashes) that keeper manifests and decoders are parity-tested against. [ADR-0005](adr/0005-cow-api-via-cached-orderbookapi.md) and [ADR-0006](adr/0006-cow-twap-ethflow-host-helpers.md) are superseded accordingly. ## Complete WIT Package Layout ``` wit/ ├── nexum-host/ -│ ├── types.wit # chain-id, block, log, tick, message, event, config, fault -│ ├── chain.wit # chain interface (consensus access + request-batch) -│ ├── identity.wit # identity interface (key management, signing) -│ ├── local-store.wit # local-store interface -│ ├── remote-store.wit # remote-store interface (Swarm) -│ ├── messaging.wit # messaging interface (Waku) -│ ├── logging.wit # logging interface +│ ├── types.wit # chain-id, block, log, tick, message, custom-event, event, config, fault +│ ├── chain.wit # chain interface (request + request-batch) +│ ├── identity.wit # identity interface (accounts, signing) +│ ├── local-store.wit +│ ├── remote-store.wit # Swarm +│ ├── messaging.wit # Waku +│ ├── logging.wit │ ├── event-module.wit # event-module world (6 imports) -│ └── query-module.wit # experimental: query-module world (no host impl in 0.2) +│ └── query-module.wit # experimental: no host impl in 0.2 │ ├── videre-value-flow/ │ └── types.wit # asset + asset-amount vocabulary -│ ├── videre-types/ │ └── types.wit # intent-header, quotation, receipt, submit-outcome, intent-status, venue-error -│ ├── videre-venue/ │ └── venue.wit # client + adapter interfaces, venue-adapter world -│ └── shepherd-cow/ └── cow-events.wit # CoW event-ABI package of record (legacy package name) ``` -The `nexum-host` package is domain-agnostic and reusable. The `videre` packages are the venue-neutral intent contract. `shepherd-cow` carries only the CoW event ABIs. New domains add adapter components, not packages: the universal and venue layers are closed. (The `ui` interface and `app-module` world are design-only and ship no WIT yet.) +The `nexum-host` package is domain-agnostic. The `videre` packages are the venue-neutral intent contract. `shepherd-cow` carries only the CoW event ABIs. New domains add adapter components, not packages. ## Platform Targets -### Server Runtime (Reference Implementation - Nexum) - -This is the current design (docs 01-07), adapted for the layered WIT. Shepherd is the Nexum composition root that registers the videre venue platform and bundles the `cow-venue` adapter. - -| Interface | Implementation | -|-----------|---------------| -| `chain` | alloy provider with tower middleware (timeout, retry, rate-limit, fallback) | -| `identity` | Keystore file, AWS KMS, or HSM - operator-configured signing backend | -| `local-store` | redb (per-module database file, ACID, MVCC, crash-safe) | -| `remote-store` | Bee API (`http://localhost:1633`) - operator runs a Bee node | -| `messaging` | Waku node (nwaku) via JSON-RPC or REST API | -| `logging` | `tracing` crate -> JSON structured logs | -| `videre:venue/client` | videre-host `VenueRegistry` -> installed venue adapter components (CoW via the bundled `cow-venue` adapter over allowlisted `wasi:http`) | -| Event sources | `eth_subscribe` (blocks, logs), cron (Tokio interval), Waku relay (messages) | -| WASM engine | wasmtime 45.x (Component Model, fuel, epoch metering) | - -### Mobile App (Flutter/Dart) - Planned - -> **Status:** No mobile host ships in 0.2. The design below is the target architecture for a future release (0.3+, conditional on a named design partner). It's retained because the WIT contract was shaped to make this implementation possible, and the `query-module` world in 0.2 is the experimental contract a mobile/wallet embedder would target. - -A Flutter application would embed a WASM runtime and provide the universal interfaces via Dart implementations: - -```mermaid -flowchart TD - subgraph FlutterApp["Flutter App"] - subgraph WASMRuntime["WASM Runtime (via FFI)"] - ENGINE["wasmtime C API or wasmer_dart or wasm3 (lightweight, C-based)"] - ModA["Module A (headless)"] - ModB["Module B (headless)"] - end - - subgraph HostAdapter["Host Adapter (Dart)"] - HA_CHAIN["chain -> HTTP client to RPC endpoint"] - HA_ID["identity -> device keychain (Keystore/Keychain) or wallet SDK"] - HA_LS["local-store -> SQLite (sqflite)"] - HA_RS["remote-store -> HTTP to Bee gateway"] - HA_MSG["messaging -> libwaku via FFI"] - HA_LOG["logging -> platform logger"] - end - - subgraph EventSources["Event Sources (Dart)"] - ES["Block polling (HTTP, no WebSocket on mobile background), timer via Dart Timer, Waku subscription via light client, push notifications (optional)"] - end - - ModA --> HostAdapter - ModB --> HostAdapter - HostAdapter --> EventSources - end -``` - -**WASM engine options:** - -| Engine | Component Model | Mobile support | Notes | -|--------|----------------|----------------|-------| -| wasmtime (C API) | Full | aarch64 (iOS/Android ARM64) | Best compatibility, largest binary size (~15 MB) | -| wasmer | Partial | Good (wasmer_dart exists) | Component Model support is partial | -| wasm3 | None | Excellent (tiny C library, ~100 KB) | Interpreter only, no Component Model - requires core module + shim | - -For full Component Model support (identical module binaries across server and mobile), **wasmtime via C API** is the recommended path. Dart's FFI (`dart:ffi`) can call the wasmtime C API directly. The binary size cost (~15 MB) is acceptable for a mobile app. - -**Mobile-specific constraints:** - -- **Background execution.** iOS and Android aggressively suspend background processes. A mobile host cannot maintain persistent WebSocket subscriptions. Event sourcing must be adapted: poll on foreground, use push notifications or local alarms for time-sensitive events. -- **Battery.** Continuous block polling drains battery. The mobile host should use adaptive polling intervals and batch event processing. -- **Connectivity.** Mobile networks are intermittent. Host functions should handle offline gracefully (queue requests, retry on reconnect). -- **Waku light client.** Mobile devices should use Waku's light push and filter protocols rather than full relay to minimise bandwidth and battery consumption. - -### WebView (Browser Engine + Injected Host Functions) - Planned - -> **Status:** No WebView host ships in 0.2. The architecture below describes a future target. The `jco`-based transpilation path is the strongest candidate, but it depends on Component Model browser support stabilising and on a concrete embedder design partner. - -A WebView host would run inside a native app (or standalone browser). The WASM module executes in the browser's native WASM engine. Host functions are injected via a JavaScript bridge. - -```mermaid -flowchart TD - subgraph NativeApp["Native App Shell"] - subgraph WebView["WebView"] - WASMModule["WASM Module (browser's WASM engine)\nCalls imported functions:\nchain.request(...)\nidentity.sign(...)\nlocalStore.get(...)\nremoteStore.download(...)\nmessaging.publish(...)\nlogging.log(...)"] - - subgraph JSBridge["JavaScript Bridge (injected)"] - JS["window.nexumRuntime = {\n chain: { request: (c, m, p) =>\n nativeBridge.call('chain', ...) },\n identity: { accounts: () =>\n nativeBridge.call('identity', ...) },\n localStore: { get: (k) =>\n nativeBridge.call('store', ..) },\n remoteStore: { download: (ref) =>\n nativeBridge.call('store', ..) },\n messaging: { publish: (t, p) =>\n nativeBridge.call('messaging', ...) },\n logging: { log: (l, m) =>\n console.log(${`[l] m`}) }\n}"] - end - - WASMModule --> JSBridge - end - - subgraph NativeHost["Native Host Adapter"] - NH_CHAIN["chain -> HTTP to RPC / wallet bridge"] - NH_ID["identity -> window.ethereum / native keychain"] - NH_LS["local-store -> SQLite / IndexedDB"] - NH_RS["remote-store -> HTTP to Bee gateway"] - NH_MSG["messaging -> Waku node / js-waku"] - NH_LOG["logging -> native logger"] - end - - JSBridge -->|"message channel"| NativeHost - end -``` - -**Component Model in the browser:** - -Browsers don't natively support the WASM Component Model (as of early 2026). Two approaches: - -1. **`jco` transpilation** (recommended). The Bytecode Alliance's `jco` tool transpiles a WASM component to a core WASM module + JavaScript glue code. The JS glue implements the canonical ABI marshalling. The result runs in any browser. This means the **same `.wasm` component** built for the server can be transpiled and run in a WebView. - -2. **Core module variant.** Compile the module as a core WASM module (not a component) with a JS shim layer that maps the WIT interface to JavaScript imports. This requires a separate build target but avoids the `jco` dependency. - -Approach 1 is preferred - it preserves the single-artifact property (one `.wasm` component, multiple platforms). - -**WebView-specific capability: `window.ethereum`** +The reference server runtime is the sole host. `shepherd` is the composition root: it registers the videre venue platform and bundles the `cow-venue` adapter over the core lattice (alloy provider pool for `chain`, redb for `local-store`, Bee for `remote-store`, Waku for `messaging`, `tracing` for `logging`, wasmtime Component Model engine). -In a browser context, the user may have a wallet extension (MetaMask, Rabby, etc.) that injects `window.ethereum`. The `chain::request` host function can optionally route through this: - -```javascript -// In the JS bridge -chain: { - request: async (chainId, method, params) => { - if (window.ethereum && useWalletProvider) { - // Route through user's wallet (gets signing capabilities too) - return await window.ethereum.request({ method, params: JSON.parse(params) }); - } else { - // Route through native bridge to configured RPC endpoint - return await nativeBridge.call('chain', { chainId, method, params }); - } - } -} -``` - -This is powerful: the same module that runs headless on a server (reading chain state via a configured RPC endpoint) can run in a WebView and read chain state via the user's wallet - gaining access to the user's connected accounts and signing capabilities. - -Similarly, the `identity` interface in a WebView context can delegate to `window.ethereum` for account enumeration and signing, providing a seamless bridge between the module's signing needs and the user's wallet extension. - -**WebView-specific capability: `js-waku`** - -For messaging in the browser, `js-waku` provides a pure JavaScript Waku client. The `messaging` host function can route through `js-waku` directly in the WebView without needing the native bridge - peer-to-peer messaging from the browser. - -### Decentralised Super App - Planned - -> **Status:** The super app is the convergence of the mobile and WebView targets. No super-app host ships in 0.2. The content below describes the target architecture for a future release once mobile and WebView are live. - -The super app is the convergence of all targets. A native shell (Flutter) that would: - -1. **Discover modules** via ENS (doc 03) - the same discovery mechanism as the server runtime. -2. **Fetch modules** from Swarm/IPFS - the same content-addressed distribution. -3. **Run event-driven modules** in an embedded WASM runtime (automation, background tasks). -4. **Run interactive modules** in WebViews (UI, dashboards, transaction builders). -5. **Provide the universal interfaces** to all modules (chain, identity, local-store, remote-store, messaging, logging). -6. **Provide the UI interface** to interactive modules. - -```mermaid -flowchart TD - subgraph SuperApp["Super App Shell (Flutter)"] - subgraph ModMgr["Module Manager"] - DISC["Discovery: ENS -> Swarm -> content store -> verify"] - LIFE["Lifecycle: Load -> Init -> Run -> Restart -> Dead"] - PERM["Permissions: per-module capability grants"] - end - - subgraph HeadlessRT["Headless WASM Runtime"] - TWAP["TWAP Monitor"] - PRICE["Price Alert"] - end - - subgraph WebViewPool["WebView Pool"] - PORTFOLIO["Portfolio Dashboard (HTML)"] - DEX["DEX Swap Interface (HTML)"] - end - - subgraph HostLayer["Host Adapter Layer"] - HL_CHAIN["chain -> HTTP to RPC endpoints"] - HL_ID["identity -> device keychain + per-module grants"] - HL_LS["local-store -> SQLite"] - HL_RS["remote-store -> Bee light node / gateway"] - HL_MSG["messaging -> Waku light client"] - HL_LOG["logging -> app logger + optional cloud"] - HL_UI["ui -> WebView bridge (interactive modules)"] - end - - subgraph ShellUI["Shell UI (Flutter)"] - SHELL["Module gallery - Navigation - Settings - Wallet"] - end - - ModMgr --> HeadlessRT - ModMgr --> WebViewPool - TWAP --> HostLayer - PRICE --> HostLayer - PORTFOLIO --> HostLayer - DEX --> HostLayer - end -``` - -**What makes this different from Telegram/WeChat mini-programs:** - -| Aspect | Telegram/WeChat | Decentralised Super App | -|--------|-----------------|------------------------| -| Distribution | Central app store / bot platform | ENS -> Swarm/IPFS (no gatekeeper) | -| Integrity | Trust the platform | Content-addressed (hash-verified) | -| Execution | JavaScript in WebView (unrestricted) | WASM sandbox (capability-based) | -| Capabilities | Platform APIs (payments, camera, etc.) | Blockchain-native (consensus, identity, state, messaging) | -| Updates | Platform-mediated | Author updates ENS -> instant propagation | -| Censorship resistance | Platform can ban apps | ENS + Swarm = no single point of removal | -| Interoperability | Walled garden | Modules from any author, any domain | -| Communication | Platform's messaging API | Waku (decentralised, privacy-preserving) | - -**Permissions model:** - -The super app adds a capability-grant layer on top of the WIT world. When a module is installed, the user reviews what it imports: - -``` -"TWAP Monitor" requests: - ✓ chain - read blockchain state (chains: 42161) - ✓ identity - sign with your accounts - ✓ local-store - store data on your device - ✓ remote-store - read/write to Swarm network - ✓ messaging - send/receive messages (topics: /nexum/1/twap-*) - ✗ ui - (not requested - event-driven module) - ✓ client - submit intents to installed venues (cow) - - [Allow] [Deny] -``` - -The host only links interfaces the user has approved. A module that doesn't import `messaging` structurally cannot publish messages - the same structural sandboxing property that the server runtime uses (doc 01). - -## Host Adapter Specification - -Any platform that wants to run modules must implement the **Host Adapter** - the set of host functions backing the WIT interfaces. The specification defines the contract: - -### Required Behaviours - -Each interface returns its own typed error over the shared `fault` vocabulary (`unsupported`, `unavailable`, `denied`, `rate-limited`, `timeout`, `invalid-input`, `internal`). The fault case is normative - embedders MUST pick the most specific case for each backend failure. See ADR-0011 for the embedder-side mapping table. - -**`chain::request` / `chain::request-batch`** (Chain) -- MUST forward the JSON-RPC request to a provider for the given chain. -- MUST return the JSON-encoded result (the `result` field from the JSON-RPC response). -- MUST return `chain-error` for provider errors, method-not-found, and transport failures. A structured JSON-RPC error (a node code plus decoded revert bytes) MUST use the `rpc` case; otherwise use a `fault`: `invalid-input` for method-not-found, `unavailable`/`timeout` for transport, `rate-limited` for 429s, `denied` for 401/403. -- SHOULD enforce a method allowlist (configurable by the operator/user). -- MAY apply middleware (timeout, retry, rate-limit, fallback) - this is platform-specific. - -**`identity::accounts/sign/sign-typed-data`** (Identity) -- `accounts` MUST return the list of available account identifiers (addresses) for the current host configuration. -- `sign` MUST produce a valid cryptographic signature over the provided data using the specified account's private key. -- `sign-typed-data` MUST produce a valid EIP-712 signature over the provided typed data structure. -- MUST return a `fault`. User rejection is `denied`; unknown account is `invalid-input`; backend offline is `unavailable`. -- MAY prompt the user for approval before signing (platform-dependent - e.g. wallet extension popup in WebView, biometric prompt on mobile). -- SHOULD NOT expose private key material to the module. The module sends data in, gets a signature out. - -**`local-store::get/set/delete/list-keys`** -- MUST provide per-module isolation (module A cannot read module B's state). -- MUST persist across module restarts within the same host process/session. -- SHOULD persist across host process restarts (platform-dependent). -- MAY enforce size quotas. If exceeded, `set` returns `fault.invalid-input` (not a trap). -- MAY provide transactional semantics. Modules SHOULD NOT rely on this across platforms. - -**`remote-store::upload/download/read-feed/write-feed`** -- MUST route to a Swarm-compatible node or gateway. -- `upload` MUST return the 32-byte content reference of the stored data. -- `download` MUST return the raw bytes for a valid reference, or `fault.unavailable` for missing/unreachable content. -- `write-feed` signs with the host's identity. The owner is implicit. -- MAY return `fault.unavailable` for offline / no-node-configured. - -**`messaging::publish/query`** -- MUST route `publish` to a Waku-compatible node. -- `publish` MUST deliver the message to the content topic's relay network on a best-effort basis. -- `query` SHOULD return historical messages if the host's Waku node supports the store protocol. -- `query` MAY return an empty list or `fault.unsupported` if store is unavailable. -- MAY apply rate limits (returning `fault.rate-limited`) to prevent message spam. - -**`logging::log`** -- MUST accept log calls without blocking or erroring. -- MAY discard logs (e.g. below a configured level threshold). -- Output destination is entirely host-specific. - -**Event dispatch (`on-event`)** -- MUST call `init(config)` exactly once before any `on-event` calls. -- MUST call `on-event` for each subscribed event (per manifest). -- MUST support all four event variants: `block`, `logs`, `tick`, `message`. -- SHOULD guarantee in-order delivery within a single module. -- MAY dispatch events concurrently across modules. -- SHOULD handle panics/traps gracefully (restart module, not crash host). - -### Optional Behaviours (Platform-Specific) - -| Capability | Server | Mobile | WebView | -|------------|--------|--------|---------| -| Fuel metering | Yes (wasmtime) | Maybe (engine-dependent) | No (browser engine) | -| Epoch interruption | Yes (Tokio task) | No | No (browser manages scheduling) | -| Memory limits | Yes (`ResourceLimiter`) | Limited (engine-dependent) | No (browser enforces its own limits) | -| Transactional state | Yes (redb write txn) | Optional (SQLite txn) | No (IndexedDB is async) | -| WebSocket subscriptions | Yes | Limited (background constraints) | Yes (if tab is active) | -| Push-based events | N/A | Yes (FCM/APNs) | N/A | -| Waku full relay | Yes | No (light client) | Maybe (js-waku) | - -## Content-Addressed Distribution: Works Everywhere - -The packaging and distribution model (doc 02, 03) is already platform-agnostic: - -``` -Module author: - 1. Build WASM component - 2. Create manifest (module.toml) - 3. Upload bundle to Swarm -> get content hash - 4. Set ENS contenthash -> content hash - -Any host (server, mobile, WebView): - 1. Resolve ENS name -> contenthash - 2. Fetch bundle from Swarm (or IPFS/OCI/HTTP gateway) - 3. Verify sha256(module.wasm) matches manifest - 4. Load module -``` - -The only platform-specific part is **how** the host fetches from Swarm: -- Server: direct Bee API -- Mobile: Bee gateway over HTTP -- WebView: fetch API to Bee gateway - -The content hash is the trust anchor. The transport is interchangeable. +Mobile (Flutter embedding a WASM runtime), WebView (browser WASM engine with host functions injected over a JavaScript bridge), and a decentralised super app (ENS discovery plus Swarm distribution over the same interfaces) are design directions the portable WIT contract is shaped to admit; none ships in 0.2. ## SDK Layering -The SDK mirrors the architecture, one crate per layer, with no re-export between them. See [doc 05](05-sdk-design.md) for the full treatment. - -- **`nexum-sdk` (shipped)** - the universal Rust SDK for any module targeting `nexum:host/event-module`. It ships the host-trait seam (`ChainHost`, `LocalStoreHost`, `LoggingHost`, supertrait `Host`), `Fault` / `ChainError`, the `bind_host_via_wit_bindgen!` adapter macro, the `#[nexum_sdk::module]` attribute macro, chain / config / address helpers, the `http` fetch seam over wasi:http, the keeper store primitives, and the guest tracing facade. Would additionally provide `HostTransport` (alloy `Transport` trait over `chain::request` / `chain::request-batch`), `provider(chain_id)`, `TypedState` (serde over `local-store`), `RemoteStore`, `Messaging`, and `Signer` typed wrappers as future direction. Any module author - CoW, DeFi, gaming, whatever - uses this. - -- **`videre-sdk` (shipped)** - the venue layer, serving both venue sides: the `VenueAdapter` trait under `#[videre_sdk::venue]` for adapter authors, and the `IntentBody` codec, typed `VenueClient` and `#[videre_sdk::keeper]` for keeper authors, plus the generic run assembler and the `videre-test` conformance kit alongside. - -- **Per-venue crates (shipped for CoW)** - each domain ships as crates on the venue layer, not as an SDK layer: `cow-venue` (body codec, typed client, adapter component) and `composable-cow` (conditional-order keeper machinery). A new domain adds its own. - -A generic automation module depends only on `nexum-sdk`; a keeper adds `videre-sdk` and the venue's body crate; a venue adapter depends on `videre-sdk` and its own crate. - -For **non-Rust** module authors (JavaScript, Python, Go, C++), the SDK is unnecessary - they use `wit-bindgen` directly against the WIT package for their target world. The WIT is the universal contract; the SDK is a Rust ergonomics layer on top. - -## Summary - -### Primitive Taxonomy - -| Primitive | Interface | Implementation | Persistence | Scope | -|-----------|-----------|---------------|-------------|-------| -| Chain | `chain` | JSON-RPC (eth_*) | Blockchain | Global (chain) | -| Identity | `identity` | Keystore / KMS / HSM | Key material | Per-account | -| Local Store | `local-store` | redb / SQLite / IndexedDB | Device-local | Per-module | -| Remote Store | `remote-store` | Ethereum Swarm | Decentralised | Global (content-addressed) | -| Messaging | `messaging` | Waku | Ephemeral | Topic-based pub/sub | -| Logging | `logging` | tracing / console | None | Diagnostic | - -### Architecture - -| Concept | Scope | -|---------|-------| -| `nexum:host` WIT package | Universal - any blockchain app, any platform | -| `event-module` world (0.2, shipping) | Event-driven modules - server today, mobile/background planned | -| `query-module` world (0.2 experimental) | Request/response modules - WIT published, no host impl in 0.2 | -| `app-module` world | Interactive modules - design only; planned hosts | -| `videre:types` / `videre:value-flow` / `videre:venue` WIT packages | Venue-neutral intent contract: types, asset vocabulary, client + adapter faces, venue-adapter world | -| `shepherd:cow` WIT package | CoW event-ABI package of record (`cow-events` only; the legacy `cow-api` read path and `shepherd` world are retired) | -| Venue adapter components | The domain-extension mechanism: `#[videre_sdk::venue]` components installed into the videre platform (`cow-venue` shipped) | -| `nexum-sdk` crate (shipped) | Universal Rust SDK: host-trait seam (ADR-0009), Fault / ChainError, bind macro, module macro, chain / config / address helpers, keeper store primitives, guest `http` helper, tracing facade | -| `videre-sdk` crate (shipped) | Venue Rust SDK: VenueAdapter + venue macro, IntentBody codec, typed VenueClient + keeper macro, run assembler; `videre-test` conformance kit alongside | -| Content-addressed distribution | Platform-agnostic (Swarm/IPFS, ENS discovery, hash verification) | -| Host Adapter | Platform-specific implementation of universal interfaces | - -The module binary is the portable artifact. The WIT contract is the universal interface. The host adapter is the platform-specific implementation. Everything else - packaging, distribution, discovery, SDK - layers cleanly on top. +[Doc 05](05-sdk-design.md) owns the SDK. In summary: `nexum-sdk` is the universal SDK for `nexum:host/event-module`; `videre-sdk` is the venue layer serving both venue faces; per-venue crates (`cow-venue`, `composable-cow`) sit on the venue layer. No crate re-exports another. Non-Rust authors use `wit-bindgen` directly against the WIT. diff --git a/docs/design/linker-extension-seam.md b/docs/design/linker-extension-seam.md index f24d4b19..c167da1e 100644 --- a/docs/design/linker-extension-seam.md +++ b/docs/design/linker-extension-seam.md @@ -1,121 +1,89 @@ # The linker extension seam -## Why +## What The core host binds the `nexum:host/event-module` world: the six core primitives (chain, identity, local-store, remote-store, messaging, logging) -plus the allowlisted wasi:http outgoing surface. A domain capability such as -cow-api is not a core seam. It plugs into the host through an extension seam -that is assembled at the composition root, so the core runtime compiles and -runs with no domain backend at all (`Ext = ()`, no hooks registered). - -An extension contributes four things that travel together: - -1. an `Ext` payload carrying its backend, held in the runtime `HostState`; -2. a linker hook that adds its WIT interfaces to each module linker; -3. a capability namespace so enforcement recognises its imports; -4. its own operator config, parsed from the `[extensions.]` table - of `engine.toml`. - -## The seam - -### `Ext` slot and the `ExtState` accessor - -`RuntimeTypes` names an associated `Ext: Clone + Send + Sync + 'static`. The -per-module `HostState` holds one `ext: T::Ext`. The generic accessor -trait is the load-bearing piece: - -```rust -pub trait ExtState { - type Ext; - fn ext(&self) -> &Self::Ext; -} -impl ExtState for HostState { - type Ext = T::Ext; - fn ext(&self) -> &Self::Ext { &self.ext } -} -``` - -An extension defines its own `bindgen!` for its world. That generates a -`Host` trait local to the extension. The extension implements it for the -foreign `HostState`, which is orphan-legal because the trait is local. -To reach its own backend without knowing the concrete lattice `T`, the impl -goes through `ExtState::ext`, then bounds the payload on an -extension-defined trait: - -```rust -pub trait CowBackend { type Cow: CowApi; fn cow(&self) -> &Self::Cow; } - -impl cow_bindings::...::Host for HostState -where T: RuntimeTypes, T::Ext: CowBackend { - async fn request(&mut self, ...) { self.ext().cow().request(...).await } -} -``` - -Two traits, two owners: `ExtState` is the runtime's generic reach into the -slot; `CowBackend` is the extension's own payload shape. The bindgen shares -`nexum:host/types` with the core bindings via `with`, so the extension's -`fault` is the same type the core host constructs, and `cow-api-error` embeds -it alongside the extension's own `http` and `rejected` cases. - -### Linker hook and capability registry - -An extension is one value: - -```rust -pub struct Extension { - pub link: LinkerHook, // Arc Result<()>> - pub capabilities: NamespaceCaps, // { prefix, ifaces } -} -``` - -`build_linker` binds the core world then runs each hook. `CapabilityRegistry` -starts from the core namespace (`nexum:host/`) and registers each extension's -namespace; `enforce_capabilities` and manifest name validation both consult -it. The composition root (`nexum-cli`'s `launch::run_from_config`) assembles -the `Extension` list once and threads it into the generic -`nexum_runtime::bootstrap::run`, which builds the linker and the registry -from it. The supervisor caches the list so the module-restart path rebuilds -an identical linker. - -An extension such as cow-api lives in its own crate (`shepherd-cow-host`) -that depends on the runtime for the seam types (`HostState`, `Extension`, -the `nexum:host/types` bindgen) and is depended on by `nexum-cli` at the -composition root. The runtime carries no dependency on any extension crate, -so the cow cone stays out of the bare engine. - -The hook takes only `&mut Linker`, never the wasmtime `Store` (which is not -`Sync`). This keeps the seam compatible with a future per-extension call -router that serialises access to a `Store`. - -### Extension config - -`engine.toml` stays domain-free. The engine deserialises every +plus the allowlisted `wasi:http` outgoing surface. A domain capability such +as a venue platform is not a core seam. It plugs into the host through an +extension assembled at the composition root, so the core runtime compiles +and runs with no domain backend at all (`Ext = ()`, no extensions +registered). + +## The `Extension` trait + +One trait, `Extension` (`host::extension`), is what a +domain contributes. Its members: + +- `namespace()`: the namespace it owns; keys its service on `HostServices`. +- `capabilities() -> NamespaceCaps`: the `{ prefix, ifaces }` merged into + enforcement so a module importing its interfaces still validates. +- `link(&mut Linker>)`: adds its WIT imports to each worker + linker, after the core interfaces and before instantiation. Takes only + `&mut Linker`, never the wasmtime `Store` (not `Sync`), so the seam stays + compatible with a future per-extension call router that serializes access + to a `Store`. +- `service() -> Option>`: a type-erased service + published under the namespace on the shared `HostServices` map and + downcast at the call site. +- `provider() -> Option>>`: a provider component + kind (e.g. the venue-adapter kind) the extension installs. +- `manifest_sections`, `admit_provider`, `admit_worker`: the non-core + manifest sections it claims and its install-time predicates over them + (an `Err` refuses the install fail-fast). +- `subscriptions`, `events`: the manifest subscription kinds it emits and + the event sources it opens once the engine is booted. + +An extension defines its own `bindgen!` for its world, generating a `Host` +trait local to the extension, and implements it for the foreign +`HostState` (orphan-legal: the trait is local). It reaches its backend +either through the `HostServices` map +(`state.services.get::(namespace)`, downcast) or, for a per-store +payload, through the `ExtState` accessor over the lattice `Ext` slot +(`RuntimeTypes::Ext`, held as `HostState.ext`). The shipped venue platform +uses the service map. The bindgen shares `nexum:host/types` with the core +bindings via `with`, so the extension's `fault` is the same type the core +host constructs. + +## Registration and enforcement + +`CapabilityRegistry` starts from the core namespace (`nexum:host/`) and +registers each extension's namespace; `enforce_capabilities` and manifest +name validation both consult it. The composition root assembles the +`Vec>>` once and threads it through the runtime +builder (`with_extensions`), which builds the linker and the registry from +it; the supervisor caches the list so the module-restart path rebuilds an +identical linker. + +An extension lives in its own crate depending on the runtime for the seam +types (`HostState`, `Extension`, the `nexum:host/types` bindgen) and +depended on by the composition-root binary. The runtime carries no +dependency on any extension crate, so a domain cone stays out of the bare +engine. The `shepherd` binary registers one extension, the videre venue +platform (`videre_host::platform`), through its `Runtime::extensions` impl. + +## Extension config + +`engine.toml` stays domain-free. The engine deserializes every `[extensions.]` table into an opaque `toml::Value` (`EngineConfig::extensions`) and never interprets it; the composition root -hands the extension its own entry to parse into a typed struct (cow-api's -`CowConfig` reads `[extensions.cow]`, today one `orderbook_urls` per-chain -map). +hands each extension its own entry to parse. Venue adapter components +install from the `[[adapters]]` table. ## Normative rule: import narrowing and boot ordering Modules built through `#[nexum_sdk::module]` compile against a per-module world derived from their manifest's `[capabilities]`, so a module that -never declares cow-api has no cow-api import and boots with a core-only -linker by construction. Hand-rolled modules compiled against the supertype -world reach the same shape a weaker way: the `wasm-tools` pipeline elides -any WIT import the produced component does not exercise. A module that DOES -import an extension interface instantiates only if, before instantiation: +never declares an extension capability has no such import and boots with a +core-only linker by construction. A module that DOES import an extension +interface instantiates only if, before instantiation: -- the extension's linker hook is registered (else an unsatisfied-import trap), AND +- the extension's linker hook is registered (else an unsatisfied-import + trap), AND - the extension's capability namespace is registered (else the manifest's - declaration of that capability is rejected as unknown, or the imported - interface is not recognised as a declared capability). + declaration of that capability is rejected as unknown). Therefore the linker hook and the capability namespace of an extension MUST be registered as a pair, from the same `Extension` value, before any module is instantiated. Registering one without the other is a boot-time failure, -not a compile-time one. This is exercised in both directions: the runtime's -supervisor tests pin the negative (a cow-importing module fails to boot with -the extension absent), and `shepherd-cow-host`'s own boot tests pin the -positive (the same module boots and dispatches with the extension present). +not a compile-time one. From 08845bf3dce1f4dc50449044f7dfe39691ae44d5 Mon Sep 17 00:00:00 2001 From: mfw78 Date: Sat, 25 Jul 2026 03:30:11 +0000 Subject: [PATCH 2/2] docs: unwrap paragraphs to one logical line for diff-friendliness Hard-wrapped prose churns diffs: a one-word edit reflows the whole paragraph. Join each paragraph onto a single logical line and let it soft-wrap. Content unchanged (word and heading counts preserved); code fences, tables, lists, blockquotes and headings untouched. --- docs/design/linker-extension-seam.md | 75 ++++++---------------------- 1 file changed, 16 insertions(+), 59 deletions(-) diff --git a/docs/design/linker-extension-seam.md b/docs/design/linker-extension-seam.md index c167da1e..45703ef1 100644 --- a/docs/design/linker-extension-seam.md +++ b/docs/design/linker-extension-seam.md @@ -2,88 +2,45 @@ ## What -The core host binds the `nexum:host/event-module` world: the six core -primitives (chain, identity, local-store, remote-store, messaging, logging) -plus the allowlisted `wasi:http` outgoing surface. A domain capability such -as a venue platform is not a core seam. It plugs into the host through an -extension assembled at the composition root, so the core runtime compiles -and runs with no domain backend at all (`Ext = ()`, no extensions -registered). +The core host binds the `nexum:host/event-module` world: the six core primitives (chain, identity, local-store, remote-store, messaging, logging) plus the allowlisted `wasi:http` outgoing surface. A domain capability such as a venue platform is not a core seam. It plugs into the host through an extension assembled at the composition root, so the core runtime compiles and runs with no domain backend at all (`Ext = ()`, no extensions registered). ## The `Extension` trait -One trait, `Extension` (`host::extension`), is what a -domain contributes. Its members: +One trait, `Extension` (`host::extension`), is what a domain contributes. Its members: - `namespace()`: the namespace it owns; keys its service on `HostServices`. - `capabilities() -> NamespaceCaps`: the `{ prefix, ifaces }` merged into - enforcement so a module importing its interfaces still validates. +enforcement so a module importing its interfaces still validates. - `link(&mut Linker>)`: adds its WIT imports to each worker - linker, after the core interfaces and before instantiation. Takes only - `&mut Linker`, never the wasmtime `Store` (not `Sync`), so the seam stays - compatible with a future per-extension call router that serializes access - to a `Store`. +linker, after the core interfaces and before instantiation. Takes only `&mut Linker`, never the wasmtime `Store` (not `Sync`), so the seam stays compatible with a future per-extension call router that serializes access to a `Store`. - `service() -> Option>`: a type-erased service - published under the namespace on the shared `HostServices` map and - downcast at the call site. +published under the namespace on the shared `HostServices` map and downcast at the call site. - `provider() -> Option>>`: a provider component - kind (e.g. the venue-adapter kind) the extension installs. +kind (e.g. the venue-adapter kind) the extension installs. - `manifest_sections`, `admit_provider`, `admit_worker`: the non-core - manifest sections it claims and its install-time predicates over them - (an `Err` refuses the install fail-fast). +manifest sections it claims and its install-time predicates over them (an `Err` refuses the install fail-fast). - `subscriptions`, `events`: the manifest subscription kinds it emits and - the event sources it opens once the engine is booted. +the event sources it opens once the engine is booted. -An extension defines its own `bindgen!` for its world, generating a `Host` -trait local to the extension, and implements it for the foreign -`HostState` (orphan-legal: the trait is local). It reaches its backend -either through the `HostServices` map -(`state.services.get::(namespace)`, downcast) or, for a per-store -payload, through the `ExtState` accessor over the lattice `Ext` slot -(`RuntimeTypes::Ext`, held as `HostState.ext`). The shipped venue platform -uses the service map. The bindgen shares `nexum:host/types` with the core -bindings via `with`, so the extension's `fault` is the same type the core -host constructs. +An extension defines its own `bindgen!` for its world, generating a `Host` trait local to the extension, and implements it for the foreign `HostState` (orphan-legal: the trait is local). It reaches its backend either through the `HostServices` map (`state.services.get::(namespace)`, downcast) or, for a per-store payload, through the `ExtState` accessor over the lattice `Ext` slot (`RuntimeTypes::Ext`, held as `HostState.ext`). The shipped venue platform uses the service map. The bindgen shares `nexum:host/types` with the core bindings via `with`, so the extension's `fault` is the same type the core host constructs. ## Registration and enforcement -`CapabilityRegistry` starts from the core namespace (`nexum:host/`) and -registers each extension's namespace; `enforce_capabilities` and manifest -name validation both consult it. The composition root assembles the -`Vec>>` once and threads it through the runtime -builder (`with_extensions`), which builds the linker and the registry from -it; the supervisor caches the list so the module-restart path rebuilds an -identical linker. +`CapabilityRegistry` starts from the core namespace (`nexum:host/`) and registers each extension's namespace; `enforce_capabilities` and manifest name validation both consult it. The composition root assembles the `Vec>>` once and threads it through the runtime builder (`with_extensions`), which builds the linker and the registry from it; the supervisor caches the list so the module-restart path rebuilds an identical linker. -An extension lives in its own crate depending on the runtime for the seam -types (`HostState`, `Extension`, the `nexum:host/types` bindgen) and -depended on by the composition-root binary. The runtime carries no -dependency on any extension crate, so a domain cone stays out of the bare -engine. The `shepherd` binary registers one extension, the videre venue -platform (`videre_host::platform`), through its `Runtime::extensions` impl. +An extension lives in its own crate depending on the runtime for the seam types (`HostState`, `Extension`, the `nexum:host/types` bindgen) and depended on by the composition-root binary. The runtime carries no dependency on any extension crate, so a domain cone stays out of the bare engine. The `shepherd` binary registers one extension, the videre venue platform (`videre_host::platform`), through its `Runtime::extensions` impl. ## Extension config -`engine.toml` stays domain-free. The engine deserializes every -`[extensions.]` table into an opaque `toml::Value` -(`EngineConfig::extensions`) and never interprets it; the composition root -hands each extension its own entry to parse. Venue adapter components -install from the `[[adapters]]` table. +`engine.toml` stays domain-free. The engine deserializes every `[extensions.]` table into an opaque `toml::Value` (`EngineConfig::extensions`) and never interprets it; the composition root hands each extension its own entry to parse. Venue adapter components install from the `[[adapters]]` table. ## Normative rule: import narrowing and boot ordering -Modules built through `#[nexum_sdk::module]` compile against a per-module -world derived from their manifest's `[capabilities]`, so a module that -never declares an extension capability has no such import and boots with a -core-only linker by construction. A module that DOES import an extension -interface instantiates only if, before instantiation: +Modules built through `#[nexum_sdk::module]` compile against a per-module world derived from their manifest's `[capabilities]`, so a module that never declares an extension capability has no such import and boots with a core-only linker by construction. A module that DOES import an extension interface instantiates only if, before instantiation: - the extension's linker hook is registered (else an unsatisfied-import - trap), AND +trap), AND - the extension's capability namespace is registered (else the manifest's - declaration of that capability is rejected as unknown). +declaration of that capability is rejected as unknown). -Therefore the linker hook and the capability namespace of an extension MUST -be registered as a pair, from the same `Extension` value, before any module -is instantiated. Registering one without the other is a boot-time failure, -not a compile-time one. +Therefore the linker hook and the capability namespace of an extension MUST be registered as a pair, from the same `Extension` value, before any module is instantiated. Registering one without the other is a boot-time failure, not a compile-time one.