From 7708023cb901a3968e0aee4a4f8f28d86a29a56e Mon Sep 17 00:00:00 2001 From: anthonyiscoding <142696453+anthonyiscoding@users.noreply.github.com> Date: Fri, 31 Jul 2026 13:03:53 -0500 Subject: [PATCH] docs(iii-directory): add worker microVM service-exposure skill Ships the verified microVM networking map, the guest-initiated tunnel pattern for reaching an in-VM HTTP/WebSocket service from a browser, and the console injectable-UI contract, as a skill leaf under iii-directory/skills/ (published as skills/worker-microvm-service.md, read as skill id iii-directory/worker-microvm-service). Co-Authored-By: Claude Opus 5 (1M context) --- .../skills/worker-microvm-service.md | 169 ++++++++++++++++++ 1 file changed, 169 insertions(+) create mode 100644 iii-directory/skills/worker-microvm-service.md diff --git a/iii-directory/skills/worker-microvm-service.md b/iii-directory/skills/worker-microvm-service.md new file mode 100644 index 000000000..cd9bc978b --- /dev/null +++ b/iii-directory/skills/worker-microvm-service.md @@ -0,0 +1,169 @@ +--- +title: Exposing a service that runs inside a worker microVM +type: how-to +description: >- + Locally-added workers run in libkrun microVMs whose network is egress-only: the + guest cannot reach its own TCP loopback and the host cannot dial in at all. + This is the verified networking map plus the guest-initiated tunnel pattern for + publishing any in-VM HTTP/WebSocket service to a browser, and the console + injectable-UI contract for embedding it. +--- + +# Exposing a service that runs inside a worker microVM + +When a worker runs a long-lived server of its own — an editor, a dev server, a +notebook, a database console, a preview renderer — the hard part is not starting +it, it is *reaching* it. A locally-added worker runs inside a libkrun microVM with +**egress-only** networking, so most of the obvious approaches fail silently. Read +the map below before writing any code that binds a port inside a worker. + +## 1. Is this worker in a VM? + +`engine::workers::info { name }` → `isolation: "libkrun"` with `os: "linux … (arm64)"` +means yes, whatever the host OS is. Workers added from a local project directory +(`worker::add { source: { kind: "local" } }`) and other managed Node/Python workers +get a VM; the always-on Rust workers (console, state, shell, cron, …) run natively +on the host. Consequences that bite immediately: + +* `process.platform` / `process.arch` are the **guest's** — on an Apple-silicon host + a worker must download `linux-arm64` artifacts, not `darwin-arm64`; +* `process.cwd()` is `/workspace` (the worker's own source, bind-mounted from the + host — writes there are real host edits and trip the source watcher), `$HOME` is + `/`, and everything else the worker writes lands in the VM's overlay; +* whatever the worker installs is installed **in the VM** — usually the point: + heavy, untrusted, or OS-specific payloads never touch the host. + +## 2. The networking map (verified, not assumed) + +The launcher (`iii worker __vm-boot --network`) runs a **userspace smoltcp stack on +the host** that proxies guest TCP **outward only**. The guest learns its addressing +from env vars: `III_INIT_IP` (e.g. `100.96.0.2`), `III_INIT_GW` (`100.96.0.1`), +`III_INIT_CIDR` (`30`), `III_INIT_DNS`. + +| from → to | works? | notes | +| --- | --- | --- | +| guest → internet (`curl`, `wget`) | **yes** | DNS via `/etc/resolv.conf` = the gateway | +| guest → **gateway** `$III_INIT_GW:PORT` | **yes** | this *is* the host's `127.0.0.1:PORT` — the only route to the host | +| guest → `127.0.0.1` (its own loopback) | **no** | there is no `lo` route; packets exit via `eth0` and time out | +| guest → its own `III_INIT_IP` | **no** | the userspace stack does not hairpin | +| host → guest, any address/port | **no** | no route, no ARP entry, no port publishing — `--network` is egress-only | +| guest → engine `ws://localhost:49134` | yes | pre-wired; the engine sees the worker as `127.0.0.1` | +| host → guest via `iii worker exec` | yes | multiplexed virtio-console channel, not TCP — the only ingress that exists | + +Two traps worth real time: + +1. **A worker cannot health-check its own service over TCP.** Binding `0.0.0.0:PORT` + and probing `127.0.0.1:PORT` times out even though `/proc/net/tcp` shows the + listener. Put the service on a **unix socket** and probe with + `curl --unix-socket http://localhost/`. +2. **Node's global `fetch` (undici) is unreliable in-guest**, failing with + `UND_ERR_CONNECT_TIMEOUT` on routes where `curl` succeeds. Shell out to + `curl`/`wget` for anything load-bearing; keep `fetch` as a fallback only. For + download progress, poll `fs.stat(dest).size` against a + `curl -sSIL | grep -i content-length` probe rather than streaming. + +## 3. The pattern: a guest-initiated tunnel + +Since ingress does not exist, **the guest must dial out** and the pairing happens on +the host. This shape carries HTTP, static assets *and* WebSocket upgrades +(verified `101 Switching Protocols`), which is why it succeeds where an +HTTP-function proxy cannot — a browser WebSocket cannot be served through function +calls, and a Service Worker cannot intercept WS. + +``` +browser ──▶ host 127.0.0.1:PUBLIC ─┐ + ├─ paired by a tiny host byte-pump +guest ──▶ host 127.0.0.1:TUNNEL ─┘ + (pool of idle outbound sockets, dialed at $III_INIT_GW:TUNNEL) + │ + ▼ + unix socket ──▶ the real service, inside the VM +``` + +1. **Service on a unix socket** in the VM (most servers support this: VS Code's + `--socket-path`, node/express `listen(path)`, gunicorn `--bind unix:…`). +2. **Host byte-pump**: two loopback listeners — `PUBLIC` for browsers, `TUNNEL` for + the guest. It copies bytes only: no application code, no state, no filesystem + access. Keep it small enough to audit at a glance. +3. **Guest keeps a pool** (≈8) of connections to `$III_INIT_GW:TUNNEL`, each opening + with a handshake line (`MAGIC \n`). The pump parks them idle; per browser + connection it pops one, writes `GO\n`, and pipes both directions. The guest end, + on `GO`, connects to the unix socket and pipes — **forwarding any bytes that + arrived in the same chunk after `GO\n`** (the classic prefix-buffer bug) — then + opens a replacement idle socket. +4. Pool hygiene: `setNoDelay` + `setKeepAlive` both ends, ≈1.5s backoff on failed + dials so a dead pump does not spin, and have the pump wait a few seconds for a + fresh tunnel socket instead of dropping a browser connection when the pool is dry. +5. Health is **two probes**: service-up (`curl --unix-socket`) and end-to-end + (`curl http://$III_INIT_GW:PUBLIC/` from the guest, which traverses pump + + tunnel + socket). Only report "running" when the second one passes — that is what + the browser will actually experience. + +### Bootstrapping the host side from inside the VM + +The worker installs and launches its own host helper over the bus — no hardcoded +host paths, no manual setup step, no separate install instructions: + +```js +// write it: stdin carries the source, bash writes it on the HOST +await worker.trigger({ function_id: "shell::exec", payload: { + command: "bash", + args: ["-lc", `PID=$(cat ${PIDFILE} 2>/dev/null || true); [ -n "$PID" ] && kill "$PID" 2>/dev/null; cat > ${REMOTE}`], + stdin: await fs.readFile(pumpSourceInsideTheVM, "utf8"), +}}); +// run it: exec_bg outlives the call; env carries ports + a per-boot secret +const { job_id } = await worker.trigger({ function_id: "shell::exec_bg", payload: { + command: "bash", args: ["-lc", `exec node ${REMOTE}`], + env: { PUBLIC_PORT: "3210", TUNNEL_PORT: "3211", SECRET: secret, PIDFILE }, +}}); +``` + +* use `bash -lc "exec node …"` so a login shell resolves the interpreter on the host; +* keep a **pidfile** and kill the previous pid on start — `shell::exec_bg` jobs are + children of the shell worker and die with it, so start must be idempotent and + survivable across worker restarts; +* bind the pump to **loopback only** and treat the handshake secret as per-boot; +* the pump is the one host-side artifact. Be explicit about it: the payload stays in + the VM, but a byte-relay on the host is unavoidable while ingress does not exist. + +## 4. Debugging playbook + +* `iii worker exec --no-tty -- /usr/bin/curl -sS -m 4 http://$GW:PORT/` — a + shell **inside** the VM; the only host→guest path. +* `cat /proc/net/route` (only `eth0` rows ⇒ no loopback) and `cat /proc/net/tcp` + (hex ports — `0C8A` = 3210, state `0A` = LISTEN) when `ip`/`ss` are missing from + the image. +* `worker::status { name }` → `stderr_tail` carries VM boot + dependency install; + the boot line also reveals mounts (`/mnt/host-src`, `/opt/iii`) and the erofs + overlay. `worker::logs` for more. +* Ship a `::diag` function returning guest addressing, routes, DNS, egress + status, socket listing and both probes. Cheaper than guessing, and it keeps the + VM's internals observable from the bus. +* Prove the round trip with the *service's own* logs (e.g. a remote-agent server + logging a management/extension-host connection) rather than a screenshot. + +## 5. Embedding it in the console + +A worker ships console UI as assets; no console rebuild: + +* register `console:script` with `config { path: "/page.js" }` and + `console:style` with `"/styles.css"`. The trigger's `function_id` is a + content function returning `{ content, content_type }` — read from disk per call + and the dev loop is edit + reload the tab. Re-registering a path overrides it. +* the script is **ESM with `export default function setup(host)`**, optionally + returning a cleanup function. Bare imports resolve through the console's static + import map — `react`, `react-dom`, `react-dom/client`, `react/jsx-runtime`, + `@iii-dev/console-ui` — so hand-written `createElement` needs **no bundler**. +* `host` = `{ iii: { trigger(fnId, payload), on, registerTrigger, browserId, + addConnectionStateListener }, components, useTheme, path, pages, functionTriggers, + configForms }`. +* `host.pages.register({ id, title, render })` → a page routed at `#/ext/`; + `host.configForms.register(configId, Component)` overrides a worker's config form; + `host.functionTriggers.register(renderer)` adds trigger-specific UI. +* verify with `console::ui-manifest` (paths, hashes, style-lint warnings, per-worker + enable state); users can disable a worker's UI via the console config + (`injectableUi.disabledWorkers`). +* an `