feat(truapi-provider): native and wasm ChainProvider with WebSocket and embedded smoldot backends - #276
feat(truapi-provider): native and wasm ChainProvider with WebSocket and embedded smoldot backends#276BigTava wants to merge 50 commits into
Conversation
…et and embedded smoldot backends
# Conflicts: # CLAUDE.md # README.md
…ork build for embedded smoldot
…ate behind a uniffi feature
…d-client locks, panic-free light-client builder, and Apache-2.0 attribution for the vendored platform
… GenericError, and require at least one backend feature
…ed platform client identity, a WS disconnect test, and doc/dep-pin notes
…inProvider since it also runs on wasm32
…e seeding and their wasm JS bindings
…asm light-client test builder call
…n/task/time/num types
…ir last parachain connection closes
…heck plus headless browser tests
|
For reference, the design so far has been
Owning the connection end to end make sense as well, but might require some research as well to understand the trade-off, especially on native platform like iOS where a networking stack built on top of the Native SDK might Anyway that should not prevent you from testing this PR out. The core should probably now own the networking settings preference that currently live in dotli |
| /// Warm-start database blob previously returned by the | ||
| /// `chainHead_unstable_finalizedDatabase` JSON-RPC function. Invalid | ||
| /// blobs are silently ignored by smoldot. | ||
| database_content: Option<String>, |
There was a problem hiding this comment.
Isn't this returned as a String in smoldot? Happy to change it.
…bundled specs borrow like smoldot's &str
…ation` to match smoldot
…g instead of a caller-supplied option
…ilding the crate to browser WASM
…e a parachain's relay as provider topology
…m publish automation with a dev-tag snapshot script
…ed and drop the caller-facing toggle
…build through the npm automation
…e public builder and construct catalog sources directly
# Conflicts: # .github/workflows/ci.yml # .github/workflows/release.yml # Cargo.lock
# Conflicts: # Cargo.lock # README.md # deny.toml
…ro and per-platform usage
# Conflicts: # Cargo.lock
# Conflicts: # README.md
…nd correct catalog genesis hashes
… genesis drift at the catalog
…are authorities at the pinned block
# Conflicts: # Cargo.lock
…ewnet checkpoint against its other validators
…and Android packages
…ckages and correct the Android distribution claims
| return false; | ||
| } | ||
| if (theirs.length === 0) { | ||
| console.log(" ? " + peer + " served no authorities, confirming the block only"); |
There was a problem hiding this comment.
I believe this fails open. If the peer cant serve the authorities we treat that as agreement, so the corroboration passes without ever comparing them. And state_call is exactly what public endpoints usually refuse or cant answer on a pruned block, so thats the normal path not the rare one.
I tested it. A peer that serves the real parent hash but refuses state_call will accept an all zero authority set. The same peer serving its real authorities rejects it correctly, so the check works when it gets an answer and just skips when it doesnt.
Nothing validates the checkpoint at runtime, we hand it straight to add_chain, so this decides the authority set every light client warp syncs from. Its testnet only right now so not urgent, but can we make an unavailable state_call a hard failure instead of counting it as agreement?
| blockHash = await rpc(peer, "chain_getBlockHash", [number]); | ||
| if (!blockHash) { | ||
| console.log(" ? " + peer + " has not reached block " + number); | ||
| continue; | ||
| } | ||
| peerParent = (await rpc(peer, "chain_getHeader", [blockHash])).parentHash; | ||
| } catch (error) { | ||
| console.log(" ? " + peer + " did not answer (" + (error.message || error) + ")"); | ||
| continue; | ||
| } | ||
| if (peerParent.toLowerCase() !== parentHash.toLowerCase()) { | ||
| console.log(" x " + peer + " disagrees at block " + number); | ||
| console.log(" checkpoint parent " + parentHash); | ||
| console.log(" peer parent " + peerParent); | ||
| process.exit(1); | ||
| } |
There was a problem hiding this comment.
I think we only ever check the parent hash here, never the checkpoint header itself. So the state root, extrinsics root and digest are never compared against the peer, which means the checkpoint's own block hash is never verified. A parent hash is shared by every sibling at that height so its cheap to reproduce.
To test this I did it with the real header from networks/paseo.json and just overwrote the state root. The peer agreed on the block, served all 56 real authorities, and it still came back corroborated. Output was identical to the unmodified run.
This also weakens the authority check above it, since the block hash we pass to state_call comes from the peer rather than from our checkpoint. So if the header is forged with the same parent were comparing authorities at a different block than the one were pinning.
Can we hash the checkpoint header and compare it to chain_getBlockHash for that number?
| /// JSON-RPC queue budgets for chains added by this backend. The provider is a | ||
| /// trusted in-process client, so the pending cap is generous (smoldot's docs | ||
| /// sanction up to `u32::MAX` for trusted callers); an overflow is still handled | ||
| /// gracefully by synthesizing an error response rather than hanging the caller. | ||
| const MAX_PENDING_REQUESTS: u32 = 1024; | ||
| const MAX_SUBSCRIPTIONS: u32 = 1024; |
There was a problem hiding this comment.
Heads up, these two constants are inert. smoldot stores them and never reads them, and the request channel is async_channel::unbounded() with their own // TODO: capacity? next to it. So the comment here about overflow being handled gracefully, and the synthetic error frame below it, are written against behaviour that doesnt exist.
I measured it. A product that calls send() in a loop and never drains responses() got 1.3 million requests queued in 3 seconds at 470MB rss, zero rejections.
I checked smoldot main too and its the same at 2.0.0, so a version bump doesnt help. ws.rs already does this properly with a bounded buffer and a test. Can we bound the light path the same way? I'll open an issue upstream for the docs.
Cut the "no test for queue full" line, the bounded(16) asymmetry, and the wasm/FFI consequence — the 470MB number carries that on its own.
Happy to share how I measured this if youd like.
| /// Yield every inbound text frame; the stream ends on transport error or EOF, | ||
| /// which is the disconnect signal consumers rely on. | ||
| fn response_stream<R: TransportReceiverT + Send>(receiver: R) -> impl Stream<Item = String> + Send { | ||
| stream::unfold(receiver, |mut receiver| async move { | ||
| loop { | ||
| match receiver.receive().await { | ||
| Ok(ReceivedMessage::Text(text)) => return Some((text, receiver)), | ||
| Ok(ReceivedMessage::Bytes(bytes)) => match String::from_utf8(bytes) { | ||
| Ok(text) => return Some((text, receiver)), | ||
| Err(_) => tracing::warn!("dropping non-UTF-8 binary WebSocket frame"), | ||
| }, | ||
| Ok(ReceivedMessage::Pong) => {} | ||
| Err(err) => { | ||
| tracing::debug!("WebSocket receive ended: {err}"); | ||
| return None; | ||
| } | ||
| } | ||
| } | ||
| }) | ||
| } |
There was a problem hiding this comment.
The comment says the stream ending is the disconnect signal consumers rely on, but on a half open socket it never ends. Theres no keepalive and no read timeout, and we handle Pong while nothing ever sends a Ping. I have a repro, the stream never terminates and the request downstream stays pending past 10s. Can we add a keepalive ping and a read timeout here?
| - name: Create Pull Request | ||
| if: steps.diff.outputs.changed == 'true' | ||
| uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1 | ||
| with: |
There was a problem hiding this comment.
Does this need a PAT or app token? With the default GITHUB_TOKEN github wont trigger workflows on the PR, so every_catalog_genesis_hash_matches_its_bundled_spec never runs on it.
| default: "" | ||
| # Push a `truapi-provider-dev` or `truapi-provider-dev-<suffix>` tag to publish | ||
| # a dev snapshot; the part after `truapi-provider-dev-` becomes the dist-tag suffix. | ||
| push: |
There was a problem hiding this comment.
Do we want this gated the same way release.yml is? Right now any truapi-provider-dev* tag push publishes to npm with no needs:?
| responses.next().await | ||
| } | ||
|
|
||
| /// Close the connection; pending `nextResponse()` calls resolve to |
There was a problem hiding this comment.
Worth documenting that calling this isnt optional.
… the native FFI surface with tests
5 participants
Summary
truapi-providercrate: a native andwasm32implementation oftruapi_platform::ChainProviderwhose per-chain backend is a remote WebSocket JSON-RPC node or an embedded smoldot light client.