diff --git a/docs/protocol/peer-network.md b/docs/protocol/peer-network.md index 5ec3fb2..193df64 100644 --- a/docs/protocol/peer-network.md +++ b/docs/protocol/peer-network.md @@ -622,6 +622,13 @@ Stream a byte range of a resource or capsule from this peer. - `chunk_index` — the index (into `chunk_lens`) of the first chunk in this frame. - `inclusion_proof` — the merkle inclusion proof of the **whole resource** against the capsule's generation `root` (first frame only), relayed verbatim ([Merkle inclusion proofs](./merkle-proofs.md)). For `capsule: true` the capsule self-verifies on install, so `inclusion_proof` is `null` (as with [`dig.getCapsule`](./dig-rpc.md)). +**Per-range proof fields (v0.4.0+, optional):** Any frame MAY additionally carry merkle proofs for the chunks in that frame — a forward-compatible extension that allows clients to verify each range against the chain-anchored root without awaiting the whole resource: + + - `range_proof` (optional, array of strings) — one base64-encoded merkle inclusion proof per chunk in this frame, in ascending chunk order. Each proof binds to the chunk's absolute index (see `first_chunk_index` below). Servers expand a byte-range request to the covering whole-chunk boundaries, so the returned frame's chunks are complete, verifiable units. + - `first_chunk_index` (optional, integer) — the absolute index (into the resource's `chunk_lens`) of the first chunk in this frame. Pairs with `range_proof[]` so each proof is indexed: `proof[i]` verifies chunk `first_chunk_index + i`. + +**Verification rule (normative):** Clients verify each range's proofs against their **own chain-anchored root** (the CHIP-0035 singleton's current on-chain `metadata.root_hash`), never against the frame's `root` field (which is advisory echo only). A frame lacking valid proofs against the client's pinned root is unverified content and MUST NOT be treated as trusted. The fields are optional for backwards compatibility with v0.3.0 servers; a frame without them still verifies via the first-frame `inclusion_proof` over the whole resource. + ### Per-range integrity — verify a range without the whole file {#range-integrity} A range fetched from one peer is **independently verifiable** against the capsule's on-chain merkle root, so a single peer cannot forge a range and multi-source pieces always reassemble correctly. Integrity aligns exactly with the [existing dig-store content model](./merkle-proofs.md): a resource is a sequence of AES-256-GCM-SIV chunks; the resource commits to the generation merkle root as a single leaf (`resource_leaf = SHA-256(concatenated chunk ciphertexts)`); `chunk_lens` fixes the chunk boundaries. @@ -728,7 +735,7 @@ The peer network is implemented by several crates that must interoperate byte-fo | **Peer exchange** | `RequestPeers`→`RespondPeers` of `TimestampedPeerInfo{host, port, timestamp}` (Chia-streamable, big-endian) | that nodes discover peers from each other identically | | **Peer RPC** | `dig.getPeers` / `dig.announce` / `dig.getNetworkInfo` + `-32006`; `dig.getAvailability` / `dig.listInventory`; `dig.fetchRange` + `-32007`, generated into [`openrpc-node.json`](https://docs.dig.net/openrpc-node.json) | the machine surface an agent drives; CI-diffable against a live node | | **Availability** | `dig.getAvailability` batch per-item answers at store / root / capsule granularity (`available` + `roots`/`total_length`/`chunk_count`/`complete`) | that a downloader can confirm a peer HOLDS content (and plan ranges) before any fetch | -| **Streaming + range** | `dig.fetchRange` streams `RangeFrame{offset,length,bytes,complete}`; first frame carries `total_length` + `chunk_lens` + `chunk_index` + `inclusion_proof`; ranges are chunk-aligned | that data streams (not buffered), and a single-peer range verifies against the chain-anchored root — so multi-source pieces reassemble and can't be forged | +| **Streaming + range** | `dig.fetchRange` streams `RangeFrame{offset,length,bytes,complete}`; first frame carries `total_length` + `chunk_lens` + `chunk_index` + `inclusion_proof`; any frame MAY carry `range_proof[]` (per-chunk proofs, v0.4.0+) + `first_chunk_index` (chunk start index); ranges are chunk-aligned | that data streams (not buffered), a single-peer range verifies against the chain-anchored root (whole-resource or per-chunk), and multi-source pieces reassemble verified — so they can't be forged | | **Range integrity** | a range maps to whole chunk(s); each verifies via `chunk_lens` + the whole-resource `inclusion_proof` against the on-chain `root` (same as [merkle-proofs](./merkle-proofs.md)) | that any peer's range is independently verifiable + a bad source is detectable without the whole file | | **NAT ladder** | the ordered strategies DIRECT → UPnP → NAT-PMP → PCP → hole-punch (relay signalling only) → RELAYED/TURN (relay carries data), relay-data-last | that every `connect(peer)` implementation prefers direct, prefers hole-punch signalling over full relaying, and proxies the stream only as a last resort | | **DHT content key** | `SHA-256(tag ‖ canonical bytes)` with tags `0x01` store (`store_id`), `0x02` root/capsule (`store_id ‖ root`), `0x03` resource (`store_id ‖ root ‖ retrieval_key`); node id = `peer_id`; distance = XOR; bucket = `255 − leading_zeros` ([§4c](#dht)) | that every node derives the identical content key for the same content, and places nodes + content in one 256-bit keyspace, so a provider record announced by one implementation is found by another | diff --git a/package.json b/package.json index 0f68e7a..cc40611 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "docs-dig-net", - "version": "0.5.2", + "version": "0.5.3", "private": true, "scripts": { "docusaurus": "docusaurus", diff --git a/scripts/dig-spec.mjs b/scripts/dig-spec.mjs index 9fbe8a8..d0502ca 100644 --- a/scripts/dig-spec.mjs +++ b/scripts/dig-spec.mjs @@ -211,6 +211,18 @@ export const schemas = { description: 'FIRST FRAME ONLY: base64 merkle inclusion proof of the WHOLE resource against the generation `root`, relayed verbatim. Verify the range against the CALLER-supplied chain-anchored root — the node is never the trust anchor. null for capsule fetches (capsule: true), which self-verify on install.', }, root: { ...hex64, description: 'FIRST FRAME ONLY: the resolved generation root the inclusion_proof is against.' }, + range_proof: { + type: 'array', + items: { type: 'string', contentEncoding: 'base64' }, + description: + 'OPTIONAL (v0.4.0+): one base64-encoded merkle inclusion proof PER CHUNK carried by this frame, in ascending chunk order — so every frame\'s chunks are independently verifiable, not just the first. Verify each proof against the CALLER\'s chain-anchored root (the frame\'s `root` is an advisory echo, never the trust anchor); a frame whose chunks lack a valid proof is unverified content. Absent from pre-0.4.0 peers (backwards compatible).', + }, + first_chunk_index: { + type: 'integer', + minimum: 0, + description: + 'OPTIONAL (v0.4.0+): the absolute index (into chunk_lens) of the FIRST chunk carried by this frame — pairs each range_proof entry with its chunk position. Absent from pre-0.4.0 peers.', + }, }, }, }; diff --git a/static/openrpc-node.json b/static/openrpc-node.json index 05ebdc8..cbe69ba 100644 --- a/static/openrpc-node.json +++ b/static/openrpc-node.json @@ -2,7 +2,7 @@ "openrpc": "1.2.6", "info": { "title": "dig RPC — node profile (local dig-node / in-process DIG Browser)", - "version": "0.5.0", + "version": "0.5.3", "description": "The NODE PROFILE: a distinct, smaller surface than the network profile. Of the byte methods it implements ONLY dig.getContent (local-first, else proxy); everything else proxies upstream or returns -32601. It ADDS node-only methods the security model depends on — chiefly dig.getAnchoredRoot (the CHIP-0035 on-chain head, the trusted root for mandatory root-pinning), dig.stage, and cache.*. Gate on dig.methods rather than assuming one uniform surface. See https://docs.dig.net/docs/protocol/dig-rpc#node-profile.", "license": { "name": "GPL-2.0", @@ -959,6 +959,19 @@ "type": "string", "pattern": "^[0-9a-f]{64}$", "description": "FIRST FRAME ONLY: the resolved generation root the inclusion_proof is against." + }, + "range_proof": { + "type": "array", + "items": { + "type": "string", + "contentEncoding": "base64" + }, + "description": "OPTIONAL (v0.4.0+): one base64-encoded merkle inclusion proof PER CHUNK carried by this frame, in ascending chunk order — so every frame's chunks are independently verifiable, not just the first. Verify each proof against the CALLER's chain-anchored root (the frame's `root` is an advisory echo, never the trust anchor); a frame whose chunks lack a valid proof is unverified content. Absent from pre-0.4.0 peers (backwards compatible)." + }, + "first_chunk_index": { + "type": "integer", + "minimum": 0, + "description": "OPTIONAL (v0.4.0+): the absolute index (into chunk_lens) of the FIRST chunk carried by this frame — pairs each range_proof entry with its chunk position. Absent from pre-0.4.0 peers." } } } diff --git a/static/openrpc.json b/static/openrpc.json index 34ac611..0dbdb76 100644 --- a/static/openrpc.json +++ b/static/openrpc.json @@ -2,7 +2,7 @@ "openrpc": "1.2.6", "info": { "title": "dig RPC — DIG Network Content Interface (network profile)", - "version": "0.5.0", + "version": "0.5.3", "description": "The network-wide read interface for DIG content over JSON-RPC 2.0 — the NETWORK PROFILE served by the canonical node at rpc.dig.net. Blind by construction (the node holds no URN and no key), verifiable without trust (merkle inclusion proofs against the chain-anchored root), and streamable at any size. There is no `decoy` field on the wire and no CDN. See https://docs.dig.net/docs/protocol/dig-rpc.", "license": { "name": "GPL-2.0", @@ -907,6 +907,19 @@ "type": "string", "pattern": "^[0-9a-f]{64}$", "description": "FIRST FRAME ONLY: the resolved generation root the inclusion_proof is against." + }, + "range_proof": { + "type": "array", + "items": { + "type": "string", + "contentEncoding": "base64" + }, + "description": "OPTIONAL (v0.4.0+): one base64-encoded merkle inclusion proof PER CHUNK carried by this frame, in ascending chunk order — so every frame's chunks are independently verifiable, not just the first. Verify each proof against the CALLER's chain-anchored root (the frame's `root` is an advisory echo, never the trust anchor); a frame whose chunks lack a valid proof is unverified content. Absent from pre-0.4.0 peers (backwards compatible)." + }, + "first_chunk_index": { + "type": "integer", + "minimum": 0, + "description": "OPTIONAL (v0.4.0+): the absolute index (into chunk_lens) of the FIRST chunk carried by this frame — pairs each range_proof entry with its chunk position. Absent from pre-0.4.0 peers." } } }