From 846412e6ec7dbf63cd4673c397c14245ec9d6edf Mon Sep 17 00:00:00 2001 From: Michael Taylor Date: Sun, 26 Jul 2026 14:26:37 -0700 Subject: [PATCH 1/4] chore: open the 0.6.0 docs coherence lane Co-Authored-By: Claude From 97a16246a6c832c15e440b919941940185ec0f3c Mon Sep 17 00:00:00 2001 From: Michael Taylor Date: Sun, 26 Jul 2026 14:33:11 -0700 Subject: [PATCH 2/4] docs(protocol): publish the paged range-frame metadata split and -32009 Range-frame metadata divides by whether it scales with the resource. The fixed-size identity set (root, total_length, chunk_count, plus chunk_index on a chunk-aligned window) rides EVERY frame, because root + total_length are what detect a wrong-generation holder as the first bytes arrive. The resource-scaling layout set (chunk_lens, inclusion_proof) rides the first frame or a paged prologue once per range stream, each page located by chunk_lens_offset, and is omitted entirely when a client sets skip_layout. chunk_lens is a decrypt input, not a verify input: per-chunk AES-GCM-SIV needs the whole array and a reader rejects one whose entries do not sum to total_length. That is why a large layout is paged rather than sliced, and it is stated on the page because an implementer who slices it ships a stream no client can decrypt. Also publishes the five frame bounds, records that a serve path splits on the payload cap rather than the per-request window, marks range_proof and first_chunk_index reserved, and adds -32009 RANGE_METADATA_UNREPRESENTABLE to the error taxonomy on both the prose page and the generated catalog (with its holder-fatal, not-a-transport-failure client rule). Co-Authored-By: Claude --- docs/protocol/peer-network.md | 59 ++++++++++++++++++++++++---------- docs/support/error-codes.md | 3 +- scripts/dig-spec.mjs | 30 +++++++++++------- static/error-codes.json | 14 ++++++++ static/openrpc-node.json | 60 +++++++++++++++++++++++++---------- static/openrpc.json | 47 ++++++++++++++++++--------- 6 files changed, 154 insertions(+), 59 deletions(-) diff --git a/docs/protocol/peer-network.md b/docs/protocol/peer-network.md index b5c1902..86a974d 100644 --- a/docs/protocol/peer-network.md +++ b/docs/protocol/peer-network.md @@ -661,8 +661,9 @@ Stream a byte range of a resource or capsule from this peer. - **Resource identity.** For a content resource: `store_id` + `retrieval_key` (+ optional `root`, defaulting to the chain-anchored tip). For a whole capsule / `.dig`: set `capsule: true` and identify it by `store_id` (+ optional `root`); `retrieval_key` is then omitted. The capsule identity is `[:]`. - **Range.** `offset` (bytes into the resource ciphertext, default `0`) and `length` (bytes to return). `length` is clamped to the node's window (3 MiB); a request whose range is not chunk-aligned is widened to whole-chunk boundaries (see integrity below), so the response may return slightly more than asked. + - **`skip_layout`** (optional, bool) — set it to `true` when you already hold the resource's layout and commitment. The peer then omits the layout metadata described below and streams data frames only. -- **result:** a **stream** ([§8](#streaming)) of `dig.fetchRange` frames. Beyond the base frame fields, the **first frame** (`offset == range start`) carries the verification metadata for the range: +- **result:** a **stream** ([§8](#streaming)) of `dig.fetchRange` frames. Frame metadata divides into two sets, by whether it scales with the resource: ```json { @@ -670,25 +671,50 @@ Stream a byte range of a resource or capsule from this peer. "length": 262144, "bytes": "", "complete": false, + "root": "<64hex>", "total_length": 10485760, - "chunk_lens": [262144, 262144, 131072], + "chunk_count": 3, "chunk_index": 0, - "inclusion_proof": "", - "root": "<64hex>" + "chunk_lens": [262144, 262144, 131072], + "chunk_lens_offset": 0, + "inclusion_proof": "" } ``` - - `total_length` — the full resource ciphertext length (so a client can plan its ranges). - - `chunk_lens` — the per-chunk ciphertext lengths of the **whole resource**, in order (first frame only) — identical to the [dig RPC `chunk_lens`](./dig-rpc.md#the-chunk-wire-object). This is how a client maps a byte range to the chunk(s) that cover it. - - `chunk_index` — the index (into `chunk_lens`) of the first chunk in this frame. - - `inclusion_proof` — the merkle inclusion proof of the **whole resource** against the capsule's generation `root` (first frame only), relayed verbatim ([Merkle inclusion proofs](./merkle-proofs.md)). For `capsule: true` the capsule self-verifies on install, so `inclusion_proof` is `null` (as with [`dig.getCapsule`](./dig-rpc.md)). +**On EVERY frame — the fixed-size identity set.** Each of these is one scalar, so it costs the same on a three-chunk resource as on a million-chunk one: + + - `root` — the generation the peer served this range from. Advisory echo (see the verification rule below). + - `total_length` — the full resource ciphertext length, so a client can plan its ranges. + - `chunk_count` — the number of chunks in the whole resource. + - `chunk_index` — the index (into `chunk_lens`) of this frame's first chunk. Present when the frame's window is chunk-aligned. + +They ride every frame because `root` + `total_length` are what detect a peer serving a **different generation** than the one you asked for, and they detect it as the first bytes arrive rather than at the end of the stream. + +**Once per range stream — the resource-scaling layout set.** These grow with the resource, so they are delivered once and reused for every subsequent frame of the stream: + + - `chunk_lens` — the per-chunk ciphertext lengths of the **whole resource**, in order — identical to the [dig RPC `chunk_lens`](./dig-rpc.md#the-chunk-wire-object). This is how a client maps a byte range to the chunk(s) that cover it. It rides the first frame when it fits there, and otherwise a **paged prologue**: successive frames each carry up to `MAX_CHUNK_LENS_PER_FRAME` entries, and the reader concatenates the pages into one array of `chunk_count` entries before it decrypts. + - `chunk_lens_offset` — the index into the resource's `chunk_lens` at which this frame's page begins. Every page states its own offset, so pages are placed without depending on arrival order. + - `inclusion_proof` — the merkle inclusion proof of the **whole resource** against the generation `root`, relayed verbatim ([Merkle inclusion proofs](./merkle-proofs.md)). For `capsule: true` the capsule self-verifies on install, so `inclusion_proof` is `null` (as with [`dig.getCapsule`](./dig-rpc.md)). A client that requested `skip_layout` receives no layout set at all. + +**Why the layout is delivered whole, and never sliced to the frame's own chunks.** `chunk_lens` is a **decrypt** input, not a verify input: per-chunk AES-256-GCM-SIV needs the *entire* array, and a reader rejects any array whose entries do not sum to `total_length`. A per-frame slice is therefore unusable on arrival — which is why a large layout is **paged**, each page carrying the offset that locates it, rather than trimmed. The arrival-time wrong-holder check does not need the layout either: `root` + `total_length` answer that, while a per-chunk proof could not, because merkle leaves commit per **resource** — a proof for a single chunk is not derivable. + +**Frame bounds.** These are byte-identical wire constants; every implementation of DIG peer framing uses these exact values: + +| Constant | Value | What it bounds | +|---|---|---| +| `MAX_FRAMED_BODY` | `65536` | The total encoded body of one frame. | +| `MAX_RANGE_FRAME_PAYLOAD` | `32768` | The raw `bytes` payload one frame may carry. | +| `MAX_INCLUSION_PROOF_B64` | `4096` | The base64 `inclusion_proof`. | +| `MAX_CHUNK_LENS_PER_FRAME` | `2048` | The `chunk_lens` entries one prologue page carries. | +| `MAX_FIRST_FRAME_CHUNK_LENS` | `2486` | The `chunk_lens` entries guaranteed to fit a first frame. | + +A serve path splits its stream on **`MAX_RANGE_FRAME_PAYLOAD`** — never on the per-request window (3 MiB), which bounds what one *request* may ask for, not what one *frame* may carry. -**Per-range proof fields (v0.4.0+, optional):** Any frame MAY additionally carry merkle proofs for the chunks in that frame — a forward-compatible extension that allows clients to verify each range against the chain-anchored root without awaiting the whole resource: +**When metadata alone cannot fit a frame.** One input is unrepresentable: an `inclusion_proof` whose base64 exceeds `MAX_INCLUSION_PROOF_B64`. Such a resource has no conforming range stream, so the holder streams **no** frame and answers the single structured error [`-32009` `RANGE_METADATA_UNREPRESENTABLE`](#range-errors). A client MUST treat it as **holder-fatal**: skip that holder, do not re-request that range from it, and do **not** count it as a transport failure — counted as transport, it makes the client retry a peer that can never succeed. - - `range_proof` (optional, array of strings) — one base64-encoded merkle inclusion proof per chunk in this frame, in ascending chunk order. Each proof binds to the chunk's absolute index (see `first_chunk_index` below). Servers expand a byte-range request to the covering whole-chunk boundaries, so the returned frame's chunks are complete, verifiable units. - - `first_chunk_index` (optional, integer) — the absolute index (into the resource's `chunk_lens`) of the first chunk in this frame. Pairs with `range_proof[]` so each proof is indexed: `proof[i]` verifies chunk `first_chunk_index + i`. +**Reserved.** `range_proof` and `first_chunk_index` are reserved frame field names. A server MUST NOT emit them and a client MUST NOT require them; range verification is the whole-resource `inclusion_proof` against the client's own pinned root. -**Verification rule (normative):** Clients verify each range's proofs against their **own chain-anchored root** (the CHIP-0035 singleton's current on-chain `metadata.root_hash`), never against the frame's `root` field (which is advisory echo only). A frame lacking valid proofs against the client's pinned root is unverified content and MUST NOT be treated as trusted. The fields are optional for backwards compatibility with v0.3.0 servers; a frame without them still verifies via the first-frame `inclusion_proof` over the whole resource. +**Verification rule (normative):** Clients verify every range against their **own chain-anchored root** (the CHIP-0035 singleton's current on-chain `metadata.root_hash`), never against the frame's `root` field, which is advisory echo. Bytes not verified against the client's pinned root are unverified content and MUST NOT be treated as trusted. ### Per-range integrity — verify a range without the whole file {#range-integrity} @@ -696,7 +722,7 @@ A range fetched from one peer is **independently verifiable** against the capsul A requested range maps to whole **chunk(s)** — the node widens the range to chunk boundaries — so each returned chunk is a complete, verifiable unit. A client verifies a fetched range as follows: -1. **Split by `chunk_lens`.** Using the first frame's `chunk_lens` and `chunk_index`, cut the reassembled range bytes into the exact chunk(s) it covers. +1. **Split by `chunk_lens`.** Using the stream's `chunk_lens` and the frame's `chunk_index`, cut the reassembled range bytes into the exact chunk(s) it covers. 2. **Verify the resource against the root.** The `inclusion_proof` proves `resource_leaf` (= `SHA-256` of the *whole* resource ciphertext, reconstructed from all chunks in `chunk_lens` order) is included under the caller-supplied **chain-anchored `root`** ([Verification & provenance](./verification-and-provenance.md)) — the node is never the trust anchor. A client that already holds this proof (from an earlier range/peer) reuses it; the proof is the same regardless of which peer or range served the bytes. 3. **Bind each chunk to the committed resource.** Because `chunk_lens` fixes each chunk's length and the resource leaf is `SHA-256` over the concatenation of all chunk ciphertexts in order, a chunk delivered for a given `chunk_index` is correct iff, when placed at its `chunk_lens` offset, the whole-resource hash still matches the proven `resource_leaf`. A chunk from a **bad source** yields a resource hash that does not match the proof — detected without downloading the whole file. 4. **Decrypt.** AES-256-GCM-SIV-open each verified chunk; a wrong key/salt or corrupted bytes fails the authentication tag ([`DIG_ERR_DECRYPT_TAG`](../support/error-codes.md)). @@ -713,7 +739,7 @@ Detecting a bad source: any of a chunk-length mismatch against `chunk_lens`, a f AVAILABILITY has_root / has_capsule (§9 availability). Keep only peers that actually HOLD the resource; read total_length + chunk_count to plan. 3. PLAN — partition the resource into chunk-aligned ranges (using chunk_lens - from the first range frame or the availability chunk_count). + from the range stream's layout or the availability chunk_count). 4. FAN OUT — request DIFFERENT ranges from DIFFERENT holders CONCURRENTLY over the multiplexed stream transport (§8), respecting each peer's backpressure. 5. VERIFY — verify each returned range independently against the chain-anchored @@ -729,10 +755,11 @@ Step 2 is the gate: a downloader never fans a range at a peer it has not confirm - **Resume.** Because each range is independently addressable and independently verifiable, an interrupted download resumes **per range** — a client re-requests only the ranges it has not yet verified, from any peer that holds the resource. No range already verified is refetched. - **Any source, one root.** Every range — whichever peer served it — is verified against the *same* on-chain generation root, so mixing sources never weakens integrity. -### Range fetch error codes +### Range fetch error codes {#range-errors} | Code | Name | Meaning | |---|---|---| +| `-32009` | `RANGE_METADATA_UNREPRESENTABLE` | The range's metadata alone cannot fit a frame (an `inclusion_proof` over `MAX_INCLUSION_PROOF_B64`), so this holder has no conforming range stream for the resource and streams no frames. **Holder-fatal:** skip this holder for this range and do not count it as a transport failure. | | `-32004` | `RESOURCE_UNAVAILABLE` | This peer does not hold the resource/capsule at the requested root, **and** it could not locate any peer that does (a genuine not-found). When it CAN locate a holder it returns the [redirect](#redirect-on-miss) (`-32008`) instead. | | `-32007` | `RANGE_NOT_SATISFIABLE` | The requested `offset`/`length` lies outside the resource (`offset >= total_length`), or the range is otherwise unsatisfiable. | | `-32008` | `CONTENT_REDIRECT` | This node does not hold the content but located peers that do — see [redirect-on-miss](#redirect-on-miss). | @@ -796,7 +823,7 @@ The peer network is implemented by several crates that must interoperate byte-fo | **Peer exchange** | `RequestPeers`→`RespondPeers` of `TimestampedPeerInfo{host, port, timestamp}` (Chia-streamable, big-endian) | that nodes discover peers from each other identically | | **Peer RPC** | `dig.getPeers` / `dig.announce` / `dig.getNetworkInfo` + `-32006`; `dig.getAvailability` / `dig.listInventory`; `dig.fetchRange` + `-32007`, generated into [`openrpc-node.json`](https://docs.dig.net/openrpc-node.json) | the machine surface an agent drives; CI-diffable against a live node | | **Availability** | `dig.getAvailability` batch per-item answers at store / root / capsule granularity (`available` + `roots`/`total_length`/`chunk_count`/`complete`) | that a downloader can confirm a peer HOLDS content (and plan ranges) before any fetch | -| **Streaming + range** | `dig.fetchRange` streams `RangeFrame{offset,length,bytes,complete}`; first frame carries `total_length` + `chunk_lens` + `chunk_index` + `inclusion_proof`; any frame MAY carry `range_proof[]` (per-chunk proofs, v0.4.0+) + `first_chunk_index` (chunk start index); ranges are chunk-aligned | that data streams (not buffered), a single-peer range verifies against the chain-anchored root (whole-resource or per-chunk), and multi-source pieces reassemble verified — so they can't be forged | +| **Streaming + range** | `dig.fetchRange` streams `RangeFrame{offset,length,bytes,complete}`; EVERY frame carries the fixed-size identity set `root` + `total_length` + `chunk_count` (+ `chunk_index` when chunk-aligned); the resource-scaling layout `chunk_lens` + `inclusion_proof` rides the first frame or a paged prologue once per stream, located by `chunk_lens_offset`, and is omitted entirely when the client sets `skip_layout`; ranges are chunk-aligned | that data streams (not buffered), a wrong-generation holder is detected as the first bytes arrive, a single-peer range verifies against the chain-anchored root, and multi-source pieces reassemble verified — so they can't be forged | | **Range integrity** | a range maps to whole chunk(s); each verifies via `chunk_lens` + the whole-resource `inclusion_proof` against the on-chain `root` (same as [merkle-proofs](./merkle-proofs.md)) | that any peer's range is independently verifiable + a bad source is detectable without the whole file | | **NAT ladder** | the ordered strategies DIRECT → UPnP → NAT-PMP → PCP → hole-punch (relay signalling only) → RELAYED/TURN (relay carries data), relay-data-last | that every `connect(peer)` implementation prefers direct, prefers hole-punch signalling over full relaying, and proxies the stream only as a last resort | | **DHT content key** | `SHA-256(tag ‖ canonical bytes)` with tags `0x01` store (`store_id`), `0x02` root/capsule (`store_id ‖ root`), `0x03` resource (`store_id ‖ root ‖ retrieval_key`); node id = `peer_id`; distance = XOR; bucket = `255 − leading_zeros` ([§4c](#dht)) | that every node derives the identical content key for the same content, and places nodes + content in one 256-bit keyspace, so a provider record announced by one implementation is found by another | diff --git a/docs/support/error-codes.md b/docs/support/error-codes.md index 0e16467..dd9d302 100644 --- a/docs/support/error-codes.md +++ b/docs/support/error-codes.md @@ -27,7 +27,7 @@ This catalog is also published as [`error-codes.json`](https://docs.dig.net/erro ## dig RPC (JSON-RPC) -The [dig RPC](../protocol/dig-rpc.md) uses the standard [JSON-RPC 2.0](https://www.jsonrpc.org/specification) error codes plus the protocol-specific `-32004`, `-32005`, the node-profile `-32006` / `-32007` / `-32008`, the shell/staging codes `-32010` (upstream) and `-32011`–`-32014` (`dig.stage`), the [private-retrieval](../protocol/onion-routing.md) codes `-32020` / `-32021` / `-32022`, and the local control-plane codes `-32030` / `-32031` / `-32032`. A content **miss is never an error** — the capsule returns its own indistinguishable, non-verifying response (there is no `decoy` field on the wire), and the client discovers the miss by inclusion-proof and/or decryption failure (see [the blind host model](../protocol/blind-host-model.md)). For any well-formed body the HTTP status is `200`; the error is carried in the JSON envelope. +The [dig RPC](../protocol/dig-rpc.md) uses the standard [JSON-RPC 2.0](https://www.jsonrpc.org/specification) error codes plus the protocol-specific `-32004`, `-32005`, the node-profile `-32006` / `-32007` / `-32008` / `-32009`, the shell/staging codes `-32010` (upstream) and `-32011`–`-32014` (`dig.stage`), the [private-retrieval](../protocol/onion-routing.md) codes `-32020` / `-32021` / `-32022`, and the local control-plane codes `-32030` / `-32031` / `-32032`. A content **miss is never an error** — the capsule returns its own indistinguishable, non-verifying response (there is no `decoy` field on the wire), and the client discovers the miss by inclusion-proof and/or decryption failure (see [the blind host model](../protocol/blind-host-model.md)). For any well-formed body the HTTP status is `200`; the error is carried in the JSON envelope. | Code | Meaning | What to do | |---|---|---| @@ -41,6 +41,7 @@ The [dig RPC](../protocol/dig-rpc.md) uses the standard [JSON-RPC 2.0](https://w | `-32006` | **Peer unreachable** — no connection to the named peer could be established: every NAT-traversal strategy (direct, UPnP/NAT-PMP/PCP mapping, relay-coordinated hole-punch, and relayed fallback) failed, or the peer is not registered on this network. Returned by the node-profile [peer methods](../protocol/peer-network.md#peer-rpc) (`dig.getPeers` / `dig.announce` / `dig.getNetworkInfo`). | Confirm the peer is online and on the same network; retry — a relayed path may become available, or a hole-punch may succeed on a later attempt. | | `-32007` | **Range not satisfiable** — the requested byte range lies outside the resource (`offset` ≥ its length) or is otherwise unsatisfiable. Returned by the node-profile [`dig.fetchRange`](../protocol/peer-network.md#range) (the streaming byte-range / multi-source content fetch). | Re-request within `total_length` (read it from the first range frame or `dig.getMetadata`); align ranges to the resource size. | | `-32008` | **Content held elsewhere — redirect** — this node does not hold the requested content, but it located peers that DO. Not a not-found: `error.data.redirect` names the holder(s) (`providers[]` = `peer_id` + candidate `addresses`), the content requested, and the redirect budget (`redirect_depth`, `max_redirects`). Returned by the node-profile content methods ([`dig.getContent`](../protocol/dig-rpc.md), [`dig.fetchRange`](../protocol/peer-network.md#range)) on a local miss when a provider exists. | Re-request the same content against a peer in `data.redirect.providers`, echoing `redirect_depth` in your `params` so the hop budget stays bounded; stop when `redirect_depth` reaches `max_redirects`. | +| `-32009` | **Range metadata unrepresentable** — the metadata for a byte range *alone* cannot fit a frame (an inclusion proof whose base64 exceeds the 4096-byte cap), so this holder has no conforming range stream for the resource and streams no frames. Returned by the node-profile [`dig.fetchRange`](../protocol/peer-network.md#range). | Treat the holder as **fatal for that range**: fetch the range from a different holder and do not re-request it from this one. Do **not** count it as a transport failure — counted that way, your client retries a peer that can never succeed. | | `-32010` | **Upstream error** — a node acting as a thin shell relayed a method it does not resolve locally to its upstream DIG RPC, and the upstream was unreachable or returned a non-JSON response. | Retry; if it persists the upstream (e.g. `rpc.dig.net`) may be unavailable — try another node. | | `-32011` | **Stage: source directory unreadable** (`dig.stage`, local control) — the source directory could not be read. | Confirm the directory exists and is readable; re-run. | | `-32012` | **Stage: no files to stage** — the source directory contains no files. | Point `dig.stage` at a directory with content. | diff --git a/scripts/dig-spec.mjs b/scripts/dig-spec.mjs index d0502ca..b31b8de 100644 --- a/scripts/dig-spec.mjs +++ b/scripts/dig-spec.mjs @@ -191,37 +191,43 @@ export const schemas = { title: 'RangeFrame', type: 'object', description: - 'One frame of a streamed dig.fetchRange response. Data is delivered as an ordered stream of these frames over a logical stream of the multiplexed peer transport — the caller reads incrementally with backpressure and reassembles by `offset`. The FIRST frame (offset == range start) additionally carries the range-verification metadata (total_length, chunk_lens, chunk_index, inclusion_proof) so a range fetched from ONE peer is independently verifiable against the capsule\'s chain-anchored merkle root without the whole file. See https://docs.dig.net/docs/protocol/peer-network#range.', + 'One frame of a streamed dig.fetchRange response. Data is delivered as an ordered stream of these frames over a logical stream of the multiplexed peer transport — the caller reads incrementally with backpressure and reassembles by `offset`. Frame metadata divides in two by whether it scales with the resource: the FIXED-SIZE identity set (root, total_length, chunk_count, plus chunk_index when the window is chunk-aligned) rides EVERY frame, so a wrong-generation holder is detected as the first bytes arrive; the RESOURCE-SCALING layout set (chunk_lens, inclusion_proof) rides the first frame or a paged prologue ONCE per range stream, each page located by chunk_lens_offset, and is omitted entirely when the caller sets skip_layout. See https://docs.dig.net/docs/protocol/peer-network#range.', required: ['offset', 'length', 'bytes', 'complete'], properties: { offset: { type: 'integer', minimum: 0, description: 'This frame\'s start offset within the requested range (bytes into the resource ciphertext).' }, - length: { type: 'integer', minimum: 0, description: 'This frame\'s byte length (= decoded `bytes` length).' }, + length: { type: 'integer', minimum: 0, description: 'This frame\'s byte length (= decoded `bytes` length). At most MAX_RANGE_FRAME_PAYLOAD (32768) — the cap a serve path splits its stream on.' }, bytes: { type: 'string', contentEncoding: 'base64', description: 'This frame\'s ciphertext bytes, standard base64.' }, complete: { type: 'boolean', description: 'true when this is the final frame of the requested range.' }, - total_length: { type: 'integer', minimum: 0, description: 'FIRST FRAME ONLY: the full resource ciphertext length (so a client can plan its ranges).' }, + total_length: { type: 'integer', minimum: 0, description: 'EVERY FRAME (identity set): the full resource ciphertext length — plan ranges against it, and with `root` it detects a holder serving a different generation. A chunk_lens array whose entries do not sum to this value is rejected.' }, + chunk_count: { type: 'integer', minimum: 0, description: 'EVERY FRAME (identity set): the number of chunks in the WHOLE resource — the entry count a reassembled chunk_lens must reach.' }, + chunk_index: { type: 'integer', minimum: 0, description: 'EVERY FRAME (identity set) when the frame\'s window is chunk-aligned: index into chunk_lens of this frame\'s first chunk.' }, + root: { ...hex64, description: 'EVERY FRAME (identity set): the generation root this range was served from. Advisory echo — verify against the CALLER\'s chain-anchored root, never this field.' }, chunk_lens: { type: 'array', items: { type: 'integer', minimum: 0 }, - description: 'FIRST FRAME ONLY: per-chunk ciphertext lengths of the WHOLE resource, in order (identical to the dig RPC chunk_lens). Maps a byte range to the chunk(s) covering it; required to split + verify + decrypt.', + description: 'ONCE PER RANGE STREAM (layout set): per-chunk ciphertext lengths of the WHOLE resource, in order (identical to the dig RPC chunk_lens). A DECRYPT input, not a verify input — per-chunk AES-256-GCM-SIV needs the entire array and a reader rejects one whose entries do not sum to total_length, so it is never sliced to a frame\'s own chunks. It rides the first frame when it fits (at most MAX_FIRST_FRAME_CHUNK_LENS = 2486 entries), otherwise a paged prologue of at most MAX_CHUNK_LENS_PER_FRAME = 2048 entries per page, each page located by chunk_lens_offset. Omitted when the caller sets skip_layout.', + }, + chunk_lens_offset: { + type: 'integer', + minimum: 0, + description: 'ONCE PER RANGE STREAM (layout set): the index into the resource\'s chunk_lens at which this frame\'s prologue page begins, so pages are placed without depending on arrival order.', }, - chunk_index: { type: 'integer', minimum: 0, description: 'FIRST FRAME ONLY: index into chunk_lens of the first chunk in this range.' }, inclusion_proof: { type: ['string', 'null'], contentEncoding: 'base64', - description: 'FIRST FRAME ONLY: base64 merkle inclusion proof of the WHOLE resource against the generation `root`, relayed verbatim. Verify the range against the CALLER-supplied chain-anchored root — the node is never the trust anchor. null for capsule fetches (capsule: true), which self-verify on install.', + description: 'ONCE PER RANGE STREAM (layout set): base64 merkle inclusion proof of the WHOLE resource against the generation `root`, relayed verbatim, at most MAX_INCLUSION_PROOF_B64 (4096) bytes. Verify the range against the CALLER-supplied chain-anchored root — the node is never the trust anchor. null for capsule fetches (capsule: true), which self-verify on install. A resource whose proof exceeds the cap has no conforming range stream and the holder answers RANGE_METADATA_UNREPRESENTABLE (-32009) instead of streaming frames. Omitted when the caller sets skip_layout.', }, - root: { ...hex64, description: 'FIRST FRAME ONLY: the resolved generation root the inclusion_proof is against.' }, range_proof: { type: 'array', items: { type: 'string', contentEncoding: 'base64' }, description: - 'OPTIONAL (v0.4.0+): one base64-encoded merkle inclusion proof PER CHUNK carried by this frame, in ascending chunk order — so every frame\'s chunks are independently verifiable, not just the first. Verify each proof against the CALLER\'s chain-anchored root (the frame\'s `root` is an advisory echo, never the trust anchor); a frame whose chunks lack a valid proof is unverified content. Absent from pre-0.4.0 peers (backwards compatible).', + 'RESERVED: a server MUST NOT emit this field and a client MUST NOT require it. Range verification is the whole-resource inclusion_proof against the caller\'s chain-anchored root; merkle leaves commit per RESOURCE, so a proof for a single chunk is not derivable.', }, first_chunk_index: { type: 'integer', minimum: 0, description: - 'OPTIONAL (v0.4.0+): the absolute index (into chunk_lens) of the FIRST chunk carried by this frame — pairs each range_proof entry with its chunk position. Absent from pre-0.4.0 peers.', + 'RESERVED: a server MUST NOT emit this field and a client MUST NOT require it. Use chunk_index (identity set) for the frame\'s first chunk position.', }, }, }, @@ -603,7 +609,7 @@ export const nodeMethods = [ name: 'dig.fetchRange', summary: 'Stream a byte range [offset, offset+length) of a resource or capsule (multi-source).', description: - 'STREAMING + BYTE-RANGE fetch: return only the requested byte range of a content resource (store_id + retrieval_key) or a whole capsule/.dig (capsule: true, identified by store_id[:root]), delivered as an ordered STREAM of RangeFrame chunks over a logical stream of the multiplexed peer transport (read incrementally with backpressure; reassemble by offset). The FIRST frame carries total_length + chunk_lens + chunk_index + inclusion_proof so the range is INDEPENDENTLY verifiable against the capsule\'s chain-anchored merkle root — a downloader fans different ranges out to different peers concurrently, verifies each, retries a bad range from another source, and resumes per-range. The range is widened to whole-chunk boundaries so each returned chunk is a complete verifiable unit; length is clamped to the node window (3 MiB). NODE-PROFILE ONLY. See https://docs.dig.net/docs/protocol/peer-network#range.', + 'STREAMING + BYTE-RANGE fetch: return only the requested byte range of a content resource (store_id + retrieval_key) or a whole capsule/.dig (capsule: true, identified by store_id[:root]), delivered as an ordered STREAM of RangeFrame chunks over a logical stream of the multiplexed peer transport (read incrementally with backpressure; reassemble by offset). EVERY frame carries the fixed-size identity set (root + total_length + chunk_count, plus chunk_index when chunk-aligned), and the resource-scaling layout (chunk_lens + inclusion_proof) rides the first frame or a paged prologue once per stream, so the range is INDEPENDENTLY verifiable against the capsule\'s chain-anchored merkle root — a downloader fans different ranges out to different peers concurrently, verifies each, retries a bad range from another source, and resumes per-range. The range is widened to whole-chunk boundaries so each returned chunk is a complete verifiable unit; length is clamped to the node window (3 MiB). NODE-PROFILE ONLY. See https://docs.dig.net/docs/protocol/peer-network#range.', paramStructure: 'by-name', params: [ { name: 'store_id', required: true, schema: { $ref: '#/components/schemas/StoreId' } }, @@ -612,9 +618,10 @@ export const nodeMethods = [ { name: 'capsule', required: false, schema: { type: 'boolean', default: false }, description: 'true to fetch the whole capsule/.dig (identified by store_id[:root]) instead of a resource.' }, { name: 'offset', required: false, schema: { type: 'integer', minimum: 0, default: 0 }, description: 'Byte offset into the resource ciphertext.' }, { name: 'length', required: true, schema: { type: 'integer', minimum: 1 }, description: 'Bytes to return; clamped to the node window (3 MiB) and widened to whole-chunk boundaries.' }, + { name: 'skip_layout', required: false, schema: { type: 'boolean' }, description: 'true when the caller already holds the resource layout + commitment; the peer then omits the layout metadata (chunk_lens, inclusion_proof) and streams data frames only.' }, ], result: { name: 'frame', schema: { $ref: '#/components/schemas/RangeFrame' } }, - errors: ['INVALID_PARAMS', 'INTERNAL_ERROR', 'RESOURCE_UNAVAILABLE', 'RANGE_NOT_SATISFIABLE'], + errors: ['INVALID_PARAMS', 'INTERNAL_ERROR', 'RESOURCE_UNAVAILABLE', 'RANGE_NOT_SATISFIABLE', 'RANGE_METADATA_UNREPRESENTABLE'], }, ]; @@ -633,6 +640,7 @@ export const rpcErrors = { ROOT_NOT_ANCHORED: { code: -32005, message: 'Root not chain-anchored', meaning: 'The requested or served generation is not the store’s current on-chain root. A content read is pinned to the CHIP-0035 singleton’s on-chain root (resolved live from the chain, never trusted from the serving node): a requested root that is not the on-chain root, an unreachable chain, or a store with no confirmed generation fails closed with this code rather than serving an unverified generation. Omit root to take the chain tip.' }, PEER_UNREACHABLE: { code: -32006, message: 'Peer unreachable', meaning: 'No connection to the named peer could be established — every NAT-traversal strategy (direct, UPnP/NAT-PMP/PCP mapping, relay-coordinated hole-punch, and relayed fallback) failed, or the peer is not registered on this network. Returned by the node-profile peer methods (dig.getPeers / dig.announce / dig.getNetworkInfo).' }, RANGE_NOT_SATISFIABLE: { code: -32007, message: 'Range not satisfiable', meaning: 'The requested byte range lies outside the resource (offset >= total_length) or is otherwise unsatisfiable. Returned by the node-profile dig.fetchRange when the offset/length cannot be served.' }, + RANGE_METADATA_UNREPRESENTABLE: { code: -32009, message: 'Range metadata unrepresentable', meaning: 'The range metadata ALONE cannot fit a frame — an inclusion_proof whose base64 exceeds MAX_INCLUSION_PROOF_B64 (4096) — so this holder has no conforming range stream for the resource and streams no frames. Returned by the node-profile dig.fetchRange. HOLDER-FATAL: a client skips this holder, does not re-request the range from it, and MUST NOT count it as a transport failure (counted as transport it retries a peer that can never succeed).' }, CONTENT_REDIRECT: { code: -32008, message: 'Content held elsewhere — redirect', meaning: 'This node does not hold the requested content, but it located peers that do (via the DHT) — a redirect, not a not-found. error.data.redirect names the holder(s) (providers[] = peer_id + candidate addresses), the requested content, and the bounded redirect budget (redirect_depth, max_redirects). Returned by the node-profile content methods (dig.getContent / dig.fetchRange) on a local miss when a provider exists; re-request against a named provider, echoing redirect_depth.' }, UPSTREAM_ERROR: { code: -32010, message: 'Upstream error', meaning: 'A thin-shell node relayed a method it does not resolve locally to its upstream DIG RPC, and the upstream was unreachable or returned a non-JSON response.' }, STAGE_DIR_UNREADABLE: { code: -32011, message: 'Stage: source directory unreadable', meaning: 'dig.stage (local control) could not read the source directory.' }, diff --git a/static/error-codes.json b/static/error-codes.json index 421968c..54b43d6 100644 --- a/static/error-codes.json +++ b/static/error-codes.json @@ -73,6 +73,13 @@ "http_or_exit": 200, "description": "Range not satisfiable — The requested byte range lies outside the resource (offset >= total_length) or is otherwise unsatisfiable. Returned by the node-profile dig.fetchRange when the offset/length cannot be served." }, + { + "surface": "dig-rpc", + "code": -32009, + "name": "RANGE_METADATA_UNREPRESENTABLE", + "http_or_exit": 200, + "description": "Range metadata unrepresentable — The range metadata ALONE cannot fit a frame — an inclusion_proof whose base64 exceeds MAX_INCLUSION_PROOF_B64 (4096) — so this holder has no conforming range stream for the resource and streams no frames. Returned by the node-profile dig.fetchRange. HOLDER-FATAL: a client skips this holder, does not re-request the range from it, and MUST NOT count it as a transport failure (counted as transport it retries a peer that can never succeed)." + }, { "surface": "dig-rpc", "code": -32008, @@ -486,6 +493,13 @@ "http_or_exit": 200, "description": "Range not satisfiable — The requested byte range lies outside the resource (offset >= total_length) or is otherwise unsatisfiable. Returned by the node-profile dig.fetchRange when the offset/length cannot be served." }, + { + "surface": "dig-rpc", + "code": -32009, + "name": "RANGE_METADATA_UNREPRESENTABLE", + "http_or_exit": 200, + "description": "Range metadata unrepresentable — The range metadata ALONE cannot fit a frame — an inclusion_proof whose base64 exceeds MAX_INCLUSION_PROOF_B64 (4096) — so this holder has no conforming range stream for the resource and streams no frames. Returned by the node-profile dig.fetchRange. HOLDER-FATAL: a client skips this holder, does not re-request the range from it, and MUST NOT count it as a transport failure (counted as transport it retries a peer that can never succeed)." + }, { "surface": "dig-rpc", "code": -32008, diff --git a/static/openrpc-node.json b/static/openrpc-node.json index cbe69ba..bc8222a 100644 --- a/static/openrpc-node.json +++ b/static/openrpc-node.json @@ -2,7 +2,7 @@ "openrpc": "1.2.6", "info": { "title": "dig RPC — node profile (local dig-node / in-process DIG Browser)", - "version": "0.5.3", + "version": "0.6.0", "description": "The NODE PROFILE: a distinct, smaller surface than the network profile. Of the byte methods it implements ONLY dig.getContent (local-first, else proxy); everything else proxies upstream or returns -32601. It ADDS node-only methods the security model depends on — chiefly dig.getAnchoredRoot (the CHIP-0035 on-chain head, the trusted root for mandatory root-pinning), dig.stage, and cache.*. Gate on dig.methods rather than assuming one uniform surface. See https://docs.dig.net/docs/protocol/dig-rpc#node-profile.", "license": { "name": "GPL-2.0", @@ -483,7 +483,7 @@ { "name": "dig.fetchRange", "summary": "Stream a byte range [offset, offset+length) of a resource or capsule (multi-source).", - "description": "STREAMING + BYTE-RANGE fetch: return only the requested byte range of a content resource (store_id + retrieval_key) or a whole capsule/.dig (capsule: true, identified by store_id[:root]), delivered as an ordered STREAM of RangeFrame chunks over a logical stream of the multiplexed peer transport (read incrementally with backpressure; reassemble by offset). The FIRST frame carries total_length + chunk_lens + chunk_index + inclusion_proof so the range is INDEPENDENTLY verifiable against the capsule's chain-anchored merkle root — a downloader fans different ranges out to different peers concurrently, verifies each, retries a bad range from another source, and resumes per-range. The range is widened to whole-chunk boundaries so each returned chunk is a complete verifiable unit; length is clamped to the node window (3 MiB). NODE-PROFILE ONLY. See https://docs.dig.net/docs/protocol/peer-network#range.", + "description": "STREAMING + BYTE-RANGE fetch: return only the requested byte range of a content resource (store_id + retrieval_key) or a whole capsule/.dig (capsule: true, identified by store_id[:root]), delivered as an ordered STREAM of RangeFrame chunks over a logical stream of the multiplexed peer transport (read incrementally with backpressure; reassemble by offset). EVERY frame carries the fixed-size identity set (root + total_length + chunk_count, plus chunk_index when chunk-aligned), and the resource-scaling layout (chunk_lens + inclusion_proof) rides the first frame or a paged prologue once per stream, so the range is INDEPENDENTLY verifiable against the capsule's chain-anchored merkle root — a downloader fans different ranges out to different peers concurrently, verifies each, retries a bad range from another source, and resumes per-range. The range is widened to whole-chunk boundaries so each returned chunk is a complete verifiable unit; length is clamped to the node window (3 MiB). NODE-PROFILE ONLY. See https://docs.dig.net/docs/protocol/peer-network#range.", "paramStructure": "by-name", "params": [ { @@ -536,6 +536,14 @@ "type": "integer", "minimum": 1 } + }, + { + "name": "skip_layout", + "required": false, + "description": "true when the caller already holds the resource layout + commitment; the peer then omits the layout metadata (chunk_lens, inclusion_proof) and streams data frames only.", + "schema": { + "type": "boolean" + } } ], "result": { @@ -556,6 +564,9 @@ }, { "$ref": "#/components/errors/RANGE_NOT_SATISFIABLE" + }, + { + "$ref": "#/components/errors/RANGE_METADATA_UNREPRESENTABLE" } ] } @@ -902,7 +913,7 @@ "RangeFrame": { "title": "RangeFrame", "type": "object", - "description": "One frame of a streamed dig.fetchRange response. Data is delivered as an ordered stream of these frames over a logical stream of the multiplexed peer transport — the caller reads incrementally with backpressure and reassembles by `offset`. The FIRST frame (offset == range start) additionally carries the range-verification metadata (total_length, chunk_lens, chunk_index, inclusion_proof) so a range fetched from ONE peer is independently verifiable against the capsule's chain-anchored merkle root without the whole file. See https://docs.dig.net/docs/protocol/peer-network#range.", + "description": "One frame of a streamed dig.fetchRange response. Data is delivered as an ordered stream of these frames over a logical stream of the multiplexed peer transport — the caller reads incrementally with backpressure and reassembles by `offset`. Frame metadata divides in two by whether it scales with the resource: the FIXED-SIZE identity set (root, total_length, chunk_count, plus chunk_index when the window is chunk-aligned) rides EVERY frame, so a wrong-generation holder is detected as the first bytes arrive; the RESOURCE-SCALING layout set (chunk_lens, inclusion_proof) rides the first frame or a paged prologue ONCE per range stream, each page located by chunk_lens_offset, and is omitted entirely when the caller sets skip_layout. See https://docs.dig.net/docs/protocol/peer-network#range.", "required": [ "offset", "length", @@ -918,7 +929,7 @@ "length": { "type": "integer", "minimum": 0, - "description": "This frame's byte length (= decoded `bytes` length)." + "description": "This frame's byte length (= decoded `bytes` length). At most MAX_RANGE_FRAME_PAYLOAD (32768) — the cap a serve path splits its stream on." }, "bytes": { "type": "string", @@ -932,7 +943,22 @@ "total_length": { "type": "integer", "minimum": 0, - "description": "FIRST FRAME ONLY: the full resource ciphertext length (so a client can plan its ranges)." + "description": "EVERY FRAME (identity set): the full resource ciphertext length — plan ranges against it, and with `root` it detects a holder serving a different generation. A chunk_lens array whose entries do not sum to this value is rejected." + }, + "chunk_count": { + "type": "integer", + "minimum": 0, + "description": "EVERY FRAME (identity set): the number of chunks in the WHOLE resource — the entry count a reassembled chunk_lens must reach." + }, + "chunk_index": { + "type": "integer", + "minimum": 0, + "description": "EVERY FRAME (identity set) when the frame's window is chunk-aligned: index into chunk_lens of this frame's first chunk." + }, + "root": { + "type": "string", + "pattern": "^[0-9a-f]{64}$", + "description": "EVERY FRAME (identity set): the generation root this range was served from. Advisory echo — verify against the CALLER's chain-anchored root, never this field." }, "chunk_lens": { "type": "array", @@ -940,12 +966,12 @@ "type": "integer", "minimum": 0 }, - "description": "FIRST FRAME ONLY: per-chunk ciphertext lengths of the WHOLE resource, in order (identical to the dig RPC chunk_lens). Maps a byte range to the chunk(s) covering it; required to split + verify + decrypt." + "description": "ONCE PER RANGE STREAM (layout set): per-chunk ciphertext lengths of the WHOLE resource, in order (identical to the dig RPC chunk_lens). A DECRYPT input, not a verify input — per-chunk AES-256-GCM-SIV needs the entire array and a reader rejects one whose entries do not sum to total_length, so it is never sliced to a frame's own chunks. It rides the first frame when it fits (at most MAX_FIRST_FRAME_CHUNK_LENS = 2486 entries), otherwise a paged prologue of at most MAX_CHUNK_LENS_PER_FRAME = 2048 entries per page, each page located by chunk_lens_offset. Omitted when the caller sets skip_layout." }, - "chunk_index": { + "chunk_lens_offset": { "type": "integer", "minimum": 0, - "description": "FIRST FRAME ONLY: index into chunk_lens of the first chunk in this range." + "description": "ONCE PER RANGE STREAM (layout set): the index into the resource's chunk_lens at which this frame's prologue page begins, so pages are placed without depending on arrival order." }, "inclusion_proof": { "type": [ @@ -953,12 +979,7 @@ "null" ], "contentEncoding": "base64", - "description": "FIRST FRAME ONLY: base64 merkle inclusion proof of the WHOLE resource against the generation `root`, relayed verbatim. Verify the range against the CALLER-supplied chain-anchored root — the node is never the trust anchor. null for capsule fetches (capsule: true), which self-verify on install." - }, - "root": { - "type": "string", - "pattern": "^[0-9a-f]{64}$", - "description": "FIRST FRAME ONLY: the resolved generation root the inclusion_proof is against." + "description": "ONCE PER RANGE STREAM (layout set): base64 merkle inclusion proof of the WHOLE resource against the generation `root`, relayed verbatim, at most MAX_INCLUSION_PROOF_B64 (4096) bytes. Verify the range against the CALLER-supplied chain-anchored root — the node is never the trust anchor. null for capsule fetches (capsule: true), which self-verify on install. A resource whose proof exceeds the cap has no conforming range stream and the holder answers RANGE_METADATA_UNREPRESENTABLE (-32009) instead of streaming frames. Omitted when the caller sets skip_layout." }, "range_proof": { "type": "array", @@ -966,12 +987,12 @@ "type": "string", "contentEncoding": "base64" }, - "description": "OPTIONAL (v0.4.0+): one base64-encoded merkle inclusion proof PER CHUNK carried by this frame, in ascending chunk order — so every frame's chunks are independently verifiable, not just the first. Verify each proof against the CALLER's chain-anchored root (the frame's `root` is an advisory echo, never the trust anchor); a frame whose chunks lack a valid proof is unverified content. Absent from pre-0.4.0 peers (backwards compatible)." + "description": "RESERVED: a server MUST NOT emit this field and a client MUST NOT require it. Range verification is the whole-resource inclusion_proof against the caller's chain-anchored root; merkle leaves commit per RESOURCE, so a proof for a single chunk is not derivable." }, "first_chunk_index": { "type": "integer", "minimum": 0, - "description": "OPTIONAL (v0.4.0+): the absolute index (into chunk_lens) of the FIRST chunk carried by this frame — pairs each range_proof entry with its chunk position. Absent from pre-0.4.0 peers." + "description": "RESERVED: a server MUST NOT emit this field and a client MUST NOT require it. Use chunk_index (identity set) for the frame's first chunk position." } } } @@ -1040,6 +1061,13 @@ "type": "object" } }, + "RANGE_METADATA_UNREPRESENTABLE": { + "code": -32009, + "message": "Range metadata unrepresentable", + "data": { + "type": "object" + } + }, "CONTENT_REDIRECT": { "code": -32008, "message": "Content held elsewhere — redirect", diff --git a/static/openrpc.json b/static/openrpc.json index 0dbdb76..bda42d2 100644 --- a/static/openrpc.json +++ b/static/openrpc.json @@ -2,7 +2,7 @@ "openrpc": "1.2.6", "info": { "title": "dig RPC — DIG Network Content Interface (network profile)", - "version": "0.5.3", + "version": "0.6.0", "description": "The network-wide read interface for DIG content over JSON-RPC 2.0 — the NETWORK PROFILE served by the canonical node at rpc.dig.net. Blind by construction (the node holds no URN and no key), verifiable without trust (merkle inclusion proofs against the chain-anchored root), and streamable at any size. There is no `decoy` field on the wire and no CDN. See https://docs.dig.net/docs/protocol/dig-rpc.", "license": { "name": "GPL-2.0", @@ -850,7 +850,7 @@ "RangeFrame": { "title": "RangeFrame", "type": "object", - "description": "One frame of a streamed dig.fetchRange response. Data is delivered as an ordered stream of these frames over a logical stream of the multiplexed peer transport — the caller reads incrementally with backpressure and reassembles by `offset`. The FIRST frame (offset == range start) additionally carries the range-verification metadata (total_length, chunk_lens, chunk_index, inclusion_proof) so a range fetched from ONE peer is independently verifiable against the capsule's chain-anchored merkle root without the whole file. See https://docs.dig.net/docs/protocol/peer-network#range.", + "description": "One frame of a streamed dig.fetchRange response. Data is delivered as an ordered stream of these frames over a logical stream of the multiplexed peer transport — the caller reads incrementally with backpressure and reassembles by `offset`. Frame metadata divides in two by whether it scales with the resource: the FIXED-SIZE identity set (root, total_length, chunk_count, plus chunk_index when the window is chunk-aligned) rides EVERY frame, so a wrong-generation holder is detected as the first bytes arrive; the RESOURCE-SCALING layout set (chunk_lens, inclusion_proof) rides the first frame or a paged prologue ONCE per range stream, each page located by chunk_lens_offset, and is omitted entirely when the caller sets skip_layout. See https://docs.dig.net/docs/protocol/peer-network#range.", "required": [ "offset", "length", @@ -866,7 +866,7 @@ "length": { "type": "integer", "minimum": 0, - "description": "This frame's byte length (= decoded `bytes` length)." + "description": "This frame's byte length (= decoded `bytes` length). At most MAX_RANGE_FRAME_PAYLOAD (32768) — the cap a serve path splits its stream on." }, "bytes": { "type": "string", @@ -880,7 +880,22 @@ "total_length": { "type": "integer", "minimum": 0, - "description": "FIRST FRAME ONLY: the full resource ciphertext length (so a client can plan its ranges)." + "description": "EVERY FRAME (identity set): the full resource ciphertext length — plan ranges against it, and with `root` it detects a holder serving a different generation. A chunk_lens array whose entries do not sum to this value is rejected." + }, + "chunk_count": { + "type": "integer", + "minimum": 0, + "description": "EVERY FRAME (identity set): the number of chunks in the WHOLE resource — the entry count a reassembled chunk_lens must reach." + }, + "chunk_index": { + "type": "integer", + "minimum": 0, + "description": "EVERY FRAME (identity set) when the frame's window is chunk-aligned: index into chunk_lens of this frame's first chunk." + }, + "root": { + "type": "string", + "pattern": "^[0-9a-f]{64}$", + "description": "EVERY FRAME (identity set): the generation root this range was served from. Advisory echo — verify against the CALLER's chain-anchored root, never this field." }, "chunk_lens": { "type": "array", @@ -888,12 +903,12 @@ "type": "integer", "minimum": 0 }, - "description": "FIRST FRAME ONLY: per-chunk ciphertext lengths of the WHOLE resource, in order (identical to the dig RPC chunk_lens). Maps a byte range to the chunk(s) covering it; required to split + verify + decrypt." + "description": "ONCE PER RANGE STREAM (layout set): per-chunk ciphertext lengths of the WHOLE resource, in order (identical to the dig RPC chunk_lens). A DECRYPT input, not a verify input — per-chunk AES-256-GCM-SIV needs the entire array and a reader rejects one whose entries do not sum to total_length, so it is never sliced to a frame's own chunks. It rides the first frame when it fits (at most MAX_FIRST_FRAME_CHUNK_LENS = 2486 entries), otherwise a paged prologue of at most MAX_CHUNK_LENS_PER_FRAME = 2048 entries per page, each page located by chunk_lens_offset. Omitted when the caller sets skip_layout." }, - "chunk_index": { + "chunk_lens_offset": { "type": "integer", "minimum": 0, - "description": "FIRST FRAME ONLY: index into chunk_lens of the first chunk in this range." + "description": "ONCE PER RANGE STREAM (layout set): the index into the resource's chunk_lens at which this frame's prologue page begins, so pages are placed without depending on arrival order." }, "inclusion_proof": { "type": [ @@ -901,12 +916,7 @@ "null" ], "contentEncoding": "base64", - "description": "FIRST FRAME ONLY: base64 merkle inclusion proof of the WHOLE resource against the generation `root`, relayed verbatim. Verify the range against the CALLER-supplied chain-anchored root — the node is never the trust anchor. null for capsule fetches (capsule: true), which self-verify on install." - }, - "root": { - "type": "string", - "pattern": "^[0-9a-f]{64}$", - "description": "FIRST FRAME ONLY: the resolved generation root the inclusion_proof is against." + "description": "ONCE PER RANGE STREAM (layout set): base64 merkle inclusion proof of the WHOLE resource against the generation `root`, relayed verbatim, at most MAX_INCLUSION_PROOF_B64 (4096) bytes. Verify the range against the CALLER-supplied chain-anchored root — the node is never the trust anchor. null for capsule fetches (capsule: true), which self-verify on install. A resource whose proof exceeds the cap has no conforming range stream and the holder answers RANGE_METADATA_UNREPRESENTABLE (-32009) instead of streaming frames. Omitted when the caller sets skip_layout." }, "range_proof": { "type": "array", @@ -914,12 +924,12 @@ "type": "string", "contentEncoding": "base64" }, - "description": "OPTIONAL (v0.4.0+): one base64-encoded merkle inclusion proof PER CHUNK carried by this frame, in ascending chunk order — so every frame's chunks are independently verifiable, not just the first. Verify each proof against the CALLER's chain-anchored root (the frame's `root` is an advisory echo, never the trust anchor); a frame whose chunks lack a valid proof is unverified content. Absent from pre-0.4.0 peers (backwards compatible)." + "description": "RESERVED: a server MUST NOT emit this field and a client MUST NOT require it. Range verification is the whole-resource inclusion_proof against the caller's chain-anchored root; merkle leaves commit per RESOURCE, so a proof for a single chunk is not derivable." }, "first_chunk_index": { "type": "integer", "minimum": 0, - "description": "OPTIONAL (v0.4.0+): the absolute index (into chunk_lens) of the FIRST chunk carried by this frame — pairs each range_proof entry with its chunk position. Absent from pre-0.4.0 peers." + "description": "RESERVED: a server MUST NOT emit this field and a client MUST NOT require it. Use chunk_index (identity set) for the frame's first chunk position." } } } @@ -988,6 +998,13 @@ "type": "object" } }, + "RANGE_METADATA_UNREPRESENTABLE": { + "code": -32009, + "message": "Range metadata unrepresentable", + "data": { + "type": "object" + } + }, "CONTENT_REDIRECT": { "code": -32008, "message": "Content held elsewhere — redirect", From ce764d7761e3cd54000a19f179faf08a5dc877aa Mon Sep 17 00:00:00 2001 From: Michael Taylor Date: Sun, 26 Jul 2026 14:43:56 -0700 Subject: [PATCH 3/4] chore(release): bump docs to 0.7.0 Co-Authored-By: Claude --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index c4aa688..b775211 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "docs-dig-net", - "version": "0.6.0", + "version": "0.7.0", "private": true, "scripts": { "docusaurus": "docusaurus", From 2e7415f6d75237ed58f4d6eee7723a75d9c191dd Mon Sep 17 00:00:00 2001 From: Michael Taylor Date: Sun, 26 Jul 2026 15:09:03 -0700 Subject: [PATCH 4/4] docs(support): order both range-error tables by code and scope total_length to any frame MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three same-class fixes on the numbered surfaces this unit publishes: - the -32007 remediation pointed at the FIRST range frame for total_length, the last first-frame-scoped framing of a field that now rides every frame; - -32009 sat above -32004 in the range-error table, leaving that one table out of numeric order — the same out-of-order declaration that hid the missing row this unit exists to close; - section 7's peer-RPC table enumerates the node-profile codes a peer method adds and omitted -32009, which dig.fetchRange adds. Regenerating also picks up the OpenRPC info.version, which tracks package.json and still read 0.6.0 after the bump — a machine artifact understating its own version. Co-Authored-By: Claude --- docs/protocol/peer-network.md | 3 ++- docs/support/error-codes.md | 2 +- static/openrpc-node.json | 2 +- static/openrpc.json | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/protocol/peer-network.md b/docs/protocol/peer-network.md index 86a974d..b065ca9 100644 --- a/docs/protocol/peer-network.md +++ b/docs/protocol/peer-network.md @@ -558,6 +558,7 @@ Alongside the standard JSON-RPC codes and the shared `-32004` (resource unavaila | `-32006` | `PEER_UNREACHABLE` | No connection to the named peer could be established — every [traversal strategy](#nat-traversal) (direct, UPnP/NAT-PMP/PCP mapping, relay-coordinated hole-punch, and relayed fallback) failed, or the peer is not registered on this network. | | `-32007` | `RANGE_NOT_SATISFIABLE` | The requested byte range lies outside the resource (`offset` ≥ its length). Returned by [`dig.fetchRange`](#range). | | `-32008` | `CONTENT_REDIRECT` | This node does **not** hold the requested content, but it located peers that do — a **redirect, not a not-found** ([§9a](#redirect-on-miss)). `error.data.redirect` names the holder(s). | +| `-32009` | `RANGE_METADATA_UNREPRESENTABLE` | The metadata for a range alone cannot fit a frame, so this holder has no conforming range stream for the resource ([§9](#range-errors)). Holder-fatal, and not a transport failure. | See the full [error catalog](../support/error-codes.md). @@ -759,10 +760,10 @@ Step 2 is the gate: a downloader never fans a range at a peer it has not confirm | Code | Name | Meaning | |---|---|---| -| `-32009` | `RANGE_METADATA_UNREPRESENTABLE` | The range's metadata alone cannot fit a frame (an `inclusion_proof` over `MAX_INCLUSION_PROOF_B64`), so this holder has no conforming range stream for the resource and streams no frames. **Holder-fatal:** skip this holder for this range and do not count it as a transport failure. | | `-32004` | `RESOURCE_UNAVAILABLE` | This peer does not hold the resource/capsule at the requested root, **and** it could not locate any peer that does (a genuine not-found). When it CAN locate a holder it returns the [redirect](#redirect-on-miss) (`-32008`) instead. | | `-32007` | `RANGE_NOT_SATISFIABLE` | The requested `offset`/`length` lies outside the resource (`offset >= total_length`), or the range is otherwise unsatisfiable. | | `-32008` | `CONTENT_REDIRECT` | This node does not hold the content but located peers that do — see [redirect-on-miss](#redirect-on-miss). | +| `-32009` | `RANGE_METADATA_UNREPRESENTABLE` | The range's metadata alone cannot fit a frame (an `inclusion_proof` over `MAX_INCLUSION_PROOF_B64`), so this holder has no conforming range stream for the resource and streams no frames. **Holder-fatal:** skip this holder for this range and do not count it as a transport failure. | ### 9a · Redirect-on-miss — never a silent not-found when a holder exists {#redirect-on-miss} diff --git a/docs/support/error-codes.md b/docs/support/error-codes.md index dd9d302..d26a83e 100644 --- a/docs/support/error-codes.md +++ b/docs/support/error-codes.md @@ -39,7 +39,7 @@ The [dig RPC](../protocol/dig-rpc.md) uses the standard [JSON-RPC 2.0](https://w | `-32004` | **Resource not available at the requested root** — a genuine infrastructure miss (no host seed, the module absent in both buckets, bad magic, oversize, a wasmtime trap, or an undecodable envelope). Distinct from a content miss, which is an indistinguishable decoy and is *never* an error. | Confirm the `root` is a confirmed generation (`dig.listCapsules`); retry or try another node. | | `-32005` | **Root not chain-anchored** — the requested or served generation is not the store's current on-chain root. A content read is pinned to the CHIP-0035 singleton's on-chain root (resolved live from the chain, never trusted from the serving node): if the requested root is not the on-chain root, or the chain cannot be reached, or the store has no confirmed generation, the read fails closed with this code rather than serving an unverified generation. | Read against the store's current on-chain root (omit `root` to take the chain tip, or resolve it yourself); if the chain was unreachable, retry. | | `-32006` | **Peer unreachable** — no connection to the named peer could be established: every NAT-traversal strategy (direct, UPnP/NAT-PMP/PCP mapping, relay-coordinated hole-punch, and relayed fallback) failed, or the peer is not registered on this network. Returned by the node-profile [peer methods](../protocol/peer-network.md#peer-rpc) (`dig.getPeers` / `dig.announce` / `dig.getNetworkInfo`). | Confirm the peer is online and on the same network; retry — a relayed path may become available, or a hole-punch may succeed on a later attempt. | -| `-32007` | **Range not satisfiable** — the requested byte range lies outside the resource (`offset` ≥ its length) or is otherwise unsatisfiable. Returned by the node-profile [`dig.fetchRange`](../protocol/peer-network.md#range) (the streaming byte-range / multi-source content fetch). | Re-request within `total_length` (read it from the first range frame or `dig.getMetadata`); align ranges to the resource size. | +| `-32007` | **Range not satisfiable** — the requested byte range lies outside the resource (`offset` ≥ its length) or is otherwise unsatisfiable. Returned by the node-profile [`dig.fetchRange`](../protocol/peer-network.md#range) (the streaming byte-range / multi-source content fetch). | Re-request within `total_length` (read it from any range frame or `dig.getMetadata`); align ranges to the resource size. | | `-32008` | **Content held elsewhere — redirect** — this node does not hold the requested content, but it located peers that DO. Not a not-found: `error.data.redirect` names the holder(s) (`providers[]` = `peer_id` + candidate `addresses`), the content requested, and the redirect budget (`redirect_depth`, `max_redirects`). Returned by the node-profile content methods ([`dig.getContent`](../protocol/dig-rpc.md), [`dig.fetchRange`](../protocol/peer-network.md#range)) on a local miss when a provider exists. | Re-request the same content against a peer in `data.redirect.providers`, echoing `redirect_depth` in your `params` so the hop budget stays bounded; stop when `redirect_depth` reaches `max_redirects`. | | `-32009` | **Range metadata unrepresentable** — the metadata for a byte range *alone* cannot fit a frame (an inclusion proof whose base64 exceeds the 4096-byte cap), so this holder has no conforming range stream for the resource and streams no frames. Returned by the node-profile [`dig.fetchRange`](../protocol/peer-network.md#range). | Treat the holder as **fatal for that range**: fetch the range from a different holder and do not re-request it from this one. Do **not** count it as a transport failure — counted that way, your client retries a peer that can never succeed. | | `-32010` | **Upstream error** — a node acting as a thin shell relayed a method it does not resolve locally to its upstream DIG RPC, and the upstream was unreachable or returned a non-JSON response. | Retry; if it persists the upstream (e.g. `rpc.dig.net`) may be unavailable — try another node. | diff --git a/static/openrpc-node.json b/static/openrpc-node.json index bc8222a..178da9f 100644 --- a/static/openrpc-node.json +++ b/static/openrpc-node.json @@ -2,7 +2,7 @@ "openrpc": "1.2.6", "info": { "title": "dig RPC — node profile (local dig-node / in-process DIG Browser)", - "version": "0.6.0", + "version": "0.7.0", "description": "The NODE PROFILE: a distinct, smaller surface than the network profile. Of the byte methods it implements ONLY dig.getContent (local-first, else proxy); everything else proxies upstream or returns -32601. It ADDS node-only methods the security model depends on — chiefly dig.getAnchoredRoot (the CHIP-0035 on-chain head, the trusted root for mandatory root-pinning), dig.stage, and cache.*. Gate on dig.methods rather than assuming one uniform surface. See https://docs.dig.net/docs/protocol/dig-rpc#node-profile.", "license": { "name": "GPL-2.0", diff --git a/static/openrpc.json b/static/openrpc.json index bda42d2..e014a56 100644 --- a/static/openrpc.json +++ b/static/openrpc.json @@ -2,7 +2,7 @@ "openrpc": "1.2.6", "info": { "title": "dig RPC — DIG Network Content Interface (network profile)", - "version": "0.6.0", + "version": "0.7.0", "description": "The network-wide read interface for DIG content over JSON-RPC 2.0 — the NETWORK PROFILE served by the canonical node at rpc.dig.net. Blind by construction (the node holds no URN and no key), verifiable without trust (merkle inclusion proofs against the chain-anchored root), and streamable at any size. There is no `decoy` field on the wire and no CDN. See https://docs.dig.net/docs/protocol/dig-rpc.", "license": { "name": "GPL-2.0",