diff --git a/docs/protocol/peer-network.md b/docs/protocol/peer-network.md index 193df64..b5c1902 100644 --- a/docs/protocol/peer-network.md +++ b/docs/protocol/peer-network.md @@ -1,7 +1,7 @@ --- sidebar_position: 13 title: "L7 · DIG Node peer network" -description: "The normative node↔node protocol: mTLS peer identity (peer_id = SHA-256(TLS SPKI DER)), the two RPC tiers (mTLS-authenticated PEER/CONTROL vs anonymous PUBLIC-READ so browsers can retrieve content), the dual-mode public gateway (rpc.dig.net's mTLS front for node-class clients — the dig-store CLI, the SDK, any DIG-identity-key holder — across the full dig.local/localhost/rpc.dig.net ladder, plus its plain-HTTPS+CORS front for browsers, with an ephemeral self-signed client certificate for channel-bound anonymous reads), the ordered NAT-traversal ladder (direct → UPnP → NAT-PMP → PCP → relay-coordinated hole-punch (signalling only) → relayed/TURN transport), the relay's four roles (STUN, introducer, hole-punch signalling, relayed transport), STUN reflexive-address discovery, introducer + gossip peer discovery, PEX peer-exchange (node↔node stream + the RLY-008 relay introducer binding), the Kademlia DHT with provider records that locate which peers hold content (find_node/find_providers/add_provider/ping over a framed dig-nat mTLS stream; content-key = SHA-256(domain-tag ‖ store_id[‖root[‖retrieval_key]])), the relay RelayMessage wire (RLY-001..RLY-008), the peer RPC methods (dig.getPeers/dig.announce/dig.getNetworkInfo/dig.getAvailability/dig.listInventory/dig.fetchRange), and the relay-last-fallback invariant (prefer hole-punch signalling over full relaying)." +description: "The normative node↔node protocol: mTLS peer identity (peer_id = SHA-256(TLS SPKI DER)), the two RPC tiers (mTLS-authenticated PEER/CONTROL vs anonymous PUBLIC-READ so browsers can retrieve content), the dual-mode public gateway (rpc.dig.net's mTLS front for node-class clients — the dig-store CLI, the SDK, any DIG-identity-key holder — across the full dig.local/localhost/rpc.dig.net ladder, plus its plain-HTTPS+CORS front for browsers, with an ephemeral self-signed client certificate for channel-bound anonymous reads), the ordered NAT-traversal ladder (direct → UPnP → NAT-PMP → PCP → relay-coordinated hole-punch (signalling only) → relayed/TURN transport), the relay's four roles (STUN, introducer, hole-punch signalling, relayed transport), STUN reflexive-address discovery, introducer + gossip peer discovery, PEX peer-exchange (node↔node stream + the RLY-008 relay introducer binding), the Kademlia DHT with provider records that locate which peers hold content (find_node/find_providers/add_provider/ping over a framed dig-nat mTLS stream; content-key = SHA-256(domain-tag ‖ store_id[‖root[‖retrieval_key]])), provider records announced promptly on an inventory gain (whichever path put a capsule on disk — so a reader that caches a capsule becomes a discoverable holder of it) and withdrawn on removal, the holdings-announce wire (gossip-pool broadcast opcode 222 — the signed, batched add/remove statement of a peer's holdings, TLS-leaf-key ECDSA with the `dig:holdings:v1` preimage and fail-closed acceptance), the gossip/peer-RPC port relationship (gossip 9445 = peer 9444 + 1 — translate before dialing), the relay RelayMessage wire (RLY-001..RLY-008), the peer RPC methods (dig.getPeers/dig.announce/dig.getNetworkInfo/dig.getAvailability/dig.listInventory/dig.fetchRange), and the relay-last-fallback invariant (prefer hole-punch signalling over full relaying)." keywords: - peer network - peer_id @@ -24,6 +24,10 @@ keywords: - Kademlia - provider record - content discovery + - holdings announce + - opcode 222 + - capsule holder discovery + - find_providers tags: - dig-node - relay @@ -289,13 +293,70 @@ A provider record is **soft state**, not a permanent entry, so an offline holder - **Absolute expiry.** `expires_at` is set to `now + TTL` in absolute Unix seconds. A record at or after its `expires_at` is treated as **absent**. - **Republish before expiry.** The holder re-announces (a fresh record with a new `expires_at`) on an interval strictly shorter than the TTL, so its records never expire while it is online. - **Withdraw on removal.** A node that no longer holds content stops announcing it; the record then ages out on its TTL (no explicit delete is required). +- **Announce the gain promptly, don't wait for the republish tick.** A holder diffs its inventory against the content-id set it last announced and PUTs a record for each newly-held key **as soon as it gains it**, so freshness is not bounded by the republish interval. This is what makes the network self-scaling: whichever path put a capsule on disk — a cache fill after a remote read, a hosted pin, a store sync — the node announces it and a reader that caches a capsule becomes a discoverable holder of it, so content replicates toward wherever it is read. - **GC drops the expired.** A responder discards expired records on read and does not return them. - **Inbound RPC populates the routing table bidirectionally.** On **every** inbound DHT RPC, the responder folds the **mTLS-verified caller** (its `Contact`) into its own routing table — every request is evidence the caller is alive, so a node that queries you teaches you about itself. The caller identity MUST come from the authenticated transport, never from a field the caller sets. +:::warning Two ports, one node — translate before you dial +A node's **gossip** listener (`DIG_GOSSIP_PORT`, default **9445**) and its **peer-RPC / DHT** listener (`DIG_PEER_PORT`, default **9444**) are different sockets on the same identity — `gossip_port = peer_port + 1`, a fixed offset. Every address a peer offers as a dial target for a DHT stream or for [`dig.getAvailability`](#availability) / [`dig.fetchRange`](#range) is a **peer-RPC** address; the addresses a node observes on its own gossip pool are **gossip** addresses. A path that seeds the DHT routing table or a download-candidate list from a gossip-pool address MUST subtract the offset first. Dialing `9445` for a peer-RPC stream reaches the gossip protocol instead, and the failure surfaces as a content miss rather than as a connection error. +::: + +#### The holdings-announce wire — opcode 222 {#holdings-announce} + +Beside the pulled provider records, the peer protocol defines a **signed holdings announce**: one message in which a peer states, under its own key, which content keys it serves and which it no longer serves. It is a gossip-pool broadcast message type — **`HoldingsAnnounce`, opcode 222** of the peer message set, classed as a Plumtree eager/lazy-push broadcast at **bulk** priority — and not one of the four [DHT RPC](#dht-wire) methods. + +The shapes below are a **frozen, byte-exact contract**: what a conforming implementation emits, and what it requires before it accepts a statement about another peer's holdings. Every field is covered by the holder's signature. + +**The message.** All integers big-endian; `⟨lp⟩` marks a `u16`-length-prefixed byte string. + +| Field | Shape | Meaning | +|---|---|---| +| `provider_peer_id` | 64 lowercase hex | the holder's `peer_id`; equals `SHA-256(provider_spki)`, and an acceptor **derives** it from the SPKI rather than trusting the carried value | +| `provider_spki` | TLS leaf `SubjectPublicKeyInfo` DER (P-256, ~91 B) | both what the `peer_id` hashes **and** the public key that verifies `signature` — no certificate and no chain are on the wire | +| `seq` | `u64` | monotonic per holder; a higher `seq` states a later position than a lower one | +| `announced_at` | `u64` | Unix seconds the announce was produced | +| `changes` | 1..**256** deltas | the batch (`MAX_CHANGES` = 256; a larger batch is rejected, not truncated) | +| `signature` | ECDSA-P256, ASN.1 DER (~70-72 B) | the holder's signature over the signing message below | + +Each delta is one of two kinds: + +```text +Add = 0x01 ‖ content_key[32] ‖ addr_count(u16) + ‖ ( host_len(u16) ‖ host ‖ port(u16) )* + ‖ expires_at(u64) — now serving content_key at these addresses +Remove = 0x02 ‖ content_key[32] — no longer serving content_key +``` + +`content_key` is the same 64-hex [content key](#dht) the DHT is keyed by (store `0x01` / capsule `0x02` / resource `0x03`), so an announce and a `find_providers` name content identically; `addresses` and `expires_at` carry the same meaning they do on a [`ProviderRecord`](#dht-wire). The delta encoding above is the **canonical encoding** — deterministic, no maps or sets — and both the wire body and the signed bytes use it, so signer and verifier never diverge: + +```text +wire = peer_id⟨lp⟩ ‖ spki⟨lp⟩ ‖ seq(u64) ‖ announced_at(u64) + ‖ change_count(u16) ‖ canonical_encode(changes) ‖ signature⟨lp⟩ + +signed = "dig:holdings:v1" ‖ peer_id[32] ‖ seq(u64) ‖ announced_at(u64) + ‖ canonical_encode(changes) +``` + +The `dig:holdings:v1` domain tag is part of the signed preimage, so a holdings signature can never be replayed as a signature over any other DIG message; the version suffix makes a future preimage change an explicit, distinguishable domain. The signature is computed over the **full preimage** (ECDSA-P256 with SHA-256 internally), not over a pre-hashed digest. A truncated, over-long, or trailing-byte frame is rejected outright. + +**Acceptance is fail-closed — five checks, in order.** An announce is acceptable only when every one of them passes; the first failure rejects the whole message, and with it every delta the message carried: + +1. `changes` holds at most 256 deltas. +2. `provider_peer_id` decodes as exactly 64 hex characters. +3. `SHA-256(provider_spki)` equals that `peer_id` — the identity is derived, never taken on the announcer's word. +4. `provider_spki` parses as a P-256 (`prime256v1`) EC public key. +5. the signature verifies over the signing message under that key. + +Because the signing key is the **TLS leaf key the `peer_id` already commits to**, the message is unforgeable standalone — no handshake, proof of possession, or certificate chain is needed, and there is no separate identity binding a third party could graft onto a copied public key. This is what the signature is for: only the holder can state holdings under the holder's `peer_id`, or name the addresses a reader would dial for them. + +A holdings announce is **public discovery data addressed to every peer**, so it is signed and mTLS-carried rather than sealed to a recipient — the same treatment public consensus broadcasts get, and unlike the recipient-sealed directed peer messages. + +**What a valid signature does and does not establish.** It establishes *who* made the statement. It does not establish that the content is there: an announced address is authenticated only as far as the announcer's `peer_id`, so addresses are untrusted dial hints that MUST NOT be amplified, and a false claim costs a reader one failed fetch attempt — content is merkle-verified against the [chain-anchored root](#range-integrity) before it is trusted or re-served. + #### How a node uses the DHT - **On content-want** (a user asks for `store_id`, `store_id:root`, or a specific resource): derive the matching content key, run `find_providers`, then reach each returned provider over the [NAT ladder](#nat-traversal) and fetch via [`dig.getAvailability`](#availability) + [`dig.fetchRange`](#range). The DHT is **step 1 of the [multi-source download](#multi-source)** — it finds the candidate holders the download then fans out across. -- **On inventory-change** (the node gains or loses content it serves): `add_provider` for each new content key, and stop announcing what it no longer holds. Run republish on the configured interval. +- **On inventory-change** (the node gains or loses content it serves): diff the current inventory's content ids against the set last announced, `add_provider` each newly-held key promptly, and withdraw each key it no longer holds. Run republish on the configured interval. A node announces its inventory at **store** and **capsule** granularity, so a peer can be found by store id or by exact `store_id:root`. - **Bootstrap** the routing table from existing discovery — the gossip peer pool ([§4b](#discovery)) or the relay introducer ([§4a](#discovery)) — then a self-lookup (`find_node` on the node's own id) fills the table. The DHT never hard-depends on a live relay. ### 4d · PEX — peer exchange {#pex} @@ -740,6 +801,7 @@ The peer network is implemented by several crates that must interoperate byte-fo | **NAT ladder** | the ordered strategies DIRECT → UPnP → NAT-PMP → PCP → hole-punch (relay signalling only) → RELAYED/TURN (relay carries data), relay-data-last | that every `connect(peer)` implementation prefers direct, prefers hole-punch signalling over full relaying, and proxies the stream only as a last resort | | **DHT content key** | `SHA-256(tag ‖ canonical bytes)` with tags `0x01` store (`store_id`), `0x02` root/capsule (`store_id ‖ root`), `0x03` resource (`store_id ‖ root ‖ retrieval_key`); node id = `peer_id`; distance = XOR; bucket = `255 − leading_zeros` ([§4c](#dht)) | that every node derives the identical content key for the same content, and places nodes + content in one 256-bit keyspace, so a provider record announced by one implementation is found by another | | **DHT RPC wire** | the four `type`-tagged methods `find_node` / `find_providers` / `add_provider` / `ping` (+ the `error` envelope), `u32`-BE length-prefixed JSON over an authenticated dig-nat mTLS stream (same framing as the relay control messages), `find_providers` always returning `closer` ([§4c](#dht-wire)) | that any node's DHT speaks the same locate-the-holders wire; `dig-nat`/`dig-dht`/`dig-node` conform | +| **Holdings-announce wire** | gossip-pool broadcast opcode **222**; wire `peer_id⟨lp⟩ ‖ spki⟨lp⟩ ‖ seq(u64) ‖ announced_at(u64) ‖ change_count(u16) ‖ canonical_encode(changes) ‖ signature⟨lp⟩` (BE, `u16` length prefixes), deltas `0x01` add (`content_key ‖ addresses ‖ expires_at`) / `0x02` remove (`content_key`), at most 256 per batch; signed `"dig:holdings:v1" ‖ peer_id[32] ‖ seq ‖ announced_at ‖ canonical_encode(changes)` with the TLS leaf key (ECDSA-P256-SHA256, ASN.1); acceptance fail-closed on the five checks, `peer_id` derived as `SHA-256(spki)` ([§4c](#holdings-announce)) | that a signed statement of holdings is byte-identical and independently verifiable across implementations, and that no peer can state holdings under another peer's identity | | **DHT shapes** | `Contact { peer_id:<64hex>, addresses:[{host,port,kind}] }` and `ProviderRecord { content_key, provider_peer_id, addresses, expires_at }`, `addresses[]` byte-compatible with `dig.getPeers`; provider records are TTL'd (absolute `expires_at`), republished before expiry, GC'd when stale, and every inbound RPC folds the mTLS-verified caller into the routing table | that returned contacts/records drop straight into a dial target and that provider state is soft state that ages out | A reimplementation of any peer crate conforms iff it reproduces these — the same discipline that keeps the [read path parity-locked](./conformance-and-parity.md). diff --git a/package.json b/package.json index cc40611..c4aa688 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "docs-dig-net", - "version": "0.5.3", + "version": "0.6.0", "private": true, "scripts": { "docusaurus": "docusaurus", diff --git a/static/llms.txt b/static/llms.txt index 7219873..3a023ad 100644 --- a/static/llms.txt +++ b/static/llms.txt @@ -103,7 +103,7 @@ The protocol is specified as SEVEN bottom-up layers, each naming its canonical c - [L4 · DIG CAT payment & pricing](https://docs.dig.net/docs/protocol/dig-cat-payment): DIG_ASSET_ID, the treasury inner ph, 3-decimal units, the dynamic USD-pegged per-capsule amount (no protocol constant), the client-side all-or-nothing commit bundle and off-chain payment reconciliation. - [L6 · The dig RPC (machine interface)](https://docs.dig.net/docs/protocol/dig-rpc): JSON-RPC 2.0 over POST, the method catalogue, the chunk wire object (chunk_lens first-window-only, NO decoy field), range/window math, error codes incl. -32004 and the node-profile -32006, the node profile vs the network profile. - [L5 · §21 transport & push](https://docs.dig.net/docs/protocol/transport-and-push): dig:// locator resolution, the REST surface, JSON-outer/Chia-codec-inner envelope, 2-leg push v1, the authenticated head (PUSH_DST), per-request §21.9 auth, tombstones, decoy indistinguishability. -- [L7 · DIG Node peer network](https://docs.dig.net/docs/protocol/peer-network): the normative node↔node protocol — mTLS peer identity (peer_id = SHA-256(TLS SubjectPublicKeyInfo DER)), the ordered NAT-traversal ladder (direct → UPnP → NAT-PMP → PCP → relay-coordinated hole-punch (signalling only) → relayed/TURN transport, relay-data-last), the relay's four roles (STUN, introducer, hole-punch signalling, relayed transport), STUN reflexive-address discovery, introducer + gossip peer discovery, the Kademlia DHT (§4c) with provider records that locate WHICH peers hold content — content-key = SHA-256(domain-tag ‖ store_id[‖root[‖retrieval_key]]) (tags 0x01/0x02/0x03), node id = peer_id, distance = XOR, bucket = 255−leading_zeros; the four DHT RPC methods (find_node / find_providers / add_provider / ping + error envelope, type-tagged JSON u32-BE length-prefixed over a dig-nat mTLS stream — NOT JSON-RPC 2.0) returning Contact / ProviderRecord shapes (addresses byte-compatible with dig.getPeers), provider records TTL'd + republished + GC'd and every inbound RPC folding the mTLS-verified caller into the routing table — the relay RelayMessage JSON wire (RLY-001..RLY-007), streaming-first content transport over a multiplexed stream connection, availability/inventory queries (dig.getAvailability batch has_store/has_root/has_capsule + dig.listInventory) as the pre-fetch step, byte-range multi-source content/.dig fetch (dig.fetchRange) with per-range merkle integrity + resume, the peer RPC methods (dig.getPeers / dig.announce / dig.getNetworkInfo / dig.getAvailability / dig.listInventory / dig.fetchRange + -32006 / -32007), and the relay-last-fallback invariant. The anchor dig-nat/dig-relay/dig-gossip/dig-dht/dig-node conform to. +- [L7 · DIG Node peer network](https://docs.dig.net/docs/protocol/peer-network): the normative node↔node protocol — mTLS peer identity (peer_id = SHA-256(TLS SubjectPublicKeyInfo DER)), the ordered NAT-traversal ladder (direct → UPnP → NAT-PMP → PCP → relay-coordinated hole-punch (signalling only) → relayed/TURN transport, relay-data-last), the relay's four roles (STUN, introducer, hole-punch signalling, relayed transport), STUN reflexive-address discovery, introducer + gossip peer discovery, the Kademlia DHT (§4c) with provider records that locate WHICH peers hold content — content-key = SHA-256(domain-tag ‖ store_id[‖root[‖retrieval_key]]) (tags 0x01/0x02/0x03), node id = peer_id, distance = XOR, bucket = 255−leading_zeros; the four DHT RPC methods (find_node / find_providers / add_provider / ping + error envelope, type-tagged JSON u32-BE length-prefixed over a dig-nat mTLS stream — NOT JSON-RPC 2.0) returning Contact / ProviderRecord shapes (addresses byte-compatible with dig.getPeers), provider records TTL'd + republished + GC'd and every inbound RPC folding the mTLS-verified caller into the routing table; a holder announcing each newly-held content key promptly on an inventory gain (whichever path put the capsule on disk — so a reader that caches a capsule becomes a discoverable holder of it) at store + capsule granularity and withdrawing what it no longer holds; the holdings-announce wire (gossip-pool broadcast opcode 222 at bulk priority, NOT a DHT method) — a signed batch of add (content_key ‖ addresses ‖ expires_at) / remove (content_key) deltas, at most 256, signed "dig:holdings:v1" ‖ peer_id ‖ seq ‖ announced_at ‖ canonical_encode(changes) with the TLS leaf key (ECDSA-P256, peer_id derived as SHA-256(spki), acceptance fail-closed on five checks); the gossip/peer-RPC port relationship (gossip 9445 = peer 9444 + 1 — a gossip-pool address is not a peer-RPC dial target, translate first) — the relay RelayMessage JSON wire (RLY-001..RLY-007), streaming-first content transport over a multiplexed stream connection, availability/inventory queries (dig.getAvailability batch has_store/has_root/has_capsule + dig.listInventory) as the pre-fetch step, byte-range multi-source content/.dig fetch (dig.fetchRange) with per-range merkle integrity + resume, the peer RPC methods (dig.getPeers / dig.announce / dig.getNetworkInfo / dig.getAvailability / dig.listInventory / dig.fetchRange + -32006 / -32007), and the relay-last-fallback invariant. The anchor dig-nat/dig-relay/dig-gossip/dig-dht/dig-node conform to. - [L6 · Verification, provenance & anchoring](https://docs.dig.net/docs/protocol/verification-and-provenance): the four ordered integrity gates (merkle → authenticated decryption → anchored-root pinning → risc0), anchored-root pinning against the singleton via coinset.org, the MOCK-by-default risc0 caveat, provenance UX. - [L6 · The blind host model](https://docs.dig.net/docs/protocol/blind-host-model): provider-blindness, serve_blind (framing-only decode), the resolver (*.on.dig.net + custom domains), the /v1 control plane (CHIP-0002 login, JWT sessions, refresh reuse-detection, action re-auth, GitHub-OIDC CI), accept-on-signature push trust. - [Conformance & parity](https://docs.dig.net/docs/protocol/conformance-and-parity): the cross-impl parity discipline (frozen canonical()/retrieval_key() goldens, C8 crypto fixtures), the URN conformance vectors, the OpenRPC-vs-server diff.