Skip to content

docs(protocol): real-time capsule-holder discovery (find_providers + holdings-announce) - #47

Merged
MichaelTaylor3d merged 4 commits into
mainfrom
docs/1430-holder-discovery
Jul 26, 2026
Merged

docs(protocol): real-time capsule-holder discovery (find_providers + holdings-announce)#47
MichaelTaylor3d merged 4 commits into
mainfrom
docs/1430-holder-discovery

Conversation

@MichaelTaylor3d

@MichaelTaylor3d MichaelTaylor3d commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Documents capsule-holder discovery on the L7 peer-network page (dig_ecosystem #1430, the docs-coherence sibling of the #1394 discovery epic / #1425 flywheel map). All additions sit inside the existing §4c "Content discovery — the DHT"; no new page, no IA change.

What changed

docs/protocol/peer-network.md

  • Provider-record lifecycle — new rule: a holder diffs its current inventory against the content-id set it last announced and PUTs a record for each newly-held key as soon as it gains it, so freshness is not bounded by the republish interval. This carries the flywheel statement: whichever path put a capsule on disk (cache fill after a remote read, hosted pin, store sync), the node announces it, so a reader that caches a capsule becomes a discoverable holder of it.
  • How a node uses the DHT — the inventory-change rule now states the diff-against-last-announced behaviour, the prompt PUT, the withdraw, and that inventory is announced at store and capsule granularity, so a peer is findable by store id or by exact store_id:root.
  • New :::warning "Two ports, one node" — gossip 9445 = peer-RPC/DHT 9444 + 1; a peer's dial targets are peer-RPC addresses while a node's own gossip-pool observations are gossip addresses, so a path that seeds DHT routing or a download-candidate list from a gossip-pool address must subtract the offset. Dialing 9445 for a peer-RPC stream reaches the gossip protocol and surfaces as a content miss rather than a connection error — the failure mode that caused two prior outages.
  • New #### The holdings-announce wire — opcode 222 subsection ({#holdings-announce}) — strictly the wire contract: that it is a gossip-pool broadcast message type at bulk priority and not one of the four DHT RPC methods; the field table (provider_peer_id / provider_spki / seq / announced_at / changes ≤256 / signature); the add/remove delta canonical encoding; the wire layout and the domain-separated dig:holdings:v1 signing preimage; the five fail-closed acceptance checks; why signing with the TLS leaf key the peer_id already commits to makes the message unforgeable standalone with no graftable identity binding; that it is a public all-peers statement, signed rather than recipient-sealed; and what a valid signature does not establish — addresses are untrusted, non-amplifiable dial hints and content is merkle-verified against the chain-anchored root before it is trusted or re-served.
  • Conformance table — new Holdings-announce wire row pinning the wire, the signed preimage, the 256-delta cap, and fail-closed acceptance with the derived peer_id.
  • Frontmatter — description extended, four keywords added.

static/llms.txt — the peer-network entry now summarizes the announce-on-gain provider-record behaviour, the holdings-announce wire, and the port relationship, matching the page.

Deliberately unchanged: static/dht-methods.json documents the four-method DHT RPC wire, and a holdings announce is a gossip-pool broadcast, not a DHT method.

How verified

  • npm ci && npm run buildgreen, all 15 locales. onBrokenLinks: "throw" + onBrokenAnchors: "throw" are enabled, so every new anchor and cross-link is build-validated. postbuild hreflang sitemaps regenerated cleanly (14 sitemaps, 1680 urls).
  • npm run typecheck (tsc) clean; npm run lint (eslint) clean; npm run test:unit 26/26.
  • Facts read from source at origin/main, not inferred: the shipped discovery path is dig-node crates/dig-node-core/src/seams/dig_peer/dht.rsannounce_inventory (per-content-id announce_provider), inventory_content_ids (store + capsule granularity), inventory_diff, sync_inventory (prompt announce of new ids, withdraw_provider for dropped ids, TTL age-out), reached via refresh_dht_inventory in seams/dig_peer/peer_network.rs. The opcode-222 wire is dig-gossip src/service/holdings_announce.rs (opcode 222, SIG_DOMAIN_TAG = b"dig:holdings:v1", MAX_CHANGES = 256, canonical_encode, holdings_signing_message, encode/decode, verify_holdings_announce) with its broadcast/priority classification in src/gossip/broadcaster.rs + src/gossip/priority.rs. Ports and the fixed offset come from the canonical port contract.

Review finding addressed

The first revision described provider-store ingest semantics, announce triggers, and a receive-side apply rule for opcode 222. git grep -lI -E "holdings_announce|HoldingsAnnounce|HOLDINGS_ANNOUNCE" origin/main -- '*.rs' returns zero hits across dig-node, and in dig-gossip the only references outside the wire module are the lib.rs re-exports and the priority/broadcast classification — so no producer emits an announce and no consumer applies one. Those passages described behaviour that does not exist and have been removed; the page now documents the provider-record path that actually carries discovery, and keeps opcode 222 only as the byte-exact wire contract a conforming implementation reproduces.

Version bump — 0.5.3 → 0.6.0 (minor)

New capability documented (a previously undocumented protocol surface, a new conformance row, and a new anchor other pages can link to), additive only, nothing removed or restated differently.

Closes #1430 (dig_ecosystem).

MichaelTaylor3d and others added 3 commits July 26, 2026 04:10
Co-Authored-By: Claude <noreply@anthropic.com>
Document the push half of the provider map on the L7 peer-network page: the
opcode-222 holdings announce — its gossip-pool transport, message + signed
byte layout, the fail-closed leaf-key verification, the ingest rules for add
and remove deltas, and when a holder announces (including the runtime cache
gain that turns a reader into a discoverable holder). Also states the
gossip/peer-RPC port relationship a dial path must translate, extends the
provider-record lifecycle and DHT usage rules, adds the conformance row, and
refreshes the llms.txt summary for machine consumers.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@MichaelTaylor3d
MichaelTaylor3d marked this pull request as ready for review July 26, 2026 11:25
…2 as wire only

Holder discovery is documented as the provider-record path it is: a holder
diffs its inventory and announces each newly-held content key promptly at
store + capsule granularity, withdrawing what it no longer holds. The
flywheel statement stays, attributed to that announce.

The opcode-222 subsection is now strictly the wire contract — fields,
canonical encoding, signing preimage, the five fail-closed acceptance checks,
and the leaf-key identity reasoning — with no claim about when a peer
announces and no provider-store semantics. Conformance row and the llms.txt
summary follow. The gossip/peer-RPC port translation warning is retained and
restated so it stands on gossip-pool addresses generally.

Co-Authored-By: Claude <noreply@anthropic.com>
@MichaelTaylor3d
MichaelTaylor3d merged commit 745c22e into main Jul 26, 2026
8 checks passed
@MichaelTaylor3d
MichaelTaylor3d deleted the docs/1430-holder-discovery branch July 26, 2026 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant