feat(content): report peer-tier attachment on every serve and on /health (#1763)

[MichaelTaylor3d]

Closes DIG-Network/dig_ecosystem#1763.

The defect

dig-node opens its HTTP surface ~30 s before the P2P content engine attaches. A read routed in that window skips Tier 2 outright and is answered by the public gateway — and said nothing about it. A fresh node, or any test that started promptly, measured the gateway and recorded the result as P2P. That makes it a correctness bug about evidence, not routing.

The decision: (c) — serve, and expose readiness explicitly

Weighed against the alternatives:

• (a) delay serving until attach — trades availability for honesty. A node that refuses all reads for half a minute is a worse product than one that serves from the gateway, and it would break the existing "spawn → poll /health → read" harness contract (SPEC §7.8). Rejected.
• (b) serve but mark the response — keeps both, but a response-only signal still forces a caller to issue a read to discover readiness, and gives an acceptance test nothing to wait on except a guessed sleep.
• (c) serve, mark the response, AND expose readiness out-of-band — chosen. It is (b) plus the thing that actually removes sleep 30 from the harness.

This is wire-visible, so it is stated normatively in SPEC.md rather than left implicit:

• X-Dig-Peer-Tier: attached | unattached on every served response. Captured once at request entry, before any tier runs, so it describes the routing decision that read faced — not whether the engine happened to attach while the read was in flight. Applied uniformly across whichever tier served the bytes. MUST NOT be derived from X-Dig-Source.
• GET /health → peer_tier: { attached } — the checkable wait condition. status: "ok" means live; it has never implied a usable peer tier, and now nothing has to infer that.

X-Dig-Source cannot carry this on its own: a gateway serve because no peer held the resource and a gateway serve because there was no peer tier yet are both rpc.

On naming (#1738). That ticket's complaint is that X-Dig-Verified reports enforcement while its name says verification. This header is deliberately narrow and named for exactly what it holds: whether the engine was attached. Not peer count, not reachability, not whether a fetch was attempted, and explicitly not verification. Both SPEC.md and the enum doc say so, so a second implementation cannot widen it by accident.

Blast radius checked

gitnexus is disabled in the loop (worktree hangs), so this was done with ripgrep + a direct read of every call site.

• PlaintextOutcome::Served gained a field. Constructed in 3 places (all in content_serve.rs, all inside serve_content_plaintext's own tiers) and destructured in 3 (two arms in server.rs, one core test). The compiler confirmed the radius: adding the field produced exactly one unmentioned-field error outside the files already being edited.
• with_serve_metadata — private to content_serve.rs, 3 call sites, all updated.
• served_response — private to server.rs, 2 call sites. Its five positional provenance args were collapsed into a ServeProvenance struct rather than growing to a sixth transposable positional (it already carried #[allow(clippy::too_many_arguments)], now removed).
• New Node::peer_tier() is additive; p2p_content() is untouched and stays pub(crate).
• EXPOSED_DIG_HEADERS 10 → 11. This is a cross-repo contract (SPEC §4.3, mirrored in the superproject SYSTEM.md and consumed by dig-urn-resolver) — see "For the orchestrator" below.
• §5.3 endpoint ladder untouched: no change to endpoint resolution, no new hardcoded rpc.dig.net.

Risk: LOW. Additive on both surfaces, nothing removed, renamed, or re-meant.

Tests — and why these fixtures

cold_start_gateway_serve_reports_the_peer_tier_as_unattached (dig-node-service, real HTTP). Not a mock of the window — build_state is the same constructor the daemon uses and does not attach the engine, so the reader node is genuinely inside it. Its Tier-3 fallback is a second real dig-node answering dig.getContent from its own cache, so the ciphertext, proof and chunk lengths come from the production serve path rather than being hand-built. It went RED on <header absent> after the 200 and x-dig-source: rpc assertions passed — which is what proves the fixture really is a cold-start gateway serve.

serve_reports_peer_tier_attachment_independently_of_the_serving_tier (dig-node-core). The nearest wrong implementation is peer_tier = if source == Rpc { Unattached } else { Attached } — indistinguishable from the real thing on any gateway-serve fixture. So this drives one capsule, locally seeded and chain-anchored, twice, varying only whether an engine is attached; both arms serve Local. Arm 1 is Local-and-Unattached (that impl says Attached) and arm 2 is Local-and-Attached, so it is satisfiable by neither the source-derived impl nor any constant.

health_reports_the_peer_tier_as_unattached_before_the_peer_network_starts — pins the out-of-band half.

Plus the existing real-provider peer-serve test now asserts Attached, so the two reads #1763 calls indistinguishable assert opposite values.

Load-bearing proof (committed first, reverted via file copy):

revert	test
peer_tier capture → constant Attached	core two-arm test RED (attach=false); cold-start e2e RED
/health field → constant true	health test RED (and correctly unaffected by the revert above)

Evidence

Real cold start, real dig-node run binary, polling /health:

t=+0s   no answer
t=+5s   ok {'attached': False}
t=+10s  ok {'attached': False}
t=+15s  ok {'attached': True}      <- engine attaches
t=+20s  ok {'attached': True}

status: ok at +5 s with the peer tier down is exactly the window the ticket describes, now observable instead of inferred. The same live node confirms the header reaches the browser contract:

access-control-expose-headers: x-dig-verified,x-dig-root,...,x-dig-source,x-dig-peer-tier,...

Response-side: the two reads that were indistinguishable now assert opposite values — rpc/unattached on the real cold-start gateway serve, peer/attached on the real provider fetch.

Gates

cargo fmt --all clean · cargo clippy --all-targets --locked -- -D warnings exit 0 · cargo test --locked full workspace green (463 + 246 + all integration suites).

Version

0.67.0 → 0.68.0 (workspace), dig-node-core 0.24.0 → 0.25.0. Minor: additive response header + additive /health field, nothing removed or changed.

For the orchestrator (superproject, not touched here)

SYSTEM.md mirrors the exposed-X-Dig-* set as a cross-repo contract with dig-urn-resolver. It needs X-Dig-Peer-Tier added to that list. I did not edit the superproject.

Deliberately left undone

• Non-Served outcomes carry no peer-tier. A cold-start read that 502s or 404s is still silent about the tier. PlaintextOutcome's error variants have no such field and error_response/not_found are shared by unrelated paths, so that is real scope rather than a one-liner. Filed as a follow-up.
• #1764 / #1765 are a different family — chain-anchor enforcement policy, not tier readiness. Assessment in the ticket.

[MichaelTaylor3d]

VERDICT: CHANGES-REQUIRED (head read: 38da9a5ad1bb36ab1c92aab96346a91dd8f595da)

The design is right and option (c) is implemented coherently: peer_tier is captured ONCE at request entry before any tier runs (content_serve.rs:431), stamped uniformly in with_serve_metadata, and the three Served constructors are the only ones (verified: decrypt_local and peer_serve_plaintext each have exactly ONE caller, both routed through with_serve_metadata, so no Served escapes with the placeholder). The None -> Unattached mapping fails CLOSED. SPEC.md is updated in the same unit (§4.3 exposed set, new §4.6 paragraph, §6.1 health shape, §7.8 harness rule) and the §5.3 ladder is untouched. ServeProvenance is a genuine readability improvement over a sixth transposable positional. The two-arm core test serve_reports_peer_tier_attachment_independently_of_the_serving_tier is non-vacuous and does kill the source-derived impl from both directions — that one is real.

What blocks: the suite only ever asserts the unattached/false half at the two WIRE surfaces. PeerTier::Attached => "attached" and /health -> peer_tier.attached: true are asserted by no test anywhere in the repo (git grep '"attached"' finds exactly one hit, and it is the production json! line). This is the shape that has already shipped here: a suite covering one wire state stays green with the wire wrong, and X-Dig-Peer-Tier's value set is a cross-repo contract with dig-urn-resolver per SPEC §4.3. The remedy is 2–3 assertions, no design change. Details inline.

Non-gating notes posted separately and resolved by me.

Also confirmed for the orchestrator (not this repo, correctly deferred): superproject SYSTEM.md:107 is now stale on BOTH counts — it lists the provenance set as X-Dig-Verified / X-Dig-Root / X-Dig-Source and the health shape as {status, mode, version, cache}. docs.dig.net does not document the X-Dig-* set at all, so §4.3 triggers nothing there.

[MichaelTaylor3d]

VERDICT: PASS — round 2, head read 090b99bf8e037e3ae255f8d21070655b9a9d8e11 (delta 38da9a5..090b99bf only; the rest of the diff was cleared in round 1).

Both gating findings are genuinely closed, and I falsified each new test rather than trusting the report:

• Finding 1 closed — server.rs:1919 served_response_reports_both_peer_tier_wire_values asserts the header string for BOTH arms through the real response builder. Delete the .header("X-Dig-Peer-Tier", …) line and both arms get None != Some(_) → RED. Swap either literal, or pin both to one constant → RED. The control claim holds for the mutant class reachable at this layer: ServeSource::Local is constant across both arms, so a served_response mutant emitting if source == Rpc { "unattached" } else { "attached" } yields attached twice and fails the Unattached arm. The source-derived mutant one layer UP (inside serve_content_plaintext) is not visible to this test and does not need to be — the core two-arm test serve_reports_peer_tier_attachment_independently_of_the_serving_tier kills it, from both directions. Non-vacuous: data.bin is non-HTML so the builder reaches the header block unconditionally.
• Finding 2 closed — the peer_tier_status extraction is behaviour-preserving: same key peer_tier, same insert position (after sync, before the return), byte-identical json! expression, and I checked all THREE status_fields callers (server.rs:520 /health, :575 the ws snapshot, :596 the heartbeat) — every one keeps the identical field, so /ws/status is unaffected. health_peer_tier_status_reports_both_directions pins both arms of the mapping; forcing either to a constant is RED.
• Finding 3 folded in — server.rs:2068 now asserts EXPOSED_DIG_HEADERS.contains("x-dig-peer-tier"); removing the entry is RED.

No visibility was widened. peer_tier_status, served_response, ServeProvenance and status_fields are all private (no pub); the tests reach them via use super:: from the in-crate mod tests, which is the correct way to do this. StorePath was already pub in content.rs:109. Node::p2p_content() is still pub(crate) (download.rs:1316) and set_p2p_content was not touched — the implementer explicitly declined the visibility widening that a fully-composed health test would have needed, which is the right trade.

On the residual unasserted line — NOT a merge blocker; a follow-up at most. It is also strictly narrower than reported: health_reports_the_peer_tier_as_unattached_before_the_peer_network_starts drives real /health → status_fields → state.node.peer_tier() → peer_tier_status, so the composition IS covered end-to-end in the Unattached direction. What remains uncovered is the Attached direction of that one branch-free delegation, whose two operands are each pinned per-arm (Node::peer_tier() in the core test, peer_tier_status in the unit test) and whose only remaining mutant is substituting a literal for the accessor's result. Closing it would require making attachment settable from dig-node-service — trading real production encapsulation for one line of test reach, on a path the PR already has live-node evidence for (attached: false at +5 s/+10 s flipping true at ~+15 s). Accept as-is.

Gates: all 14 checks SUCCESS (0 non-success, 1 skipped release job), Test + coverage green so the 80% floor holds mechanically at the reported 83.28%. Zero unresolved review threads. Merge is unblocked from this gate's side; the orchestrator still owns the squash-merge, and the superproject SYSTEM.md:107 sweep (add X-Dig-Peer-Tier to the provenance list, refresh the /health shape) remains its action.