fix: IPFS peering never persists, and bitswap hangs even when peered

[EnriqueL8]

Background

#359 fixed the initial "IPFS nodes never peer" bug (private-network connection bug in go-ipfs:v0.10.0, mDNS unreliable on Linux Docker bridge networks in CI) by bumping to ipfs/kubo:v0.42.0 and adding an explicit peering step (swarm/peering/add + swarm/connect) after startup.

It's still failing intermittently in CI and in the firefly Core E2E suite (see hyperledger-firefly/firefly#1766 run failures on v1.5.0-rc.3) with the same symptom: Waiting for 2 orgs to appear timing out after 10 minutes, IPFS shared storage downloads failing with context deadline exceeded.

Root causes (two separate bugs)

1. swarm/peering/add returns success but never persists anything.

Verified directly: called the endpoint, got {"Status":"success"}, then immediately checked ipfs config Peering.Peers on the same node - it was null. The endpoint only registers the peer with the running daemon's in-memory peering service for that process's lifetime. It does not write to the on-disk config the way ipfs config (the CLI command) does.

This matters because runFirstTimeSetup does a full docker compose stop + restart partway through, to apply finalized FireFly config. Since the peering relationship was never actually saved anywhere, that restart silently wipes it. From then on the nodes are back to relying on mDNS, which is unreliable in CI - so every run was a coin flip depending on whether FireFly's first broadcast happened to land before that restart.

Fix: POST /api/v0/config?arg=Peering.Peers&arg=<json> instead - the same endpoint the ipfs config CLI command uses under the hood. Verified this persists to disk and survives a full container restart, with both nodes reconnecting automatically within ~30s.

2. Even genuinely peered, content fetches can still hang.

This is a long-standing upstream Kubo/bitswap bug: ipfs/kubo#8346, filed 2021 against go-ipfs 0.9.1, still open and unaddressed (labeled need/analysis, no maintainer activity since 2021). In a small private network, bitswap finds a provider via the DHT but never sends it a WANT message if the swarm connection to that peer already existed - so it looks connected, ipfs routing findprovs correctly identifies the provider, but ipfs cat/the gateway just hangs.

Reproduced directly: with Routing.Type: dht, ipfs bitswap stat showed partners [0] on both nodes despite ipfs swarm peers showing an active, bitswap-protocol-negotiated connection - even right after a fresh, verified-persisted peering + restart.

Fix: set Routing.Type: none instead of dht. With no DHT to depend on, bitswap has no choice but to broadcast wants directly to its connected peers, which is all a small private swarm actually needs - there's no "crowd" to search when you already know and are directly connected to the only other member. Verified 5/5 sequential cross-node ipfs cat calls plus the HTTP gateway path all succeed reliably with this change, versus 0/many succeeding with dht.

Test plan

• Fresh 2-member private-mode stack: Peering.Peers persists and survives a full manual container restart, verified via ipfs config Peering.Peers before/after
• 5/5 sequential cross-node ipfs cat calls succeed with Routing.Type: none (0/many succeeded with dht, reproduced repeatedly)
• HTTP gateway cross-node fetch succeeds
• go build ./... passes
• CI Build/E2E