Skip to content

Devnet e2e: full finalize → post-payment failure → resume → success round-trip (#140 follow-up) #144

Description

@Nic-dorman

Context

PR #143 (fixes #140) makes external-signer finalize_upload* retryable after payment via PaidRetryState / Error::FinalizeStorePaidFailed / Client::finalize_resume.

Coverage today:

  • Unit tests (pure assemble_wave_result / assemble_merkle_result): prove the retry-state assembly and the failure → resume → success logic deterministically, without a network.
  • Tier 1 e2e (e2e_file.rs::test_finalize_after_payment_failure_yields_retryable_state_wave): proves the real store path emits FinalizeStorePaidFailed with a correct PaidRetryState when the network is collapsed below quorum after payment, that the failed finalize does not spend more tokens, and that finalize_resume re-drives the real path without re-paying.

Gap (this issue)

No live test of the success-after-resume half: first store attempt fails post-payment, then the network recovers and finalize_resume actually lands the chunks and they are retrievable (including the public data_map_address).

The blocker is harness capability, not the feature:

  • MiniTestnet (in-process) has shutdown_node(i) but no restart — and the wave path re-targets the same quoted_peers baked into each paid chunk, so those exact peers (same PeerId, same data dir) must come back for resume to succeed.
  • LocalDevnet (real antnode processes) only exposes whole-devnet shutdown(), no per-node stop/start.

Proposed work

  1. Add a per-node control primitive — either:
    • MiniTestnet::restart_node(i) (re-spawn at the same port + data dir + bootstrap set, wait for DHT re-convergence), or
    • LocalDevnet::stop_node(i) / start_node(i) proxying the upstream ant_node::devnet::Devnet (preferred — real processes restart cleanly), or
    • a reversible network-block helper (iptables/nftables drop the storage ports, Linux-CI only) — cleanest for the wave same-peer requirement since nodes never die.
  2. Write the round-trip test:
    prepare (public) → pay quotes → collapse network → finalize → assert FinalizeStorePaidFailed + retry → restore capacity → wait for convergence → finalize_resume → assert success → data_map_fetch + file_download → bytes == original.
  3. Add the merkle variant (needs 35+ nodes for 16-per-pool quorum).

Assert throughout that wallet token balance is unchanged across the failed finalize and the resume (no second payment).

Notes

  • Expect this to be slower / more flake-prone than Tier 1 (port reuse, DHT re-convergence). Use short store timeouts on the doomed phase and generous convergence waits on recovery.
  • The reversible-network-block approach avoids the same-PeerId requirement entirely and is the least flaky, at the cost of being Linux-only.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions