Skip to content

Chain-log poller reconnect resumes past the last delivered block, missing reorgs during a disconnect window #62

Description

@lgahdl

Summary

When a chain-log subscription's poller reconnects after a transient error (not a fresh boot), it resumes at last_seen_block + 1 rather than re-validating the last delivered block. If a reorg touches exactly that block while the connection was down, the module is never told the earlier delivery is now stale — no removed event is ever produced for it.

Affected code

  • crates/nexum-runtime/src/runtime/event_loop.rsreconnecting_chain_log_task(), poller_resume_block()
  • crates/nexum-runtime/src/host/provider_pool.rsProviderPool::watch_chain_logs()

Finding

While a stream stays connected, reorgs are handled correctly: watch_chain_logs() delegates canonical-chain tracking to alloy_provider's watch_canonical_logs_from, which emits CanonicalEvent::Added/Removed, and the host correctly stamps removed on each log, all the way through to the guest-visible ChainLog.removed field.

Across a reconnect, though, poller_resume_block() computes the new stream's start block as:

fn poller_resume_block(last_seen_block: Option<u64>, head: u64) -> u64 {
    match last_seen_block {
        None => head,
        Some(last) => last.saturating_add(1),   // resumes AFTER, not AT
    }
}

— already asserted by the crate's own test, poller_resume_block_resumes_after_last_delivered: poller_resume_block(Some(90), 100) == 91.

Because each reconnect opens a brand-new watch_canonical_logs_from instance with no memory of the previous instance's view, if last_seen_block itself gets reorged out during the disconnected window, the new stream never re-fetches or re-validates that height — it just moves forward from last_seen_block + 1, building on top of a height whose canonical status was never reconfirmed.

Contrast with the persisted-cursor boot path (boot_resume), which explicitly resumes at the cursor block (not after it) for exactly this reason, per its own comment: "so a mid-block crash before the restart loses nothing." The in-process reconnect path has no equivalent safeguard.

Impact

A malicious or flaky RPC provider that briefly drops the connection right after delivering a block, then presents a fork at exactly that height on reconnect, can feed a module state that's inconsistent with any chain state that was ever real — with no removed signal ever raised. Any watch, submitted-order marker, or decision built off the stale delivery is never corrected.

Suggested fix

On reconnect, resume at last_seen_block (not last_seen_block + 1) and re-deliver/re-diff that height, mirroring what the persisted-cursor boot path already does.

Found via

Internal red-team exercise (malicious/compromised RPC provider persona). Confirmed via code inspection plus the crate's own existing unit test asserting the exact resume arithmetic described above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions