StateCache: fills can go stale when a snapshot download extends file visibility with never-applied state

[yperbasis]

Found while qualifying the Frontier stale-low contract in #22444 (review follow-up). Same family as #22463: the cache's latest-value claim is anchored to the apply stream, and state that arrives outside that stream can invalidate fills without anything noticing. Code-inspection level — no reproduction built yet; the startup ordering needs verification.

Mechanism

Fill admission (frontier >= appliedEnd) is sound as long as every txNum beyond the filling view's frontier eventually flows through Applier.Apply. Snapshot download breaks that assumption: ForceReopenUnderlyingFilesTx / files reopen after a download extends visibility with state the process never applied, so appliedEnd knows nothing about it and no cache entry is invalidated.

Concrete shape (negative fill, the sharpest case):

1. Node starts with an incomplete snapshot set; a fill-enabled StateCache is wired at ExecModule construction.
2. A read at latest (embedded RPC, read-ahead) misses account K — its data lives in a segment still being downloaded. The read legitimately returns "absent"; the negative is filled, admitted (cold cache: appliedEnd = 0).
3. The download completes; the files view extends; K now exists — via files, not via any Apply.
4. Every later GetLatest(K) hits the cached negative and serves "absent" for an existing account, until K's next in-process write.

Positive fills have the same shape (an old value cached as latest while the downloaded segment holds a newer one).

The dependency-clamp instance of this family was closed in #22444 (DomainVisibleEnd answers ok=false for clamped values views) — but the download case is not a clamped view; the filling view is perfectly coherent at fill time. The problem is purely that the future arrived outside the apply stream.

Why the existing machinery doesn't catch it

• Admission compares against appliedEnd, which only applies advance; a files extension bypasses it.
• Unwind invalidation (epochs + floor) fires on unwinds, not on visibility raises.
• The visibility-lowering guard (BindAggregator → recalcVisibleFiles assert) forbids lowering; raising is allowed and is exactly this path.
• No production path calls StateCache Clear/invalidation on ForceReopenUnderlyingFilesTx or OnFilesChange.

Exposure

Narrow in practice: fills at latest while segments are still downloading (initial sync, catch-up after downtime, new segment types). A freshly syncing node serves little meaningful latest traffic, and locally built segments cover data that already flowed through applies (safe). But the invariant violation is real, and negative entries make it user-visible (eth_getBalance returning 0/absent for an existing account after sync completes).

Fix directions

1. Invalidate on extension: bump the cache epochs (or Clear) from the files-change hook when a reopen extends a cached state domain's visible end beyond appliedEnd — precise trigger, no cost in the steady state where extensions only cover applied ranges.
2. Wire late: attach the fill-enabled cache only once initial download is complete (apply-only until then). Simple, but doesn't cover later catch-up downloads.
3. Advance appliedEnd on extension: treat a download-driven extension as an authority advance (rejects the stale fills' admission window going forward, but does not evict entries already admitted — insufficient alone).

Direction 1 looks right; direction 3 alone is not enough.

[VBulikov]

yperbasis added Imp2 as Importance