Skip to content

execution: abort cache transitions after interrupted read-ahead drain - #22469

Draft
yperbasis wants to merge 4 commits into
mainfrom
yperbasis/warmup-epoch-fences
Draft

execution: abort cache transitions after interrupted read-ahead drain#22469
yperbasis wants to merge 4 commits into
mainfrom
yperbasis/warmup-epoch-fences

Conversation

@yperbasis

@yperbasis yperbasis commented Jul 14, 2026

Copy link
Copy Markdown
Member

Problem

Block read-ahead runs asynchronously and can populate StateCache from a committed read-only snapshot. The execution paths already wait for read-ahead before an unwind epoch bump or FCU cache apply, but WaitForWarmup did not report whether the warmup completed or the module context cancelled the wait. A caller could therefore continue without satisfying the drain precondition.

Solution

  • WaitForWarmup and drainReadAhead report whether the warmup fully drained.
  • unwindToCommonCanonical, updateForkChoice, and SetHead stop when shutdown interrupts the drain.
  • Under ASSERT_STATE_CACHE, StateCache.Unwind rejects an unwind while a directly cache-populating read-ahead warmup is counted as in flight.
  • A warmup uses one launch-time StateCache instance for both its diagnostic gauge and cache writes.

The database-close path intentionally ignores the result because it only needs a bounded wait before shutdown.

Scope

This PR covers only the direct BlockReadAheader lifecycle. It does not address:

#21414 proposes routing read-ahead through the published SharedDomains. That implementation must preserve the requirement that read-ahead fully drains before an unwind or cache apply.

Testing

  • TestStateCache_UnwindAssertsWarmupInFlight
  • TestWaitForWarmupReportsDrained
  • go test ./execution/cache ./execution/exec ./execution/execmodule
  • go test -race -count=1 ./execution/cache ./execution/exec ./execution/execmodule
  • make erigon integration
  • repeated clean make lint

…pulating warmups against epoch bumps and clears

Split from #22159 (StateCache review findings #22120).

- WarmupStarted/WarmupDone bracket every fire-and-forget warmup and
  StateCache.Unwind panics under ASSERT_STATE_CACHE if one is still in
  flight — the drain-before-epoch-bump convention becomes a loud
  invariant. The warmup binds its gauge and its puts to one launch-time
  cache snapshot so a racing SetStateCache cannot split the pair.
- WaitForWarmup and drainReadAhead report whether the warmup fully
  drained (false only when the module context is cancelled), and every
  epoch-bump or Clear call site treats an interrupted drain as a failed
  precondition instead of proceeding; the DB-close caller keeps its
  bounded wait.
- ExecModule.Start drains and clears the state cache under the module
  semaphore before ProcessFrozenBlocks, so pre-start payload validation
  cannot leave pre-catchup entries live across frozen-block processing.
- The read-ahead getter carries a snapshot-progress oracle: negatives
  are stamped with the domain's progress at observation time (a getter
  without an oracle fills nothing), and a side-effect-free
  CodeCache.ContainsLive probe lets it skip the keccak+copy for
  already-live address bindings.
@yperbasis
yperbasis requested a review from mh0lt as a code owner July 14, 2026 12:04
@yperbasis
yperbasis marked this pull request as draft July 14, 2026 12:12
@yperbasis yperbasis changed the title execution/exec, execution/execmodule, execution/cache: fence cache-populating warmups against epoch bumps and clears execution: fence cache-populating warmups against epoch bumps and clears Jul 14, 2026
@yperbasis
yperbasis requested a review from Copilot July 14, 2026 14:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the lifecycle of asynchronous cache-populating warmups so they cannot race cache epoch bumps (unwind/clear) and contaminate canonical state, and it reduces unnecessary work in read-ahead warm paths.

Changes:

  • Make read-ahead warmup draining report “fully drained vs interrupted”, and gate unwind/clear/FCU paths on a successful drain.
  • Track in-flight warmups in StateCache and (under ASSERT_STATE_CACHE) assert that Unwind never runs while a cache-populating warmup is still running.
  • Add side-effect-free “live binding” probes (CodeCache.ContainsLive / StateCache.HasLiveCode) so read-ahead can skip keccak+copy when a conditional put would no-op; add tests for the new behaviors.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
execution/execmodule/set_head.go Bail out of SetHead unwind if read-ahead warmup drain is interrupted.
execution/execmodule/forkchoice.go Treat interrupted warmup drain as shutdown and abort FCU handling early.
execution/execmodule/exec_module.go Make drainReadAhead return a drained/undrained boolean; drain+clear on Start; enforce drain before unwind.
execution/exec/blocks_read_ahead.go Add warmup Started/Done bracketing + cache snapshot binding; make WaitForWarmup return bool; implement negative stamping and “skip for live code” logic.
execution/exec/blocks_read_ahead_test.go Add coverage for drained reporting, live-binding skip, and negative invalidation behavior.
execution/cache/state_cache.go Add warmup gauge + assert in Unwind; expose HasLiveCode.
execution/cache/grow_lru.go Add Peek passthrough for side-effect-free LRU probes.
execution/cache/code_cache.go Add ContainsLive probe for “live binding” detection without recency/counter side effects.
execution/cache/cache_test.go Add tests for CodeCache.ContainsLive and the unwind-vs-warmup invariant assertion.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread execution/exec/blocks_read_ahead.go
Comment thread execution/exec/blocks_read_ahead.go Outdated
@mh0lt

mh0lt commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Heads-up: #21414's latest commit reworks the same read-ahead prefetch this PR hardens, taking a different approach — flagging so we can coordinate direction before either merges.

What #21414 does: warmBody now prefetches through the published SharedDomains (its own coordinated tx, sd.AsGetter) instead of the raw tx, and deletes cachePopulatingGetter — cache population becomes the SD's own read-fill. Driven by two invariants: the state cache is a SharedDomains implementation feature (only the SD writes it), and all reads go through the in-flight SD. This removes the one cache-writer that sat outside the SD, and reads now see in-flight tip state rather than committed-behind-tip state.

Direct conflict: both PRs edit blocks_read_ahead.go (+ its test, exec_module.go, forkchoice.go, set_head.go). #21414 deletes cachePopulatingGetter and its test; this PR extends both.

Complementary vs obviated, if the SD-routed prefetch is the agreed direction:

My read: the SD-routed prefetch (remove the anomaly) with this PR's drain/gauge/finding-10 hardening layered on top is the clean combination — but it's your call on direction. Related: #22520 (SD-owned run-task API + dropping the parallel-exec start queue).

…h-fences

# Conflicts:
#	execution/cache/cache_test.go
#	execution/exec/blocks_read_ahead.go
#	execution/exec/blocks_read_ahead_test.go
@yperbasis yperbasis changed the title execution: fence cache-populating warmups against epoch bumps and clears execution: fence cache-populating warmups before unwinds and flushes Jul 31, 2026
@yperbasis yperbasis changed the title execution: fence cache-populating warmups before unwinds and flushes execution: enforce warmup drains before unwinds and flushes Jul 31, 2026
@yperbasis

yperbasis commented Jul 31, 2026

Copy link
Copy Markdown
Member Author

@mh0lt Agreed. On current main, the normal read-ahead drains before unwinds and FCU cache apply are already present. #22469 is therefore limited to making an interrupted drain a failed precondition and adding an optional ASSERT_STATE_CACHE diagnostic for direct read-ahead warmups.

The getter-specific optimization and behavior are no longer part of the PR. #21414 should preserve the successful-drain requirement when it routes read-ahead through published SharedDomains.

The broader cache-coherence cases remain separate: startup processing is #22925, forward stale read-view fills are #22444, and pre-unwind stale read-view fills are #22463.

@yperbasis yperbasis changed the title execution: enforce warmup drains before unwinds and flushes execution: abort cache transitions after interrupted read-ahead drain Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants