System information
Commit hash: 7566757e37 (main)
This is a code-level concurrency problem and is not specific to an OS, network, or node configuration.
Expected behaviour
After ProcessFrozenBlocks advances durable account, storage, or code state during startup, StateCache must not serve or accept values from an older temporal snapshot.
Actual behaviour
Engine endpoints are live before ExecModule.Start. Early payload validation or a reader using a published SharedDomains can populate the process-wide StateCache from the pre-catchup state.
ExecModule.Start then calls ProcessFrozenBlocks. That path advances durable domain state through SharedDomains instances that are not connected to StateCache, so existing pre-catchup entries are not updated. Later execution can therefore receive stale account, storage, or code values from the cache and compute the wrong state.
A clear by itself is not a complete fix unless it also fences every cache-populating reader. A temporal read opened before the clear can finish afterward and refill an old value. Waiting for BlockReadAheader only covers asynchronous read-ahead; regular SharedDomains read-fill can still cross the reset.
Steps to reproduce the behaviour
- Commit account value
v1 and open a temporal read transaction at that state.
- Advance the durable domain to
v2 through a path that does not apply updates to StateCache, matching frozen-block startup processing.
- Clear the cache at the startup boundary.
- Complete the old temporal read so it offers
v1 to the empty cache.
- Read the account from a new temporal transaction.
Without a snapshot-generation or domain-progress fence, the new read can hit cached v1 instead of durable v2.
Regression coverage should also test the simpler main-branch case where v1 is already cached before uncached frozen-block processing advances the database to v2.
Context
Found while reviewing #22469. That PR should remain focused on cache-populating warmup lifecycle and interrupted-drain handling; startup cache coherence should be fixed and tested under this issue.
System information
Commit hash:
7566757e37(main)This is a code-level concurrency problem and is not specific to an OS, network, or node configuration.
Expected behaviour
After
ProcessFrozenBlocksadvances durable account, storage, or code state during startup,StateCachemust not serve or accept values from an older temporal snapshot.Actual behaviour
Engine endpoints are live before
ExecModule.Start. Early payload validation or a reader using a publishedSharedDomainscan populate the process-wideStateCachefrom the pre-catchup state.ExecModule.Startthen callsProcessFrozenBlocks. That path advances durable domain state throughSharedDomainsinstances that are not connected toStateCache, so existing pre-catchup entries are not updated. Later execution can therefore receive stale account, storage, or code values from the cache and compute the wrong state.A clear by itself is not a complete fix unless it also fences every cache-populating reader. A temporal read opened before the clear can finish afterward and refill an old value. Waiting for
BlockReadAheaderonly covers asynchronous read-ahead; regularSharedDomainsread-fill can still cross the reset.Steps to reproduce the behaviour
v1and open a temporal read transaction at that state.v2through a path that does not apply updates toStateCache, matching frozen-block startup processing.v1to the empty cache.Without a snapshot-generation or domain-progress fence, the new read can hit cached
v1instead of durablev2.Regression coverage should also test the simpler main-branch case where
v1is already cached before uncached frozen-block processing advances the database tov2.Context
Found while reviewing #22469. That PR should remain focused on cache-populating warmup lifecycle and interrupted-drain handling; startup cache coherence should be fixed and tested under this issue.