From b64b55056e5fabe391f2781c1084faa6661a3210 Mon Sep 17 00:00:00 2001 From: Andrew Ashikhmin <34320705+yperbasis@users.noreply.github.com> Date: Wed, 5 Aug 2026 15:09:22 +0200 Subject: [PATCH] execution, db: bind StateCache fills to transaction views and reject stale fills (#22444) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #22356 — the forward stale-fill direction: a fill from a read view older than the latest apply is rejected. The reverse direction around unwind is tracked in #22463 and stays open (details in the scope section). `StateCache` is one process-global in-memory cache of the **latest committed values** of accounts, storage and code. It exists to skip the file-accessor/MDBX stack on repeated `GetLatest` reads. It is not a snapshot: per key it holds one value — the newest committed value known to the process, whether applied by execution or filled from a read. It is on by default; `USE_STATE_CACHE=false` constructs no cache at all (readers go straight to the backing tx), and `STATE_CACHE_FILLS=false` disables every reader write (apply-only mode) — the kill switch and the A/B lever for measuring what fills contribute. | Actor | Reads | Writes | When | |---|---|---|---| | Canonical execution (`SharedDomains.Commit`) | via `GetLatest` | **apply**: put committed updates, physically delete deletions | after the RwTx commit succeeds — `sd.Commit` walks the pending updates into the cache only once `tx.Commit` returns | | Embedded RPC (`execmodule.CacheView` → `SharedDomains.AsGetter`) | via `GetLatest` | **fill**: on a miss, offer the value read from its own tx | during any request at `latest` | | Read-ahead warmup (`blocks_read_ahead`) | — | **fill**: same, to pre-warm keys for exec | while exec runs | A **fill** is a cache write performed on behalf of a *database* reader (fill-on-miss); an **apply** is the authoritative post-commit write from `SharedDomains.Commit`. A **read view** is what a read-only temporal tx sees (its MDBX read tx + pinned files view); its **frontier** is the exclusive txNum end of what it can see — a view with frontier N sees txNums < N. This sharing — including the RPC fills — is `main`'s code today (wiring: `node/eth/backend.go` hands `execmodule.Cache` to the embedded rpcdaemon as its kvcache; `CacheView.Get` → `SharedDomains.AsGetter` → `getLatestMetered`, which consults and fills the shared cache). The PR does not add the sharing; it adds the missing ordering. 1. **Reader** begins a read-only tx; its frontier is N. 2. **Exec** at txNum M >= N deletes key K and commits; the post-commit apply empties K's slot. 3. **Reader** reads K: cache miss → reads its own older tx → gets the pre-delete value → its fill re-inserts it (the slot is empty, so `PutIfAbsent` lands). 4. Anyone's next `GetLatest(K)` serves the deleted value from the cache. Each step is consistent inside its own tx. The bug is the unordered hand-off between two txs through the shared cache — no per-tx consistency property can prevent it. ```mermaid sequenceDiagram participant Reader as Reader with frontier N participant Cache as shared StateCache participant Exec as Exec commit Exec->>Cache: apply at txNum M >= N. delete K, appliedEnd = M+1 Reader->>Cache: GetLatest(K). miss, the slot is empty Reader->>Reader: falls back to its own older tx. pre-delete value Reader->>Cache: fill(K, pre-delete value) Note over Cache: main. PutIfAbsent lands, K is resurrected Note over Cache: this PR. rejected, frontier N < appliedEnd ``` `StateCache` has no data methods (like `db`). Access goes through two handles (like `tx`): - **`ReadView`** — reads plus admission-gated fills, bound to one tx's read view; it must not outlive the tx. `SharedDomains.AsGetter*(tx)` builds one per getter; the plain `GetLatest` wrappers use a frontier-less view and bind a frontier only on the miss path. The view resolves its frontier internally, so a fill can no longer be paired with another tx's frontier. - **`Applier`** — `Apply` / `Unwind` / `Clear`. Used by the authoritative `SharedDomains` commit/unwind paths (every SD holds a handle; the operations serialize under the admission lock). The cache tracks, per domain, the exclusive end of what has been applied (`appliedEnd`), guarded by one `RWMutex`: - `Applier.Apply` (write lock) advances `appliedEnd` and mutates the cache in the same critical section. Applying txNum X records `appliedEnd = X+1`. - `ReadView.Fill` (read lock) rechecks its view's frontier and inserts before releasing: admitted iff `frontier >= appliedEnd`. Fills are put-if-absent, so they never replace a live authoritative entry. Ordering proof, two cases: if the fill takes the lock first, the later apply overwrites or deletes it; if the apply takes the lock first, the stale fill is rejected. Equality is safe because a view whose frontier equals the applied end has seen the apply. Checking freshness only at view creation would not be enough — an apply can land between the check and the fill — which is why admission happens at fill time, under the same lock applies take. `DomainVisibleEnd` reports the exact exclusive frontier of a tx's domain read view: a file covering `[0,N)` reports N; a hot-DB view containing txNum N reports N+1. Views without an exact frontier answer `ok=false` and never fill — remote and history-disabled backends, and dependency-clamped values views (their reads mix ages, and the clamped-away state can appear later without any apply); such reads still work, they just skip cache population. Two memos, one per path: read-only temporal txs memoize the frontier per domain in a tx-local cache, reset only by `ForceReopenUnderlyingFilesTx` (which can only extend it); the writable path memoizes in `SharedDomains`, reset at flush and on `ViewID` change. A compile-time assert guards each memo's bitmask against domain-count growth. Positive entries are stamped with their step-derived txNum bound. A negative entry is stamped with the last txNum its read view included, `max(frontier-1, 0)`: since `Unwind(N)` treats N as the first rolled-back txNum, the negative survives `Unwind(N)` and is invalidated by `Unwind(N-1)`. Admission safety needs one more invariant: a view's frontier never decreases in a process that fills the cache (the DB component is frozen at tx begin; a files reopen only extends it). This is enforced, not just documented: wiring a fill-enabled `StateCache` forbids visibility lowering on that aggregator, and `recalcVisibleFiles` panics on the one transition that breaks admission — a cached state domain's visible end decreasing — whichever entry point causes it. Raising visibility (unaligning a lagging entity) stays allowed, and apply-only caches (`STATE_CACHE_FILLS=false`) skip the forbid: with no fills there is nothing for a lowered frontier to poison. `Apply` physically deletes account, storage and code entries; on the authoritative side absence is represented by absence, with no deletion markers (a marker can be evicted, re-opening the fill window). Reader-side negative entries do exist — they are ordinary admission-gated fills, stamped as below. An account update invalidates the derived address→codeHash mapping; an account deletion also removes the address's code binding; an account-only deletion does not advance the code-domain frontier (that would suppress valid code fills for unrelated contracts). On authoritative code apply, bytes are cloned before hashing so cached code and its codeHash cannot diverge. `SeedAddrCodeHash` accepts only view-sourced account records (a cache-sourced record can lag the latest apply and would carry pre-apply state past the gate); a read error does not seed the zero-hash sentinel. - **Reads are not snapshot-isolated**: a `ReadView` hit can be newer than the view's tx. That is the same direction the Overlay already serves (embedded RPC at `latest` reads exec's newest state on purpose), and a single-version LRU cannot give stable per-view reads without becoming a second kvcache — `node/shards` already provides that model for the remote daemon. The cache's contract, in the forward direction, is monotonicity: content never regresses behind the applied frontier; unwinds invalidate by epoch and floor. - **The reverse direction around unwind** — a view opened before an unwind refilling a value from the discarded fork — is tracked separately in #22463. - Cache hits are unchanged: no frontier query, no admission lock. - The memoized frontier read is `~2.4 ns/op`, 0 allocs (Apple M2 Max); the memo adds `+64 B/op` to a read-only temporal tx (`BenchmarkBeginTemporalRo/WithBlockSnaps`; absolute numbers drift with unrelated changes on `main`). - Getter paths (exec workers, RPC) build their frontier-carrying `ReadView` once per getter; frontier-less views elsewhere (`GetCode`/`GetCodeSize` fast paths) are by-value and allocation-free, and a frontier is bound — one allocation — only on cold fill/seed paths. A cold negative fill through the plain `GetLatest` wrappers is `~0.3 µs/op`, 48 B/op, 2 allocs: one allocation is the fill itself, one is binding the frontier on the miss path — both amortize against the backing read they follow (the no-cache baseline of the same read is `~0.1 µs`). Production-path reproductions through embedded RPC (`execmodule.Cache.View` over a published `SharedDomains` overlay and a real temporal DB): account, storage and code resurrection from an old RPC read view, and code-of-a-deleted-account refill (pinning both the SD-level paired deletion and the cache-level accounts-frontier check on code fills). Plus: admission at the exact frontier boundary and a 20,000-round concurrent apply/delete vs fill interleaving; physical deletion and derived code-mapping invalidation; account-only deletion not suppressing unrelated code fills; the address→codeHash mapping seeding only from view-sourced records; exact file/DB frontier calculation and memo re-derivation after files-view reopen and after flush; read-ahead freshness, no-exact-frontier backends never filling; negatives surviving `Unwind(N)` but not `Unwind(N-1)`; the admission frontier surviving `Clear`; the `STATE_CACHE_FILLS=false` apply-only switch; `USE_STATE_CACHE=false` constructing no cache; node close releasing every cache-budget reservation (a full EngineApiTester lifecycle); and the visibility-lowering assert. All cross-package tests exercise the public `ReadView`/`Applier` API. The three resurrection tests use no API introduced by this PR: ported unchanged to `main`, all three fail with the resurrected values served — they pin the bug, not the refactor. Verified with `go test` (and `-race`) on `execution/cache`, `execution/exec`, `db/state/execctx`, `db/kv/temporal`, full-tree build, benchmarks with `-benchmem`, and repeated clean `make lint` runs. Suggested order: 1. `execution/cache/cache.go` (package doc) and `execution/cache/view.go` — the contract and the API: `Frontier`, `ReadView`, `Applier`. Small files; everything else implements them. 2. `execution/cache/state_cache.go` — admission internals: `appliedEnd`, `fillIfFresh`, `apply`. 3. `db/state/execctx/domain_shared.go` — wiring: getters hold a `ReadView`, commit/unwind hold the `Applier`; `getLatestMetered` shows the full read chain (mem → cache → backing tx → fill). 4. `execution/exec/blocks_read_ahead.go` — the read-ahead fill path. 5. `db/state/execctx/statecache_rpc_integration_test.go` — the resurrection reproduced end-to-end through the embedded-RPC path, and the fix pinned. --------- Co-authored-by: Alexey Sharov --- db/kv/kv_interface.go | 7 + db/kv/remotedb/kv_remote.go | 3 + db/kv/temporal/kv_temporal.go | 72 ++++ db/kv/temporal/kv_temporal_test.go | 127 +++++++ db/state/aggregator.go | 63 +++- db/state/aggregator_align_test.go | 101 +++++ db/state/execctx/codehash_routing_test.go | 101 ++++- db/state/execctx/domain_shared.go | 353 ++++++++++++------ .../execctx/domain_visible_end_memo_test.go | 103 +++++ db/state/execctx/export_test.go | 3 +- db/state/execctx/flush_storage_cache_test.go | 4 +- .../execctx/statecache_readfill_bench_test.go | 45 ++- db/state/execctx/statecache_readfill_test.go | 163 +++++++- .../statecache_rpc_integration_test.go | 342 +++++++++++++++++ db/state/inverted_index.go | 26 +- db/state/inverted_index_test.go | 26 ++ execution/cache/cache.go | 20 +- execution/cache/cache_test.go | 303 +++++++++++++-- execution/cache/code_cache.go | 2 + execution/cache/state_cache.go | 255 +++++++++---- execution/cache/view.go | 197 ++++++++++ .../engineapi/engine_api_cache_budget_test.go | 57 +++ execution/exec/blocks_read_ahead.go | 87 ++--- execution/exec/blocks_read_ahead_test.go | 98 +++-- execution/execmodule/exec_module.go | 76 ++-- .../execmodule/exec_module_internal_test.go | 46 +++ .../execmoduletester/exec_module_tester.go | 12 +- execution/execmodule/forkchoice.go | 7 +- node/eth/backend.go | 15 +- 29 files changed, 2292 insertions(+), 422 deletions(-) create mode 100644 db/state/execctx/domain_visible_end_memo_test.go create mode 100644 db/state/execctx/statecache_rpc_integration_test.go create mode 100644 execution/cache/view.go create mode 100644 execution/engineapi/engine_api_cache_budget_test.go create mode 100644 execution/execmodule/exec_module_internal_test.go diff --git a/db/kv/kv_interface.go b/db/kv/kv_interface.go index 990a35492f4..78c4fdd76ef 100644 --- a/db/kv/kv_interface.go +++ b/db/kv/kv_interface.go @@ -517,7 +517,14 @@ type TemporalDebugTx interface { // HistoryStartFrom return the earliest known txnum in history of a given domain HistoryStartFrom(domainName Domain) uint64 + // DomainProgress is a best-effort progress number for reporting: it mixes + // an exclusive files end with an inclusive DB txNum (so it is ±1 depending + // on which side wins) and falls back to step granularity when history is + // disabled. For an exact bound use DomainVisibleEnd. DomainProgress(domain Domain) (txNum uint64) + // DomainVisibleEnd returns the exact exclusive txNum bound of the tx's + // domain read view. ok is false when the backend cannot provide an exact bound. + DomainVisibleEnd(domain Domain) (visibleEnd uint64, ok bool) IIProgress(name InvertedIdx) (txNum uint64) StepSize() uint64 // Retire retires frozen history files entirely below their diff --git a/db/kv/remotedb/kv_remote.go b/db/kv/remotedb/kv_remote.go index 4022845c4f1..1e7260a7fbd 100644 --- a/db/kv/remotedb/kv_remote.go +++ b/db/kv/remotedb/kv_remote.go @@ -255,6 +255,9 @@ func (tx *tx) Retire(ctx context.Context, cutoffs kv.RetireCutoffs) (int, error) } func (tx *tx) DomainFiles(domain ...kv.Domain) kv.VisibleFiles { panic("not implemented") } func (tx *tx) DomainProgress(domain kv.Domain) uint64 { panic("not implemented") } +func (tx *tx) DomainVisibleEnd(domain kv.Domain) (uint64, bool) { + return 0, false +} func (tx *tx) GetLatestFromDB(domain kv.Domain, k []byte) (v []byte, step kv.Step, found bool, err error) { panic("not implemented") } diff --git a/db/kv/temporal/kv_temporal.go b/db/kv/temporal/kv_temporal.go index df1f198f729..f6c43df0aeb 100644 --- a/db/kv/temporal/kv_temporal.go +++ b/db/kv/temporal/kv_temporal.go @@ -21,6 +21,7 @@ import ( "errors" "fmt" "sync" + "sync/atomic" "time" "github.com/erigontech/erigon/db/datadir" @@ -267,6 +268,7 @@ type tx struct { type Tx struct { kv.Tx tx + visibleEnds domainVisibleEnds } type RwTx struct { @@ -274,6 +276,63 @@ type RwTx struct { tx } +type domainVisibleEnds struct { + // ends is atomic so a lock-free read can overlap a reset-and-reload of + // the same slot without a data race. A torn read (state bit from one + // generation, end from another) can only be stale-low, which merely + // over-rejects fills: a view's frontier never decreases in a process that + // fills a cache — the DB component is frozen at tx begin, and a files + // reopen only extends it, an invariant the aggregator enforces once a + // fill-enabled cache is wired over it (ForbidVisibilityLowering). + ends [kv.DomainLen]atomic.Uint64 + mu sync.Mutex + state atomic.Uint32 +} + +// state packs two bits per domain into one word so a single atomic load +// returns a consistent (loaded, ok) pair: loadedBit says ends[domain] is +// memoized, okBit is the memoized ok answer of DomainVisibleEnd. The array +// size asserts at compile time that both halves fit in uint32. +var _ [32 - 2*int(kv.DomainLen)]struct{} + +func visibleEndBits(domain kv.Domain) (loadedBit, okBit uint32) { + loadedBit = uint32(1) << uint32(domain) + return loadedBit, loadedBit << uint32(kv.DomainLen) +} + +func (v *domainVisibleEnds) get(tx *Tx, domain kv.Domain) (uint64, bool) { + loadedBit, okBit := visibleEndBits(domain) + state := v.state.Load() + if state&loadedBit != 0 { + return v.ends[domain].Load(), state&okBit != 0 + } + return v.load(tx, domain, loadedBit, okBit) +} + +func (v *domainVisibleEnds) load(tx *Tx, domain kv.Domain, loadedBit, okBit uint32) (uint64, bool) { + v.mu.Lock() + defer v.mu.Unlock() + + state := v.state.Load() + if state&loadedBit == 0 { + end, ok := tx.aggtx.DomainVisibleEnd(domain, tx.Tx) + v.ends[domain].Store(end) + state |= loadedBit + if ok { + state |= okBit + } + v.state.Store(state) + } + return v.ends[domain].Load(), state&okBit != 0 +} + +// reset takes mu so an in-flight load can't re-store pre-reset bits. +func (v *domainVisibleEnds) reset() { + v.mu.Lock() + defer v.mu.Unlock() + v.state.Store(0) +} + func (tx *tx) ForceReopenUnderlyingFilesTx() { if tx.blocktx != nil { tx.blocktx.Close() @@ -284,6 +343,13 @@ func (tx *tx) ForceReopenUnderlyingFilesTx() { } tx.aggtx = tx.Agg().BeginFilesRo() } + +// ForceReopenUnderlyingFilesTx swaps in a fresh files view, which can extend +// the visible frontier — drop the memoized ends so they are re-derived. +func (tx *Tx) ForceReopenUnderlyingFilesTx() { + tx.tx.ForceReopenUnderlyingFilesTx() + tx.visibleEnds.reset() +} func (tx *tx) FreezeInfo() kv.FreezeInfo { return tx.aggtx } func (tx *tx) AggTx() any { return tx.aggtx } @@ -724,6 +790,12 @@ func (tx *Tx) DomainProgress(domain kv.Domain) uint64 { func (tx *RwTx) DomainProgress(domain kv.Domain) uint64 { return tx.aggtx.DomainProgress(domain, tx.RwTx) } +func (tx *Tx) DomainVisibleEnd(domain kv.Domain) (uint64, bool) { + return tx.visibleEnds.get(tx, domain) +} +func (tx *RwTx) DomainVisibleEnd(domain kv.Domain) (uint64, bool) { + return tx.aggtx.DomainVisibleEnd(domain, tx.RwTx) +} func (tx *Tx) IIProgress(domain kv.InvertedIdx) uint64 { return tx.aggtx.IIProgress(domain, tx.Tx) } diff --git a/db/kv/temporal/kv_temporal_test.go b/db/kv/temporal/kv_temporal_test.go index 7d28aacea2d..ab7bb830d07 100644 --- a/db/kv/temporal/kv_temporal_test.go +++ b/db/kv/temporal/kv_temporal_test.go @@ -2,6 +2,7 @@ package temporal import ( "encoding/binary" + "sync" "testing" "time" @@ -257,6 +258,132 @@ func TestTemporalTx_PinsBlockFilesView(t *testing.T) { require.NotNil(t, roTx2.(*Tx).blocktx) } +// DomainVisibleEnd's memo serves repeat readers lock-free while first loads +// run under the memo mutex. Fresh txs each round make the two paths +// interleave across goroutines; results must stay stable (run with -race). +func TestTemporalTx_DomainVisibleEndConcurrent(t *testing.T) { + t.Parallel() + ctx := t.Context() + + mdbxDb := memdb.NewTestDB(t, dbcfg.ChainDB) + dirs := datadir.New(t.TempDir()) + agg := state.NewTest(dirs).StepSize(1).MustOpen(ctx, mdbxDb) + defer agg.Close() + temporalDb, err := New(mdbxDb, agg, nil) + require.NoError(t, err) + defer temporalDb.Close() + + acc := common.HexToAddress("0x1234567890123456789012345678901234567890") + slot := common.HexToHash("0x0000000000000000000000000000000000000000000000000000000000000001") + storageK := append(append([]byte{}, acc[:]...), slot[:]...) + + rwTtx, err := temporalDb.BeginTemporalRw(ctx) + require.NoError(t, err) + defer rwTtx.Rollback() + sd, err := execctx.NewSharedDomains(ctx, rwTtx, log.Root()) + require.NoError(t, err) + defer sd.Close() + require.NoError(t, sd.DomainPut(kv.StorageDomain, rwTtx, storageK, []byte{1}, 1, nil)) + require.NoError(t, sd.Flush(ctx, rwTtx)) + require.NoError(t, rwTtx.Commit()) + + var expectedEnd [kv.DomainLen]uint64 + var expectedOk [kv.DomainLen]bool + baseTtx, err := temporalDb.BeginTemporalRo(ctx) + require.NoError(t, err) + defer baseTtx.Rollback() + for d := range kv.DomainLen { + expectedEnd[d], expectedOk[d] = baseTtx.Debug().DomainVisibleEnd(d) + } + baseTtx.Rollback() + require.Equal(t, uint64(2), expectedEnd[kv.StorageDomain]) + require.True(t, expectedOk[kv.StorageDomain]) + + for range 25 { + require.NoError(t, temporalDb.ViewTemporal(ctx, func(roTtx kv.TemporalTx) error { + var wg sync.WaitGroup + for range 8 { + wg.Go(func() { + for range 4 { + for d := range kv.DomainLen { + end, ok := roTtx.Debug().DomainVisibleEnd(d) + if end != expectedEnd[d] || ok != expectedOk[d] { + t.Errorf("domain %v: got (%d, %t), want (%d, %t)", d, end, ok, expectedEnd[d], expectedOk[d]) + } + } + } + }) + } + wg.Wait() + return nil + })) + } +} + +// A read-only temporal tx memoizes DomainVisibleEnd, while +// ForceReopenUnderlyingFilesTx swaps in a fresh files view that can extend the +// frontier — the memo must be re-derived after the swap. +func TestTemporalTx_ForceReopenRefreshesDomainVisibleEnd(t *testing.T) { + t.Parallel() + ctx := t.Context() + + mdbxDb := memdb.NewTestDB(t, dbcfg.ChainDB) + dirs := datadir.New(t.TempDir()) + agg := state.NewTest(dirs).StepSize(1).MustOpen(ctx, mdbxDb) + defer agg.Close() + temporalDb, err := New(mdbxDb, agg, nil) + require.NoError(t, err) + defer temporalDb.Close() + + acc := common.HexToAddress("0x1234567890123456789012345678901234567890") + slot := common.HexToHash("0x0000000000000000000000000000000000000000000000000000000000000001") + storageK := append(append([]byte{}, acc[:]...), slot[:]...) + + rwTtx1, err := temporalDb.BeginTemporalRw(ctx) + require.NoError(t, err) + defer rwTtx1.Rollback() + sd, err := execctx.NewSharedDomains(ctx, rwTtx1, log.Root()) + require.NoError(t, err) + defer sd.Close() + require.NoError(t, sd.DomainPut(kv.StorageDomain, rwTtx1, storageK, []byte{1}, 1, nil)) + require.NoError(t, sd.Flush(ctx, rwTtx1)) + require.NoError(t, rwTtx1.Commit()) + + roTtx, err := temporalDb.BeginTemporalRo(ctx) + require.NoError(t, err) + defer roTtx.Rollback() + end, ok := roTtx.Debug().DomainVisibleEnd(kv.StorageDomain) + require.True(t, ok) + require.Equal(t, uint64(2), end) + + // Write past the RO tx's MVCC view and move the data into files, which are + // visible regardless of the DB read view. + for txNum := uint64(2); txNum <= 3; txNum++ { + rwTtx, err := temporalDb.BeginTemporalRw(ctx) + require.NoError(t, err) + defer rwTtx.Rollback() + require.NoError(t, sd.DomainPut(kv.StorageDomain, rwTtx, storageK, []byte{byte(txNum)}, txNum, nil)) + require.NoError(t, sd.Flush(ctx, rwTtx)) + require.NoError(t, rwTtx.Commit()) + } + require.NoError(t, agg.BuildFiles(3)) + + freshRoTtx, err := temporalDb.BeginTemporalRo(ctx) + require.NoError(t, err) + defer freshRoTtx.Rollback() + filesEnd := freshRoTtx.Debug().TxNumsInFiles(kv.StorageDomain) + require.Greater(t, filesEnd, uint64(2), "the new files must extend past the memoized frontier") + + end, ok = roTtx.Debug().DomainVisibleEnd(kv.StorageDomain) + require.True(t, ok) + require.Equal(t, uint64(2), end, "the pinned files view cannot see the new files before reopen") + + roTtx.(*Tx).ForceReopenUnderlyingFilesTx() + end, ok = roTtx.Debug().DomainVisibleEnd(kv.StorageDomain) + require.True(t, ok) + require.Equal(t, filesEnd, end, "the frontier must reflect the fresh files view after reopen") +} + func TestTemporalTx_RangeAsOf_StorageDomain(t *testing.T) { t.Parallel() ctx := t.Context() diff --git a/db/state/aggregator.go b/db/state/aggregator.go index 9a1711be428..ff4edbdec86 100644 --- a/db/state/aggregator.go +++ b/db/state/aggregator.go @@ -88,9 +88,15 @@ type Aggregator struct { oldestVisible *aggregatorVisible // unaligned entities are left out of the shared visible-file ceiling while tooling // regenerates them. Guarded by dirtyFilesLock. - unalignedDomain [kv.DomainLen]bool - unalignedIdx [kv.StandaloneIdxLen]bool - snapshotBuildSema *semaphore.Weighted + unalignedDomain [kv.DomainLen]bool + unalignedIdx [kv.StandaloneIdxLen]bool + // visibilityLoweringForbidden: a fill-enabled StateCache is wired over + // this aggregator, and its fill admission relies on view frontiers never + // decreasing. recalcVisibleFiles refuses to lower the cached state + // domains' visible ends while set; Close clears it (shutdown is not a + // fill window). + visibilityLoweringForbidden atomic.Bool + snapshotBuildSema *semaphore.Weighted disableHistory bool branchCacheDisabled bool @@ -542,6 +548,17 @@ func (a *Aggregator) UnalignIdx(name kv.InvertedIdx) (realign func()) { return func() {} } +// ForbidVisibilityLowering marks this aggregator as backing a fill-enabled +// StateCache: from then on recalcVisibleFiles panics instead of lowering a +// cached state domain's visible end, whichever entry point caused it. +// Serialized with recalcVisibleFiles via dirtyFilesLock so "from then on" +// holds against a recalculation already in flight. +func (a *Aggregator) ForbidVisibilityLowering() { + a.dirtyFilesLock.Lock() + defer a.dirtyFilesLock.Unlock() + a.visibilityLoweringForbidden.Store(true) +} + func (a *Aggregator) setUnalignedDomain(d kv.Domain, v bool) { a.dirtyFilesLock.Lock() defer a.dirtyFilesLock.Unlock() @@ -683,6 +700,9 @@ func (a *Aggregator) WaitForFiles() { } func (a *Aggregator) Close() { + a.dirtyFilesLock.Lock() + a.visibilityLoweringForbidden.Store(false) // shutdown is not a fill window + a.dirtyFilesLock.Unlock() a.WaitForFiles() if !a.background.BeginClose() { // idempotent: safe to call Close multiple times return @@ -1852,6 +1872,28 @@ func (a *Aggregator) recalcVisibleFiles(retired retiredFiles) { } next.minimaxTxNum = next.stateMinimaxTxNum() + if a.visibilityLoweringForbidden.Load() { + prev := a.visible.Load() + for _, d := range []kv.Domain{kv.AccountsDomain, kv.StorageDomain, kv.CodeDomain} { + if prev.d[d] == nil || next.d[d] == nil { + continue + } + prevEnd := visibleFiles(prev.d[d].files).EndTxNum() + nextEnd := visibleFiles(next.d[d].files).EndTxNum() + if nextEnd < prevEnd { + panic(fmt.Sprintf("assert: %s visible end lowered %d -> %d while a fill-enabled StateCache is wired — fill admission relies on view frontiers never decreasing", d, prevEnd, nextEnd)) + } + if prev.dhii[d] == nil || next.dhii[d] == nil { + continue + } + prevII := prev.dhii[d].files.EndTxNum() + nextII := next.dhii[d].files.EndTxNum() + if nextII < prevII { + panic(fmt.Sprintf("assert: %s history-II visible end lowered %d -> %d while a fill-enabled StateCache is wired — DomainVisibleEnd derives view frontiers from it", d, prevII, nextII)) + } + } + } + old := a.visible.Load() old.retired = retired old.next = next @@ -2594,6 +2636,21 @@ func (at *AggregatorRoTx) DomainProgress(name kv.Domain, tx kv.Tx) uint64 { } return at.d[name].ht.iit.Progress(tx) } +func (at *AggregatorRoTx) DomainVisibleEnd(name kv.Domain, tx kv.Tx) (uint64, bool) { + d := at.d[name] + if d.d.HistoryDisabled { + return 0, false + } + // A dependency checker can clamp the values view below the history-II end. + // Such a view has no exact frontier: reads mix fresh DB-resident keys with + // older file values for the gap, and raising the dependent file's + // visibility later reveals state without any cache apply — a fill made + // during the clamp would never be invalidated. + if d.files.EndTxNum() < d.ht.iit.files.EndTxNum() { + return 0, false + } + return d.ht.iit.visibleEnd(tx), true +} func (at *AggregatorRoTx) IIProgress(name kv.InvertedIdx, tx kv.Tx) uint64 { return at.searchII(name).Progress(tx) } diff --git a/db/state/aggregator_align_test.go b/db/state/aggregator_align_test.go index 3df1e407143..2d4be50b798 100644 --- a/db/state/aggregator_align_test.go +++ b/db/state/aggregator_align_test.go @@ -17,6 +17,7 @@ package state import ( + "context" "testing" "github.com/stretchr/testify/require" @@ -159,3 +160,103 @@ func TestUnalign_RejectsStateDomain(t *testing.T) { require.Panics(t, func() { agg.Unalign(d) }, "domain %s", d) } } + +// Fill admission relies on view frontiers never decreasing. Raising +// visibility (unaligning a lagging entity) is allowed even on a forbidden +// aggregator; the transition that lowers a cached state domain's visible end +// (here: realigning while receipt still lags, which drops the shared ceiling) +// must panic, whichever entry point caused it. +func TestVisibilityLowering_ForbiddenAggregatorPanicsOnLoweringOnly(t *testing.T) { + t.Parallel() + _, agg := testDbAndAggregatorv3(t, alignStepSize) + + generateStateFiles(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + generateCommitmentFile(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + generateStandaloneIIFiles(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + generateDomainFiles(t, "receipt", agg.Dirs(), []testFileRange{{0, 1}}) + require.NoError(t, agg.OpenFolder()) + + agg.ForbidVisibilityLowering() + realign := agg.Unalign(kv.ReceiptDomain) // raises the ceiling: allowed + require.Panics(t, func() { realign() }, "realigning a still-lagging receipt lowers the state domains' ends") +} + +// craftedClampedVisible replaces the current visible bundle with one where +// every state domain's values files end one segment below its history-II end +// — the divergence a dependency checker produces when a dependent file is +// missing. +func craftedClampedVisible(t *testing.T, agg *Aggregator) { + t.Helper() + agg.dirtyFilesLock.Lock() + defer agg.dirtyFilesLock.Unlock() + v := agg.visible.Load() + crafted := &aggregatorVisible{minimaxTxNum: v.minimaxTxNum} + crafted.d, crafted.dh, crafted.dhii, crafted.iis = v.d, v.dh, v.dhii, v.iis + for _, dom := range []kv.Domain{kv.AccountsDomain, kv.StorageDomain, kv.CodeDomain} { + files := v.d[dom].files + require.GreaterOrEqual(t, len(files), 2) + crafted.d[dom] = newDomainVisible(dom, files[:len(files)-1]) + } + v.next = crafted + agg.visible.Store(crafted) +} + +// A dependency-clamped values view has no exact frontier: reads mix fresh +// DB-resident keys with older file values for gap keys, and raising the +// dependent file's visibility later reveals state without any cache apply — +// nothing would invalidate a fill made during the clamp. DomainVisibleEnd +// must report ok=false so such views never fill. +func TestDomainVisibleEnd_ClampedViewHasNoExactFrontier(t *testing.T) { + t.Parallel() + db, agg := testDbAndAggregatorv3(t, alignStepSize) + + generateStateFiles(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + generateCommitmentFile(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + generateStandaloneIIFiles(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + require.NoError(t, agg.OpenFolder()) + + craftedClampedVisible(t, agg) + + at := agg.BeginFilesRo() + defer at.Close() + tx, err := db.BeginRo(context.Background()) + require.NoError(t, err) + defer tx.Rollback() + + _, ok := at.DomainVisibleEnd(kv.AccountsDomain, tx) + require.False(t, ok, "a dependency-clamped values view has no exact frontier") +} + +// The forbid assert must also watch the history-II ends: they are the base of +// what DomainVisibleEnd reports, and with values dependency-clamped below the +// ceiling they can lower while every values end stays put. +func TestVisibilityLowering_GuardsHistoryIIEnd(t *testing.T) { + t.Parallel() + _, agg := testDbAndAggregatorv3(t, alignStepSize) + + generateStateFiles(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + generateCommitmentFile(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + generateStandaloneIIFiles(t, agg.Dirs(), []testFileRange{{0, 1}, {1, 2}}) + require.NoError(t, agg.OpenFolder()) + + craftedClampedVisible(t, agg) + agg.ForbidVisibilityLowering() + + // Drop the accounts history-II {1,2} segment in memory rather than from + // disk (Windows forbids removing a mapped file): the recalculation lowers + // the ii end while every values end stays put. + agg.dirtyFilesLock.Lock() + defer agg.dirtyFilesLock.Unlock() + dropped := 0 + agg.d[kv.AccountsDomain].History.InvertedIndex.dirtyFiles.CloseIf(func(item *FilesItem) bool { + if item.endTxNum == 2*alignStepSize { + dropped++ + return true + } + return false + }) + require.Equal(t, 1, dropped) + + require.Panics(t, func() { agg.recalcVisibleFiles(nil) }, + "lowering a history-II end while values ends stay put must trip the forbid assert") +} diff --git a/db/state/execctx/codehash_routing_test.go b/db/state/execctx/codehash_routing_test.go index c11d892884d..211e3cadaea 100644 --- a/db/state/execctx/codehash_routing_test.go +++ b/db/state/execctx/codehash_routing_test.go @@ -45,7 +45,7 @@ func TestCodeHashForAddr_InBatchAccountWinsOverStaleLRU(t *testing.T) { } var staleArr [32]byte copy(staleArr[:], stale[:]) - sc.PutAddrCodeHash(addr[:], staleArr, 0) + sc.View(frontierAt(0)).SeedAddrCodeHash(addr[:], staleArr, 0) t.Run("empty in-batch account wins (codeHash-no-code repro)", func(t *testing.T) { acc := accounts.Account{Nonce: 7, CodeHash: accounts.EmptyCodeHash} @@ -68,3 +68,102 @@ func TestCodeHashForAddr_InBatchAccountWinsOverStaleLRU(t *testing.T) { require.NotEqual(t, stale[:], got) }) } + +// The addr→codeHash admission gate vouches for the tx read view's frontier, but +// resolve() may serve the account record from the shared accounts cache, which +// lags a just-committed flush until the apply loop reaches the key. A +// cache-sourced record must therefore never seed the mapping — an apply +// interleaved between the read and the fill would leave a mapping derived from +// the pre-apply record. +func TestCodeHashForAddr_CacheSourcedRecordDoesNotSeedMapping(t *testing.T) { + t.Parallel() + + ctx := t.Context() + db := newTestDb(t, 16) + sc := cache.NewDefaultStateCache() + t.Cleanup(sc.Close) + + var addr common.Address + addr[0] = 0xab + var codeHash common.Hash + for i := range codeHash { + codeHash[i] = 0x11 + } + acc := accounts.Account{Nonce: 7, CodeHash: accounts.InternCodeHash(codeHash)} + + seedTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer seedTx.Rollback() + seedSD, err := execctx.NewSharedDomains(ctx, seedTx, log.New()) + require.NoError(t, err) + defer seedSD.Close() + seedSD.SetStateCacheForTest(sc) + seedSD.SetTxNum(10) + require.NoError(t, seedSD.DomainPut(kv.AccountsDomain, seedTx, addr[:], accounts.SerialiseV3(&acc), 10, nil)) + require.NoError(t, seedSD.Commit(ctx, seedTx)) + seedSD.Close() + + _, ok := sc.View(nil).Get(kv.AccountsDomain, addr[:]) + require.True(t, ok, "the committed record must be served by the accounts cache") + _, ok = sc.View(nil).GetAddrCodeHash(addr[:]) + require.False(t, ok, "the post-commit apply must leave the derived mapping empty") + + roTx, err := db.BeginTemporalRo(ctx) + require.NoError(t, err) + defer roTx.Rollback() + sd, err := execctx.NewSharedDomains(ctx, roTx, log.New()) + require.NoError(t, err) + defer sd.Close() + sd.SetStateCacheForTest(sc) + + got := sd.CodeHashForAddr(roTx, addr[:], 20) + require.Equal(t, codeHash[:], got) + _, ok = sc.View(nil).GetAddrCodeHash(addr[:]) + require.False(t, ok, "a cache-sourced account record must not seed the addr→codeHash mapping") +} + +// A record read from the tx's read view (accounts-cache miss) is exactly what +// the admission gate vouches for, so it still seeds the mapping. +func TestCodeHashForAddr_ViewSourcedRecordSeedsMapping(t *testing.T) { + t.Parallel() + + ctx := t.Context() + db := newTestDb(t, 16) + + var addr common.Address + addr[0] = 0xcd + var codeHash common.Hash + for i := range codeHash { + codeHash[i] = 0x22 + } + acc := accounts.Account{Nonce: 3, CodeHash: accounts.InternCodeHash(codeHash)} + + // Seed without a state cache so the record lands in the DB only. + seedTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer seedTx.Rollback() + seedSD, err := execctx.NewSharedDomains(ctx, seedTx, log.New()) + require.NoError(t, err) + defer seedSD.Close() + seedSD.SetTxNum(10) + require.NoError(t, seedSD.DomainPut(kv.AccountsDomain, seedTx, addr[:], accounts.SerialiseV3(&acc), 10, nil)) + require.NoError(t, seedSD.Commit(ctx, seedTx)) + seedSD.Close() + + sc := cache.NewDefaultStateCache() + t.Cleanup(sc.Close) + + roTx, err := db.BeginTemporalRo(ctx) + require.NoError(t, err) + defer roTx.Rollback() + sd, err := execctx.NewSharedDomains(ctx, roTx, log.New()) + require.NoError(t, err) + defer sd.Close() + sd.SetStateCacheForTest(sc) + + got := sd.CodeHashForAddr(roTx, addr[:], 20) + require.Equal(t, codeHash[:], got) + h, ok := sc.View(nil).GetAddrCodeHash(addr[:]) + require.True(t, ok, "a view-sourced record must seed the mapping") + require.Equal(t, [32]byte(codeHash), h) +} diff --git a/db/state/execctx/domain_shared.go b/db/state/execctx/domain_shared.go index e777d3063b2..b670d5460e5 100644 --- a/db/state/execctx/domain_shared.go +++ b/db/state/execctx/domain_shared.go @@ -81,6 +81,111 @@ type accHolder interface { SetChangesetAccumulator(acc *changeset.StateChangeSet) } +// domainVisibleEndMemo caches DomainVisibleEnd per domain for one view at a time. +// Its sequence counter keeps lock-free reads coherent across view changes. +type domainVisibleEndMemo struct { + ends [kv.DomainLen]atomic.Uint64 + mu sync.Mutex + seq atomic.Uint64 + viewID atomic.Uint64 + state atomic.Uint32 +} + +// state packs two bits per domain into one word so a single atomic load +// returns a consistent (loaded, ok) pair: loadedBit says ends[domain] is +// memoized, okBit is the memoized ok answer of DomainVisibleEnd. The array +// size asserts at compile time that both halves fit in uint32. +var _ [32 - 2*int(kv.DomainLen)]struct{} + +func visibleEndBits(domain kv.Domain) (loadedBit, okBit uint32) { + loadedBit = uint32(1) << uint32(domain) + return loadedBit, loadedBit << uint32(kv.DomainLen) +} + +func (m *domainVisibleEndMemo) get(tx kv.TemporalTx, domain kv.Domain) (uint64, bool) { + viewID := tx.ViewID() + loadedBit, okBit := visibleEndBits(domain) + seq := m.seq.Load() + if seq&1 == 0 && m.viewID.Load() == viewID { + if state := m.state.Load(); state&loadedBit != 0 { + end := m.ends[domain].Load() + if m.seq.Load() == seq { + return end, state&okBit != 0 + } + } + } + return m.load(tx, domain, viewID, loadedBit, okBit) +} + +func (m *domainVisibleEndMemo) load(tx kv.TemporalTx, domain kv.Domain, viewID uint64, loadedBit, okBit uint32) (uint64, bool) { + m.mu.Lock() + defer m.mu.Unlock() + + cachedViewID := m.viewID.Load() + state := m.state.Load() + if cachedViewID == viewID && state&loadedBit != 0 { + return m.ends[domain].Load(), state&okBit != 0 + } + + m.seq.Add(1) + defer m.seq.Add(1) + + if cachedViewID != viewID { + state = 0 + m.viewID.Store(viewID) + } + end, ok := tx.Debug().DomainVisibleEnd(domain) + m.ends[domain].Store(end) + state |= loadedBit + if ok { + state |= okBit + } + m.state.Store(state) + return end, ok +} + +// reset takes mu so an in-flight load can't re-store pre-reset bits. +func (m *domainVisibleEndMemo) reset() { + m.mu.Lock() + m.seq.Add(1) + m.state.Store(0) + m.seq.Add(1) + m.mu.Unlock() +} + +func (sd *SharedDomains) domainVisibleEnd(tx kv.TemporalTx, domain kv.Domain) (uint64, bool) { + if _, ok := tx.(kv.TemporalRwTx); ok { + return sd.visibleEnds.get(tx, domain) + } + return tx.Debug().DomainVisibleEnd(domain) +} + +// sdFrontier adapts one (SharedDomains, tx) pair to cache.Frontier: writable +// txs go through the SD's flush-coherent memo, read-only txs use their own +// tx-local memo. +type sdFrontier struct { + sd *SharedDomains + tx kv.TemporalTx +} + +func (f sdFrontier) DomainVisibleEnd(domain kv.Domain) (uint64, bool) { + return f.sd.domainVisibleEnd(f.tx, domain) +} + +// cacheViewFor binds the shared state cache to tx's read view. Boxing the +// frontier allocates, so per-read paths hold the view in their getter instead +// of rebuilding it per call. +func (sd *SharedDomains) cacheViewFor(tx kv.TemporalTx) cache.ReadView { + if sd.stateCache == nil { + return cache.ReadView{} + } + return sd.stateCache.View(sdFrontier{sd: sd, tx: tx}) +} + +// cacheReader is a frontier-less view: admission-gated fills are disabled, +// content-addressed fills still work. Safe on a nil cache. +func (sd *SharedDomains) cacheReader() cache.ReadView { return sd.stateCache.View(nil) } + func IsDomainAheadOfBlocks(ctx context.Context, tx kv.TemporalRwTx, logger log.Logger) bool { doms, err := NewSharedDomains(ctx, tx, logger) if doms != nil { @@ -123,8 +228,14 @@ type SharedDomains struct { // to read from the FCU's published SD without writing to it. parent *SharedDomains - // stateCache is an optional cache for state data (accounts, storage, code) - stateCache *cache.StateCache + // stateCache is an optional cache for state data (accounts, storage, code); + // cacheApplier is its authoritative writer handle (commit/unwind only). + stateCache *cache.StateCache + cacheApplier cache.Applier + + // Backing frontiers stay fixed while writes and staged unwinds remain in + // mem; both reach the transaction during flush, which resets the memo. + visibleEnds domainVisibleEndMemo // codeStore is the optional two-tier (in-mem + MDBX) codehash-keyed code // cache, reached via temporalGetter so an addr-keyed reader can serve a @@ -410,6 +521,9 @@ func (sd *SharedDomains) domainPutNoLock(domain kv.Domain, roTx kv.TemporalTx, k type temporalGetter struct { sd *SharedDomains tx kv.TemporalTx + // view binds the shared state cache to tx's read view once per getter, + // keeping the per-read path allocation-free. + view cache.ReadView // m is an optional per-worker metrics instance to record reads into. nil // (the AsGetter default) collects nothing — there is no process-wide // accumulator, since AsGetter is used by many concurrent goroutines (RPC, @@ -419,7 +533,7 @@ type temporalGetter struct { } func (gt *temporalGetter) GetLatest(name kv.Domain, k []byte) (v []byte, step kv.Step, err error) { - return gt.sd.getLatestMetered(name, gt.tx, k, gt.m) + return gt.sd.getLatestMetered(name, gt.tx, k, gt.m, gt.view) } // GetLatestContext is the context-aware read: it records into the per-worker, @@ -429,7 +543,7 @@ func (gt *temporalGetter) GetLatest(name kv.Domain, k []byte) (v []byte, step kv // lock. Optional method — callers type-assert for it (mirrors the existing // AggregatorRoTx.MeteredGetLatest pattern). func (gt *temporalGetter) GetLatestContext(ctx context.Context, name kv.Domain, k []byte) (v []byte, step kv.Step, err error) { - return gt.sd.getLatestMetered(name, gt.tx, k, kvmetrics.MetricsFromContext(ctx)) + return gt.sd.getLatestMetered(name, gt.tx, k, kvmetrics.MetricsFromContext(ctx), gt.view) } // GetCodeSize returns the length of the code at addr without loading the @@ -441,7 +555,7 @@ func (gt *temporalGetter) GetLatestContext(ctx context.Context, name kv.Domain, // so the existing kv.TemporalGetter interface is unchanged. txNum is the // caller's read txNum, used to stamp any cache entry it populates. func (gt *temporalGetter) GetCodeSize(addr []byte, txNum uint64) (int, bool, error) { - return gt.sd.GetCodeSize(gt.tx, addr, txNum) + return gt.sd.getCodeSize(gt.tx, gt.view, addr, txNum) } // GetCode returns contract code via the content-addressed fast path (see @@ -451,7 +565,7 @@ func (gt *temporalGetter) GetCodeSize(addr []byte, txNum uint64) (int, bool, err // (they resolve prevVal through GetLatest, which is addr-keyed). txNum is the // caller's read txNum, used to stamp any cache entry it populates. func (gt *temporalGetter) GetCode(addr []byte, txNum uint64) ([]byte, bool, error) { - return gt.sd.GetCode(gt.tx, addr, txNum) + return gt.sd.getCode(gt.tx, gt.view, addr, txNum) } func (gt *temporalGetter) HasPrefix(name kv.Domain, prefix []byte) (firstKey []byte, firstVal []byte, ok bool, err error) { @@ -463,13 +577,13 @@ func (gt *temporalGetter) StepsInFiles(entitySet ...kv.Domain) kv.Step { } func (sd *SharedDomains) AsGetter(tx kv.TemporalTx) kv.TemporalGetter { - return &temporalGetter{sd: sd, tx: tx} + return &temporalGetter{sd: sd, tx: tx, view: sd.cacheViewFor(tx)} } // AsGetterNoMetrics is an explicit-intent alias of AsGetter (collects no // metrics), for concurrent callers (RPC/engine) where that is deliberate. func (sd *SharedDomains) AsGetterNoMetrics(tx kv.TemporalTx) kv.TemporalGetter { - return &temporalGetter{sd: sd, tx: tx} + return &temporalGetter{sd: sd, tx: tx, view: sd.cacheViewFor(tx)} } // AsGetterMetered returns a getter that records reads into the caller's own @@ -477,7 +591,7 @@ func (sd *SharedDomains) AsGetterNoMetrics(tx kv.TemporalTx) kv.TemporalGetter { // caller hands it off via MergeMetrics at task end (a lock per task, not per // read) and allocates a fresh instance. Used by parallel-exec workers. func (sd *SharedDomains) AsGetterMetered(tx kv.TemporalTx, m *kvmetrics.DomainMetrics) kv.TemporalGetter { - return &temporalGetter{sd: sd, tx: tx, m: m} + return &temporalGetter{sd: sd, tx: tx, m: m, view: sd.cacheViewFor(tx)} } // MergeMetrics hands a boundary producer's accumulator to BOTH sinks: the @@ -666,12 +780,10 @@ func (sd *SharedDomains) Unwind(txNumUnwindTo uint64, changeset *[kv.DomainLen][ } } // Invalidate the state cache for everything above the unwind point. txNum/epoch - // based and diffset-free (see StateCache.Unwind), so it runs unconditionally — + // based and diffset-free (see Applier.Unwind), so it runs unconditionally — // independent of whether changesets were generated for the unwound range, which // they are not below the reorg window. Matches the domain overlay's maxtx prune. - if sd.stateCache != nil { - sd.stateCache.Unwind(txNumUnwindTo) - } + sd.cacheApplier.Unwind(txNumUnwindTo) } func (sd *SharedDomains) GetMemBatch() kv.TemporalMemBatch { return sd.mem } @@ -728,21 +840,39 @@ func (sd *SharedDomains) GetCommitmentCtx() *commitmentdb.SharedDomainsCommitmen func (sd *SharedDomains) Logger() log.Logger { return sd.logger } -// SetStateCache hands this SD the process-global state cache to manage. -// -// Coherence is structural, enforced by the architecture rather than by -// remembering to call this: app components reach state only through the SD, and -// the SD owns cache population (on flush) and invalidation (sd.Unwind → -// stateCache.Unwind). It is not *additionally* type-enforced only because the -// cache crosses the app/storage boundary — the storage layer can't depend on an -// app-level cache type. The single desync vector is a component that -// deliberately bypasses the SD (raw domain reads + direct cache writes, e.g. -// read-ahead warmup), which then owns its cache coherence explicitly. +// SetStateCache hands this SD the process-global state cache to manage: +// Commit applies committed updates after a successful DB commit, Unwind +// invalidates them, and the SD's reads populate it through admission-gated +// fills. No-op when USE_STATE_CACHE is off or the cache is nil. func (sd *SharedDomains) SetStateCache(stateCache *cache.StateCache) { if !dbg.UseStateCache || stateCache == nil { return } sd.stateCache = stateCache + sd.cacheApplier = stateCache.Applier() +} + +// GuardAggregatorForCache forbids visibility lowering on db's aggregator when +// sc is a fill-enabled StateCache: fill admission relies on view frontiers +// never decreasing. This is the one place that binds the invariant — call it +// wherever a fill-enabled cache is wired over a DB. Duck-typed so the storage +// layer need not know the cache type (and vice versa) — but load-bearing, so +// a db that cannot produce its aggregator fails loudly instead of silently +// dropping the guard. A nil or apply-only cache needs no guard. +func GuardAggregatorForCache(db any, sc *cache.StateCache) { + if sc == nil || !sc.FillsEnabled() { + return + } + h, ok := db.(interface{ Agg() any }) + if !ok { + panic(fmt.Sprintf("assert: fill-enabled StateCache wired over %T, which cannot produce its aggregator — the visibility-lowering guard would be silently dropped", db)) + } + agg := h.Agg() + f, ok := agg.(interface{ ForbidVisibilityLowering() }) + if !ok { + panic(fmt.Sprintf("assert: aggregator %T lacks ForbidVisibilityLowering — the visibility-lowering guard would be silently dropped", agg)) + } + f.ForbidVisibilityLowering() } // SetCodeStore sets the persistent codehash-keyed code cache. @@ -841,11 +971,12 @@ func (sd *SharedDomains) Close() { sd.sdCtx = nil } -// The state caches (the account/storage StateCache and the commitment -// BranchCache) are an internal implementation detail of SharedDomains. No -// external entity accesses or mutates them directly — callers drive state -// through Flush / Commit / GetLatest / DomainPut, and the cache lifecycle -// (population, invalidation, commit-gating) is owned entirely here. +// SharedDomains owns the cache lifecycle for the account/storage StateCache +// and the commitment BranchCache: population, invalidation and commit-gating +// all happen here, and callers drive state through Flush / Commit / +// GetLatest / DomainPut. The one exception is read-ahead warmup, which fills +// the StateCache directly through its own ReadView, under the same +// admission. // Flush writes the in-memory batch into tx without committing. It deliberately // does NOT touch the caches: plain Flush leaves the commit to the caller (who @@ -855,15 +986,21 @@ func (sd *SharedDomains) Close() { // conservative upper-bound txNum. It is that txNum stamp, not population // timing, that keeps the cache correct: an unwind lowers the floor so every // entry reflecting a now-dead fork is evicted, and mem-first masking means a -// later in-memory write shadows a stale cached read. Callers that flush a tx -// they commit themselves get a cache-safe (cold-but-correct) result; use -// Commit to also keep the cache warm. +// later in-memory write shadows a stale cached read. +// +// An SD with an attached state cache must route every flush through Commit: +// Flush neither applies nor invalidates, so a populated cache would keep +// serving pre-flush values for the flushed keys after the caller's own +// commit — and Commit collects its cache updates only from its own flush, so +// an earlier plain Flush's keys would never be applied. Cache-less callers +// may Flush and commit themselves. func (sd *SharedDomains) Flush(ctx context.Context, tx kv.RwTx) error { defer mxFlushTook.ObserveDuration(time.Now()) return sd.flushMem(ctx, tx) } func (sd *SharedDomains) flushMem(ctx context.Context, tx kv.RwTx, opts ...kv.FlushOption) error { + defer sd.visibleEnds.reset() if sd.sdCtx.HasPendingUpdate() { if ttx, ok := tx.(kv.TemporalTx); ok { if err := sd.FlushPendingUpdates(ctx, ttx); err != nil { @@ -926,7 +1063,9 @@ func (sd *SharedDomains) Commit(ctx context.Context, tx kv.RwTx, validate ...fun // Stash every cache-bound domain tuple during the flush; apply them only // after the commit succeeds. On a failed commit the stash is discarded, so - // no cache is ever advanced past durable MDBX state. + // no cache apply ever runs ahead of durable MDBX state. (Reads through + // this SD between flush and a failed commit can still fill flushed + // values; a failed commit is fatal, so they die with the process.) var pending []cacheUpdate stash := func(domain kv.Domain) kv.FlushOption { return kv.WithFlushCallback(domain, func(k []byte, v []byte, step kv.Step, txNum uint64) { @@ -1035,41 +1174,15 @@ func (sd *SharedDomains) Commit(ctx context.Context, tx kv.RwTx, validate ...fun } for i := range pending { u := &pending[i] - switch u.domain { - case kv.CommitmentDomain: + if u.domain == kv.CommitmentDomain { if len(u.val) == 0 { sd.branchCache.Invalidate(u.key) } else { sd.branchCache.Put(u.key, u.val, uint64(u.step), u.txN) } - case kv.AccountsDomain: - if len(u.val) == 0 { - sd.stateCache.Delete(kv.AccountsDomain, u.key) - sd.stateCache.Delete(kv.CodeDomain, u.key) - sd.stateCache.DeleteAddrCodeHash(u.key) - } else { - sd.stateCache.Put(kv.AccountsDomain, u.key, u.val, u.txN) - sd.stateCache.DeleteAddrCodeHash(u.key) - } - case kv.StorageDomain: - if len(u.val) == 0 { - sd.stateCache.Delete(kv.StorageDomain, u.key) - } else { - sd.stateCache.Put(kv.StorageDomain, u.key, u.val, u.txN) - } - case kv.CodeDomain: - if len(u.val) == 0 { - sd.stateCache.Delete(kv.CodeDomain, u.key) - } else { - // Validated committed code: populate the addr layer AND the - // content-addressed codeHash->code map, keyed by keccak(v) so each - // entry is self-consistent by construction. The read-fill path - // (PutCodeWithHash on a cold GetLatest, below) populates the same - // way — both key on keccak(v), never a separately-read account - // codeHash, so the shared map only ever holds self-consistent entries. - sd.stateCache.PutCodeWithHash(u.key, u.val, crypto.Keccak256(u.val), u.txN) - } + continue } + sd.cacheApplier.Apply(u.domain, u.key, u.val, u.txN) } return nil } @@ -1077,7 +1190,7 @@ func (sd *SharedDomains) Commit(ctx context.Context, tx kv.RwTx, validate ...fun // TemporalDomain satisfaction. Collects no read metrics — see // temporalGetter.GetLatest for why there is no process-wide accumulator. func (sd *SharedDomains) GetLatest(domain kv.Domain, tx kv.TemporalTx, k []byte) (v []byte, step kv.Step, err error) { - return sd.getLatestMetered(domain, tx, k, nil) + return sd.getLatestMetered(domain, tx, k, nil, sd.cacheReader()) } // GetLatestContext is the context-aware read for callers that read on behalf of @@ -1086,7 +1199,7 @@ func (sd *SharedDomains) GetLatest(domain kv.Domain, tx kv.TemporalTx, k []byte) // without any shared accumulator or lock. Mirrors temporalGetter.GetLatestContext // for readers that hold the SD directly (e.g. the committer's asOfStateReader). func (sd *SharedDomains) GetLatestContext(ctx context.Context, domain kv.Domain, tx kv.TemporalTx, k []byte) (v []byte, step kv.Step, err error) { - return sd.getLatestMetered(domain, tx, k, kvmetrics.MetricsFromContext(ctx)) + return sd.getLatestMetered(domain, tx, k, kvmetrics.MetricsFromContext(ctx), sd.cacheReader()) } // servableUnderBound gates a cached entry against an in-flight unwind's @@ -1103,7 +1216,7 @@ func servableUnderBound(cStep, maxStep kv.Step) bool { // per-task/per-worker metrics accumulator (nil disables metrics for the call). // No global metrics lock is taken on this hot path — accumulators are combined // into the shared DomainMetrics later via Merge. -func (sd *SharedDomains) getLatestMetered(domain kv.Domain, tx kv.TemporalTx, k []byte, wm *kvmetrics.DomainMetrics) (v []byte, step kv.Step, err error) { +func (sd *SharedDomains) getLatestMetered(domain kv.Domain, tx kv.TemporalTx, k []byte, wm *kvmetrics.DomainMetrics, view cache.ReadView) (v []byte, step kv.Step, err error) { if tx == nil { return nil, 0, errors.New("sd.GetLatest: unexpected nil tx") } @@ -1157,13 +1270,12 @@ func (sd *SharedDomains) getLatestMetered(domain kv.Domain, tx kv.TemporalTx, k MeteredGetLatestWithTxN(domain kv.Domain, k []byte, tx kv.Tx, maxStep kv.Step, metrics *kvmetrics.DomainMetrics, start time.Time) (v []byte, step kv.Step, txN uint64, ok bool, err error) } - // stateCache holds in-flight values from previous transactions in the same batch - // that haven't been flushed to DB yet. Early return keeps correctness AND performance. + // stateCache holds committed values shared across domain readers. if sd.stateCache != nil { - v, cTxNum, ok := sd.stateCache.GetWithTxNum(domain, k) + v, cTxNum, ok := view.GetWithTxNum(domain, k) // The cache stamps txNums — divide to get the step the entry reflects. - // An empty value is stamped with the domain's progress at fill time, so - // its cStep is progress-derived, not the step of any deletion. + // A negative uses the last txNum included by its read-view frontier, not + // the step of a deletion. cStep := kv.Step(cTxNum / sd.StepSize()) if ok && !servableUnderBound(cStep, maxStep) { ok = false @@ -1236,31 +1348,18 @@ func (sd *SharedDomains) getLatestMetered(domain kv.Domain, tx kv.TemporalTx, k return nil, 0, fmt.Errorf("storage %x read error: %w", k, err) } - // Populate the cache with if-absent semantics: a read-fill never carries - // newer information than a flush-apply, so it must not overwrite one - // (e.g. an embedded-RPC read straddling an FCU commit). Stamp with the - // last txNum of the step the value came from — an upper bound on its - // write txNum — so an unwind below it can't leave the entry stale. A - // negative carries no step; stamp it with the domain's progress at - // observation time so any unwind drops it. - if sd.stateCache != nil { + // View freshness is rechecked while the fill is serialized against + // committed cache updates. + if sd.stateCache != nil && sd.stateCache.Caches(domain) { readTxNum := (uint64(step)+1)*sd.StepSize() - 1 - if domain == kv.CodeDomain { - if len(v) > 0 { - // This SD getter is the single place that populates the code cache - // on a read. Key the content-addressed entry by the code's OWN hash, - // keccak(v) — NEVER a separately-read account codeHash, which under - // parallel exec can be a skewed or cross-account value and would - // poison the shared codeHash→code map for every account sharing - // that hash. - sd.stateCache.PutCodeWithHashIfAbsent(k, v, crypto.Keccak256(v), readTxNum) - } - } else { - if len(v) == 0 && sd.stateCache.GetCache(domain) != nil { - readTxNum = tx.Debug().DomainProgress(domain) - } - sd.stateCache.PutIfAbsent(domain, k, v, readTxNum) + fillView := view + if !fillView.CanFill() { + // Frontier-less view from the plain GetLatest wrappers: bind a + // frontier here, on the miss path, where the boxing amortizes + // against the backing read it follows. + fillView = sd.cacheViewFor(tx) } + fillView.Fill(domain, k, v, readTxNum) } // Only cache a branch when the read's txN is known: a txN=0 entry would // be treated as immortal by UnwindTo, so skip the Put rather than insert @@ -1294,6 +1393,10 @@ func (sd *SharedDomains) getLatestMetered(domain kv.Domain, tx kv.TemporalTx, k // Returns (size, true, nil) on success and (0, false, nil) only when // CodeDomain itself confirms no code. func (sd *SharedDomains) GetCodeSize(tx kv.TemporalTx, addr []byte, txNum uint64) (int, bool, error) { + return sd.getCodeSize(tx, sd.cacheReader(), addr, txNum) +} + +func (sd *SharedDomains) getCodeSize(tx kv.TemporalTx, view cache.ReadView, addr []byte, txNum uint64) (int, bool, error) { if tx == nil { return 0, false, errors.New("sd.GetCodeSize: unexpected nil tx") } @@ -1301,14 +1404,14 @@ func (sd *SharedDomains) GetCodeSize(tx kv.TemporalTx, addr []byte, txNum uint64 // Fast path: when we can resolve codeHash from the account cache AND // the size is in the size cache, return without loading bytes. if sd.stateCache != nil { - if codeHash := sd.codeHashForAddr(tx, addr, txNum); len(codeHash) > 0 { - if size, ok := sd.stateCache.GetCodeSizeByHash(codeHash); ok { + if codeHash := sd.codeHashForAddr(tx, view, addr, txNum); len(codeHash) > 0 { + if size, ok := view.GetCodeSizeByHash(codeHash); ok { return size, true, nil } - if cv, ok := sd.stateCache.GetCodeByHash(codeHash); ok { + if cv, ok := view.GetCodeByHash(codeHash); ok { // txNum is a conservative upper bound: >= the live code's write // txNum, so the size drops on any unwind that drops the code. - sd.stateCache.PutCodeSizeByHash(codeHash, len(cv), txNum) + view.FillCodeSize(codeHash, len(cv), txNum) return len(cv), true, nil } } @@ -1317,7 +1420,7 @@ func (sd *SharedDomains) GetCodeSize(tx kv.TemporalTx, addr []byte, txNum uint64 // Cold path: authoritative read via the normal SD.GetLatest chain. // Populates L1, codeHashToCode, and (via PutWithCodeHash) the size layer for // future callers. - v, _, err := sd.GetLatest(kv.CodeDomain, tx, addr) + v, _, err := sd.getLatestMetered(kv.CodeDomain, tx, addr, nil, view) if err != nil { return 0, false, err } @@ -1342,6 +1445,10 @@ func (sd *SharedDomains) GetCodeSize(tx kv.TemporalTx, addr []byte, txNum uint64 // the write. Setters therefore resolve prevVal through GetLatest, which is // addr-keyed (domain-faithful); only getters use this codeHash shortcut. func (sd *SharedDomains) GetCode(tx kv.TemporalTx, addr []byte, txNum uint64) ([]byte, bool, error) { + return sd.getCode(tx, sd.cacheReader(), addr, txNum) +} + +func (sd *SharedDomains) getCode(tx kv.TemporalTx, view cache.ReadView, addr []byte, txNum uint64) ([]byte, bool, error) { if tx == nil { return nil, false, errors.New("sd.GetCode: unexpected nil tx") } @@ -1352,9 +1459,9 @@ func (sd *SharedDomains) GetCode(tx kv.TemporalTx, addr []byte, txNum uint64) ([ // a stateObject's stale snapshot) is reorg-safe. var codeHash []byte if sd.stateCache != nil || sd.codeStore != nil { - if codeHash = sd.codeHashForAddr(tx, addr, txNum); len(codeHash) > 0 { + if codeHash = sd.codeHashForAddr(tx, view, addr, txNum); len(codeHash) > 0 { if sd.stateCache != nil { - if cv, ok := sd.stateCache.GetCodeByHash(codeHash); ok { + if cv, ok := view.GetCodeByHash(codeHash); ok { return cv, true, nil } } @@ -1367,7 +1474,7 @@ func (sd *SharedDomains) GetCode(tx kv.TemporalTx, addr []byte, txNum uint64) ([ } // Cold path: authoritative addr-keyed read (also populates the caches). - v, _, err := sd.GetLatest(kv.CodeDomain, tx, addr) + v, _, err := sd.getLatestMetered(kv.CodeDomain, tx, addr, nil, view) if err != nil { return nil, false, err } @@ -1389,7 +1496,7 @@ func (sd *SharedDomains) GetCode(tx kv.TemporalTx, addr []byte, txNum uint64) ([ // unwind invalidation). It is passed in by the caller — never read from the // shared sd.txNum, which a parallel exec worker on this read path must not // touch (the exec loop advances it concurrently). -func (sd *SharedDomains) codeHashForAddr(tx kv.TemporalTx, addr []byte, txNum uint64) []byte { +func (sd *SharedDomains) codeHashForAddr(tx kv.TemporalTx, view cache.ReadView, addr []byte, txNum uint64) []byte { if len(addr) == 0 { return nil } @@ -1410,7 +1517,7 @@ func (sd *SharedDomains) codeHashForAddr(tx kv.TemporalTx, addr []byte, txNum ui // (flush-invalidated). The zero-hash sentinel means "no code / missing // account" (negative cache). if sd.stateCache != nil { - if h, ok := sd.stateCache.GetAddrCodeHash(addr); ok { + if h, ok := view.GetAddrCodeHash(addr); ok { if h == ([32]byte{}) { return nil } @@ -1418,32 +1525,44 @@ func (sd *SharedDomains) codeHashForAddr(tx kv.TemporalTx, addr []byte, txNum ui } } - // Resolve from the committed layers (stateCache → MDBX/files) and populate - // the LRU. mem is intentionally not consulted here — it was checked above. - resolve := func() []byte { + // Resolve from the committed layers (stateCache → MDBX/files). mem is + // intentionally not consulted here — it was checked above. fromReadView + // reports whether the record was read from the tx's read view. + resolve := func() ([]byte, bool) { if sd.stateCache != nil { - if v, ok := sd.stateCache.Get(kv.AccountsDomain, addr); ok { - return accounts.DeserialiseV3CodeHash(v) + if v, ok := view.Get(kv.AccountsDomain, addr); ok { + return accounts.DeserialiseV3CodeHash(v), false } } v, _, err := tx.GetLatest(kv.AccountsDomain, addr) - if err != nil || len(v) == 0 { - return nil + if err != nil { + return nil, false } - return accounts.DeserialiseV3CodeHash(v) + if len(v) == 0 { + return nil, true + } + return accounts.DeserialiseV3CodeHash(v), true } - h := resolve() - if sd.stateCache != nil { + h, fromReadView := resolve() + if fromReadView && sd.stateCache != nil { var fixed [32]byte if len(h) == 32 { copy(fixed[:], h) } - // Always offer the mapping, including the zero-hash sentinel for - // misses — repeat lookups skip the whole resolve() chain. txNum is a - // conservative upper bound (>= the resolved account's write txNum), so - // the mapping drops on any unwind that reverts that account. - sd.stateCache.PutAddrCodeHash(addr, fixed, txNum) + // Only a view-sourced record (including the zero-hash sentinel for + // misses) may seed the mapping: the admission gate vouches for the tx's + // frontier, and a cache-sourced record can lag a just-committed flush, + // slipping pre-apply state past the gate. txNum is a conservative upper + // bound (>= the resolved account's write txNum), so the mapping drops + // on any unwind that reverts that account. + seedView := view + if !seedView.CanFill() { + // Frontier-less view from the plain wrappers: bind one on this cold + // seed path, where the boxing amortizes against the account read. + seedView = sd.cacheViewFor(tx) + } + seedView.SeedAddrCodeHash(addr, fixed, txNum) } return h } diff --git a/db/state/execctx/domain_visible_end_memo_test.go b/db/state/execctx/domain_visible_end_memo_test.go new file mode 100644 index 00000000000..0ad82a7132a --- /dev/null +++ b/db/state/execctx/domain_visible_end_memo_test.go @@ -0,0 +1,103 @@ +// Copyright 2026 The Erigon Authors +// This file is part of Erigon. +// +// Erigon is free software: you can redistribute it and/or modify +// it under the terms of the GNU Lesser General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Erigon is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Lesser General Public License for more details. +// +// You should have received a copy of the GNU Lesser General Public License +// along with Erigon. If not, see . + +package execctx + +import ( + "sync" + "sync/atomic" + "testing" + + "github.com/stretchr/testify/require" + + "github.com/erigontech/erigon/db/kv" +) + +type stubVisibleEndTx struct { + kv.TemporalTx + viewID uint64 +} + +func (tx *stubVisibleEndTx) ViewID() uint64 { return tx.viewID } +func (tx *stubVisibleEndTx) Debug() kv.TemporalDebugTx { return stubVisibleEndDebug{viewID: tx.viewID} } + +type stubVisibleEndDebug struct { + kv.TemporalDebugTx + viewID uint64 +} + +func (d stubVisibleEndDebug) DomainVisibleEnd(kv.Domain) (uint64, bool) { + return d.viewID * 100, true +} + +// Parallel-exec workers share one SharedDomains and one view, so the memo +// must tolerate concurrent gets interleaved with resets, and must re-derive +// after a sequential view rotation. +func TestDomainVisibleEndMemoConcurrent(t *testing.T) { + t.Parallel() + + var memo domainVisibleEndMemo + var wg sync.WaitGroup + for range 8 { + tx := &stubVisibleEndTx{viewID: 7} + wg.Go(func() { + for range 512 { + for d := range kv.DomainLen { + end, ok := memo.get(tx, d) + if !ok || end != 700 { + t.Errorf("domain %v: got (%d, %t)", d, end, ok) + return + } + } + } + }) + } + wg.Go(func() { + for range 512 { + memo.reset() + } + }) + wg.Wait() + + rotated := &stubVisibleEndTx{viewID: 8} + end, ok := memo.get(rotated, kv.AccountsDomain) + require.True(t, ok) + require.Equal(t, uint64(800), end) +} + +func TestDomainVisibleEndMemoConcurrentViews(t *testing.T) { + t.Parallel() + + var memo domainVisibleEndMemo + var mismatches atomic.Uint64 + var wg sync.WaitGroup + for _, viewID := range []uint64{7, 8} { + for range 8 { + tx := &stubVisibleEndTx{viewID: viewID} + wg.Go(func() { + for range 100_000 { + end, ok := memo.get(tx, kv.AccountsDomain) + if !ok || end != viewID*100 { + mismatches.Add(1) + } + } + }) + } + } + wg.Wait() + + require.Zero(t, mismatches.Load(), "memo returned a frontier from another view") +} diff --git a/db/state/execctx/export_test.go b/db/state/execctx/export_test.go index 74607252d79..868dacd5a98 100644 --- a/db/state/execctx/export_test.go +++ b/db/state/execctx/export_test.go @@ -9,7 +9,7 @@ import ( // external test package (which cannot import db/state to build a SharedDomains // internally without an import cycle). func (sd *SharedDomains) CodeHashForAddr(tx kv.TemporalTx, addr []byte, txNum uint64) []byte { - return sd.codeHashForAddr(tx, addr, txNum) + return sd.codeHashForAddr(tx, sd.cacheReader(), addr, txNum) } // SetStateCacheForTest attaches a cache unconditionally, bypassing the @@ -18,4 +18,5 @@ func (sd *SharedDomains) CodeHashForAddr(tx kv.TemporalTx, addr []byte, txNum ui // — without mutating the process-global flag (which would race t.Parallel tests). func (sd *SharedDomains) SetStateCacheForTest(sc *cache.StateCache) { sd.stateCache = sc + sd.cacheApplier = sc.Applier() } diff --git a/db/state/execctx/flush_storage_cache_test.go b/db/state/execctx/flush_storage_cache_test.go index 4df663dfd7a..fd9feb429a0 100644 --- a/db/state/execctx/flush_storage_cache_test.go +++ b/db/state/execctx/flush_storage_cache_test.go @@ -77,14 +77,14 @@ func TestCommit_UpdatesStorageStateCache(t *testing.T) { // First commit: the storage callback must fire and populate the cache. commit(1, val1, nil) - got, ok := sc.Get(kv.StorageDomain, key) + got, ok := sc.View(nil).Get(kv.StorageDomain, key) require.True(t, ok, "storage cache must be populated by the commit callback") require.Equal(t, val1, got) // Overwrite in a second tx: the callback must fire again and refresh the // entry — not leave the stale val1 behind. commit(stepSize+1, val2, val1) - got, ok = sc.Get(kv.StorageDomain, key) + got, ok = sc.View(nil).Get(kv.StorageDomain, key) require.True(t, ok) require.Equal(t, val2, got, "commit must refresh the storage cache; stale value served on hit was the bug") } diff --git a/db/state/execctx/statecache_readfill_bench_test.go b/db/state/execctx/statecache_readfill_bench_test.go index a5868cf2933..8d13ae8face 100644 --- a/db/state/execctx/statecache_readfill_bench_test.go +++ b/db/state/execctx/statecache_readfill_bench_test.go @@ -49,33 +49,42 @@ func benchSeedDb(b *testing.B) kv.TemporalRwDB { return db } -// BenchmarkDomainProgress isolates the negative-stamp source: one -// files.EndTxNum read plus an MDBX LastKey on the domain's keys table. -func BenchmarkDomainProgress(b *testing.B) { +// BenchmarkDomainVisibleEnd isolates the transaction-local cached frontier +// lookup used by repeated cache fills. +func BenchmarkDomainVisibleEnd(b *testing.B) { db := benchSeedDb(b) roTx, err := db.BeginTemporalRo(b.Context()) require.NoError(b, err) defer roTx.Rollback() + _, _ = roTx.Debug().DomainVisibleEnd(kv.AccountsDomain) b.ResetTimer() for i := 0; i < b.N; i++ { - _ = roTx.Debug().DomainProgress(kv.AccountsDomain) + _, _ = roTx.Debug().DomainVisibleEnd(kv.AccountsDomain) } } // benchColdNegativeReads drives the full cold-negative SD read: the whole -// miss stack, plus — when a cache is wired — the progress stamp and the -// if-absent fill. -func benchColdNegativeReads(b *testing.B, withCache bool) { +// miss stack, plus — when a cache is wired — the exact-frontier lookup and +// freshness-checked fill. +func benchColdNegativeReads(b *testing.B, withCache, writable bool) { db := benchSeedDb(b) ctx := b.Context() - roTx, err := db.BeginTemporalRo(ctx) + var tx kv.TemporalTx + var err error + if writable { + tx, err = db.BeginTemporalRw(ctx) + } else { + tx, err = db.BeginTemporalRo(ctx) + } require.NoError(b, err) - defer roTx.Rollback() - sd, err := execctx.NewSharedDomains(ctx, roTx, log.New()) + defer tx.Rollback() + sd, err := execctx.NewSharedDomains(ctx, tx, log.New()) require.NoError(b, err) defer sd.Close() if withCache { - sd.SetStateCacheForTest(newSmallStateCache()) + stateCache := newSmallStateCache() + defer stateCache.Close() + sd.SetStateCacheForTest(stateCache) } key := make([]byte, 20) @@ -83,7 +92,7 @@ func benchColdNegativeReads(b *testing.B, withCache bool) { b.ResetTimer() for i := 0; i < b.N; i++ { binary.BigEndian.PutUint64(key[12:], uint64(i)+1) - v, _, err := sd.GetLatest(kv.AccountsDomain, roTx, key) + v, _, err := sd.GetLatest(kv.AccountsDomain, tx, key) if err != nil { b.Fatal(err) } @@ -93,7 +102,15 @@ func benchColdNegativeReads(b *testing.B, withCache bool) { } } -func BenchmarkGetLatestColdNegative(b *testing.B) { benchColdNegativeReads(b, true) } +func BenchmarkGetLatestColdNegative(b *testing.B) { benchColdNegativeReads(b, true, false) } // The baseline the stamp+fill cost adds to. -func BenchmarkGetLatestColdNegativeNoCache(b *testing.B) { benchColdNegativeReads(b, false) } +func BenchmarkGetLatestColdNegativeNoCache(b *testing.B) { + benchColdNegativeReads(b, false, false) +} + +func BenchmarkGetLatestColdNegativeRw(b *testing.B) { benchColdNegativeReads(b, true, true) } + +func BenchmarkGetLatestColdNegativeRwNoCache(b *testing.B) { + benchColdNegativeReads(b, false, true) +} diff --git a/db/state/execctx/statecache_readfill_test.go b/db/state/execctx/statecache_readfill_test.go index 95d63fbafd7..7c198a3034e 100644 --- a/db/state/execctx/statecache_readfill_test.go +++ b/db/state/execctx/statecache_readfill_test.go @@ -18,6 +18,7 @@ package execctx_test import ( "encoding/binary" + "math" "testing" "github.com/c2h5oh/datasize" @@ -75,6 +76,88 @@ func newSmallStateCache() *cache.StateCache { return cache.NewStateCache(b, b, b, b) } +func frontierAt(end uint64) cache.Frontier { + return cache.FrontierFunc(func(kv.Domain) (uint64, bool) { return end, true }) +} + +// seed places an entry with an exact txNum stamp through the public fill API +// without moving the applied frontier. A positive passes admission at any +// applied end; a negative is stamped frontier-1 by the fill path, so it must +// be seeded while the applied end is at most txNum+1. +func seed(sc *cache.StateCache, domain kv.Domain, k, v []byte, txNum uint64) { + end := uint64(math.MaxUint64) + if len(v) == 0 { + end = txNum + 1 + } + sc.View(frontierAt(end)).Fill(domain, k, v, txNum) +} + +type visibleEndCountingDebugTx struct { + kv.TemporalDebugTx + calls uint64 + last uint64 +} + +func (tx *visibleEndCountingDebugTx) DomainVisibleEnd(domain kv.Domain) (uint64, bool) { + tx.calls++ + end, ok := tx.TemporalDebugTx.DomainVisibleEnd(domain) + tx.last = end + return end, ok +} + +type visibleEndCountingRwTx struct { + kv.TemporalRwTx + debug *visibleEndCountingDebugTx +} + +func (tx *visibleEndCountingRwTx) Debug() kv.TemporalDebugTx { + return tx.debug +} + +func TestReadFill_MemoizesWritableVisibleEndUntilFlush(t *testing.T) { + t.Parallel() + + const stepSize = uint64(16) + ctx := t.Context() + db := newTestDb(t, stepSize) + + baseTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer baseTx.Rollback() + debug := &visibleEndCountingDebugTx{TemporalDebugTx: baseTx.Debug()} + rwTx := &visibleEndCountingRwTx{TemporalRwTx: baseTx, debug: debug} + domains, err := execctx.NewSharedDomains(ctx, rwTx, log.New()) + require.NoError(t, err) + defer domains.Close() + stateCache := newSmallStateCache() + t.Cleanup(stateCache.Close) + domains.SetStateCacheForTest(stateCache) + + for i := byte(2); i <= 3; i++ { + missing := make([]byte, 20) + missing[0] = i + value, _, err := domains.GetLatest(kv.AccountsDomain, rwTx, missing) + require.NoError(t, err) + require.Empty(t, value) + } + require.Equal(t, uint64(1), debug.calls) + initialEnd := debug.last + + written := make([]byte, 20) + written[0] = 4 + domains.SetTxNum(20) + require.NoError(t, domains.DomainPut(kv.AccountsDomain, rwTx, written, encAccount(2), 20, nil)) + require.NoError(t, domains.Flush(ctx, rwTx)) + + missing := make([]byte, 20) + missing[0] = 5 + value, _, err := domains.GetLatest(kv.AccountsDomain, rwTx, missing) + require.NoError(t, err) + require.Empty(t, value) + require.Equal(t, uint64(2), debug.calls) + require.Greater(t, debug.last, initialEnd) +} + // During an in-flight unwind the mem overlay bounds reads of an affected key // by maxStep while MDBX still holds the not-yet-deleted dying row inside that // bound. A cache hit legitimately below the unwind floor then diverges from @@ -99,9 +182,10 @@ func TestAssertStateCache_NoFalsePanicDuringInFlightUnwind(t *testing.T) { defer sd.Close() sd.SetStateCacheForTest(sc) - // A live cache entry below the unwind floor: the restored (correct) value. - sc.Put(kv.AccountsDomain, key, v1, 5) sd.Unwind(10, &diffs) // in-flight: mem publishes maxStep=1; MDBX still holds the step-1 row + // A live cache entry below the unwind floor: the restored (correct) value, + // as a post-unwind fill would insert it. + seed(sc, kv.AccountsDomain, key, v1, 5) old := dbg.AssertStateCache dbg.AssertStateCache = true @@ -156,8 +240,8 @@ func TestAssertStateCache_NoFalsePanicDuringInFlightUnwindStepZero(t *testing.T) defer sd2.Close() sd2.SetStateCacheForTest(sc) - sc.Put(kv.AccountsDomain, key, nil, 2) sd2.Unwind(3, &diffs) + seed(sc, kv.AccountsDomain, key, nil, 2) old := dbg.AssertStateCache dbg.AssertStateCache = true @@ -172,7 +256,7 @@ func TestAssertStateCache_NoFalsePanicDuringInFlightUnwindStepZero(t *testing.T) } // The read-fill after a fall-through read must not replace a live cache -// entry: it never carries newer information than a flush-apply, and during an +// entry: it never carries newer information than a post-commit apply, and during an // in-flight unwind the bounded DB read can even return the not-yet-deleted // dying row. func TestReadFill_DoesNotClobberLiveEntry(t *testing.T) { @@ -197,21 +281,20 @@ func TestReadFill_DoesNotClobberLiveEntry(t *testing.T) { // A live (current-epoch) entry above the read bound: the maxStep gate turns // the hit into a miss, so the read falls through to the bounded DB read. v3 := encAccount(3) - sc.Put(kv.AccountsDomain, key, v3, 40) + seed(sc, kv.AccountsDomain, key, v3, 40) v, _, err := sd.GetLatest(kv.AccountsDomain, roTx, key) require.NoError(t, err) require.Equal(t, v2, v, "fall-through read serves the maxStep-bounded DB row") - got, ok := sc.Get(kv.AccountsDomain, key) + got, ok := sc.View(nil).Get(kv.AccountsDomain, key) require.True(t, ok) require.Equal(t, v3, got, "read-fill must not clobber the live entry") } -// Negative results (missing account) must be stamped with the domain's -// progress at observation time, not a synthetic step-0 bound that survives -// every unwind. -func TestReadFill_NegativeStampedWithProgress(t *testing.T) { +// A negative reflects transactions below the read view's exclusive frontier, +// so its unwind stamp is the last included txNum. +func TestReadFill_NegativeUsesLastVisibleTxNum(t *testing.T) { t.Parallel() const stepSize = uint64(16) @@ -237,6 +320,9 @@ func TestReadFill_NegativeStampedWithProgress(t *testing.T) { roTx, err := db.BeginTemporalRo(ctx) require.NoError(t, err) defer roTx.Rollback() + visibleEnd, ok := roTx.Debug().DomainVisibleEnd(kv.AccountsDomain) + require.True(t, ok) + require.NotZero(t, visibleEnd) sd2, err := execctx.NewSharedDomains(ctx, roTx, log.New()) require.NoError(t, err) defer sd2.Close() @@ -247,12 +333,57 @@ func TestReadFill_NegativeStampedWithProgress(t *testing.T) { v, _, err := sd2.GetLatest(kv.AccountsDomain, roTx, missing) require.NoError(t, err) require.Empty(t, v) - _, ok := sc.Get(kv.AccountsDomain, missing) + _, ok = sc.View(nil).Get(kv.AccountsDomain, missing) require.True(t, ok, "the negative result must be cached") - // The domain's progress is 100 (the committed write), so any unwind at or - // below it must drop the negative instead of letting it outlive the fact. - sc.Unwind(50) - _, ok = sc.Get(kv.AccountsDomain, missing) - require.False(t, ok, "a negative observed at progress 100 must not survive an unwind to 50") + sc.Applier().Unwind(visibleEnd) + _, ok = sc.View(nil).Get(kv.AccountsDomain, missing) + require.True(t, ok, "an unwind starting after the read view must preserve the negative") + + sc.Applier().Unwind(visibleEnd - 1) + _, ok = sc.View(nil).Get(kv.AccountsDomain, missing) + require.False(t, ok, "an unwind of the view's last included txNum must invalidate the negative") +} + +type fakeForbidder struct{ called bool } + +func (f *fakeForbidder) ForbidVisibilityLowering() { f.called = true } + +type fakeHasAgg struct{ f *fakeForbidder } + +func (h fakeHasAgg) Agg() any { return h.f } + +type fakeHasBadAgg struct{} + +func (fakeHasBadAgg) Agg() any { return struct{}{} } + +// The guard is load-bearing: for a fill-enabled cache it must either bind the +// invariant or fail loudly — never silently drop it on a DB shape mismatch. +// A nil or apply-only cache needs no guard at all. +func TestGuardAggregatorForCache(t *testing.T) { + sc := newSmallStateCache() + t.Cleanup(sc.Close) + + f := &fakeForbidder{} + execctx.GuardAggregatorForCache(fakeHasAgg{f}, sc) + require.True(t, f.called) + + require.NotPanics(t, func() { execctx.GuardAggregatorForCache(struct{}{}, nil) }, + "no cache, no invariant to bind — shape is irrelevant") + require.Panics(t, func() { execctx.GuardAggregatorForCache(struct{}{}, sc) }, + "a db that cannot produce its aggregator must fail loudly, not drop the guard") + require.Panics(t, func() { execctx.GuardAggregatorForCache(fakeHasBadAgg{}, sc) }, + "an aggregator without ForbidVisibilityLowering must fail loudly, not drop the guard") +} + +// An apply-only cache (STATE_CACHE_FILLS=false) has no fills for a lowered +// frontier to poison, so the guard must not constrain the aggregator. +func TestGuardAggregatorForCache_ApplyOnlySkips(t *testing.T) { + t.Setenv("STATE_CACHE_FILLS", "false") + sc := newSmallStateCache() + t.Cleanup(sc.Close) + + f := &fakeForbidder{} + execctx.GuardAggregatorForCache(fakeHasAgg{f}, sc) + require.False(t, f.called) } diff --git a/db/state/execctx/statecache_rpc_integration_test.go b/db/state/execctx/statecache_rpc_integration_test.go new file mode 100644 index 00000000000..789bea870df --- /dev/null +++ b/db/state/execctx/statecache_rpc_integration_test.go @@ -0,0 +1,342 @@ +// Copyright 2026 The Erigon Authors +// This file is part of Erigon. +// +// Erigon is free software: you can redistribute it and/or modify +// it under the terms of the GNU Lesser General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Erigon is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Lesser General Public License for more details. +// +// You should have received a copy of the GNU Lesser General Public License +// along with Erigon. If not, see . + +package execctx_test + +import ( + "testing" + + "github.com/c2h5oh/datasize" + "github.com/holiman/uint256" + "github.com/stretchr/testify/require" + + "github.com/erigontech/erigon/common/crypto" + "github.com/erigontech/erigon/common/log/v3" + "github.com/erigontech/erigon/db/kv" + "github.com/erigontech/erigon/db/state/execctx" + "github.com/erigontech/erigon/execution/cache" + "github.com/erigontech/erigon/execution/execmodule" + "github.com/erigontech/erigon/execution/types/accounts" + "github.com/erigontech/erigon/node/shards" +) + +func TestEmbeddedRPCCacheViewDoesNotResurrectDeletedAccount(t *testing.T) { + testEmbeddedRPCCacheViewDoesNotResurrectDeletedValue(t, kv.AccountsDomain) +} + +func TestEmbeddedRPCCacheViewDoesNotResurrectDeletedStorage(t *testing.T) { + testEmbeddedRPCCacheViewDoesNotResurrectDeletedValue(t, kv.StorageDomain) +} + +func TestEmbeddedRPCCacheViewDoesNotResurrectDeletedCode(t *testing.T) { + testEmbeddedRPCCacheViewDoesNotResurrectDeletedValue(t, kv.CodeDomain) +} + +func TestAccountOnlyDeleteDoesNotBlockUnrelatedCodeFill(t *testing.T) { + const stepSize = uint64(16) + ctx := t.Context() + db := newTestDb(t, stepSize) + + contractAddr := make([]byte, 20) + contractAddr[0] = 0xaa + deletedAddr := make([]byte, 20) + deletedAddr[0] = 0xbb + code := []byte{0xcc, 1, 2, 3} + account := accounts.SerialiseV3(&accounts.Account{ + Nonce: 1, + Balance: *uint256.NewInt(1), + CodeHash: accounts.InternCodeHash(crypto.Keccak256Hash(code)), + }) + codelessAccount := accounts.SerialiseV3(&accounts.Account{ + Nonce: 1, + Balance: *uint256.NewInt(1), + }) + + seedTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer seedTx.Rollback() + seedDomains, err := execctx.NewSharedDomains(ctx, seedTx, log.New()) + require.NoError(t, err) + defer seedDomains.Close() + seedDomains.SetTxNum(10) + require.NoError(t, seedDomains.DomainPut(kv.AccountsDomain, seedTx, contractAddr, account, 10, nil)) + require.NoError(t, seedDomains.DomainPut(kv.CodeDomain, seedTx, contractAddr, code, 10, nil)) + require.NoError(t, seedDomains.DomainPut(kv.AccountsDomain, seedTx, deletedAddr, codelessAccount, 10, nil)) + require.NoError(t, seedDomains.Commit(ctx, seedTx)) + seedDomains.Close() + + budget := 1 * datasize.MB + stateCache := cache.NewStateCache(budget, budget, budget, budget) + t.Cleanup(stateCache.Close) + + deleteTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer deleteTx.Rollback() + deleteDomains, err := execctx.NewSharedDomains(ctx, deleteTx, log.New()) + require.NoError(t, err) + defer deleteDomains.Close() + deleteDomains.SetStateCacheForTest(stateCache) + deleteDomains.SetTxNum(20) + require.NoError(t, deleteDomains.DomainDel(kv.AccountsDomain, deleteTx, deletedAddr, 20, nil)) + require.NoError(t, deleteDomains.Commit(ctx, deleteTx)) + deleteDomains.Close() + + freshTx, err := db.BeginTemporalRo(ctx) + require.NoError(t, err) + defer freshTx.Rollback() + codeEnd, ok := freshTx.Debug().DomainVisibleEnd(kv.CodeDomain) + require.True(t, ok) + accountsEnd, ok := freshTx.Debug().DomainVisibleEnd(kv.AccountsDomain) + require.True(t, ok) + require.Less(t, codeEnd, accountsEnd) + + freshDomains, err := execctx.NewSharedDomains(ctx, freshTx, log.New()) + require.NoError(t, err) + defer freshDomains.Close() + freshDomains.SetStateCacheForTest(stateCache) + got, _, err := freshDomains.GetLatest(kv.CodeDomain, freshTx, contractAddr) + require.NoError(t, err) + require.Equal(t, code, got) + + cached, ok := stateCache.View(nil).Get(kv.CodeDomain, contractAddr) + require.True(t, ok, "an account-only deletion must not block unrelated code fills") + require.Equal(t, code, cached) +} + +func TestSharedDomainsNegativeCacheEntryUsesLastVisibleTxNum(t *testing.T) { + const stepSize = uint64(16) + ctx := t.Context() + db := newTestDb(t, stepSize) + + presentKey := make([]byte, 20) + presentKey[0] = 0xaa + missingKey := make([]byte, 20) + missingKey[0] = 0xbb + account := accounts.SerialiseV3(&accounts.Account{ + Nonce: 1, + Balance: *uint256.NewInt(1), + }) + + seedTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer seedTx.Rollback() + seedDomains, err := execctx.NewSharedDomains(ctx, seedTx, log.New()) + require.NoError(t, err) + defer seedDomains.Close() + seedDomains.SetTxNum(10) + require.NoError(t, seedDomains.DomainPut(kv.AccountsDomain, seedTx, presentKey, account, 10, nil)) + require.NoError(t, seedDomains.Commit(ctx, seedTx)) + seedDomains.Close() + + budget := 1 * datasize.MB + stateCache := cache.NewStateCache(budget, budget, budget, budget) + t.Cleanup(stateCache.Close) + + readTx, err := db.BeginTemporalRo(ctx) + require.NoError(t, err) + defer readTx.Rollback() + visibleEnd, ok := readTx.Debug().DomainVisibleEnd(kv.AccountsDomain) + require.True(t, ok) + require.NotZero(t, visibleEnd) + + readDomains, err := execctx.NewSharedDomains(ctx, readTx, log.New()) + require.NoError(t, err) + defer readDomains.Close() + readDomains.SetStateCacheForTest(stateCache) + got, _, err := readDomains.GetLatest(kv.AccountsDomain, readTx, missingKey) + require.NoError(t, err) + require.Empty(t, got) + + cached, ok := stateCache.View(nil).Get(kv.AccountsDomain, missingKey) + require.True(t, ok) + require.Empty(t, cached) + + stateCache.Applier().Unwind(visibleEnd) + _, ok = stateCache.View(nil).Get(kv.AccountsDomain, missingKey) + require.True(t, ok, "a negative observed before the unwind floor must remain cached") + + stateCache.Applier().Unwind(visibleEnd - 1) + _, ok = stateCache.View(nil).Get(kv.AccountsDomain, missingKey) + require.False(t, ok, "a negative observed at the unwind floor must be invalidated") +} + +func testEmbeddedRPCCacheViewDoesNotResurrectDeletedValue(t *testing.T, domain kv.Domain) { + t.Helper() + const stepSize = uint64(16) + ctx := t.Context() + db := newTestDb(t, stepSize) + + budget := 1 * datasize.MB + stateCache := cache.NewStateCache(budget, budget, budget, budget) + t.Cleanup(stateCache.Close) + + keyLen := 20 + if domain == kv.StorageDomain { + keyLen = 52 + } + key := make([]byte, keyLen) + key[0] = 0xab + value := accounts.SerialiseV3(&accounts.Account{ + Nonce: 1, + Balance: *uint256.NewInt(1), + }) + switch domain { + case kv.StorageDomain: + value = []byte{0x01} + case kv.CodeDomain: + value = []byte{0xaa, 1, 2, 3} + } + + seedTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer seedTx.Rollback() + seedDomains, err := execctx.NewSharedDomains(ctx, seedTx, log.New()) + require.NoError(t, err) + defer seedDomains.Close() + seedDomains.SetStateCacheForTest(stateCache) + seedDomains.SetTxNum(10) + require.NoError(t, seedDomains.DomainPut(domain, seedTx, key, value, 10, nil)) + require.NoError(t, seedDomains.Commit(ctx, seedTx)) + seedDomains.Close() + + rpcTx, err := db.BeginTemporalRo(ctx) + require.NoError(t, err) + defer rpcTx.Rollback() + + deleteTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer deleteTx.Rollback() + deleteDomains, err := execctx.NewSharedDomains(ctx, deleteTx, log.New()) + require.NoError(t, err) + defer deleteDomains.Close() + deleteDomains.SetStateCacheForTest(stateCache) + deleteDomains.SetTxNum(20) + require.NoError(t, deleteDomains.DomainDel(domain, deleteTx, key, 20, value)) + + events := shards.NewEvents() + events.PublishOverlay(deleteDomains) + rpcCache := &execmodule.Cache{} + rpcCache.SetPublishedSD(events.LatestSD) + rpcView, err := rpcCache.View(ctx, rpcTx) + require.NoError(t, err) + + require.NoError(t, deleteDomains.Commit(ctx, deleteTx)) + events.PublishOverlay(nil) + // The view outlives the overlay teardown on purpose: a production RPC view + // built during a background commit keeps reading after PublishOverlay(nil) + // and the SD's Close. + deleteDomains.Close() + + oldValue, _, err := rpcTx.GetLatest(domain, key) + require.NoError(t, err) + require.Equal(t, value, oldValue) + + freshTx, err := db.BeginTemporalRo(ctx) + require.NoError(t, err) + defer freshTx.Rollback() + freshValue, _, err := freshTx.GetLatest(domain, key) + require.NoError(t, err) + require.Empty(t, freshValue) + + if domain == kv.CodeDomain { + _, err = rpcView.GetCode(key) + } else { + _, err = rpcView.Get(key) + } + require.NoError(t, err) + + freshDomains, err := execctx.NewSharedDomains(ctx, freshTx, log.New()) + require.NoError(t, err) + defer freshDomains.Close() + freshDomains.SetStateCacheForTest(stateCache) + got, _, err := freshDomains.GetLatest(domain, freshTx, key) + require.NoError(t, err) + require.Empty(t, got, "the old RPC read view must not repopulate the shared cache after the deletion") +} + +// The account-deletion mirror of TestEmbeddedRPCCacheViewDoesNotResurrectDeletedCode. +// DomainDel(AccountsDomain) cascades a code-domain delete at the SD layer, so the +// commit applies it and the code frontier advances past every pre-deletion view — +// and the cache-level code-fill admission also checks the accounts frontier. This +// pins both layers: losing either must not let a pre-deletion RPC view refill the +// deleted account's code. +func TestEmbeddedRPCCacheViewDoesNotRefillCodeOfDeletedAccount(t *testing.T) { + const stepSize = uint64(16) + ctx := t.Context() + db := newTestDb(t, stepSize) + + budget := 1 * datasize.MB + stateCache := cache.NewStateCache(budget, budget, budget, budget) + t.Cleanup(stateCache.Close) + + addr := make([]byte, 20) + addr[0] = 0xab + code := []byte{0xaa, 1, 2, 3} + account := accounts.SerialiseV3(&accounts.Account{ + Nonce: 1, + Balance: *uint256.NewInt(1), + CodeHash: accounts.InternCodeHash(crypto.Keccak256Hash(code)), + }) + + seedTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer seedTx.Rollback() + seedDomains, err := execctx.NewSharedDomains(ctx, seedTx, log.New()) + require.NoError(t, err) + defer seedDomains.Close() + seedDomains.SetStateCacheForTest(stateCache) + seedDomains.SetTxNum(10) + require.NoError(t, seedDomains.DomainPut(kv.AccountsDomain, seedTx, addr, account, 10, nil)) + require.NoError(t, seedDomains.DomainPut(kv.CodeDomain, seedTx, addr, code, 10, nil)) + require.NoError(t, seedDomains.Commit(ctx, seedTx)) + seedDomains.Close() + + rpcTx, err := db.BeginTemporalRo(ctx) + require.NoError(t, err) + defer rpcTx.Rollback() + + deleteTx, err := db.BeginTemporalRw(ctx) + require.NoError(t, err) + defer deleteTx.Rollback() + deleteDomains, err := execctx.NewSharedDomains(ctx, deleteTx, log.New()) + require.NoError(t, err) + defer deleteDomains.Close() + deleteDomains.SetStateCacheForTest(stateCache) + deleteDomains.SetTxNum(20) + require.NoError(t, deleteDomains.DomainDel(kv.AccountsDomain, deleteTx, addr, 20, account)) + + events := shards.NewEvents() + events.PublishOverlay(deleteDomains) + rpcCache := &execmodule.Cache{} + rpcCache.SetPublishedSD(events.LatestSD) + rpcView, err := rpcCache.View(ctx, rpcTx) + require.NoError(t, err) + + require.NoError(t, deleteDomains.Commit(ctx, deleteTx)) + events.PublishOverlay(nil) + // The view outlives the overlay teardown on purpose, as above. + deleteDomains.Close() + + _, ok := stateCache.View(nil).Get(kv.CodeDomain, addr) + require.False(t, ok, "the account deletion must drop the cached code entry") + + got, err := rpcView.GetCode(addr) + require.NoError(t, err) + require.Equal(t, code, got, "the pre-deletion view still reads the code from its own tx") + + _, ok = stateCache.View(nil).Get(kv.CodeDomain, addr) + require.False(t, ok, "a pre-deletion RPC view must not refill the deleted account's code") +} diff --git a/db/state/inverted_index.go b/db/state/inverted_index.go index f65439f2443..0cff2c7bf65 100644 --- a/db/state/inverted_index.go +++ b/db/state/inverted_index.go @@ -1238,15 +1238,31 @@ func (ii *InvertedIndex) minTxNumInDB(tx kv.Tx) uint64 { return 0 } -func (ii *InvertedIndex) maxTxNumInDB(tx kv.Tx) uint64 { +func (ii *InvertedIndex) lastTxNumInDB(tx kv.Tx) (uint64, bool) { lst, _ := kv.LastKey(tx, ii.KeysTable) - if len(lst) > 0 { - lstInDb := binary.BigEndian.Uint64(lst) - return lstInDb + if len(lst) == 0 { + return 0, false } - return 0 + return binary.BigEndian.Uint64(lst), true +} + +func (ii *InvertedIndex) maxTxNumInDB(tx kv.Tx) uint64 { + txNum, _ := ii.lastTxNumInDB(tx) + return txNum } func (iit *InvertedIndexRoTx) Progress(tx kv.Tx) uint64 { return max(iit.files.EndTxNum(), iit.ii.maxTxNumInDB(tx)) } + +// visibleEnd is the exclusive txNum bound of what this view can see: the max +// of its two components, because GetLatest reads their union. Both sides are +// required — on a snapshot-synced or fully-pruned datadir the DB side is empty +// and the files carry the whole bound. +func (iit *InvertedIndexRoTx) visibleEnd(tx kv.Tx) uint64 { + dbEnd, ok := iit.ii.lastTxNumInDB(tx) + if ok && dbEnd < math.MaxUint64 { + dbEnd++ + } + return max(iit.files.EndTxNum(), dbEnd) +} diff --git a/db/state/inverted_index_test.go b/db/state/inverted_index_test.go index cf0e86d852d..85f77df954c 100644 --- a/db/state/inverted_index_test.go +++ b/db/state/inverted_index_test.go @@ -87,6 +87,32 @@ func testDbAndInvertedIndex(tb testing.TB, aggStep uint64, logger log.Logger) (k return db, ii } +func TestInvertedIndexVisibleEnd(t *testing.T) { + db, ii := testDbAndInvertedIndex(t, 16, log.New()) + tx, err := db.BeginRw(t.Context()) + require.NoError(t, err) + defer tx.Rollback() + + iit := ii.beginForTests() + defer iit.Close() + require.Zero(t, iit.Progress(tx)) + require.Zero(t, iit.visibleEnd(tx)) + + var txNum [8]byte + require.NoError(t, tx.Put(ii.KeysTable, txNum[:], []byte{1})) + require.Zero(t, iit.Progress(tx)) + require.Equal(t, uint64(1), iit.visibleEnd(tx)) + + binary.BigEndian.PutUint64(txNum[:], 100) + require.NoError(t, tx.Put(ii.KeysTable, txNum[:], []byte{1})) + require.Equal(t, uint64(100), iit.Progress(tx)) + require.Equal(t, uint64(101), iit.visibleEnd(tx)) + + iit.files = visibleFiles{{endTxNum: 200}} + require.Equal(t, uint64(200), iit.Progress(tx)) + require.Equal(t, uint64(200), iit.visibleEnd(tx)) +} + func TestInvIndexPruningCorrectness(t *testing.T) { t.Parallel() diff --git a/execution/cache/cache.go b/execution/cache/cache.go index fd59c16f2d5..d30c41a0f4d 100644 --- a/execution/cache/cache.go +++ b/execution/cache/cache.go @@ -14,6 +14,24 @@ // You should have received a copy of the GNU Lesser General Public License // along with Erigon. If not, see . +// Package cache provides the process-global caches of latest committed state. +// +// StateCache holds the newest applied value per key for the accounts, storage +// and code domains, so repeated GetLatest reads skip the file-accessor/MDBX +// stack. It is not a snapshot and gives readers no isolation: a hit can be +// newer than the reader's tx (snapshot-isolated caching is kvcache's job, +// node/shards). In the forward direction its invariant is monotonicity: +// content never regresses behind what has been applied. Unwinds invalidate +// by epoch and floor instead. +// +// StateCache itself has no data methods. A ReadView — bound to one tx's read +// view and not outliving it — serves reads and fills (cache writes made on +// behalf of a database reader after a miss); admission compares the view's +// frontier — the exclusive txNum end of what its tx can see, so a view with +// frontier N sees txNums < N — against the applied end, under the same lock +// applies take. The Applier handle, held by the SharedDomains +// commit/unwind path, performs the authoritative writes: post-commit +// applies, unwinds, clears. package cache // Cache is the interface for domain caches. @@ -31,7 +49,7 @@ type Cache interface { Put(key []byte, value []byte, txNum uint64) // PutIfAbsent is Put except that a live entry for key is left untouched - // (a stale one is replaced) — for prefetch writers, whose snapshot may + // (a stale one is replaced) — for fill writers, whose read view may // already be superseded by an authoritative Put. PutIfAbsent(key []byte, value []byte, txNum uint64) diff --git a/execution/cache/cache_test.go b/execution/cache/cache_test.go index b860779309d..42e0e740eab 100644 --- a/execution/cache/cache_test.go +++ b/execution/cache/cache_test.go @@ -423,23 +423,23 @@ func TestStateCache_NewStateCache(t *testing.T) { c := closeOnCleanup(t, NewStateCache(10, 20, 30, 40)) require.NotNil(t, c) - // Account, Storage, Code, Commitment should be initialized - assert.NotNil(t, c.GetCache(kv.AccountsDomain)) - assert.NotNil(t, c.GetCache(kv.StorageDomain)) - assert.NotNil(t, c.GetCache(kv.CodeDomain)) + // Account, Storage, Code should be initialized + assert.NotNil(t, c.getCache(kv.AccountsDomain)) + assert.NotNil(t, c.getCache(kv.StorageDomain)) + assert.NotNil(t, c.getCache(kv.CodeDomain)) // Other domains should be nil - assert.Nil(t, c.GetCache(kv.ReceiptDomain)) - assert.Nil(t, c.GetCache(kv.RCacheDomain)) + assert.Nil(t, c.getCache(kv.ReceiptDomain)) + assert.Nil(t, c.getCache(kv.RCacheDomain)) } func TestStateCache_NewDefaultStateCache(t *testing.T) { c := closeOnCleanup(t, NewDefaultStateCache()) require.NotNil(t, c) - assert.NotNil(t, c.GetCache(kv.AccountsDomain)) - assert.NotNil(t, c.GetCache(kv.StorageDomain)) - assert.NotNil(t, c.GetCache(kv.CodeDomain)) + assert.NotNil(t, c.getCache(kv.AccountsDomain)) + assert.NotNil(t, c.getCache(kv.StorageDomain)) + assert.NotNil(t, c.getCache(kv.CodeDomain)) } func TestStateCache_GetPut_Account(t *testing.T) { @@ -449,13 +449,13 @@ func TestStateCache_GetPut_Account(t *testing.T) { value := makeValue(1) // Get non-existent - v, ok := c.Get(kv.AccountsDomain, addr) + v, ok := c.get(kv.AccountsDomain, addr) assert.False(t, ok) assert.Nil(t, v) // Put and Get - c.Put(kv.AccountsDomain, addr, value, 0) - v, ok = c.Get(kv.AccountsDomain, addr) + c.put(kv.AccountsDomain, addr, value, 0) + v, ok = c.get(kv.AccountsDomain, addr) assert.True(t, ok) assert.Equal(t, value, v) } @@ -468,8 +468,8 @@ func TestStateCache_GetPut_Storage(t *testing.T) { key[51] = 1 value := makeValue(1) - c.Put(kv.StorageDomain, key, value, 0) - v, ok := c.Get(kv.StorageDomain, key) + c.put(kv.StorageDomain, key, value, 0) + v, ok := c.get(kv.StorageDomain, key) assert.True(t, ok) assert.Equal(t, value, v) } @@ -480,8 +480,8 @@ func TestStateCache_GetPut_Code(t *testing.T) { addr := makeAddr(1) code := makeCode(1) - c.Put(kv.CodeDomain, addr, code, 0) - v, ok := c.Get(kv.CodeDomain, addr) + c.put(kv.CodeDomain, addr, code, 0) + v, ok := c.get(kv.CodeDomain, addr) assert.True(t, ok) assert.Equal(t, code, v) } @@ -490,8 +490,8 @@ func TestStateCache_GetPut_UnsupportedDomain(t *testing.T) { c := closeOnCleanup(t, NewStateCache(100, 100, 100, 100)) // ReceiptDomain is not supported - c.Put(kv.ReceiptDomain, makeAddr(1), makeValue(1), 0) - v, ok := c.Get(kv.ReceiptDomain, makeAddr(1)) + c.put(kv.ReceiptDomain, makeAddr(1), makeValue(1), 0) + v, ok := c.get(kv.ReceiptDomain, makeAddr(1)) assert.False(t, ok) assert.Nil(t, v) } @@ -500,10 +500,10 @@ func TestStateCache_Delete(t *testing.T) { c := closeOnCleanup(t, NewStateCache(100, 100, 100, 100)) addr := makeAddr(1) - c.Put(kv.AccountsDomain, addr, makeValue(1), 0) - c.Delete(kv.AccountsDomain, addr) + c.put(kv.AccountsDomain, addr, makeValue(1), 0) + c.deleteKey(kv.AccountsDomain, addr) - _, ok := c.Get(kv.AccountsDomain, addr) + _, ok := c.get(kv.AccountsDomain, addr) assert.False(t, ok) } @@ -517,9 +517,9 @@ func TestStateCache_PutEmpty_ThenGet_IsCacheHit(t *testing.T) { key[0] = 0x1d key[51] = 0xa2 - c.Put(kv.StorageDomain, key, nil, 0) + c.put(kv.StorageDomain, key, nil, 0) - v, ok := c.Get(kv.StorageDomain, key) + v, ok := c.get(kv.StorageDomain, key) assert.True(t, ok, "Get after Put(nil) must be a cache hit, not a miss") assert.Empty(t, v, "cached value for a deleted key must be empty") } @@ -532,9 +532,9 @@ func TestStateCache_PutEmptySlice_ThenGet_IsCacheHit(t *testing.T) { key[0] = 0x1d key[51] = 0xa2 - c.Put(kv.StorageDomain, key, []byte{}, 0) + c.put(kv.StorageDomain, key, []byte{}, 0) - v, ok := c.Get(kv.StorageDomain, key) + v, ok := c.get(kv.StorageDomain, key) assert.True(t, ok, "Get after Put([]byte{}) must be a cache hit") assert.Empty(t, v) } @@ -543,21 +543,21 @@ func TestStateCache_Delete_UnsupportedDomain(t *testing.T) { c := closeOnCleanup(t, NewStateCache(100, 100, 100, 100)) // Should not panic - c.Delete(kv.ReceiptDomain, makeAddr(1)) + c.deleteKey(kv.ReceiptDomain, makeAddr(1)) } func TestStateCache_Clear(t *testing.T) { c := closeOnCleanup(t, NewStateCache(100, 100, 100, 100)) - c.Put(kv.AccountsDomain, makeAddr(1), makeValue(1), 0) - c.Put(kv.StorageDomain, makeAddr(2), makeValue(2), 0) - c.Put(kv.CodeDomain, makeAddr(3), makeCode(3), 0) + c.put(kv.AccountsDomain, makeAddr(1), makeValue(1), 0) + c.put(kv.StorageDomain, makeAddr(2), makeValue(2), 0) + c.put(kv.CodeDomain, makeAddr(3), makeCode(3), 0) - c.Clear() + c.clear() - _, ok1 := c.Get(kv.AccountsDomain, makeAddr(1)) - _, ok2 := c.Get(kv.StorageDomain, makeAddr(2)) - _, ok3 := c.Get(kv.CodeDomain, makeAddr(3)) + _, ok1 := c.get(kv.AccountsDomain, makeAddr(1)) + _, ok2 := c.get(kv.StorageDomain, makeAddr(2)) + _, ok3 := c.get(kv.CodeDomain, makeAddr(3)) assert.False(t, ok1) assert.False(t, ok2) @@ -568,10 +568,10 @@ func TestStateCache_GetCache_OutOfBounds(t *testing.T) { c := closeOnCleanup(t, NewStateCache(100, 100, 100, 100)) // Domain >= DomainLen should return nil - cache := c.GetCache(kv.DomainLen) + cache := c.getCache(kv.DomainLen) assert.Nil(t, cache) - cache = c.GetCache(kv.Domain(100)) + cache = c.getCache(kv.Domain(100)) assert.Nil(t, cache) } @@ -641,13 +641,13 @@ func TestStateCache_DomainIsolation(t *testing.T) { storageData := []byte("storage") codeData := []byte{0x60, 0x00, 0x60, 0x00} // valid code - c.Put(kv.AccountsDomain, addr, accountData, 0) - c.Put(kv.StorageDomain, addr, storageData, 0) - c.Put(kv.CodeDomain, addr, codeData, 0) + c.put(kv.AccountsDomain, addr, accountData, 0) + c.put(kv.StorageDomain, addr, storageData, 0) + c.put(kv.CodeDomain, addr, codeData, 0) - v1, ok1 := c.Get(kv.AccountsDomain, addr) - v2, ok2 := c.Get(kv.StorageDomain, addr) - v3, ok3 := c.Get(kv.CodeDomain, addr) + v1, ok1 := c.get(kv.AccountsDomain, addr) + v2, ok2 := c.get(kv.StorageDomain, addr) + v3, ok3 := c.get(kv.CodeDomain, addr) assert.True(t, ok1) assert.True(t, ok2) @@ -890,6 +890,126 @@ func TestDomainCache_PutIfAbsentAtomicWithPut(t *testing.T) { } } +func TestStateCache_AppliedEndLifecycle(t *testing.T) { + b := 1 * datasize.MB + sc := NewStateCache(b, b, b, b) + t.Cleanup(sc.Close) + require.Zero(t, sc.appliedEnd[kv.AccountsDomain]) + + sc.apply(kv.AccountsDomain, makeAddr(1), makeValue(1), 20) + sc.apply(kv.AccountsDomain, makeAddr(2), makeValue(2), 10) + require.Equal(t, uint64(21), sc.appliedEnd[kv.AccountsDomain]) + require.Zero(t, sc.appliedEnd[kv.StorageDomain]) + + sc.unwind(15) + require.Equal(t, uint64(15), sc.appliedEnd[kv.AccountsDomain]) + + sc.clear() + require.Equal(t, uint64(15), sc.appliedEnd[kv.AccountsDomain], + "clear drops entries, not admission history") +} + +func TestStateCache_StaleViewCannotFillAfterDelete(t *testing.T) { + b := 1 * datasize.MB + sc := NewStateCache(b, b, b, b) + t.Cleanup(sc.Close) + + key := makeAddr(1) + stale := makeValue(1) + sc.apply(kv.AccountsDomain, key, stale, 10) + sc.apply(kv.AccountsDomain, key, nil, 20) + _, ok := sc.get(kv.AccountsDomain, key) + require.False(t, ok, "an authoritative deletion must physically remove the entry") + + sc.fillIfFresh(kv.AccountsDomain, key, stale, 10, 11) + _, ok = sc.get(kv.AccountsDomain, key) + require.False(t, ok, "a view older than the deletion must not fill afterward") +} + +func TestStateCache_FileEndViewCannotFillAtAppliedTx(t *testing.T) { + b := 1 * datasize.MB + sc := NewStateCache(b, b, b, b) + t.Cleanup(sc.Close) + + key := makeAddr(1) + stale := makeValue(1) + sc.apply(kv.AccountsDomain, key, nil, 100) + + sc.fillIfFresh(kv.AccountsDomain, key, stale, 99, 100) + _, ok := sc.get(kv.AccountsDomain, key) + require.False(t, ok, "a [0,100) view does not contain the applied tx 100") + + fresh := makeValue(2) + sc.fillIfFresh(kv.AccountsDomain, key, fresh, 100, 101) + got, ok := sc.get(kv.AccountsDomain, key) + require.True(t, ok) + require.Equal(t, fresh, got) +} + +func TestStateCache_ApplyDeleteAtomicWithFill(t *testing.T) { + b := 1 * datasize.MB + sc := NewStateCache(b, b, b, b) + t.Cleanup(sc.Close) + + progressKey := makeAddr(1) + key := makeAddr(2) + value := makeValue(1) + for round := range 20000 { + appliedTxNum := uint64(round*2 + 1) + visibleEnd := appliedTxNum + 1 + sc.apply(kv.AccountsDomain, progressKey, value, appliedTxNum) + + var wg sync.WaitGroup + wg.Add(2) + go func() { + defer wg.Done() + sc.apply(kv.AccountsDomain, key, nil, visibleEnd) + }() + go func() { + defer wg.Done() + sc.fillIfFresh(kv.AccountsDomain, key, value, appliedTxNum, visibleEnd) + }() + wg.Wait() + + _, ok := sc.get(kv.AccountsDomain, key) + require.False(t, ok, "round %d: stale fill survived the authoritative delete", round) + } +} + +func TestStateCache_ApplyCodeDeleteDropsAddrCodeHash(t *testing.T) { + b := 1 * datasize.MB + sc := NewStateCache(b, b, b, b) + t.Cleanup(sc.Close) + + addr := makeAddr(1) + var h [32]byte + h[0] = 0xaa + sc.seedAddrCodeHash(addr, h, 10, 0) + _, ok := sc.getAddrCodeHash(addr) + require.True(t, ok) + + sc.apply(kv.CodeDomain, addr, nil, 20) + _, ok = sc.getAddrCodeHash(addr) + require.False(t, ok, "a code deletion must drop the derived addr→codeHash mapping") +} + +func TestStateCache_AccountDeleteDropsCodeBinding(t *testing.T) { + b := 1 * datasize.MB + sc := NewStateCache(b, b, b, b) + t.Cleanup(sc.Close) + + addr := makeAddr(1) + code := makeCode(1) + + sc.apply(kv.CodeDomain, addr, code, 10) + _, ok := sc.get(kv.CodeDomain, addr) + require.True(t, ok) + + sc.apply(kv.AccountsDomain, addr, nil, 20) + _, ok = sc.get(kv.CodeDomain, addr) + require.False(t, ok, "an account deletion must drop the addr→code binding") +} + // A Delete racing an update-in-place put must not double-subtract the // displaced entry's size: freelru's OnEvict subtracts it for the Remove, and // put's update delta subtracts it again unless the two writers share the @@ -952,3 +1072,104 @@ func TestDomainCache_ClearAtomicWithPut_NoSizeDrift(t *testing.T) { require.Equal(t, wantSize, c.SizeBytes(), "round %d: size accounting drifted", round) } } + +// STATE_CACHE_FILLS=false turns off the admission-gated read fills (apply-only +// mode): the A/B lever for measuring what fills contribute, and the ops kill +// switch. Applies keep working. +func TestStateCacheFillsSwitchDisablesReadFills(t *testing.T) { + t.Setenv("STATE_CACHE_FILLS", "false") + b := 1 * datasize.MB + c := NewStateCache(b, b, b, b) + defer c.Close() + + key := make([]byte, 20) + key[0] = 0xaa + view := c.View(FrontierFunc(func(kv.Domain) (uint64, bool) { return 100, true })) + + view.Fill(kv.AccountsDomain, key, []byte("value"), 10) + _, ok := c.View(nil).Get(kv.AccountsDomain, key) + require.False(t, ok, "fills must be disabled") + + view.SeedAddrCodeHash(key, [32]byte{1}, 10) + _, ok = c.View(nil).GetAddrCodeHash(key) + require.False(t, ok, "mapping seeds must be disabled") + + codeHash := crypto.Keccak256([]byte{0xaa, 1, 2, 3}) + view.FillCodeSize(codeHash, 4, 10) + _, ok = c.View(nil).GetCodeSizeByHash(codeHash) + require.False(t, ok, "content-addressed fills must be disabled too: the switch means no reader writes at all") + + c.Applier().Apply(kv.AccountsDomain, key, []byte("applied"), 20) + got, ok := c.View(nil).Get(kv.AccountsDomain, key) + require.True(t, ok, "applies must keep working") + require.Equal(t, []byte("applied"), got) +} + +// Clearing entries does not rewind canonical state, so the admission frontier +// must survive Clear: a still-live older ReadView must not refill pre-apply +// data into the emptied cache. +func TestStateCache_StaleViewCannotFillAfterClear(t *testing.T) { + b := 1 * datasize.MB + sc := NewStateCache(b, b, b, b) + t.Cleanup(sc.Close) + + key := makeAddr(1) + oldView := sc.View(FrontierFunc(func(kv.Domain) (uint64, bool) { return 11, true })) + + sc.Applier().Apply(kv.AccountsDomain, key, nil, 20) // canonical delete + sc.Applier().Clear() + + oldView.Fill(kv.AccountsDomain, key, []byte("pre-delete"), 10) + _, ok := sc.View(nil).Get(kv.AccountsDomain, key) + require.False(t, ok, "a pre-apply view must not resurrect the deleted value through Clear") + + freshView := sc.View(FrontierFunc(func(kv.Domain) (uint64, bool) { return 21, true })) + freshView.Fill(kv.AccountsDomain, key, []byte("current"), 20) + got, ok := sc.View(nil).Get(kv.AccountsDomain, key) + require.True(t, ok, "a view at the applied frontier must still fill after Clear") + require.Equal(t, []byte("current"), got) +} + +// An addr-keyed code entry derives from the account: an account deletion drops +// it without advancing the code frontier, so code-fill admission must check the +// accounts frontier too — otherwise a pre-deletion view refills the dead code. +func TestStateCache_AccountDeletionGatesStaleCodeFill(t *testing.T) { + b := 1 * datasize.MB + c := NewStateCache(b, b, b, b) + t.Cleanup(c.Close) + addr, code := makeAddr(1), makeCode(1) + other, otherCode := makeAddr(2), makeCode(2) + + c.Applier().Apply(kv.CodeDomain, addr, code, 100) + c.Applier().Apply(kv.AccountsDomain, addr, nil, 200) + + stale := c.View(FrontierFunc(func(kv.Domain) (uint64, bool) { return 101, true })) + stale.Fill(kv.CodeDomain, addr, code, 100) + _, ok := c.View(nil).Get(kv.CodeDomain, addr) + require.False(t, ok, "code of a deleted account must not be refillable from a pre-deletion view") + + fresh := c.View(FrontierFunc(func(d kv.Domain) (uint64, bool) { + if d == kv.AccountsDomain { + return 201, true + } + return 101, true + })) + fresh.Fill(kv.CodeDomain, other, otherCode, 100) + _, ok = c.View(nil).Get(kv.CodeDomain, other) + require.True(t, ok, "unrelated code fills from a current view must stay admitted") +} + +// An apply-only cache (STATE_CACHE_FILLS=false) has no fill for a lowered +// frontier to poison; wire-up code keys the aggregator forbid on this. +func TestApplyOnlyCacheReportsFillsDisabled(t *testing.T) { + t.Setenv("STATE_CACHE_FILLS", "false") + b := 1 * datasize.MB + c := NewStateCache(b, b, b, b) + t.Cleanup(c.Close) + require.False(t, c.FillsEnabled()) + + t.Setenv("STATE_CACHE_FILLS", "true") + c2 := NewStateCache(b, b, b, b) + t.Cleanup(c2.Close) + require.True(t, c2.FillsEnabled()) +} diff --git a/execution/cache/code_cache.go b/execution/cache/code_cache.go index 17f029212be..45f7297113e 100644 --- a/execution/cache/code_cache.go +++ b/execution/cache/code_cache.go @@ -559,7 +559,9 @@ func (c *CodeCache) putCodeSizeByCodeHashLocked(codeHash []byte, size int, hcs, // Delete removes the address → code mapping for addr. func (c *CodeCache) Delete(addr []byte) { + c.addrBindMu.Lock() c.addrToHash.Remove(common.BytesToAddress(addr)) + c.addrBindMu.Unlock() } // Clear removes every layer, resets accounting, and starts a new coherence diff --git a/execution/cache/state_cache.go b/execution/cache/state_cache.go index df4a86cc31a..276871b07cc 100644 --- a/execution/cache/state_cache.go +++ b/execution/cache/state_cache.go @@ -18,14 +18,16 @@ package cache import ( "bytes" + "math" "strings" + "sync" "github.com/c2h5oh/datasize" + "github.com/erigontech/erigon/common/crypto" "github.com/erigontech/erigon/common/dbg" "github.com/erigontech/erigon/common/log/v3" "github.com/erigontech/erigon/db/kv" - "github.com/erigontech/erigon/execution/commitment/commitmentdb" ) const ( @@ -48,10 +50,22 @@ const ( // Uses an array indexed by kv.Domain. Only Account, Storage, and Code domains // are supported; other indices are nil. // +// StateCache itself exposes no data methods: reads and admission-gated fills +// go through a ReadView bound to one tx's read view, committed updates through +// the Applier handle (see view.go). +// // Account and Storage use GenericCache. // Code uses CodeCache (two-level for deduplication). type StateCache struct { caches [kv.DomainLen]Cache + // admissionMu makes Apply's frontier advance + cache mutation atomic + // against concurrent read-fills, which recheck freshness under RLock. + admissionMu sync.RWMutex + appliedEnd [kv.DomainLen]uint64 + // disableFills (STATE_CACHE_FILLS=false) turns off every reader fill + // (including the content-addressed ones), leaving applies as the only + // writer ("apply-only" mode) — an A/B lever and an operational kill switch. + disableFills bool } // NewStateCache creates a new StateCache with the specified byte capacities. @@ -61,9 +75,16 @@ type StateCache struct { func NewStateCache(accountBytes, storageBytes, codeBytes, addrBytes datasize.ByteSize) *StateCache { mode := stateCacheModeFromEnv() sc := &StateCache{} + if !dbg.EnvBool("STATE_CACHE_FILLS", true) { + sc.disableFills = true + log.Info("[cache] STATE_CACHE_FILLS=false — read fills disabled, only post-commit applies populate the cache") + } sc.caches[kv.AccountsDomain] = newDomainCacheBytes(accountBytes, avgAccountEntryBytes, mode) sc.caches[kv.StorageDomain] = newDomainCacheBytes(storageBytes, avgStorageEntryBytes, mode) sc.caches[kv.CodeDomain] = NewCodeCache(codeBytes, addrBytes) + // CommitmentDomain deliberately gets no cache: commitment data lives in the + // BranchCache, and the nil slot short-circuits every StateCache path for it + // (including writes of commitmentdb.KeyCommitmentState). return sc } @@ -95,9 +116,9 @@ func newDomainCacheBytes(capacityBytes datasize.ByteSize, avgBytes uint32, mode } // NewDefaultStateCache creates a new StateCache with the production byte budgets -// (Account 1GB, Storage 150MB, Code 512MB, Addr 16MB). Test/CLI harnesses that -// build many short-lived ExecModules pass an explicit small cache instead — via -// ExecModuleTester, or via ethconfig.Config.StateCacheBudget for the eth.New path. +// (Account 1GB, Storage 150MB, Code 512MB, Addr 16MB). Harnesses that build +// many short-lived ExecModules set a small ethconfig.Config.StateCacheBudget +// instead. func NewDefaultStateCache() *StateCache { return NewStateCache( DefaultAccountCacheBytes, @@ -107,10 +128,10 @@ func NewDefaultStateCache() *StateCache { ) } -// Get retrieves data for the given domain and key. -// Returns (value, true) on cache hit — including (nil, true) for deleted keys — +// get retrieves data for the given domain and key. +// Returns (value, true) on cache hit — including (nil, true) for cached negatives — // and (nil, false) on cache miss. -func (c *StateCache) Get(domain kv.Domain, key []byte) ([]byte, bool) { +func (c *StateCache) get(domain kv.Domain, key []byte) ([]byte, bool) { cache := c.caches[domain] if cache == nil { return nil, false @@ -118,9 +139,9 @@ func (c *StateCache) Get(domain kv.Domain, key []byte) ([]byte, bool) { return cache.Get(key) } -// GetWithTxNum is Get plus the txNum the cached value reflects, so the read +// getWithTxNum is get plus the txNum the cached value reflects, so the read // path can bound a hit by step against an in-flight unwind's maxStep. -func (c *StateCache) GetWithTxNum(domain kv.Domain, key []byte) ([]byte, uint64, bool) { +func (c *StateCache) getWithTxNum(domain kv.Domain, key []byte) ([]byte, uint64, bool) { cache := c.caches[domain] if cache == nil { return nil, 0, false @@ -128,7 +149,7 @@ func (c *StateCache) GetWithTxNum(domain kv.Domain, key []byte) ([]byte, uint64, return cache.GetWithTxNum(key) } -// GetCodeByHash retrieves code bytes by their Ethereum codeHash (keccak256), +// getCodeByHash retrieves code bytes by their Ethereum codeHash (keccak256), // bypassing the addr-keyed CodeDomain lookup. Returns (nil, false) on miss or // when the code domain cache is not a CodeCache (defensive fallback). // @@ -136,7 +157,7 @@ func (c *StateCache) GetWithTxNum(domain kv.Domain, key []byte) ([]byte, uint64, // for EXTCODESIZE / EXTCODEHASH / CALL targets. Lets many-addrs-one-code // patterns (proxies, factory clones, ERC-20 holders) share a single codeHashToCode // entry. -func (c *StateCache) GetCodeByHash(codeHash []byte) ([]byte, bool) { +func (c *StateCache) getCodeByHash(codeHash []byte) ([]byte, bool) { cc, ok := c.caches[kv.CodeDomain].(*CodeCache) if !ok { return nil, false @@ -144,35 +165,10 @@ func (c *StateCache) GetCodeByHash(codeHash []byte) ([]byte, bool) { return cc.GetByCodeHash(codeHash) } -// PutCodeWithHash stores code populating both the addr-keyed path and the -// codeHash-keyed codeHashToCode layer. Callers should prefer this over Put when they -// have the codeHash from the account record — avoids a redundant keccak. -func (c *StateCache) PutCodeWithHash(addr, code, codeHash []byte, txNum uint64) { - c.putCodeWithHash(addr, code, codeHash, txNum, true) -} - -// PutCodeWithHashIfAbsent is PutCodeWithHash with if-absent binding semantics -// (see Cache.PutIfAbsent). -func (c *StateCache) PutCodeWithHashIfAbsent(addr, code, codeHash []byte, txNum uint64) { - c.putCodeWithHash(addr, code, codeHash, txNum, false) -} - -func (c *StateCache) putCodeWithHash(addr, code, codeHash []byte, txNum uint64, overwrite bool) { - cc, ok := c.caches[kv.CodeDomain].(*CodeCache) - if !ok { - return - } - if overwrite { - cc.PutWithCodeHash(addr, bytes.Clone(code), codeHash, txNum) - } else { - cc.PutWithCodeHashIfAbsent(addr, bytes.Clone(code), codeHash, txNum) - } -} - -// GetCodeSizeByHash returns the size of code by its Ethereum codeHash +// getCodeSizeByHash returns the size of code by its Ethereum codeHash // without loading the bytes. Returns (0, false) when the size-only layer // is not populated for this hash. -func (c *StateCache) GetCodeSizeByHash(codeHash []byte) (int, bool) { +func (c *StateCache) getCodeSizeByHash(codeHash []byte) (int, bool) { cc, ok := c.caches[kv.CodeDomain].(*CodeCache) if !ok { return 0, false @@ -180,10 +176,10 @@ func (c *StateCache) GetCodeSizeByHash(codeHash []byte) (int, bool) { return cc.GetCodeSizeByCodeHash(codeHash) } -// PutCodeSizeByHash records the code size for a given codeHash. Useful when +// putCodeSizeByHash records the code size for a given codeHash. Useful when // the caller has the size in hand (e.g. from an account-domain probe that // resolved a sibling addr to the same code) but doesn't have the bytes. -func (c *StateCache) PutCodeSizeByHash(codeHash []byte, size int, txNum uint64) { +func (c *StateCache) putCodeSizeByHash(codeHash []byte, size int, txNum uint64) { cc, ok := c.caches[kv.CodeDomain].(*CodeCache) if !ok { return @@ -191,9 +187,16 @@ func (c *StateCache) PutCodeSizeByHash(codeHash []byte, size int, txNum uint64) cc.PutCodeSizeByCodeHash(codeHash, size, txNum) } -// GetAddrCodeHash returns the Ethereum codeHash for addr without an +// FillsEnabled reports whether reader fills are active (STATE_CACHE_FILLS). +// Wire-up code uses it to decide whether the backing aggregator must forbid +// visibility lowering: fill admission relies on view frontiers never +// decreasing, and apply-only caches have nothing for a lowered frontier to +// poison. +func (c *StateCache) FillsEnabled() bool { return !c.disableFills } + +// getAddrCodeHash returns the Ethereum codeHash for addr without an // account-domain round-trip. The hash is zero when ok is false. -func (c *StateCache) GetAddrCodeHash(addr []byte) ([32]byte, bool) { +func (c *StateCache) getAddrCodeHash(addr []byte) ([32]byte, bool) { cc, ok := c.caches[kv.CodeDomain].(*CodeCache) if !ok { return [32]byte{}, false @@ -201,19 +204,23 @@ func (c *StateCache) GetAddrCodeHash(addr []byte) ([32]byte, bool) { return cc.GetAddrCodeHash(addr) } -// PutAddrCodeHash records a committed-state addr → codeHash mapping in the -// addr-keyed LRU. An existing live mapping remains authoritative until -// DeleteAddrCodeHash invalidates it; an unwind-stale mapping can be replaced. -func (c *StateCache) PutAddrCodeHash(addr []byte, h [32]byte, txNum uint64) { +// seedAddrCodeHash conditionally records an addr → codeHash mapping. +// The mapping derives from an account record, so admission checks the accounts +// frontier even though the mapping lives in the code cache. +func (c *StateCache) seedAddrCodeHash(addr []byte, h [32]byte, txNum, visibleEnd uint64) { cc, ok := c.caches[kv.CodeDomain].(*CodeCache) if !ok { return } + c.admissionMu.RLock() + defer c.admissionMu.RUnlock() + if visibleEnd < c.appliedEnd[kv.AccountsDomain] { + return + } cc.PutAddrCodeHash(addr, h, txNum) } -// DeleteAddrCodeHash removes the mapping when its account record is invalidated. -func (c *StateCache) DeleteAddrCodeHash(addr []byte) { +func (c *StateCache) deleteAddrCodeHash(addr []byte) { cc, ok := c.caches[kv.CodeDomain].(*CodeCache) if !ok { return @@ -221,34 +228,66 @@ func (c *StateCache) DeleteAddrCodeHash(addr []byte) { cc.DeleteAddrCodeHash(addr) } -// Put stores data for the given domain and key, stamped with the txNum the -// value reflects (for txNum/epoch unwind invalidation). -func (c *StateCache) Put(domain kv.Domain, key []byte, value []byte, txNum uint64) { - c.put(domain, key, value, txNum, true) -} - -// PutIfAbsent is Put with if-absent semantics (see Cache.PutIfAbsent). -func (c *StateCache) PutIfAbsent(domain kv.Domain, key []byte, value []byte, txNum uint64) { - c.put(domain, key, value, txNum, false) +// put stores data for the given domain and key, stamped with the txNum the +// value reflects (for txNum/epoch unwind invalidation). It bypasses fill +// admission: committed updates go through Applier.Apply, read fills through +// ReadView.Fill. +func (c *StateCache) put(domain kv.Domain, key []byte, value []byte, txNum uint64) { + cache := c.caches[domain] + if cache == nil { + return + } + cache.Put(key, bytes.Clone(value), txNum) } -func (c *StateCache) put(domain kv.Domain, key []byte, value []byte, txNum uint64, overwrite bool) { +// fillIfFresh conditionally inserts an accounts or storage value read from a +// read view without replacing an authoritative entry. Negatives use the view's +// last included txNum. Code goes through fillCodeIfFresh. +func (c *StateCache) fillIfFresh(domain kv.Domain, key []byte, value []byte, readTxNum, visibleEnd uint64) { cache := c.caches[domain] if cache == nil { return } - if domain == kv.CommitmentDomain && bytes.Equal(key, commitmentdb.KeyCommitmentState) { + // Clone outside the lock: a rejected fill wastes one copy (rare), but + // Apply's write lock never waits on a fill's memcpy. + cloned := bytes.Clone(value) + if len(value) == 0 { + readTxNum = 0 + if visibleEnd > 0 { + readTxNum = visibleEnd - 1 + } + } + c.admissionMu.RLock() + defer c.admissionMu.RUnlock() + if visibleEnd < c.appliedEnd[domain] { return } - if overwrite { - cache.Put(key, bytes.Clone(value), txNum) - } else { - cache.PutIfAbsent(key, bytes.Clone(value), txNum) + cache.PutIfAbsent(key, cloned, readTxNum) +} + +// fillCodeIfFresh is fillIfFresh for the code domain. An addr-keyed code entry +// derives from the account — an account deletion drops it without advancing the +// code frontier — so admission also checks the accounts frontier. Code +// negatives are not cached here: "no code" is cached at the addr→codeHash +// mapping instead (the zero-hash sentinel seeded by SeedAddrCodeHash). +func (c *StateCache) fillCodeIfFresh(key []byte, value []byte, readTxNum, visibleEnd, accountsVisibleEnd uint64) { + codeCache, ok := c.caches[kv.CodeDomain].(*CodeCache) + if !ok || len(value) == 0 { + return } + codeHash := crypto.Keccak256(value) + cloned := bytes.Clone(value) + c.admissionMu.RLock() + defer c.admissionMu.RUnlock() + if visibleEnd < c.appliedEnd[kv.CodeDomain] || accountsVisibleEnd < c.appliedEnd[kv.AccountsDomain] { + return + } + codeCache.PutWithCodeHashIfAbsent(key, cloned, codeHash, readTxNum) } -// Delete removes the data for the given domain and key. -func (c *StateCache) Delete(domain kv.Domain, key []byte) { +// deleteKey removes the data for the given domain and key. Authoritative +// deletions go through apply, which also advances the fill-admission frontier. +func (c *StateCache) deleteKey(domain kv.Domain, key []byte) { cache := c.caches[domain] if cache == nil { return @@ -256,8 +295,72 @@ func (c *StateCache) Delete(domain kv.Domain, key []byte) { cache.Delete(key) } -// Clear removes all mutable entries from all caches. -func (c *StateCache) Clear() { +// apply makes a committed domain update authoritative for subsequent fills. +func (c *StateCache) apply(domain kv.Domain, key, value []byte, txNum uint64) { + cache := c.caches[domain] + if cache == nil { + return + } + var codeHash []byte + if domain == kv.CodeDomain && len(value) > 0 { + // Clone before hashing so the stored bytes and their codeHash cannot + // diverge if the caller reuses its buffer. + value = bytes.Clone(value) + codeHash = crypto.Keccak256(value) + } + + c.admissionMu.Lock() + defer c.admissionMu.Unlock() + c.noteApplied(domain, txNum) + + switch domain { + case kv.AccountsDomain: + putOrDelete(cache, key, value, txNum) + c.deleteAddrCodeHash(key) + if len(value) == 0 { + // SharedDomains pairs an account deletion with a code-domain apply; + // that paired apply is what advances the code frontier — this cascade + // only drops the entry. Code-fill admission also checks the accounts + // frontier (fillCodeIfFresh), so the cache holds even for a caller + // that does not pair the deletes. + c.deleteKey(kv.CodeDomain, key) + } + case kv.CodeDomain: + if len(value) == 0 { + cache.Delete(key) + c.deleteAddrCodeHash(key) + } else if codeCache, ok := cache.(*CodeCache); ok { + codeCache.PutWithCodeHash(key, value, codeHash, txNum) + } + default: + putOrDelete(cache, key, value, txNum) + } +} + +func putOrDelete(cache Cache, key, value []byte, txNum uint64) { + if len(value) == 0 { + cache.Delete(key) + return + } + cache.Put(key, bytes.Clone(value), txNum) +} + +func (c *StateCache) noteApplied(domain kv.Domain, txNum uint64) { + end := txNum + if end < math.MaxUint64 { + end++ + } + if end > c.appliedEnd[domain] { + c.appliedEnd[domain] = end + } +} + +// clear removes all mutable entries from all caches. The admission frontier +// survives: clearing drops entries, it does not rewind canonical state, and a +// zeroed frontier would let a still-live older ReadView refill pre-apply data. +func (c *StateCache) clear() { + c.admissionMu.Lock() + defer c.admissionMu.Unlock() for _, cache := range c.caches { if cache != nil { cache.Clear() @@ -275,22 +378,30 @@ func (c *StateCache) Close() { } } -// Unwind invalidates, across all caches, entries reflecting state above +// unwind invalidates, across all caches, entries reflecting state above // unwindToTxNum on a now-dead fork. Diffset-free and O(1): every cache (the // GenericCaches and the CodeCache, all layers) bumps an epoch + lowers a floor // and drops stale entries lazily on read. This is the sole cache-invalidation // path on unwind — the executor never touches the cache during forward execution. -func (c *StateCache) Unwind(unwindToTxNum uint64) { +func (c *StateCache) unwind(unwindToTxNum uint64) { + c.admissionMu.Lock() + defer c.admissionMu.Unlock() for _, cache := range c.caches { if cache != nil { cache.Unwind(unwindToTxNum) } } + for i := range c.appliedEnd { + c.appliedEnd[i] = min(c.appliedEnd[i], unwindToTxNum) + } +} + +// Caches reports whether the given domain has a cache attached. +func (c *StateCache) Caches(domain kv.Domain) bool { + return domain < kv.DomainLen && c.caches[domain] != nil } -// GetCache returns the cache for the given domain. -// Returns nil if the domain is not supported. -func (c *StateCache) GetCache(domain kv.Domain) Cache { +func (c *StateCache) getCache(domain kv.Domain) Cache { if domain >= kv.DomainLen { return nil } diff --git a/execution/cache/view.go b/execution/cache/view.go new file mode 100644 index 00000000000..0de781920d6 --- /dev/null +++ b/execution/cache/view.go @@ -0,0 +1,197 @@ +// Copyright 2026 The Erigon Authors +// This file is part of Erigon. +// +// Erigon is free software: you can redistribute it and/or modify +// it under the terms of the GNU Lesser General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Erigon is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Lesser General Public License for more details. +// +// You should have received a copy of the GNU Lesser General Public License +// along with Erigon. If not, see . + +package cache + +import ( + "github.com/erigontech/erigon/db/kv" +) + +// Frontier reports the exclusive txNum bound of one transaction's read view +// per domain. ok=false means the view has no exact frontier for the domain +// (remote or history-disabled backends, dependency-clamped values views); +// fills sourced from such a view are skipped. +// +// An implementation may report a stale-low bound only for a coherent, +// monotonically extended view — then it merely over-rejects fills. A view +// serving mixed-age reads has no exact frontier and must answer ok=false. +// Overstating what the tx can currently read is never safe: admission rests +// on that. +type Frontier interface { + DomainVisibleEnd(domain kv.Domain) (visibleEnd uint64, ok bool) +} + +// FrontierFunc adapts a function to the Frontier interface. +type FrontierFunc func(domain kv.Domain) (visibleEnd uint64, ok bool) + +func (f FrontierFunc) DomainVisibleEnd(domain kv.Domain) (uint64, bool) { return f(domain) } + +// ReadView is the read-and-fill handle of a StateCache, bound to one +// transaction's read view: values filled through it are vouched for by that +// view's frontier alone, and it must not outlive the transaction. A nil +// frontier disables the admission-gated fills (Fill, SeedAddrCodeHash); +// FillCodeSize is content-addressed and works on any view. The zero value is +// inert: reads miss, fills no-op. +// +// A ReadView does not isolate reads: the cache holds latest-applied state, so +// a hit can be newer than the view — the same direction the exec overlay +// already serves. In the forward direction the cache's invariant is +// monotonicity (content never regresses behind the applied frontier), +// enforced on the fill side; unwinds invalidate by epoch and floor. +// Snapshot-isolated caching is kvcache's job (node/shards). +type ReadView struct { + c *StateCache + frontier Frontier +} + +// View creates a ReadView vouched for by f. A nil f disables admission-gated fills. +func (c *StateCache) View(f Frontier) ReadView { return ReadView{c: c, frontier: f} } + +// Get retrieves data for the given domain and key. +// Returns (value, true) on cache hit — including (nil, true) for cached negatives — +// and (nil, false) on cache miss. +func (v ReadView) Get(domain kv.Domain, key []byte) ([]byte, bool) { + if v.c == nil { + return nil, false + } + return v.c.get(domain, key) +} + +// GetWithTxNum is Get plus the txNum the cached value reflects, so the read +// path can bound a hit by step against an in-flight unwind's maxStep. +func (v ReadView) GetWithTxNum(domain kv.Domain, key []byte) ([]byte, uint64, bool) { + if v.c == nil { + return nil, 0, false + } + return v.c.getWithTxNum(domain, key) +} + +// GetCodeByHash retrieves code bytes by their Ethereum codeHash (keccak256), +// bypassing the addr-keyed CodeDomain lookup. Returns (nil, false) on miss. +func (v ReadView) GetCodeByHash(codeHash []byte) ([]byte, bool) { + if v.c == nil { + return nil, false + } + return v.c.getCodeByHash(codeHash) +} + +// GetCodeSizeByHash returns the cached code length for codeHash. +func (v ReadView) GetCodeSizeByHash(codeHash []byte) (int, bool) { + if v.c == nil { + return 0, false + } + return v.c.getCodeSizeByHash(codeHash) +} + +// GetAddrCodeHash returns the Ethereum codeHash for addr without an +// account-domain round-trip. The hash is zero when ok is false. +func (v ReadView) GetAddrCodeHash(addr []byte) ([32]byte, bool) { + if v.c == nil { + return [32]byte{}, false + } + return v.c.getAddrCodeHash(addr) +} + +// CanFill reports whether this view carries a frontier, i.e. Fill and +// SeedAddrCodeHash can admit values through it. +func (v ReadView) CanFill() bool { return v.c != nil && v.frontier != nil } + +// Fill offers a value read from this view without replacing an authoritative +// entry. Admission is checked against the view's frontier for the domain; +// views without an exact frontier skip the fill. A code fill also checks the +// accounts frontier: an addr-keyed code entry derives from the account — an +// account deletion drops it without advancing the code frontier — so a view +// that predates the deletion must not refill it (mirrors SeedAddrCodeHash). +func (v ReadView) Fill(domain kv.Domain, key []byte, value []byte, readTxNum uint64) { + if v.c == nil || v.c.disableFills || v.frontier == nil { + return + } + visibleEnd, ok := v.frontier.DomainVisibleEnd(domain) + if !ok { + return + } + if domain == kv.CodeDomain { + accountsEnd, ok := v.frontier.DomainVisibleEnd(kv.AccountsDomain) + if !ok { + return + } + v.c.fillCodeIfFresh(key, value, readTxNum, visibleEnd, accountsEnd) + return + } + v.c.fillIfFresh(domain, key, value, readTxNum, visibleEnd) +} + +// SeedAddrCodeHash offers an addr → codeHash mapping derived from an account +// record read from this view, so admission checks the accounts frontier even +// though the mapping lives in the code cache. +func (v ReadView) SeedAddrCodeHash(addr []byte, h [32]byte, txNum uint64) { + if v.c == nil || v.c.disableFills || v.frontier == nil { + return + } + visibleEnd, ok := v.frontier.DomainVisibleEnd(kv.AccountsDomain) + if !ok { + return + } + v.c.seedAddrCodeHash(addr, h, txNum, visibleEnd) +} + +// FillCodeSize records the code length for codeHash. Content-addressed and +// immutable for a given hash, so it needs no admission and no frontier — but +// it is still a reader write, so the fills switch covers it. +func (v ReadView) FillCodeSize(codeHash []byte, size int, txNum uint64) { + if v.c == nil || v.c.disableFills { + return + } + v.c.putCodeSizeByHash(codeHash, size, txNum) +} + +// Applier is the authoritative writer handle of a StateCache: post-commit +// applies, unwinds and clears. It belongs to the authoritative mutation path +// — the SharedDomains commit/unwind code. The zero value is a no-op. +type Applier struct { + c *StateCache +} + +// Applier creates the writer handle. +func (c *StateCache) Applier() Applier { return Applier{c: c} } + +// Apply makes a committed domain update authoritative for subsequent fills: +// it advances the domain's applied frontier and mutates the cache in the same +// critical section, so a fill from an older read view can never land on top. +func (a Applier) Apply(domain kv.Domain, key, value []byte, txNum uint64) { + if a.c == nil { + return + } + a.c.apply(domain, key, value, txNum) +} + +// Unwind invalidates, across all caches, entries reflecting state above +// unwindToTxNum on a now-dead fork, and lowers the applied frontiers. +func (a Applier) Unwind(unwindToTxNum uint64) { + if a.c == nil { + return + } + a.c.unwind(unwindToTxNum) +} + +// Clear removes all mutable entries from all caches. The applied frontiers +// survive — clearing is not a canonical-state rewind (that is Unwind). +func (a Applier) Clear() { + if a.c == nil { + return + } + a.c.clear() +} diff --git a/execution/engineapi/engine_api_cache_budget_test.go b/execution/engineapi/engine_api_cache_budget_test.go new file mode 100644 index 00000000000..9d78047d7d1 --- /dev/null +++ b/execution/engineapi/engine_api_cache_budget_test.go @@ -0,0 +1,57 @@ +// Copyright 2026 The Erigon Authors +// This file is part of Erigon. +// +// Erigon is free software: you can redistribute it and/or modify +// it under the terms of the GNU Lesser General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Erigon is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Lesser General Public License for more details. +// +// You should have received a copy of the GNU Lesser General Public License +// along with Erigon. If not, see . + +package engineapi_test + +import ( + "testing" + + "github.com/stretchr/testify/require" + + "github.com/erigontech/erigon/common/cachebudget" + "github.com/erigontech/erigon/common/log/v3" + "github.com/erigontech/erigon/common/testlog" + "github.com/erigontech/erigon/execution/engineapi/engineapitester" +) + +// Node close must return every cache reservation to the process-wide envelope; +// otherwise each per-fixture node in a test binary leaks its slice of +// cachebudget.Global and later caches size against phantom concurrency. +func TestEngineApiNodeCloseReleasesCacheBudget(t *testing.T) { + if testing.Short() { + t.Skip("long-running test") + } + ctx := t.Context() + logger := testlog.Logger(t, log.LvlError) + genesis, coinbaseKey, err := engineapitester.DefaultEngineApiTesterGenesis() + require.NoError(t, err) + + usedBefore := cachebudget.Global.Used() + eat, err := engineapitester.InitialiseEngineApiTester(ctx, engineapitester.EngineApiTesterInitArgs{ + Logger: logger, + DataDir: t.TempDir(), + Genesis: genesis, + CoinbaseKey: coinbaseKey, + }) + require.NoError(t, err) + t.Cleanup(func() { _ = eat.Close() }) + require.Greater(t, cachebudget.Global.Used(), usedBefore, + "a running node must hold cache-budget reservations") + + require.NoError(t, eat.Close()) + require.Equal(t, usedBefore, cachebudget.Global.Used(), + "node close must release every cache-budget reservation") +} diff --git a/execution/exec/blocks_read_ahead.go b/execution/exec/blocks_read_ahead.go index baa7cd78823..87c44d3664b 100644 --- a/execution/exec/blocks_read_ahead.go +++ b/execution/exec/blocks_read_ahead.go @@ -11,7 +11,6 @@ import ( "golang.org/x/sync/errgroup" "github.com/erigontech/erigon/common" - "github.com/erigontech/erigon/common/crypto" "github.com/erigontech/erigon/common/dbg" "github.com/erigontech/erigon/common/length" "github.com/erigontech/erigon/common/log/v3" @@ -73,70 +72,36 @@ func (bra *BlockReadAheader) SetStateCache(sc *cache.StateCache) { bra.stateCache = sc } -// cachePopulatingGetter wraps a kv.TemporalGetter and writes successful -// reads through to a cache.StateCache as a side effect. Used by warmBody -// to make read-ahead prefetches populate the same in-process cache layer -// that SharedDomains.GetLatest consults — eliminating the file-accessor -// stack cost on the EVM's first touch of any prefetched address. +// cachePopulatingGetter wraps a kv.TemporalGetter and fills a StateCache +// ReadView as a side effect. Used by warmBody to make read-ahead prefetches +// populate the same in-process cache layer that SharedDomains.GetLatest +// consults — eliminating the file-accessor stack cost on the EVM's first +// touch of any prefetched address. // -// For the CodeDomain the wrapper also populates the codeHashToCode -// (codeHash→bytes) + size-cache layers via PutCodeWithHashIfAbsent, keyed by -// the code's own keccak hash so every cached pair is self-consistent. +// Code reads also populate the content-addressed and size-cache layers. type cachePopulatingGetter struct { - g kv.TemporalGetter - sc *cache.StateCache - progress func(kv.Domain) uint64 // domain progress source for stamping negative fills - stepSize uint64 // for the read txNum upper bound (last txNum of the read's step) + kv.TemporalGetter + view cache.ReadView + stepSize uint64 // for the read txNum upper bound (last txNum of the read's step) } -func newCachePopulatingGetter(tx kv.TemporalTx, sc *cache.StateCache) *cachePopulatingGetter { - debug := tx.Debug() - return &cachePopulatingGetter{g: tx, sc: sc, progress: debug.DomainProgress, stepSize: debug.StepSize()} +func readAheadGetter(ttx kv.TemporalTx, sc *cache.StateCache) kv.TemporalGetter { + if sc == nil { + return ttx + } + debug := ttx.Debug() + return &cachePopulatingGetter{TemporalGetter: ttx, view: sc.View(debug), stepSize: debug.StepSize()} } func (cpg *cachePopulatingGetter) GetLatest(name kv.Domain, k []byte) ([]byte, kv.Step, error) { - v, step, err := cpg.g.GetLatest(name, k) - if err == nil && cpg.sc != nil { - // If-absent writes only: this runs in a fire-and-forget goroutine over a - // committed snapshot, so an unconditional Put racing an FCU flush's - // cache-apply could replace the flushed value with the pre-flush one. - if name == kv.CodeDomain && len(v) > 0 { - // Key the content cache by the code's OWN hash, never a separately - // read account codeHash: under parallel/speculative exec that hash - // can be skewed or cross-account, and a (hash, code) pair that - // doesn't satisfy keccak(code)==hash poisons every account sharing - // the hash. keccak(v) makes each entry self-consistent. - cpg.sc.PutCodeWithHashIfAbsent(k, v, crypto.Keccak256(v), (uint64(step)+1)*cpg.stepSize-1) - } else { - // Cache including nil/empty results: a probe returning no - // bytes is a valid negative answer (missing account, empty - // storage slot; empty code lands here too but CodeCache drops - // zero-length puts) and caching it lets repeated probes - // skip the file accessor stack. Mirrors revm's CacheAccount - // { account: None, status: LoadedNotExisting } pattern. - // Stamp with an upper bound on the value's write txNum (last txNum - // of the step it came from) so unwind invalidation is correct. A - // negative carries no step — stamp it with the domain's progress - // at observation time so any unwind drops it (as the SD read-fill - // does). - readTxNum := (uint64(step)+1)*cpg.stepSize - 1 - if len(v) == 0 && name != kv.CodeDomain && cpg.sc.GetCache(name) != nil { - readTxNum = cpg.progress(name) - } - cpg.sc.PutIfAbsent(name, k, v, readTxNum) - } + v, step, err := cpg.TemporalGetter.GetLatest(name, k) + if err == nil { + readTxNum := (uint64(step)+1)*cpg.stepSize - 1 + cpg.view.Fill(name, k, v, readTxNum) } return v, step, err } -func (cpg *cachePopulatingGetter) HasPrefix(name kv.Domain, prefix []byte) ([]byte, []byte, bool, error) { - return cpg.g.HasPrefix(name, prefix) -} - -func (cpg *cachePopulatingGetter) StepsInFiles(entitySet ...kv.Domain) kv.Step { - return cpg.g.StepsInFiles(entitySet...) -} - func (bra *BlockReadAheader) AddHeaderAndBody(ctx context.Context, db kv.RoDB, header *types.Header, body *types.Body) { blockHash := header.Hash() bra.headers.Add(blockHash, header) @@ -239,11 +204,7 @@ func (bra *BlockReadAheader) warmBody(ctx context.Context, db kv.RoDB, header *t if !ok { return nil } - var getter kv.TemporalGetter = ttx - if bra.stateCache != nil { - getter = newCachePopulatingGetter(ttx, bra.stateCache) - } - stateReader := state.NewReaderV3(getter) + stateReader := state.NewReaderV3(readAheadGetter(ttx, bra.stateCache)) for idx := workerStart; idx < workerEnd; idx++ { select { @@ -309,13 +270,7 @@ func (bra *BlockReadAheader) warmBody(ctx context.Context, db kv.RoDB, header *t if !ok { return nil } - var getter kv.TemporalGetter = ttx - var cpg *cachePopulatingGetter - if bra.stateCache != nil { - cpg = newCachePopulatingGetter(ttx, bra.stateCache) - getter = cpg - } - stateReader := state.NewReaderV3(getter) + stateReader := state.NewReaderV3(readAheadGetter(ttx, bra.stateCache)) for txIdx := workerStart; txIdx < workerEnd; txIdx++ { select { diff --git a/execution/exec/blocks_read_ahead_test.go b/execution/exec/blocks_read_ahead_test.go index 3566bd6bb1e..b5d153feef3 100644 --- a/execution/exec/blocks_read_ahead_test.go +++ b/execution/exec/blocks_read_ahead_test.go @@ -27,7 +27,7 @@ import ( "github.com/erigontech/erigon/execution/cache" ) -// stubTemporalGetter stands in for the committed-state snapshot a warmup +// stubTemporalGetter stands in for the committed-state read view a warmup // goroutine reads: every GetLatest returns the same fixed value. type stubTemporalGetter struct { v []byte @@ -49,24 +49,30 @@ func newTestStateCache() *cache.StateCache { return cache.NewStateCache(b, b, b, b) } +// seedFill places an entry with an exact txNum stamp through the public fill +// API without moving the applied frontier. +func seedFill(sc *cache.StateCache, domain kv.Domain, k, v []byte, txNum uint64) { + sc.View(cache.FrontierFunc(func(kv.Domain) (uint64, bool) { return txNum + 1, true })).Fill(domain, k, v, txNum) +} + // A warmup read-through must never replace a fresher entry an authoritative // writer (the FCU flush cache-apply) has already put: the warmup reads a -// pre-flush snapshot, so a laggard Put landing after the flush would pin stale -// state in the cache and corrupt the next block's execution. +// pre-flush read view, so a laggard Put landing after the flush would pin +// stale state in the cache and corrupt the next block's execution. func TestCachePopulatingGetterKeepsFresherEntry(t *testing.T) { key := []byte("\x11\x22\x33\x44\x55\x66\x77\x88\x99\xaa\xbb\xcc\xdd\xee\xff\x00\x11\x22\x33\x44") fresh := []byte("account-record-nonce-5") stale := []byte("account-record-nonce-4") for _, domain := range []kv.Domain{kv.AccountsDomain, kv.StorageDomain} { sc := newTestStateCache() - sc.Put(domain, key, fresh, 54) - cpg := &cachePopulatingGetter{g: stubTemporalGetter{v: stale}, sc: sc, stepSize: 1_562_500} + seedFill(sc, domain, key, fresh, 54) + cpg := &cachePopulatingGetter{TemporalGetter: stubTemporalGetter{v: stale}, view: sc.View(cache.FrontierFunc(emptyVisibleEnd)), stepSize: 1_562_500} v, _, err := cpg.GetLatest(domain, key) require.NoError(t, err) - require.Equal(t, stale, v, "read-through must still return the snapshot value") + require.Equal(t, stale, v, "read-through must still return the view's value") - got, ok := sc.Get(domain, key) + got, ok := sc.View(nil).Get(domain, key) require.True(t, ok, "domain %s", domain) require.Equal(t, fresh, got, "domain %s: warmup must not clobber the fresher entry", domain) } @@ -79,13 +85,13 @@ func TestCachePopulatingGetterKeepsFresherCodeBinding(t *testing.T) { freshCode := []byte{0xaa, 0x01, 0x02, 0x03} staleCode := []byte{0xbb, 0x04, 0x05, 0x06} sc := newTestStateCache() - sc.PutCodeWithHash(addr, freshCode, crypto.Keccak256(freshCode), 54) - cpg := &cachePopulatingGetter{g: stubTemporalGetter{v: staleCode}, sc: sc, stepSize: 1_562_500} + seedFill(sc, kv.CodeDomain, addr, freshCode, 54) + cpg := &cachePopulatingGetter{TemporalGetter: stubTemporalGetter{v: staleCode}, view: sc.View(cache.FrontierFunc(emptyVisibleEnd)), stepSize: 1_562_500} _, _, err := cpg.GetLatest(kv.CodeDomain, addr) require.NoError(t, err) - got, ok := sc.Get(kv.CodeDomain, addr) + got, ok := sc.View(nil).Get(kv.CodeDomain, addr) require.True(t, ok) require.Equal(t, freshCode, got, "warmup must not rebind addr to older code") } @@ -98,48 +104,84 @@ func TestCachePopulatingGetterWarmsColdKeys(t *testing.T) { for _, domain := range []kv.Domain{kv.AccountsDomain, kv.StorageDomain} { sc := newTestStateCache() - cpg := &cachePopulatingGetter{g: stubTemporalGetter{v: val}, sc: sc, stepSize: 1_562_500} + cpg := &cachePopulatingGetter{TemporalGetter: stubTemporalGetter{v: val}, view: sc.View(cache.FrontierFunc(emptyVisibleEnd)), stepSize: 1_562_500} _, _, err := cpg.GetLatest(domain, key) require.NoError(t, err) - got, ok := sc.Get(domain, key) + got, ok := sc.View(nil).Get(domain, key) require.True(t, ok, "domain %s", domain) require.Equal(t, val, got, "domain %s", domain) } sc := newTestStateCache() - cpg := &cachePopulatingGetter{g: stubTemporalGetter{v: code}, sc: sc, stepSize: 1_562_500} + cpg := &cachePopulatingGetter{TemporalGetter: stubTemporalGetter{v: code}, view: sc.View(cache.FrontierFunc(emptyVisibleEnd)), stepSize: 1_562_500} _, _, err := cpg.GetLatest(kv.CodeDomain, key) require.NoError(t, err) - got, ok := sc.Get(kv.CodeDomain, key) + got, ok := sc.View(nil).Get(kv.CodeDomain, key) + require.True(t, ok) + require.Equal(t, code, got) + got, ok = sc.View(nil).GetCodeByHash(crypto.Keccak256(code)) require.True(t, ok) require.Equal(t, code, got) // Negative results (missing account, empty slot) are cached as nil hits. sc = newTestStateCache() - cpg = &cachePopulatingGetter{g: stubTemporalGetter{v: nil}, sc: sc, stepSize: 1_562_500, progress: func(kv.Domain) uint64 { return 100 }} + cpg = &cachePopulatingGetter{TemporalGetter: stubTemporalGetter{v: nil}, view: sc.View(cache.FrontierFunc(emptyVisibleEnd)), stepSize: 1_562_500} _, _, err = cpg.GetLatest(kv.AccountsDomain, key) require.NoError(t, err) - got, ok = sc.Get(kv.AccountsDomain, key) + got, ok = sc.View(nil).Get(kv.AccountsDomain, key) require.True(t, ok) require.Empty(t, got) } -// A negative (missing account, empty slot) carries no write step, so a -// step-derived stamp pins it at the start of history where no unwind can drop -// it. It must be stamped with the domain's progress at observation time — -// any unwind at or below that progress then invalidates it (mirroring the SD -// read-fill). -func TestCachePopulatingGetterNegativeDroppedByUnwind(t *testing.T) { +func TestCachePopulatingGetterNegativeUsesLastVisibleTxNum(t *testing.T) { + const visibleEnd = uint64(10_000_001) key := []byte("\x11\x22\x33\x44\x55\x66\x77\x88\x99\xaa\xbb\xcc\xdd\xee\xff\x00\x11\x22\x33\x44") sc := newTestStateCache() - cpg := &cachePopulatingGetter{g: stubTemporalGetter{v: nil}, sc: sc, stepSize: 16, progress: func(kv.Domain) uint64 { return 100 }} + cpg := &cachePopulatingGetter{ + TemporalGetter: stubTemporalGetter{v: nil}, stepSize: 1_562_500, + view: sc.View(cache.FrontierFunc(func(kv.Domain) (uint64, bool) { return visibleEnd, true })), + } + _, _, err := cpg.GetLatest(kv.AccountsDomain, key) + require.NoError(t, err) + _, ok := sc.View(nil).Get(kv.AccountsDomain, key) + require.True(t, ok) + + sc.Applier().Unwind(visibleEnd) + _, ok = sc.View(nil).Get(kv.AccountsDomain, key) + require.True(t, ok, "a negative observed before the unwind floor must remain cached") + + sc.Applier().Unwind(visibleEnd - 1) + _, ok = sc.View(nil).Get(kv.AccountsDomain, key) + require.False(t, ok, "a negative observed at the unwind floor must be invalidated") +} +func TestCachePopulatingGetterUnavailableVisibleEndNeverFills(t *testing.T) { + key := []byte("\x11\x22\x33\x44\x55\x66\x77\x88\x99\xaa\xbb\xcc\xdd\xee\xff\x00\x11\x22\x33\x44") + sc := newTestStateCache() + cpg := &cachePopulatingGetter{ + TemporalGetter: stubTemporalGetter{v: nil}, stepSize: 1_562_500, + view: sc.View(cache.FrontierFunc(func(kv.Domain) (uint64, bool) { return 0, false })), + } _, _, err := cpg.GetLatest(kv.AccountsDomain, key) require.NoError(t, err) - _, ok := sc.Get(kv.AccountsDomain, key) - require.True(t, ok, "the negative result must be cached") + _, ok := sc.View(nil).Get(kv.AccountsDomain, key) + require.False(t, ok, "no exact frontier — nothing may be cached") +} - sc.Unwind(50) - _, ok = sc.Get(kv.AccountsDomain, key) - require.False(t, ok, "a negative must not survive an unwind below the progress at which it was observed") +func TestCachePopulatingGetterStaleViewDoesNotFill(t *testing.T) { + key := []byte("\x11\x22\x33\x44\x55\x66\x77\x88\x99\xaa\xbb\xcc\xdd\xee\xff\x00\x11\x22\x33\x44") + sc := newTestStateCache() + sc.Applier().Apply(kv.AccountsDomain, key, nil, 20) + cpg := &cachePopulatingGetter{ + TemporalGetter: stubTemporalGetter{v: []byte("pre-delete-record")}, + stepSize: 1_562_500, + view: sc.View(cache.FrontierFunc(func(kv.Domain) (uint64, bool) { return 11, true })), + } + + _, _, err := cpg.GetLatest(kv.AccountsDomain, key) + require.NoError(t, err) + _, ok := sc.View(nil).Get(kv.AccountsDomain, key) + require.False(t, ok) } + +func emptyVisibleEnd(kv.Domain) (uint64, bool) { return 0, true } diff --git a/execution/execmodule/exec_module.go b/execution/execmodule/exec_module.go index 6bf8ed4f6c7..9a5b3204633 100644 --- a/execution/execmodule/exec_module.go +++ b/execution/execmodule/exec_module.go @@ -24,6 +24,7 @@ import ( "sync" "time" + "github.com/c2h5oh/datasize" "github.com/holiman/uint256" "golang.org/x/sync/semaphore" @@ -128,7 +129,11 @@ func (c *Cache) View(_ context.Context, tx kv.TemporalTx) (kvcache.CacheView, er context = c.publishedSD() } - return &CacheView{context: context, tx: tx}, nil + view := &CacheView{context: context, getter: tx} + if context != nil { + view.getter = context.AsGetter(tx) + } + return view, nil } func (c *Cache) OnNewBlock(sc *remoteproto.StateChangeBatch) {} func (c *Cache) Evict() int { return 0 } @@ -139,27 +144,21 @@ func (c *Cache) ValidateCurrentRoot(_ context.Context, _ kv.TemporalTx) (*kvcach type CacheView struct { context *execctx.SharedDomains - tx kv.TemporalTx + // getter is built once per view: it carries the per-tx cache ReadView, so + // per-read getter construction would cost an allocation on every call. + getter kv.TemporalGetter } func (c *CacheView) Get(k []byte) ([]byte, error) { - var getter kv.TemporalGetter = c.tx - if c.context != nil { - getter = c.context.AsGetter(c.tx) - } if len(k) == 20 { - v, _, err := getter.GetLatest(kv.AccountsDomain, k) + v, _, err := c.getter.GetLatest(kv.AccountsDomain, k) return v, err } - v, _, err := getter.GetLatest(kv.StorageDomain, k) + v, _, err := c.getter.GetLatest(kv.StorageDomain, k) return v, err } func (c *CacheView) GetCode(k []byte) ([]byte, error) { - var getter kv.TemporalGetter = c.tx - if c.context != nil { - getter = c.context.AsGetter(c.tx) - } - v, _, err := getter.GetLatest(kv.CodeDomain, k) + v, _, err := c.getter.GetLatest(kv.CodeDomain, k) return v, err } @@ -174,11 +173,7 @@ func (c *CacheView) GetAsOf(key []byte, ts uint64) (v []byte, ok bool, err error } func (c *CacheView) HasStorage(address common.Address) (bool, error) { - var getter kv.TemporalGetter = c.tx - if c.context != nil { - getter = c.context.AsGetter(c.tx) - } - _, _, hasStorage, err := getter.HasPrefix(kv.StorageDomain, address[:]) + _, _, hasStorage, err := c.getter.HasPrefix(kv.StorageDomain, address[:]) return hasStorage, err } @@ -249,7 +244,7 @@ func NewExecModule( hook *stageloop.Hook, accum *Accumulation, stateCache *Cache, - domainStateCache *cache.StateCache, + stateCacheBudget datasize.ByteSize, logger log.Logger, engine rules.Engine, syncCfg ethconfig.Sync, @@ -259,14 +254,8 @@ func NewExecModule( readAheader *exec.BlockReadAheader, stopNode func() error, ) *ExecModule { - // Production passes nil → full-size default cache. Test/CLI harnesses pass a - // small cache so building one ExecModule per fixture doesn't allocate - // hundreds of MB of LRU tables each (which stalled the parallel eest - // blocktest). Per-instance, so it never mutates the process-wide default. - domainCache := domainStateCache - if domainCache == nil { - domainCache = cache.NewDefaultStateCache() - } + domainCache := newDomainStateCache(stateCacheBudget) + execctx.GuardAggregatorForCache(db, domainCache) var codeStore *cache.CodeStore if dbg.UseCodeStore { codeStore = cache.NewCodeStore(cache.DefaultCodeStoreMemBytes, cache.DefaultCodeStoreTableBytes) @@ -320,6 +309,29 @@ func (e *ExecModule) WaitIdle(ctx context.Context) { e.semaphore.Release(1) } +// newDomainStateCache is the module's one construction site of the domain +// state cache: USE_STATE_CACHE=false builds none, so nothing upstream can +// allocate a cache that would only be discarded. A budget > 0 overrides the +// production per-domain byte budget (test harnesses keep per-fixture modules +// small); 0 means the production default. +func newDomainStateCache(budget datasize.ByteSize) *cache.StateCache { + if !dbg.UseStateCache { + return nil + } + if budget > 0 { + return cache.NewStateCache(budget, budget, budget, budget) + } + return cache.NewDefaultStateCache() +} + +// Close releases the domain state cache's reservation in the shared memory +// envelope. +func (e *ExecModule) Close() { + if e.stateCache != nil { + e.stateCache.Close() + } +} + // closeModuleContext closes and clears e.currentContext. The nil swap happens // under e.lock first, so getters holding the read lock (beginOverlayOrRo) can // never obtain a SharedDomains that is about to be closed. @@ -385,12 +397,12 @@ func (e *ExecModule) canonicalHash(ctx context.Context, tx kv.Tx, blockNumber ui } // drainReadAhead blocks until any in-flight block-assembly warmup finishes. -// warmBody is fire-and-forget and populates the shared state/branch caches; if -// it is still running when an unwind bumps the cache epoch, it can Put a +// warmBody is fire-and-forget and fills the shared state cache; if +// it is still running when an unwind bumps the cache epoch, it can fill a // pre-unwind (dead-fork) value stamped with the post-unwind epoch — IsStale then -// returns false and the stale value is served as canonical (wrong root). A -// laggard Put can likewise land after a flush's cache-apply and pin the -// pre-flush snapshot. Call before any unwind epoch-bump or flush cache-apply. +// returns false and the stale value is served as canonical (wrong root). Fill +// admission does not cover this direction: an unwind lowers the applied +// frontier, so a pre-unwind view passes. Call before any unwind epoch-bump. func (e *ExecModule) drainReadAhead() { if e.readAheader == nil { return diff --git a/execution/execmodule/exec_module_internal_test.go b/execution/execmodule/exec_module_internal_test.go new file mode 100644 index 00000000000..f2420c0ff57 --- /dev/null +++ b/execution/execmodule/exec_module_internal_test.go @@ -0,0 +1,46 @@ +// Copyright 2026 The Erigon Authors +// This file is part of Erigon. +// +// Erigon is free software: you can redistribute it and/or modify +// it under the terms of the GNU Lesser General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Erigon is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Lesser General Public License for more details. +// +// You should have received a copy of the GNU Lesser General Public License +// along with Erigon. If not, see . + +package execmodule + +import ( + "testing" + + "github.com/c2h5oh/datasize" + "github.com/stretchr/testify/require" + + "github.com/erigontech/erigon/common/dbg" +) + +// The module is the one owner of the domain state cache: callers pass a byte +// budget, never a constructed cache, so a disabled cache cannot be built +// upstream and leak its memory-envelope reservation. +func TestNewDomainStateCacheRespectsUseStateCache(t *testing.T) { + prev := dbg.UseStateCache + t.Cleanup(func() { dbg.SetUseStateCache(prev) }) + + dbg.SetUseStateCache(false) + require.Nil(t, newDomainStateCache(0), "disabled mode must construct no cache") + require.Nil(t, newDomainStateCache(16*datasize.MB), "a budget must not override the kill switch") + + dbg.SetUseStateCache(true) + sc := newDomainStateCache(16 * datasize.MB) + require.NotNil(t, sc) + sc.Close() + scDefault := newDomainStateCache(0) + require.NotNil(t, scDefault, "zero budget means the production default, not no cache") + scDefault.Close() +} diff --git a/execution/execmodule/execmoduletester/exec_module_tester.go b/execution/execmodule/execmoduletester/exec_module_tester.go index 1ecbd77dbe1..35b9373bbe6 100644 --- a/execution/execmodule/execmoduletester/exec_module_tester.go +++ b/execution/execmodule/execmoduletester/exec_module_tester.go @@ -54,7 +54,6 @@ import ( "github.com/erigontech/erigon/db/snaptype" dbstate "github.com/erigontech/erigon/db/state" "github.com/erigontech/erigon/execution/builder" - "github.com/erigontech/erigon/execution/cache" "github.com/erigontech/erigon/execution/chain" "github.com/erigontech/erigon/execution/exec" "github.com/erigontech/erigon/execution/execmodule" @@ -124,7 +123,6 @@ type ExecModuleTester struct { ForkValidator *execmodule.ForkValidator ExecModule *execmodule.ExecModule StateCache *execmodule.Cache - domainCache *cache.StateCache retirementStart chan bool retirementDone chan struct{} retirementWg sync.WaitGroup @@ -163,8 +161,8 @@ func (emt *ExecModuleTester) Close() { if emt.DB != nil { emt.DB.Close() } - if emt.domainCache != nil { - emt.domainCache.Close() + if emt.ExecModule != nil { + emt.ExecModule.Close() } if emt.tb == nil && emt.Dirs.DataDir != "" { dir.RemoveAll(emt.Dirs.DataDir) @@ -754,10 +752,6 @@ func New(tb testing.TB, opts ...Option) *ExecModuleTester { Accumulator: mock.Notifications.Accumulator, RecentReceipts: mock.Notifications.RecentReceipts, } - // Per-instance domain cache, held on the tester so Close releases its - // envelope reservation. Uses the production default — the caches jump-grow on - // demand, so a small-working-set fixture stays small. - mock.domainCache = cache.NewDefaultStateCache() mock.ExecModule = execmodule.NewExecModule( ctx, mock.BlockReader, @@ -769,7 +763,7 @@ func New(tb testing.TB, opts ...Option) *ExecModuleTester { hook, accum, mock.StateCache, - mock.domainCache, + 0, // stateCacheBudget: production default; the caches jump-grow on demand logger, engine, cfg.Sync, diff --git a/execution/execmodule/forkchoice.go b/execution/execmodule/forkchoice.go index 280959cb907..f67f0d8b20d 100644 --- a/execution/execmodule/forkchoice.go +++ b/execution/execmodule/forkchoice.go @@ -360,10 +360,9 @@ func (e *ExecModule) updateForkChoice(ctx context.Context, originalBlockHash, sa }) defer cleanupBeforeSemaRelease() - // Drain any warmup a preceding newPayload spawned: its Puts reflect a - // pre-FCU snapshot and must land before this FCU's unwind epoch-bump and - // flush cache-apply, not after them (no new warmup starts while we hold - // the semaphore). + // Drain any warmup a preceding newPayload spawned: a fill from a pre-unwind + // view would survive this FCU's possible unwind epoch-bump as a live entry + // (see drainReadAhead). No new warmup starts while we hold the semaphore. e.drainReadAhead() var validationError string diff --git a/node/eth/backend.go b/node/eth/backend.go index 9be1008a771..949589b41e7 100644 --- a/node/eth/backend.go +++ b/node/eth/backend.go @@ -73,7 +73,6 @@ import ( "github.com/erigontech/erigon/diagnostics/diaglib" "github.com/erigontech/erigon/diagnostics/mem" "github.com/erigontech/erigon/execution/builder" - "github.com/erigontech/erigon/execution/cache" "github.com/erigontech/erigon/execution/chain" chainspec "github.com/erigontech/erigon/execution/chain/spec" "github.com/erigontech/erigon/execution/engineapi" @@ -959,14 +958,6 @@ func New(ctx context.Context, stack *node.Node, config *ethconfig.Config, logger Accumulator: backend.notifications.Accumulator, RecentReceipts: backend.notifications.RecentReceipts, } - // Test harnesses (e.g. EngineApiTester) set StateCacheBudget small so each - // per-fixture ExecModule doesn't allocate the full production cache; 0 keeps - // the production default. - var domainStateCache *cache.StateCache - if config.StateCacheBudget > 0 { - b := config.StateCacheBudget - domainStateCache = cache.NewStateCache(b, b, b, b) - } backend.execModule = execmodule.NewExecModule( ctx, blockReader, @@ -978,7 +969,7 @@ func New(ctx context.Context, stack *node.Node, config *ethconfig.Config, logger hook, accum, execmoduleCache, - domainStateCache, + config.StateCacheBudget, logger, backend.engine, config.Sync, @@ -1565,6 +1556,10 @@ func (s *Ethereum) Stop() error { s.chainDB.Close() + if s.execModule != nil { + s.execModule.Close() + } + if s.config.Downloader != nil { _ = s.config.Downloader.CloseTorrentLogFile() }