execution: FCU background-commit worker + generation-chain reader consistency - #21414

Open

mh0lt wants to merge 186 commits into

mainfrom

mh/perf-semaphore-pr-v2

Commits on May 21, 2026

commitment: extract canonical DecodeBranchInto from unfoldBranchNode

Mark Holt
and
claude
committed
ddc6df7
commitment: add BranchCache with pinned root + bounded LRU tail

Mark Holt
and
claude
committed
31fdf57
commitment: document BranchCache concurrency contract

Mark Holt
and
claude
committed
1593f0e
commitment: wire BranchCache into trie read + encoder write paths

Mark Holt
and
claude
committed
8a2c565
commitment, db/state: lift BranchCache lifetime to aggregator scope
Mark Holt
committed
2849165
commitment: BranchCache divergence detector + per-block fingerprint log
Mark Holt
committed
f87937e
commitment: disable deferred-encoding mode (use inline CollectUpdate)
Mark Holt
committed
b61655c
commitment: add DISABLE_BRANCH_CACHE_READS env kill-switch
Mark Holt
committed
7e21628
commitment: multi-layer state probe at BranchCache divergence site
Mark Holt
committed
8d6d19a
commitment: tag BranchCache writes with origin + write-seq + timestamp
Mark Holt
committed
87f12be

Commits on May 22, 2026

commitment, db/state/execctx: move BranchCache behind sd.mem chain

Mark Holt
and
claude
committed
6989427
commitment/branch_cache: document responsibility split + post-WarmupCache state
Mark Holt
committed
99823bb
commitment/branch_cache: add pinned tier (PinEntry / PinnedCount)
Mark Holt
committed
4410704
commitment/preload: PreloadContractTrunk recursive enumeration
Mark Holt
committed
9b7c02a
state/execctx, commitment: PIN_CONTRACT_TRUNKS env hook
Mark Holt
committed
a60b052
state/execctx: trunk-preload sync (was async, hit cgo-pointer panic)
Mark Holt
committed
71d8480
commitment, state/execctx: trunk-preload progress logs + max-branches cap
Mark Holt
committed
f85308d
state/execctx: run trunk-preload AFTER SeekCommitment
Mark Holt
committed
a289adc
state/execctx: trunk-preload async with own tx, triggered from EnableParaTrieDB
Mark Holt
committed
29ebc42
execution/commitment: PIN_TRUNK_MAX_DEPTH env + raise maxBranches cap
Mark Holt
committed
db656cc
stagedsync/exec3_parallel: re-enable trie warmup on parallel path

Mark Holt
and
claude
committed
ce8cce4
commitment: WarmupCache hit/miss/evict observability counters

Mark Holt
and
claude
committed
e97c95f
commitment: extract unfoldKeyPath as per-key traversal primitive

Mark Holt
and
claude
committed
af2ef68
commitment: WarmupCache dirty-flag + PutBranchIfClean invariants

Mark Holt
and
claude
committed
4074bcc
commitment: WarmupCache GetBranchDecoded — lazy-decode read path

Mark Holt
and
claude
committed
4be9c86
execmodule: chain ValidateChain SD to currentContext for head-extending payloads
Mark Holt
committed
733dad5
db/state: H0 baseline benchmark for snapshot-vs-MDBX read cost

Mark Holt
and
claude
committed
a823097
db/state: H0 real-datadir bootstrap; first production numbers

Mark Holt
and
claude
committed
118298a
db/state, commitment: H_GetAsOf bench + per-block calculator timing

Mark Holt
and
claude
committed
005790f
commitment: surface per-block telemetry on cache-fp log

Mark Holt
and
claude
committed
4276219
stagedsync/calc_state: skip wasted ensureStorage in StoragePath ApplyWrites

Mark Holt
and
claude
committed
1e8c25c
commitment: split disk-load counter from EVM-write plumbing
Mark Holt
committed
a263695
state, commitment: scaffold to quantify EIP-684 HasPrefix cost on snapshot storage
Mark Holt
committed
fd593ca
commitment, dbg: warmer scales 1:1 with cores; document pooling strategy
Mark Holt
committed
d2ac8ef
existence, state, commitment: add measurement counters for assumption check
Mark Holt
committed
20b020d
commitment/warmuper: stop walking past leaf cells
Mark Holt
committed
fefec01
commitment: drop BranchEncoder's WarmupCache fast path
Mark Holt
committed
c57420b
commitment/hph: drop hph.cache (WarmupCache read-through layer)
Mark Holt
committed
63e1124
commitment, db/state, stagedsync: delete WarmupCache type and toggle chain
Mark Holt
committed
39b758e
commitment: mark BranchCache dirty in CollectDeferredUpdate too
Mark Holt
committed
b99c192
commitment/warmuper: scope warmer to branches only — drop account/storage prefetch
Mark Holt
committed
80ea439
state, commitment: add unique-prefix file-read counter
Mark Holt
committed
614fb7b
state, commitment: add unique-prefix length histogram
Mark Holt
committed
4b95d45
state, commitment: refine length histogram inside storage subtree
Mark Holt
committed
d566e9b
state/changeset: trunk-probe log to identify hot contracts
Mark Holt
committed
65fbd14
commitment: surface pinned-tier counters on cache-fp log
Mark Holt
committed
c07a688
db/state, db/kv: add TemporalMemBatch.FlushWithCallback
Mark Holt
committed
a62cbaf
commitment, state/execctx: BFS / budget-driven trunk preload
Mark Holt
committed
8bad508
state/execctx: raise PIN_TRUNK_RAM_BUDGET_MB default 32 → 64
Mark Holt
committed
b29f3cf
state/execctx: clamp PIN_TRUNK_RAM_BUDGET_MB at 64 MiB per contract
Mark Holt
committed
d494eec
commitment/branch_cache: pin-aware Invalidate, GetWithOrigin, Clear
Mark Holt
committed
6ea7ff8
state/execctx: invalidate stale BranchCache entries on SD.Unwind
Mark Holt
committed
9ef3b9c
commitment/preload: split BFS into resumable ContractTrunkPreload
Mark Holt
committed
878f04a
commitment/branch_cache: add miss-callback hook for adaptive controller
Mark Holt
committed
a2b6ab5
commitment: AdaptivePinController for promotion / extension / demotion
Mark Holt
committed
be2616e
state/execctx: wire AdaptivePinController into SharedDomains
Mark Holt
committed
b082c93
commitment: Prometheus metrics for trunk-pin layer
Mark Holt
committed
2c6ee4e
commitment: tests for trunk-pin layer + delete orphaned warmup test
Mark Holt
committed
f48c411
commitment, db/state: drop bench-only env knobs and cache-fp log
Mark Holt
committed
e69e9bf
lint: gofmt fixes + drop snapshot_vs_mdbx bench test
Mark Holt
committed
e0b0189
perf(r3): bulk storage-trunk preload via two-parity range scan (core + tests)

Mark Holt
and
claude
committed
fa4d7f7
perf(r3): wire triggerTrunkPreload to the bulk range-scan path

Mark Holt
and
claude
committed
b4591a2
perf(r3): adaptive controller uses LoadBulk (range scan) when available

Mark Holt
and
claude
committed
ae86359
perf(preload): retire the broken R3 range-scan preload

Mark Holt
and
claude
committed
4ca9284
perf(preload): parallel file-only wave-BFS preload (cut-down)

Mark Holt
and
claude
committed
3e0154f
perf(preload): cap per-wave fetch by remaining budget

Mark Holt
and
claude
committed
4c9968a
perf(preload): phase 1 — MDBX-resident branch prefetch (operational correctness)

Mark Holt
and
claude
committed
eb0c0c5
db/state: test for the phase-1 dbPrefetch cursor walk

Mark Holt
and
claude
committed
a3374fe
commitment, db/state: resumable parallel preload + adaptive parallel-mode wiring

Mark Holt
and
claude
committed
9db0d59
execution/commitment: keep the trie checkpoint key out of BranchCache

Mark Holt
and
claude
committed
ae9b3f3
execution: parent fork-validation SD into the canonical generation

Mark Holt
and
claude
committed
3065d89
execution/commitment: prealloc rest slice in preload_parallel

Mark Holt
and
claude
committed
ae711df
db/state/execctx, execution/execmodule: clear BranchCache on SetHead unwind

Mark Holt
and
claude
committed
b2a9d64

Commits on May 23, 2026

Merge remote-tracking branch 'origin/main' into mh/perf-caches-pr
Mark Holt
committed
9f5086b
execution/stagedsync/rawdbreset: clear branchCache on ResetExec (#21138)

Mark Holt
and
claude
committed
f151829
common/dbg, node/ethconfig: default ExecWorkerCount = NumCPU-1

Mark Holt
and
claude
committed
9898f3c
Revert "common/dbg, node/ethconfig: default ExecWorkerCount = NumCPU-1"
Mark Holt
committed
6454b7b

Commits on May 25, 2026

cmd/evm, execution/tests: per-subtest ephemeral SharedDomains for state tests
mh0lt
committed
e8ba812
execution/tests: read pre-state via sd.AsGetter so tx execution sees sd.mem
mh0lt
committed
54a9029
execution/engineapi/engineapitester: reset aggregator branchCache between cached-tester runs
mh0lt
committed
57dd0d6
execution/cache, db/state/execctx: SD-transparent ethHash bypass for CodeDomain

authored and
mh0lt
committed
5a415cf
execution/cache, db/state, execution/state: codeSizeCache for EXTCODESIZE / EXTCODEHASH

authored and
mh0lt
committed
4499ad5
execution/exec, execution/execmodule: BlockReadAheader populates cache.StateCache

authored and
mh0lt
committed
ad565fe
execution/state, execution/cache: stateObject.code populate + addrToHash LRU

authored and
mh0lt
committed
2d82956
execution/cache, db/state/execctx: addr → codeHash LRU above SD

authored and
mh0lt
committed
987f741
execution/exec: cachePopulatingGetter caches negative results

authored and
mh0lt
committed
934082b
execution/cache: surface fill-and-freeze cliff via inserts/dropped counters

authored and
mh0lt
committed
e96df71
execution/cache: replace GenericCache map with sharded LRU + Mode

authored and
mh0lt
committed
7b5a71c
execution/cache: STATE_CACHE_MODE env override at NewStateCache time

authored and
mh0lt
committed
3fec476
execution/cache: correct the LFU rationale in Mode docstring

authored and
mh0lt
committed
1a56794
execution/cache: reduce default cache caps to 100 MB each (bench knob)

authored and
mh0lt
committed
4a512ce
Parallel-exec correctness fixes (PR #3 of the perf stack) (#21387)

authored
da4b091
Merge remote-tracking branch 'origin/main' into mh/perf-caches-pr
mh0lt
committed
3b11e0e
Merge remote-tracking branch 'origin/mh/perf-caches-pr' into mh/perf-statecache-lru-pr
mh0lt
committed
b7d89ad
execution/state: drop tautological nil checks in code-loading paths (govet)
mh0lt
committed
9b15362

Commits on May 26, 2026

Commits on May 27, 2026

Commits on Jun 4, 2026

db/state/execctx: decode AccountsDomain codeHash with DeserialiseV3

mh0lt
and
claude
committed
fd74979

Commits on Jun 5, 2026

db/state, db/kv: unify FlushWithCallback — flush-only refresh for branchCache + stateCache

authored and
mh0lt
committed
ba6c67a
execution/execmodule: FCU semaphore decouple + foreground-priority bg-commit worker (gate item 2, core)

authored and
mh0lt
committed
a5f973b
execution/execmodule: gate-2 overlay-continuity wiring (FCU/ValidateChain/InsertBlocks)

authored and
mh0lt
committed
f54eb60
execution/execmodule: commitWorker lifecycle — drain + exit before DB-close

authored and
mh0lt
committed
980898d
execution, rpc, txpool: route latest-state reads through the in-flight SD under async commit

authored and
mh0lt
committed
a1dbe0b
execution: published-leaf SD reads for the outer layer under async commit

authored and
mh0lt
committed
c2ee3fc
execution/cache, state, stagedsync: txNum/epoch cache invalidation; remove the schedule-time ValidateAndPrepare purge

mh0lt
and
claude
committed
2a41192
execution: wire state cache into SetHead unwind (fixes BadBlock on re-exec)

mh0lt
and
claude
committed
42a2aaf
commitment: LatestStateReader.Clone keeps its source tx, not the compute tx

mh0lt
and
claude
committed
2683865

Commits on Jun 6, 2026

db/state, commitment: lock-free per-worker read metrics (+ tx-aware BranchCache)

mh0lt
and
claude
committed
44355fd
db/state: drop process-wide metrics accumulator (metrics-on safety)

mh0lt
and
claude
committed
a027002
commitment: meter the main-fold reads per-ComputeCommitment

mh0lt
and
claude
committed
943968d
db/state, commitment: collapse metrics to per-worker DomainMetrics instances

mh0lt
and
claude
committed
f375aa8
exec: meter parallel-exec worker reads (per-worker DomainMetrics, per-task merge)

mh0lt
and
claude
committed
cd1fb81

Commits on Jun 7, 2026

execution/cache: fix off-by-one in state-cache unwind floor

mh0lt
and
claude
committed
bd6097e
execution: address #21380 review feedback

mh0lt
and
claude
committed
67d0f33
Merge origin/main into mh/perf-caches-pr (#21380)

mh0lt
and
claude
committed
6f9f737

Commits on Jun 8, 2026

Merge mh/perf-caches-pr into mh/perf-statecache-lru-pr (#21386)

mh0lt
and
claude
committed
5d16a64
Merge mh/perf-statecache-lru-pr into mh/perf-semaphore-pr-v2 (#21414)

mh0lt
and
claude
committed
2542895

Commits on Jul 9, 2026

Merge remote-tracking branch 'origin/main' into mh/perf-semaphore-pr-v2
mh0lt
committed
14604c0

Commits on Jul 13, 2026

Commits on Jul 14, 2026

Commits on Jul 15, 2026

Commits on Jul 16, 2026

Commits on Jul 17, 2026

Commits on Jul 18, 2026

execution/stagedsync: post-block-consistent fold reader (fix wrong root on storage clear)
mh0lt
committed
c323df0

Commits on Jul 20, 2026

Merge remote-tracking branch 'origin/main' into mh/perf-semaphore-pr-v2
mh0lt
committed
32370ba

Commits on Jul 22, 2026

Commits on Jul 28, 2026

Commits on Jul 29, 2026

Commits on Aug 5, 2026

Merge remote-tracking branch 'origin/main' into mh/perf-semaphore-pr-v2
mh0lt
committed
607ca28