Flaky gas mismatch: EIP-8037 per-authorization state-gas refund (StateGasAuthBase) intermittently skipped
Summary
eest-spec-blocktests-devnet intermittently fails a single Amsterdam EIP-7702 test with a gas-used mismatch of exactly 35,190 gas — which is StateGasAuthBase (EIP-8037: StateBytesAuthBase(23) × CostPerStateByte(1530)). The block executes with the per-authorization state-gas charged but not refunded, when the spec/header expects it to be refunded. The failure is non-deterministic: it passes in isolation and across two full local corpus passes, and it passes on main. Because the shard has max-allowed-failures: 0, a single occurrence reds the whole job (and via the workflow's Cancel workflow run on failure step, SIGTERMs the other in-flight jobs).
Affected component
- Fork: Amsterdam (devnet); EIP-7702 (set-code txns) × EIP-8037 (state-creation gas / multidimensional metering).
- Code:
execution/protocol/txn_executor.go — the EIP-8037 per-authorization state-gas refund.
- Surfaces via:
cmd/evm blocktest on eest_devnet (tests-bal@v7.3.2).
Symptom (confirmed)
Failing case:
tests/prague/eip7702_set_code_tx/test_set_code_txs.py::test_set_code_to_sstore_then_sload[fork_Amsterdam-blockchain_test-auth_signer_in_access_list]
error: invalid block, gas used by execution: 449820, in header: 414630
449820 − 414630 = 35190 = StateGasAuthBase = StateBytesAuthBase(23) × CostPerStateByte(1530).
CI run: 1 failure out of 83,782 tests; blocktests-devnet shard budget is max-allowed-failures: 0.
Where (confirmed)
execution/protocol/txn_executor.go (EIP-8037 refund in the authorization loop):
exists, _ := st.state.Exist(authority)
if st.evm.ChainRules().IsAmsterdam {
if exists {
stateIgasRefund += params.StateGasNewAccount // 120×1530
}
if hasDelegation || auth.Address == (common.Address{}) {
stateIgasRefund += params.StateGasAuthBase // 23×1530 = 35190 ← intermittently skipped
}
}hasDelegation is derived a few lines above from st.state.GetCodeHash(authority) + st.state.GetDelegatedDesignation(authority). The missing 35,190 means hasDelegation evaluated false when it should have been true for that authority.
Trigger shape (confirmed via instrumentation)
Logging each Amsterdam authorization's refund inputs for the failing test shows the same authority is processed multiple times in the block, with its delegation state evolving:
auth=0xc0F6…992B exists=false hasDeleg=false → no StateGasAuthBase refund
auth=0xc0F6…992B exists=false hasDeleg=false → no refund
auth=0xc0F6…992B exists=true hasDeleg=true → +StateGasNewAccount +StateGasAuthBase (correct)
The flake is a run where the last processing reads hasDeleg=false instead of true — i.e. it does not observe the delegation a prior authorization/tx set on the same authority — so the 35,190 refund is skipped.
Root-cause hypothesis (NOT yet reproduced)
A speculative/parallel-execution stale read in exec3_parallel: a transaction reads the authority's delegation state before a predecessor's delegation write has settled, computes hasDelegation=false, and skips the StateGasAuthBase refund. The executor's own model supports this (ErrExecAbortError doc in txn_executor.go: "a versioned read observes an unsettled predecessor … stale speculative input"). The conflict machinery resolves the state version, but the derived refund gas computed during the speculative pass appears to slip through.
Most likely fix area: ensure the authority reads feeding the refund decision (GetCodeHash / GetDelegatedDesignation / Exist) are version-tracked in the parallel executor so a stale speculative read triggers re-execution, or recompute the refund after the read set settles. The fix is in the parallel read-validation path, not the refund line itself (which is correct given accurate inputs).
This is a hypothesis: I could not reproduce the failure to confirm it (see below).
Reproduction attempts (all clean — flake is rare/timing-dependent)
- Failing case in isolation, serial and parallel exec: 30/30 pass.
- EIP-7928 / EIP-7702 set-code subsets at
--workers 8 under -race: no data race, no failure.
- Two full devnet-corpus passes (
--workers 16, parallel exec, USE_STATE_CACHE=true): 0 failures / 167,564 executions; refund path deterministic (3,240 auth events per pass, identical both passes).
So local reproduction failed; the event is rare and likely hardware/scheduling-sensitive (it fired once on CI). A -race run over the full devnet corpus on CI-class hardware, or many more passes with VersionMap.validateRead instrumented, is the path to a live repro.
Impact
- Mainnet: not affected for the normal activation case (gas accounting is correct given accurate reads; the failure requires the speculative stale-read window).
- CI: any single occurrence reds
blocktests-devnet (0-failure budget) and cancels the rest of the run, producing confusing "runner shutdown / exit 143" failures on unrelated jobs (the heavy test/race shards) as collateral.
Suggested mitigations
- Fix the parallel read-validation so the authorization refund inputs are conflict-tracked (real fix).
- Consider a small non-zero
max-allowed-failures for the in-development devnet blocktest shard until Amsterdam stabilizes, so a single rare flake doesn't red unrelated PRs.
Not related to
yperbasis/commitment-warmup-scaling (PR #21982). The failure is gas-accounting in EIP-8037/7702 and is independent of the commitment BranchCache: identical with USE_STATE_CACHE on/off, and the BranchCache is commitment-domain-only (cannot affect execution gas).
Flaky gas mismatch: EIP-8037 per-authorization state-gas refund (
StateGasAuthBase) intermittently skippedSummary
eest-spec-blocktests-devnetintermittently fails a single Amsterdam EIP-7702 test with a gas-used mismatch of exactly 35,190 gas — which isStateGasAuthBase(EIP-8037:StateBytesAuthBase(23) × CostPerStateByte(1530)). The block executes with the per-authorization state-gas charged but not refunded, when the spec/header expects it to be refunded. The failure is non-deterministic: it passes in isolation and across two full local corpus passes, and it passes onmain. Because the shard hasmax-allowed-failures: 0, a single occurrence reds the whole job (and via the workflow's Cancel workflow run on failure step, SIGTERMs the other in-flight jobs).Affected component
execution/protocol/txn_executor.go— the EIP-8037 per-authorization state-gas refund.cmd/evm blocktestoneest_devnet(tests-bal@v7.3.2).Symptom (confirmed)
Failing case:
449820 − 414630 = 35190 = StateGasAuthBase = StateBytesAuthBase(23) × CostPerStateByte(1530).CI run: 1 failure out of 83,782 tests;
blocktests-devnetshard budget ismax-allowed-failures: 0.Where (confirmed)
execution/protocol/txn_executor.go(EIP-8037 refund in the authorization loop):hasDelegationis derived a few lines above fromst.state.GetCodeHash(authority)+st.state.GetDelegatedDesignation(authority). The missing 35,190 meanshasDelegationevaluated false when it should have been true for that authority.Trigger shape (confirmed via instrumentation)
Logging each Amsterdam authorization's refund inputs for the failing test shows the same authority is processed multiple times in the block, with its delegation state evolving:
The flake is a run where the last processing reads
hasDeleg=falseinstead oftrue— i.e. it does not observe the delegation a prior authorization/tx set on the same authority — so the 35,190 refund is skipped.Root-cause hypothesis (NOT yet reproduced)
A speculative/parallel-execution stale read in
exec3_parallel: a transaction reads the authority's delegation state before a predecessor's delegation write has settled, computeshasDelegation=false, and skips theStateGasAuthBaserefund. The executor's own model supports this (ErrExecAbortErrordoc intxn_executor.go: "a versioned read observes an unsettled predecessor … stale speculative input"). The conflict machinery resolves the state version, but the derived refund gas computed during the speculative pass appears to slip through.Most likely fix area: ensure the authority reads feeding the refund decision (
GetCodeHash/GetDelegatedDesignation/Exist) are version-tracked in the parallel executor so a stale speculative read triggers re-execution, or recompute the refund after the read set settles. The fix is in the parallel read-validation path, not the refund line itself (which is correct given accurate inputs).This is a hypothesis: I could not reproduce the failure to confirm it (see below).
Reproduction attempts (all clean — flake is rare/timing-dependent)
--workers 8under-race: no data race, no failure.--workers 16, parallel exec,USE_STATE_CACHE=true): 0 failures / 167,564 executions; refund path deterministic (3,240 auth events per pass, identical both passes).So local reproduction failed; the event is rare and likely hardware/scheduling-sensitive (it fired once on CI). A
-racerun over the full devnet corpus on CI-class hardware, or many more passes withVersionMap.validateReadinstrumented, is the path to a live repro.Impact
blocktests-devnet(0-failure budget) and cancels the rest of the run, producing confusing "runner shutdown / exit 143" failures on unrelated jobs (the heavy test/race shards) as collateral.Suggested mitigations
max-allowed-failuresfor the in-development devnet blocktest shard until Amsterdam stabilizes, so a single rare flake doesn't red unrelated PRs.Not related to
yperbasis/commitment-warmup-scaling(PR #21982). The failure is gas-accounting in EIP-8037/7702 and is independent of the commitmentBranchCache: identical withUSE_STATE_CACHEon/off, and the BranchCache is commitment-domain-only (cannot affect execution gas).