From c388e2bfa65c6ea5f9a2f0f2fb3a42c84373eb36 Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 09:37:59 +0200 Subject: [PATCH 01/12] feat(test-fill): add a sync phase for framework-injected payloads The stateful fill introduced per-payload phase tags (setup / execution / cleanup) so consumers can partition a fixture's payloads by role. Add a `sync` phase for payloads the framework injects purely to change the chain's shape: they prepare no state a test depends on, so neither of the existing phases describes them, and borrowing `setup` would send a reader looking for state that is not there. The first users are the empty blocks the filler adds to engine_x chains so that sync-based consumers can trigger a devp2p sync: a salted empty block appended above a valid chain's head, and one prepended between genesis and a single invalid block. Consumers that replay payloads through the Engine API can treat a sync payload like any other block; the tag exists so a framework-injected payload is distinguishable from the test's own after the fixture is reloaded. --- .../src/execution_testing/test_types/phase_manager.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/packages/testing/src/execution_testing/test_types/phase_manager.py b/packages/testing/src/execution_testing/test_types/phase_manager.py index cdd4fb182d5..22e14f77bfc 100644 --- a/packages/testing/src/execution_testing/test_types/phase_manager.py +++ b/packages/testing/src/execution_testing/test_types/phase_manager.py @@ -15,6 +15,16 @@ class TestPhase(str, Enum): # compatibility EXECUTION = "testing" CLEANUP = "cleanup" + SYNC = "sync" + """ + A framework-injected payload that exists only to make the chain + syncable, such as the empty block the filler adds to a test's + chain - appended above a valid chain's head, or prepended between + genesis and a single invalid block - so that sync-based consumers + can trigger a devp2p sync. Unlike ``SETUP``, a sync payload + prepares no state a test depends on; consumers that replay + payloads through the Engine API can treat it like any other block. + """ class TestPhaseManager: From 38795a5487318c53e8f917d8cc722c3c8e5bf7aa Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 16:11:14 +0200 Subject: [PATCH 02/12] feat(test-fill): add per-class sync blocks for engine_x fixtures A client only starts a devp2p sync when the announced head's parent is unknown to it, and only blocks below the head are guaranteed to travel devp2p: the head's payload is always delivered through `engine_newPayload`, and whether a client also re-fetches it from the peer is an implementation choice - measured across four engaged clients it goes both ways (geth, ethrex and nethermind fetch the head's body, reth executes it from the announcement). A single-block chain built on the client's own genesis never syncs at all - and 97% of the Cancun engine_x corpus is single-block chains. The new --sync-block option gives every eligible chain one framework-built empty block, placed per test by the chain's own statically declared structure (`SyncBlockPolicy`): - append (fully valid chains): the block is built on the test's post-state and is what a sync-based consumer announces. Every one of the test's own blocks is then an ancestor of the announced head, which a syncing client must fetch and execute through its sync pipeline: the test's content reaches the client over the wire by chain structure, not by client courtesy, on any client. - prepend (a single expected-invalid or Engine API-refused block): nothing can be built on such a block, so the empty block lands between genesis and the test's block, giving the sync a reason to start before the client judges the announced head. A block pinning an absolute number is shifted up by one; a pinned timestamp the empty block does not clear fails the fill loudly (timestamps are semantic - fork activation, TIMESTAMP expectations - and are never shifted). - none (everything else): an invalid multi-block chain already carries valid ancestors that travel the wire and needs no help; chains with an Engine API error code on any block keep their own announcement. An Engine API-refused head must never take an appended block even though it is consensus-valid to the fill: the appended block would become the announced head, the refused payload would arrive over devp2p instead of through `newPayload`, and the refusal the test asserts would never happen. Refused singletons take the prepend rather than nothing because sync-based consumers skip single-block chains: bare, these tests would silently drop out of every sync run; prepended, their announcement stays the test's own payload and the expected refusal stays exercised. Both placements guard their timestamp boundary. The prepend refuses pinned timestamps that cannot clear the extra block below it; the append refuses a head whose timestamp leaves no uint64 headroom for the block above it - at or next to 2**64 - 1 the trailer's timestamp would not fit uint64 and no client could even parse the payload, while nothing fill-side notices (Python integers do not overflow and t8n accepts the value). Timestamps are semantic and are never clamped or shifted; both directions fail the fill loudly. The appended block is serialized out-of-chain, in a new optional `syncPayload` field - the exact representation, field name and builder call `BlockchainEngineSyncFixture` has always used - rather than in-chain as the last payload: - The fixture's semantic surface stays the author's: `payloads`, `lastblockhash` and the post state describe exactly the chain the test wrote, and the fill verifies the test's post conditions before the extra block exists. In-chain placement would move all three onto framework-built state (post-Prague the appended block executes real system work, e.g. writing the head's hash into EIP-2935 history). - Engine API consumers are untouched: engine_x replays the same payloads as today at the same cost, and the format ignores unknown fields, so older readers skip the field wholesale and every append-class fixture stays consumable by them. - The fill-time consistency check compares the append-class payload list against the engine sibling at full strictness - nothing is dropped or scrubbed, because nothing shifts. Only the prepend class needs the leading-payload drop and the position scrub, keyed off the in-chain sync-phase tag. The prepended block stays in-chain as `payloads[0]`, tagged with the `sync` phase: there it is load-bearing ancestry - skipping it leaves the test's own block with an unknown parent - so every consumer must replay it. Both placements carry a digest of the pytest node id in their `extra_data`. The tests of a pre-allocation group share one reused client, and every announced head must be a block that client has never seen, so each sync attempt is attributable to its own test: a prepended block builds on the shared group genesis and would otherwise be identical across the group, and an appended block usually inherits uniqueness from its parent but two group tests with byte-identical chains would share it. The digest is taken over the test's own id: the fixture format and the xdist group suffix both ride along in the raw node id, and neither may reach the block, so a parallel fill builds the same chains as a sequential one. The option is scoped per fixture format - only blockchain_test_engine_x opts in, and formats sharing a non-empty t8n cache key must agree on it, which a unit test enforces - and is withheld from spec types that measure per-block (benchmark tests) and from any measuring session. It is off by default until the marker sweeps land: a fill without it is byte-identical to one from the unmodified code. Verified: unit tests pin the policy resolution for every chain class (including the refused-head cases), the prepend transform (salt digest, number shift, timestamp guard with its expected-invalid exemption), the append-side uint64 headroom guard at its exact boundary, and the consistency check's strict append-class comparison; pytester fills of a two-test pre-allocation group and an invalid singleton assert the appended block's placement, parentage, block number, salt length and untouched `lastblockhash`, the prepend shape, distinct per-test salts, and that a fill without the option emits no sync block in any format. `just static` and `just test-tests` pass. --- .../pytest_commands/plugins/filler/filler.py | 71 +++++ .../filler/tests/test_sync_block_fill.py | 276 +++++++++++++++++ .../plugins/filler/tests/test_t8n_cache.py | 36 ++- .../src/execution_testing/fixtures/base.py | 12 + .../execution_testing/fixtures/blockchain.py | 51 ++++ .../fixtures/engine_x_checks.py | 70 ++++- .../fixtures/tests/test_base.py | 32 ++ .../fixtures/tests/test_engine_x_checks.py | 188 +++++++++++- .../src/execution_testing/specs/base.py | 55 ++++ .../src/execution_testing/specs/benchmark.py | 7 + .../src/execution_testing/specs/blockchain.py | 289 +++++++++++++++++- .../specs/tests/test_sync_block.py | 269 ++++++++++++++++ 12 files changed, 1338 insertions(+), 18 deletions(-) create mode 100644 packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py create mode 100644 packages/testing/src/execution_testing/specs/tests/test_sync_block.py diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py index 7db3702718c..c2d97baf329 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py @@ -623,6 +623,38 @@ def pytest_addoption(parser: pytest.Parser) -> None: default=False, help="Fill tests using existing pre-allocation groups (phase 2 only).", ) + test_group.addoption( + "--sync-block", + action="store_true", + dest="sync_block", + default=False, + help=( + "Add one framework-built empty block to every blockchain " + "test's chain for fixture formats that opt in " + "(blockchain_test_engine_x), so that sync-based consumers " + "can trigger a devp2p sync. The block's placement is " + "resolved per test from the chain's structure: appended " + "above a fully valid chain's head (stored out-of-chain in " + "the fixture's `syncPayload` field, making every test " + "block a wire-guaranteed ancestor), prepended below a " + "single expected-invalid or Engine API-refused block " + "(stored in-chain as the first payload, tagged with the " + "`sync` phase), and omitted otherwise. Spec types that " + "opt out (benchmark tests) are filled without it. " + "Prepending shifts the invalid singleton's number and " + "hash, so those fixtures are not comparable with fixtures " + "filled without the option." + ), + ) + test_group.addoption( + "--no-sync-block", + action="store_false", + dest="sync_block", + help=( + "Do not add sync blocks to any fixture format; every " + "chain is built exactly as the test defines it." + ), + ) test_group.addoption( "--generate-all-formats", action="store_true", @@ -1504,6 +1536,23 @@ def _strip_xdist_group_suffix(s: str) -> str: return s +def _node_id_without_xdist_group(nodeid: str) -> str: + """ + Return the node id without any xdist group suffix. + + Under ``--dist=loadgroup`` the xdist worker appends ``@`` to + every grouped item's node id, so anything derived from the raw id + depends on whether the fill ran in parallel. Every group name the + fill sets is a bare word, while a parametrized node id always ends + in ``]``, so a trailing ``@`` segment without one is a group name + and never part of the test's own id. + """ + base, separator, suffix = nodeid.rpartition("@") + if separator and base and "]" not in suffix: + return base + return nodeid + + def node_to_test_info(node: pytest.Item) -> TestInfo: """Return test info of the current node item.""" # Strip xdist group suffix (@groupname) that may be added during execution. @@ -1614,6 +1663,28 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: kwargs["fork"] = fork op_mode: OpMode = request.config.op_mode # type: ignore kwargs["operation_mode"] = op_mode + # The sync block only applies to fixture formats that + # opt in (sync-based consumers need it); it is further + # withheld from spec types that measure per-block and + # from any session that is measuring. Where the block + # lands in an eligible test's chain - appended, + # prepended, or not at all - is resolved by the spec + # itself from the chain's structure. + kwargs["sync_block"] = ( + request.config.getoption("sync_block", False) + and fixture_format.sync_block + and cls.supports_sync_block + and op_mode != OpMode.BENCHMARKING + ) + # Salt with the test's own id, not with the raw node + # id: the fixture format and the xdist group suffix + # both ride along in the latter, and every format of + # one test must build the same chain (they share a + # t8n output cache) whether or not the fill ran in + # parallel. + kwargs["sync_block_salt"] = _node_id_without_xdist_group( + strip_fixture_format_from_node(request.node) + ) kwargs["is_tx_gas_heavy_test"] = is_tx_gas_heavy_test kwargs["is_exception_test"] = is_exception_test if ( diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py new file mode 100644 index 00000000000..a3445e6f05c --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py @@ -0,0 +1,276 @@ +""" +Test that the sync block reaches exactly the fixture formats that opt +in, in the placement each chain class calls for. + +The sync block is scoped per fixture format: only +``blockchain_test_engine_x`` declares ``sync_block`` true, so a fill +with ``--sync-block`` must emit engine_x fixtures carrying the extra +block - appended out-of-chain (the ``syncPayload`` field) for a valid +chain, prepended in-chain (a leading payload tagged with the ``sync`` +phase) for an invalid singleton - while every other format's chains +are byte-for-byte what the test defines. These tests fill single-block +tests and read the fixtures back, so a sync block that leaks into the +wrong format or placement, loses its phase tag, or fails to salt per +test fails here rather than in a consumer. +""" + +import json +import textwrap +from pathlib import Path +from typing import Any, Dict + +valid_test_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction + + + # Two tests of one pre-allocation group: a client reused across + # them must not already know either announced sync block. + @pytest.mark.parametrize("value", [1, 2]) + def test_single_block(blockchain_test, pre, value) -> None: + tx = Transaction( + to=0, + value=value, + gas_limit=21_000, + sender=pre.fund_eoa(), + ) + blockchain_test(pre=pre, post={}, blocks=[Block(txs=[tx])]) + """ +) + +invalid_singleton_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction, TransactionException + + + @pytest.mark.exception_test + def test_invalid_singleton(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=20_999, + sender=pre.fund_eoa(), + error=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + blockchain_test( + pre=pre, + post={}, + blocks=[ + Block( + txs=[tx], + exception=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + ], + ) + """ +) + + +def make_test_module(pytester: Any, source: str, name: str) -> Any: + """Write a test module into a pytester tests tree.""" + tests_dir = pytester.mkdir("tests") + cancun_tests_dir = tests_dir / "cancun" + cancun_tests_dir.mkdir() + module_dir = cancun_tests_dir / "sync_block_fill_module" + module_dir.mkdir() + test_module = module_dir / name + test_module.write_text(source) + pytester.copy_example( + name="src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-fill.ini" + ) + return test_module + + +def fill(pytester: Any, test_module: Any, *args: str) -> Path: + """ + Fill the module into a fresh output directory and return it. + + An all-formats fill is two pytest sessions - the `fill` CLI runs + phase 1 (pre-allocation grouping) and phase 2 (fixture filling) + back to back - so both are run here the same way. + """ + output = pytester.path / f"fixtures{len(args)}" + common = ( + "-c", + "pytest-fill.ini", + "--fork", + "Cancun", + "--generate-all-formats", + "--skip-index", + "--no-html", + f"--output={output}", + *args, + str(test_module.relative_to(pytester.path)), + ) + result = pytester.runpytest_subprocess( + "--generate-pre-alloc-groups", *common + ) + assert result.ret == 0, "fill phase 1 was expected to succeed" + result = pytester.runpytest_subprocess("--use-pre-alloc-groups", *common) + assert result.ret == 0, "fill phase 2 was expected to succeed" + return output + + +# Fixture directory to the format name that appears in a test id. +FORMATS = { + "blockchain_tests": "blockchain_test", + "blockchain_tests_engine": "blockchain_test_engine", + "blockchain_tests_engine_x": "blockchain_test_engine_x", +} + + +def fixtures_of_format( + output: Path, format_dir: str +) -> Dict[str, Dict[str, Any]]: + """ + Return the fixtures emitted in a format's directory, keyed by test + id with the format's own name stripped out so the same test's + fixtures line up across formats. + """ + fixtures: Dict[str, Dict[str, Any]] = {} + for path in sorted((output / format_dir).rglob("*.json")): + if "pre_alloc" in path.parts: + continue + for test_id, fixture in json.loads(path.read_text()).items(): + key = test_id.replace(f"-{FORMATS[format_dir]}", "") + fixtures[key] = fixture + assert fixtures, f"no {format_dir} fixtures were emitted" + return fixtures + + +def test_valid_chain_appends_out_of_chain(pytester: Any) -> None: + """ + A valid chain's sync block is appended: the engine_x payload list + stays exactly the test's own chain (as does every other format's), + and the extra block rides out-of-chain in ``syncPayload``, built + on the test's head. + """ + test_module = make_test_module( + pytester, valid_test_module, "test_single_block.py" + ) + output = fill(pytester, test_module, "--sync-block") + + for fixture in fixtures_of_format(output, "blockchain_tests").values(): + assert len(fixture["blocks"]) == 1 + for fixture in fixtures_of_format( + output, "blockchain_tests_engine" + ).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 1 + assert payloads[0].get("phase") is None + assert "syncPayload" not in fixture + + for fixture in fixtures_of_format( + output, "blockchain_tests_engine_x" + ).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 1, ( + "the appended sync block must not enter the payload list" + ) + assert payloads[0].get("phase") is None + own = payloads[0]["params"][0] + + sync_payload = fixture["syncPayload"] + assert sync_payload["phase"] == "sync", ( + "the sync payload must carry the sync phase tag" + ) + appended = sync_payload["params"][0] + assert appended["transactions"] == [], ( + "the sync block carries no transactions" + ) + assert appended["parentHash"] == own["blockHash"], ( + "the sync block must build on the test's own head" + ) + assert int(appended["blockNumber"], 16) == 2 + assert int(appended["timestamp"], 16) > int(own["timestamp"], 16) + assert len(appended["extraData"]) == 2 + 2 * 16, ( + "the sync block is salted with a digest of the test id" + ) + assert fixture["lastblockhash"] == own["blockHash"], ( + "the fixture's head stays the author's own block" + ) + + +def test_appended_sync_block_is_salted_per_test(pytester: Any) -> None: + """ + Each test's sync block must be unique to it, so every announced + head is a block the group's reused client has never seen and each + sync attempt is attributable to its own test. + """ + test_module = make_test_module( + pytester, valid_test_module, "test_single_block.py" + ) + output = fill(pytester, test_module, "--sync-block") + + salted = { + fixture["syncPayload"]["params"][0]["extraData"] + for fixture in fixtures_of_format( + output, "blockchain_tests_engine_x" + ).values() + } + assert len(salted) == 2 + + +def test_invalid_singleton_prepends_in_chain(pytester: Any) -> None: + """ + An invalid singleton's sync block is prepended: it is load-bearing + ancestry every consumer must replay, so it lives in-chain as the + first payload, tagged with the sync phase, and the test's own + block shifts up by one. No format gains an out-of-chain payload. + """ + test_module = make_test_module( + pytester, invalid_singleton_module, "test_invalid_singleton.py" + ) + output = fill(pytester, test_module, "--sync-block") + + for fixture in fixtures_of_format(output, "blockchain_tests").values(): + assert len(fixture["blocks"]) == 1 + for fixture in fixtures_of_format( + output, "blockchain_tests_engine" + ).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 1 + assert payloads[0].get("phase") is None + + for fixture in fixtures_of_format( + output, "blockchain_tests_engine_x" + ).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 2 + prepended, own = (payload["params"][0] for payload in payloads) + assert payloads[0].get("phase") == "sync", ( + "the prepended payload must carry the sync phase tag" + ) + assert payloads[1].get("phase") is None + assert prepended["transactions"] == [], ( + "the prepended block carries no transactions" + ) + assert int(prepended["blockNumber"], 16) == 1 + assert int(own["blockNumber"], 16) == 2 + assert own["parentHash"] == prepended["blockHash"] + assert "syncPayload" not in fixture, ( + "nothing can be appended above an invalid head" + ) + + +def test_sync_block_is_off_by_default(pytester: Any) -> None: + """Without the option no format gains a sync block.""" + test_module = make_test_module( + pytester, valid_test_module, "test_single_block.py" + ) + output = fill(pytester, test_module) + + for fixture in fixtures_of_format(output, "blockchain_tests").values(): + assert len(fixture["blocks"]) == 1 + for format_dir in ( + "blockchain_tests_engine", + "blockchain_tests_engine_x", + ): + for fixture in fixtures_of_format(output, format_dir).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 1 + assert payloads[0].get("phase") is None + assert "syncPayload" not in fixture diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py index 912c3a1d983..8d51a3ffe6a 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py @@ -17,7 +17,7 @@ ) from ...shared.helpers import labeled_format_parameter_set -from ..filler import _strip_xdist_group_suffix +from ..filler import _node_id_without_xdist_group, _strip_xdist_group_suffix class MockItem: @@ -238,6 +238,40 @@ def test_at_in_params_preserved(self) -> None: assert _strip_xdist_group_suffix(nodeid) == expected +class TestNodeIdWithoutXdistGroup: + """ + Test cases for _node_id_without_xdist_group. + + Values derived from the node id must not change when the same fill + runs in parallel, so every group suffix is stripped here, unlike + _strip_xdist_group_suffix which preserves the deliberate ones. + """ + + @pytest.mark.parametrize( + "group", ["t8n-cache-12345678", "bigmem", "custom_group"] + ) + def test_strips_every_group_suffix(self, group: str) -> None: + """Test that any group xdist appends is stripped.""" + expected = "test.py::test[params]" + assert _node_id_without_xdist_group(f"{expected}@{group}") == expected + + def test_no_suffix_unchanged(self) -> None: + """Test that nodeids without @ are unchanged.""" + nodeid = "test.py::test[params]" + assert _node_id_without_xdist_group(nodeid) == nodeid + + def test_at_in_params_preserved(self) -> None: + """Test that a parameter's own @ is not mistaken for a group.""" + nodeid = "test.py::test[email@example.com]" + assert _node_id_without_xdist_group(nodeid) == nodeid + + def test_at_in_params_with_group_suffix(self) -> None: + """Test that a group is stripped from a parameter containing @.""" + nodeid = "test.py::test[email@example.com]@bigmem" + expected = "test.py::test[email@example.com]" + assert _node_id_without_xdist_group(nodeid) == expected + + class TestCacheExecutionOrder: """Test that execution order maximizes cache hits.""" diff --git a/packages/testing/src/execution_testing/fixtures/base.py b/packages/testing/src/execution_testing/fixtures/base.py index 9b2fb475669..0af93daab97 100644 --- a/packages/testing/src/execution_testing/fixtures/base.py +++ b/packages/testing/src/execution_testing/fixtures/base.py @@ -91,6 +91,18 @@ class BaseFixture(CamelModel): FixtureFillingPhase.FILL_AFTER_PRE_ALLOC_GENERATION, } transition_tool_cache_key: ClassVar[str] = "" + sync_block: ClassVar[bool] = False + """ + Whether the filler's sync-block option applies to fixtures of this + format. + + Only formats whose consumers may need to trigger a devp2p sync + (``blockchain_test_engine_x``) opt in; every other format builds + the test's chain exactly as written. Formats that share a + non-empty ``transition_tool_cache_key`` must agree on this value: + the t8n output cache is positional, so formats sharing a key must + build byte-identical chains. + """ @classmethod def output_base_dir_name(cls) -> str: diff --git a/packages/testing/src/execution_testing/fixtures/blockchain.py b/packages/testing/src/execution_testing/fixtures/blockchain.py index a7331378786..726636c5009 100644 --- a/packages/testing/src/execution_testing/fixtures/blockchain.py +++ b/packages/testing/src/execution_testing/fixtures/blockchain.py @@ -922,6 +922,17 @@ class BlockchainEngineXFixture(BlockchainEngineFixtureCommon): FixtureFillingPhase.PRE_ALLOC_GENERATION, } transition_tool_cache_key: ClassVar[str] = "" + sync_block: ClassVar[bool] = True + """ + Engine X fixtures may be consumed by sync-based simulators, which + can only trigger a devp2p sync on a chain whose announced head has + an unknown parent; the filler therefore adds one framework-built + empty block to this format's chains, placed per test by + ``BlockchainTest.sync_block_policy``. The format opts out of the + t8n output cache (its phase-2 chains build on a group genesis), so + its chains diverging from the other blockchain formats' costs no + extra t8n work. + """ pre_hash: str """Hash of the pre-allocation group this test belongs to.""" @@ -937,6 +948,46 @@ class BlockchainEngineXFixture(BlockchainEngineFixtureCommon): ) """Engine API payloads for blockchain execution.""" + sync_payload: FixtureEngineNewPayload | None = None + """ + Framework-built empty payload appended above a fully valid chain's + head so that every payload in ``payloads`` is a wire-guaranteed + ancestor for sync-based consumers - the same out-of-chain + representation ``BlockchainEngineSyncFixture`` uses. + + Kept out of ``payloads`` because it is scaffolding, not test + content: ``payloads``, ``last_block_hash`` and the post state all + keep describing exactly the chain the test author wrote, and + consumers that replay payloads through the Engine API need not + know the field exists (this format ignores unknown fields, so + older readers skip it wholesale). A sync-based consumer announces + it instead of ``payloads[-1]`` and treats the chain as complete + when the client accepts it as head. + + ``None`` for chains that carry no appended block: a single + expected-invalid or Engine API-refused block gets a *prepended* + sync payload instead - in-chain as ``payloads[0]``, because there + the extra block is load-bearing ancestry every consumer must + replay - and every other ineligible chain is exactly the test's + own. + """ + + @property + def has_sync_payload(self) -> bool: + """ + Return whether the fixture carries a framework-injected sync + payload, appended (``sync_payload``) or prepended (a leading + ``payloads`` entry tagged with the sync phase). + + Derived from the payloads rather than stored, so the JSON + schema carries no separate switch. + """ + if self.sync_payload is not None: + return True + return ( + len(self.payloads) > 0 and self.payloads[0].phase == TestPhase.SYNC + ) + class BlockchainEngineStatefulFixture(BlockchainEngineFixtureCommon): """ diff --git a/packages/testing/src/execution_testing/fixtures/engine_x_checks.py b/packages/testing/src/execution_testing/fixtures/engine_x_checks.py index b81eaf38363..4bfceb69699 100644 --- a/packages/testing/src/execution_testing/fixtures/engine_x_checks.py +++ b/packages/testing/src/execution_testing/fixtures/engine_x_checks.py @@ -12,6 +12,25 @@ # payload is a pure function of the test's execution. _STATE_ROOT_DERIVED_FIELDS = ("stateRoot", "blockHash", "parentHash") +# Fields the prepended sync block additionally shifts in every payload +# above it: block numbers move up by one, unpinned timestamps move by +# the empty block's own second, slot numbers chain from the extra +# block's slot, and the block access list embeds position-derived +# content outright (the EIP-2935 system write records the parent's +# hash, which the prepend changes). Scrubbed only when the compared +# Engine X fixture carries a leading in-chain sync payload (the +# prepend class); every other fixture keeps the stricter comparison - +# an *appended* sync payload lives out-of-chain in `syncPayload`, +# shifts nothing, and never enters this comparison. +_SYNC_SHIFTED_FIELDS = ( + "blockNumber", + "timestamp", + "slotNumber", + "blockAccessList", +) + +_SYNC_PHASE = "sync" + class EngineXExecutionDriftError(Exception): """ @@ -67,14 +86,31 @@ def summary(self) -> str: return summary -def _scrubbed_payloads(fixture: Dict[str, Any]) -> List[Any]: - """Return the fixture's payload entries minus state-root-derived fields.""" +def _scrubbed_payloads( + fixture: Dict[str, Any], *, scrub_sync_shift: bool = False +) -> List[Any]: + """ + Return the fixture's payload entries minus state-root-derived + fields, and minus the position-derived fields when + ``scrub_sync_shift`` is set (the prepended empty block shifts them + in every payload above it). + + A leading sync-phase payload (the prepended empty block) is + dropped: it is framework-injected and has no sibling to compare + against. + """ + entries = fixture.get("engineNewPayloads", []) + if entries and entries[0].get("phase") == _SYNC_PHASE: + entries = entries[1:] + scrubbed_fields = _STATE_ROOT_DERIVED_FIELDS + ( + _SYNC_SHIFTED_FIELDS if scrub_sync_shift else () + ) payloads = [] - for entry in fixture.get("engineNewPayloads", []): + for entry in entries: entry = json.loads(json.dumps(entry)) params = entry.get("params") if params and isinstance(params[0], dict): - for field in _STATE_ROOT_DERIVED_FIELDS: + for field in scrubbed_fields: params[0].pop(field, None) payloads.append(entry) return payloads @@ -116,6 +152,19 @@ def verify_engine_x_execution( Engine X fixtures never share the transition tool output cache). All payload fields except the state-root-derived ones must match exactly. + An Engine X fixture whose chain starts with a sync-phase payload (the + empty block the filler prepends below a single invalid block for + sync-based consumers - the sibling formats never carry it) is compared + without that payload, and without the position-derived fields the + shift changes in every payload above it: `blockNumber`, unpinned + `timestamp`, `slotNumber`, and `blockAccessList`, whose EIP-2935 + system-write entry embeds the parent's hash; the genesis fee + compensation makes everything else identical. An *appended* sync + payload needs no such handling: it lives out-of-chain in + `syncPayload`, so the fixture's payload list is the author's own + chain - block access list included - and compares at full + strictness. + Return the comparison counts, or ``None`` when one of the two fixture format trees was not generated at all (e.g. when filling with ``-m blockchain_test_engine_x``, which produces no siblings). @@ -156,8 +205,17 @@ def verify_engine_x_execution( skipped += 1 continue compared += 1 - base = _scrubbed_payloads(sibling) - packed = _scrubbed_payloads(fixture) + payload_entries = fixture.get("engineNewPayloads", []) + has_sync_payload = bool( + payload_entries + and payload_entries[0].get("phase") == _SYNC_PHASE + ) + base = _scrubbed_payloads( + sibling, scrub_sync_shift=has_sync_payload + ) + packed = _scrubbed_payloads( + fixture, scrub_sync_shift=has_sync_payload + ) if base != packed: mismatches.append((test_id, _describe_mismatch(base, packed))) diff --git a/packages/testing/src/execution_testing/fixtures/tests/test_base.py b/packages/testing/src/execution_testing/fixtures/tests/test_base.py index a05dd2210d2..d8d7f3846b6 100644 --- a/packages/testing/src/execution_testing/fixtures/tests/test_base.py +++ b/packages/testing/src/execution_testing/fixtures/tests/test_base.py @@ -159,3 +159,35 @@ def test_base_fixtures_parsing(fixture: BaseFixture) -> None: json_dump = fixture.json_dict_with_info() assert json_dump is not None Fixtures.model_validate({"fixture": json_dump}) + + +def test_formats_sharing_a_cache_key_agree_on_sync_block() -> None: + """ + Fixture formats sharing a non-empty t8n cache key must agree on + ``sync_block``. + + The t8n output cache is positional: formats sharing a key replay + each other's transition results call by call, so they must build + byte-identical chains. A format that prepends a sync block below + an invalid singleton while a cache-sharing sibling does not would + consume the sibling's results one position off and emit chains no + client can execute. + """ + formats_by_cache_key: dict[str, list[type[BaseFixture]]] = {} + for format_class in BaseFixture.formats.values(): + cache_key = format_class.transition_tool_cache_key + if cache_key: + formats_by_cache_key.setdefault(cache_key, []).append(format_class) + assert formats_by_cache_key, "no fixture format declares a cache key" + for cache_key, format_classes in formats_by_cache_key.items(): + sync_block_values = { + format_class.sync_block for format_class in format_classes + } + assert len(sync_block_values) == 1, ( + f"formats sharing the t8n cache key {cache_key!r} disagree " + "on sync_block: " + + ", ".join( + f"{format_class.format_name}={format_class.sync_block}" + for format_class in format_classes + ) + ) diff --git a/packages/testing/src/execution_testing/fixtures/tests/test_engine_x_checks.py b/packages/testing/src/execution_testing/fixtures/tests/test_engine_x_checks.py index 7e44afee03c..ebbf35d7478 100644 --- a/packages/testing/src/execution_testing/fixtures/tests/test_engine_x_checks.py +++ b/packages/testing/src/execution_testing/fixtures/tests/test_engine_x_checks.py @@ -49,11 +49,15 @@ def _write_fixture( fixture_dir: str, test_id: str, payloads: List[Dict[str, Any]], + sync_payload: Dict[str, Any] | None = None, ) -> None: """Write a single-fixture file into a format tree.""" file = folder / fixture_dir / "prague" / "module" / "test_a.json" file.parent.mkdir(parents=True, exist_ok=True) - file.write_text(json.dumps({test_id: {"engineNewPayloads": payloads}})) + fixture: Dict[str, Any] = {"engineNewPayloads": payloads} + if sync_payload is not None: + fixture["syncPayload"] = sync_payload + file.write_text(json.dumps({test_id: fixture})) def test_identical_execution_passes(tmp_path: Path) -> None: @@ -211,3 +215,185 @@ def test_no_matching_siblings_reports_skip_count(tmp_path: Path) -> None: assert result is not None assert result.compared == 0 assert result.skipped == 1 + + +def _sync_payload() -> Dict[str, Any]: + """Build a framework sync block's newPayload entry.""" + payload = _payload(gas_used="0x0", state_root="0x33", block_hash="0x44") + payload["params"][0]["transactions"] = [] + payload["phase"] = "sync" + return payload + + +def _shifted( + payload: Dict[str, Any], *, number: str, timestamp: str +) -> Dict[str, Any]: + """Return a copy of `payload` at a shifted chain position.""" + shifted = json.loads(json.dumps(payload)) + shifted["params"][0]["blockNumber"] = number + shifted["params"][0]["timestamp"] = timestamp + return shifted + + +def test_sync_payload_is_ignored(tmp_path: Path) -> None: + """ + An Engine X fixture prepended with a sync-phase payload compares + clean against its unprepended sibling: the extra payload is + dropped and the block numbers and timestamps it shifts are + scrubbed. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + _write_fixture( + tmp_path, + SIBLING_FIXTURES_DIR, + SIBLING_ID, + [_shifted(base, number="0x1", timestamp="0xc")], + ) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [_sync_payload(), _shifted(base, number="0x2", timestamp="0xd")], + ) + + result = verify_engine_x_execution(tmp_path) + + assert result is not None + assert result.compared == 1 + + +def test_sync_payload_does_not_mask_drift(tmp_path: Path) -> None: + """A real execution difference still fails on a prepended fixture.""" + _write_fixture( + tmp_path, + SIBLING_FIXTURES_DIR, + SIBLING_ID, + [_payload(gas_used="0x5208", state_root="0x01", block_hash="0x02")], + ) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [ + _sync_payload(), + _payload(gas_used="0xbeef", state_root="0xaa", block_hash="0xbb"), + ], + ) + + with pytest.raises(EngineXExecutionDriftError) as exc_info: + verify_engine_x_execution(tmp_path) + + assert "gasUsed" in str(exc_info.value) + + +def test_appended_sync_payload_never_enters_the_comparison( + tmp_path: Path, +) -> None: + """ + An append-class fixture (out-of-chain ``syncPayload``) compares + its payload list against the sibling untouched: the appended block + shifts nothing, so nothing is dropped or scrubbed. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + _write_fixture( + tmp_path, + SIBLING_FIXTURES_DIR, + SIBLING_ID, + [_shifted(base, number="0x1", timestamp="0xc")], + ) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [_shifted(base, number="0x1", timestamp="0xc")], + sync_payload=_sync_payload(), + ) + + result = verify_engine_x_execution(tmp_path) + + assert result is not None + assert result.compared == 1 + + +def test_appended_sync_payload_keeps_full_strictness(tmp_path: Path) -> None: + """ + An append-class fixture must not inherit the prepend class's + position scrubbing: a shifted block number is a real difference. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + _write_fixture( + tmp_path, + SIBLING_FIXTURES_DIR, + SIBLING_ID, + [_shifted(base, number="0x1", timestamp="0xc")], + ) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [_shifted(base, number="0x2", timestamp="0xc")], + sync_payload=_sync_payload(), + ) + + with pytest.raises(EngineXExecutionDriftError) as exc_info: + verify_engine_x_execution(tmp_path) + + assert "blockNumber" in str(exc_info.value) + + +def test_prepended_fixture_scrubs_position_derived_content( + tmp_path: Path, +) -> None: + """ + A prepend-class fixture's shifted slot numbers and block access + list (whose EIP-2935 system write embeds the parent's hash) are + scrubbed alongside the block number and timestamp; a real + execution difference (gas used) still fails. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + sibling = _shifted(base, number="0x1", timestamp="0xc") + sibling["params"][0]["slotNumber"] = "0x1" + sibling["params"][0]["blockAccessList"] = "0xaa" + engine_x = _shifted(base, number="0x2", timestamp="0xd") + engine_x["params"][0]["slotNumber"] = "0x2" + engine_x["params"][0]["blockAccessList"] = "0xbb" + _write_fixture(tmp_path, SIBLING_FIXTURES_DIR, SIBLING_ID, [sibling]) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [_sync_payload(), engine_x], + ) + + result = verify_engine_x_execution(tmp_path) + + assert result is not None + assert result.compared == 1 + + +def test_appended_fixture_keeps_access_list_strictness( + tmp_path: Path, +) -> None: + """ + The append class must not inherit the prepend class's access-list + scrub: nothing shifts under an appended sync payload, so a block + access list difference is a real execution difference. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + sibling = json.loads(json.dumps(base)) + sibling["params"][0]["blockAccessList"] = "0xaa" + engine_x = json.loads(json.dumps(base)) + engine_x["params"][0]["blockAccessList"] = "0xbb" + _write_fixture(tmp_path, SIBLING_FIXTURES_DIR, SIBLING_ID, [sibling]) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [engine_x], + sync_payload=_sync_payload(), + ) + + with pytest.raises(EngineXExecutionDriftError) as exc_info: + verify_engine_x_execution(tmp_path) + + assert "blockAccessList" in str(exc_info.value) diff --git a/packages/testing/src/execution_testing/specs/base.py b/packages/testing/src/execution_testing/specs/base.py index bf0585396f8..66a9708ab2f 100644 --- a/packages/testing/src/execution_testing/specs/base.py +++ b/packages/testing/src/execution_testing/specs/base.py @@ -113,6 +113,50 @@ class BaseTest(BaseModel): # instead of each test having to set it ) operation_mode: OpMode | None = None + sync_block: bool = False + """ + Add one framework-built empty block to a blockchain test's chain + when filling for sync-based consumption. + + A client only starts a devp2p sync when the announced head's + parent is unknown to it, and blocks below the head can only reach + the client over devp2p - there is no Engine API path for them. The + placement of the extra block therefore decides what the wire + guarantees, and is resolved per test from the chain's own + structure (see ``BlockchainTest.sync_block_policy``): + + - a fully valid chain gets the block *appended* above its head, + making every one of the test's own blocks a wire-guaranteed + ancestor; + - a single expected-invalid (or Engine API-refused) block gets it + *prepended* between genesis and itself, giving the sync a reason + to start so the client can judge the announced head; + - every other chain is built exactly as the test defines it. + + Set by the filler only when the fixture format being filled opts + in via ``BaseFixture.sync_block`` (currently + ``blockchain_test_engine_x``), the spec type's + ``supports_sync_block`` is true, and the session is not measuring + gas. Ignored by test specs that do not build a chain of blocks, + and by stateful fixtures, whose chains continue a live client's + own head instead of a genesis the framework builds. + """ + sync_block_salt: str = "" + """ + Value mixed into the sync block's ``extra_data`` so its hash is + unique to this test. + + The tests of a pre-allocation group share a genesis and are served + to one reused client, and every announced head must be a block + that client has never seen, so each sync attempt is attributable + to its own test. A prepended block builds on the shared genesis + and would be identical across the group without the salt; an + appended block usually inherits uniqueness from its parent (the + test's own head), but two tests of a group whose chains are + byte-identical would share it. A per-test ``extra_data`` keeps the + block's state transition identical while making its hash unique. + The filler sets this to the pytest node id. + """ gas_optimization_max_gas_limit: int | None = None expected_benchmark_gas_used: int | None = None skip_gas_used_validation: bool = False @@ -121,6 +165,17 @@ class BaseTest(BaseModel): is_exception_test: bool = False # Class variables, to be set by subclasses + supports_sync_block: ClassVar[bool] = True + """ + Whether the filler's ``--sync-block`` option applies to this spec + type. + + Consensus test specs support it so sync-based consumers can + trigger a devp2p sync on their chains. Spec types whose + measurements the extra block would distort (benchmark tests) set + this to false and are filled without it even when the option is + given. + """ spec_types: ClassVar[Dict[str, Type["BaseTest"]]] = {} supported_fixture_formats: ClassVar[ Sequence[FixtureFormat | LabeledFixtureFormat] diff --git a/packages/testing/src/execution_testing/specs/benchmark.py b/packages/testing/src/execution_testing/specs/benchmark.py index c1c2d5358e6..d23cbd4f829 100644 --- a/packages/testing/src/execution_testing/specs/benchmark.py +++ b/packages/testing/src/execution_testing/specs/benchmark.py @@ -341,6 +341,13 @@ class BenchmarkTest(BaseTest): include_full_post_state_in_output: bool = False include_tx_receipts_in_output: bool = False + supports_sync_block: ClassVar[bool] = False + """ + Benchmark tests measure per-block gas and timing; a framework + block in the chain would distort those measurements, so the + filler's ``--sync-block`` option never applies to them. + """ + supported_fixture_formats: ClassVar[ Sequence[FixtureFormat | LabeledFixtureFormat] ] = [ diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index ba0f1f191fd..a4111868013 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -1,5 +1,7 @@ """Ethereum blockchain test spec definition and filler.""" +from enum import Enum +from hashlib import sha256 from pprint import pprint from typing import ( Any, @@ -103,6 +105,50 @@ from .debugging import print_traces from .helpers import verify_block, verify_transactions +DEFAULT_TIMESTAMP_INCREMENT = 12 +""" +Seconds between a block and its parent when the block does not pin a +timestamp of its own (see ``Block.set_environment``). +""" + + +class SyncBlockPolicy(Enum): + """ + Placement of the framework-built empty block that makes a chain + devp2p-syncable, resolved per test from the chain's own structure + (see ``BlockchainTest.sync_block_policy``). + + A client only starts a sync when the announced head's parent is + unknown to it, and only blocks *below* the head are guaranteed to + travel devp2p - the head's payload is always announced through the + Engine API, and whether a client also re-fetches it from a peer is + an implementation choice. The placement therefore decides which + blocks the wire guarantees. + """ + + NONE = "none" + """ + The chain is built exactly as the test defines it: it is either + already syncable (an invalid multi-block chain judges its head + after the valid ancestors arrive), cannot take an extra block in + either position, or the sync-block option is off. + """ + PREPEND = "prepend" + """ + One empty block between genesis and a single expected-invalid (or + Engine API-refused) block: nothing can be built on that block, so + the extra block below it is the only way to give the sync a reason + to start before the client judges the announced head. + """ + APPEND = "append" + """ + One empty block above a fully valid chain's head: the appended + block becomes the announced head and every one of the test's own + blocks becomes an ancestor, which a syncing client must fetch and + execute through its sync pipeline - the test's content is + wire-guaranteed by chain structure, on any client. + """ + def environment_from_parent_header(parent: "FixtureHeader") -> "Environment": """Instantiate new environment with the provided header as parent.""" @@ -442,7 +488,7 @@ def set_environment(self, env: Environment) -> Environment: else: assert env.parent_timestamp is not None new_env_values["timestamp"] = int( - Number(env.parent_timestamp) + 12 + Number(env.parent_timestamp) + DEFAULT_TIMESTAMP_INCREMENT ) return env.copy(**new_env_values) @@ -577,8 +623,20 @@ def engine_payload_modifier( ) return None - def get_fixture_engine_new_payload(self) -> FixtureEngineNewPayload: - """Get a FixtureEngineNewPayload from the built block.""" + def get_fixture_engine_new_payload( + self, phase: TestPhase | None = None + ) -> FixtureEngineNewPayload: + """ + Get a FixtureEngineNewPayload from the built block. + + ``phase`` overrides the phase auto-derived from the block's + transactions; the filler passes ``TestPhase.SYNC`` for the + empty sync block it builds itself, which carries no + transactions to derive a phase from. + """ + kwargs: Dict[str, Any] = {} + if phase is not None: + kwargs["phase"] = phase return FixtureEngineNewPayload.from_fixture_header( fork=self.fork, header=self.header, @@ -591,6 +649,7 @@ def get_fixture_engine_new_payload(self) -> FixtureEngineNewPayload: execution_payload_modifier=self.engine_payload_modifier(), validation_error=self.expected_exception, error_code=self.engine_api_error_code, + **kwargs, ) def verify_transactions( @@ -782,9 +841,12 @@ def discard_fixture_format_by_marks( and "blockchain_test_only" in marker_names ): return True + engine_formats: List[FixtureFormat] = [ + BlockchainEngineFixture, + BlockchainEngineXFixture, + ] if ( - fixture_format - not in [BlockchainEngineFixture, BlockchainEngineXFixture] + fixture_format not in engine_formats and "blockchain_test_engine_only" in marker_names ): return True @@ -1131,6 +1193,178 @@ def verify_post_state( print_traces(t8n.get_traces()) raise e + def sync_block_policy(self) -> SyncBlockPolicy: + """ + Resolve where this test's chain takes its sync block, from the + chain's statically declared structure. + + The resolution reads only what every ``Block`` declares at + collection time - its expected exceptions and Engine API error + code - so phase 1 of the two-phase fill (which hashes the + genesis environment for pre-allocation grouping) and phase 2 + (which builds the chain) always agree. + + - **Append** when every block is valid and none is refused at + the Engine API: the appended block announces the chain and + makes all of the test's own blocks wire-guaranteed + ancestors. + - **Prepend** for a single expected-invalid block, and equally + for a single block whose ``engine_api_error_code`` expects + the announcement itself to be refused: nothing can be built + on either, and without a block below them no sync can start. + An error-code head must keep its announcement - a block + appended above it would announce the chain instead and the + refusal the test asserts would never happen. + - **None** otherwise: an invalid multi-block chain already + carries valid ancestors that travel the wire, and its head + is judged after they arrive; a multi-block chain with an + error-code block keeps its own announcement for the same + reason the singleton does; an empty chain has nothing to + announce. + """ + if not self.sync_block or not self.blocks: + return SyncBlockPolicy.NONE + engine_refused = any( + block.engine_api_error_code is not None for block in self.blocks + ) + invalid = any(block.exception is not None for block in self.blocks) + if len(self.blocks) == 1 and (invalid or engine_refused): + return SyncBlockPolicy.PREPEND + if not invalid and not engine_refused: + return SyncBlockPolicy.APPEND + return SyncBlockPolicy.NONE + + def sync_block_extra_data(self) -> Bytes: + """ + Return the sync block's ``extra_data``: a digest of the test's + salt, making the block's hash unique to this test (see + ``BaseTest.sync_block_salt``). + """ + return Bytes(sha256(self.sync_block_salt.encode()).digest()[:16]) + + def blocks_to_build(self) -> List[Block]: + """ + Return the chain's block list, honoring the resolved sync-block + policy. + + Under the prepend policy, one empty block is inserted between + genesis and the test's single expected-invalid (or Engine + API-refused) block, so the announced head has an unknown + parent and sync-based consumers can trigger a devp2p sync + before the client judges it. + + The prepended block's timestamp is pinned to one second after + genesis, below any timestamp a test is likely to pin. Its + ``extra_data`` carries the per-test salt digest (see + ``BaseTest.sync_block_salt``). A block that pins an absolute + ``number`` (state tests converted to blockchain tests always + do) is shifted up by one so the chain stays contiguous; a + deliberately wrong number stays wrong relative to the shifted + chain. + + A pinned timestamp the prepended block's own timestamp does + not clear fails the fill loudly: timestamps are semantic (fork + activation, TIMESTAMP expectations), so they are never + shifted, and building the chain anyway would produce a + non-monotonic - consensus-invalid - sequence. + + The appended sync block is deliberately *not* part of this + list: it is built on the chain's post-state after the test's + own blocks (and their post-state verification) are done, and + lives out-of-chain in the fixture's ``sync_payload`` field, so + the chain built here is byte-for-byte the author's. + """ + if self.sync_block_policy() is not SyncBlockPolicy.PREPEND: + return self.blocks + genesis_timestamp = int(self.get_genesis_environment().timestamp) + self._verify_timestamps_clear_prepended_block(genesis_timestamp) + blocks: List[Block] = [ + Block( + timestamp=HexNumber(genesis_timestamp + 1), + extra_data=self.sync_block_extra_data(), + ) + ] + for block in self.blocks: + if block.number is not None: + block = block.model_copy( + update={"number": HexNumber(block.number + 1)} + ) + blocks.append(block) + return blocks + + def _verify_timestamps_clear_prepended_block( + self, genesis_timestamp: int + ) -> None: + """ + Refuse to prepend when a pinned timestamp cannot follow the + empty block. + + The walk mirrors the block builder: an unpinned block gets its + parent's timestamp plus ``DEFAULT_TIMESTAMP_INCREMENT``, which + can never violate monotonicity, so only pinned timestamps can + collide. A block that declares a block exception is exempt - a + deliberately invalid timestamp stays invalid relative to the + shifted chain - and does not advance the walk, since an + invalid block is rolled back and its successor builds on the + previous head. + + The prepend policy currently confines the walk to single-block + chains, where the check only ever fires for an Engine + API-refused block (an expected-invalid one is exempt). The + walk keeps its general form so it stays correct for any chain + the policy may hand it. + """ + previous_timestamp = genesis_timestamp + 1 + for index, block in enumerate(self.blocks): + if block.exception is not None: + continue + if block.timestamp is None: + previous_timestamp += DEFAULT_TIMESTAMP_INCREMENT + continue + pinned = int(block.timestamp) + if pinned <= previous_timestamp: + raise ValueError( + f"the test's block {index + 1} pins timestamp " + f"{pinned}, which does not clear its parent's " + f"{previous_timestamp}: the chain would be " + "non-monotonic and consensus-invalid. The prepended " + "sync block occupies timestamp " + f"{genesis_timestamp + 1} (genesis + 1) and every " + "block above it must clear its own parent, so raise " + "the pinned timestamps to leave the extra block " + "room." + ) + previous_timestamp = pinned + + def _verify_sync_block_timestamp_headroom( + self, head_timestamp: int + ) -> None: + """ + Refuse to append when the chain's head leaves the sync block + no room below the uint64 ceiling. + + The appended block takes its parent's timestamp plus + ``DEFAULT_TIMESTAMP_INCREMENT``, and a block timestamp must + fit uint64: above a head pinned at or next to ``2**64 - 1`` + the trailer's timestamp does not, and no client can even + parse the resulting payload. Nothing downstream notices + otherwise - Python integers do not overflow and t8n accepts + the value - so the fill refuses loudly instead of emitting an + unusable fixture. The mirror of the prepend side's + monotonicity walk: timestamps are semantic and are never + clamped or shifted. + """ + sync_block_timestamp = head_timestamp + DEFAULT_TIMESTAMP_INCREMENT + if sync_block_timestamp > 2**64 - 1: + raise ValueError( + f"the chain's head pins timestamp {head_timestamp}, " + "leaving no uint64 headroom for the appended sync " + f"block: {head_timestamp} + " + f"{DEFAULT_TIMESTAMP_INCREMENT} exceeds 2**64 - 1, " + "and no client can parse a block whose timestamp " + "does not fit uint64." + ) + def make_fixture( self, t8n: FillerBackend, @@ -1148,7 +1382,8 @@ def make_fixture( benchmark_gas_used: int | None = None benchmark_block_gas_used: int | None = None benchmark_opcode_count: OpcodeCount | None = None - for block in self.blocks: + blocks = self.blocks_to_build() + for index, block in enumerate(blocks): # This is the most common case, the RLP needs to be constructed # based on the transactions to be included in the block. # Set the environment according to the block to execute. @@ -1159,7 +1394,7 @@ def make_fixture( previous_alloc=alloc, ) block_number = int(built_block.header.number) - is_last_block = block is self.blocks[-1] + is_last_block = index == len(blocks) - 1 if is_last_block and self.operation_mode == OpMode.BENCHMARKING: benchmark_gas_used = built_block.cumulative_gas_used() benchmark_block_gas_used = built_block.block_gas_used() @@ -1253,7 +1488,9 @@ def make_hive_fixture( benchmark_gas_used: int | None = None benchmark_block_gas_used: int | None = None benchmark_opcode_count: OpcodeCount | None = None - for block in self.blocks: + sync_block_policy = self.sync_block_policy() + blocks = self.blocks_to_build() + for index, block in enumerate(blocks): built_block = self.generate_block_data( t8n=t8n, block=block, @@ -1261,7 +1498,7 @@ def make_hive_fixture( previous_alloc=alloc, ) block_number = int(built_block.header.number) - is_last_block = block is self.blocks[-1] + is_last_block = index == len(blocks) - 1 if is_last_block and self.operation_mode == OpMode.BENCHMARKING: benchmark_gas_used = built_block.cumulative_gas_used() benchmark_block_gas_used = built_block.block_gas_used() @@ -1272,7 +1509,15 @@ def make_hive_fixture( block_number=block_number, ) fixture_payloads.append( - built_block.get_fixture_engine_new_payload() + built_block.get_fixture_engine_new_payload( + # The prepended sync block is always the first + # block built; tag it so consumers can tell the + # framework-injected payload from the test's own. + phase=TestPhase.SYNC + if sync_block_policy is SyncBlockPolicy.PREPEND + and index == 0 + else None + ) ) if block.exception is None: alloc = built_block.alloc @@ -1333,6 +1578,30 @@ def make_hive_fixture( "pre_hash": "", # Will be set by BaseTestWrapper } ) + if sync_block_policy is SyncBlockPolicy.APPEND: + assert env.parent_timestamp is not None + self._verify_sync_block_timestamp_headroom( + int(env.parent_timestamp) + ) + # Build the salted empty sync block on the test's + # post-state, after the post-state verification above, + # and keep it out of the payload list: the fixture's + # payloads, head and post state stay exactly the + # author's chain, and the appended block makes every + # one of them a wire-guaranteed ancestor (same + # representation as BlockchainEngineSyncFixture's + # sync_payload below). + sync_built_block = self.generate_block_data( + t8n=t8n, + block=Block(extra_data=self.sync_block_extra_data()), + previous_env=env, + previous_alloc=alloc, + ) + fixture_data["sync_payload"] = ( + sync_built_block.get_fixture_engine_new_payload( + phase=TestPhase.SYNC + ) + ) fixture = BlockchainEngineXFixture(**fixture_data) elif fixture_format == BlockchainEngineSyncFixture: # Sync fixture format diff --git a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py new file mode 100644 index 00000000000..8454f730e13 --- /dev/null +++ b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py @@ -0,0 +1,269 @@ +""" +Tests for the per-class sync-block policy and its chain transforms. + +The filler adds one framework-built empty block to engine_x chains so +sync-based consumers can trigger a devp2p sync; where the block lands +is resolved per test from the chain's statically declared structure: +appended above a fully valid chain (out-of-chain, the fixture's +``sync_payload``), prepended below a single expected-invalid or Engine +API-refused block (in-chain, shifting the singleton up), and nowhere +otherwise. These tests pin the resolution and the prepend transform; +the appended block is built by ``make_hive_fixture`` against a real +backend and is covered by the filler plugin's pytester tests. +""" + +from hashlib import sha256 +from typing import List + +import pytest + +from execution_testing.base_types import HexNumber +from execution_testing.exceptions import BlockException, EngineAPIError +from execution_testing.forks import Cancun +from execution_testing.specs.benchmark import BenchmarkTest +from execution_testing.specs.blockchain import ( + Block, + BlockchainTest, + SyncBlockPolicy, +) +from execution_testing.specs.state import StateTest +from execution_testing.test_types import Alloc, Environment, Transaction + +SALT = "tests/cancun/test_x.py::test_y[fork_Cancun-blockchain_test]" + +INVALID = Block( + timestamp=1_000, exception=BlockException.INCORRECT_BLOCK_FORMAT +) +REFUSED = Block( + timestamp=1_000, engine_api_error_code=EngineAPIError.InvalidParams +) +VALID = Block(timestamp=1_000) + + +def make_test( + *, blocks: List[Block], salt: str = SALT, sync_block: bool = True +) -> BlockchainTest: + """Create a Cancun blockchain test over the given blocks.""" + return BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=blocks, + sync_block=sync_block, + sync_block_salt=salt, + ) + + +@pytest.mark.parametrize( + "blocks,policy", + [ + pytest.param([VALID], SyncBlockPolicy.APPEND, id="valid_singleton"), + pytest.param( + [VALID, Block(timestamp=2_000)], + SyncBlockPolicy.APPEND, + id="valid_multi_block", + ), + pytest.param( + [INVALID], SyncBlockPolicy.PREPEND, id="invalid_singleton" + ), + pytest.param( + [REFUSED], + SyncBlockPolicy.PREPEND, + id="engine_refused_singleton", + ), + pytest.param( + [ + Block( + timestamp=1_000, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + engine_api_error_code=EngineAPIError.InvalidParams, + ) + ], + SyncBlockPolicy.PREPEND, + id="invalid_and_refused_singleton", + ), + pytest.param( + [ + VALID, + Block( + timestamp=2_000, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + ), + ], + SyncBlockPolicy.NONE, + id="invalid_multi_block", + ), + pytest.param( + [ + Block( + timestamp=1_000, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + ), + Block(timestamp=2_000), + ], + SyncBlockPolicy.NONE, + id="mid_chain_invalid_block", + ), + pytest.param( + [ + VALID, + Block( + timestamp=2_000, + engine_api_error_code=EngineAPIError.InvalidParams, + ), + ], + SyncBlockPolicy.NONE, + id="engine_refused_multi_block_head", + ), + pytest.param([], SyncBlockPolicy.NONE, id="empty_chain"), + ], +) +def test_sync_block_policy_resolution( + blocks: List[Block], policy: SyncBlockPolicy +) -> None: + """ + The policy must follow the chain's statically declared structure. + + An Engine API-refused block must never sit below an appended sync + block: the appended block would become the announced head, the + refused block would arrive over devp2p instead of through + `newPayload`, and the refusal the test asserts would never happen. + """ + assert make_test(blocks=blocks).sync_block_policy() is policy + + +def test_sync_block_disabled_resolves_to_none() -> None: + """Without the option every chain keeps its own structure.""" + test = make_test(blocks=[VALID], sync_block=False) + assert test.sync_block_policy() is SyncBlockPolicy.NONE + assert test.blocks_to_build() is test.blocks + + +def test_append_class_chain_is_untouched() -> None: + """ + The appended sync block must not appear in the chain's block list: + it is built on the test's post-state after the chain (and its + post-state verification) is done, and lives out-of-chain in the + fixture's ``sync_payload`` field. + """ + test = make_test(blocks=[VALID]) + assert test.blocks_to_build() is test.blocks + + +def test_prepend_inserts_one_salted_empty_block() -> None: + """ + The prepend-class chain gains one empty block at genesis + 1 + carrying a digest of the test's salt in its ``extra_data``. + """ + test = make_test(blocks=[INVALID]) + blocks = test.blocks_to_build() + assert len(blocks) == 2 + prepended = blocks[0] + assert prepended.timestamp == 1 + assert prepended.txs == [] + assert prepended.exception is None + assert prepended.extra_data == sha256(SALT.encode()).digest()[:16] + assert blocks[1] is test.blocks[0], "the test's own block passes through" + + +def test_sync_block_differs_per_test() -> None: + """ + Tests of a pre-allocation group must not share the sync block: a + client that already knows the announced head never starts a sync. + """ + first = make_test(blocks=[INVALID], salt="a").blocks_to_build() + second = make_test(blocks=[INVALID], salt="b").blocks_to_build() + assert first[0].extra_data != second[0].extra_data + + +def test_prepend_shifts_a_pinned_block_number() -> None: + """A block pinning an absolute number shifts up to stay contiguous.""" + test = make_test( + blocks=[ + Block( + number=1, + timestamp=1_000, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + ) + ] + ) + numbers = [block.number for block in test.blocks_to_build()] + assert numbers == [None, 2] + assert test.blocks[0].number == 1, "the test's own block is untouched" + + +def test_expected_invalid_block_is_exempt_from_the_timestamp_walk() -> None: + """ + An expected-invalid block pinning the prepended block's own + timestamp still fills: it is rejected and rolled back anyway, and + the collision only adds one more reason. + """ + test = make_test( + blocks=[ + Block(timestamp=1, exception=BlockException.INCORRECT_BLOCK_FORMAT) + ] + ) + assert len(test.blocks_to_build()) == 2 + + +def test_engine_refused_block_timestamp_collision_is_refused() -> None: + """ + An Engine API-refused block is consensus-valid, so a pinned + timestamp the prepended block does not clear would build a + non-monotonic - consensus-invalid - chain and must fail loudly. + """ + test = make_test( + blocks=[ + Block( + timestamp=1, + engine_api_error_code=EngineAPIError.InvalidParams, + ) + ] + ) + with pytest.raises(ValueError, match="does not clear its parent"): + test.blocks_to_build() + + +def test_benchmark_tests_opt_out_of_the_sync_block() -> None: + """ + Benchmark tests must never receive a sync block: a framework block + in the chain would distort their per-block measurements. + """ + assert BenchmarkTest.supports_sync_block is False + assert BlockchainTest.supports_sync_block is True + assert StateTest.supports_sync_block is True + + +def test_sync_block_timestamp_headroom() -> None: + """ + A head timestamp must leave the appended sync block room below + the uint64 ceiling: the last parent the increment clears passes, + and the first that overflows - and the ceiling itself - are + refused loudly. + """ + test = make_test(blocks=[VALID]) + test._verify_sync_block_timestamp_headroom(2**64 - 13) + with pytest.raises(ValueError, match="no uint64 headroom"): + test._verify_sync_block_timestamp_headroom(2**64 - 12) + with pytest.raises(ValueError, match="no uint64 headroom"): + test._verify_sync_block_timestamp_headroom(2**64 - 1) + + +def test_state_test_conversion_carries_the_sync_block_fields() -> None: + """ + A converted state test resolves its policy like the blockchain + test it becomes: its single valid block appends, and the salt + rides along. + """ + state_test = StateTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + tx=Transaction(sender=HexNumber(0), to=HexNumber(0), nonce=0), + env=Environment(number=1, timestamp=1_000), + sync_block=True, + sync_block_salt=SALT, + ) + blockchain_test = state_test.generate_blockchain_test() + assert blockchain_test.sync_block_policy() is SyncBlockPolicy.APPEND + assert blockchain_test.sync_block_salt == SALT From 622b2ea781bf5643ab2748bb091b77f19420754d Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 16:22:52 +0200 Subject: [PATCH 03/12] feat(test-fill): compensate genesis fees for the prepended sync block The prepended sync block participates in fee mechanics: it decays the base fee by one EIP-1559 step and the excess blob gas by one target, so a prepend-class test pinning those values would fill against a fee environment one step below the one its author specified. On the prepend-everywhere design this class dominated the Cancun refill catalogue (674 of 681 failures); under the per-class policy it is confined to invalid singletons, where fee precision is often the test's whole point (insufficient-balance variants pin single-wei sender balances against exact blob fee values). Wind genesis one progression step up in get_genesis_environment when the resolved policy is prepend, so the empty block consumes exactly the step it introduces: it lands precisely on the author's genesis fee values and the test's own block derives its fee context from an identical parent. The preimages are exact, found by a bounded scan and verified forward with the fork's own calculators; a value with no preimage fails the fill loudly instead of producing a semantically shifted fixture. The compensation lives in get_genesis_environment because phase 1 of the two-phase fill hashes that environment for pre-allocation grouping - both phases must see the same genesis, and the policy resolution reads only statically declared block fields, so they always agree. Append-class chains are untouched: nothing sits below the test's own blocks, so the author's genesis passes through and valid tests share pre-allocation groups exactly as before the sync-block option existed. State tests compose transparently: their conversion already winds genesis one step up from the pinned block environment, and this adds the one further step the prepended block consumes. Verified: the 188 insufficient-balance blob variants at Cancun (single-wei balance precision plus exact excess blob gas, all resolving to the prepend class) fill clean with --sync-block; the same fill without this commit's compensation fails loudly, the execution-consistency check reporting 144 of the 188 engine_x fixtures executing differently from their engine siblings - the check doubles as a tripwire for any future fee-environment drift. An option-off fill of ported stRefundTest at Cancun is content-identical to one from the branch's base across all 104 fixtures of every format (per-fixture content hashes compared). Unit tests round-trip both preimages through the fork's own calculators, pin the refusal for values a fee floor makes unreachable, the prepend-class compensation (blob and pre-blob forks), the untouched append-class genesis, and the two-step state-test composition; `just static` and `just test-tests` pass. --- .../src/execution_testing/specs/blockchain.py | 147 +++++++++++- .../specs/tests/test_sync_block.py | 217 +++++++++++++++++- 2 files changed, 361 insertions(+), 3 deletions(-) diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index a4111868013..73f495176e8 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -185,6 +185,79 @@ def apply_new_parent( return env.copy(**updated) +def empty_block_base_fee_preimage( + *, fork: Fork, base_fee_per_gas: int, gas_limit: int +) -> int: + """ + Return the parent base fee from which an empty block derives + ``base_fee_per_gas``. + + An empty parent is below target, so the EIP-1559 progression is a + pure decay step; every value has an exact preimage at or near + ``value * d // (d - 1)`` for change denominator ``d``. Scan the + candidates and verify each with the fork's own calculator so the + result is exact by construction, never approximated. + + The parent here is genesis, which consumes no gas; a test pinning a + nonzero ``gas_used`` on its genesis environment would decay from a + different point than this scan assumes, and its own fee + expectations would then fail the fill. + """ + calculate_base_fee = fork.base_fee_per_gas_calculator() + denominator = fork.base_fee_max_change_denominator() + guess = base_fee_per_gas * denominator // (denominator - 1) + for candidate in range(max(base_fee_per_gas, guess - 2), guess + 3): + derived = calculate_base_fee( + parent_base_fee_per_gas=candidate, + parent_gas_used=0, + parent_gas_limit=gas_limit, + ) + if derived == base_fee_per_gas: + return candidate + raise ValueError( + f"no parent base fee decays to {base_fee_per_gas} on an empty " + f"block at fork {fork.name()}; the prepended sync block cannot " + "preserve this test's fee environment" + ) + + +def empty_block_excess_blob_gas_preimage( + *, fork: Fork, excess_blob_gas: int, parent_base_fee_per_gas: int +) -> int: + """ + Return the parent excess blob gas from which an empty block derives + ``excess_blob_gas``. + + An empty parent uses no blob gas, so the progression either decays + the excess by one target (EIP-4844) or, on forks where a fee floor + holds the value in place, leaves it unchanged; the preimage is one + of two candidates, each verified with the fork's own calculator. + Genesis is that parent and uses no blob gas either, so a test + pinning a nonzero ``blob_gas_used`` on its genesis environment + would progress from a different point than these candidates assume. + + Some values are unreachable: under a fee floor the decay is skipped + while the floor holds, so a small nonzero excess has no parent at + all. Such a test cannot take the prepended sync block and its fill + is refused loudly. + """ + calculate_excess_blob_gas = fork.excess_blob_gas_calculator() + target = fork.target_blobs_per_block() * fork.blob_gas_per_blob() + for candidate in (excess_blob_gas + target, excess_blob_gas): + derived = calculate_excess_blob_gas( + parent_excess_blob_gas=candidate, + parent_blob_gas_used=0, + parent_base_fee_per_gas=parent_base_fee_per_gas, + ) + if derived == excess_blob_gas: + return candidate + raise ValueError( + f"no parent excess blob gas decays to {excess_blob_gas} on an " + f"empty block at fork {fork.name()}; the prepended sync block " + "cannot preserve this test's fee environment" + ) + + def count_blobs(txs: List[Transaction]) -> int: """Return number of blobs in a list of transactions.""" return sum( @@ -853,11 +926,81 @@ def discard_fixture_format_by_marks( return False def get_genesis_environment(self) -> Environment: - """Get the genesis environment for pre-allocation groups.""" + """ + Get the genesis environment for pre-allocation groups. + + When the resolved sync-block policy is prepend, the genesis fee + fields are wound one progression step up so that the prepended + sync block consumes exactly the step it introduces (see + ``_compensate_genesis_fees``). This must happen here rather + than in ``make_genesis``: phase 1 of the two-phase fill hashes + this environment for pre-allocation grouping, so both phases + must see the same compensated genesis. Append-class chains need + no compensation - nothing sits below the test's own blocks, so + the author's genesis passes through untouched. + """ modified_values = self.genesis_environment.set_fork_requirements( self.fork.transitions_from() ).model_dump(exclude_unset=True) - return Environment(**(GENESIS_ENVIRONMENT_DEFAULTS | modified_values)) + env = Environment(**(GENESIS_ENVIRONMENT_DEFAULTS | modified_values)) + if self.sync_block_policy() is SyncBlockPolicy.PREPEND: + env = self._compensate_genesis_fees(env) + return env + + def _compensate_genesis_fees(self, env: Environment) -> Environment: + """ + Wind the genesis fee fields one progression step up to cancel + the prepended sync block's step. + + The prepended block (see ``blocks_to_build``) sits between + genesis and the test's single block and participates in fee + mechanics: it decays the base fee by one EIP-1559 step and the + excess blob gas by one target. Starting genesis one exact + preimage step higher makes the prepended block land precisely + on the fee values the test author gave for genesis, so the + test's own block derives its fee context from an identical + parent and executes in exactly the environment the author + specified. State tests compose transparently: their conversion + already winds genesis one step up from the pinned block + environment, and this adds the one further step the prepended + block consumes. + + Fee fields the genesis fork does not require are left alone; a + value with no preimage fails the fill loudly rather than + producing a semantically shifted fixture. + """ + fork = self.fork.fork_at( + block_number=int(env.number) + 1, + timestamp=int(env.timestamp) + 1, + ) + updates: Dict[str, Any] = {} + base_fee_per_gas: int | None = ( + None if env.base_fee_per_gas is None else int(env.base_fee_per_gas) + ) + if base_fee_per_gas is not None and fork.header_base_fee_required(): + base_fee_per_gas = empty_block_base_fee_preimage( + fork=fork, + base_fee_per_gas=base_fee_per_gas, + gas_limit=int(env.gas_limit), + ) + updates["base_fee_per_gas"] = HexNumber(base_fee_per_gas) + if ( + env.excess_blob_gas is not None + and fork.header_excess_blob_gas_required() + ): + assert base_fee_per_gas is not None, ( + "excess blob gas compensation requires a genesis base fee" + ) + updates["excess_blob_gas"] = HexNumber( + empty_block_excess_blob_gas_preimage( + fork=fork, + excess_blob_gas=int(env.excess_blob_gas), + parent_base_fee_per_gas=base_fee_per_gas, + ) + ) + if not updates: + return env + return env.copy(**updates) def make_genesis( self, *, apply_pre_allocation_blockchain: bool diff --git a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py index 8454f730e13..fd31e8bf974 100644 --- a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py +++ b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py @@ -19,12 +19,14 @@ from execution_testing.base_types import HexNumber from execution_testing.exceptions import BlockException, EngineAPIError -from execution_testing.forks import Cancun +from execution_testing.forks import Cancun, Fork, London, Osaka, Shanghai from execution_testing.specs.benchmark import BenchmarkTest from execution_testing.specs.blockchain import ( Block, BlockchainTest, SyncBlockPolicy, + empty_block_base_fee_preimage, + empty_block_excess_blob_gas_preimage, ) from execution_testing.specs.state import StateTest from execution_testing.test_types import Alloc, Environment, Transaction @@ -267,3 +269,216 @@ def test_state_test_conversion_carries_the_sync_block_fields() -> None: blockchain_test = state_test.generate_blockchain_test() assert blockchain_test.sync_block_policy() is SyncBlockPolicy.APPEND assert blockchain_test.sync_block_salt == SALT + + +GAS_LIMIT = 100_000_000 +CANCUN_TARGET_BLOB_GAS = ( + Cancun.target_blobs_per_block() * Cancun.blob_gas_per_blob() +) + + +@pytest.mark.parametrize( + "base_fee_per_gas", + [0, 1, 6, 7, 8, 9, 10, 100, 1_000, 875_000_000, 1_000_000_000], +) +@pytest.mark.parametrize("fork", [London, Cancun]) +def test_base_fee_preimage_round_trip( + fork: Fork, base_fee_per_gas: int +) -> None: + """An empty block must decay the preimage to the intended value.""" + preimage = empty_block_base_fee_preimage( + fork=fork, + base_fee_per_gas=base_fee_per_gas, + gas_limit=GAS_LIMIT, + ) + derived = fork.base_fee_per_gas_calculator()( + parent_base_fee_per_gas=preimage, + parent_gas_used=0, + parent_gas_limit=GAS_LIMIT, + ) + assert derived == base_fee_per_gas + + +@pytest.mark.parametrize( + "excess_blob_gas", + [ + 0, + 0x20000, + 0x60000, + 0x80000, + 0xE0000, + 0x12345, # not a multiple of the blob gas quantum + ], +) +def test_excess_blob_gas_preimage_round_trip(excess_blob_gas: int) -> None: + """An empty block must decay the preimage to the intended value.""" + preimage = empty_block_excess_blob_gas_preimage( + fork=Cancun, + excess_blob_gas=excess_blob_gas, + parent_base_fee_per_gas=7, + ) + derived = Cancun.excess_blob_gas_calculator()( + parent_excess_blob_gas=preimage, + parent_blob_gas_used=0, + parent_base_fee_per_gas=7, + ) + assert derived == excess_blob_gas + + +@pytest.mark.parametrize("excess_blobs", [1, 2, 3, 4, 5]) +def test_excess_blob_gas_with_no_preimage_is_refused( + excess_blobs: int, +) -> None: + """ + A value the reserve price makes unreachable must be refused. + + Above the reserve price an empty block leaves the excess where it + is, so a genesis winding up to a small nonzero excess does not + exist. Refusing loudly here is what keeps a silently shifted fee + environment out of the fixtures. + """ + with pytest.raises(ValueError, match="no parent excess blob gas"): + empty_block_excess_blob_gas_preimage( + fork=Osaka, + excess_blob_gas=excess_blobs * Osaka.blob_gas_per_blob(), + parent_base_fee_per_gas=17, + ) + + +def make_fee_pinning_test(*, blocks: List[Block]) -> BlockchainTest: + """Create a Cancun blockchain test pinning genesis fee values.""" + return BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=blocks, + genesis_environment=Environment( + base_fee_per_gas=HexNumber(10), + excess_blob_gas=HexNumber(0x80000), + ), + sync_block=True, + sync_block_salt="test", + ) + + +def test_append_class_genesis_is_not_compensated() -> None: + """ + A valid chain's genesis passes through untouched: nothing sits + below the test's own blocks, so there is no fee step to cancel. + """ + env = make_fee_pinning_test(blocks=[VALID]).get_genesis_environment() + assert env.base_fee_per_gas == 10 + assert env.excess_blob_gas == 0x80000 + + +def test_prepend_class_genesis_compensation() -> None: + """ + One empty-block step from an invalid singleton's compensated + genesis lands on the author's genesis values. + """ + env = make_fee_pinning_test(blocks=[INVALID]).get_genesis_environment() + assert env.base_fee_per_gas is not None + assert env.excess_blob_gas is not None + derived_base_fee = Cancun.base_fee_per_gas_calculator()( + parent_base_fee_per_gas=int(env.base_fee_per_gas), + parent_gas_used=0, + parent_gas_limit=int(env.gas_limit), + ) + assert derived_base_fee == 10 + derived_excess = Cancun.excess_blob_gas_calculator()( + parent_excess_blob_gas=int(env.excess_blob_gas), + parent_blob_gas_used=0, + parent_base_fee_per_gas=int(env.base_fee_per_gas), + ) + assert derived_excess == 0x80000 + + +def test_prepend_class_pre_blob_genesis_compensation() -> None: + """A pre-Cancun invalid singleton compensates the base fee only.""" + test = BlockchainTest( + fork=Shanghai, + pre=Alloc(), + post=Alloc(), + blocks=[INVALID], + genesis_environment=Environment(base_fee_per_gas=HexNumber(1000)), + sync_block=True, + sync_block_salt="test", + ) + env = test.get_genesis_environment() + assert env.base_fee_per_gas is not None + assert env.excess_blob_gas is None + derived_base_fee = Shanghai.base_fee_per_gas_calculator()( + parent_base_fee_per_gas=int(env.base_fee_per_gas), + parent_gas_used=0, + parent_gas_limit=int(env.gas_limit), + ) + assert derived_base_fee == 1000 + + +def make_invalid_state_test(*, sync_block: bool) -> StateTest: + """ + Create a Cancun state test that converts to an invalid singleton + while pinning its block fee values. + """ + return StateTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + tx=Transaction(sender=HexNumber(0), to=HexNumber(0), nonce=0), + env=Environment( + number=1, + timestamp=1_000, + base_fee_per_gas=HexNumber(10), + excess_blob_gas=HexNumber(0xE0000), + ), + block_exception=BlockException.INCORRECT_BLOCK_FORMAT, + sync_block=sync_block, + sync_block_salt="test", + ) + + +def test_invalid_state_test_composition() -> None: + """ + Two empty-block steps from a converted invalid state test's + compensated genesis land on the pinned block environment. + + The state-test conversion winds genesis one step up from the + pinned block environment; the prepend compensation adds the one + further step the prepended sync block consumes. + """ + blockchain_test = make_invalid_state_test( + sync_block=True + ).generate_blockchain_test() + assert blockchain_test.sync_block_policy() is SyncBlockPolicy.PREPEND + env = blockchain_test.get_genesis_environment() + assert env.base_fee_per_gas is not None + assert env.excess_blob_gas is not None + + base_fee = int(env.base_fee_per_gas) + excess = int(env.excess_blob_gas) + base_fee_calculator = Cancun.base_fee_per_gas_calculator() + excess_calculator = Cancun.excess_blob_gas_calculator() + for _ in range(2): + new_excess = excess_calculator( + parent_excess_blob_gas=excess, + parent_blob_gas_used=0, + parent_base_fee_per_gas=base_fee, + ) + base_fee = base_fee_calculator( + parent_base_fee_per_gas=base_fee, + parent_gas_used=0, + parent_gas_limit=int(env.gas_limit), + ) + excess = new_excess + assert base_fee == 10 + assert excess == 0xE0000 + + +def test_state_test_conversion_unchanged_without_sync_block() -> None: + """Without the option the conversion winds exactly one step up.""" + blockchain_test = make_invalid_state_test( + sync_block=False + ).generate_blockchain_test() + env = blockchain_test.get_genesis_environment() + assert env.base_fee_per_gas == 10 * 8 // 7 + assert env.excess_blob_gas == 0xE0000 + CANCUN_TARGET_BLOB_GAS From 9f4207ae24308fb0e8650d8a4146aa544e3b5229 Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 16:25:17 +0200 Subject: [PATCH 04/12] fix(test-fill): reject invalid-header blocks whose modifier changes nothing An `rlp_modifier` on an expected-invalid block is what makes the block invalid when the expected exception is header level, so exception verification is deliberately skipped for it. That leaves a silent hole: when the chain context shifts under a test, the "wrong" value the modifier pins can become the correct one, the modifier no-ops, and the fill emits a valid chain whose fixture still claims invalidity - no fill-side check notices, and consumers disagree with the fixture at run time. The ethrex engagement caught exactly this on the prepend-everywhere design: the extra block's excess blob gas decay turned two `test_invalid_static_excess_blob_gas` fixtures valid while their expectation stayed INVALID, discovered only because two clients answered VALID on a refill and INVALID on the release. Raise a fill error when a block whose expected exceptions are all block level applies an `rlp_modifier` that leaves the header unchanged: the fixture's invalidity expectation no longer tests anything. Blocks expecting a transaction exception are excluded - they are invalid regardless of their header, and the state test conversion routinely pins header fields (e.g. `blob_gas_used`) to values that legitimately match the computed ones, which the full Cancun prepend refill confirmed on 84 such tests. Under the per-class policy the exposure is confined to the prepend class - append-class chains are built byte-for-byte as authored, so no context shifts under them - and the genesis fee compensation keeps the prepend class's pinned values wrong exactly as their authors meant them. The check makes any future recurrence of this class fail loudly at fill time instead of surfacing as a cross-client disagreement. Verified: pytester coverage pins both directions - a no-op modifier on a block-level exception fails the fill naming the block, and a transaction-exception block with a legitimately matching pinned header field still fills; `just static` and `just test-tests` pass. --- .../filler/tests/test_noop_rlp_modifier.py | 160 ++++++++++++++++++ .../src/execution_testing/specs/blockchain.py | 56 +++++- 2 files changed, 215 insertions(+), 1 deletion(-) create mode 100644 packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_noop_rlp_modifier.py diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_noop_rlp_modifier.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_noop_rlp_modifier.py new file mode 100644 index 00000000000..d90d2828e04 --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_noop_rlp_modifier.py @@ -0,0 +1,160 @@ +""" +Test that filling fails loudly when an expected-invalid block's +``rlp_modifier`` does not actually change the header. + +A modifier that pins a header field to the value the header already +holds produces a block that is valid while the fixture claims it is +invalid. This can happen silently when the chain context shifts under +a test, e.g. when a fee progression is moved by the prepended sync +block, so the fill must refuse instead of emitting the fixture. + +The refusal only applies when every expected exception is a block +exception: a block carrying an invalid transaction is invalid +regardless of its header, and the state test conversion routinely +pins header fields to values that legitimately match the computed +ones. +""" + +import textwrap +from typing import Any + +noop_rlp_modifier_test_module = textwrap.dedent( + """\ + from execution_testing import Block + from execution_testing.exceptions.exceptions import BlockException + from execution_testing.specs.blockchain import Header + + + def test_noop_rlp_modifier(blockchain_test, pre) -> None: + blockchain_test( + pre=pre, + post={}, + blocks=[ + Block( + timestamp=12, + rlp_modifier=Header(timestamp=12), + exception=BlockException.INVALID_BLOCK_HASH, + ) + ], + ) + """ +) + + +noop_modifier_tx_exception_test_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction + from execution_testing.exceptions.exceptions import ( + TransactionException, + ) + from execution_testing.specs.blockchain import Header + + + @pytest.mark.exception_test + def test_noop_rlp_modifier_tx_exception(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=21_000, + gas_price=10**9, + sender=pre.fund_eoa(amount=1), + error=TransactionException.INSUFFICIENT_ACCOUNT_FUNDS, + ) + blockchain_test( + pre=pre, + post={}, + blocks=[ + Block( + timestamp=12, + txs=[tx], + rlp_modifier=Header(timestamp=12), + exception=( + TransactionException.INSUFFICIENT_ACCOUNT_FUNDS + ), + ) + ], + ) + """ +) + + +def test_fill_rejects_noop_rlp_modifier_on_invalid_block( + pytester: Any, capsys: Any, pytestconfig: Any +) -> None: + """A no-op modifier on an expected-invalid block must fail the fill.""" + tests_dir = pytester.mkdir("tests") + cancun_tests_dir = tests_dir / "cancun" + cancun_tests_dir.mkdir() + module_dir = cancun_tests_dir / "noop_rlp_modifier_module" + module_dir.mkdir() + test_module = module_dir / "test_noop_rlp_modifier.py" + test_module.write_text(noop_rlp_modifier_test_module) + + pytester.copy_example( + name="src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-fill.ini" + ) + + result = pytester.runpytest_subprocess( + "-c", + "pytest-fill.ini", + "--fork", + "Cancun", + "-m", + "blockchain_test", + "--no-html", + "--output=stdout", + str(test_module.relative_to(pytester.path)), + ) + # Suppress the expected inner pytest failure output from the outer test + capsys.readouterr() + + assert result.ret != 0, "Fill command was expected to fail" + + output = "\n".join(result.outlines + result.errlines) + expected_message = "`rlp_modifier` changed nothing" + assert expected_message in output + + error_line = next( + line for line in output.splitlines() if expected_message in line + ) + # show print but only when -s is passed + if pytestconfig.getoption("capture") == "no": + with capsys.disabled(): + print(error_line) + + +def test_fill_accepts_noop_rlp_modifier_on_tx_exception_block( + pytester: Any, +) -> None: + """ + A no-op modifier on a block whose invalidity comes from a + transaction must fill: the block is invalid regardless of its + header. + """ + tests_dir = pytester.mkdir("tests") + cancun_tests_dir = tests_dir / "cancun" + cancun_tests_dir.mkdir() + module_dir = cancun_tests_dir / "noop_rlp_modifier_tx_module" + module_dir.mkdir() + test_module = module_dir / "test_noop_rlp_modifier_tx_exception.py" + test_module.write_text(noop_modifier_tx_exception_test_module) + + pytester.copy_example( + name="src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-fill.ini" + ) + + result = pytester.runpytest_subprocess( + "-c", + "pytest-fill.ini", + "--fork", + "Cancun", + "-m", + "blockchain_test", + "--no-html", + "--output=stdout", + str(test_module.relative_to(pytester.path)), + ) + outcomes = result.parseoutcomes() + assert outcomes.get("failed", 0) == 0 + assert outcomes.get("passed", 0) > 0 diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index 73f495176e8..4098636c7ca 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -1213,7 +1213,13 @@ def generate_block_data( if block.rlp_modifier is not None: # Modify any parameter specified in the `rlp_modifier` after # transition tool processing. - header = block.rlp_modifier.apply(header) + modified_header = block.rlp_modifier.apply(header) + self._verify_modifier_still_invalidates( + block=block, + header=header, + modified_header=modified_header, + ) + header = modified_header header.fork = fork # Deleted during `apply` because `exclude=True` # Process block access list - apply transformer if present for invalid @@ -1320,6 +1326,54 @@ def generate_block_data( return built_block + def _verify_modifier_still_invalidates( + self, + *, + block: Block, + header: FixtureHeader, + modified_header: FixtureHeader, + ) -> None: + """ + Refuse an expected-invalid block whose ``rlp_modifier`` is a + no-op. + + A block whose expected invalidity is purely header level rests + on the modifier corrupting the header, so exception + verification is skipped for it. A modifier that changes nothing + therefore means the "wrong" value the test pinned has become + the correct one: the block is valid in this chain context while + the fixture still claims it is invalid. This happens when the + chain context shifts under the test, e.g. a fee progression + moved by the prepended sync block. + + Blocks whose exception list names a transaction exception are + excluded: their invalidity comes from a transaction, and the + state test conversion routinely pins header fields to values + that legitimately match the computed ones. + """ + if block.exception is None: + return + expected_exceptions = ( + block.exception + if isinstance(block.exception, list) + else [block.exception] + ) + if not all( + isinstance(exception, BlockException) + for exception in expected_exceptions + ): + return + if modified_header.model_dump() != header.model_dump(): + return + raise ValueError( + f"block {header.number}'s `rlp_modifier` changed " + "nothing: the block expects " + f"`{block.exception}` but its header already holds " + "the pinned values, so the block is valid in this " + "chain context and the fixture's invalidity " + "expectation no longer tests anything" + ) + def verify_post_state( self, t8n: FillerBackend, From 3c722d13dba21985631df497d059799bc0ddc1c6 Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 16:35:12 +0200 Subject: [PATCH 05/12] feat(test-fill): add a marker for tests pinned to absolute block positions A handful of tests assert values derived from absolute block numbers or block hashes (BLOCKHASH lookups, storage keyed by NUMBER). The prepended sync block shifts every block position by one, so no fill transformation can preserve what these tests verify when they are invalid singletons; they are permanently ineligible for the prepend, by construction. They would otherwise surface as fill failures, indistinguishable from bugs in the prepend transformation itself. Register an `absolute_block_position` marker and the mechanism every sync-block eligibility marker shares: the filler collects the markers present on a test node into the spec's `sync_block_ineligibilities`, and the policy resolution vetoes only the placement a marker is about - a marked invalid singleton resolves to no sync block and fills with exactly its own chain (genesis compensation included: the author's pinned fee environment passes through untouched, in both fill phases), while a marked valid chain keeps its appended trailer, because the append shifts nothing and the marker means nothing to it. A marked test is never skipped, so no test leaves the fixture release; the eligibility rule becomes explicit at the test definition, and refill failure lists stay reserved for real regressions. Under the per-class policy this marker's constituency shrinks from every position-pinned test (the prepend-everywhere design) to position-pinned *invalid singletons*; the marker sweep commits re-adjudicate every existing site against that rule. Verified: unit tests pin the veto (marked invalid singleton resolves to none, bare chain, untouched genesis) and its irrelevance to valid chains; a pytester fill of one module asserts all three behaviors side by side - the marked invalid singleton fills bare, the unmarked one gains the prepended block, and the marked valid test keeps its trailer. `just static` and `just test-tests` pass. --- .../pytest_commands/plugins/filler/filler.py | 33 +++- .../filler/tests/test_sync_block_markers.py | 181 ++++++++++++++++++ .../plugins/shared/execute_fill.py | 9 + .../src/execution_testing/specs/base.py | 15 ++ .../src/execution_testing/specs/blockchain.py | 25 +++ .../specs/tests/test_sync_block.py | 60 ++++++ 6 files changed, 322 insertions(+), 1 deletion(-) create mode 100644 packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py index c2d97baf329..321eb099d2d 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py @@ -21,7 +21,17 @@ import warnings from dataclasses import dataclass, field from pathlib import Path -from typing import TYPE_CHECKING, Any, Dict, Generator, List, Self, Set, Type +from typing import ( + TYPE_CHECKING, + Any, + Dict, + FrozenSet, + Generator, + List, + Self, + Set, + Type, +) import pytest import xdist @@ -79,6 +89,9 @@ ) from execution_testing.specs import BaseTest from execution_testing.specs.base import FillResult, OpMode +from execution_testing.specs.blockchain import ( + PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS, +) from execution_testing.test_types import EnvironmentDefaults from execution_testing.test_types.chain_config_types import ( DEFAULT_CHAIN_ID, @@ -107,6 +120,15 @@ if TYPE_CHECKING: from .pre_alloc import Alloc +SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = ( + PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS +) +""" +All sync-block ineligibility markers the filler collects from a test +node. Each vetoes one placement in the spec's policy resolution; the +spec knows which marker belongs to which placement. +""" + # Fixture output dir for keyboard interrupt cleanup (set in pytest_configure). # Used by _merge_on_exit to merge partial JSONL files on Ctrl+C or SIGTERM. _fixture_output_dir: Path | None = None @@ -1676,6 +1698,15 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: and cls.supports_sync_block and op_mode != OpMode.BENCHMARKING ) + # Each ineligibility marker vetoes one sync-block + # placement in the spec's policy resolution; a marked + # test fills without the extra block rather than being + # skipped, so no test ever leaves the fixture release. + kwargs["sync_block_ineligibilities"] = frozenset( + marker + for marker in SYNC_BLOCK_INELIGIBILITY_MARKERS + if request.node.get_closest_marker(marker) is not None + ) # Salt with the test's own id, not with the raw node # id: the fixture format and the xdist group suffix # both ride along in the latter, and every format of diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py new file mode 100644 index 00000000000..3558beb59f8 --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py @@ -0,0 +1,181 @@ +""" +Test the eligibility markers' interaction with the sync block. + +A prepend-class marker names a reason the prepended sync block cannot +preserve what an invalid singleton verifies: the extra block shifts +every position, executes the test's own setup, or cannot reproduce a +pinned fee value. A marked test is not skipped - it fills without the +extra block, so no test ever leaves the fixture release; sync-based +consumers skip its single-block chain at consume time instead. The +markers are irrelevant to valid chains: the appended sync block +changes nothing below itself, so a marked valid test keeps its +trailer. +""" + +import json +import textwrap +from pathlib import Path +from typing import Any, Dict + +import pytest + +marked_test_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction, TransactionException + + + @pytest.mark.{marker} + @pytest.mark.exception_test + def test_marked_invalid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=20_999, + sender=pre.fund_eoa(), + error=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + blockchain_test( + pre=pre, + post={{}}, + blocks=[ + Block( + txs=[tx], + exception=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + ], + ) + + + @pytest.mark.exception_test + def test_unmarked_invalid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=20_999, + sender=pre.fund_eoa(), + error=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + blockchain_test( + pre=pre, + post={{}}, + blocks=[ + Block( + txs=[tx], + exception=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + ], + ) + + + @pytest.mark.{marker} + def test_marked_valid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=21_000, + sender=pre.fund_eoa(), + ) + blockchain_test(pre=pre, post={{}}, blocks=[Block(txs=[tx])]) + """ +) + + +def make_test_module(pytester: Any, marker: str) -> Any: + """Write a test module into a pytester tests tree.""" + tests_dir = pytester.mkdir("tests") + cancun_tests_dir = tests_dir / "cancun" + cancun_tests_dir.mkdir() + module_dir = cancun_tests_dir / "sync_block_markers_module" + module_dir.mkdir() + test_module = module_dir / "test_marked.py" + test_module.write_text(marked_test_module.format(marker=marker)) + pytester.copy_example( + name="src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-fill.ini" + ) + return test_module + + +def fill(pytester: Any, test_module: Any) -> Path: + """Fill the module in both phases and return the output directory.""" + output = pytester.path / "fixtures" + common = ( + "-c", + "pytest-fill.ini", + "--fork", + "Cancun", + "--generate-all-formats", + "--sync-block", + "--skip-index", + "--no-html", + f"--output={output}", + str(test_module.relative_to(pytester.path)), + ) + result = pytester.runpytest_subprocess( + "--generate-pre-alloc-groups", *common + ) + assert result.ret == 0, "fill phase 1 was expected to succeed" + result = pytester.runpytest_subprocess("--use-pre-alloc-groups", *common) + assert result.ret == 0, "fill phase 2 was expected to succeed" + return output + + +def engine_x_fixtures(output: Path) -> Dict[str, Dict[str, Any]]: + """Return the emitted engine_x fixtures keyed by test id.""" + fixtures: Dict[str, Dict[str, Any]] = {} + for path in sorted((output / "blockchain_tests_engine_x").rglob("*.json")): + if "pre_alloc" in path.parts: + continue + fixtures.update(json.loads(path.read_text())) + assert fixtures, "no engine_x fixtures were emitted" + return fixtures + + +@pytest.mark.parametrize( + "marker", + [ + "absolute_block_position", + ], +) +def test_marked_test_fills_without_the_sync_block( + pytester: Any, marker: str +) -> None: + """ + Each marker must veto the prepend for its own test only: the + marked invalid singleton fills with exactly its own single block, + the unmarked invalid singleton in the same module gains the + prepended one, and the marked valid test keeps its appended + trailer - the marker means nothing to a chain the extra block + cannot shift. + """ + test_module = make_test_module(pytester, marker=marker) + output = fill(pytester, test_module) + + fixtures = engine_x_fixtures(output) + by_name = {} + for test_id, fixture in fixtures.items(): + for name in ("marked_invalid", "unmarked_invalid", "marked_valid"): + if f"test_{name}[" in test_id: + by_name[name] = fixture + assert set(by_name) == { + "marked_invalid", + "unmarked_invalid", + "marked_valid", + } + + marked_payloads = by_name["marked_invalid"]["engineNewPayloads"] + assert len(marked_payloads) == 1, ( + "the marked invalid singleton must fill without the sync block" + ) + assert marked_payloads[0].get("phase") is None + assert "syncPayload" not in by_name["marked_invalid"] + + unmarked_payloads = by_name["unmarked_invalid"]["engineNewPayloads"] + assert len(unmarked_payloads) == 2, ( + "the unmarked invalid singleton must still gain the prepended block" + ) + assert unmarked_payloads[0].get("phase") == "sync" + + valid_payloads = by_name["marked_valid"]["engineNewPayloads"] + assert len(valid_payloads) == 1 + assert "syncPayload" in by_name["marked_valid"], ( + "a prepend-class marker must not cost a valid chain its trailer" + ) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py index 4c82a3adbca..a6a46b54e78 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py @@ -240,6 +240,15 @@ def pytest_configure(config: pytest.Config) -> None: "pre_alloc_mutable: Marks a test to allow impossible mutations in the " "pre-state.", ) + config.addinivalue_line( + "markers", + "absolute_block_position: Marks a test whose logic or expectations " + "depend on absolute block numbers or block hashes; an invalid " + "singleton so marked fills without the prepended sync block, because " + "the extra block would shift every block position and silently " + "change what the test verifies. Irrelevant to appended sync blocks, " + "which shift nothing.", + ) config.addinivalue_line( "markers", "fixture_format_id: ID used to describe the fixture format.", diff --git a/packages/testing/src/execution_testing/specs/base.py b/packages/testing/src/execution_testing/specs/base.py index 66a9708ab2f..7876a7434d4 100644 --- a/packages/testing/src/execution_testing/specs/base.py +++ b/packages/testing/src/execution_testing/specs/base.py @@ -10,6 +10,7 @@ Callable, ClassVar, Dict, + FrozenSet, Generator, List, Sequence, @@ -141,6 +142,20 @@ class BaseTest(BaseModel): and by stateful fixtures, whose chains continue a live client's own head instead of a genesis the framework builds. """ + sync_block_ineligibilities: FrozenSet[str] = frozenset() + """ + Names of the sync-block ineligibility markers present on the test. + + Each marker declares the test permanently incompatible with one + sync-block placement, and vetoes only that placement's resolution + (see ``BlockchainTest.sync_block_policy``): a marked test is not + skipped - it fills without the sync block, so no test ever leaves + the fixture release. Its chain is then only syncable if the test's + own blocks make it so; sync-based consumers skip the rest at + consume time, which is honest: a chain that cannot take the extra + block cannot trigger a devp2p sync no matter how it is filled. + Set by the filler from the markers on the pytest node. + """ sync_block_salt: str = "" """ Value mixed into the sync block's ``extra_data`` so its hash is diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index 4098636c7ca..8cf97019834 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -8,6 +8,7 @@ Callable, ClassVar, Dict, + FrozenSet, Generator, List, Sequence, @@ -111,6 +112,20 @@ timestamp of its own (see ``Block.set_environment``). """ +PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = frozenset( + { + "absolute_block_position", + } +) +""" +Markers declaring a test permanently ineligible for the *prepended* +sync block, each naming the reason the transformation cannot preserve +what the test verifies. A marked test is not skipped - it fills +without the extra block (see +``BaseTest.sync_block_ineligibilities``). The markers are irrelevant +to the appended sync block, which changes nothing below itself. +""" + class SyncBlockPolicy(Enum): """ @@ -1418,6 +1433,11 @@ def sync_block_policy(self) -> SyncBlockPolicy: error-code block keeps its own announcement for the same reason the singleton does; an empty chain has nothing to announce. + + A test marked ineligible for its class's placement (see + ``BaseTest.sync_block_ineligibilities``) also resolves to + none: it fills with exactly its own chain instead of being + skipped, so no test ever leaves the fixture release. """ if not self.sync_block or not self.blocks: return SyncBlockPolicy.NONE @@ -1426,6 +1446,11 @@ def sync_block_policy(self) -> SyncBlockPolicy: ) invalid = any(block.exception is not None for block in self.blocks) if len(self.blocks) == 1 and (invalid or engine_refused): + if ( + self.sync_block_ineligibilities + & PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS + ): + return SyncBlockPolicy.NONE return SyncBlockPolicy.PREPEND if not invalid and not engine_refused: return SyncBlockPolicy.APPEND diff --git a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py index fd31e8bf974..116b2f942b7 100644 --- a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py +++ b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py @@ -482,3 +482,63 @@ def test_state_test_conversion_unchanged_without_sync_block() -> None: env = blockchain_test.get_genesis_environment() assert env.base_fee_per_gas == 10 * 8 // 7 assert env.excess_blob_gas == 0xE0000 + CANCUN_TARGET_BLOB_GAS + + +def test_prepend_ineligibility_marker_falls_back_bare() -> None: + """ + A marked invalid singleton fills with exactly its own chain: the + marker vetoes the prepend, and the test is never skipped. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[INVALID], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset({"absolute_block_position"}), + ) + assert test.sync_block_policy() is SyncBlockPolicy.NONE + assert test.blocks_to_build() is test.blocks + + +def test_prepend_ineligibility_marker_is_irrelevant_to_append() -> None: + """ + A prepend-class marker on a valid chain changes nothing: the + appended sync block shifts no position, so the chain keeps its + trailer. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[VALID], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset({"absolute_block_position"}), + ) + assert test.sync_block_policy() is SyncBlockPolicy.APPEND + + +def test_prepend_ineligibility_restores_the_author_genesis() -> None: + """ + The marker's veto must reach the genesis compensation: a marked + invalid singleton's pinned fee environment passes through + untouched, in both fill phases. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[INVALID], + genesis_environment=Environment( + base_fee_per_gas=HexNumber(10), + excess_blob_gas=HexNumber(0x80000), + ), + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset({"absolute_block_position"}), + ) + env = test.get_genesis_environment() + assert env.base_fee_per_gas == 10 + assert env.excess_blob_gas == 0x80000 From 9f42332952e65e3e5956bc624b4665700371971e Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 16:42:10 +0200 Subject: [PATCH 06/12] feat(test-fill): add a marker for pre-state the prepended block executes The Prague sweep of the prepend-everywhere design found a class the position marker does not cover: tests that sabotage a system contract every post-Prague block calls. A prepended sync block runs the same system calls, so on such pre-state it either fails to build outright (the block-invalidating error modes) or consumes the contract's one-shot behavior before the test's own block runs - either way the transformation cannot be neutral. Pre-state is not the only setup that can reach the prepended block: a test pinning a genesis gas limit below a minimal block's own system work makes the block invalid outright, with no compensation available, so the marker's description names the genesis environment alongside pre-state rather than splitting the same decision across two markers. Like its position sibling, `pre_state_affects_empty_block` vetoes only the prepend: a marked invalid singleton fills with exactly its own chain rather than being skipped. Under the per-class policy the class it describes is exactly the sabotage-style *invalid singletons* - the block-invalidating sabotage variants that remain prepend class keep needing it, while sabotage tests whose own block is valid resolve to append, where the extra block sits *above* the test's blocks and meets the same broken contract from the other side; that mirror class gets its own marker in the follow-up commit, and the sweep commits re-adjudicate every existing site. Verified: the pytester marker coverage runs over both prepend-class markers (marked invalid singleton fills bare, unmarked prepends, marked valid keeps its trailer); `just static` and `just test-tests` pass. --- .../plugins/filler/tests/test_sync_block_markers.py | 1 + .../pytest_commands/plugins/shared/execute_fill.py | 11 +++++++++++ .../testing/src/execution_testing/specs/blockchain.py | 1 + 3 files changed, 13 insertions(+) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py index 3558beb59f8..82bef4bbdfb 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py @@ -133,6 +133,7 @@ def engine_x_fixtures(output: Path) -> Dict[str, Dict[str, Any]]: "marker", [ "absolute_block_position", + "pre_state_affects_empty_block", ], ) def test_marked_test_fills_without_the_sync_block( diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py index a6a46b54e78..d4b09e50805 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py @@ -249,6 +249,17 @@ def pytest_configure(config: pytest.Config) -> None: "change what the test verifies. Irrelevant to appended sync blocks, " "which shift nothing.", ) + config.addinivalue_line( + "markers", + "pre_state_affects_empty_block: Marks a test whose pre-state or " + "genesis environment changes what any block executes - e.g. a " + "deliberately broken system contract that every post-Prague block " + "calls, or a genesis gas limit too small for a minimal block's own " + "system work; an invalid singleton so marked fills without the " + "prepended sync block because the extra block would either fail on " + "that setup or consume its one-shot behavior before the test's own " + "block runs.", + ) config.addinivalue_line( "markers", "fixture_format_id: ID used to describe the fixture format.", diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index 8cf97019834..447c2fb6728 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -115,6 +115,7 @@ PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = frozenset( { "absolute_block_position", + "pre_state_affects_empty_block", } ) """ From 76ae92849cf6c8d08f9ce03ed724e927d28c659e Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 16:44:14 +0200 Subject: [PATCH 07/12] feat(test-fill): add a marker for fee values the prepended block cannot reproduce The prepend compensates a test's genesis fee environment so the empty block leaves the test's own block seeing what it pinned. Some pinned values have no preimage at all: under EIP-7918's reserve price the excess-blob-gas decay is `0` below target, identity while the reserve is active, and `x - target` otherwise, so a small nonzero value is unreachable from any parent. The fill already refuses these loudly, which is correct but leaves the "genuinely ineligible" class with no way to record the verdict in-tree. Add `no_empty_block_fee_preimage` alongside the position and pre-state markers so the exclusion is explicit and reviewable rather than a command-line ignore, and name it in the refusal the preimage scan raises, the way the marker registrations name their own remedies. Like its siblings, the marker vetoes only the prepend: a marked invalid singleton fills with its pinned fee environment preserved and stays in the release. Valid chains cannot hit the preimage scan at all under the per-class policy - their genesis is never compensated - so the marker's constituency is exactly the prepend class's fee-pinning tests; the sweep commits re-adjudicate the existing sites. Verified: the pytester marker coverage runs over all three prepend-class markers; unit tests keep pinning the refusal for unreachable values, whose message now names the marker; `just static` and `just test-tests` pass. --- .../plugins/filler/tests/test_sync_block_markers.py | 1 + .../pytest_commands/plugins/shared/execute_fill.py | 9 +++++++++ .../src/execution_testing/specs/blockchain.py | 13 +++++++++---- 3 files changed, 19 insertions(+), 4 deletions(-) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py index 82bef4bbdfb..223586ec8fc 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py @@ -133,6 +133,7 @@ def engine_x_fixtures(output: Path) -> Dict[str, Dict[str, Any]]: "marker", [ "absolute_block_position", + "no_empty_block_fee_preimage", "pre_state_affects_empty_block", ], ) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py index d4b09e50805..d76bd4b72c1 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py @@ -260,6 +260,15 @@ def pytest_configure(config: pytest.Config) -> None: "that setup or consume its one-shot behavior before the test's own " "block runs.", ) + config.addinivalue_line( + "markers", + "no_empty_block_fee_preimage: Marks a test pinning a fee value the " + "prepended sync block cannot reproduce - e.g. under EIP-7918's " + "reserve price, a small nonzero excess blob gas has no parent value " + "that decays to it across an empty block; an invalid singleton so " + "marked fills without the prepended sync block because no genesis " + "compensation exists that preserves the test's fee environment.", + ) config.addinivalue_line( "markers", "fixture_format_id: ID used to describe the fixture format.", diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index 447c2fb6728..3040c990b21 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -115,6 +115,7 @@ PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = frozenset( { "absolute_block_position", + "no_empty_block_fee_preimage", "pre_state_affects_empty_block", } ) @@ -233,7 +234,9 @@ def empty_block_base_fee_preimage( raise ValueError( f"no parent base fee decays to {base_fee_per_gas} on an empty " f"block at fork {fork.name()}; the prepended sync block cannot " - "preserve this test's fee environment" + "preserve this test's fee environment, so the test is " + "ineligible for the prepend and belongs marked " + "`no_empty_block_fee_preimage`" ) @@ -254,8 +257,8 @@ def empty_block_excess_blob_gas_preimage( Some values are unreachable: under a fee floor the decay is skipped while the floor holds, so a small nonzero excess has no parent at - all. Such a test cannot take the prepended sync block and its fill - is refused loudly. + all. Such a test is ineligible for the prepended sync block and is + marked ``no_empty_block_fee_preimage`` rather than filled. """ calculate_excess_blob_gas = fork.excess_blob_gas_calculator() target = fork.target_blobs_per_block() * fork.blob_gas_per_blob() @@ -270,7 +273,9 @@ def empty_block_excess_blob_gas_preimage( raise ValueError( f"no parent excess blob gas decays to {excess_blob_gas} on an " f"empty block at fork {fork.name()}; the prepended sync block " - "cannot preserve this test's fee environment" + "cannot preserve this test's fee environment, so the test is " + "ineligible for the prepend and belongs marked " + "`no_empty_block_fee_preimage`" ) From 3fb36baaa93dde505f8e81a2e21748b73abe61ed Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 16:47:31 +0200 Subject: [PATCH 08/12] feat(test-fill): add a marker for post-state the appended block cannot survive The appended sync block is built by t8n on the test's *post*-state, and post-Prague it executes the same per-block system calls as any real block. A valid test that ends with a sabotaged system contract (the EIP-7002/7251 error-mode families), or one-shot state the extra block would consume, breaks that build: the fill fails loudly inside the trailer construction, indistinguishable from a bug in the append itself. This is the mirror image of `pre_state_affects_empty_block`, on the other side of the chain: the same broken contract stops a prepended block below an invalid singleton and an appended block above a valid chain. Add `post_state_affects_sync_block`, the append class's one eligibility marker: a marked valid chain resolves to no sync block and fills as exactly the author's chain - fallback bare, never skip, so no test leaves the fixture release; sync-based consumers skip what cannot sync at consume time. The marker vetoes only the append; an invalid singleton keeps its prepend regardless, because the prepended block never sees the test's post-state. The trailer build now names the marker when it fails, the way the preimage refusal names its own, so a failing fill points straight at its remedy. Nothing that the fixture records changes for a marked test: under the out-of-chain representation the payload list, head and post state are the author's whether or not a trailer exists, so the marker's only observable effect is the absent `syncPayload`. Verified: unit tests pin the veto (a marked valid chain resolves to none and its chain is untouched; the marker is irrelevant to an invalid singleton's prepend), and the pytester marker coverage asserts the marked valid test emits no `syncPayload` while its unmarked sibling keeps one. The trailer-build failure context is exercised on the real sabotage families by the marker sweep that follows, whose commit records the observed refusals. `just static` and `just test-tests` pass. --- .../pytest_commands/plugins/filler/filler.py | 2 + .../filler/tests/test_sync_block_markers.py | 70 +++++++++++++++++-- .../plugins/shared/execute_fill.py | 9 +++ .../src/execution_testing/specs/blockchain.py | 43 ++++++++++-- .../specs/tests/test_sync_block.py | 40 +++++++++++ 5 files changed, 152 insertions(+), 12 deletions(-) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py index 321eb099d2d..3f11c94c09d 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py @@ -90,6 +90,7 @@ from execution_testing.specs import BaseTest from execution_testing.specs.base import FillResult, OpMode from execution_testing.specs.blockchain import ( + APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS, PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS, ) from execution_testing.test_types import EnvironmentDefaults @@ -122,6 +123,7 @@ SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = ( PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS + | APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS ) """ All sync-block ineligibility markers the filler collects from a test diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py index 223586ec8fc..e814bc56b9d 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py @@ -4,12 +4,14 @@ A prepend-class marker names a reason the prepended sync block cannot preserve what an invalid singleton verifies: the extra block shifts every position, executes the test's own setup, or cannot reproduce a -pinned fee value. A marked test is not skipped - it fills without the -extra block, so no test ever leaves the fixture release; sync-based -consumers skip its single-block chain at consume time instead. The -markers are irrelevant to valid chains: the appended sync block -changes nothing below itself, so a marked valid test keeps its -trailer. +pinned fee value. The append-class marker names the one reason a +valid chain cannot take its trailer: the test's final state breaks +the empty block built on top of it. A marked test is not skipped - +it fills without the extra block, so no test ever leaves the fixture +release; sync-based consumers skip what cannot sync at consume time +instead. Each marker vetoes only its own placement: a prepend-class +marker is irrelevant to a valid chain's trailer, and the +append-class marker is irrelevant to an invalid singleton's prepend. """ import json @@ -181,3 +183,59 @@ def test_marked_test_fills_without_the_sync_block( assert "syncPayload" in by_name["marked_valid"], ( "a prepend-class marker must not cost a valid chain its trailer" ) + + +append_marked_test_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction + + + @pytest.mark.post_state_affects_sync_block + def test_marked_valid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=21_000, + sender=pre.fund_eoa(), + ) + blockchain_test(pre=pre, post={}, blocks=[Block(txs=[tx])]) + + + def test_unmarked_valid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=21_000, + sender=pre.fund_eoa(), + ) + blockchain_test(pre=pre, post={}, blocks=[Block(txs=[tx])]) + """ +) + + +def test_append_marked_test_fills_without_the_trailer(pytester: Any) -> None: + """ + The append-class marker must cost exactly its own test the + trailer: the marked valid test fills as the author's bare chain + while the unmarked one in the same module keeps its + ``syncPayload``. + """ + test_module = make_test_module(pytester, marker="unused") + test_module.write_text(append_marked_test_module) + output = fill(pytester, test_module) + + fixtures = engine_x_fixtures(output) + by_name = {} + for test_id, fixture in fixtures.items(): + for name in ("marked_valid", "unmarked_valid"): + if f"test_{name}[" in test_id: + by_name[name] = fixture + assert set(by_name) == {"marked_valid", "unmarked_valid"} + + assert "syncPayload" not in by_name["marked_valid"], ( + "the marked valid test must fill without the appended sync block" + ) + assert len(by_name["marked_valid"]["engineNewPayloads"]) == 1 + assert "syncPayload" in by_name["unmarked_valid"], ( + "the unmarked valid test must keep its appended sync block" + ) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py index d76bd4b72c1..83bc4b3b8df 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py @@ -269,6 +269,15 @@ def pytest_configure(config: pytest.Config) -> None: "marked fills without the prepended sync block because no genesis " "compensation exists that preserves the test's fee environment.", ) + config.addinivalue_line( + "markers", + "post_state_affects_sync_block: Marks a test whose final state " + "breaks the empty block the filler would append above its chain - " + "e.g. it ends with a sabotaged system contract that every " + "subsequent block calls, or one-shot state the extra block would " + "consume; a valid chain so marked fills without the appended sync " + "block, as exactly the author's chain.", + ) config.addinivalue_line( "markers", "fixture_format_id: ID used to describe the fixture format.", diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index 3040c990b21..b25982f09b7 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -128,6 +128,20 @@ to the appended sync block, which changes nothing below itself. """ +APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = frozenset( + { + "post_state_affects_sync_block", + } +) +""" +Markers declaring a test permanently ineligible for the *appended* +sync block: its final state breaks the extra block t8n would build on +top of it (e.g. a sabotaged system contract every block calls, or +one-shot state the block would consume). A marked test fills as +exactly the author's chain - fallback bare, never skip. Irrelevant to +the prepended sync block, which never sees the test's post-state. +""" + class SyncBlockPolicy(Enum): """ @@ -1459,6 +1473,11 @@ def sync_block_policy(self) -> SyncBlockPolicy: return SyncBlockPolicy.NONE return SyncBlockPolicy.PREPEND if not invalid and not engine_refused: + if ( + self.sync_block_ineligibilities + & APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS + ): + return SyncBlockPolicy.NONE return SyncBlockPolicy.APPEND return SyncBlockPolicy.NONE @@ -1819,12 +1838,24 @@ def make_hive_fixture( # one of them a wire-guaranteed ancestor (same # representation as BlockchainEngineSyncFixture's # sync_payload below). - sync_built_block = self.generate_block_data( - t8n=t8n, - block=Block(extra_data=self.sync_block_extra_data()), - previous_env=env, - previous_alloc=alloc, - ) + try: + sync_built_block = self.generate_block_data( + t8n=t8n, + block=Block(extra_data=self.sync_block_extra_data()), + previous_env=env, + previous_alloc=alloc, + ) + except Exception as e: + raise Exception( + "the appended sync block could not be built on " + "this test's post-state: the state the test " + "ends with breaks the empty block's own " + "execution (e.g. a sabotaged system contract " + "or consumed one-shot state). If that state is " + "the test's purpose, the test is ineligible " + "for the appended sync block and belongs " + "marked `post_state_affects_sync_block`." + ) from e fixture_data["sync_payload"] = ( sync_built_block.get_fixture_engine_new_payload( phase=TestPhase.SYNC diff --git a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py index 116b2f942b7..62a36b318b5 100644 --- a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py +++ b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py @@ -542,3 +542,43 @@ def test_prepend_ineligibility_restores_the_author_genesis() -> None: env = test.get_genesis_environment() assert env.base_fee_per_gas == 10 assert env.excess_blob_gas == 0x80000 + + +def test_append_ineligibility_marker_falls_back_bare() -> None: + """ + A marked valid chain fills as exactly the author's chain: the + marker vetoes the append, and the test is never skipped. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[VALID], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset( + {"post_state_affects_sync_block"} + ), + ) + assert test.sync_block_policy() is SyncBlockPolicy.NONE + assert test.blocks_to_build() is test.blocks + + +def test_append_ineligibility_marker_is_irrelevant_to_prepend() -> None: + """ + The append-class marker on an invalid singleton changes nothing: + the prepended sync block never sees the test's post-state, so the + chain keeps its prepend. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[INVALID], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset( + {"post_state_affects_sync_block"} + ), + ) + assert test.sync_block_policy() is SyncBlockPolicy.PREPEND From f0a836b54f36247edd527cae02f927fd1dfeb7a2 Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 23:36:28 +0200 Subject: [PATCH 09/12] feat(test-fill): add a marker for head timestamps that leave no sync-block room The appended sync block takes its parent's timestamp plus the default increment, and a block timestamp must fit uint64. A valid chain whose head pins a timestamp at or next to `2**64 - 1` - the beacon-root families test exactly this boundary - therefore gets a trailer whose timestamp does not fit, and no client can even parse the fixture's sync payload: geth refuses at the JSON-RPC layer (`hex number > 64 bits`) before any consensus logic runs. The fill itself noticed nothing - Python integers do not overflow, t8n accepts the value, and the consistency check compares payload lists, which the out-of-chain trailer is not part of - so the transformation commit's headroom guard now fails such fills loudly, and this commit gives the verdict its in-tree record. Add `no_sync_block_timestamp_headroom`, the append class's second eligibility marker: a marked valid chain resolves to no sync block and fills as exactly the author's chain - fallback bare, never skip, so no test leaves the fixture release; sync-based consumers skip what cannot sync at consume time. The marker vetoes only the append; an invalid singleton keeps its prepend regardless, because the prepended block sits below the chain, where the uint64 ceiling above the head is not its problem. The headroom guard names the marker in its refusal, the way the preimage refusals name theirs, so a failing fill points straight at its remedy. Clamping the trailer's step instead (`min(head + 12, 2**64 - 1)`) was considered and set aside: it would rescue sync coverage for the near-max case only, at the cost of a special-cased trailer environment for a handful of fixtures and of breaking the "built through the normal block machinery" property every other trailer holds; the max case is unfixable regardless, and timestamps are semantic - never clamped, never shifted - on both sides of the chain. Verified: unit tests pin the veto (a marked maximal-timestamp valid chain resolves to none and its chain is untouched; the marker is irrelevant to an invalid singleton's prepend) and the guard's exact boundary; the append-class pytester marker coverage now runs over both append markers, asserting the marked valid test emits no `syncPayload` while its unmarked sibling keeps one. `just static` and `just test-tests` pass. --- .../filler/tests/test_sync_block_markers.py | 21 ++++++--- .../plugins/shared/execute_fill.py | 9 ++++ .../src/execution_testing/specs/blockchain.py | 14 ++++-- .../specs/tests/test_sync_block.py | 46 +++++++++++++++++++ 4 files changed, 80 insertions(+), 10 deletions(-) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py index e814bc56b9d..c4a45acf573 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py @@ -192,14 +192,14 @@ def test_marked_test_fills_without_the_sync_block( from execution_testing import Block, Transaction - @pytest.mark.post_state_affects_sync_block + @pytest.mark.{marker} def test_marked_valid(blockchain_test, pre) -> None: tx = Transaction( to=0, gas_limit=21_000, sender=pre.fund_eoa(), ) - blockchain_test(pre=pre, post={}, blocks=[Block(txs=[tx])]) + blockchain_test(pre=pre, post={{}}, blocks=[Block(txs=[tx])]) def test_unmarked_valid(blockchain_test, pre) -> None: @@ -208,20 +208,29 @@ def test_unmarked_valid(blockchain_test, pre) -> None: gas_limit=21_000, sender=pre.fund_eoa(), ) - blockchain_test(pre=pre, post={}, blocks=[Block(txs=[tx])]) + blockchain_test(pre=pre, post={{}}, blocks=[Block(txs=[tx])]) """ ) -def test_append_marked_test_fills_without_the_trailer(pytester: Any) -> None: +@pytest.mark.parametrize( + "marker", + [ + "post_state_affects_sync_block", + "no_sync_block_timestamp_headroom", + ], +) +def test_append_marked_test_fills_without_the_trailer( + pytester: Any, marker: str +) -> None: """ - The append-class marker must cost exactly its own test the + Each append-class marker must cost exactly its own test the trailer: the marked valid test fills as the author's bare chain while the unmarked one in the same module keeps its ``syncPayload``. """ test_module = make_test_module(pytester, marker="unused") - test_module.write_text(append_marked_test_module) + test_module.write_text(append_marked_test_module.format(marker=marker)) output = fill(pytester, test_module) fixtures = engine_x_fixtures(output) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py index 83bc4b3b8df..aaa65d6b32b 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py @@ -278,6 +278,15 @@ def pytest_configure(config: pytest.Config) -> None: "consume; a valid chain so marked fills without the appended sync " "block, as exactly the author's chain.", ) + config.addinivalue_line( + "markers", + "no_sync_block_timestamp_headroom: Marks a test whose head pins a " + "timestamp at or next to the uint64 ceiling (2**64 - 1), leaving " + "no room for the sync block the filler would append above its " + "chain - the extra block's timestamp would not fit uint64 and no " + "client could parse it; a valid chain so marked fills without the " + "appended sync block, as exactly the author's chain.", + ) config.addinivalue_line( "markers", "fixture_format_id: ID used to describe the fixture format.", diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index b25982f09b7..b845ff0c7e7 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -130,6 +130,7 @@ APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = frozenset( { + "no_sync_block_timestamp_headroom", "post_state_affects_sync_block", } ) @@ -137,9 +138,11 @@ Markers declaring a test permanently ineligible for the *appended* sync block: its final state breaks the extra block t8n would build on top of it (e.g. a sabotaged system contract every block calls, or -one-shot state the block would consume). A marked test fills as -exactly the author's chain - fallback bare, never skip. Irrelevant to -the prepended sync block, which never sees the test's post-state. +one-shot state the block would consume), or its head pins a timestamp +so close to the uint64 ceiling that no block fits above it. A marked +test fills as exactly the author's chain - fallback bare, never skip. +Irrelevant to the prepended sync block, which sits below everything +the markers describe. """ @@ -1609,7 +1612,10 @@ def _verify_sync_block_timestamp_headroom( f"block: {head_timestamp} + " f"{DEFAULT_TIMESTAMP_INCREMENT} exceeds 2**64 - 1, " "and no client can parse a block whose timestamp " - "does not fit uint64." + "does not fit uint64. If the maximal timestamp is the " + "test's purpose, the test is ineligible for the " + "appended sync block and belongs marked " + "`no_sync_block_timestamp_headroom`." ) def make_fixture( diff --git a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py index 62a36b318b5..b3174c03056 100644 --- a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py +++ b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py @@ -582,3 +582,49 @@ def test_append_ineligibility_marker_is_irrelevant_to_prepend() -> None: ), ) assert test.sync_block_policy() is SyncBlockPolicy.PREPEND + + +def test_timestamp_headroom_marker_falls_back_bare() -> None: + """ + A marked maximal-timestamp valid chain fills as exactly the + author's chain: the marker vetoes the append, and the test is + never skipped. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[Block(timestamp=2**64 - 1)], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset( + {"no_sync_block_timestamp_headroom"} + ), + ) + assert test.sync_block_policy() is SyncBlockPolicy.NONE + assert test.blocks_to_build() is test.blocks + + +def test_timestamp_headroom_marker_is_irrelevant_to_prepend() -> None: + """ + The headroom marker on an invalid singleton changes nothing: the + prepended sync block sits below the chain, where the uint64 + ceiling above the head is not its problem. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[ + Block( + timestamp=2**64 - 1, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + ) + ], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset( + {"no_sync_block_timestamp_headroom"} + ), + ) + assert test.sync_block_policy() is SyncBlockPolicy.PREPEND From 9928657bf13bd64e69aa0e39859c2734a3db00e2 Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 17:15:52 +0200 Subject: [PATCH 10/12] chore(tests): mark system-contract sabotage tests ineligible for the prepended block Re-adjudicate every site the prepend-everywhere design had marked (35 absolute_block_position, 6 pre_state_affects_empty_block, 2 no_empty_block_fee_preimage) against the per-class policy's rules, by filling every affected file with --sync-block and judging each failure. The verdict: three sites need a mark, everything else fills clean unmarked - the append placement dissolves the position, fee and one-shot classes wholesale, because nothing shifts under a valid chain and its genesis is never compensated. The three marks, all `pre_state_affects_empty_block`, all on `test_system_contract_errors` (EIP-7002 withdrawals, EIP-7251 consolidations, EIP-8282 builder requests): their generated error modes (system_contract_reverts / _throws / _out_of_gas) are invalid singletons whose sabotaged pre-state contract is called by every post-Prague (or builder) block, so the prepended sync block itself fails to build - 12 engine_x fill failures (SYSTEM_CONTRACT_CALL_FAILED building block 1, six at Prague, six at Amsterdam), exactly the class the marker names. The mark is whole-test, and costs the generated *passing* params nothing: the veto is class-scoped, so the valid params keep their appended trailers - the over-skip the prepend-everywhere design accepted for this family ("a mark cannot name a parameter combination") does not exist under the hybrid. EIP-6110's sabotage family needs no mark: deposits parse transaction logs, no per-block system call reaches the modified contract, and its 23 invalid-layout singletons prepend cleanly with the genesis compensation. The census of everything else, per sweep fill: - Cancun-era (the 16 files carrying the 19 position marks: stRandom x10, stWalletTest, vmTests block_info, scenarios, blockhash, Shanghai withdrawals, point-evaluation transition): 1115 passed, zero failures; 295 append-class fixtures, 4 prepend-class (test_withdrawals_root x3, test_use_value_in_tx x1, compensated, clean). BLOCKHASH expectations, NUMBER-keyed storage and number-embedding wallet hashes are all untouched by a trailer above the head. - Osaka (the 2 fee-preimage sites): 258 passed; all 86 reserve-price variants append - their genesis is never compensated, so the unreachable-preimage refusal cannot arise and the reference's over-skip of 86 variants for 35 affected drops to zero. - Prague (the 2935 history families, both sabotage families, the extra_* trio): 210 passed, the 6 failures marked here, 3 pre-existing skips; 43 append-class, 23 prepend-class. The history-pinning tests (test_block_hashes_history*, test_invalid_history_contract_calls current/future_block) keep their absolute meaning - "current block" stays current when nothing sits below it. The extra_* trio's record-returning modified contracts are harmless above the chain too: the appended block simply carries the records its system call returns, t8n- verified (16 append-class fixtures). - Amsterdam (EIP-8024/7843/7928/8282 families): 8282's 6 error modes marked as above; 307 append-class, 14 prepend-class, 1 none-class fixture. The five prepend-class consistency-check drifts (shifted slotNumber, and the EIP-2935 system write embedding the parent hash inside blockAccessList) are handled by the transformation commit's position scrub, not by marks. Separately disclosed, not addressed here: 274 Amsterdam append-class fixtures fail the fill-time execution-consistency check on this branch *and byte-identically on the untouched base* (exit 1 at 5b2b22c75f filling test_block_access_lists_eip2935.py alone, default options): their blockAccessList embeds genesis-hash-derived data, which cannot survive pre-alloc grouping. Append-class payload lists are byte-identical to option-off fills, so this is a pre-existing condition surfaced by the sweep, reported upstream rather than worked around. `no_sync_block_timestamp_headroom` has exactly one constituency, found by consume-side validation of the filled corpus rather than by the fill: the two beacon-root families (test_beacon_root_contract_timestamps, test_beacon_root_equal_to_timestamp) parametrize head timestamps at 2**64 - 1 and 2**64 - 2, and the trailers appended above them carried timestamps that do not fit uint64 - 48 such fixtures in the full --until=BPO4 corpus (12 + 4 per fork at Cancun, Prague and Osaka), each refused by geth at the JSON-RPC layer before any consensus logic. Param-level marks on the max and near-max variants of both families fill exactly those bare while every other variant keeps its trailer: the Cancun refill of the two families yields 16 appended + 16 bare, 96 fixtures across all formats, zero failures, zero overflowing trailers. `post_state_affects_sync_block` gained no constituents anywhere: every valid chain in the sweeps built its trailer, including the sabotage families' passing params - the one-shot-consumption worry was an artifact of the prepend running *before* the test; after it, the author's chain is already complete. The marker and its build-failure naming stay as machinery for the class the full corpus may yet contain. Verified: refilling the Prague sabotage files with the marks turns the 6 failures into 24 clean fills (6 error modes bare, 2 passing params trailered, exit 0), and the 8282 refill mirrors it exactly; re-running the consistency check over the full Amsterdam sweep output with the extended position scrub leaves drift on exactly the 274 append-class fixtures of the disclosed base condition and none of the prepend class; `just static` and `just test-tests` pass. --- .../test_modified_builder_contract.py | 6 ++++ .../test_beacon_root_contract.py | 28 ++++++++++++++++--- .../test_modified_withdrawal_contract.py | 5 ++++ .../test_modified_consolidation_contract.py | 5 ++++ 4 files changed, 40 insertions(+), 4 deletions(-) diff --git a/tests/amsterdam/eip8282_builder_execution_requests/test_modified_builder_contract.py b/tests/amsterdam/eip8282_builder_execution_requests/test_modified_builder_contract.py index ead0b94e08b..22da4780023 100644 --- a/tests/amsterdam/eip8282_builder_execution_requests/test_modified_builder_contract.py +++ b/tests/amsterdam/eip8282_builder_execution_requests/test_modified_builder_contract.py @@ -198,6 +198,12 @@ def test_extra_builder_exits( ) +# Every post-Amsterdam block calls the (deliberately broken or +# gas-hungry one-shot) builder contracts, so the generated error modes +# are invalid singletons on whose pre-state the prepended sync block +# cannot build; the passing params are valid chains and keep their +# appended trailer. +@pytest.mark.pre_state_affects_empty_block @pytest.mark.parametrize( "system_contract", [ diff --git a/tests/cancun/eip4788_beacon_root/test_beacon_root_contract.py b/tests/cancun/eip4788_beacon_root/test_beacon_root_contract.py index 3150ca50ee7..c799fd22387 100644 --- a/tests/cancun/eip4788_beacon_root/test_beacon_root_contract.py +++ b/tests/cancun/eip4788_beacon_root/test_beacon_root_contract.py @@ -111,8 +111,18 @@ def test_beacon_root_contract_calls( [ (0x0C, True), # twelve (2**32, True), # arbitrary - (2**64 - 2, True), # near-max - (2**64 - 1, True), # max + # The (near-)max heads leave no uint64 room for a block above + # them, so no sync block can be appended to their chains. + pytest.param( + 2**64 - 2, # near-max + True, + marks=pytest.mark.no_sync_block_timestamp_headroom, + ), + pytest.param( + 2**64 - 1, # max + True, + marks=pytest.mark.no_sync_block_timestamp_headroom, + ), # TODO: Update t8n to un marshal > 64-bit int # Exception: failed to evaluate: ERROR(10): failed un marshaling stdin # (2**64, False), # overflow @@ -209,8 +219,18 @@ def test_calldata_lengths( [ (12, 12), # twelve (2**32, 2**32), # arbitrary - (2**64 - 2, 2**64 - 2), # near-max - (2**64 - 1, 2**64 - 1), # max + # The (near-)max heads leave no uint64 room for a block above + # them, so no sync block can be appended to their chains. + pytest.param( + 2**64 - 2, # near-max + 2**64 - 2, + marks=pytest.mark.no_sync_block_timestamp_headroom, + ), + pytest.param( + 2**64 - 1, # max + 2**64 - 1, + marks=pytest.mark.no_sync_block_timestamp_headroom, + ), ], indirect=["beacon_root"], ) diff --git a/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py b/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py index f83b13d5082..5c4ef096336 100644 --- a/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py +++ b/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py @@ -137,6 +137,11 @@ def test_extra_withdrawals( ) +# Every block calls the (deliberately broken or gas-hungry one-shot) +# withdrawal contract, so the generated error modes are invalid +# singletons on whose pre-state the prepended sync block cannot build; +# the passing params are valid chains and keep their appended trailer. +@pytest.mark.pre_state_affects_empty_block @pytest.mark.parametrize( "system_contract", [Address(Spec_EIP7002.WITHDRAWAL_REQUEST_PREDEPLOY_ADDRESS)], diff --git a/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py b/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py index ef63436c21c..ca2ec30bae4 100644 --- a/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py +++ b/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py @@ -136,6 +136,11 @@ def test_extra_consolidations( ) +# Every block calls the (deliberately broken or gas-hungry one-shot) +# consolidation contract, so the generated error modes are invalid +# singletons on whose pre-state the prepended sync block cannot build; +# the passing params are valid chains and keep their appended trailer. +@pytest.mark.pre_state_affects_empty_block @pytest.mark.parametrize( "system_contract", [Address(Spec_EIP7251.CONSOLIDATION_REQUEST_PREDEPLOY_ADDRESS)], From fad1a49797b2b6cecb912a9fafa5bd1826cef13b Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 17:19:08 +0200 Subject: [PATCH 11/12] feat(test-fill): enable sync blocks by default for engine_x fixtures The sync block exists so that sync-based consumers can trigger a devp2p sync on every engine_x chain, and the per-class scoping work confined it to exactly that format and to the placement each chain class can take: a valid chain keeps its payload list, head and post state byte-identical to the author's and gains only the out-of-chain `syncPayload` trailer; a single invalid block gains the prepended in-chain block with its compensated genesis; every other chain, every marked test, every other fixture format, benchmark specs and measuring sessions are untouched. There is therefore no reason left to fill engine_x fixtures without it: a corpus without sync blocks is strictly less useful - 97% of its chains are single blocks that can never trigger a sync - while consumers that replay payloads through the Engine API read the valid-chain majority identically either way. Flip the option's default to on and keep --no-sync-block as the opt-out, e.g. for comparing against corpora filled before the option existed. The pytester coverage now exercises the default path and the opt-out. Verified: the pytester fill suite asserts the default path emits the per-class shapes without the flag (valid chains carry `syncPayload`, an invalid singleton carries the tagged in-chain prepend) and that --no-sync-block restores chains byte-for-byte as the tests define them in every format; `just static` and `just test-tests` pass. --- .../pytest_commands/plugins/filler/filler.py | 11 ++++--- .../filler/tests/test_sync_block_fill.py | 31 ++++++++++--------- .../filler/tests/test_sync_block_markers.py | 1 - 3 files changed, 22 insertions(+), 21 deletions(-) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py index 3f11c94c09d..40611ae4056 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py @@ -651,7 +651,7 @@ def pytest_addoption(parser: pytest.Parser) -> None: "--sync-block", action="store_true", dest="sync_block", - default=False, + default=True, help=( "Add one framework-built empty block to every blockchain " "test's chain for fixture formats that opt in " @@ -664,10 +664,11 @@ def pytest_addoption(parser: pytest.Parser) -> None: "single expected-invalid or Engine API-refused block " "(stored in-chain as the first payload, tagged with the " "`sync` phase), and omitted otherwise. Spec types that " - "opt out (benchmark tests) are filled without it. " - "Prepending shifts the invalid singleton's number and " - "hash, so those fixtures are not comparable with fixtures " - "filled without the option." + "opt out (benchmark tests) are filled without it. On by " + "default; --no-sync-block disables it. Prepending shifts " + "the invalid singleton's number and hash, so those " + "fixtures are not comparable with fixtures filled " + "without the option." ), ) test_group.addoption( diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py index a3445e6f05c..95c61352d09 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py @@ -3,15 +3,16 @@ in, in the placement each chain class calls for. The sync block is scoped per fixture format: only -``blockchain_test_engine_x`` declares ``sync_block`` true, so a fill -with ``--sync-block`` must emit engine_x fixtures carrying the extra -block - appended out-of-chain (the ``syncPayload`` field) for a valid -chain, prepended in-chain (a leading payload tagged with the ``sync`` -phase) for an invalid singleton - while every other format's chains -are byte-for-byte what the test defines. These tests fill single-block -tests and read the fixtures back, so a sync block that leaks into the -wrong format or placement, loses its phase tag, or fails to salt per -test fails here rather than in a consumer. +``blockchain_test_engine_x`` declares ``sync_block`` true, so a +default fill must emit engine_x fixtures carrying the extra block - +appended out-of-chain (the ``syncPayload`` field) for a valid chain, +prepended in-chain (a leading payload tagged with the ``sync`` phase) +for an invalid singleton - while every other format's chains are +byte-for-byte what the test defines, and ``--no-sync-block`` must +restore every chain as written. These tests fill single-block tests +and read the fixtures back, so a sync block that leaks into the wrong +format or placement, loses its phase tag, or fails to salt per test +fails here rather than in a consumer. """ import json @@ -151,7 +152,7 @@ def test_valid_chain_appends_out_of_chain(pytester: Any) -> None: test_module = make_test_module( pytester, valid_test_module, "test_single_block.py" ) - output = fill(pytester, test_module, "--sync-block") + output = fill(pytester, test_module) for fixture in fixtures_of_format(output, "blockchain_tests").values(): assert len(fixture["blocks"]) == 1 @@ -203,7 +204,7 @@ def test_appended_sync_block_is_salted_per_test(pytester: Any) -> None: test_module = make_test_module( pytester, valid_test_module, "test_single_block.py" ) - output = fill(pytester, test_module, "--sync-block") + output = fill(pytester, test_module) salted = { fixture["syncPayload"]["params"][0]["extraData"] @@ -224,7 +225,7 @@ def test_invalid_singleton_prepends_in_chain(pytester: Any) -> None: test_module = make_test_module( pytester, invalid_singleton_module, "test_invalid_singleton.py" ) - output = fill(pytester, test_module, "--sync-block") + output = fill(pytester, test_module) for fixture in fixtures_of_format(output, "blockchain_tests").values(): assert len(fixture["blocks"]) == 1 @@ -256,12 +257,12 @@ def test_invalid_singleton_prepends_in_chain(pytester: Any) -> None: ) -def test_sync_block_is_off_by_default(pytester: Any) -> None: - """Without the option no format gains a sync block.""" +def test_no_sync_block_disables_the_sync_block(pytester: Any) -> None: + """With the opt-out no format gains a sync block.""" test_module = make_test_module( pytester, valid_test_module, "test_single_block.py" ) - output = fill(pytester, test_module) + output = fill(pytester, test_module, "--no-sync-block") for fixture in fixtures_of_format(output, "blockchain_tests").values(): assert len(fixture["blocks"]) == 1 diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py index c4a45acf573..c8e0aec5dfa 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py @@ -105,7 +105,6 @@ def fill(pytester: Any, test_module: Any) -> Path: "--fork", "Cancun", "--generate-all-formats", - "--sync-block", "--skip-index", "--no-html", f"--output={output}", From 06471b1f05c57d2589da29e786d16da18d92c6e0 Mon Sep 17 00:00:00 2001 From: danceratopz Date: Tue, 11 Aug 2026 17:32:31 +0200 Subject: [PATCH 12/12] docs(test-fill): document the per-class sync block and its markers The sync block changes what filled engine_x chains look like and the five eligibility markers are decisions test authors have to make, so both belong in the user-facing documentation rather than only in `--help` and the marker registrations. Add a section to the fill command-line page - the two protocol facts the placement follows from, the chain-class table (append out-of-chain in `syncPayload` for valid chains, prepend in-chain for invalid singletons, nothing otherwise), why the appended block makes the test's own blocks wire-guaranteed, the per-test salt, the prepend-only genesis compensation, that it applies to engine_x fixtures only and is on by default - and document `absolute_block_position`, `pre_state_affects_empty_block`, `no_empty_block_fee_preimage`, `post_state_affects_sync_block` and `no_sync_block_timestamp_headroom` alongside the other test markers, including which placement each vetoes and that marked tests fill without the extra block instead of being skipped. --- .../filling_tests_command_line.md | 27 +++++++++++++++++ docs/writing_tests/test_markers.md | 30 +++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/docs/filling_tests/filling_tests_command_line.md b/docs/filling_tests/filling_tests_command_line.md index 599c93dc1d0..061f4bc4198 100644 --- a/docs/filling_tests/filling_tests_command_line.md +++ b/docs/filling_tests/filling_tests_command_line.md @@ -107,6 +107,33 @@ This flag automatically performs a two-phase execution: uv run fill --generate-all-formats --output=fixtures.tar.gz tests/shanghai/ ``` +## The Sync Block + +By default the filler adds one framework-built empty block to every blockchain test's chain **in `blockchain_test_engine_x` fixtures only**; every other fixture format's chains are exactly what the test defines. `--no-sync-block` disables it: + +```console +uv run fill --no-sync-block --generate-all-formats tests/cancun/ +``` + +A consumer that makes the client download and execute a test's own blocks over devp2p needs the client to actually sync, and two facts about the protocol decide what such a consumer can guarantee: a client only starts a sync when the announced head's parent is unknown to it, and only blocks *below* the head must travel devp2p - the head's payload is always delivered through `engine_newPayload`, and whether a client also re-fetches it from a peer is an implementation choice. The sync block's placement is therefore resolved per test from the chain's own structure: + +| Chain class | Sequence | Placement | Fixture representation | +| ----------- | -------- | --------- | ---------------------- | +| Fully valid (single or multi-block) | `G → T₁…Tₙ → S*` | **appended** above the head | out-of-chain, the fixture's `syncPayload` field | +| Single expected-invalid or Engine API-refused block | `G → S → T₁*` | **prepended** below it | in-chain, `engineNewPayloads[0]` tagged `"phase": "sync"` | +| Invalid multi-block | `G → T₁…Tₙᵢ*` | none | - | + +(`*` marks the block a sync-based consumer announces; `ᵢ` the intentionally invalid block.) + +For a valid chain the appended block makes every one of the test's own blocks an ancestor of the announced head, which a syncing client must fetch and execute through its sync pipeline - the test's content is wire-guaranteed by chain structure, on any client. The fixture's payload list, `lastblockhash` and post state keep describing exactly the chain the test author wrote; the trailer rides out-of-chain in `syncPayload`, the same representation [`consume sync`](../running_tests/running.md#sync)'s fixture format has always used, and consumers that replay payloads through the Engine API ignore it. For a single invalid block nothing can be built on top, so the extra block lands below, giving the sync a reason to start before the client judges the announced head; there it is load-bearing ancestry, so it lives in-chain and every consumer replays it. An invalid multi-block chain needs no help: its valid ancestors already travel the wire. + +The sync block is a real block, built through the same machinery as every other block, and carries a per-test digest in its `extra_data` so every announced head is a block the client has never seen, even across tests sharing a pre-allocation group. Prepend-class chains get their genesis fee fields wound one progression step up to cancel the step the extra block introduces, so the test's own block executes in the fee environment its author specified; append-class chains need no compensation at all. Timestamps are never shifted: a prepend-class test pinning a timestamp the extra block cannot clear fails the fill instead of producing a non-monotonic chain. + +!!! note "Only engine_x fixtures are affected" + Prepending shifts the invalid singleton's block number and hash, so engine_x fixtures filled with and without the option are not comparable. The other blockchain formats never carry the extra block: they share a positional `t8n` output cache and must build byte-identical chains, while engine_x fixtures opt out of that cache and pay no extra `t8n` work for the divergence. + +Spec types the extra block would distort opt out and are filled without it (benchmark tests), so a combined fill needs no extra options. Tests that cannot take their class's placement are marked in the tree and fill without the extra block instead of being skipped - no test leaves the fixture release; sync-based consumers skip chains that cannot sync at consume time. See [`absolute_block_position`](../writing_tests/test_markers.md#pytestmarkabsolute_block_position) and its sibling markers. + ## Debugging the `t8n` Command The `--evm-dump-dir` flag can be used to dump the inputs and outputs of every call made to the `t8n` command for debugging purposes, see [Debugging Transition Tools](./debugging_t8n_tools.md). diff --git a/docs/writing_tests/test_markers.md b/docs/writing_tests/test_markers.md index 22c613ab498..def670f5b48 100644 --- a/docs/writing_tests/test_markers.md +++ b/docs/writing_tests/test_markers.md @@ -371,6 +371,36 @@ Examples of this include: - Contracts having zero-nonce - Deploying a contract to a hard-coded address +### `@pytest.mark.absolute_block_position` + +This marker is used to mark tests whose logic or expectations depend on absolute block numbers or block hashes, e.g. `BLOCKHASH` lookups with static expectations, storage keyed by `NUMBER`, or EIP-2935 history contents. + +An invalid singleton so marked fills without the [prepended sync block](../filling_tests/filling_tests_command_line.md#the-sync-block): the prepended block shifts every block position, and the expectations are derived from the position rather than merely equal to it, so no fill transformation can preserve what the test verifies. The marker is irrelevant to valid chains - their sync block is appended above the head and shifts nothing - and only affects `blockchain_test_engine_x` fixtures (the only format that carries the extra block); the test itself always fills. + +### `@pytest.mark.pre_state_affects_empty_block` + +This marker is used to mark tests whose pre-state or genesis environment changes what *any* block executes, for example a deliberately broken system contract that every post-Prague block calls, one-shot pre-state that an extra block would consume, or a genesis gas limit too small for a minimal block's own system work. + +An invalid singleton so marked fills without the prepended sync block: the extra block would either fail on that setup or consume its one-shot behavior before the test's own block runs. + +### `@pytest.mark.no_empty_block_fee_preimage` + +This marker is used to mark tests pinning a fee value that no parent value decays to across an empty block, e.g. a small nonzero excess blob gas while EIP-7918's reserve price is active. + +An invalid singleton so marked fills without the prepended sync block: no genesis compensation preserves its fee environment, so the fill would otherwise refuse it loudly. Valid chains never need this marker - their genesis is never compensated. + +### `@pytest.mark.post_state_affects_sync_block` + +This marker is used to mark tests whose *final* state breaks the empty block the filler would append above their chain, for example a test that ends with a sabotaged system contract that every subsequent block calls, or with one-shot state the extra block would consume. + +A valid chain so marked fills as exactly the author's chain, without the appended sync block (`syncPayload` is absent from its fixture). This is the mirror image of `pre_state_affects_empty_block` on the other side of the chain; it is irrelevant to invalid singletons, whose prepended block never sees the test's post-state. + +### `@pytest.mark.no_sync_block_timestamp_headroom` + +This marker is used to mark tests whose head pins a timestamp at or next to the uint64 ceiling (`2**64 - 1`), for example the EIP-4788 beacon-root tests that exercise exactly that boundary. The appended sync block takes its parent's timestamp plus the default increment, so above such a head its timestamp would not fit uint64 and no client could parse the fixture's `syncPayload`. + +A valid chain so marked fills as exactly the author's chain, without the appended sync block; the fill refuses loudly (naming this marker) when an unmarked test hits the boundary. Timestamps are semantic and are never clamped or shifted, on either side of the chain. The marker is irrelevant to invalid singletons, whose prepended block sits below the chain. + ### `@pytest.mark.skip()` This marker can be used to skip a test.