diff --git a/docs/filling_tests/filling_tests_command_line.md b/docs/filling_tests/filling_tests_command_line.md index 599c93dc1d0..061f4bc4198 100644 --- a/docs/filling_tests/filling_tests_command_line.md +++ b/docs/filling_tests/filling_tests_command_line.md @@ -107,6 +107,33 @@ This flag automatically performs a two-phase execution: uv run fill --generate-all-formats --output=fixtures.tar.gz tests/shanghai/ ``` +## The Sync Block + +By default the filler adds one framework-built empty block to every blockchain test's chain **in `blockchain_test_engine_x` fixtures only**; every other fixture format's chains are exactly what the test defines. `--no-sync-block` disables it: + +```console +uv run fill --no-sync-block --generate-all-formats tests/cancun/ +``` + +A consumer that makes the client download and execute a test's own blocks over devp2p needs the client to actually sync, and two facts about the protocol decide what such a consumer can guarantee: a client only starts a sync when the announced head's parent is unknown to it, and only blocks *below* the head must travel devp2p - the head's payload is always delivered through `engine_newPayload`, and whether a client also re-fetches it from a peer is an implementation choice. The sync block's placement is therefore resolved per test from the chain's own structure: + +| Chain class | Sequence | Placement | Fixture representation | +| ----------- | -------- | --------- | ---------------------- | +| Fully valid (single or multi-block) | `G → T₁…Tₙ → S*` | **appended** above the head | out-of-chain, the fixture's `syncPayload` field | +| Single expected-invalid or Engine API-refused block | `G → S → T₁*` | **prepended** below it | in-chain, `engineNewPayloads[0]` tagged `"phase": "sync"` | +| Invalid multi-block | `G → T₁…Tₙᵢ*` | none | - | + +(`*` marks the block a sync-based consumer announces; `ᵢ` the intentionally invalid block.) + +For a valid chain the appended block makes every one of the test's own blocks an ancestor of the announced head, which a syncing client must fetch and execute through its sync pipeline - the test's content is wire-guaranteed by chain structure, on any client. The fixture's payload list, `lastblockhash` and post state keep describing exactly the chain the test author wrote; the trailer rides out-of-chain in `syncPayload`, the same representation [`consume sync`](../running_tests/running.md#sync)'s fixture format has always used, and consumers that replay payloads through the Engine API ignore it. For a single invalid block nothing can be built on top, so the extra block lands below, giving the sync a reason to start before the client judges the announced head; there it is load-bearing ancestry, so it lives in-chain and every consumer replays it. An invalid multi-block chain needs no help: its valid ancestors already travel the wire. + +The sync block is a real block, built through the same machinery as every other block, and carries a per-test digest in its `extra_data` so every announced head is a block the client has never seen, even across tests sharing a pre-allocation group. Prepend-class chains get their genesis fee fields wound one progression step up to cancel the step the extra block introduces, so the test's own block executes in the fee environment its author specified; append-class chains need no compensation at all. Timestamps are never shifted: a prepend-class test pinning a timestamp the extra block cannot clear fails the fill instead of producing a non-monotonic chain. + +!!! note "Only engine_x fixtures are affected" + Prepending shifts the invalid singleton's block number and hash, so engine_x fixtures filled with and without the option are not comparable. The other blockchain formats never carry the extra block: they share a positional `t8n` output cache and must build byte-identical chains, while engine_x fixtures opt out of that cache and pay no extra `t8n` work for the divergence. + +Spec types the extra block would distort opt out and are filled without it (benchmark tests), so a combined fill needs no extra options. Tests that cannot take their class's placement are marked in the tree and fill without the extra block instead of being skipped - no test leaves the fixture release; sync-based consumers skip chains that cannot sync at consume time. See [`absolute_block_position`](../writing_tests/test_markers.md#pytestmarkabsolute_block_position) and its sibling markers. + ## Debugging the `t8n` Command The `--evm-dump-dir` flag can be used to dump the inputs and outputs of every call made to the `t8n` command for debugging purposes, see [Debugging Transition Tools](./debugging_t8n_tools.md). diff --git a/docs/writing_tests/test_markers.md b/docs/writing_tests/test_markers.md index 22c613ab498..def670f5b48 100644 --- a/docs/writing_tests/test_markers.md +++ b/docs/writing_tests/test_markers.md @@ -371,6 +371,36 @@ Examples of this include: - Contracts having zero-nonce - Deploying a contract to a hard-coded address +### `@pytest.mark.absolute_block_position` + +This marker is used to mark tests whose logic or expectations depend on absolute block numbers or block hashes, e.g. `BLOCKHASH` lookups with static expectations, storage keyed by `NUMBER`, or EIP-2935 history contents. + +An invalid singleton so marked fills without the [prepended sync block](../filling_tests/filling_tests_command_line.md#the-sync-block): the prepended block shifts every block position, and the expectations are derived from the position rather than merely equal to it, so no fill transformation can preserve what the test verifies. The marker is irrelevant to valid chains - their sync block is appended above the head and shifts nothing - and only affects `blockchain_test_engine_x` fixtures (the only format that carries the extra block); the test itself always fills. + +### `@pytest.mark.pre_state_affects_empty_block` + +This marker is used to mark tests whose pre-state or genesis environment changes what *any* block executes, for example a deliberately broken system contract that every post-Prague block calls, one-shot pre-state that an extra block would consume, or a genesis gas limit too small for a minimal block's own system work. + +An invalid singleton so marked fills without the prepended sync block: the extra block would either fail on that setup or consume its one-shot behavior before the test's own block runs. + +### `@pytest.mark.no_empty_block_fee_preimage` + +This marker is used to mark tests pinning a fee value that no parent value decays to across an empty block, e.g. a small nonzero excess blob gas while EIP-7918's reserve price is active. + +An invalid singleton so marked fills without the prepended sync block: no genesis compensation preserves its fee environment, so the fill would otherwise refuse it loudly. Valid chains never need this marker - their genesis is never compensated. + +### `@pytest.mark.post_state_affects_sync_block` + +This marker is used to mark tests whose *final* state breaks the empty block the filler would append above their chain, for example a test that ends with a sabotaged system contract that every subsequent block calls, or with one-shot state the extra block would consume. + +A valid chain so marked fills as exactly the author's chain, without the appended sync block (`syncPayload` is absent from its fixture). This is the mirror image of `pre_state_affects_empty_block` on the other side of the chain; it is irrelevant to invalid singletons, whose prepended block never sees the test's post-state. + +### `@pytest.mark.no_sync_block_timestamp_headroom` + +This marker is used to mark tests whose head pins a timestamp at or next to the uint64 ceiling (`2**64 - 1`), for example the EIP-4788 beacon-root tests that exercise exactly that boundary. The appended sync block takes its parent's timestamp plus the default increment, so above such a head its timestamp would not fit uint64 and no client could parse the fixture's `syncPayload`. + +A valid chain so marked fills as exactly the author's chain, without the appended sync block; the fill refuses loudly (naming this marker) when an unmarked test hits the boundary. Timestamps are semantic and are never clamped or shifted, on either side of the chain. The marker is irrelevant to invalid singletons, whose prepended block sits below the chain. + ### `@pytest.mark.skip()` This marker can be used to skip a test. diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py index 7db3702718c..40611ae4056 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py @@ -21,7 +21,17 @@ import warnings from dataclasses import dataclass, field from pathlib import Path -from typing import TYPE_CHECKING, Any, Dict, Generator, List, Self, Set, Type +from typing import ( + TYPE_CHECKING, + Any, + Dict, + FrozenSet, + Generator, + List, + Self, + Set, + Type, +) import pytest import xdist @@ -79,6 +89,10 @@ ) from execution_testing.specs import BaseTest from execution_testing.specs.base import FillResult, OpMode +from execution_testing.specs.blockchain import ( + APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS, + PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS, +) from execution_testing.test_types import EnvironmentDefaults from execution_testing.test_types.chain_config_types import ( DEFAULT_CHAIN_ID, @@ -107,6 +121,16 @@ if TYPE_CHECKING: from .pre_alloc import Alloc +SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = ( + PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS + | APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS +) +""" +All sync-block ineligibility markers the filler collects from a test +node. Each vetoes one placement in the spec's policy resolution; the +spec knows which marker belongs to which placement. +""" + # Fixture output dir for keyboard interrupt cleanup (set in pytest_configure). # Used by _merge_on_exit to merge partial JSONL files on Ctrl+C or SIGTERM. _fixture_output_dir: Path | None = None @@ -623,6 +647,39 @@ def pytest_addoption(parser: pytest.Parser) -> None: default=False, help="Fill tests using existing pre-allocation groups (phase 2 only).", ) + test_group.addoption( + "--sync-block", + action="store_true", + dest="sync_block", + default=True, + help=( + "Add one framework-built empty block to every blockchain " + "test's chain for fixture formats that opt in " + "(blockchain_test_engine_x), so that sync-based consumers " + "can trigger a devp2p sync. The block's placement is " + "resolved per test from the chain's structure: appended " + "above a fully valid chain's head (stored out-of-chain in " + "the fixture's `syncPayload` field, making every test " + "block a wire-guaranteed ancestor), prepended below a " + "single expected-invalid or Engine API-refused block " + "(stored in-chain as the first payload, tagged with the " + "`sync` phase), and omitted otherwise. Spec types that " + "opt out (benchmark tests) are filled without it. On by " + "default; --no-sync-block disables it. Prepending shifts " + "the invalid singleton's number and hash, so those " + "fixtures are not comparable with fixtures filled " + "without the option." + ), + ) + test_group.addoption( + "--no-sync-block", + action="store_false", + dest="sync_block", + help=( + "Do not add sync blocks to any fixture format; every " + "chain is built exactly as the test defines it." + ), + ) test_group.addoption( "--generate-all-formats", action="store_true", @@ -1504,6 +1561,23 @@ def _strip_xdist_group_suffix(s: str) -> str: return s +def _node_id_without_xdist_group(nodeid: str) -> str: + """ + Return the node id without any xdist group suffix. + + Under ``--dist=loadgroup`` the xdist worker appends ``@`` to + every grouped item's node id, so anything derived from the raw id + depends on whether the fill ran in parallel. Every group name the + fill sets is a bare word, while a parametrized node id always ends + in ``]``, so a trailing ``@`` segment without one is a group name + and never part of the test's own id. + """ + base, separator, suffix = nodeid.rpartition("@") + if separator and base and "]" not in suffix: + return base + return nodeid + + def node_to_test_info(node: pytest.Item) -> TestInfo: """Return test info of the current node item.""" # Strip xdist group suffix (@groupname) that may be added during execution. @@ -1614,6 +1688,37 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: kwargs["fork"] = fork op_mode: OpMode = request.config.op_mode # type: ignore kwargs["operation_mode"] = op_mode + # The sync block only applies to fixture formats that + # opt in (sync-based consumers need it); it is further + # withheld from spec types that measure per-block and + # from any session that is measuring. Where the block + # lands in an eligible test's chain - appended, + # prepended, or not at all - is resolved by the spec + # itself from the chain's structure. + kwargs["sync_block"] = ( + request.config.getoption("sync_block", False) + and fixture_format.sync_block + and cls.supports_sync_block + and op_mode != OpMode.BENCHMARKING + ) + # Each ineligibility marker vetoes one sync-block + # placement in the spec's policy resolution; a marked + # test fills without the extra block rather than being + # skipped, so no test ever leaves the fixture release. + kwargs["sync_block_ineligibilities"] = frozenset( + marker + for marker in SYNC_BLOCK_INELIGIBILITY_MARKERS + if request.node.get_closest_marker(marker) is not None + ) + # Salt with the test's own id, not with the raw node + # id: the fixture format and the xdist group suffix + # both ride along in the latter, and every format of + # one test must build the same chain (they share a + # t8n output cache) whether or not the fill ran in + # parallel. + kwargs["sync_block_salt"] = _node_id_without_xdist_group( + strip_fixture_format_from_node(request.node) + ) kwargs["is_tx_gas_heavy_test"] = is_tx_gas_heavy_test kwargs["is_exception_test"] = is_exception_test if ( diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_noop_rlp_modifier.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_noop_rlp_modifier.py new file mode 100644 index 00000000000..d90d2828e04 --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_noop_rlp_modifier.py @@ -0,0 +1,160 @@ +""" +Test that filling fails loudly when an expected-invalid block's +``rlp_modifier`` does not actually change the header. + +A modifier that pins a header field to the value the header already +holds produces a block that is valid while the fixture claims it is +invalid. This can happen silently when the chain context shifts under +a test, e.g. when a fee progression is moved by the prepended sync +block, so the fill must refuse instead of emitting the fixture. + +The refusal only applies when every expected exception is a block +exception: a block carrying an invalid transaction is invalid +regardless of its header, and the state test conversion routinely +pins header fields to values that legitimately match the computed +ones. +""" + +import textwrap +from typing import Any + +noop_rlp_modifier_test_module = textwrap.dedent( + """\ + from execution_testing import Block + from execution_testing.exceptions.exceptions import BlockException + from execution_testing.specs.blockchain import Header + + + def test_noop_rlp_modifier(blockchain_test, pre) -> None: + blockchain_test( + pre=pre, + post={}, + blocks=[ + Block( + timestamp=12, + rlp_modifier=Header(timestamp=12), + exception=BlockException.INVALID_BLOCK_HASH, + ) + ], + ) + """ +) + + +noop_modifier_tx_exception_test_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction + from execution_testing.exceptions.exceptions import ( + TransactionException, + ) + from execution_testing.specs.blockchain import Header + + + @pytest.mark.exception_test + def test_noop_rlp_modifier_tx_exception(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=21_000, + gas_price=10**9, + sender=pre.fund_eoa(amount=1), + error=TransactionException.INSUFFICIENT_ACCOUNT_FUNDS, + ) + blockchain_test( + pre=pre, + post={}, + blocks=[ + Block( + timestamp=12, + txs=[tx], + rlp_modifier=Header(timestamp=12), + exception=( + TransactionException.INSUFFICIENT_ACCOUNT_FUNDS + ), + ) + ], + ) + """ +) + + +def test_fill_rejects_noop_rlp_modifier_on_invalid_block( + pytester: Any, capsys: Any, pytestconfig: Any +) -> None: + """A no-op modifier on an expected-invalid block must fail the fill.""" + tests_dir = pytester.mkdir("tests") + cancun_tests_dir = tests_dir / "cancun" + cancun_tests_dir.mkdir() + module_dir = cancun_tests_dir / "noop_rlp_modifier_module" + module_dir.mkdir() + test_module = module_dir / "test_noop_rlp_modifier.py" + test_module.write_text(noop_rlp_modifier_test_module) + + pytester.copy_example( + name="src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-fill.ini" + ) + + result = pytester.runpytest_subprocess( + "-c", + "pytest-fill.ini", + "--fork", + "Cancun", + "-m", + "blockchain_test", + "--no-html", + "--output=stdout", + str(test_module.relative_to(pytester.path)), + ) + # Suppress the expected inner pytest failure output from the outer test + capsys.readouterr() + + assert result.ret != 0, "Fill command was expected to fail" + + output = "\n".join(result.outlines + result.errlines) + expected_message = "`rlp_modifier` changed nothing" + assert expected_message in output + + error_line = next( + line for line in output.splitlines() if expected_message in line + ) + # show print but only when -s is passed + if pytestconfig.getoption("capture") == "no": + with capsys.disabled(): + print(error_line) + + +def test_fill_accepts_noop_rlp_modifier_on_tx_exception_block( + pytester: Any, +) -> None: + """ + A no-op modifier on a block whose invalidity comes from a + transaction must fill: the block is invalid regardless of its + header. + """ + tests_dir = pytester.mkdir("tests") + cancun_tests_dir = tests_dir / "cancun" + cancun_tests_dir.mkdir() + module_dir = cancun_tests_dir / "noop_rlp_modifier_tx_module" + module_dir.mkdir() + test_module = module_dir / "test_noop_rlp_modifier_tx_exception.py" + test_module.write_text(noop_modifier_tx_exception_test_module) + + pytester.copy_example( + name="src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-fill.ini" + ) + + result = pytester.runpytest_subprocess( + "-c", + "pytest-fill.ini", + "--fork", + "Cancun", + "-m", + "blockchain_test", + "--no-html", + "--output=stdout", + str(test_module.relative_to(pytester.path)), + ) + outcomes = result.parseoutcomes() + assert outcomes.get("failed", 0) == 0 + assert outcomes.get("passed", 0) > 0 diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py new file mode 100644 index 00000000000..95c61352d09 --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_fill.py @@ -0,0 +1,277 @@ +""" +Test that the sync block reaches exactly the fixture formats that opt +in, in the placement each chain class calls for. + +The sync block is scoped per fixture format: only +``blockchain_test_engine_x`` declares ``sync_block`` true, so a +default fill must emit engine_x fixtures carrying the extra block - +appended out-of-chain (the ``syncPayload`` field) for a valid chain, +prepended in-chain (a leading payload tagged with the ``sync`` phase) +for an invalid singleton - while every other format's chains are +byte-for-byte what the test defines, and ``--no-sync-block`` must +restore every chain as written. These tests fill single-block tests +and read the fixtures back, so a sync block that leaks into the wrong +format or placement, loses its phase tag, or fails to salt per test +fails here rather than in a consumer. +""" + +import json +import textwrap +from pathlib import Path +from typing import Any, Dict + +valid_test_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction + + + # Two tests of one pre-allocation group: a client reused across + # them must not already know either announced sync block. + @pytest.mark.parametrize("value", [1, 2]) + def test_single_block(blockchain_test, pre, value) -> None: + tx = Transaction( + to=0, + value=value, + gas_limit=21_000, + sender=pre.fund_eoa(), + ) + blockchain_test(pre=pre, post={}, blocks=[Block(txs=[tx])]) + """ +) + +invalid_singleton_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction, TransactionException + + + @pytest.mark.exception_test + def test_invalid_singleton(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=20_999, + sender=pre.fund_eoa(), + error=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + blockchain_test( + pre=pre, + post={}, + blocks=[ + Block( + txs=[tx], + exception=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + ], + ) + """ +) + + +def make_test_module(pytester: Any, source: str, name: str) -> Any: + """Write a test module into a pytester tests tree.""" + tests_dir = pytester.mkdir("tests") + cancun_tests_dir = tests_dir / "cancun" + cancun_tests_dir.mkdir() + module_dir = cancun_tests_dir / "sync_block_fill_module" + module_dir.mkdir() + test_module = module_dir / name + test_module.write_text(source) + pytester.copy_example( + name="src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-fill.ini" + ) + return test_module + + +def fill(pytester: Any, test_module: Any, *args: str) -> Path: + """ + Fill the module into a fresh output directory and return it. + + An all-formats fill is two pytest sessions - the `fill` CLI runs + phase 1 (pre-allocation grouping) and phase 2 (fixture filling) + back to back - so both are run here the same way. + """ + output = pytester.path / f"fixtures{len(args)}" + common = ( + "-c", + "pytest-fill.ini", + "--fork", + "Cancun", + "--generate-all-formats", + "--skip-index", + "--no-html", + f"--output={output}", + *args, + str(test_module.relative_to(pytester.path)), + ) + result = pytester.runpytest_subprocess( + "--generate-pre-alloc-groups", *common + ) + assert result.ret == 0, "fill phase 1 was expected to succeed" + result = pytester.runpytest_subprocess("--use-pre-alloc-groups", *common) + assert result.ret == 0, "fill phase 2 was expected to succeed" + return output + + +# Fixture directory to the format name that appears in a test id. +FORMATS = { + "blockchain_tests": "blockchain_test", + "blockchain_tests_engine": "blockchain_test_engine", + "blockchain_tests_engine_x": "blockchain_test_engine_x", +} + + +def fixtures_of_format( + output: Path, format_dir: str +) -> Dict[str, Dict[str, Any]]: + """ + Return the fixtures emitted in a format's directory, keyed by test + id with the format's own name stripped out so the same test's + fixtures line up across formats. + """ + fixtures: Dict[str, Dict[str, Any]] = {} + for path in sorted((output / format_dir).rglob("*.json")): + if "pre_alloc" in path.parts: + continue + for test_id, fixture in json.loads(path.read_text()).items(): + key = test_id.replace(f"-{FORMATS[format_dir]}", "") + fixtures[key] = fixture + assert fixtures, f"no {format_dir} fixtures were emitted" + return fixtures + + +def test_valid_chain_appends_out_of_chain(pytester: Any) -> None: + """ + A valid chain's sync block is appended: the engine_x payload list + stays exactly the test's own chain (as does every other format's), + and the extra block rides out-of-chain in ``syncPayload``, built + on the test's head. + """ + test_module = make_test_module( + pytester, valid_test_module, "test_single_block.py" + ) + output = fill(pytester, test_module) + + for fixture in fixtures_of_format(output, "blockchain_tests").values(): + assert len(fixture["blocks"]) == 1 + for fixture in fixtures_of_format( + output, "blockchain_tests_engine" + ).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 1 + assert payloads[0].get("phase") is None + assert "syncPayload" not in fixture + + for fixture in fixtures_of_format( + output, "blockchain_tests_engine_x" + ).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 1, ( + "the appended sync block must not enter the payload list" + ) + assert payloads[0].get("phase") is None + own = payloads[0]["params"][0] + + sync_payload = fixture["syncPayload"] + assert sync_payload["phase"] == "sync", ( + "the sync payload must carry the sync phase tag" + ) + appended = sync_payload["params"][0] + assert appended["transactions"] == [], ( + "the sync block carries no transactions" + ) + assert appended["parentHash"] == own["blockHash"], ( + "the sync block must build on the test's own head" + ) + assert int(appended["blockNumber"], 16) == 2 + assert int(appended["timestamp"], 16) > int(own["timestamp"], 16) + assert len(appended["extraData"]) == 2 + 2 * 16, ( + "the sync block is salted with a digest of the test id" + ) + assert fixture["lastblockhash"] == own["blockHash"], ( + "the fixture's head stays the author's own block" + ) + + +def test_appended_sync_block_is_salted_per_test(pytester: Any) -> None: + """ + Each test's sync block must be unique to it, so every announced + head is a block the group's reused client has never seen and each + sync attempt is attributable to its own test. + """ + test_module = make_test_module( + pytester, valid_test_module, "test_single_block.py" + ) + output = fill(pytester, test_module) + + salted = { + fixture["syncPayload"]["params"][0]["extraData"] + for fixture in fixtures_of_format( + output, "blockchain_tests_engine_x" + ).values() + } + assert len(salted) == 2 + + +def test_invalid_singleton_prepends_in_chain(pytester: Any) -> None: + """ + An invalid singleton's sync block is prepended: it is load-bearing + ancestry every consumer must replay, so it lives in-chain as the + first payload, tagged with the sync phase, and the test's own + block shifts up by one. No format gains an out-of-chain payload. + """ + test_module = make_test_module( + pytester, invalid_singleton_module, "test_invalid_singleton.py" + ) + output = fill(pytester, test_module) + + for fixture in fixtures_of_format(output, "blockchain_tests").values(): + assert len(fixture["blocks"]) == 1 + for fixture in fixtures_of_format( + output, "blockchain_tests_engine" + ).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 1 + assert payloads[0].get("phase") is None + + for fixture in fixtures_of_format( + output, "blockchain_tests_engine_x" + ).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 2 + prepended, own = (payload["params"][0] for payload in payloads) + assert payloads[0].get("phase") == "sync", ( + "the prepended payload must carry the sync phase tag" + ) + assert payloads[1].get("phase") is None + assert prepended["transactions"] == [], ( + "the prepended block carries no transactions" + ) + assert int(prepended["blockNumber"], 16) == 1 + assert int(own["blockNumber"], 16) == 2 + assert own["parentHash"] == prepended["blockHash"] + assert "syncPayload" not in fixture, ( + "nothing can be appended above an invalid head" + ) + + +def test_no_sync_block_disables_the_sync_block(pytester: Any) -> None: + """With the opt-out no format gains a sync block.""" + test_module = make_test_module( + pytester, valid_test_module, "test_single_block.py" + ) + output = fill(pytester, test_module, "--no-sync-block") + + for fixture in fixtures_of_format(output, "blockchain_tests").values(): + assert len(fixture["blocks"]) == 1 + for format_dir in ( + "blockchain_tests_engine", + "blockchain_tests_engine_x", + ): + for fixture in fixtures_of_format(output, format_dir).values(): + payloads = fixture["engineNewPayloads"] + assert len(payloads) == 1 + assert payloads[0].get("phase") is None + assert "syncPayload" not in fixture diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py new file mode 100644 index 00000000000..c8e0aec5dfa --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_sync_block_markers.py @@ -0,0 +1,249 @@ +""" +Test the eligibility markers' interaction with the sync block. + +A prepend-class marker names a reason the prepended sync block cannot +preserve what an invalid singleton verifies: the extra block shifts +every position, executes the test's own setup, or cannot reproduce a +pinned fee value. The append-class marker names the one reason a +valid chain cannot take its trailer: the test's final state breaks +the empty block built on top of it. A marked test is not skipped - +it fills without the extra block, so no test ever leaves the fixture +release; sync-based consumers skip what cannot sync at consume time +instead. Each marker vetoes only its own placement: a prepend-class +marker is irrelevant to a valid chain's trailer, and the +append-class marker is irrelevant to an invalid singleton's prepend. +""" + +import json +import textwrap +from pathlib import Path +from typing import Any, Dict + +import pytest + +marked_test_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction, TransactionException + + + @pytest.mark.{marker} + @pytest.mark.exception_test + def test_marked_invalid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=20_999, + sender=pre.fund_eoa(), + error=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + blockchain_test( + pre=pre, + post={{}}, + blocks=[ + Block( + txs=[tx], + exception=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + ], + ) + + + @pytest.mark.exception_test + def test_unmarked_invalid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=20_999, + sender=pre.fund_eoa(), + error=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + blockchain_test( + pre=pre, + post={{}}, + blocks=[ + Block( + txs=[tx], + exception=TransactionException.INTRINSIC_GAS_TOO_LOW, + ) + ], + ) + + + @pytest.mark.{marker} + def test_marked_valid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=21_000, + sender=pre.fund_eoa(), + ) + blockchain_test(pre=pre, post={{}}, blocks=[Block(txs=[tx])]) + """ +) + + +def make_test_module(pytester: Any, marker: str) -> Any: + """Write a test module into a pytester tests tree.""" + tests_dir = pytester.mkdir("tests") + cancun_tests_dir = tests_dir / "cancun" + cancun_tests_dir.mkdir() + module_dir = cancun_tests_dir / "sync_block_markers_module" + module_dir.mkdir() + test_module = module_dir / "test_marked.py" + test_module.write_text(marked_test_module.format(marker=marker)) + pytester.copy_example( + name="src/execution_testing/cli/pytest_commands/pytest_ini_files/pytest-fill.ini" + ) + return test_module + + +def fill(pytester: Any, test_module: Any) -> Path: + """Fill the module in both phases and return the output directory.""" + output = pytester.path / "fixtures" + common = ( + "-c", + "pytest-fill.ini", + "--fork", + "Cancun", + "--generate-all-formats", + "--skip-index", + "--no-html", + f"--output={output}", + str(test_module.relative_to(pytester.path)), + ) + result = pytester.runpytest_subprocess( + "--generate-pre-alloc-groups", *common + ) + assert result.ret == 0, "fill phase 1 was expected to succeed" + result = pytester.runpytest_subprocess("--use-pre-alloc-groups", *common) + assert result.ret == 0, "fill phase 2 was expected to succeed" + return output + + +def engine_x_fixtures(output: Path) -> Dict[str, Dict[str, Any]]: + """Return the emitted engine_x fixtures keyed by test id.""" + fixtures: Dict[str, Dict[str, Any]] = {} + for path in sorted((output / "blockchain_tests_engine_x").rglob("*.json")): + if "pre_alloc" in path.parts: + continue + fixtures.update(json.loads(path.read_text())) + assert fixtures, "no engine_x fixtures were emitted" + return fixtures + + +@pytest.mark.parametrize( + "marker", + [ + "absolute_block_position", + "no_empty_block_fee_preimage", + "pre_state_affects_empty_block", + ], +) +def test_marked_test_fills_without_the_sync_block( + pytester: Any, marker: str +) -> None: + """ + Each marker must veto the prepend for its own test only: the + marked invalid singleton fills with exactly its own single block, + the unmarked invalid singleton in the same module gains the + prepended one, and the marked valid test keeps its appended + trailer - the marker means nothing to a chain the extra block + cannot shift. + """ + test_module = make_test_module(pytester, marker=marker) + output = fill(pytester, test_module) + + fixtures = engine_x_fixtures(output) + by_name = {} + for test_id, fixture in fixtures.items(): + for name in ("marked_invalid", "unmarked_invalid", "marked_valid"): + if f"test_{name}[" in test_id: + by_name[name] = fixture + assert set(by_name) == { + "marked_invalid", + "unmarked_invalid", + "marked_valid", + } + + marked_payloads = by_name["marked_invalid"]["engineNewPayloads"] + assert len(marked_payloads) == 1, ( + "the marked invalid singleton must fill without the sync block" + ) + assert marked_payloads[0].get("phase") is None + assert "syncPayload" not in by_name["marked_invalid"] + + unmarked_payloads = by_name["unmarked_invalid"]["engineNewPayloads"] + assert len(unmarked_payloads) == 2, ( + "the unmarked invalid singleton must still gain the prepended block" + ) + assert unmarked_payloads[0].get("phase") == "sync" + + valid_payloads = by_name["marked_valid"]["engineNewPayloads"] + assert len(valid_payloads) == 1 + assert "syncPayload" in by_name["marked_valid"], ( + "a prepend-class marker must not cost a valid chain its trailer" + ) + + +append_marked_test_module = textwrap.dedent( + """\ + import pytest + + from execution_testing import Block, Transaction + + + @pytest.mark.{marker} + def test_marked_valid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=21_000, + sender=pre.fund_eoa(), + ) + blockchain_test(pre=pre, post={{}}, blocks=[Block(txs=[tx])]) + + + def test_unmarked_valid(blockchain_test, pre) -> None: + tx = Transaction( + to=0, + gas_limit=21_000, + sender=pre.fund_eoa(), + ) + blockchain_test(pre=pre, post={{}}, blocks=[Block(txs=[tx])]) + """ +) + + +@pytest.mark.parametrize( + "marker", + [ + "post_state_affects_sync_block", + "no_sync_block_timestamp_headroom", + ], +) +def test_append_marked_test_fills_without_the_trailer( + pytester: Any, marker: str +) -> None: + """ + Each append-class marker must cost exactly its own test the + trailer: the marked valid test fills as the author's bare chain + while the unmarked one in the same module keeps its + ``syncPayload``. + """ + test_module = make_test_module(pytester, marker="unused") + test_module.write_text(append_marked_test_module.format(marker=marker)) + output = fill(pytester, test_module) + + fixtures = engine_x_fixtures(output) + by_name = {} + for test_id, fixture in fixtures.items(): + for name in ("marked_valid", "unmarked_valid"): + if f"test_{name}[" in test_id: + by_name[name] = fixture + assert set(by_name) == {"marked_valid", "unmarked_valid"} + + assert "syncPayload" not in by_name["marked_valid"], ( + "the marked valid test must fill without the appended sync block" + ) + assert len(by_name["marked_valid"]["engineNewPayloads"]) == 1 + assert "syncPayload" in by_name["unmarked_valid"], ( + "the unmarked valid test must keep its appended sync block" + ) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py index 912c3a1d983..8d51a3ffe6a 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py @@ -17,7 +17,7 @@ ) from ...shared.helpers import labeled_format_parameter_set -from ..filler import _strip_xdist_group_suffix +from ..filler import _node_id_without_xdist_group, _strip_xdist_group_suffix class MockItem: @@ -238,6 +238,40 @@ def test_at_in_params_preserved(self) -> None: assert _strip_xdist_group_suffix(nodeid) == expected +class TestNodeIdWithoutXdistGroup: + """ + Test cases for _node_id_without_xdist_group. + + Values derived from the node id must not change when the same fill + runs in parallel, so every group suffix is stripped here, unlike + _strip_xdist_group_suffix which preserves the deliberate ones. + """ + + @pytest.mark.parametrize( + "group", ["t8n-cache-12345678", "bigmem", "custom_group"] + ) + def test_strips_every_group_suffix(self, group: str) -> None: + """Test that any group xdist appends is stripped.""" + expected = "test.py::test[params]" + assert _node_id_without_xdist_group(f"{expected}@{group}") == expected + + def test_no_suffix_unchanged(self) -> None: + """Test that nodeids without @ are unchanged.""" + nodeid = "test.py::test[params]" + assert _node_id_without_xdist_group(nodeid) == nodeid + + def test_at_in_params_preserved(self) -> None: + """Test that a parameter's own @ is not mistaken for a group.""" + nodeid = "test.py::test[email@example.com]" + assert _node_id_without_xdist_group(nodeid) == nodeid + + def test_at_in_params_with_group_suffix(self) -> None: + """Test that a group is stripped from a parameter containing @.""" + nodeid = "test.py::test[email@example.com]@bigmem" + expected = "test.py::test[email@example.com]" + assert _node_id_without_xdist_group(nodeid) == expected + + class TestCacheExecutionOrder: """Test that execution order maximizes cache hits.""" diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py index 4c82a3adbca..aaa65d6b32b 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/execute_fill.py @@ -240,6 +240,53 @@ def pytest_configure(config: pytest.Config) -> None: "pre_alloc_mutable: Marks a test to allow impossible mutations in the " "pre-state.", ) + config.addinivalue_line( + "markers", + "absolute_block_position: Marks a test whose logic or expectations " + "depend on absolute block numbers or block hashes; an invalid " + "singleton so marked fills without the prepended sync block, because " + "the extra block would shift every block position and silently " + "change what the test verifies. Irrelevant to appended sync blocks, " + "which shift nothing.", + ) + config.addinivalue_line( + "markers", + "pre_state_affects_empty_block: Marks a test whose pre-state or " + "genesis environment changes what any block executes - e.g. a " + "deliberately broken system contract that every post-Prague block " + "calls, or a genesis gas limit too small for a minimal block's own " + "system work; an invalid singleton so marked fills without the " + "prepended sync block because the extra block would either fail on " + "that setup or consume its one-shot behavior before the test's own " + "block runs.", + ) + config.addinivalue_line( + "markers", + "no_empty_block_fee_preimage: Marks a test pinning a fee value the " + "prepended sync block cannot reproduce - e.g. under EIP-7918's " + "reserve price, a small nonzero excess blob gas has no parent value " + "that decays to it across an empty block; an invalid singleton so " + "marked fills without the prepended sync block because no genesis " + "compensation exists that preserves the test's fee environment.", + ) + config.addinivalue_line( + "markers", + "post_state_affects_sync_block: Marks a test whose final state " + "breaks the empty block the filler would append above its chain - " + "e.g. it ends with a sabotaged system contract that every " + "subsequent block calls, or one-shot state the extra block would " + "consume; a valid chain so marked fills without the appended sync " + "block, as exactly the author's chain.", + ) + config.addinivalue_line( + "markers", + "no_sync_block_timestamp_headroom: Marks a test whose head pins a " + "timestamp at or next to the uint64 ceiling (2**64 - 1), leaving " + "no room for the sync block the filler would append above its " + "chain - the extra block's timestamp would not fit uint64 and no " + "client could parse it; a valid chain so marked fills without the " + "appended sync block, as exactly the author's chain.", + ) config.addinivalue_line( "markers", "fixture_format_id: ID used to describe the fixture format.", diff --git a/packages/testing/src/execution_testing/fixtures/base.py b/packages/testing/src/execution_testing/fixtures/base.py index 9b2fb475669..0af93daab97 100644 --- a/packages/testing/src/execution_testing/fixtures/base.py +++ b/packages/testing/src/execution_testing/fixtures/base.py @@ -91,6 +91,18 @@ class BaseFixture(CamelModel): FixtureFillingPhase.FILL_AFTER_PRE_ALLOC_GENERATION, } transition_tool_cache_key: ClassVar[str] = "" + sync_block: ClassVar[bool] = False + """ + Whether the filler's sync-block option applies to fixtures of this + format. + + Only formats whose consumers may need to trigger a devp2p sync + (``blockchain_test_engine_x``) opt in; every other format builds + the test's chain exactly as written. Formats that share a + non-empty ``transition_tool_cache_key`` must agree on this value: + the t8n output cache is positional, so formats sharing a key must + build byte-identical chains. + """ @classmethod def output_base_dir_name(cls) -> str: diff --git a/packages/testing/src/execution_testing/fixtures/blockchain.py b/packages/testing/src/execution_testing/fixtures/blockchain.py index a7331378786..726636c5009 100644 --- a/packages/testing/src/execution_testing/fixtures/blockchain.py +++ b/packages/testing/src/execution_testing/fixtures/blockchain.py @@ -922,6 +922,17 @@ class BlockchainEngineXFixture(BlockchainEngineFixtureCommon): FixtureFillingPhase.PRE_ALLOC_GENERATION, } transition_tool_cache_key: ClassVar[str] = "" + sync_block: ClassVar[bool] = True + """ + Engine X fixtures may be consumed by sync-based simulators, which + can only trigger a devp2p sync on a chain whose announced head has + an unknown parent; the filler therefore adds one framework-built + empty block to this format's chains, placed per test by + ``BlockchainTest.sync_block_policy``. The format opts out of the + t8n output cache (its phase-2 chains build on a group genesis), so + its chains diverging from the other blockchain formats' costs no + extra t8n work. + """ pre_hash: str """Hash of the pre-allocation group this test belongs to.""" @@ -937,6 +948,46 @@ class BlockchainEngineXFixture(BlockchainEngineFixtureCommon): ) """Engine API payloads for blockchain execution.""" + sync_payload: FixtureEngineNewPayload | None = None + """ + Framework-built empty payload appended above a fully valid chain's + head so that every payload in ``payloads`` is a wire-guaranteed + ancestor for sync-based consumers - the same out-of-chain + representation ``BlockchainEngineSyncFixture`` uses. + + Kept out of ``payloads`` because it is scaffolding, not test + content: ``payloads``, ``last_block_hash`` and the post state all + keep describing exactly the chain the test author wrote, and + consumers that replay payloads through the Engine API need not + know the field exists (this format ignores unknown fields, so + older readers skip it wholesale). A sync-based consumer announces + it instead of ``payloads[-1]`` and treats the chain as complete + when the client accepts it as head. + + ``None`` for chains that carry no appended block: a single + expected-invalid or Engine API-refused block gets a *prepended* + sync payload instead - in-chain as ``payloads[0]``, because there + the extra block is load-bearing ancestry every consumer must + replay - and every other ineligible chain is exactly the test's + own. + """ + + @property + def has_sync_payload(self) -> bool: + """ + Return whether the fixture carries a framework-injected sync + payload, appended (``sync_payload``) or prepended (a leading + ``payloads`` entry tagged with the sync phase). + + Derived from the payloads rather than stored, so the JSON + schema carries no separate switch. + """ + if self.sync_payload is not None: + return True + return ( + len(self.payloads) > 0 and self.payloads[0].phase == TestPhase.SYNC + ) + class BlockchainEngineStatefulFixture(BlockchainEngineFixtureCommon): """ diff --git a/packages/testing/src/execution_testing/fixtures/engine_x_checks.py b/packages/testing/src/execution_testing/fixtures/engine_x_checks.py index b81eaf38363..4bfceb69699 100644 --- a/packages/testing/src/execution_testing/fixtures/engine_x_checks.py +++ b/packages/testing/src/execution_testing/fixtures/engine_x_checks.py @@ -12,6 +12,25 @@ # payload is a pure function of the test's execution. _STATE_ROOT_DERIVED_FIELDS = ("stateRoot", "blockHash", "parentHash") +# Fields the prepended sync block additionally shifts in every payload +# above it: block numbers move up by one, unpinned timestamps move by +# the empty block's own second, slot numbers chain from the extra +# block's slot, and the block access list embeds position-derived +# content outright (the EIP-2935 system write records the parent's +# hash, which the prepend changes). Scrubbed only when the compared +# Engine X fixture carries a leading in-chain sync payload (the +# prepend class); every other fixture keeps the stricter comparison - +# an *appended* sync payload lives out-of-chain in `syncPayload`, +# shifts nothing, and never enters this comparison. +_SYNC_SHIFTED_FIELDS = ( + "blockNumber", + "timestamp", + "slotNumber", + "blockAccessList", +) + +_SYNC_PHASE = "sync" + class EngineXExecutionDriftError(Exception): """ @@ -67,14 +86,31 @@ def summary(self) -> str: return summary -def _scrubbed_payloads(fixture: Dict[str, Any]) -> List[Any]: - """Return the fixture's payload entries minus state-root-derived fields.""" +def _scrubbed_payloads( + fixture: Dict[str, Any], *, scrub_sync_shift: bool = False +) -> List[Any]: + """ + Return the fixture's payload entries minus state-root-derived + fields, and minus the position-derived fields when + ``scrub_sync_shift`` is set (the prepended empty block shifts them + in every payload above it). + + A leading sync-phase payload (the prepended empty block) is + dropped: it is framework-injected and has no sibling to compare + against. + """ + entries = fixture.get("engineNewPayloads", []) + if entries and entries[0].get("phase") == _SYNC_PHASE: + entries = entries[1:] + scrubbed_fields = _STATE_ROOT_DERIVED_FIELDS + ( + _SYNC_SHIFTED_FIELDS if scrub_sync_shift else () + ) payloads = [] - for entry in fixture.get("engineNewPayloads", []): + for entry in entries: entry = json.loads(json.dumps(entry)) params = entry.get("params") if params and isinstance(params[0], dict): - for field in _STATE_ROOT_DERIVED_FIELDS: + for field in scrubbed_fields: params[0].pop(field, None) payloads.append(entry) return payloads @@ -116,6 +152,19 @@ def verify_engine_x_execution( Engine X fixtures never share the transition tool output cache). All payload fields except the state-root-derived ones must match exactly. + An Engine X fixture whose chain starts with a sync-phase payload (the + empty block the filler prepends below a single invalid block for + sync-based consumers - the sibling formats never carry it) is compared + without that payload, and without the position-derived fields the + shift changes in every payload above it: `blockNumber`, unpinned + `timestamp`, `slotNumber`, and `blockAccessList`, whose EIP-2935 + system-write entry embeds the parent's hash; the genesis fee + compensation makes everything else identical. An *appended* sync + payload needs no such handling: it lives out-of-chain in + `syncPayload`, so the fixture's payload list is the author's own + chain - block access list included - and compares at full + strictness. + Return the comparison counts, or ``None`` when one of the two fixture format trees was not generated at all (e.g. when filling with ``-m blockchain_test_engine_x``, which produces no siblings). @@ -156,8 +205,17 @@ def verify_engine_x_execution( skipped += 1 continue compared += 1 - base = _scrubbed_payloads(sibling) - packed = _scrubbed_payloads(fixture) + payload_entries = fixture.get("engineNewPayloads", []) + has_sync_payload = bool( + payload_entries + and payload_entries[0].get("phase") == _SYNC_PHASE + ) + base = _scrubbed_payloads( + sibling, scrub_sync_shift=has_sync_payload + ) + packed = _scrubbed_payloads( + fixture, scrub_sync_shift=has_sync_payload + ) if base != packed: mismatches.append((test_id, _describe_mismatch(base, packed))) diff --git a/packages/testing/src/execution_testing/fixtures/tests/test_base.py b/packages/testing/src/execution_testing/fixtures/tests/test_base.py index a05dd2210d2..d8d7f3846b6 100644 --- a/packages/testing/src/execution_testing/fixtures/tests/test_base.py +++ b/packages/testing/src/execution_testing/fixtures/tests/test_base.py @@ -159,3 +159,35 @@ def test_base_fixtures_parsing(fixture: BaseFixture) -> None: json_dump = fixture.json_dict_with_info() assert json_dump is not None Fixtures.model_validate({"fixture": json_dump}) + + +def test_formats_sharing_a_cache_key_agree_on_sync_block() -> None: + """ + Fixture formats sharing a non-empty t8n cache key must agree on + ``sync_block``. + + The t8n output cache is positional: formats sharing a key replay + each other's transition results call by call, so they must build + byte-identical chains. A format that prepends a sync block below + an invalid singleton while a cache-sharing sibling does not would + consume the sibling's results one position off and emit chains no + client can execute. + """ + formats_by_cache_key: dict[str, list[type[BaseFixture]]] = {} + for format_class in BaseFixture.formats.values(): + cache_key = format_class.transition_tool_cache_key + if cache_key: + formats_by_cache_key.setdefault(cache_key, []).append(format_class) + assert formats_by_cache_key, "no fixture format declares a cache key" + for cache_key, format_classes in formats_by_cache_key.items(): + sync_block_values = { + format_class.sync_block for format_class in format_classes + } + assert len(sync_block_values) == 1, ( + f"formats sharing the t8n cache key {cache_key!r} disagree " + "on sync_block: " + + ", ".join( + f"{format_class.format_name}={format_class.sync_block}" + for format_class in format_classes + ) + ) diff --git a/packages/testing/src/execution_testing/fixtures/tests/test_engine_x_checks.py b/packages/testing/src/execution_testing/fixtures/tests/test_engine_x_checks.py index 7e44afee03c..ebbf35d7478 100644 --- a/packages/testing/src/execution_testing/fixtures/tests/test_engine_x_checks.py +++ b/packages/testing/src/execution_testing/fixtures/tests/test_engine_x_checks.py @@ -49,11 +49,15 @@ def _write_fixture( fixture_dir: str, test_id: str, payloads: List[Dict[str, Any]], + sync_payload: Dict[str, Any] | None = None, ) -> None: """Write a single-fixture file into a format tree.""" file = folder / fixture_dir / "prague" / "module" / "test_a.json" file.parent.mkdir(parents=True, exist_ok=True) - file.write_text(json.dumps({test_id: {"engineNewPayloads": payloads}})) + fixture: Dict[str, Any] = {"engineNewPayloads": payloads} + if sync_payload is not None: + fixture["syncPayload"] = sync_payload + file.write_text(json.dumps({test_id: fixture})) def test_identical_execution_passes(tmp_path: Path) -> None: @@ -211,3 +215,185 @@ def test_no_matching_siblings_reports_skip_count(tmp_path: Path) -> None: assert result is not None assert result.compared == 0 assert result.skipped == 1 + + +def _sync_payload() -> Dict[str, Any]: + """Build a framework sync block's newPayload entry.""" + payload = _payload(gas_used="0x0", state_root="0x33", block_hash="0x44") + payload["params"][0]["transactions"] = [] + payload["phase"] = "sync" + return payload + + +def _shifted( + payload: Dict[str, Any], *, number: str, timestamp: str +) -> Dict[str, Any]: + """Return a copy of `payload` at a shifted chain position.""" + shifted = json.loads(json.dumps(payload)) + shifted["params"][0]["blockNumber"] = number + shifted["params"][0]["timestamp"] = timestamp + return shifted + + +def test_sync_payload_is_ignored(tmp_path: Path) -> None: + """ + An Engine X fixture prepended with a sync-phase payload compares + clean against its unprepended sibling: the extra payload is + dropped and the block numbers and timestamps it shifts are + scrubbed. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + _write_fixture( + tmp_path, + SIBLING_FIXTURES_DIR, + SIBLING_ID, + [_shifted(base, number="0x1", timestamp="0xc")], + ) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [_sync_payload(), _shifted(base, number="0x2", timestamp="0xd")], + ) + + result = verify_engine_x_execution(tmp_path) + + assert result is not None + assert result.compared == 1 + + +def test_sync_payload_does_not_mask_drift(tmp_path: Path) -> None: + """A real execution difference still fails on a prepended fixture.""" + _write_fixture( + tmp_path, + SIBLING_FIXTURES_DIR, + SIBLING_ID, + [_payload(gas_used="0x5208", state_root="0x01", block_hash="0x02")], + ) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [ + _sync_payload(), + _payload(gas_used="0xbeef", state_root="0xaa", block_hash="0xbb"), + ], + ) + + with pytest.raises(EngineXExecutionDriftError) as exc_info: + verify_engine_x_execution(tmp_path) + + assert "gasUsed" in str(exc_info.value) + + +def test_appended_sync_payload_never_enters_the_comparison( + tmp_path: Path, +) -> None: + """ + An append-class fixture (out-of-chain ``syncPayload``) compares + its payload list against the sibling untouched: the appended block + shifts nothing, so nothing is dropped or scrubbed. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + _write_fixture( + tmp_path, + SIBLING_FIXTURES_DIR, + SIBLING_ID, + [_shifted(base, number="0x1", timestamp="0xc")], + ) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [_shifted(base, number="0x1", timestamp="0xc")], + sync_payload=_sync_payload(), + ) + + result = verify_engine_x_execution(tmp_path) + + assert result is not None + assert result.compared == 1 + + +def test_appended_sync_payload_keeps_full_strictness(tmp_path: Path) -> None: + """ + An append-class fixture must not inherit the prepend class's + position scrubbing: a shifted block number is a real difference. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + _write_fixture( + tmp_path, + SIBLING_FIXTURES_DIR, + SIBLING_ID, + [_shifted(base, number="0x1", timestamp="0xc")], + ) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [_shifted(base, number="0x2", timestamp="0xc")], + sync_payload=_sync_payload(), + ) + + with pytest.raises(EngineXExecutionDriftError) as exc_info: + verify_engine_x_execution(tmp_path) + + assert "blockNumber" in str(exc_info.value) + + +def test_prepended_fixture_scrubs_position_derived_content( + tmp_path: Path, +) -> None: + """ + A prepend-class fixture's shifted slot numbers and block access + list (whose EIP-2935 system write embeds the parent's hash) are + scrubbed alongside the block number and timestamp; a real + execution difference (gas used) still fails. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + sibling = _shifted(base, number="0x1", timestamp="0xc") + sibling["params"][0]["slotNumber"] = "0x1" + sibling["params"][0]["blockAccessList"] = "0xaa" + engine_x = _shifted(base, number="0x2", timestamp="0xd") + engine_x["params"][0]["slotNumber"] = "0x2" + engine_x["params"][0]["blockAccessList"] = "0xbb" + _write_fixture(tmp_path, SIBLING_FIXTURES_DIR, SIBLING_ID, [sibling]) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [_sync_payload(), engine_x], + ) + + result = verify_engine_x_execution(tmp_path) + + assert result is not None + assert result.compared == 1 + + +def test_appended_fixture_keeps_access_list_strictness( + tmp_path: Path, +) -> None: + """ + The append class must not inherit the prepend class's access-list + scrub: nothing shifts under an appended sync payload, so a block + access list difference is a real execution difference. + """ + base = _payload(gas_used="0x5208", state_root="0x01", block_hash="0x02") + sibling = json.loads(json.dumps(base)) + sibling["params"][0]["blockAccessList"] = "0xaa" + engine_x = json.loads(json.dumps(base)) + engine_x["params"][0]["blockAccessList"] = "0xbb" + _write_fixture(tmp_path, SIBLING_FIXTURES_DIR, SIBLING_ID, [sibling]) + _write_fixture( + tmp_path, + ENGINE_X_FIXTURES_DIR, + ENGINE_X_ID, + [engine_x], + sync_payload=_sync_payload(), + ) + + with pytest.raises(EngineXExecutionDriftError) as exc_info: + verify_engine_x_execution(tmp_path) + + assert "blockAccessList" in str(exc_info.value) diff --git a/packages/testing/src/execution_testing/specs/base.py b/packages/testing/src/execution_testing/specs/base.py index bf0585396f8..7876a7434d4 100644 --- a/packages/testing/src/execution_testing/specs/base.py +++ b/packages/testing/src/execution_testing/specs/base.py @@ -10,6 +10,7 @@ Callable, ClassVar, Dict, + FrozenSet, Generator, List, Sequence, @@ -113,6 +114,64 @@ class BaseTest(BaseModel): # instead of each test having to set it ) operation_mode: OpMode | None = None + sync_block: bool = False + """ + Add one framework-built empty block to a blockchain test's chain + when filling for sync-based consumption. + + A client only starts a devp2p sync when the announced head's + parent is unknown to it, and blocks below the head can only reach + the client over devp2p - there is no Engine API path for them. The + placement of the extra block therefore decides what the wire + guarantees, and is resolved per test from the chain's own + structure (see ``BlockchainTest.sync_block_policy``): + + - a fully valid chain gets the block *appended* above its head, + making every one of the test's own blocks a wire-guaranteed + ancestor; + - a single expected-invalid (or Engine API-refused) block gets it + *prepended* between genesis and itself, giving the sync a reason + to start so the client can judge the announced head; + - every other chain is built exactly as the test defines it. + + Set by the filler only when the fixture format being filled opts + in via ``BaseFixture.sync_block`` (currently + ``blockchain_test_engine_x``), the spec type's + ``supports_sync_block`` is true, and the session is not measuring + gas. Ignored by test specs that do not build a chain of blocks, + and by stateful fixtures, whose chains continue a live client's + own head instead of a genesis the framework builds. + """ + sync_block_ineligibilities: FrozenSet[str] = frozenset() + """ + Names of the sync-block ineligibility markers present on the test. + + Each marker declares the test permanently incompatible with one + sync-block placement, and vetoes only that placement's resolution + (see ``BlockchainTest.sync_block_policy``): a marked test is not + skipped - it fills without the sync block, so no test ever leaves + the fixture release. Its chain is then only syncable if the test's + own blocks make it so; sync-based consumers skip the rest at + consume time, which is honest: a chain that cannot take the extra + block cannot trigger a devp2p sync no matter how it is filled. + Set by the filler from the markers on the pytest node. + """ + sync_block_salt: str = "" + """ + Value mixed into the sync block's ``extra_data`` so its hash is + unique to this test. + + The tests of a pre-allocation group share a genesis and are served + to one reused client, and every announced head must be a block + that client has never seen, so each sync attempt is attributable + to its own test. A prepended block builds on the shared genesis + and would be identical across the group without the salt; an + appended block usually inherits uniqueness from its parent (the + test's own head), but two tests of a group whose chains are + byte-identical would share it. A per-test ``extra_data`` keeps the + block's state transition identical while making its hash unique. + The filler sets this to the pytest node id. + """ gas_optimization_max_gas_limit: int | None = None expected_benchmark_gas_used: int | None = None skip_gas_used_validation: bool = False @@ -121,6 +180,17 @@ class BaseTest(BaseModel): is_exception_test: bool = False # Class variables, to be set by subclasses + supports_sync_block: ClassVar[bool] = True + """ + Whether the filler's ``--sync-block`` option applies to this spec + type. + + Consensus test specs support it so sync-based consumers can + trigger a devp2p sync on their chains. Spec types whose + measurements the extra block would distort (benchmark tests) set + this to false and are filled without it even when the option is + given. + """ spec_types: ClassVar[Dict[str, Type["BaseTest"]]] = {} supported_fixture_formats: ClassVar[ Sequence[FixtureFormat | LabeledFixtureFormat] diff --git a/packages/testing/src/execution_testing/specs/benchmark.py b/packages/testing/src/execution_testing/specs/benchmark.py index c1c2d5358e6..d23cbd4f829 100644 --- a/packages/testing/src/execution_testing/specs/benchmark.py +++ b/packages/testing/src/execution_testing/specs/benchmark.py @@ -341,6 +341,13 @@ class BenchmarkTest(BaseTest): include_full_post_state_in_output: bool = False include_tx_receipts_in_output: bool = False + supports_sync_block: ClassVar[bool] = False + """ + Benchmark tests measure per-block gas and timing; a framework + block in the chain would distort those measurements, so the + filler's ``--sync-block`` option never applies to them. + """ + supported_fixture_formats: ClassVar[ Sequence[FixtureFormat | LabeledFixtureFormat] ] = [ diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index ba0f1f191fd..b845ff0c7e7 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -1,11 +1,14 @@ """Ethereum blockchain test spec definition and filler.""" +from enum import Enum +from hashlib import sha256 from pprint import pprint from typing import ( Any, Callable, ClassVar, Dict, + FrozenSet, Generator, List, Sequence, @@ -103,6 +106,83 @@ from .debugging import print_traces from .helpers import verify_block, verify_transactions +DEFAULT_TIMESTAMP_INCREMENT = 12 +""" +Seconds between a block and its parent when the block does not pin a +timestamp of its own (see ``Block.set_environment``). +""" + +PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = frozenset( + { + "absolute_block_position", + "no_empty_block_fee_preimage", + "pre_state_affects_empty_block", + } +) +""" +Markers declaring a test permanently ineligible for the *prepended* +sync block, each naming the reason the transformation cannot preserve +what the test verifies. A marked test is not skipped - it fills +without the extra block (see +``BaseTest.sync_block_ineligibilities``). The markers are irrelevant +to the appended sync block, which changes nothing below itself. +""" + +APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS: FrozenSet[str] = frozenset( + { + "no_sync_block_timestamp_headroom", + "post_state_affects_sync_block", + } +) +""" +Markers declaring a test permanently ineligible for the *appended* +sync block: its final state breaks the extra block t8n would build on +top of it (e.g. a sabotaged system contract every block calls, or +one-shot state the block would consume), or its head pins a timestamp +so close to the uint64 ceiling that no block fits above it. A marked +test fills as exactly the author's chain - fallback bare, never skip. +Irrelevant to the prepended sync block, which sits below everything +the markers describe. +""" + + +class SyncBlockPolicy(Enum): + """ + Placement of the framework-built empty block that makes a chain + devp2p-syncable, resolved per test from the chain's own structure + (see ``BlockchainTest.sync_block_policy``). + + A client only starts a sync when the announced head's parent is + unknown to it, and only blocks *below* the head are guaranteed to + travel devp2p - the head's payload is always announced through the + Engine API, and whether a client also re-fetches it from a peer is + an implementation choice. The placement therefore decides which + blocks the wire guarantees. + """ + + NONE = "none" + """ + The chain is built exactly as the test defines it: it is either + already syncable (an invalid multi-block chain judges its head + after the valid ancestors arrive), cannot take an extra block in + either position, or the sync-block option is off. + """ + PREPEND = "prepend" + """ + One empty block between genesis and a single expected-invalid (or + Engine API-refused) block: nothing can be built on that block, so + the extra block below it is the only way to give the sync a reason + to start before the client judges the announced head. + """ + APPEND = "append" + """ + One empty block above a fully valid chain's head: the appended + block becomes the announced head and every one of the test's own + blocks becomes an ancestor, which a syncing client must fetch and + execute through its sync pipeline - the test's content is + wire-guaranteed by chain structure, on any client. + """ + def environment_from_parent_header(parent: "FixtureHeader") -> "Environment": """Instantiate new environment with the provided header as parent.""" @@ -139,6 +219,83 @@ def apply_new_parent( return env.copy(**updated) +def empty_block_base_fee_preimage( + *, fork: Fork, base_fee_per_gas: int, gas_limit: int +) -> int: + """ + Return the parent base fee from which an empty block derives + ``base_fee_per_gas``. + + An empty parent is below target, so the EIP-1559 progression is a + pure decay step; every value has an exact preimage at or near + ``value * d // (d - 1)`` for change denominator ``d``. Scan the + candidates and verify each with the fork's own calculator so the + result is exact by construction, never approximated. + + The parent here is genesis, which consumes no gas; a test pinning a + nonzero ``gas_used`` on its genesis environment would decay from a + different point than this scan assumes, and its own fee + expectations would then fail the fill. + """ + calculate_base_fee = fork.base_fee_per_gas_calculator() + denominator = fork.base_fee_max_change_denominator() + guess = base_fee_per_gas * denominator // (denominator - 1) + for candidate in range(max(base_fee_per_gas, guess - 2), guess + 3): + derived = calculate_base_fee( + parent_base_fee_per_gas=candidate, + parent_gas_used=0, + parent_gas_limit=gas_limit, + ) + if derived == base_fee_per_gas: + return candidate + raise ValueError( + f"no parent base fee decays to {base_fee_per_gas} on an empty " + f"block at fork {fork.name()}; the prepended sync block cannot " + "preserve this test's fee environment, so the test is " + "ineligible for the prepend and belongs marked " + "`no_empty_block_fee_preimage`" + ) + + +def empty_block_excess_blob_gas_preimage( + *, fork: Fork, excess_blob_gas: int, parent_base_fee_per_gas: int +) -> int: + """ + Return the parent excess blob gas from which an empty block derives + ``excess_blob_gas``. + + An empty parent uses no blob gas, so the progression either decays + the excess by one target (EIP-4844) or, on forks where a fee floor + holds the value in place, leaves it unchanged; the preimage is one + of two candidates, each verified with the fork's own calculator. + Genesis is that parent and uses no blob gas either, so a test + pinning a nonzero ``blob_gas_used`` on its genesis environment + would progress from a different point than these candidates assume. + + Some values are unreachable: under a fee floor the decay is skipped + while the floor holds, so a small nonzero excess has no parent at + all. Such a test is ineligible for the prepended sync block and is + marked ``no_empty_block_fee_preimage`` rather than filled. + """ + calculate_excess_blob_gas = fork.excess_blob_gas_calculator() + target = fork.target_blobs_per_block() * fork.blob_gas_per_blob() + for candidate in (excess_blob_gas + target, excess_blob_gas): + derived = calculate_excess_blob_gas( + parent_excess_blob_gas=candidate, + parent_blob_gas_used=0, + parent_base_fee_per_gas=parent_base_fee_per_gas, + ) + if derived == excess_blob_gas: + return candidate + raise ValueError( + f"no parent excess blob gas decays to {excess_blob_gas} on an " + f"empty block at fork {fork.name()}; the prepended sync block " + "cannot preserve this test's fee environment, so the test is " + "ineligible for the prepend and belongs marked " + "`no_empty_block_fee_preimage`" + ) + + def count_blobs(txs: List[Transaction]) -> int: """Return number of blobs in a list of transactions.""" return sum( @@ -442,7 +599,7 @@ def set_environment(self, env: Environment) -> Environment: else: assert env.parent_timestamp is not None new_env_values["timestamp"] = int( - Number(env.parent_timestamp) + 12 + Number(env.parent_timestamp) + DEFAULT_TIMESTAMP_INCREMENT ) return env.copy(**new_env_values) @@ -577,8 +734,20 @@ def engine_payload_modifier( ) return None - def get_fixture_engine_new_payload(self) -> FixtureEngineNewPayload: - """Get a FixtureEngineNewPayload from the built block.""" + def get_fixture_engine_new_payload( + self, phase: TestPhase | None = None + ) -> FixtureEngineNewPayload: + """ + Get a FixtureEngineNewPayload from the built block. + + ``phase`` overrides the phase auto-derived from the block's + transactions; the filler passes ``TestPhase.SYNC`` for the + empty sync block it builds itself, which carries no + transactions to derive a phase from. + """ + kwargs: Dict[str, Any] = {} + if phase is not None: + kwargs["phase"] = phase return FixtureEngineNewPayload.from_fixture_header( fork=self.fork, header=self.header, @@ -591,6 +760,7 @@ def get_fixture_engine_new_payload(self) -> FixtureEngineNewPayload: execution_payload_modifier=self.engine_payload_modifier(), validation_error=self.expected_exception, error_code=self.engine_api_error_code, + **kwargs, ) def verify_transactions( @@ -782,20 +952,93 @@ def discard_fixture_format_by_marks( and "blockchain_test_only" in marker_names ): return True + engine_formats: List[FixtureFormat] = [ + BlockchainEngineFixture, + BlockchainEngineXFixture, + ] if ( - fixture_format - not in [BlockchainEngineFixture, BlockchainEngineXFixture] + fixture_format not in engine_formats and "blockchain_test_engine_only" in marker_names ): return True return False def get_genesis_environment(self) -> Environment: - """Get the genesis environment for pre-allocation groups.""" + """ + Get the genesis environment for pre-allocation groups. + + When the resolved sync-block policy is prepend, the genesis fee + fields are wound one progression step up so that the prepended + sync block consumes exactly the step it introduces (see + ``_compensate_genesis_fees``). This must happen here rather + than in ``make_genesis``: phase 1 of the two-phase fill hashes + this environment for pre-allocation grouping, so both phases + must see the same compensated genesis. Append-class chains need + no compensation - nothing sits below the test's own blocks, so + the author's genesis passes through untouched. + """ modified_values = self.genesis_environment.set_fork_requirements( self.fork.transitions_from() ).model_dump(exclude_unset=True) - return Environment(**(GENESIS_ENVIRONMENT_DEFAULTS | modified_values)) + env = Environment(**(GENESIS_ENVIRONMENT_DEFAULTS | modified_values)) + if self.sync_block_policy() is SyncBlockPolicy.PREPEND: + env = self._compensate_genesis_fees(env) + return env + + def _compensate_genesis_fees(self, env: Environment) -> Environment: + """ + Wind the genesis fee fields one progression step up to cancel + the prepended sync block's step. + + The prepended block (see ``blocks_to_build``) sits between + genesis and the test's single block and participates in fee + mechanics: it decays the base fee by one EIP-1559 step and the + excess blob gas by one target. Starting genesis one exact + preimage step higher makes the prepended block land precisely + on the fee values the test author gave for genesis, so the + test's own block derives its fee context from an identical + parent and executes in exactly the environment the author + specified. State tests compose transparently: their conversion + already winds genesis one step up from the pinned block + environment, and this adds the one further step the prepended + block consumes. + + Fee fields the genesis fork does not require are left alone; a + value with no preimage fails the fill loudly rather than + producing a semantically shifted fixture. + """ + fork = self.fork.fork_at( + block_number=int(env.number) + 1, + timestamp=int(env.timestamp) + 1, + ) + updates: Dict[str, Any] = {} + base_fee_per_gas: int | None = ( + None if env.base_fee_per_gas is None else int(env.base_fee_per_gas) + ) + if base_fee_per_gas is not None and fork.header_base_fee_required(): + base_fee_per_gas = empty_block_base_fee_preimage( + fork=fork, + base_fee_per_gas=base_fee_per_gas, + gas_limit=int(env.gas_limit), + ) + updates["base_fee_per_gas"] = HexNumber(base_fee_per_gas) + if ( + env.excess_blob_gas is not None + and fork.header_excess_blob_gas_required() + ): + assert base_fee_per_gas is not None, ( + "excess blob gas compensation requires a genesis base fee" + ) + updates["excess_blob_gas"] = HexNumber( + empty_block_excess_blob_gas_preimage( + fork=fork, + excess_blob_gas=int(env.excess_blob_gas), + parent_base_fee_per_gas=base_fee_per_gas, + ) + ) + if not updates: + return env + return env.copy(**updates) def make_genesis( self, *, apply_pre_allocation_blockchain: bool @@ -1008,7 +1251,13 @@ def generate_block_data( if block.rlp_modifier is not None: # Modify any parameter specified in the `rlp_modifier` after # transition tool processing. - header = block.rlp_modifier.apply(header) + modified_header = block.rlp_modifier.apply(header) + self._verify_modifier_still_invalidates( + block=block, + header=header, + modified_header=modified_header, + ) + header = modified_header header.fork = fork # Deleted during `apply` because `exclude=True` # Process block access list - apply transformer if present for invalid @@ -1115,6 +1364,54 @@ def generate_block_data( return built_block + def _verify_modifier_still_invalidates( + self, + *, + block: Block, + header: FixtureHeader, + modified_header: FixtureHeader, + ) -> None: + """ + Refuse an expected-invalid block whose ``rlp_modifier`` is a + no-op. + + A block whose expected invalidity is purely header level rests + on the modifier corrupting the header, so exception + verification is skipped for it. A modifier that changes nothing + therefore means the "wrong" value the test pinned has become + the correct one: the block is valid in this chain context while + the fixture still claims it is invalid. This happens when the + chain context shifts under the test, e.g. a fee progression + moved by the prepended sync block. + + Blocks whose exception list names a transaction exception are + excluded: their invalidity comes from a transaction, and the + state test conversion routinely pins header fields to values + that legitimately match the computed ones. + """ + if block.exception is None: + return + expected_exceptions = ( + block.exception + if isinstance(block.exception, list) + else [block.exception] + ) + if not all( + isinstance(exception, BlockException) + for exception in expected_exceptions + ): + return + if modified_header.model_dump() != header.model_dump(): + return + raise ValueError( + f"block {header.number}'s `rlp_modifier` changed " + "nothing: the block expects " + f"`{block.exception}` but its header already holds " + "the pinned values, so the block is valid in this " + "chain context and the fixture's invalidity " + "expectation no longer tests anything" + ) + def verify_post_state( self, t8n: FillerBackend, @@ -1131,6 +1428,196 @@ def verify_post_state( print_traces(t8n.get_traces()) raise e + def sync_block_policy(self) -> SyncBlockPolicy: + """ + Resolve where this test's chain takes its sync block, from the + chain's statically declared structure. + + The resolution reads only what every ``Block`` declares at + collection time - its expected exceptions and Engine API error + code - so phase 1 of the two-phase fill (which hashes the + genesis environment for pre-allocation grouping) and phase 2 + (which builds the chain) always agree. + + - **Append** when every block is valid and none is refused at + the Engine API: the appended block announces the chain and + makes all of the test's own blocks wire-guaranteed + ancestors. + - **Prepend** for a single expected-invalid block, and equally + for a single block whose ``engine_api_error_code`` expects + the announcement itself to be refused: nothing can be built + on either, and without a block below them no sync can start. + An error-code head must keep its announcement - a block + appended above it would announce the chain instead and the + refusal the test asserts would never happen. + - **None** otherwise: an invalid multi-block chain already + carries valid ancestors that travel the wire, and its head + is judged after they arrive; a multi-block chain with an + error-code block keeps its own announcement for the same + reason the singleton does; an empty chain has nothing to + announce. + + A test marked ineligible for its class's placement (see + ``BaseTest.sync_block_ineligibilities``) also resolves to + none: it fills with exactly its own chain instead of being + skipped, so no test ever leaves the fixture release. + """ + if not self.sync_block or not self.blocks: + return SyncBlockPolicy.NONE + engine_refused = any( + block.engine_api_error_code is not None for block in self.blocks + ) + invalid = any(block.exception is not None for block in self.blocks) + if len(self.blocks) == 1 and (invalid or engine_refused): + if ( + self.sync_block_ineligibilities + & PREPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS + ): + return SyncBlockPolicy.NONE + return SyncBlockPolicy.PREPEND + if not invalid and not engine_refused: + if ( + self.sync_block_ineligibilities + & APPEND_SYNC_BLOCK_INELIGIBILITY_MARKERS + ): + return SyncBlockPolicy.NONE + return SyncBlockPolicy.APPEND + return SyncBlockPolicy.NONE + + def sync_block_extra_data(self) -> Bytes: + """ + Return the sync block's ``extra_data``: a digest of the test's + salt, making the block's hash unique to this test (see + ``BaseTest.sync_block_salt``). + """ + return Bytes(sha256(self.sync_block_salt.encode()).digest()[:16]) + + def blocks_to_build(self) -> List[Block]: + """ + Return the chain's block list, honoring the resolved sync-block + policy. + + Under the prepend policy, one empty block is inserted between + genesis and the test's single expected-invalid (or Engine + API-refused) block, so the announced head has an unknown + parent and sync-based consumers can trigger a devp2p sync + before the client judges it. + + The prepended block's timestamp is pinned to one second after + genesis, below any timestamp a test is likely to pin. Its + ``extra_data`` carries the per-test salt digest (see + ``BaseTest.sync_block_salt``). A block that pins an absolute + ``number`` (state tests converted to blockchain tests always + do) is shifted up by one so the chain stays contiguous; a + deliberately wrong number stays wrong relative to the shifted + chain. + + A pinned timestamp the prepended block's own timestamp does + not clear fails the fill loudly: timestamps are semantic (fork + activation, TIMESTAMP expectations), so they are never + shifted, and building the chain anyway would produce a + non-monotonic - consensus-invalid - sequence. + + The appended sync block is deliberately *not* part of this + list: it is built on the chain's post-state after the test's + own blocks (and their post-state verification) are done, and + lives out-of-chain in the fixture's ``sync_payload`` field, so + the chain built here is byte-for-byte the author's. + """ + if self.sync_block_policy() is not SyncBlockPolicy.PREPEND: + return self.blocks + genesis_timestamp = int(self.get_genesis_environment().timestamp) + self._verify_timestamps_clear_prepended_block(genesis_timestamp) + blocks: List[Block] = [ + Block( + timestamp=HexNumber(genesis_timestamp + 1), + extra_data=self.sync_block_extra_data(), + ) + ] + for block in self.blocks: + if block.number is not None: + block = block.model_copy( + update={"number": HexNumber(block.number + 1)} + ) + blocks.append(block) + return blocks + + def _verify_timestamps_clear_prepended_block( + self, genesis_timestamp: int + ) -> None: + """ + Refuse to prepend when a pinned timestamp cannot follow the + empty block. + + The walk mirrors the block builder: an unpinned block gets its + parent's timestamp plus ``DEFAULT_TIMESTAMP_INCREMENT``, which + can never violate monotonicity, so only pinned timestamps can + collide. A block that declares a block exception is exempt - a + deliberately invalid timestamp stays invalid relative to the + shifted chain - and does not advance the walk, since an + invalid block is rolled back and its successor builds on the + previous head. + + The prepend policy currently confines the walk to single-block + chains, where the check only ever fires for an Engine + API-refused block (an expected-invalid one is exempt). The + walk keeps its general form so it stays correct for any chain + the policy may hand it. + """ + previous_timestamp = genesis_timestamp + 1 + for index, block in enumerate(self.blocks): + if block.exception is not None: + continue + if block.timestamp is None: + previous_timestamp += DEFAULT_TIMESTAMP_INCREMENT + continue + pinned = int(block.timestamp) + if pinned <= previous_timestamp: + raise ValueError( + f"the test's block {index + 1} pins timestamp " + f"{pinned}, which does not clear its parent's " + f"{previous_timestamp}: the chain would be " + "non-monotonic and consensus-invalid. The prepended " + "sync block occupies timestamp " + f"{genesis_timestamp + 1} (genesis + 1) and every " + "block above it must clear its own parent, so raise " + "the pinned timestamps to leave the extra block " + "room." + ) + previous_timestamp = pinned + + def _verify_sync_block_timestamp_headroom( + self, head_timestamp: int + ) -> None: + """ + Refuse to append when the chain's head leaves the sync block + no room below the uint64 ceiling. + + The appended block takes its parent's timestamp plus + ``DEFAULT_TIMESTAMP_INCREMENT``, and a block timestamp must + fit uint64: above a head pinned at or next to ``2**64 - 1`` + the trailer's timestamp does not, and no client can even + parse the resulting payload. Nothing downstream notices + otherwise - Python integers do not overflow and t8n accepts + the value - so the fill refuses loudly instead of emitting an + unusable fixture. The mirror of the prepend side's + monotonicity walk: timestamps are semantic and are never + clamped or shifted. + """ + sync_block_timestamp = head_timestamp + DEFAULT_TIMESTAMP_INCREMENT + if sync_block_timestamp > 2**64 - 1: + raise ValueError( + f"the chain's head pins timestamp {head_timestamp}, " + "leaving no uint64 headroom for the appended sync " + f"block: {head_timestamp} + " + f"{DEFAULT_TIMESTAMP_INCREMENT} exceeds 2**64 - 1, " + "and no client can parse a block whose timestamp " + "does not fit uint64. If the maximal timestamp is the " + "test's purpose, the test is ineligible for the " + "appended sync block and belongs marked " + "`no_sync_block_timestamp_headroom`." + ) + def make_fixture( self, t8n: FillerBackend, @@ -1148,7 +1635,8 @@ def make_fixture( benchmark_gas_used: int | None = None benchmark_block_gas_used: int | None = None benchmark_opcode_count: OpcodeCount | None = None - for block in self.blocks: + blocks = self.blocks_to_build() + for index, block in enumerate(blocks): # This is the most common case, the RLP needs to be constructed # based on the transactions to be included in the block. # Set the environment according to the block to execute. @@ -1159,7 +1647,7 @@ def make_fixture( previous_alloc=alloc, ) block_number = int(built_block.header.number) - is_last_block = block is self.blocks[-1] + is_last_block = index == len(blocks) - 1 if is_last_block and self.operation_mode == OpMode.BENCHMARKING: benchmark_gas_used = built_block.cumulative_gas_used() benchmark_block_gas_used = built_block.block_gas_used() @@ -1253,7 +1741,9 @@ def make_hive_fixture( benchmark_gas_used: int | None = None benchmark_block_gas_used: int | None = None benchmark_opcode_count: OpcodeCount | None = None - for block in self.blocks: + sync_block_policy = self.sync_block_policy() + blocks = self.blocks_to_build() + for index, block in enumerate(blocks): built_block = self.generate_block_data( t8n=t8n, block=block, @@ -1261,7 +1751,7 @@ def make_hive_fixture( previous_alloc=alloc, ) block_number = int(built_block.header.number) - is_last_block = block is self.blocks[-1] + is_last_block = index == len(blocks) - 1 if is_last_block and self.operation_mode == OpMode.BENCHMARKING: benchmark_gas_used = built_block.cumulative_gas_used() benchmark_block_gas_used = built_block.block_gas_used() @@ -1272,7 +1762,15 @@ def make_hive_fixture( block_number=block_number, ) fixture_payloads.append( - built_block.get_fixture_engine_new_payload() + built_block.get_fixture_engine_new_payload( + # The prepended sync block is always the first + # block built; tag it so consumers can tell the + # framework-injected payload from the test's own. + phase=TestPhase.SYNC + if sync_block_policy is SyncBlockPolicy.PREPEND + and index == 0 + else None + ) ) if block.exception is None: alloc = built_block.alloc @@ -1333,6 +1831,42 @@ def make_hive_fixture( "pre_hash": "", # Will be set by BaseTestWrapper } ) + if sync_block_policy is SyncBlockPolicy.APPEND: + assert env.parent_timestamp is not None + self._verify_sync_block_timestamp_headroom( + int(env.parent_timestamp) + ) + # Build the salted empty sync block on the test's + # post-state, after the post-state verification above, + # and keep it out of the payload list: the fixture's + # payloads, head and post state stay exactly the + # author's chain, and the appended block makes every + # one of them a wire-guaranteed ancestor (same + # representation as BlockchainEngineSyncFixture's + # sync_payload below). + try: + sync_built_block = self.generate_block_data( + t8n=t8n, + block=Block(extra_data=self.sync_block_extra_data()), + previous_env=env, + previous_alloc=alloc, + ) + except Exception as e: + raise Exception( + "the appended sync block could not be built on " + "this test's post-state: the state the test " + "ends with breaks the empty block's own " + "execution (e.g. a sabotaged system contract " + "or consumed one-shot state). If that state is " + "the test's purpose, the test is ineligible " + "for the appended sync block and belongs " + "marked `post_state_affects_sync_block`." + ) from e + fixture_data["sync_payload"] = ( + sync_built_block.get_fixture_engine_new_payload( + phase=TestPhase.SYNC + ) + ) fixture = BlockchainEngineXFixture(**fixture_data) elif fixture_format == BlockchainEngineSyncFixture: # Sync fixture format diff --git a/packages/testing/src/execution_testing/specs/tests/test_sync_block.py b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py new file mode 100644 index 00000000000..b3174c03056 --- /dev/null +++ b/packages/testing/src/execution_testing/specs/tests/test_sync_block.py @@ -0,0 +1,630 @@ +""" +Tests for the per-class sync-block policy and its chain transforms. + +The filler adds one framework-built empty block to engine_x chains so +sync-based consumers can trigger a devp2p sync; where the block lands +is resolved per test from the chain's statically declared structure: +appended above a fully valid chain (out-of-chain, the fixture's +``sync_payload``), prepended below a single expected-invalid or Engine +API-refused block (in-chain, shifting the singleton up), and nowhere +otherwise. These tests pin the resolution and the prepend transform; +the appended block is built by ``make_hive_fixture`` against a real +backend and is covered by the filler plugin's pytester tests. +""" + +from hashlib import sha256 +from typing import List + +import pytest + +from execution_testing.base_types import HexNumber +from execution_testing.exceptions import BlockException, EngineAPIError +from execution_testing.forks import Cancun, Fork, London, Osaka, Shanghai +from execution_testing.specs.benchmark import BenchmarkTest +from execution_testing.specs.blockchain import ( + Block, + BlockchainTest, + SyncBlockPolicy, + empty_block_base_fee_preimage, + empty_block_excess_blob_gas_preimage, +) +from execution_testing.specs.state import StateTest +from execution_testing.test_types import Alloc, Environment, Transaction + +SALT = "tests/cancun/test_x.py::test_y[fork_Cancun-blockchain_test]" + +INVALID = Block( + timestamp=1_000, exception=BlockException.INCORRECT_BLOCK_FORMAT +) +REFUSED = Block( + timestamp=1_000, engine_api_error_code=EngineAPIError.InvalidParams +) +VALID = Block(timestamp=1_000) + + +def make_test( + *, blocks: List[Block], salt: str = SALT, sync_block: bool = True +) -> BlockchainTest: + """Create a Cancun blockchain test over the given blocks.""" + return BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=blocks, + sync_block=sync_block, + sync_block_salt=salt, + ) + + +@pytest.mark.parametrize( + "blocks,policy", + [ + pytest.param([VALID], SyncBlockPolicy.APPEND, id="valid_singleton"), + pytest.param( + [VALID, Block(timestamp=2_000)], + SyncBlockPolicy.APPEND, + id="valid_multi_block", + ), + pytest.param( + [INVALID], SyncBlockPolicy.PREPEND, id="invalid_singleton" + ), + pytest.param( + [REFUSED], + SyncBlockPolicy.PREPEND, + id="engine_refused_singleton", + ), + pytest.param( + [ + Block( + timestamp=1_000, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + engine_api_error_code=EngineAPIError.InvalidParams, + ) + ], + SyncBlockPolicy.PREPEND, + id="invalid_and_refused_singleton", + ), + pytest.param( + [ + VALID, + Block( + timestamp=2_000, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + ), + ], + SyncBlockPolicy.NONE, + id="invalid_multi_block", + ), + pytest.param( + [ + Block( + timestamp=1_000, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + ), + Block(timestamp=2_000), + ], + SyncBlockPolicy.NONE, + id="mid_chain_invalid_block", + ), + pytest.param( + [ + VALID, + Block( + timestamp=2_000, + engine_api_error_code=EngineAPIError.InvalidParams, + ), + ], + SyncBlockPolicy.NONE, + id="engine_refused_multi_block_head", + ), + pytest.param([], SyncBlockPolicy.NONE, id="empty_chain"), + ], +) +def test_sync_block_policy_resolution( + blocks: List[Block], policy: SyncBlockPolicy +) -> None: + """ + The policy must follow the chain's statically declared structure. + + An Engine API-refused block must never sit below an appended sync + block: the appended block would become the announced head, the + refused block would arrive over devp2p instead of through + `newPayload`, and the refusal the test asserts would never happen. + """ + assert make_test(blocks=blocks).sync_block_policy() is policy + + +def test_sync_block_disabled_resolves_to_none() -> None: + """Without the option every chain keeps its own structure.""" + test = make_test(blocks=[VALID], sync_block=False) + assert test.sync_block_policy() is SyncBlockPolicy.NONE + assert test.blocks_to_build() is test.blocks + + +def test_append_class_chain_is_untouched() -> None: + """ + The appended sync block must not appear in the chain's block list: + it is built on the test's post-state after the chain (and its + post-state verification) is done, and lives out-of-chain in the + fixture's ``sync_payload`` field. + """ + test = make_test(blocks=[VALID]) + assert test.blocks_to_build() is test.blocks + + +def test_prepend_inserts_one_salted_empty_block() -> None: + """ + The prepend-class chain gains one empty block at genesis + 1 + carrying a digest of the test's salt in its ``extra_data``. + """ + test = make_test(blocks=[INVALID]) + blocks = test.blocks_to_build() + assert len(blocks) == 2 + prepended = blocks[0] + assert prepended.timestamp == 1 + assert prepended.txs == [] + assert prepended.exception is None + assert prepended.extra_data == sha256(SALT.encode()).digest()[:16] + assert blocks[1] is test.blocks[0], "the test's own block passes through" + + +def test_sync_block_differs_per_test() -> None: + """ + Tests of a pre-allocation group must not share the sync block: a + client that already knows the announced head never starts a sync. + """ + first = make_test(blocks=[INVALID], salt="a").blocks_to_build() + second = make_test(blocks=[INVALID], salt="b").blocks_to_build() + assert first[0].extra_data != second[0].extra_data + + +def test_prepend_shifts_a_pinned_block_number() -> None: + """A block pinning an absolute number shifts up to stay contiguous.""" + test = make_test( + blocks=[ + Block( + number=1, + timestamp=1_000, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + ) + ] + ) + numbers = [block.number for block in test.blocks_to_build()] + assert numbers == [None, 2] + assert test.blocks[0].number == 1, "the test's own block is untouched" + + +def test_expected_invalid_block_is_exempt_from_the_timestamp_walk() -> None: + """ + An expected-invalid block pinning the prepended block's own + timestamp still fills: it is rejected and rolled back anyway, and + the collision only adds one more reason. + """ + test = make_test( + blocks=[ + Block(timestamp=1, exception=BlockException.INCORRECT_BLOCK_FORMAT) + ] + ) + assert len(test.blocks_to_build()) == 2 + + +def test_engine_refused_block_timestamp_collision_is_refused() -> None: + """ + An Engine API-refused block is consensus-valid, so a pinned + timestamp the prepended block does not clear would build a + non-monotonic - consensus-invalid - chain and must fail loudly. + """ + test = make_test( + blocks=[ + Block( + timestamp=1, + engine_api_error_code=EngineAPIError.InvalidParams, + ) + ] + ) + with pytest.raises(ValueError, match="does not clear its parent"): + test.blocks_to_build() + + +def test_benchmark_tests_opt_out_of_the_sync_block() -> None: + """ + Benchmark tests must never receive a sync block: a framework block + in the chain would distort their per-block measurements. + """ + assert BenchmarkTest.supports_sync_block is False + assert BlockchainTest.supports_sync_block is True + assert StateTest.supports_sync_block is True + + +def test_sync_block_timestamp_headroom() -> None: + """ + A head timestamp must leave the appended sync block room below + the uint64 ceiling: the last parent the increment clears passes, + and the first that overflows - and the ceiling itself - are + refused loudly. + """ + test = make_test(blocks=[VALID]) + test._verify_sync_block_timestamp_headroom(2**64 - 13) + with pytest.raises(ValueError, match="no uint64 headroom"): + test._verify_sync_block_timestamp_headroom(2**64 - 12) + with pytest.raises(ValueError, match="no uint64 headroom"): + test._verify_sync_block_timestamp_headroom(2**64 - 1) + + +def test_state_test_conversion_carries_the_sync_block_fields() -> None: + """ + A converted state test resolves its policy like the blockchain + test it becomes: its single valid block appends, and the salt + rides along. + """ + state_test = StateTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + tx=Transaction(sender=HexNumber(0), to=HexNumber(0), nonce=0), + env=Environment(number=1, timestamp=1_000), + sync_block=True, + sync_block_salt=SALT, + ) + blockchain_test = state_test.generate_blockchain_test() + assert blockchain_test.sync_block_policy() is SyncBlockPolicy.APPEND + assert blockchain_test.sync_block_salt == SALT + + +GAS_LIMIT = 100_000_000 +CANCUN_TARGET_BLOB_GAS = ( + Cancun.target_blobs_per_block() * Cancun.blob_gas_per_blob() +) + + +@pytest.mark.parametrize( + "base_fee_per_gas", + [0, 1, 6, 7, 8, 9, 10, 100, 1_000, 875_000_000, 1_000_000_000], +) +@pytest.mark.parametrize("fork", [London, Cancun]) +def test_base_fee_preimage_round_trip( + fork: Fork, base_fee_per_gas: int +) -> None: + """An empty block must decay the preimage to the intended value.""" + preimage = empty_block_base_fee_preimage( + fork=fork, + base_fee_per_gas=base_fee_per_gas, + gas_limit=GAS_LIMIT, + ) + derived = fork.base_fee_per_gas_calculator()( + parent_base_fee_per_gas=preimage, + parent_gas_used=0, + parent_gas_limit=GAS_LIMIT, + ) + assert derived == base_fee_per_gas + + +@pytest.mark.parametrize( + "excess_blob_gas", + [ + 0, + 0x20000, + 0x60000, + 0x80000, + 0xE0000, + 0x12345, # not a multiple of the blob gas quantum + ], +) +def test_excess_blob_gas_preimage_round_trip(excess_blob_gas: int) -> None: + """An empty block must decay the preimage to the intended value.""" + preimage = empty_block_excess_blob_gas_preimage( + fork=Cancun, + excess_blob_gas=excess_blob_gas, + parent_base_fee_per_gas=7, + ) + derived = Cancun.excess_blob_gas_calculator()( + parent_excess_blob_gas=preimage, + parent_blob_gas_used=0, + parent_base_fee_per_gas=7, + ) + assert derived == excess_blob_gas + + +@pytest.mark.parametrize("excess_blobs", [1, 2, 3, 4, 5]) +def test_excess_blob_gas_with_no_preimage_is_refused( + excess_blobs: int, +) -> None: + """ + A value the reserve price makes unreachable must be refused. + + Above the reserve price an empty block leaves the excess where it + is, so a genesis winding up to a small nonzero excess does not + exist. Refusing loudly here is what keeps a silently shifted fee + environment out of the fixtures. + """ + with pytest.raises(ValueError, match="no parent excess blob gas"): + empty_block_excess_blob_gas_preimage( + fork=Osaka, + excess_blob_gas=excess_blobs * Osaka.blob_gas_per_blob(), + parent_base_fee_per_gas=17, + ) + + +def make_fee_pinning_test(*, blocks: List[Block]) -> BlockchainTest: + """Create a Cancun blockchain test pinning genesis fee values.""" + return BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=blocks, + genesis_environment=Environment( + base_fee_per_gas=HexNumber(10), + excess_blob_gas=HexNumber(0x80000), + ), + sync_block=True, + sync_block_salt="test", + ) + + +def test_append_class_genesis_is_not_compensated() -> None: + """ + A valid chain's genesis passes through untouched: nothing sits + below the test's own blocks, so there is no fee step to cancel. + """ + env = make_fee_pinning_test(blocks=[VALID]).get_genesis_environment() + assert env.base_fee_per_gas == 10 + assert env.excess_blob_gas == 0x80000 + + +def test_prepend_class_genesis_compensation() -> None: + """ + One empty-block step from an invalid singleton's compensated + genesis lands on the author's genesis values. + """ + env = make_fee_pinning_test(blocks=[INVALID]).get_genesis_environment() + assert env.base_fee_per_gas is not None + assert env.excess_blob_gas is not None + derived_base_fee = Cancun.base_fee_per_gas_calculator()( + parent_base_fee_per_gas=int(env.base_fee_per_gas), + parent_gas_used=0, + parent_gas_limit=int(env.gas_limit), + ) + assert derived_base_fee == 10 + derived_excess = Cancun.excess_blob_gas_calculator()( + parent_excess_blob_gas=int(env.excess_blob_gas), + parent_blob_gas_used=0, + parent_base_fee_per_gas=int(env.base_fee_per_gas), + ) + assert derived_excess == 0x80000 + + +def test_prepend_class_pre_blob_genesis_compensation() -> None: + """A pre-Cancun invalid singleton compensates the base fee only.""" + test = BlockchainTest( + fork=Shanghai, + pre=Alloc(), + post=Alloc(), + blocks=[INVALID], + genesis_environment=Environment(base_fee_per_gas=HexNumber(1000)), + sync_block=True, + sync_block_salt="test", + ) + env = test.get_genesis_environment() + assert env.base_fee_per_gas is not None + assert env.excess_blob_gas is None + derived_base_fee = Shanghai.base_fee_per_gas_calculator()( + parent_base_fee_per_gas=int(env.base_fee_per_gas), + parent_gas_used=0, + parent_gas_limit=int(env.gas_limit), + ) + assert derived_base_fee == 1000 + + +def make_invalid_state_test(*, sync_block: bool) -> StateTest: + """ + Create a Cancun state test that converts to an invalid singleton + while pinning its block fee values. + """ + return StateTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + tx=Transaction(sender=HexNumber(0), to=HexNumber(0), nonce=0), + env=Environment( + number=1, + timestamp=1_000, + base_fee_per_gas=HexNumber(10), + excess_blob_gas=HexNumber(0xE0000), + ), + block_exception=BlockException.INCORRECT_BLOCK_FORMAT, + sync_block=sync_block, + sync_block_salt="test", + ) + + +def test_invalid_state_test_composition() -> None: + """ + Two empty-block steps from a converted invalid state test's + compensated genesis land on the pinned block environment. + + The state-test conversion winds genesis one step up from the + pinned block environment; the prepend compensation adds the one + further step the prepended sync block consumes. + """ + blockchain_test = make_invalid_state_test( + sync_block=True + ).generate_blockchain_test() + assert blockchain_test.sync_block_policy() is SyncBlockPolicy.PREPEND + env = blockchain_test.get_genesis_environment() + assert env.base_fee_per_gas is not None + assert env.excess_blob_gas is not None + + base_fee = int(env.base_fee_per_gas) + excess = int(env.excess_blob_gas) + base_fee_calculator = Cancun.base_fee_per_gas_calculator() + excess_calculator = Cancun.excess_blob_gas_calculator() + for _ in range(2): + new_excess = excess_calculator( + parent_excess_blob_gas=excess, + parent_blob_gas_used=0, + parent_base_fee_per_gas=base_fee, + ) + base_fee = base_fee_calculator( + parent_base_fee_per_gas=base_fee, + parent_gas_used=0, + parent_gas_limit=int(env.gas_limit), + ) + excess = new_excess + assert base_fee == 10 + assert excess == 0xE0000 + + +def test_state_test_conversion_unchanged_without_sync_block() -> None: + """Without the option the conversion winds exactly one step up.""" + blockchain_test = make_invalid_state_test( + sync_block=False + ).generate_blockchain_test() + env = blockchain_test.get_genesis_environment() + assert env.base_fee_per_gas == 10 * 8 // 7 + assert env.excess_blob_gas == 0xE0000 + CANCUN_TARGET_BLOB_GAS + + +def test_prepend_ineligibility_marker_falls_back_bare() -> None: + """ + A marked invalid singleton fills with exactly its own chain: the + marker vetoes the prepend, and the test is never skipped. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[INVALID], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset({"absolute_block_position"}), + ) + assert test.sync_block_policy() is SyncBlockPolicy.NONE + assert test.blocks_to_build() is test.blocks + + +def test_prepend_ineligibility_marker_is_irrelevant_to_append() -> None: + """ + A prepend-class marker on a valid chain changes nothing: the + appended sync block shifts no position, so the chain keeps its + trailer. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[VALID], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset({"absolute_block_position"}), + ) + assert test.sync_block_policy() is SyncBlockPolicy.APPEND + + +def test_prepend_ineligibility_restores_the_author_genesis() -> None: + """ + The marker's veto must reach the genesis compensation: a marked + invalid singleton's pinned fee environment passes through + untouched, in both fill phases. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[INVALID], + genesis_environment=Environment( + base_fee_per_gas=HexNumber(10), + excess_blob_gas=HexNumber(0x80000), + ), + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset({"absolute_block_position"}), + ) + env = test.get_genesis_environment() + assert env.base_fee_per_gas == 10 + assert env.excess_blob_gas == 0x80000 + + +def test_append_ineligibility_marker_falls_back_bare() -> None: + """ + A marked valid chain fills as exactly the author's chain: the + marker vetoes the append, and the test is never skipped. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[VALID], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset( + {"post_state_affects_sync_block"} + ), + ) + assert test.sync_block_policy() is SyncBlockPolicy.NONE + assert test.blocks_to_build() is test.blocks + + +def test_append_ineligibility_marker_is_irrelevant_to_prepend() -> None: + """ + The append-class marker on an invalid singleton changes nothing: + the prepended sync block never sees the test's post-state, so the + chain keeps its prepend. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[INVALID], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset( + {"post_state_affects_sync_block"} + ), + ) + assert test.sync_block_policy() is SyncBlockPolicy.PREPEND + + +def test_timestamp_headroom_marker_falls_back_bare() -> None: + """ + A marked maximal-timestamp valid chain fills as exactly the + author's chain: the marker vetoes the append, and the test is + never skipped. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[Block(timestamp=2**64 - 1)], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset( + {"no_sync_block_timestamp_headroom"} + ), + ) + assert test.sync_block_policy() is SyncBlockPolicy.NONE + assert test.blocks_to_build() is test.blocks + + +def test_timestamp_headroom_marker_is_irrelevant_to_prepend() -> None: + """ + The headroom marker on an invalid singleton changes nothing: the + prepended sync block sits below the chain, where the uint64 + ceiling above the head is not its problem. + """ + test = BlockchainTest( + fork=Cancun, + pre=Alloc(), + post=Alloc(), + blocks=[ + Block( + timestamp=2**64 - 1, + exception=BlockException.INCORRECT_BLOCK_FORMAT, + ) + ], + sync_block=True, + sync_block_salt="test", + sync_block_ineligibilities=frozenset( + {"no_sync_block_timestamp_headroom"} + ), + ) + assert test.sync_block_policy() is SyncBlockPolicy.PREPEND diff --git a/packages/testing/src/execution_testing/test_types/phase_manager.py b/packages/testing/src/execution_testing/test_types/phase_manager.py index cdd4fb182d5..22e14f77bfc 100644 --- a/packages/testing/src/execution_testing/test_types/phase_manager.py +++ b/packages/testing/src/execution_testing/test_types/phase_manager.py @@ -15,6 +15,16 @@ class TestPhase(str, Enum): # compatibility EXECUTION = "testing" CLEANUP = "cleanup" + SYNC = "sync" + """ + A framework-injected payload that exists only to make the chain + syncable, such as the empty block the filler adds to a test's + chain - appended above a valid chain's head, or prepended between + genesis and a single invalid block - so that sync-based consumers + can trigger a devp2p sync. Unlike ``SETUP``, a sync payload + prepares no state a test depends on; consumers that replay + payloads through the Engine API can treat it like any other block. + """ class TestPhaseManager: diff --git a/tests/amsterdam/eip8282_builder_execution_requests/test_modified_builder_contract.py b/tests/amsterdam/eip8282_builder_execution_requests/test_modified_builder_contract.py index ead0b94e08b..22da4780023 100644 --- a/tests/amsterdam/eip8282_builder_execution_requests/test_modified_builder_contract.py +++ b/tests/amsterdam/eip8282_builder_execution_requests/test_modified_builder_contract.py @@ -198,6 +198,12 @@ def test_extra_builder_exits( ) +# Every post-Amsterdam block calls the (deliberately broken or +# gas-hungry one-shot) builder contracts, so the generated error modes +# are invalid singletons on whose pre-state the prepended sync block +# cannot build; the passing params are valid chains and keep their +# appended trailer. +@pytest.mark.pre_state_affects_empty_block @pytest.mark.parametrize( "system_contract", [ diff --git a/tests/cancun/eip4788_beacon_root/test_beacon_root_contract.py b/tests/cancun/eip4788_beacon_root/test_beacon_root_contract.py index 3150ca50ee7..c799fd22387 100644 --- a/tests/cancun/eip4788_beacon_root/test_beacon_root_contract.py +++ b/tests/cancun/eip4788_beacon_root/test_beacon_root_contract.py @@ -111,8 +111,18 @@ def test_beacon_root_contract_calls( [ (0x0C, True), # twelve (2**32, True), # arbitrary - (2**64 - 2, True), # near-max - (2**64 - 1, True), # max + # The (near-)max heads leave no uint64 room for a block above + # them, so no sync block can be appended to their chains. + pytest.param( + 2**64 - 2, # near-max + True, + marks=pytest.mark.no_sync_block_timestamp_headroom, + ), + pytest.param( + 2**64 - 1, # max + True, + marks=pytest.mark.no_sync_block_timestamp_headroom, + ), # TODO: Update t8n to un marshal > 64-bit int # Exception: failed to evaluate: ERROR(10): failed un marshaling stdin # (2**64, False), # overflow @@ -209,8 +219,18 @@ def test_calldata_lengths( [ (12, 12), # twelve (2**32, 2**32), # arbitrary - (2**64 - 2, 2**64 - 2), # near-max - (2**64 - 1, 2**64 - 1), # max + # The (near-)max heads leave no uint64 room for a block above + # them, so no sync block can be appended to their chains. + pytest.param( + 2**64 - 2, # near-max + 2**64 - 2, + marks=pytest.mark.no_sync_block_timestamp_headroom, + ), + pytest.param( + 2**64 - 1, # max + 2**64 - 1, + marks=pytest.mark.no_sync_block_timestamp_headroom, + ), ], indirect=["beacon_root"], ) diff --git a/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py b/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py index f83b13d5082..5c4ef096336 100644 --- a/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py +++ b/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py @@ -137,6 +137,11 @@ def test_extra_withdrawals( ) +# Every block calls the (deliberately broken or gas-hungry one-shot) +# withdrawal contract, so the generated error modes are invalid +# singletons on whose pre-state the prepended sync block cannot build; +# the passing params are valid chains and keep their appended trailer. +@pytest.mark.pre_state_affects_empty_block @pytest.mark.parametrize( "system_contract", [Address(Spec_EIP7002.WITHDRAWAL_REQUEST_PREDEPLOY_ADDRESS)], diff --git a/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py b/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py index ef63436c21c..ca2ec30bae4 100644 --- a/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py +++ b/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py @@ -136,6 +136,11 @@ def test_extra_consolidations( ) +# Every block calls the (deliberately broken or gas-hungry one-shot) +# consolidation contract, so the generated error modes are invalid +# singletons on whose pre-state the prepended sync block cannot build; +# the passing params are valid chains and keep their appended trailer. +@pytest.mark.pre_state_affects_empty_block @pytest.mark.parametrize( "system_contract", [Address(Spec_EIP7251.CONSOLIDATION_REQUEST_PREDEPLOY_ADDRESS)],