diff --git a/docs/navigation.md b/docs/navigation.md index db82611d077..09d93b182d9 100644 --- a/docs/navigation.md +++ b/docs/navigation.md @@ -67,6 +67,7 @@ * [Consume Cache & Fixture Inputs](running_tests/consume/cache.md) * [Consume Direct](running_tests/consume/direct.md) * [Consume Simulators](running_tests/consume/simulators.md) + * [Consume WireX](running_tests/consume/wirex.md) * [Exception Tests](running_tests/consume/exceptions.md) * [Execute Commands](./running_tests/execute/index.md) * [Execute Hive](./running_tests/execute/hive.md) diff --git a/docs/running_tests/consume/wirex.md b/docs/running_tests/consume/wirex.md new file mode 100644 index 00000000000..fad982eb496 --- /dev/null +++ b/docs/running_tests/consume/wirex.md @@ -0,0 +1,137 @@ +# Consume WireX + +The WireX simulator (`eels/consume-wirex`) makes the client under test full sync each test's chain from a deterministic mock devp2p peer implemented inside the testing framework. + +The intent is to verify that clients can receive and propagate blocks over devp2p using the consensus test corpus; it is not intended to be a complete test of historical sync. WireX intends to replace [`consume rlp`](../running.md#rlp) for post-Merge forks: instead of loading RLP-encoded blocks through a client-specific offline import mode at startup, the client downloads and executes the same blocks through its production peer-to-peer ingestion path. + +## Command Syntax + +```bash +uv run consume wirex [OPTIONS] +``` + +WireX consumes the [Blockchain Engine X Test](../test_formats/blockchain_test_engine_x.md) fixture format and keeps the client topology that [`consume enginex`](../running.md#enginex) established: one client per pre-allocation group, reused across all of the group's tests. Only the way blocks arrive changes. Each test is an independent chain that forks at the group's shared genesis. + +To see the WireX-specific options, run: + +```bash +uv run consume wirex --help +``` + +## The Control Plane and the Data Plane + +A post-Merge client does not choose its own head; something must tell it what to sync to. WireX splits this deliberately: + +- The control plane is the Engine API. One `engine_newPayload` carries the head block and one `engine_forkchoiceUpdated` names it as the head. That is the entire non-devp2p surface of a test. +- The data plane is devp2p. The client downloads headers and bodies from the mock peer over RLPx and executes the blocks itself. + +For a chain of N blocks, the Engine API carries only the announced head (inside its payload); devp2p carries the remaining headers and every non-empty body; and the client's full-sync path executes all N blocks. `engine_newPayload` for a block whose parent is unknown executes nothing; it only caches the header and answers `SYNCING`. As long as the announced head's parent is unknown to the client, every block below it is really fetched from the peer and executed by the sync path. + +Which block is announced is decided by the fixture's chain class (see [The Sync Block and Chain Classes](#the-sync-block-and-chain-classes)): a fixture carrying an appended sync payload has that trailer announced, so every block the test author wrote is an ancestor the client must download — on every client, by chain structure. Only blocks *below* the announced head are guaranteed to travel devp2p: the head's payload always arrives through the Engine API, and whether a client also re-fetches the head's body from a peer is an implementation choice that measured clients answer both ways. + +There is deliberately no rewind between tests. Every test's chain forks at the group's genesis, so announcing the new head is all a consensus client would ever do; a backward forkchoice update is not part of the flow because clients that honor it can enter recovery modes that bypass block execution. + +## Process Diagram + +```mermaid +sequenceDiagram + participant S as Simulator (pytest) + participant E as Client: Engine API + participant D as Client: devp2p + participant P as MockPeer + + note over S,P: once per pre-allocation group + S->>E: start client (group genesis + pre-alloc) + S->>P: connect(first chain) + P->>D: dial, RLPx auth/ack, Hello (eth/69-71, p2p v5) + D-->>P: Hello (capabilities), highest common eth version wins, Snappy on + P->>D: eth Status (fork id, earliest/latest, head hash) + S->>E: eth_getBlockByNumber(0), verify genesis, once per client + + note over S,P: per test (chain forks at group genesis) + S->>P: set_chain(chain), BlockRangeUpdate, old chains stay served + S->>E: newPayload(announced head: syncPayload, or the chain's own) + E-->>S: SYNCING (parent unknown, nothing executes) + S->>E: fcU(head, safe=finalized=genesis) + E-->>S: SYNCING + D->>P: GetBlockHeaders + P-->>D: headers + D->>P: GetBlockBodies (non-empty bodies) + P-->>D: bodies + note over D: full sync executes every block,
head included (real EVM work) + loop poll until synced or timeout (re-announce on a slow cadence) + S->>E: eth_getBlockByHash(head) + E-->>S: null ... then the block + end + S->>E: fcU(head, safe=finalized=genesis), confirming + E-->>S: VALID + S->>E: eth_getBlockByNumber(latest), verify head hash + + opt client hung up on the peer + P->>D: redial + full handshake, chains stay served + end +``` + +A sync is awaited by polling for the head block itself (`eth_getBlockByHash`), never by repeating the forkchoice update: in some clients every forkchoice update restarts the sync cycle, so polling faster than a cycle completes prevents the sync from ever finishing. The announcement is re-sent only on a slow cadence (`--wirex-announce-interval`), and its `engine_newPayload` response is read each time: an `INVALID` answer fails the test immediately with the client's `validationError` instead of waiting for the sync timeout. + +## The Mock Peer + +The peer is implemented in `execution_testing.devp2p`: an RLPx transport (ECIES handshake, frame MACs, Snappy compression, p2p v5) and the eth wire protocol in versions 69 through 71. The wire dialect is negotiated per the RLPx rule (highest shared version wins) and recorded in every test's transcript; `--wirex-eth-version` pins the advertised set, so an explicit version makes a client that lacks it fail the handshake loudly. + +The peer is deliberately honest. It never withholds, reorders or corrupts a response, so a sync failure is a finding about the client or the fixture rather than about the peer. Its behavior in detail: + +- Receipts, and from eth/71 block access lists, are counted and left unanswered, never invented. A full-syncing client derives both by executing blocks, so a request for them means the client chose a path this simulator cannot honestly serve. Serving them would convert real failures into silent no-coverage passes; a nonzero unanswered-request count in the transcript is a finding, never noise. +- Chains already served stay served. A client's downloader does not drop the chain it was syncing when a test ends, so the peer answers each request from the chain the requested hash belongs to. +- Responses are bounded by serialized bytes (2 MiB, matching the limit clients themselves serve under), because clients cap the size of every message they read and drop peers that exceed it. +- The peer redials the client after a disconnect, as a real peer would; liveness is checked at each re-announcement. +- Every request and response is recorded in a per-test transcript, which is what makes a stalled sync diagnosable after the fact. + +## Rejection Tests + +Fixtures containing an intentionally invalid block are not skipped; they run as rejection tests. The peer serves the chain as-is — for an invalid singleton that chain is `G → S → T₁*`, the prepended sync block giving the sync a reason to start below the block under judgement — and once the ancestry has arrived over devp2p, the client must answer `INVALID` to `engine_newPayload` for the head. Accepting an invalid chain fails the test — but only a `VALID` that holds for half a second counts as acceptance. A client answers `newPayload` from the database state of that instant, so while the chain is still arriving that answer can be an artifact rather than a judgement: geth has been observed answering a well-formed `VALID` for a block its own beacon backfill rejected fifteen milliseconds later, and `INVALID` on every ask thereafter. A verdict that outlives the sync is a real one. + +Only the fact of rejection is asserted: the Engine API's `validationError` is free-form client text and devp2p carries no error reason at all, so matching the fixture's specific exception over this path is deliberately not attempted (the client's reason text is logged for debugging). The invalid block is executed by the client's sync path, which is distinct coverage from the Engine simulators executing the same fixture via `engine_newPayload`. + +One class of fixture cannot travel this path and skips with an explicit reason: a payload whose declared block hash does not match its own header (corrupted at fill time via `rlp_modifier`), because devp2p cannot present a block whose hash differs from its header's keccak. + +A rejection target strictly below a reused client's head does not travel the wire either. Such a client refuses to walk its head backwards — geth's sync declines the announcement outright (`chain reorged, tail: 3, head: 3, newHead: 2`) and fetches the same header and body once per re-announcement without ever concluding — so the target's valid ancestry is handed over the Engine API instead, and its head is never named in a forkchoice update at all, because naming it only starts that unfinishable sync. The hand-over precedes the announcement: an idle client executes each ancestor against its parent's state, while a client already syncing towards the head answers for the ancestor without executing it and leaves the head permanently unjudgeable. `engine_newPayload` is idempotent, so the delivery is repeated at every re-announcement and a client that was busy on one attempt executes it on a later one. + +If the fixture declares an Engine API error code for the head payload, the client refusing `engine_newPayload` at the RPC layer is itself the expected rejection. + +## Test Ordering and Client Reuse + +Tests inside each pre-allocation group are ordered by default: valid chains before invalid ones, each by ascending chain length. This exists because a reused client's head number must never decrease (some clients stall permanently when asked to sync a chain shorter than one they already synced) and because serving a bad block can leave a client's sync machinery in a failure state that a following valid sync collides with. `--wirex-no-sort-by-chain-length` disables the ordering, which is useful for reproducing those stalls and for comparison runs. + +## The Sync Block and Chain Classes + +The fill gives every eligible `blockchain_test_engine_x` chain one framework-built empty block `S` (on by default at fill time; `--no-sync-block` disables it), placed by the chain's own statically declared structure. In the sequences below, `G` is genesis, `T₁…Tₙ` the test's own blocks, `*` marks the block this simulator announces, and `ᵢ` the intentionally invalid block: + +| Chain class | Sequence | Extra block | What is wire-guaranteed | +| ----------- | -------- | ----------- | ----------------------- | +| Valid (single or multi-block) | `G → T₁…Tₙ → S*` | appended, out-of-chain (`syncPayload`) | all of `T₁…Tₙ`, on every client | +| Invalid singleton (expected exception or Engine API error code) | `G → S → T₁*` | prepended, in-chain (`payloads[0]`, tagged `"phase": "sync"`) | `S` only; `T₁` is judged after `S` arrives | +| Invalid multi-block | `G → T₁…Tₙᵢ*` | none | `T₁…Tₙ₋₁` already travel the wire | +| Marked ineligible for its class's placement | `G → T₁*` | none | nothing — skipped below the block minimum | + +The appended `S` is scaffolding, not test content: `engineNewPayloads`, `lastblockhash` and the post state keep describing exactly the chain the test author wrote, and the sync completes when the client reports `S` as its head. The prepended `S` is load-bearing ancestry — without it the invalid singleton's block has a known parent and no sync ever starts — so it lives in-chain and every consumer replays it. + +An appended sync payload counts toward a fixture's chain length: a valid single-block test plus its trailer is a two-block chain, both for the skip accounting and for the chain-length ordering. Fixtures whose chain is still shorter than `--wirex-min-blocks` (default 2) are skipped. + +## Wire Coverage + +Reaching the head is necessary but not sufficient: the test also asserts that the blocks got there over devp2p. Every block below the announced head whose body cannot be derived from its header alone (an empty transactions trie and an empty withdrawals root leave nothing to download) must have had its body served by this peer, block by block — the failure names the blocks whose bodies never traveled. The announced head is exempt by protocol, and head-body service stays visible in the per-test transcript without being asserted, so a client changing its fetch shape shows up in logs rather than as a false failure. + +The serving evidence is cumulative per client, not per test. Valid chains carry no per-test salt, so two tests of one pre-allocation group may declare byte-identical chains; the reused client re-syncs nothing for the second one, because it already imported those blocks when the first one synced. A body that traveled this client's wire connection once satisfies the requirement for every later test of the group, and the run logs when a test was satisfied by earlier service. + +## Relationship to Other Simulators + +| | `consume rlp` | `consume sync` | `consume wirex` | +| ---------------------- | ----------------------------------- | --------------------------------------- | -------------------------------------------------- | +| Block delivery | RLP files imported at startup | devp2p, from a second client | devp2p, from a framework-controlled mock peer | +| Client code path | Client-specific offline import | Production sync path | Production sync path | +| Determinism | Deterministic | Depends on the serving client | Deterministic peer, transcripted | +| Clients per test | One per fixture | Two per fixture | One per pre-allocation group (reused) | +| Fork support | All forks | Post-Merge only | Post-Merge only | +| Invalid-block fixtures | Rejection implied by final head | Rejected via Engine API, never synced | Invalid block travels devp2p, rejection asserted | + +`consume sync` validates client-to-client interoperability on a handful of fixtures; WireX runs the whole test corpus on post-Merge forks against a single deterministic peer. `consume rlp` remains the only simulator covering pre-Merge forks. diff --git a/docs/running_tests/running.md b/docs/running_tests/running.md index 2515cb3677e..e94d0c25cb9 100644 --- a/docs/running_tests/running.md +++ b/docs/running_tests/running.md @@ -16,6 +16,7 @@ Both `consume` and `execute` provide sub-commands which correspond to different | [`consume engine`](#engine) | Client imports blocks via Engine API `EngineNewPayload` in Hive | EVM, block processing, Engine API | Staging, Hive | System test | | [`consume enginex`](#enginex) | Client imports blocks via Engine API in Hive, optimized by client reuse | EVM, block processing, Engine API, chain reorgs (implicit\*\*) | Staging, Hive | System test | | [`consume sync`](#sync) | Client syncs from another client using Engine API in Hive | EVM, block processing, Engine API, P2P sync | Staging, Hive | System test | +| [`consume wirex`](#wirex) | Client full syncs fixture blocks from a mock devp2p peer in Hive, with client reuse | EVM, block processing, Engine API (sync trigger), devp2p | Staging, Hive | System test | | [`consume rlp`](#rlp) | Client imports RLP-encoded blocks upon start-up in Hive | EVM, block processing, RLP import (sync\*) | Staging, Hive | System test | | [`build-block`](#block-building) | Client builds blocks via `testing_buildBlockV1` in Hive, validated against fixture | EVM, block production, Engine API (testing namespace) | Staging, Hive | System test | | [`execute hive`](./execute/hive.md) | Tests executed against a client via JSON RPC `eth_sendRawTransaction` in Hive | EVM, JSON RPC, mempool | Staging, Hive | System test | @@ -173,6 +174,33 @@ The `consume sync` command: 5. **Monitors sync progress** and validates that the sync client reaches the same state. 6. **Verifies final state** matches between both clients. +## WireX + +| Nomenclature | | +| -------------- | -------------------------- | +| Command | `consume wirex` | +| Simulator | `eels/consume-wirex` | +| Fixture format | `blockchain_test_engine_x` | + +The WireX method makes the client under test full sync each test's chain from a deterministic mock devp2p peer implemented inside the testing framework. The intent is to verify that clients can receive and propagate blocks over devp2p using the consensus test corpus; it is not intended to be a complete test of historical sync. WireX intends to replace `consume rlp` for post-Merge forks: the same workload moves from a client-specific offline import onto the client's production peer-to-peer block ingestion path, and EngineX-style client reuse amortizes the client startup cost that dominates `consume rlp` runs. + +The `consume wirex` command, for each pre-allocation group: + +1. **Initializes the execution client** with the group's shared genesis state. +2. **Connects a mock devp2p peer** to the client (RLPx and eth handshakes). +3. **Executes all tests in the group** against the same client. Each test: + + - Installs the test's chain on the peer and announces its block range. + - Names the sync target over the Engine API: one `engine_newPayload` for the announced head — the fixture's appended sync payload when it carries one, the chain's own head otherwise — then one `engine_forkchoiceUpdated`. + - Waits while the client downloads headers and bodies from the peer and executes every block through its full-sync path. + - Verifies the head via `eth_getBlockByNumber` and that every non-derivable body below the announced head was served over devp2p (evidence cumulative per reused client). + +4. **Stops the client** when all tests in the group complete. + +Engine X fixtures carry a per-class sync block from fill time (on by default): a fully valid chain `G → T₁…Tₙ → S*` gets an appended trailer `S`, which is the block WireX announces, so every test block is an ancestor the client must fetch over devp2p on every client; a single expected-invalid block `G → S → T₁*` gets `S` prepended in-chain to give the sync a reason to start. Fixtures whose chain ends in an intentionally invalid block run as rejection tests: the peer serves the chain as-is and the client passes by refusing it. + +See [Consume WireX](./consume/wirex.md) for the full flow, including a process diagram, the peer's behavior, rejection tests, and command options. + ## Block Building | Nomenclature | | diff --git a/packages/testing/pyproject.toml b/packages/testing/pyproject.toml index f8d980872ad..3807df98529 100644 --- a/packages/testing/pyproject.toml +++ b/packages/testing/pyproject.toml @@ -34,6 +34,8 @@ dependencies = [ "pytest-metadata>=3,<4", "pytest-xdist>=3.3.1,<4", "spec256k1>=0.2.3,<0.3", + "pycryptodome>=3.22,<4", + "cryptography>=45.0.1,<46", "trie>=3.1.0,<4", "semver>=3.0.1,<4", "pydantic>=2.12.3,<3", diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/consume.py b/packages/testing/src/execution_testing/cli/pytest_commands/consume.py index 3059f463c05..83166557cde 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/consume.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/consume.py @@ -51,9 +51,12 @@ def get_command_logic_test_paths(command_name: str) -> List[Path]: / "simulator_logic" / f"test_via_{test_command}.py" ] - elif command_name == "sync": + elif command_name in ["sync", "wirex"]: command_logic_test_paths = [ - base_path / "simulators" / "simulator_logic" / "test_via_sync.py" + base_path + / "simulators" + / "simulator_logic" + / f"test_via_{command_name}.py" ] elif command_name == "direct": command_logic_test_paths = [ @@ -132,6 +135,12 @@ def sync() -> None: pass +@consume_command(is_hive=True) +def wirex() -> None: + """Client full syncs fixture blocks from a mock devp2p peer.""" + pass + + @consume.command( context_settings={"ignore_unknown_options": True}, ) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/base.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/base.py index 23d6fbd5ad0..b0319bc72ac 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/base.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/base.py @@ -66,6 +66,7 @@ def check_live_port(test_suite_name: str) -> Literal[8545, 8551]: "eels/consume-engine", "eels/consume-enginex", "eels/consume-sync", + "eels/consume-wirex", "eels/build-block", }: return 8551 diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/enginex/conftest.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/enginex/conftest.py index f0275269e97..0f6ff5ca625 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/enginex/conftest.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/enginex/conftest.py @@ -7,29 +7,15 @@ pre-alloc group. """ -import io -import json import logging import time -from typing import TYPE_CHECKING, Generator, cast +from typing import Generator import pytest -from hive.client import Client, ClientType -from hive.testing import HiveTest from execution_testing.fixtures import BlockchainEngineXFixture -from execution_testing.fixtures.blockchain import FixtureHeader -from execution_testing.fixtures.pre_alloc_groups import PreAllocGroup -from ..helpers.test_tracker import ( - PreAllocGroupTestTracker, - enginex_group_counts_key, - make_group_identifier, -) - -if TYPE_CHECKING: - from ..multi_test_client import MultiTestClientManager - from ..timing_data import TimingData +from ..helpers.test_tracker import count_tests_per_group logger = logging.getLogger(__name__) @@ -68,24 +54,7 @@ def pytest_collection_modifyitems( if BlockchainEngineXFixture not in supported_formats: return - group_counts: dict[str, int] = {} - - for item in items: - for marker in item.iter_markers("xdist_group"): - if "name" in marker.kwargs: - group_identifier = marker.kwargs["name"] - break - else: - continue - group_counts[group_identifier] = ( - group_counts.get(group_identifier, 0) + 1 - ) - - session.stash[enginex_group_counts_key] = group_counts - logger.info( - f"Counted {len(group_counts)} pre-alloc groups with " - f"{sum(group_counts.values())} total tests" - ) + group_counts = count_tests_per_group(session, items) def sort_key(item: pytest.Item) -> tuple[int, str]: """Return sort key: largest group first, then by group id.""" @@ -146,15 +115,6 @@ def pytest_runtest_protocol( _GroupDispatchTracker.last_protocol_end = time.perf_counter() -@pytest.fixture(scope="session", autouse=True) -def _configure_client_manager( - multi_test_client_manager: "MultiTestClientManager", - pre_alloc_group_test_tracker: PreAllocGroupTestTracker, -) -> None: - """Wire the test tracker to the client manager at session start.""" - multi_test_client_manager.set_test_tracker(pre_alloc_group_test_tracker) - - @pytest.fixture(scope="module") def test_suite_name() -> str: """The name of the hive test suite used in this simulator.""" @@ -168,109 +128,3 @@ def test_suite_description() -> str: "Execute blockchain tests against clients using the Engine API with " "pre-allocation group optimization using Engine X fixtures." ) - - -@pytest.fixture(scope="function", autouse=True) -def _per_test_reporting( - client: Client, - hive_test: HiveTest, -) -> None: - """ - Register a test for execution against a multi-test client. - - Activate log segment capturing in the Hive backend for correct - client log reporting in the multi-test client case. - - Parameter order matters: `client` listed before `hive_test` - ensures pytest sets up `client` first and tears it down last. - This guarantees `hive_test` teardown (`test.end()`) runs while - the hive node still exists, before `client` teardown calls - `mark_test_completed` / `client.stop()`. - """ - hive_test.register_multi_test_client(client) - - -@pytest.fixture(scope="function") -def client( - multi_test_hive_test: HiveTest, - multi_test_client_manager: "MultiTestClientManager", - fixture: BlockchainEngineXFixture, - client_type: ClientType, - environment: dict, - client_genesis: dict, - total_timing_data: "TimingData", - request: pytest.FixtureRequest, -) -> Generator[Client, None, None]: - """ - Get or create a multi-test client for this pre-allocation group. - - Called for each test, but reuses clients across tests that - share the same pre-allocation group. - """ - group_identifier = make_group_identifier( - fixture.pre_hash, client_type.name - ) - test_id = request.node.nodeid - - resolved_client = multi_test_client_manager.get_client(group_identifier) - if resolved_client is not None: - logger.info(f"♻️ Reusing client for group {group_identifier}") - else: - # Start new client; calculate genesis - serialize_start = time.perf_counter() - genesis_bytes = json.dumps(client_genesis).encode("utf-8") - buffered_genesis = io.BufferedReader( - cast(io.RawIOBase, io.BytesIO(genesis_bytes)) - ) - logger.info( - f"⏱ phase=genesis_serialize group={group_identifier} " - f"ms={(time.perf_counter() - serialize_start) * 1000:.1f}" - ) - - logger.info( - f"🚀 Starting client ({client_type.name}) " - f"for group {group_identifier}" - ) - - start_requested = time.perf_counter() - with total_timing_data.time("Start client"): - resolved_client = multi_test_hive_test.start_client( - client_type=client_type, - environment=environment, - files={"/genesis.json": buffered_genesis}, - ) - - assert resolved_client is not None, ( - f"Unable to connect to client ({client_type.name}) via " - "Hive. Check the client or Hive server logs for more " - "information." - ) - - # The hive start-client API only returns once the client answers - # its liveness check, so this duration spans container creation, - # client boot and the check-live wait. - logger.info( - f"⏱ phase=client_start group={group_identifier} " - f"ms={(time.perf_counter() - start_requested) * 1000:.1f}" - ) - logger.info( - f"Client ({client_type.name}) ready for group {group_identifier}" - ) - - multi_test_client_manager.register_client( - group_identifier, resolved_client - ) - resolved_client.multi_test = True - - try: - yield resolved_client - finally: - multi_test_client_manager.mark_test_completed( - group_identifier, test_id - ) - - -@pytest.fixture(scope="function") -def genesis_header(pre_alloc_group: PreAllocGroup) -> FixtureHeader: - """Provide the genesis header from the pre-allocation group.""" - return pre_alloc_group.genesis diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/helpers/test_tracker.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/helpers/test_tracker.py index 21c4a35d22e..82031813ca6 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/helpers/test_tracker.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/helpers/test_tracker.py @@ -11,6 +11,35 @@ enginex_group_counts_key: StashKey[dict[str, int]] = StashKey() +def count_tests_per_group( + session: pytest.Session, items: list[pytest.Item] +) -> dict[str, int]: + """ + Count the collected tests of each pre-allocation group and stash + the counts on the session. + + The xdist_group markers are set during parametrization. The counts + feed the largest-group-first sort of the simulators' collection + hooks and, via the session stash, the test tracker's client + teardown accounting. + """ + group_counts: dict[str, int] = {} + for item in items: + for marker in item.iter_markers("xdist_group"): + if "name" in marker.kwargs: + group_counts[marker.kwargs["name"]] = ( + group_counts.get(marker.kwargs["name"], 0) + 1 + ) + break + + session.stash[enginex_group_counts_key] = group_counts + logger.info( + f"Counted {len(group_counts)} pre-alloc groups with " + f"{sum(group_counts.values())} total tests" + ) + return group_counts + + def make_group_identifier(pre_hash: str, client_name: str) -> str: """Build xdist group key from pre-alloc hash and client name.""" return f"{pre_hash}-{client_name}" diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/multi_test_client.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/multi_test_client.py index c28bb60e085..641aa73cd00 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/multi_test_client.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/multi_test_client.py @@ -1,19 +1,29 @@ """Pytest fixtures for multi-test client architecture.""" +import io +import json import logging import time -from typing import Generator +from typing import TYPE_CHECKING, Generator, cast import pytest -from hive.client import Client +from hive.client import Client, ClientType +from hive.testing import HiveTest from execution_testing.base_types import to_json from execution_testing.fixtures import BlockchainEngineXFixture +from execution_testing.fixtures.blockchain import FixtureHeader from execution_testing.fixtures.pre_alloc_groups import PreAllocGroup from ..consume import FixturesSource from .helpers.ruleset import ruleset -from .helpers.test_tracker import PreAllocGroupTestTracker +from .helpers.test_tracker import ( + PreAllocGroupTestTracker, + make_group_identifier, +) + +if TYPE_CHECKING: + from .timing_data import TimingData logger = logging.getLogger(__name__) @@ -276,3 +286,105 @@ def environment( environment_cache[pre_hash] = env return env + + +@pytest.fixture(scope="session", autouse=True) +def _configure_client_manager( + multi_test_client_manager: MultiTestClientManager, + pre_alloc_group_test_tracker: PreAllocGroupTestTracker, +) -> None: + """Wire the test tracker to the client manager at session start.""" + multi_test_client_manager.set_test_tracker(pre_alloc_group_test_tracker) + + +@pytest.fixture(scope="function", autouse=True) +def _per_test_reporting( + client: Client, + hive_test: HiveTest, +) -> None: + """ + Register a test for execution against a multi-test client. + + Activate log segment capturing in the Hive backend for correct + client log reporting in the multi-test client case. + + Parameter order matters: `client` listed before `hive_test` + ensures pytest sets up `client` first and tears it down last. + This guarantees `hive_test` teardown (`test.end()`) runs while + the hive node still exists, before `client` teardown calls + `mark_test_completed` / `client.stop()`. + """ + hive_test.register_multi_test_client(client) + + +@pytest.fixture(scope="function") +def client( + multi_test_hive_test: HiveTest, + multi_test_client_manager: MultiTestClientManager, + fixture: BlockchainEngineXFixture, + client_type: ClientType, + environment: dict, + client_genesis: dict, + total_timing_data: "TimingData", + request: pytest.FixtureRequest, +) -> Generator[Client, None, None]: + """ + Get or create a multi-test client for this pre-allocation group. + + Called for each test, but reuses clients across tests that + share the same pre-allocation group. + """ + group_identifier = make_group_identifier( + fixture.pre_hash, client_type.name + ) + test_id = request.node.nodeid + + resolved_client = multi_test_client_manager.get_client(group_identifier) + if resolved_client is not None: + logger.info(f"♻️ Reusing client for group {group_identifier}") + else: + # Start new client; calculate genesis + genesis_bytes = json.dumps(client_genesis).encode("utf-8") + buffered_genesis = io.BufferedReader( + cast(io.RawIOBase, io.BytesIO(genesis_bytes)) + ) + + logger.info( + f"🚀 Starting client ({client_type.name}) " + f"for group {group_identifier}" + ) + + with total_timing_data.time("Start client"): + resolved_client = multi_test_hive_test.start_client( + client_type=client_type, + environment=environment, + files={"/genesis.json": buffered_genesis}, + ) + + assert resolved_client is not None, ( + f"Unable to connect to client ({client_type.name}) via " + "Hive. Check the client or Hive server logs for more " + "information." + ) + + logger.info( + f"Client ({client_type.name}) ready for group {group_identifier}" + ) + + multi_test_client_manager.register_client( + group_identifier, resolved_client + ) + resolved_client.multi_test = True + + try: + yield resolved_client + finally: + multi_test_client_manager.mark_test_completed( + group_identifier, test_id + ) + + +@pytest.fixture(scope="function") +def genesis_header(pre_alloc_group: PreAllocGroup) -> FixtureHeader: + """Provide the genesis header from the pre-allocation group.""" + return pre_alloc_group.genesis diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/simulator_logic/test_via_wirex.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/simulator_logic/test_via_wirex.py new file mode 100644 index 00000000000..0cb4239eb74 --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/simulator_logic/test_via_wirex.py @@ -0,0 +1,566 @@ +""" +A hive based simulator that makes clients full sync fixture blocks. + +Where `consume rlp` hands a client its blocks through a client specific +offline import mode, and `consume engine` hands them over one +`engine_newPayload` call at a time, this simulator makes the client +fetch them itself, from a mock peer speaking the production devp2p +protocols. + +The control plane and the data plane are deliberately separate: + +- The control plane is the Engine API. A post-merge client does not + choose its own head, so it is told which block to sync to. That takes + one `engine_newPayload` for the head block, which gives the client the + header, and one `engine_forkchoiceUpdated` naming that head. +- The data plane is devp2p. Every block before the head is downloaded + from the mock peer over RLPx and executed by the client's full sync + path. + +Because only the blocks before the announced head are guaranteed to +travel over devp2p - whether a client also re-fetches the head's body +from a peer is an implementation choice, and measured clients go both +ways - the filler gives every eligible engine_x chain one extra empty +block, placed by the chain's own structure. A fully valid chain gets +it *appended*, out-of-chain in the fixture's `syncPayload` field: this +simulator announces that trailer instead of the test's own head, which +makes every one of the test's blocks an ancestor whose header and body +a full-syncing client must fetch from the peer, on every client, by +chain structure rather than client courtesy. A single expected-invalid +block gets the extra block *prepended* in-chain instead (tagged with +the `sync` phase), giving the sync a reason to start below the block +the client is expected to refuse. Chains still too short to put any +block on the wire - single-block fixtures the extra block cannot +survive - are skipped here, where the limitation actually lives. +""" + +import time + +from hive.client import Client + +from execution_testing.devp2p.chain import Block, Chain +from execution_testing.devp2p.peer import MockPeer +from execution_testing.fixtures import BlockchainEngineXFixture +from execution_testing.fixtures.blockchain import ( + FixtureEngineNewPayload, + FixtureHeader, +) +from execution_testing.logging import get_logger +from execution_testing.rpc import ( + EngineRPC, + EthRPC, + ForkchoiceUpdateTimeoutError, +) +from execution_testing.rpc.rpc_types import ( + ForkchoiceState, + JSONRPCError, + PayloadStatus, + PayloadStatusEnum, +) + +from ..helpers.exceptions import ( + GenesisBlockMismatchExceptionError, + LoggedError, +) +from ..helpers.timing import TimingData + +logger = get_logger(__name__) + +ACCEPTANCE_HOLD_TIME = 0.5 +""" +Seconds a VALID verdict on an invalid head must hold to be believed. + +A client answers `engine_newPayload` from the database state of that +instant, so while a chain is still arriving its answer can be a +transient artifact rather than a judgement: geth has been observed +answering VALID for a block its own backfill rejected fifteen +milliseconds later. Only a verdict that outlives its sync fails the +test, which costs this wait once per genuinely accepted chain. +""" + + +def announced_payload( + fixture: BlockchainEngineXFixture, +) -> FixtureEngineNewPayload: + """ + Return the payload this simulator announces as the sync target. + + The appended sync payload when the fixture carries one - the + trailer exists precisely to be announced, so that every payload of + the test's own chain is an ancestor the client must fetch from the + peer - and the chain's own head otherwise (prepend-class fixtures + must announce the test's block: their assertion is the client's + judgement of it). + """ + return fixture.sync_payload or fixture.payloads[-1] + + +def required_wire_bodies(chain: Chain) -> list[Block]: + """ + Return the blocks whose bodies must have traveled the wire. + + Every block below the announced head, except those whose body a + client may derive from the header alone (empty transactions trie, + empty withdrawals root). The head itself is exempt by protocol: + its payload arrives through the Engine API, and whether a client + also re-fetches its body from a peer is an implementation choice + that measured clients answer both ways. + """ + return [ + block + for block in chain.blocks[:-1] + if block.transactions or block.withdrawals + ] + + +def test_blockchain_via_wirex( + timing_data: TimingData, + eth_rpc: EthRPC, + engine_rpc: EngineRPC, + client: Client, + genesis_verified_clients: set[str], + fixture: BlockchainEngineXFixture, + genesis_header: FixtureHeader, + chain: Chain, + mock_peer: MockPeer, + wirex_sync_timeout: float, + wirex_poll_interval: float, + wirex_announce_interval: float, +) -> None: + """ + Make a client full sync one test's chain from the mock peer. + + The sequence is: + + 1. Verify the client's genesis matches the group's, once per client. + 2. Deliver the announced head over the Engine API so the client + knows which chain to sync to, and name it in a forkchoice + update. For an appended-class fixture that head is the sync + trailer riding above the test's own chain, so every block the + test author wrote is an ancestor the client must fetch from the + peer. + 3. Wait for the client to download and execute the ancestors from the + mock peer, polling the same forkchoice update until it is VALID. + Each announcement's `newPayload` answer is read as the client's + verdict: an INVALID means the client has executed the ancestry + and refused the chain, so the test fails immediately with the + client's reason instead of waiting out the sync timeout. + 4. Check the client's head really is the expected block. + + There is deliberately no rewind between tests. Every test's chain + forks at genesis, so announcing the new head is all a consensus + client would do, and a backwards forkchoice update is actively + harmful to clients that act on it: nethermind moves its head back + to genesis while its persisted state stays at the previous chain's + tip, which lands it in a crash-recovery edge case where it fetches + receipts instead of executing blocks (`BlockDownloader. + ReceiptEdgeCase`); geth ignores the rewind entirely. + + Fixtures containing an intentionally invalid block are rejection + tests: the peer serves the chain as-is and the client passes by + refusing it - `engine_newPayload` for the head must answer INVALID + once the ancestry is available over devp2p, and a VALID that holds + fails the test. Only the fact of rejection is asserted: a devp2p + peer observes acceptance or rejection, not error causes, so + matching the fixture's specific exception over the wire is + deliberately left for later. Fixtures whose invalid block cannot + even be represented on the wire (declared hash inconsistent with + the header) are skipped by the `chain` fixture. When the rejection + target sits below the reused client's head, the valid ancestry is + delivered over the Engine API instead of the wire, before the head + is announced and again at every re-announcement - the client's + refusal to walk its head backwards would otherwise starve the + verdict (see the comment at the ancestry block). + """ + head_payload = announced_payload(fixture) + head_hash = head_payload.params[0].block_hash + + if client.id not in genesis_verified_clients: + with timing_data.time("Verify genesis"): + genesis_block = eth_rpc.get_block_by_number(0) + if genesis_block is None: + raise LoggedError("Client returned no genesis block") + if genesis_block["hash"] != str(genesis_header.block_hash): + raise GenesisBlockMismatchExceptionError( + expected_header=genesis_header, + got_genesis_block=genesis_block, + ) + genesis_verified_clients.add(client.id) + + expected_head = "0x" + chain.head.block_hash.hex() + + head_state = ForkchoiceState( + head_block_hash=head_hash, + safe_block_hash=genesis_header.block_hash, + finalized_block_hash=genesis_header.block_hash, + ) + + # Whether the head sits below the reused client's own head, decided + # for rejection targets below and read by `announce`. + below_client_head = False + + def announce() -> PayloadStatus: + """ + Tell the client which block to sync to. + + The `newPayload` response is returned because it carries the + client's verdict on the head: SYNCING while the ancestry is + still traveling, VALID once imported, and INVALID as soon as + the client has executed the ancestry and refused the chain. + + No forkchoice update is sent for a head that sits below the + client's own head. Naming such a head only starts a sync the + client cannot finish, because its sync machinery refuses to + walk its head backwards, and that unfinishable sync is what + stops the ancestry from taking effect (see the rejection + block, which hands that ancestry over separately). + """ + payload_status = engine_rpc.new_payload( + *head_payload.params, version=head_payload.new_payload_version + ) + if not below_client_head: + engine_rpc.forkchoice_updated( + forkchoice_state=head_state, + payload_attributes=None, + version=head_payload.forkchoice_updated_version, + ) + return payload_status + + def deliver_ancestry() -> None: + """ + Hand a rejection target's valid ancestors to the client over + the Engine API. + + Only the head of a rejection chain is expected to be refused, + so every ancestor must come back VALID. A client that is busy + syncing answers for the payload without executing it, leaving + the head unjudgeable; the delivery is idempotent, so anything + but VALID is logged and retried by the caller once the client + has had time to settle. + """ + for ancestor_payload in fixture.payloads[:-1]: + ancestor_status = engine_rpc.new_payload( + *ancestor_payload.params, + version=ancestor_payload.new_payload_version, + ) + if ancestor_status.status != PayloadStatusEnum.VALID: + logger.warning( + f"Client answered {ancestor_status.status} for the " + f"valid ancestor " + f"{ancestor_payload.params[0].block_hash} instead of " + "executing it; the delivery will be repeated" + ) + + def expected_rpc_refusal(error: JSONRPCError) -> bool: + """ + Return whether `error` is the rejection the fixture declares. + + A head payload that violates the Engine API's rules for the + fork (e.g. a pre-fork block carrying blob fields) is refused + at the RPC layer before any chain context matters. When the + fixture declares that error code, the refusal is the expected + rejection - but only with the declared code, so a client + failing for an unrelated reason still fails the test. + """ + if head_payload.error_code is None: + return False + if error.code != head_payload.error_code: + raise LoggedError( + f"Client refused the head with the wrong error code: " + f"got {error.code}, expected {head_payload.error_code}" + ) + logger.info( + f"Client refused the invalid head at the RPC layer with " + f"the expected error code {head_payload.error_code} " + f"({error})" + ) + return True + + # A declared Engine API error code is itself a rejection: the + # client must refuse the head, whether over the RPC layer or with + # an INVALID verdict, even when every payload is semantically + # valid. + expect_rejection = ( + any(not payload.valid() for payload in fixture.payloads) + or head_payload.error_code is not None + ) + + if expect_rejection: + with timing_data.time("Prepare rejection"): + # A rejection target below the reused client's head cannot + # reach it over devp2p: the sync machinery of geth-like + # clients refuses to walk its head backwards, so the + # ancestry never arrives and the head stays unjudgeable. + # Equal-height targets sync fine (the reused-client common + # case) and stay on the wire; only a strictly-below target + # has its valid ancestry handed over the Engine API. + # + # That hand-over happens here, before the head is + # announced, because an idle client executes each ancestor + # against its parent's state, while a client already + # syncing towards the head answers for the ancestor + # without executing it and leaves the head unjudgeable for + # good. + client_head_block = eth_rpc.get_block_by_number("latest") + client_head_number = ( + int(client_head_block["number"], 16) + if client_head_block + else 0 + ) + below_client_head = chain.head.number < client_head_number + if below_client_head: + logger.info( + f"Rejection target {chain.head.number} is below the " + f"client head {client_head_number}; delivering the " + f"{len(fixture.payloads) - 1} valid ancestor(s) over " + "the Engine API instead of the wire" + ) + deliver_ancestry() + + announce_status: PayloadStatus | None = None + with timing_data.time("Announce sync target"): + logger.info( + f"Announcing head block {chain.head.number} to trigger a sync " + f"of {len(chain.blocks) - 1} ancestor block(s) over devp2p" + ) + try: + announce_status = announce() + except JSONRPCError as error: + if expected_rpc_refusal(error): + return + raise + + if expect_rejection: + with timing_data.time("Reject invalid chain"): + deadline = time.monotonic() + wirex_sync_timeout + next_announcement = time.monotonic() + wirex_announce_interval + status: PayloadStatusEnum | None = None + validation_error: object = None + accepted_since: float | None = None + while time.monotonic() < deadline: + # Once the ancestry has arrived over devp2p the client + # can judge the head; until then it answers SYNCING (or + # ACCEPTED if it merely stored the payload). + try: + payload_status = engine_rpc.new_payload( + *head_payload.params, + version=head_payload.new_payload_version, + ) + except JSONRPCError as error: + if expected_rpc_refusal(error): + return + raise + status = payload_status.status + validation_error = payload_status.validation_error + if status in ( + PayloadStatusEnum.INVALID, + PayloadStatusEnum.INVALID_BLOCK_HASH, + ): + break + if status != PayloadStatusEnum.VALID: + accepted_since = None + elif accepted_since is None: + # A client whose sync is still in flight answers + # about the database state of that instant, and a + # lone VALID is not proof that it accepted the + # chain: geth has been observed answering a + # well-formed VALID for a block its own backfill + # rejected milliseconds later, then INVALID on + # every ask thereafter. A real acceptance holds, so + # the verdict is read only once it has. + accepted_since = time.monotonic() + logger.warning( + f"Client answered VALID for the invalid head " + f"{expected_head} (latestValidHash " + f"{payload_status.latest_valid_hash}) while the " + "chain may still be arriving; confirming before " + "failing the test" + ) + elif time.monotonic() - accepted_since >= ACCEPTANCE_HOLD_TIME: + raise LoggedError( + f"Client accepted the invalid chain: head " + f"{expected_head} returned VALID for " + f"{ACCEPTANCE_HOLD_TIME}s (latestValidHash " + f"{payload_status.latest_valid_hash}) but the " + "fixture expects the block to be rejected" + ) + if time.monotonic() >= next_announcement: + if not mock_peer.alive: + # A client may drop a peer that served it a bad + # chain; a real peer would simply redial. + logger.warning("Peer dropped mid-rejection; redialing") + mock_peer.reconnect(chain) + if below_client_head: + # A whole announcement interval has passed, so a + # delivery the client answered without executing + # takes effect on this attempt. It goes first so + # that the re-announcement's own answer already + # reflects it. + deliver_ancestry() + logger.info("Re-announcing the invalid sync target") + try: + announce() + except JSONRPCError as error: + if expected_rpc_refusal(error): + return + raise + next_announcement = ( + time.monotonic() + wirex_announce_interval + ) + time.sleep(wirex_poll_interval) + if status not in ( + PayloadStatusEnum.INVALID, + PayloadStatusEnum.INVALID_BLOCK_HASH, + ): + raise LoggedError( + f"Client never rejected the invalid head " + f"{expected_head} within {wirex_sync_timeout}s (last " + f"status: {status}). Peer transcript: " + f"{mock_peer.statistics.transcript}" + ) + statistics = mock_peer.statistics + logger.info( + f"Client rejected the invalid head at block " + f"{chain.head.number} with {status} " + f"(validationError: {validation_error}) after the peer " + f"served {statistics.headers_served} header(s) and " + f"{statistics.bodies_served} body/bodies" + ) + return + + def raise_if_rejected(payload_status: PayloadStatus | None) -> None: + """ + Fail immediately when the client has refused the chain. + + An INVALID verdict for the head means the client has already + downloaded and executed the ancestry and decided against it - + waiting out the sync timeout cannot import the chain, and the + verdict carries the client's reason while a timeout carries + nothing. Reading the verdict from the announcement that was + sent anyway costs no extra requests. + """ + if payload_status is not None and payload_status.status in ( + PayloadStatusEnum.INVALID, + PayloadStatusEnum.INVALID_BLOCK_HASH, + ): + raise LoggedError( + f"Client rejected the chain at head {expected_head}: " + f"{payload_status.status} (validationError: " + f"{payload_status.validation_error}). Peer transcript: " + f"{mock_peer.statistics.transcript}" + ) + + with timing_data.time("Sync from peer"): + # Wait by watching for the block rather than by repeating the + # forkchoice update. A repeated update restarts the client's sync + # cycle, and repeating it faster than a cycle takes prevents the + # sync from ever finishing. The announcement is repeated on a much + # slower cadence, as a consensus client would each slot, because a + # client whose sync state was still settling may have ignored the + # first one. + raise_if_rejected(announce_status) + deadline = time.monotonic() + wirex_sync_timeout + next_announcement = time.monotonic() + wirex_announce_interval + synced = False + while time.monotonic() < deadline: + if eth_rpc.get_block_by_hash(head_hash, full_txs=False): + synced = True + break + if time.monotonic() >= next_announcement: + if not mock_peer.alive: + # A mid-sync drop would otherwise strand the test + # peerless until its timeout; redial like a real + # peer would. + logger.warning("Peer dropped mid-sync; redialing") + mock_peer.reconnect(chain) + logger.info("Re-announcing the sync target") + raise_if_rejected(announce()) + next_announcement = time.monotonic() + wirex_announce_interval + time.sleep(wirex_poll_interval) + if not synced: + raise LoggedError( + f"Client never imported the fixture head {expected_head} " + f"within {wirex_sync_timeout}s. Peer transcript: " + f"{mock_peer.statistics.transcript}" + ) + + with timing_data.time("Confirm head"): + try: + response = engine_rpc.forkchoice_updated_with_retry( + forkchoice_state=head_state, + forkchoice_version=head_payload.forkchoice_updated_version, + max_attempts=10, + wait_fixed=0.5, + ) + except ForkchoiceUpdateTimeoutError as error: + raise LoggedError( + f"Client imported {expected_head} but never made it " + f"canonical: {error}" + ) from None + if response.payload_status.status != PayloadStatusEnum.VALID: + raise LoggedError( + f"Client failed to sync to {expected_head}: " + f"{response.payload_status.status}. Peer transcript: " + f"{mock_peer.statistics.transcript}" + ) + + with timing_data.time("Verify head"): + head_block = eth_rpc.get_block_by_number("latest") + if head_block is None: + raise LoggedError("Client returned no head block") + if head_block["hash"] != expected_head: + raise LoggedError( + f"Client head is {head_block['hash']}, expected " + f"{expected_head}" + ) + + statistics = mock_peer.statistics + logger.info( + f"Synced to block {chain.head.number}: peer served " + f"{statistics.headers_served} header(s) in " + f"{statistics.header_requests} request(s) and " + f"{statistics.bodies_served} body/bodies in " + f"{statistics.body_requests} request(s)" + ) + # Every non-derivable body below the announced head must have + # traveled the wire, block by block: an aggregate count would let + # one downloaded body vouch for a chain whose other bodies arrived + # some other way. The evidence is cumulative per client, not per + # test: valid chains carry no per-test salt, so two tests of one + # group may declare byte-identical chains, and the reused client + # re-syncs nothing for the second - its blocks already traveled + # the wire during the first. + required = required_wire_bodies(chain) + ever_served = mock_peer.body_hashes_ever_served + missing_bodies = [ + block.number + for block in required + if block.block_hash not in ever_served + ] + prior_served = sum( + 1 + for block in required + if block.block_hash not in statistics.body_hashes_served + and block.block_hash in ever_served + ) + if prior_served: + logger.info( + f"{prior_served} of {len(required)} required body/bodies " + "already traveled the wire during an earlier test of this " + "client (byte-identical chain content); the wire-coverage " + "evidence is cumulative per client" + ) + if missing_bodies: + raise LoggedError( + "The client reached the expected head, but the non-empty " + "body/bodies of block(s) " + f"{', '.join(str(n) for n in missing_bodies)} never " + "traveled this client's wire connection, so those blocks " + "were not verified over devp2p." + ) + if statistics.receipt_requests: + logger.warning( + f"Client made {statistics.receipt_requests} receipt request(s), " + "which this peer does not serve; the client may not be " + "executing the blocks it downloads." + ) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/wirex/__init__.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/wirex/__init__.py new file mode 100644 index 00000000000..8b365f972f0 --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/wirex/__init__.py @@ -0,0 +1 @@ +"""Pytest plugin for the `consume wirex` simulator.""" diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/wirex/conftest.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/wirex/conftest.py new file mode 100644 index 00000000000..76ca2a9891e --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/wirex/conftest.py @@ -0,0 +1,433 @@ +""" +Pytest fixtures for the `consume wirex` simulator. + +The client topology is the one `consume enginex` established: one client +per pre-allocation group, reused by every test in that group. What +differs is how a test's blocks reach the client: instead of being handed +over one `engine_newPayload` call at a time, they are downloaded by the +client from a mock devp2p peer, so the blocks travel the client's +production full sync path. There is no rewind between tests - every +chain forks at the group's genesis, and the new head is announced. + +The peer is created once per client and re-pointed at each test's chain, +which keeps the RLPx handshake out of the per-test cost. +""" + +import json +import logging +import os +import time +from typing import TYPE_CHECKING, Dict, Generator + +import pytest +from hive.client import Client + +from execution_testing.devp2p.chain import ( + Chain, + ChainReconstructionError, + chain_from_payloads, +) +from execution_testing.devp2p.peer import MockPeer +from execution_testing.devp2p.protocol import ETH_PROTOCOLS +from execution_testing.fixtures import BlockchainEngineXFixture +from execution_testing.fixtures.blockchain import ( + FixtureEngineNewPayload, + FixtureHeader, +) + +from ..helpers.test_tracker import count_tests_per_group + +if TYPE_CHECKING: + from ..timing_data import TimingData + +logger = logging.getLogger(__name__) + +pytest_plugins = ( + "execution_testing.cli.pytest_commands.plugins.pytest_hive.pytest_hive", + "execution_testing.cli.pytest_commands.plugins.consume.simulators.base", + "execution_testing.cli.pytest_commands.plugins.consume.simulators.multi_test_client", + "execution_testing.cli.pytest_commands.plugins.consume.simulators.test_case_description", + "execution_testing.cli.pytest_commands.plugins.consume.simulators.timing_data", + "execution_testing.cli.pytest_commands.plugins.consume.simulators.exceptions", + "execution_testing.cli.pytest_commands.plugins.consume.simulators.helpers.test_tracker", + "execution_testing.cli.pytest_commands.plugins.consume.simulators.engine_api", +) + +DEFAULT_NETWORK_ID = 1 +"""Network identifier the client is started with, and the peer claims.""" + + +def pytest_addoption(parser: pytest.Parser) -> None: + """Add the wirex specific command line options.""" + group = parser.getgroup("wirex", "Arguments for the wirex simulator") + group.addoption( + "--wirex-min-blocks", + action="store", + dest="wirex_min_blocks", + type=int, + default=2, + help=( + "Skip fixtures whose chain is shorter than this. A fixture's " + "announced head is delivered over the Engine API to name the " + "sync target, so only the blocks before it travel over " + "devp2p; at the default of 2 every executed test syncs at " + "least one block from the peer. An appended sync payload " + "counts toward the length: a valid single-block test plus " + "its trailer is a two-block chain." + ), + ) + group.addoption( + "--wirex-no-sort-by-chain-length", + action="store_false", + dest="wirex_sort_by_chain_length", + default=True, + help=( + "Do not order the tests inside each pre-allocation group " + "(valid chains before invalid ones, each by ascending " + "chain length). The ordering is on by default because a " + "reused client's head number must never decrease and no " + "valid sync should follow a served bad block: geth's " + "beacon sync has been observed to stall when asked to " + "sync a chain shorter than one the same client already " + "synced, and to back off after syncing a chain with a bad " + "block in a way that starves the next sync. Disabling is " + "for reproducing those stalls and for comparing against " + "unordered runs." + ), + ) + group.addoption( + "--wirex-eth-version", + action="store", + dest="wirex_eth_version", + choices=["auto"] + [str(v) for v in sorted(ETH_PROTOCOLS)], + default="auto", + help=( + "eth protocol version the mock peer advertises. 'auto' " + "(default) advertises every implemented version and lets " + "RLPx negotiation pick the highest the client shares; an " + "explicit version advertises exactly that one, so the " + "client either speaks it or the handshake fails loudly. " + "The negotiated version is recorded in every test's peer " + "transcript." + ), + ) + group.addoption( + "--wirex-sync-timeout", + action="store", + dest="wirex_sync_timeout", + type=float, + default=60.0, + help="Seconds to wait for a client to reach the fixture head.", + ) + group.addoption( + "--wirex-announce-interval", + action="store", + dest="wirex_announce_interval", + type=float, + default=3.0, + help=( + "Seconds between repeats of the sync target announcement " + "while waiting for a client to reach it." + ), + ) + group.addoption( + "--wirex-poll-interval", + action="store", + dest="wirex_poll_interval", + type=float, + default=0.005, + help=( + "Seconds between result polls while waiting for a verdict: " + "eth_getBlockByHash during a valid sync, the head's " + "newPayload for a rejection. Short chains sync in tens of " + "milliseconds, so a long interval measures the poller " + "rather than the client." + ), + ) + + +def pytest_configure(config: pytest.Config) -> None: + """Set the supported fixture formats for the wirex simulator.""" + config.supported_fixture_formats = [BlockchainEngineXFixture] # type: ignore[attr-defined] + + +def _chain_properties( + config: pytest.Config, items: list[pytest.Item] +) -> Dict[str, tuple[bool, int]]: + """ + Read each collected test case's chain properties for ordering: does + the chain contain an invalid payload, and how long is it. + + An appended sync payload counts toward the length: it is a real + block of the served chain, and the ordering must agree with the + skip accounting, which counts it too. The fixture index records + neither property, so the fixture files are read directly, one at a + time: each file is parsed once, mined for all of its collected + test cases, and dropped before the next is opened, so the peak + footprint is one parsed file rather than the whole corpus. + """ + properties: Dict[str, tuple[bool, int]] = {} + cases_by_path: Dict[str, list[tuple[str, str]]] = {} + source_path = getattr(config, "fixtures_source", None) + for item in items: + callspec = getattr(item, "callspec", None) + if callspec is None: + continue + test_case = callspec.params.get("test_case") + fixture = getattr(test_case, "fixture", None) + if fixture is not None: # stdin: the fixture is already loaded + properties[item.nodeid] = ( + any(not payload.valid() for payload in fixture.payloads), + len(sync_chain_payloads(fixture)), + ) + continue + json_path = getattr(test_case, "json_path", None) + if json_path is None or source_path is None: + continue + path = str(source_path.path / json_path) + cases_by_path.setdefault(path, []).append((item.nodeid, test_case.id)) + for path, cases in cases_by_path.items(): + with open(path) as file: + raw = json.load(file) + for nodeid, case_id in cases: + raw_fixture = raw.get(case_id) + if raw_fixture is None: + continue + payloads = raw_fixture.get("engineNewPayloads", []) + properties[nodeid] = ( + any( + payload.get("validationError") is not None + for payload in payloads + ), + len(payloads) + (1 if raw_fixture.get("syncPayload") else 0), + ) + return properties + + +@pytest.hookimpl(trylast=True) +def pytest_collection_modifyitems( + session: pytest.Session, config: pytest.Config, items: list[pytest.Item] +) -> None: + """Count tests per pre-allocation group and sort largest first.""" + supported_formats = getattr(config, "supported_fixture_formats", []) + if BlockchainEngineXFixture not in supported_formats: + return + + group_counts = count_tests_per_group(session, items) + + chain_properties: Dict[str, tuple[bool, int]] = {} + if config.getoption("wirex_sort_by_chain_length", True): + chain_properties = _chain_properties(config, items) + logger.info( + "Ordering tests inside each pre-allocation group: valid " + "chains before invalid ones, each by ascending chain length" + ) + + def sort_key(item: pytest.Item) -> tuple[int, str, bool, int, str]: + """ + Return sort key: largest group first, then by group id, then + (when enabled) valid chains before invalid ones, each by + ascending chain length inside the group. + + Invalid chains run last because serving a chain with a bad + block leaves a client's sync machinery in a failure state that + a following valid sync on the same client collides with + (observed on geth as a backfill backoff whose delayed header + retries race the peer's chain switches); once the group's + valid tests are done, that state poisons nothing. + """ + has_invalid, chain_length = chain_properties.get( + item.nodeid, (False, 0) + ) + for marker in item.iter_markers("xdist_group"): + if "name" in marker.kwargs: + group = marker.kwargs["name"] + return ( + -group_counts[group], + group, + has_invalid, + chain_length, + item.nodeid, + ) + return (0, "", has_invalid, chain_length, item.nodeid) + + items.sort(key=sort_key) + + +@pytest.fixture(scope="module") +def test_suite_name() -> str: + """The name of the hive test suite used in this simulator.""" + return "eels/consume-wirex" + + +@pytest.fixture(scope="module") +def test_suite_description() -> str: + """The description of the hive test suite used in this simulator.""" + return ( + "Execute blockchain tests against clients by making them full sync " + "the fixture blocks from a mock devp2p peer." + ) + + +@pytest.fixture(scope="session") +def wirex_min_blocks(request: pytest.FixtureRequest) -> int: + """Return the smallest chain length worth syncing.""" + return int(request.config.getoption("wirex_min_blocks")) + + +@pytest.fixture(scope="session") +def wirex_eth_versions(request: pytest.FixtureRequest) -> tuple[int, ...]: + """Return the eth capability versions the mock peer advertises.""" + option = str(request.config.getoption("wirex_eth_version")) + if option == "auto": + return tuple(sorted(ETH_PROTOCOLS)) + return (int(option),) + + +@pytest.fixture(scope="session") +def wirex_sync_timeout(request: pytest.FixtureRequest) -> float: + """Return how long to wait for a client to reach the fixture head.""" + return float(request.config.getoption("wirex_sync_timeout")) + + +@pytest.fixture(scope="session") +def wirex_announce_interval(request: pytest.FixtureRequest) -> float: + """Return how often to repeat the sync target announcement.""" + return float(request.config.getoption("wirex_announce_interval")) + + +@pytest.fixture(scope="session") +def wirex_poll_interval(request: pytest.FixtureRequest) -> float: + """Return the interval between forkchoice updates while syncing.""" + return float(request.config.getoption("wirex_poll_interval")) + + +def sync_chain_payloads( + fixture: BlockchainEngineXFixture, +) -> list[FixtureEngineNewPayload]: + """ + Return the payload sequence a sync-based consumer serves. + + The author's chain, plus the appended sync payload when the fixture + carries one: the trailer is a real block above the test's head, and + it is the block this simulator announces, so the peer must hold it + like any other. Prepend-class fixtures need no assembly - their + extra block is already ``payloads[0]``. + """ + payloads = list(fixture.payloads) + if fixture.sync_payload is not None: + payloads.append(fixture.sync_payload) + return payloads + + +@pytest.fixture(scope="function") +def chain( + genesis_header: FixtureHeader, + fixture: BlockchainEngineXFixture, + wirex_min_blocks: int, +) -> Chain: + """ + Rebuild the chain of blocks this test expects a client to hold. + + The chain is the author's payloads plus the appended sync payload + when the fixture carries one, so an appended-class single-block + test is a two-block chain here. Chains too short to put any block + on the wire skip here, before any reconstruction or peer setup is + spent on them. + + Fixtures whose payloads are flagged invalid still reconstruct and + are served as rejection tests (see ``test_blockchain_via_wirex``): + their blocks are semantically invalid but hash-consistent, so they + travel the wire like any other block. The exception is a payload + whose declared block hash does not match its own header (a header + corrupted at fill via ``rlp_modifier``): devp2p has no way to + present a block whose hash differs from its header's keccak, so + such fixtures are skipped rather than reported as setup errors. + """ + payloads = sync_chain_payloads(fixture) + if len(payloads) < wirex_min_blocks: + pytest.skip( + f"chain has {len(payloads)} block(s); at least " + f"{wirex_min_blocks} are needed for any block to be " + "transferred over devp2p rather than the Engine API" + ) + try: + return chain_from_payloads(genesis_header, payloads) + except ChainReconstructionError as error: + if any(not payload.valid() for payload in fixture.payloads): + pytest.skip( + f"invalid fixture cannot be represented over devp2p: {error}" + ) + raise + + +@pytest.fixture(scope="session") +def mock_peers() -> Generator[Dict[str, MockPeer], None, None]: + """Hold one peer per client for the lifetime of the session.""" + peers: Dict[str, MockPeer] = {} + yield peers + for peer in peers.values(): + peer.close() + + +@pytest.fixture(scope="function") +def mock_peer( + client: Client, + chain: Chain, + mock_peers: Dict[str, MockPeer], + wirex_eth_versions: tuple[int, ...], + total_timing_data: "TimingData", +) -> MockPeer: + """ + Return the peer connected to this test's client. + + The connection is established once per client and then re-pointed at + each test's chain, so the RLPx handshake is paid once per group + rather than once per test. + """ + peer = mock_peers.get(client.id) + if peer is None: + enode = client.enode() + logger.info(f"Connecting mock peer to {enode}") + peer = MockPeer( + host=str(client.ip), + port=enode.port, + remote_public_key=bytes.fromhex(enode.id), + private_key=os.urandom(32), + network_id=DEFAULT_NETWORK_ID, + eth_versions=wirex_eth_versions, + ) + with total_timing_data.time("Connect mock peer"): + # The readiness gate behind `client` waits on the Engine + # API port only; a freshly started client may open its + # devp2p listener a moment later, so the first dial gets + # a deadline rather than a single attempt. + deadline = time.monotonic() + 10.0 + while True: + try: + peer.connect(chain) + break + except OSError: + if time.monotonic() >= deadline: + raise + time.sleep(0.25) + peer.start() + mock_peers[client.id] = peer + logger.info(f"Mock peer connected to {peer.remote_name}") + return peer + + # A client may hang up mid-group (nethermind drops peers it deems + # idle); a dead connection would otherwise fail every remaining test + # in the group, so redial exactly as a real peer would. + if not peer.alive: + logger.warning("Peer connection lost; redialing the client") + with total_timing_data.time("Reconnect mock peer"): + peer.reconnect(chain) + return peer + try: + peer.set_chain(chain) + except OSError: + logger.warning("Connection died announcing the chain; redialing") + with total_timing_data.time("Reconnect mock peer"): + peer.reconnect(chain) + return peer diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/tests/test_wirex_sync_contract.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/tests/test_wirex_sync_contract.py new file mode 100644 index 00000000000..c56aeccdd3d --- /dev/null +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/tests/test_wirex_sync_contract.py @@ -0,0 +1,172 @@ +"""Tests for the wirex simulator's per-class sync-block contract.""" + +from dataclasses import dataclass +from typing import cast + +from execution_testing.base_types import Bytes, Hash +from execution_testing.devp2p.chain import Block, Chain +from execution_testing.fixtures import BlockchainEngineXFixture +from execution_testing.fixtures.blockchain import ( + FixtureEngineNewPayload, + FixtureHeader, +) + +from ..simulators.simulator_logic.test_via_wirex import ( + announced_payload, + required_wire_bodies, +) +from ..simulators.wirex.conftest import sync_chain_payloads + + +def _payload() -> FixtureEngineNewPayload: + """Return a bare payload sentinel; the tests read identity only.""" + return cast(FixtureEngineNewPayload, object()) + + +@dataclass +class _StubFixture: + """The two fixture fields the contract helpers read.""" + + payloads: list[FixtureEngineNewPayload] + sync_payload: FixtureEngineNewPayload | None + + +def _fixture( + payloads: list[FixtureEngineNewPayload], + sync_payload: FixtureEngineNewPayload | None = None, +) -> BlockchainEngineXFixture: + """Return a fixture carrying exactly the fields the helpers read.""" + return cast(BlockchainEngineXFixture, _StubFixture(payloads, sync_payload)) + + +@dataclass +class _StubHeader: + """The two header fields a served block is read by.""" + + number: int + block_hash: Hash + + +def _block(number: int, *, empty: bool) -> Block: + """Return a block at `number`, with or without body content.""" + header = _StubHeader(number=number, block_hash=Hash(number)) + return Block( + header=cast(FixtureHeader, header), + transactions=[] if empty else [Bytes(b"\x01")], + withdrawals=None, + ) + + +def _chain(blocks: list[Block]) -> Chain: + """Return a chain of `blocks` under a stub genesis.""" + genesis = _StubHeader(number=0, block_hash=Hash(0)) + return Chain(genesis=cast(FixtureHeader, genesis), blocks=blocks) + + +class TestAnnouncedPayload: + """Which block the simulator announces, per chain class.""" + + def test_appended_class_announces_the_trailer(self) -> None: + """A fixture with a sync payload announces it, not its head.""" + test_block, trailer = _payload(), _payload() + assert announced_payload(_fixture([test_block], trailer)) is trailer + + def test_prepended_class_announces_its_own_head(self) -> None: + """An in-chain prepend keeps the test's block as the target.""" + prepended, invalid_head = _payload(), _payload() + fixture = _fixture([prepended, invalid_head]) + assert announced_payload(fixture) is invalid_head + + def test_bare_chain_announces_its_own_head(self) -> None: + """A chain with no extra block announces the author's head.""" + first, head = _payload(), _payload() + assert announced_payload(_fixture([first, head])) is head + + +class TestSyncChainPayloads: + """The served sequence, and the chain length skips are judged by.""" + + def test_appended_singleton_becomes_two_blocks(self) -> None: + """The trailer joins the served chain, above the test's head.""" + test_block, trailer = _payload(), _payload() + payloads = sync_chain_payloads(_fixture([test_block], trailer)) + assert payloads == [test_block, trailer] + + def test_prepended_singleton_is_already_two_blocks(self) -> None: + """An in-chain prepend needs no assembly.""" + prepended, invalid_head = _payload(), _payload() + fixture = _fixture([prepended, invalid_head]) + assert sync_chain_payloads(fixture) == [prepended, invalid_head] + + def test_bare_singleton_stays_one_block(self) -> None: + """No extra block, nothing to add: skipped below the minimum.""" + assert len(sync_chain_payloads(_fixture([_payload()]))) == 1 + + def test_bare_chain_keeps_its_own_length(self) -> None: + """An invalid multi-block chain is served exactly as written.""" + payloads = [_payload(), _payload()] + assert sync_chain_payloads(_fixture(payloads)) == payloads + + def test_the_author_payload_list_is_not_mutated(self) -> None: + """Assembly returns a new list; the fixture stays the author's.""" + test_block, trailer = _payload(), _payload() + fixture = _fixture([test_block], trailer) + sync_chain_payloads(fixture) + assert fixture.payloads == [test_block] + + +class TestRequiredWireBodies: + """Which bodies the wire-coverage check demands, per chain shape.""" + + def test_the_announced_head_is_exempt(self) -> None: + """ + A reth-shaped client passes: ancestors served, head never. + + The head's payload arrives through the Engine API, and whether + a client also re-fetches its body from a peer goes both ways + across measured clients - so the head is never required. + """ + ancestors = [_block(1, empty=False), _block(2, empty=False)] + chain = _chain([*ancestors, _block(3, empty=True)]) + required = required_wire_bodies(chain) + assert required == ancestors + served = {block.block_hash for block in ancestors} + assert [b.number for b in required if b.block_hash not in served] == [] + + def test_a_missing_ancestor_is_named(self) -> None: + """An ancestor body that never traveled fails, by number.""" + chain = _chain( + [ + _block(1, empty=False), + _block(2, empty=False), + _block(3, empty=True), + ] + ) + served = {_block(1, empty=False).block_hash} + missing = [ + block.number + for block in required_wire_bodies(chain) + if block.block_hash not in served + ] + assert missing == [2] + + def test_derivable_ancestor_bodies_are_exempt(self) -> None: + """An empty body a client derives from its header is not owed.""" + chain = _chain( + [ + _block(1, empty=True), + _block(2, empty=False), + _block(3, empty=True), + ] + ) + assert [b.number for b in required_wire_bodies(chain)] == [2] + + def test_prepend_class_check_is_vacuous(self) -> None: + """ + Below an invalid head sits only the empty prepended block. + + Its body is derivable, so nothing is owed over the wire - the + test's own block is judged through the Engine API instead. + """ + chain = _chain([_block(1, empty=True), _block(2, empty=False)]) + assert required_wire_bodies(chain) == [] diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/help/help.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/help/help.py index 0b501956faa..5b03d8e3b21 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/help/help.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/help/help.py @@ -134,6 +134,10 @@ def pytest_configure(config: pytest.Config) -> None: "pytest-consume.ini", [ "consuming", + # Per-simulator option groups; each exists only when its + # subcommand loaded the matching plugin, so unmatched + # substrings are harmless for the other subcommands. + "wirex", ], ) elif config.getoption("show_execute_help"): @@ -215,6 +219,8 @@ def show_specific_help( } if isinstance(action, argparse._StoreTrueAction): kwargs["action"] = "store_true" + elif isinstance(action, argparse._StoreFalseAction): + kwargs["action"] = "store_false" else: kwargs["type"] = action.type if action.nargs: diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/processors.py b/packages/testing/src/execution_testing/cli/pytest_commands/processors.py index d23174fc34e..c5f6140dae7 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/processors.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/processors.py @@ -106,11 +106,13 @@ def process_args(self, args: List[str]) -> List[str]: ] and not self._has_parallelism_flag(args): modified_args.extend(["-n", str(hive_parallelism)]) - # For enginex: ensure xdist uses loadgroup distribution so tests with - # the same xdist_group marker (pre-alloc group) run on the same worker - if self.command_name == "enginex" and self._has_parallelism_flag( - modified_args - ): + # For enginex and wirex: ensure xdist uses loadgroup distribution so + # tests with the same xdist_group marker (pre-alloc group) run on the + # same worker + if self.command_name in ( + "enginex", + "wirex", + ) and self._has_parallelism_flag(modified_args): if "--dist" not in modified_args: modified_args.extend(["--dist", "loadgroup"]) @@ -126,6 +128,7 @@ def process_args(self, args: List[str]) -> List[str]: "engine", "enginex", "sync", + "wirex", "rlp", "build_block", } diff --git a/packages/testing/src/execution_testing/devp2p/__init__.py b/packages/testing/src/execution_testing/devp2p/__init__.py new file mode 100644 index 00000000000..1083c54c912 --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/__init__.py @@ -0,0 +1,8 @@ +""" +A deterministic devp2p peer that serves fixture blocks to a client. + +The modules here implement just enough of RLPx and the eth wire protocol +for an execution client to full sync a fixture backed chain from this +framework, so that historical blocks reach the client through its +production peer-to-peer ingestion path instead of an offline import. +""" diff --git a/packages/testing/src/execution_testing/devp2p/chain.py b/packages/testing/src/execution_testing/devp2p/chain.py new file mode 100644 index 00000000000..77b0fe105ed --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/chain.py @@ -0,0 +1,316 @@ +""" +Reconstruction of consensus blocks from Engine API payloads. + +An Engine X fixture stores each block as the `engine_newPayload` request +that delivers it. A payload omits the header fields the client is +expected to derive - the two transaction and withdrawal tries, and the +constants a post-merge header carries - so serving those blocks over the +wire means putting them back. + +The reconstruction is self checking: a rebuilt header is only accepted if +its hash equals the block hash the payload already claims. +""" + +import hashlib +import logging +from dataclasses import dataclass +from typing import Dict, List, Sequence + +import ethereum_rlp as eth_rlp +from ethereum_types.numeric import Uint +from trie import HexaryTrie + +from execution_testing.base_types import Bytes, Hash +from execution_testing.fixtures.blockchain import ( + FixtureEngineNewPayload, + FixtureExecutionPayload, + FixtureHeader, +) +from execution_testing.test_types.block_types import Withdrawal + +from .protocol import encode_list, encode_transactions + +logger = logging.getLogger(__name__) + +EMPTY_OMMERS_HASH = Hash( + 0x1DCC4DE8DEC75D7AAB85B567B6CCD41AD312451B948A7413F0A142FD40D49347 +) +"""Keccak-256 of an empty ommer list, the only value a merged chain has.""" + +EMPTY_OMMERS_RLP = b"\xc0" + + +class ChainReconstructionError(Exception): + """Raised when a payload cannot be turned back into a valid block.""" + + +def _requests_hash(execution_requests: Sequence[Bytes]) -> Hash: + """ + Return the EIP-7685 requests hash over `execution_requests`. + + Each request arrives from the payload as its type byte followed by + the request data, and the header commits to the flat sha256 scheme: + `sha256(sha256(r_0) ++ sha256(r_1) ++ ...)`. + """ + digest = hashlib.sha256() + for request in execution_requests: + digest.update(hashlib.sha256(bytes(request)).digest()) + return Hash(digest.digest()) + + +def _transactions_root(transactions: Sequence[Bytes]) -> bytes: + """Return the transactions trie root over raw transaction bytes.""" + trie = HexaryTrie(db={}) + for index, transaction in enumerate(transactions): + trie.set(eth_rlp.encode(Uint(index)), transaction) + return trie.root_hash + + +@dataclass +class Block: + """One reconstructed block, ready to be served to a peer.""" + + header: FixtureHeader + transactions: List[Bytes] + withdrawals: List[Withdrawal] | None + + @property + def number(self) -> int: + """Return the block number.""" + return int(self.header.number) + + @property + def block_hash(self) -> bytes: + """Return the block hash.""" + return bytes(self.header.block_hash) + + def header_rlp(self) -> bytes: + """Return the RLP encoded header.""" + return bytes(self.header.rlp) + + def body_rlp(self) -> bytes: + """Return the RLP encoded block body.""" + items = [ + encode_transactions([bytes(t) for t in self.transactions]), + EMPTY_OMMERS_RLP, + ] + if self.withdrawals is not None: + items.append( + encode_list( + [ + eth_rlp.encode(withdrawal.to_serializable_list()) + for withdrawal in self.withdrawals + ] + ) + ) + return encode_list(items) + + +def block_from_payload(payload: FixtureEngineNewPayload) -> Block: + """ + Rebuild the block that `payload` delivers. + + Raise `ChainReconstructionError` if the rebuilt header does not hash + to the block hash the payload declares, which is what would happen if + a fork introduced a header field this reconstruction does not know + about. + """ + execution_payload = payload.params[0] + if not isinstance(execution_payload, FixtureExecutionPayload): + raise ChainReconstructionError("payload has no execution payload") + + withdrawals = execution_payload.withdrawals + beacon_root = payload.params[2] if len(payload.params) > 2 else None + execution_requests = payload.params[3] if len(payload.params) > 3 else None + block_access_list = execution_payload.block_access_list + + header = FixtureHeader( + parent_hash=execution_payload.parent_hash, + ommers_hash=EMPTY_OMMERS_HASH, + fee_recipient=execution_payload.fee_recipient, + state_root=execution_payload.state_root, + transactions_trie=Hash( + _transactions_root(execution_payload.transactions) + ), + receipts_root=execution_payload.receipts_root, + logs_bloom=execution_payload.logs_bloom, + difficulty=0, + number=execution_payload.number, + gas_limit=execution_payload.gas_limit, + gas_used=execution_payload.gas_used, + timestamp=execution_payload.timestamp, + extra_data=execution_payload.extra_data, + prev_randao=execution_payload.prev_randao, + nonce=0, + base_fee_per_gas=execution_payload.base_fee_per_gas, + withdrawals_root=( + None + if withdrawals is None + else Hash(Withdrawal.list_root(withdrawals)) + ), + blob_gas_used=execution_payload.blob_gas_used, + excess_blob_gas=execution_payload.excess_blob_gas, + parent_beacon_block_root=beacon_root, + requests_hash=( + None + if execution_requests is None + else _requests_hash(execution_requests) + ), + block_access_list_hash=( + None + if block_access_list is None + else block_access_list.keccak256() + ), + slot_number=execution_payload.slot_number, + ) + + if header.block_hash != execution_payload.block_hash: + raise ChainReconstructionError( + f"reconstructed block {execution_payload.number} hashes to " + f"{header.block_hash} but the payload declares " + f"{execution_payload.block_hash}" + ) + + return Block( + header=header, + transactions=list(execution_payload.transactions), + withdrawals=None if withdrawals is None else list(withdrawals), + ) + + +class Chain: + """ + The canonical chain a mock peer serves for one test. + + Blocks are indexed by both number and hash because a syncing client + walks headers backwards by hash from the head it was told to reach, + then asks for bodies by hash. + """ + + def __init__(self, genesis: FixtureHeader, blocks: Sequence[Block]): + """Build a chain of `blocks` descending from `genesis`.""" + self.genesis = genesis + self.blocks = list(blocks) + self._by_number: Dict[int, Block] = { + block.number: block for block in self.blocks + } + self._by_hash: Dict[bytes, Block] = { + block.block_hash: block for block in self.blocks + } + + @property + def head(self) -> Block: + """Return the last block of the chain.""" + return self.blocks[-1] + + def header_rlp_by_hash(self, block_hash: bytes) -> bytes | None: + """Return the RLP of the header with `block_hash`, if held.""" + if block_hash == bytes(self.genesis.block_hash): + return bytes(self.genesis.rlp) + block = self._by_hash.get(block_hash) + return None if block is None else block.header_rlp() + + def header_rlp_by_number(self, number: int) -> bytes | None: + """Return the RLP of the header at `number`, if held.""" + if number == int(self.genesis.number): + return bytes(self.genesis.rlp) + block = self._by_number.get(number) + return None if block is None else block.header_rlp() + + def number_of(self, block_hash: bytes) -> int | None: + """Return the number of the block with `block_hash`, if held.""" + if block_hash == bytes(self.genesis.block_hash): + return int(self.genesis.number) + block = self._by_hash.get(block_hash) + return None if block is None else block.number + + def body_rlp_by_hash(self, block_hash: bytes) -> bytes | None: + """ + Return the RLP of the body with `block_hash`, if held. + + The genesis body is empty by definition, but its shape follows + its header: a genesis whose header commits to a withdrawals + root must serve the empty withdrawals list too, or the body + cannot validate against the header. + """ + if block_hash == bytes(self.genesis.block_hash): + items = [encode_list([]), EMPTY_OMMERS_RLP] + if self.genesis.withdrawals_root is not None: + items.append(encode_list([])) + return encode_list(items) + block = self._by_hash.get(block_hash) + return None if block is None else block.body_rlp() + + +class ServedChains: + """ + Every chain a peer has served over one connection. + + A client is reused across the tests of a pre-allocation group, and + each test installs its own chain. The client's downloader does not + forget the previous one that fast: it keeps asking for blocks of the + chain it was syncing when the test ended. A real peer would still + hold those blocks, so this one does too, and answers from whichever + installed chain a requested hash belongs to. + + Requests that name a block by number are answered from the current + chain only, since a number alone does not identify a chain. + """ + + def __init__(self) -> None: + """Start with no chain installed.""" + self._chain_by_hash: Dict[bytes, Chain] = {} + self._current: Chain | None = None + + def install(self, chain: Chain) -> None: + """Serve `chain` from now on, keeping earlier chains available.""" + self._current = chain + for block in chain.blocks: + self._chain_by_hash.setdefault(block.block_hash, chain) + + @property + def current(self) -> Chain: + """Return the chain currently being served.""" + assert self._current is not None, "no chain installed" + return self._current + + def chain_for_hash(self, block_hash: bytes) -> Chain | None: + """Return the chain holding `block_hash`, if any.""" + if self._current is not None and block_hash == bytes( + self._current.genesis.block_hash + ): + return self._current + return self._chain_by_hash.get(block_hash) + + def header_rlp_by_hash(self, block_hash: bytes) -> bytes | None: + """Return the RLP of the header with `block_hash`, if held.""" + chain = self.chain_for_hash(block_hash) + return None if chain is None else chain.header_rlp_by_hash(block_hash) + + def body_rlp_by_hash(self, block_hash: bytes) -> bytes | None: + """Return the RLP of the body with `block_hash`, if held.""" + chain = self.chain_for_hash(block_hash) + return None if chain is None else chain.body_rlp_by_hash(block_hash) + + +def chain_from_payloads( + genesis: FixtureHeader, payloads: Sequence[FixtureEngineNewPayload] +) -> Chain: + """ + Build the chain that `payloads` describe, rooted at `genesis`. + + Every payload must reconstruct, and every block must name its + predecessor: a chain served with a gap in it would make a client's + sync failure look like a client bug. + """ + blocks = [block_from_payload(payload) for payload in payloads] + + parent_hash = bytes(genesis.block_hash) + for block in blocks: + if bytes(block.header.parent_hash) != parent_hash: + raise ChainReconstructionError( + f"block {block.number} does not extend its predecessor" + ) + parent_hash = block.block_hash + + return Chain(genesis, blocks) diff --git a/packages/testing/src/execution_testing/devp2p/ecies.py b/packages/testing/src/execution_testing/devp2p/ecies.py new file mode 100644 index 00000000000..dfed6dd27fc --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/ecies.py @@ -0,0 +1,112 @@ +""" +The ECIES scheme that protects the RLPx handshake messages. + +RLPx encrypts its `auth` and `ack` messages with ECIES: an ephemeral +Diffie-Hellman agreement feeds a concatenation KDF, whose output is split +into an AES-128-CTR key and a HMAC-SHA256 key. The two byte big endian +length prefix of the message is authenticated alongside the ciphertext as +shared MAC data. +""" + +import hashlib +import hmac +import os + +from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes + +from .secp256k1 import agree, public_key_bytes + +PUBLIC_KEY_LENGTH = 64 +"""Length of an uncompressed public key without its SEC tag.""" + +_IV_LENGTH = 16 +_MAC_LENGTH = 32 +_OVERHEAD = 1 + PUBLIC_KEY_LENGTH + _IV_LENGTH + _MAC_LENGTH + + +class DecryptionError(Exception): + """Raised when an ECIES message fails its authentication check.""" + + +def _concat_kdf(shared_secret: bytes) -> bytes: + """Derive 32 key bytes from `shared_secret` (NIST SP 800-56 KDF).""" + output = b"" + counter = 1 + while len(output) < 32: + output += hashlib.sha256( + counter.to_bytes(4, "big") + shared_secret + ).digest() + counter += 1 + return output[:32] + + +def _aes_ctr(key: bytes, initialization_vector: bytes, data: bytes) -> bytes: + """Return `data` run through AES-128-CTR under `key`.""" + cipher = Cipher( + algorithms.AES(key), modes.CTR(initialization_vector) + ).encryptor() + return cipher.update(data) + cipher.finalize() + + +def encrypt( + remote_public_key: bytes, plaintext: bytes, shared_mac_data: bytes +) -> bytes: + """ + Encrypt `plaintext` to `remote_public_key`. + + The result is `ephemeral-public-key || iv || ciphertext || tag`, with + `shared_mac_data` covered by the tag but not included in the output. + """ + ephemeral_private_key = os.urandom(32) + shared_secret = agree(ephemeral_private_key, remote_public_key) + key_material = _concat_kdf(shared_secret) + encryption_key = key_material[:16] + mac_key = hashlib.sha256(key_material[16:]).digest() + + initialization_vector = os.urandom(_IV_LENGTH) + ciphertext = _aes_ctr(encryption_key, initialization_vector, plaintext) + tag = hmac.new( + mac_key, + initialization_vector + ciphertext + shared_mac_data, + hashlib.sha256, + ).digest() + + return ( + b"\x04" + + public_key_bytes(ephemeral_private_key) + + initialization_vector + + ciphertext + + tag + ) + + +def decrypt( + private_key: bytes, message: bytes, shared_mac_data: bytes +) -> bytes: + """ + Decrypt an ECIES `message` addressed to `private_key`. + + Raise `DecryptionError` if the message is truncated or its tag does + not authenticate the ciphertext and `shared_mac_data`. + """ + if len(message) < _OVERHEAD or message[0] != 0x04: + raise DecryptionError("malformed ECIES message") + + ephemeral_public_key = message[1 : 1 + PUBLIC_KEY_LENGTH] + initialization_vector = message[ + 1 + PUBLIC_KEY_LENGTH : 1 + PUBLIC_KEY_LENGTH + _IV_LENGTH + ] + ciphertext = message[1 + PUBLIC_KEY_LENGTH + _IV_LENGTH : -_MAC_LENGTH] + tag = message[-_MAC_LENGTH:] + + key_material = _concat_kdf(agree(private_key, ephemeral_public_key)) + mac_key = hashlib.sha256(key_material[16:]).digest() + expected_tag = hmac.new( + mac_key, + initialization_vector + ciphertext + shared_mac_data, + hashlib.sha256, + ).digest() + if not hmac.compare_digest(tag, expected_tag): + raise DecryptionError("ECIES tag mismatch") + + return _aes_ctr(key_material[:16], initialization_vector, ciphertext) diff --git a/packages/testing/src/execution_testing/devp2p/keccak.py b/packages/testing/src/execution_testing/devp2p/keccak.py new file mode 100644 index 00000000000..2e11cdeee53 --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/keccak.py @@ -0,0 +1,54 @@ +""" +Incremental Keccak-256 used by the RLPx frame MACs. + +The RLPx transport keeps a running Keccak-256 state per direction, and +reads a digest from it after every frame *without* finalizing it. +``hashlib`` cannot express that at all, since it implements SHA3, a +different padding rule. ``pycryptodome`` can, but only when asked: a +Keccak hash built with ``update_after_digest`` pads and squeezes a copy +of the sponge, leaving the absorbing state free to take the next frame. + +That flag is worth the note it takes to explain, because the tempting +alternative - a hand-rolled pure-Python sponge - absorbs at roughly +0.2 MiB/s, some 880 times slower than pycryptodome's compiled Keccak. +At that rate, MAC-ing one frame carrying an eight mebibyte EIP-7934 +block takes the best part of a minute, long enough for a syncing +client to give up on the peer and drop it mid-transfer. +""" + +from Crypto.Hash import keccak as _pycryptodome_keccak + + +class Keccak256: + """ + A Keccak-256 sponge that can be digested and then updated again. + + `digest` leaves the absorbing state untouched, which is what allows a + single instance to act as the running egress or ingress MAC of an + RLPx connection. + """ + + def __init__(self, data: bytes = b"") -> None: + """Initialize the sponge, optionally absorbing `data`.""" + self._hash = _pycryptodome_keccak.new( + digest_bits=256, update_after_digest=True + ) + self.update(data) + + def update(self, data: bytes) -> None: + """Absorb `data` into the sponge.""" + self._hash.update(data) + + def digest(self) -> bytes: + """ + Return the 32 byte digest of everything absorbed so far. + + The sponge remains usable: further `update` calls continue from + the pre-padding state. + """ + return bytes(self._hash.digest()) + + +def keccak256(data: bytes) -> bytes: + """Return the Keccak-256 digest of `data`.""" + return Keccak256(data).digest() diff --git a/packages/testing/src/execution_testing/devp2p/peer.py b/packages/testing/src/execution_testing/devp2p/peer.py new file mode 100644 index 00000000000..d447c790e8b --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/peer.py @@ -0,0 +1,516 @@ +""" +The deterministic peer that serves fixture chains over devp2p. + +The peer dials the client under test, completes the RLPx and eth +handshakes, and then answers header and body requests from whichever +fixture chain is currently installed. It is deliberately honest: it never +withholds, reorders or corrupts a response, so a sync failure is a +finding about the client or the fixture rather than about the peer. + +Every request and response is recorded in a transcript, which is what +makes a stalled sync diagnosable after the fact. +""" + +import logging +import threading +from dataclasses import dataclass, field +from typing import Dict, List, Optional, Sequence, Set + +from .chain import Chain, ServedChains +from .protocol import ( + BLOCK_BODIES, + BLOCK_HEADERS, + BLOCK_RANGE_UPDATE, + DISCONNECT, + ETH_PROTOCOLS, + GET_BLOCK_ACCESS_LISTS, + GET_BLOCK_BODIES, + GET_BLOCK_HEADERS, + GET_RECEIPTS, + HELLO, + MESSAGE_NAMES, + P2P_VERSION, + PING, + PONG, + STATUS, + BlockHeadersRequest, + EthProtocol, + ProtocolError, + Status, + decode_disconnect, + decode_get_block_access_lists, + decode_get_block_bodies, + decode_get_block_headers, + decode_hello, + encode_block_range_update, + encode_hello, + encode_response, + highest_common_eth_version, +) +from .rlpx import RLPxError, RLPxSession, connect +from .secp256k1 import public_key_bytes + +logger = logging.getLogger(__name__) + +CLIENT_IDENTIFIER = "wirex-peer/v0" +"""Client identifier advertised in the base protocol handshake.""" + +MAX_HEADERS_PER_RESPONSE = 1024 +"""Cap on headers served in one response, matching common client limits.""" + +MAX_BODIES_PER_RESPONSE = 256 +"""Cap on bodies served in one response.""" + +SOFT_RESPONSE_LIMIT = 2 * 1024 * 1024 +""" +Target ceiling on the serialized bytes of one response. + +A count of items is no bound at all when one item may be megabytes, and +a client caps the size of every message it reads: geth refuses anything +over ten mebibytes and drops the peer rather than truncating it. Two +EIP-7934 blocks are roughly sixteen, so stop filling a response once it +reaches this many bytes and let the client ask for the rest. The value +is the one clients themselves stop at, and it is a target rather than a +cap: a single body is always served, however large it is alone. +""" + +EMPTY_LIST_PAYLOAD = b"\xc0" + + +@dataclass +class PeerStatistics: + """Counts of what a client asked for during one test.""" + + header_requests: int = 0 + headers_served: int = 0 + body_requests: int = 0 + bodies_served: int = 0 + body_hashes_served: Set[bytes] = field(default_factory=set) + """ + The block hashes whose bodies were served, not only their count. + + A count cannot tell the consumer *which* bodies traveled the wire, + and its wire-coverage claim is per block: every block with a + non-derivable body must have been downloaded from this peer, not + just some block. + """ + unknown_requests: int = 0 + unanswered_requests: Dict[str, int] = field(default_factory=dict) + """ + Requests the peer deliberately left unanswered, counted by message + name. A nonzero count is a finding, never noise: it means the + client asked for data it should derive by executing blocks. + """ + transcript: List[str] = field(default_factory=list) + + @property + def receipt_requests(self) -> int: + """Return how many GetReceipts requests went unanswered.""" + return self.unanswered_requests.get("GetReceipts", 0) + + def record(self, line: str) -> None: + """Append `line` to the request transcript.""" + self.transcript.append(line) + + +class MockPeer: + """ + A single connection to the client under test. + + The peer runs its message loop on a background thread so that the + test body can drive the Engine API while the client is downloading + the chain. + """ + + def __init__( + self, + host: str, + port: int, + remote_public_key: bytes, + private_key: bytes, + network_id: int, + eth_versions: Sequence[int] | None = None, + ) -> None: + """ + Record where to dial and under which network identity. + + `eth_versions` is the set of eth capability versions to + advertise; the default advertises every implemented version. + Passing exactly one version forces the client to speak it or + fail the handshake loudly, which is what probing a client's + version matrix wants. + """ + self.host = host + self.port = port + self.remote_public_key = remote_public_key + self.private_key = private_key + self.network_id = network_id + if eth_versions is None: + eth_versions = tuple(ETH_PROTOCOLS) + unknown = set(eth_versions).difference(ETH_PROTOCOLS) + if unknown: + raise ValueError( + f"unimplemented eth version(s) {sorted(unknown)}; " + f"implemented: {sorted(ETH_PROTOCOLS)}" + ) + self.eth_versions = tuple(sorted(eth_versions)) + self.protocol: Optional[EthProtocol] = None + + self._session: Optional[RLPxSession] = None + self._chains = ServedChains() + self._lock = threading.Lock() + self._stop = threading.Event() + self._dead = threading.Event() + self._thread: Optional[threading.Thread] = None + self.statistics = PeerStatistics() + self.body_hashes_ever_served: Set[bytes] = set() + """ + Every block hash whose body this peer served the client, across + the peer's whole lifetime. + + `statistics.body_hashes_served` is reset for each test so its + transcript stays attributable, but the client it describes is + reused across a whole group of tests and keeps every block it + ever imported. Two tests of one group may declare byte-identical + chains, and the client re-syncs nothing for the second one - its + blocks already traveled the wire during the first. This set is + the evidence that they did: the consumer's wire-coverage check + consults it, so a body is required to cross the wire once per + client, not once per test. + """ + self.remote_name = "" + self.disconnect_reason: Optional[int] = None + + @property + def alive(self) -> bool: + """Whether the connection's message loop is still running.""" + return self._thread is not None and not self._dead.is_set() + + def connect(self, chain: Chain, timeout: float = 30.0) -> None: + """ + Dial the client and complete both handshakes. + + `chain` is the chain advertised in the eth Status message and the + one served until `set_chain` replaces it. + """ + self._chains.install(chain) + session = connect( + self.host, + self.port, + self.remote_public_key, + self.private_key, + timeout=timeout, + ) + self._session = session + + session.write_message( + HELLO, + encode_hello( + CLIENT_IDENTIFIER, + public_key_bytes(self.private_key), + self.eth_versions, + ), + ) + code, payload = session.read_message() + if code == DISCONNECT: + raise RLPxError( + f"client refused the connection: reason " + f"{decode_disconnect(payload)}" + ) + if code != HELLO: + raise RLPxError(f"expected Hello, got message {code}") + remote_version, self.remote_name, capabilities = decode_hello(payload) + logger.info( + "Connected to %s (p2p version %d) advertising %s", + self.remote_name, + remote_version, + capabilities, + ) + negotiated = highest_common_eth_version( + self.eth_versions, capabilities + ) + if negotiated is None: + raise RLPxError( + f"no common eth version: this peer advertises " + f"{list(self.eth_versions)}, {self.remote_name or 'client'} " + f"advertises {capabilities}" + ) + self.protocol = ETH_PROTOCOLS[negotiated] + logger.info("Negotiated eth/%d", negotiated) + with self._lock: + self.statistics.record(f"eth/{negotiated} negotiated") + if min(remote_version, P2P_VERSION) >= 5: + # Every message after Hello is Snappy compressed once both + # sides have advertised base protocol version 5 or higher. + session.enable_snappy() + + session.write_message( + STATUS, self.protocol.encode_status(self._status(chain)) + ) + session.set_timeout(1.0) + + def _status(self, chain: Chain) -> Status: + """Return the Status message describing `chain`.""" + return Status( + network_id=self.network_id, + genesis_hash=bytes(chain.genesis.block_hash), + fork_activations=[], + earliest_block=0, + latest_block=chain.head.number, + latest_block_hash=chain.head.block_hash, + ) + + def start(self) -> None: + """Start the background message loop.""" + self._thread = threading.Thread( + target=self._run, name="wirex-peer", daemon=True + ) + self._thread.start() + + def set_chain(self, chain: Chain) -> None: + """ + Serve `chain` from now on and announce its range. + + Installing a new chain is how one client is reused across the + tests of a pre-allocation group: each test is an independent + chain from the same genesis. + """ + with self._lock: + self._chains.install(chain) + self.statistics = PeerStatistics() + if self.protocol is not None: + # Every test's transcript documents which wire dialect + # it exercised. + self.statistics.record( + f"eth/{self.protocol.version} negotiated" + ) + session = self._session + if session is not None: + session.write_message( + BLOCK_RANGE_UPDATE, + encode_block_range_update( + 0, chain.head.number, chain.head.block_hash + ), + ) + + def reconnect(self, chain: Chain, timeout: float = 30.0) -> None: + """ + Dial the client again after it dropped the previous session. + + A client is free to hang up on a peer at any time (nethermind + does, with `Disconnect(0x00)`); a real peer would simply redial, + so this one does too. Chains installed on the previous session + stay installed: the client's downloader may still ask for them. + """ + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=5.0) + if self._session is not None: + self._session.close() + self._stop.clear() + self._dead.clear() + self.disconnect_reason = None + self.connect(chain, timeout=timeout) + self.start() + + def _run(self) -> None: + """Read and answer messages until stopped or disconnected.""" + session = self._session + assert session is not None + try: + while not self._stop.is_set(): + try: + code, payload = session.read_message() + except TimeoutError: + continue + except (OSError, RLPxError) as error: + logger.info("Peer connection ended: %s", error) + return + + try: + self._handle(session, code, payload) + except OSError as error: + # The client closed the socket while the answer was + # being written. Ending the loop lets the owner + # notice via `alive` and redial, rather than leaving + # a traceback in a thread nobody joins. + logger.info( + "Peer connection ended while answering message %d: %s", + code, + error, + ) + return + except (ProtocolError, RLPxError) as error: + logger.warning( + "Failed to answer message %d: %s", code, error + ) + return + if self.disconnect_reason is not None: + # The client said goodbye; the socket is as good as + # closed. Ending the loop lets the owner notice via + # `alive` and redial. + return + finally: + self._dead.set() + + def _handle(self, session: RLPxSession, code: int, payload: bytes) -> None: + """Answer one message from the client.""" + protocol = self.protocol + assert protocol is not None, "message received before negotiation" + if code == PING: + session.write_message(PONG, EMPTY_LIST_PAYLOAD) + elif code == GET_BLOCK_HEADERS: + self._serve_headers(session, decode_get_block_headers(payload)) + elif code == GET_BLOCK_BODIES: + self._serve_bodies(session, *decode_get_block_bodies(payload)) + elif code in protocol.unanswered_requests: + # A full syncing client derives receipts - and, from + # Amsterdam, block access lists - by executing the block, + # so a request here means the client chose a path this peer + # cannot honestly serve. Leave it unanswered and make the + # silence visible. + name = protocol.unanswered_requests[code] + if code == GET_RECEIPTS: + description = protocol.decode_get_receipts(payload).describe() + elif code == GET_BLOCK_ACCESS_LISTS: + _, hashes = decode_get_block_access_lists(payload) + description = f"access lists for {len(hashes)} hashes" + else: + description = name + with self._lock: + self.statistics.unanswered_requests[name] = ( + self.statistics.unanswered_requests.get(name, 0) + 1 + ) + self.statistics.record(f"{description} (unanswered)") + logger.warning("Client requested %s; not served", name) + elif code == DISCONNECT: + self.disconnect_reason = decode_disconnect(payload) + logger.info( + "Client disconnected: reason %d", self.disconnect_reason + ) + elif code in (STATUS, HELLO, PONG): + pass + else: + with self._lock: + self.statistics.unknown_requests += 1 + logger.debug( + "Ignoring %s", MESSAGE_NAMES.get(code, f"message {code}") + ) + + def _serve_headers( + self, session: RLPxSession, request: BlockHeadersRequest + ) -> None: + """Answer a GetBlockHeaders request from the current chain.""" + with self._lock: + # The test's statistics object is captured under the lock: + # `set_chain` swaps in a fresh one when the next test + # starts, and a straggler request from the previous test + # must land in the previous test's statistics, not poison + # the next test's transcript (or satisfy its wire-coverage + # check with another chain's bodies). + statistics = self.statistics + statistics.header_requests += 1 + if request.origin_hash is None: + chain = self._chains.current + start = request.origin_number + else: + origin_chain = self._chains.chain_for_hash(request.origin_hash) + chain = ( + self._chains.current + if origin_chain is None + else origin_chain + ) + start = chain.number_of(request.origin_hash) + + headers: List[bytes] = [] + if start is not None: + step = request.skip + 1 + number = start + while len(headers) < min(request.amount, MAX_HEADERS_PER_RESPONSE): + header = chain.header_rlp_by_number(number) + if header is None: + break + headers.append(header) + number = number - step if request.reverse else number + step + if number < 0: + break + + # Written before recording, for the same reason as in + # `_serve_bodies`: a failed send must not read as service. + session.write_message( + BLOCK_HEADERS, encode_response(request.request_id, headers) + ) + + with self._lock: + statistics.headers_served += len(headers) + statistics.record( + f"{request.describe()} -> {len(headers)} headers" + ) + + def _serve_bodies( + self, session: RLPxSession, request_id: int, hashes: List[bytes] + ) -> None: + """ + Answer a GetBlockBodies request from the served chains. + + A hash this peer does not hold is skipped, as the eth wire + protocol allows and real peers do; every held body in the + request is still served, up to the response bounds. Ending + the response at the first unknown hash instead would withhold + held bodies whenever a client mixes hashes from an abandoned + chain into a request, and starve its sync. + """ + with self._lock: + # Captured for the same straggler-attribution reason as in + # `_serve_headers`. + statistics = self.statistics + statistics.body_requests += 1 + chain = self._chains.current + + bodies: List[bytes] = [] + served_hashes: List[bytes] = [] + unknown: List[bytes] = [] + served_bytes = 0 + for block_hash in hashes[:MAX_BODIES_PER_RESPONSE]: + if served_bytes >= SOFT_RESPONSE_LIMIT: + break + body = self._chains.body_rlp_by_hash(block_hash) + if body is None: + unknown.append(block_hash) + continue + bodies.append(body) + served_hashes.append(block_hash) + served_bytes += len(body) + + # The response is written before anything is recorded as served: + # `body_hashes_served` is the evidence behind the consumer's + # per-block wire-coverage assertion, and a `write_message` that + # raises mid-send must not leave statistics claiming the bodies + # reached the client. + session.write_message( + BLOCK_BODIES, encode_response(request_id, bodies) + ) + + with self._lock: + statistics.bodies_served += len(bodies) + statistics.body_hashes_served.update(served_hashes) + self.body_hashes_ever_served.update(served_hashes) + detail = ( + "" + if not unknown + else f", {len(unknown)} unknown, first " + f"0x{unknown[0].hex()[:16]} (chain head " + f"#{chain.head.number} 0x{chain.head.block_hash.hex()[:16]})" + ) + statistics.record( + f"bodies for {len(hashes)} hashes -> " + f"{len(bodies)} served ({served_bytes} bytes){detail}" + ) + + def close(self) -> None: + """Stop the message loop and close the connection.""" + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=5.0) + if self._session is not None: + self._session.close() diff --git a/packages/testing/src/execution_testing/devp2p/protocol.py b/packages/testing/src/execution_testing/devp2p/protocol.py new file mode 100644 index 00000000000..cef0b4c7937 --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/protocol.py @@ -0,0 +1,430 @@ +""" +Message encoding for the devp2p base protocol and the eth capability. + +Only the subset a full syncing client asks of a serving peer is +implemented: the base protocol handshake and liveness messages, the eth +status exchange, and the header, body and range messages that carry +chain data. Announcement and transaction pool messages are decoded far +enough to be recognized and ignored. +""" + +import zlib +from dataclasses import dataclass +from typing import Dict, List, Mapping, Sequence, Tuple + +import ethereum_rlp as eth_rlp +from ethereum_types.numeric import Uint + +P2P_VERSION = 5 +""" +Base protocol version advertised to the remote node. + +Version 5 enables Snappy compression of every message payload after the +Hello exchange. Compression is negotiated: it is only used when both +sides advertise version 5 or higher, so a version 4 remote still gets +an uncompressed connection. +""" + +ETH_OFFSET = 16 +"""Message code offset of the first capability after the base protocol.""" + +# Base protocol message codes. +HELLO = 0x00 +DISCONNECT = 0x01 +PING = 0x02 +PONG = 0x03 + +# eth capability message codes, already offset onto the wire. +STATUS = ETH_OFFSET + 0x00 +TRANSACTIONS = ETH_OFFSET + 0x02 +GET_BLOCK_HEADERS = ETH_OFFSET + 0x03 +BLOCK_HEADERS = ETH_OFFSET + 0x04 +GET_BLOCK_BODIES = ETH_OFFSET + 0x05 +BLOCK_BODIES = ETH_OFFSET + 0x06 +NEW_POOLED_TRANSACTION_HASHES = ETH_OFFSET + 0x08 +GET_POOLED_TRANSACTIONS = ETH_OFFSET + 0x09 +GET_RECEIPTS = ETH_OFFSET + 0x0F +RECEIPTS = ETH_OFFSET + 0x10 +BLOCK_RANGE_UPDATE = ETH_OFFSET + 0x11 +GET_BLOCK_ACCESS_LISTS = ETH_OFFSET + 0x12 +BLOCK_ACCESS_LISTS = ETH_OFFSET + 0x13 + +MESSAGE_NAMES = { + HELLO: "Hello", + DISCONNECT: "Disconnect", + PING: "Ping", + PONG: "Pong", + STATUS: "Status", + TRANSACTIONS: "Transactions", + GET_BLOCK_HEADERS: "GetBlockHeaders", + BLOCK_HEADERS: "BlockHeaders", + GET_BLOCK_BODIES: "GetBlockBodies", + BLOCK_BODIES: "BlockBodies", + NEW_POOLED_TRANSACTION_HASHES: "NewPooledTransactionHashes", + GET_POOLED_TRANSACTIONS: "GetPooledTransactions", + GET_RECEIPTS: "GetReceipts", + RECEIPTS: "Receipts", + BLOCK_RANGE_UPDATE: "BlockRangeUpdate", + GET_BLOCK_ACCESS_LISTS: "GetBlockAccessLists", + BLOCK_ACCESS_LISTS: "BlockAccessLists", +} +"""Human readable names used in the peer's request transcript.""" + + +class ProtocolError(Exception): + """Raised when a peer message cannot be decoded as expected.""" + + +def encode_list(encoded_items: Sequence[bytes]) -> bytes: + """ + Wrap already encoded RLP `encoded_items` in a list header. + + Chain data arrives pre-encoded: a fixture header knows its own RLP, + and a legacy transaction is carried as the RLP list it already is. + Re-encoding those through a generic encoder would nest them one level + too deep, so lists of raw items are assembled here instead. + """ + payload = b"".join(encoded_items) + if len(payload) < 56: + return bytes([0xC0 + len(payload)]) + payload + length = len(payload).to_bytes((len(payload).bit_length() + 7) // 8, "big") + return bytes([0xF7 + len(length)]) + length + payload + + +def encode_transactions(transactions: Sequence[bytes]) -> bytes: + """ + Encode a block's transactions as they appear in a block body. + + A legacy transaction is an RLP list and is spliced in unchanged; a + typed transaction is an opaque byte string holding its type prefix + and payload, and is encoded as such. + """ + items = [] + for transaction in transactions: + if transaction and transaction[0] >= 0xC0: + items.append(transaction) + else: + items.append(eth_rlp.encode(transaction)) + return encode_list(items) + + +def fork_id(genesis_hash: bytes, fork_activations: Sequence[int]) -> List: + """ + Return the EIP-2124 fork identifier `[fork-hash, fork-next]`. + + `fork_activations` holds the block numbers and timestamps at which + forks activate after genesis, in order. Activations at genesis are + part of the genesis rule set and must already be excluded. + """ + checksum = zlib.crc32(genesis_hash) + for activation in fork_activations: + checksum = zlib.crc32(activation.to_bytes(8, "big"), checksum) + return [checksum.to_bytes(4, "big"), Uint(0)] + + +def encode_hello( + client_id: str, + public_key: bytes, + eth_versions: Sequence[int], + listen_port: int = 0, +) -> bytes: + """ + Encode the base protocol Hello message. + + One `("eth", version)` pair is advertised per entry of + `eth_versions`, in ascending order. The remote applies the RLPx + rule to the advertised set: the shared capability with the highest + version wins. + """ + return eth_rlp.encode( + [ + Uint(P2P_VERSION), + client_id.encode(), + [[b"eth", Uint(version)] for version in sorted(eth_versions)], + Uint(listen_port), + public_key, + ] + ) + + +def highest_common_eth_version( + local_versions: Sequence[int], + remote_capabilities: Sequence[Tuple[str, int]], +) -> int | None: + """ + Return the eth version RLPx negotiation selects, if any. + + Both sides list `(name, version)` pairs in Hello and the shared + capability with the highest version wins. Capabilities other than + eth (e.g. snap) are not implemented and never join the count, so + message-id offsets stay fixed at `ETH_OFFSET`. + """ + remote_versions = { + version for name, version in remote_capabilities if name == "eth" + } + common = remote_versions.intersection(local_versions) + return max(common) if common else None + + +def decode_hello( + payload: bytes, +) -> Tuple[int, str, List[Tuple[str, int]]]: + """ + Return the remote base protocol version, client identifier and + capabilities. + """ + fields = eth_rlp.decode(payload) + if not isinstance(fields, list) or len(fields) < 3: + raise ProtocolError("malformed Hello message") + version = int.from_bytes(bytes(fields[0]), "big") + name = bytes(fields[1]).decode(errors="replace") + capabilities = [] + for capability in fields[2]: + capabilities.append( + ( + bytes(capability[0]).decode(errors="replace"), + int.from_bytes(bytes(capability[1]), "big"), + ) + ) + return version, name, capabilities + + +def decode_disconnect(payload: bytes) -> int: + """Return the reason code of a Disconnect message.""" + fields = eth_rlp.decode(payload) + if isinstance(fields, bytes): + return int.from_bytes(fields, "big") + if isinstance(fields, list) and fields: + return int.from_bytes(bytes(fields[0]), "big") + return -1 + + +@dataclass +class Status: + """The content of the eth capability handshake message.""" + + network_id: int + genesis_hash: bytes + fork_activations: Sequence[int] + earliest_block: int + latest_block: int + latest_block_hash: bytes + + +@dataclass +class GetReceiptsRequest: + """A decoded GetReceipts request.""" + + request_id: int + block_hashes: List[bytes] + first_block_receipt_index: int | None + """ + Receipt offset into the first block, letting a response continue a + block whose receipt list exceeded one message. Added by eth/70 + (EIP-7975); `None` on eth/69. + """ + + def describe(self) -> str: + """Return a one line description for the request transcript.""" + offset = ( + "" + if self.first_block_receipt_index is None + else f" from receipt {self.first_block_receipt_index}" + ) + return f"receipts for {len(self.block_hashes)} hashes{offset}" + + +@dataclass(frozen=True) +class EthProtocol: + """ + One implemented version of the eth capability. + + A protocol object owns exactly the things versions change: the + version number, the codecs whose wire shape differs between + versions, and the set of requests this peer deliberately leaves + unanswered. Everything version independent - RLP helpers, the fork + id, the header and body request codecs, which have been stable + since eth/66 - stays at module level. + """ + + version: int + + receipts_request_has_offset: bool + """ + Whether GetReceipts carries `firstBlockReceiptIndex` between the + request id and the block hashes. Added by eth/70 (EIP-7975) so a + receipts response can be resumed mid-block. + """ + + unanswered_requests: Mapping[int, str] + """ + Wire code to message name of every request this peer deliberately + never answers. Receipts, and from eth/71 block access lists, are + data a client could import instead of deriving by execution; + serving either would let a failing test pass with no coverage, so + the silence is a recorded decision per message type rather than an + omission. + """ + + def encode_status(self, status: Status) -> bytes: + """ + Encode the Status message. + + The layout was set by eth/69 (EIP-7642) and is unchanged + through eth/72 - the later versions' deltas live in other + messages - so all implemented versions share this codec and + differ only in the version they declare. + """ + return eth_rlp.encode( + [ + Uint(self.version), + Uint(status.network_id), + status.genesis_hash, + fork_id(status.genesis_hash, status.fork_activations), + Uint(status.earliest_block), + Uint(status.latest_block), + status.latest_block_hash, + ] + ) + + def decode_get_receipts(self, payload: bytes) -> GetReceiptsRequest: + """Decode a GetReceipts request as this version shapes it.""" + fields = eth_rlp.decode(payload) + expected = 3 if self.receipts_request_has_offset else 2 + if not isinstance(fields, list) or len(fields) != expected: + raise ProtocolError( + f"malformed GetReceipts message for eth/{self.version}" + ) + request_id = int.from_bytes(bytes(fields[0]), "big") + if self.receipts_request_has_offset: + first_index = int.from_bytes(bytes(fields[1]), "big") + hashes = fields[2] + else: + first_index = None + hashes = fields[1] + return GetReceiptsRequest( + request_id=request_id, + block_hashes=[bytes(item) for item in hashes], + first_block_receipt_index=first_index, + ) + + +ETH_PROTOCOLS: Dict[int, EthProtocol] = { + 69: EthProtocol( + version=69, + receipts_request_has_offset=False, + unanswered_requests={GET_RECEIPTS: "GetReceipts"}, + ), + 70: EthProtocol( + version=70, + receipts_request_has_offset=True, + unanswered_requests={GET_RECEIPTS: "GetReceipts"}, + ), + 71: EthProtocol( + version=71, + receipts_request_has_offset=True, + unanswered_requests={ + GET_RECEIPTS: "GetReceipts", + GET_BLOCK_ACCESS_LISTS: "GetBlockAccessLists", + }, + ), +} +""" +Every eth capability version this peer implements, by version number. + +eth/70 (EIP-7975) changes only the receipts pair, which this peer never +serves, so implementing it means decoding the new request shape. eth/71 +(EIP-8159) adds the block access list request pair; a block access list +carries post-state values a client could import instead of executing, +so the receipts rule generalizes and the requests are counted but never +answered. +""" + + +def encode_block_range_update( + earliest_block: int, latest_block: int, latest_block_hash: bytes +) -> bytes: + """Encode the range of blocks whose bodies this peer can serve.""" + return eth_rlp.encode( + [Uint(earliest_block), Uint(latest_block), latest_block_hash] + ) + + +@dataclass +class BlockHeadersRequest: + """A decoded GetBlockHeaders request.""" + + request_id: int + origin_hash: bytes | None + origin_number: int | None + amount: int + skip: int + reverse: bool + + def describe(self) -> str: + """Return a one line description for the request transcript.""" + origin = ( + f"#{self.origin_number}" + if self.origin_hash is None + else f"0x{self.origin_hash.hex()[:12]}" + ) + direction = "reverse" if self.reverse else "forward" + return ( + f"headers from {origin} amount={self.amount} " + f"skip={self.skip} {direction}" + ) + + +def decode_get_block_headers(payload: bytes) -> BlockHeadersRequest: + """Decode a GetBlockHeaders request.""" + fields = eth_rlp.decode(payload) + if not isinstance(fields, list) or len(fields) != 2: + raise ProtocolError("malformed GetBlockHeaders message") + request_id = int.from_bytes(bytes(fields[0]), "big") + query = fields[1] + if not isinstance(query, list) or len(query) != 4: + raise ProtocolError("malformed GetBlockHeaders query") + + origin = bytes(query[0]) + return BlockHeadersRequest( + request_id=request_id, + origin_hash=origin if len(origin) == 32 else None, + origin_number=( + None if len(origin) == 32 else int.from_bytes(origin, "big") + ), + amount=int.from_bytes(bytes(query[1]), "big"), + skip=int.from_bytes(bytes(query[2]), "big"), + reverse=bool(int.from_bytes(bytes(query[3]), "big")), + ) + + +def _decode_hash_list_request( + payload: bytes, name: str +) -> Tuple[int, List[bytes]]: + """Decode a `[request-id, [hash, ...]]` shaped request.""" + fields = eth_rlp.decode(payload) + if not isinstance(fields, list) or len(fields) != 2: + raise ProtocolError(f"malformed {name} message") + request_id = int.from_bytes(bytes(fields[0]), "big") + hashes = [bytes(item) for item in fields[1]] + return request_id, hashes + + +def decode_get_block_bodies(payload: bytes) -> Tuple[int, List[bytes]]: + """Decode a GetBlockBodies request into its identifier and hashes.""" + return _decode_hash_list_request(payload, "GetBlockBodies") + + +def decode_get_block_access_lists(payload: bytes) -> Tuple[int, List[bytes]]: + """ + Decode a GetBlockAccessLists request (eth/71, EIP-8159) into its + identifier and block hashes. + """ + return _decode_hash_list_request(payload, "GetBlockAccessLists") + + +def encode_response(request_id: int, encoded_items: Sequence[bytes]) -> bytes: + """Encode a request identifier and a list of pre-encoded items.""" + return encode_list( + [eth_rlp.encode(Uint(request_id)), encode_list(encoded_items)] + ) diff --git a/packages/testing/src/execution_testing/devp2p/rlpx.py b/packages/testing/src/execution_testing/devp2p/rlpx.py new file mode 100644 index 00000000000..3e1107a2a00 --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/rlpx.py @@ -0,0 +1,393 @@ +""" +The RLPx transport: encrypted, authenticated framing over TCP. + +This implements the initiator half of the handshake described in the +devp2p RLPx specification, and the frame codec that carries every +subsequent message. Only what a deterministic test peer needs is +present; there is no support for acting as the recipient of a dial. +""" + +import logging +import os +import select +import socket +import threading +from typing import Tuple + +import ethereum_rlp as eth_rlp +from cryptography.hazmat.primitives.ciphers import ( + Cipher, + CipherContext, + algorithms, + modes, +) +from ethereum_types.numeric import Uint +from spec256k1 import PrivateKey + +from .ecies import decrypt, encrypt +from .keccak import Keccak256, keccak256 +from .secp256k1 import agree, public_key_bytes +from .snappy import SnappyError, compress, decompress, decompressed_length + +logger = logging.getLogger(__name__) + +AUTH_VERSION = 4 +"""RLPx handshake version advertised in auth and ack messages.""" + +MAX_FRAME_SIZE = 1 << 24 +"""Largest frame this peer will write, and the ceiling for the +decompressed payload size a compressed message may claim on read. A +frame header expresses its length in three bytes, so an outgoing frame +at or above this limit cannot be described at all.""" + +FRAME_READ_TIMEOUT = 30.0 +""" +Socket timeout for the reads inside one frame. + +A frame read, once begun, cannot be paused and resumed: the ingress +cipher and MAC have already advanced over the bytes consumed so far. A +remote that stalls mid-frame for this long has abandoned the +connection, and the session with it. +""" + +_MAC_LENGTH = 16 + + +class RLPxError(Exception): + """Raised when the transport cannot maintain a valid session.""" + + +def _xor(left: bytes, right: bytes) -> bytes: + """Return the byte-wise exclusive-or of two equal length strings.""" + return bytes(a ^ b for a, b in zip(left, right, strict=True)) + + +def _pad_to_block(data: bytes) -> bytes: + """Zero pad `data` to a whole number of 16 byte blocks.""" + remainder = len(data) % 16 + return data if remainder == 0 else data + bytes(16 - remainder) + + +def _recv_exactly(connection: socket.socket, length: int) -> bytes: + """Read exactly `length` bytes or raise `RLPxError` on a close.""" + buffer = b"" + while len(buffer) < length: + chunk = connection.recv(length - len(buffer)) + if not chunk: + raise RLPxError("connection closed by peer") + buffer += chunk + return buffer + + +class _Mac: + """ + One direction of the RLPx frame MAC. + + The MAC is a running Keccak-256 state, seeded from the shared MAC + secret and both handshake messages, that is advanced by every header + and frame body. Each advance mixes in an AES-ECB encryption of the + state's own current digest, which binds the MAC to the connection's + key material rather than to the ciphertext alone. + """ + + def __init__(self, secret: bytes, seed: bytes) -> None: + """Seed the MAC state with `seed` under the MAC `secret`.""" + self._secret = secret + self._state = Keccak256(seed) + + def _encrypt_seed(self, seed: bytes) -> bytes: + """Return `seed` encrypted with AES-ECB under the MAC secret.""" + cipher = Cipher(algorithms.AES(self._secret), modes.ECB()).encryptor() + return cipher.update(seed) + cipher.finalize() + + def update_header(self, header_ciphertext: bytes) -> bytes: + """Advance the MAC over a frame header and return its tag.""" + digest = self._state.digest()[:_MAC_LENGTH] + self._state.update(_xor(self._encrypt_seed(digest), header_ciphertext)) + return self._state.digest()[:_MAC_LENGTH] + + def update_body(self, frame_ciphertext: bytes) -> bytes: + """Advance the MAC over a frame body and return its tag.""" + self._state.update(frame_ciphertext) + digest = self._state.digest()[:_MAC_LENGTH] + self._state.update(_xor(self._encrypt_seed(digest), digest)) + return self._state.digest()[:_MAC_LENGTH] + + +class RLPxSession: + """ + An established RLPx connection to a remote node. + + Construct with `connect`, which performs the encryption handshake and + returns a session whose `read_message` and `write_message` speak in + devp2p message codes and RLP encoded payloads. + """ + + _connection: socket.socket + _egress_aes: CipherContext + _ingress_aes: CipherContext + + def __init__( + self, + connection: socket.socket, + aes_secret: bytes, + mac_secret: bytes, + egress_seed: bytes, + ingress_seed: bytes, + ) -> None: + """Initialize the frame codec from the derived session secrets.""" + self._connection = connection + zero_counter = bytes(16) + self._egress_aes = Cipher( + algorithms.AES(aes_secret), modes.CTR(zero_counter) + ).encryptor() + self._ingress_aes = Cipher( + algorithms.AES(aes_secret), modes.CTR(zero_counter) + ).decryptor() + self._egress_mac = _Mac(mac_secret, egress_seed) + self._ingress_mac = _Mac(mac_secret, ingress_seed) + self._snappy = False + self._write_lock = threading.Lock() + self._read_poll_timeout: float = FRAME_READ_TIMEOUT + + def enable_snappy(self) -> None: + """ + Compress every message payload from now on. + + Called after the Hello exchange when both sides advertised base + protocol version 5 or higher. Hello itself is always exchanged + uncompressed, so enabling is a one-way switch made between the + handshake and the first capability message. + """ + self._snappy = True + + def _read_exactly(self, length: int) -> bytes: + """ + Read exactly `length` bytes of an in-flight frame. + + A timeout here is fatal to the session rather than retryable: + the ingress cipher and MAC have already advanced over the bytes + consumed so far, so the read cannot be resumed later. + """ + try: + return _recv_exactly(self._connection, length) + except TimeoutError: + raise RLPxError( + "connection stalled mid-frame; the stream cannot be resumed" + ) from None + + def write_message(self, code: int, payload: bytes) -> None: + """ + Write one devp2p message as a single RLPx frame. + + Serialized with a lock: the peer's serving thread and the test + thread (chain announcements) both write to the session, and the + egress cipher and MAC are stateful - an interleaved write would + corrupt the running MAC and the frame stream. + + A frame too large to describe in the header's three byte length + field is refused here. Without this the length would overflow + inside a serving thread, far from the caller that assembled an + oversized response. + """ + if self._snappy: + payload = compress(payload) + frame = eth_rlp.encode(Uint(code)) + payload + if len(frame) >= MAX_FRAME_SIZE: + raise RLPxError( + f"frame of {len(frame)} bytes exceeds the " + f"{MAX_FRAME_SIZE - 1} bytes a frame header can express" + ) + header = _pad_to_block( + len(frame).to_bytes(3, "big") + eth_rlp.encode([Uint(0), Uint(0)]) + ) + + with self._write_lock: + header_ciphertext = self._egress_aes.update(header) + header_mac = self._egress_mac.update_header(header_ciphertext) + frame_ciphertext = self._egress_aes.update(_pad_to_block(frame)) + frame_mac = self._egress_mac.update_body(frame_ciphertext) + + self._connection.sendall( + header_ciphertext + header_mac + frame_ciphertext + frame_mac + ) + + def read_message(self) -> Tuple[int, bytes]: + """ + Read one devp2p message and return its code and payload. + + Waits up to the `set_timeout` interval for a frame to begin, + raising `TimeoutError` while nothing has been consumed - the + only point where a read may time out and leave the session + usable, because resuming a partially read frame is impossible + once the ingress cipher and MAC have advanced over its start. + """ + ready, _, _ = select.select( + [self._connection], [], [], self._read_poll_timeout + ) + if not ready: + raise TimeoutError("no message within the read timeout") + header_ciphertext = self._read_exactly(16) + header_mac = self._read_exactly(_MAC_LENGTH) + if self._ingress_mac.update_header(header_ciphertext) != header_mac: + raise RLPxError("frame header MAC mismatch") + header = self._ingress_aes.update(header_ciphertext) + + frame_size = int.from_bytes(header[:3], "big") + if frame_size == 0: + raise RLPxError("zero-length frame") + + padded_size = (frame_size + 15) // 16 * 16 + frame_ciphertext = self._read_exactly(padded_size) + frame_mac = self._read_exactly(_MAC_LENGTH) + if self._ingress_mac.update_body(frame_ciphertext) != frame_mac: + raise RLPxError("frame body MAC mismatch") + frame = self._ingress_aes.update(frame_ciphertext)[:frame_size] + + # The message code is a single RLP encoded integer: either a + # literal byte below 0x80, or 0x80 for a code of zero. + code = 0 if frame[0] == 0x80 else frame[0] + payload = frame[1:] + if self._snappy: + try: + if decompressed_length(payload) > MAX_FRAME_SIZE: + raise RLPxError( + "compressed message claims a decompressed size " + "over the frame limit" + ) + payload = decompress(payload) + except SnappyError as error: + raise RLPxError( + f"invalid snappy payload in message {code}: {error}" + ) from None + return code, payload + + def set_timeout(self, timeout: float) -> None: + """ + Set how long `read_message` waits for a message to begin. + + The wait applies ahead of a frame, where timing out is + harmless; the reads inside a frame run under the fixed + `FRAME_READ_TIMEOUT`, because a partial frame cannot be + resumed and abandoning one ends the session. + """ + self._read_poll_timeout = timeout + + def close(self) -> None: + """Close the underlying socket.""" + try: + self._connection.close() + except OSError: + pass + + +def _build_auth( + private_key: bytes, + ephemeral_private_key: bytes, + nonce: bytes, + remote_public_key: bytes, +) -> bytes: + """Return the encrypted auth message for the handshake initiator.""" + static_shared_secret = agree(private_key, remote_public_key) + signature = PrivateKey(ephemeral_private_key).sign_recoverable( + _xor(static_shared_secret, nonce) + ) + body = eth_rlp.encode( + [ + signature, + public_key_bytes(private_key), + nonce, + Uint(AUTH_VERSION), + ] + ) + # Random padding places the message in the EIP-8 encoding, which is + # the only auth format current clients accept. + body += os.urandom(100 + int.from_bytes(os.urandom(1), "big") % 100) + + encrypted_size = len(body) + 1 + 64 + 16 + 32 + prefix = encrypted_size.to_bytes(2, "big") + return prefix + encrypt(remote_public_key, body, prefix) + + +def _first_rlp_item(data: bytes) -> bytes: + """ + Return the leading RLP item of `data`, ignoring what follows it. + + Handshake messages carry random padding after their RLP body, so the + body has to be delimited by its own length rather than by the end of + the message. + """ + prefix = data[0] + if prefix < 0x80: + return data[:1] + if prefix <= 0xB7: + return data[: 1 + prefix - 0x80] + if prefix <= 0xBF: + header = 1 + (prefix - 0xB7) + length = int.from_bytes(data[1:header], "big") + return data[: header + length] + if prefix <= 0xF7: + return data[: 1 + prefix - 0xC0] + header = 1 + (prefix - 0xF7) + length = int.from_bytes(data[1:header], "big") + return data[: header + length] + + +def _parse_ack(private_key: bytes, message: bytes) -> Tuple[bytes, bytes]: + """Return the remote ephemeral public key and nonce from an ack.""" + body = decrypt(private_key, message[2:], message[:2]) + fields = eth_rlp.decode(_first_rlp_item(body)) + if not isinstance(fields, list) or len(fields) < 2: + raise RLPxError("malformed ack message") + ephemeral_public_key, remote_nonce = fields[0], fields[1] + if not isinstance(ephemeral_public_key, bytes) or not isinstance( + remote_nonce, bytes + ): + raise RLPxError("malformed ack message fields") + return ephemeral_public_key, remote_nonce + + +def connect( + host: str, + port: int, + remote_public_key: bytes, + private_key: bytes, + timeout: float = 30.0, +) -> RLPxSession: + """ + Dial `host`:`port` and complete the RLPx encryption handshake. + + `remote_public_key` is the 64 byte node identity taken from the + remote's enode URL. + """ + ephemeral_private_key = os.urandom(32) + nonce = os.urandom(32) + + connection = socket.create_connection((host, port), timeout=timeout) + connection.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) + + auth = _build_auth( + private_key, ephemeral_private_key, nonce, remote_public_key + ) + connection.sendall(auth) + + size_prefix = _recv_exactly(connection, 2) + ack = size_prefix + _recv_exactly( + connection, int.from_bytes(size_prefix, "big") + ) + remote_ephemeral_public_key, remote_nonce = _parse_ack(private_key, ack) + + ephemeral_key = agree(ephemeral_private_key, remote_ephemeral_public_key) + shared_secret = keccak256(ephemeral_key + keccak256(remote_nonce + nonce)) + aes_secret = keccak256(ephemeral_key + shared_secret) + mac_secret = keccak256(ephemeral_key + aes_secret) + + logger.debug("RLPx handshake complete with %s:%d", host, port) + connection.settimeout(FRAME_READ_TIMEOUT) + return RLPxSession( + connection, + aes_secret, + mac_secret, + egress_seed=_xor(mac_secret, remote_nonce) + auth, + ingress_seed=_xor(mac_secret, nonce) + ack, + ) diff --git a/packages/testing/src/execution_testing/devp2p/secp256k1.py b/packages/testing/src/execution_testing/devp2p/secp256k1.py new file mode 100644 index 00000000000..058392341df --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/secp256k1.py @@ -0,0 +1,83 @@ +""" +The secp256k1 operations required by the RLPx handshake. + +Transaction signing elsewhere in the framework only needs to sign and +recover, which `spec256k1` provides. RLPx additionally needs a raw +Diffie-Hellman agreement (a point multiplication by a foreign public +key), which it does not expose, so the group arithmetic lives here. +""" + +from typing import Tuple + +from spec256k1 import PrivateKey + +FIELD_PRIME = 2**256 - 2**32 - 977 +"""Prime of the field the curve is defined over.""" + +Point = Tuple[int, int] +"""An affine curve point. The point at infinity is `(0, 0)`.""" + + +def _add(left: Point, right: Point) -> Point: + """Return the sum of two affine curve points.""" + if left == (0, 0): + return right + if right == (0, 0): + return left + + left_x, left_y = left + right_x, right_y = right + + if left_x == right_x: + if (left_y + right_y) % FIELD_PRIME == 0: + return (0, 0) + slope = ( + 3 * left_x * left_x * pow(2 * left_y, FIELD_PRIME - 2, FIELD_PRIME) + ) % FIELD_PRIME + else: + slope = ( + (right_y - left_y) + * pow(right_x - left_x, FIELD_PRIME - 2, FIELD_PRIME) + ) % FIELD_PRIME + + sum_x = (slope * slope - left_x - right_x) % FIELD_PRIME + sum_y = (slope * (left_x - sum_x) - left_y) % FIELD_PRIME + return (sum_x, sum_y) + + +def _multiply(point: Point, scalar: int) -> Point: + """Return `scalar` times `point` by double-and-add.""" + result: Point = (0, 0) + addend = point + while scalar: + if scalar & 1: + result = _add(result, addend) + addend = _add(addend, addend) + scalar >>= 1 + return result + + +def public_key_bytes(private_key: bytes) -> bytes: + """ + Return the 64 byte uncompressed public key for `private_key`. + + The leading `0x04` tag of the SEC encoding is stripped: RLPx carries + public keys as bare coordinate pairs. + """ + return PrivateKey(private_key).public_key.format(compressed=False)[1:] + + +def agree(private_key: bytes, public_key: bytes) -> bytes: + """ + Return the x coordinate of `private_key` times `public_key`. + + This is the raw Diffie-Hellman agreement used to derive the RLPx + static and ephemeral shared secrets. `public_key` is the 64 byte + coordinate pair form. + """ + point = ( + int.from_bytes(public_key[:32], "big"), + int.from_bytes(public_key[32:], "big"), + ) + shared_x, _ = _multiply(point, int.from_bytes(private_key, "big")) + return shared_x.to_bytes(32, "big") diff --git a/packages/testing/src/execution_testing/devp2p/snappy.py b/packages/testing/src/execution_testing/devp2p/snappy.py new file mode 100644 index 00000000000..e4b6003366b --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/snappy.py @@ -0,0 +1,132 @@ +""" +Snappy raw-block codec for RLPx frame payloads. + +RLPx compresses every message payload after the base protocol handshake +with Snappy's raw block format when both peers advertise base protocol +version 5 or higher. Pure Python keeps the peer dependency-free; the +payloads are protocol messages of at most a few hundred kilobytes, so +codec speed is immaterial next to the network round trip. + +Compression emits the payload as literal elements without searching for +back-references, which is a valid Snappy encoding of any input: the +format's compression is optional, its framing is not. Decompression +implements the full format, because the remote end really compresses. + +Format reference: google/snappy `format_description.txt`. +""" + +_MAX_LITERAL_LENGTH = 1 << 16 +"""Payload bytes carried per literal element when compressing.""" + + +class SnappyError(Exception): + """Raised when a Snappy block cannot be decoded.""" + + +def _encode_length(length: int) -> bytes: + """Encode a payload length as the preamble's little-endian varint.""" + out = bytearray() + while length >= 0x80: + out.append((length & 0x7F) | 0x80) + length >>= 7 + out.append(length) + return bytes(out) + + +def _decode_length(data: bytes) -> tuple[int, int]: + """Return the preamble's payload length and the offset after it.""" + length = 0 + shift = 0 + for position, byte in enumerate(data): + length |= (byte & 0x7F) << shift + if byte < 0x80: + return length, position + 1 + shift += 7 + if shift > 31: + break + raise SnappyError("malformed length preamble") + + +def decompressed_length(data: bytes) -> int: + """ + Return the decompressed size a Snappy block claims. + + Read without decompressing anything, which lets a reader enforce + its size limit before allocating. + """ + return _decode_length(data)[0] + + +def compress(data: bytes) -> bytes: + """Encode `data` as a Snappy block of literal elements.""" + out = bytearray(_encode_length(len(data))) + for start in range(0, len(data), _MAX_LITERAL_LENGTH): + chunk = data[start : start + _MAX_LITERAL_LENGTH] + stored = len(chunk) - 1 + if stored < 60: + out.append(stored << 2) + elif stored < (1 << 8): + out.append(60 << 2) + out.append(stored) + else: + out.append(61 << 2) + out += stored.to_bytes(2, "little") + out += chunk + return bytes(out) + + +def decompress(data: bytes) -> bytes: + """Decode one Snappy block.""" + expected_length, position = _decode_length(data) + out = bytearray() + while position < len(data): + tag = data[position] + position += 1 + element_type = tag & 0b11 + if element_type == 0b00: # Literal. + size = tag >> 2 + if size >= 60: + extra = size - 59 + if position + extra > len(data): + raise SnappyError("truncated literal length") + size = int.from_bytes( + data[position : position + extra], "little" + ) + position += extra + size += 1 + if position + size > len(data): + raise SnappyError("truncated literal") + out += data[position : position + size] + position += size + continue + if element_type == 0b01: # Copy with a 1 byte offset. + if position >= len(data): + raise SnappyError("truncated copy offset") + size = ((tag >> 2) & 0b111) + 4 + offset = ((tag >> 5) << 8) | data[position] + position += 1 + elif element_type == 0b10: # Copy with a 2 byte offset. + if position + 2 > len(data): + raise SnappyError("truncated copy offset") + size = (tag >> 2) + 1 + offset = int.from_bytes(data[position : position + 2], "little") + position += 2 + else: # Copy with a 4 byte offset. + if position + 4 > len(data): + raise SnappyError("truncated copy offset") + size = (tag >> 2) + 1 + offset = int.from_bytes(data[position : position + 4], "little") + position += 4 + if offset == 0 or offset > len(out): + raise SnappyError("copy offset outside decoded output") + # A copy may overlap its own output (offset < size), which is + # how the format expresses run length encoding; appending byte + # by byte reproduces that semantic exactly. + for _ in range(size): + out.append(out[-offset]) + if len(out) != expected_length: + raise SnappyError( + f"decompressed to {len(out)} bytes, preamble claimed " + f"{expected_length}" + ) + return bytes(out) diff --git a/packages/testing/src/execution_testing/devp2p/tests/__init__.py b/packages/testing/src/execution_testing/devp2p/tests/__init__.py new file mode 100644 index 00000000000..9f974a3d0d4 --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/tests/__init__.py @@ -0,0 +1 @@ +"""Unit tests for the `execution_testing.devp2p` package.""" diff --git a/packages/testing/src/execution_testing/devp2p/tests/test_keccak.py b/packages/testing/src/execution_testing/devp2p/tests/test_keccak.py new file mode 100644 index 00000000000..bfa4c16aa96 --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/tests/test_keccak.py @@ -0,0 +1,78 @@ +""" +Tests for the incremental Keccak-256 behind the RLPx frame MACs. + +Two properties matter here and nothing else does. The hash has to be +Keccak-256 rather than SHA3-256, which differ only in a padding byte and +so agree on nothing - a session built on the wrong one fails its first +frame MAC. And `digest` has to leave the sponge absorbing, because the +MAC reads a tag out of it after every frame and then keeps going. +""" + +import pytest + +from ..keccak import Keccak256, keccak256 + +VECTORS = [ + ( + b"", + "c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470", + ), + ( + b"abc", + "4e03657aea45a94fc7d47ba826c8d667c0d1e6e33a64a036ec44f58fa12d6c45", + ), +] +"""Keccak-256 of the empty string and of `abc`. SHA3-256 of the same +inputs differs in every byte, so these pin the padding rule.""" + +RATE_BYTES = 136 +"""The Keccak-256 bitrate, where the sponge absorbs a block.""" + + +class TestDigest: + """The hash is Keccak-256, not SHA3-256.""" + + @pytest.mark.parametrize("data, expected", VECTORS) + def test_known_vector(self, data: bytes, expected: str) -> None: + """A published digest is reproduced.""" + assert keccak256(data).hex() == expected + + def test_constructor_absorbs(self) -> None: + """Data passed at construction is absorbed.""" + assert Keccak256(b"abc").digest() == keccak256(b"abc") + + +class TestIncrementalUse: + """A digest is a peek: it must not finalize the sponge.""" + + @pytest.mark.parametrize( + "chunks", + [ + [b"a", b"b", b"c"], + [b"x" * RATE_BYTES, b"y" * RATE_BYTES], + [b"x" * (RATE_BYTES - 1), b"y", b"z" * (RATE_BYTES + 1)], + [b"", b"abc", b""], + ], + ids=["tiny", "whole_blocks", "block_boundary", "empty_updates"], + ) + def test_digest_after_every_chunk(self, chunks: list[bytes]) -> None: + """Peeking after each chunk agrees with hashing the whole.""" + sponge = Keccak256() + absorbed = b"" + for chunk in chunks: + sponge.update(chunk) + absorbed += chunk + assert sponge.digest() == keccak256(absorbed) + + def test_repeated_digest_is_stable(self) -> None: + """Digesting twice without absorbing returns the same tag.""" + sponge = Keccak256(b"abc") + assert sponge.digest() == sponge.digest() + + def test_chunking_does_not_change_the_digest(self) -> None: + """A split update matches the same bytes absorbed at once.""" + data = bytes(range(256)) * 5 + split = Keccak256() + for start in range(0, len(data), 7): + split.update(data[start : start + 7]) + assert split.digest() == keccak256(data) diff --git a/packages/testing/src/execution_testing/devp2p/tests/test_peer.py b/packages/testing/src/execution_testing/devp2p/tests/test_peer.py new file mode 100644 index 00000000000..150c2b9d2a0 --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/tests/test_peer.py @@ -0,0 +1,237 @@ +""" +Tests for how the peer bounds the responses it serves. + +A response is bounded by serialized bytes, not only by a count of items. +Clients cap the size of every message they read - geth's `maxMessageSize` +is ten mebibytes, and exceeding it drops the peer rather than truncating +the message - so two multi-megabyte bodies have to travel as two +responses. EIP-7934 caps a block at eight mebibytes, which puts two of +them in one response over that cap, and blocks that large are exactly +what the `eip7934_block_rlp_limit` fixtures serve. +""" + +import socket +from dataclasses import dataclass, field +from typing import Any, Dict, List, Tuple, cast + +import pytest + +from ..peer import ( + BLOCK_BODIES, + MAX_BODIES_PER_RESPONSE, + SOFT_RESPONSE_LIMIT, + MockPeer, +) +from ..rlpx import MAX_FRAME_SIZE, RLPxError, RLPxSession + +LARGE_BODY_SIZE = 8 * 1024 * 1024 +"""A body the size EIP-7934 allows a block to reach.""" + + +def _hash(index: int) -> bytes: + """Return a distinct block hash for `index`.""" + return index.to_bytes(32, "big") + + +@dataclass +class _StubHead: + """The head fields the peer reads when a hash is unknown.""" + + number: int = 1 + block_hash: bytes = b"\xaa" * 32 + + +@dataclass +class _StubChain: + """A chain that only has to answer for its head.""" + + head: _StubHead = field(default_factory=_StubHead) + + +class _StubChains: + """The bodies a peer holds, keyed by block hash.""" + + def __init__(self, bodies: Dict[bytes, bytes]) -> None: + """Hold `bodies` and present a chain to report as current.""" + self._bodies = bodies + self.current = _StubChain() + + def install(self, chain: Any) -> None: + """Accept a new chain without changing the held bodies.""" + del chain + + def body_rlp_by_hash(self, block_hash: bytes) -> bytes | None: + """Return the body held under `block_hash`, if any.""" + return self._bodies.get(block_hash) + + +class _RecordingSession: + """A session that keeps what was written instead of sending it.""" + + def __init__(self) -> None: + """Start with nothing written.""" + self.messages: List[Tuple[int, bytes]] = [] + + def write_message(self, code: int, payload: bytes) -> None: + """Record one message.""" + self.messages.append((code, payload)) + + +class _FailingSession: + """A session whose socket dies on every write.""" + + def write_message(self, code: int, payload: bytes) -> None: + """Fail the way a closed socket fails `sendall`.""" + del code, payload + raise OSError(32, "Broken pipe") + + +def _serve(bodies: Dict[bytes, bytes], hashes: List[bytes]) -> int: + """ + Ask a peer holding `bodies` for `hashes`, and return how many it sent. + + The response is counted rather than decoded: these payloads reach + megabytes, and a pure Python RLP decode of one takes long enough to + dominate the suite. + """ + peer = MockPeer( + host="127.0.0.1", + port=30303, + remote_public_key=b"\x00" * 64, + private_key=b"\x01" * 32, + network_id=1, + ) + peer._chains = cast(Any, _StubChains(bodies)) + session = _RecordingSession() + peer._serve_bodies(cast(RLPxSession, session), 1, hashes) + + assert len(session.messages) == 1 + code, payload = session.messages[0] + assert code == BLOCK_BODIES + served = peer.statistics.bodies_served + # The response carries every served body and nothing else of size. + # Unknown hashes are skipped, so the served bodies are the first + # `served` known hashes of the request. + known = [block_hash for block_hash in hashes if block_hash in bodies] + expected = sum(len(bodies[block_hash]) for block_hash in known[:served]) + assert expected <= len(payload) <= expected + 16 + # Exactly the served hashes are recorded - the per-block coverage + # the consumer asserts - never an unknown or a bounded-out one. + assert peer.statistics.body_hashes_served == set(known[:served]) + return served + + +class TestBodyResponseSize: + """One response never carries more bytes than a client will read.""" + + def test_two_large_bodies_are_split(self) -> None: + """Two eight mebibyte bodies do not share one response.""" + bodies = { + _hash(1): b"\x00" * LARGE_BODY_SIZE, + _hash(2): b"\x00" * LARGE_BODY_SIZE, + } + assert _serve(bodies, [_hash(1), _hash(2)]) == 1 + + def test_one_oversized_body_is_still_served(self) -> None: + """A body larger than the limit is served rather than withheld.""" + body = b"\x00" * LARGE_BODY_SIZE + assert len(body) > SOFT_RESPONSE_LIMIT + assert _serve({_hash(1): body}, [_hash(1)]) == 1 + + def test_small_bodies_share_one_response(self) -> None: + """Bodies that fit are still batched, as they always were.""" + bodies = {_hash(index): bytes([index]) for index in range(1, 33)} + assert _serve(bodies, sorted(bodies)) == 32 + + def test_unknown_hash_is_skipped(self) -> None: + """An unheld hash is skipped; held bodies after it still serve.""" + bodies = {_hash(1): b"\x01", _hash(3): b"\x03"} + assert _serve(bodies, [_hash(1), _hash(2), _hash(3)]) == 2 + + def test_item_cap_still_applies(self) -> None: + """The count cap bounds a request for many tiny bodies.""" + wanted = MAX_BODIES_PER_RESPONSE + 10 + bodies = {_hash(index): b"\x01" for index in range(wanted)} + hashes = [_hash(index) for index in range(wanted)] + assert _serve(bodies, hashes) == MAX_BODIES_PER_RESPONSE + + def test_lifetime_service_survives_a_chain_switch(self) -> None: + """ + `body_hashes_ever_served` accumulates across `set_chain`. + + The per-test statistics reset with every chain switch, but the + reused client keeps every block it imported, so the evidence + that a body once traveled the wire must outlive the test that + made it travel: a later test declaring a byte-identical chain + syncs nothing, and its coverage check reads this set. + """ + peer = MockPeer( + host="127.0.0.1", + port=30303, + remote_public_key=b"\x00" * 64, + private_key=b"\x01" * 32, + network_id=1, + ) + peer._chains = cast(Any, _StubChains({_hash(1): b"\x01"})) + peer._serve_bodies( + cast(RLPxSession, _RecordingSession()), 1, [_hash(1)] + ) + assert peer.statistics.body_hashes_served == {_hash(1)} + + peer.set_chain(cast(Any, _StubChain())) + assert peer.statistics.body_hashes_served == set() + assert peer.body_hashes_ever_served == {_hash(1)} + + def test_failed_write_records_nothing_as_served(self) -> None: + """ + A response whose socket write raises is not service. + + `body_hashes_served` is the evidence behind the consumer's + per-block wire-coverage assertion, so a body that never left + the peer must not appear in it - and the transcript must not + claim it was served. The request itself is still counted: it + arrived, whatever became of the answer. + """ + peer = MockPeer( + host="127.0.0.1", + port=30303, + remote_public_key=b"\x00" * 64, + private_key=b"\x01" * 32, + network_id=1, + ) + peer._chains = cast(Any, _StubChains({_hash(1): b"\x01"})) + with pytest.raises(OSError): + peer._serve_bodies( + cast(RLPxSession, _FailingSession()), 1, [_hash(1)] + ) + assert peer.statistics.body_requests == 1 + assert peer.statistics.bodies_served == 0 + assert peer.statistics.body_hashes_served == set() + assert peer.body_hashes_ever_served == set() + assert not any("served" in line for line in peer.statistics.transcript) + + +class TestFrameSizeGuard: + """A frame too large to describe is refused, not silently mangled.""" + + def test_oversized_frame_is_refused(self) -> None: + """A frame at the three byte length ceiling raises.""" + + class _RefusingSocket: + """A socket that fails the test if it is ever written to.""" + + def sendall(self, data: bytes) -> None: + """Reject a write the size guard should have stopped.""" + del data + raise AssertionError("oversized frame reached the socket") + + session = RLPxSession( + cast(socket.socket, _RefusingSocket()), + aes_secret=b"\x02" * 32, + mac_secret=b"\x03" * 32, + egress_seed=b"\x04" * 32, + ingress_seed=b"\x05" * 32, + ) + # One byte for the message code brings the frame to the ceiling. + with pytest.raises(RLPxError, match="frame of"): + session.write_message(0x10, b"\x00" * (MAX_FRAME_SIZE - 1)) diff --git a/packages/testing/src/execution_testing/devp2p/tests/test_protocol.py b/packages/testing/src/execution_testing/devp2p/tests/test_protocol.py new file mode 100644 index 00000000000..5c666925f0e --- /dev/null +++ b/packages/testing/src/execution_testing/devp2p/tests/test_protocol.py @@ -0,0 +1,183 @@ +""" +Tests for the version-switchable eth protocol layer. + +The wire shapes asserted here are taken from the devp2p specification +(`caps/eth.md`) and cross-checked against geth's +`eth/protocols/eth/protocol.go`: the Status layout is shared by every +version since eth/69, eth/70 (EIP-7975) inserts a receipt offset into +GetReceipts, and eth/71 (EIP-8159) adds the block access list request +pair. +""" + +import ethereum_rlp as eth_rlp +import pytest +from ethereum_types.numeric import Uint + +from ..protocol import ( + ETH_PROTOCOLS, + GET_BLOCK_ACCESS_LISTS, + GET_RECEIPTS, + ProtocolError, + Status, + decode_get_block_access_lists, + decode_hello, + encode_hello, + highest_common_eth_version, +) + +GETH_LIKE_CAPABILITIES = [ + ("eth", 69), + ("eth", 70), + ("eth", 71), + ("eth", 72), + ("snap", 1), +] +"""What a current geth advertises (eth 69-72 plus snap/1).""" + + +def _decode_status(payload: bytes) -> tuple[int, int, bytes]: + """ + Return the version, network identifier and genesis hash sent. + + The production peer never decodes a Status - it ignores the one the + client sends back - so the encoder's inverse lives here, where it + pins what each version's codec writes. + """ + fields = eth_rlp.decode(payload) + assert isinstance(fields, list) and len(fields) >= 3 + return ( + int.from_bytes(bytes(fields[0]), "big"), + int.from_bytes(bytes(fields[1]), "big"), + bytes(fields[2]), + ) + + +class TestNegotiation: + """The RLPx rule: highest shared version of the shared capability.""" + + def test_auto_picks_highest_implemented(self) -> None: + """Advertising every implemented version negotiates eth/71.""" + assert ( + highest_common_eth_version( + tuple(ETH_PROTOCOLS), GETH_LIKE_CAPABILITIES + ) + == 71 + ) + + def test_pinned_version_wins_when_shared(self) -> None: + """A single advertised version negotiates exactly itself.""" + for version in ETH_PROTOCOLS: + assert ( + highest_common_eth_version((version,), GETH_LIKE_CAPABILITIES) + == version + ) + + def test_no_common_version_is_none(self) -> None: + """A client without any shared version cannot negotiate.""" + assert highest_common_eth_version((71,), [("eth", 69)]) is None + + def test_other_capabilities_are_ignored(self) -> None: + """A snap capability version never joins the eth negotiation.""" + assert highest_common_eth_version((69, 70, 71), [("snap", 71)]) is None + + +class TestHello: + """The advertised capability set is data.""" + + def test_hello_advertises_one_pair_per_version(self) -> None: + """Each version becomes its own ("eth", version) pair.""" + payload = encode_hello("peer/v0", b"\x01" * 64, (71, 69, 70)) + _, _, capabilities = decode_hello(payload) + assert capabilities == [("eth", 69), ("eth", 70), ("eth", 71)] + + def test_hello_single_version(self) -> None: + """Pinning a version advertises exactly that one.""" + payload = encode_hello("peer/v0", b"\x01" * 64, (70,)) + _, _, capabilities = decode_hello(payload) + assert capabilities == [("eth", 70)] + + +class TestStatus: + """One Status layout since eth/69; only the declared version moves.""" + + @pytest.mark.parametrize("version", sorted(ETH_PROTOCOLS)) + def test_status_declares_negotiated_version(self, version: int) -> None: + """The vsn field is the negotiated version, layout unchanged.""" + status = Status( + network_id=1, + genesis_hash=b"\xaa" * 32, + fork_activations=[], + earliest_block=0, + latest_block=7, + latest_block_hash=b"\xbb" * 32, + ) + encoded = ETH_PROTOCOLS[version].encode_status(status) + assert _decode_status(encoded) == (version, 1, b"\xaa" * 32) + # The layout is the seven-field eth/69 one for every version. + fields = eth_rlp.decode(encoded) + assert isinstance(fields, list) and len(fields) == 7 + + +class TestGetReceipts: + """eth/70 (EIP-7975) inserts the first-block receipt offset.""" + + def test_eth69_shape(self) -> None: + """[request-id, [hashes]] decodes without an offset.""" + payload = eth_rlp.encode([Uint(7), [b"\xcc" * 32]]) + request = ETH_PROTOCOLS[69].decode_get_receipts(payload) + assert request.request_id == 7 + assert request.block_hashes == [b"\xcc" * 32] + assert request.first_block_receipt_index is None + assert request.describe() == "receipts for 1 hashes" + + @pytest.mark.parametrize("version", [70, 71]) + def test_eth70_shape(self, version: int) -> None: + """[request-id, firstBlockReceiptIndex, [hashes]] from eth/70.""" + payload = eth_rlp.encode([Uint(7), Uint(3), [b"\xcc" * 32]]) + request = ETH_PROTOCOLS[version].decode_get_receipts(payload) + assert request.request_id == 7 + assert request.block_hashes == [b"\xcc" * 32] + assert request.first_block_receipt_index == 3 + assert request.describe() == "receipts for 1 hashes from receipt 3" + + def test_wrong_shape_for_version_is_loud(self) -> None: + """A request in the other version's shape is a protocol error.""" + eth70_shaped = eth_rlp.encode([Uint(7), Uint(3), [b"\xcc" * 32]]) + with pytest.raises(ProtocolError): + ETH_PROTOCOLS[69].decode_get_receipts(eth70_shaped) + eth69_shaped = eth_rlp.encode([Uint(7), [b"\xcc" * 32]]) + with pytest.raises(ProtocolError): + ETH_PROTOCOLS[70].decode_get_receipts(eth69_shaped) + + +class TestBlockAccessLists: + """eth/71 (EIP-8159) adds the request pair; the peer stays silent.""" + + def test_request_decodes(self) -> None: + """[request-id, [hashes]], the GetBlockBodies shape.""" + payload = eth_rlp.encode([Uint(9), [b"\xdd" * 32, b"\xee" * 32]]) + request_id, hashes = decode_get_block_access_lists(payload) + assert request_id == 9 + assert hashes == [b"\xdd" * 32, b"\xee" * 32] + + def test_only_eth71_defers_the_request(self) -> None: + """The silence is a per-version decision, not a global one.""" + for version, protocol in ETH_PROTOCOLS.items(): + expected = version >= 71 + assert ( + GET_BLOCK_ACCESS_LISTS in protocol.unanswered_requests + ) is expected + + +class TestRegistry: + """The registry is the single source of what the peer implements.""" + + def test_versions_match_keys(self) -> None: + """Each protocol object declares the version it is keyed by.""" + for version, protocol in ETH_PROTOCOLS.items(): + assert protocol.version == version + + def test_receipts_are_never_answered(self) -> None: + """The receipts rule holds on every implemented version.""" + for protocol in ETH_PROTOCOLS.values(): + assert protocol.unanswered_requests[GET_RECEIPTS] == "GetReceipts" diff --git a/packages/testing/src/execution_testing/fixtures/blockchain.py b/packages/testing/src/execution_testing/fixtures/blockchain.py index a7331378786..98f0363f1ad 100644 --- a/packages/testing/src/execution_testing/fixtures/blockchain.py +++ b/packages/testing/src/execution_testing/fixtures/blockchain.py @@ -937,6 +937,46 @@ class BlockchainEngineXFixture(BlockchainEngineFixtureCommon): ) """Engine API payloads for blockchain execution.""" + sync_payload: FixtureEngineNewPayload | None = None + """ + Framework-built empty payload appended above a fully valid chain's + head so that every payload in ``payloads`` is a wire-guaranteed + ancestor for sync-based consumers - the same out-of-chain + representation ``BlockchainEngineSyncFixture`` uses. + + Kept out of ``payloads`` because it is scaffolding, not test + content: ``payloads``, ``last_block_hash`` and the post state all + keep describing exactly the chain the test author wrote, and + consumers that replay payloads through the Engine API need not + know the field exists (this format ignores unknown fields, so + older readers skip it wholesale). A sync-based consumer announces + it instead of ``payloads[-1]`` and treats the chain as complete + when the client accepts it as head. + + ``None`` for chains that carry no appended block: a single + expected-invalid or Engine API-refused block gets a *prepended* + sync payload instead - in-chain as ``payloads[0]``, because there + the extra block is load-bearing ancestry every consumer must + replay - and every other ineligible chain is exactly the test's + own. + """ + + @property + def has_sync_payload(self) -> bool: + """ + Return whether the fixture carries a framework-injected sync + payload, appended (``sync_payload``) or prepended (a leading + ``payloads`` entry tagged with the sync phase). + + Derived from the payloads rather than stored, so the JSON + schema carries no separate switch. + """ + if self.sync_payload is not None: + return True + return ( + len(self.payloads) > 0 and self.payloads[0].phase == TestPhase.SYNC + ) + class BlockchainEngineStatefulFixture(BlockchainEngineFixtureCommon): """ diff --git a/packages/testing/src/execution_testing/specs/blockchain.py b/packages/testing/src/execution_testing/specs/blockchain.py index ba0f1f191fd..420917b54ab 100644 --- a/packages/testing/src/execution_testing/specs/blockchain.py +++ b/packages/testing/src/execution_testing/specs/blockchain.py @@ -782,9 +782,12 @@ def discard_fixture_format_by_marks( and "blockchain_test_only" in marker_names ): return True + engine_formats: List[FixtureFormat] = [ + BlockchainEngineFixture, + BlockchainEngineXFixture, + ] if ( - fixture_format - not in [BlockchainEngineFixture, BlockchainEngineXFixture] + fixture_format not in engine_formats and "blockchain_test_engine_only" in marker_names ): return True diff --git a/packages/testing/src/execution_testing/test_types/phase_manager.py b/packages/testing/src/execution_testing/test_types/phase_manager.py index cdd4fb182d5..22e14f77bfc 100644 --- a/packages/testing/src/execution_testing/test_types/phase_manager.py +++ b/packages/testing/src/execution_testing/test_types/phase_manager.py @@ -15,6 +15,16 @@ class TestPhase(str, Enum): # compatibility EXECUTION = "testing" CLEANUP = "cleanup" + SYNC = "sync" + """ + A framework-injected payload that exists only to make the chain + syncable, such as the empty block the filler adds to a test's + chain - appended above a valid chain's head, or prepended between + genesis and a single invalid block - so that sync-based consumers + can trigger a devp2p sync. Unlike ``SETUP``, a sync payload + prepares no state a test depends on; consumers that replay + payloads through the Engine API can treat it like any other block. + """ class TestPhaseManager: diff --git a/uv.lock b/uv.lock index c00273eacbf..f3776b9f3eb 100644 --- a/uv.lock +++ b/uv.lock @@ -1065,6 +1065,7 @@ dependencies = [ { name = "ckzg" }, { name = "click" }, { name = "colorlog" }, + { name = "cryptography" }, { name = "eth-abi" }, { name = "eth-remerkleable" }, { name = "ethereum-execution" }, @@ -1077,6 +1078,7 @@ dependencies = [ { name = "jinja2" }, { name = "joblib" }, { name = "platformdirs" }, + { name = "pycryptodome" }, { name = "pydantic" }, { name = "pyjwt" }, { name = "pytest" }, @@ -1119,6 +1121,7 @@ requires-dist = [ { name = "ckzg", specifier = ">=2.1.3,<3" }, { name = "click", specifier = ">=8.1.0,<9" }, { name = "colorlog", specifier = ">=6.7.0,<7" }, + { name = "cryptography", specifier = ">=45.0.1,<46" }, { name = "eth-abi", specifier = ">=5.2.0" }, { name = "eth-remerkleable", specifier = "==0.1.31" }, { name = "ethereum-execution", editable = "." }, @@ -1131,6 +1134,7 @@ requires-dist = [ { name = "jinja2", specifier = ">=3,<4" }, { name = "joblib", specifier = ">=1.4.2" }, { name = "platformdirs", specifier = ">=4.2,<5" }, + { name = "pycryptodome", specifier = ">=3.22,<4" }, { name = "pydantic", specifier = ">=2.12.3,<3" }, { name = "pyjwt", specifier = ">=2.3.0,<3" }, { name = "pytest", specifier = ">=8,<9" },