feat(test-consume): add consume wirex, a devp2p full-sync simulator - #3365
Open
danceratopz wants to merge 18 commits into
Open
feat(test-consume): add consume wirex, a devp2p full-sync simulator#3365danceratopz wants to merge 18 commits into
danceratopz wants to merge 18 commits into
Conversation
Add the three primitives that the RLPx transport needs and that no existing dependency provides ready-made: - An incremental Keccak-256 that can be digested and then updated again. RLPx reads a digest from its running frame MAC after every frame without finalizing it; hashlib cannot express that at all (it implements SHA3, a different padding rule), and pycryptodome can only when asked: `keccak.new(update_after_digest=True)` pads and squeezes a copy of the sponge and leaves the absorbing state free. pycryptodome is already a root dependency - the spec's own `ethereum.crypto.hash.keccak256` uses it - and its compiled Keccak absorbs at around 176 MiB/s; a naive pure-Python sponge measures roughly 880x slower and cannot keep pace with multi-megabyte block bodies, so the module docstring warns against ever swapping one in. The tests pin the two properties that matter and nothing else: that the hash is Keccak-256 rather than SHA3-256, which differ only in a padding byte and so agree on no input, and that a digest leaves the sponge absorbing. - A secp256k1 Diffie-Hellman agreement. spec256k1 signs and recovers but does not expose a raw point multiplication by a foreign public key. - The ECIES scheme that protects the auth and ack handshake messages.
Dial a node, complete the ECIES encryption handshake as the initiator, derive the session secrets, and frame subsequent messages under the running egress and ingress MACs. Handshake bodies are delimited by their own RLP length rather than by the end of the message, because the EIP-8 encoding appends random padding after the body. Frame writes are serialized by a session-owned lock: the peer's serving thread and the test thread announcing chains both write to one session, and the egress cipher and MAC are stateful, so an unserialized pair of writes would corrupt the running MAC and interleave frame bytes on the socket - a failure the remote could only attribute to a broken peer. Frame reads are timeout-safe by construction: `read_message` waits for a frame to begin under `select`, where timing out consumes nothing and the caller simply tries again, while the reads inside a frame run under a generous fixed socket timeout that ends the session when it fires. A partially read frame can never be resumed - the ingress cipher and MAC have already advanced over its start - so a mid-frame timeout must be fatal rather than retryable, and a retryable timeout must never begin a frame. `write_message` also refuses a frame at or above the 2^24 byte ceiling that a frame header's three byte length field can express, raising `RLPxError` naming the size instead of overflowing inside a serving thread, far from the caller that assembled the oversized message. No length check is needed on the read side, where three bytes cannot express an excessive size; a zero-length frame, which cannot carry a message code, is refused instead. The write guard's test lands with the peer's test suite once that exists.
Encode and decode the subset of messages a serving peer needs: the base protocol handshake and liveness messages, the eth status exchange with its EIP-2124 fork identifier, and the header, body and block range messages that carry chain data. Advertise base protocol version 4 so the connection stays uncompressed. Clients select Snappy from the version their peer announces, so this avoids a compression implementation without violating the protocol.
An Engine X fixture stores each block as the newPayload request that delivers it, which omits the header fields a client derives for itself. Serving those blocks to a peer means restoring them: the transactions and withdrawals tries, and the constants and commitments a post-merge header carries - through Amsterdam, the EIP-7685 sha256 requests hash of Prague/Osaka payloads and the block access list hash and slot number Amsterdam adds. The reconstruction is self checking. A rebuilt header is accepted only if it hashes to the block hash the payload already declares, so a fork that adds a header field this does not know about fails loudly instead of serving a subtly wrong chain. Validated offline against the full local EngineX corpus (57,859 valid payloads, Paris through Osaka/BPO2 including transition forks): zero refusals on valid payloads. The genesis body follows its header's shape: a genesis whose header commits to a withdrawals root is served with the empty withdrawals list alongside the empty transaction and ommer lists, so the body a client downloads validates against the header it already holds. `ServedChains` holds every chain installed on a connection, not just the newest, and can answer which chain a block hash belongs to. A client's downloader does not drop the chain it was syncing when a test ends - it keeps asking for those blocks while the next test runs - so the peer serving from this container can keep answering, as a real peer holding those blocks would.
Dial the client under test, complete both handshakes, and answer header and body requests from whichever fixture chains are installed. Installing a new chain is how one client serves every test of a pre-allocation group: each test is an independent chain from the same genesis. Chains already served stay served: a client's downloader does not drop the chain it was syncing when a test ends - it keeps asking for those blocks while the next test runs - so each request is answered from the chain the requested hash belongs to, as a real peer holding those blocks would. For the same reason a serving thread records into the statistics object it captured when the request arrived: a straggler answer for the previous test lands in the previous test's transcript instead of polluting - or, worse, satisfying the wire-coverage check of - the next one's. The peer is deliberately honest and never withholds, reorders or corrupts a response, so that a sync failure is a finding about the client or the fixture. Receipt requests are counted and left unanswered: a full syncing client derives receipts by executing the block, so a request means the client took a path this peer cannot serve, and that should be visible rather than papered over with an empty response. A body request that mixes unknown hashes with held ones is answered for everything held: the eth wire protocol lets a response skip unavailable entries, and ending the response at the first unknown hash would withhold blocks a syncing client is entitled to. Served bodies are recorded by hash as well as by count. The consumer's wire-coverage claim is per block - every block whose body cannot be derived from its header alone must have been downloaded from this peer - and an aggregate count cannot say which bodies traveled, only that some body did. Because that set is assertion evidence rather than a diagnostic, service is recorded only after the response's socket write returns: a `sendall` that raises mid-send must not leave statistics claiming the bodies reached the client. Requests are still counted on arrival, whatever becomes of the answer, and headers follow the same write-before-record order so the transcript never says "served" about a response that died on the socket. Body service is additionally accumulated for the peer's whole lifetime in `body_hashes_ever_served`, because the per-test set resets with each chain switch while the client it describes keeps every block it ever imported. Two tests of one group may declare byte-identical chains - valid chains carry no per-test salt - and the client re-syncs nothing for the second one; the lifetime set is the evidence that those bodies did travel the wire for this client, so the consumer's coverage check can require each body to cross the wire once per client rather than once per test. The lifetime set observes the same write-before-record rule, and a unit test pins each property: accumulation across `set_chain`, the per-test reset, and a failed write recording nothing in either set. Verified with `just static` and `just test-tests`.
Until now the peer advertised base protocol version 4 to keep the connection uncompressed, which clients honour but which is not what a production peer looks like. The peer now advertises version 5 and compresses every message payload after the Hello exchange whenever both sides advertise 5 or higher, exactly as the RLPx specification requires; a version 4 remote still gets the uncompressed connection. The codec is pure Python to keep the peer dependency free. The compressor emits the payload as Snappy literal elements, which is a valid encoding of any input - the format's compression is optional, its framing is not. The decompressor implements the full format, copies included, because the remote end really compresses. Verified in both directions against cramjam's reference codec, and end to end against geth 1.17.6: the multi-block beacon-root group syncs 5/5 over a compressed v5 connection with per-test timings unchanged. Readers guard the claimed decompressed size against the frame size limit before allocating anything.
A protocol version is now a value: EthProtocol objects own exactly what versions change (the version number, the Status codec's declared vsn, the GetReceipts shape, the deliberately unanswered request set), and ETH_PROTOCOLS enumerates what the peer implements. The peer advertises a configurable set of versions and stores the object RLPx negotiation selects; every test's transcript records the negotiated dialect. eth/70 (EIP-7975) changes only the receipts pair, which this peer never serves, so implementing it means decoding the offset-bearing request. eth/71 (EIP-8159) adds the block access list pair; a BAL carries post-state values a client could import instead of executing, so the receipts rule generalizes: counted, recorded, never answered. Wire shapes derived from the devp2p spec and cross-checked against geth's eth/protocols/eth/protocol.go.
The fill side gives every eligible engine_x chain one framework-built empty block, placed per test by the chain's statically declared structure: appended above a fully valid chain - out-of-chain, in a new optional `syncPayload` field, so `engineNewPayloads`, `lastblockhash` and the post state keep describing exactly the chain the test author wrote - and prepended in-chain (tagged with the `sync` test phase) below a single expected-invalid or Engine API-refused block, where the extra block is load-bearing ancestry every consumer must replay. Invalid multi-block chains carry no extra block at all. Declare the vocabulary this simulator needs to consume that corpus: the `sync` value on `TestPhase`, so prepend-class fixtures parse (the enum is strict - an unknown value would fail the whole corpus at load), and the `sync_payload` field with the `has_sync_payload` property on `BlockchainEngineXFixture`, so append-class fixtures surface their trailer instead of dropping it in the format's ignore-unknown-fields parse. Both hunks are byte-identical duplicates of the fill branch's declarations and drop out when this branch rebases onto it. The fill side's `sync_block` format classvar is deliberately not duplicated: it steers fill-time placement policy, which does not exist on this branch, and dead vocabulary would only invite drift. Verified with `just static` and `just test-tests`.
Add a simulator that makes a client fetch a test's blocks for itself instead of being handed them. It keeps the client topology consume enginex established - one client per pre-allocation group, reused across the group's tests - and changes only how the blocks arrive. The control plane and the data plane are separate. A post-merge client does not choose its own head, so the Engine API is used to name the sync target: one newPayload for the announced head and one forkchoiceUpdated naming it. Everything before that head is downloaded from the mock peer over devp2p and executed by the client's full sync path. The announced head is chosen by the fixture's chain class. Only blocks below the announced head are guaranteed to travel devp2p - the head's payload always arrives through the Engine API, and whether a client also re-fetches its body from a peer is an implementation choice measured clients answer both ways - so a fixture carrying an appended sync payload has that trailer announced instead of the test's own head, which makes every block the test author wrote an ancestor the client must fetch and execute through its sync pipeline, on every client, by chain structure rather than client courtesy. The trailer joins the served chain (an appended-class single-block test is a two-block chain here), the sync completes when the client reports the trailer as head, and a fixture without one announces its own head as before. Prepend-class fixtures keep announcing the test's own block: their extra block is in-chain ancestry below it. There is deliberately no rewind between tests. Every test's chain forks at the group's genesis, so announcing the new head is all a consensus client would ever do. A rewind-to-genesis forkchoice update would be a no-op on geth, which ignores backward updates, but it breaks clients that honour it: nethermind moves its head back to genesis while its persisted state stays at the previous chain's tip, which its BlockDownloader.ReceiptEdgeCase treats as a crash-recovery situation - it downloads receipts instead of executing blocks, and this peer serves no receipts. Verified against geth, ethrex, and nethermind dup-group smokes, 19/19 each. A sync is awaited by polling for the head block itself (eth_getBlockByHash), never by repeating the forkchoice update: in go-ethereum every update restarts the sync cycle, so polling faster than a cycle completes stops the sync from ever finishing. The announcement is re-sent only on a slow cadence, as a consensus client would. The devp2p leg is verified block by block, ancestors only: a client can derive an empty block body from its header alone (an empty transactions trie and an empty withdrawals root leave nothing to download), so every below-head block whose body has content must have had that body served by the peer, and the assertion names the blocks whose bodies did not travel. An aggregate served-bodies count would let one downloaded body vouch for a chain whose other bodies arrived some other way. The evidence is the peer's lifetime service record rather than the current test's: valid chains carry no per-test salt, so two tests of one group may declare byte-identical chains, and the reused client re-syncs nothing for the second - its blocks already traveled the wire during the first, which is logged when it happens. Head-body service stays visible in the per-test transcript without being asserted, so a client changing its shape shows up in logs instead of as a mystery. Chains still too short to put any block on the wire are skipped by default. The skip lives in the chain fixture, ahead of chain reconstruction and peer setup, so a skipped fixture costs neither; an appended sync payload counts toward the length, so the skip class is the fixtures whose chain the filler could not extend. The first dial of a new client runs under a ten second deadline rather than as a single attempt: Hive's readiness gate waits on the Engine API port only, and a freshly started client may open its RLPx listener a moment after it. The simulator's option group is registered with the filtered help so that `consume wirex --consume-help` lists the wirex options alongside the shared consume ones; the options exist only on this subcommand (the plugin is injected per command). Unit tests pin the contract per chain class: which payload is announced (trailer / own invalid head / own head), the served sequence and the length the skip is judged by (including that the author's payload list is never mutated), and the wire-coverage requirement (announced head exempt, derivable ancestor bodies exempt, a missing ancestor named by number, the prepend class vacuous by design). Verified with `just static` and `just test-tests`.
When a reused client is asked to sync a chain shorter than one it already synced, geth's beacon sync downloads the new headers and then never requests bodies, and the test times out. Equal or growing chain lengths have never shown the stall. --wirex-sort-by-chain-length orders the tests inside each pre-allocation group by ascending chain length, so the head number a client is asked to sync to never decreases over the client's lifetime. Test ordering inside a group is a runtime scheduling policy - fixtures declare no ordering dependency - so this belongs to the simulator, not the fill. The chain lengths are read from the fixture files at collection time because the fixture index does not record them; each file is parsed once. An appended sync payload counts toward a chain's length on both paths, loaded fixture and raw file, so the ordering agrees with the skip accounting: a valid single-block test plus its trailer sorts as the two-block chain the client will actually be asked to sync. Off by default so runs stay comparable with earlier measurements and the stall can still be reproduced for reporting.
A client may hang up on a peer at any time - nethermind sends Disconnect(0x00) mid-group when it deems the peer idle - and the mock peer connected only once per client, so a single drop failed every remaining test in the group: the next tests found an empty transcript and the chain announcement hit a broken pipe. A real peer redials, so the peer now exposes liveness and a reconnect, and the simulator's mock_peer fixture redials before announcing a chain on a dead session. Detecting death also needed two fixes in the message loop: a non-timeout OSError was swallowed by the read-timeout handler (socket timeouts are TimeoutError, itself an OSError, so the catches must be ordered), and a received Disconnect message left the loop running on a session the client had already abandoned. Observed in the nethermind full Cancun run: one mid-group drop failed the three longest fork-transition tests (two 60s timeouts and one BrokenPipeError at set_chain); all three pass on rerun.
Fixtures containing an intentionally invalid block were skipped
wholesale - 651 tests of the Cancun corpus with no sync-path
coverage. But rejection is observable over the production interfaces:
the peer serves the chain as-is, and once the ancestry has arrived
over devp2p the client must answer INVALID to `engine_newPayload` for
the head. Accepting an invalid chain fails the test. Only the fact of
rejection is asserted: the Engine API's validationError is free-form,
client-specific text and devp2p carries no reason at all, so matching
the fixture's specific exception over the wire is deliberately
deferred; the client's reason text is logged for debugging. Rejection
resolves via the same newPayload poll the valid flow uses to watch a
sync, so a rejection test costs ~0.5 s, not the 60 s sync timeout.
The corpus supports this cleanly: all 764 invalid fixtures of the
compensated Cancun refill are linear chains with the invalid payload
at the head, and all reconstruct - their blocks are semantically
invalid but hash-consistent, so they travel the wire like any other
block. The one unservable class is a payload whose declared hash does
not match its own header (`rlp_modifier`-corrupted): devp2p cannot
present a block whose hash differs from its header's keccak, so the
`chain` fixture skips exactly those, ahead of reconstruction and with
the hash mismatch in the skip reason - a deliberate refusal to
reconstruct must read as a skip, not a setup error.
A verdict is read from a client's database, not from an oracle, and
`engine_newPayload` answers about the instant it is asked. While a
chain is still arriving, both directions of that answer can be an
artifact, and the two rules below are what make the verdict
trustworthy; without them roughly one rejection test in 4,600 fails
spuriously, in whichever direction the timing falls.
- A VALID must hold to count as acceptance. geth has been observed
answering a well-formed VALID, latestValidHash and all, for a block
its own beacon backfill rejected fifteen milliseconds later, and
answering INVALID stably on every re-ask thereafter. A VALID therefore starts a timer instead of failing the
test, and only a verdict that survives ACCEPTANCE_HOLD_TIME is
believed - which costs that wait once per genuinely accepted chain
and nothing otherwise. The transient itself is a client-side
wrongness direction worth filing upstream: over this path a client
is asked to judge a head whose ancestry is arriving underneath it,
which no Engine API simulator does.
- A below-head target is judged over the Engine API alone. A
rejection target strictly below the reused client's head cannot
reach it over devp2p, because the sync machinery of geth-like
clients refuses to walk its head backwards: geth declines the
announcement outright ("chain reorged, tail: 3, head: 3, newHead:
2") and then re-fetches the same header and body once per
re-announcement without ever concluding. Equal-height targets (the
all-2-block common case) sync fine and stay on the wire; only the
strictly-below case has its valid ancestry handed over via
newPayload, and its head is never named in a forkchoice update at
all, since naming it only starts that unfinishable sync. The
hand-over precedes the announcement, because an idle client
executes each ancestor against its parent's state while a client
already syncing towards the head answers for the ancestor without
executing it - VALID, yet with the ancestor's state still
unavailable, so the head reports a missing parent for the full
timeout. newPayload is idempotent, so the delivery is repeated
ahead of every re-announcement and a client busy on one attempt
executes it on a later one; re-delivering after a stalled 60 s wait
was confirmed to make the head judgeable on the next call.
Two more behaviours close the reused-client cases, both learned from
the full Cancun mixed-group and big-group runs:
- Redial mid-test. Connection liveness is checked at each
re-announcement, in both the valid-sync and rejection wait loops,
so a client hanging up during a test's wait no longer strands it
peerless until the sync timeout.
- Invalid chains run last. Serving a chain with a bad block leaves a
client's beacon backfill in a failure state (geth logs "Beacon
backfilling failed: retrieved hash chain is invalid" per
rejection); running each group's rejection tests after its valid
tests means that state can poison nothing that follows on the
reused client. Folded into --wirex-sort-by-chain-length: valid
chains first, then invalid ones, each ascending by chain length.
A declared Engine API error is itself the rejection: six
transition-fork fixtures declare an `errorCode` on the head payload
(a pre-fork block carrying blob fields, or a post-fork block missing
them, violates the Engine API's payload rules for the fork), so the
client refuses `engine_newPayload` at the RPC layer with `-32602`
before any chain context matters. When the fixture declares an error
code and the client answers the head's newPayload with a JSON-RPC
error carrying that code, the refusal passes the test; a different
code fails it, exactly as consume engine treats the same fixtures,
and payloads without a declared error code still propagate RPC errors
as genuine failures. A head that declares an error code runs as a
rejection test even when every payload is semantically valid: the
declared refusal is the expected outcome, whichever layer delivers
it. Every `newPayload` of the head - the initial announcement, the
verdict poll, and each re-announcement - applies the same rule.
Verified against geth on the compensated Cancun refill: all 15
invalid_static_excess_blob_gas fixtures pass as rejections, two
formerly wrong v1 fixtures fail loudly with "Client accepted the
invalid chain", the 3 hash-inconsistent fixtures skip, the six
test_invalid_{pre,post}_fork_block_* fixtures pass in 3 s, and the
mixed-group (34/34 with and without ordering), big-group-slice (a
38-block sync followed by two below-head rejections, seconds instead
of two timeouts), and 19-test dup-group smokes pass. The two verdict
rules were measured on a 621-test rejection-heavy Osaka slice looped
at -n 6, ~116 below-head rejections per iteration: 4 spurious
failures in 30 iterations (18,630 tests) before, three of them a
transient VALID and one a 60 s no-verdict stall, and zero in 45
iterations (27,945 tests) after. The full fork_Osaka slice of the
until-Osaka corpus then ran 17,226 tests over 699 groups in 11:02
with 12 failures, all of them the EIP-7610 create-collision
divergence this simulator is supposed to keep reporting, plus the 22
devp2p-unrepresentable skips: no timeouts and no accepted chains.
The valid-sync wait watched for the head block and nothing else, so a client that executed the ancestry and refused the chain (the EIP-7610 divergence class) burned the full 60 s sync timeout per test and the failure carried no reason. The announcement resent every 3 s already contains the verdict: its newPayload response answers SYNCING while the ancestry travels, VALID once imported, and INVALID the moment the client has decided against the chain. Read it and fail immediately with the client's validationError - a refusal is a verdict, not a timeout. The forkchoiceUpdated no-poll rule is untouched: the announcement cadence is unchanged, only its previously discarded response is used.
'auto' (default) advertises every implemented version and negotiates the highest the client shares; an explicit version advertises exactly that one, so a client that does not speak it fails the handshake loudly - which is what probing a client's version matrix wants.
Ascending chain-length ordering inside pre-allocation groups (valid chains first, then invalid ones) is required for geth and nethermind - both stall permanently when a reused client is asked to sync a chain whose head is below one it already synced - and costs nothing on clients that tolerate shrinking heads. Every validated full-corpus run enables it, so the flag was pure foot-gun surface: a run without it silently opts into a known cross-client failure mode. Make it the default and replace the flag with --wirex-no-sort-by-chain-length, kept so the stalls stay reproducible for upstream reports and unordered comparison runs stay possible. The filtered help renderer only knew how to reconstruct store_true actions, so a store_false flag would have been shown expecting a value; teach it store_false alongside.
The poll sleeps between result reads - eth_getBlockByHash during a valid sync, the head's newPayload for a rejection - not between forkchoice updates; re-announcements ride their own slower cadence (--wirex-announce-interval), so polling faster cannot restart a client's sync cycle. Short chains sync in tens of milliseconds, so at the old 50 ms default the poller dominated the measured per-test cost on the corpus's all-2-block majority. Fix the option's help text to describe what the sleep actually paces.
The enginex and wirex conftests carried near-verbatim copies of the client lifecycle plumbing: the client fixture, the genesis_header fixture, the tracker wiring and the per-test Hive reporting hook, plus the group-counting half of the collection hook. The copies had already drifted in log lines, and any fix to the client lifecycle would have had to land twice. Move the four fixtures into multi_test_client, which both simulators already load and which owns the rest of the group-scoped plumbing (pre_alloc_group, client_genesis, environment), and extract the group counting into a helper next to the stash key it fills. Each simulator's conftest keeps only its own collection policy: enginex sorts largest group first, wirex additionally orders the tests inside each group.
Add wirex to the running-methods comparison table with a short section in running.md, and a dedicated Consume WireX page. The page states the intent up front: verify that clients can receive and propagate blocks over devp2p using the consensus test corpus. It is not intended to be a complete test of historical sync, and it intends to replace consume rlp for post-Merge forks. It then covers the control-plane/data-plane split and which block is announced per chain class, the per-group process as a sequence diagram, the honest-peer rules (receipts and block access lists counted but never served, chains stay served, byte-bounded responses, redial), rejection tests with the two rules that make their verdict trustworthy, their single skip class and the below-head Engine API hand-over, the default chain-length ordering, the per-class sync block in the fill's own notation (G -> T1..Tn -> S* appended for valid chains, G -> S -> T1* prepended for invalid singletons, bare invalid multi-block chains, with * marking the announced block), the ancestors-only wire-coverage check and its per-client cumulative evidence, and a comparison against consume rlp and consume sync.
danceratopz
added
C-feat
2 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #3365 +/- ##
================================================
Coverage 93.53% 93.53%
================================================
Files 624 624
Lines 37070 37074 +4
Branches 3394 3394
================================================
+ Hits 34675 34679 +4
Misses 1645 1645
Partials 750 750
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
What
Adds
consume wirex, a Hive simulator that makes the client under test full sync each test's chain from a deterministic mock devp2p peer, plus theexecution_testing.devp2plibrary the peer runs on (RLPx transport with ECIES handshake and Snappy compression, eth/69–71, Engine-payload-to-consensus-block reconstruction). Client topology isconsume enginex's, i.e., one client per pre-allocation group, reused across the group's tests.Test case blocks get played against a client via devp2p:
engine_newPayload+ oneengine_forkchoiceUpdated). That is the whole control plane.Which block gets announced is decided by the fixture's chain class (
Ggenesis,T₁…Tₙthe test's blocks,*the announced block,ᵢthe intentionally invalid block):G → T₁…Tₙ → S*ST₁…Tₙ, on every clientG → S → T₁*S;T₁is judged afterSarrivesG → T₁…Tₙᵢ*T₁…Tₙ₋₁Process diagram (one group, one test — from the docs page)
sequenceDiagram participant S as Simulator (pytest) participant E as Client: Engine API participant D as Client: devp2p participant P as MockPeer note over S,P: once per pre-allocation group S->>E: start client (group genesis + pre-alloc) S->>P: connect(first chain) P->>D: dial, RLPx auth/ack, Hello (eth/69-71, p2p v5) D-->>P: Hello (capabilities), highest common eth version wins, Snappy on P->>D: eth Status (fork id, earliest/latest, head hash) S->>E: eth_getBlockByNumber(0), verify genesis, once per client note over S,P: per test (chain forks at group genesis) S->>P: set_chain(chain), BlockRangeUpdate, old chains stay served S->>E: newPayload(announced head: syncPayload, or the chain's own) E-->>S: SYNCING (parent unknown, nothing executes) S->>E: fcU(head, safe=finalized=genesis) E-->>S: SYNCING D->>P: GetBlockHeaders P-->>D: headers D->>P: GetBlockBodies (non-empty bodies) P-->>D: bodies note over D: full sync executes every block (real EVM work) loop poll until synced or timeout (re-announce on a slow cadence) S->>E: eth_getBlockByHash(head) E-->>S: null ... then the block end S->>E: fcU(head, safe=finalized=genesis), confirming E-->>S: VALID S->>E: eth_getBlockByNumber(latest), verify head hash opt client hung up on the peer P->>D: redial + full handshake, chains stay served endWhy
After EngineX,
consume rlpis the dominant remaining cost in Hive execution testing (one client boot per fixture) and the least production-representative path in the suite: mainnet clients never run their offline importer. A post-merge client has exactly two production ways blocks enter it —engine_newPayloadand devp2p sync. EngineX covers the first; WireX covers the second, putting the whole corpus on production interfaces and opening the path to deprecatingconsume rlp. Measured against geth, WireX costs about 1.5× EngineX wall clock on a 54k-fixture corpus while executing every block through the client's real sync pipeline.The one protocol fact that shaped this PR: only blocks below the announced head must travel devp2p. The head's payload always arrives through the Engine API, and whether a client also re-fetches the head's body from a peer is an implementation choice — measured across six clients it splits 3–3 (geth, nethermind and besu fetch it; reth, erigon and ethrex execute it from the announcement, ethrex having switched sides between 2026-08-10 and 2026-08-11). Announcing the fill side's appended sync payload dissolves the problem: every block the test author wrote becomes an ancestor that must cross the wire, on every client, by chain structure rather than client courtesy — and the simulator's per-block wire-coverage check becomes enforceable client-agnostically.
Beyond replacing
consume rlp: WireX turns the consensus corpus into a reusable devp2p conformance workload. It already exercises sync-facing upgrades such aseth/71BAL exchange (geth requests block access lists over eth/71, is declined by design, and completes its full sync), and provides the foundation for extending the corpus to future receipt, history-sync, and transaction-propagation changes.Key design decisions
The control plane (engine API) and the data plane (mock peer) are separate. The Engine API names the sync target; devp2p carries the chain. There is deliberately no rewind between tests: every chain forks at the group's genesis, so announcing the new head is all a consensus client would ever do, and clients that honor a backward forkchoice update break under one (nethermind falls into a crash-recovery mode that downloads receipts instead of executing).
The corpus contract: per-class sync blocks, consumed here. The fill gives every eligible engine_x chain one framework-built empty block, placed by the chain's own structure (table above). This simulator announces the appended
syncPayloadwhen present — the served chain includes it, the sync completes when the client reports it as head, and it counts toward chain length for both the skip accounting and the chain-length ordering, so a valid single-block test is a two-block chain. Prepend-class fixtures announce the test's own block, exactly as before. Fixtures whose chain is still a single block skip here via--wirex-min-blocks(default 2) — honestly, since a single-block chain cannot trigger a devp2p sync no matter how it was filled.The peer is deliberately honest. It never withholds, reorders or corrupts a response, so a failure is a finding about the client or the fixture. Receipts — and, from eth/71, block access lists — are counted and left unanswered, never invented: a full-syncing client derives both by executing blocks, and serving them would convert real failures into silent no-coverage passes. Service is recorded only after the response's socket write returns, chains already served stay served, the peer redials when a client hangs up mid-group, and body responses are byte-bounded (2 MiB soft limit, matching geth's own).
Wire coverage is asserted per block, with per-client cumulative evidence. A valid test fails unless every below-head block whose body cannot be derived from its header alone was downloaded from this peer — the failure names the missing blocks. The announced head is exempt by protocol (its service stays visible in the transcript without being asserted, so a client changing its fetch shape shows up in logs, not as a false failure). The serving evidence is cumulative per client rather than per test: valid chains carry no per-test salt, so two tests of one group may declare byte-identical chains, and the reused client re-syncs nothing for the second one — its blocks already crossed the wire during the first, which the run logs when it happens (560 such tests in the fork_Osaka geth run).
Invalid chains run as rejection tests. The peer serves the chain as-is and the client passes by answering INVALID to the head's
newPayloadonce the ancestry has arrived — accepting the chain fails the test. Only the fact of rejection is asserted. One class skips explicitly: a payload whose declared hash mismatches its own header cannot travel devp2p at all. A below-head rejection target on a reused client has its valid ancestry handed over the Engine API before the head is announced (geth-like sync machinery refuses to walk its head backwards), and its head is never named in a forkchoice update.Tests are ordered inside each group, by default. Valid chains first, then invalid ones, each ascending by chain length (the appended sync payload counted): a reused client's head number must never decrease, and no valid sync should follow a served bad block.
The wire dialect is negotiated, not fixed. eth/69, 70 and 71 as
EthProtocolvalues owning exactly what versions change;--wirex-eth-versionpins the advertised set; every transcript records the negotiated dialect.A refusal is a verdict, not a timeout. An INVALID answer fails a valid test immediately with the client's
validationError; a VALID must hold half a second before it fails a rejection test (geth has been observed answering a transient VALID for a block its own backfill rejected fifteen milliseconds later).Validation
Full
fork_Osakarun against seven clients at eth/69 (same corpus, same flags) fixtures from: https://github.com/ethereum/execution-specs/actions/runs/31539180927test_bad_v_r_sanswered-32603 'Failed to recover the signer'instead of INVALID (deterministic strictness divergence) + 1 cascade-adjacent; the 187 errors are the known upstream sparse-trie SELFDESTRUCT panic poisoning its pre-alloc groupstest_invalid_blob_gas_used_in_header: besu refuses to decode the invalid payload (-32602 Invalid engine payload parameter, deterministic) where the other clients decode it and answer INVALID — an Engine API strictness divergence, rejection at the wrong layer per the fixture's declarationEIP-7610 stands geth+reth vs ethrex+nethermind+erigon+besu+nimbus-el (5-vs-2) on this corpus. Skips are 44 everywhere (22 single-block chains + 22 devp2p-unrepresentable), by design.
Follow-up tasks
--wirex-sync-timeout, so a client whose Engine API stops answering serializes its remaining tests at ~5 minutes each; the simulator's own timeout should bound every call.EthProtocolentry plus its message deltas).forkchoiceUpdatednaming an unknown head, never from a barenewPayload(spec-permitted), so rejection tests cannot elicit a verdict from it as designed. Proposed fallback: if after a grace period the head is still SYNCING and the peer transcript shows zero header requests, send one forkchoiceUpdated naming the invalid head, then resume the verdict poll. Self-triggering only on clients that do not sync from the announcement.Related Issues or PRs
Companion to the fill-side PR #3364 (per-class sync blocks for engine_x fixtures).
Checklist
just static<type>(<area>): <title>matchingC-*/A-*labels; the title matches the target squash commit message.Cute Animal Picture