frametx-fuzz: fuzz frame transactions by coverage, not by verdict - #281
Open
pk910 wants to merge 14 commits into
Open
frametx-fuzz: fuzz frame transactions by coverage, not by verdict#281pk910 wants to merge 14 commits into
pk910 wants to merge 14 commits into
Conversation
Two gaps found by comparing against an independent implementation. The envelope is composable. EIP-8141's payload has seven fields with a scalar nonce; EIP-8250 replaces that nonce with nonce_keys and nonce_seq, and EIP-8272 appends recent_root_references. Each amends the base independently, so a chain may activate either, both or neither, and all four shapes occur. FrameTx encoded only the nine-field both-extensions layout, and go-ethereum's RLP struct decoder is strict about element count, so on a chain running anything else every frame transaction failed to decode. That reads as an indexing hole rather than a decoder mismatch, which is why testing against a devnet running all three hid it. The shape is now read off the payload: the field count separates three cases, and the ambiguous pair -- both eight fields -- differ in their second element, since nonce_keys is an RLP list where nonce is an integer. FrameTx carries a FrameExtensions set so re-encoding round-trips and callers can tell a chain without keyed nonces from a transaction that used key zero, which UsesLegacyNonce alone conflated. The signature hash, both gas formulas and the JSON codec all follow the selected shape; EIP-8272's per-reference intrinsic charge and its calldata pricing are applied. EIP-7906 adds POST_TX as frame mode 3: STATICCALL semantics with no APPROVE exception, constrained to a trailing suffix of the frame list, and its failure reverts the whole execution body rather than unwinding one atomic batch. Mode 3 is accepted, the suffix rule is enforced as the static property it is, and the mode is classified for display. The consequential part is DurableFrames. A frame receipt's status says whether the frame executed, not whether its effects lasted: an unrolled atomic batch and a failed POST_TX both discard the effects of frames that report success, and a consumer reading statuses alone reports durable successes where nothing survived. DurableFrames answers that from the transaction and its receipt.
The scenario built one envelope layout and had no shape exercising EIP-7906, so neither of the cases the decoder now handles could be generated as traffic. --envelope selects the payload shape: base, keyed, roots, full, or auto. Auto, the default, probes the chain at startup and reports what it found, which turns the shape from an assumption into an observation. Validating this against the current frames-testnet devnet showed it runs base EIP-8141, seven fields with a scalar nonce -- the shape that failed to decode before, confirming the gap was live rather than theoretical. Two POST_TX shapes: post-tx, whose assertion passes, and post-tx-revert, whose failure reverts the execution body while the user operation's own receipt still reports success. Both use the expiry verifier predeploy as their assertion contract, so they need nothing deployed: with an 8-byte future deadline it stops successfully, and with any other calldata length it reverts. A chain without EIP-7906 is detected at startup and those shapes fall back rather than producing a stream of rejections. The confirm log and the checks now cover durability as well as status. A frame can report success and still have had its effects discarded, so the scenario reports both and treats a body-revert expectation as its own assertion.
Envelope auto-detection classified a probe transaction by string-matching the client's error message. A client rejecting a wrong-sized envelope with different wording was read as "this shape decodes", locking the wrong shape and turning a loud startup failure into a per-transaction one -- the failure the probe existed to prevent. It was also only ever calibrated against one client's phrasing. All three EIPs install a predeploy at activation and require the address to be empty beforehand, so the account is an exact, client-independent signal: EXPIRY_VERIFIER for EIP-8141, NONCE_MANAGER for EIP-8250, RECENT_ROOT_ADDRESS for EIP-8272. Detection now reads code and nonce at those addresses. Nonce is checked alongside code because one of the codes is still TBD in its EIP, while both specify nonce 1 at activation. EIP-7906 installs no predeploy, so nothing in chain state says whether it is active. Rather than guess, the POST_TX shapes are excluded from "all" and selected by name; sending transactions that cannot succeed is worse than requiring one word of configuration. No error strings are parsed anywhere in the scenario now.
Adds what a consumer needs to work with the two EIPs that extend the frame
transaction envelope, all pure functions of the transaction fields:
EIP-8250 NonceManagerSlot, NonceKeysHash, WithNonceKeys, the first-use gas
constant, and the NONCE_MANAGER runtime code
EIP-8272 the source id, entry hash and storage key derivations, the 64-byte
write calldata, and the reference window rule
Sequences and committed roots live in protocol storage that no contract exposes
a read for, so deriving the slot is the only way to look either of them up.
Three further changes:
IndependentNonceTx lets a transaction say its nonce does not address the
sender's account nonce. A frame transaction on a non-zero EIP-8250 key set is
sequenced under NONCE_MANAGER and leaves the account nonce untouched, so
anything tracking transactions by nonce has to ask before assuming.
A frame's state gas limit is read under both spellings ethrex has shipped.
Reading only one decodes every frame from the other build with a state limit of
zero, and nothing says so: the transaction re-encodes to a different hash while
round-tripping against ourselves perfectly.
SignEntryP256 filled in the entry's signer after computing the digest, but the
signer is part of the canonical signature hash, so the entry was signed over a
hash the transaction no longer had. The existing test never verified the
signature, which is why it passed; it does now.
Frame support is a property of the chain, like the gas model and the block gas limit, so the txpool owns it: GetFrameSupportWithInit reports whether the chain implements EIP-8141 and which envelope extensions it activates, read from the predeploys each of the three EIPs installs at activation rather than inferred from a client's error text. A negative result is deliberately not cached. The predeploys appear at the fork and not at genesis, so a spammer started before activation reads an inactive chain and would otherwise stay stuck on that reading forever, encoding an envelope every client rejects. Transactions whose nonce is not the sender's account nonce are now tracked by hash instead. Sending an EIP-8250 keyed nonce transaction today would make the pool set the wallet's confirmed count from a sequence in an unrelated domain and close every nonce channel below it, breaking everything else that wallet sends. The hash-tracked path covers confirmation, the stale sweep and rebroadcast; the nonce path is untouched for every existing type. Also adds GetStorageAt, which is how protocol-managed storage is read, and splits frame transaction building into PrepareFrameTx and SignFrameTx. The canonical signature hash covers the sender, nonce and frame data, so a caller that needs the transaction's own contents before signing needs the two halves apart; BuildFrameTxWithSigners uses them to let a paymaster sign its own entry.
evm-fuzz's seeded generator moves to utils so that more than one fuzzer can draw from the same stream. evm-fuzz keeps its names as aliases and is otherwise unchanged.
Drops the scenario's own predeploy probing in favour of the txpool's, so the answer is shared and probed once. Behaviour is unchanged except that a chain which activates the fork mid-run is now picked up.
A new scenario that generates EIP-8141 frame transactions across every dimension the type and its extension EIPs define, and reports what it reached. It deliberately does not judge outcomes. Whether a frame should have failed, what an instruction should have returned, whether a shape ought to propagate -- each is a reading of a specification that is still moving, and a fuzzer that bakes in its author's reading turns a client disagreement into a false alarm about the client that disagreed. On a network of more than one node a real disagreement splits the chain by itself. So a run reports a coverage histogram, and a dimension with a zero count is the actionable result: it means the generator never got there. Generation is seeded per transaction index, as evm-fuzz is. Every draw happens in one pass producing a recipe with no chain state in it, so the seed and index regenerate it anywhere; the recipe is logged alongside for replay on a chain that has moved on. Frames that only address wallets never reach code, so none of the seven instructions EIP-8141 introduces would ever run. The scenario deploys a small geas contract whose calldata is a script, and whose introspection operations execute each instruction and discard the result. The same code plays paymaster and contract sender through an EIP-7702 delegation: EIP-8141's own answer is a deploy frame, but a CREATE2 account deployment costs about 224,000 gas against the 100,000 execution cap on the whole validation prefix. Malformed transactions are drawn from the same stream at a low rate rather than living in a separate mode, and are fired raw from a burner wallet: a transaction that never lands would otherwise stall its sender's nonce. Structural violations are applied before signing so the signature covers them, or every case would be refused for a bad signature instead of for the thing being exercised. Two features no predeploy announces -- EIP-7906 POST_TX frames and blob-carrying frame transactions -- are settled by sending a pair of transactions differing only in the feature. The signal is the accept or reject of the pair, never the text of an error. Validated against an ethrex eip8141-v2 devnet running all three EIPs: keyed nonce domains, recent root references and their window edges, P256 signature entries, contract senders and the introspection sweep all reach the chain.
This comment has been minimized.
This comment has been minimized.
A fixed contract can only execute the sequences someone wrote into it. The instructions EIP-8141 and its extensions introduce are interesting in combination -- with each other, with ordinary EVM, and at depth inside nested calls -- so the code is generated, by the same stack-aware generator evm-fuzz uses with the frame instructions added to its table. One frame deploys through the CREATE2 factory and a later frame calls the result. The address is a function of the code, so it is known before the transaction is sent and a frame can name a contract the transaction has not created yet; a recipe that calls generated code without deploying any promotes its first frame to the deployment, since leaving the pairing to chance almost never produces it. Frames also reach contracts earlier transactions deployed, which the run keeps in a bounded registry. evm-fuzz gains an AddOpcodes hook so a scenario can teach the generator instructions that only exist inside its own transaction type, and spamoor's deployment factory exposes its address for callers assembling their own deployment calls. Three fixes from watching a soak run against the devnet: Sponsorship spreads across every delegated wallet. The public mempool caps how many pending transactions one non-canonical paymaster may sponsor, so pointing every sponsored recipe at a single account refused everything behind the first few. The keyed nonce first-use budget is derived from the prefix rather than assumed. A transaction that also carried a P256 entry or a two-frame sponsored prefix tipped over MAX_VERIFY_GAS. Recent root edge cases are drawn at a fifth rather than being the common case. All but one of them are refused by design, so they dominated the stream and the plain reference path that has to keep landing barely ran.
This comment has been minimized.
This comment has been minimized.
A slice of the delegated wallets points at generated account code instead of the probe contract: fuzzed code ending in an APPROVE that reads its scope from calldata, so a validation frame runs arbitrary code before it can approve anything. This is the only thing that puts generated code inside the validation prefix, where EIP-8141's banned-opcode and storage rules apply and a public mempool node has to simulate what it finds. The same wallets serve both roles, as the transaction's sender and as the account a pay frame targets. Drawn rarely, out of the same band as the fixed contract sender: a fuzzed prologue often halts before it reaches the APPROVE, and such a transaction never lands, so the fixed contract keeps playing both roles most of the time. Validation-prefix frames running fuzzed code are budgeted well inside MAX_VERIFY_GAS rather than at the generated-code budget, which on its own exceeds the cap that covers the whole prefix.
SummaryAdds frametx-fuzz, a coverage-driven (not verdict-driven) fuzzer for EIP-8141 frame transactions plus the EIP-8250/8272/7906 extensions, moves the deterministic RNG into utils, and hoists frame-support detection into the txpool with hash-based confirmation tracking for keyed-nonce txs. The probe contract, seprated draw/render pipeline and sharing with evm-fuzz are well thought out; the two findings are both in the scenario's own EIP-8250 keyed-nonce bookkeeping, where the keyed sequence is mishandled in the account-nonce tracker. Issues
Reviewed @ |
Adds TxPool.WatchTransaction, a one-shot callback fired when a transaction with a given hash is included in a processed block. Block processing matches confirmations to wallets by their from-address, so a transaction from an address the pool does not track -- a keyless EIP-8141 contract sender, whose sender is a contract and whose transaction carries no signature -- would otherwise be invisible. The watch is keyed by hash, removed when it fires or when its context is cancelled, and gated by an atomic count so block processing skips it entirely when nothing is watching.
…ding Generated account contracts now play the sender and paymaster roles, and their funding is recovered rather than stranded. A contract sender is keyless: its sender is a contract that approves in its own code, so the transaction carries no signature and there is no wallet or nonce to manage. It is submitted raw and tracked by hash through TxPool.WatchTransaction, since the pool has no wallet to match its inclusion through. Each account is used once -- a created contract is at nonce one, and re-reading it per transaction is a live sync a spammer cannot afford -- so it is used at nonce one and retired. Every account contract carries a dispatch: a non-zero scope runs the fuzzed prologue and APPROVEs, a zero scope sends the contract's whole balance to CALLER. After a contract is used, or dropped from the ready ring unused, it is queued and later reclaimed by a SENDER-mode frame that calls it with an empty preamble, so CALLER is that transaction's sender and the funding returns to a tracked wallet. Account deployments are appended to ordinary transactions rather than drawn as body frames, so they add supply without displacing fuzz frames, and their rate follows how depleted the pool is: the buffer is topped up toward a target, harder when it is lower, so a transaction that needs a fresh sender and paymaster at once usually finds both.
Ten frame transaction spammers of increasing complexity, meant to be run one after another against a devnet to find the readiness stage at which a client breaks. Each rung adds one dimension over the last: 0 self-verify minimal self-signed frame transactions 1 transfers value transfers 2 expiry the expiry verifier predeploy 3 batches atomic batches and their rolled-back statuses 4 full static every enumerated shape, incl. the probe contract 5 fuzz structure the fuzzer: prefixes, batches, starved failures 6 fuzz code & sigs P256/witness signatures, generated code, introspection 7 fuzz extensions EIP-8250 keyed nonces and EIP-8272 recent roots 8 fuzz all every dimension, well-formed only 9 fuzz all + invalid the full fuzzer with a share of invalid transactions Every stage's config is validated against its scenario's option parser.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A fuzzer for EIP-8141 frame transactions and the EIPs stacked on them — EIP-8250 keyed nonces, EIP-8272 recent roots, EIP-7906 POST_TX.
Based on #280, so review that first; this PR's own diff is the five commits above it.
The design decision worth reviewing first
The fuzzer triggers combinations. It does not verify outcomes.
An earlier version carried an oracle: predicted per-frame statuses, predicted durability, in-EVM assertions comparing each introspection instruction against an expected value. It was built and then deleted, because the first live run produced five "findings" that were really one open question about whether a build charges the frame-entry access.
Every such prediction is a reading of a specification that is still moving, and a fuzzer that bakes in its author's reading turns a client disagreement into a false alarm about the client that disagreed. On a network of more than one node a real disagreement splits the chain by itself, which is a stronger signal than any comparison the generator could make.
So a run reports a coverage histogram. A dimension with a zero count is the actionable result — it means the generator never got there.
Rejection reasons are recorded per shape and compared with digit runs stripped, so a chain that starts refusing the same shape for a different reason is visible without anyone deciding which reason is right.
What's in it
Reproducibility. Seeded per transaction index like
evm-fuzz. Every draw happens in one pass producing a recipe with no chain state in it, so--payload-seed X --tx-id-offset N -c 1regenerates it on any chain; the recipe JSON is logged alongside for replay on a chain that has moved on.The probe contract. Frames that only address wallets never reach code, so none of the seven instructions EIP-8141 introduces would ever execute. A small geas contract whose calldata is a script — revert, sstore, log, burn,
APPROVE, and one read per introspection instruction. The reads discard their results on purpose. The same code plays paymaster and contract sender via an EIP-7702 delegation, because EIP-8141's own answer (a deploy frame) costs ~224k gas against the 100k execution cap on the whole validation prefix.Invalid combinations are drawn from the same stream at
--invalid-ratio(default 0.05) rather than living in a separate mode, and fired raw from a burner wallet — a transaction that never lands would otherwise stall its sender's nonce. Structural violations are applied before signing so the signature covers them.Capability probes. POST_TX and blobs are settled by sending a pair of transactions differing only in the feature. The signal is accept/reject of the pair, never error text.
Engine and txtypes prep
TxPool.GetFrameSupportWithInit— frame support as a chain property, next toIsAmsterdam. Negatives are re-probed on a throttle because the predeploys appear at the fork, not at genesis.IndependentNonceTxplus hash-keyed confirmation; the nonce path is untouched for every existing type. This is what makes EIP-8250 reachable at all — per dora's handover it had never been exercised by anything.txtypes,Client.GetStorageAt,PrepareFrameTx/SignFrameTx/BuildFrameTxWithSigners, andutils.DeterministicRNGshared withevm-fuzz.frametxis otherwise untouched: it stays a static list of shapes, and only loses its local predeploy probing.Bugs found on the way
SignEntryP256signed the wrong digest. It set the entry's signer after computing the digest, but the signer is part of the canonical signature hash, so the entry was signed over a hash the transaction no longer had. The existing test never verified the signature, which is why it passed. Fixed, and the test now verifies against the final sig-hash.stateLimitdecodes every frame from the other ethrex build with a state limit of zero, silently, and the transaction re-encodes to a different hash.expiry-not-firstandverify-after-prefixare mempool policy, not payload validity. Caught by a test asserting each violation is actually invalid, which is the guard that keeps a violation that stops being invalid from being reported as a client gap.Devnet validation
Against an ethrex
eip8141-v2devnet running all three EIPs (8141+8250+8272 envelope detected from the predeploys):scheme=0x2, 128 bytes, derived signer), and a corrupted-signature-with-intact-key case is refused, so the client verifies rather than merely parsesEvery refusal in a clean run was accounted for: deliberate root edge cases, deliberate violations, and one real mempool rule (a non-canonical paymaster's pending-sponsorship limit).
Two spec conflicts worth raising upstream
0xB5is assigned twice:SIGDATACOPYin EIP-8141 andRECENTROOTREFLOADin EIP-8272, whose justification note predates 8141 growingSIGDATACOPY.TXPARAM 0x0Clikewise: 8141'sstate_gas_leftagainst 8250's pre-state legacy nonce.The introspection sweep executes both of each, so a chain running both EIPs meets the collision.
One thing to weigh
The probe contract's interpreter test runs the compiled geas in go-ethereum's
core/vm/runtime, which adds 11 indirect deps togo.mod. Without it the hand-written assembly is untestable, since no local devnet runs EIP-8141. Happy to drop it if the dependency weight isn't worth it.