Clarifications on builder spec - #165
Conversation
The slot in a signed request auth is the slot the validator will propose in, not the slot at which the request is signed or sent. Anchor the bid channel check to the slot path parameter, define the preferences channel semantics (the proposal slot the preferences apply to), and reject preferences for past slots so a replayed request cannot roll preferences back to a stale value.
data identifies the builder, not an API resource or a transport detail, so its canonical form is just the scheme and host: lowercased, with no port, path, query, fragment, or userinfo. The validator applies the rules before signing and the builder applies them to its own URL before the byte comparison, so both sides derive the same bytes.
…V1 so cached auths are easily reusable
05e1563 to
3dadb3a
Compare
|
From the SSV side, the proposal-slot clarification and reuse of one auth for the AOT and JIT channels are compatible with our pre-signing design. I think one signing-contract issue remains.
More importantly for SSV, normalizing inside verification authenticates a reconstructed message rather than the exact Could we instead canonicalize the configured builder URL before constructing This maps onto the existing spec structure: steps 1 and 2 belong in
|
Eth-Builder-Url header
|
5078eab decouples the request authorization data from the builder's URL. The motivation is that request authorizations require data unique to the builder. A builder's URL is just one instance of this, so rather than lock into one format, the idea is that the authorization data is just opaque bytes. These can be pre-negotiated between the builder and proposer, or very well just be the builder's URL. This change just makes it more flexible. Related to the conversation here ethereum/beacon-APIs/pull/625 |
- Eth-Builder-Url to submitSignedBeaconBlock - Specify clearly max_execution_payment in Gwei - additional clarity on what auth data SHOULD default to
|
Thanks @JasonVranek ! Very supportive of changes that:
|
| - [Builder Preferences](#builder-preferences) | ||
| - [`max_execution_payment`](#max_execution_payment) | ||
| - [Per-request Validator Inputs](#per-request-validator-inputs) | ||
| - [Routing through a proxy](#routing-through-a-proxy) |
There was a problem hiding this comment.
I think all explicit mentions to a proxy should be completely removed from the spec. This is out of protocol and unnecessary to be mentioned.
We just need to have two fields, one is the url to send the request, and another is arbitrary data. The latter is part of the signed auth, the former is not.
These two fields are already enough to enable proxying if people want to use that.
There was a problem hiding this comment.
Agreed, will remove explicit mentions of proxy across the PRs. The proxy was included because of the change to make auth data opaque bytes and not explicitly carry the builder's url. For a proxy to exist under this format, they must set url to their own and require data to carry the builder's URL to route the request. An easily fixable issue is that the constraint that per-key, URLs must be unique should be relaxed to (url, data) pairs should be unique. This would allow for example, a relay to receive N requests to different builders that carry real auth data.
For example the config would need to support:
{
"0xa057816155ad77931185101128655c0191bd0214c201ca48ed887f6c4c6adf334070efcd75140eada5ac83a92506dd7a": {
"fee_recipient": "0x50155530FCE8a85ec7055A5F8b2bE214B3DaeFd3",
"target_gas_limit": "45000000",
"builder": {
"enabled": true,
"builders": [
{
"url": "https://proxy.example.com",
"data": "https://builder-a.example.com",
"pubkey": "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a",
"max_execution_payment": "250000000",
"min_bid": "10000000",
"builder_boost_factor": "100"
}
],
"max_execution_payment": "500000000"
},
{
"url": "https://proxy.example.com", // this should be allowed (will update PRs)
"data": "https://builder-b.example.com",
...
}
}- the auth body on getExecutionPayloadBid is required and builders MUST verify it: signature failure is a 401, data/slot mismatch a 400, missing or malformed body a 400 -- the same auth story as submitBuilderPreferences; proposer duties are known an epoch ahead so auths pre-sign off the proposal hot path - delete the proxy section and the Eth-Builder-Url header from all endpoints; url and auth data are sufficient to support optional proxies - relax the no-preferences default: a served bid MUST honor the max_execution_payment cap from stored preferences, but without them the builder MAY serve any execution_payment; the proposer's locally configured per-builder limits are the backstop and discard bids that exceed them - data default convention: UTF-8 bytes of the builder's own advertised URL, hex pattern tightened to whole bytes - error surface aligned: 401 is signature failure only (InvalidAuth), 400 gains MissingBody, WrongBuilder renamed DataMismatch, 204 is any non-served bid; submitBuilderPreferences notes JSON/SSZ bodies need no Eth-Consensus-Version header (not fork-versioned) - unify the path parameter name to proposer_pubkey across both endpoints
| - Builder | ||
| parameters: | ||
| - name: validator_pubkey | ||
| - name: proposer_pubkey |
There was a problem hiding this comment.
related to ethereum/beacon-APIs#630 (comment), maybe there was some discussion around this but I don't really understand why we wanna sent a single request per pubkey, maybe someone can explain this
There was a problem hiding this comment.
I don't think I understood the question the first time. Is the assumption if a node operator has more than one proposer in the lookahead they can save on a call if batching was supported?
There was a problem hiding this comment.
yes, you only do a single call no matter how many validators are proposers in the lookahead, that matches how the proposer preferences api works
as far as I am aware all apis support batching, this one seems like an exception that decided to do a request per pubkey, but I don't see why it needs to be
to be clear, I don't feel strongly about this, on mainnet that will not make a large difference, but it would be kinda nice if the builder preferences work alongside the proposer preferences more closely unless there is a good reason to have a different api design
There was a problem hiding this comment.
after thinking about this more and reading up on the current spec, although I haven't found that mentioned as an explicit argument during the initial spec design, I believe it makes sense due to signed auth requests since we allow pubkeys to sign over different auth, so there can be partial failures, however per the spec here this should return a 401 which is communicated back via http status code which doesn't really allow reporting partial failures. It seems favorable to send individual requests per-pubkey to me.
builder spec endpoints
…prior" This reverts commit e27dfe3.
nflaig
left a comment
There was a problem hiding this comment.
generally looks good to me, just some comments and few self-reminders
| code: 400 | ||
| message: "Invalid SignedRequestAuthV1: auth.message.slot does not match the requested slot" | ||
| WrongBuilder: | ||
| message: "Invalid SignedRequestAuth: auth.message.slot has already passed" |
There was a problem hiding this comment.
I guess another reason for SlotMismatch, or actually the more accurate reason would be if the validator is not proposer of the given slot?
also compared to ProposerPreferences, the BuilderPreferencesRequest doesn't contain the dependent_root so in case the builder and proposer have a different view of the chain, it's not trivial for them to validate it
but this might be fine for this message since it's not propagated via gossip and builders can decide to keep BuilderPreferencesRequest even if it doesn't match their current view of the shuffling
| The `SignedRequestAuth` body is required. If the body is missing or | ||
| malformed, the request is invalid and the builder MUST return a 400 | ||
| response. If its `auth.message.data` or `auth.message.slot` does not match, | ||
| the builder MUST return a 400 response; if its signature fails to verify, | ||
| the builder MUST return a 401 response. Builders MAY use the authenticated | ||
| identity to apply per-validator policy (e.g. rate-limiting). |
There was a problem hiding this comment.
to be clear on this, so the CL client is expected to always send this, but a builder may choose to ignore it, right?
| - A [`SignedRequestAuth`][signed-request-auth] in the request body | ||
| authenticating the requesting validator. The body is required and MAY be | ||
| encoded as JSON (`Content-Type: application/json`) or SSZ | ||
| (`Content-Type: application/octet-stream`); `RequestAuth` is fork-versioned, | ||
| so the `Eth-Consensus-Version` header is required. | ||
| - A required `Date-Milliseconds` header with the Unix timestamp in milliseconds | ||
| at which the request was sent, and a required `X-Timeout-Ms` header with the | ||
| proposer's timeout for the request, measured from `Date-Milliseconds`. The | ||
| timestamp lets the builder estimate the transit delay from proposer to | ||
| builder; the builder MUST respond by `Date-Milliseconds` plus `X-Timeout-Ms`, | ||
| and the proposer discards later responses. | ||
|
|
||
| The `Eth-Consensus-Version` header is required on every request that carries a | ||
| body, naming the fork of the body's type. The builder MUST return a 400 response | ||
| when the header is absent or names a fork it does not recognize. |
There was a problem hiding this comment.
this whole section seems redundant to me, but fine with keeping it in case someone only looks at the .md file(s)
| [proposer-preferences]: https://github.com/ethereum/consensus-specs/blob/master/specs/gloas/p2p-interface.md#new-proposerpreferences | ||
| [proposer-preferences-topic]: https://github.com/ethereum/consensus-specs/blob/master/specs/gloas/p2p-interface.md#new-proposer_preferences |
There was a problem hiding this comment.
all of these links should probably be fixed later by using a tag/release instead of master as reference
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
| ``` | ||
|
|
||
| #### `SignedRequestAuthV1` | ||
| #### `SignedRequestAuth` |
There was a problem hiding this comment.
since we are renaming these, I think it would be good to use less generic names, what about the following
RequestAuth/SignedRequestAuth->BuilderRequestAuth/SignedBuilderRequestAuthDOMAIN_REQUEST_AUTH->DOMAIN_BUILDER_REQUEST_AUTHMAX_DATA_SIZE->MAX_BUILDER_AUTH_DATA_SIZEget_/verify_request_auth_signature->get_/verify_builder_request_auth_signature
when reading the builder-specs itself, the naming is fine, but in client code it's a bit too generic
cc @markolazic01 who brought this up
DOMAIN_BUILDER_REQUEST_AUTH - clarfications
| [`submitSignedBeaconBlock`][submit-signed-beacon-block] API call. The | ||
| validator does not call builders directly. |
There was a problem hiding this comment.
The validator does not call builders directly.
I don't know why we have such details here in the spec, in case of vero/vouch they might directly talk to builders, in any case, this note isn't harmful, I am mostly questioning this whole section here, not just this specific line
| def validate_bid( | ||
| state: BeaconState, | ||
| proposer_preferences: ProposerPreferences, | ||
| max_execution_payment: uint64, | ||
| signed_bid: SignedExecutionPayloadBid, | ||
| fee_recipient: ExecutionAddress, | ||
| ) -> bool: |
There was a problem hiding this comment.
min_bid is missing here, might be something we can add in a follow-up if desired, or it's fine as is since it's a local selection rule, and not related to bid validity
|
LGTM, thanks for the renames @nflaig @JasonVranek |
…rwarding (#630) The Beacon API side of the Gloas builder flow, aligned with [ethereum/builder-specs#165](ethereum/builder-specs#165) and [ethereum/keymanager-APIs#88](ethereum/keymanager-APIs#88) so the three specs describe one proposer flow. Supersedes #625. - `produceBlockV4` takes a required `BuilderConfig` body: a `builders` list with one `BuilderEntry` per builder-API bid request, plus a top-level `min_bid` and `builder_boost_factor` that apply to p2p bids. A bid is valued at `value + min(execution_payment, max_execution_payment)`, surviving bids are weighted by their `builder_boost_factor` and compared without division, and the local build's value is weighted by `100`. - `submitBuilderPreferences` (`POST /eth/v1/validator/builder_preferences`) lets the validator client push `BuilderPreferencesEntry` objects, batched across its proposers, so the beacon node can submit each builder's `max_execution_payment` cap ahead of the slot. - Block forwarding: `produceBlockV4` returns `Eth-Builder-Url` when a builder-API bid wins; `publishBlockV2` echoes it so the beacon node forwards the signed block to that builder via `submitSignedBeaconBlock`, letting it learn the result without waiting on gossip. - Both request bodies support JSON and SSZ and require `Eth-Consensus-Version`; builder failures are handled per entry so one unusable builder does not fail the block-production request or prevent other preference submissions. Reference implementation: [ChainSafe/lodestar#9832](ChainSafe/lodestar#9832)
Implement the Gloas builder API flow end to end so validators can source execution payload bids directly from builders post-ePBS, as specified in: - ethereum/builder-specs#138 - ethereum/builder-specs#165 - ethereum/beacon-APIs#630 - ethereum/keymanager-APIs#88 Builder configuration is owned by the validator client, the beacon node requests and validates bids on its behalf and ranks builder api and p2p bids uniformly during selection. - `produceBlockV4` is now a `POST` with a required `BuilderConfig` body, one bid request per builder entry plus top-level `min_bid` and `builder_boost_factor` governing p2p bids - the validator client signs and caches a `BuilderRequestAuth` for each proposal-slot/auth-data pair ahead of time; bid requests authenticate with the data agreed with the builder, defaulting to the UTF-8 bytes of the builder URL - the beacon node fans out `getExecutionPayloadBid` requests with a fixed 500ms deadline, validates bids against proposer preferences and the entry config with the execution payment counted up to the configured cap, and ranks bids by their boosted counted total. The p2p bid is selected after the same deadline so bids received over gossip have time to bid up - the signed block is forwarded via `Eth-Builder-Url` to the winning builder so it can help disseminate the block and promptly learn that its bid won; only previously authenticated builder clients are eligible and redirects are disabled - builder preferences are submitted ahead of proposals by the validator client and forwarded to each builder via the batched `submitBuilderPreferences` beacon api - keymanager `/eth/v1/validator/{pubkey}/builder_config` endpoints allow per-key configuration with per-builder overrides - new validator flags `--builder.urls` (auth data can be appended as a hex fragment), `--builder.minBid` and `--builder.maxExecutionPayment`, configuring a max execution payment above `0` requires the explicit `--allowDangerousTrustedPayments` opt-in as trusted payments are only backed by the builder's promise to pay - new `lodestar_builder_api_*` metrics - verified e2e against buildoor with kurtosis, both with the builder connected to the proposer's beacon node and to a separate node The beacon and builder openapi spec tests still ignore the new endpoints until builder-specs and beacon-APIs cut a release, keymanager is already checked against `v1.2.0-alpha.0`. Closes #6242 Closes #6602 Closes #9298 Closes #9918
Renames the Gloas builder request authentication containers, domain, and size constants to the names in ethereum/builder-specs#165 at commit 0bac9997b1dedad40910220883e63e0eb4f9aa63: RequestAuth -> BuilderRequestAuth SignedRequestAuth -> SignedBuilderRequestAuth DOMAIN_REQUEST_AUTH -> DOMAIN_BUILDER_REQUEST_AUTH MAX_DATA_SIZE -> MAX_BUILDER_AUTH_DATA_SIZE The rename is carried through the generated protobuf/SSZ code, the SignRequest.builder_request_auth keymanager oneof field, the HTTP structs, and all call sites; request_auth.go moves to builder_request_auth.go. Only the spec key is user-visible: /eth/v1/config/spec now reports DOMAIN_BUILDER_REQUEST_AUTH, and custom config YAML files must use the new key. The domain value (0x0B000001), the 4096-byte cap, JSON field names, SSZ encodings, and endpoints are all unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
**What type of PR is this?** > Other **What does this PR do? Why is it needed?** - ethereum/builder-specs#165 (comment) Builder request authentication types and containers are renamed. This PR matches with the latest `builder-specs` change, including - `RequestAuth` / `SignedRequestAuth` -> `BuilderRequestAuth` / `SignedBuilderRequestAuth` - `DOMAIN_REQUEST_AUTH` -> `DOMAIN_BUILDER_REQUEST_AUTH` Note that only user-facing change for this PR is "`DOMAIN_REQUEST_AUTH` -> `DOMAIN_BUILDER_REQUEST_AUTH`". Else, it is rather for readability and more conformant to the spec. **Which issue(s) does this PR fix?** N/A **Other notes for review** **Acknowledgements** - [x] I have read [CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md). - [x] I have included a uniquely named [changelog fragment file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd). - [x] I have added a description with sufficient context for reviewers to understand this PR. - [x] I have tested that my changes work as expected and I added a testing plan to the PR description (if applicable). Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
There have been some ambiguities when reading the builder specs around slot numbers, URL encodings, request auth signing/verifying, plus some broken links. This PR aims to clarify them.