Integrate engine_getInclusionListV1 - #53
Closed
conache wants to merge 28 commits into
Closed
Conversation
conache
marked this pull request as ready for review
July 15, 2026 14:14
conache
force-pushed
the
support-get-il-v1
branch
from
July 24, 2026 09:28
dfd1ec9 to
60b7bf8
Compare
## Issue Addressed sigp#8922 ## Proposed Changes - Early payload envelopes (arriving before their slot is current) are now queued for reprocessing and retried at the start of their slot, instead of being dropped - Follows the same reprocess-queue pattern used by early blocks: deduplicated by block root and bounded in size. Gossip propagation is not affected - Includes unit tests for the queue behaviour and an e2e test covering the full requeue-and-retry flow
## Issue Addressed In many places we currently measure how long callers have to *wait* to acquire a fork choice lock, but we do not have any visibility into how long specific callers actually *hold* the lock for. ## Proposed Changes - Adds a wrapper around the fork choice lock with built in timing instrumentation. - Adds histograms for read/write lock hold times (including upgradeable). - Also adds a log indicating when and where a long-lasting lock was held, e.g. ``` DEBUG Fork choice lock held for a long time held=2.4s lock="write" acquired_from="beacon_node/beacon_chain/src/beacon_chain.rs:4322:47" ``` ## Additional Info There's a very small performance penalty for the lock, in the 100ns range which I believe is acceptable.
Add fork boilerplate for Heze Written by AI, but I did a self review Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
conache
force-pushed
the
support-get-il-v1
branch
from
July 28, 2026 08:20
60b7bf8 to
090c84f
Compare
An ENR advertising a `tcp`/`tcp6`/`quic`/`quic6` port of `0` produced an undialable `/ip4/…/tcp/0` (or `/udp/0/quic-v1`) address that was then attempted as a dial. This makes every ENR-derived dial path treat a zero port as absent, so no addressless record reaches the swarm. Found during a cross-client discv5 audit; the same class was fixed in Erigon's caplin sentinel (erigontech/erigon#22271). ### Changes **`common/network_utils/src/enr_ext.rs`** — the two dial-multiaddr builders skip zero ports. Renamed `multiaddr_tcp`/`multiaddr_quic` to `dialable_multiaddrs_tcp`/`dialable_multiaddrs_quic`, since they now return only dialable addresses rather than all of them. The unfiltered `multiaddr()` / `multiaddr_p2p*()` helpers are deliberately left alone — their remaining callers are HTTP API reporting and logging, which should show what the ENR actually says. **`beacon_node/lighthouse_network/src/peer_manager/mod.rs`** — `dial_peer` now rejects ENRs with no dialable address up front. Filtering at the discovery predicate alone is not enough, because not every ENR reaching the dial queue comes from a discovery query: - `dial_cached_enrs_in_subnet` filters cached ENRs by `subnet_predicate` alone, and cached ENRs also arrive via `Discovery::add_enr` (persisted DHT records, bootnodes), which never saw the query predicate. - `dial_trusted_peer` → `maintain_trusted_peers` runs every heartbeat, so an operator-added trusted ENR with no usable port was retried forever, each time producing a `ToSwarm::Dial` with an empty address list → `DialError::NoAddresses`. The "which addresses can we dial this on" logic was lifted out of the behaviour poll into a shared `PeerManager::dialable_multiaddrs`, so the guard and the actual dial cannot disagree. **`beacon_node/lighthouse_network/src/discovery/mod.rs`** — the query predicate now requires a dialable address instead of a merely present TCP port. It asks the canonical helpers rather than checking raw ports, because a raw check is family-blind: an ENR with `ip6` + `tcp6: 0` + `tcp4: 9000` and no `ip4` passes `tcp4().is_some_and(|p| p != 0)` while both helpers return empty. This also stops rejecting QUIC-only peers, which the old `tcp4().is_some() || tcp6().is_some()` filtered out even though the peer manager prefers QUIC when dialing; that arm is gated on `!disable_quic_support` so `--disable-quic` does not start admitting peers we have no transport for. **`beacon_node/lighthouse_network/src/service/mod.rs`** — bootnode dialing uses `dialable_multiaddrs_tcp()`, which made the manual `components[1]` / `MProtocol::Udp` filter dead code, so it is gone along with its unchecked index. Address-set-equivalent to before minus zero ports: `multiaddr()` emitted udp4/quic4/tcp4/udp6/quic6/tcp6 and the old filter dropped everything whose second protocol was `Udp` (both plain UDP and the `/udp/N/quic-v1` entries), leaving exactly tcp4 + tcp6. ### Tests Four tests in `enr_ext.rs` covering zero ports across tcp/tcp6/quic/quic6, and `test_dial_peer_skips_zero_port` covering the peer-manager guard. Each has a non-zero-port control so it cannot pass vacuously. The discovery predicate has no direct unit test — it is an inline closure in `start_query`, unreachable without extracting a helper, and the zero-port semantics it delegates to are covered by the `enr_ext.rs` tests. --- _Disclosure: this change — the finding, patch, and tests — was prepared with AI assistance and reviewed by me before submission._
## Issue Addressed We were not emitting sse events when processing attestations by batch. We also werent emitting sse events for aggregate attestations during batch processing. I've also deleted any pre-electra related logic since that is no longer supported
## Issue Addressed Every shuffling cache lookup currently takes an exclusive write lock, including cache hits. But practically 100% of lookups are cache hits. ## Proposed Changes Add `ShufflingCache::get_shuffling_if_cached`, a which serves cached shufflings, allowing us to avoid the write lock on cache hits. We check the cache with a read lock and only fallback to the write lock if the cache misses. ## Additional Info This also fixes the lock wait timer since the `let _` actually drops the timer immediately, so the metric used to always record 0
## Proposed Changes Use `ChainSpec::get_attestation_due` for regular VC attestation timers and fallback after a failed head-event request. This changes the deadline from 3,999 ms before Gloas to 3,000 ms from Gloas onward. Eager production on head events remains unchanged. ## Additional Info Aggregate attestation timing is unchanged. Verified with `cargo nextest run -p validator_services attestation_service::tests`, `cargo fmt --all -- --check`, `cargo check`, and scoped Clippy.
…umn_sidecar` event (sigp#9597) ## Issue Addressed sigp#9045 ## Proposed Changes - Add `head_v2` event - remove the fields `kzg_commitments` and `versioned_hashes` in `data_column_sidecar` The other new task for the events API that has yet to be implemented include: - ~~add `fast_confirmation` event, pending as it seems that Lighthouse still doesn't implement this~~ (update: implemented in sigp#8951 ) - `Removed parent_block_number field in payload_attributes event` is to be done after Gloas, as highlighted below, because current (Fulu) PayloadAttributes still has `parent_block_number` https://github.com/sigp/lighthouse/blob/81d576943835d443f43b6dda52769784ed82bd47/common/eth2/src/types.rs#L1157-L1159 AI-assisted, self-reviewed
sigp#9695) ## Issue Addressed Fixes sigp#9679 ## Proposed Changes - Add `PendingComponents::is_blob_data_available(num_expected_columns)` which is true when there are no blobs required, or when `num_completed_columns() >= num_expected_columns` - Add `PendingPayloadCache::is_blob_data_available(block_root)`: peeks pending components, computes expected columns via the existing custody helper, and delegates to the method above - Update `produce_payload_attestation_data` to use `pending_payload_cache.is_blob_data_available` instead of `fork_choice.is_payload_received` - `payload_present` is unchanged: still derived from `envelope_times_cache` vs `get_payload_due()` ## Tests - Existing: `gloas_payload_attestation_seen_but_data_unavailable` (envelope on time, columns incomplete). - New: `gloas_payload_attestation_blob_data_available_without_payload` (columns fully custodied, envelope withheld)
## Issue Addressed N/A ## Proposed Changes `make audit-CI` was failing. https://github.com/sigp/lighthouse/actions/runs/30526874910/job/90976616534?pr=9708 ~~Updated ruint.~~ Edit: ruint 1.20 requires msrv >=1.90. I think its safe to ignore the advisory for ruint 1.17 since we don't use the affected path. We should remove the ignore and update ruint once we update msrv.
## Issue Addressed This PR adds the main types and constants needed for FOCIL. ## Proposed Changes The changes included in this PR are purely additive, so no existing container should be modified. Main changes: - Add `InclusionList` and `SignedInclusionList` containers - Add `InclusionListCommitteeSize` to `EthSpec` typenum - Add Heze presets - Add `DOMAIN_INCLUSION_LIST_COMMITTEE` and FOCIL config values: `inclusion_list_due_bps`, `max_bytes_per_inclusion_list`, `max_request_inclusion_list`, `min_slots_for_inclusion_lists_requests`. ## Additional Info The following will be handled in follow-up PRs: - Add `inclusion_list_bits` on `ExecutionPayloadBid` - Swap of `InclusionList.transactions` to a `ProgressiveList`
## Issue Addressed sigp#8828 This implements the API but does not change the VC to leverage stateless block production. In a separate PR we should make updates to the VC
## Description
Implements the standard Gloas `POST /eth/v1/beacon/states/{state_id}/builders` endpoint from [ethereum/beacon-APIs#614](ethereum/beacon-APIs#614).
The endpoint follows the existing validator-state API patterns for state loading, ID and status filtering, response metadata, and error handling. Requests without a body or with empty filters return all builders, while pre-Gloas states return `400`.
## Additional Info
Builder-state requests use the P1 API queue, including requests for `head`, because listing the builder registry is not required for validator duties.
## Issue Addressed `LightClientUpdatesByRange` was treated as a non-streaming RPC protocol even though a request may produce multiple responses. After sending the first light client update, the inbound request was removed from the active request map. Subsequent updates for the same request were therefore rejected. Closes sigp#9619. ## Proposed Changes - Mark `LightClientUpdatesByRange` as a streaming protocol with the existing `ResponseTermination::LightClientUpdatesByRange` terminator. - Add a regression test covering the protocol's streaming semantics. ## Additional Info - `cargo fmt --all -- --check` - `cargo check` - `cargo test -p lighthouse_network --lib` (93 tests passed) Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
…p#9736) 🤖 automated (dapplion's agent): opened on dapplion's request after local testing. ## Issue Addressed Closes sigp#3108 Alternative to sigp#9488 and sigp#7802. ## Proposed Changes Instead of hand-rolling the counter update (sigp#9488) or resetting the counter on each scrape (sigp#7802, racy under concurrent scrapes), register the `prometheus` crate's built-in `ProcessCollector`, which exports the canonical process metrics with the correct types and reads `/proc` at gather time: - `process_cpu_seconds_total` (counter — fixes sigp#3108) - `process_resident_memory_bytes` - `process_virtual_memory_bytes` - `process_start_time_seconds`, `process_open_fds`, `process_max_fds`, `process_threads` (new) The hand-rolled `process_cpu_seconds_total`, `process_resident_memory_bytes` and `process_virtual_memory_bytes` gauges are removed; their names (and values) are unchanged, so existing dashboards keep working. `process_num_threads` and `process_shared_memory_bytes` are kept as the collector does not provide them. ## Additional Info - The collector is Linux-only (like the existing `ProcessHealth`-based metrics); registration is a no-op elsewhere, and the `process_collector` module is cfg-gated inside `prometheus` so non-Linux builds are unaffected. - The `process` feature pulls in `procfs 0.16` alongside the workspace's `procfs 0.18`; `deny.toml` allows multiple versions and `procfs` is not on the deny list. - Tested on a live mainnet BN: `process_cpu_seconds_total` is exported with `# TYPE ... counter` and increases monotonically across scrapes, including 4 concurrent scrapes.
## Issue Addressed The `SlotAssignment` cache can be leveraged for fork choice calculations (gloas `is_head_weak`). This PR moves the cache `CanonicalHead` so that it can be used outside of FCR Also found a potential bug in the cache? in `restore_from_store`, we load the head state via `get_advanced_hot_state` and w/ FCR enabled we rebuild FCR from it. We don't prime any of the caches when restoring from the store. If the committee cache was unbuilt FCR errors w/ `CommitteeCacheUninitialized` and we fail `restore_from_store`.
## Relevant Links - https://eips.ethereum.org/EIPS/eip-7688 - ethereum/consensus-specs#4630 ## Proposed Changes Adds the necessary types and components to support EIP-7688, along with the cargo patches to pull in the `progressive` versions of the crates in our SSZ stack. ## Testing ```yaml participants_matrix: el: - el_type: nethermind el_image: ethpandaops/nethermind:master cl: - cl_type: nimbus cl_image: ethpandaops/nimbus-eth2:glamsterdam-devnet-7-minimal - cl_type: lodestar cl_image: ethpandaops/lodestar:glamsterdam-devnet-7 - cl_type: lighthouse cl_image: ethpandaops/lighthouse:glamsterdam-devnet-7 - cl_type: teku cl_image: ethpandaops/teku:glamsterdam-devnet-7 ethereum_genesis_generator_params: image: "ethpandaops/ethereum-genesis-generator:glamsterdam-devnet-7" dora_params: image: ethpandaops/dora:glamsterdam-devnet-7 global_log_level: debug network_params: preset: minimal gloas_fork_epoch: 1 additional_services: - dora ``` ## Additional Info Based on the original work by @michaelsproul in sigp#8505 Co-authored-by: Michael Sproul <michael@sigmaprime.io>
…pport-get-il-v1
Author
|
Moved to sigp#9749 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Changes
Integrates the
engine_getInclusionListV1engine API method.Engine API specification of the method
Additional Info
GET /eth/v1/validator/inclusion_listBeacon API endpoints, which will be added in a follow-up PR.