Skip to content

Partial data column containers for Gloas - #17348

Open
aarshkshah1992 wants to merge 18 commits into
alpha13from
feat/gloas-partial-column-container-changes
Open

Partial data column containers for Gloas #17348
aarshkshah1992 wants to merge 18 commits into
alpha13from
feat/gloas-partial-column-container-changes

Conversation

@aarshkshah1992

@aarshkshah1992 aarshkshah1992 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What

Stacks the Gloas partial data column container work on top of #17382.

Kurtosis Testing

Cherry-pick commit 9b36b8f on top of this branch which simulates private blobs/blobs missing in the EL.
Kurtosis config file:

Hey ! Ofcourse. Something like this works :

participants:
  - el_type: nethermind
    el_image: ethpandaops/nethermind:glamsterdam-devnet-8
    cl_type: prysm
    cl_image: prysm-bn-custom-image:latest
    vc_type: prysm
    vc_image: prysm-vc-custom-image:latest
    cl_log_level: "debug"
    count: 2
    validator_count: 128
    cl_extra_params:
      - --supernode
      - --subscribe-all-subnets
      - --partial-data-columns
      - --simulate-partial-el-blobs
    vc_extra_params:
      - --verbosity=debug
  - el_type: nethermind
    el_image: ethpandaops/nethermind:glamsterdam-devnet-8
    cl_type: prysm
    cl_image: prysm-bn-custom-image:latest
    vc_type: prysm
    vc_image: prysm-vc-custom-image:latest
    cl_log_level: "debug"
    count: 2
    validator_count: 4
    cl_extra_params:
      - --partial-data-columns
      - --simulate-partial-el-blobs
    vc_extra_params:
      - --verbosity=debug
  - el_type: nethermind
    el_image: ethpandaops/nethermind:glamsterdam-devnet-8
    cl_type: prysm
    cl_image: prysm-bn-custom-image:latest
    vc_image: prysm-vc-custom-image:latest
    count: 1
    validator_count: 0
    cl_extra_params:
      - --supernode
      - --partial-data-columns
      - --subscribe-all-subnets
    vc_extra_params:
      - --verbosity=debug
  # 2x Lighthouse supernodes for Gloas partial-columns interop, 4 validators each.
  # No simulate flag (Prysm-only); --enable-partial-columns is default-on for custom
  # networks on this branch, passed explicitly for clarity.
  - el_type: nethermind
    el_image: ethpandaops/nethermind:glamsterdam-devnet-8
    cl_type: lighthouse
    cl_image: ethpandaops/lighthouse:glamsterdam-devnet-8
    cl_log_level: "debug"
    count: 2
    validator_count: 4
    cl_extra_params:
      - --supernode
      - --subscribe-all-subnets
      - --enable-partial-columns

network_params:
  gloas_fork_epoch: 1
  preset: mainnet
  min_validator_withdrawability_delay: 1
  shard_committee_period: 1
  churn_limit_quotient: 16
  seconds_per_slot: 12
  gas_limit: 150000000

additional_services:
  - dora
  - spamoor

spamoor_params:
  image: ethpandaops/spamoor:master
  spammers:
    - scenario: eoatx
      config:
        throughput: 10
    - scenario: blobs
      config:
        throughput: 6

dora_params:
  image: ethpandaops/dora:master

global_log_level: debug

@aarshkshah1992 aarshkshah1992 changed the title Partial data column containers for Gloas (stacked on spectest-alpha-12-compliance) Partial data column containers for Gloas Aug 13, 2026
@aarshkshah1992
aarshkshah1992 force-pushed the feat/gloas-partial-column-container-changes branch from 8fc0aa2 to 86df8fd Compare August 13, 2026 15:59
@kasey
kasey force-pushed the spectest-alpha-12-compliance branch from 9a4debc to 3e89799 Compare August 14, 2026 20:23
@kasey
kasey requested a review from prestonvanloon as a code owner August 14, 2026 20:23
@aarshkshah1992
aarshkshah1992 force-pushed the feat/gloas-partial-column-container-changes branch from 86df8fd to bab7dc6 Compare August 19, 2026 07:32
@kasey
kasey force-pushed the spectest-alpha-12-compliance branch 10 times, most recently from c3c534c to ce6a7a9 Compare August 25, 2026 19:28
@kasey
kasey force-pushed the spectest-alpha-12-compliance branch 4 times, most recently from 2f9751a to c6049a2 Compare August 27, 2026 19:10
@kasey
kasey requested review from a team and kasey as code owners August 27, 2026 19:10
Base automatically changed from spectest-alpha-12-compliance to develop August 27, 2026 20:29
terencechain and others added 10 commits September 2, 2026 09:01
The alpha.14 parent_execution_payload vectors set latest_execution_payload_bid.slot without latest_block_header.slot, so the header slot only becomes usable here with the alpha.15 vectors (consensus-specs#5554).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Squash of the feat/gloas-partial-column-container-changes work (origin tip
bab7dc6, i.e. everything on top of the spectest-alpha-12 chain
3e89799, which is already in develop via #17310) rebased onto alpha13.

Merge notes vs alpha13:
- proposer_payload_envelope.go: partial columns ride along the cache-first
  publish path from #17391 (cachedDataColumns returns them too);
  sidecarsFromContents returns sidecars plus partial columns.
- two_node_test.go: the Gloas copy of TestTwoNodePartialColumnExchange gets
  the connect-early fix from #17361, same as the Fulu test.
- blockchain/testing/mock.go: only RecentBlockSlotErr is added.
- encoding/ssz/BUILD.bazel: stale fastssz test dep dropped by gazelle.
- proto/prysm/v1alpha1/gloas.yaml: partial column types re-added and SSZ
  regenerated with make gen (no drift).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The minimal.tar.gz asset of the v1.7.0-beta.0 consensus-specs release was
re-uploaded at 2026-09-04T02:24Z, after alpha13 pinned it, so the pinned
sha256 no longer matches (mainnet and general are unchanged). Belongs in
#17382; kept as a separate commit here so this branch builds and runs the
spectests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
broadcastDataColumnSidecars joined each column topic lazily inside the batch
full-sidecar path while the partial publish for the same column ran
concurrently. When the partial publish won the race on a topic the node is not
subscribed to, the pubsub router had no local topic state yet, MeshPeers
selected no peers, and the column was never sent. A full-node proposer hit
this on its first proposal after a fork: peers ended the slot holding exactly
the proposer's subscribed columns and the payload went empty network-wide.

Join every column topic up front, before dispatching either publish path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@aarshkshah1992
aarshkshah1992 force-pushed the feat/gloas-partial-column-container-changes branch from bab7dc6 to 6c384e3 Compare September 4, 2026 11:07
@aarshkshah1992
aarshkshah1992 changed the base branch from develop to alpha13 September 4, 2026 11:10

// Field 2: KzgProofs
dst = ssz.WriteOffset(dst, offset)
offset += len(c.KzgProofs) * 48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants