Skip to content

Add multiverifier folding to the recursive tree prover#381

Open
YairVaknin-starkware wants to merge 1 commit into
mainfrom
yairv/add_multiverifier_support_for_recursive_tree_prover
Open

Add multiverifier folding to the recursive tree prover#381
YairVaknin-starkware wants to merge 1 commit into
mainfrom
yairv/add_multiverifier_support_for_recursive_tree_prover

Conversation

@YairVaknin-starkware

@YairVaknin-starkware YairVaknin-starkware commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Replace the per-pair Cairo simple-bootloader run with a circuit-world fold: for each pair, build a circuit_multiverifier circuit that verifies both child circuit proofs, prove it, and make the resulting proof the parent for the next layer. The tree is balanced with odd-carry; there is no Cairo bootloader or wrapping step anymore.

The tree is homogeneous: the leaf cairo-verifier circuit and the multiverifier circuit are both padded to one shared TARGET_PADDING_SIZES, so they share a preprocessed-trace layout and a single SharedConfig verifies every layer (leaf proofs at layer 0, multiverifier proofs above).

Module layout:

  • canonical: one-time CanonicalCircuit setup (circuit shape + shared config).
  • fold: LayerEntry + reduce_pair (deserialize children, build/prove the multiverifier, reserialize the parent proof in memory).
  • leaf_io: typed LeafInput manifest — inline output_values and preprocessed_root plus a path to a pure Proof<QM31>. Both are inline because neither is recoverable from the serialized proof; the per-leaf root also keeps the fold agnostic to the leaf's circuit type.
  • output: root proof + flat output-values JSON + nested PackedNode tree.

Intermediate proofs are held in memory (no scratch dir). Drops the Cairo bootloader machinery and its resources/.

The circuit is TEMPORARILY configured to the privacy cairo-verifier fixture so the slow-tests e2e can fold a pre-generated leaf proof (duped 2/3/4x) without running leaf_prover; wiring in leaf_prover as the real leaf producer is the follow-up.

Type

  • feature
  • bugfix
  • dev (no functional changes, no API changes)
  • fmt (formatting, renaming)
  • build
  • docs
  • testing

Description

Breaking changes?

  • yes
  • no

This change is Reviewable

@cursor

cursor Bot commented Jul 2, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Introduces a new end-to-end proving and circuit-verification path with pinned cryptographic dependencies; incorrect component/trace configuration would produce invalid proofs, though changes are largely additive with slow integration tests.

Overview
Adds a leaf-prover pipeline that runs a Cairo program, produces a Cairo STARK proof, builds and checks a cairo-verifier circuit over that proof, then proves that circuit and writes JSON via SerializedLeafProof (circuit_preprocessed_root + base64-encoded circuit proof). A small leaf-proof-format crate defines that wire type for downstream consumers (e.g. the recursive tree fold).

The workspace gains pinned stwo-circuits and stwo git dependencies (with a large Cargo.lock refresh, including Cairo 2.19.3), plus serde_with for base64 serialization. CI slow-tests now run leaf-prover with slow-tests alongside the existing proving crates.

prove_leaf still has a TODO to pad the verifier circuit to multiverifier size so leaf proofs align with the homogeneous tree layout described in the PR goal.

Reviewed by Cursor Bugbot for commit 270e1b8. Bugbot is set up for automated code reviews on this repo. Configure here.

@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.47909% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.10%. Comparing base (56b4a3a) to head (270e1b8).

Files with missing lines Patch % Lines
...ates/stwo_run_and_prove_recursive_tree/src/fold.rs 98.03% 2 Missing ⚠️
...stwo_run_and_prove_recursive_tree/src/canonical.rs 98.36% 1 Missing ⚠️
...ates/stwo_run_and_prove_recursive_tree/src/main.rs 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #381      +/-   ##
==========================================
- Coverage   66.52%   65.10%   -1.43%     
==========================================
  Files          41       45       +4     
  Lines        5807     5834      +27     
==========================================
- Hits         3863     3798      -65     
- Misses       1944     2036      +92     
Files with missing lines Coverage Δ
...s/stwo_run_and_prove_recursive_tree/src/leaf_io.rs 100.00% <100.00%> (ø)
...rates/stwo_run_and_prove_recursive_tree/src/lib.rs 96.55% <100.00%> (+4.21%) ⬆️
...es/stwo_run_and_prove_recursive_tree/src/output.rs 100.00% <100.00%> (ø)
...stwo_run_and_prove_recursive_tree/src/canonical.rs 98.36% <98.36%> (ø)
...ates/stwo_run_and_prove_recursive_tree/src/main.rs 88.88% <80.00%> (-7.41%) ⬇️
...ates/stwo_run_and_prove_recursive_tree/src/fold.rs 98.03% <98.03%> (ø)

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@YairVaknin-starkware
YairVaknin-starkware force-pushed the yairv/add_multiverifier_support_for_recursive_tree_prover branch from 558da8a to 985cba7 Compare July 2, 2026 14:31
@az-starkware
az-starkware force-pushed the adar/leaf_prover branch 14 times, most recently from 50c33f4 to dd68cc3 Compare July 6, 2026 14:13

@Gali-StarkWare Gali-StarkWare left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have all the leaf prover stuff here, rebase it on Adar's branch

@Gali-StarkWare partially reviewed 7 files and made 1 comment.
Reviewable status: 5 of 19 files reviewed, all discussions resolved (waiting on OmriEshhar1).

@YairVaknin-starkware
YairVaknin-starkware force-pushed the yairv/add_multiverifier_support_for_recursive_tree_prover branch 4 times, most recently from 6cced69 to 229036b Compare July 9, 2026 10:22
@az-starkware
az-starkware force-pushed the adar/leaf_prover branch 3 times, most recently from ed97add to 33675fe Compare July 9, 2026 11:27
@YairVaknin-starkware
YairVaknin-starkware force-pushed the yairv/add_multiverifier_support_for_recursive_tree_prover branch from 229036b to 1b1121a Compare July 9, 2026 12:50
@az-starkware
az-starkware force-pushed the adar/leaf_prover branch 2 times, most recently from f9ad153 to 74b51d2 Compare July 9, 2026 13:03
@YairVaknin-starkware
YairVaknin-starkware force-pushed the yairv/add_multiverifier_support_for_recursive_tree_prover branch 2 times, most recently from b7e0fd0 to f61a696 Compare July 12, 2026 10:05
@az-starkware
az-starkware force-pushed the adar/leaf_prover branch 3 times, most recently from 0098157 to efd1d3c Compare July 12, 2026 11:55
@YairVaknin-starkware
YairVaknin-starkware force-pushed the yairv/add_multiverifier_support_for_recursive_tree_prover branch from f61a696 to 9a12dd8 Compare July 12, 2026 13:35

@Gali-StarkWare Gali-StarkWare left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gali-StarkWare reviewed 10 files.
Reviewable status: 14 of 27 files reviewed, all discussions resolved (waiting on OmriEshhar1).

@OmriEshhar1 OmriEshhar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OmriEshhar1 reviewed 20 files and all commit messages, and made 3 comments.
Reviewable status: 20 of 27 files reviewed, 2 unresolved discussions (waiting on Gali-StarkWare and YairVaknin-starkware).


crates/stwo_run_and_prove_recursive_tree/src/fold.rs line 90 at r8 (raw file):

/// Merkle channel (internal folds use the M31 channel the multiverifier circuit verifies) and
/// serialized as the felt252 hex-string JSON stream `scarb execute --arguments-file` expects. A
/// single-leaf tree performs no reduction at all, so its root proof stays the untouched leaf proof.

maybe move the comment about single-leaf tree to lib.rs.
if it's a single-leaf tree it will never call reduce_pair() so the comment is out of context here.

Code quote:

single-leaf tree performs no reduction at all,

crates/stwo_run_and_prove_recursive_tree/src/lib.rs line 95 at r8 (raw file):

/// root-layer output files.
pub fn stwo_run_and_prove_recursive_tree(
    config: RecursiveTreeConfig,

isn't it better to pass the function 4 arguments, instead of 1 'config' argument (which you unpack immediately anyway)?

Code quote:

config: RecursiveTreeConfig

crates/stwo_run_and_prove_recursive_tree/tests/cli_test.rs line 11 at r5 (raw file):

/// must exit non-zero (the `run_binary` error arm) rather than panicking or silently succeeding.
#[test]
fn invalid_program_input_exits_nonzero() {

test names should start with 'test_' (x2)

@OmriEshhar1 OmriEshhar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OmriEshhar1 reviewed 8 files.
Reviewable status: 14 of 28 files reviewed, 2 unresolved discussions (waiting on Gali-StarkWare and YairVaknin-starkware).

@OmriEshhar1 OmriEshhar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OmriEshhar1 reviewed 14 files and all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on YairVaknin-starkware).

@YairVaknin-starkware
YairVaknin-starkware force-pushed the yairv/add_multiverifier_support_for_recursive_tree_prover branch 2 times, most recently from 6205132 to 66e3666 Compare July 13, 2026 14:06

@OmriEshhar1 OmriEshhar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OmriEshhar1 reviewed 21 files and all commit messages, and resolved 1 discussion.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on YairVaknin-starkware).

@YairVaknin-starkware YairVaknin-starkware left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@YairVaknin-starkware made 3 comments and resolved 1 discussion.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on YairVaknin-starkware).


crates/stwo_run_and_prove_recursive_tree/src/fold.rs line 90 at r8 (raw file):

Previously, OmriEshhar1 wrote…

maybe move the comment about single-leaf tree to lib.rs.
if it's a single-leaf tree it will never call reduce_pair() so the comment is out of context here.

Done
(dropped since the single-leaf behavior was already documented in the right place too)


crates/stwo_run_and_prove_recursive_tree/src/lib.rs line 95 at r8 (raw file):

Previously, OmriEshhar1 wrote…

isn't it better to pass the function 4 arguments, instead of 1 'config' argument (which you unpack immediately anyway)?

Done.


crates/stwo_run_and_prove_recursive_tree/tests/cli_test.rs line 11 at r5 (raw file):

Previously, OmriEshhar1 wrote…

test names should start with 'test_' (x2)

Done.

@az-starkware
az-starkware force-pushed the adar/leaf_prover branch 2 times, most recently from d169493 to cf22973 Compare July 16, 2026 12:36
Replace the per-pair Cairo simple-bootloader run with a circuit-world fold:
for each pair, build a `circuit_multiverifier` circuit that verifies both
child circuit proofs, prove it, and make the resulting proof the parent for
the next layer. The tree is balanced with odd-carry; there is no Cairo
bootloader or wrapping step anymore.

The tree is homogeneous: the leaf cairo-verifier circuit and the multiverifier
circuit are both padded to one shared `TARGET_PADDING_SIZES`, so they share a
preprocessed-trace layout and a single `SharedConfig` verifies every layer
(leaf proofs at layer 0, multiverifier proofs above).

Module layout:
- canonical: one-time `CanonicalCircuit` setup (circuit shape + shared config).
- fold: `LayerEntry` + `reduce_pair` (deserialize children, build/prove the
  multiverifier, reserialize the parent proof in memory) plus the `PackedNode`
  output tree — the circuit-world analogue of Cairo `PackedOutput`.
- leaf_io: leaves are `LeafInput`: the `leaf_proof_format::SerializedLeafProof`
  `leaf_prover` produces (the shared output type — preprocessed root and the
  serialized `Proof<QM31>` inline as base64) flattened together with the
  bootloader-level context `leaf_prover` doesn't know about —
  `output_preimage`, the raw preimage of the leaf bootloader's hashed output,
  injected by the backend as a sibling JSON key. Loaded from a
  `{"leaves": ["<path>", ...]}` manifest of per-leaf input files.
  `LeafProofExt` adds the fold-side conversions. Each leaf declares its own
  preprocessed root, so the fold is agnostic to the leaf's circuit type.
- output: root proof + flat circuit-output JSON + the nested `PackedNode` tree.

The wire formats carry no derivable data: `SerializedLeafProof` has no output
fields, so a leaf's circuit output words are recomputed from `output_preimage`
by replaying its hash chain out-of-circuit (`LeafInput::output_values`:
cairo0-encoded Blake2s to the Uint256 `H1`, then the circuit's Blake2s over
the two halves' 9-bit-limb encoding), and the packed tree carries only
`Composite` topology down to each leaf's `Plain(output_preimage)` reveal — a
future unpacker recomputes every digest bottom-up and checks one root value
against verified data. Output values ride each in-memory `LayerEntry` (leaf:
derived; internal: the fresh proof's claim, as raw u32 digest words) for the
next layer's multiverifier inputs.

The tree's final reduction is special: its proof is consumed by the Cairo
circuit verifier (as the applicative root task), not by another fold, so it is
proven with the standard lossless Blake2s Merkle channel (internal folds use
the M31 channel the multiverifier circuit verifies) and with
`min_lifting_log_size = 0` — the wire format omits the field and the Cairo
verifier mixes 0 into its channel — and written as the verifier's felt252
`--arguments-file` stream. A single-leaf tree performs no reduction, so its
root proof is the untouched leaf proof.

Intermediate proofs are held in memory (no scratch dir). Drops the Cairo
bootloader machinery and its `resources/`.

The circuit is TEMPORARILY configured to the privacy cairo-verifier fixture
(`circuit_multiverifier`'s `proof_cairo.bin`) so the `slow-tests` e2e can fold
a pre-generated leaf proof (duped 2/3/4x) without running `leaf_prover`; the
e2e builds its layer-0 entries directly since the privacy fixture's outputs
are not derivable by the leaf-bootloader chain. Wiring in `leaf_prover` as the
real leaf producer is the follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@YairVaknin-starkware
YairVaknin-starkware force-pushed the yairv/add_multiverifier_support_for_recursive_tree_prover branch from 66e3666 to 270e1b8 Compare July 16, 2026 21:20
@YairVaknin-starkware
YairVaknin-starkware changed the base branch from adar/leaf_prover to main July 17, 2026 08:04

@OmriEshhar1 OmriEshhar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OmriEshhar1 reviewed 15 files and all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on YairVaknin-starkware).

@Gali-StarkWare Gali-StarkWare left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gali-StarkWare reviewed 16 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on YairVaknin-starkware).


Cargo.toml line 87 at r11 (raw file):

# stwo (used directly by the recursive-tree circuit folding; rev must match what the stwo-circuits
# rev above depends on — see Cargo.lock).

Remove comment

Code quote:

# stwo (used directly by the recursive-tree circuit folding; rev must match what the stwo-circuits
# rev above depends on — see Cargo.lock).

crates/stwo_run_and_prove_recursive_tree/Cargo.toml line 59 at r11 (raw file):

tracing-subscriber.workspace = true
# Tests stage leaf-proof fixtures and collect outputs under a throwaway directory.
tempfile.workspace = true

Same

Code quote:

# Gates the end-to-end fold test: it proves a real multiverifier circuit per pair (over a
# pre-generated leaf-proof fixture), too time/memory-heavy for the regular `tests` CI job.
slow-tests = []

[dependencies]
# Host Blake2s for recomputing a leaf's output digests from its output preimage.
blake2 = "0.10"
clap.workspace = true
leaf-proof-format.workspace = true
serde.workspace = true
serde_json.workspace = true
sonic-rs.workspace = true
# The Cairo `encode_felt252_data_and_calc_blake2s` felt-to-u32-words encoding.
starknet-types-core.workspace = true
stwo-cairo-common.workspace = true
stwo-cairo-utils.workspace = true

thiserror.workspace = true
tracing.workspace = true

# stwo-circuits crates, version-managed by the workspace so leaf proofs (de)serialize and verify
# against the exact same types they were produced with.
circuit-cairo-serialize.workspace = true
circuit-cairo-verifier.workspace = true
circuit-multiverifier.workspace = true
circuit-verifier.workspace = true
circuit-common = { workspace = true, features = ["prover"] }
circuit-prover.workspace = true
circuit-serialize.workspace = true
circuits.workspace = true
circuits-stark-verifier.workspace = true
stwo.workspace = true

[dev-dependencies]
# The e2e test (slow-tests) folds a pre-generated leaf circuit proof fixture; it does not run
# leaf_prover, so it needs no Cairo VM / bootloader dependencies.
tracing-subscriber.workspace = true
# Tests stage leaf-proof fixtures and collect outputs under a throwaway directory.
tempfile.workspace = true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants