Add multiverifier folding to the recursive tree prover#381
Add multiverifier folding to the recursive tree prover#381YairVaknin-starkware wants to merge 1 commit into
Conversation
PR SummaryMedium Risk Overview The workspace gains pinned
Reviewed by Cursor Bugbot for commit 270e1b8. Bugbot is set up for automated code reviews on this repo. Configure here. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #381 +/- ##
==========================================
- Coverage 66.52% 65.10% -1.43%
==========================================
Files 41 45 +4
Lines 5807 5834 +27
==========================================
- Hits 3863 3798 -65
- Misses 1944 2036 +92
🚀 New features to boost your workflow:
|
558da8a to
985cba7
Compare
50c33f4 to
dd68cc3
Compare
Gali-StarkWare
left a comment
There was a problem hiding this comment.
You have all the leaf prover stuff here, rebase it on Adar's branch
@Gali-StarkWare partially reviewed 7 files and made 1 comment.
Reviewable status: 5 of 19 files reviewed, all discussions resolved (waiting on OmriEshhar1).
6cced69 to
229036b
Compare
ed97add to
33675fe
Compare
229036b to
1b1121a
Compare
f9ad153 to
74b51d2
Compare
74b51d2 to
fb876d7
Compare
b7e0fd0 to
f61a696
Compare
0098157 to
efd1d3c
Compare
f61a696 to
9a12dd8
Compare
Gali-StarkWare
left a comment
There was a problem hiding this comment.
@Gali-StarkWare reviewed 10 files.
Reviewable status: 14 of 27 files reviewed, all discussions resolved (waiting on OmriEshhar1).
efd1d3c to
6aea293
Compare
OmriEshhar1
left a comment
There was a problem hiding this comment.
@OmriEshhar1 reviewed 20 files and all commit messages, and made 3 comments.
Reviewable status: 20 of 27 files reviewed, 2 unresolved discussions (waiting on Gali-StarkWare and YairVaknin-starkware).
crates/stwo_run_and_prove_recursive_tree/src/fold.rs line 90 at r8 (raw file):
/// Merkle channel (internal folds use the M31 channel the multiverifier circuit verifies) and /// serialized as the felt252 hex-string JSON stream `scarb execute --arguments-file` expects. A /// single-leaf tree performs no reduction at all, so its root proof stays the untouched leaf proof.
maybe move the comment about single-leaf tree to lib.rs.
if it's a single-leaf tree it will never call reduce_pair() so the comment is out of context here.
Code quote:
single-leaf tree performs no reduction at all,crates/stwo_run_and_prove_recursive_tree/src/lib.rs line 95 at r8 (raw file):
/// root-layer output files. pub fn stwo_run_and_prove_recursive_tree( config: RecursiveTreeConfig,
isn't it better to pass the function 4 arguments, instead of 1 'config' argument (which you unpack immediately anyway)?
Code quote:
config: RecursiveTreeConfigcrates/stwo_run_and_prove_recursive_tree/tests/cli_test.rs line 11 at r5 (raw file):
/// must exit non-zero (the `run_binary` error arm) rather than panicking or silently succeeding. #[test] fn invalid_program_input_exits_nonzero() {
test names should start with 'test_' (x2)
6aea293 to
894154a
Compare
OmriEshhar1
left a comment
There was a problem hiding this comment.
@OmriEshhar1 reviewed 8 files.
Reviewable status: 14 of 28 files reviewed, 2 unresolved discussions (waiting on Gali-StarkWare and YairVaknin-starkware).
OmriEshhar1
left a comment
There was a problem hiding this comment.
@OmriEshhar1 reviewed 14 files and all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on YairVaknin-starkware).
894154a to
34454b1
Compare
6205132 to
66e3666
Compare
OmriEshhar1
left a comment
There was a problem hiding this comment.
@OmriEshhar1 reviewed 21 files and all commit messages, and resolved 1 discussion.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on YairVaknin-starkware).
YairVaknin-starkware
left a comment
There was a problem hiding this comment.
@YairVaknin-starkware made 3 comments and resolved 1 discussion.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on YairVaknin-starkware).
crates/stwo_run_and_prove_recursive_tree/src/fold.rs line 90 at r8 (raw file):
Previously, OmriEshhar1 wrote…
maybe move the comment about single-leaf tree to lib.rs.
if it's a single-leaf tree it will never call reduce_pair() so the comment is out of context here.
Done
(dropped since the single-leaf behavior was already documented in the right place too)
crates/stwo_run_and_prove_recursive_tree/src/lib.rs line 95 at r8 (raw file):
Previously, OmriEshhar1 wrote…
isn't it better to pass the function 4 arguments, instead of 1 'config' argument (which you unpack immediately anyway)?
Done.
crates/stwo_run_and_prove_recursive_tree/tests/cli_test.rs line 11 at r5 (raw file):
Previously, OmriEshhar1 wrote…
test names should start with 'test_' (x2)
Done.
d169493 to
cf22973
Compare
Replace the per-pair Cairo simple-bootloader run with a circuit-world fold:
for each pair, build a `circuit_multiverifier` circuit that verifies both
child circuit proofs, prove it, and make the resulting proof the parent for
the next layer. The tree is balanced with odd-carry; there is no Cairo
bootloader or wrapping step anymore.
The tree is homogeneous: the leaf cairo-verifier circuit and the multiverifier
circuit are both padded to one shared `TARGET_PADDING_SIZES`, so they share a
preprocessed-trace layout and a single `SharedConfig` verifies every layer
(leaf proofs at layer 0, multiverifier proofs above).
Module layout:
- canonical: one-time `CanonicalCircuit` setup (circuit shape + shared config).
- fold: `LayerEntry` + `reduce_pair` (deserialize children, build/prove the
multiverifier, reserialize the parent proof in memory) plus the `PackedNode`
output tree — the circuit-world analogue of Cairo `PackedOutput`.
- leaf_io: leaves are `LeafInput`: the `leaf_proof_format::SerializedLeafProof`
`leaf_prover` produces (the shared output type — preprocessed root and the
serialized `Proof<QM31>` inline as base64) flattened together with the
bootloader-level context `leaf_prover` doesn't know about —
`output_preimage`, the raw preimage of the leaf bootloader's hashed output,
injected by the backend as a sibling JSON key. Loaded from a
`{"leaves": ["<path>", ...]}` manifest of per-leaf input files.
`LeafProofExt` adds the fold-side conversions. Each leaf declares its own
preprocessed root, so the fold is agnostic to the leaf's circuit type.
- output: root proof + flat circuit-output JSON + the nested `PackedNode` tree.
The wire formats carry no derivable data: `SerializedLeafProof` has no output
fields, so a leaf's circuit output words are recomputed from `output_preimage`
by replaying its hash chain out-of-circuit (`LeafInput::output_values`:
cairo0-encoded Blake2s to the Uint256 `H1`, then the circuit's Blake2s over
the two halves' 9-bit-limb encoding), and the packed tree carries only
`Composite` topology down to each leaf's `Plain(output_preimage)` reveal — a
future unpacker recomputes every digest bottom-up and checks one root value
against verified data. Output values ride each in-memory `LayerEntry` (leaf:
derived; internal: the fresh proof's claim, as raw u32 digest words) for the
next layer's multiverifier inputs.
The tree's final reduction is special: its proof is consumed by the Cairo
circuit verifier (as the applicative root task), not by another fold, so it is
proven with the standard lossless Blake2s Merkle channel (internal folds use
the M31 channel the multiverifier circuit verifies) and with
`min_lifting_log_size = 0` — the wire format omits the field and the Cairo
verifier mixes 0 into its channel — and written as the verifier's felt252
`--arguments-file` stream. A single-leaf tree performs no reduction, so its
root proof is the untouched leaf proof.
Intermediate proofs are held in memory (no scratch dir). Drops the Cairo
bootloader machinery and its `resources/`.
The circuit is TEMPORARILY configured to the privacy cairo-verifier fixture
(`circuit_multiverifier`'s `proof_cairo.bin`) so the `slow-tests` e2e can fold
a pre-generated leaf proof (duped 2/3/4x) without running `leaf_prover`; the
e2e builds its layer-0 entries directly since the privacy fixture's outputs
are not derivable by the leaf-bootloader chain. Wiring in `leaf_prover` as the
real leaf producer is the follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
66e3666 to
270e1b8
Compare
OmriEshhar1
left a comment
There was a problem hiding this comment.
@OmriEshhar1 reviewed 15 files and all commit messages.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on YairVaknin-starkware).
Gali-StarkWare
left a comment
There was a problem hiding this comment.
@Gali-StarkWare reviewed 16 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on YairVaknin-starkware).
Cargo.toml line 87 at r11 (raw file):
# stwo (used directly by the recursive-tree circuit folding; rev must match what the stwo-circuits # rev above depends on — see Cargo.lock).
Remove comment
Code quote:
# stwo (used directly by the recursive-tree circuit folding; rev must match what the stwo-circuits
# rev above depends on — see Cargo.lock).crates/stwo_run_and_prove_recursive_tree/Cargo.toml line 59 at r11 (raw file):
tracing-subscriber.workspace = true # Tests stage leaf-proof fixtures and collect outputs under a throwaway directory. tempfile.workspace = true
Same
Code quote:
# Gates the end-to-end fold test: it proves a real multiverifier circuit per pair (over a
# pre-generated leaf-proof fixture), too time/memory-heavy for the regular `tests` CI job.
slow-tests = []
[dependencies]
# Host Blake2s for recomputing a leaf's output digests from its output preimage.
blake2 = "0.10"
clap.workspace = true
leaf-proof-format.workspace = true
serde.workspace = true
serde_json.workspace = true
sonic-rs.workspace = true
# The Cairo `encode_felt252_data_and_calc_blake2s` felt-to-u32-words encoding.
starknet-types-core.workspace = true
stwo-cairo-common.workspace = true
stwo-cairo-utils.workspace = true
thiserror.workspace = true
tracing.workspace = true
# stwo-circuits crates, version-managed by the workspace so leaf proofs (de)serialize and verify
# against the exact same types they were produced with.
circuit-cairo-serialize.workspace = true
circuit-cairo-verifier.workspace = true
circuit-multiverifier.workspace = true
circuit-verifier.workspace = true
circuit-common = { workspace = true, features = ["prover"] }
circuit-prover.workspace = true
circuit-serialize.workspace = true
circuits.workspace = true
circuits-stark-verifier.workspace = true
stwo.workspace = true
[dev-dependencies]
# The e2e test (slow-tests) folds a pre-generated leaf circuit proof fixture; it does not run
# leaf_prover, so it needs no Cairo VM / bootloader dependencies.
tracing-subscriber.workspace = true
# Tests stage leaf-proof fixtures and collect outputs under a throwaway directory.
tempfile.workspace = true
Replace the per-pair Cairo simple-bootloader run with a circuit-world fold: for each pair, build a
circuit_multiverifiercircuit that verifies both child circuit proofs, prove it, and make the resulting proof the parent for the next layer. The tree is balanced with odd-carry; there is no Cairo bootloader or wrapping step anymore.The tree is homogeneous: the leaf cairo-verifier circuit and the multiverifier circuit are both padded to one shared
TARGET_PADDING_SIZES, so they share a preprocessed-trace layout and a singleSharedConfigverifies every layer (leaf proofs at layer 0, multiverifier proofs above).Module layout:
CanonicalCircuitsetup (circuit shape + shared config).LayerEntry+reduce_pair(deserialize children, build/prove the multiverifier, reserialize the parent proof in memory).LeafInputmanifest — inlineoutput_valuesandpreprocessed_rootplus a path to a pureProof<QM31>. Both are inline because neither is recoverable from the serialized proof; the per-leaf root also keeps the fold agnostic to the leaf's circuit type.PackedNodetree.Intermediate proofs are held in memory (no scratch dir). Drops the Cairo bootloader machinery and its
resources/.The circuit is TEMPORARILY configured to the privacy cairo-verifier fixture so the
slow-testse2e can fold a pre-generated leaf proof (duped 2/3/4x) without runningleaf_prover; wiring inleaf_proveras the real leaf producer is the follow-up.Type
Description
Breaking changes?
This change is