1.98 memory usage regression

[sveitser]

Overall memory requirement for compiling our espresso-node crate roughly doubles from 10GB to 20GB from 1.97.1 to 1.98. The issue was discovered because our CI runners have 16GB RAM and fail to build on 1.98

Example: https://github.com/EspressoSystems/espresso-network/actions/runs/32478615863/job/96760043575

Exit code: 143

LLM assisted bisection yielded #156417 as the cause. I can provide more of the LLMs outputs if useful.

Code

I tried this code:

git clone https://github.com/EspressoSystems/espresso-network
cd espresso-network
RUSTC_BOOTSTRAP=espresso_network cargo rustc -p espresso-node --lib -- -Ztime-passes

I expected to see this happen: compiles with roughly the same memory requirement as 1.97.1

$ RUSTC_BOOTSTRAP=espresso_network cargo rustc -p espresso-node --lib -- -Ztime-passes
..
time:   0.001; rss: 1372MB -> 1372MB (   +0MB)	monomorphization_collector_root_collections
time:  57.142; rss: 1372MB -> 3067MB (+1695MB)	monomorphization_collector_graph_walk
time:   3.256; rss: 3061MB -> 3344MB ( +283MB)	partition_and_assert_distinct_symbols
time:  71.973; rss: 1358MB -> 3291MB (+1933MB)	generate_crate_metadata
time:   0.000; rss: 3291MB -> 3291MB (   +0MB)	find_cgu_reuse
time:  16.406; rss: 3291MB -> 3838MB ( +547MB)	codegen_to_LLVM_IR
time:  16.450; rss: 3291MB -> 3838MB ( +547MB)	codegen_crate
time:   0.000; rss: 3839MB -> 3839MB (   +0MB)	incr_comp_persist_dep_graph
time:  14.824; rss: 3666MB -> 3834MB ( +169MB)	LLVM_passes
..

Instead, this happened: Uses significantly more memory on 1.98

$ RUSTC_BOOTSTRAP=espresso_network cargo rustc -p espresso-node --lib -- -Ztime-passes
..
time:   0.001; rss: 1345MB -> 1345MB (   +0MB)	monomorphization_collector_root_collections
time:  65.445; rss: 1345MB -> 14974MB (+13629MB)	monomorphization_collector_graph_walk
time:   3.033; rss: 14987MB -> 15276MB ( +289MB)	partition_and_assert_distinct_symbols
time:  78.328; rss: 1350MB -> 15220MB (+13870MB)	generate_crate_metadata
time:   0.000; rss: 15220MB -> 15220MB (   +0MB)	find_cgu_reuse
time:  16.254; rss: 15220MB -> 17343MB (+2123MB)	codegen_to_LLVM_IR
time:  16.256; rss: 15220MB -> 17344MB (+2124MB)	codegen_crate
time:   0.000; rss: 17344MB -> 17344MB (   +0MB)	serialize_dep_graph
time:  18.253; rss: 15418MB -> 6046MB (-9372MB)	LLVM_passes
..

Version it worked on

rustc 1.97.1 (8bab26f4f 2026-07-14)
binary: rustc
commit-hash: 8bab26f4f68e0e26f0bb7960be334d5b520ea452
commit-date: 2026-07-14
host: x86_64-unknown-linux-gnu
release: 1.97.1
LLVM version: 22.1.6

Version with regression

rustc 1.98.0 (88d9e12ae 2026-08-18)
binary: rustc
commit-hash: 88d9e12ae178fab0fb5cc050a94da85685d449ea
commit-date: 2026-08-18
host: x86_64-unknown-linux-gnu
release: 1.98.0
LLVM version: 22.1.8

Backtrace

no crash

[apiraino]

@sveitser the licensing of your project is a bit unclear:

[...] While we plan to adopt an open source license, we have not yet selected one. As such, all rights are reserved for the time being. Please reach out to us if you have thoughts on licensing.

I assume that this text is just not updated? Asking just to be sure we don't risk any license infringement by downloading the code and compiling to try to reproduce the issue

[sveitser]

@apiraino thanks for pointing out this issue with our lack of official license. We simply never got around to it but it's being worked on now. I just can't really say when the license will be available/decided.

In the meantime I think the below LLM generated snippet could serve as a repro instead. It triggers time and memory regressions at the same releases and nightly commits as the original code. It's inspired by the NodeType trait that is used everywhere in the code base.

$ RUSTC_BOOTSTRAP=1 cargo +1.97.1 rustc -- -Ztime-passes 2>&1 | grep collector_graph_walk
time:   1.557; rss:   65MB ->   68MB (   +3MB)	monomorphization_collector_graph_walk

$ RUSTC_BOOTSTRAP=1 cargo +1.98 rustc -- -Ztime-passes 2>&1 | grep collector_graph_walk
time:   6.785; rss:   68MB ->  369MB ( +301MB)	monomorphization_collector_graph_walk

$ cargo +bisector-ci-b3f7e3204640c7458c9f5096f3e9c5ad9971a51c-x86_64-unknown-linux-gnu rustc -- -Ztime-passes 2>&1 | grep collector_graph_walk
time:   1.869; rss:   66MB ->   70MB (   +4MB)	monomorphization_collector_graph_walk

$ cargo +bisector-ci-688ab44c7439a226d792990f81cd8df0603747d8-x86_64-unknown-linux-gnu rustc -- -Ztime-passes 2>&1 | grep collector_graph_walk
time:   4.076; rss:   66MB ->  368MB ( +303MB)	monomorphization_collector_graph_walk

#![allow(dead_code)]

pub trait Bound {}
pub trait Extra {}
impl<T: Bound> Extra for T {}

pub struct Pair<A, B>(A, B);
impl<A: Bound, B: Bound> Bound for Pair<A, B> {}

pub struct P<const I: usize, const S: usize>;
impl<const I: usize, const S: usize> Bound for P<I, S> {}

pub struct Node<const I: usize>;

pub struct W8<A, B, C, D, E, F, G, H>(A, B, C, D, E, F, G, H);
pub type L1<T> = W8<T, T, T, T, T, T, T, T>;
pub type L2<T> = W8<L1<T>, L1<T>, L1<T>, L1<T>, L1<T>, L1<T>, L1<T>, L1<T>>;
pub type L3<T> = W8<L2<T>, L2<T>, L2<T>, L2<T>, L2<T>, L2<T>, L2<T>, L2<T>>;
pub type L4<T> = W8<L3<T>, L3<T>, L3<T>, L3<T>, L3<T>, L3<T>, L3<T>, L3<T>>;
pub type L5<T> = W8<L4<T>, L4<T>, L4<T>, L4<T>, L4<T>, L4<T>, L4<T>, L4<T>>;

pub trait NodeType: 'static {
    type A0: Bound + Extra + Send + Sync;
    type A1: Bound + Extra + Send + Sync;
    type A2: Bound + Extra + Send + Sync;
    type A3: Bound + Extra + Send + Sync;
    type A4: Bound + Extra + Send + Sync;
}

impl<const I: usize> NodeType for Node<I> {
    type A0 = P<I, 0>;
    type A1 = P<I, 1>;
    type A2 = P<I, 2>;
    type A3 = P<I, 3>;
    type A4 = P<I, 4>;
}

macro_rules! level {
    ($name:ident, $method:ident, [$(($a:ident, $b:ident)),*], $auto:ident $(, $sup:ident)?) => {
        pub trait $name<T: NodeType> $(: $sup<T>)?
        where
            T::A0: Bound + Extra, T::A1: Bound + Extra, T::A2: Bound + Extra,
            T::A3: Bound + Extra, T::A4: Bound + Extra,
            $( Pair<T::$a, T::$b>: Bound + Extra, )*
            L5<T::$auto>: Send + Sync,
        {
            fn $method(&self) -> usize { 0 }
        }

        impl<X, T: NodeType> $name<T> for X
        where
            X: 'static $(+ $sup<T>)?,
            T::A0: Bound + Extra, T::A1: Bound + Extra, T::A2: Bound + Extra,
            T::A3: Bound + Extra, T::A4: Bound + Extra,
            $( Pair<T::$a, T::$b>: Bound + Extra, )*
            L5<T::$auto>: Send + Sync,
        {}
    };
}

// Each level states different bounds so the goals do not dedupe across the chain.
level!(S0, s0, [(A0, A1), (A2, A3), (A4, A0)], A0);
level!(S1, s1, [(A0, A2), (A1, A3), (A4, A1)], A1, S0);
level!(Api, m0, [(A0, A3), (A1, A4), (A2, A0)], A2, S1);

macro_rules! build_vtables {
    ($($i:literal)*) => {
        $( { let _d: &dyn Api<Node<$i>> = &0u8; } )*
    };
}

fn main() {
    build_vtables!(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19);
}

[apiraino]

cc @jakubadamw do you anything relevant to #156417 about this regression?

thanks