Question about cross-microbatch expert-weight reuse in the dual-lane path

[sinpeyw]

Hi, I have been learning a lot from the implementation, and I am particularly interested in the cross-microbatch weight reuse described in MegaMoEM2NAfdAdapter.eg_moe_dual(). My understanding is that the mb=2 persistent path exercises the same two-lane behavior through eg_moe_persistent():

lane 1 reuses lane 0's L2-resident weights

I would appreciate some clarification on how this reuse behaves in the evaluated workloads and how it interacts with the other MegaMoE optimizations.

To check my understanding, I made a rough estimate. For an FP8 SwiGLU expert, the weight size is approximately:

2 * hidden * intermediate + hidden * intermediate
= 3 * hidden * intermediate bytes

Using the published configurations with FFN EP=4 gives:

Model	Local experts / FFN GPU	Weight / expert	Weight / layer / FFN GPU	Weight / 16-expert wave	Routed assignments / local expert / microbatch
Qwen3-235B	32	18 MiB	576 MiB	288 MiB	66–84
MiniMax-M2.5	64	13.5 MiB	864 MiB	216 MiB	38.25–76.5

The weight estimates exclude scale storage. The routing estimates assume balanced routing and use the published 8K/16K batch sizes and A:F ratios. If this reasoning is correct, nearly all local experts would be active in each microbatch, while the per-wave and per-layer weight working sets would be larger than GB200's approximately 126 MB L2 cache. This made me curious about how much useful weight data remains resident between the two lanes in practice.

Would you mind sharing:

• whether the contribution of cross-microbatch weight reuse was measured separately from the complete MegaMoE optimization bundle;
• any lane-level observations, such as TMA-B time, expert-GEMM time, L2 hit rate, or HBM read bytes for lane 0 versus lane 1.

These are only back-of-the-envelope estimates, and I may be missing an important detail of the MegaMoE scheduler or Blackwell cache behavior. I would be grateful for any correction, clarification, or measurement you can share.