Gradient norm instability after overfitting

[KindXiaoming]

Problem description

After overfitting, gradient norm slightly increases and may transition into chaos.

How to reproduce

1. open comfyResearch: https://online.comfy-research.com/
2. Use Lottery mode and Type in '41' in the input field
3. draw the random graph and run

[KindXiaoming]

The gradient norm chaos is more severe when Muon is changed to Adam

[KindXiaoming]

Related to #5