Skip to content

[mxfp8 training] Add fused grouped GEMM + SwiGLU + MXFP8 quantization ops for MoE routed experts #4820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wolfcomos wants to merge 13 commits into
base: main
Choose a base branch
from
+2,233 −0
Open

[mxfp8 training] Add fused grouped GEMM + SwiGLU + MXFP8 quantization ops for MoE routed experts #4820

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
f8a1ac9
Add MXFP8 grouped-MLP kernel family foundation and the wgrad kernel
wolfcomos Aug 17, 2026
b822dbe
Rewrite the MXFP8 grouped-MLP family: public-DSL kernels A/B/C, ops, …
wolfcomos Aug 17, 2026
758dd04
Simplify cutedsl_grouped_mlp internals: fewer helpers, flatter bodies
wolfcomos Aug 17, 2026
c6fa589
Add cuDNN-frontend MXFP8 grouped-MLP custom ops (glu/mm/dglu/wgrad)
wolfcomos Aug 18, 2026
1c5bceb
Add unit tests for the cuDNN-frontend MXFP8 grouped-MLP ops
wolfcomos Aug 18, 2026
e03489a
Remove the archived pure-CuTe-DSL MXFP8 grouped-MLP kernel family (A/…
wolfcomos Aug 19, 2026
ddd8846
Merge branch 'pytorch:main' into agent/mxfp8-cudnn-grouped-mlp
wolfcomos Aug 19, 2026
c478ba5
Rename the fused grouped-MLP surface off the cudnn prefix
wolfcomos Aug 19, 2026
4d4079d
Merge remote-tracking branch 'origin/agent/mxfp8-cudnn-grouped-mlp' i…
wolfcomos Aug 19, 2026
683b370
Memoize the grouped-MLP ops' metadata validation by signature
wolfcomos Aug 19, 2026
7bc1755
Fold the grouped-MLP validation and public wrappers into the ops module
wolfcomos Aug 19, 2026
b3510d6
Rename grouped_mlp_ops.py to cutedsl_grouped_mlp.py
wolfcomos Aug 19, 2026
5a50196
Fix review findings in the grouped-MLP ops module and its tests
wolfcomos Aug 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading