Skip to content

[gluon] Add fuse sigmoid+mul+gemm_splitk#9

Draft
apinge wants to merge 4 commits into
tingqli:mainfrom
apinge:gluon_fusion_gfx950
Draft

[gluon] Add fuse sigmoid+mul+gemm_splitk#9
apinge wants to merge 4 commits into
tingqli:mainfrom
apinge:gluon_fusion_gfx950

Conversation

@apinge

@apinge apinge commented Apr 14, 2026

Copy link
Copy Markdown
Collaborator

Perf on MI350

mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 10.401 us   0.8 TFLOPS   806.7 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 10.120 us   0.8 TFLOPS   829.1 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 10.921 us   0.8 TFLOPS   768.3 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 10.000 us   0.8 TFLOPS   839.1 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 11.880 us   0.7 TFLOPS   706.3 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 9.880 us   0.8 TFLOPS   849.3 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 10.281 us   0.8 TFLOPS   816.1 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 9.880 us   0.8 TFLOPS   849.3 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 9.960 us   0.8 TFLOPS   842.4 GB/s 
mxn_splitk_2s[M=1,bfloat16] : 8 MFLOP / 10.121 us   0.8 TFLOPS   829.0 GB/s 
mxn_splitk_2s[M=1 weight_type=torch.bfloat16] acc OK
final selected TILE_M=16, TILE_N=32
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 8.840 us   3.8 TFLOPS   949.9 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 10.160 us   3.3 TFLOPS   826.5 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 10.200 us   3.3 TFLOPS   823.2 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 10.321 us   3.3 TFLOPS   813.6 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 10.000 us   3.4 TFLOPS   839.7 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 13.440 us   2.5 TFLOPS   624.8 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 11.000 us   3.1 TFLOPS   763.3 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 11.041 us   3.0 TFLOPS   760.5 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 10.960 us   3.1 TFLOPS   766.1 GB/s 
mxn_splitk_2s[M=4,bfloat16] : 34 MFLOP / 11.240 us   3.0 TFLOPS   747.0 GB/s 
mxn_splitk_2s[M=4 weight_type=torch.bfloat16] acc OK
final selected TILE_M=16, TILE_N=32
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 8.960 us   15.0 TFLOPS   939.9 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.200 us   13.2 TFLOPS   825.6 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.441 us   12.9 TFLOPS   806.6 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.000 us   13.4 TFLOPS   842.1 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.560 us   12.7 TFLOPS   797.5 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.640 us   12.6 TFLOPS   791.5 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.961 us   12.2 TFLOPS   768.3 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.600 us   12.7 TFLOPS   794.5 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.920 us   12.3 TFLOPS   771.2 GB/s 
mxn_splitk_2s[M=16,bfloat16] : 134 MFLOP / 10.880 us   12.3 TFLOPS   774.0 GB/s 
mxn_splitk_2s[M=16 weight_type=torch.bfloat16] acc OK
dict_tile_mn {'1': (16, 32), '4': (16, 32), '16': (16, 32)}

summary:
mxn_splitk_2s[torch.bfloat16 B=1   (16x32)]:    10 us,  814.3 GB/s,  0.8 tflops
mxn_splitk_2s[torch.bfloat16 B=4   (16x32)]:    11 us,  773.9 GB/s,  3.1 tflops
mxn_splitk_2s[torch.bfloat16 B=16  (16x32)]:    11 us,  796.8 GB/s, 12.7 tflops

apinge added 4 commits April 13, 2026 11:46
Signed-off-by: apinge <tong.qiu2@amd.com>
Signed-off-by: apinge <tong.qiu2@amd.com>
Signed-off-by: apinge <tong.qiu2@amd.com>
Signed-off-by: apinge <tong.qiu2@amd.com>
@apinge apinge changed the title [gluon ] Add fuse sigmoid+mul+gemm_splitk [gluon] Add fuse sigmoid+mul+gemm_splitk Apr 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant