Skip to content

[ARK] Optimize SYCL-TLA woqgemm_s8 - #2133

Draft
Zhenzhong1 wants to merge 2 commits into
mainfrom
zhenzhong/woqgemm_s8_update
Draft

[ARK] Optimize SYCL-TLA woqgemm_s8#2133
Zhenzhong1 wants to merge 2 commits into
mainfrom
zhenzhong/woqgemm_s8_update

Conversation

@Zhenzhong1

Copy link
Copy Markdown
Contributor

Desc

Perf

SYCL-TLA on B60

=== W4A8===

 m=4096, n=4096, k=4096, blocksize=64, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.062500, Mean Diff: 0.008240, ref mean:7.480469, OUT mean:7.480469
[Performance] Time: 1.5445 ms
              GFLOPS: 88987.05
              Bandwidth: 49.22 GB/s

 m=4096, n=4096, k=4096, blocksize=32, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.007812, Mean Diff: 0.001297, ref mean:7.511719, OUT mean:7.511719
[Performance] Time: 1.7683 ms
              GFLOPS: 77723.89
              Bandwidth: 43.29 GB/s

 m=4096, n=4096, k=4096, blocksize=32, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.007812, Mean Diff: 0.001297, ref mean:7.511719, OUT mean:7.511719
[Performance] Time: 1.7685 ms
              GFLOPS: 77714.57
              Bandwidth: 43.28 GB/s

 m=4096, n=4096, k=4096, blocksize=64, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.062500, Mean Diff: 0.008240, ref mean:7.480469, OUT mean:7.480469
[Performance] Time: 1.5443 ms
              GFLOPS: 88997.73
              Bandwidth: 49.23 GB/s

 m=4096, n=4096, k=4096, blocksize=128, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.062500, Mean Diff: 0.008240, ref mean:7.500000, OUT mean:7.500000
[Performance] Time: 1.4031 ms
              GFLOPS: 97953.18
              Bandwidth: 53.99 GB/s

Reference Joint_matrix on B60

=== W4A8 / A8W4 ===

 m=4096, n=4096, k=4096, blocksize=64, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.062500, Mean Diff: 0.008240, ref mean:7.480469, OUT mean:7.480469
[Performance] Time: 1.9193 ms
              GFLOPS: 71607.50
              Bandwidth: 39.61 GB/s

 m=4096, n=4096, k=4096, blocksize=32, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.007812, Mean Diff: 0.001297, ref mean:7.511719, OUT mean:7.511719
[Performance] Time: 1.5900 ms
              GFLOPS: 86441.91
              Bandwidth: 48.14 GB/s

 m=4096, n=4096, k=4096, blocksize=32, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.007812, Mean Diff: 0.001297, ref mean:7.511719, OUT mean:7.511719
[Performance] Time: 1.5905 ms
              GFLOPS: 86410.68
              Bandwidth: 48.13 GB/s

 m=4096, n=4096, k=4096, blocksize=64, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.062500, Mean Diff: 0.008240, ref mean:7.480469, OUT mean:7.480469
[Performance] Time: 1.9193 ms
              GFLOPS: 71609.52
              Bandwidth: 39.61 GB/s

 m=4096, n=4096, k=4096, blocksize=128, compute_type=int8, weight_type=int4, scale_type=fp16, asym=False, device=xpu
  Max Diff: 0.062500, Mean Diff: 0.008240, ref mean:7.500000, OUT mean:7.500000
[Performance] Time: 1.5447 ms
              GFLOPS: 88972.04
              Bandwidth: 49.04 GB/s

Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant