Skip to content

onnx bench rework#18

Merged
enthropy7 merged 2 commits into
enthropy7:mainfrom
Human9000-bit:better-bench
Jul 14, 2026
Merged

onnx bench rework#18
enthropy7 merged 2 commits into
enthropy7:mainfrom
Human9000-bit:better-bench

Conversation

@Human9000-bit

@Human9000-bit Human9000-bit commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator
  • Add onnx model criterion benches
  • ONNX BENCH REWORK !!!!111!!

there were 2 onnx benches:

  • new one, uses criterion and various
  • old one, used for onnx models profiling

Those 2 guys had some shared logic, which has been moved into benchmarks/onnx-models/src/lib.rs

updated CI as well

New commands to run benches:

  • Criterion:
cargo bench --manifest-path benchmarks/onnx-models/Cargo.toml --bench onnx_models
  • Detailed profiling:
cargo run --release --manifest-path benchmarks/onnx-models/Cargo.toml --example bench_yolo

@Human9000-bit Human9000-bit marked this pull request as ready for review July 14, 2026 16:08
@Human9000-bit Human9000-bit requested a review from enthropy7 July 14, 2026 16:08
Repository owner deleted a comment from github-actions Bot Jul 14, 2026
there were 2 onnx benches:
* new one, uses criterion and various models
* old one, used for onnx models profiling

Those 2 guys had some shared logic, which is moved into
`benchmarks/onnx-models/src/lib.rs`
@Human9000-bit

Copy link
Copy Markdown
Collaborator Author

/bench

@github-actions

Copy link
Copy Markdown

ONNX CPU benchmark

GitHub-hosted runner signal, advisory only. Base and head are compared within the same architecture job.

arch case base p50 head p50 delta

Head op/kernel summary

arch case ops dispatched kernels

Missing paired result(s): aarch64/gemm_relu_256x512x512, aarch64/mobilenet_v3_small_224, aarch64/resnet18_224, aarch64/small_conv_relu_64, aarch64/small_dw_pw_64, aarch64/small_gemm_relu_1x128, aarch64/small_residual_conv_64, aarch64/winograd_3x3_20x20_c256, aarch64/winograd_3x3_80x80_c64, aarch64/yolo11n_bus_640, aarch64/yolov8n_bus_640, x86_64/gemm_relu_256x512x512, x86_64/mobilenet_v3_small_224, x86_64/resnet18_224, x86_64/small_conv_relu_64, x86_64/small_dw_pw_64, x86_64/small_gemm_relu_1x128, x86_64/small_residual_conv_64, x86_64/winograd_3x3_20x20_c256, x86_64/winograd_3x3_80x80_c64, x86_64/yolo11n_bus_640, x86_64/yolov8n_bus_640

Raw JSON artifacts include per-run min/p50/avg/p95/p99, optimized node counts, graph-cost summaries, yscv CPU dispatch report, and per-node runner profile summaries.

@Human9000-bit

Copy link
Copy Markdown
Collaborator Author

ig we need to merge this for bench to work

@enthropy7 enthropy7 merged commit a747ffd into enthropy7:main Jul 14, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants