Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,35 @@ Please see the [MLPerf Inference benchmark paper](https://arxiv.org/abs/1911.025

Please see [here](https://docs.mlcommons.org/inference/benchmarks/) for the MLPerf inference documentation website which includes automated commands to run MLPerf inference benchmarks using different implementations.

## MLPerf Inference v6.1 (submission deadline July 31 2026)

For submissions, please use the master branch and any commit since the [v6.1 seed release](https://github.com/mlcommons/inference/commit/b0662794362bc2c22c33c3e4a0c66ef83411ecf0) although it is best to use the latest commit in the [master branch](https://github.com/mlcommons/inference).

For power submissions please use [SPEC PTD 1.11.1](https://github.com/mlcommons/power) (needs special access) and any commit of the power-dev repository after the [code-freeze](https://github.com/mlcommons/power-dev/commit/c4b3ad8202fbd8ac28d77149e5e7aeadb725bbf2)


| model | reference app | framework | dataset | category
| ---- | ---- | ---- | ---- | ---- |
| resnet50-v1.5 | [vision/classification_and_detection](https://github.com/mlcommons/inference/tree/master/vision/classification_and_detection) | tensorflow, onnx, tvm, ncnn | imagenet2012 | edge |
| yolo v11 | [vision/classification_and_detection](https://github.com/mlcommons/inference/tree/master/vision/classification_and_detection/yolo) | pytorch, onnx | COCO safe subset | edge |
| bert | [language/bert](https://github.com/mlcommons/inference/tree/master/language/bert) | tensorflow, pytorch, onnx | squad-1.1 | edge |
| dlrm-v3 | [recommendation/dlrm_v3](https://github.com/mlcommons/inference/tree/master/recommendation/dlrm_v3/pytorch) | pytorch | Synthetic dataset | datacenter |
| 3d-unet | [vision/medical_imaging/3d-unet-kits19](https://github.com/mlcommons/inference/tree/master/vision/medical_imaging/3d-unet-kits19) | pytorch, tensorflow, onnx | KiTS19 | edge |
| stable-diffusion-xl | [text_to_image](https://github.com/mlcommons/inference/tree/master/text_to_image) | pytorch | COCO 2014 | edge |
| Wan2.2-T2V-A14B-Diffusers | [text_to_video](https://github.com/mlcommons/inference/tree/master/text_to_video) | pytorch | COCO 2014 | datacenter |
| llama2-70b | [language/llama2-70b](https://github.com/mlcommons/inference/tree/master/language/llama2-70b) | pytorch | OpenOrca | datacenter |
| rgat | [graph/rgat](https://github.com/mlcommons/inference/tree/master/graph/R-GAT) | pytorch | IGBH | datacenter |
| llama3.1-8b | [language/llama3.1-8b](https://github.com/mlcommons/inference/tree/master/language/llama3.1-8b)| pytorch | CNN-Daily Mail | edge,datacenter |
| deepseek-r1 | [language/deepseek-r1](https://github.com/mlcommons/inference/tree/master/language/deepseek-r1)| pytorch | AIME, MATH500, gpqa, MMLU-Pro, livecodebench(code_generation_lite) | datacenter |
| whisper | [speech2text](https://github.com/mlcommons/inference/tree/master/speech2text)| pytorch | LibriSpeech | edge,datacenter |
| GPT–OSS | [language/gpt-oss-120b](https://github.com/mlcommons/inference/tree/master/language/gpt-oss-120b)| pytorch | mlperf_gpt_oss_performance, mlperf_gpt_oss_accuracy | datacenter |
| VLM (qwen3-vl-235b-a22b) | [multimodal/qwen3-vl](https://github.com/mlcommons/inference/tree/master/multimodal/qwen3-vl)| pytorch | Shopify-product-catalogue | datacenter |
| qwen3.6-27b | [language/edge-agentic](https://github.com/mlcommons/inference/tree/master/language/edge-agentic) | pytorch | BFCL v4 single-turn (accuracy) + recorded agentic-coding replay (performance) | edge |
| e2e-rag (qna, db) | [e2e-rag](https://github.com/mlcommons/inference/tree/master/e2e-rag) | pytorch | FRAMES | datacenter |

* Framework here is given for the reference implementation. Submitters are free to use their own frameworks to run the benchmark.
* pointpainting, llama3.1-405b and mixtral-8x7b are not part of the v6.1 model list.

## MLPerf Inference v6.0 (submission deadline February 13, 2026)

For submissions, please use the master branch and any commit since the [6.0 seed release](https://github.com/mlcommons/inference/commit/f131a0d29ccae9a967d93ffe96f66b1be3537d3b) although it is best to use the latest commit in the [master branch](https://github.com/mlcommons/inference).
Expand Down
Loading