MLPerf Tiny benchmark for oneliner on Ariel OS

Reproducible latency + memory-footprint benchmark of the official MLPerf Tiny int8 models (from mlcommons/tiny, benchmark/training), running the oneliner TinyML inference stack on Ariel OS targeting the Raspberry Pi Pico (RP2040, Cortex-M0+).

No datasets are downloaded and no accuracy is measured: this benchmark measures per-inference latency and flash / RAM memory usage only.

Benchmarked models

Task	Official name	Model file	Input
Keyword Spotting	kws_ref_model (DS-CNN)	models/kws01_int8.tflite	(1, 49, 10, 1) i8
Image Classification	trainedResnet_3blk_480k	models/ic01_int8.tflite	(1, 32, 32, 3) i8
Visual Wake Words	vww_96 (MobileNetV1)	models/vww01_int8.tflite	(1, 96, 96, 3) i8
Anomaly Detection	ad01 (deep autoencoder)	models/ad01_int8.tflite	(1, 640) i8
Streaming Wake Word	str_ww_ref_model	models/str_ww01_int8.tflite	(1, 30, 1, 40) i8

Models are the int8 quantized pretrained artifacts from mlcommons/tiny. See models/provenance.md.

Requirements

Preliminary requirements of oneliner please check oneliner.

• ariel-os v0.5.0
• laze (≥ 0.1.37): cargo install laze
• For flashing/running on a real Pico: probe-rs (used by laze/Ariel OS)

Build for the Raspberry Pi Pico

laze build -b rpi-pico          # all five apps in benchmark/

or per app, e.g.:

cd benchmark/kws01 && laze build -b rpi-pico

Each build compiles the TFLite model with IREE for the thumbv6m-none-eabi target and prints the model memory footprint, e.g.:

[oneliner-profiler] Model memory footprint:
  Flash Usage: params = 28736 B (28 KiB), text(code) = 13796 B (13 KiB), rodata = 8990 B (8 KiB), total = 51522 B (50 KiB)
  RAM Usage: arena = 32896 B (32 KiB), input = 490 B (0 KiB), output = 12 B (0 KiB)

All five models fit the RP2040 (2 MB flash / 264 KB RAM).

Run on the Pico (measure latency)

With a Pico connected via SWD:

cd benchmark/kws01 && laze build -b rpi-pico run

The app runs 1000 inferences and reports per-inference latency via the debug console (RTT/USB):

[INFO] mlperf-tiny bench | model=.../models/kws01_int8.tflite | board=rpi-pico
[INFO] memory: flash(params=28736 code=13796 rodata=8990 total=51522) ram(arena=32896 input=490 output=12)
[INFO] latency: samples=1000 avg_us=... min_us=... max_us=...

Host reference run

The same apps also build and run on the host via Ariel OS's native board. This exercises the full measurement pipeline and serves as a reference (host CPU, not representative of the Pico):

scripts/run-native.sh      # runs all five, logs to results/native/
scripts/build-and-run.sh rpi-pico  # captures target footprints to results/rpi-pico/
scripts/report.py          # combines everything into results/summary.json

Current results

See results/summary.json (generated by scripts/report.py) and report.log.