From 05abb4561595a3013c32431078f2fd19d63f1e14 Mon Sep 17 00:00:00 2001 From: Erick Garcia Date: Tue, 2 Jun 2026 00:15:36 -0600 Subject: [PATCH] docs: define benchmark results artifact policy --- README.md | 8 ++++---- results/README.md | 39 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+), 4 deletions(-) create mode 100644 results/README.md diff --git a/README.md b/README.md index 90e7592..d55b3f1 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ compose-vs-views/ ├── app-view/ → XML View + RecyclerView implementation ├── shared/ → Shared data models and fake repository ├── benchmark/ → AndroidX Macrobenchmark tests - ├── results/ → JSON / CSV benchmark outputs + ├── results/ → Tracked benchmark result policy and curated summaries └── paper.md → Research write-up (draft or published version) ``` @@ -110,9 +110,9 @@ compose-vs-views/ adb pull /sdcard/Android/media/dev.egarcia.andperf.benchmark/additional_test_output ./benchmark/build/outputs/connected_android_test_additional_output/ ``` -5. Aggregate and analyze +5. Preserve, aggregate, and analyze - Collect the JSON/CSV outputs from the additional output directory or use the HTML report to inspect per-test timings, then aggregate medians/p90/p95 for final analysis. + Keep generated build artifacts out of Git. For each verified physical-device run, copy only curated summaries/manifests into `results/` and record where the raw JSON, HTML, and perfetto artifacts were retained. See [`results/README.md`](results/README.md) for the artifact policy. --- @@ -174,7 +174,7 @@ Both implementations use: ./gradlew :app-compose:assembleRelease :app-view:assembleRelease :benchmark:assembleBenchmark ``` 3. Install and run benchmarks on connected physical device(s). -4. Export results from `/results/` and compare using your favorite data-analysis tool. +4. Export generated benchmark outputs from `benchmark/build/outputs/connected_android_test_additional_output/`, retain the raw artifacts, and add a curated summary or manifest under `results/` before comparing results. --- diff --git a/results/README.md b/results/README.md new file mode 100644 index 0000000..36776e1 --- /dev/null +++ b/results/README.md @@ -0,0 +1,39 @@ +# Benchmark Results Artifact Policy + +This directory is the tracked index for benchmark evidence. It is not a dump +folder for every generated Macrobenchmark artifact. + +## What is tracked here + +- Curated result summaries, such as small JSON, CSV, Markdown, or manifest files + that are intentionally reviewed and suitable for version control. +- A per-run manifest that records the source location for raw artifacts when the + raw files are too large or too transient to commit. +- Methodology notes needed to connect README result values to a reproducible run. + +## What stays out of Git + +- Generated Android build outputs under + `benchmark/build/outputs/connected_android_test_additional_output/`. +- HTML reports under `benchmark/build/reports/androidTests/connected/`. +- Large perfetto traces (`*.perfetto-trace`) unless a future task explicitly + approves committing a small, curated sample. +- Device-local files that still need to be pulled from + `/sdcard/Android/media/dev.egarcia.andperf.benchmark/additional_test_output`. + +## Required manifest fields for a verified run + +Before publishing README numbers, add a manifest or summary in this directory +with at least: + +- Run date and operator. +- Device model, Android/API version, and device serial handling policy. +- Exact Gradle command(s), benchmark class/method filters, and target package. +- Thermal, animation, network, and battery assumptions. +- Raw artifact retention location for benchmark JSON, HTML reports, and perfetto + traces. +- Curated Compose and View values used in documentation, with units and sample + counts. + +README result values should cite one of these tracked summaries/manifests so +future reviewers can trace every published number back to retained evidence. \ No newline at end of file