Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ compose-vs-views/
├── app-view/ → XML View + RecyclerView implementation
├── shared/ → Shared data models and fake repository
├── benchmark/ → AndroidX Macrobenchmark tests
├── results/ → JSON / CSV benchmark outputs
├── results/ → Tracked benchmark result policy and curated summaries
└── paper.md → Research write-up (draft or published version)
```

Expand Down Expand Up @@ -110,9 +110,9 @@ compose-vs-views/
adb pull /sdcard/Android/media/dev.egarcia.andperf.benchmark/additional_test_output ./benchmark/build/outputs/connected_android_test_additional_output/
```

5. Aggregate and analyze
5. Preserve, aggregate, and analyze

Collect the JSON/CSV outputs from the additional output directory or use the HTML report to inspect per-test timings, then aggregate medians/p90/p95 for final analysis.
Keep generated build artifacts out of Git. For each verified physical-device run, copy only curated summaries/manifests into `results/` and record where the raw JSON, HTML, and perfetto artifacts were retained. See [`results/README.md`](results/README.md) for the artifact policy.

---

Expand Down Expand Up @@ -174,7 +174,7 @@ Both implementations use:
./gradlew :app-compose:assembleRelease :app-view:assembleRelease :benchmark:assembleBenchmark
```
3. Install and run benchmarks on connected physical device(s).
4. Export results from `/results/` and compare using your favorite data-analysis tool.
4. Export generated benchmark outputs from `benchmark/build/outputs/connected_android_test_additional_output/`, retain the raw artifacts, and add a curated summary or manifest under `results/` before comparing results.

---

Expand Down
39 changes: 39 additions & 0 deletions results/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Benchmark Results Artifact Policy

This directory is the tracked index for benchmark evidence. It is not a dump
folder for every generated Macrobenchmark artifact.

## What is tracked here

- Curated result summaries, such as small JSON, CSV, Markdown, or manifest files
that are intentionally reviewed and suitable for version control.
- A per-run manifest that records the source location for raw artifacts when the
raw files are too large or too transient to commit.
- Methodology notes needed to connect README result values to a reproducible run.

## What stays out of Git

- Generated Android build outputs under
`benchmark/build/outputs/connected_android_test_additional_output/`.
- HTML reports under `benchmark/build/reports/androidTests/connected/`.
- Large perfetto traces (`*.perfetto-trace`) unless a future task explicitly
approves committing a small, curated sample.
- Device-local files that still need to be pulled from
`/sdcard/Android/media/dev.egarcia.andperf.benchmark/additional_test_output`.

## Required manifest fields for a verified run

Before publishing README numbers, add a manifest or summary in this directory
with at least:

- Run date and operator.
- Device model, Android/API version, and device serial handling policy.
- Exact Gradle command(s), benchmark class/method filters, and target package.
- Thermal, animation, network, and battery assumptions.
- Raw artifact retention location for benchmark JSON, HTML reports, and perfetto
traces.
- Curated Compose and View values used in documentation, with units and sample
counts.

README result values should cite one of these tracked summaries/manifests so
future reviewers can trace every published number back to retained evidence.