Skip to content

Add release build statistics workflows#1043

Open
arrayka wants to merge 30 commits intomainfrom
u/arrayka/release_builds_ci
Open

Add release build statistics workflows#1043
arrayka wants to merge 30 commits intomainfrom
u/arrayka/release_builds_ci

Conversation

@arrayka
Copy link
Copy Markdown
Contributor

@arrayka arrayka commented May 8, 2026

Why

We need to track release build time and binary size trends to catch regressions early and review weekly. Currently we have no visibility into how build times or output sizes change over time.

What

Two new workflows:

  • build-release.yml — runs on every push to main (on post-checkin). Performs a clean release build with --timings, logs crate build times and binary sizes to the console, runs cargo bloat and cargo llvm-lines, and uploads structured JSON artifacts (90-day retention).
  • produce-build-stats.yml — runs daily at 8am UTC. Aggregates the last 30 days of build-release artifacts and produces an HTML dashboard with:
    • Total build time trend
    • Per-crate build time trend (top 15)
    • Total binary size trend
    • Per-binary size trend
    • Latest cargo bloat output
    • Latest cargo llvm-lines output

Example report

See: https://github.com/microsoft/DiskANN/actions/runs/25590491134

To view:

  • download thebuild-stats-report artifact zip
  • unpack it
  • open build-stats-report.html in a browser. The build-stats-data.js file must be in the same directory.

@arrayka arrayka requested review from a team and Copilot May 8, 2026 01:47
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds automated collection and reporting for release build time + binary size trends to make regressions visible over time via GitHub Actions artifacts and an HTML dashboard.

Changes:

  • Introduces a build-release.yml workflow to run clean --release --timings, collect per-crate timings, binary sizes, and cargo bloat, and upload artifacts.
  • Introduces a produce-build-stats.yml workflow to download recent artifacts and generate an HTML/Vue/Chart.js report.
  • Adds Python scripts to parse timing HTML + binaries into JSON and to aggregate artifacts into a build-stats-data.js payload consumed by the report.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
.github/workflows/produce-build-stats.yml Scheduled/manual workflow to fetch recent build-release artifacts and publish an HTML report artifact.
.github/workflows/build-release.yml Push/manual workflow to perform a clean release build with timings, compute size/bloat metrics, and upload artifacts.
.github/scripts/parse-build-stats.py Parses Cargo timing HTML and release binaries into JSON artifacts for later aggregation.
.github/scripts/generate-stats-data.py Aggregates per-run JSON/text artifacts into a JS data file used by the HTML dashboard.
.github/scripts/build-stats-report.html HTML dashboard (Vue 3 + Chart.js) that visualizes build-time and size trends and shows latest cargo-bloat output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/produce-build-stats.yml Outdated
Comment thread .github/scripts/build-stats-report-data.py
Comment thread .github/workflows/produce-build-stats.yml Outdated
Comment thread .github/workflows/build-release.yml Outdated
Comment thread .github/reports/build-stats-report.html
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.47%. Comparing base (9ca5928) to head (d7a4a86).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1043      +/-   ##
==========================================
- Coverage   89.47%   89.47%   -0.01%     
==========================================
  Files         460      461       +1     
  Lines       85369    85494     +125     
==========================================
+ Hits        76387    76498     +111     
- Misses       8982     8996      +14     
Flag Coverage Δ
miri 89.47% <ø> (-0.01%) ⬇️
unittests 89.32% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Alex Razumov (from Dev Box) added 4 commits May 7, 2026 19:38
Copy link
Copy Markdown
Contributor

@hildebrandmw hildebrandmw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Alex! Any idea on how long all this takes to run in CI?

tool: cargo-bloat,cargo-llvm-lines

- name: Build release with timings
run: cargo clean && cargo build --workspace --release --locked --timings
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you hit this with a cargo clean, there's no reason to cache anything Swatinem/rust-cache.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed Swatinem/rust-cache


- name: Collect artifacts from recent runs
run: |
set -euo pipefail
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on making this an independent script that could also be run locally? Something to help with local debugging/running?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. extracted it into .github/scripts/build-stats-report.sh

@@ -0,0 +1,196 @@
<!DOCTYPE html>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move to .github/reports or something?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@arrayka
Copy link
Copy Markdown
Contributor Author

arrayka commented May 8, 2026

Thanks Alex! Any idea on how long all this takes to run in CI?

Here is an example: https://github.com/microsoft/DiskANN/actions/runs/25533992329/job/74945898287
Total run time: 5m 43s

  • Run cargo clean && cargo build --workspace --release --locked --timings - 3m 1s
  • Run cargo bloat --release --package diskann-benchmark -n 100 | tee cargo-bloat.txt - 1m 52s
  • Run cargo llvm-lines --release --package diskann-benchmark | head -100 | tee cargo-llvm-lines.txt - 24s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants