Skip to content

Separate concurrency benchmark lifecycle measurements - #130

Draft
reneklacan wants to merge 1 commit into
mainfrom
reneklacan/longer-concurrency-benches
Draft

Separate concurrency benchmark lifecycle measurements#130
reneklacan wants to merge 1 commit into
mainfrom
reneklacan/longer-concurrency-benches

Conversation

@reneklacan

@reneklacan reneklacan commented Jul 30, 2026

Copy link
Copy Markdown
Member

Summary

  • split concurrency benchmarks into end-to-end and steady-state suites
  • prestart and gate steady-state workers so startup and shutdown stay outside throughput timing
  • increase high-concurrency workloads until samples take roughly 2-5 seconds
  • report item throughput and extend the CI timeout for the larger repeated suite

Measurements

High-concurrency cases now take approximately 2.2-3.7 seconds locally, up from roughly 0.3-0.5 seconds.

Verification

  • cargo fmt --all --check
  • cargo test --workspace (200 passed, 3 ignored)
  • cargo clippy --all-features --workspace --all-targets -- -D warnings
  • repeated steady-state benchmark samples with no queued or in-flight jobs left behind
  • workflow YAML parsing and git diff --check

Note

Low Risk
Changes are limited to benchmark harness code and CI timeout; no production queue/runtime behavior is modified.

Overview
Reworks oxana concurrency Divan benchmarks so throughput numbers separate full runtime lifecycle from steady-state processing, and CI can finish the heavier suite.

End-to-end benches still time starting the runtime, draining the queue, and shutdown. Steady-state benches pre-start workers, block the first worker cohort until setup completes, then only time the gated job batch; shutdown and RunStats checks run in SteadyStateCleanup after Divan stops the timer. Cases are parameterized via BenchmarkCase (worker count + job count), with larger job counts at high concurrency so samples run ~2–5s instead of sub-second.

Benchmark setup uses enqueue_list instead of per-job enqueue, reports items/sec via Divan ItemsCount, and asserts success counts through shared assert_stats. The Test workflow job timeout rises from 10 to 30 minutes to match repeated cargo bench in CI.

Reviewed by Cursor Bugbot for commit 79dd9f3. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 79dd9f3. Configure here.

async fn measure(&self) {
self.control.start_measurement();
self.control.wait_until_finished().await;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Measure path can hang forever

Medium Severity

measure waits only on the finished semaphore and never watches the runtime task. prepare already uses select! so a setup-time runtime exit fails fast, but if the runtime exits during the timed section before every job calls record_completion, wait_until_finished blocks forever. Cleanup never runs, so CI sits until the 30-minute timeout instead of surfacing the runtime failure.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 79dd9f3. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant