From 14d793d02a073606192a5c2d9390fea5fd1c4c81 Mon Sep 17 00:00:00 2001 From: Arnaud Gourlay Date: Sat, 22 Aug 2026 07:54:18 +0200 Subject: [PATCH] --rps reaped at most one completion per rate-limiter tick `process_with_rps` drives its in-flight set from a `tokio::select!` whose completion branch was written as a pattern: Some(Err(err)) = in_flight.next(), if !in_flight.is_empty() => { ... } A successful request returns `Some(Ok(()))`, which does not match, and `select!` disables a branch whose pattern fails for the remainder of that invocation. So every time a request succeeded the loop stopped looking at `in_flight` and went back to waiting on `interval.tick()`. Together with `FuturesUnordered::poll_next` returning at the *first* ready child and leaving the rest of the ready queue unpolled, that capped the whole loop at roughly one completion reaped per tick. A request's elapsed time is taken inside its own future, so a child that has been woken by its response but not yet polled accumulates the wait for its turn and charges it to the request. The reported latency is therefore mostly the drain backlog, which is why it was *worst at the lowest offered rate* -- fewer ticks per second, fewer chances to drain -- and shrank monotonically as the rate rose. Queueing does the opposite, and that inversion is what made the numbers unusable rather than merely pessimistic. Binding the result instead of pattern-matching it keeps the branch enabled, so the loop drains completions as fast as they arrive. Measured against a server serving the same load, 50,000 queries, `-t 16 -c 2`, `--rps 2000`, closed-loop saturation of that server 2,387/s: before client p50 1663.71 ms server p50 80.80 ms gap 1582.91 ms after client p50 3.29 ms server p50 3.09 ms gap 0.20 ms The server-side figure moves too: starved of reaping, the loop delivers its requests in bursts, and the server was reporting the queue those bursts made. Above saturation both builds agree (`--rps 4000` on the same server: 1.76 s either way), which is a real queue and should not move. `process_with_parallel` is unaffected: `buffer_unordered` bounds concurrency and its `while let` reaps every completion. Co-Authored-By: Claude Opus 5 (1M context) --- src/stats.rs | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/src/stats.rs b/src/stats.rs index 96794d6..648f627 100644 --- a/src/stats.rs +++ b/src/stats.rs @@ -260,14 +260,24 @@ async fn process_with_rps( let future = processor.make_request(req_id, args, progress_bar); in_flight.push(future); } - // Process completed requests - Some(Err(err)) = in_flight.next(), if !in_flight.is_empty() => { - if args.ignore_errors { - progress_bar.println(format!("Error: {err}")); - } else if first_error.is_none() { - first_error = Some(err); - // Stop sending new requests on error - break; + // Process completed requests. + // + // Bind the result here — do not match on `Some(Err(err))`. + // `select!` disables a branch whose pattern fails, so a successful + // request would switch this branch off and leave the loop draining + // `in_flight` at the rate limiter's cadence instead of as fast as + // completions arrive. Each request times itself, so a + // woken-but-unpolled future charges the wait for its turn to the + // request and inflates the reported latency. + res = in_flight.next(), if !in_flight.is_empty() => { + if let Some(Err(err)) = res { + if args.ignore_errors { + progress_bar.println(format!("Error: {err}")); + } else if first_error.is_none() { + first_error = Some(err); + // Stop sending new requests on error + break; + } } } else => {