Explore a borrower-aware capacity limiter

[tisonkun]

Status

This is a backlog investigation, not an implementation-ready request. It should remain behind the more clearly differentiated primitives tracked by #218 until a concrete workload establishes that Asyncband's existing weighted semaphore is insufficient.

Background

Trio and AnyIO expose a CapacityLimiter for controlling how many borrowers may perform an operation concurrently. It resembles a semaphore but adds identity and resizing semantics:

• one borrower may hold at most one token from a limiter;
• a token may be acquired or released on behalf of an explicit borrower;
• the total capacity can change while tokens are borrowed;
• diagnostics can identify current borrowers and waiters.

See Trio CapacityLimiter and AnyIO capacity limiters.

This can detect accidental double acquisition, transfer admission authority between parts of a workflow, and expose useful overload diagnostics. Typical examples include bounding blocking-thread usage, per-service request execution, or access to a resource where each logical borrower must occupy no more than one slot.

Possible runtime-agnostic contract

Asyncband cannot infer a current task identity without depending on a runtime. A compatible design would therefore require caller-supplied borrower identities, for example a generic B: Eq + Hash or an opaque borrower token created by the limiter.

A possible contract would include:

• acquire one token for a borrower;
• reject a second simultaneous acquisition for the same borrower;
• release only a token held by that borrower;
• optionally acquire and release on behalf of another caller-supplied borrower;
• adjust total capacity without revoking already borrowed tokens;
• report borrowed count, configured total, queued waiter count, and possibly borrower snapshots;
• preserve FIFO waiter ordering and cleanly remove cancelled acquisitions.

This sketch is intentionally provisional. The issue should first prove that borrower identity is worth its cost and that the API is not merely admission policy layered over Semaphore.

Relationship with the existing semaphore

Asyncband's weighted Semaphore already provides:

• borrowed and owned RAII permits;
• acquisition of one or several permits;
• FIFO queued acquisition;
• explicit permit release and permanent permit forgetting;
• increasing capacity through release and decreasing the logical balance through permit reduction;
• cancellation that removes the pending acquirer at the cost of its queue position.

A capacity limiter would be differentiated mainly by borrower identity, one-token-per-borrower validation, explicit total-capacity resizing, and diagnostics. If those features are not required together, a thin wrapper around Semaphore is preferable to another core primitive.

Design considerations

Borrower identity

Using the current task as the borrower is convenient in Trio, but it is incompatible with Asyncband's runtime independence. Requiring B: Eq + Hash makes identity explicit but adds hashing, key ownership, and an internal map to every acquire and release. An opaque RAII borrower handle avoids arbitrary keys but loses acquire_on_behalf_of semantics.

The API must not pretend to provide task ownership when it only records a caller-supplied token.

Dynamic resizing

Increasing capacity can wake queued borrowers immediately. Decreasing capacity while more tokens are borrowed than the new total cannot revoke existing work, so the limiter enters a deficit state until enough borrowers release. The contract must define:

• whether zero and fractional capacity are allowed;
• whether shrink takes priority over already queued acquisitions;
• how available capacity is reported while borrowed exceeds total;
• whether resizing preserves strict FIFO order.

Asyncband should use integer capacity unless a real use case requires Trio's fractional token count.

Release authority and RAII

A normal RAII permit makes accidental leaks less likely, while release-on-behalf-of intentionally separates the code that borrows from the code that releases. Supporting both can make double release and ownership transfer difficult to reason about. The design should choose a primary ownership model rather than exposing every Trio operation by default.

Diagnostics and retained keys

Tracking borrower identities can retain user keys for the entire borrow and wait duration. This adds allocation, hashing, cleanup races, and potentially sensitive diagnostic data. Snapshot APIs also need a clear consistency contract and can become an accidental synchronization interface.

Cancellation and fairness

Cancelling a queued acquisition must remove its borrower reservation so a later retry is not reported as a duplicate. Cancellation racing with grant must resolve to either a returned RAII permit or a fully released capacity slot. Weighted semaphore internals may be reusable, but borrower-map and grant order must remain one atomic logical transition.

Potential negative factors

• Substantial overlap: most capacity-control workloads are already served by the weighted semaphore.
• No portable task identity: callers must carry borrower keys or handles explicitly, weakening the ergonomic advantage seen in Trio and AnyIO.
• Policy creep: borrower-aware admission and diagnostics move the abstraction toward workload policy; Asyncband recently removed admission::FairShare, so a new admission primitive needs concrete evidence rather than taxonomy completeness.
• Additional state and cost: exact borrower tracking requires a hash table, key retention, and cleanup on completion, error, panic, and cancellation.
• Resizing complexity: shrinking below current usage creates a deficit state that users may misunderstand as revocation.
• Misuse surface: acquire-on-behalf-of and release-on-behalf-of permit authority transfer that RAII normally prevents.
• Unclear Rust ecosystem gap: a semaphore plus caller-owned permits composes naturally in Rust, so the extra contract must solve an observed problem rather than mirror a Python API.

Key trade-offs

• Runtime-independent explicit borrower keys versus runtime-provided task identity.
• Misuse detection and diagnostics versus hash-map cost and retained user state.
• Dynamic total capacity versus a simpler fixed-capacity RAII contract.
• Release-on-behalf-of flexibility versus ownership clarity.
• A first-class primitive versus a higher-level wrapper over Semaphore.

Non-goals

This backlog item does not propose:

• spawning or limiting tasks on the caller's behalf;
• selecting a runtime's current task identity;
• weighted fair sharing, priorities, or tenant scheduling;
• deadlines or rate limiting;
• distributed admission control;
• implementation before the differentiation criteria below are met.

Exit criteria for the backlog

Move this issue to implementation design only when all of the following are available:

• a concrete Asyncband or downstream workload where weighted Semaphore is materially insufficient;
• a runtime-independent borrower identity model that callers can use ergonomically;
• a documented resize and deficit contract;
• evidence that exact identity tracking and diagnostics justify their memory and contention cost;
• a clear explanation of why a wrapper or companion crate is not the better placement.