feat: make remote evaluation connection pool size configurable

[cmyui]

Summary

The remote evaluation client hardcodes its connection pool to a single connection:

# remote/client.py
self._connection_pool = HTTPConnectionPool(host, max_size=1, idle_timeout=30, ...)

and acquires it with no timeout, so every fetch_v2 in a process serializes through one HTTP connection and concurrent callers block indefinitely in pool.acquire(). fetch_timeout_millis bounds the socket read only — not the queue wait — so under concurrent server-side usage during a period of elevated evaluation-API latency, queued callers can be blocked far longer than any configured timeout, pinning worker threads/greenlets. We hit exactly this in production: request handlers stalled inside connection_pool.acquire() while the fetch timeout never applied.

Observed impact

To illustrate the magnitude: during a ~10-minute window of elevated evaluation-API latency, this serialization amplified the slowdown into tens of thousands of failed requests across several of our services — the worst-affected service logged ~13,000 request errors, and a downstream dependent that had correctly bounded its own call timeouts logged ~4,700 more as cascade. Two properties made it unusually hard to diagnose:

1. It's invisible to fetch-level telemetry. Callers queued in acquire() never reach the socket, so no fetch error or timeout is ever recorded — monitoring built on fetch outcomes shows a healthy SDK while worker threads/greenlets starve underneath it.
2. It outlives the upstream event. The acquire queue is a backlog: ours kept failing requests for ~7 minutes after the evaluation API had fully recovered, extending the impact window ~70% past the underlying degradation.

Because the pool size is hardcoded, no combination of the SDK's existing config options can mitigate this today.

Change

This PR adds connection_pool_max_size to RemoteEvaluationConfig (default 1, preserving existing behavior exactly) and passes it through to the pool, so concurrent server-side callers can size the pool to their needs:

config = RemoteEvaluationConfig(
    fetch_timeout_millis=1500,
    connection_pool_max_size=8,
)

This also aligns the Python SDK with the Node server SDK, whose https.Agent({ keepAlive: true }) does not limit concurrent sockets.

The pool implementation already supports arbitrary sizes — this only exposes the existing parameter through config. Tests added for the config default, config pass-through, and the resulting pool size on the client.

A natural follow-up would be bounding the acquire() wait itself (e.g. applying fetch_timeout_millis to queue wait, or a separate acquire_timeout_millis) so callers fail fast instead of queueing unboundedly — happy to open that separately if there's interest.

Checklist

• Does your PR title have the correct title format?
• Does your PR have a breaking change?: no

---

Note

Low Risk
Small, backward-compatible config surface with default behavior unchanged; only affects fetch concurrency and connection reuse.

Overview
Adds connection_pool_max_size to RemoteEvaluationConfig (default 1, unchanged from the previous hardcoded value) and wires it into RemoteEvaluationClient’s HTTPConnectionPool instead of a fixed max_size=1.

Callers can raise the limit so multiple concurrent fetch_v2 calls can use separate keep-alive connections instead of blocking on pool.acquire() when the single connection is busy. Docs on the config field describe that extra concurrent fetches wait for a released connection.

Unit tests cover the config default/storage and that the client pool’s max_size reflects the setting.

^{Reviewed by Cursor Bugbot for commit 12eb4b8. Bugbot is set up for automated code reviews on this repo. Configure here.}