The Bug
In engine/sdks/typescript/runner/src/tunnel.ts, the Tunnel class uses an array to map incoming requests to their respective actors:
#requestToActor: Array<{
gatewayId: GatewayId;
requestId: RequestId;
actorId: string;
}> = [];Every single time a message or request chunk is received, the code does an O(N) linear scan over this array using .find() and arraysEqual():
getRequestActor(
gatewayId: GatewayId,
requestId: RequestId,
): RunnerActor | undefined {
const entry = this.#requestToActor.find(
(entry) =>
arraysEqual(entry.gatewayId, gatewayId) &&
arraysEqual(entry.requestId, requestId),
);
// ...Additionally, cleaning up the request performs an O(N) .findIndex() followed by an O(N) .splice().
The Impact
If there are 10,000 active concurrent requests/WebSockets on a single runner, every incoming event (e.g. HTTP body chunk, WS message) will iterate through 10,000 array elements doing byte-by-byte comparisons. This results in O(N^2) processing overhead and will easily peg the Node.js event loop at 100% CPU under load, severely degrading throughput.
The Fix
We should refactor #requestToActor to use a Map<string, string> (using a composite key like ${idToStr(gatewayId)}:${idToStr(requestId)}), converting all lookups, inserts, and deletes to O(1).
Hey @jog1t, I noticed this bottleneck while studying the TypeScript SDK runner architecture! Could you please assign this issue to me? I'd love to submit a PR to refactor this into an O(1) Map!
The Bug
In
engine/sdks/typescript/runner/src/tunnel.ts, theTunnelclass uses an array to map incoming requests to their respective actors:Every single time a message or request chunk is received, the code does an
O(N)linear scan over this array using.find()andarraysEqual():Additionally, cleaning up the request performs an
O(N).findIndex()followed by anO(N).splice().The Impact
If there are 10,000 active concurrent requests/WebSockets on a single runner, every incoming event (e.g. HTTP body chunk, WS message) will iterate through 10,000 array elements doing byte-by-byte comparisons. This results in
O(N^2)processing overhead and will easily peg the Node.js event loop at 100% CPU under load, severely degrading throughput.The Fix
We should refactor
#requestToActorto use aMap<string, string>(using a composite key like${idToStr(gatewayId)}:${idToStr(requestId)}), converting all lookups, inserts, and deletes toO(1).Hey @jog1t, I noticed this bottleneck while studying the TypeScript SDK runner architecture! Could you please assign this issue to me? I'd love to submit a PR to refactor this into an
O(1)Map!