Inference: Prevent the coordinator from choking at large concurrency - #6497

Open

sidsingh-nvidia wants to merge 2 commits into

NVIDIA:mainfrom

sidsingh-nvidia:siddharth/co-ordinator-fixes

Commits on Aug 19, 2026

modify coordinator wire format - split into two chunks - first chunk is metadata of constant size. This is the only thing that the coordinator needs to unpack/read and pack

Siddharth Singh
authored and
sidsingh-nvidia
committed
815842c
refactor(inference): declare the coordinator wire schema in one place
sidsingh-nvidia
committed
fe4e618