Add TensorRT Multi-Device (multi-GPU) inference support

[pkisfaludi-nv]

Summary

Adds TensorRT Multi-Device (MD) support to the tensorrt backend: run a single TensorRT engine sharded across multiple GPUs via NCCL DistCollective + IExecutionContext::setCommunicator (GA in TensorRT 11), transparently to clients (same gRPC/HTTP API).

This mirrors an internal change; submitting upstream as requested by the backend maintainers.

Usage

Enable per model with a KIND_MODEL instance group + parameters:

instance_group [ { kind: KIND_MODEL count: 1 } ]
parameters [
  { key: "enable_multi_device"           value: { string_value: "true" } },
  { key: "multi_device_gpus"             value: { string_value: "0,1" } },
  { key: "multi_device_per_rank_engines" value: { string_value: "true" } }
]

See docs/multi_device.md.

Implementation

• ncclCommInitAll, per-rank deserialize + concurrent setCommunicator (sequential deadlocks), adaptive P2P/host input replication, fan-out enqueueV3, rank-0 output.
• Supports offline-sharded engines and per-rank weight-shard (tensor-parallel) engines (multi_device_per_rank_engines).
• Built behind -DTRITON_ENABLE_TENSORRT_MULTI_DEVICE=ON (TensorRT >= 11 + NCCL); default off, so non-MD builds/models are unchanged.
• docs/ includes the engine builders used for testing.

Validation

Validated on 2× and 8× B200 (NVLink): a sharded model across 2 GPUs matches the 1-GPU baseline (rel_max ~3.6e-3); server logs TensorRT Multi-Device ready: N ranks; both GPUs active.

Notes

• Requires TensorRT >= 11.
• DCO signed-off. Happy to split into smaller commits or adjust naming per maintainer preference.

[pkisfaludi-nv]

cc @mc-nv @whoisj — this is the upstream version of the internal MR you reviewed (TRT-28040, TensorRT Multi-Device / multi-GPU support), submitted here at @mc-nv's request. Would appreciate your review when you have a chance. I couldn't add you as formal reviewers from a fork — please assign yourselves (or let me know who should own it). Thanks!

[mc-nv]

@pkisfaludi-nv
I've change the target brach to 26.06, default one doesn't support the TensorRT 11.
I been tested your changes today and they went through the build process.
#121

cc: @whoisj