Skip to content

[BUG]: examples/global_planner/global-planner-vllm-test.yaml /v1/chat/completions 404 #13636

Description

@thinkeng

Describe the Bug

examples/global_planner/global-planner-vllm-test.yaml

Steps to Reproduce

kubectl get pod -n share-dynamo
NAME READY STATUS RESTARTS AGE
gp-ctrl-0-frontend-kwxlq 1/1 Running 0 16m
gp-ctrl-0-globalplanner-xk97x 1/1 Running 0 18m
gp-ctrl-0-globalrouter-9p2fv 1/1 Running 0 16m
gp-decode-0-0-localrouter-h228m 1/1 Running 0 9m6s
gp-decode-0-0-planner-ljr7w 1/1 Running 0 18m
gp-decode-0-0-vllmdecodeworker-vzfd6 1/1 Running 1 (18m ago) 18m
gp-prefill-0-0-localrouter-pt78p 1/1 Running 0 9m7s
gp-prefill-0-0-planner-pdx5b 1/1 Running 0 18m
gp-prefill-0-0-vllmprefillworker-lhl2b 1/1 Running 0 18m
gp-prefill-1-0-localrouter-f2976 1/1 Running 0 7m10s
gp-prefill-1-0-planner-n4njs 1/1 Running 0 18m
gp-prefill-1-0-vllmprefillworker-4xddx 1/1 Running 2 (17m ago) 18m
:/work/deploy/trtllm/qwen$ kubectl get svc -n share-dynamo
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gp-ctrl-0 ClusterIP None 18m
gp-ctrl-frontend ClusterIP 10.43.233.68 8000/TCP 18m
gp-ctrl-globalplanner ClusterIP 10.43.242.242 9090/TCP 18m
gp-ctrl-globalrouter ClusterIP 10.43.195.91 9090/TCP 18m
gp-decode-0-0 ClusterIP None 18m
gp-decode-0-localrouter ClusterIP 10.43.98.122 9090/TCP 18m
gp-decode-0-planner ClusterIP 10.43.250.60 9090/TCP 18m
gp-decode-0-vllmdecodeworker ClusterIP 10.43.42.147 9090/TCP 18m
gp-prefill-0-0 ClusterIP None 18m
gp-prefill-0-localrouter ClusterIP 10.43.250.131 9090/TCP 18m
gp-prefill-0-planner ClusterIP 10.43.148.5 9090/TCP 18m
gp-prefill-0-vllmprefillworker ClusterIP 10.43.94.244 9090/TCP 18m
gp-prefill-1-0 ClusterIP None 18m
gp-prefill-1-localrouter ClusterIP 10.43.238.92 9090/TCP 18m
gp-prefill-1-planner ClusterIP 10.43.47.195 9090/TCP 18m
gp-prefill-1-vllmprefillworker ClusterIP 10.43.46.50 9090/TCP 18m
:
/work/deploy/trtllm/qwen$ curl http://10.43.233.68:8000/v1/models | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 27 100 27 0 0 39764 0 --:--:-- --:--:-- --:--:-- 27000
{
"object": "list",
"data": []
}

Expected Behavior

get models list

Actual Behavior

curl http://10.43.233.68:8000/v1/models | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 27 100 27 0 0 39764 0 --:--:-- --:--:-- --:--:-- 27000
{
"object": "list",
"data": []
}

Environment

export DYNAMO_IMAGE="nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.4.0"
export DYNAMO_VLLM_IMAGE="nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0"
export MODEL_NAME="Qwen/Qwen3-0.6B"

Additional Context

No response

Screenshots

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    QA Bugbackend::vllmRelates to the vllm backendbugSomething isn't workingdeployment::k8sRelates to dynamo deployment in kubernetesdynamo-plannerdynamo-runtimeRelates to the dynamo-runtime componentfrontend`python -m dynamo.frontend` and `dynamo-run in=http|text|grpc`planner

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions