test(vllm): restore decode migration coverage

[alec-flowers]

Summary

• Stack this change on test(vllm): re-enable aggregate migration matrix #13684 and restore the vLLM disaggregated decode-worker migration test.
• Replace the inherited eight-row parameter surface with four explicit streaming cases: one for each migration policy outcome, while retaining both shutdown paths, both OpenAI APIs, and both request-plane transports.
• Supply the explicit NIXL kv_both transfer configuration now required by disaggregated vLLM workers.
• Bound the workload to a 1,024-token context and 512 output tokens, then start the independent prefill and decode workers concurrently.
• Require five streamed responses and prove the request is still in flight before injecting the worker fault; a missed causal window now fails instead of logging a warning.
• Add deterministic KV-cache and measured VRAM markers so the cases participate safely in the pre-merge GPU scheduler during qualification.
• Apply a measured 350-second pytest timeout and leave a TODO to return the restored matrix to nightly after no-retry pre-merge CI passes.

Scope reduction and traceability

The inherited covering array supplied eight rows to every migration test. Decode migration can only inject its fault after a response stream has started, so the four unary rows were not applicable and immediately self-skipped.

Original equivalence class	Permanent decode case
Migration enabled, no sequence cap	migration_enabled-worker_failure-chat-stream-nats
Migration disabled	migration_disabled-graceful_shutdown-completion-stream-nats
Migration sequence cap exceeded	max_seq_len_exceeded-worker_failure-completion-stream-tcp
Finite sequence cap not exceeded	max_seq_len_not_exceeded-graceful_shutdown-chat-stream-tcp
Four unary rows	Omitted: the complete response leaves no in-flight decode request to fault

This changes decode collection from eight outer-skipped rows (and four additional inner skips if the outer skip was bypassed) to four named executable cases with no skip.

Live root cause

Forced execution in the exact nightly image failed before model readiness:

ValueError: --connector is deprecated and the default is no longer nixl.
When using --disaggregation-mode prefill, you must explicitly provide
--kv-transfer-config.

The test harness had not followed the current disaggregated vLLM CLI contract. Adding the canonical NixlConnector/kv_both configuration lets the test reach and validate the complete decode migration chain. The enabled chat case also exercises the historical request-ID/prefix-cache regression described by the removed skip.

Validation

Exact image:

210086341041.dkr.ecr.us-west-2.amazonaws.com/ai-dynamo/dynamo:10d71514e9c9c1c6a1b902be4b692378e78bff8a-vllm-runtime-nightly-test

• Exact-image collection: four decode node IDs collected, zero target skips.
• Focused repaired case with sequential worker startup: 1/1 passed first attempt in 183.29s.
• Focused case with concurrent startup: 1/1 passed first attempt in 104.47s, a 43% reduction.
• VRAM profile with the deterministic 331,711,000-byte cache cap: 1/1 passed; 6.8 GiB peak; 0 MiB leaked (668 MiB baseline and final).
• All retained cases together using -n 4 --max-vram-gib=41: 4/4 passed first attempt, zero retries/skips, in 150.05s (2m30s); scheduler reported 3.5x versus 520s sequential and observed an 18.1/48 GiB suite peak.
• Post-suite GPU process query returned no compute processes.
• isort, direct single-worker black --check, flake8, codespell, ruff, whitespace checks, py_compile, and git diff --check passed.
• The repository-wide marker report found zero missing marker sets across 4,806 tests, but its local hook exited nonzero because unrelated collection exhausted the shared workstation port allocator after GPU runs. Exact-image target collection above passed.

No retry-assisted result is included, and no new test was added: this restores the existing behaviorally justified end-to-end test at its smallest meaningful surface.

Stack navigation

1. #13684 — Aggregate migration
2. #13692 — Prefill removal · DYN-4118
3. #13693 — Decode migration · DYN-4119
4. #13694 — KV-transfer migration · DYN-4120

This PR is layer 3 and targets codex/vllm-prefill-migration. Review and merge the stack in order.

---

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.