Fix checkpointed prefix-cache result reconstruction

[lmcafee-nvidia]

What does this PR do?

This PR correctly reconstructs the final result for a request split across multiple checkpoint records. The old optional-list merge decided the entire result from the first record, so an initial None discarded values produced by later records. Routing reconstruction was also limited to requests with exactly one record. The fix merges every non-None segment in order, returns None only when every segment is absent, and reconstructs per-block routing regardless of checkpoint count.

Example failure scenario(s)

• request A has 512 prompt tokens and produces 4 output tokens.
• the engine pauses and resumes it, creating 2 result records.
• record 1 contains 0 output scores, while record 2 contains 4.
• before the fix, merging returns 0 scores and 0 routing rows.
• after the fix, merging returns 4 scores and 515 routing rows.

How was this bug discovered?

A focused prefix-cache scenario paused and resumed a request before its first checkpoint contained scores, then inspected the joined result at completion. Scores produced after the resume were missing, and the recorded MoE expert choices were omitted; the expected result was the ordered union of all available score segments plus routing for every completed block. The detection oracle was exact final-result reconstruction: score presence and order, token alignment, and per-block routing indices.

Dependency and scope

This draft is based on prefix-cache-core-stress (NVIDIA #6418). Relative to that base, it changes only:

• megatron/core/inference/engines/dynamic_engine.py
• megatron/core/inference/inference_request.py
• tests/unit_tests/inference/contexts/test_dynamic_prefix_caching.py (+78/-2)

It adds no functional tests, recipes, or unrelated production changes.

Validation

Exact signed tip: 0c05dba3fb0595a548d65cde896462b5a290522d

• DFW Slurm job 15520293: passed
  • focused 8-rank run: 1 passed, 69 deselected, 0 failed/skipped on each rank
  • 8 rank reports collected
  • static checks passed