Support GOP-grouped multi-frame extraction to avoid redundant demux and NVDEC

[KazukiKomon]

Summary

When a training sample requests several frames from the same video, the current
GetGOPList(filepaths, frame_ids) / DecodeFromGOPListRGB(...) interface is
frame-oriented. The caller has to repeat the filepath:

gop_list = demuxer.GetGOPList(
    [camera_path] * 4,
    [6, 7, 8, 9],
)
gop_data = [item[0] for item in gop_list]
frames = decoder.DecodeFromGOPListRGB(
    gop_data,
    [camera_path] * 4,
    [6, 7, 8, 9],
)

If frames 6, 7, 8, and 9 belong to the same GOP, this produces multiple copies
of the same serialized packet bundle and schedules multiple decode tasks for
the same GOP. NVDEC consequently replays the same encoded I/P/B packet sequence
once per requested frame instead of decoding the GOP once and retaining all
requested outputs.

A cache between samples or batches does not eliminate this work: the
duplication already exists inside one sample/request. The cost becomes
especially visible with long GOPs and contributed several seconds of redundant
work to the first batch in our training workload.

The same abstraction is also awkward when one source request spans several
GOPs. The natural result is one payload per unique (source, GOP), not one
payload per requested frame.

Proposed grouped API

We implemented and validated the following request-oriented API in our fork:

groups = demuxer.GetGOPGroups(
    [
        {
            "filepath": camera_path,
            "frame_ids": [9, 6, 8, 6],
        }
    ]
)

decoded_groups = decoder.DecodeFromGOPGroupsRGB(groups, as_bgr=False)

GetGOPGroups returns one dictionary per unique source/GOP:

{
    "gop_data": serialized_packet_bundle,
    "source_index": 0,
    "source_name": camera_path,
    "frame_ids": [6, 8, 9],
    "frame_positions": [[1, 3], [2], [0]],
    "first_frame_id": 0,
    "gop_len": 12,
}

Here:

• gop_data contains the encoded packet bytes and packet metadata for one GOP,
  not decoded pixels.
• frame_ids contains sorted, unique decode targets within that GOP.
• frame_positions[i] records every position at which frame_ids[i] appeared
  in the original request, so duplicate targets and original ordering can be
  reconstructed exactly.
• source_index maps a group back to the input request. If one request spans
  three GOPs, three dictionaries are returned with the same source_index.

For example, requesting four frames that span three GOPs produces three
serialized bundles and three NVDEC tasks. Requesting four frames from one GOP
produces one bundle and one NVDEC task.

Implementation approach

Our implementation:

1. Normalizes each source request into sorted unique frame IDs while retaining
   all original positions.
2. Uses the existing GOP extractor with the first unassigned target as a
   representative, then assigns every remaining target covered by the returned
   [first_frame_id, first_frame_id + gop_len) range to that group.
3. Repeats only when the same source has targets in another GOP.
4. Parses and decodes each serialized GOP bundle once, collecting all requested
   frames during that decode pass.
5. Returns group metadata unchanged with the decoded frames so the caller can
   scatter results back to source/request order.

We also found two related implementation details important:

• Persistent decoder slots must be matched by codec and native stream shape
  rather than by list position. This safely handles batches containing videos
  with different resolutions and group orders.
• maxfiles should be a capacity limit, not an eager thread count. Demux/decode
  runners should be created on demand for active sources/groups; constructing a
  decoder with maxfiles=256 should not create 256 threads for a request that
  touches one video.

Validation performed

We added regression coverage for:

• multiple requested frames in one GOP;
• duplicate and non-monotonic frame IDs;
• one source request spanning multiple GOPs;
• open-GOP boundary/overlap cases;
• mixed resolutions and changing group order;
• lazy thread creation;
• output comparison against the existing random-access decode path.

The grouped path produced the same requested RGB frames while removing repeated
GOP payloads and repeated NVDEC passes.

Question

Would the maintainers be interested in a grouped multi-frame API along these
lines? We have a working implementation, tests, and API documentation and can
prepare an upstream pull request if this direction fits the project.

[xupinjie]

Fixed in #47