Skip to content

Add grouped GOP multi-frame decode API - #47

Merged
xupinjie merged 3 commits into
NVIDIA:mainfrom
KazukiKomon:zhy/grouped-gop-api
Aug 6, 2026
Merged

Add grouped GOP multi-frame decode API#47
xupinjie merged 3 commits into
NVIDIA:mainfrom
KazukiKomon:zhy/grouped-gop-api

Conversation

@KazukiKomon

Copy link
Copy Markdown
Contributor

Description

UniVerse requests several frames from one camera with repeated GOP bundles. The existing list path serializes the same GOP once per output frame and creates one NVDEC task per frame.

This change adds GetGOPGroups(requests) and DecodeFromGOPGroupsRGB(groups) so one serialized payload and one decoder task cover all requested frames in a source/GOP. Cross-GOP requests are split automatically, while source indices, original order, and duplicate positions are preserved. Grouped payloads are validated and decoder slots are pooled by native stream configuration.

The port is based on the current upstream main: it preserves the newer skip_final_sync path and does not restore legacy APIs removed upstream.

Affected component: accvlab.on_demand_video_decoder.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation / examples / tutorials / demos
  • Supporting functionality change
  • Refactoring / internal change

Testing

  • Tests added or updated if/as needed
  • Repository test runner executed: scripts/run_tests.sh

Validation performed for this upstream port:

  • CPython 3.14 / CUDA 13 wheel built successfully with accvlab-build:py3.14-cu130.
  • git diff --check passed.
  • GPU tests could not be rerun on this host because the NVIDIA driver is unavailable.

Validation of the original change before porting:

  • Full on-demand decoder GPU suite: 188 passed.
  • Unordered duplicate request [45, 6, 25, 6] split across three GOPs and matched the legacy output byte-for-byte.
  • Four-camera frames 2728-2731 produced identical images, camera indices, and timestamps.
  • Serialized payload decreased from 217,657,968 to 54,414,504 bytes (-75.0%).
  • NVDEC instances decreased from 16 to 4; cuvidDecodePicture calls decreased from 2,672 to 674.

Documentation, Examples, Tutorials, Demos

  • User-facing API docstrings updated
  • Examples / tutorials / demos updated or added
  • Limitations and constraints documented in API docstrings
  • Performance documented in this PR
  • Documentation build executed

Code Quality

  • No dependency changes required
  • git diff --check passed

Related Issues / Context

This is a clean single-commit port of the previously reviewed grouped GOP implementation onto the latest upstream main.

DCO / Sign-Off

The commit includes Signed-off-by: hongyizhang <hongyizhang02@deeproute.ai>.

@KazukiKomon
KazukiKomon force-pushed the zhy/grouped-gop-api branch from 5f06af1 to c14ca1a Compare July 22, 2026 11:25
Add grouped GOP extraction and decoding so multiple requested frames from one source GOP share a serialized payload and NVDEC traversal.

Preserve source, order, and duplicate mappings; split cross-GOP requests; validate grouped payloads; and pool decoders by native stream configuration.

Signed-off-by: hongyizhang <29877338+KazukiKomon@users.noreply.github.com>
@KazukiKomon
KazukiKomon force-pushed the zhy/grouped-gop-api branch from c14ca1a to c86e617 Compare July 22, 2026 11:27
assert group["first_frame_id"] == 0
assert group["gop_len"] == 20

native_demuxer = nvc.PyNvGopDecoder(maxfiles=1, iGpu=0)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use this API nvc.CreateGopDecoder(...). Direct construction of a decoder via its constructor is currently not supported.

):
assert native_groups[0][key] == group[key]

legacy_data, _, _ = demuxer.GetGOP([OPEN_GOP_SAMPLE] * len(expected_ids), expected_ids)

@xupinjie xupinjie Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GetGOP API has been deprecated. Please use GetGOPList instead.

@xupinjie

Copy link
Copy Markdown
Collaborator

Fixes #45

Comment on lines +774 to +776
R"pbdoc(
Extract one serialized payload for each unique source/GOP.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest explicitly mentioning in the API documentation that each group may contain a different number of frame_ids; the lengths do not need to be aligned. This is the key distinction from conventional batch-style APIs in the ACCV-Lab.

@xupinjie

xupinjie commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

GetGOPGroups / DecodeFromGOPGroupsRGB Example

These two APIs are designed to be used together:

groups = decoder.GetGOPGroups(requests)
decoded_groups = decoder.DecodeFromGOPGroupsRGB(groups)

Assume that video_a.mp4 has two GOP ranges: [0, 20) and [20, 40).

requests = [
    {
        "filepath": "video_a.mp4",
        "frame_ids": [9, 6, 7, 7, 25],
    },
    {
        "filepath": "video_b.mp4",
        "frame_ids": [3],
    },
]

groups = decoder.GetGOPGroups(requests)

GetGOPGroups sorts and deduplicates the frame IDs, then splits them by GOP. Its output is approximately:

groups = [
    {
        "gop_data": gop_data_a_0_20,
        "source_index": 0,
        "source_name": "video_a.mp4",
        "frame_ids": [6, 7, 9],
        "frame_positions": [[1], [2, 3], [0]],
        "first_frame_id": 0,
        "gop_len": 20,
    },
    {
        "gop_data": gop_data_a_20_40,
        "source_index": 0,
        "source_name": "video_a.mp4",
        "frame_ids": [25],
        "frame_positions": [[4]],
        "first_frame_id": 20,
        "gop_len": 20,
    },
    {
        "gop_data": gop_data_b_0_20,
        "source_index": 1,
        "source_name": "video_b.mp4",
        "frame_ids": [3],
        "frame_positions": [[0]],
        "first_frame_id": 0,
        "gop_len": 20,
    },
]

The returned groups can be passed directly to the decode API:

decoded_groups = decoder.DecodeFromGOPGroupsRGB(groups, as_bgr=False)

The result preserves the mapping metadata, removes gop_data, and adds frames:

decoded_groups = [
    {
        "source_index": 0,
        "source_name": "video_a.mp4",
        "frame_ids": [6, 7, 9],
        "frame_positions": [[1], [2, 3], [0]],
        "first_frame_id": 0,
        "gop_len": 20,
        "frames": [frame_6, frame_7, frame_9],
    },
    {
        "source_index": 0,
        "source_name": "video_a.mp4",
        "frame_ids": [25],
        "frame_positions": [[4]],
        "first_frame_id": 20,
        "gop_len": 20,
        "frames": [frame_25],
    },
    {
        "source_index": 1,
        "source_name": "video_b.mp4",
        "frame_ids": [3],
        "frame_positions": [[0]],
        "first_frame_id": 0,
        "gop_len": 20,
        "frames": [frame_3],
    },
]

frame_ids, frames, and frame_positions correspond element by element. For example:

"frame_ids":       [6,       7,        9]
"frames":          [frame_6, frame_7,  frame_9]
"frame_positions": [[1],     [2, 3],   [0]]

Frame 7 appears twice in the original request but is decoded only once. [2, 3] means that the decoded frame_7 belongs at positions 2 and 3 in the original request.

The original request order can be restored as follows:

restored = [[None] * len(request["frame_ids"]) for request in requests]

for group in decoded_groups:
    destination = restored[group["source_index"]]
    for frame, positions in zip(group["frames"], group["frame_positions"]):
        for position in positions:
            destination[position] = frame

# restored:
# [
#     [frame_9, frame_6, frame_7, frame_7, frame_25],
#     [frame_3],
# ]

@KazukiKomon
KazukiKomon force-pushed the zhy/grouped-gop-api branch from 1cd6132 to bda1b71 Compare August 6, 2026 10:03
Signed-off-by: hongyizhang <hongyizhang02@deeproute.ai>
@KazukiKomon
KazukiKomon force-pushed the zhy/grouped-gop-api branch from bda1b71 to 4edff94 Compare August 6, 2026 10:04
@xupinjie

xupinjie commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

/build

@xupinjie

xupinjie commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

/build

@xupinjie xupinjie left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution.

@xupinjie
xupinjie merged commit 3797549 into NVIDIA:main Aug 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants