docs(datasets): document Daily-Omni and Video-R1-260k prep for Qwen3-Omni RL - #292
Merged
Merged
Conversation
…Omni RL The two Qwen3-Omni GSPO recipes ship conversion scripts but no instructions, so the upstream download, the intermediate format each converter expects, and the pipeline constraints behind every conversion choice were undocumented. Add a README per dataset covering the source and download commands, the cooking steps, and why each output field is shaped that way: MultimodalRLDataSource only accepts (image, condition), (video, condition) and (video, prompt), so audio reaches Qwen3-Omni through use_audio_in_video rather than an audio media ref, and MCExactMatchRewardScorer returns 0.0 instead of raising on a malformed metadata.answer. Both also recommend pre-filtering prompts whose whole group scores identically, since group-normalized advantages are exactly 0 there and UniRL has no DAPO-style dynamic sampling to skip them at runtime. Note that .gitignore's datasets/* rule covers these files, so they need git add -f, as did the already-tracked converters and image_edit README. Co-authored-by: Cursor <cursoragent@cursor.com>
github-actions
Bot
added
approved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
qa.jsondirectly, resolve the releasedVideos/layout, split byvideo_id, and append the answer format required by the recipes.Related Issue
N/A
Test Plan
SKIP=no-commit-to-branch pre-commit run --files datasets/daily_omni_av/README.md datasets/daily_omni_av/convert_daily_omni_dataset_format_to_unirl.py datasets/video_r1_260k/README.md --show-diff-on-failurepython3 datasets/daily_omni_av/convert_daily_omni_dataset_format_to_unirl.py --helppython3 datasets/video_r1_260k/convert_video_r1_260k_to_unirl.py --help--val-ratio 0.5; verified video-disjoint train/val splits, unique prompt IDs, normalized answers, absolute media URIs, and the required answer instruction.Compatibility / Risk
The Daily-Omni converter CLI changes from pre-split
--train-input/--val-inputJSONL files to the official--qa-jsoninput plus optional--videos-root. The UniRL output schema is unchanged. Video-R1 runtime behavior is unchanged.Reviewer Notes
AI-assisted review and documentation cleanup (Codex); the maintainer reviewed the full diff and ran the checks above. This updates the existing PR only; no duplicate PR was opened.
Checklist