Skip to content

fix(render): account for rotation metadata in portrait detection - #137

Open
sidorovanthon wants to merge 2 commits into
browser-use:mainfrom
sidorovanthon:fix/rotation-aware-portrait
Open

fix(render): account for rotation metadata in portrait detection#137
sidorovanthon wants to merge 2 commits into
browser-use:mainfrom
sidorovanthon:fix/rotation-aware-portrait

Conversation

@sidorovanthon

@sidorovanthon sidorovanthon commented Aug 24, 2026

Copy link
Copy Markdown

Problem

Portrait detection currently compares coded width and height only. Phone and camera files commonly store landscape-coded frames plus 90 or 270 degree display rotation. ffmpeg autorotates those frames before filters run, but render.py still selects the landscape scale axis, producing an oversized vertical output.

This edge case was identified by cubic during review of #29.

Fix

  • Probe coded dimensions, display-matrix rotation, and the legacy rotate tag in one ffprobe call.
  • Swap the effective dimensions for 90 and 270 degree rotations before selecting the scale axis.
  • Preserve existing behavior for native portrait and landscape sources.

Verification

  • Six unit tests cover native portrait/landscape, modern side-data rotation, legacy rotate tags, reverse orientation, and invalid probe output.
  • Verified with a real synthetic MP4 carrying a 90 degree display matrix: ffprobe reported 320x180 plus rotation=90 and is_portrait_source returned true.
  • python -m unittest discover -s tests -v
  • python -m py_compile helpers/render.py tests/test_render_orientation.py

Summary by cubic

Fixes portrait detection to honor display-matrix rotation side data so vertical sources scale on the correct axis. Previously we compared coded dimensions only, treating 90/270° rotated videos as landscape.

  • Uses one ffprobe JSON probe to read width/height and stream_side_data=rotation; ignores plain rotate tags because they do not guarantee autorotation.
  • Swaps effective dimensions for quarter-turn rotations before choosing the scale axis; preserves behavior for native portrait and landscape.
  • Defaults to landscape on invalid probe output; adds tests for side-data rotations, ignoring rotate, reverse orientation, and error handling.

Written for commit 4ff8e60. Summary will update on new commits.

Review in cubic

Read display-matrix rotation and legacy rotate tags alongside coded dimensions. Swap dimensions for quarter-turn rotations so ffmpeg's autorotated filter input uses the correct scale axis.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread helpers/render.py Outdated
Comment thread tests/test_render_orientation.py Outdated
Ignore plain rotate tags that do not guarantee ffmpeg autorotation and assert that the ffprobe query continues to request display-matrix rotation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant