Skip to content

Refactor av2_rd_pick_inter_mode_sb() - #5342

Merged
urvangjoshi merged 5 commits into
AOMediaCodec:av2-encfrom
mudassir-sg:refactor_rd_pick_inter
Aug 28, 2026
Merged

Refactor av2_rd_pick_inter_mode_sb()#5342
urvangjoshi merged 5 commits into
AOMediaCodec:av2-encfrom
mudassir-sg:refactor_rd_pick_inter

Conversation

@mudassir-sg

Copy link
Copy Markdown
Contributor

No description provided.

@mudassir-sg

Copy link
Copy Markdown
Contributor Author

@yunqingwang1 , @urvangjoshi
/cc @harishdm @ranjit-tulabandu @cherma-a @deepa-kg @RemyaPrakasan @ram-mohan @vikasprasad10

We have verified that this PR is bit-exact with it's parent commit for speeds 0 to 3 across RA, LD, and AI configurations for few test clips.
Note: Each commit in this PR is logically independent and has its own commit message. Please do not squash them.

@yeqing-wu yeqing-wu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for refactoring the code, the code looks good to me.

This change unifies the rectangular sub-block search to improve the
readabilty and avoid code duplication. Unified code is extracted
to new function has_searched_rect_subblock() which scans HORZ/VERT
sub-blocks for a previously searched partition.

No stats changed

Change-Id: I331cf4a82fadc549122a91f2acb7f1e9eef1b2cd
- Abstracted the TPL stage inter/intra cost accumulation to
  calculate_cost_from_tpl_data().
- Abstracted the IntraBC prediction code to
  try_intrabc_after_inter_search().

No stats changed

Change-Id: Icc70b559709078236cc96b4d3af34c5d10cff6ac
- Hoist the frame/sequence level reads out of the mode loop
  into locals.
- Fold the loop-invariant BLOCK_4X4 check into the mode loop
  bound.
- Flatten the BRU gate from three nesting levels to two.

No stats changed

Change-Id: I7d9e8fa4794e1959185e9aa39aedd7aad0cfc319
- Drop is_comp_mode, a second name for comp_pred used
  interchangeably with it in the same loop body.
- Drop the no-op guard around the search_state.best_rd assignment.
- Compute num_amvd_modes and amvd_inverted just above the use_amvd
  loop that consumes them.
- Scope the loop counter i to the loops that use it.
- Initialize the motion mode prune pool with a single flat loop.
- Use the local aliases consistently (e.g: sf-> over cpi->sf etc).
- Add the missing const to the pointer locals that are never
  reassigned.
- Move the main loop and hdres threshold comments next to the code
  they describe.
- Use prefix increment in the two loops that used postfix.

No stats changed

Change-Id: Ic88588aa0ce5986bdabb471cf4322141975a00c5
Gcc reports "array subscript 8 is above array bounds of
unsigned int[8][8]" for ref_costs_comp[ref_frame_index]. The index
only reaches TIP_FRAME_INDEX(8) for TIP, which is single reference
only, so the compound branch is never evaluated with that index.
ref_costs_single is sized to include the TIP slot, ref_costs_comp is
not.

Added the bounds check to the condition so that the invariant is
visible to the compiler.

No stats changed

Change-Id: I518f7414c5958f886e6750ee7402a48ecc1a895a
@urvangjoshi
urvangjoshi force-pushed the refactor_rd_pick_inter branch from 60079d2 to 27e69f9 Compare August 28, 2026 21:53
@urvangjoshi
urvangjoshi merged commit 175be20 into AOMediaCodec:av2-enc Aug 28, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants