Tune variance partition thresholds for low resolutions in RTC - #5256
Draft
jianj-g wants to merge 1 commit into
Draft
Tune variance partition thresholds for low resolutions in RTC#5256jianj-g wants to merge 1 commit into
jianj-g wants to merge 1 commit into
Conversation
jianj-g
marked this pull request as draft
August 12, 2026 19:08
jianj-g
force-pushed
the
rtc-vbp-tune-thresh
branch
2 times, most recently
from
August 28, 2026 14:03
013571e to
2242cf9
Compare
Tune variance-based partition (VBP) thresholds and sub-block split evaluation for real-time (RTC) encoding mode: 1. Update threshold_base formula to properly model AV2's exponential quantizer step size across QP, replacing the linear AC quantizer scale. 2. Refine resolution-based threshold scaling across <= 288p, 360p, 720p, and 1080p+, and smooth the high-QP scaling slope. 3. Enhance sub-block variance spread evaluation to avoid unnecessary 8x8 splits on static/homogeneous blocks while preserving splits for fine details and screen content. 4. Guard 64x64 block splitting based on maximum sub-block variance. BD-rate and speedup summary: rtc_derf (20 clips): avg_psnr: -2.17% (median: -2.29%) ovr_psnr: -2.13% (median: -2.29%) vmaf: -3.60% (median: -5.14%) encoding speedup: +1.25% (median: +0.47%) rtc (28 clips): avg_psnr: -3.04% (median: -2.80%) ovr_psnr: -3.03% (median: -2.79%) vmaf: -4.97% (median: -5.01%) encoding speedup: +2.15% (median: +2.55%) STATS_CHANGED for RTC mode
jianj-g
force-pushed
the
rtc-vbp-tune-thresh
branch
from
August 28, 2026 14:13
2242cf9 to
4df3421
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tune variance-based partition (VBP) thresholds and sub-block split
evaluation for real-time (RTC) encoding mode:
quantizer step size across QP, replacing the linear AC quantizer scale.
and 1080p+, and smooth the high-QP scaling slope.
splits on static/homogeneous blocks while preserving splits for fine
details and screen content.
BD-rate and speedup summary:
rtc_derf (20 clips):
avg_psnr: -2.17% (median: -2.29%)
ovr_psnr: -2.13% (median: -2.29%)
vmaf: -3.60% (median: -5.14%)
encoding speedup: +1.25% (median: +0.47%)
rtc (28 clips):
avg_psnr: -3.04% (median: -2.80%)
ovr_psnr: -3.03% (median: -2.79%)
vmaf: -4.97% (median: -5.01%)
encoding speedup: +2.15% (median: +2.55%)
STATS_CHANGED for RTC mode