docs(bench): profile the grain ladder against 86 real titles - #51
Merged
Conversation
--grain N was an ffmpeg noise value with no physical meaning and no stated safe range. Profiled it against a real library: 86 titles (56 films + 30 TV series, one episode each) stratified across the 1960s-2020s, 24 frames each at a fixed seek, measured both as a native centre crop (true per-pixel grain) and scaled to 640x360 (what the corpus actually feeds the filters). Findings: - Real-content grain_sigma spans 0.0011-0.0265, median 0.0067. - Grain COLLAPSES in the 2020s: film 0.0022, TV 0.0028, against ~0.008 for the 1960s-80s. Median grain_flat for 2020s film is 1.000 -- the estimator finds the frame essentially entirely flat. This is consistent with the repeated honest negatives on clean content and is why the grain axis must be injected. - TV is grainier than film from the 1990s on (1990s TV 0.0105 vs film 0.0064). Film-sourced series retain grain that feature Blu-rays often have remastered out, so TV is the better hunting ground -- thanks to the user for the prompt. - Downscaling to 640x360 HALVES the interquartile range (0.0064 -> 0.0038): it lifts the floor with resampling residual and lowers the ceiling by averaging real grain away. Benchmarking at the corpus resolution therefore under-detects grain differences, which is a caveat on any BD-rate delta attributed to grain. Actionable output: use netflix-bar as the grain base and stay at or below --grain 16 (flat >= 0.44). Do NOT use bbb -- it starts at only 33% flat coverage and its axis goes non-monotonic past 16, sigma peaking at 0.0244 then falling to 0.0181 by grain 24. A non-monotonic axis is not an axis. Also records honestly that grain_sigma measures high-frequency residual in flat areas, of which codec noise is a contributor alongside film grain: both corpus clips read ABOVE nearly all real content (0.0124-0.0131 vs a 0.0067 median) despite being the clean sources, because they are compressed low-resolution distribution encodes. Cross-source absolute comparison is only safe between comparably-encoded sources. Only aggregate statistics are recorded; no content is redistributed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--grain Nwas an ffmpegnoise=alls=value with no physical meaning and no stated safe range. This profiles it against a real library so a value can be read as "about the grain of X", and marks where the measurement stops being trustworthy.Method. 86 titles — 56 films + 30 TV series (one episode each) — stratified across the 1960s–2020s, 24 frames each at a fixed seek, measured two ways: a native centre
crop=640:360(true per-pixel grain) andscale=640:360(what the corpus feeds the filters). Only aggregate statistics recorded; no content redistributed.What real content measures (native crop, n=84)
grain_sigmaMedian by decade:
Grain collapses in the 2020s — 0.0022 film, with median
grain_flatof 1.000: the estimator finds the frame essentially entirely flat. That is consistent with the repeated honest negatives on clean content, and it's why the grain axis has to be injected rather than selected.TV is grainier than film from the 1990s on (1990s TV 0.0105 vs film 0.0064) — film-sourced series keep grain that feature Blu-rays often have remastered out. Thanks for the nudge; folding TV in materially changed the picture.
Downscaling hides grain differences
Medians barely move but the interquartile range halves (0.0064 → 0.0038): downscaling lifts the floor with resampling residual and lowers the ceiling by averaging real grain away. Benchmarking at the corpus resolution therefore under-detects grain differences — a caveat on any BD-rate delta attributed to grain.
The actionable output
Use
netflix-baras the grain base, stay at or below--grain 16:--graingrain_sigmagrain_flatDo not use
bbb— it starts at only 33% flat coverage and goes non-monotonic past 16: sigma peaks at 0.0244 then falls to 0.0181 by grain 24. A non-monotonic axis is not an axis.Honest caveat
Both corpus clips read higher than nearly all real content (0.0124–0.0131 vs a 0.0067 median) despite being the clean sources — they're compressed low-resolution distribution encodes, and codec artifacts put high-frequency residual exactly where the estimator looks. So
grain_sigmais high-frequency residual in flat areas, with grain one contributor and codec noise another. Fine for deciding whether removable impairment exists; not safe for cross-source absolute comparison unless encodes are comparable.🤖 Generated with Claude Code