docs(bench): re-measure the grain ladder at 9x sample — several published figures were wrong - #53
Merged
Merged
Conversation
…es were wrong The first profile was 86 titles at one scene each. That was too weak to publish, and re-running it properly shows why. Now 752 titles across 2230 scenes: 418 films + 334 TV episodes drawn from all 183 series, THREE scenes per title (films 20/40/60 min, episodes 5/15/25 min), 12 frames each, per-title value the median across its scenes. Native centre crops throughout, so per-pixel grain survives. What changed. Individual decade buckets moved by up to 78% and the overall median by 38% (0.00666 -> 0.00484). Two published conclusions were false: the 1960s are NOT the grainiest decade (mid-pack at 0.0049; the 1970s-80s peak at ~0.0072), and TV does NOT stay grainier than film into the 2010s (they converge at ~0.0046). Why the small probe failed, quantified: the within-title scene-to-scene spread has a median of 0.71x the title's own median, p90 2.02x. A dark interior and a bright exterior in one film differ more than two different films do, so a single-scene probe typically misestimates by ~36% and for a tenth of titles by 100%+. The largest effect in the data was invisible at n=86: the 1080p tier is 2.2x grainier than the 4K tier (median 0.00724 vs 0.00323, like-for-like native crops). 4K masters are denoised hard enough to compress, so they have already had the impairment removed. For a project whose thesis is that reductive filtering wins in proportion to removable impairment, that is the actionable line: target 1080p sources for bench material. The ladder itself and the netflix-bar/--grain<=16 recommendation are unchanged -- those were measured directly, not sampled. The document now carries a revision section recording the wrong numbers rather than quietly replacing them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first profile was 86 titles at one scene each. That was too weak to publish, and re-running it properly shows exactly why.
Now 752 titles across 2230 scenes: 418 films + 334 TV episodes from all 183 series, three scenes per title (films 20/40/60 min, episodes 5/15/25 min), 12 frames each, per-title value = median across its scenes. Native centre crops throughout, so per-pixel grain survives.
What the small probe got wrong
Two published conclusions were simply false:
Why, quantified
Within-title scene-to-scene spread (max-min, relative to that title's own median):
A dark interior and a bright exterior in the same film differ more than two different films do. So a single-scene probe typically misestimates a title by ~36%, and for a tenth of titles by 100%+. Three scenes per title is the minimum honest design.
The largest effect was invisible at n=86
1080p masters are 2.2x grainier than 4K masters — like-for-like native crops, so not a scaling artifact. 4K releases are denoised hard enough to compress, meaning someone else has already removed the impairment.
For a project whose thesis is that reductive filtering wins in proportion to removable impairment, that is the most actionable line in the document: target 1080p sources for bench material.
Unchanged
The ladder table and the
netflix-barbase /--grain <= 16ceiling stand — those were measured directly, not sampled.The document now carries a revision-history section recording the wrong numbers rather than quietly replacing them.
🤖 Generated with Claude Code