feat(core): percentile pooling and Windows UTF-8 path contract (ADR-1181/1182)

[lusoris]

Summary

Two additive public-API / compatibility features that close long-standing upstream issues Netflix#818 and Netflix#1568.

Part A — percentile pooling (ADR-1181). Adds VMAF_POOL_METHOD_MEDIAN, VMAF_POOL_METHOD_PERC5, VMAF_POOL_METHOD_PERC10 and VMAF_POOL_METHOD_PERC20 to enum VmafPoolingMethod. The streaming accumulators in pool_reduce() cannot express a percentile, so vmaf_feature_score_pooled() routes the percentile family to pool_percentile(), which materialises the (subsample-filtered) per-frame scores, sorts them and interpolates linearly between neighbouring ranks — the same convention NumPy's percentile uses by default, which is what the Python harness compares against. core/src/pooling_percentile.h holds the interpolation itself. XML and JSON reports emit every registered pooling method, so every <metric> row and JSON object gains four attributes; FFmpeg filter patches 0005 / 0006 / 0013 and the Go bindings in pkg/libvmaf/ follow the enum.

Part B — Windows UTF-8 path contract (ADR-1182). core/src/compat/path_utf8.{h,c} adds vmaf_fopen_utf8 and vmaf_open_utf8. On Windows they convert the UTF-8 path (and mode) to UTF-16 and call _wfopen / _wopen, mapping ERROR_NO_UNICODE_TRANSLATION to EILSEQ and ERROR_INSUFFICIENT_BUFFER to ENAMETOOLONG; on POSIX they delegate straight to fopen / open, so the byte-sequence behaviour is unchanged. 17 narrow fopen / _open call sites across libvmaf.c, the model loaders and the fork-added tools move onto the shims, which makes UTF-8 the documented contract for every path the library and CLI accept.

Follow-up already tracked: T-WINDOWS-CLI-WMAIN-ARGV-UTF8-2026-09-05 — the Windows CLI still receives narrow ACP argv from the OS runtime, which is a separate wmain change.

Type

• feat — percentile pooling methods and Windows UTF-8 path contract shim

Checklist

• Commits follow Conventional Commits.
• pre-commit run --files green on every touched file; clang-tidy -p build reports zero findings on core/src/compat/path_utf8.c, core/src/libvmaf.c, core/test/test_path_utf8.c and core/test/test_pooling_percentile.c (the ADR-0141 / ADR-1142 requirement for the files this PR adds or moves off their baseline).
• Unit tests: core/test/test_pooling_percentile.c and core/test/test_path_utf8.c.
• Docs in the same PR: docs/api/index.md (the VmafPoolingMethod table and the UTF-8 path contract section) and docs/usage/cli.md (the pooled-metrics output-schema change).
• SIMD/GPU, twins, new C sources, breaking change, ADR — new C sources core/src/compat/path_utf8.{h,c} and core/src/pooling_percentile.h; no SIMD/GPU twin changes; additive, non-breaking API; ADR-1181 and ADR-1182.

Bug-status hygiene (ADR-0165)

• docs/state.md — moved T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 and T-UPSTREAM-1568-WINDOWS-NARROW-PATH-API-2026-09-03 to Recently closed; added follow-up T-WINDOWS-CLI-WMAIN-ARGV-UTF8-2026-09-05 to Open.

Netflix golden-data gate (ADR-0024)

• I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.

Deep-dive deliverables (ADR-0108)

• Research digest — no digest needed: two targeted additive changes whose specification is the upstream issue itself (Netflix#818, Netflix#1568) plus documented standard-library / Win32 APIs.
• Decision matrix — docs/adr/1181-percentile-pooling-methods.md § Alternatives considered and docs/adr/1182-windows-utf8-path-contract.md § Alternatives considered
• AGENTS.md invariant note — core/src/AGENTS.md
• Reproducer / smoke-test command — below.
• CHANGELOG fragment — changelog.d/added/pooling-percentile-methods.md, changelog.d/fixed/windows-utf8-paths.md
• Rebase note — docs/rebase-notes.md entry

Reproducer

meson setup build core -Denable_cuda=false -Denable_sycl=false -Db_lto=false
ninja -C build
meson test -C build --suite=fast
# Ok: 116  Fail: 0
#   8/116 fast - libvmaf:test_path_utf8            OK
#  44/116 fast - libvmaf:test_pooling_percentile   OK

# The new pooled attributes on the documented 576x324 pair:
./build/tools/vmaf --reference python/test/resource/yuv/src01_hrc00_576x324.yuv \
  --distorted python/test/resource/yuv/src01_hrc01_576x324.yuv \
  --width 576 --height 324 --pixel_format 420 --bitdepth 8 \
  --model version=vmaf_v0.6.1 --output scores.xml --xml
grep 'name="vmaf"' scores.xml
# <metric name="vmaf" min="71.174772" max="87.180960" mean="76.667831"
#         harmonic_mean="76.508907" median="76.091664" perc5="72.351853"
#         perc10="72.717340" perc20="73.357468" />

🤖 Generated with Claude Code

[lusoris]

Duplicate resolution: the percentile-pooling half of this PR is superseded by #1340

This PR is not being closed — its Part B (Windows UTF-8 path contract, ADR-1182,
Netflix#1568, core/src/compat/path_utf8.{h,c} + 17 narrow-fopen/_open call sites)
is unique work that nothing else in flight covers, and closing it would destroy that.

But Part A (percentile pooling, ADR-1181) duplicates #1340 and loses on the evidence.
Concrete, diff-level differences:

1. Heap buffer overflow on the fork's own documented idiom.
vmaf_feature_score_pooled() here sizes the score buffer as

const unsigned capacity = (index_high - index_low + 1);
double *scores = (double *)malloc(capacity * sizeof(double));

docs/api/index.md:611 documents and demonstrates
vmaf_feature_score_pooled(vmaf, "psnr_y", ..., 0, UINT_MAX), which makes
capacity == UINT_MAX - 0 + 1 == 0 in unsigned arithmetic. malloc(0) returns a
non-NULL zero-length block and the loop then writes scores[0..n_frames-1] into it —
8 * n_frames bytes past the end. Any intermediate index_high is also sized eagerly
(a 10-million-index range on a 100-frame clip demands 80 MB). #1340 grows the buffer
geometrically from 64 with an explicit wrap check and -ENOMEM, and its
core/src/libvmaf.c comment names this exact idiom as the reason.

2. Silent output-schema widening. This PR raises VMAF_POOL_METHOD_NB 5 → 9 but
leaves both report writers looping for (unsigned j = 1; j < VMAF_POOL_METHOD_NB; j++)
(core/src/output.cpp xml_write_one_metric_pools and json_write_pooled_entry), so
every pooled_metrics JSON object and XML <metric> element silently gains median,
perc5, perc10, perc20 keys. That is a consumed output schema and the PR body does
not mention it. #1340 introduces pool_report_order[], pins the reported set to the
historical four, and documents that as a deliberate decision in ADR-1188, leaving the
log bytes identical.

3. Tests. This PR ships 2 tests: a 5-value math unit test and one end-to-end test
that needs the src01_hrc00/01_576x324.yuv fixture plus a loaded vmaf_v0.6.1 model,
asserting at 1e-4. #1340 ships 6 fixture-free tests driven through
vmaf_import_feature_score: exact numpy.percentile(method="linear") agreement at
1e-12 on the golden pair's 48 real per-frame scores, an interpolation-vs-nearest-rank
discriminator, arrival-order independence, single-frame pooling, n_subsample parity
between MEDIAN and MEAN, rejection of UNKNOWN / out-of-range / NULL, and a pin that
the pre-existing enumerator values did not move. Both implementations agree numerically
(this PR's 72.71734120 vs #1340's 72.717340155042217), so nothing here is a
correctness dispute about the formula — only about coverage.

4. Smaller points. #1340 namespaces the shared helpers (vmaf_percentile /
vmaf_score_compare) rather than putting bare percentile() / score_compare() into a
header included by two translation units; factors the frame-selection loop into one
shared pool_accumulate() instead of duplicating it; adds a
VMAF_HAVE_PERCENTILE_POOLING feature macro so downstream can #ifdef instead of
version-compare; ships the ADR-0108 research digest
(docs/research/1188-percentile-pooling-methods.md) rather than waiving it; and adds
the ffmpeg mapping as a new appended patch 0018 (macro-guarded, so the filter still
builds against an upstream Netflix libvmaf, and it also maps the previously-unmapped
max) instead of mutating patches 0005/0006/0013 in place. All three of those
patches touch the same libavfilter/vf_libvmaf.c, so a single appended patch covers the
sycl / vulkan / metal filter variants too.

5. Duplicate ledger row. Both PRs move
T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03 from Open to Recently closed in
docs/state.md. Only one may — scripts/ci/check-state-md-rows.sh rejects a duplicate
id. #1340 keeps that row.

Requested action on this PR

De-scope it to Part B only: drop commit ff9b4bdc7 ("feat(core): percentile pooling
methods"), the percentile half of 079b2fb3f, ADR-1181 + its index fragment +
changelog.d/added/pooling-percentile-methods.md, and the
T-UPSTREAM-818-... row change in docs/state.md. The title/body should then describe
only the Windows UTF-8 path contract (ADR-1182 / Netflix#1568), which is worth landing
on its own.

I did not perform that de-scope myself: it requires removing files under docs/adr/ and
changelog.d/, which my operating rules forbid unconditionally.

One thing this PR has that #1340 does not

The Go binding surface in pkg/libvmaf — PoolMethod, String(), ParsePoolMethod,
toC(), the ScoreDirectRequest.PoolMethod / StreamConfig.PoolMethod fields wiring
vmaf_score_pooled off its hardcoded VMAF_POOL_METHOD_MEAN, plus
TestParsePoolMethod / TestPoolMethod_String. #1340 exposes the equivalent through
the Rust bindings (bindings/rust/vmafx/src/score.rs) instead. That Go work should be
re-targeted onto #1340 or a follow-up PR rather than dropped — it is the only copy.

[lusoris]

Queue audit (read-only verification pass, 2026-09-06): Part A of this PR duplicates PR #1340.

Both add the same four enumerators to enum VmafPoolingMethod (VMAF_POOL_METHOD_MEDIAN / _PERC5 / _PERC10 / _PERC20), both compute them by sorting the per-frame scores and interpolating linearly between ranks (the numpy.percentile(method="linear") rule), and both close the same ledger row T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES-2026-09-03. Each carries its own ADR with the identical slug — docs/adr/1181-percentile-pooling-methods.md here, docs/adr/1188-percentile-pooling-methods.md on #1340 — plus its own helper header (core/src/pooling_percentile.h vs core/src/percentile.h) and its own test (core/test/test_pooling_percentile.c vs core/test/test_pool_percentile.c).

git merge-tree between the two heads (9ba5beb / 37c3da6) conflicts in 8 files: core/include/libvmaf/libvmaf.h, core/src/libvmaf.c, core/src/output.cpp, core/src/predict.c, docs/api/index.md, docs/state.md, docs/adr/README.md, docs/adr/_index_fragments/_order.txt. Whichever lands second will not merge cleanly, and the loser also has to give up its ADR number and its T-UPSTREAM-818 state.md row. This needs a maintainer decision (as with #1300/#1338, #1303/#1339, #1310/#1336), not a rebase.

Differences worth weighing: #1340 is percentile-only and ships Rust-binding and FFmpeg-patch coverage (bindings/rust/vmafx/, ffmpeg-patches/0018-*) plus a research digest; this PR bundles percentile pooling with an unrelated Part B (Windows UTF-8 path contract, ADR-1182) and updates FFmpeg patches 0005/0006/0013 and the Go bindings instead. If Part A is dropped here, Part B is independent and stands on its own.

Both PRs are otherwise clean: no conflict against current master, six-deliverable gate passes locally, and all new C files measure 0 clang-tidy findings.

[lusoris]

Drafted — two blockers, neither of them the fixture issue I just fixed in 37b7b89.

1. Functional regression (blocking). Ubuntu clang+DNN fails three vmafexec CLI tests: test_run_vmafexec, test_run_vmafexec_with_frame_skipping, test_run_vmafexec_with_frame_skipping_unequal, all with Lists differ: [] != ['False is not true']. This PR touches libvmaf.c, output.cpp, predict.c, read_json_model.{c,cpp} and the public header, so there is real surface here. Needs diagnosis before this can merge.

2. Pre-existing lint debt in touched files (scope decision). Tidy Changed fails on ~99 findings across four files this PR touches. Checked against scripts/ci/tidy-baseline-cpu.json: this PR introduces none of them.

file	baseline	CI found	dominant check
core/src/dnn/model_loader.c	29	29	21x modernize-use-nullptr, 6x readability-function-size
core/tools/vmaf_roi.c	22	20	modernize-use-nullptr
core/tools/vmaf_per_shot.c	20	20	modernize-use-nullptr
core/src/output.cpp	29	28	misc-use-anonymous-namespace, bugprone-suspicious-stringview-data-usage

CLAUDE.md rule 12 (ADR-0141 / ADR-1142) requires a touched file to be left lint-clean, so the gate is behaving correctly — but discharging it means ~61 mechanical nullptr conversions plus structural work (oversized functions, anonymous namespaces) that has nothing to do with percentile pooling or UTF-8 paths.

Fixed and pushed meanwhile: test_pooling_percentile now skips instead of failing when the untracked Netflix YUVs are absent, and no longer leaks its model/context on early returns — that was the Sanitizers x4 / Coverage Gate / ARM clang failure.

[lusoris]

Superseded by the split you approved — closing so the ADR-1181/1182 numbers stop colliding with their successors.

Why this PR is closed rather than rebased: it bundled two independent changes, ADR-1181 (percentile pooling) and ADR-1182 (Windows UTF-8 path contract). Bundling them dragged model_loader.c, vmaf_roi.c and vmaf_per_shot.c into the touched-file set purely to route their fopen calls through the shim — three files carrying ~71 pre-existing clang-tidy findings that rule 12 then made this PR's problem, on top of a schema decision that needed making on its own terms.

Successors:

PR	Carries	State
#1392	ADR-1181 percentile pooling, closes T-UPSTREAM-818-POOLING-ENUM-NO-PERCENTILES	in CI
fix/windows-utf8-path-contract	ADR-1182 UTF-8 path contract, closes T-UPSTREAM-1568-WINDOWS-NARROW-PATH-API	branch pushed, PR opening once its tidy debt clears

Both carry the work verbatim — #1392 by file split, the UTF-8 branch by cherry-picking e925878a0 — so nothing here is lost.

What the split changed on the way through:

• The XML/JSON schema question is settled and documented: percentile pools ship by default, flagged BREAKING, with the three command_line_test.py assertions updated against measured output. The four pre-existing values are byte-identical; only four attributes are appended.
• output.cpp went from 29 clang-tidy findings to 0. Eight of those were bugprone-suspicious-stringview-data-usage: fmt_or_default() returned a std::string_view whose .data() was handed to std::fprintf as a format string, which carries no null-termination guarantee. Latent rather than live, but real.
• test_pooling_percentile now skips instead of failing when the untracked Netflix YUVs are absent, and no longer leaks its model and context on early returns — that was the Sanitizers ×4 / Coverage / ARM failure here.
• On the UTF-8 side, 61 of its 70 inherited tidy findings are already cleared mechanically; 6 oversized functions and 2 getenv sites remain.

Reopen this if the split turns out wrong — but the two halves are independent and the ADR numbers can only live in one open PR each.