Skip to content

fix(cli): split --model/--feature option strings escape-aware (ADR-1190) - #1336

Draft
lusoris wants to merge 7 commits into
masterfrom
fix/t-upstream-766-cli-option-string-delimit
Draft

fix(cli): split --model/--feature option strings escape-aware (ADR-1190)#1336
lusoris wants to merge 7 commits into
masterfrom
fix/t-upstream-766-cli-option-string-delimit

Conversation

@lusoris

@lusoris lusoris commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Summary

core/tools/cli_parse.cpp split the --model / --feature option strings with raw strsep at nine sites and carried no escape state, so every : and = in the string was a separator whatever the user meant — Netflix/vmaf#766, ledger row T-UPSTREAM-766-CLI-OPTION-STRING-DELIMITERS-2026-09-03. The worst failure was silent: -m 'path=/a/dir=eq/m.json' truncated to /a/dir and reported a phantom path the user never typed. This PR replaces all nine sites with an escape-aware cli_split() + cli_unescape() pair (ADR-1190), adds a Windows drive-letter affordance so path=C:\... needs no escaping, makes a pair's value the whole remainder after the first unescaped =, and escapes the paths the Go surfaces synthesise via the new pkg/cliopt.EscapeValue.

Type

  • fix — bug fix

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally (clang-format clean; core/tools/cli_parse.cpp measures 0 clang-tidy warnings and core/test/test_cli_parse.c 24, exactly its ADR-1142 cpu baseline).
  • Unit tests pass: meson test -C build-cpu --suite=fast114/114 Ok, 0 Fail.
  • No SIMD/GPU code path touched — the change is confined to the CLI option-string parser.
  • No feature extractor touched, so no twin gap.
  • New files (pkg/cliopt/escape.go, pkg/cliopt/escape_test.go) carry the Copyright 2026 Lusoris header.
  • Not a breaking change for any documented option string: version=…:name=…:disable_clip, psnr=reduced_hbd_peak=true:… and adm.<option>=<value> parse byte-identically (regression tests cover all three). The only behaviour deltas are the ones that used to fail or truncate.
  • The ADR row lives in docs/adr/_index_fragments/1190-cli-option-string-escape-grammar.md with the slug appended to _order.txt; docs/adr/README.md was regenerated by scripts/docs/concat-adr-index.sh --write (--check clean).

Bug-status hygiene (ADR-0165)

  • docs/state.md updated in this PR: T-UPSTREAM-766-CLI-OPTION-STRING-DELIMITERS-2026-09-03 moved out of Open bugs into Recently closed, citing this PR.

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • No scoring path is touched. The Python harness only ever emits name=vmaf:path=model/<file>.json (compat/python-vmaf/core/quality_runner.py:1191-1198) and <feature>=<k>=<v>:<k>=<v> (compat/python-vmaf/__init__.py:265-273) — none of those contain a backslash or a second = in a value, so they parse byte-identically under the new grammar.

Cross-backend numerical results

Not applicable — no backend, kernel or scoring code is touched.

Reproducer

Build: meson setup build-cpu core -Denable_cuda=false -Denable_sycl=false -Db_lto=false && ninja -C build-cpu -j 4 tools/vmaf test/test_cli_parse

Pre-fix (git show origin/master:core/tools/cli_parse.cpp compiled into the same build dir):

$ ./build-cpu/tools/vmaf -r /dev/null -d /dev/null -w 16 -h 16 -p 420 -b 8 -m 'path=/a/dir=eq/m.json'
libvmaf WARNING could not read model from path: "/a/dir"
libvmaf ERROR could not read model collection from path: "/a/dir"
$ ./build-cpu/tools/vmaf ... -m 'path=C:\models\vmaf_v0.6.1.json'
Problem parsing model, bad option string "\models\vmaf_v0.6.1.json".
$ ./build-cpu/tools/vmaf ... --feature 'psnr=some_path=C:\x'
Problem parsing feature "psnr", bad option string "\x".
$ ./build-cpu/tools/vmaf ... -m 'path=/a/dir\:colon/m.json'
Problem parsing model, bad option string "colon/m.json".
$ ./build-cpu/test/test_cli_parse
...
test_model_path_keeps_inner_equals: fail
T-UPSTREAM-766: everything after the FIRST unescaped '=' is the value; `path=/a/dir=eq/m.json` used to be truncated to "/a/dir"
25 tests run, 1 failed

Post-fix, same commands:

$ ./build-cpu/tools/vmaf ... -m 'path=/a/dir=eq/m.json'
libvmaf WARNING could not read model from path: "/a/dir=eq/m.json"
$ ./build-cpu/tools/vmaf ... -m 'path=C:\models\vmaf_v0.6.1.json'
libvmaf WARNING could not read model from path: "C:\models\vmaf_v0.6.1.json"
$ ./build-cpu/tools/vmaf ... --feature 'psnr=some_path=C:\x'      # -w 320 -h 240
libvmaf ERROR feature extractor 'psnr': unknown option 'some_path'
$ ./build-cpu/tools/vmaf ... -m 'path=/a/dir\:colon/m.json'
libvmaf WARNING could not read model from path: "/a/dir:colon/m.json"
$ ./build-cpu/test/test_cli_parse
32 tests run, 32 passed
$ meson test -C build-cpu --suite=fast
Ok: 114   Fail: 0
$ go test ./pkg/cliopt/
ok  	github.com/VMAFx/vmafx/pkg/cliopt	0.001s

Every path in the "post-fix" block reaches the model/feature loader with the exact bytes the user typed; the remaining errors are the expected "this file does not exist" / "this option is not a psnr option" ones.

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: the grammar question is a single parser decision and its full option matrix lives in ADR-1190's ## Alternatives considered; there is no external literature to survey.
  • Decision matrixdocs/adr/1190-cli-option-string-escape-grammar.md ## Alternatives considered (six options: escapes+drive-letter, escapes only, drive-letter only, hard-error on a second =, dropping \\ from the escape set, quote-based grammar).
  • AGENTS.md invariant notecore/tools/AGENTS.md, new ADR-1190 block: split-then-unescape is two passes, the value is the remainder after the first unescaped =, the overload key is split on . before unescaping, the drive-letter rule is load-bearing ergonomics, and the Go escaper is the same grammar in another language.
  • Reproducer / smoke-test command — pasted above under "Reproducer".
  • CHANGELOG fragmentchangelog.d/fixed/T-UPSTREAM-766-cli-option-string-delimiters.md; CHANGELOG.md re-rendered with bash scripts/release/concat-changelog-fragments.sh --write.
  • Rebase notedocs/rebase-notes.md, section fix/t-upstream-766-cli-option-string-delimit.

Documentation (ADR-0100)

docs/usage/cli.md gains an Option-string grammar section (delimiters, the escape table, worked shell examples, the drive-letter rule, the UNC caveat and the shell-quoting caveat), and both the --model and --feature sections link to it from the sentences that previously said only "colon-delimited key/value string".

ffmpeg patch stack (CLAUDE.md §12 r14)

No patch change is needed and none is made. The rule covers public C-API entry points, --enable-libvmaf-* configure flags, LIBVMAFContext fields and vf_libvmaf.c variants; this PR touches none of them — cli_parse.cpp is the CLI tool, not the library. ffmpeg-patches/0008-add-libvmaf_tune-filter.patch's load_model() takes the whole remainder after the first = and never splits on :, so it is already immune to both failure modes, and ffmpeg's filtergraph parser owns escaping at that layer — teaching it the CLI grammar would double-unescape.

Known follow-ups

  • The Python harness (compat/python-vmaf/) still concatenates path= strings without escaping. It only ever passes repo-relative model paths, so nothing is reachable today; escaping there would touch the Netflix-mirrored harness and is deliberately out of scope.

Supersedes #1310 (same bug, same ledger row)

#1310 (fix/cli-option-string-escapes, ADR-1180) implemented the same fix and
was closed in favour of this PR. Both binaries were built from their own branch
tips and probed side by side (build-cpu/tools/vmaf, CPU-only, -Db_lto=false):

Case #1310 (ee7e13e3b) this PR (f87f190ba)
-m 'path=/a/dir=eq/m.json' could not read model from path: "/a/dir"still silently truncated … "/a/dir=eq/m.json"
--feature 'psnr=some_path=C:\x' Problem parsing feature "psnr", bad option string "\x"still rejected unknown option 'some_path' (path reached the extractor whole)
-m 'path=C:\models\vmaf_v0.6.1.json' ok ok
-m 'path=/a/dir\:colon/m.json' ok ok

#1310 kept the second strsep-style split on = (so a value still ends at the
next unescaped =) and put its drive-letter affordance inside
parse_model_config() behind !strcmp(key, "path"), so --feature never got
it. It also left vmaf_cli_strsep and the HAVE_STRSEP probe in the tree as
dead code, exported vmaf_cli_split from cli_parse.h, and duplicated its Go
escaper verbatim in pkg/libvmaf and pkg/corpus.

Ported from #1310: core/test/fuzz/cli_parse_corpus/seed_escapes.argv
(commit f87f190ba), rewritten NUL-separated — tokenise_argv() in
core/test/fuzz/fuzz_cli_parse.c splits on NUL, and #1310's space-joined,
newline-terminated seed collapsed to a single argv token that never reached the
option-string parsers.

Deliberately not ported: #1310's edit to
ffmpeg-patches/0008-add-libvmaf_tune-filter.patch. It is corrupt — the
new-file hunk header still declares @@ -0,0 +1,427 @@ while the body carries
429 + lines, so git apply writes a 427-line vf_libvmaf_tune.c that is
missing the closing FILTER_QUERY_FUNC2(query_formats); and }; of
ff_vf_libvmaf_tune and cannot compile. Its intent is also wrong for that
layer: unescaping \\ inside the filter would turn a UNC \\server\share
into \server\share, and ffmpeg's filtergraph parser already owns escaping
there (see the "ffmpeg patch stack" section above).

@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 5, 2026
lusoris pushed a commit that referenced this pull request Sep 5, 2026
)

Move the ledger row out of "Open bugs" into "Recently closed" per
ADR-0165, citing PR #1336 and recording the pre-/post-fix reproducer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris pushed a commit that referenced this pull request Sep 6, 2026
)

Move the ledger row out of "Open bugs" into "Recently closed" per
ADR-0165, citing PR #1336 and recording the pre-/post-fix reproducer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-766-cli-option-string-delimit branch from 611d5ee to b17aed7 Compare September 6, 2026 03:01
lusoris pushed a commit that referenced this pull request Sep 6, 2026
)

Move the ledger row out of "Open bugs" into "Recently closed" per
ADR-0165, citing PR #1336 and recording the pre-/post-fix reproducer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-766-cli-option-string-delimit branch from b17aed7 to bde4513 Compare September 6, 2026 06:38
lusoris pushed a commit that referenced this pull request Sep 6, 2026
)

Move the ledger row out of "Open bugs" into "Recently closed" per
ADR-0165, citing PR #1336 and recording the pre-/post-fix reproducer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-766-cli-option-string-delimit branch from bde4513 to f87f190 Compare September 6, 2026 09:30
lusoris pushed a commit that referenced this pull request Sep 6, 2026
)

Move the ledger row out of "Open bugs" into "Recently closed" per
ADR-0165, citing PR #1336 and recording the pre-/post-fix reproducer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-766-cli-option-string-delimit branch from f87f190 to 6b33af3 Compare September 6, 2026 09:45
lusoris pushed a commit that referenced this pull request Sep 6, 2026
)

Move the ledger row out of "Open bugs" into "Recently closed" per
ADR-0165, citing PR #1336 and recording the pre-/post-fix reproducer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-766-cli-option-string-delimit branch from 6b33af3 to 46c16a4 Compare September 6, 2026 11:10
lusoris pushed a commit that referenced this pull request Sep 6, 2026
)

Move the ledger row out of "Open bugs" into "Recently closed" per
ADR-0165, citing PR #1336 and recording the pre-/post-fix reproducer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-766-cli-option-string-delimit branch from 46c16a4 to 59442cf Compare September 6, 2026 12:55
lusoris pushed a commit that referenced this pull request Sep 6, 2026
My rebase dedup kept the wrong side. The rule it applied -- when a bug id
appears twice after a keep-both rebase, keep the row that matches origin/master
-- is right when the branch is merely restating a row master already carries.
It is WRONG when the branch is the PR that closes the bug, which is exactly
this case: #1336 fixes Netflix/vmaf#766, so its past-tense Recently-closed row
is the correct survivor and master's present-tense Open row is the stale one.

Restored the branch's closure row and left a tombstone comment where the Open
row was, matching the convention used elsewhere in this file. Exactly one row
per bug id; scripts/ci/check-state-md-rows.sh reports OK (361 rows).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lusoris and others added 7 commits September 6, 2026 20:21
`core/tools/cli_parse.cpp` split the `--model` and `--feature` option
strings with raw `strsep` at nine sites and carried no escape state, so
every `:` and `=` in the string was a separator whatever the user meant.
Reproduced against `cd52f2670`:

  -m 'path=/a/dir=eq/m.json'          -> could not read model from path: "/a/dir"
  -m 'path=C:\models\vmaf_v0.6.1.json'-> bad option string "\models\vmaf_v0.6.1.json"
  --feature 'psnr=some_path=C:\x'     -> bad option string "\x"

The first is the worst: a silent truncation to a phantom path the user
never typed. The second is the case Netflix/vmaf#766 reports, and the
fork widens the blast radius because `pkg/libvmaf` and `pkg/corpus`
paste user-supplied paths straight into `path=<path>`.

Replace all nine `vmaf_cli_strsep` call sites with `cli_split()`, which
breaks on the first *unescaped* separator and leaves backslash sequences
in place, plus `cli_unescape()`, which drops the backslash in `\:`,
`\=`, `\.` and `\\` once, at the leaf. Every other backslash is data, so
`C:\models\m.json` survives verbatim, and a `:` that spells a Windows
drive letter is data rather than a separator. A pair's value is now the
whole remainder after the first unescaped `=`, so an inner `=` can no
longer truncate a path. The `strsep` / `HAVE_STRSEP` portability shim
and its meson probe are deleted with the last call site, which also ends
a POSIX-vs-MSVC disagreement about trailing separators.

Go callers escape through the new `pkg/cliopt.EscapeValue`, whose
round-trip test pins it to the C grammar.

Tests: eight cases in `core/test/test_cli_parse.c` (three reproduced
failures, `\=`/`\\` escapes, and no-change regressions over
`version=...:name=...:disable_clip`, the `--aom_ctc v1.0` psnr option
string and a `adm.<option>` overload). Pre-fix, the truncation case
fails the assertion and the two backslash cases exit through `usage()`.
114/114 `meson test --suite=fast` pass; `core/tools/cli_parse.cpp` stays
at 0 clang-tidy warnings and `core/test/test_cli_parse.c` at its
baseline 24 (ADR-1142). No scoring path is touched — the Netflix golden
assertions are untouched and the Python harness only ever emits
`name=vmaf:path=model/*.json`, which parses byte-identically.

Closes T-UPSTREAM-766-CLI-OPTION-STRING-DELIMITERS-2026-09-03.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
)

Move the ledger row out of "Open bugs" into "Recently closed" per
ADR-0165, citing PR #1336 and recording the pre-/post-fix reproducer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The seven `cli_parse_corpus/` seeds all predate the escape-aware splitter, so
the fuzzer had no starting point that exercises `cli_split()` /
`cli_unescape()`. Add one NUL-separated argv seed covering, in a single input,
the three constructs the new grammar introduces: an unescaped inner `=` in a
value (`path=/dir=eq/...`, the silent-truncation case), a backslash-escaped
delimiter (`m\:colon.json`), and a Windows drive-letter colon inside a
`--feature` option value (`some_path=C:\x`).

Force-added, like the seven seeds already tracked beside it: the
`core/test/fuzz/*_corpus/` ignore rule exists to keep fuzzer-generated corpus
growth out of the tree, not the checked-in seeds.

Ported from PR #1310, which proposed the same idea but wrote the seed as one
space-joined line with a trailing newline. `tokenise_argv()` in
core/test/fuzz/fuzz_cli_parse.c splits on NUL, so that form collapses to a
single argv token and never reaches the option-string parsers at all; this
seed uses the NUL separation the harness and the other seeds use.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…190)

The harness comment still described the option-string sub-parsers as
running `strsep` chains. ADR-1190 replaced all nine split sites with
`cli_split()` / `cli_unescape()` in the same PR, so the rationale now
names the current splitter and keeps `strsep` only as the historical
note. Comment-only; no behaviour, no new symbols, no tidy delta (the
fuzz targets are not in the CPU lane's compile_commands.json).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each dropped row restates one origin/master already carries; master is the
authoritative record. Verified with scripts/ci/check-state-md-rows.sh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each dropped row restates one origin/master already carries; master is the
authoritative record. Verified with scripts/ci/check-state-md-rows.sh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
My rebase dedup kept the wrong side. The rule it applied -- when a bug id
appears twice after a keep-both rebase, keep the row that matches origin/master
-- is right when the branch is merely restating a row master already carries.
It is WRONG when the branch is the PR that closes the bug, which is exactly
this case: #1336 fixes Netflix/vmaf#766, so its past-tense Recently-closed row
is the correct survivor and master's present-tense Open row is the stale one.

Restored the branch's closure row and left a tombstone comment where the Open
row was, matching the convention used elsewhere in this file. Exactly one row
per bug id; scripts/ci/check-state-md-rows.sh reports OK (361 rows).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/t-upstream-766-cli-option-string-delimit branch from 690d685 to bd38e6a Compare September 6, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant