Skip to content

M2.5: wit logic-report tool for the empty-verdict reality gate - #32

Merged
sepehrsadri merged 1 commit into
mainfrom
feature/m2.5-logic-report-tool
Aug 8, 2026
Merged

M2.5: wit logic-report tool for the empty-verdict reality gate#32
sepehrsadri merged 1 commit into
mainfrom
feature/m2.5-logic-report-tool

Conversation

@sepehrsadri

Copy link
Copy Markdown
Contributor

Progress on #15.

What this PR delivers

Issue #15 asks for the empty-verdict rate across a 30-project / 26 GB real
Logic library. This environment does not have that library — it has exactly
one real .logicx project. Fabricating a "the answer is X%" finding from n=1
would violate this repo's own reproducibility doctrine (AGENTS.md), so this
PR delivers the tool the issue needs, runs it for real on the one project
available, and publishes that honestly-labeled n=1 result — it does not
close #15, because the actual reality-gate finding still requires the real
library.

New: wit logic-report <library-root>

$ wit logic-report "/path/to/YourLibrary"
  scanned 1 project(s), 1 alternative(s), 9 consecutive save pair(s)
  44.4% of save pairs show a structural change Wit can see (4 of 9)
  distribution of change counts per save pair (0 = no visible structural change):
    0 change(s): 5 pair(s)
    1 change(s): 1 pair(s)
    2 change(s): 1 pair(s)
    14 change(s): 1 pair(s)
    17 change(s): 1 pair(s)
  5 pair(s) (55.6%) are byte-different but structurally identical

Walks every discovered Logic/GarageBand alternative's backup chain
(Project File Backups/0009 oldest-first, then current — reusing M3's
wit_index::discover_logic_projects), compares every consecutive pair at
wit-logic's Structure honesty tier, and reports the three statistics #15
asks for: % of saves with any visible structural change, the distribution of
change counts per save, and how often adjacent saves are byte-different but
structurally identical.

New library pieces

  • wit_logic::change_count(a, b) (crates/wit-logic/src/lib.rs) — a new
    diagnostic counting distinct differing signals between two walks (one per
    differing census tag, one per name added/removed across the three
    extracted-name lists, one for a tempo change). Guaranteed 0 exactly when
    semantic_equal reports NoStructuralChange — not part of the Verdict
    itself, just granularity for the "how much changed" question M2.5: Measure the empty-verdict rate on a real Logic library #15 asks.
  • wit_index::logic_report / LogicLibraryReport
    (crates/wit-index/src/report.rs) — orchestrates discovery + wit-logic
    walks/comparisons across a whole library, the same way scan.rs already
    orchestrates discovery + the store. Read-only, following discover.rs's
    and dupes.rs's existing discipline.
  • wit-cli's LogicReport subcommand follows scan/dupes's privacy
    convention: only project/alternative names in output, never a full path
    (assert_no_home_paths asserted on the rendered report, same as dupes).

Tests

  • Synthetic-fixture unit tests in wit-index/src/report.rs (deterministic,
    crafted ProjectData containers, no real material) proving the three
    statistics compute correctly, plus a read-error and an empty-library case.
  • wit-logic/src/lib.rs gains unit tests for change_count (zero iff
    NoStructuralChange, symmetric, counts exactly one per differing tag).
  • An opt-in real-material test, crates/wit-index/tests/real_fixtures.rs,
    gated on WIT_LOGIC_LIBRARY (mirrors the existing WIT_LOGIC_PROJECT /
    WIT_FIXTURES loud-skip pattern), for a library root rather than a
    single bundle.

Docs

  • docs/EXPERIMENTS.md §11 — the n=1 result above, explicitly labeled
    measured and explicitly scoped: this is a data point, not the issue's
    own >50%-of-30-projects decision.
  • docs/ROADMAP.md's M2.5 row — tool landed, finding still open, with the
    exact command to run against the real library.

For the repo owner: run this against the real library

WIT_LOGIC_LIBRARY="/path/to/YourLibrary" cargo test -p wit-index --test real_fixtures -- --nocapture --ignored

or directly via the CLI:

cargo run -p wit-cli -- logic-report "/path/to/YourLibrary"

That run is what actually resolves #15's exit criterion (M5 vs. the M2
stretch goal of mapping Logic's volume-fader field) — this PR only gets the
tool in place for it.

Verification

  • cargo build --workspace — clean
  • cargo test --workspace --locked — clean, all green (opt-in real-material
    tests loudly skip as expected)
  • cargo fmt --all --check — clean
  • cargo clippy --workspace --all-targets --locked -- -D warnings — clean
  • cargo deny check licenses advisories — clean (pre-existing unrelated
    license-not-encountered warning for ISC, not an error)
  • python3 .github/workflows/scripts/check_personal_paths.py — clean

🤖 Generated with Claude Code

Adds the library-wide comparison tool issue #15 asks for, without
fabricating the 30-project/26GB finding this environment cannot produce:

- wit_logic::change_count: a new diagnostic counting distinct differing
  signals between two walks (census tags + name adds/removes + tempo),
  0 exactly when semantic_equal reports NoStructuralChange.
- wit_index::logic_report / LogicLibraryReport: walks every discovered
  Logic/GarageBand alternative's backup chain (reusing wit-index's M3
  discover_logic_projects) and computes the three statistics the issue
  asks for: % of saves with a visible structural change, the
  change_count distribution, and how often adjacent saves are
  byte-different but structurally identical.
- wit-cli: new `wit logic-report <library-root>` subcommand, following
  the scan/dupes privacy discipline (names only, assert_no_home_paths).
- Tests: synthetic-fixture unit tests in wit-index::report, plus an
  opt-in WIT_LOGIC_LIBRARY-gated real-material test
  (crates/wit-index/tests/real_fixtures.rs), mirroring the
  WIT_LOGIC_PROJECT/WIT_FIXTURES pattern used elsewhere in this repo.

Ran the tool for real against the one local .logicx project available
in this environment (n=1 project, 9 consecutive save pairs): 55.6% show
no visible structural change. Published as a measured, explicitly-
labeled n=1 finding in docs/EXPERIMENTS.md §11 and docs/ROADMAP.md's
M2.5 row — NOT the issue's own >50%-of-30-projects decision, which
still requires running this tool against the real 26 GB/30-project
library on a machine that has it:

  WIT_LOGIC_LIBRARY=/path/to/YourLibrary cargo test -p wit-index \
    --test real_fixtures -- --nocapture --ignored

Progress on #15 — does not close it; the reality-gate finding itself
is still outstanding pending the real library.

Verification: cargo build/test/fmt/clippy/deny all clean;
check_personal_paths.py clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@sepehrsadri
sepehrsadri merged commit 194e206 into main Aug 8, 2026
14 checks passed
@sepehrsadri
sepehrsadri deleted the feature/m2.5-logic-report-tool branch August 8, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

M2.5: Measure the empty-verdict rate on a real Logic library

1 participant