Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ Building the 0.0 pilot — no released artifact yet.
policy, issue/PR templates, CI

### Fixed
- `wit dupes` formatted byte counts with a 1024-based divisor while printing decimal unit
labels ("GB"), understating its own totals by 7.4% and making its output silently
incomparable to `docs/EXPERIMENTS.md`, which quotes decimal GB throughout. Now decimal,
with a test pinning the boundary
- All 13 bugs documented as `xfail` in the Python prototype test suite (21 xfail markers
total), spanning robustness (billion-laughs expansion, unbounded tree allocation,
quadratic varint DoS in `flp_parse`, a leaked file descriptor, an oversized-payload
Expand All @@ -51,6 +55,14 @@ Building the 0.0 pilot — no released artifact yet.
`EXPERIMENTS.md` §6a) — [PR #28](https://github.com/sep-lab/Wit/pull/28), closes #10

### Findings that shaped the design
- **M2.5 reality gate — the Logic empty-verdict rate, measured at scale and passed.**
Across 32 real Logic projects (28 GB, 100 consecutive save pairs), **33.0%** of save
pairs show no structural change Wit can see — against issue #15's >50% threshold for
stopping M5 in favour of mapping Logic's volume-fader field first. **M5 proceeds**
([#18](https://github.com/sep-lab/Wit/issues/18) unblocked). 28.0% of pairs are
byte-different but structurally identical, reconfirming at library scale M2's finding
that byte comparison is useless on this format — see
[docs/EXPERIMENTS.md](docs/EXPERIMENTS.md) §11, closes #15
- Byte-level versioning of rendered audio is not viable: a global gain change leaves
**0.00%** of bytes reusable, and costs git a full extra copy
- Delta chains beat content-defined chunking **29×** on project-file history
Expand Down
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
<h1 align="center">Wit</h1>

<p align="center">
Expand Down Expand Up @@ -168,6 +168,17 @@
delivered it. (An earlier figure of 8.1 GB / 3.2× was wrong: it applied FLAC to 7.4 GB of
`.caf` that is already ALAC-compressed.)

You can reproduce the duplicate half of that on your own library in about half a minute —
it is read-only, prints basenames only, and there is no delete button:

```
cargo run --release -p wit-cli -- dupes /path/to/YourLibrary
```

On the library above, since grown to 32 projects, that reports **5.5 GB of 23.9 GB (23.2%)**
— an independent Rust reimplementation agreeing with the Python analysis
([EXPERIMENTS.md §9](docs/EXPERIMENTS.md)).

## How it works

A music project is already a program: immutable source recordings plus a tree of
Expand Down
33 changes: 31 additions & 2 deletions crates/wit-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -339,12 +339,21 @@ fn dupes(path: &std::path::Path) -> ExitCode {
ExitCode::SUCCESS
}

/// Format a byte count in **decimal** units — 1 GB = 1,000,000,000 bytes.
///
/// This is deliberately not the 1024-based convention. Every published
/// figure in `docs/EXPERIMENTS.md` is decimal GB (§9 says so explicitly),
/// and `wit dupes` output is meant to be directly comparable to it — a
/// user pasting this tool's number into a Measurement issue (the ask in
/// [#4]) must be quoting the same unit the docs quote. Dividing by 1024
/// while printing "GB" understated the library total by 7.4% and made the
/// two numbers silently incomparable.
fn human_bytes(bytes: u64) -> String {
const UNITS: &[&str] = &["B", "KB", "MB", "GB", "TB"];
let mut size = bytes as f64;
let mut unit = 0;
while size >= 1024.0 && unit < UNITS.len() - 1 {
size /= 1024.0;
while size >= 1000.0 && unit < UNITS.len() - 1 {
size /= 1000.0;
unit += 1;
}
if unit == 0 {
Expand Down Expand Up @@ -415,3 +424,23 @@ fn logic_report(path: &std::path::Path) -> ExitCode {
print!("{out}");
ExitCode::SUCCESS
}

#[cfg(test)]
mod tests {
use super::human_bytes;

#[test]
fn byte_counts_are_formatted_in_decimal_units_not_binary() {
// The boundary that matters: 1000 B is 1.0 KB, and 1024 B is also
// 1.0 KB rather than the binary convention's "1.0 KiB".
assert_eq!(human_bytes(999), "999 B");
assert_eq!(human_bytes(1_000), "1.0 KB");
assert_eq!(human_bytes(1_024), "1.0 KB");
// The unit the published numbers are actually quoted in. A 1024-based
// divisor would render this as "20.7 GB" — a 7.4% understatement, and
// the exact discrepancy that made `wit dupes` output incomparable to
// EXPERIMENTS.md §9.
assert_eq!(human_bytes(22_200_000_000), "22.2 GB");
assert_eq!(human_bytes(0), "0 B");
}
}
130 changes: 101 additions & 29 deletions docs/EXPERIMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ fixtures:
| `Artefakt - Undertow` (Ableton) | A commercial-grade Live 12 set, 28 tracks, 688 audio clips, 5,112 warp markers. **30 sequential autosaves** of the same project spanning Feb–May 2026. |
| `You make my crazy!` (Logic) | A real Logic 12 project, 459 MB, 33 audio files. **10 sequential saves** (`Project File Backups/00..08` + current). |
| `Aston Martin Music Remake` (FL Studio) | A real FL 10-era `.flp`, 136 KB, 18 channels. |
| A 30-project Logic library | 26 GB across 30 real projects — used for the duplication analysis. |
| A real Logic library | One person's working library, used for the duplication analysis (§9) and the M2.5 reality gate (§11). It grows: **30 projects / 26 GB** when §9 walked it, **32 projects / 28 GB** at §11's 2026-08-16 run. Each section states the snapshot it measured. |

Labels used throughout: **measured** (we ran it), **cited** (someone else's result, with
a source), **inferred** (reasoning, not measurement).
Expand Down Expand Up @@ -538,6 +538,28 @@ exist today**.
> it would work today. That is an argument for shipping that part first, not evidence that
> version control produced it.

**Independent cross-check — measured, 2026-08-16.** The figures above come from the Python
walk on the 30-project / 26 GB snapshot. `wit dupes` (the Rust reimplementation shipped in
M3, `wit-index::duplicate_report`) was run against the same library after it had grown to 32
projects:

```
$ wit dupes "/path/to/YourLibrary"
found 5.5 GB of duplicate audio (23.2% of 23.9 GB scanned)
```

**5.5 GB of 23.9 GB (23.2%)**, against Python's 5.38 GB of 21.95 GB (24.5%) — the two
implementations agree, on a library that gained ~2 GB of audio in between, with the duplicate
share drifting down about a point. This closes M3's flagged follow-up (the ROADMAP noted the
`wit dupes` verification had only ever run against a deliberately-planted duplicate, never the
full library). The top duplicate groups are the same `Save As` triples §9 named — three copies
each of `Vintage Upright Piano_*_consolidated.caf` at ~300 MB apiece.

Both numbers are decimal GB. `wit dupes` divided by 1024 while printing "GB" until this
change, which understated its own totals by 7.4% and made its output silently incomparable to
this section; it now formats in decimal units, with a test pinning the boundary
(`crates/wit-cli/src/main.rs`).

**Limits.** Exact-duplicate detection only; sub-file chunk dedup would find more
([issue #6](https://github.com/sep-lab/Wit/issues/6)). The FLAC ratio is extrapolated from
a 10-file random sample, so treat 1.8× as approximate. Figures are decimal GB throughout;
Expand Down Expand Up @@ -599,40 +621,90 @@ consecutive pair with `wit_logic::walk` and reports, per pair: `semantic_equal`'
extracted name, or a tempo change; `0` exactly when the verdict is `NoStructuralChange`),
and raw byte identity.

**Result — measured, n = 1 project.**
**Result — measured, n = 32 projects, 2026-08-16.**

```
$ wit logic-report "/path/to/YourLibrary"
scanned 1 project(s), 1 alternative(s), 9 consecutive save pair(s)
44.4% of save pairs show a structural change Wit can see (4 of 9)
scanned 32 project(s), 32 alternative(s), 100 consecutive save pair(s)
67.0% of save pairs show a structural change Wit can see (67 of 100)
distribution of change counts per save pair (0 = no visible structural change):
0 change(s): 5 pair(s)
1 change(s): 1 pair(s)
2 change(s): 1 pair(s)
14 change(s): 1 pair(s)
17 change(s): 1 pair(s)
5 pair(s) (55.6%) are byte-different but structurally identical
0 change(s): 33 pair(s)
1 change(s): 12 pair(s)
2 change(s): 4 pair(s)
3 change(s): 6 pair(s)
4 change(s): 1 pair(s)
5 change(s): 1 pair(s)
6 change(s): 3 pair(s)
7 change(s): 4 pair(s)
9 change(s): 1 pair(s)
11 change(s): 3 pair(s)
13 change(s): 2 pair(s)
14 change(s): 2 pair(s)
15 change(s): 2 pair(s)
16 change(s): 1 pair(s)
17 change(s): 1 pair(s)
18 change(s): 1 pair(s)
19 change(s): 1 pair(s)
25 change(s): 2 pair(s)
26 change(s): 1 pair(s)
29 change(s): 1 pair(s)
30 change(s): 1 pair(s)
33 change(s): 1 pair(s)
36 change(s): 1 pair(s)
37 change(s): 1 pair(s)
44 change(s): 1 pair(s)
47 change(s): 1 pair(s)
56 change(s): 1 pair(s)
63 change(s): 1 pair(s)
65 change(s): 1 pair(s)
67 change(s): 1 pair(s)
68 change(s): 1 pair(s)
71 change(s): 1 pair(s)
76 change(s): 1 pair(s)
93 change(s): 1 pair(s)
116 change(s): 2 pair(s)
145 change(s): 1 pair(s)
235 change(s): 1 pair(s)
28 pair(s) (28.0%) are byte-different but structurally identical
```

Run against `You make my crazy!` (§0's fixture table) — the same 10-save chain M2 used.
5 of 9 pairs show no structural change (**55.6%**), and every one of those 5 is also
byte-different from its neighbor (the M2 finding that raw byte comparison is useless on
this format, reconfirmed: 100% of the no-visible-change pairs would look "changed" to a
byte diff). Every pair with a nonzero `change_count` was correctly called
`StructuralChange`, and every zero-count pair was correctly called `NoStructuralChange` —
`change_count` and the verdict never disagreed, which is expected since both derive from
the same census/extracted equality (`wit-logic/src/lib.rs`), but is a useful sanity check
on the new counting logic itself.

**Limits — this is a tool delivery, not the finding issue #15 asks for.** The issue's own
exit criterion is a decision at **30 projects, 26 GB** — whether >50% of real saves show
no visible structural change. **n = 1 project cannot answer that question either way**;
55.6% on one project's one alternative is a data point, not a verdict on a library. This
environment does not have the 30-project/26 GB library (`daw-vcs-adoption-evidence`
memory) the issue was scoped against — only the one `.logicx` bundle in the fixture table
above. The exit-criterion decision (M5 vs. the M2 stretch goal — mapping Logic's
volume-fader field) is **not resolved by this entry** and still requires running the
command below against the real library.
Run against the full library in §0's fixture table (32 projects, 28 GB — it has grown by
two projects since the §9 duplication analysis walked it). Every `ProjectData` in the
library walked cleanly: the report printed no read-error line, so all 100 pairs are real
comparisons rather than partial ones.

**The three statistics issue #15 asked for:**

- **33 of 100 pairs (33.0%) show no structural change Wit can see.** The complement — 67.0%
— do.
- **The distribution is long-tailed, not bimodal.** The mode above zero is a single change
(12 pairs), 26 pairs sit in the 1–7 band, and a thin tail runs out to 235. So the common
case is not "nothing" and not "everything"; it is a handful of visible changes per save.
- **28 pairs (28.0%) are byte-different but structurally identical** — 28 of the 33
empty-verdict pairs. The M2 finding holds at library scale: a byte diff would report a
change on 28% of all saves where there is nothing structural to show. The remaining 5
empty pairs were byte-identical too, so *every* pair the byte comparison called changed
and the walker called empty is accounted for.

**The decision this triggers.** Issue #15's exit criterion was: if **>50%** of real saves
show no visible structural change, do not start M5 — spend that time on the M2 stretch
goal (mapping Logic's volume-fader field) instead, because no UI copy survives a dominant
"nothing changed" first impression. **Measured 33.0%, so the gate passes and M5 proceeds**
([#18](https://github.com/sep-lab/Wit/issues/18) is no longer blocked).

For calibration against the other format: Ableton's semantically-empty rate is 24% (§1).
Logic's 33% is higher, as expected from the Structure tier's narrower vision, but it is not
the majority outcome the gate was written to catch.

**Limits.** One person's library, one machine — the same acknowledged breadth weakness every
number in this document carries, and exactly what [issue #4](https://github.com/sep-lab/Wit/issues/4)
exists to fix. It is also a library that has never been used with a version-control tool, so
its save cadence reflects Logic's own backup behaviour rather than deliberate committing. The
33% is a rate over *save pairs*, not over sessions: a project with 20 backups contributes 19
pairs and weighs more than a project with 2. And "no structural change Wit can see" remains a
statement about the Structure honesty tier, not about the music — a save that moved only a
fader is in that 33% and is a real musical change. That gap is what the stretch goal closes,
and it is still worth doing; the gate result means it does not have to come *before* M5.

**Reproduce (opt-in, real material, never committed):**

Expand Down
6 changes: 3 additions & 3 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,10 @@ understand — without ever risking a project file.**
| M0 ✅ | [PR #13](https://github.com/sep-lab/Wit/pull/13) | ADR-0006, Rust workspace scaffolding, CI (`rust` + `licenses` jobs) | Landed |
| M1 ✅ | [PR #24](https://github.com/sep-lab/Wit/pull/24) | Ableton `.als` parity port (`wit-als`) — Rust port of the working Python differ | Landed — golden byte-for-byte (`crates/wit-diff/tests/golden.rs`); spot-checked against a real 29-save `Backup/` chain via `WIT_FIXTURES` (28 pairs parse clean, no panics; 2 pairs cross-checked line-for-line against Python's actual output, exact match). The formal corpus-agreement gate against the specific 7 zero-change / 3 knob-only pairs named in `wit-planning/PLAN.md` has not been re-run — flagged for a follow-up pass |
| M2 ✅ | [PR #25](https://github.com/sep-lab/Wit/pull/25) | Logic/GarageBand `ProjectData` walker (`wit-logic`) | Landed — spot-checked against a real `.logicx` project + its 9 on-disk `Project File Backups` (10 files, clean EOF on all, tempo matches `MetaData.plist` exactly on all); 5 of 9 real consecutive pairs correctly report `NoStructuralChange` despite differing bytes. The 30-fixture `jonkubis/LogicProFormatWriter` corpus fetch (pinned SHA `1f77c5c37d49ccd9551cc8e9107750e8db2f1fed`) has not been run — network-gated, opt-in, flagged for a follow-up pass |
| M2.5 | [#15](https://github.com/sep-lab/Wit/issues/15) | **Reality gate** — measure the empty-verdict rate on a real Logic library | **Tool landed, finding still open.** `wit logic-report <library-root>` (`wit-index::logic_report`) now walks every alternative's backup chain across a whole library and publishes the three statistics the issue asks for — see EXPERIMENTS.md §11. Run against the one real `.logicx` project available in this environment (n=1, 9 pairs): 55.6% show no visible structural change. That is a data point, not the issue's own >50%-of-30-projects decision — the real 26 GB/30-project run (`WIT_LOGIC_LIBRARY=/path/to/YourLibrary cargo test -p wit-index --test real_fixtures -- --nocapture --ignored`) still needs to happen before M5-vs-stretch-goal is decided |
| M3 ✅ | [PR #29](https://github.com/sep-lab/Wit/pull/29) | Index, content-addressed store, CLI (`wit-index`, `wit-cli`) — covers Logic, GarageBand, and Ableton discovery | Landed — `wit scan` verified against real local Ableton (5 lineages / 30 versions, exact match) and Logic (10 versions) libraries, rescans idempotent through the actual CLI. `wit dupes` verified correct against a deliberate real duplicate; the README's ~5.4 GB figure needs the full 26 GB library this environment doesn't have, flagged for a follow-up pass. (Originally reviewed and merged as [PR #26](https://github.com/sep-lab/Wit/pull/26), which was opened against the wrong base branch and never reached `main` — PR #29 is the corrected landing.) |
| M2.5 | [#15](https://github.com/sep-lab/Wit/issues/15) | **Reality gate** — measure the empty-verdict rate on a real Logic library | **Landed, and the gate passes.** `wit logic-report <library-root>` (`wit-index::logic_report`) walks every alternative's backup chain across a whole library and publishes the three statistics the issue asks for — see EXPERIMENTS.md §11. Run at the scale the issue specified (32 projects, 28 GB, 100 consecutive save pairs, 2026-08-16): **33.0% of save pairs show no structural change Wit can see**, against the issue's >50% "stop and map the fader first" threshold — so **M5 proceeds**. 28.0% of pairs are byte-different but structurally identical, reconfirming M2's finding at library scale. The M2 volume-fader stretch goal stays worth doing; it just no longer has to come first |
| M3 ✅ | [PR #29](https://github.com/sep-lab/Wit/pull/29) | Index, content-addressed store, CLI (`wit-index`, `wit-cli`) — covers Logic, GarageBand, and Ableton discovery | Landed — `wit scan` verified against real local Ableton (5 lineages / 30 versions, exact match) and Logic (10 versions) libraries, rescans idempotent through the actual CLI. `wit dupes` follow-up now **closed**: run against the full library it reports 5.5 GB of 23.9 GB (23.2%), agreeing with §9's independent Python walk (5.38 GB of 21.95 GB) — see EXPERIMENTS.md §9's cross-check. (Originally reviewed and merged as [PR #26](https://github.com/sep-lab/Wit/pull/26), which was opened against the wrong base branch and never reached `main` — PR #29 is the corrected landing.) |
| M4 ✅ | [PR #27](https://github.com/sep-lab/Wit/pull/27) | Audio engine — decode, peaks, alignment, null-diff (`wit-audio`) | Landed — injected sample shifts (+1, −1, +4800, −12000) recovered exactly via FFT cross-correlation on seeded broadband noise (a pure tone can't exercise misalignment, per the issue's own reasoning); the confidence gate (refuse below 1.5) verified both ways — a real aligned pair scores tens-to-hundreds, two independently-seeded noise buffers score 0.9 and are correctly refused; real `afconvert`-generated PCM-CAF, ALAC-in-CAF, and ALAC-in-M4A fixtures decode sample-exact against the synthetic source (macOS, measured on this machine — loud-skips via `WIT_AUDIO_AFCONVERT`/`cfg!(target_os)` elsewhere, mirroring `wit-logic`'s `WIT_FIXTURES` doctrine); perf measured at 1.57 s for two 5-minute mono bounces end-to-end (decode+align+null-diff, release build, single-threaded, this machine) against the 10 s budget. Not yet exercised: the `afconvert` test running inside GitHub Actions' macOS runner itself, since it is intentionally opt-in (`#[ignore]`) rather than a default CI assertion — flagged for a follow-up pass |
| M5 | [#18](https://github.com/sep-lab/Wit/issues/18) | Tauri app alpha — Shelf, Story, Compare, watcher | Installs on a second Mac, reads a diff on the demo library, a real Logic folder, **and one Ableton `.als` lineage**, clicks Reveal |
| M5 | [#18](https://github.com/sep-lab/Wit/issues/18) | Tauri app alpha — Shelf, Story, Compare, watcher | **Unblocked** — M2.5's gate passed (33.0%, threshold >50%). Installs on a second Mac, reads a diff on the demo library, a real Logic folder, **and one Ableton `.als` lineage**, clicks Reveal |
| M6 | [#22](https://github.com/sep-lab/Wit/issues/22) | Share bundle + zero-install viewer (`wit-share`, `viewer/`) | AirDrop the `.html` to another Mac **and** open it on a physical iPhone via iMessage/Quick Look — sentences, waveform, and audio all render with JS off; zero network requests |
| M7-lite | [#19](https://github.com/sep-lab/Wit/issues/19) | **Unsigned** pilot package + PILOT.md | Clean Mac installs from the release link using only PILOT.md — recipients will see macOS's unsigned-app warning and use System Settings → Privacy & Security → "Open Anyway"; PILOT.md walks through it. Notarized distribution (Apple Developer ID) is a 0.1 upgrade, not a 0.0 blocker |

Expand Down
Loading