Add a data profile for generic large-data transfers - #1
Conversation
The copy engine was never camera-specific — it reads every byte once,
checksums it, fans it out to N destinations and reads each copy back off
disk — but the layer above it assumed camera originals: ffprobe metadata,
contact-sheet thumbnails, the BRAW container check. This generalises the
tool to any large one-way transfer without touching that engine.
Add a Profile { media, data } selected with `--profile` (shorthand
`--generic`). The data profile turns the media layer off: no ffprobe, no
thumbnails, no BRAW check, so a dataset, disk image, render output or
backup is copied, verified and documented with nothing depending on
ffmpeg. The default stays `media`, so the camera-card workflow is
unchanged.
Profile is a first-class field on OffloadOptions (whose __post_init__
enforces the media knobs off for a data transfer, so a library caller
that sets only the profile still gets a clean generic copy), on Job, and
on saved presets (round-tripped through JSON, tolerant of a missing or
garbage key). It is selectable in the desktop app's Simple mode and
preset editor, where the media-only thumbnails control greys out for a
data transfer. The CLI summary drops the "N video" line under the data
profile, where it would only ever read "0 video".
Docs reframe the tool as verified copy for large data transfers with
camera-card offload built in, and the ROADMAP's "not a sync tool" entry
now distinguishes the supported one-way transfer from the two-way sync
that stays out of scope.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017D2mLdX5ABymqURvDQEJsY
owenpkent
left a comment
There was a problem hiding this comment.
Ran the branch's tests in a worktree — 44 passed across test_profile.py, test_presets.py and test_cli.py. This looks good to merge.
Enforcing the profile in OffloadOptions.__post_init__ rather than at the CLI is the right shape: it means the guarantee holds for a library caller and a preset, not just for --generic. The loader tolerating both a missing and a garbage profile key is the right amount of paranoia for something that reads a user-editable JSON file.
Two small things inline, neither blocking.
One cross-PR note: #2 adds companions.group() to engine.run unconditionally, so once both land a --profile data transfer will still label .xmp files and anything under a Proxy/ folder as companions of a "clip". That wants gating on profile.probes_media. The two branches also conflict textually in CHANGELOG.md, README.md (3 hunks), cli.py (_summarize — this PR adds the profile gate, #2 appends the paranoid suffix) and preset_editor.py; engine.py, models.py, presets.py and simple_mode.py auto-merge. Since #2 has a blocker and this doesn't, merging this one first and rebasing #2 seems the cheaper order.
| # The video count is meaningful only when media was probed; a generic | ||
| # data transfer never looks inside a file, so reporting "0 video" would | ||
| # be noise rather than information. | ||
| video = f" ({job.video_files} video)" if job.profile.probes_media else "" |
There was a problem hiding this comment.
The CLI summary is gated on probes_media, but the reports aren't: pdf.py:232 and html.py:163 both emit job.video_files unconditionally, so a --profile data job still gets a Video Files: 0 row in its header block. Same reasoning as the comment above it applies — it's noise rather than information — and the README says the reports "render a plain file listing".
| return f"{where} · {self.algorithm} · {verify} · {', '.join(self.reports) or 'no reports'}" | ||
| parts = [where, self.algorithm, verify, ", ".join(self.reports) or "no reports"] | ||
| if self.profile is Profile.DATA: | ||
| parts.insert(0, "data") |
There was a problem hiding this comment.
Inserting at 0 puts the profile ahead of the destination count, so the summary reads data · 2 destinations · sha256 · full verify. Reads a little oddly with the leading bare word — appending it, or prefixing where itself, would scan better.
What this changes
Generalises the tool from camera-card offload to any large one-way transfer, without disturbing the camera workflow.
The copy engine was never actually camera-specific — it reads every byte once, checksums it, fans out to N destinations, and reads each copy back off disk. Only the layer above it (ffprobe metadata, thumbnails, the BRAW container check) assumed camera originals. This change makes that layer switchable rather than rewriting the core.
New
Profile { media, data }, selected with--profile data(shorthand--generic):dataturns the media layer off — no ffprobe, no thumbnails, no BRAW check — so a dataset, disk image, render output or backup is copied, checksummed, verified and documented (CSV / MHL / ASC MHL / PDF / HTML) with nothing depending on ffmpeg.media, so the camera-card workflow is byte-for-byte unchanged.Threaded through as a first-class field, not a CLI-only flag:
OffloadOptions.__post_init__enforces the media knobs off for a data transfer, so a library caller that sets onlyprofile=Profile.DATAstill gets a clean generic copy.Job; the CLI summary drops the "N video" line under the data profile (where it would only ever read "0 video").Docs reframe the tool as verified copy for large data transfers with camera-card offload built in, and the
ROADMAP.md"not a sync tool" entry now distinguishes the supported one-way verified transfer from the two-way sync (reconciliation, conflict resolution, partial-file updates) that stays out of scope.What you verified
engine.probe_mod.probeto raise and runs an offload underProfile.DATA; it passes, proving ffprobe is never reached. The same job still copies and verifies (final_status == "Verified", every fileVERIFIED), so switching the media layer off does not weaken the copy/verify path.Profile.MEDIAleavesextra_probe/thumbnail_countexactly as passed; the default CLI path still resolves toProfile.MEDIAand still prints the "N video" count.__post_init__is the single source of truth. Settingprofile=Profile.DATAalongsideextra_probe=True, thumbnail_count=4still yieldsextra_probe is Falseandthumbnail_count == 0, so no caller (CLI, preset, or library) can accidentally probe a data transfer.to_dict/from_dictpreserve the profile; a config with noprofilekey or a garbage value falls back tomediarather than bricking the load.--generic --verify full --report csv,mhl) over a 5 MB file plus a subfolder, thenoffloader verifyon the resulting MHL — reports "VERIFIED — safe to erase the source". Confirmed the summary omits the video count fordataand keeps it formedia.This does not make a "Verified" verdict easier to reach: the data profile removes only metadata collection, not any read, hash, or read-back step.
docs/data-safety.mddescribes the copy/verify/delete path, which is untouched.Checklist
pytestpasses (409 tests, 8 skipped — addedtests/test_profile.py)ruff check src testspassesIf this touches the copy, verification, or delete path:
docs/data-safety.mdstill describes realityIf this changes documented behaviour:
README.md,ROADMAP.md,CHANGELOG.mdandpyproject.tomlupdatedREADME.mdis updated (400 → 409, coverage 83% → 82%)🤖 Generated with Claude Code
Generated by Claude Code