feat(generators): add curated macOS preferences generator - #30
feat(generators): add curated macOS preferences generator#30wgordon17 wants to merge 35 commits into
Conversation
Curated 7-domain preferences generator covering dock, Finder, screensaver, trackpad, keyboard shortcuts, keyboard text replacements, battery/power, and desktop wallpaper. Renders all four classification tiers (native system.defaults options, CustomUserPreferences, a postActivation wallpaper script, and non-skipped manual-report comments) into one preferences.nix module per host. Adds SystemConfig.wallpaper_path (read from desktoppicture.db, schema verified against a real macOS Tahoe machine) and classify_wallpaper() routing it to a Tier 3 activation script. Coerces raw pmset power values to the types nix-darwin's power.sleep.* option actually expects (null | positive-int | "never"), caught by a real nix build failure during UAT.
Introduces generate_all(), GenerateResult, and GenerateError in generators/__init__.py, scoped to the preferences domain for now -- sibling `if` blocks and a (filename, import_line) list are structured so Tasks 6/7 can extend this incrementally without restructuring. Regenerates a host's configuration.nix generated-imports section from actual on-disk file existence, with hash-based hand-edit detection mirroring add-host's flake.nix mechanism. Fills in the `mac2nix generate --hostname` CLI stub: validates the target is a scaffolded framework and the host is registered, loads a scan file or runs one inline, and reports which domains ran/were skipped/were unrecognized. Covered by a real `nix build` integration test and a real VM-based apply-and-verify test (nix run nix-darwin -- switch against a disposable Tart VM, confirming the specific curated settings this generator writes actually survive a real switch).
Validator._scan_vm()'s local-checkout override (added for this plan's
own nix_vm tests) runs `nix run <mac2nix_source> -- scan` inside the
VM, but mac2nix's own repo has never had a flake.nix -- confirmed via
`git log --all -- flake.nix` and a real `nix run` failure ("not part
of a flake"). This affected the already-merged default path too
(`github:gordon-code/mac2nix` has never had one either), so `mac2nix
validate`'s re-scan step has apparently never been exercised for real
end-to-end.
Adds a minimal flake.nix exposing a `nix run` app that delegates to
`uv run --project <self> mac2nix`, redirecting uv's venv to a fresh
writable temp directory (the flake's own source is a read-only Nix
store path once fetched, which uv must never try to write a venv
into). Verified for real: `nix run . -- scan` builds a working venv
and produces a real scan.
Also excludes `.cache` (a local dev venv/cache directory that can be
tens to hundreds of MB) from the local-checkout SCP/copy scope.
Extracts the duplicated `_register_fake_host()` helper (byte-identical across test_generate_all.py and test_generate.py) into tests/_generate_helpers.py, mirroring the existing tests/_scaffold_helpers.py convention.
mac2nix ships a minimal flake.nix (used internally to re-scan a VM during `mac2nix validate`) that also works as a standalone entry point for anyone without a local uv install.
Its read-modify-write cycle on configuration.nix/.mac2nix-meta.json assumes single-operator, sequential use, matching add_host()'s own documented limitation on flake.nix/.sops.yaml.
`flake_path.read_text()` and `host_dir.exists()` in `mac2nix generate` were unguarded — verified empirically that `Path.exists()` on Python 3.13 propagates `PermissionError` for an unreadable intermediate directory rather than returning `False`. Either call could leak a raw traceback instead of a clean `click.ClickException`, violating this command's own "never a raw traceback" contract. Extracted into `_check_scaffolded_framework()`/`_check_host_registered()` (also resolves a ruff PLR0912 branch-count violation for real, rather than suppressing it). Also adds `ac_power.autorestart` to the nix_build integration fixture and verifies it for real: `power.restartAfterPowerFailure`'s boolean coercion had only ever been confirmed by a unit test, never by an actual nix build, unlike its sibling `networking.wakeOnLan.enable`.
A `#` comment ends at the first newline -- free-text that derives from scanned, attacker-writable data (e.g. a plist key name set via `defaults write`) could otherwise embed a newline and break out of a rendered `# not automated: ...` comment, turning the remainder of the string into live Nix syntax.
Embedding the path directly into the AppleScript source string let a path containing a literal `"` terminate the string early and inject arbitrary AppleScript (including `do shell script`). Passing it as an osascript argument (argv) instead means the path is never parsed as script source. Also switches the manual-report comment line to the new nix_comment filter, and adds coverage confirming the CustomSystemPreferences bucket (reachable via any curated domain's system-scoped plist) renders correctly.
A secret is sometimes embedded in the key itself rather than the value (e.g. a literal API key used as a dict key). The sensitive-key manual-report path already redacted the value but still echoed the raw key name into `destination`, which generators surface verbatim into real, on-disk output.
`--hostname` fed directly into `host_dir = output_dir / "hosts" / "darwin" / hostname` without validation -- a value like "../../../../tmp" escapes output_dir entirely. Routes it through the same _validate_hostname callback add-host already uses, and switches _check_scaffolded_framework to the shared _HOSTS_BEGIN/_HOSTS_END sentinel constants so it can't silently drift from add_host()'s own check.
sqlite3.Connection's own context manager only commits/rolls back the pending transaction on exit -- it doesn't close the connection or its file descriptor. Wraps it in contextlib.closing() so the handle is actually released.
A stripped or corrupted MAC2NIX:GENERATE:BEGIN/END marker pair previously surfaced as a raw `ValueError: substring not found` instead of a purpose-written, actionable GenerateError.
A fresh `mktemp -d` on every `nix run` invocation forced a full dependency install each time and left orphaned venv directories behind in $TMPDIR indefinitely. uv creates any missing parent directories itself, so a stable, XDG-respecting cache path makes repeat invocations a fast no-op sync and leaves no litter. Verified for real: two consecutive `nix run . -- --version` calls, second one instant.
Before this, flake.nix's only consumer was Validator._scan_vm()'s local-source override, exercised only by nix_vm-marked tests -- which always skip in CI (no tart on GitHub-hosted runners). A syntax or evaluation error in flake.nix would pass `make test`/`make test-nix` completely undetected, silently reproducing the "nix run mechanism doesn't actually work" gap this PR found and fixed.
str(float('inf')) renders as the bare word `inf`, which Nix parses as
an undefined variable reference rather than a number literal -- a
scanned preference value that happened to be IEEE 754 infinity or NaN
(structurally possible via plistlib) would otherwise surface as an
opaque nix-instantiate syntax error with no indication of which
preference value caused it. Fails loud and immediately instead,
consistent with this function's existing TypeError for other
unsupported types.
_coerce_power_native_value() matched power booleans against a known false-value set and defaulted everything else -- including an empty string or an unrecognized future pmset value -- to true, the opposite of this generator's mkDefault-everywhere conservatism. Switches to a positive match against known true-values instead. The wallpaper activation script's osascript call talks to the logged- in user's WindowServer session, which doesn't exist during a headless or SSH-only activation. Under nix-darwin's `set -e`, that failure could abort the entire darwin-rebuild switch over a cosmetic setting. Adds a non-fatal `|| echo ... >&2` fallback so it degrades loudly instead of failing the whole activation.
…ting darwin-rebuild switch power.restartAfterPowerFailure and networking.wakeOnLan.enable were auto-applied as NATIVE options from scanned pmset values, but nix-darwin's own activation script (modules/system/checks.nix) hard-aborts the entire darwin-rebuild switch on hardware that doesn't support these features -- confirmed against nix-darwin's actual source and matching upstream issues #1236/#1244. Neither setting can be safely known-supported from a source-machine scan, so both now route to a manual-report comment instead. Also fixes two real duplicate-comment bugs confirmed via actual pmset -g custom output on real hardware (some keys report under both AC Power and Battery Power sections with different values), and makes NSGlobalDomain (cfprefsd, live) deterministically win over .GlobalPreferences (on-disk snapshot) when a scan reports both for the same curated key -- which real scans always do, since the on-disk plist's file stem never textually matches the literal domain name cfprefsd reports. Verified against Apple's own documentation that cfprefsd's in-memory cache is authoritative. Re-verified against a real Tart VM: the nix_vm integration test failed before this fix and passes after.
…te URI An unescaped '?' or '#' in the desktoppicture.db path (e.g. from an unusual macOS username) would be misparsed as the start of the file: URI's query string or fragment, silently truncating the path. Percent-encoding via urllib.parse.quote(path, safe="/") prevents this while still round-tripping through sqlite3's URI decoder correctly. Also downgrades the two soft-failure log calls in this method from warning to debug, matching this file's existing convention for every other best-effort scanner capability (missing file, corrupt db, and schema mismatch are all expected outcomes, not warning-worthy ones).
… source risk
Validator.validate()'s first step copied the caller's real flake directory
into the VM with no exclusion -- .git (full history, sometimes with secrets
committed before later encryption) and .env were copied wholesale, right
before an unpinned re-scan source runs in that same VM, which also has
outbound network access (TartVMManager configures a public DNS resolver).
A compromised upstream main would have had both read access to the copied
flake and a network path to exfiltrate it.
Fixed with a new _USER_FLAKE_EXCLUDE = frozenset({".git", ".env"}) --
deliberately narrower than the existing _LOCAL_SOURCE_EXCLUDE, which is
scoped to mac2nix's own dev-tree conventions ("data", "hack" are this
project's own directory names, not general nix-darwin flake conventions).
Reusing the broader set for an arbitrary caller's real flake risked silently
dropping a legitimately-named user directory. _LOCAL_SOURCE_EXCLUDE remains
unchanged for _scan_vm()'s own local-source-checkout copy.
Also rewrites the SECURITY comment on _DEFAULT_MAC2NIX_SOURCE to state the
risk accurately (the VM is not network-isolated, and "the user owns the
upstream repo" only holds for the mac2nix maintainer) and raises the
unpinned-source disclosure from a debug log to a warning, since it's a real
supply-chain fact the caller should see by default.
…ivate scaffold constants cli.py imported scaffold.py's private _HOSTS_BEGIN/_HOSTS_END sentinel constants directly, the only cross-module import of an underscore-prefixed name anywhere in the codebase. Adds a public has_hosts_sentinels() accessor in scaffold.py, mirroring the existing age_key_path() public-wrapper pattern, and uses it from both add_host() and cli.py's _check_scaffolded_framework() -- removing a literal duplicated boolean check between the two in the process.
…idempotency guarantees - Asserts preferences.nix survives on disk when a later sentinel-parsing failure aborts generate_all(), verifying the documented partial-write guarantee - Adds a corrupt .mac2nix-meta.json test mirroring the existing test_corrupt_state_file_handled_gracefully coverage for add_host()'s analogous mechanism - Adds a CLI-level test driving a GenerateError through to a clean ClickException, exercising the one exception-wrap branch no existing test reached - Tightens a CLI happy-path assertion from a bare "preferences" substring (which passes identically for both the success and skip code paths) to the literal "Generated: preferences" success string - Updates test_generate_integration.py's fixture comments to describe the corrected MANUAL_REPORT routing for hardware-dependent power/networking settings
A prior commit on this branch used scope "tests" (refactor(tests): extracts shared generate test helper), which wasn't in the documented scope list. Extends the list to match real, already-in-use usage rather than rewording pushed history.
92e3122 to
c42f38e
Compare
UAT — Curated macOS Preferences GeneratorReal commands, run against a scaffolded framework. The scan data is synthetic (fake dock settings, a fake API key, a fake wallpaper image) rather than a real Mac's personal data — chosen to exercise every output type in one pass, not to represent anyone's actual settings. 1. Generate a host's preferences from a scanA host was already registered via 2. Generated
|
Summary
generate_all()/mac2nix generateCLI foundation scoped to the preferences domainflake.nix, soValidator's VM-based re-scan mechanism (used by bothmac2nix validateand this PR'snix_vmtest) could never actually runnix run ... -- scaninside a VMnix buildagainst the assembled flake and a real Tart-VM apply-and-verify run (nix run nix-darwin -- switch, confirmed the generateddock.tilesizesetting survives), not just unit tests