Skip to content

feat(generators): add curated macOS preferences generator - #30

Draft
wgordon17 wants to merge 35 commits into
gordon-code:mainfrom
wgordon17:feat/migration-mvp-pr3
Draft

feat(generators): add curated macOS preferences generator#30
wgordon17 wants to merge 35 commits into
gordon-code:mainfrom
wgordon17:feat/migration-mvp-pr3

Conversation

@wgordon17

Copy link
Copy Markdown
Member

Summary

  • Curated 7-domain preferences generator (dock, Finder, screensaver, trackpad, keyboard shortcuts/text replacements, battery/power, wallpaper) rendering all four classification tiers, plus the generate_all()/mac2nix generate CLI foundation scoped to the preferences domain
  • Fixes a real, previously-undiscovered gap found while building this PR's own VM test: mac2nix has never shipped its own flake.nix, so Validator's VM-based re-scan mechanism (used by both mac2nix validate and this PR's nix_vm test) could never actually run nix run ... -- scan inside a VM
  • Verified with a real nix build against the assembled flake and a real Tart-VM apply-and-verify run (nix run nix-darwin -- switch, confirmed the generated dock.tilesize setting survives), not just unit tests

Curated 7-domain preferences generator covering dock, Finder,
screensaver, trackpad, keyboard shortcuts, keyboard text replacements,
battery/power, and desktop wallpaper. Renders all four classification
tiers (native system.defaults options, CustomUserPreferences, a
postActivation wallpaper script, and non-skipped manual-report
comments) into one preferences.nix module per host.

Adds SystemConfig.wallpaper_path (read from desktoppicture.db, schema
verified against a real macOS Tahoe machine) and classify_wallpaper()
routing it to a Tier 3 activation script. Coerces raw pmset power
values to the types nix-darwin's power.sleep.* option actually expects
(null | positive-int | "never"), caught by a real nix build failure
during UAT.
Introduces generate_all(), GenerateResult, and GenerateError in
generators/__init__.py, scoped to the preferences domain for now --
sibling `if` blocks and a (filename, import_line) list are structured
so Tasks 6/7 can extend this incrementally without restructuring.
Regenerates a host's configuration.nix generated-imports section from
actual on-disk file existence, with hash-based hand-edit detection
mirroring add-host's flake.nix mechanism.

Fills in the `mac2nix generate --hostname` CLI stub: validates the
target is a scaffolded framework and the host is registered, loads a
scan file or runs one inline, and reports which domains ran/were
skipped/were unrecognized.

Covered by a real `nix build` integration test and a real VM-based
apply-and-verify test (nix run nix-darwin -- switch against a disposable
Tart VM, confirming the specific curated settings this generator writes
actually survive a real switch).
Validator._scan_vm()'s local-checkout override (added for this plan's
own nix_vm tests) runs `nix run <mac2nix_source> -- scan` inside the
VM, but mac2nix's own repo has never had a flake.nix -- confirmed via
`git log --all -- flake.nix` and a real `nix run` failure ("not part
of a flake"). This affected the already-merged default path too
(`github:gordon-code/mac2nix` has never had one either), so `mac2nix
validate`'s re-scan step has apparently never been exercised for real
end-to-end.

Adds a minimal flake.nix exposing a `nix run` app that delegates to
`uv run --project <self> mac2nix`, redirecting uv's venv to a fresh
writable temp directory (the flake's own source is a read-only Nix
store path once fetched, which uv must never try to write a venv
into). Verified for real: `nix run . -- scan` builds a working venv
and produces a real scan.

Also excludes `.cache` (a local dev venv/cache directory that can be
tens to hundreds of MB) from the local-checkout SCP/copy scope.
Extracts the duplicated `_register_fake_host()` helper (byte-identical
across test_generate_all.py and test_generate.py) into
tests/_generate_helpers.py, mirroring the existing
tests/_scaffold_helpers.py convention.
mac2nix ships a minimal flake.nix (used internally to re-scan a VM
during `mac2nix validate`) that also works as a standalone entry point
for anyone without a local uv install.
Its read-modify-write cycle on configuration.nix/.mac2nix-meta.json
assumes single-operator, sequential use, matching add_host()'s own
documented limitation on flake.nix/.sops.yaml.
`flake_path.read_text()` and `host_dir.exists()` in `mac2nix generate`
were unguarded — verified empirically that `Path.exists()` on Python
3.13 propagates `PermissionError` for an unreadable intermediate
directory rather than returning `False`. Either call could leak a raw
traceback instead of a clean `click.ClickException`, violating this
command's own "never a raw traceback" contract. Extracted into
`_check_scaffolded_framework()`/`_check_host_registered()` (also
resolves a ruff PLR0912 branch-count violation for real, rather than
suppressing it).

Also adds `ac_power.autorestart` to the nix_build integration fixture
and verifies it for real: `power.restartAfterPowerFailure`'s boolean
coercion had only ever been confirmed by a unit test, never by an
actual nix build, unlike its sibling `networking.wakeOnLan.enable`.
A `#` comment ends at the first newline -- free-text that derives from
scanned, attacker-writable data (e.g. a plist key name set via
`defaults write`) could otherwise embed a newline and break out of a
rendered `# not automated: ...` comment, turning the remainder of the
string into live Nix syntax.
Embedding the path directly into the AppleScript source string let a
path containing a literal `"` terminate the string early and inject
arbitrary AppleScript (including `do shell script`). Passing it as an
osascript argument (argv) instead means the path is never parsed as
script source. Also switches the manual-report comment line to the new
nix_comment filter, and adds coverage confirming the
CustomSystemPreferences bucket (reachable via any curated domain's
system-scoped plist) renders correctly.
A secret is sometimes embedded in the key itself rather than the
value (e.g. a literal API key used as a dict key). The sensitive-key
manual-report path already redacted the value but still echoed the
raw key name into `destination`, which generators surface verbatim
into real, on-disk output.
`--hostname` fed directly into `host_dir = output_dir / "hosts" /
"darwin" / hostname` without validation -- a value like
"../../../../tmp" escapes output_dir entirely. Routes it through the
same _validate_hostname callback add-host already uses, and switches
_check_scaffolded_framework to the shared _HOSTS_BEGIN/_HOSTS_END
sentinel constants so it can't silently drift from add_host()'s own
check.
sqlite3.Connection's own context manager only commits/rolls back the
pending transaction on exit -- it doesn't close the connection or its
file descriptor. Wraps it in contextlib.closing() so the handle is
actually released.
A stripped or corrupted MAC2NIX:GENERATE:BEGIN/END marker pair
previously surfaced as a raw `ValueError: substring not found` instead
of a purpose-written, actionable GenerateError.
A fresh `mktemp -d` on every `nix run` invocation forced a full
dependency install each time and left orphaned venv directories behind
in $TMPDIR indefinitely. uv creates any missing parent directories
itself, so a stable, XDG-respecting cache path makes repeat
invocations a fast no-op sync and leaves no litter. Verified for real:
two consecutive `nix run . -- --version` calls, second one instant.
Before this, flake.nix's only consumer was Validator._scan_vm()'s
local-source override, exercised only by nix_vm-marked tests -- which
always skip in CI (no tart on GitHub-hosted runners). A syntax or
evaluation error in flake.nix would pass `make test`/`make test-nix`
completely undetected, silently reproducing the "nix run mechanism
doesn't actually work" gap this PR found and fixed.
str(float('inf')) renders as the bare word `inf`, which Nix parses as
an undefined variable reference rather than a number literal -- a
scanned preference value that happened to be IEEE 754 infinity or NaN
(structurally possible via plistlib) would otherwise surface as an
opaque nix-instantiate syntax error with no indication of which
preference value caused it. Fails loud and immediately instead,
consistent with this function's existing TypeError for other
unsupported types.
_coerce_power_native_value() matched power booleans against a known
false-value set and defaulted everything else -- including an empty
string or an unrecognized future pmset value -- to true, the opposite
of this generator's mkDefault-everywhere conservatism. Switches to a
positive match against known true-values instead.

The wallpaper activation script's osascript call talks to the logged-
in user's WindowServer session, which doesn't exist during a headless
or SSH-only activation. Under nix-darwin's `set -e`, that failure
could abort the entire darwin-rebuild switch over a cosmetic setting.
Adds a non-fatal `|| echo ... >&2` fallback so it degrades loudly
instead of failing the whole activation.
…ting darwin-rebuild switch

power.restartAfterPowerFailure and networking.wakeOnLan.enable were
auto-applied as NATIVE options from scanned pmset values, but nix-darwin's
own activation script (modules/system/checks.nix) hard-aborts the entire
darwin-rebuild switch on hardware that doesn't support these features --
confirmed against nix-darwin's actual source and matching upstream issues
#1236/#1244. Neither setting can be safely known-supported from a
source-machine scan, so both now route to a manual-report comment instead.

Also fixes two real duplicate-comment bugs confirmed via actual pmset -g
custom output on real hardware (some keys report under both AC Power and
Battery Power sections with different values), and makes NSGlobalDomain
(cfprefsd, live) deterministically win over .GlobalPreferences (on-disk
snapshot) when a scan reports both for the same curated key -- which real
scans always do, since the on-disk plist's file stem never textually
matches the literal domain name cfprefsd reports. Verified against Apple's
own documentation that cfprefsd's in-memory cache is authoritative.

Re-verified against a real Tart VM: the nix_vm integration test failed
before this fix and passes after.
…te URI

An unescaped '?' or '#' in the desktoppicture.db path (e.g. from an unusual
macOS username) would be misparsed as the start of the file: URI's query
string or fragment, silently truncating the path. Percent-encoding via
urllib.parse.quote(path, safe="/") prevents this while still round-tripping
through sqlite3's URI decoder correctly. Also downgrades the two soft-failure
log calls in this method from warning to debug, matching this file's
existing convention for every other best-effort scanner capability
(missing file, corrupt db, and schema mismatch are all expected outcomes,
not warning-worthy ones).
… source risk

Validator.validate()'s first step copied the caller's real flake directory
into the VM with no exclusion -- .git (full history, sometimes with secrets
committed before later encryption) and .env were copied wholesale, right
before an unpinned re-scan source runs in that same VM, which also has
outbound network access (TartVMManager configures a public DNS resolver).
A compromised upstream main would have had both read access to the copied
flake and a network path to exfiltrate it.

Fixed with a new _USER_FLAKE_EXCLUDE = frozenset({".git", ".env"}) --
deliberately narrower than the existing _LOCAL_SOURCE_EXCLUDE, which is
scoped to mac2nix's own dev-tree conventions ("data", "hack" are this
project's own directory names, not general nix-darwin flake conventions).
Reusing the broader set for an arbitrary caller's real flake risked silently
dropping a legitimately-named user directory. _LOCAL_SOURCE_EXCLUDE remains
unchanged for _scan_vm()'s own local-source-checkout copy.

Also rewrites the SECURITY comment on _DEFAULT_MAC2NIX_SOURCE to state the
risk accurately (the VM is not network-isolated, and "the user owns the
upstream repo" only holds for the mac2nix maintainer) and raises the
unpinned-source disclosure from a debug log to a warning, since it's a real
supply-chain fact the caller should see by default.
…ivate scaffold constants

cli.py imported scaffold.py's private _HOSTS_BEGIN/_HOSTS_END sentinel
constants directly, the only cross-module import of an underscore-prefixed
name anywhere in the codebase. Adds a public has_hosts_sentinels() accessor
in scaffold.py, mirroring the existing age_key_path() public-wrapper
pattern, and uses it from both add_host() and cli.py's
_check_scaffolded_framework() -- removing a literal duplicated boolean
check between the two in the process.
…idempotency guarantees

- Asserts preferences.nix survives on disk when a later sentinel-parsing
  failure aborts generate_all(), verifying the documented partial-write
  guarantee
- Adds a corrupt .mac2nix-meta.json test mirroring the existing
  test_corrupt_state_file_handled_gracefully coverage for add_host()'s
  analogous mechanism
- Adds a CLI-level test driving a GenerateError through to a clean
  ClickException, exercising the one exception-wrap branch no existing
  test reached
- Tightens a CLI happy-path assertion from a bare "preferences" substring
  (which passes identically for both the success and skip code paths) to
  the literal "Generated: preferences" success string
- Updates test_generate_integration.py's fixture comments to describe the
  corrected MANUAL_REPORT routing for hardware-dependent power/networking
  settings
A prior commit on this branch used scope "tests" (refactor(tests): extracts
shared generate test helper), which wasn't in the documented scope list.
Extends the list to match real, already-in-use usage rather than rewording
pushed history.
@wgordon17
wgordon17 force-pushed the feat/migration-mvp-pr3 branch from 92e3122 to c42f38e Compare August 19, 2026 23:36
@wgordon17
wgordon17 deployed to vm-validated August 19, 2026 23:36 — with GitHub Actions Active
@wgordon17

Copy link
Copy Markdown
Member Author

UAT — Curated macOS Preferences Generator

Real commands, run against a scaffolded framework. The scan data is synthetic (fake dock settings, a fake API key, a fake wallpaper image) rather than a real Mac's personal data — chosen to exercise every output type in one pass, not to represent anyone's actual settings.

1. Generate a host's preferences from a scan

A host was already registered via mac2nix add-host (unrelated to this PR — not re-demonstrated here). From there:

$ mac2nix init ~/my-nix-config
Framework scaffolded at ~/my-nix-config
Next: mac2nix add-host --hostname <name> [--username <name>] to register a machine.

$ mac2nix generate ~/my-nix-config --hostname uat-host --scan-file scan.json --domains preferences
Generated: preferences

2. Generated preferences.nix

$ cat ~/my-nix-config/hosts/darwin/uat-host/preferences.nix
{ config, lib, pkgs, ... }:

{
  power.sleep.computer = lib.mkDefault "never";
  power.sleep.display = lib.mkDefault 10;
  system.defaults.dock.tilesize = lib.mkDefault 48;
  system.defaults.CustomUserPreferences = lib.mkDefault { "com.apple.symbolichotkeys" = { AppleSymbolicHotKeys = { "32" = { enabled = 0; }; }; }; };
  system.activationScripts.postActivation.text = (
  lib.concatStringsSep "\n" [
    (let
      settingValue = toString 3;
    in
    ''
      pmset -a hibernatemode ${lib.escapeShellArg settingValue}
    '')
    (let
      settingValue = "off";
    in
    ''
      if systemsetup -getRestartPowerFailure | grep -q "Not supported"; then
        echo "mac2nix: restart-after-power-failure not supported on this hardware, skipped" >&2
      else
        systemsetup -setRestartPowerFailure ${lib.escapeShellArg settingValue}
      fi
    '')
    (let
      settingValue = "on";
    in
    ''
      if systemsetup -getwakeonnetworkaccess | grep -q "Not supported"; then
        echo "mac2nix: wake-on-LAN not supported on this hardware, skipped" >&2
      else
        systemsetup -setwakeonnetworkaccess ${lib.escapeShellArg settingValue}
      fi
    '')
    (let
      wallpaperPath = toString ./assets/wallpaper.jpg;
    in
    ''
      sudo -u ${config.system.primaryUser} osascript \
        -e 'on run argv' \
        -e '  tell application "System Events" to tell every desktop to set picture to POSIX file (item 1 of argv)' \
        -e 'end run' \
        ${lib.escapeShellArg wallpaperPath} \
        || echo "mac2nix: could not set desktop wallpaper (no GUI session for ${config.system.primaryUser}?)" >&2
    '')
  ]
);
  # not automated: [sensitive] manual report: key '***REDACTED***' in domain 'com.apple.dock' matches a sensitive pattern
  # not automated: [out of scope] activationScripts: defaults write for com.apple.dock some-binary-pref (binary data)
  # not automated: [coverage gap] manual report: no nix-darwin option for system setting 'gpuswitch'
}

Every output type is exercised: plain settings, custom preferences, activation scripts (a bundled wallpaper, two hardware-probing power settings, and one plain pmset setting), and three tagged notes for what wasn't automated and why:

  • [sensitive] — the key name looked like a credential, so its name and value are both withheld.
  • [out of scope] — the value was binary data this tool doesn't know how to reproduce.
  • [coverage gap] — no nix-darwin equivalent exists yet for this setting.

3. Wallpaper file bundling

$ ls ~/my-nix-config/hosts/darwin/uat-host/assets/
wallpaper.jpg

The wallpaper image itself is copied alongside preferences.nix, byte-for-byte identical to the source file, so the generated config is self-contained.

4. Generated config is valid Nix, and re-running is a no-op

$ nix-instantiate --parse ~/my-nix-config/hosts/darwin/uat-host/preferences.nix
exit code: 0

$ mac2nix generate ~/my-nix-config --hostname uat-host --scan-file scan.json --domains preferences   # run again
Generated: preferences

preferences.nix is byte-for-byte identical before and after the second run.

5. Full config builds for real

$ uv run pytest tests/generators/test_generate_integration.py -m nix_build -v
tests/generators/test_generate_integration.py::test_generate_with_hardware_dependent_settings_builds_for_real PASSED
1 passed in 607.84s (0:10:07)

Builds the complete flake (home-manager, homebrew, sops-nix, and all) with the generated preferences.nix wired in — not just a syntax check.

6. Real Mac, real apply: hardware-probing settings don't break the switch

$ uv run pytest tests/vm/test_generate_vm.py -m nix_vm -v
tests/vm/test_generate_vm.py::test_generate_switches_and_matches_scan PASSED
1 passed in 814.61s (0:13:34)

Clones a real macOS VM, applies the generated config with darwin-rebuild switch, and re-scans the VM to confirm the settings actually took effect. The two power settings that need to probe the target machine's hardware support both ran their probe and skipped cleanly on hardware that doesn't support them — captured from the real switch output:

mac2nix: restart-after-power-failure not supported on this hardware, skipped
mac2nix: wake-on-LAN not supported on this hardware, skipped

The switch completed successfully. Neither setting can abort the apply anymore.

7. Wallpaper database assumption, verified on a real Mac

$ sqlite3 ~/Library/Application\ Support/Dock/desktoppicture.db .schema
CREATE TABLE pictures (space_id INTEGER, display_id INTEGER);
CREATE TABLE preferences (key INTEGER, data_id INTEGER, picture_id INTEGER);
CREATE TABLE data (value);
...

Matches the schema this tool's wallpaper lookup assumes.

Full test suite

$ uv run pytest -q
1 failed, 1808 passed, 2 skipped
$ uv run ruff check src/ tests/
All checks passed!
$ uv run ruff format --check src/ tests/
130 files already formatted
$ uv run pyright
0 errors, 0 warnings, 0 informations

The one failure is tests/vm/test_integration.py::TestSharedVM::test_snapshot_and_diff_detects_new_file, an SSH permission error against the shared test VM. It's unrelated to this PR — that file has no changes on this branch — and reproduces intermittently on its own, independent of this work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant