Skip to content

Add CoreUpgrade skill: scoped, reversible major-version upgrade for existing installs#1580

Open
asdf8675309 wants to merge 3 commits into
danielmiessler:mainfrom
asdf8675309:feat/coreupgrade-skill
Open

Add CoreUpgrade skill: scoped, reversible major-version upgrade for existing installs#1580
asdf8675309 wants to merge 3 commits into
danielmiessler:mainfrom
asdf8675309:feat/coreupgrade-skill

Conversation

@asdf8675309

Copy link
Copy Markdown
Contributor

The problem

The installer is deliberately additive — DeployCore copies only missing files and never overwrites a populated target. That's correct for a fresh install, but it means there is no supported path to move an existing install across a major version. Deploying a new payload over an old install leaves every old SYSTEM file in place, producing a broken hybrid where new and retired machinery coexist. The only alternatives today are a full reinstall (loses local state) or hand-deleting directories (one wrong rm -rf and USER data is gone). There's a missing primitive: a scoped, reversible clear-and-replace with the preserve boundary encoded as tested code rather than operator care.

This is the migration gap raised in discussion #1417: installing a newer version over a customized install force-overwrites existing hook scripts with no backup — the installer backs up settings.json but not the hooks it replaces.

The change

A new CoreUpgrade skill (skills/CoreUpgrade/) with one tool, LifeosUpgrade.ts:

  1. Plan — the clear-list is enumerated from the live install; an entry is cleared only if it passes the preserve check and the new payload ships a replacement for it. A wrong or malicious payload can neither inject a path nor clear a preserved zone; an empty payload clears nothing. Upstream deletions are honored; local content the payload doesn't know about survives.
  2. Backup — exactly the to-be-cleared entries are copied to a backup dir outside the install (enforced), verified per-entry, before any delete.
  3. Clear + deploy — every delete goes through a guarded remove (preserve-denylist + path-escape guard, case-insensitive for APFS, failing toward preservation). The payload then fills the cleared slots.
  4. Rollback — on any error, deploy-created artifacts are removed and cleared entries are restored from the backup, with partial failures named explicitly (with their backup path) rather than reported as success.

Preserved untouched: LIFEOS/USER/, LIFEOS/MEMORY/, .env*, settings*.json, private skills/_*, and any command/agent/skill the payload does not ship.

Optional --split-claude-md ends CLAUDE.md drift: it diffs your CLAUDE.md against the payload's CLAUDE.template.md (deterministic markdown block-diff, no model calls), auto-extracts pure additions into an imported LIFEOS/USER/CUSTOMIZATIONS/GLOBAL.md, and flags modified base lines for manual placement rather than guessing which version you meant. So the base CLAUDE.md can be pulled fresh on every future upgrade.

Dry-run is the default; nothing mutates without the literal --apply.

Why it's safe

  • Dry-run by default--apply is the only mutating path, and no delete happens unless its backup is verified on disk first. Every cleared entry — the hook scripts Adopting LifeOS on a machine that already has a customized PAI install — how do you handle existing hooks? #1417 flagged included — is copied to an external backup and verified before any delete.
  • Payload-aware clearing — only entries the payload ships are ever cleared; the preserve denylist + path-escape guard gate every single delete and fail toward preservation.
  • Refuses to run on the wrong thing — a LifeOS source checkout (dev tree) and an ambiguous target (a bare dir with a stray CLAUDE.md) are both rejected.
  • Named-failure rollback — anything that couldn't be auto-restored is listed with its backup path, never silently dropped.
  • Symlink-correct — payload symlinks are never deployed (matching the installer), and backup/rollback verification is lstat-based so symlink-bearing zones round-trip.

Verification

Validated on a real upgrade, not only in tests. This tool performed the actual v6.0.5 → v7.1.1 upgrade of a heavily-customized live install (--apply --split-claude-md): it created a scoped backup of only the cleared SYSTEM entries, ran the CLAUDE.md split (additions extracted to GLOBAL.md), preserved USER / MEMORY / private _* skills / .env / settings intact, and the install came up healthy on v7.1.1.

  • bun test — 27 tests / 138 assertions, green (macOS 25.5, Bun 1.3.14). Covers: preserve-zone protection (incl. case variants), payload-aware clearing, empty/wrong-payload no-op, dev-tree + ambiguous-target + missing-payload refusal, symlink safety, backup scoping + pre-existing-backup-dir refusal, rollback (restore + new-artifact removal + named failures with causes), the CLAUDE.md split (additions vs flagged modifications vs noise, no-clobber with a write-time re-check, failure atomicity), full split+rollback composition under a simulated deploy failure, and two subprocess integration tests of main() itself.
  • Clone rehearsal — the whole cut was rehearsed on a throwaway clone (--config-root <clone>) before the live apply.
  • Transpiles clean (bun build --no-bundle --target=bun, exit 0).

Honest scope of the live proof: the live apply succeeded, so the rollback path was exercised by the unit/e2e tests and the clone rehearsal, not fired in the production run. And the tool does the scoped core replacement + optional CLAUDE.md split only — post-upgrade steps (deps install/build, InstallHooks, settings reconcile, re-applying any custom edits made inside replaced SYSTEM dirs) are manual and documented in the skill's "Post-Upgrade Steps," printed by the tool at the end of a run.

Scope

Additive — one new skill directory (SKILL.md + Tools/LifeosUpgrade.ts + Tools/LifeosUpgrade.test.ts) and one line added to CLAUDE.template.md (the commented @…/CUSTOMIZATIONS/GLOBAL.md import the split activates). No existing behavior changes; the installer and every current flow are untouched. disable-model-invocation: true — it only runs when a human calls it.

asdf8675309 and others added 3 commits July 23, 2026 15:44
…UDE.md customization split

The installer is additive (DeployCore never overwrites), so no path exists
to replace the core across a major version. CoreUpgrade is that primitive:
payload-aware clear of only the SYSTEM entries the new payload ships,
per-entry-verified scoped backup before any delete, guarded rollback with
named failures, dry-run by default.

Optional --split-claude-md diffs the user's CLAUDE.md against
CLAUDE.template.md, auto-extracts pure additions into an imported
USER/CUSTOMIZATIONS/GLOBAL.md, flags modified base lines for manual
placement, and replaces CLAUDE.md with the fresh base — so future base
updates pull cleanly. Template gains the commented GLOBAL import, activated
by the existing ActivateImports.ts flow.

20 tests / 111 assertions, standalone via bun test.

Co-authored-by: Claude <noreply@anthropic.com>
…gration tests

- copyMissing now SKIPS payload symlinks (matches upstream DeployCore, which
  never deploys them) — closes a symlink-target-injection surface a tampered
  payload could exploit, and makes the 'mirrors DeployCore' claim true.
- Backup/rollback verification is lstat-based (lexists): a dangling symlink
  in a cleared zone no longer spuriously aborts a successful backup or
  mis-reports a clean rollback as PARTIAL (existsSync follows links).
- Default config-root honors CLAUDE_CONFIG_DIR (PR danielmiessler#696 convention, matches
  ActivateImports.ts) before falling back to ~/.claude.
- Rollback/split-undo failures now log their cause instead of a bare path.
- Comments: deliberate inline dev-tree predicate (skill self-containment);
  lexical backup-dir guard limitation noted at the check site.
- Tests 20→27 (111→138 assertions): two subprocess integration tests of
  main() (dry-run inert; apply proves backup-precedes-delete + preserve
  boundary), forced-throw split atomicity (before/after GLOBAL write),
  dangling-symlink backup/rollback round-trip, payload-symlink skip,
  preflight missing-payload/ambiguous-target, pre-existing-backup-dir
  refusal, tracked temp-dir cleanup.

Co-authored-by: Claude <noreply@anthropic.com>
Surfaced by running the tool on a real v6->v7 upgrade: the tool copies
source but not node_modules or built frontend output, so a deployed
component with a new dependency or a build step fails until the user runs
its bun install (+ build). Concrete case hit: LIFEOS/PULSE needs bun
install, and its Observability dashboard needs bun install && bun run
build, or the dashboard 503s.

- Add it as SKILL.md Post-Upgrade Step 2 (before InstallHooks, since tools
  may need deps too).
- Add a runtime warning to planUpgrade() so every dry-run/apply prints it.

Docs + one warning string; no logic change. 27 tests / 138 assertions green.

Co-authored-by: Claude <noreply@anthropic.com>
@danielmiessler

Copy link
Copy Markdown
Owner

Not closing this one either way yet — it's under active consideration. The engineering is excellent (the test matrix, verified backup-before-delete, and the honest-scope section especially), and the migration gap it targets is the loudest recurring theme in these discussions. The open question is doctrine, not code: the supported cross-major path has deliberately been fresh-install + port USER, and adopting this means owning an in-place upgrade primitive and its support surface long-term. I want to decide that deliberately rather than in a triage pass — you'll hear back here either way.

@asdf8675309

Copy link
Copy Markdown
Contributor Author

Sounds good - I worked on it so that I could use it to go from v6 to v7. After going through a pretty manual upgrade path to go from v5 to v6 -- with 77 customizations. So shifting some of the customizations into a portion that doesn't change alongside working on what could potentially go upstream was the approach this time around. Perfect - nah - but if nothing else I was hoping it would give a start to some thoughts about a potential upgrade script.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants