Skip to content

fix(storage): stabilize Accounts cloud registry routing#12

Merged
andrei-hasna merged 15 commits into
mainfrom
codewith/466d092b-storage-stabilization
Jul 14, 2026
Merged

fix(storage): stabilize Accounts cloud registry routing#12
andrei-hasna merged 15 commits into
mainfrom
codewith/466d092b-storage-stabilization

Conversation

@andrei-hasna

@andrei-hasna andrei-hasna commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Tasks

  • Accounts: 8b45b77f-9218-4aa3-b59d-b0b695c59cea
  • Alumia launch: 466d092b-b6d6-4882-a120-851e553e6142

Restack Strategy

Scope And Blast Radius

  • AccountsStore routing with fail-closed explicit self_hosted/cloud configuration.
  • Cold-process custom-tool hydration across add, launch, login selection, import, supervisor, and explicit CLI lookup paths.
  • Name/path validation, failed API-write cleanup, partial import-copy cleanup, and atomic local JSON replacement.
  • Raw machine-local current/applied/tool-lock reconciliation on API rename/remove.
  • PostgreSQL selection FK/row locking plus advisory serialization of custom-tool add/remove against account creation.
  • No publish, merge, deployment, installed-package, secret, DNS, or cloud-state change.

Compatibility

No breaking removal is approved. Deprecated storage constants, types, status/snapshot functions, ensureProfileForLogin, and accounts storage status|push|pull|sync remain.

accountsStorageSnapshotKey, storagePush, storagePull, and storageSync preserve their old optional NodeJS.ProcessEnv argument. Retired CLI push|pull|sync commands parse --json before returning the deterministic retirement diagnostic. A dedicated TypeScript compile fixture and CLI regressions enforce both contracts.

Old-client/new-server remains compatible. New-client/old-server accepts minimal additive Tool responses and existing endpoints; rename/custom-tool mutations require the new server and fail with an actionable version diagnostic.

Migration And Deployment

  1. Take and verify a restorable database backup.
  2. Run the new accounts-migrate through migrations 0003 and 0004.
  3. Inspect current_selection_orphan_archive; 0004 archives orphan selections before removing them, preserves valid rows, and adds the cascading FK.
  4. Start the server and require health/readiness/version/tools/OpenAPI probes.
  5. Roll clients only after server readiness.

Never run a pre-0003 migrator after 0003/0004 are recorded. The migration status/ledger rejects unknown applied migrations as a deterministic downgrade guard. Application rollback keeps the new migrator job with the older app image; otherwise forward-fix. Do not drop additive tables/FKs as an application rollback.

Validation

  • Frozen install with 259200-second release-age policy: passed, no lock changes.
  • Typecheck and dedicated legacy TypeScript compatibility compile: passed.
  • Sanitized full suite: 276 pass, 11 expected platform/PostgreSQL conditional skips, 0 fail.
  • Required live PostgreSQL: 6 pass, 0 fail, using a disposable local database that was dropped afterward.
  • Build: passed; generated declaration preserves legacy storage signatures and the original public importProfile arity.
  • Build-then-no-cloud scan: passed.
  • Generated SDK/contracts: 13 pass; contract examples, vendor-kit, and repository conformance passed.
  • Migration coverage: 0003/0004 upgrade, orphan archive, valid-row preservation, restart idempotency, legacy-manifest downgrade rejection, rollback/forward-fix, FK/transaction rollback, selection races, and custom-tool/account-create race.
  • git diff --check: passed.
  • Staged gitleaks: 16.37 KB before commit and 0-byte clean index before push; no findings.
  • Base-to-head gitleaks: 13 commits / 170.84 KB; no findings.
  • Current source/docs contain no stale 0.2.x claims.
  • Commit identity: Andrei Hasna <andrei@hasna.com>; no Co-Authored-By trailer.

Adversarial Self-Review

No agent was spawned because the task explicitly prohibited it. The labeled self-review challenged migration data loss, no-op downgrade bypass, cooperative-lock ordering, remove-first/create-first race outcomes, copy failure after partial mutation, public declaration drift, ambient Accounts env contamination, authorship, and PR #11 provenance.

Reconciled findings: archive orphans before cleanup; enforce migration status compatibility before no-op; lock custom-tool/account mutations by tool id and validate custom-tool existence; keep copy inside cleanup; preserve the two-argument public import declaration; run full tests with CI-equivalent Accounts env sanitization.

Residual risk: migration 0004 changes live orphan rows and retains the backup-first requirement. Migration 0005 must remain installed during application rollback; pre-0005 migrators remain downgrade-blocked.

Final P1 Revision (980e75e)

  • Additive, idempotent migration 0005 creates durable custom-tool tombstones and database lock/guard triggers.
  • Old-client wire POST with an unseen legacy custom tool id succeeds without Tool registration.
  • Explicit removal persists a tombstone; account creation cannot resurrect it. Explicit Tool registration is the only reactivation path.
  • Repository and database writers share the same tool advisory key. Live tests force both create-first and remove-first orderings and prove no dangling account/tombstone state.
  • Older direct account/custom_tools writers are protected by migration triggers; rollback retains migrations 0003/0004/0005 and the new migrator.
  • Final validation: frozen install, typecheck, legacy compile/CLI, 276-pass full suite, build, no-cloud, generated SDK/contracts (13 pass), contract examples, vendor-kit, and repository conformance.
  • Required disposable PostgreSQL: 8 pass, 0 fail; test database removed.
  • Gitleaks: staged 16.41 KB and base-to-head 14 commits / 187.25 KB, no findings.

Runtime Role Review (1b4db0c)

  • Migration and serving identities are distinct dedicated LOGIN NOINHERIT roles with no elevated attributes or memberships. The migrator fails closed without HASNA_ACCOUNTS_RUNTIME_ROLE, verifies ownership of the dedicated schema and all relations/functions, revokes PUBLIC access, and reapplies direct least-privilege grants after migrations and on current-schema runs.
  • Migration 0005 trigger functions remain SECURITY INVOKER, are migration-owner controlled, pin search_path to pg_catalog plus the dedicated Accounts schema, and grant neither PUBLIC nor runtime-role EXECUTE.
  • The runtime role receives schema USAGE; account/current/custom-tool DML; tombstone SELECT/INSERT/DELETE only; and ledger/API-key SELECT only. It receives no schema CREATE or tombstone UPDATE.
  • Required live PostgreSQL creates a non-superuser owner and separate DML-only app role, migrates as owner, reconnects as app, proves normal account/custom-tool flows, and forces both raw INSERT accounts versus DELETE custom_tools orderings without AccountsRepo locks. Result: 10 pass, 0 fail.
  • CI adds a gitleaks v8.30.1 PR-range job using the official Linux x64 archive pinned to SHA-256 551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb. It has read-only contents permission, no persisted checkout credential or secrets, full redaction, and ignores PR-controlled config/ignore/allow directives.
  • Exact source validation: frozen Bun install, typecheck, legacy compile/CLI, 276-pass full suite, build then no-cloud, SDK/contracts (13 pass), contract examples, vendor-kit, repository conformance, and live role-separated PostgreSQL (10 pass).
  • Independent scans: staged 26.61 KB, post-commit index 0 bytes, and full base-to-head 15 commits / 213.85 KB; no findings.

andrei-hasna and others added 15 commits July 14, 2026 16:53
…ore + ApiStore)

Consolidate the registry behind one Store abstraction: LocalStore (on-box JSON)
and ApiStore (self-hosted/cloud <API_URL>/v1 + bearer). resolveStore() is the
mode resolver from env. CLI commands and MCP tools now read/write the registry
only through the Store — no scattered cloud/local branches, no raw fetch/sqlite
outside the Store/server. Extends cloud-awareness to list/show/use/active/current/
env/launch/shell/set/rename/detect/path/remove/doctor.

- New src/lib/store.ts: AccountsStore interface + LocalStore + ApiStore + resolveStore
- Add update()/rename() to the cloud client; add PATCH-backed update + new
  POST /v1/accounts/:tool/:name/rename server endpoint (repo + schema + OpenAPI + SDK).
  NOTE: rename endpoint requires an ECS redeploy of accounts-serve.
- Depend on @hasna/contracts@0.5.2
- Tests for Store transport selection/routing, cloud update/rename, server rename
- Bump version to 0.2.0

Machine-local orchestration (apply/switch/run/login/pick/supervisor) and the
applied/toolLocks maps remain per-machine by design; they resolve the profile
record through the Store.
Complete the single-AccountsStore refactor so no CLI command, MCP tool, or
SDK method reads/writes the accounts registry via the local JSON store in
self_hosted/cloud (api) mode.

- resolveSupervisorLaunch is now async and resolves profiles + the active
  selection through the Store (was calling getProfile/currentProfile/
  appliedProfile directly from profiles.js). The machine-local applied
  pointer name still lives on-box, but its record is resolved via the Store.
- Supervised launch/restart marks the active selection via store.useProfile
  instead of the local-only useProfile, eliminating split-brain current
  writes in api mode.
- Delete dead/duplicate ensureProfileForLogin + findProfileByName from
  import-profile.ts (login uses the Store-routed existingOrCreateProfile);
  drop the stale export and tests.
…rage through the Store

Removes the dead dual code path flagged by adversarial review:
- storage.ts kept ONLY the local registry primitives (accountsHome, storePath,
  profilesDir, loadStore, saveStore). Deleted AccountsStorageMode
  local|remote|hybrid, parseMode, the S3 client, storagePush/Pull/Sync,
  snapshots, and all HASNA_ACCOUNTS_S3_* / STORAGE_MODE env plumbing. The only
  storage abstraction is now AccountsStore (LocalStore + ApiStore).
- Removed the CLI `storage status|push|pull|sync` command group and its print
  helpers (they bypassed the Store and used forbidden remote/hybrid words).
- Dropped the @aws-sdk/client-s3 dependency and its build-script external flag.

Fixes the live crash: a stale HASNA_ACCOUNTS_STORAGE_MODE=remote|hybrid|s3
(left from the S3 era) made every registry command throw via the contracts
resolver. deriveEnv now only honors the canonical local|self_hosted|cloud
words (mode's sole authority is to force local) and strips any other value so
it can never reach the resolver; URL+KEY presence remains the cloud toggle.

Reworked health/readiness storage section to report local vs self_hosted (api)
via the Store resolver instead of the deleted S3 status, and rewrote
storage.test.ts to cover the local primitives plus the legacy-mode crash
regression.
In cloud mode 'accounts tools add/remove' wrote to the local accounts.json
and never reached the shared cloud registry (split-brain). Add a cloud
custom_tools table + POST/DELETE/GET /v1/tools server routes, route the CLI
and MCP tool commands through AccountsStore.listTools/addTool/removeTool, and
have ApiStore refresh a machine-local resolution cache so synchronous
getTool/listTools still resolve cloud-registered tools for launch/apply.

Bump 0.2.1.
The health/readiness command and doctor cross-checks read profiles, custom
tools, and the per-tool 'current' selection from the local JSON file via
loadStore(), even in api (self_hosted/cloud) mode where those are cloud-owned.
On a flipped machine with an empty/stale local file this reported empty or
stale profiles and a wrong active selection while claiming transport=api — a
split-brain / routing-bar violation.

- getAccountsReadiness is now async and reads profiles, tools, and current
  through resolveStore(); 'applied' stays machine-local.
- doctor reads current through the Store and applied via loadAppliedMap().
- add loadAppliedMap(): reads the machine-local applied pointer without the
  local-profile pruning loadStore() applies, so it stays correct in api mode
  (profiles live in the cloud) and lets doctor still flag stale pointers.
- delete the dead toolsFromStore() local-merge helper.
- regression test proves api-mode readiness reflects the Store, not the file.
accounts list crashed (exit 1, "unknown tool") against the shared cloud
registry whenever it held profiles for tools not in the client's local
tool set (e.g. aicopilot, browserplan): per-profile prelaunch rendering
called getTool(), which threw and aborted the whole listing.

- prelaunchSummaryFor now degrades an unknown tool to an unsupported
  summary instead of throwing.
- list hydrates the local tool cache from the active Store first, so
  cloud-registered custom tools resolve fully (best-effort).
- action() wrapper now surfaces cloud HasnaHttpError as a single clean
  error line + exit 1 (with a redeploy hint on 404) instead of dumping
  an uncaught Node stack trace, which is what rename/tools add/tools
  remove did when they hit a stale server missing those routes.

Adds a regression test that lists profiles referencing unknown tools.
…oint

Mutating registry calls (accounts rename, tools add, tools remove) route
correctly through the Store to endpoints that already exist in src/server.
When the connected accounts-serve is an older deployed build, those routes
are absent and return a generic route-missing 404 ({"error":"not found"}).

Surface an actionable 'redeploy accounts-serve' diagnostic for that case
instead of a raw HTTP failure, while leaving genuine entity-404s untouched.
Adds regression coverage. Bump 0.2.4.
…istent profile

The ApiStore.remove path threw a plain Error for entity-not-found and
multi-tool ambiguity, so the CLI action() wrapper (which only cleans up
AccountsError + HasnaHttpError) re-threw it as an unhandled exception,
dumping a stack trace that leaked the internal cli.js path. Local mode
already threw AccountsError. Now cloud mode matches: a single clean
'error: no profile named ...' line + exit 1. Adds regression tests.
@andrei-hasna
andrei-hasna merged commit ae89e37 into main Jul 14, 2026
2 checks passed
@andrei-hasna
andrei-hasna deleted the codewith/466d092b-storage-stabilization branch July 14, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant