diff --git a/.gitleaksignore b/.gitleaksignore index 6d17ea827..af4b83ece 100644 --- a/.gitleaksignore +++ b/.gitleaksignore @@ -37,3 +37,11 @@ baa7a0bb4b5f4c3112680c37ab7572663bb87551:apps/desktop/vite.webclient.config.ts:g # that introduced the literal still carries it in its own patch, and gitleaks # fingerprints are commit-scoped. Scoped to that one commit and finding. 359ae24fd3fdf638b1e4b28cbeea34c068285aad:apps/ade-cli/src/services/diagnostics/diagnosticReport.test.ts:generic-api-key:54 + +# cli.test.ts's formatDiagnosticError redaction tests need secret-shaped +# fixtures to prove they are stripped. The working tree assembles them from +# segments at runtime, but the commit that introduced the literals still +# carries them in its own patch, and gitleaks fingerprints are commit-scoped. +# Scoped to that one commit and its two findings. +7ec89db49bbd5f9724915222008be6fc0b264682:apps/ade-cli/src/cli.test.ts:generic-api-key:11792 +7ec89db49bbd5f9724915222008be6fc0b264682:apps/ade-cli/src/cli.test.ts:generic-api-key:11808 diff --git a/apps/account-directory/README.md b/apps/account-directory/README.md index 98b2a5ae3..a964c6793 100644 --- a/apps/account-directory/README.md +++ b/apps/account-directory/README.md @@ -59,11 +59,10 @@ nothing at all. Two proofs are accepted, either one sufficient: 2. **A pairing grant (fallback).** 32 random bytes minted at `POST /device/token` — the one interactive sign-in this Worker runs end to end — stored as a SHA-256 digest in `machine_pairing_grants`, bound to the signing-in user and - to the `machine_key` declared back at `POST /device/code`, valid for - `PAIRING_GRANT_TTL_MS` (10 minutes), and redeemed by a single conditional - `DELETE` so it is spendable exactly once. A removed machine holding only an - old access token cannot obtain one: minting requires completing the browser - half of the device flow. + to the `machine_key` declared back at `POST /device/code`, and valid for + `PAIRING_GRANT_TTL_MS` (10 minutes). A removed machine holding only an old + access token cannot obtain one: minting requires completing the browser half + of the device flow. The fallback exists because path 1 fails closed and ADE's brain authenticates with a Clerk **OAuth access token**, whose documented claim set does not include @@ -75,11 +74,161 @@ actionable message rather than a bare status. An accepted re-pair clears the relay's revocation first, so a machine is never back on the roster while still unable to publish. +### Spending a grant takes two phases + +A grant is spendable exactly once, but a spend is not a single `DELETE`. The +relay hand-off that follows can fail, and destroying the grant before knowing +the outcome meant a relay outage burned the only credential a reinstalled +machine had — the same lockout the grant exists to prevent, moved one step +later. So redemption is: + +1. **Reserve.** One atomic `UPDATE ... SET reserved_at` whose `WHERE` still + carries every rule (this user, this machine, inside its TTL, not already + held). `changes === 1` is the whole proof, so two concurrent registrations + can no more both spend it than they could before. +2. **Consume** (`DELETE`, scoped to that reservation) once the relay agrees, or + **release** (`SET reserved_at = null`) when it does not. + +A release restores the row exactly as it was. `expires_at` is never rewritten, +so an attacker who can force relay failures gains nothing beyond the TTL the +grant was minted with. A reservation older than `PAIRING_GRANT_RESERVATION_MS` +(60 s) is ignored, so a Worker that dies mid-hand-off strands the grant for a +minute rather than until it expires. + +## Superseding a rotated machine key + +Machines are keyed `(user_id, machine_key)`, so a client that rotates its +identity file — a reinstall, a wiped config directory, a restored backup — +arrives as a **second row for one physical computer**. The user then removes the +row that looks stale, and half the time that is the live install. + +A register call whose `deviceId` **or** `hardwareId` matches other rows on the +same account therefore deletes those rows and reports them: + +```json +{ "machineKey": "...", "supersededMachineKeys": [""] } +``` + +The field is additive and omitted when nothing was superseded, so existing +clients are unaffected. Three rules bound it: + +- **Two identifiers, one union.** `deviceId` catches an in-place reinstall, + where `~/.ade/secrets` survived. `hardwareId` — an optional, per-account + sha256 of an OS-level machine identifier (`IOPlatformUUID`, `MachineGuid`, + `/etc/machine-id`) — catches a full `~/.ade` wipe, where the device id was + minted fresh alongside the machine key and matches nothing. It is salted with + the account id, so one machine seen by two accounts stores two unrelated + values and the column cannot correlate users. Rows with a null `hardware_id` + (written before it shipped, or by a host that cannot read one) are matched by + `deviceId` only, and nothing back-fills them. +- **Same trust bar as a re-pair.** `deviceId` and `hardwareId` are both + caller-supplied and forgeable, so on a plain token they authorize nothing — + otherwise any machine could claim another's identifiers and delete its row. + The call must carry proven-fresh interactive authentication or spend a pairing + grant, exactly as un-revoking does. A grant is only spendable on + `pairing: true`; the claim is honored on any register, because it is a + property of a token this Worker verified. +- **At most 5 rows per call** across both identifiers, oldest-seen first. The + rest go on the next proven re-pair. + +It **folds**, it does not merely delete: the one thing a superseded row holds +that the new one cannot rebuild is `custom_name`, the name the user typed. The +most recently seen superseded name is carried onto the surviving row, and only +when that row has no name of its own — a name set on the new row is the fresher +statement of intent. The carry-forward and the deletes go out as a single +`DB.batch()`, because the pairing grant is already spent by the time they run +and a half-finished loop would leave phantoms behind with no credential left to +clear them. + +Superseded keys get **no** `revoked_machines` row. The physical device holds the +new key, and blocking the old one would trapdoor any client that rolls its +identity file back into a permanent refusal; an absent key simply registers +again. The relay is not called either — the device never left the account, so +its Activity is still the user's own. + +## Refusal logs + +Every refusal on this Worker is a user who cannot get their computer back onto +their account, and by the time they ask for help the request is gone. Each +refusal path emits exactly one structured line to `console.log` (Workers +observability runs at `head_sampling_rate: 1`): + +```json +{"event":"directory.register_refused","userId":"user_…","machineKeyPrefix":"abcdef12", + "deviceIdPrefix":"01234567","code":"machine_revoked","correlationId":"…"} +``` + +`event` is one of `directory.register_refused`, `directory.remove_refused`, or +`directory.supersede_refused`; `code` is the wire code the client received +(`machine_revoked`, `pairing_authentication_required`, +`activity_relay_unavailable`, `activity_purge_failed`, +`supersede_authentication_required`), and an optional `reason` carries the finer +classification support actually needs — `no_proof` versus `grant_rejected`, or +the relay's own failure text. `correlationId` joins the line to the request the +client logged. + +Identifiers appear as **8-character prefixes only**. A machine key is +capability-shaped and a grant is a live credential; no full key, token, or grant +is ever logged. + +There is no admin route for restoring a machine by hand, and this change did not +add one: the Worker has no secret-gated inbound surface to extend +(`DIRECTORY_AUTH_SECRET` is outbound provenance for the relay, not an inbound +credential), and adding one would be a new authentication boundary guarding +exactly the tables `wrangler d1 execute --env production` already reaches. +Support recovery is a direct D1 statement — typically +`delete from revoked_machines where user_id = ? and machine_key = ?` — after the +refusal logs above identify the row. + Machine registration and list records may carry a `pubkey` string. Current ADE hosts publish `ed25519:` so clients can verify and seal account adoption on direct or relay routes. The Worker treats the value as opaque metadata and rejects values longer than 128 characters. +## Diagnostic report uploads + +`POST /diagnostics/upload` is the destination for ADE's "Send to ADE" button and +`ade report-issue --send`. It exists because support round-trips were the real +cost of a broken install: the report is already built and fully redacted on the +user's machine, and asking someone whose ADE will not start to run terminal +commands and paste output is where most of them stalled. + +**Contract** + +| | | +|---|---| +| Method | `POST` (plus `OPTIONS` preflight; anything else is `405`) | +| Body | `text/plain` — the report itself; or `application/json` — `{ report, installId?, appVersion? }` | +| Metadata on `text/plain` | `?installId=` / `?appVersion=` query parameters | +| Auth | **Optional** `Authorization: Bearer `, verified exactly as the account routes verify it. Absent, the upload is anonymous. A header that is sent and does not verify — or does not even parse as `Bearer ` — is `401`, never silently downgraded. A Worker with no Clerk configuration answers `503`, exactly as the account routes do | +| Origin | `403` when the browser reports `sec-fetch-site: cross-site` from a real remote origin. ADE's own senders are unaffected: the CLI sends no fetch-metadata header, and the Electron renderer's `null` (packaged `file://`) and loopback (development) origins are exempt | +| Size | `413` above 512 KB. `content-length` is checked first, then the stream is counted as it arrives, so a missing or dishonest length changes nothing | +| Rate limit | 5 per UTC day per user (signed in) or per `cf-connecting-ip` (anonymous) → `429` with `retry-after: 86400`. Off Cloudflare there is no trustworthy address, so anonymous callers share one bucket; `x-forwarded-for` is caller-controlled and is never read | +| Success | `200 {"ok": true, "id": ""}`. The report is **never** echoed back | +| Storage | `reports///.md` in the `DIAGNOSTICS` R2 bucket, with `userId` / `installId` / `appVersion` as custom metadata | +| No binding | `503`, and the in-app button says sending is unavailable | + +The key's identity segment is `u-` when signed in and +`anon-` otherwise — the *same* segment the quota is counted +on, so one prefix listing answers both "where does this go" and "has this caller +had enough today". + +CORS is `*` on this route only. The desktop button runs in Electron's renderer, +whose origin is `file://` (`Origin: null`) in a packaged build, so no fixed +allow-list can name it; `*` is safe here because the route reads no account +state, returns only an opaque id, and cannot be used with +`credentials: "include"`. Every `/account/*` route keeps its exact-origin rule. + +**Rate limiting without a migration.** The device flow counts attempts in the +`device_approval_rate_limits` D1 table. This route deliberately does not: it +ships without touching `migrations/`, so the quota is enforced by a per-isolate +counter (fast, but lost when Cloudflare recycles the isolate) backed by an R2 +prefix listing (durable and global, one class-A operation per upload). The +listing is not transactional, so genuinely simultaneous requests can land a +couple of objects over five. For a bound whose only job is "one person cannot +fill the bucket", that is an acceptable trade; if volume ever justifies exact +counting, move it to the D1 pattern the device flow already uses. + ## Local checks ```sh @@ -114,7 +263,33 @@ deployment: `DIRECTORY_AUTH_SECRET` (`npx wrangler secret put DIRECTORY_AUTH_SECRET`) to the same value configured on the push relay; machine removal and re-pairing both fail loudly without it. -3. Apply the remote migrations and deploy the Worker. Use +3. Create the R2 bucket behind the `DIAGNOSTICS` binding, **before** the deploy + that first references it — `wrangler deploy` does not create buckets, and a + Worker bound to a bucket that does not exist fails to start: + + ```sh + npx wrangler r2 bucket create ade-diagnostics # default environment + npx wrangler r2 bucket create ade-diagnostics-production # production + ``` + + The binding is optional in code, so an already-deployed Worker whose bucket + was removed answers `503` on `/diagnostics/upload` and keeps every other + route working. +4. Give both diagnostics buckets an expiry lifecycle rule. **Nothing in the + Worker ever deletes a report**, so without this the bucket grows forever and + every report a user ever sent stays readable indefinitely. Ninety days is the + default because it is far longer than any support thread and far shorter than + "forever" — shorten it if your retention policy says so: + + ```sh + npx wrangler r2 bucket lifecycle add ade-diagnostics \ + expire-reports reports/ --expire-days 90 + npx wrangler r2 bucket lifecycle add ade-diagnostics-production \ + expire-reports reports/ --expire-days 90 + ``` + + Confirm with `npx wrangler r2 bucket lifecycle list `. +5. Apply the remote migrations and deploy the Worker. Use `npm run d1:migrate:production` and `npm run deploy:production` for the production environment. Each deploy script validates only the environment it is about to publish, so an unconfigured development Worker cannot block a diff --git a/apps/account-directory/migrations/0007_pairing_grant_reservations.sql b/apps/account-directory/migrations/0007_pairing_grant_reservations.sql new file mode 100644 index 000000000..8cf5139d8 --- /dev/null +++ b/apps/account-directory/migrations/0007_pairing_grant_reservations.sql @@ -0,0 +1,38 @@ +-- Two-phase redemption for pairing grants. +-- +-- Redemption used to be a single DELETE: the grant was destroyed BEFORE the +-- activity relay was asked to lift the machine's publish block, and it was +-- deliberately not put back when that hand-off failed. The reasoning was sound +-- as far as it went — restoring a deleted grant means either a non-atomic +-- read-then-write (two concurrent registrations each see it unspent) or +-- re-issuing it with a fresh expiry (an attacker who can force relay failures +-- keeps one alive indefinitely) — but the cost landed on real users: a relay +-- outage during a re-pair burned the only credential a reinstalled machine had, +-- and the account token it still held could not mint another. That is the +-- lockout the revocation work was supposed to have removed. +-- +-- `reserved_at` splits the spend in two so neither horn of that dilemma +-- applies. Phase one is an atomic UPDATE that claims the grant — same single +-- statement, same `changes === 1` proof, so concurrency is unchanged and no +-- second registration can hold it at the same time. Phase two either deletes +-- the row (relay agreed) or clears `reserved_at` back to null (relay failed). +-- +-- The release restores the row EXACTLY as it was: `expires_at` is never +-- rewritten, so forcing relay failures buys an attacker nothing beyond the +-- original TTL the grant was minted with. +-- +-- A null `reserved_at` means unheld. A reservation older than the worker's +-- crash-safety bound also counts as unheld, so a worker that dies mid-relay +-- strands the grant for at most that long rather than until it expires; the +-- bound lives in the worker (`PAIRING_GRANT_RESERVATION_MS`) because it is a +-- property of one relay round trip, not of the schema. +-- +-- Additive and backfill-free: every existing row reads as unheld, which is what +-- an unspent grant already was. +alter table machine_pairing_grants add column reserved_at integer; + +-- The register path already selects duplicate rows for one (user, device) pair +-- when a proven re-pair supersedes a rotated machine key. The existing index is +-- (user_id, last_seen_at), which does not serve that predicate. +create index if not exists idx_machines_user_device + on machines(user_id, device_id); diff --git a/apps/account-directory/migrations/0008_machine_hardware_anchor.sql b/apps/account-directory/migrations/0008_machine_hardware_anchor.sql new file mode 100644 index 000000000..3f0367d4a --- /dev/null +++ b/apps/account-directory/migrations/0008_machine_hardware_anchor.sql @@ -0,0 +1,26 @@ +-- A machine identifier that survives a full `~/.ade` wipe. +-- +-- Supersede-by-device dedup (0007's index, and the register path that uses it) +-- assumed the device id outlives a reinstall. It does not: `sync-device-id` +-- lives in `~/.ade/secrets` next to the machine key, so the user who deletes +-- `~/.ade` and signs in again mints BOTH halves fresh. There is then nothing to +-- match on, and the account keeps a row for a computer the user owns once. +-- +-- `hardware_id` is the client's per-account hash of an OS-level machine +-- identifier — `IOPlatformUUID`, `MachineGuid`, `/etc/machine-id` — that no ADE +-- uninstall can remove. It is HASHED WITH THE ACCOUNT ID, so the same physical +-- machine registered under two accounts stores two unrelated values and this +-- column cannot be used to correlate users. +-- +-- Nullable and never backfilled. Rows written before this shipped, and rows +-- from clients that cannot read an anchor, simply keep matching on device id — +-- which is still the common case for an in-place reinstall. +alter table machines add column hardware_id text; + +-- Serves the second half of the supersede predicate. Same shape and reasoning +-- as `idx_machines_user_device`: the register path selects duplicate rows for +-- one (user, anchor) pair, and the (user_id, last_seen_at) index does not serve +-- that. Rows with a null anchor are still indexed by SQLite but are never +-- matched, because `hardware_id = null` is never true. +create index if not exists idx_machines_user_hardware + on machines(user_id, hardware_id); diff --git a/apps/account-directory/src/activityRelay.ts b/apps/account-directory/src/activityRelay.ts new file mode 100644 index 000000000..b48d0990b --- /dev/null +++ b/apps/account-directory/src/activityRelay.ts @@ -0,0 +1,125 @@ +import { logActivityRelayFailure } from "./logging"; +import { trustedHttpsOrigin } from "./trustedOrigin"; + +/** + * The slice of the Worker env this hand-off needs. Declared here rather than + * imported from `directory.ts` so the relay call has no dependency on the + * routing module that uses it. + */ +export type ActivityRelayEnv = { + /** + * Push relay origin. The relay is a different worker over a different D1, so + * machine membership changes have to be forwarded to it explicitly: it owns + * the Activity feed and the roster of machines allowed to publish into it. + */ + PUSH_RELAY_URL?: string; + /** Optional service binding used in place of a public fetch to the relay. */ + ACTIVITY_RELAY?: { fetch: typeof fetch }; + /** + * REQUIRED. Shared secret proving to the relay that a machine membership + * change came from this worker and not from a machine holding an account + * token. + */ + DIRECTORY_AUTH_SECRET?: string; +}; + +/** Options a caller (or a test) can inject around the relay hand-off. */ +export type ActivityRelayOptions = { + fetchImpl?: typeof fetch; + retryDelayMs?: number; +}; + +export type ActivityRelayOutcome = + | { ok: true } + | { ok: false; reason: string }; + +/** + * The relay is addressed by origin and the paths below are appended to it, so + * a configured value that carries one is trusted for its origin rather than + * refused — the one way this differs from the CORS allow-list. + */ +function trustedActivityRelayBaseUrl(env: ActivityRelayEnv): string | null { + return trustedHttpsOrigin(env.PUSH_RELAY_URL); +} + +/** + * Forward a machine membership change to the push relay. + * + * Two credentials travel together, and they answer different questions: + * + * - The caller's already-verified bearer token says WHICH ACCOUNT this is for. + * The relay re-verifies it and derives its own account id from it, so this + * worker can never act on an account it was not called for. + * - `x-ade-directory-auth` says THIS CAME FROM THE DIRECTORY. A removed machine + * keeps a valid account token by design (that is the whole premise of the + * revocation tables), so the token alone cannot authorize un-revoking a + * machine — otherwise the removed machine clears its own revocation and + * resumes publishing. Only the directory knows this secret. + * + * The secret is required for both operations rather than only the re-pair, so a + * half-configured deployment fails loudly on the first machine removal instead + * of silently leaving the security-critical route unauthenticated. It is sent + * over whichever transport is configured — the `ACTIVITY_RELAY` service binding + * or a public HTTPS fetch — because a service binding carries no attestable + * provenance marker the relay could check on its own. + * + * A failure here is never swallowed: it is logged, retried once, and returned + * so the caller can report it. + */ +export async function callActivityRelay( + request: Request, + env: ActivityRelayEnv, + args: { + operation: "purge" | "restore"; + machineKey: string; + correlationId: string; + options: ActivityRelayOptions; + }, +): Promise { + const baseUrl = trustedActivityRelayBaseUrl(env); + if (!baseUrl) return { ok: false, reason: "activity relay is not configured" }; + const directoryAuth = env.DIRECTORY_AUTH_SECRET?.trim(); + if (!directoryAuth) { + return { ok: false, reason: "directory relay authentication is not configured" }; + } + const authorization = request.headers.get("authorization"); + if (!authorization) return { ok: false, reason: "missing caller authorization" }; + const path = `/attention/account/machines/${encodeURIComponent(args.machineKey)}`; + const url = args.operation === "purge" ? `${baseUrl}${path}` : `${baseUrl}${path}/pairing`; + const fetchImpl = args.options.fetchImpl + ?? (env.ACTIVITY_RELAY ? env.ACTIVITY_RELAY.fetch.bind(env.ACTIVITY_RELAY) : fetch); + const retryDelayMs = Math.max(0, args.options.retryDelayMs ?? 250); + let reason = "activity relay is unreachable"; + for (let attempt = 1; attempt <= 2; attempt += 1) { + if (attempt > 1 && retryDelayMs > 0) { + await new Promise((resolve) => setTimeout(resolve, retryDelayMs)); + } + try { + const response = await fetchImpl(url, { + method: args.operation === "purge" ? "DELETE" : "POST", + headers: { + accept: "application/json", + authorization, + "x-ade-directory-auth": directoryAuth, + "x-ade-correlation-id": args.correlationId, + }, + redirect: "error", + }); + await response.body?.cancel().catch(() => {}); + if (response.ok) return { ok: true }; + reason = `activity relay returned ${response.status}`; + // A rejected token or a refused request will not heal on a retry. + if (response.status < 500 && response.status !== 429) break; + } catch (error) { + reason = error instanceof Error ? error.message : String(error); + } + } + logActivityRelayFailure({ + correlationId: args.correlationId, + operation: args.operation, + machineKey: args.machineKey, + reason, + attempts: 2, + }); + return { ok: false, reason }; +} diff --git a/apps/account-directory/src/callerToken.ts b/apps/account-directory/src/callerToken.ts new file mode 100644 index 000000000..69a13d287 --- /dev/null +++ b/apps/account-directory/src/callerToken.ts @@ -0,0 +1,217 @@ +import { createRemoteJWKSet, errors, jwtVerify, type JWTPayload } from "jose"; + +/** + * Clerk token verification, shared by every route that takes a caller bearer. + * + * It declares the slice of the Worker env it needs rather than importing the + * full `Env`, exactly as `deviceAuthorization.ts` does: this module has no + * business knowing about D1, the relay, or the diagnostics bucket, and keeping + * it that way is what stops an import cycle back into `directory.ts`. + */ +export interface CallerTokenEnv { + CLERK_JWKS_URL: string; + CLERK_ISSUER: string; + CLERK_OAUTH_CLIENT_ID: string; +} + +/** + * How recent the caller's INTERACTIVE authentication must be for a + * `pairing: true` registration to be allowed to clear a revocation. + * + * `pairing` arrives in the request body, so on its own it is an unauthenticated + * client boolean: a removed-but-still-signed-in machine can simply set it on + * its next 30 s heartbeat and the directory would then call the relay's + * `/pairing` route with its own `DIRECTORY_AUTH_SECRET` — a confused deputy + * clearing both halves of the removal. Un-revoking therefore has to be bound to + * a credential a removed machine cannot mint. + * + * Authentication TIME is that credential. A background heartbeat carries an old + * authentication even after its access token is refreshed (a refresh renews + * `exp`/`iat`, never the moment the human authenticated), while a user who + * genuinely signs in again carries a new one. Ten minutes is long enough to + * cover a sign-in followed by the auto-repair and any retry, and short enough + * that a token sitting on a removed machine never qualifies. + * + * This check FAILS CLOSED — a token with no such claim proves nothing — which + * is why it is not the only way back. See `PAIRING_GRANT_TTL_MS`: a grant this + * worker mints at the end of a `/device/*` sign-in proves the same fact for + * token shapes that carry neither claim, so a removal stays durable without + * becoming permanent. + */ +export const PAIRING_AUTH_FRESHNESS_MS = 10 * 60_000; + +const remoteJwksByUrl = new Map>(); + +export type CallerTokenFailureReason = + | "authentication unavailable" + | "invalid audience" + | "invalid issuer" + | "invalid token" + | "missing bearer token" + | "missing token subject" + | "token expired"; + +export class CallerTokenValidationError extends Error { + constructor(readonly reason: CallerTokenFailureReason) { + super(reason); + this.name = "CallerTokenValidationError"; + } +} + +/** + * Was the token refused because THIS WORKER is misconfigured? + * + * That is a 503 on every route that takes a bearer, never a 401: telling a user + * their token is invalid when the deployment simply has no JWKS URL sends them + * to sign in again forever. Exported so the routes outside this module classify + * it the same way rather than each matching on a message string. + */ +export function isAuthenticationUnavailableError(error: unknown): boolean { + return error instanceof CallerTokenValidationError + && error.reason === "authentication unavailable"; +} + +function getRemoteJwks(rawUrl: string): ReturnType { + const url = new URL(rawUrl); + const cacheKey = url.toString(); + const cached = remoteJwksByUrl.get(cacheKey); + if (cached) return cached; + const jwks = createRemoteJWKSet(url); + remoteJwksByUrl.set(cacheKey, jwks); + return jwks; +} + +function audienceIncludes(audience: JWTPayload["aud"], expected: string): boolean { + return typeof audience === "string" ? audience === expected : Array.isArray(audience) && audience.includes(expected); +} + +function isAllowedCallerToken(payload: JWTPayload, oauthClientId: string): boolean { + // Clerk's native session tokens have no audience. Their `azp` may be absent, + // empty, or origin-based, so `azp` alone must not reject that token shape. + if (payload.aud === undefined) return true; + + // OAuth access tokens are audience/authorized-party bound to the ADE client. + // Future fixed audiences (for example `ade-relay`) can be added to this list. + const allowedAudiences = [oauthClientId]; + return allowedAudiences.some((allowed) => + audienceIncludes(payload.aud, allowed) || payload.azp === allowed + ); +} + +export function readBearerToken(request: Request): string | null { + const authorization = request.headers.get("authorization") ?? ""; + const match = authorization.match(/^Bearer\s+(\S+)\s*$/i); + return match?.[1] ?? null; +} + +/** + * Milliseconds since the caller last passed an interactive authentication, or + * `null` when the verified token carries no authentication-time claim at all. + * + * Two shapes are understood, because ADE accepts both Clerk token shapes: + * + * - `auth_time` — the standard OIDC claim, seconds since the epoch. + * - `fva` — Clerk's equivalent ("factor verification age"), a two-element array + * of MINUTES since the first- and second-factor verifications. `-1` means the + * factor was never verified. Clerk caps the counter at 99, which is far above + * the freshness bound, so the cap never turns a stale token into a fresh one. + * + * Never derived from `iat`: a background token refresh mints a new `iat` while + * the human authentication behind it stays exactly as old as it was. + */ +function interactiveAuthenticationAgeMs(payload: JWTPayload, nowMs: number): number | null { + const authTime = (payload as { auth_time?: unknown }).auth_time; + if (typeof authTime === "number" && Number.isFinite(authTime) && authTime > 0) { + return Math.max(0, nowMs - authTime * 1_000); + } + const factorVerificationAge = (payload as { fva?: unknown }).fva; + if (Array.isArray(factorVerificationAge)) { + const firstFactorMinutes = factorVerificationAge[0]; + if ( + typeof firstFactorMinutes === "number" + && Number.isFinite(firstFactorMinutes) + && firstFactorMinutes >= 0 + ) { + return firstFactorMinutes * 60_000; + } + } + return null; +} + +/** + * Did this token prove an interactive authentication inside the freshness + * bound? Fails closed: a token with no authentication-time claim is treated as + * unproven, because the alternative is accepting the client's word for it. + */ +function hasFreshInteractiveAuthentication(payload: JWTPayload, nowMs: number): boolean { + const ageMs = interactiveAuthenticationAgeMs(payload, nowMs); + return ageMs !== null && ageMs <= PAIRING_AUTH_FRESHNESS_MS; +} + +async function verifyCallerTokenPayload(token: string, env: CallerTokenEnv): Promise { + const jwksUrl = typeof env.CLERK_JWKS_URL === "string" ? env.CLERK_JWKS_URL.trim() : ""; + const issuer = typeof env.CLERK_ISSUER === "string" ? env.CLERK_ISSUER.trim() : ""; + const oauthClientId = typeof env.CLERK_OAUTH_CLIENT_ID === "string" + ? env.CLERK_OAUTH_CLIENT_ID.trim() + : ""; + if (!jwksUrl || !issuer || !oauthClientId) { + throw new CallerTokenValidationError("authentication unavailable"); + } + + const { payload } = await jwtVerify(token, getRemoteJwks(jwksUrl), { + issuer, + algorithms: ["RS256"], + clockTolerance: 5, + }); + if (typeof payload.sub !== "string" || !payload.sub.trim()) { + throw new CallerTokenValidationError("missing token subject"); + } + if (!isAllowedCallerToken(payload, oauthClientId)) { + throw new CallerTokenValidationError("invalid audience"); + } + return payload; +} + +export async function verifyCallerToken(token: string, env: CallerTokenEnv): Promise { + const payload = await verifyCallerTokenPayload(token, env); + return payload.sub as string; +} + +function callerTokenFailureReason(error: unknown): CallerTokenFailureReason { + if (error instanceof CallerTokenValidationError) return error.reason; + if (error instanceof errors.JWTExpired) return "token expired"; + if (error instanceof errors.JWTClaimValidationFailed) { + if (error.claim === "exp") return "token expired"; + if (error.claim === "iss") return "invalid issuer"; + if (error.claim === "aud") return "invalid audience"; + } + return "invalid token"; +} + +export type CallerAuthenticationResult = + | { + ok: true; + userId: string; + /** Proven-recent interactive sign-in; the only thing `pairing` is honored on. */ + freshInteractiveAuthentication: boolean; + } + | { ok: false; reason: CallerTokenFailureReason }; + +export async function authenticate( + request: Request, + env: CallerTokenEnv, +): Promise { + const token = readBearerToken(request); + if (!token) return { ok: false, reason: "missing bearer token" }; + try { + const payload = await verifyCallerTokenPayload(token, env); + return { + ok: true, + userId: payload.sub as string, + freshInteractiveAuthentication: hasFreshInteractiveAuthentication(payload, Date.now()), + }; + } catch (error) { + // Return only a fixed classification, never JOSE details or token claims. + return { ok: false, reason: callerTokenFailureReason(error) }; + } +} diff --git a/apps/account-directory/src/deviceAuthorization.ts b/apps/account-directory/src/deviceAuthorization.ts index 48d1ab4c9..26dea0ba4 100644 --- a/apps/account-directory/src/deviceAuthorization.ts +++ b/apps/account-directory/src/deviceAuthorization.ts @@ -1,3 +1,5 @@ +import { isLoopbackHostname } from "./trustedOrigin"; + const DEVICE_CODE_TTL_SECONDS = 10 * 60; const DEVICE_POLL_INTERVAL_SECONDS = 5; const DEVICE_SLOW_DOWN_INCREMENT_SECONDS = 5; @@ -108,7 +110,7 @@ function positiveInteger(value: unknown): number | null { function normalizeIssuer(raw: string): string { const issuer = raw.trim().replace(/\/+$/, ""); const parsed = new URL(issuer); - if (parsed.protocol !== "https:" && !(parsed.protocol === "http:" && ["localhost", "127.0.0.1", "[::1]"].includes(parsed.hostname))) { + if (parsed.protocol !== "https:" && !(parsed.protocol === "http:" && isLoopbackHostname(parsed.hostname))) { throw new Error("CLERK_ISSUER must use https (http is only allowed for localhost)"); } return issuer; diff --git a/apps/account-directory/src/diagnostics.ts b/apps/account-directory/src/diagnostics.ts new file mode 100644 index 000000000..5744a642e --- /dev/null +++ b/apps/account-directory/src/diagnostics.ts @@ -0,0 +1,420 @@ +import { isAuthenticationUnavailableError, verifyCallerToken } from "./callerToken"; +import type { Env } from "./directory"; +import { logDiagnosticsUpload } from "./logging"; +import { isLoopbackHostname } from "./trustedOrigin"; + +/** + * `POST /diagnostics/upload` — the one-click destination for ADE's already + * redacted diagnostic report. + * + * The report is built and redacted entirely on the user's machine + * (`apps/ade-cli/src/services/diagnostics/diagnosticReport.ts`), so this route + * is a write-only sink: it stores bytes it never parses, never indexes and + * never echoes back. That property is what lets it accept anonymous uploads at + * all — the alternative is asking a user whose ADE will not start to sign in + * first, which is exactly the support round-trip this route exists to remove. + */ + +/** R2 is optional in the binding type so a Worker deployed before the bucket exists degrades instead of crashing. */ +export type DiagnosticsEnv = Env & { DIAGNOSTICS?: R2Bucket }; + +export const DIAGNOSTICS_UPLOAD_PATH = "/diagnostics/upload"; + +/** + * Hard cap on one upload. Real reports are tens of kilobytes (log tails are + * already truncated by the collector), so half a megabyte is far above the + * honest ceiling and far below anything worth storing by accident. + * + * Applied to the WHOLE request body, because the bytes have to be bounded as + * they arrive, before anything is parsed. Senders mirror this number in + * `apps/desktop/src/shared/diagnosticsUpload.ts` and weigh their serialized + * body against it for that reason — a report that fits with no room for the + * `{"report":...}` envelope does not fit here. + */ +export const MAX_DIAGNOSTIC_REPORT_BYTES = 512 * 1024; + +/** Uploads one identity may store per UTC day. */ +export const MAX_DIAGNOSTIC_UPLOADS_PER_DAY = 5; + +const MAX_METADATA_CHARS = 200; +/** + * Bound on the per-isolate counter map. An isolate that has seen more distinct + * uploaders than this is being probed, not used; dropping the whole map costs + * only the fast path, because the R2 listing below is the authority. + */ +const MAX_TRACKED_IDENTITIES = 5_000; + +type DailyUploadCount = { dayKey: string; count: number }; + +/** + * Per-isolate upload counters. + * + * TRADEOFF, deliberately taken: the device flow rate-limits through a D1 table + * (`device_approval_rate_limits`), which is durable and global but needs a + * migration. This route is additive and ships without one, so the limit is + * enforced in two weaker layers that together are good enough for an abuse + * bound on a write-only sink: + * + * 1. This map — free and exact, but scoped to one isolate and lost when + * Cloudflare recycles it. + * 2. An R2 `list()` on the identity's prefix for the day (below) — durable and + * global, and the layer that actually holds the line. It costs one class-A + * operation per upload and is not transactional, so a burst of genuinely + * simultaneous requests can land a few objects over the cap. For a quota + * whose purpose is "one person cannot fill the bucket", overshooting five + * by a couple is not a failure mode worth a migration. + * + * If diagnostics volume ever justifies exact global counting, move this to the + * D1 table pattern the device flow already uses. + */ +const isolateUploadCounts = new Map(); + +/** Same `(value, init)` shape as `directory.ts`'s `json`, so the name means one thing here. */ +function json(value: unknown, init: ResponseInit = {}): Response { + return new Response(JSON.stringify(value), { + ...init, + headers: { + "content-type": "application/json", + ...corsHeaders(), + ...(init.headers ?? {}), + }, + }); +} + +/** + * Open CORS, on purpose and only here. + * + * The desktop "Send to ADE" button runs in Electron's renderer, whose origin is + * `file://` in a packaged build (`Origin: null`) and `http://localhost:5173` in + * development, so no fixed allow-list can name it. `*` is safe for this route + * specifically: it accepts a body and returns an opaque id, it never reads + * account state, and `*` is incompatible with `credentials: "include"`, so no + * browser will ever attach ambient cookies to it. Every account route keeps its + * exact-origin rule in `directory.ts`. + */ +function corsHeaders(): Record { + return { + "access-control-allow-origin": "*", + vary: "Origin", + }; +} + +export function isDiagnosticsRequest(url: URL): boolean { + return url.pathname.replace(/\/+$/, "") === DIAGNOSTICS_UPLOAD_PATH; +} + +/** + * Cloudflare sets `cf-connecting-ip` and a client cannot influence it. Off + * Cloudflare — a local dev run, a proxy in front — there is no trustworthy + * address at all, so everyone shares one bucket rather than falling back to + * `x-forwarded-for`: a quota keyed on a header the caller writes is not a quota. + */ +function clientIdentity(request: Request): string { + return request.headers.get("cf-connecting-ip")?.trim() || "unknown-client"; +} + +/** + * A drive-by POST from some other site's page, which no real ADE client is. + * + * `sec-fetch-site` is attached by the browser and cannot be set by the page, so + * it is the one honest signal here. ADE's own senders are unaffected: the CLI + * and any non-browser fetch send no such header at all, and the desktop button + * runs in Electron's renderer, whose two origin shapes — `null` from `file://` + * in a packaged build and loopback in development — are exempted rather than + * named by hostname, because that renderer is cross-site to this Worker too. + */ +function isCrossSiteBrowserUpload(request: Request): boolean { + if (request.headers.get("sec-fetch-site")?.trim().toLowerCase() !== "cross-site") return false; + const origin = request.headers.get("origin")?.trim() ?? ""; + if (!origin || origin === "null") return false; + try { + return !isLoopbackHostname(new URL(origin).hostname); + } catch { + return true; + } +} + +async function sha256Hex(value: string): Promise { + const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(value)); + return Array.from(new Uint8Array(digest)) + .map((byte) => byte.toString(16).padStart(2, "0")) + .join(""); +} + +/** + * The key segment that both names the uploader and *is* the rate-limit bucket. + * + * Signed in, it is the Clerk user id, so support can find every report a user + * sent. Anonymous, it is a hash of the caller's IP rather than the install id + * the report carries: an install id is client-supplied and a spammer would + * simply mint a new one per request, whereas the address is the thing the quota + * is actually meant to bound. The install id still travels, as metadata. + */ +async function uploadIdentity(request: Request, userId: string | null): Promise { + if (userId) { + const safe = userId.replace(/[^A-Za-z0-9_-]/g, "").slice(0, 80); + if (safe) return `u-${safe}`; + } + return `anon-${(await sha256Hex(clientIdentity(request))).slice(0, 16)}`; +} + +function utcDayKey(nowMs: number): string { + return new Date(nowMs).toISOString().slice(0, 10); +} + +function boundedMetadata(value: unknown): string | undefined { + if (typeof value !== "string") return undefined; + const trimmed = value.trim(); + if (!trimmed) return undefined; + // R2 stores custom metadata as HTTP header values, so a control character + // would be rejected (or silently mangled) at write time. Replaced rather + // than escaped: this is a label a human reads off the object, not data. + return trimmed.replace(/[\u0000-\u001f\u007f]/g, " ").slice(0, MAX_METADATA_CHARS); +} + +/** + * Reads at most `MAX_DIAGNOSTIC_REPORT_BYTES + 1` bytes. + * + * `content-length` is checked first because it makes the common rejection free, + * but it is never trusted on its own: a chunked upload carries no length at + * all, so the stream is counted as it arrives and abandoned the moment it + * crosses the cap. Buffering whatever the client claimed to send would be the + * bug the cap exists to prevent. + */ +async function readBoundedBody( + request: Request, +): Promise<{ ok: true; text: string } | { ok: false; reason: "too_large" }> { + const declared = Number(request.headers.get("content-length") ?? ""); + if (Number.isFinite(declared) && declared > MAX_DIAGNOSTIC_REPORT_BYTES) { + return { ok: false, reason: "too_large" }; + } + const body = request.body; + if (!body) return { ok: true, text: "" }; + const reader = body.getReader(); + const chunks: Uint8Array[] = []; + let total = 0; + try { + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + if (!value) continue; + total += value.byteLength; + if (total > MAX_DIAGNOSTIC_REPORT_BYTES) { + await reader.cancel().catch(() => {}); + return { ok: false, reason: "too_large" }; + } + chunks.push(value); + } + } finally { + reader.releaseLock(); + } + const joined = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + joined.set(chunk, offset); + offset += chunk.byteLength; + } + return { ok: true, text: new TextDecoder().decode(joined) }; +} + +type ParsedUpload = { + report: string; + installId?: string; + appVersion?: string; +}; + +/** + * Two body shapes, because the two senders want different things: the CLI and + * the desktop button post JSON so they can name the install and app version, + * and `text/plain` stays supported so a report can be piped straight in with + * `curl --data-binary @report.md` when someone is debugging this route. + */ +function parseUpload(contentType: string, raw: string, url: URL): ParsedUpload | null { + const isJson = contentType.toLowerCase().includes("application/json"); + let fields: Record = {}; + let report: string | null = raw; + if (isJson) { + let parsed: unknown; + try { + parsed = JSON.parse(raw); + } catch { + return null; + } + if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) return null; + fields = parsed as Record; + report = typeof fields.report === "string" ? fields.report : null; + } + if (report === null || !report.trim()) return null; + return { + report, + installId: boundedMetadata(fields.installId ?? url.searchParams.get("installId")), + appVersion: boundedMetadata(fields.appVersion ?? url.searchParams.get("appVersion")), + }; +} + +async function withinDailyLimit( + bucket: R2Bucket, + identity: string, + prefix: string, + dayKey: string, +): Promise { + const remembered = isolateUploadCounts.get(identity); + const rememberedCount = remembered?.dayKey === dayKey ? remembered.count : 0; + if (rememberedCount >= MAX_DIAGNOSTIC_UPLOADS_PER_DAY) return false; + + // The durable half of the limit. `limit` stops one greedy prefix from + // listing an unbounded page just to answer a yes/no question. + const listed = await bucket.list({ prefix, limit: MAX_DIAGNOSTIC_UPLOADS_PER_DAY + 1 }); + const stored = listed.objects.length; + const count = Math.max(stored, rememberedCount); + if (isolateUploadCounts.size >= MAX_TRACKED_IDENTITIES) isolateUploadCounts.clear(); + isolateUploadCounts.set(identity, { + dayKey, + count: count >= MAX_DIAGNOSTIC_UPLOADS_PER_DAY ? count : count + 1, + }); + return count < MAX_DIAGNOSTIC_UPLOADS_PER_DAY; +} + +export type DiagnosticsRequestOptions = { + now?: () => number; + randomId?: () => string; +}; + +export async function handleDiagnosticsRequest( + request: Request, + env: DiagnosticsEnv, + options: DiagnosticsRequestOptions = {}, +): Promise { + if (request.method === "OPTIONS") { + return new Response(null, { + status: 204, + headers: { + ...corsHeaders(), + "access-control-allow-methods": "POST, OPTIONS", + "access-control-allow-headers": "authorization, content-type, x-ade-correlation-id", + "access-control-max-age": "600", + }, + }); + } + if (request.method !== "POST") return json({ error: "method not allowed" }, { status: 405 }); + if (isCrossSiteBrowserUpload(request)) { + return json({ error: "cross-site upload not allowed" }, { status: 403 }); + } + + const bucket = env.DIAGNOSTICS; + if (!bucket) { + // The bucket is created as a deploy step (see the README). Until it exists + // the button must fail politely rather than 500. + return json({ error: "diagnostics upload unavailable" }, { status: 503 }); + } + + // Authentication is OPTIONAL, but a token that was SENT and does not verify + // is an error rather than a downgrade: silently storing that report as + // anonymous would hide a broken sign-in from the very user reporting it. + // That applies to the header shape as much as to the signature — an + // `Authorization` this route cannot parse is a client that believes it is + // signed in, so it is refused rather than quietly demoted to anonymous. + let userId: string | null = null; + const authorization = request.headers.get("authorization")?.trim() ?? ""; + const bearer = authorization.match(/^Bearer\s+(\S+)\s*$/i)?.[1] ?? null; + if (authorization && !bearer) return json({ error: "invalid token" }, { status: 401 }); + if (bearer) { + try { + userId = await verifyCallerToken(bearer, env); + } catch (error) { + // A Worker with no JWKS URL configured is a deployment fault, not a bad + // token; the account routes answer 503 for it and so does this one. + return isAuthenticationUnavailableError(error) + ? json({ error: "authentication unavailable" }, { status: 503 }) + : json({ error: "invalid token" }, { status: 401 }); + } + } + + const body = await readBoundedBody(request); + const identity = await uploadIdentity(request, userId); + if (!body.ok) { + logDiagnosticsUpload({ + outcome: "rejected", + status: 413, + reason: "too_large", + identity, + authenticated: Boolean(userId), + bytes: MAX_DIAGNOSTIC_REPORT_BYTES, + }); + return json({ error: "report too large" }, { status: 413 }); + } + + const url = new URL(request.url); + const parsed = parseUpload(request.headers.get("content-type") ?? "", body.text, url); + if (!parsed) { + logDiagnosticsUpload({ + outcome: "rejected", + status: 400, + reason: "empty_report", + identity, + authenticated: Boolean(userId), + bytes: body.text.length, + }); + return json({ error: "missing report" }, { status: 400 }); + } + + const now = options.now?.() ?? Date.now(); + const dayKey = utcDayKey(now); + const prefix = `reports/${dayKey}/${identity}/`; + if (!(await withinDailyLimit(bucket, identity, prefix, dayKey))) { + logDiagnosticsUpload({ + outcome: "rejected", + status: 429, + reason: "rate_limited", + identity, + authenticated: Boolean(userId), + bytes: parsed.report.length, + }); + return json({ error: "rate limited" }, { + status: 429, + headers: { "retry-after": "86400" }, + }); + } + + const id = options.randomId?.() ?? crypto.randomUUID(); + try { + await bucket.put(`${prefix}${id}.md`, parsed.report, { + httpMetadata: { contentType: "text/markdown; charset=utf-8" }, + customMetadata: { + ...(userId ? { userId } : {}), + ...(parsed.installId ? { installId: parsed.installId } : {}), + ...(parsed.appVersion ? { appVersion: parsed.appVersion } : {}), + }, + }); + } catch { + // R2 refused the write. Left unhandled this is the one path that answers + // without a line, which defeats the point of the log: support could no + // longer tell "the report never arrived" from "it arrived and the store + // dropped it". `rejected` because the outcome vocabulary has exactly two + // values and nothing was stored; the reason carries that this one is ours, + // not the caller's. 502 rather than the 503 the missing-binding path uses, + // so a configured bucket having a bad minute stays distinguishable from a + // bucket that was never created. + logDiagnosticsUpload({ + outcome: "rejected", + status: 502, + reason: "storage_write_failed", + identity, + authenticated: Boolean(userId), + bytes: parsed.report.length, + }); + return json({ error: "diagnostics upload failed" }, { status: 502 }); + } + + logDiagnosticsUpload({ + outcome: "stored", + status: 200, + identity, + authenticated: Boolean(userId), + bytes: parsed.report.length, + }); + // Only the id goes back. The report is never echoed: a route that returned + // what it stored would be a way to read other people's uploads the moment an + // id leaked. + return json({ ok: true, id }); +} diff --git a/apps/account-directory/src/directory.ts b/apps/account-directory/src/directory.ts index a63729724..0803e1857 100644 --- a/apps/account-directory/src/directory.ts +++ b/apps/account-directory/src/directory.ts @@ -1,38 +1,27 @@ -import { createRemoteJWKSet, errors, jwtVerify, type JWTPayload } from "jose"; import { handleDeviceAuthorizationRequest, type DeviceAuthorizationRequestOptions, } from "./deviceAuthorization"; +import { authenticate, type CallerTokenEnv } from "./callerToken"; +import { + callActivityRelay, + type ActivityRelayEnv, + type ActivityRelayOptions, +} from "./activityRelay"; +import { + createPairingProofBroker, + mintPairingGrant, + type PairingProofBroker, +} from "./pairingGrants"; +import { logDirectoryLifecycle, logDirectoryRefusal } from "./logging"; +import { trustedHttpsOrigin } from "./trustedOrigin"; -export interface Env { +export type { ActivityRelayOptions } from "./activityRelay"; + +export type Env = CallerTokenEnv & ActivityRelayEnv & { DB: D1Database; - CLERK_JWKS_URL: string; - CLERK_ISSUER: string; - CLERK_OAUTH_CLIENT_ID: string; WEB_CLIENT_ORIGIN?: string; ONLINE_WINDOW_MS?: string; - /** - * Push relay origin. The relay is a different worker over a different D1, so - * machine membership changes have to be forwarded to it explicitly: it owns - * the Activity feed and the roster of machines allowed to publish into it. - */ - PUSH_RELAY_URL?: string; - /** Optional service binding used in place of a public fetch to the relay. */ - ACTIVITY_RELAY?: { fetch: typeof fetch }; - /** - * REQUIRED. Shared secret proving to the relay that a machine membership - * change came from this worker and not from a machine holding an account - * token. Set it as a wrangler secret on BOTH workers with the same value: - * `wrangler secret put DIRECTORY_AUTH_SECRET`. Unset fails closed — machine - * removal and re-pairing both report a relay failure rather than proceed. - */ - DIRECTORY_AUTH_SECRET?: string; -} - -/** Options a caller (or a test) can inject around the relay hand-off. */ -export type ActivityRelayOptions = { - fetchImpl?: typeof fetch; - retryDelayMs?: number; }; export type DirectoryRequestOptions = DeviceAuthorizationRequestOptions & { @@ -56,6 +45,18 @@ type MachineRow = { created_at: number | null; }; +/** + * Every column `machineRecord` reads, in one place. + * + * Four routes select the same machine row and all four must stay in step with + * `MachineRow`: a column added to one select and forgotten in another surfaces + * as a field that is present on the register response and missing from the + * list. + */ +const MACHINE_ROW_COLUMNS = `user_id, machine_key, device_id, name, custom_name, platform, device_type, + pubkey, reachable_endpoints, power, sleep_state, sleep_state_at, + last_seen_at, created_at`; + type ReachableEndpoint = { kind: "lan" | "tailnet" | "relay"; url?: string; @@ -92,6 +93,21 @@ type MachineSleepStateValue = "awake" | "asleep"; type RegisterInput = { machineKey: string; deviceId: string; + /** + * A per-account hash of an OS-level machine identifier, when the client can + * read one. Null on older clients, on hosts with no such identifier, and on + * any registration made without an account id to salt with. + * + * It exists because `deviceId` does NOT survive a `~/.ade` wipe — it lives in + * the same secrets directory as the machine key — so a reinstall produces two + * fresh identifiers and dedup has nothing to match. This one is derived from + * the hardware and the OS install, so it is the same after the wipe. + * + * Caller-supplied and therefore forgeable, exactly like `deviceId`: on a + * plain token it authorizes nothing. See `supersedePhantomDuplicates` for the + * proof bar that makes acting on it safe. + */ + hardwareId: string | null; name: string; platform: string; deviceType: string; @@ -150,64 +166,29 @@ type AccountRoute = export const DEFAULT_ONLINE_WINDOW_MS = 90_000; /** - * How recent the caller's INTERACTIVE authentication must be for a - * `pairing: true` registration to be allowed to clear a revocation. - * - * `pairing` arrives in the request body, so on its own it is an unauthenticated - * client boolean: a removed-but-still-signed-in machine can simply set it on - * its next 30 s heartbeat and the directory would then call the relay's - * `/pairing` route with its own `DIRECTORY_AUTH_SECRET` — a confused deputy - * clearing both halves of the removal. Un-revoking therefore has to be bound to - * a credential a removed machine cannot mint. - * - * Authentication TIME is that credential. A background heartbeat carries an old - * authentication even after its access token is refreshed (a refresh renews - * `exp`/`iat`, never the moment the human authenticated), while a user who - * genuinely signs in again carries a new one. Ten minutes is long enough to - * cover a sign-in followed by the auto-repair and any retry, and short enough - * that a token sitting on a removed machine never qualifies. - * - * This check FAILS CLOSED — a token with no such claim proves nothing — which - * is why it is not the only way back. See `PAIRING_GRANT_TTL_MS`: a grant this - * worker mints at the end of a `/device/*` sign-in proves the same fact for - * token shapes that carry neither claim, so a removal stays durable without - * becoming permanent. - */ -export const PAIRING_AUTH_FRESHNESS_MS = 10 * 60_000; -/** - * How long a minted pairing grant stays spendable. + * Most rows one register call may supersede. * - * Deliberately the same order as `PAIRING_AUTH_FRESHNESS_MS`: both answer "did - * a human just authenticate?", so a grant must not outlive the claim it stands - * in for. It covers the sign-in, the automatic re-pair that follows it, and one - * retry — nothing longer. + * A device with more phantom keys than this is either pathological or hostile, + * and either way there is no reason to let a single request delete an unbounded + * slice of the account's roster. The rest are cleaned up by the next proven + * re-pair, oldest first. */ -export const PAIRING_GRANT_TTL_MS = 10 * 60_000; +export const MAX_SUPERSEDED_MACHINES = 5; const MAX_PUBKEY_CHARS = 128; const MAX_MACHINE_KEY_CHARS = 128; +/** + * A hardware anchor is a sha256 hex digest (64 chars); the cap is slack, not a + * shape check. Validating the shape would buy nothing — the value is + * caller-supplied either way, and what makes it safe to act on is the pairing + * proof, not its formatting. + */ +const MAX_HARDWARE_ID_CHARS = 128; /** A grant is 32 random bytes in base64url (43 chars); the cap is slack, not a shape check. */ const MAX_PAIRING_GRANT_CHARS = 256; const MAX_CUSTOM_NAME_CHARS = 80; -const remoteJwksByUrl = new Map>(); const CORRELATION_ID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; -type CallerTokenFailureReason = - | "authentication unavailable" - | "invalid audience" - | "invalid issuer" - | "invalid token" - | "missing bearer token" - | "missing token subject" - | "token expired"; - -class CallerTokenValidationError extends Error { - constructor(readonly reason: CallerTokenFailureReason) { - super(reason); - this.name = "CallerTokenValidationError"; - } -} - function json(value: unknown, init: ResponseInit = {}): Response { return new Response(JSON.stringify(value), { ...init, @@ -334,6 +315,7 @@ function parseRegisterInput(value: unknown): RegisterInput | null { if (!isRecord(value)) return null; const machineKey = requiredString(value, "machineKey"); const deviceId = requiredString(value, "deviceId"); + const hardwareId = optionalString(value, "hardwareId"); const name = requiredString(value, "name"); const platform = requiredString(value, "platform"); const deviceType = requiredString(value, "deviceType"); @@ -349,6 +331,8 @@ function parseRegisterInput(value: unknown): RegisterInput | null { !machineKey || machineKey.length > MAX_MACHINE_KEY_CHARS || !deviceId + || hardwareId === undefined + || (hardwareId !== null && hardwareId.length > MAX_HARDWARE_ID_CHARS) || !name || !platform || !deviceType @@ -365,6 +349,7 @@ function parseRegisterInput(value: unknown): RegisterInput | null { return { machineKey, deviceId, + hardwareId, name, platform, deviceType, @@ -381,151 +366,6 @@ function parseRegisterInput(value: unknown): RegisterInput | null { }; } -function getRemoteJwks(rawUrl: string): ReturnType { - const url = new URL(rawUrl); - const cacheKey = url.toString(); - const cached = remoteJwksByUrl.get(cacheKey); - if (cached) return cached; - const jwks = createRemoteJWKSet(url); - remoteJwksByUrl.set(cacheKey, jwks); - return jwks; -} - -function audienceIncludes(audience: JWTPayload["aud"], expected: string): boolean { - return typeof audience === "string" ? audience === expected : Array.isArray(audience) && audience.includes(expected); -} - -function isAllowedCallerToken(payload: JWTPayload, oauthClientId: string): boolean { - // Clerk's native session tokens have no audience. Their `azp` may be absent, - // empty, or origin-based, so `azp` alone must not reject that token shape. - if (payload.aud === undefined) return true; - - // OAuth access tokens are audience/authorized-party bound to the ADE client. - // Future fixed audiences (for example `ade-relay`) can be added to this list. - const allowedAudiences = [oauthClientId]; - return allowedAudiences.some((allowed) => - audienceIncludes(payload.aud, allowed) || payload.azp === allowed - ); -} - -function readBearerToken(request: Request): string | null { - const authorization = request.headers.get("authorization") ?? ""; - const match = authorization.match(/^Bearer\s+(\S+)\s*$/i); - return match?.[1] ?? null; -} - -/** - * Milliseconds since the caller last passed an interactive authentication, or - * `null` when the verified token carries no authentication-time claim at all. - * - * Two shapes are understood, because ADE accepts both Clerk token shapes: - * - * - `auth_time` — the standard OIDC claim, seconds since the epoch. - * - `fva` — Clerk's equivalent ("factor verification age"), a two-element array - * of MINUTES since the first- and second-factor verifications. `-1` means the - * factor was never verified. Clerk caps the counter at 99, which is far above - * the freshness bound, so the cap never turns a stale token into a fresh one. - * - * Never derived from `iat`: a background token refresh mints a new `iat` while - * the human authentication behind it stays exactly as old as it was. - */ -function interactiveAuthenticationAgeMs(payload: JWTPayload, nowMs: number): number | null { - const authTime = (payload as { auth_time?: unknown }).auth_time; - if (typeof authTime === "number" && Number.isFinite(authTime) && authTime > 0) { - return Math.max(0, nowMs - authTime * 1_000); - } - const factorVerificationAge = (payload as { fva?: unknown }).fva; - if (Array.isArray(factorVerificationAge)) { - const firstFactorMinutes = factorVerificationAge[0]; - if ( - typeof firstFactorMinutes === "number" - && Number.isFinite(firstFactorMinutes) - && firstFactorMinutes >= 0 - ) { - return firstFactorMinutes * 60_000; - } - } - return null; -} - -/** - * Did this token prove an interactive authentication inside the freshness - * bound? Fails closed: a token with no authentication-time claim is treated as - * unproven, because the alternative is accepting the client's word for it. - */ -function hasFreshInteractiveAuthentication(payload: JWTPayload, nowMs: number): boolean { - const ageMs = interactiveAuthenticationAgeMs(payload, nowMs); - return ageMs !== null && ageMs <= PAIRING_AUTH_FRESHNESS_MS; -} - -async function verifyCallerTokenPayload(token: string, env: Env): Promise { - const jwksUrl = typeof env.CLERK_JWKS_URL === "string" ? env.CLERK_JWKS_URL.trim() : ""; - const issuer = typeof env.CLERK_ISSUER === "string" ? env.CLERK_ISSUER.trim() : ""; - const oauthClientId = typeof env.CLERK_OAUTH_CLIENT_ID === "string" - ? env.CLERK_OAUTH_CLIENT_ID.trim() - : ""; - if (!jwksUrl || !issuer || !oauthClientId) { - throw new CallerTokenValidationError("authentication unavailable"); - } - - const { payload } = await jwtVerify(token, getRemoteJwks(jwksUrl), { - issuer, - algorithms: ["RS256"], - clockTolerance: 5, - }); - if (typeof payload.sub !== "string" || !payload.sub.trim()) { - throw new CallerTokenValidationError("missing token subject"); - } - if (!isAllowedCallerToken(payload, oauthClientId)) { - throw new CallerTokenValidationError("invalid audience"); - } - return payload; -} - -export async function verifyCallerToken(token: string, env: Env): Promise { - const payload = await verifyCallerTokenPayload(token, env); - return payload.sub as string; -} - -function callerTokenFailureReason(error: unknown): CallerTokenFailureReason { - if (error instanceof CallerTokenValidationError) return error.reason; - if (error instanceof errors.JWTExpired) return "token expired"; - if (error instanceof errors.JWTClaimValidationFailed) { - if (error.claim === "exp") return "token expired"; - if (error.claim === "iss") return "invalid issuer"; - if (error.claim === "aud") return "invalid audience"; - } - return "invalid token"; -} - -type CallerAuthenticationResult = - | { - ok: true; - userId: string; - /** Proven-recent interactive sign-in; the only thing `pairing` is honored on. */ - freshInteractiveAuthentication: boolean; - } - | { ok: false; reason: CallerTokenFailureReason }; - -async function authenticate( - request: Request, - env: Env, -): Promise { - const token = readBearerToken(request); - if (!token) return { ok: false, reason: "missing bearer token" }; - try { - const payload = await verifyCallerTokenPayload(token, env); - return { - ok: true, - userId: payload.sub as string, - freshInteractiveAuthentication: hasFreshInteractiveAuthentication(payload, Date.now()), - }; - } catch (error) { - // Return only a fixed classification, never JOSE details or token claims. - return { ok: false, reason: callerTokenFailureReason(error) }; - } -} - function routeAccount(pathname: string): AccountRoute | null { const parts = pathname.split("/").filter(Boolean); if (parts[0] !== "account" || parts[1] !== "machines") return null; @@ -584,211 +424,57 @@ function onlineWindowMs(env: Env): number { : DEFAULT_ONLINE_WINDOW_MS; } -function trustedActivityRelayBaseUrl(env: Env): string | null { - const raw = env.PUSH_RELAY_URL?.trim(); - if (!raw) return null; - try { - const url = new URL(raw); - const loopback = url.hostname === "localhost" - || url.hostname === "127.0.0.1" - || url.hostname === "[::1]"; - if (url.protocol !== "https:" && !(url.protocol === "http:" && loopback)) return null; - if (url.username || url.password || url.search || url.hash) return null; - return url.origin; - } catch { - return null; - } -} - -type ActivityRelayOutcome = - | { ok: true } - | { ok: false; reason: string }; - -function logActivityRelayFailure(args: { - correlationId: string; - operation: "purge" | "restore"; - machineKey: string; - reason: string; - attempts: number; -}): void { - console.error(JSON.stringify({ - ts: new Date().toISOString(), - svc: "ade-account-directory", - kind: "activity_relay_failed", - correlationId: args.correlationId, - operation: args.operation, - // The machine key is a capability-shaped secret; log only a tail marker. - machine: args.machineKey.slice(-6), - attempts: args.attempts, - reason: args.reason.slice(0, 300), - })); -} +type RefusalLogger = ( + event: "directory.register_refused" | "directory.supersede_refused", + code: string, + reason?: string, +) => void; /** - * Forward a machine membership change to the push relay. + * Other machine rows this account already has for the SAME physical device. * - * Two credentials travel together, and they answer different questions: + * This is the phantom-duplicate query. A reinstall (or any client-side identity + * rotation) mints a fresh machine key, and rows are keyed `(user_id, + * machine_key)`, so the old row survives forever as a machine the user never + * owned twice. Ordered oldest-seen first so that when the cap bites, it is the + * stalest rows that go — and so the newest surviving `custom_name` is the last + * one the caller sees. * - * - The caller's already-verified bearer token says WHICH ACCOUNT this is for. - * The relay re-verifies it and derives its own account id from it, so this - * worker can never act on an account it was not called for. - * - `x-ade-directory-auth` says THIS CAME FROM THE DIRECTORY. A removed machine - * keeps a valid account token by design (that is the whole premise of the - * revocation tables), so the token alone cannot authorize un-revoking a - * machine — otherwise the removed machine clears its own revocation and - * resumes publishing. Only the directory knows this secret. + * TWO identifiers can name the same device, and the union is the point. The + * device id catches an in-place reinstall, where `~/.ade/secrets` survived. The + * hardware anchor catches the case that motivated it — a full `~/.ade` wipe, + * where the device id was minted fresh alongside the machine key and matches + * nothing. Either one alone leaves a phantom row behind in the other's case. * - * The secret is required for both operations rather than only the re-pair, so a - * half-configured deployment fails loudly on the first machine removal instead - * of silently leaving the security-critical route unauthenticated. It is sent - * over whichever transport is configured — the `ACTIVITY_RELAY` service binding - * or a public HTTPS fetch — because a service binding carries no attestable - * provenance marker the relay could check on its own. + * Null-anchor rows can only ever be matched by device id. That is expected: a + * row written before this shipped, or by a host that cannot read an identifier, + * is folded in the first time the same physical machine re-registers with fresh + * auth AND a surviving device id, and otherwise ages out by hand from the + * machine list. Nothing here back-fills an anchor onto a row it did not send. * - * A failure here is never swallowed: it is logged, retried once, and returned - * so the caller can report it. + * One statement rather than two, so `order by`/`limit` apply to the UNION: two + * capped queries merged in the worker could return six rows, or drop the oldest + * of one set in favour of a newer row from the other. */ -async function callActivityRelay( - request: Request, +async function duplicateMachinesForDevice( env: Env, - args: { - operation: "purge" | "restore"; - machineKey: string; - correlationId: string; - options: ActivityRelayOptions; - }, -): Promise { - const baseUrl = trustedActivityRelayBaseUrl(env); - if (!baseUrl) return { ok: false, reason: "activity relay is not configured" }; - const directoryAuth = env.DIRECTORY_AUTH_SECRET?.trim(); - if (!directoryAuth) { - return { ok: false, reason: "directory relay authentication is not configured" }; - } - const authorization = request.headers.get("authorization"); - if (!authorization) return { ok: false, reason: "missing caller authorization" }; - const path = `/attention/account/machines/${encodeURIComponent(args.machineKey)}`; - const url = args.operation === "purge" ? `${baseUrl}${path}` : `${baseUrl}${path}/pairing`; - const fetchImpl = args.options.fetchImpl - ?? (env.ACTIVITY_RELAY ? env.ACTIVITY_RELAY.fetch.bind(env.ACTIVITY_RELAY) : fetch); - const retryDelayMs = Math.max(0, args.options.retryDelayMs ?? 250); - let reason = "activity relay is unreachable"; - for (let attempt = 1; attempt <= 2; attempt += 1) { - if (attempt > 1 && retryDelayMs > 0) { - await new Promise((resolve) => setTimeout(resolve, retryDelayMs)); - } - try { - const response = await fetchImpl(url, { - method: args.operation === "purge" ? "DELETE" : "POST", - headers: { - accept: "application/json", - authorization, - "x-ade-directory-auth": directoryAuth, - "x-ade-correlation-id": args.correlationId, - }, - redirect: "error", - }); - await response.body?.cancel().catch(() => {}); - if (response.ok) return { ok: true }; - reason = `activity relay returned ${response.status}`; - // A rejected token or a refused request will not heal on a retry. - if (response.status < 500 && response.status !== 429) break; - } catch (error) { - reason = error instanceof Error ? error.message : String(error); - } - } - logActivityRelayFailure({ - correlationId: args.correlationId, - operation: args.operation, - machineKey: args.machineKey, - reason, - attempts: 2, - }); - return { ok: false, reason }; -} - -function bytesToBase64Url(bytes: Uint8Array): string { - let binary = ""; - for (const byte of bytes) binary += String.fromCharCode(byte); - return btoa(binary).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/g, ""); -} - -async function sha256Base64Url(value: string): Promise { - const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(value)); - return bytesToBase64Url(new Uint8Array(digest)); -} - -/** - * Mint the pairing grant for a just-completed `/device/*` sign-in. - * - * `accessToken` is the token this worker itself fetched from Clerk moments ago, - * but it is re-verified rather than decoded: the user id the grant is bound to - * decides whose revocation it can lift, and that must come from a signature - * check, not from a base64 payload. Verification failure yields `null` — the - * sign-in still succeeds, only the second proof path is unavailable. - * - * Only the hash is stored. The plaintext exists in one response body and in the - * signing-in machine's memory; a dump of this D1 yields nothing spendable. - */ -async function mintPairingGrant( - env: Env, - args: { accessToken: string; machineKey: string; nowMs: number }, -): Promise { - let userId: string; - try { - userId = await verifyCallerToken(args.accessToken, env); - } catch { - return null; - } - const grant = bytesToBase64Url(crypto.getRandomValues(new Uint8Array(32))); - await env.DB.prepare(` - insert into machine_pairing_grants (grant_hash, user_id, machine_key, created_at, expires_at) - values (?, ?, ?, ?, ?) - on conflict(grant_hash) do nothing - `).bind( - await sha256Base64Url(grant), - userId, - args.machineKey, - args.nowMs, - args.nowMs + PAIRING_GRANT_TTL_MS, - ).run(); - return grant; -} - -/** - * Spend a pairing grant, or refuse it. - * - * One statement carries every rule the grant exists to enforce — it must belong - * to this caller, name this machine, still be inside its TTL, and never have - * been spent — because splitting them into a read and a later write would let - * two concurrent registrations both observe an unspent row. `changes === 1` is - * therefore the whole proof: the delete both consumes and validates. - */ -async function redeemPairingGrant( - env: Env, - args: { userId: string; machineKey: string; grant: string; nowMs: number }, -): Promise { - const result = await env.DB.prepare(` - delete from machine_pairing_grants - where grant_hash = ? and user_id = ? and machine_key = ? and expires_at > ? + args: { userId: string; deviceId: string | null; hardwareId: string | null; machineKey: string }, +): Promise> { + return (await env.DB.prepare(` + select machine_key, custom_name + from machines + where user_id = ? + and machine_key <> ? + and (device_id = ? or hardware_id = ?) + order by last_seen_at asc + limit ? `).bind( - await sha256Base64Url(args.grant), args.userId, args.machineKey, - args.nowMs, - ).run(); - return (result.meta.changes ?? 0) === 1; -} - -/** Cron sweep: an unspent grant is dead weight the moment it expires. */ -export async function cleanupExpiredPairingGrants( - env: Pick, - nowMs = Date.now(), -): Promise { - const result = await env.DB - .prepare("delete from machine_pairing_grants where expires_at <= ?") - .bind(nowMs) - .run(); - return result.meta.changes ?? 0; + args.deviceId, + args.hardwareId, + MAX_SUPERSEDED_MACHINES, + ).all<{ machine_key: string; custom_name: string | null }>()).results ?? []; } async function machineRevocation( @@ -803,106 +489,125 @@ async function machineRevocation( `).bind(userId, machineKey).first(); } -async function handleRegister( +/** + * Step one of a register call: may this machine be on this account at all? + * + * Removal is only durable if the removed machine cannot re-register itself. Its + * heartbeat carries a valid account token for as long as it stays signed in, so + * the revocation — not the token — is what decides. Returns the response to + * send when the answer is no, and `null` when registration may proceed. + */ +async function enforceRevocationGate( request: Request, env: Env, - userId: string, - correlationId: string, - relayOptions: ActivityRelayOptions, - freshInteractiveAuthentication: boolean, -): Promise { - if (request.method !== "POST") return text("method not allowed", 405); - let raw: unknown; - try { - raw = await request.json(); - } catch { - return json({ error: "invalid request body" }, { status: 400 }); - } - const input = parseRegisterInput(raw); - if (!input) return json({ error: "invalid request body" }, { status: 400 }); - - // Removal is only durable if the removed machine cannot re-register itself. - // Its heartbeat carries a valid account token for as long as it stays signed - // in, so the revocation — not the token — is what decides. + args: { + userId: string; + input: RegisterInput; + correlationId: string; + relayOptions: ActivityRelayOptions; + pairingProof: PairingProofBroker; + refuse: RefusalLogger; + }, +): Promise { + const { userId, input, refuse } = args; const revocation = await machineRevocation(env, userId, input.machineKey); - if (revocation) { - // `pairing` is honored ONLY together with proof of a freshly completed - // interactive sign-in (either proof below). Everything else in this request - // — including `deviceId`, which used to carry a "reinstalled machine" - // recovery clause — is supplied by the caller, so it can be forged by - // exactly the removed machine this gate exists to stop. A genuine reinstall - // recovers the same way every other re-pair does: the user signs in and ADE - // re-pairs immediately after. - if (!input.pairing) { - return json({ - error: "machine removed from account", - code: "machine_revoked", - revokedAt: revocation.revoked_at, - }, { status: 403 }); - } - // Two independent proofs, either of which is sufficient, neither of which a - // removed machine can produce from the token it is still holding. - // - // The claim is the fast path and is checked first so a genuinely fresh - // sign-in never spends a grant it does not need. The grant is the fallback, - // and it exists because the claim path fails CLOSED: ADE's brain - // authenticates with a Clerk OAuth access token, and `auth_time`/`fva` are - // not in the documented default claim set for that token shape. If they are - // absent in production, claim-only freshness would make every removal - // permanent — the original Blocker wearing a different hat. - if ( - !freshInteractiveAuthentication - && !(input.pairingGrant && await redeemPairingGrant(env, { - userId, - machineKey: input.machineKey, - grant: input.pairingGrant, - nowMs: Date.now(), - })) - ) { - return json({ - error: "Sign in again on this computer to reconnect it to your ADE account", - code: "pairing_authentication_required", - revokedAt: revocation.revoked_at, - }, { status: 403 }); - } - // A grant is spent above, before the relay is called, and it is NOT put - // back if the relay hand-off then fails. That ordering is deliberate: - // restoring it would mean either a non-atomic read-then-write (two - // concurrent registrations could each see it unspent) or re-issuing it with - // a fresh expiry (an attacker who can force relay failures could keep one - // alive indefinitely). A relay outage costs the user another sign-in — - // exactly what the refusal already tells them to do — and correctness of - // single-use wins over convenience on a rare failure path. - // - // Clear the relay's revocation first: a machine back on the roster but - // unable to publish is a worse state than one that retries the re-pair. - const restored = await callActivityRelay(request, env, { - operation: "restore", - machineKey: input.machineKey, - correlationId, - options: relayOptions, - }); - if (!restored.ok) { - return json({ - error: "activity relay unavailable", - code: "activity_relay_unavailable", - detail: restored.reason, - }, { status: 503 }); - } - await env.DB - .prepare("delete from revoked_machines where user_id = ? and machine_key = ?") - .bind(userId, input.machineKey) - .run(); + if (!revocation) return null; + + // `pairing` is honored ONLY together with proof of a freshly completed + // interactive sign-in (either proof below). Everything else in this request + // — including `deviceId`, which used to carry a "reinstalled machine" + // recovery clause — is supplied by the caller, so it can be forged by + // exactly the removed machine this gate exists to stop. A genuine reinstall + // recovers the same way every other re-pair does: the user signs in and ADE + // re-pairs immediately after. + if (!input.pairing) { + refuse("directory.register_refused", "machine_revoked"); + return json({ + error: "machine removed from account", + code: "machine_revoked", + revokedAt: revocation.revoked_at, + }, { status: 403 }); + } + // Two independent proofs, either of which is sufficient, neither of which a + // removed machine can produce from the token it is still holding. + // + // The claim is the fast path and is checked first so a genuinely fresh + // sign-in never spends a grant it does not need. The grant is the fallback, + // and it exists because the claim path fails CLOSED: ADE's brain + // authenticates with a Clerk OAuth access token, and `auth_time`/`fva` are + // not in the documented default claim set for that token shape. If they are + // absent in production, claim-only freshness would make every removal + // permanent — the original Blocker wearing a different hat. + const proof = await args.pairingProof.prove(); + if (proof.kind === "none") { + refuse( + "directory.register_refused", + "pairing_authentication_required", + // Support's first question is always which of the two it was: a machine + // that never presented a grant is a client-side problem, a rejected one + // is expired, replayed, or minted for another machine. + input.pairingGrant ? "grant_rejected" : "no_proof", + ); + return json({ + error: "Sign in again on this computer to reconnect it to your ADE account", + code: "pairing_authentication_required", + revokedAt: revocation.revoked_at, + }, { status: 403 }); } + // A grant is RESERVED above, not destroyed, and the reservation is what makes + // the next few lines recoverable. Reserving is one atomic statement, so + // single-use is enforced exactly as strictly as the old delete enforced it — + // two concurrent registrations still cannot both hold it — but a relay outage + // no longer burns the user's only credential. On failure it goes back with its + // ORIGINAL expiry, so forcing relay failures buys nothing: the grant still + // dies at the moment it was always going to die. + // + // Clear the relay's revocation first: a machine back on the roster but unable + // to publish is a worse state than one that retries the re-pair. + const restored = await callActivityRelay(request, env, { + operation: "restore", + machineKey: input.machineKey, + correlationId: args.correlationId, + options: args.relayOptions, + }); + if (!restored.ok) { + // Nothing is proven any more: the credential is back in circulation. + await args.pairingProof.release(); + refuse("directory.register_refused", "activity_relay_unavailable", restored.reason); + return json({ + error: "activity relay unavailable", + code: "activity_relay_unavailable", + detail: restored.reason, + }, { status: 503 }); + } + // Still proof for anything later in this request, never spendable again. + await args.pairingProof.consume(); + await env.DB + .prepare("delete from revoked_machines where user_id = ? and machine_key = ?") + .bind(userId, input.machineKey) + .run(); + return null; +} - const now = Date.now(); +/** Step two: write the row. The only unconditional write a heartbeat makes. */ +async function upsertMachine( + env: Env, + args: { userId: string; input: RegisterInput; nowMs: number }, +): Promise { + const { input } = args; await env.DB.prepare(` insert into machines ( user_id, machine_key, device_id, name, platform, device_type, pubkey, - reachable_endpoints, power, sleep_state, sleep_state_at, last_seen_at, created_at - ) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + reachable_endpoints, power, sleep_state, sleep_state_at, + last_seen_at, created_at, hardware_id + ) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) on conflict(user_id, machine_key) do update set device_id = excluded.device_id, + -- coalesce, and not excluded.hardware_id: an anchor is optional on every + -- request, so a single heartbeat from a host that momentarily could not + -- read one -- a sandbox, a slow probe, a downgrade -- must not erase the + -- anchor this row already learned. A new non-null value still wins. + hardware_id = coalesce(excluded.hardware_id, machines.hardware_id), name = excluded.name, platform = excluded.platform, device_type = excluded.device_type, @@ -941,7 +646,7 @@ async function handleRegister( end, last_seen_at = excluded.last_seen_at `).bind( - userId, + args.userId, input.machineKey, input.deviceId, input.name, @@ -952,19 +657,200 @@ async function handleRegister( input.power ? JSON.stringify(input.power) : null, input.sleepState, input.sleepStateAt, - now, - now, + args.nowMs, + args.nowMs, + input.hardwareId, input.retainRelayEndpoints ? 1 : 0, ).run(); +} + +/** + * Step three: phantom duplicates — the OTHER half of the reinstall story. + * + * Machines are keyed `(user_id, machine_key)`, so a client that rotates its + * identity file — a reinstall, a wiped config, a restored backup — lands as a + * second row for the same physical computer. The user sees a duplicate, + * removes the one that looks stale, and if they guess wrong they have just + * revoked the live install. Nothing about that is recoverable by the user, so + * the directory folds the old rows into the new one instead of letting them + * accumulate. + * + * FOLDS, not just deletes: the one thing a superseded row holds that the new + * one cannot rebuild is the name the user typed. Losing "Studio Mac" because a + * reinstall rotated a key is a small betrayal of a deliberate act, so the most + * recently seen superseded name is carried onto the survivor — and only when + * the survivor has none of its own, because a name set on the new row is the + * fresher statement of intent. + * + * The gate is the SAME proof that lifts a revocation, for the same reason: + * `deviceId` is caller-supplied and forgeable, so on a plain token it + * authorizes nothing — otherwise any machine could claim another's device id + * and delete that machine's row. With a proven-fresh human behind the call it + * is exactly the signal we want. `hardwareId` is caller-supplied in exactly + * the same way and gets exactly the same treatment: it is a better identifier + * (it survives the `~/.ade` wipe that invalidates the device id), not a more + * trustworthy one, and nothing about it is attested. The fresh-auth bar is + * the whole of what makes acting on either of them safe. + * + * Superseded keys get NO revocation row. The physical device holds the new + * key; blocking the old one would trapdoor any client that rolls its identity + * file back (a restored snapshot, a failed migration) into a permanent + * refusal. An absent key simply registers again. For the same reason the + * relay is not called: the device did not leave the account, so its Activity + * is still the user's own. + */ +async function supersedePhantomDuplicates( + env: Env, + args: { + userId: string; + input: RegisterInput; + pairingProof: PairingProofBroker; + refuse: RefusalLogger; + }, +): Promise { + const { userId, input } = args; + // Empty identifiers are normalized to null before they reach the query, and + // that is load-bearing: `device_id = ''` would match every row a future + // relaxation of the parser let through with an unset device id, while + // `device_id = null` matches nothing at all. `hardwareId` is already null + // whenever the client sent none. + const matchDeviceId = input.deviceId || null; + const matchHardwareId = input.hardwareId || null; + if (!matchDeviceId && !matchHardwareId) return []; + + const duplicates = await duplicateMachinesForDevice(env, { + userId, + deviceId: matchDeviceId, + hardwareId: matchHardwareId, + machineKey: input.machineKey, + }); + if (duplicates.length === 0) return []; + + const proof = await args.pairingProof.prove(); + if (proof.kind === "none") { + // Not a refused REGISTRATION — the machine is registered, the duplicate + // simply stays. Logged because "why is my Mac listed twice" is the support + // question this whole path exists to answer. + args.refuse( + "directory.supersede_refused", + "supersede_authentication_required", + `duplicates=${duplicates.length}`, + ); + return []; + } + // Spend before deleting, not after: a consume that failed once the rows were + // already gone would leave the grant reservable again in a minute, and a + // credential that can be spent twice is the worse of the two failures. Both + // are D1 writes on one database, so the window is a hypothetical either way. + await args.pairingProof.consume(); + + // Rows arrive oldest-seen first, so the last non-null name is the one the + // user set most recently. + const carriedName = duplicates.reduce( + (carried, row) => row.custom_name ?? carried, + null, + ); + // ONE batch, not a loop of independent writes. The grant is already spent by + // the time these run, so a D1 failure partway through a sequential loop would + // leave half the phantoms deleted with no credential left to finish the job. + await env.DB.batch([ + // `custom_name is null` is the whole carry-forward rule, in the statement + // rather than in a read-then-write: a name the user set on the surviving + // row must win over an inherited one, and nothing may clobber it. + ...(carriedName + ? [ + env.DB.prepare(` + update machines + set custom_name = ? + where user_id = ? and machine_key = ? and custom_name is null + `).bind(carriedName, userId, input.machineKey), + ] + : []), + // The identifiers stay in the predicate, and it mirrors the select exactly: + // the row was chosen a moment ago because it matched one of them, and that + // match is the only thing that authorized deleting it. A delete narrower + // than the select (device id alone) would silently leave every + // anchor-matched row in place. + ...duplicates.map((row) => + env.DB.prepare(` + delete from machines + where user_id = ? and machine_key = ? and (device_id = ? or hardware_id = ?) + `).bind(userId, row.machine_key, matchDeviceId, matchHardwareId) + ), + ]); + return duplicates.map((row) => row.machine_key); +} + +async function handleRegister( + request: Request, + env: Env, + userId: string, + correlationId: string, + relayOptions: ActivityRelayOptions, + freshInteractiveAuthentication: boolean, +): Promise { + if (request.method !== "POST") return text("method not allowed", 405); + let raw: unknown; + try { + raw = await request.json(); + } catch { + return json({ error: "invalid request body" }, { status: 400 }); + } + const input = parseRegisterInput(raw); + if (!input) return json({ error: "invalid request body" }, { status: 400 }); + + const nowMs = Date.now(); + const refuse: RefusalLogger = (event, code, reason) => logDirectoryRefusal({ + event, + userId, + machineKey: input.machineKey, + deviceId: input.deviceId, + code, + correlationId, + reason, + }); + // One owner for the two privileged operations below, so the grant behind them + // is reserved at most once and spent at most once across both. + const pairingProof = createPairingProofBroker(env, { + userId, + machineKey: input.machineKey, + pairing: input.pairing, + pairingGrant: input.pairingGrant, + freshInteractiveAuthentication, + nowMs, + }); + + const refusal = await enforceRevocationGate(request, env, { + userId, + input, + correlationId, + relayOptions, + pairingProof, + refuse, + }); + if (refusal) return refusal; + + await upsertMachine(env, { userId, input, nowMs }); + + const supersededMachineKeys = await supersedePhantomDuplicates(env, { + userId, + input, + pairingProof, + refuse, + }); const row = await env.DB.prepare(` - select user_id, machine_key, device_id, name, custom_name, platform, device_type, pubkey, - reachable_endpoints, power, sleep_state, sleep_state_at, last_seen_at, created_at + select ${MACHINE_ROW_COLUMNS} from machines where user_id = ? and machine_key = ? `).bind(userId, input.machineKey).first(); if (!row) return json({ error: "machine was not stored" }, { status: 500 }); - return json(machineRecord(row)); + // `supersededMachineKeys` is additive and omitted when empty, so a heartbeat + // response is byte-identical to what every deployed client already parses. + return json({ + ...machineRecord(row), + ...(supersededMachineKeys.length > 0 ? { supersededMachineKeys } : {}), + }); } async function handleList( @@ -976,8 +862,7 @@ async function handleList( if (request.method !== "GET") return text("method not allowed", 405); const dbStartedAt = performance.now(); const rows = (await env.DB.prepare(` - select user_id, machine_key, device_id, name, custom_name, platform, device_type, pubkey, - reachable_endpoints, power, sleep_state, sleep_state_at, last_seen_at, created_at + select ${MACHINE_ROW_COLUMNS} from machines where user_id = ? -- 500 is the machine-directory cap, matching the client's effective cap. @@ -1014,8 +899,7 @@ async function handleDelete( ): Promise { if (request.method !== "DELETE") return text("method not allowed", 405); const existing = await env.DB.prepare(` - select user_id, machine_key, device_id, name, custom_name, platform, device_type, pubkey, - reachable_endpoints, power, sleep_state, sleep_state_at, last_seen_at, created_at + select ${MACHINE_ROW_COLUMNS} from machines where user_id = ? and machine_key = ? `).bind(userId, machineKey).first(); @@ -1051,6 +935,15 @@ async function handleDelete( // The machine is off the roster and blocked, but its Activity is still // there. Say so instead of reporting a clean removal the user can see is // untrue the next time they open Activity. + logDirectoryRefusal({ + event: "directory.remove_refused", + userId, + machineKey, + deviceId: existing?.device_id ?? null, + code: "activity_purge_failed", + correlationId, + reason: purged.reason, + }); return json({ ok: false, error: "activity purge failed", @@ -1095,8 +988,7 @@ async function handleRename( } const row = await env.DB.prepare(` - select user_id, machine_key, device_id, name, custom_name, platform, device_type, pubkey, - reachable_endpoints, power, sleep_state, sleep_state_at, last_seen_at, created_at + select ${MACHINE_ROW_COLUMNS} from machines where user_id = ? and machine_key = ? `).bind(userId, machineKey).first(); @@ -1108,18 +1000,13 @@ async function handleRename( }); } +/** + * This value is compared against an `Origin` header, which is always bare, so + * anything more than an origin means the setting is not what its author thought + * it was and is refused rather than truncated. + */ function trustedWebClientOrigin(env: Env): string | null { - const raw = env.WEB_CLIENT_ORIGIN?.trim(); - if (!raw) return null; - try { - const url = new URL(raw); - const loopback = url.hostname === "localhost" || url.hostname === "127.0.0.1" || url.hostname === "[::1]"; - if (url.protocol !== "https:" && !(url.protocol === "http:" && loopback)) return null; - if (url.origin !== raw || url.username || url.password || url.search || url.hash) return null; - return url.origin; - } catch { - return null; - } + return trustedHttpsOrigin(env.WEB_CLIENT_ORIGIN, { requireExactOrigin: true }); } function withCors(response: Response, origin: string): Response { @@ -1150,31 +1037,6 @@ function withCorrelationId(response: Response, correlationId: string): Response }); } -function logDirectoryLifecycle(args: { - correlationId: string; - route: AccountRoute | null; - method: string; - status: number; - durationMs: number; -}): void { - const outcome = args.status < 400 - ? "ok" - : args.status < 500 - ? "client_error" - : "server_error"; - console.log(JSON.stringify({ - ts: new Date().toISOString(), - svc: "ade-account-directory", - kind: "request_completed", - correlationId: args.correlationId, - route: args.route?.kind ?? "other", - method: args.method, - status: args.status, - outcome, - durationMs: Math.max(0, Math.round(args.durationMs)), - })); -} - async function handleRequestCore( request: Request, env: Env, @@ -1188,9 +1050,9 @@ async function handleRequestCore( const deviceResponse = await handleDeviceAuthorizationRequest(request, env, { ...options, - // The device flow owns the sign-in; the grant it hands back is this + // The device flow owns the sign-in; the grant it hands back is the pairing // module's concern, so the minting (and the Clerk verification it needs) - // stays here rather than being duplicated into the device module. + // stays out of the device module. mintPairingGrant: options.mintPairingGrant ?? ((args) => mintPairingGrant(env, { ...args, nowMs: (options.now ?? Date.now)(), @@ -1260,7 +1122,7 @@ export async function handleRequest( const correlatedResponse = withCorrelationId(response, correlationId); logDirectoryLifecycle({ correlationId, - route, + route: route?.kind ?? null, method: request.method, status: correlatedResponse.status, durationMs: performance.now() - startedAt, diff --git a/apps/account-directory/src/index.ts b/apps/account-directory/src/index.ts index 45d03f9ed..7a0d57372 100644 --- a/apps/account-directory/src/index.ts +++ b/apps/account-directory/src/index.ts @@ -1,8 +1,17 @@ -import { cleanupExpiredPairingGrants, handleRequest, type Env } from "./directory"; +import { handleRequest, type Env } from "./directory"; +import { cleanupExpiredPairingGrants } from "./pairingGrants"; import { cleanupExpiredDeviceAuthorizations } from "./deviceAuthorization"; +import { handleDiagnosticsRequest, isDiagnosticsRequest, type DiagnosticsEnv } from "./diagnostics"; export default { - fetch(request: Request, env: Env): Promise { + fetch(request: Request, env: DiagnosticsEnv): Promise { + // Diagnostics is matched before the directory because it is the one route + // here that is not account-scoped: `handleRequest` answers an unknown + // OPTIONS with 404 and applies the directory's exact-origin CORS rule, + // neither of which fits a write-only sink an unauthenticated Electron + // renderer has to be able to reach. + const url = new URL(request.url); + if (isDiagnosticsRequest(url)) return handleDiagnosticsRequest(request, env); return handleRequest(request, env); }, diff --git a/apps/account-directory/src/logging.ts b/apps/account-directory/src/logging.ts new file mode 100644 index 000000000..acc6f67f2 --- /dev/null +++ b/apps/account-directory/src/logging.ts @@ -0,0 +1,124 @@ +/** + * Every structured line this Worker emits, in one place. + * + * They share a service name and a correlation id on purpose: support joins them + * to the request the client already logged, and to each other. Nothing here + * ever carries a full machine key, a token, or a pairing grant. + */ + +const SERVICE = "ade-account-directory"; + +export function logActivityRelayFailure(args: { + correlationId: string; + operation: "purge" | "restore"; + machineKey: string; + reason: string; + attempts: number; +}): void { + console.error(JSON.stringify({ + ts: new Date().toISOString(), + svc: SERVICE, + kind: "activity_relay_failed", + correlationId: args.correlationId, + operation: args.operation, + // The machine key is a capability-shaped secret; log only a tail marker. + machine: args.machineKey.slice(-6), + attempts: args.attempts, + reason: args.reason.slice(0, 300), + })); +} + +/** + * One line per refused machine-membership change. + * + * Every refusal on this worker is a user who cannot get their computer back + * onto their account, and the request that produced it is long gone by the time + * they ask for help. Support has no other window into that: the client reports + * only the code, and the D1 tables record what the state IS, never why a call + * was turned away. So each refusal path emits exactly one line, and the fields + * are chosen to be joinable — `correlationId` ties it to the request the client + * logged, `userId` to the account, the prefixes to the specific install. + * + * PREFIXES ONLY. A machine key is a capability-shaped secret and a pairing + * grant is a live credential; eight characters identify a row for a human + * reading logs and are useless to anyone who reads them. Nothing here ever + * carries a full key, a token, or a grant in any form. + */ +export function logDirectoryRefusal(args: { + event: "directory.register_refused" | "directory.remove_refused" | "directory.supersede_refused"; + userId: string; + machineKey: string; + deviceId: string | null; + code: string; + correlationId: string; + /** Optional finer classification for support; the wire `code` stays the contract. */ + reason?: string; +}): void { + console.log(JSON.stringify({ + ts: new Date().toISOString(), + svc: SERVICE, + event: args.event, + userId: args.userId, + machineKeyPrefix: args.machineKey.slice(0, 8), + deviceIdPrefix: args.deviceId ? args.deviceId.slice(0, 8) : null, + code: args.code, + correlationId: args.correlationId, + ...(args.reason ? { reason: args.reason.slice(0, 300) } : {}), + })); +} + +/** + * One line per diagnostics upload, stored or refused. + * + * The route stores bytes it never parses, so this line is the only record that + * an upload happened at all — and the only way to tell "the user's report never + * arrived" from "it arrived and was refused for being too large". + */ +export function logDiagnosticsUpload(args: { + outcome: "stored" | "rejected"; + status: number; + reason?: string; + identity: string; + authenticated: boolean; + bytes: number; +}): void { + console.log(JSON.stringify({ + ts: new Date().toISOString(), + svc: SERVICE, + kind: "diagnostics_upload", + outcome: args.outcome, + status: args.status, + ...(args.reason ? { reason: args.reason } : {}), + // The identity is already a hash for anonymous callers; a signed-in one is + // truncated for the same reason every other log line here truncates. + identity: args.identity.slice(0, 24), + authenticated: args.authenticated, + bytes: args.bytes, + })); +} + +export function logDirectoryLifecycle(args: { + correlationId: string; + /** The matched account route's kind, or null for anything else. */ + route: string | null; + method: string; + status: number; + durationMs: number; +}): void { + const outcome = args.status < 400 + ? "ok" + : args.status < 500 + ? "client_error" + : "server_error"; + console.log(JSON.stringify({ + ts: new Date().toISOString(), + svc: SERVICE, + kind: "request_completed", + correlationId: args.correlationId, + route: args.route ?? "other", + method: args.method, + status: args.status, + outcome, + durationMs: Math.max(0, Math.round(args.durationMs)), + })); +} diff --git a/apps/account-directory/src/pairingGrants.ts b/apps/account-directory/src/pairingGrants.ts new file mode 100644 index 000000000..b8a164fcb --- /dev/null +++ b/apps/account-directory/src/pairingGrants.ts @@ -0,0 +1,302 @@ +import { verifyCallerToken, type CallerTokenEnv } from "./callerToken"; + +/** + * The pairing-grant lifecycle: mint, reserve, consume, release, sweep. + * + * A grant is the second of the two proofs that a human just authenticated ON A + * SPECIFIC MACHINE, and the only one available to token shapes that carry no + * `auth_time`/`fva` claim. Because it is the user's only way back onto their + * own account, every state transition it can make lives here rather than being + * spread through the register handler — `createPairingProofBroker` gives the + * mutable "has this request proven anything yet, and is a grant still + * restorable" state exactly one owner. + */ + +/** The slice of the Worker env this module needs. */ +export type PairingGrantEnv = CallerTokenEnv & { DB: D1Database }; + +/** + * How long a minted pairing grant stays spendable. + * + * Deliberately the same order as `PAIRING_AUTH_FRESHNESS_MS`: both answer "did + * a human just authenticate?", so a grant must not outlive the claim it stands + * in for. It covers the sign-in, the automatic re-pair that follows it, and one + * retry — nothing longer. + */ +export const PAIRING_GRANT_TTL_MS = 10 * 60_000; + +/** + * How long one registration may hold a pairing grant reserved before another is + * allowed to take it. + * + * This is a CRASH-SAFETY bound, not a lease anyone waits on. A reservation is + * held only across a single activity-relay hand-off — two attempts with a short + * backoff — so a minute is generous for the happy path. What it actually bounds + * is the bad path: a worker that dies between reserving a grant and either + * consuming or releasing it would otherwise strand the row as permanently + * unspendable until it expired, which is the same lockout the two-phase scheme + * exists to remove. After this long the reservation simply does not count. + */ +export const PAIRING_GRANT_RESERVATION_MS = 60_000; + +function bytesToBase64Url(bytes: Uint8Array): string { + let binary = ""; + for (const byte of bytes) binary += String.fromCharCode(byte); + return btoa(binary).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/g, ""); +} + +async function sha256Base64Url(value: string): Promise { + const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(value)); + return bytesToBase64Url(new Uint8Array(digest)); +} + +/** + * Mint the pairing grant for a just-completed `/device/*` sign-in. + * + * `accessToken` is the token this worker itself fetched from Clerk moments ago, + * but it is re-verified rather than decoded: the user id the grant is bound to + * decides whose revocation it can lift, and that must come from a signature + * check, not from a base64 payload. Verification failure yields `null` — the + * sign-in still succeeds, only the second proof path is unavailable. + * + * Only the hash is stored. The plaintext exists in one response body and in the + * signing-in machine's memory; a dump of this D1 yields nothing spendable. + */ +export async function mintPairingGrant( + env: PairingGrantEnv, + args: { accessToken: string; machineKey: string; nowMs: number }, +): Promise { + let userId: string; + try { + userId = await verifyCallerToken(args.accessToken, env); + } catch { + return null; + } + const grant = bytesToBase64Url(crypto.getRandomValues(new Uint8Array(32))); + await env.DB.prepare(` + insert into machine_pairing_grants (grant_hash, user_id, machine_key, created_at, expires_at) + values (?, ?, ?, ?, ?) + on conflict(grant_hash) do nothing + `).bind( + await sha256Base64Url(grant), + userId, + args.machineKey, + args.nowMs, + args.nowMs + PAIRING_GRANT_TTL_MS, + ).run(); + return grant; +} + +/** + * Reserve a pairing grant: phase one of spending it. + * + * One statement still carries every rule the grant exists to enforce — it must + * belong to this caller, name this machine, still be inside its TTL, and not + * already be held by another in-flight registration — because splitting them + * into a read and a later write would let two concurrent registrations both + * observe a spendable row. `changes === 1` is therefore the whole proof. + * + * What the reservation buys over the plain delete this replaced is a spend that + * can be undone. The relay hand-off that follows can fail, and destroying the + * grant before knowing the outcome made a relay outage cost the user their only + * way back onto the account. A reserved grant can be put back byte for byte + * (`releaseReservedPairingGrant`) — no fresh expiry, no read-then-write. + */ +async function reservePairingGrant( + env: PairingGrantEnv, + args: { userId: string; machineKey: string; grant: string; nowMs: number }, +): Promise { + const result = await env.DB.prepare(` + update machine_pairing_grants + set reserved_at = ? + where grant_hash = ? and user_id = ? and machine_key = ? and expires_at > ? + and (reserved_at is null or reserved_at <= ?) + `).bind( + args.nowMs, + await sha256Base64Url(args.grant), + args.userId, + args.machineKey, + args.nowMs, + args.nowMs - PAIRING_GRANT_RESERVATION_MS, + ).run(); + return (result.meta.changes ?? 0) === 1; +} + +/** + * Phase two, success: the grant is spent for good. + * + * `reserved_at` is part of the predicate so this only ever deletes the + * reservation THIS request took. A stale reservation another registration has + * since claimed is not ours to consume. + */ +async function consumeReservedPairingGrant( + env: PairingGrantEnv, + args: { userId: string; machineKey: string; grant: string; reservedAt: number }, +): Promise { + await env.DB.prepare(` + delete from machine_pairing_grants + where grant_hash = ? and user_id = ? and machine_key = ? and reserved_at = ? + `).bind( + await sha256Base64Url(args.grant), + args.userId, + args.machineKey, + args.reservedAt, + ).run(); +} + +/** + * Phase two, failure: put the grant back exactly as it was. + * + * Only `reserved_at` is cleared. `expires_at` is untouched on purpose — an + * attacker who can force relay failures must not be able to keep a grant alive + * past the TTL it was minted with, so a release restores spendability without + * extending the window. Same `reserved_at` predicate as the consume, for the + * same reason. + */ +async function releaseReservedPairingGrant( + env: PairingGrantEnv, + args: { userId: string; machineKey: string; grant: string; reservedAt: number }, +): Promise { + await env.DB.prepare(` + update machine_pairing_grants + set reserved_at = null + where grant_hash = ? and user_id = ? and machine_key = ? and reserved_at = ? + `).bind( + await sha256Base64Url(args.grant), + args.userId, + args.machineKey, + args.reservedAt, + ).run(); +} + +/** + * How a register call proved that a human just authenticated ON THIS MACHINE. + * + * Two privileged operations sit behind this one bar — lifting a revocation, and + * superseding the rows a rotated machine key left behind — and they must sit + * behind the SAME bar. Everything else in a register request (`deviceId`, + * `pairing`, the machine key itself) is caller-supplied and therefore forgeable + * by exactly the removed machine these gates exist to stop. + */ +export type PairingProof = + /** An `auth_time`/`fva` claim on the caller's own verified token. Costs nothing. */ + | { kind: "claim" } + /** A grant, reserved and still restorable. Must be consumed or released before the response. */ + | { kind: "grant"; grant: string; reservedAt: number } + /** A grant already consumed earlier in this request: still proof, no longer spendable. */ + | { kind: "spent_grant" } + /** Nothing was proven. */ + | { kind: "none" }; + +/** + * Establish the proof, spending a grant only if the claim path cannot answer. + * + * The claim is checked first so a genuinely fresh sign-in never burns a grant + * it does not need, and it is honored on any register call because it is a + * property of the token this worker already verified — no client assertion is + * involved. The grant is the fallback, and it is honored ONLY on a deliberate + * `pairing: true` link: it is a single-use credential, and a background + * heartbeat that happens to still be carrying one must leave it untouched. + */ +async function acquirePairingProof( + env: PairingGrantEnv, + args: PairingProofArgs, +): Promise { + if (args.freshInteractiveAuthentication) return { kind: "claim" }; + if (!args.pairing || !args.pairingGrant) return { kind: "none" }; + const reserved = await reservePairingGrant(env, { + userId: args.userId, + machineKey: args.machineKey, + grant: args.pairingGrant, + nowMs: args.nowMs, + }); + return reserved + ? { kind: "grant", grant: args.pairingGrant, reservedAt: args.nowMs } + : { kind: "none" }; +} + +export type PairingProofArgs = { + userId: string; + machineKey: string; + /** The register call's `pairing` flag: a grant is only ever spent on a deliberate link. */ + pairing: boolean; + pairingGrant: string | null; + freshInteractiveAuthentication: boolean; + nowMs: number; +}; + +/** + * One owner for the "what has this request proven, and is a grant still + * restorable" state. + * + * A single register call can need the proof twice — once to lift a revocation, + * once to supersede phantom duplicates — and the reserved grant behind it must + * be spendable exactly once across both. Handing the register handler a broker + * instead of four loose functions and a mutable local is what makes that + * impossible to get wrong: the grant is reserved at most once, and `consume` + * and `release` are the only ways out of that reservation. + */ +export type PairingProofBroker = { + /** Establish the proof, at most once per request. Cheap on every later call. */ + prove(): Promise; + /** Spend a reserved grant for good. Still proof afterwards, never spendable again. */ + consume(): Promise; + /** Put a reserved grant back byte for byte; nothing is proven any more. */ + release(): Promise; +}; + +export function createPairingProofBroker( + env: PairingGrantEnv, + args: PairingProofArgs, +): PairingProofBroker { + // Established at most once, and only when something actually needs it: a + // plain heartbeat — the overwhelming majority of calls — must not touch the + // grants table at all, and a request that clears a revocation and then + // supersedes duplicates in the same breath must not pay for the proof twice. + let proof: PairingProof | null = null; + return { + async prove(): Promise { + proof ??= await acquirePairingProof(env, args); + return proof; + }, + async consume(): Promise { + if (proof?.kind !== "grant") return; + await consumeReservedPairingGrant(env, { + userId: args.userId, + machineKey: args.machineKey, + grant: proof.grant, + reservedAt: proof.reservedAt, + }); + proof = { kind: "spent_grant" }; + }, + async release(): Promise { + if (proof?.kind !== "grant") return; + await releaseReservedPairingGrant(env, { + userId: args.userId, + machineKey: args.machineKey, + grant: proof.grant, + reservedAt: proof.reservedAt, + }); + proof = { kind: "none" }; + }, + }; +} + +/** + * Cron sweep: an unspent grant is dead weight the moment it expires. + * + * Reservations are ignored on purpose. A grant that expires while a + * registration holds it reserved was already unspendable by the time the sweep + * ran — every phase checks `expires_at` — so removing it costs nothing, and the + * release that follows simply matches no row. + */ +export async function cleanupExpiredPairingGrants( + env: { DB: D1Database }, + nowMs = Date.now(), +): Promise { + const result = await env.DB + .prepare("delete from machine_pairing_grants where expires_at <= ?") + .bind(nowMs) + .run(); + return result.meta.changes ?? 0; +} diff --git a/apps/account-directory/src/trustedOrigin.ts b/apps/account-directory/src/trustedOrigin.ts new file mode 100644 index 000000000..5591ca1c3 --- /dev/null +++ b/apps/account-directory/src/trustedOrigin.ts @@ -0,0 +1,61 @@ +/** + * What this Worker will accept as a trusted origin, decided once. + * + * Three call sites had grown their own near-identical copy of this: the push + * relay base URL, the hosted web client's CORS origin, and the diagnostics + * route's "is this a drive-by browser upload" exemption. They agreed by + * coincidence, not by construction, and the next edit to any one of them would + * have made a request trusted on one route and refused on another. + */ + +/** + * The three spellings of "this machine". + * + * `URL.hostname` keeps the brackets on an IPv6 literal, so `[::1]` is the form + * that actually shows up here — not `::1`. + */ +export function isLoopbackHostname(hostname: string): boolean { + return hostname === "localhost" || hostname === "127.0.0.1" || hostname === "[::1]"; +} + +export type TrustedHttpsOriginOptions = { + /** + * Require the caller to have written a bare origin and nothing else. + * + * A CORS allow-list is compared against `Origin`, which is always bare, so a + * configured value carrying a path is a misconfiguration worth refusing + * rather than silently truncating. A base URL that gets a path appended to it + * is not: there the origin is what the caller meant, whatever else it wrote. + */ + requireExactOrigin?: boolean; +}; + +/** + * The origin of `raw`, or null when `raw` is not something to trust. + * + * HTTPS only, with one exception: loopback over plain HTTP, because a local + * `wrangler dev` run has no certificate and refusing it would mean the + * development path could never exercise these routes at all. + * + * Credentials, a query, and a fragment are all refused outright rather than + * dropped. Each of them means the configured value is not the thing the author + * thought it was, and a Worker that quietly discards half of a setting is how a + * deployment ends up trusting an origin nobody chose. + */ +export function trustedHttpsOrigin( + raw: string | null | undefined, + options: TrustedHttpsOriginOptions = {}, +): string | null { + const trimmed = raw?.trim(); + if (!trimmed) return null; + try { + const url = new URL(trimmed); + const loopback = isLoopbackHostname(url.hostname); + if (url.protocol !== "https:" && !(url.protocol === "http:" && loopback)) return null; + if (url.username || url.password || url.search || url.hash) return null; + if (options.requireExactOrigin && url.origin !== trimmed) return null; + return url.origin; + } catch { + return null; + } +} diff --git a/apps/account-directory/test/deviceAuthorization.test.ts b/apps/account-directory/test/deviceAuthorization.test.ts new file mode 100644 index 000000000..ecdbcfb5e --- /dev/null +++ b/apps/account-directory/test/deviceAuthorization.test.ts @@ -0,0 +1,443 @@ +import { describe, expect, it, vi } from "vitest"; +import { handleRequest } from "../src/directory"; +import worker from "../src/index"; +import { + deviceConfirmationRequest, + ISSUER, + makeEnv, + mintToken, + OAUTH_CLIENT_ID, + request, +} from "./helpers"; + +/** + * The `/device/*` sign-in bridge: code creation, browser confirmation, the + * Clerk OAuth + PKCE round trip, and one-time redemption. Split out of + * `directory.test.ts` verbatim — it is a self-contained protocol with its own + * failure modes, and it was the largest single thing in that file. + */ + +describe("device authorization bridge", () => { + it("creates, approves with Clerk OAuth + PKCE, and one-time redeems a secret-bound device code", async () => { + const env = makeEnv(); + let now = Date.parse("2026-07-14T12:00:00.000Z"); + const deviceSecret = "daemon-device-secret-with-at-least-32-bytes"; + const created = await handleRequest( + request("POST", "/device/code", undefined, { device_secret: deviceSecret }), + env, + { now: () => now }, + ); + expect(created.status).toBe(200); + const device = await created.json() as Record; + expect(device).toMatchObject({ + device_code: expect.any(String), + user_code: expect.stringMatching(/^[A-Z2-9]{4}-[A-Z2-9]{4}$/), + verification_uri: "https://directory.test/device", + verification_uri_complete: expect.stringContaining("https://directory.test/device?user_code="), + expires_in: 600, + interval: 5, + }); + + const pending = await handleRequest( + request("POST", "/device/token", undefined, { + device_code: device.device_code, + device_secret: deviceSecret, + }), + env, + { now: () => now }, + ); + expect(pending.status).toBe(400); + expect(await pending.json()).toEqual({ error: "authorization_pending", interval: 5 }); + + const approval = await handleRequest( + deviceConfirmationRequest(String(device.user_code)), + env, + { now: () => now }, + ); + expect(approval.status).toBe(302); + const clerkAuthorizeUrl = new URL(approval.headers.get("location")!); + expect(clerkAuthorizeUrl.origin + clerkAuthorizeUrl.pathname).toBe(`${ISSUER}/oauth/authorize`); + expect(clerkAuthorizeUrl.searchParams.get("client_id")).toBe(OAUTH_CLIENT_ID); + expect(clerkAuthorizeUrl.searchParams.get("code_challenge_method")).toBe("S256"); + expect(clerkAuthorizeUrl.searchParams.get("redirect_uri")).toBe("https://directory.test/device/callback"); + const state = clerkAuthorizeUrl.searchParams.get("state")!; + const duplicateApproval = await handleRequest( + deviceConfirmationRequest(String(device.user_code)), + env, + { now: () => now }, + ); + expect(duplicateApproval.status).toBe(409); + expect(await duplicateApproval.text()).toContain("Sign-in already started"); + const tokenExchange = vi.fn(async (_input: string, init?: RequestInit) => { + const body = Object.fromEntries(new URLSearchParams(String(init?.body))); + expect(body).toMatchObject({ + grant_type: "authorization_code", + code: "clerk-authorization-code", + client_id: OAUTH_CLIENT_ID, + redirect_uri: "https://directory.test/device/callback", + }); + expect(body.code_verifier).toEqual(expect.any(String)); + return new Response(JSON.stringify({ + access_token: "approved-access-token", + refresh_token: "approved-refresh-token", + token_type: "Bearer", + expires_in: 3600, + }), { status: 200, headers: { "content-type": "application/json" } }); + }); + const callback = await handleRequest( + new Request(`https://directory.test/device/callback?code=clerk-authorization-code&state=${encodeURIComponent(state)}`), + env, + { now: () => now, fetchImpl: tokenExchange as typeof fetch }, + ); + expect(callback.status).toBe(200); + expect(await callback.text()).toContain("Signed in to ADE"); + + const wrongSecret = await handleRequest( + request("POST", "/device/token", undefined, { + device_code: device.device_code, + device_secret: "wrong-device-secret-with-at-least-32-bytes", + }), + env, + { now: () => now }, + ); + expect(wrongSecret.status).toBe(401); + expect(await wrongSecret.json()).toEqual({ error: "invalid_grant" }); + + now += 6_000; + const redeemed = await handleRequest( + request("POST", "/device/token", undefined, { + device_code: device.device_code, + device_secret: deviceSecret, + }), + env, + { now: () => now }, + ); + expect(redeemed.status).toBe(200); + expect(await redeemed.json()).toEqual({ + access_token: "approved-access-token", + refresh_token: "approved-refresh-token", + token_type: "Bearer", + expires_in: 3594, + oauth_issuer: ISSUER, + oauth_client_id: OAUTH_CLIENT_ID, + }); + expect(env.DB.deviceRows[0]).toMatchObject({ + status: "consumed", + access_token: null, + refresh_token: null, + }); + + const replay = await handleRequest( + request("POST", "/device/token", undefined, { + device_code: device.device_code, + device_secret: deviceSecret, + }), + env, + { now: () => now + 6_000 }, + ); + expect(replay.status).toBe(401); + expect(await replay.json()).toEqual({ error: "invalid_grant" }); + + const expiredReplay = await handleRequest( + request("POST", "/device/token", undefined, { + device_code: device.device_code, + device_secret: deviceSecret, + }), + env, + { now: () => now + 595_000 }, + ); + expect(expiredReplay.status).toBe(400); + expect(await expiredReplay.json()).toEqual({ error: "expired" }); + expect(env.DB.deviceRows[0]?.status).toBe("consumed"); + }); + + it("claims concurrent duplicate callbacks before the one-time OAuth exchange", async () => { + const env = makeEnv(); + const now = Date.parse("2026-07-14T12:00:00.000Z"); + const created = await handleRequest( + request("POST", "/device/code", undefined, { + device_secret: "daemon-device-secret-with-at-least-32-bytes", + }), + env, + { now: () => now }, + ); + const device = await created.json() as Record; + const approval = await handleRequest( + deviceConfirmationRequest(String(device.user_code)), + env, + { now: () => now }, + ); + const state = new URL(approval.headers.get("location")!).searchParams.get("state")!; + const callbackUrl = `https://directory.test/device/callback?code=one-time-code&state=${encodeURIComponent(state)}`; + env.DB.synchronizeOAuthStateReads(2); + + let resolveSuccessfulExchange: ((response: Response) => void) | null = null; + const successfulExchange = new Promise((resolve) => { + resolveSuccessfulExchange = resolve; + }); + let exchangeCalls = 0; + const fetchImpl = vi.fn((): Promise => { + exchangeCalls += 1; + return exchangeCalls === 1 + ? successfulExchange + : Promise.resolve(new Response(JSON.stringify({ error: "invalid_grant" }), { + status: 400, + headers: { "content-type": "application/json" }, + })); + }); + + const callbacks = Promise.all([ + handleRequest(new Request(callbackUrl), env, { now: () => now, fetchImpl: fetchImpl as typeof fetch }), + handleRequest(new Request(callbackUrl), env, { now: () => now, fetchImpl: fetchImpl as typeof fetch }), + ]); + await vi.waitFor(() => expect(fetchImpl).toHaveBeenCalled()); + resolveSuccessfulExchange!(new Response(JSON.stringify({ + access_token: "winner-access-token", + refresh_token: "winner-refresh-token", + token_type: "Bearer", + expires_in: 3600, + }), { status: 200, headers: { "content-type": "application/json" } })); + const responses = await callbacks; + + expect(fetchImpl).toHaveBeenCalledTimes(1); + expect(responses.map((response) => response.status).sort()).toEqual([200, 409]); + expect(env.DB.deviceRows[0]).toMatchObject({ + status: "approved", + access_token: "winner-access-token", + refresh_token: "winner-refresh-token", + error_message: null, + code_verifier: null, + oauth_state_hash: null, + }); + }); + + it("keeps verification-link GET previews read-only until explicit confirmation", async () => { + const env = makeEnv(); + const now = Date.parse("2026-07-14T12:00:00.000Z"); + const created = await handleRequest( + request("POST", "/device/code", undefined, { + device_secret: "daemon-device-secret-with-at-least-32-bytes", + }), + env, + { now: () => now }, + ); + const device = await created.json() as Record; + const rowBeforePreview = { ...env.DB.deviceRows[0]! }; + const limitsBeforePreview = Array.from( + env.DB.approvalRateLimits, + ([key, value]) => [key, { ...value }] as const, + ); + + const firstPreview = await handleRequest( + new Request(String(device.verification_uri_complete)), + env, + { now: () => now }, + ); + const repeatedPreview = await handleRequest( + new Request(String(device.verification_uri_complete)), + env, + { now: () => now }, + ); + + expect([firstPreview.status, repeatedPreview.status]).toEqual([200, 200]); + expect(await firstPreview.text()).toContain('form method="post" action="/device"'); + expect(env.DB.deviceRows[0]).toEqual(rowBeforePreview); + expect(Array.from(env.DB.approvalRateLimits)).toEqual(limitsBeforePreview); + + const crossSiteSubmit = await handleRequest( + deviceConfirmationRequest(String(device.user_code), { origin: "https://preview.test" }), + env, + { now: () => now }, + ); + expect(crossSiteSubmit.status).toBe(403); + expect(env.DB.deviceRows[0]).toEqual(rowBeforePreview); + expect(Array.from(env.DB.approvalRateLimits)).toEqual(limitsBeforePreview); + + const confirmed = await handleRequest( + deviceConfirmationRequest(String(device.user_code)), + env, + { now: () => now }, + ); + expect(confirmed.status).toBe(302); + expect(env.DB.deviceRows[0]).toMatchObject({ + status: "pending", + code_verifier: expect.any(String), + oauth_state_hash: expect.any(String), + }); + expect(env.DB.approvalRateLimits.size).toBe(2); + }); + + it("returns expired for a device code after its short TTL", async () => { + const env = makeEnv(); + const startedAt = Date.parse("2026-07-14T12:00:00.000Z"); + const deviceSecret = "daemon-device-secret-with-at-least-32-bytes"; + const created = await handleRequest( + request("POST", "/device/code", undefined, { device_secret: deviceSecret }), + env, + { now: () => startedAt }, + ); + const device = await created.json() as Record; + + const expired = await handleRequest( + request("POST", "/device/token", undefined, { + device_code: device.device_code, + device_secret: deviceSecret, + }), + env, + { now: () => startedAt + 601_000 }, + ); + expect(expired.status).toBe(400); + expect(await expired.json()).toEqual({ error: "expired" }); + expect(env.DB.deviceRows[0]?.status).toBe("expired"); + }); + + it("clears expired approved credentials from the scheduled worker without client polling", async () => { + const env = makeEnv(); + const startedAt = Date.parse("2026-07-14T12:00:00.000Z"); + await handleRequest( + request("POST", "/device/code", undefined, { + device_secret: "daemon-device-secret-with-at-least-32-bytes", + }), + env, + { now: () => startedAt }, + ); + Object.assign(env.DB.deviceRows[0]!, { + status: "approved", + code_verifier: "temporary-pkce-verifier", + oauth_state_hash: "temporary-state-hash", + access_token: "abandoned-access-token", + refresh_token: "abandoned-refresh-token", + }); + vi.spyOn(Date, "now").mockReturnValue(startedAt + 601_000); + let cleanup: Promise | undefined; + + await worker.scheduled( + {} as ScheduledEvent, + env, + { waitUntil: (promise) => { cleanup = promise; } } as ExecutionContext, + ); + await cleanup; + + expect(env.DB.deviceRows[0]).toMatchObject({ + status: "expired", + code_verifier: null, + oauth_state_hash: null, + access_token: null, + refresh_token: null, + }); + expect(env.DB.approvalRateLimits.size).toBe(0); + + vi.mocked(Date.now).mockReturnValue(startedAt + 4_201_000); + cleanup = undefined; + await worker.scheduled( + {} as ScheduledEvent, + env, + { waitUntil: (promise) => { cleanup = promise; } } as ExecutionContext, + ); + await cleanup; + expect(env.DB.deviceRows).toHaveLength(0); + }); + + it("rate-limits device-code issuance separately from approval lookups", async () => { + const env = makeEnv(); + const now = Date.parse("2026-07-14T12:00:00.000Z"); + const headers = { + "cf-connecting-ip": "203.0.113.8", + "content-type": "application/json", + }; + for (let attempt = 0; attempt < 10; attempt += 1) { + const response = await handleRequest( + new Request("https://directory.test/device/code", { + method: "POST", + headers, + body: JSON.stringify({ device_secret: "daemon-device-secret-with-at-least-32-bytes" }), + }), + env, + { now: () => now }, + ); + expect(response.status).toBe(200); + } + + const blocked = await handleRequest( + new Request("https://directory.test/device/code", { + method: "POST", + headers, + body: JSON.stringify({ device_secret: "daemon-device-secret-with-at-least-32-bytes" }), + }), + env, + { now: () => now }, + ); + expect(blocked.status).toBe(429); + expect(blocked.headers.get("retry-after")).toBe("60"); + expect(env.DB.deviceRows).toHaveLength(10); + + const approval = await handleRequest( + deviceConfirmationRequest("ZZZZ-ZZZZ", { "cf-connecting-ip": "203.0.113.8" }), + env, + { now: () => now }, + ); + expect(approval.status).toBe(404); + expect(env.DB.approvalRateLimits.size).toBe(2); + expect(Array.from(env.DB.approvalRateLimits.values(), (entry) => entry.attempts).sort()).toEqual([1, 10]); + }); + + it("atomically admits at most ten concurrent device-code issuances per client", async () => { + const env = makeEnv(); + let now = Date.parse("2026-07-14T12:00:00.000Z"); + env.DB.synchronizeRateLimitReads(25); + const responses = await Promise.all(Array.from({ length: 25 }, () => handleRequest( + new Request("https://directory.test/device/code", { + method: "POST", + headers: { + "cf-connecting-ip": "203.0.113.9", + "content-type": "application/json", + }, + body: JSON.stringify({ device_secret: "daemon-device-secret-with-at-least-32-bytes" }), + }), + env, + { now: () => now }, + ))); + + expect(responses.filter((response) => response.status === 200)).toHaveLength(10); + expect(responses.filter((response) => response.status === 429)).toHaveLength(15); + expect(new Set(responses.map((response) => response.status))).toEqual(new Set([200, 429])); + expect(env.DB.deviceRows).toHaveLength(10); + expect(Array.from(env.DB.approvalRateLimits.values(), (entry) => entry.attempts)).toEqual([10]); + + now += 60_000; + const nextWindow = await handleRequest( + new Request("https://directory.test/device/code", { + method: "POST", + headers: { + "cf-connecting-ip": "203.0.113.9", + "content-type": "application/json", + }, + body: JSON.stringify({ device_secret: "daemon-device-secret-with-at-least-32-bytes" }), + }), + env, + { now: () => now }, + ); + expect(nextWindow.status).toBe(200); + expect(env.DB.deviceRows).toHaveLength(11); + expect(Array.from(env.DB.approvalRateLimits.values(), (entry) => entry.attempts)).toEqual([1]); + }); + + it("rate-limits user-code confirmations on the hosted approval page", async () => { + const env = makeEnv(); + const now = Date.parse("2026-07-14T12:00:00.000Z"); + for (let attempt = 0; attempt < 10; attempt += 1) { + const response = await handleRequest( + deviceConfirmationRequest("ABCD-EFGH", { "cf-connecting-ip": "203.0.113.7" }), + env, + { now: () => now }, + ); + expect(response.status).toBe(404); + } + const blocked = await handleRequest( + deviceConfirmationRequest("ABCD-EFGH", { "cf-connecting-ip": "203.0.113.7" }), + env, + { now: () => now }, + ); + expect(blocked.status).toBe(429); + }); +}); diff --git a/apps/account-directory/test/diagnostics.test.ts b/apps/account-directory/test/diagnostics.test.ts new file mode 100644 index 000000000..486fc1179 --- /dev/null +++ b/apps/account-directory/test/diagnostics.test.ts @@ -0,0 +1,496 @@ +import { createHash } from "node:crypto"; +import { describe, expect, it, vi } from "vitest"; +import { + handleDiagnosticsRequest, + isDiagnosticsRequest, + MAX_DIAGNOSTIC_REPORT_BYTES, + MAX_DIAGNOSTIC_UPLOADS_PER_DAY, + type DiagnosticsEnv, +} from "../src/diagnostics"; +import worker from "../src/index"; +import { ISSUER, jwksEndpoint, mintToken, OAUTH_CLIENT_ID } from "./jwks"; + +const UPLOAD_URL = "https://directory.test/diagnostics/upload"; + +/** + * Fake R2, in the same spirit as the fake D1 in `./fakeD1`: enough of + * the real surface to hold the contract (prefix listing, custom metadata, + * stored bytes) and nothing else, so a test failure points at the route rather + * than at the fake. + */ +class FakeR2Bucket { + readonly objects = new Map< + string, + { body: string; customMetadata: Record; contentType: string | undefined } + >(); + + listCalls: Array<{ prefix?: string; limit?: number }> = []; + + /** Set to make every `put` reject, the way a bucket having a bad minute does. */ + putFailure: Error | null = null; + + async put( + key: string, + value: string | ArrayBuffer | ArrayBufferView, + options?: { + httpMetadata?: { contentType?: string }; + customMetadata?: Record; + }, + ): Promise { + if (this.putFailure) throw this.putFailure; + const body = typeof value === "string" + ? value + : new TextDecoder().decode(value as ArrayBuffer); + this.objects.set(key, { + body, + customMetadata: { ...(options?.customMetadata ?? {}) }, + contentType: options?.httpMetadata?.contentType, + }); + } + + async list( + options?: { prefix?: string; limit?: number }, + ): Promise<{ objects: Array<{ key: string }>; truncated: boolean }> { + this.listCalls.push({ prefix: options?.prefix, limit: options?.limit }); + const prefix = options?.prefix ?? ""; + const keys = [...this.objects.keys()].filter((key) => key.startsWith(prefix)).sort(); + const limited = options?.limit === undefined ? keys : keys.slice(0, options.limit); + return { + objects: limited.map((key) => ({ key })), + truncated: limited.length < keys.length, + }; + } + + keys(): string[] { + return [...this.objects.keys()]; + } +} + +function makeEnv( + overrides: Partial = {}, +): DiagnosticsEnv & { DIAGNOSTICS: FakeR2Bucket } { + return { + DB: {} as unknown as D1Database, + CLERK_JWKS_URL: jwksEndpoint(), + CLERK_ISSUER: ISSUER, + CLERK_OAUTH_CLIENT_ID: OAUTH_CLIENT_ID, + DIAGNOSTICS: new FakeR2Bucket(), + ...overrides, + } as unknown as DiagnosticsEnv & { DIAGNOSTICS: FakeR2Bucket }; +} + +/** A distinct address per test: the anonymous quota is keyed on the caller IP. */ +let addressCounter = 0; +function nextIp(): string { + addressCounter += 1; + return `203.0.113.${addressCounter}`; +} + +function uploadRequest(args: { + body: string; + contentType?: string; + token?: string; + /** `null` omits the header entirely — what a request that never crossed Cloudflare looks like. */ + ip?: string | null; + url?: string; + headers?: Record; +}): Request { + const ip = args.ip === undefined ? nextIp() : args.ip; + return new Request(args.url ?? UPLOAD_URL, { + method: "POST", + headers: { + "content-type": args.contentType ?? "application/json", + ...(ip ? { "cf-connecting-ip": ip } : {}), + ...(args.token ? { authorization: `Bearer ${args.token}` } : {}), + ...(args.headers ?? {}), + }, + body: args.body, + }); +} + +const REPORT = "# ADE diagnostic report\n\n- surface: brain_repair\n- installId: abc123\n"; + +const KEY_SHAPE = + /^reports\/\d{4}-\d{2}-\d{2}\/(u-[A-Za-z0-9_-]+|anon-[0-9a-f]{16})\/[0-9a-f-]{36}\.md$/; + +/** + * A fixed clock for every test that reasons about the day bucket. + * + * The quota key is the UTC day, so a test that spans midnight — either by + * deriving the prefix itself while the route derives its own, or by making + * several requests in a row — would silently be asking about two different + * days. `handleDiagnosticsRequest` takes `now` for exactly this. + */ +const FIXED_NOW = Date.UTC(2026, 7, 18, 12, 0, 0); +const FIXED_DAY_KEY = new Date(FIXED_NOW).toISOString().slice(0, 10); +const FIXED_CLOCK = { now: () => FIXED_NOW }; + +describe("diagnostics upload route", () => { + it("matches only the upload path", () => { + expect(isDiagnosticsRequest(new URL(UPLOAD_URL))).toBe(true); + expect(isDiagnosticsRequest(new URL(`${UPLOAD_URL}/`))).toBe(true); + expect(isDiagnosticsRequest(new URL("https://directory.test/diagnostics"))).toBe(false); + expect(isDiagnosticsRequest(new URL("https://directory.test/account/machines"))).toBe(false); + }); + + it("stores an anonymous upload under a dated per-caller key", async () => { + const env = makeEnv(); + const response = await handleDiagnosticsRequest( + uploadRequest({ + body: JSON.stringify({ report: REPORT, installId: "install-9", appVersion: "1.2.60" }), + ip: "203.0.113.200", + }), + env, + ); + + expect(response.status).toBe(200); + const payload = await response.json() as { ok: boolean; id: string }; + expect(payload.ok).toBe(true); + expect(payload.id).toMatch(/^[0-9a-f-]{36}$/); + + const keys = env.DIAGNOSTICS.keys(); + expect(keys).toHaveLength(1); + expect(keys[0]).toMatch(KEY_SHAPE); + expect(keys[0]).toContain("/anon-"); + expect(keys[0]?.endsWith(`${payload.id}.md`)).toBe(true); + + const stored = env.DIAGNOSTICS.objects.get(keys[0]!)!; + // The body is the report byte-for-byte: redaction happens upstream, on the + // machine, and this route must not touch what it was handed. + expect(stored.body).toBe(REPORT); + expect(stored.contentType).toBe("text/markdown; charset=utf-8"); + expect(stored.customMetadata).toEqual({ installId: "install-9", appVersion: "1.2.60" }); + }); + + it("never echoes the report back", async () => { + const env = makeEnv(); + const response = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }) }), + env, + ); + const raw = await response.text(); + expect(raw).not.toContain("ADE diagnostic report"); + expect(JSON.parse(raw)).toEqual({ ok: true, id: expect.any(String) }); + }); + + it("keys an authenticated upload by Clerk user id and records it in metadata", async () => { + const env = makeEnv(); + const response = await handleDiagnosticsRequest( + uploadRequest({ + body: JSON.stringify({ report: REPORT, installId: "install-7" }), + token: await mintToken({ sub: "user_42" }), + }), + env, + ); + + expect(response.status).toBe(200); + const keys = env.DIAGNOSTICS.keys(); + expect(keys[0]).toMatch(KEY_SHAPE); + expect(keys[0]).toContain("/u-user_42/"); + expect(env.DIAGNOSTICS.objects.get(keys[0]!)!.customMetadata).toEqual({ + userId: "user_42", + installId: "install-7", + }); + }); + + it("rejects a bearer token that does not verify instead of storing it anonymously", async () => { + const env = makeEnv(); + const response = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }), token: "not-a-jwt" }), + env, + ); + expect(response.status).toBe(401); + expect(env.DIAGNOSTICS.keys()).toHaveLength(0); + }); + + it("refuses an Authorization header it cannot parse instead of downgrading it", async () => { + const env = makeEnv(); + // A client that believes it is signed in and is not. Storing this anonymously + // would hide the broken sign-in from the one user in a position to report it. + const response = await handleDiagnosticsRequest( + uploadRequest({ + body: JSON.stringify({ report: REPORT }), + headers: { authorization: "Token abc123" }, + }), + env, + ); + expect(response.status).toBe(401); + expect(env.DIAGNOSTICS.keys()).toHaveLength(0); + }); + + it("answers 503, not 401, when this Worker has no Clerk configuration", async () => { + // A deployment fault, not a bad token: 401 would send the user to sign in + // again forever. The account routes classify it the same way. + const env = makeEnv({ CLERK_JWKS_URL: "" }); + const response = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }), token: await mintToken() }), + env, + ); + expect(response.status).toBe(503); + expect(env.DIAGNOSTICS.keys()).toHaveLength(0); + }); + + it("refuses a cross-site browser upload but not ADE's own renderer", async () => { + const env = makeEnv(); + const hostile = await handleDiagnosticsRequest( + uploadRequest({ + body: JSON.stringify({ report: REPORT }), + headers: { "sec-fetch-site": "cross-site", origin: "https://evil.test" }, + }), + env, + ); + expect(hostile.status).toBe(403); + expect(env.DIAGNOSTICS.keys()).toHaveLength(0); + + // Electron's renderer is cross-site to this Worker too: `file://` in a + // packaged build sends `Origin: null`, development sends loopback. Both are + // the real "Send to ADE" button and neither may be caught by this. + for (const origin of ["null", "http://localhost:5173"]) { + const renderer = await handleDiagnosticsRequest( + uploadRequest({ + body: JSON.stringify({ report: REPORT }), + headers: { "sec-fetch-site": "cross-site", origin }, + }), + env, + ); + expect(renderer.status).toBe(200); + } + // The CLI and every other non-browser sender set no fetch-metadata header. + const cli = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }) }), + env, + ); + expect(cli.status).toBe(200); + expect(env.DIAGNOSTICS.keys()).toHaveLength(3); + }); + + it("never lets a caller-set forwarding header buy a fresh quota", async () => { + // Off Cloudflare there is no trustworthy address, so everyone shares one + // bucket. Trusting `x-forwarded-for` would make the quota opt-out. + const env = makeEnv(); + for (let attempt = 0; attempt < MAX_DIAGNOSTIC_UPLOADS_PER_DAY; attempt += 1) { + const accepted = await handleDiagnosticsRequest( + uploadRequest({ + body: JSON.stringify({ report: REPORT }), + ip: null, + headers: { "x-forwarded-for": `198.51.100.${attempt}` }, + }), + env, + FIXED_CLOCK, + ); + expect(accepted.status).toBe(200); + } + const refused = await handleDiagnosticsRequest( + uploadRequest({ + body: JSON.stringify({ report: REPORT }), + ip: null, + headers: { "x-forwarded-for": "198.51.100.99" }, + }), + env, + FIXED_CLOCK, + ); + expect(refused.status).toBe(429); + expect(env.DIAGNOSTICS.keys()).toHaveLength(MAX_DIAGNOSTIC_UPLOADS_PER_DAY); + }); + + it("accepts a text/plain body with query metadata", async () => { + const env = makeEnv(); + const response = await handleDiagnosticsRequest( + uploadRequest({ + body: REPORT, + contentType: "text/plain; charset=utf-8", + url: `${UPLOAD_URL}?installId=install-plain&appVersion=9.9.9`, + }), + env, + ); + expect(response.status).toBe(200); + const stored = env.DIAGNOSTICS.objects.get(env.DIAGNOSTICS.keys()[0]!)!; + expect(stored.body).toBe(REPORT); + expect(stored.customMetadata).toEqual({ + installId: "install-plain", + appVersion: "9.9.9", + }); + }); + + it("rejects an empty or unparseable report", async () => { + const env = makeEnv(); + const blank = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: " " }) }), + env, + ); + expect(blank.status).toBe(400); + const garbage = await handleDiagnosticsRequest( + uploadRequest({ body: "{not json" }), + env, + ); + expect(garbage.status).toBe(400); + expect(env.DIAGNOSTICS.keys()).toHaveLength(0); + }); + + it("rejects a report over the size cap without storing it", async () => { + const env = makeEnv(); + const oversized = "x".repeat(MAX_DIAGNOSTIC_REPORT_BYTES + 1_024); + const response = await handleDiagnosticsRequest( + uploadRequest({ body: oversized, contentType: "text/plain" }), + env, + ); + expect(response.status).toBe(413); + expect(await response.json()).toEqual({ error: "report too large" }); + expect(env.DIAGNOSTICS.keys()).toHaveLength(0); + }); + + it("counts the streamed body rather than trusting content-length", async () => { + const env = makeEnv(); + const chunk = new TextEncoder().encode("y".repeat(64 * 1024)); + const stream = new ReadableStream({ + start(controller) { + for (let index = 0; index < 9; index += 1) controller.enqueue(chunk); + controller.close(); + }, + }); + const request = new Request(UPLOAD_URL, { + method: "POST", + headers: { + "content-type": "text/plain", + "content-length": "10", + "cf-connecting-ip": nextIp(), + }, + body: stream, + // Node's fetch requires this for a streaming request body. + duplex: "half", + } as RequestInit); + + const response = await handleDiagnosticsRequest(request, env); + expect(response.status).toBe(413); + expect(env.DIAGNOSTICS.keys()).toHaveLength(0); + }); + + it("allows five uploads a day per caller and refuses the sixth", async () => { + const env = makeEnv(); + const ip = "198.51.100.7"; + for (let attempt = 0; attempt < MAX_DIAGNOSTIC_UPLOADS_PER_DAY; attempt += 1) { + const accepted = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }), ip }), + env, + FIXED_CLOCK, + ); + expect(accepted.status).toBe(200); + } + const refused = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }), ip }), + env, + FIXED_CLOCK, + ); + expect(refused.status).toBe(429); + expect(refused.headers.get("retry-after")).toBe("86400"); + expect(env.DIAGNOSTICS.keys()).toHaveLength(MAX_DIAGNOSTIC_UPLOADS_PER_DAY); + + // A different caller is unaffected, and the durable half of the limit is a + // prefix listing scoped to that caller's day. + const other = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }), ip: "198.51.100.8" }), + env, + FIXED_CLOCK, + ); + expect(other.status).toBe(200); + expect(env.DIAGNOSTICS.listCalls.at(-1)?.prefix).toBe( + `reports/${FIXED_DAY_KEY}/anon-${createHash("sha256").update("198.51.100.8").digest("hex").slice(0, 16)}/`, + ); + expect(env.DIAGNOSTICS.listCalls.at(-1)?.limit).toBe(MAX_DIAGNOSTIC_UPLOADS_PER_DAY + 1); + }); + + it("enforces the day quota from stored objects when this isolate has no memory of them", async () => { + // The isolate counter is only a fast path; a recycled isolate must still + // refuse a caller who already spent the day's quota, which is what the R2 + // prefix listing is for. + const env = makeEnv(); + const ip = "198.51.100.30"; + const identity = `anon-${createHash("sha256").update(ip).digest("hex").slice(0, 16)}`; + const prefix = `reports/${FIXED_DAY_KEY}/${identity}/`; + for (let index = 0; index < MAX_DIAGNOSTIC_UPLOADS_PER_DAY; index += 1) { + await env.DIAGNOSTICS.put(`${prefix}seeded-${index}.md`, "stored by an earlier isolate"); + } + + const refused = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }), ip }), + env, + FIXED_CLOCK, + ); + expect(refused.status).toBe(429); + expect(env.DIAGNOSTICS.keys()).toHaveLength(MAX_DIAGNOSTIC_UPLOADS_PER_DAY); + }); + + it("answers a bounded status and still logs one line when the store refuses the write", async () => { + // The log line is the only record that an upload happened, so the one path + // where the store itself fails must not be the path that answers silently. + const env = makeEnv(); + env.DIAGNOSTICS.putFailure = new Error("R2 unavailable"); + const lines: string[] = []; + const logged = vi.spyOn(console, "log").mockImplementation((line: unknown) => { + lines.push(String(line)); + }); + let response: Response; + try { + response = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }) }), + env, + ); + } finally { + logged.mockRestore(); + } + + expect(response.status).toBe(502); + expect(await response.json()).toEqual({ error: "diagnostics upload failed" }); + expect(env.DIAGNOSTICS.keys()).toHaveLength(0); + + const uploadLines = lines + .map((line) => JSON.parse(line) as Record) + .filter((entry) => entry.kind === "diagnostics_upload"); + expect(uploadLines).toHaveLength(1); + expect(uploadLines[0]).toMatchObject({ + outcome: "rejected", + status: 502, + reason: "storage_write_failed", + authenticated: false, + }); + }); + + it("answers 503 when the bucket binding is missing", async () => { + const env = makeEnv({ DIAGNOSTICS: undefined }); + const response = await handleDiagnosticsRequest( + uploadRequest({ body: JSON.stringify({ report: REPORT }) }), + env, + ); + expect(response.status).toBe(503); + }); + + it("answers the browser preflight and refuses other methods", async () => { + const env = makeEnv(); + const preflight = await handleDiagnosticsRequest( + new Request(UPLOAD_URL, { + method: "OPTIONS", + headers: { origin: "null", "access-control-request-method": "POST" }, + }), + env, + ); + expect(preflight.status).toBe(204); + expect(preflight.headers.get("access-control-allow-origin")).toBe("*"); + expect(preflight.headers.get("access-control-allow-headers")).toContain("authorization"); + + const wrongMethod = await handleDiagnosticsRequest( + new Request(UPLOAD_URL, { method: "GET" }), + env, + ); + expect(wrongMethod.status).toBe(405); + }); + + it("is reachable through the Worker entry point without account authentication", async () => { + const env = makeEnv(); + const response = await worker.fetch( + uploadRequest({ body: JSON.stringify({ report: REPORT }) }), + env, + ); + expect(response.status).toBe(200); + expect(env.DIAGNOSTICS.keys()).toHaveLength(1); + }); +}); diff --git a/apps/account-directory/test/directory.test.ts b/apps/account-directory/test/directory.test.ts index 5e536bd22..07ee32d5f 100644 --- a/apps/account-directory/test/directory.test.ts +++ b/apps/account-directory/test/directory.test.ts @@ -1,676 +1,23 @@ -import { createServer, type Server } from "node:http"; -import type { AddressInfo } from "node:net"; -import { exportJWK, generateKeyPair, SignJWT } from "jose"; -import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; -import { - cleanupExpiredPairingGrants, - handleRequest, - PAIRING_GRANT_TTL_MS, - verifyCallerToken, - type Env, -} from "../src/directory"; +import { describe, expect, it, vi } from "vitest"; +import { verifyCallerToken } from "../src/callerToken"; +import { handleRequest, type Env } from "../src/directory"; import worker from "../src/index"; - -type StoredMachine = { - user_id: string; - machine_key: string; - device_id: string | null; - name: string | null; - custom_name: string | null; - platform: string | null; - device_type: string | null; - pubkey: string | null; - reachable_endpoints: string | null; - power: string | null; - sleep_state: string | null; - sleep_state_at: number | null; - last_seen_at: number | null; - created_at: number | null; -}; - -type StoredDeviceAuthorization = { - device_code: string; - user_code: string; - device_secret_hash: string; - machine_key: string | null; - status: "pending" | "approved" | "consumed" | "expired" | "error"; - code_verifier: string | null; - oauth_state_hash: string | null; - access_token: string | null; - refresh_token: string | null; - token_type: string | null; - expires_in: number | null; - error_message: string | null; - poll_interval_seconds: number; - last_polled_at: number | null; - created_at: number; - expires_at: number; - approved_at: number | null; - consumed_at: number | null; -}; - -class FakeD1Statement { - private values: unknown[] = []; - - constructor( - private readonly sql: string, - private readonly db: FakeD1Database, - ) {} - - bind(...values: unknown[]): this { - this.values = values; - return this; - } - - async first(): Promise { - const result = this.db.first(this.sql, this.values); - await this.db.waitForConcurrentReads(this.sql); - return result; - } - - async all(): Promise<{ results: T[] }> { - return { results: this.db.all(this.sql, this.values) }; - } - - async run(): Promise<{ success: boolean; meta: { changes: number } }> { - const changes = this.db.run(this.sql, this.values); - return { success: true, meta: { changes } }; - } -} - -type StoredRevocation = { - user_id: string; - machine_key: string; - device_id: string | null; - revoked_at: number; -}; - -type StoredPairingGrant = { - grant_hash: string; - user_id: string; - machine_key: string; - created_at: number; - expires_at: number; -}; - -class FakeD1Database { - rows: StoredMachine[] = []; - revocations: StoredRevocation[] = []; - deviceRows: StoredDeviceAuthorization[] = []; - pairingGrants: StoredPairingGrant[] = []; - approvalRateLimits = new Map(); - private rateLimitReadBarrier: { - remaining: number; - promise: Promise; - release: () => void; - } | null = null; - private oauthStateReadBarrier: { - remaining: number; - promise: Promise; - release: () => void; - } | null = null; - - synchronizeRateLimitReads(expectedReads: number): void { - let release = () => {}; - const promise = new Promise((resolve) => { - release = resolve; - }); - this.rateLimitReadBarrier = { remaining: expectedReads, promise, release }; - } - - synchronizeOAuthStateReads(expectedReads: number): void { - let release = () => {}; - const promise = new Promise((resolve) => { - release = resolve; - }); - this.oauthStateReadBarrier = { remaining: expectedReads, promise, release }; - } - - async waitForConcurrentReads(sql: string): Promise { - const normalized = sql.toLowerCase(); - const barrier = normalized.includes("from device_approval_rate_limits") - ? this.rateLimitReadBarrier - : normalized.includes("from device_authorizations") && normalized.includes("where oauth_state_hash") - ? this.oauthStateReadBarrier - : null; - if (!barrier) return; - barrier.remaining -= 1; - if (barrier.remaining === 0) barrier.release(); - await barrier.promise; - } - - prepare(sql: string): FakeD1Statement { - return new FakeD1Statement(sql, this); - } - - first(sql: string, values: unknown[]): T | null { - const normalized = sql.toLowerCase(); - if (normalized.includes("from revoked_machines")) { - const [userId, machineKey] = values; - return (this.revocations.find((row) => - row.user_id === userId && row.machine_key === machineKey - ) ?? null) as T | null; - } - if (normalized.includes("from machines")) { - const [userId, machineKey] = values; - return (this.rows.find((row) => row.user_id === userId && row.machine_key === machineKey) ?? null) as T | null; - } - if (normalized.includes("from device_authorizations")) { - const [value] = values; - const key = normalized.includes("where device_code") - ? "device_code" - : normalized.includes("where user_code") - ? "user_code" - : "oauth_state_hash"; - return (this.deviceRows.find((row) => row[key] === value) ?? null) as T | null; - } - if (normalized.includes("from device_approval_rate_limits")) { - return (this.approvalRateLimits.get(String(values[0])) ?? null) as T | null; - } - return null; - } - - all(sql: string, values: unknown[]): T[] { - const normalized = sql.toLowerCase(); - if (!normalized.includes("from machines")) return []; - const [userId] = values; - let rows = this.rows.filter((row) => row.user_id === userId); - if (normalized.includes("order by last_seen_at desc")) { - rows = [...rows].sort((left, right) => - Number(right.last_seen_at ?? 0) - Number(left.last_seen_at ?? 0) - ); - } - if (normalized.includes("limit 500")) rows = rows.slice(0, 500); - return rows as T[]; - } - - run(sql: string, values: unknown[]): number { - const normalized = sql.toLowerCase(); - if (normalized.includes("insert into machine_pairing_grants")) { - const [grantHash, userId, machineKey, createdAt, expiresAt] = values; - if (this.pairingGrants.some((row) => row.grant_hash === grantHash)) return 0; - this.pairingGrants.push({ - grant_hash: String(grantHash), - user_id: String(userId), - machine_key: String(machineKey), - created_at: Number(createdAt), - expires_at: Number(expiresAt), - }); - return 1; - } - if (normalized.includes("delete from machine_pairing_grants")) { - // Mirror the source's single-statement redemption exactly: user, machine, - // and expiry are all part of the WHERE clause, so a test that loosens any - // of them in the worker fails here instead of being absorbed. - if (normalized.includes("grant_hash = ?")) { - const [grantHash, userId, machineKey, nowMs] = values; - const before = this.pairingGrants.length; - this.pairingGrants = this.pairingGrants.filter((row) => - !(row.grant_hash === grantHash - && row.user_id === userId - && row.machine_key === machineKey - && row.expires_at > Number(nowMs)) - ); - return before - this.pairingGrants.length; - } - const cutoff = Number(values[0]); - const before = this.pairingGrants.length; - this.pairingGrants = this.pairingGrants.filter((row) => row.expires_at > cutoff); - return before - this.pairingGrants.length; - } - if (normalized.includes("insert into revoked_machines")) { - const [userId, machineKey, deviceId, revokedAt] = values; - // Mirror whichever conflict clause the source actually uses, so a revert - // to a bare `device_id = excluded.device_id` fails the retry test rather - // than being papered over here. - const preservesDeviceId = normalized.includes("coalesce(excluded.device_id"); - const row = this.revocations.find((entry) => - entry.user_id === userId && entry.machine_key === machineKey - ); - if (row) { - const next = deviceId == null ? null : String(deviceId); - row.device_id = preservesDeviceId ? next ?? row.device_id : next; - row.revoked_at = Number(revokedAt); - return 1; - } - this.revocations.push({ - user_id: String(userId), - machine_key: String(machineKey), - device_id: deviceId == null ? null : String(deviceId), - revoked_at: Number(revokedAt), - }); - return 1; - } - if (normalized.includes("delete from revoked_machines")) { - const [userId, machineKey] = values; - const before = this.revocations.length; - this.revocations = this.revocations.filter((row) => - row.user_id !== userId || row.machine_key !== machineKey - ); - return before - this.revocations.length; - } - if (normalized.includes("insert into machines")) { - const retainRelayEndpoints = values[13] === 1; - const row: StoredMachine = { - user_id: String(values[0]), - machine_key: String(values[1]), - device_id: values[2] == null ? null : String(values[2]), - name: values[3] == null ? null : String(values[3]), - custom_name: null, - platform: values[4] == null ? null : String(values[4]), - device_type: values[5] == null ? null : String(values[5]), - pubkey: values[6] == null ? null : String(values[6]), - reachable_endpoints: values[7] == null ? null : String(values[7]), - power: values[8] == null ? null : String(values[8]), - sleep_state: values[9] == null ? null : String(values[9]), - sleep_state_at: values[10] == null ? null : Number(values[10]), - last_seen_at: values[11] == null ? null : Number(values[11]), - created_at: values[12] == null ? null : Number(values[12]), - }; - const existing = this.rows.find((entry) => - entry.user_id === row.user_id && entry.machine_key === row.machine_key - ); - if (existing) { - if (retainRelayEndpoints) { - const nextEndpoints = JSON.parse(row.reachable_endpoints ?? "[]") as Array<{ kind?: string }>; - const existingRelayEndpoints = ( - JSON.parse(existing.reachable_endpoints ?? "[]") as Array<{ kind?: string }> - ).filter((endpoint) => endpoint.kind === "relay"); - if ( - !nextEndpoints.some((endpoint) => endpoint.kind === "relay") - && existingRelayEndpoints.length > 0 - ) { - row.reachable_endpoints = JSON.stringify([ - ...nextEndpoints, - ...existingRelayEndpoints, - ]); - } - } - Object.assign(existing, row, { - created_at: existing.created_at, - custom_name: existing.custom_name, - // Mirror the source's `coalesce(excluded.x, machines.x)` exactly, so - // a revert to a bare overwrite fails the old-host test here rather - // than being absorbed by the fake. - power: row.power ?? existing.power, - sleep_state: row.sleep_state ?? existing.sleep_state, - sleep_state_at: row.sleep_state_at ?? existing.sleep_state_at, - }); - } else { - this.rows.push(row); - } - return 1; - } - if (normalized.includes("update machines") && normalized.includes("set custom_name")) { - const [customName, userId, machineKey] = values; - const row = this.rows.find((entry) => - entry.user_id === userId && entry.machine_key === machineKey - ); - if (!row) return 0; - row.custom_name = customName == null ? null : String(customName); - return 1; - } - if (normalized.includes("delete from machines")) { - const [userId, machineKey] = values; - const before = this.rows.length; - this.rows = this.rows.filter((row) => row.user_id !== userId || row.machine_key !== machineKey); - return before - this.rows.length; - } - if (normalized.includes("delete from device_authorizations")) { - const cutoff = Number(values[0]); - const before = this.deviceRows.length; - this.deviceRows = this.deviceRows.filter((row) => - row.expires_at > cutoff || !["expired", "consumed", "error"].includes(row.status) - ); - return before - this.deviceRows.length; - } - if (normalized.includes("delete from device_approval_rate_limits")) { - const cutoff = Number(values[0]); - let changes = 0; - for (const [clientHash, record] of this.approvalRateLimits) { - if (record.window_started_at > cutoff) continue; - this.approvalRateLimits.delete(clientHash); - changes += 1; - } - return changes; - } - if (normalized.includes("insert into device_authorizations")) { - const userCode = String(values[1]); - if (this.deviceRows.some((row) => row.user_code === userCode)) { - throw new Error("UNIQUE constraint failed: device_authorizations.user_code"); - } - this.deviceRows.push({ - device_code: String(values[0]), - user_code: userCode, - device_secret_hash: String(values[2]), - machine_key: values[6] == null ? null : String(values[6]), - status: "pending", - code_verifier: null, - oauth_state_hash: null, - access_token: null, - refresh_token: null, - token_type: null, - expires_in: null, - error_message: null, - poll_interval_seconds: Number(values[3]), - last_polled_at: null, - created_at: Number(values[4]), - expires_at: Number(values[5]), - approved_at: null, - consumed_at: null, - }); - return 1; - } - if (normalized.includes("insert into device_approval_rate_limits")) { - const clientHash = String(values[0]); - const now = Number(values[1]); - const windowMs = Number(values[2]); - const maxAttempts = Number(values[5]); - const record = this.approvalRateLimits.get(clientHash); - if (!record) { - this.approvalRateLimits.set(clientHash, { window_started_at: now, attempts: 1 }); - return 1; - } - if (now - record.window_started_at >= windowMs) { - record.window_started_at = now; - record.attempts = 1; - return 1; - } - if (record.attempts >= maxAttempts) return 0; - record.attempts += 1; - return 1; - } - if (normalized.includes("update device_authorizations")) { - if (normalized.includes("where expires_at <= ?")) { - const now = Number(values[0]); - let changes = 0; - for (const row of this.deviceRows) { - if (row.expires_at > now || (row.status !== "pending" && row.status !== "approved")) continue; - row.status = "expired"; - row.code_verifier = null; - row.oauth_state_hash = null; - row.access_token = null; - row.refresh_token = null; - changes += 1; - } - return changes; - } - if (normalized.includes("set code_verifier")) { - const row = this.deviceRows.find((entry) => - entry.device_code === values[2] - && entry.status === "pending" - && entry.expires_at > Number(values[3]) - ); - if (!row) return 0; - row.code_verifier = String(values[0]); - row.oauth_state_hash = String(values[1]); - return 1; - } - if (normalized.includes("set oauth_state_hash = null")) { - const row = this.deviceRows.find((entry) => - entry.device_code === values[0] - && entry.status === "pending" - && entry.oauth_state_hash === values[1] - && entry.code_verifier !== null - && entry.expires_at > Number(values[2]) - ); - if (!row) return 0; - row.oauth_state_hash = null; - return 1; - } - if (normalized.includes("set status = 'approved'")) { - const row = this.deviceRows.find((entry) => - entry.device_code === values[5] - && entry.status === "pending" - && entry.expires_at > Number(values[6]) - ); - if (!row) return 0; - row.status = "approved"; - row.access_token = String(values[0]); - row.refresh_token = values[1] == null ? null : String(values[1]); - row.token_type = String(values[2]); - row.expires_in = Number(values[3]); - row.approved_at = Number(values[4]); - row.code_verifier = null; - row.oauth_state_hash = null; - return 1; - } - if (normalized.includes("set status = 'consumed'")) { - const row = this.deviceRows.find((entry) => - entry.device_code === values[1] - && entry.device_secret_hash === values[2] - && entry.status === "approved" - ); - if (!row) return 0; - row.status = "consumed"; - row.consumed_at = Number(values[0]); - row.access_token = null; - row.refresh_token = null; - return 1; - } - if (normalized.includes("set status = 'error'")) { - const row = this.deviceRows.find((entry) => entry.device_code === values[1] && entry.status === "pending"); - if (!row) return 0; - row.status = "error"; - row.error_message = String(values[0]); - return 1; - } - if (normalized.includes("set status = 'expired'")) { - const row = this.deviceRows.find((entry) => entry.device_code === values[0]); - const pendingOnly = /status\s*=\s*'pending'/.test(normalized); - const pendingOrApproved = /status\s+in\s*\(\s*'pending'\s*,\s*'approved'\s*\)/.test(normalized); - if ( - !row - || (pendingOnly && row.status !== "pending") - || (pendingOrApproved && row.status !== "pending" && row.status !== "approved") - ) return 0; - row.status = "expired"; - row.code_verifier = null; - row.oauth_state_hash = null; - row.access_token = null; - row.refresh_token = null; - return 1; - } - if (normalized.includes("set last_polled_at = ?, poll_interval_seconds = ?")) { - const row = this.deviceRows.find((entry) => entry.device_code === values[2]); - if (!row) return 0; - row.last_polled_at = Number(values[0]); - row.poll_interval_seconds = Number(values[1]); - return 1; - } - if (normalized.includes("set last_polled_at = ?")) { - const row = this.deviceRows.find((entry) => entry.device_code === values[1]); - if (!row) return 0; - row.last_polled_at = Number(values[0]); - return 1; - } - } - return 0; - } -} - -const ISSUER = "https://clerk.test"; -const OAUTH_CLIENT_ID = "client_ade"; -let jwksServer: Server; -let jwksUrl = ""; -let signingKey: Awaited>["privateKey"]; -let badSigningKey: Awaited>["privateKey"]; - -beforeAll(async () => { - const primary = await generateKeyPair("RS256", { extractable: true }); - const bad = await generateKeyPair("RS256", { extractable: true }); - signingKey = primary.privateKey; - badSigningKey = bad.privateKey; - const publicJwk = await exportJWK(primary.publicKey); - const jwks = { keys: [{ ...publicJwk, alg: "RS256", kid: "test-key", use: "sig" }] }; - - jwksServer = createServer((_request, response) => { - response.writeHead(200, { "content-type": "application/json" }); - response.end(JSON.stringify(jwks)); - }); - await new Promise((resolve, reject) => { - jwksServer.once("error", reject); - jwksServer.listen(0, "127.0.0.1", resolve); - }); - const address = jwksServer.address() as AddressInfo; - jwksUrl = `http://127.0.0.1:${address.port}/jwks`; -}); - -afterAll(async () => { - await new Promise((resolve, reject) => { - jwksServer.close((error) => error ? reject(error) : resolve()); - }); -}); - -afterEach(() => { - vi.restoreAllMocks(); -}); - -function makeEnv(overrides: Partial = {}): Env & { DB: FakeD1Database } { - return { - DB: new FakeD1Database(), - CLERK_JWKS_URL: jwksUrl, - CLERK_ISSUER: ISSUER, - CLERK_OAUTH_CLIENT_ID: OAUTH_CLIENT_ID, - PUSH_RELAY_URL: RELAY_URL, - DIRECTORY_AUTH_SECRET, - ...overrides, - } as unknown as Env & { DB: FakeD1Database }; -} - -const RELAY_URL = "https://relay.test"; -/** Shared with the push relay; proves a membership change came from here. */ -const DIRECTORY_AUTH_SECRET = "directory-shared-secret"; - -/** - * Stands in for the push relay so machine membership changes can be asserted - * without a network. Machine removal and re-pairing are the only routes that - * reach it, and both must report a relay failure rather than absorb it. - */ -function activityRelayStub( - respond: (url: string, init?: RequestInit) => Response = () => - new Response(JSON.stringify({ ok: true }), { status: 200 }), -): { - options: { activityRelay: { fetchImpl: typeof fetch; retryDelayMs: number } }; - calls: Array<{ - url: string; - method: string; - authorization: string | null; - directoryAuth: string | null; - }>; -} { - const calls: Array<{ - url: string; - method: string; - authorization: string | null; - directoryAuth: string | null; - }> = []; - const fetchImpl = (async (input: RequestInfo | URL, init?: RequestInit) => { - calls.push({ - url: String(input), - method: init?.method ?? "GET", - authorization: new Headers(init?.headers).get("authorization"), - directoryAuth: new Headers(init?.headers).get("x-ade-directory-auth"), - }); - return respond(String(input), init); - }) as typeof fetch; - return { options: { activityRelay: { fetchImpl, retryDelayMs: 0 } }, calls }; -} - -async function mintToken(args: { - sub?: string | null; - issuer?: string; - audience?: string | string[]; - azp?: string; - expired?: boolean; - useBadKey?: boolean; - /** Standard OIDC authentication time, in seconds since the epoch. */ - authTime?: number; - /** Clerk's factor-verification-age claim: [firstFactorMinutes, secondFactorMinutes]. */ - fva?: unknown; -} = {}): Promise { - const now = Math.floor(Date.now() / 1000); - let token = new SignJWT({ - ...(args.azp === undefined ? {} : { azp: args.azp }), - ...(args.authTime === undefined ? {} : { auth_time: args.authTime }), - ...(args.fva === undefined ? {} : { fva: args.fva }), - }) - .setProtectedHeader({ alg: "RS256", kid: "test-key" }) - .setIssuer(args.issuer ?? ISSUER) - .setIssuedAt(now) - .setExpirationTime(args.expired ? now - 60 : now + 600); - if (args.sub !== null) token = token.setSubject(args.sub ?? "user_1"); - if (args.audience !== undefined) token = token.setAudience(args.audience); - return token.sign(args.useBadKey ? badSigningKey : signingKey); -} - -/** - * A token that proves the user just signed in interactively — the only kind the - * directory accepts `pairing: true` on. `fva[0] = 0` is Clerk's "first factor - * verified within the last minute"; `-1` is "no second factor registered". - */ -async function mintFreshAuthToken(sub = "user_1"): Promise { - return mintToken({ sub, fva: [0, -1] }); -} - -function registerBody(machineKey: string, endpoints: unknown = [{ kind: "lan", host: "mac.local", port: 8787 }]) { - return { - machineKey, - deviceId: `device-${machineKey}`, - name: `Machine ${machineKey}`, - platform: "macOS", - deviceType: "desktop", - pubkey: `pubkey-${machineKey}`, - reachableEndpoints: endpoints, - }; -} - -function registrationWithRelayRetention(machineKey: string, endpoints: unknown) { - return { - ...registerBody(machineKey, endpoints), - retainRelayEndpoints: true, - }; -} - -function request( - method: string, - pathname: string, - token?: string, - body?: unknown, -): Request { - return new Request(`https://directory.test${pathname}`, { - method, - headers: { - ...(token ? { authorization: `Bearer ${token}` } : {}), - ...(body === undefined ? {} : { "content-type": "application/json" }), - }, - ...(body === undefined ? {} : { body: JSON.stringify(body) }), - }); -} - -function deviceConfirmationRequest( - userCode: string, - headers: Record = {}, -): Request { - return new Request("https://directory.test/device", { - method: "POST", - headers: { - "content-type": "application/x-www-form-urlencoded", - origin: "https://directory.test", - ...headers, - }, - body: new URLSearchParams({ user_code: userCode }), - }); -} - -async function register( - env: Env, - token: string, - machineKey: string, - endpoints?: unknown, -): Promise { - return handleRequest(request("POST", "/account/machines/register", token, registerBody(machineKey, endpoints)), env); -} +import { + activityRelayStub, + DIRECTORY_AUTH_SECRET, + deviceConfirmationRequest, + FakeD1Database, + ISSUER, + makeEnv, + mintFreshAuthToken, + mintToken, + OAUTH_CLIENT_ID, + register, + registerBody, + registrationWithRelayRetention, + RELAY_URL, + request, +} from "./helpers"; describe("Clerk JWKS authentication", () => { it("extracts sub from a valid OAuth token whose aud is the Clerk client id", async () => { @@ -734,431 +81,6 @@ describe("Clerk JWKS authentication", () => { }); }); -describe("device authorization bridge", () => { - it("creates, approves with Clerk OAuth + PKCE, and one-time redeems a secret-bound device code", async () => { - const env = makeEnv(); - let now = Date.parse("2026-07-14T12:00:00.000Z"); - const deviceSecret = "daemon-device-secret-with-at-least-32-bytes"; - const created = await handleRequest( - request("POST", "/device/code", undefined, { device_secret: deviceSecret }), - env, - { now: () => now }, - ); - expect(created.status).toBe(200); - const device = await created.json() as Record; - expect(device).toMatchObject({ - device_code: expect.any(String), - user_code: expect.stringMatching(/^[A-Z2-9]{4}-[A-Z2-9]{4}$/), - verification_uri: "https://directory.test/device", - verification_uri_complete: expect.stringContaining("https://directory.test/device?user_code="), - expires_in: 600, - interval: 5, - }); - - const pending = await handleRequest( - request("POST", "/device/token", undefined, { - device_code: device.device_code, - device_secret: deviceSecret, - }), - env, - { now: () => now }, - ); - expect(pending.status).toBe(400); - expect(await pending.json()).toEqual({ error: "authorization_pending", interval: 5 }); - - const approval = await handleRequest( - deviceConfirmationRequest(String(device.user_code)), - env, - { now: () => now }, - ); - expect(approval.status).toBe(302); - const clerkAuthorizeUrl = new URL(approval.headers.get("location")!); - expect(clerkAuthorizeUrl.origin + clerkAuthorizeUrl.pathname).toBe(`${ISSUER}/oauth/authorize`); - expect(clerkAuthorizeUrl.searchParams.get("client_id")).toBe(OAUTH_CLIENT_ID); - expect(clerkAuthorizeUrl.searchParams.get("code_challenge_method")).toBe("S256"); - expect(clerkAuthorizeUrl.searchParams.get("redirect_uri")).toBe("https://directory.test/device/callback"); - const state = clerkAuthorizeUrl.searchParams.get("state")!; - const duplicateApproval = await handleRequest( - deviceConfirmationRequest(String(device.user_code)), - env, - { now: () => now }, - ); - expect(duplicateApproval.status).toBe(409); - expect(await duplicateApproval.text()).toContain("Sign-in already started"); - const tokenExchange = vi.fn(async (_input: string, init?: RequestInit) => { - const body = Object.fromEntries(new URLSearchParams(String(init?.body))); - expect(body).toMatchObject({ - grant_type: "authorization_code", - code: "clerk-authorization-code", - client_id: OAUTH_CLIENT_ID, - redirect_uri: "https://directory.test/device/callback", - }); - expect(body.code_verifier).toEqual(expect.any(String)); - return new Response(JSON.stringify({ - access_token: "approved-access-token", - refresh_token: "approved-refresh-token", - token_type: "Bearer", - expires_in: 3600, - }), { status: 200, headers: { "content-type": "application/json" } }); - }); - const callback = await handleRequest( - new Request(`https://directory.test/device/callback?code=clerk-authorization-code&state=${encodeURIComponent(state)}`), - env, - { now: () => now, fetchImpl: tokenExchange as typeof fetch }, - ); - expect(callback.status).toBe(200); - expect(await callback.text()).toContain("Signed in to ADE"); - - const wrongSecret = await handleRequest( - request("POST", "/device/token", undefined, { - device_code: device.device_code, - device_secret: "wrong-device-secret-with-at-least-32-bytes", - }), - env, - { now: () => now }, - ); - expect(wrongSecret.status).toBe(401); - expect(await wrongSecret.json()).toEqual({ error: "invalid_grant" }); - - now += 6_000; - const redeemed = await handleRequest( - request("POST", "/device/token", undefined, { - device_code: device.device_code, - device_secret: deviceSecret, - }), - env, - { now: () => now }, - ); - expect(redeemed.status).toBe(200); - expect(await redeemed.json()).toEqual({ - access_token: "approved-access-token", - refresh_token: "approved-refresh-token", - token_type: "Bearer", - expires_in: 3594, - oauth_issuer: ISSUER, - oauth_client_id: OAUTH_CLIENT_ID, - }); - expect(env.DB.deviceRows[0]).toMatchObject({ - status: "consumed", - access_token: null, - refresh_token: null, - }); - - const replay = await handleRequest( - request("POST", "/device/token", undefined, { - device_code: device.device_code, - device_secret: deviceSecret, - }), - env, - { now: () => now + 6_000 }, - ); - expect(replay.status).toBe(401); - expect(await replay.json()).toEqual({ error: "invalid_grant" }); - - const expiredReplay = await handleRequest( - request("POST", "/device/token", undefined, { - device_code: device.device_code, - device_secret: deviceSecret, - }), - env, - { now: () => now + 595_000 }, - ); - expect(expiredReplay.status).toBe(400); - expect(await expiredReplay.json()).toEqual({ error: "expired" }); - expect(env.DB.deviceRows[0]?.status).toBe("consumed"); - }); - - it("claims concurrent duplicate callbacks before the one-time OAuth exchange", async () => { - const env = makeEnv(); - const now = Date.parse("2026-07-14T12:00:00.000Z"); - const created = await handleRequest( - request("POST", "/device/code", undefined, { - device_secret: "daemon-device-secret-with-at-least-32-bytes", - }), - env, - { now: () => now }, - ); - const device = await created.json() as Record; - const approval = await handleRequest( - deviceConfirmationRequest(String(device.user_code)), - env, - { now: () => now }, - ); - const state = new URL(approval.headers.get("location")!).searchParams.get("state")!; - const callbackUrl = `https://directory.test/device/callback?code=one-time-code&state=${encodeURIComponent(state)}`; - env.DB.synchronizeOAuthStateReads(2); - - let resolveSuccessfulExchange: ((response: Response) => void) | null = null; - const successfulExchange = new Promise((resolve) => { - resolveSuccessfulExchange = resolve; - }); - let exchangeCalls = 0; - const fetchImpl = vi.fn((): Promise => { - exchangeCalls += 1; - return exchangeCalls === 1 - ? successfulExchange - : Promise.resolve(new Response(JSON.stringify({ error: "invalid_grant" }), { - status: 400, - headers: { "content-type": "application/json" }, - })); - }); - - const callbacks = Promise.all([ - handleRequest(new Request(callbackUrl), env, { now: () => now, fetchImpl: fetchImpl as typeof fetch }), - handleRequest(new Request(callbackUrl), env, { now: () => now, fetchImpl: fetchImpl as typeof fetch }), - ]); - await vi.waitFor(() => expect(fetchImpl).toHaveBeenCalled()); - resolveSuccessfulExchange!(new Response(JSON.stringify({ - access_token: "winner-access-token", - refresh_token: "winner-refresh-token", - token_type: "Bearer", - expires_in: 3600, - }), { status: 200, headers: { "content-type": "application/json" } })); - const responses = await callbacks; - - expect(fetchImpl).toHaveBeenCalledTimes(1); - expect(responses.map((response) => response.status).sort()).toEqual([200, 409]); - expect(env.DB.deviceRows[0]).toMatchObject({ - status: "approved", - access_token: "winner-access-token", - refresh_token: "winner-refresh-token", - error_message: null, - code_verifier: null, - oauth_state_hash: null, - }); - }); - - it("keeps verification-link GET previews read-only until explicit confirmation", async () => { - const env = makeEnv(); - const now = Date.parse("2026-07-14T12:00:00.000Z"); - const created = await handleRequest( - request("POST", "/device/code", undefined, { - device_secret: "daemon-device-secret-with-at-least-32-bytes", - }), - env, - { now: () => now }, - ); - const device = await created.json() as Record; - const rowBeforePreview = { ...env.DB.deviceRows[0]! }; - const limitsBeforePreview = Array.from( - env.DB.approvalRateLimits, - ([key, value]) => [key, { ...value }] as const, - ); - - const firstPreview = await handleRequest( - new Request(String(device.verification_uri_complete)), - env, - { now: () => now }, - ); - const repeatedPreview = await handleRequest( - new Request(String(device.verification_uri_complete)), - env, - { now: () => now }, - ); - - expect([firstPreview.status, repeatedPreview.status]).toEqual([200, 200]); - expect(await firstPreview.text()).toContain('form method="post" action="/device"'); - expect(env.DB.deviceRows[0]).toEqual(rowBeforePreview); - expect(Array.from(env.DB.approvalRateLimits)).toEqual(limitsBeforePreview); - - const crossSiteSubmit = await handleRequest( - deviceConfirmationRequest(String(device.user_code), { origin: "https://preview.test" }), - env, - { now: () => now }, - ); - expect(crossSiteSubmit.status).toBe(403); - expect(env.DB.deviceRows[0]).toEqual(rowBeforePreview); - expect(Array.from(env.DB.approvalRateLimits)).toEqual(limitsBeforePreview); - - const confirmed = await handleRequest( - deviceConfirmationRequest(String(device.user_code)), - env, - { now: () => now }, - ); - expect(confirmed.status).toBe(302); - expect(env.DB.deviceRows[0]).toMatchObject({ - status: "pending", - code_verifier: expect.any(String), - oauth_state_hash: expect.any(String), - }); - expect(env.DB.approvalRateLimits.size).toBe(2); - }); - - it("returns expired for a device code after its short TTL", async () => { - const env = makeEnv(); - const startedAt = Date.parse("2026-07-14T12:00:00.000Z"); - const deviceSecret = "daemon-device-secret-with-at-least-32-bytes"; - const created = await handleRequest( - request("POST", "/device/code", undefined, { device_secret: deviceSecret }), - env, - { now: () => startedAt }, - ); - const device = await created.json() as Record; - - const expired = await handleRequest( - request("POST", "/device/token", undefined, { - device_code: device.device_code, - device_secret: deviceSecret, - }), - env, - { now: () => startedAt + 601_000 }, - ); - expect(expired.status).toBe(400); - expect(await expired.json()).toEqual({ error: "expired" }); - expect(env.DB.deviceRows[0]?.status).toBe("expired"); - }); - - it("clears expired approved credentials from the scheduled worker without client polling", async () => { - const env = makeEnv(); - const startedAt = Date.parse("2026-07-14T12:00:00.000Z"); - await handleRequest( - request("POST", "/device/code", undefined, { - device_secret: "daemon-device-secret-with-at-least-32-bytes", - }), - env, - { now: () => startedAt }, - ); - Object.assign(env.DB.deviceRows[0]!, { - status: "approved", - code_verifier: "temporary-pkce-verifier", - oauth_state_hash: "temporary-state-hash", - access_token: "abandoned-access-token", - refresh_token: "abandoned-refresh-token", - }); - vi.spyOn(Date, "now").mockReturnValue(startedAt + 601_000); - let cleanup: Promise | undefined; - - await worker.scheduled( - {} as ScheduledEvent, - env, - { waitUntil: (promise) => { cleanup = promise; } } as ExecutionContext, - ); - await cleanup; - - expect(env.DB.deviceRows[0]).toMatchObject({ - status: "expired", - code_verifier: null, - oauth_state_hash: null, - access_token: null, - refresh_token: null, - }); - expect(env.DB.approvalRateLimits.size).toBe(0); - - vi.mocked(Date.now).mockReturnValue(startedAt + 4_201_000); - cleanup = undefined; - await worker.scheduled( - {} as ScheduledEvent, - env, - { waitUntil: (promise) => { cleanup = promise; } } as ExecutionContext, - ); - await cleanup; - expect(env.DB.deviceRows).toHaveLength(0); - }); - - it("rate-limits device-code issuance separately from approval lookups", async () => { - const env = makeEnv(); - const now = Date.parse("2026-07-14T12:00:00.000Z"); - const headers = { - "cf-connecting-ip": "203.0.113.8", - "content-type": "application/json", - }; - for (let attempt = 0; attempt < 10; attempt += 1) { - const response = await handleRequest( - new Request("https://directory.test/device/code", { - method: "POST", - headers, - body: JSON.stringify({ device_secret: "daemon-device-secret-with-at-least-32-bytes" }), - }), - env, - { now: () => now }, - ); - expect(response.status).toBe(200); - } - - const blocked = await handleRequest( - new Request("https://directory.test/device/code", { - method: "POST", - headers, - body: JSON.stringify({ device_secret: "daemon-device-secret-with-at-least-32-bytes" }), - }), - env, - { now: () => now }, - ); - expect(blocked.status).toBe(429); - expect(blocked.headers.get("retry-after")).toBe("60"); - expect(env.DB.deviceRows).toHaveLength(10); - - const approval = await handleRequest( - deviceConfirmationRequest("ZZZZ-ZZZZ", { "cf-connecting-ip": "203.0.113.8" }), - env, - { now: () => now }, - ); - expect(approval.status).toBe(404); - expect(env.DB.approvalRateLimits.size).toBe(2); - expect(Array.from(env.DB.approvalRateLimits.values(), (entry) => entry.attempts).sort()).toEqual([1, 10]); - }); - - it("atomically admits at most ten concurrent device-code issuances per client", async () => { - const env = makeEnv(); - let now = Date.parse("2026-07-14T12:00:00.000Z"); - env.DB.synchronizeRateLimitReads(25); - const responses = await Promise.all(Array.from({ length: 25 }, () => handleRequest( - new Request("https://directory.test/device/code", { - method: "POST", - headers: { - "cf-connecting-ip": "203.0.113.9", - "content-type": "application/json", - }, - body: JSON.stringify({ device_secret: "daemon-device-secret-with-at-least-32-bytes" }), - }), - env, - { now: () => now }, - ))); - - expect(responses.filter((response) => response.status === 200)).toHaveLength(10); - expect(responses.filter((response) => response.status === 429)).toHaveLength(15); - expect(new Set(responses.map((response) => response.status))).toEqual(new Set([200, 429])); - expect(env.DB.deviceRows).toHaveLength(10); - expect(Array.from(env.DB.approvalRateLimits.values(), (entry) => entry.attempts)).toEqual([10]); - - now += 60_000; - const nextWindow = await handleRequest( - new Request("https://directory.test/device/code", { - method: "POST", - headers: { - "cf-connecting-ip": "203.0.113.9", - "content-type": "application/json", - }, - body: JSON.stringify({ device_secret: "daemon-device-secret-with-at-least-32-bytes" }), - }), - env, - { now: () => now }, - ); - expect(nextWindow.status).toBe(200); - expect(env.DB.deviceRows).toHaveLength(11); - expect(Array.from(env.DB.approvalRateLimits.values(), (entry) => entry.attempts)).toEqual([1]); - }); - - it("rate-limits user-code confirmations on the hosted approval page", async () => { - const env = makeEnv(); - const now = Date.parse("2026-07-14T12:00:00.000Z"); - for (let attempt = 0; attempt < 10; attempt += 1) { - const response = await handleRequest( - deviceConfirmationRequest("ABCD-EFGH", { "cf-connecting-ip": "203.0.113.7" }), - env, - { now: () => now }, - ); - expect(response.status).toBe(404); - } - const blocked = await handleRequest( - deviceConfirmationRequest("ABCD-EFGH", { "cf-connecting-ip": "203.0.113.7" }), - env, - { now: () => now }, - ); - expect(blocked.status).toBe(429); - }); -}); - describe("machine directory", () => { it("serves an unauthenticated health check", async () => { const response = await handleRequest(request("GET", "/health"), makeEnv()); @@ -1569,6 +491,7 @@ describe("machine directory", () => { user_id: "user_1", machine_key: `machine-${index}`, device_id: `device-${index}`, + hardware_id: null, name: `Machine ${index}`, custom_name: null, platform: "macOS", @@ -1925,81 +848,55 @@ describe("machine directory", () => { }); /** - * The second, independent proof that a `pairing: true` registration is backed - * by a human who just signed in. - * - * It exists because the first proof — an `auth_time`/`fva` claim on the - * caller's own token — fails CLOSED, and the ADE brain authenticates with a - * Clerk OAuth access token whose documented claim set contains neither. A - * claim-only gate would therefore risk making every account removal permanent. - * Every test here mints its tokens WITHOUT a freshness claim, so the grant is - * the only thing that can be doing the work. + * Every refusal here is a user who cannot get their computer back onto their + * account, and by the time they ask for help the request is long gone. These + * lines are the only window support has into WHY a call was turned away — the + * tables record what the state is, never that. */ -describe("device-login pairing grants", () => { - const DEVICE_SECRET = "daemon-device-secret-with-at-least-32-bytes"; - - /** - * Drive a real `/device/*` sign-in end to end and return the grant it hands - * back. Nothing here is faked past the Clerk token endpoint: the browser - * confirmation, the OAuth state round-trip, and the one-time redemption all - * run, because those steps are exactly what a removed machine cannot perform. - */ - async function completeDeviceLogin( - env: Env & { DB: FakeD1Database }, - args: { machineKey?: string; sub?: string; now?: number } = {}, - ): Promise<{ grant: string | null; accessToken: string }> { - const now = args.now ?? Date.now(); - const accessToken = await mintToken({ sub: args.sub ?? "user_1" }); - const created = await handleRequest( - request("POST", "/device/code", undefined, { - device_secret: DEVICE_SECRET, - ...(args.machineKey ? { machine_key: args.machineKey } : {}), - }), - env, - { now: () => now }, - ); - const device = await created.json() as Record; - const approval = await handleRequest( - deviceConfirmationRequest(String(device.user_code)), - env, - { now: () => now }, - ); - const state = new URL(approval.headers.get("location")!).searchParams.get("state")!; - const tokenExchange = (async () => new Response(JSON.stringify({ - access_token: accessToken, - refresh_token: "approved-refresh-token", - token_type: "Bearer", - expires_in: 3600, - }), { status: 200, headers: { "content-type": "application/json" } })) as typeof fetch; - await handleRequest( - new Request(`https://directory.test/device/callback?code=one-time-code&state=${encodeURIComponent(state)}`), - env, - { now: () => now, fetchImpl: tokenExchange }, - ); - const redeemed = await handleRequest( - request("POST", "/device/token", undefined, { - device_code: device.device_code, - device_secret: DEVICE_SECRET, +describe("refusal observability", () => { + const LONG_MACHINE_KEY = "machine-key-0123456789abcdef"; + const LONG_DEVICE_ID = "device-id-0123456789abcdef"; + const CORRELATION_ID = "3f1d9c4e-2b7a-4c8d-9e5f-6a1b2c3d4e5f"; + + /** Structured lines only; the request-lifecycle log shares the channel. */ + function captureRefusals(): Array> { + const lines: Array> = []; + vi.spyOn(console, "log").mockImplementation((value: unknown) => { + if (typeof value !== "string") return; + try { + const parsed = JSON.parse(value) as Record; + if (typeof parsed.event === "string") lines.push(parsed); + } catch { + // Not one of ours. + } + }); + return lines; + } + + function correlatedRegister(token: string, body: Record): Request { + return new Request("https://directory.test/account/machines/register", { + method: "POST", + headers: { + authorization: `Bearer ${token}`, + "content-type": "application/json", + "x-ade-correlation-id": CORRELATION_ID, + }, + body: JSON.stringify({ + ...registerBody(LONG_MACHINE_KEY), + deviceId: LONG_DEVICE_ID, + ...body, }), - env, - { now: () => now + 6_000 }, - ); - const payload = await redeemed.json() as Record; - return { - grant: typeof payload.pairing_grant === "string" ? payload.pairing_grant : null, - accessToken, - }; + }); } - /** Register `machine-a`, then remove it, leaving a live revocation. */ - async function removedMachine( + async function removedLongKeyMachine( env: Env & { DB: FakeD1Database }, token: string, ): Promise> { - await register(env, token, "machine-a"); + await handleRequest(correlatedRegister(token, {}), env); const relay = activityRelayStub(); await handleRequest( - request("DELETE", "/account/machines/machine-a", token), + request("DELETE", `/account/machines/${LONG_MACHINE_KEY}`, token), env, relay.options, ); @@ -2007,227 +904,112 @@ describe("device-login pairing grants", () => { return relay; } - function pairingRequest(token: string, body: Record): Request { - return request("POST", "/account/machines/register", token, { - ...registerBody("machine-a"), - pairing: true, - ...body, - }); - } - - it("mints a grant only for a device login that declared a machine key", async () => { - const withMachine = makeEnv(); - const withMachineResult = await completeDeviceLogin(withMachine, { machineKey: "machine-a" }); - expect(withMachineResult.grant).toEqual(expect.any(String)); - expect(withMachine.DB.pairingGrants).toHaveLength(1); - expect(withMachine.DB.pairingGrants[0]).toMatchObject({ - user_id: "user_1", - machine_key: "machine-a", - }); - // Only the hash is stored: a dump of this table must yield nothing spendable. - expect(withMachine.DB.pairingGrants[0]?.grant_hash).not.toBe(withMachineResult.grant); - - const withoutMachine = makeEnv(); - const withoutMachineResult = await completeDeviceLogin(withoutMachine); - expect(withoutMachineResult.grant).toBeNull(); - expect(withoutMachine.DB.pairingGrants).toEqual([]); - }); - - it("accepts a grant as proof for a machine whose token carries no freshness claim", async () => { + it("emits one joinable line per refused registration and never a whole key", async () => { const env = makeEnv(); const token = await mintToken({ sub: "user_1" }); - const relay = await removedMachine(env, token); - const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + const relay = await removedLongKeyMachine(env, token); + const lines = captureRefusals(); - // Same stale token that was just refused a moment ago — only the grant is new. - const staleWithoutGrant = await handleRequest(pairingRequest(token, {}), env, relay.options); - expect(staleWithoutGrant.status).toBe(403); - expect(await staleWithoutGrant.json()).toMatchObject({ - code: "pairing_authentication_required", - }); - expect(relay.calls).toEqual([]); + const heartbeat = await handleRequest(correlatedRegister(token, {}), env, relay.options); - const repaired = await handleRequest( - pairingRequest(token, { pairingGrant: grant }), - env, - relay.options, - ); - expect(repaired.status).toBe(200); - expect(env.DB.revocations).toEqual([]); - expect(relay.calls.at(-1)).toMatchObject({ - url: `${RELAY_URL}/attention/account/machines/machine-a/pairing`, - method: "POST", - }); + expect(heartbeat.status).toBe(403); + expect(lines).toEqual([{ + ts: expect.any(String), + svc: "ade-account-directory", + event: "directory.register_refused", + userId: "user_1", + machineKeyPrefix: "machine-", + deviceIdPrefix: "device-i", + code: "machine_revoked", + // Ties the line to the request the client already logged. + correlationId: CORRELATION_ID, + }]); + // Prefixes, not identifiers: a machine key is capability-shaped. + expect(JSON.stringify(lines)).not.toContain(LONG_MACHINE_KEY); + expect(JSON.stringify(lines)).not.toContain(LONG_DEVICE_ID); }); - it("refuses a replayed grant", async () => { + it("separates a missing proof from a rejected one", async () => { const env = makeEnv(); const token = await mintToken({ sub: "user_1" }); - const relay = await removedMachine(env, token); - const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + const relay = await removedLongKeyMachine(env, token); + const lines = captureRefusals(); - const first = await handleRequest( - pairingRequest(token, { pairingGrant: grant }), - env, - relay.options, - ); - expect(first.status).toBe(200); - expect(env.DB.pairingGrants).toEqual([]); - - // Remove it again, then try to re-pair with the grant that was already spent. + await handleRequest(correlatedRegister(token, { pairing: true }), env, relay.options); await handleRequest( - request("DELETE", "/account/machines/machine-a", token), - env, - relay.options, - ); - relay.calls.length = 0; - const replay = await handleRequest( - pairingRequest(token, { pairingGrant: grant }), + correlatedRegister(token, { pairing: true, pairingGrant: "not-a-real-grant" }), env, relay.options, ); - expect(replay.status).toBe(403); - expect(await replay.json()).toMatchObject({ code: "pairing_authentication_required" }); - expect(relay.calls).toEqual([]); - expect(env.DB.revocations).toHaveLength(1); - }); - it("refuses an expired grant", async () => { - const env = makeEnv(); - const token = await mintToken({ sub: "user_1" }); - const relay = await removedMachine(env, token); - const { grant } = await completeDeviceLogin(env, { - machineKey: "machine-a", - now: Date.now() - (PAIRING_GRANT_TTL_MS + 60_000), - }); - expect(grant).toEqual(expect.any(String)); - - const attempt = await handleRequest( - pairingRequest(token, { pairingGrant: grant }), - env, - relay.options, - ); - expect(attempt.status).toBe(403); - expect(await attempt.json()).toMatchObject({ code: "pairing_authentication_required" }); - expect(relay.calls).toEqual([]); - // Refused, not silently spent: an expired grant is not a usable credential - // for anyone, so leaving it for the cron sweep changes nothing. - expect(env.DB.revocations).toHaveLength(1); + // Same wire code both times — the contract is unchanged — but support's + // first question is always which of the two it was: a client that never + // presented a grant, or one whose grant was expired, replayed, or foreign. + expect(lines.map((line) => [line.code, line.reason])).toEqual([ + ["pairing_authentication_required", "no_proof"], + ["pairing_authentication_required", "grant_rejected"], + ]); + expect(JSON.stringify(lines)).not.toContain("not-a-real-grant"); }); - it("refuses a grant minted for a different machine key", async () => { + it("reports a relay outage on both the re-pair and the removal that needs it", async () => { const env = makeEnv(); const token = await mintToken({ sub: "user_1" }); - const relay = await removedMachine(env, token); - const { grant } = await completeDeviceLogin(env, { machineKey: "machine-b" }); + const relay = await removedLongKeyMachine(env, token); + const failing = activityRelayStub(() => new Response("down", { status: 503 })); + const lines = captureRefusals(); - const attempt = await handleRequest( - pairingRequest(token, { pairingGrant: grant }), + await handleRequest( + correlatedRegister(await mintFreshAuthToken("user_1"), { pairing: true }), env, - relay.options, + failing.options, ); - expect(attempt.status).toBe(403); - expect(await attempt.json()).toMatchObject({ code: "pairing_authentication_required" }); - expect(relay.calls).toEqual([]); - expect(env.DB.revocations).toHaveLength(1); - // Untouched: another machine's re-pair must still be able to spend it. - expect(env.DB.pairingGrants).toHaveLength(1); - }); - - it("refuses a grant minted for a different user", async () => { - const env = makeEnv(); - const token = await mintToken({ sub: "user_1" }); - const relay = await removedMachine(env, token); - // A second account signs in on a machine that happens to share the key. - const { grant } = await completeDeviceLogin(env, { - machineKey: "machine-a", - sub: "user_2", - }); + expect(lines).toEqual([expect.objectContaining({ + event: "directory.register_refused", + code: "activity_relay_unavailable", + reason: expect.stringContaining("503"), + })]); - const attempt = await handleRequest( - pairingRequest(token, { pairingGrant: grant }), + lines.length = 0; + // Put the machine back so there is a live row to remove, this time with a + // relay that answers. + await handleRequest( + correlatedRegister(await mintFreshAuthToken("user_1"), { pairing: true }), env, relay.options, ); - expect(attempt.status).toBe(403); - expect(await attempt.json()).toMatchObject({ code: "pairing_authentication_required" }); - expect(relay.calls).toEqual([]); - expect(env.DB.revocations).toHaveLength(1); - expect(env.DB.pairingGrants).toHaveLength(1); - }); - - it("refuses a forged grant and never lets one substitute for the pairing flag", async () => { - const env = makeEnv(); - const token = await mintToken({ sub: "user_1" }); - const relay = await removedMachine(env, token); - const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); - - const forged = await handleRequest( - pairingRequest(token, { pairingGrant: "not-a-real-grant" }), + await handleRequest( + request("DELETE", `/account/machines/${LONG_MACHINE_KEY}`, token), env, - relay.options, + failing.options, ); - expect(forged.status).toBe(403); - expect(await forged.json()).toMatchObject({ code: "pairing_authentication_required" }); - - // A valid grant on a plain heartbeat is still just a heartbeat: `pairing` - // is what declares intent, and its absence means the machine stays removed. - const heartbeat = await handleRequest(request( - "POST", - "/account/machines/register", - token, - { ...registerBody("machine-a"), pairingGrant: grant }, - ), env, relay.options); - expect(heartbeat.status).toBe(403); - expect(await heartbeat.json()).toMatchObject({ code: "machine_revoked" }); - expect(relay.calls).toEqual([]); - expect(env.DB.pairingGrants).toHaveLength(1); + expect(lines).toEqual([expect.objectContaining({ + event: "directory.remove_refused", + machineKeyPrefix: "machine-", + deviceIdPrefix: "device-i", + code: "activity_purge_failed", + })]); }); - it("does not spend a grant when the token already proves a fresh sign-in", async () => { + it("records a duplicate the caller could not prove its way past", async () => { const env = makeEnv(); const token = await mintToken({ sub: "user_1" }); - const relay = await removedMachine(env, token); - const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + await handleRequest(correlatedRegister(token, {}), env); + const lines = captureRefusals(); - const repaired = await handleRequest( - pairingRequest(await mintFreshAuthToken("user_1"), { pairingGrant: grant }), + const rotated = await handleRequest( + correlatedRegister(token, { machineKey: "machine-key-rotated" }), env, - relay.options, ); - expect(repaired.status).toBe(200); - // The claim is the fast path; the fallback must stay unspent behind it. - expect(env.DB.pairingGrants).toHaveLength(1); - }); - - it("tells the user what to do instead of only refusing", async () => { - const env = makeEnv(); - const token = await mintToken({ sub: "user_1" }); - const relay = await removedMachine(env, token); - - const refusal = await handleRequest(pairingRequest(token, {}), env, relay.options); - expect(refusal.status).toBe(403); - // This string is what the desktop's reconnect banner and `ade` both surface, - // so it has to name the action rather than describe the failure. - expect(await refusal.json()).toEqual({ - error: "Sign in again on this computer to reconnect it to your ADE account", - code: "pairing_authentication_required", - revokedAt: expect.any(Number), - }); - }); - it("sweeps expired grants and keeps live ones", async () => { - const env = makeEnv(); - const now = Date.now(); - await completeDeviceLogin(env, { machineKey: "machine-live", now }); - await completeDeviceLogin(env, { - machineKey: "machine-dead", - now: now - (PAIRING_GRANT_TTL_MS + 60_000), - }); - expect(env.DB.pairingGrants).toHaveLength(2); - - await expect(cleanupExpiredPairingGrants(env, now)).resolves.toBe(1); - expect(env.DB.pairingGrants.map((row) => row.machine_key)).toEqual(["machine-live"]); + // The registration itself succeeded — only the dedup was refused — and + // "why is my Mac listed twice" is the question this line answers. + expect(rotated.status).toBe(200); + expect(lines).toEqual([expect.objectContaining({ + event: "directory.supersede_refused", + code: "supersede_authentication_required", + reason: "duplicates=1", + })]); }); }); diff --git a/apps/account-directory/test/fakeD1.ts b/apps/account-directory/test/fakeD1.ts new file mode 100644 index 000000000..51fb598d2 --- /dev/null +++ b/apps/account-directory/test/fakeD1.ts @@ -0,0 +1,582 @@ +/** + * The fake D1 the whole account-directory suite runs on. + * + * It mirrors the SQL the Worker actually issues — predicates included — so a + * statement that narrows or widens shows up as a failing assertion rather than + * being absorbed. It lives on its own because it is the single largest thing in + * the harness and every suite that touches machines, revocations, device + * authorizations, or pairing grants needs it. + */ + +export type StoredMachine = { + user_id: string; + machine_key: string; + device_id: string | null; + /** Null for every row written before the anchor shipped, and never back-filled. */ + hardware_id: string | null; + name: string | null; + custom_name: string | null; + platform: string | null; + device_type: string | null; + pubkey: string | null; + reachable_endpoints: string | null; + power: string | null; + sleep_state: string | null; + sleep_state_at: number | null; + last_seen_at: number | null; + created_at: number | null; +}; + +export type StoredDeviceAuthorization = { + device_code: string; + user_code: string; + device_secret_hash: string; + machine_key: string | null; + status: "pending" | "approved" | "consumed" | "expired" | "error"; + code_verifier: string | null; + oauth_state_hash: string | null; + access_token: string | null; + refresh_token: string | null; + token_type: string | null; + expires_in: number | null; + error_message: string | null; + poll_interval_seconds: number; + last_polled_at: number | null; + created_at: number; + expires_at: number; + approved_at: number | null; + consumed_at: number | null; +}; + +export class FakeD1Statement { + private values: unknown[] = []; + + constructor( + private readonly sql: string, + private readonly db: FakeD1Database, + ) {} + + bind(...values: unknown[]): this { + this.values = values; + return this; + } + + async first(): Promise { + const result = this.db.first(this.sql, this.values); + await this.db.waitForConcurrentReads(this.sql); + return result; + } + + async all(): Promise<{ results: T[] }> { + return { results: this.db.all(this.sql, this.values) }; + } + + async run(): Promise<{ success: boolean; meta: { changes: number } }> { + const changes = this.db.run(this.sql, this.values); + return { success: true, meta: { changes } }; + } +} + +export type StoredRevocation = { + user_id: string; + machine_key: string; + device_id: string | null; + revoked_at: number; +}; + +export type StoredPairingGrant = { + grant_hash: string; + user_id: string; + machine_key: string; + created_at: number; + expires_at: number; + /** Non-null while one in-flight registration holds the grant. */ + reserved_at: number | null; +}; + +export class FakeD1Database { + rows: StoredMachine[] = []; + revocations: StoredRevocation[] = []; + deviceRows: StoredDeviceAuthorization[] = []; + pairingGrants: StoredPairingGrant[] = []; + approvalRateLimits = new Map(); + private rateLimitReadBarrier: { + remaining: number; + promise: Promise; + release: () => void; + } | null = null; + private oauthStateReadBarrier: { + remaining: number; + promise: Promise; + release: () => void; + } | null = null; + + synchronizeRateLimitReads(expectedReads: number): void { + let release = () => {}; + const promise = new Promise((resolve) => { + release = resolve; + }); + this.rateLimitReadBarrier = { remaining: expectedReads, promise, release }; + } + + synchronizeOAuthStateReads(expectedReads: number): void { + let release = () => {}; + const promise = new Promise((resolve) => { + release = resolve; + }); + this.oauthStateReadBarrier = { remaining: expectedReads, promise, release }; + } + + async waitForConcurrentReads(sql: string): Promise { + const normalized = sql.toLowerCase(); + const barrier = normalized.includes("from device_approval_rate_limits") + ? this.rateLimitReadBarrier + : normalized.includes("from device_authorizations") && normalized.includes("where oauth_state_hash") + ? this.oauthStateReadBarrier + : null; + if (!barrier) return; + barrier.remaining -= 1; + if (barrier.remaining === 0) barrier.release(); + await barrier.promise; + } + + prepare(sql: string): FakeD1Statement { + return new FakeD1Statement(sql, this); + } + + /** + * D1 runs a batch as one implicit transaction. The fake cannot fail halfway — + * that is the point of the source using a batch — so it only has to prove the + * statements were handed over TOGETHER: a caller that reverts to a loop of + * `.run()` calls stops going through here at all. + */ + async batch( + statements: FakeD1Statement[], + ): Promise> { + const results: Array<{ success: boolean; meta: { changes: number } }> = []; + for (const statement of statements) results.push(await statement.run()); + this.batchedStatementCounts.push(statements.length); + return results; + } + + /** One entry per `batch()` call, so a test can assert the deletes were not a loop. */ + batchedStatementCounts: number[] = []; + + first(sql: string, values: unknown[]): T | null { + const normalized = sql.toLowerCase(); + if (normalized.includes("from revoked_machines")) { + const [userId, machineKey] = values; + return (this.revocations.find((row) => + row.user_id === userId && row.machine_key === machineKey + ) ?? null) as T | null; + } + if (normalized.includes("from machines")) { + const [userId, machineKey] = values; + return (this.rows.find((row) => row.user_id === userId && row.machine_key === machineKey) ?? null) as T | null; + } + if (normalized.includes("from device_authorizations")) { + const [value] = values; + const key = normalized.includes("where device_code") + ? "device_code" + : normalized.includes("where user_code") + ? "user_code" + : "oauth_state_hash"; + return (this.deviceRows.find((row) => row[key] === value) ?? null) as T | null; + } + if (normalized.includes("from device_approval_rate_limits")) { + return (this.approvalRateLimits.get(String(values[0])) ?? null) as T | null; + } + return null; + } + + all(sql: string, values: unknown[]): T[] { + const normalized = sql.toLowerCase(); + if (!normalized.includes("from machines")) return []; + // Duplicate lookup for one physical device. Mirrors the source predicate + // exactly — including the `machine_key <> ?` self-exclusion and the OR over + // both identifiers — so a worker that drops either deletes the row it just + // wrote, or stops folding reinstalls, and the tests say so. A null bind + // matches nothing, exactly as SQL comparison to null does. + if (normalized.includes("device_id = ?")) { + const [userId, machineKey, deviceId, hardwareId, limit] = values; + return this.rows + .filter((row) => + row.user_id === userId + && row.machine_key !== machineKey + && ( + (deviceId != null && row.device_id === deviceId) + || (hardwareId != null && row.hardware_id === hardwareId) + ) + ) + .sort((left, right) => Number(left.last_seen_at ?? 0) - Number(right.last_seen_at ?? 0)) + .slice(0, Number(limit)) as T[]; + } + const [userId] = values; + let rows = this.rows.filter((row) => row.user_id === userId); + if (normalized.includes("order by last_seen_at desc")) { + rows = [...rows].sort((left, right) => + Number(right.last_seen_at ?? 0) - Number(left.last_seen_at ?? 0) + ); + } + if (normalized.includes("limit 500")) rows = rows.slice(0, 500); + return rows as T[]; + } + + run(sql: string, values: unknown[]): number { + const normalized = sql.toLowerCase(); + if (normalized.includes("insert into machine_pairing_grants")) { + const [grantHash, userId, machineKey, createdAt, expiresAt] = values; + if (this.pairingGrants.some((row) => row.grant_hash === grantHash)) return 0; + this.pairingGrants.push({ + grant_hash: String(grantHash), + user_id: String(userId), + machine_key: String(machineKey), + created_at: Number(createdAt), + expires_at: Number(expiresAt), + reserved_at: null, + }); + return 1; + } + if (normalized.includes("update machine_pairing_grants")) { + // Phase two, failure: put the row back exactly as it was. Only the + // reservation this request took may be cleared, and `expires_at` is not + // in the statement at all — an extended TTL would be the bug. + if (normalized.includes("set reserved_at = null")) { + const [grantHash, userId, machineKey, reservedAt] = values; + const row = this.pairingGrants.find((entry) => + entry.grant_hash === grantHash + && entry.user_id === userId + && entry.machine_key === machineKey + && entry.reserved_at === Number(reservedAt) + ); + if (!row) return 0; + row.reserved_at = null; + return 1; + } + // Phase one: the same single-statement guarantee the delete used to give. + // User, machine, and expiry are all in the WHERE clause, plus "not held + // by another in-flight registration", so a worker that loosens any of + // them fails here instead of being absorbed. + const [reservedAt, grantHash, userId, machineKey, nowMs, staleBefore] = values; + const row = this.pairingGrants.find((entry) => + entry.grant_hash === grantHash + && entry.user_id === userId + && entry.machine_key === machineKey + && entry.expires_at > Number(nowMs) + && (entry.reserved_at === null || entry.reserved_at <= Number(staleBefore)) + ); + if (!row) return 0; + row.reserved_at = Number(reservedAt); + return 1; + } + if (normalized.includes("delete from machine_pairing_grants")) { + // Phase two, success. Bound to the reservation this request took: a + // consume that dropped `reserved_at` from the predicate could destroy a + // grant another registration is holding. + if (normalized.includes("grant_hash = ?")) { + const [grantHash, userId, machineKey, reservedAt] = values; + const before = this.pairingGrants.length; + this.pairingGrants = this.pairingGrants.filter((row) => + !(row.grant_hash === grantHash + && row.user_id === userId + && row.machine_key === machineKey + && row.reserved_at === Number(reservedAt)) + ); + return before - this.pairingGrants.length; + } + const cutoff = Number(values[0]); + const before = this.pairingGrants.length; + this.pairingGrants = this.pairingGrants.filter((row) => row.expires_at > cutoff); + return before - this.pairingGrants.length; + } + if (normalized.includes("insert into revoked_machines")) { + const [userId, machineKey, deviceId, revokedAt] = values; + // Mirror whichever conflict clause the source actually uses, so a revert + // to a bare `device_id = excluded.device_id` fails the retry test rather + // than being papered over here. + const preservesDeviceId = normalized.includes("coalesce(excluded.device_id"); + const row = this.revocations.find((entry) => + entry.user_id === userId && entry.machine_key === machineKey + ); + if (row) { + const next = deviceId == null ? null : String(deviceId); + row.device_id = preservesDeviceId ? next ?? row.device_id : next; + row.revoked_at = Number(revokedAt); + return 1; + } + this.revocations.push({ + user_id: String(userId), + machine_key: String(machineKey), + device_id: deviceId == null ? null : String(deviceId), + revoked_at: Number(revokedAt), + }); + return 1; + } + if (normalized.includes("delete from revoked_machines")) { + const [userId, machineKey] = values; + const before = this.revocations.length; + this.revocations = this.revocations.filter((row) => + row.user_id !== userId || row.machine_key !== machineKey + ); + return before - this.revocations.length; + } + if (normalized.includes("insert into machines")) { + const retainRelayEndpoints = values[14] === 1; + const row: StoredMachine = { + user_id: String(values[0]), + machine_key: String(values[1]), + device_id: values[2] == null ? null : String(values[2]), + hardware_id: values[13] == null ? null : String(values[13]), + name: values[3] == null ? null : String(values[3]), + custom_name: null, + platform: values[4] == null ? null : String(values[4]), + device_type: values[5] == null ? null : String(values[5]), + pubkey: values[6] == null ? null : String(values[6]), + reachable_endpoints: values[7] == null ? null : String(values[7]), + power: values[8] == null ? null : String(values[8]), + sleep_state: values[9] == null ? null : String(values[9]), + sleep_state_at: values[10] == null ? null : Number(values[10]), + last_seen_at: values[11] == null ? null : Number(values[11]), + created_at: values[12] == null ? null : Number(values[12]), + }; + const existing = this.rows.find((entry) => + entry.user_id === row.user_id && entry.machine_key === row.machine_key + ); + if (existing) { + if (retainRelayEndpoints) { + const nextEndpoints = JSON.parse(row.reachable_endpoints ?? "[]") as Array<{ kind?: string }>; + const existingRelayEndpoints = ( + JSON.parse(existing.reachable_endpoints ?? "[]") as Array<{ kind?: string }> + ).filter((endpoint) => endpoint.kind === "relay"); + if ( + !nextEndpoints.some((endpoint) => endpoint.kind === "relay") + && existingRelayEndpoints.length > 0 + ) { + row.reachable_endpoints = JSON.stringify([ + ...nextEndpoints, + ...existingRelayEndpoints, + ]); + } + } + Object.assign(existing, row, { + created_at: existing.created_at, + custom_name: existing.custom_name, + // Mirrors `coalesce(excluded.hardware_id, machines.hardware_id)`: one + // heartbeat that could not read an anchor must not erase the one this + // row already has, or a single bad sample undoes the dedup. + hardware_id: row.hardware_id ?? existing.hardware_id, + // Mirror the source's `coalesce(excluded.x, machines.x)` exactly, so + // a revert to a bare overwrite fails the old-host test here rather + // than being absorbed by the fake. + power: row.power ?? existing.power, + sleep_state: row.sleep_state ?? existing.sleep_state, + sleep_state_at: row.sleep_state_at ?? existing.sleep_state_at, + }); + } else { + this.rows.push(row); + } + return 1; + } + if (normalized.includes("update machines") && normalized.includes("set custom_name")) { + const [customName, userId, machineKey] = values; + const row = this.rows.find((entry) => + entry.user_id === userId && entry.machine_key === machineKey + ); + if (!row) return 0; + // The supersede carry-forward only ever FILLS an empty name; the rename + // route carries no such predicate. Mirrored rather than ignored, so a + // carry-forward that lost the guard clobbers a user's rename here. + if (normalized.includes("custom_name is null") && row.custom_name !== null) return 0; + row.custom_name = customName == null ? null : String(customName); + return 1; + } + if (normalized.includes("delete from machines")) { + const [userId, machineKey, deviceId, hardwareId] = values; + // The supersede delete carries the identifiers it was authorized by; the + // removal delete does not. Honoring them here means a worker that narrows + // the predicate — dropping it entirely, or keeping only the device id and + // silently sparing every anchor-matched row — stops being covered. + const scopedToDevice = normalized.includes("device_id = ?"); + const matchesScope = (row: StoredMachine): boolean => + (deviceId != null && row.device_id === deviceId) + || (hardwareId != null && row.hardware_id === hardwareId); + const before = this.rows.length; + this.rows = this.rows.filter((row) => + row.user_id !== userId + || row.machine_key !== machineKey + || (scopedToDevice && !matchesScope(row)) + ); + return before - this.rows.length; + } + if (normalized.includes("delete from device_authorizations")) { + const cutoff = Number(values[0]); + const before = this.deviceRows.length; + this.deviceRows = this.deviceRows.filter((row) => + row.expires_at > cutoff || !["expired", "consumed", "error"].includes(row.status) + ); + return before - this.deviceRows.length; + } + if (normalized.includes("delete from device_approval_rate_limits")) { + const cutoff = Number(values[0]); + let changes = 0; + for (const [clientHash, record] of this.approvalRateLimits) { + if (record.window_started_at > cutoff) continue; + this.approvalRateLimits.delete(clientHash); + changes += 1; + } + return changes; + } + if (normalized.includes("insert into device_authorizations")) { + const userCode = String(values[1]); + if (this.deviceRows.some((row) => row.user_code === userCode)) { + throw new Error("UNIQUE constraint failed: device_authorizations.user_code"); + } + this.deviceRows.push({ + device_code: String(values[0]), + user_code: userCode, + device_secret_hash: String(values[2]), + machine_key: values[6] == null ? null : String(values[6]), + status: "pending", + code_verifier: null, + oauth_state_hash: null, + access_token: null, + refresh_token: null, + token_type: null, + expires_in: null, + error_message: null, + poll_interval_seconds: Number(values[3]), + last_polled_at: null, + created_at: Number(values[4]), + expires_at: Number(values[5]), + approved_at: null, + consumed_at: null, + }); + return 1; + } + if (normalized.includes("insert into device_approval_rate_limits")) { + const clientHash = String(values[0]); + const now = Number(values[1]); + const windowMs = Number(values[2]); + const maxAttempts = Number(values[5]); + const record = this.approvalRateLimits.get(clientHash); + if (!record) { + this.approvalRateLimits.set(clientHash, { window_started_at: now, attempts: 1 }); + return 1; + } + if (now - record.window_started_at >= windowMs) { + record.window_started_at = now; + record.attempts = 1; + return 1; + } + if (record.attempts >= maxAttempts) return 0; + record.attempts += 1; + return 1; + } + if (normalized.includes("update device_authorizations")) { + if (normalized.includes("where expires_at <= ?")) { + const now = Number(values[0]); + let changes = 0; + for (const row of this.deviceRows) { + if (row.expires_at > now || (row.status !== "pending" && row.status !== "approved")) continue; + row.status = "expired"; + row.code_verifier = null; + row.oauth_state_hash = null; + row.access_token = null; + row.refresh_token = null; + changes += 1; + } + return changes; + } + if (normalized.includes("set code_verifier")) { + const row = this.deviceRows.find((entry) => + entry.device_code === values[2] + && entry.status === "pending" + && entry.expires_at > Number(values[3]) + ); + if (!row) return 0; + row.code_verifier = String(values[0]); + row.oauth_state_hash = String(values[1]); + return 1; + } + if (normalized.includes("set oauth_state_hash = null")) { + const row = this.deviceRows.find((entry) => + entry.device_code === values[0] + && entry.status === "pending" + && entry.oauth_state_hash === values[1] + && entry.code_verifier !== null + && entry.expires_at > Number(values[2]) + ); + if (!row) return 0; + row.oauth_state_hash = null; + return 1; + } + if (normalized.includes("set status = 'approved'")) { + const row = this.deviceRows.find((entry) => + entry.device_code === values[5] + && entry.status === "pending" + && entry.expires_at > Number(values[6]) + ); + if (!row) return 0; + row.status = "approved"; + row.access_token = String(values[0]); + row.refresh_token = values[1] == null ? null : String(values[1]); + row.token_type = String(values[2]); + row.expires_in = Number(values[3]); + row.approved_at = Number(values[4]); + row.code_verifier = null; + row.oauth_state_hash = null; + return 1; + } + if (normalized.includes("set status = 'consumed'")) { + const row = this.deviceRows.find((entry) => + entry.device_code === values[1] + && entry.device_secret_hash === values[2] + && entry.status === "approved" + ); + if (!row) return 0; + row.status = "consumed"; + row.consumed_at = Number(values[0]); + row.access_token = null; + row.refresh_token = null; + return 1; + } + if (normalized.includes("set status = 'error'")) { + const row = this.deviceRows.find((entry) => entry.device_code === values[1] && entry.status === "pending"); + if (!row) return 0; + row.status = "error"; + row.error_message = String(values[0]); + return 1; + } + if (normalized.includes("set status = 'expired'")) { + const row = this.deviceRows.find((entry) => entry.device_code === values[0]); + const pendingOnly = /status\s*=\s*'pending'/.test(normalized); + const pendingOrApproved = /status\s+in\s*\(\s*'pending'\s*,\s*'approved'\s*\)/.test(normalized); + if ( + !row + || (pendingOnly && row.status !== "pending") + || (pendingOrApproved && row.status !== "pending" && row.status !== "approved") + ) return 0; + row.status = "expired"; + row.code_verifier = null; + row.oauth_state_hash = null; + row.access_token = null; + row.refresh_token = null; + return 1; + } + if (normalized.includes("set last_polled_at = ?, poll_interval_seconds = ?")) { + const row = this.deviceRows.find((entry) => entry.device_code === values[2]); + if (!row) return 0; + row.last_polled_at = Number(values[0]); + row.poll_interval_seconds = Number(values[1]); + return 1; + } + if (normalized.includes("set last_polled_at = ?")) { + const row = this.deviceRows.find((entry) => entry.device_code === values[1]); + if (!row) return 0; + row.last_polled_at = Number(values[0]); + return 1; + } + } + return 0; + } +} diff --git a/apps/account-directory/test/helpers.ts b/apps/account-directory/test/helpers.ts new file mode 100644 index 000000000..a64e5d887 --- /dev/null +++ b/apps/account-directory/test/helpers.ts @@ -0,0 +1,216 @@ +import { afterEach, vi } from "vitest"; +import { handleRequest, type Env } from "../src/directory"; +import { FakeD1Database } from "./fakeD1"; +import { ISSUER, jwksEndpoint, mintToken, OAUTH_CLIENT_ID } from "./jwks"; + +/** + * Shared harness for the suites that drive the directory Worker end to end. + * + * The two primitives every one of them needs live next door rather than here: + * the fake D1 in `./fakeD1` and the JWKS keypair in `./jwks`. What stays is the + * glue that is specific to THIS Worker — its env, its relay stub, and the + * request builders that speak its routes. + */ + +export { + FakeD1Database, + FakeD1Statement, + type StoredDeviceAuthorization, + type StoredMachine, + type StoredPairingGrant, + type StoredRevocation, +} from "./fakeD1"; +export { ISSUER, mintFreshAuthToken, mintToken, OAUTH_CLIENT_ID } from "./jwks"; + +afterEach(() => { + vi.restoreAllMocks(); +}); + +export function makeEnv(overrides: Partial = {}): Env & { DB: FakeD1Database } { + return { + DB: new FakeD1Database(), + CLERK_JWKS_URL: jwksEndpoint(), + CLERK_ISSUER: ISSUER, + CLERK_OAUTH_CLIENT_ID: OAUTH_CLIENT_ID, + PUSH_RELAY_URL: RELAY_URL, + DIRECTORY_AUTH_SECRET, + ...overrides, + } as unknown as Env & { DB: FakeD1Database }; +} + +export const RELAY_URL = "https://relay.test"; +/** Shared with the push relay; proves a membership change came from here. */ +export const DIRECTORY_AUTH_SECRET = "directory-shared-secret"; + +/** + * Stands in for the push relay so machine membership changes can be asserted + * without a network. Machine removal and re-pairing are the only routes that + * reach it, and both must report a relay failure rather than absorb it. + */ +export function activityRelayStub( + respond: (url: string, init?: RequestInit) => Response = () => + new Response(JSON.stringify({ ok: true }), { status: 200 }), +): { + options: { activityRelay: { fetchImpl: typeof fetch; retryDelayMs: number } }; + calls: Array<{ + url: string; + method: string; + authorization: string | null; + directoryAuth: string | null; + }>; +} { + const calls: Array<{ + url: string; + method: string; + authorization: string | null; + directoryAuth: string | null; + }> = []; + const fetchImpl = (async (input: RequestInfo | URL, init?: RequestInit) => { + calls.push({ + url: String(input), + method: init?.method ?? "GET", + authorization: new Headers(init?.headers).get("authorization"), + directoryAuth: new Headers(init?.headers).get("x-ade-directory-auth"), + }); + return respond(String(input), init); + }) as typeof fetch; + return { options: { activityRelay: { fetchImpl, retryDelayMs: 0 } }, calls }; +} + +export function registerBody(machineKey: string, endpoints: unknown = [{ kind: "lan", host: "mac.local", port: 8787 }]) { + return { + machineKey, + deviceId: `device-${machineKey}`, + name: `Machine ${machineKey}`, + platform: "macOS", + deviceType: "desktop", + pubkey: `pubkey-${machineKey}`, + reachableEndpoints: endpoints, + }; +} + +export function registrationWithRelayRetention(machineKey: string, endpoints: unknown) { + return { + ...registerBody(machineKey, endpoints), + retainRelayEndpoints: true, + }; +} + +export function request( + method: string, + pathname: string, + token?: string, + body?: unknown, +): Request { + return new Request(`https://directory.test${pathname}`, { + method, + headers: { + ...(token ? { authorization: `Bearer ${token}` } : {}), + ...(body === undefined ? {} : { "content-type": "application/json" }), + }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }), + }); +} + +export function deviceConfirmationRequest( + userCode: string, + headers: Record = {}, +): Request { + return new Request("https://directory.test/device", { + method: "POST", + headers: { + "content-type": "application/x-www-form-urlencoded", + origin: "https://directory.test", + ...headers, + }, + body: new URLSearchParams({ user_code: userCode }), + }); +} + +export async function register( + env: Env, + token: string, + machineKey: string, + endpoints?: unknown, +): Promise { + return handleRequest(request("POST", "/account/machines/register", token, registerBody(machineKey, endpoints)), env); +} + +export const DEVICE_SECRET = "daemon-device-secret-with-at-least-32-bytes"; + +/** + * Drive a real `/device/*` sign-in end to end and return the grant it hands + * back. Nothing here is faked past the Clerk token endpoint: the browser + * confirmation, the OAuth state round-trip, and the one-time redemption all + * run, because those steps are exactly what a removed machine cannot perform. + */ +export async function completeDeviceLogin( + env: Env & { DB: FakeD1Database }, + args: { machineKey?: string; sub?: string; now?: number } = {}, +): Promise<{ grant: string | null; accessToken: string }> { + const now = args.now ?? Date.now(); + const accessToken = await mintToken({ sub: args.sub ?? "user_1" }); + const created = await handleRequest( + request("POST", "/device/code", undefined, { + device_secret: DEVICE_SECRET, + ...(args.machineKey ? { machine_key: args.machineKey } : {}), + }), + env, + { now: () => now }, + ); + const device = await created.json() as Record; + const approval = await handleRequest( + deviceConfirmationRequest(String(device.user_code)), + env, + { now: () => now }, + ); + const state = new URL(approval.headers.get("location")!).searchParams.get("state")!; + const tokenExchange = (async () => new Response(JSON.stringify({ + access_token: accessToken, + refresh_token: "approved-refresh-token", + token_type: "Bearer", + expires_in: 3600, + }), { status: 200, headers: { "content-type": "application/json" } })) as typeof fetch; + await handleRequest( + new Request(`https://directory.test/device/callback?code=one-time-code&state=${encodeURIComponent(state)}`), + env, + { now: () => now, fetchImpl: tokenExchange }, + ); + const redeemed = await handleRequest( + request("POST", "/device/token", undefined, { + device_code: device.device_code, + device_secret: DEVICE_SECRET, + }), + env, + { now: () => now + 6_000 }, + ); + const payload = await redeemed.json() as Record; + return { + grant: typeof payload.pairing_grant === "string" ? payload.pairing_grant : null, + accessToken, + }; +} + +/** Register `machine-a`, then remove it, leaving a live revocation. */ +export async function removedMachine( + env: Env & { DB: FakeD1Database }, + token: string, +): Promise> { + await register(env, token, "machine-a"); + const relay = activityRelayStub(); + await handleRequest( + request("DELETE", "/account/machines/machine-a", token), + env, + relay.options, + ); + relay.calls.length = 0; + return relay; +} + +export function pairingRequest(token: string, body: Record): Request { + return request("POST", "/account/machines/register", token, { + ...registerBody("machine-a"), + pairing: true, + ...body, + }); +} diff --git a/apps/account-directory/test/jwks.ts b/apps/account-directory/test/jwks.ts new file mode 100644 index 000000000..836672887 --- /dev/null +++ b/apps/account-directory/test/jwks.ts @@ -0,0 +1,89 @@ +import { createServer, type Server } from "node:http"; +import type { AddressInfo } from "node:net"; +import { exportJWK, generateKeyPair, SignJWT } from "jose"; +import { afterAll, beforeAll } from "vitest"; + +/** + * One JWKS keypair and one loopback server per test file that needs to mint a + * Clerk-shaped token. + * + * Importing this module registers the `beforeAll`/`afterAll` that own the + * server, so a suite gets the keys by importing and nothing else. It carries + * NO env builder on purpose: the suites disagree about what an env is (a D1 + * fake here, an R2 fake there) and folding that in is what would force one + * suite to drag in the other's fixtures just to sign a token. + */ + +export const ISSUER = "https://clerk.test"; +export const OAUTH_CLIENT_ID = "client_ade"; + +let jwksServer: Server; +let jwksUrl = ""; +let signingKey: Awaited>["privateKey"]; +let badSigningKey: Awaited>["privateKey"]; + +beforeAll(async () => { + const primary = await generateKeyPair("RS256", { extractable: true }); + const bad = await generateKeyPair("RS256", { extractable: true }); + signingKey = primary.privateKey; + badSigningKey = bad.privateKey; + const publicJwk = await exportJWK(primary.publicKey); + const jwks = { keys: [{ ...publicJwk, alg: "RS256", kid: "test-key", use: "sig" }] }; + + jwksServer = createServer((_request, response) => { + response.writeHead(200, { "content-type": "application/json" }); + response.end(JSON.stringify(jwks)); + }); + await new Promise((resolve, reject) => { + jwksServer.once("error", reject); + jwksServer.listen(0, "127.0.0.1", resolve); + }); + jwksUrl = `http://127.0.0.1:${(jwksServer.address() as AddressInfo).port}/jwks`; +}); + +afterAll(async () => { + await new Promise((resolve, reject) => { + jwksServer.close((error) => (error ? reject(error) : resolve())); + }); +}); + +/** Read inside a test or an env builder — it is empty until `beforeAll` runs. */ +export function jwksEndpoint(): string { + return jwksUrl; +} + +export async function mintToken(args: { + sub?: string | null; + issuer?: string; + audience?: string | string[]; + azp?: string; + expired?: boolean; + useBadKey?: boolean; + /** Standard OIDC authentication time, in seconds since the epoch. */ + authTime?: number; + /** Clerk's factor-verification-age claim: [firstFactorMinutes, secondFactorMinutes]. */ + fva?: unknown; +} = {}): Promise { + const now = Math.floor(Date.now() / 1000); + let token = new SignJWT({ + ...(args.azp === undefined ? {} : { azp: args.azp }), + ...(args.authTime === undefined ? {} : { auth_time: args.authTime }), + ...(args.fva === undefined ? {} : { fva: args.fva }), + }) + .setProtectedHeader({ alg: "RS256", kid: "test-key" }) + .setIssuer(args.issuer ?? ISSUER) + .setIssuedAt(now) + .setExpirationTime(args.expired ? now - 60 : now + 600); + if (args.sub !== null) token = token.setSubject(args.sub ?? "user_1"); + if (args.audience !== undefined) token = token.setAudience(args.audience); + return token.sign(args.useBadKey ? badSigningKey : signingKey); +} + +/** + * A token that proves the user just signed in interactively — the only kind the + * directory accepts `pairing: true` on. `fva[0] = 0` is Clerk's "first factor + * verified within the last minute"; `-1` is "no second factor registered". + */ +export async function mintFreshAuthToken(sub = "user_1"): Promise { + return mintToken({ sub, fva: [0, -1] }); +} diff --git a/apps/account-directory/test/machineSupersede.test.ts b/apps/account-directory/test/machineSupersede.test.ts new file mode 100644 index 000000000..31b53e1b1 --- /dev/null +++ b/apps/account-directory/test/machineSupersede.test.ts @@ -0,0 +1,589 @@ +import { describe, expect, it } from "vitest"; +import { handleRequest, MAX_SUPERSEDED_MACHINES } from "../src/directory"; +import { + completeDeviceLogin, + makeEnv, + mintFreshAuthToken, + mintToken, + registerBody, + request, +} from "./helpers"; + +/** + * Phantom duplicates. A client that rotates its machine key — a reinstall, a + * wiped config directory, a restored backup — lands as a SECOND row for one + * physical computer, because rows are keyed `(user_id, machine_key)`. The user + * then removes the row that looks stale, and half the time that is the live + * install: the incident this whole change exists to stop. + */ +describe("device supersede", () => { + const DEVICE_ID = "device-macbook"; + + function registerForDevice( + machineKey: string, + body: Record = {}, + ): Record { + return { ...registerBody(machineKey), deviceId: DEVICE_ID, ...body }; + } + + it("folds a reinstalled machine's old key into the new one on a proven re-pair", async () => { + const env = makeEnv(); + const staleToken = await mintToken({ sub: "user_1" }); + await handleRequest( + request("POST", "/account/machines/register", staleToken, registerForDevice("machine-old")), + env, + ); + + // The reinstall: same computer, new identity file, and a human who just + // signed in to produce it. + const reinstalled = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerForDevice("machine-new"), + ), + env, + ); + + expect(reinstalled.status).toBe(200); + expect(await reinstalled.json()).toMatchObject({ + machineKey: "machine-new", + supersededMachineKeys: ["machine-old"], + }); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + // No revocation for the superseded key. The physical device holds the new + // one; blocking the old would trapdoor any client that rolls its identity + // file back into a permanent refusal. + expect(env.DB.revocations).toEqual([]); + }); + + it("carries the name the user typed onto the row that replaces it", async () => { + const env = makeEnv(); + const staleToken = await mintToken({ sub: "user_1" }); + await handleRequest( + request("POST", "/account/machines/register", staleToken, registerForDevice("machine-old")), + env, + ); + await handleRequest( + request("PATCH", "/account/machines/machine-old", staleToken, { customName: "Studio Mac" }), + env, + ); + + const reinstalled = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerForDevice("machine-new"), + ), + env, + ); + + // A rename is a deliberate act the client cannot reconstruct, so folding + // the row without it silently renames the user's computer back to its + // hostname — the reinstall looks like it lost something, because it did. + expect(await reinstalled.json()).toMatchObject({ + machineKey: "machine-new", + customName: "Studio Mac", + supersededMachineKeys: ["machine-old"], + }); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + }); + + it("never overwrites a name already set on the surviving row", async () => { + const env = makeEnv(); + const staleToken = await mintToken({ sub: "user_1" }); + for (const machineKey of ["machine-old", "machine-new"]) { + await handleRequest( + request("POST", "/account/machines/register", staleToken, registerForDevice(machineKey)), + env, + ); + } + await handleRequest( + request("PATCH", "/account/machines/machine-old", staleToken, { customName: "Old Name" }), + env, + ); + await handleRequest( + request("PATCH", "/account/machines/machine-new", staleToken, { customName: "New Name" }), + env, + ); + + const reinstalled = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerForDevice("machine-new"), + ), + env, + ); + + // The name on the surviving row is the fresher statement of intent; the + // carry-forward only ever fills an empty one. + expect(await reinstalled.json()).toMatchObject({ + customName: "New Name", + supersededMachineKeys: ["machine-old"], + }); + }); + + it("takes the newest name when several superseded rows carry one", async () => { + const env = makeEnv(); + const staleToken = await mintToken({ sub: "user_1" }); + for (const machineKey of ["machine-older", "machine-newer"]) { + await handleRequest( + request("POST", "/account/machines/register", staleToken, registerForDevice(machineKey)), + env, + ); + await handleRequest( + request("PATCH", `/account/machines/${machineKey}`, staleToken, { + customName: `name of ${machineKey}`, + }), + env, + ); + } + + const reinstalled = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerForDevice("machine-new"), + ), + env, + ); + + // Rows arrive oldest-seen first, so the most recently used phantom is the + // one whose name the user still recognizes. + expect(await reinstalled.json()).toMatchObject({ customName: "name of machine-newer" }); + }); + + it("deletes every superseded row in one batch, name carry-forward included", async () => { + const env = makeEnv(); + const staleToken = await mintToken({ sub: "user_1" }); + for (let index = 0; index < 3; index += 1) { + await handleRequest( + request("POST", "/account/machines/register", staleToken, registerForDevice(`machine-old-${index}`)), + env, + ); + } + await handleRequest( + request("PATCH", "/account/machines/machine-old-0", staleToken, { customName: "Studio Mac" }), + env, + ); + + await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerForDevice("machine-new"), + ), + env, + ); + + // The grant is already spent by the time these run, so a sequential loop + // that failed partway would leave half the phantoms deleted with no + // credential left to finish the job. One batch, one transaction. + expect(env.DB.batchedStatementCounts).toEqual([4]); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + }); + + it("leaves duplicates alone for a plain-token register", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + await handleRequest( + request("POST", "/account/machines/register", token, registerForDevice("machine-old")), + env, + ); + + // `deviceId` is caller-supplied. Without proof of a fresh human, honoring + // it would let any machine name another's device id and delete its row. + const rotated = await handleRequest( + request("POST", "/account/machines/register", token, registerForDevice("machine-new", { + // Even asserting intent changes nothing: the flag is client-supplied too. + pairing: true, + })), + env, + ); + + expect(rotated.status).toBe(200); + expect(await rotated.json()).not.toHaveProperty("supersededMachineKeys"); + expect(env.DB.rows.map((row) => row.machine_key).sort()).toEqual(["machine-new", "machine-old"]); + }); + + it("supersedes on a grant-backed pairing register and spends the grant exactly once", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + await handleRequest( + request("POST", "/account/machines/register", token, registerForDevice("machine-old")), + env, + ); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-new" }); + + const rotated = await handleRequest( + request("POST", "/account/machines/register", token, registerForDevice("machine-new", { + pairing: true, + pairingGrant: grant, + })), + env, + ); + + expect(rotated.status).toBe(200); + expect(await rotated.json()).toMatchObject({ supersededMachineKeys: ["machine-old"] }); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + expect(env.DB.pairingGrants).toEqual([]); + }); + + it("never supersedes across accounts", async () => { + const env = makeEnv(); + const other = await mintToken({ sub: "user_2" }); + await handleRequest( + request("POST", "/account/machines/register", other, registerForDevice("machine-old")), + env, + ); + + const mine = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerForDevice("machine-new"), + ), + env, + ); + + expect(await mine.json()).not.toHaveProperty("supersededMachineKeys"); + expect(env.DB.rows.map((row) => row.machine_key).sort()).toEqual(["machine-new", "machine-old"]); + }); + + it("supersedes at most five rows per call and finishes the job on the next one", async () => { + const env = makeEnv(); + const staleToken = await mintToken({ sub: "user_1" }); + for (let index = 0; index < 7; index += 1) { + await handleRequest( + request("POST", "/account/machines/register", staleToken, registerForDevice(`machine-old-${index}`)), + env, + ); + } + + const first = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerForDevice("machine-new"), + ), + env, + ); + + // One request must never delete an unbounded slice of the roster. + expect(((await first.json()) as { supersededMachineKeys: string[] }).supersededMachineKeys) + .toHaveLength(MAX_SUPERSEDED_MACHINES); + expect(env.DB.rows).toHaveLength(3); + + const second = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerForDevice("machine-new"), + ), + env, + ); + expect(((await second.json()) as { supersededMachineKeys: string[] }).supersededMachineKeys) + .toHaveLength(2); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + }); +}); + +/** + * The other half of the reinstall story, and the half device-supersede could + * not reach. + * + * `sync-device-id` lives in `~/.ade/secrets` next to the machine key, so the + * user who follows the oldest support instruction there is — delete `~/.ade`, + * install again, sign in — arrives with BOTH identifiers freshly minted and + * nothing for the directory to match. `hardware_id` is derived from the machine + * itself and is the same on the other side of that wipe. + * + * It is caller-supplied like the device id, so it is gated by the identical + * proof and nothing here relaxes that. + */ +describe("hardware anchor supersede", () => { + // What a client actually sends: sha256 hex, salted per account, so these two + // stand for the same computer seen by two different accounts. + const ANCHOR = "a".repeat(64); + const OTHER_ACCOUNT_ANCHOR = "b".repeat(64); + + function registerWithAnchor( + machineKey: string, + body: Record = {}, + ): Record { + return { ...registerBody(machineKey), hardwareId: ANCHOR, ...body }; + } + + it("persists the anchor and never lets a blank heartbeat erase it", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + + await handleRequest( + request("POST", "/account/machines/register", token, registerWithAnchor("machine-a")), + env, + ); + expect(env.DB.rows[0]?.hardware_id).toBe(ANCHOR); + + // A host that momentarily cannot read its identifier still heartbeats. The + // stored anchor is what a later reinstall matches on, so one bad sample + // must not cost the row its only durable identity. + const blank = await handleRequest( + request("POST", "/account/machines/register", token, registerBody("machine-a")), + env, + ); + expect(blank.status).toBe(200); + expect(env.DB.rows[0]?.hardware_id).toBe(ANCHOR); + // Nor is it echoed back: the roster has no use for it and the response is + // the one place it could leak to another surface. + expect(await blank.json()).not.toHaveProperty("hardwareId"); + }); + + it("folds a wiped install's row in when the device id did not survive either", async () => { + const env = makeEnv(); + await handleRequest( + request( + "POST", + "/account/machines/register", + await mintToken({ sub: "user_1" }), + registerWithAnchor("machine-old", { deviceId: "device-before-the-wipe" }), + ), + env, + ); + + // `rm -rf ~/.ade`, reinstall, sign in: new machine key, NEW DEVICE ID, and + // the same computer underneath. Device-supersede alone matches nothing here. + const reinstalled = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerWithAnchor("machine-new", { deviceId: "device-after-the-wipe" }), + ), + env, + ); + + expect(reinstalled.status).toBe(200); + expect(await reinstalled.json()).toMatchObject({ + machineKey: "machine-new", + supersededMachineKeys: ["machine-old"], + }); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + expect(env.DB.revocations).toEqual([]); + }); + + it("refuses an anchor match on a plain token", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + await handleRequest( + request("POST", "/account/machines/register", token, registerWithAnchor("machine-old", { + deviceId: "device-before-the-wipe", + })), + env, + ); + + // An anchor is no more attested than a device id. Honoring one without a + // proven-fresh human would let any machine claim another's hardware and + // delete its row. + const rotated = await handleRequest( + request("POST", "/account/machines/register", token, registerWithAnchor("machine-new", { + deviceId: "device-after-the-wipe", + pairing: true, + })), + env, + ); + + expect(rotated.status).toBe(200); + expect(await rotated.json()).not.toHaveProperty("supersededMachineKeys"); + expect(env.DB.rows.map((row) => row.machine_key).sort()).toEqual(["machine-new", "machine-old"]); + }); + + it("supersedes an anchor match on a grant-backed pairing register", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + await handleRequest( + request("POST", "/account/machines/register", token, registerWithAnchor("machine-old", { + deviceId: "device-before-the-wipe", + })), + env, + ); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-new" }); + + const rotated = await handleRequest( + request("POST", "/account/machines/register", token, registerWithAnchor("machine-new", { + deviceId: "device-after-the-wipe", + pairing: true, + pairingGrant: grant, + })), + env, + ); + + expect(rotated.status).toBe(200); + expect(await rotated.json()).toMatchObject({ supersededMachineKeys: ["machine-old"] }); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + expect(env.DB.pairingGrants).toEqual([]); + }); + + it("still folds a null-anchor row in through the device id", async () => { + const env = makeEnv(); + // Written before this shipped: no anchor at all, and nothing back-fills one. + await handleRequest( + request( + "POST", + "/account/machines/register", + await mintToken({ sub: "user_1" }), + { ...registerBody("machine-legacy"), deviceId: "device-macbook" }, + ), + env, + ); + expect(env.DB.rows[0]?.hardware_id).toBeNull(); + + // An in-place reinstall keeps `~/.ade/secrets`, so the device id is still + // the identifier that matches. The anchor path must not have cost it that. + const reinstalled = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerWithAnchor("machine-new", { deviceId: "device-macbook" }), + ), + env, + ); + + expect(await reinstalled.json()).toMatchObject({ supersededMachineKeys: ["machine-legacy"] }); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + }); + + it("never matches an anchor across accounts", async () => { + const env = makeEnv(); + // The salt makes this impossible to produce in the field; the query being + // user-scoped is what makes it impossible even if it were produced. + await handleRequest( + request( + "POST", + "/account/machines/register", + await mintToken({ sub: "user_2" }), + registerWithAnchor("machine-theirs", { deviceId: "device-theirs" }), + ), + env, + ); + + const mine = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerWithAnchor("machine-mine", { deviceId: "device-mine" }), + ), + env, + ); + + expect(await mine.json()).not.toHaveProperty("supersededMachineKeys"); + expect(env.DB.rows.map((row) => row.machine_key).sort()) + .toEqual(["machine-mine", "machine-theirs"]); + }); + + it("leaves another account's row alone even when it holds a different anchor", async () => { + const env = makeEnv(); + await handleRequest( + request( + "POST", + "/account/machines/register", + await mintToken({ sub: "user_2" }), + registerWithAnchor("machine-theirs", { + deviceId: "device-macbook", + hardwareId: OTHER_ACCOUNT_ANCHOR, + }), + ), + env, + ); + + // Same physical machine, second account: the device id is genuinely shared, + // and the row still belongs to somebody else. + const mine = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerWithAnchor("machine-mine", { deviceId: "device-macbook" }), + ), + env, + ); + + expect(await mine.json()).not.toHaveProperty("supersededMachineKeys"); + expect(env.DB.rows).toHaveLength(2); + }); + + it("caps the union of both identifiers at five rows per call", async () => { + const env = makeEnv(); + const staleToken = await mintToken({ sub: "user_1" }); + // Three phantoms reachable only by device id, three only by anchor: the cap + // has to bind across the union, not once per identifier. + for (let index = 0; index < 3; index += 1) { + await handleRequest( + request("POST", "/account/machines/register", staleToken, registerWithAnchor(`machine-device-${index}`, { + deviceId: "device-macbook", + hardwareId: null, + })), + env, + ); + await handleRequest( + request("POST", "/account/machines/register", staleToken, registerWithAnchor(`machine-anchor-${index}`, { + deviceId: `device-wiped-${index}`, + })), + env, + ); + } + + const first = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerWithAnchor("machine-new", { deviceId: "device-macbook" }), + ), + env, + ); + + expect(((await first.json()) as { supersededMachineKeys: string[] }).supersededMachineKeys) + .toHaveLength(MAX_SUPERSEDED_MACHINES); + expect(env.DB.rows).toHaveLength(2); + + const second = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintFreshAuthToken("user_1"), + registerWithAnchor("machine-new", { deviceId: "device-macbook" }), + ), + env, + ); + expect(((await second.json()) as { supersededMachineKeys: string[] }).supersededMachineKeys) + .toHaveLength(1); + expect(env.DB.rows.map((row) => row.machine_key)).toEqual(["machine-new"]); + }); + + it("rejects an oversized anchor rather than storing it", async () => { + const env = makeEnv(); + const response = await handleRequest( + request( + "POST", + "/account/machines/register", + await mintToken({ sub: "user_1" }), + registerWithAnchor("machine-a", { hardwareId: "x".repeat(129) }), + ), + env, + ); + + expect(response.status).toBe(400); + expect(env.DB.rows).toEqual([]); + }); +}); diff --git a/apps/account-directory/test/pairingGrants.test.ts b/apps/account-directory/test/pairingGrants.test.ts new file mode 100644 index 000000000..93819cad4 --- /dev/null +++ b/apps/account-directory/test/pairingGrants.test.ts @@ -0,0 +1,328 @@ +import { describe, expect, it } from "vitest"; +import { handleRequest } from "../src/directory"; +import { + cleanupExpiredPairingGrants, + PAIRING_GRANT_RESERVATION_MS, + PAIRING_GRANT_TTL_MS, +} from "../src/pairingGrants"; +import { + activityRelayStub, + completeDeviceLogin, + makeEnv, + mintFreshAuthToken, + mintToken, + pairingRequest, + registerBody, + RELAY_URL, + removedMachine, + request, +} from "./helpers"; + +/** + * The second, independent proof that a `pairing: true` registration is backed + * by a human who just signed in. + * + * It exists because the first proof — an `auth_time`/`fva` claim on the + * caller's own token — fails CLOSED, and the ADE brain authenticates with a + * Clerk OAuth access token whose documented claim set contains neither. A + * claim-only gate would therefore risk making every account removal permanent. + * Every test here mints its tokens WITHOUT a freshness claim, so the grant is + * the only thing that can be doing the work. + */ +describe("device-login pairing grants", () => { + it("mints a grant only for a device login that declared a machine key", async () => { + const withMachine = makeEnv(); + const withMachineResult = await completeDeviceLogin(withMachine, { machineKey: "machine-a" }); + expect(withMachineResult.grant).toEqual(expect.any(String)); + expect(withMachine.DB.pairingGrants).toHaveLength(1); + expect(withMachine.DB.pairingGrants[0]).toMatchObject({ + user_id: "user_1", + machine_key: "machine-a", + }); + // Only the hash is stored: a dump of this table must yield nothing spendable. + expect(withMachine.DB.pairingGrants[0]?.grant_hash).not.toBe(withMachineResult.grant); + + const withoutMachine = makeEnv(); + const withoutMachineResult = await completeDeviceLogin(withoutMachine); + expect(withoutMachineResult.grant).toBeNull(); + expect(withoutMachine.DB.pairingGrants).toEqual([]); + }); + + it("accepts a grant as proof for a machine whose token carries no freshness claim", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + + // Same stale token that was just refused a moment ago — only the grant is new. + const staleWithoutGrant = await handleRequest(pairingRequest(token, {}), env, relay.options); + expect(staleWithoutGrant.status).toBe(403); + expect(await staleWithoutGrant.json()).toMatchObject({ + code: "pairing_authentication_required", + }); + expect(relay.calls).toEqual([]); + + const repaired = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(repaired.status).toBe(200); + expect(env.DB.revocations).toEqual([]); + expect(relay.calls.at(-1)).toMatchObject({ + url: `${RELAY_URL}/attention/account/machines/machine-a/pairing`, + method: "POST", + }); + }); + + it("refuses a replayed grant", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + + const first = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(first.status).toBe(200); + expect(env.DB.pairingGrants).toEqual([]); + + // Remove it again, then try to re-pair with the grant that was already spent. + await handleRequest( + request("DELETE", "/account/machines/machine-a", token), + env, + relay.options, + ); + relay.calls.length = 0; + const replay = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(replay.status).toBe(403); + expect(await replay.json()).toMatchObject({ code: "pairing_authentication_required" }); + expect(relay.calls).toEqual([]); + expect(env.DB.revocations).toHaveLength(1); + }); + + it("refuses an expired grant", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + const { grant } = await completeDeviceLogin(env, { + machineKey: "machine-a", + now: Date.now() - (PAIRING_GRANT_TTL_MS + 60_000), + }); + expect(grant).toEqual(expect.any(String)); + + const attempt = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(attempt.status).toBe(403); + expect(await attempt.json()).toMatchObject({ code: "pairing_authentication_required" }); + expect(relay.calls).toEqual([]); + // Refused, not silently spent: an expired grant is not a usable credential + // for anyone, so leaving it for the cron sweep changes nothing. + expect(env.DB.revocations).toHaveLength(1); + }); + + it("refuses a grant minted for a different machine key", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-b" }); + + const attempt = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(attempt.status).toBe(403); + expect(await attempt.json()).toMatchObject({ code: "pairing_authentication_required" }); + expect(relay.calls).toEqual([]); + expect(env.DB.revocations).toHaveLength(1); + // Untouched: another machine's re-pair must still be able to spend it. + expect(env.DB.pairingGrants).toHaveLength(1); + }); + + it("refuses a grant minted for a different user", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + // A second account signs in on a machine that happens to share the key. + const { grant } = await completeDeviceLogin(env, { + machineKey: "machine-a", + sub: "user_2", + }); + + const attempt = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(attempt.status).toBe(403); + expect(await attempt.json()).toMatchObject({ code: "pairing_authentication_required" }); + expect(relay.calls).toEqual([]); + expect(env.DB.revocations).toHaveLength(1); + expect(env.DB.pairingGrants).toHaveLength(1); + }); + + it("refuses a forged grant and never lets one substitute for the pairing flag", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + + const forged = await handleRequest( + pairingRequest(token, { pairingGrant: "not-a-real-grant" }), + env, + relay.options, + ); + expect(forged.status).toBe(403); + expect(await forged.json()).toMatchObject({ code: "pairing_authentication_required" }); + + // A valid grant on a plain heartbeat is still just a heartbeat: `pairing` + // is what declares intent, and its absence means the machine stays removed. + const heartbeat = await handleRequest(request( + "POST", + "/account/machines/register", + token, + { ...registerBody("machine-a"), pairingGrant: grant }, + ), env, relay.options); + expect(heartbeat.status).toBe(403); + expect(await heartbeat.json()).toMatchObject({ code: "machine_revoked" }); + expect(relay.calls).toEqual([]); + expect(env.DB.pairingGrants).toHaveLength(1); + }); + + it("does not spend a grant when the token already proves a fresh sign-in", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + + const repaired = await handleRequest( + pairingRequest(await mintFreshAuthToken("user_1"), { pairingGrant: grant }), + env, + relay.options, + ); + expect(repaired.status).toBe(200); + // The claim is the fast path; the fallback must stay unspent behind it. + expect(env.DB.pairingGrants).toHaveLength(1); + }); + + it("tells the user what to do instead of only refusing", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + + const refusal = await handleRequest(pairingRequest(token, {}), env, relay.options); + expect(refusal.status).toBe(403); + // This string is what the desktop's reconnect banner and `ade` both surface, + // so it has to name the action rather than describe the failure. + expect(await refusal.json()).toEqual({ + error: "Sign in again on this computer to reconnect it to your ADE account", + code: "pairing_authentication_required", + revokedAt: expect.any(Number), + }); + }); + + it("sweeps expired grants and keeps live ones", async () => { + const env = makeEnv(); + const now = Date.now(); + await completeDeviceLogin(env, { machineKey: "machine-live", now }); + await completeDeviceLogin(env, { + machineKey: "machine-dead", + now: now - (PAIRING_GRANT_TTL_MS + 60_000), + }); + expect(env.DB.pairingGrants).toHaveLength(2); + + await expect(cleanupExpiredPairingGrants(env, now)).resolves.toBe(1); + expect(env.DB.pairingGrants.map((row) => row.machine_key)).toEqual(["machine-live"]); + }); +}); + +/** + * A grant is the ONLY way back for a machine whose token carries no freshness + * claim, so destroying one on a failure the user did not cause is the same + * lockout the grant was introduced to prevent. Redemption is therefore two + * phases — reserve, then consume or release — and these tests hold both the + * safety property (single-use survives) and the liveness property (an outage + * costs a retry, not a credential) at once. + */ +describe("two-phase pairing-grant redemption", () => { + it("returns the grant unchanged when the relay hand-off fails, then lets the retry spend it", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + const mintedExpiry = env.DB.pairingGrants[0]!.expires_at; + + const failing = activityRelayStub(() => new Response("down", { status: 503 })); + const outage = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + failing.options, + ); + + expect(outage.status).toBe(503); + expect(await outage.json()).toMatchObject({ code: "activity_relay_unavailable" }); + expect(env.DB.revocations).toHaveLength(1); + // Back exactly as it was. `expires_at` is the load-bearing assertion: a + // release that re-issued the grant with a fresh TTL would let anyone who + // can force relay failures keep one alive indefinitely. + expect(env.DB.pairingGrants).toHaveLength(1); + expect(env.DB.pairingGrants[0]).toMatchObject({ + reserved_at: null, + expires_at: mintedExpiry, + }); + + // The retry the refusal implies now works, with no second sign-in. + const retry = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(retry.status).toBe(200); + expect(env.DB.revocations).toEqual([]); + // Consumed on success: two phases must not become two chances. + expect(env.DB.pairingGrants).toEqual([]); + }); + + it("ignores a reservation left behind by a crashed worker but honors a live one", async () => { + const env = makeEnv(); + const token = await mintToken({ sub: "user_1" }); + const relay = await removedMachine(env, token); + const { grant } = await completeDeviceLogin(env, { machineKey: "machine-a" }); + + // Another registration is mid-relay with this grant right now. Single-use + // is what the reservation is for, so this one gets nothing. + env.DB.pairingGrants[0]!.reserved_at = Date.now(); + const contended = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(contended.status).toBe(403); + expect(await contended.json()).toMatchObject({ code: "pairing_authentication_required" }); + expect(relay.calls).toEqual([]); + expect(env.DB.pairingGrants).toHaveLength(1); + + // Same row, but the holder died without consuming or releasing it. Without + // the staleness bound the grant would now be unspendable until it expired + // — a lockout with extra steps. + env.DB.pairingGrants[0]!.reserved_at = Date.now() - (PAIRING_GRANT_RESERVATION_MS + 1_000); + const recovered = await handleRequest( + pairingRequest(token, { pairingGrant: grant }), + env, + relay.options, + ); + expect(recovered.status).toBe(200); + expect(env.DB.pairingGrants).toEqual([]); + expect(env.DB.revocations).toEqual([]); + }); +}); diff --git a/apps/account-directory/test/trustedOrigin.test.ts b/apps/account-directory/test/trustedOrigin.test.ts new file mode 100644 index 000000000..be2a70b0b --- /dev/null +++ b/apps/account-directory/test/trustedOrigin.test.ts @@ -0,0 +1,61 @@ +import { describe, expect, it } from "vitest"; +import { isLoopbackHostname, trustedHttpsOrigin } from "../src/trustedOrigin"; + +/** + * The accept/reject sets these three call sites used to each decide for + * themselves — the push relay base URL, the hosted web client's CORS origin, + * and the diagnostics route's loopback exemption. Pinned here so the shared + * helper cannot loosen any of them by accident. + */ +describe("trustedHttpsOrigin", () => { + it.each([ + ["https origin", "https://relay.ade.dev", "https://relay.ade.dev"], + ["https with a port", "https://relay.ade.dev:8443", "https://relay.ade.dev:8443"], + ["http on localhost", "http://localhost:8787", "http://localhost:8787"], + ["http on 127.0.0.1", "http://127.0.0.1:8787", "http://127.0.0.1:8787"], + ["http on the IPv6 loopback", "http://[::1]:8787", "http://[::1]:8787"], + ["surrounding whitespace", " https://relay.ade.dev ", "https://relay.ade.dev"], + ])("accepts %s", (_label, raw, expected) => { + expect(trustedHttpsOrigin(raw)).toBe(expected); + }); + + it.each([ + ["nothing configured", undefined], + ["an empty string", " "], + ["plain http off loopback", "http://relay.ade.dev"], + // A hostname that merely *contains* a loopback name is a different host. + ["a lookalike hostname", "http://localhost.evil.test"], + ["a non-http scheme", "ws://relay.ade.dev"], + ["embedded credentials", "https://user:pass@relay.ade.dev"], + ["a query string", "https://relay.ade.dev/?token=abc"], + ["a fragment", "https://relay.ade.dev/#frag"], + ["a value that is not a URL", "relay.ade.dev"], + ])("rejects %s", (_label, raw) => { + expect(trustedHttpsOrigin(raw)).toBeNull(); + }); + + it("keeps the origin of a base URL that carries a path, and refuses it as an exact origin", () => { + // The one difference between the two call sites: the relay gets paths + // appended to its base, the CORS allow-list is matched against a bare + // `Origin` header and so must not silently truncate one. + expect(trustedHttpsOrigin("https://relay.ade.dev/push")).toBe("https://relay.ade.dev"); + expect(trustedHttpsOrigin("https://relay.ade.dev/push", { requireExactOrigin: true })).toBeNull(); + expect(trustedHttpsOrigin("https://app.ade.dev", { requireExactOrigin: true })) + .toBe("https://app.ade.dev"); + // `new URL` normalizes a bare host to a trailing slash, which is not the + // origin — an allow-list entry written that way is a misconfiguration. + expect(trustedHttpsOrigin("https://app.ade.dev/", { requireExactOrigin: true })).toBeNull(); + }); +}); + +describe("isLoopbackHostname", () => { + it("names the three spellings of this machine and nothing else", () => { + for (const hostname of ["localhost", "127.0.0.1", "[::1]"]) { + expect(isLoopbackHostname(hostname)).toBe(true); + } + // `URL.hostname` keeps the brackets, so the bare form never reaches here. + for (const hostname of ["::1", "127.0.0.2", "localhost.evil.test", "app.ade.dev", ""]) { + expect(isLoopbackHostname(hostname)).toBe(false); + } + }); +}); diff --git a/apps/account-directory/wrangler.jsonc b/apps/account-directory/wrangler.jsonc index 35e4a808b..2d9318f30 100644 --- a/apps/account-directory/wrangler.jsonc +++ b/apps/account-directory/wrangler.jsonc @@ -41,6 +41,16 @@ "migrations_dir": "migrations" } ], + // Opt-in diagnostic report uploads (`POST /diagnostics/upload`). The bucket + // is NOT created by deploying — run `wrangler r2 bucket create` first (see + // README, "Diagnostic report uploads"). Missing binding is handled: the route + // answers 503 and the in-app button reports that sending is unavailable. + "r2_buckets": [ + { + "binding": "DIAGNOSTICS", + "bucket_name": "ade-diagnostics" + } + ], "env": { "production": { "name": "ade-account-directory-production", @@ -58,6 +68,12 @@ "database_id": "38ebe0bb-ac4d-4b39-b73e-bab2f0092971", "migrations_dir": "migrations" } + ], + "r2_buckets": [ + { + "binding": "DIAGNOSTICS", + "bucket_name": "ade-diagnostics-production" + } ] } } diff --git a/apps/ade-cli/README.md b/apps/ade-cli/README.md index d27db7ee3..dac3c0663 100644 --- a/apps/ade-cli/README.md +++ b/apps/ade-cli/README.md @@ -471,6 +471,7 @@ ade doctor --json ade doctor --online --text # also check the latest desktop release over the network ade report-issue --text # print a redacted diagnostic report + a prefilled GitHub issue URL (local files only; no brain needed) ade report-issue --open # also copy the report to the clipboard and open that issue URL in the browser +ade report-issue --send # also upload the same redacted report to ADE and print its reference id ade tools status --text # pinned agent CLIs: installed version + entry path per tool, plus the machine tools root ade tools ensure --text # fetch whatever this build pins and is missing (no names = all); streams progress to stderr ade tools ensure codex --text # one tool; an unknown name is a usage error listing the pinned set @@ -777,7 +778,21 @@ desktop "Report issue" button: it reads only local files — it never starts or contacts the brain — so it still works on the machine where ADE itself will not come up, and on Windows where there is no desktop error screen to press. It prints a redacted diagnostic report plus a prefilled GitHub issue URL (`--open` -also opens that URL, `--json` returns `{ installId, issueUrl, report }`). +copies the report to the clipboard and opens that URL, `--json` returns +`{ installId, issueUrl, copied, report }`). + +`--send` is the one part of this command that leaves the machine, and it is +opt-in: it uploads the same redacted report to ADE over HTTPS and prints the +reference id support quotes back. It is the headless counterpart to the desktop +button's "Send to ADE", and it reads everything it needs — the account session, +the directory origin — from local files, so it still works on a machine whose +brain will not start. A signed-in machine attaches its account token; a +signed-out one uploads anonymously against the install id already in the report. +A failed or rate-limited send never changes the printed report or the exit code: +it prints one line saying so and leaves the user holding everything they need to +file the issue by hand. With `--json`, `sent` is present only when `--send` was +asked for, as `{ ok: true, reference }` or `{ ok: false, reason }`, so a script +can tell "not requested" from "requested and failed". There is no `ade recovery diagnose` / `ade recovery repair`: those are Electron-main IPC (`ade.recovery.diagnose` / `ade.recovery.repair`) backed by diff --git a/apps/ade-cli/src/bootstrap.ts b/apps/ade-cli/src/bootstrap.ts index adbb3fee4..fc902144f 100644 --- a/apps/ade-cli/src/bootstrap.ts +++ b/apps/ade-cli/src/bootstrap.ts @@ -13,6 +13,12 @@ import { recordLastFailure, } from "../../desktop/src/main/services/runtime/lastFailureStore"; import { mapKvDbOpenErrorCode } from "../../desktop/src/shared/types/recovery"; +import { codedError } from "../../desktop/src/shared/codedError"; +import { + detectCloudPlaceholderFile, + detectCloudStorageProvider, + storageUnreadableMessage, +} from "../../desktop/src/main/services/storage/cloudPlaceholder"; import { detectDefaultBaseRef, toProjectInfo, upsertProjectRow } from "../../desktop/src/main/services/projects/projectService"; import { cleanupLegacyAdeSkills } from "../../desktop/src/main/services/skills/legacySkillCleanupService"; import { @@ -626,6 +632,16 @@ export async function createAdeRuntime(args: { }); let db: AdeDb; try { + // Preflight before the open, so a cloud-evicted database fails with the + // sentence that names the fix instead of the platform's uninterpretable + // errno ("Unknown system error -11, read" on macOS). + const placeholder = detectCloudPlaceholderFile(paths.dbPath); + if (placeholder) { + throw codedError( + storageUnreadableMessage(placeholder.path, placeholder.provider), + "storage_read_failed", + ); + } db = await openKvDb(paths.dbPath, logger, { hasSyncPeers, }); @@ -634,14 +650,23 @@ export async function createAdeRuntime(args: { const detail = error instanceof Error ? error.message : String(error); const failure = { code, - message: "ADE could not open the project data store.", + message: code === "storage_read_failed" + ? storageUnreadableMessage(paths.dbPath, detectCloudStorageProvider(paths.dbPath)) + : "ADE could not open the project data store.", detail, projectRoot, component: "project_db_open" as const, }; recordLastFailure({ kind: "project", projectRoot }, failure); recordLastFailure({ kind: "machine" }, failure); - throw error; + // Rethrowing the raw error discarded the classification computed one line + // above and handed the renderer a bare libuv message. Carry the code, the + // offending path and the raw errno instead — the code picks the recovery + // copy, and `detail` stays for logs and `ade report-issue`. + throw Object.assign( + codedError(failure.message, code), + { dbPath: paths.dbPath, projectRoot, detail }, + ); } clearLastFailure({ kind: "project", projectRoot }); diff --git a/apps/ade-cli/src/cli.test.ts b/apps/ade-cli/src/cli.test.ts index 22608cb7e..6092ae64a 100644 --- a/apps/ade-cli/src/cli.test.ts +++ b/apps/ade-cli/src/cli.test.ts @@ -18,6 +18,7 @@ import { describeLastFailureForStartupLog, detectUnmergedLaneCreateNudge, findProjectRoots, + formatDiagnosticError, formatOutput, graphWaitState, inferFormatter, @@ -11779,3 +11780,67 @@ describe("unlinkOwnedRuntimeSocket", () => { expect(unlinked).toEqual([]); }); }); + +describe("formatDiagnosticError", () => { + it("never prints the fields of a thrown non-Error, which can carry a token", () => { + // sync.connectToBrain hands the draft's token to the request it builds; a + // rejection that carried that request used to be JSON.stringify'd whole + // into launchd.err.log, which `ade report-issue` tails into a report the + // user is told to paste into a public GitHub issue. + // Assembled from segments so the working tree never carries a + // secret-shaped literal the secret scanner would flag (same convention as + // diagnosticReport.test.ts). + const fakeAdeToken = ["ade", "live", "9f3c1b7d24a54e6f8c0b1d2e3f4a5b6c"].join("_"); + const fakeSkToken = ["sk", "live", "abcdefghijklmnopqrstuvwxyz"].join("-"); + const thrown = { + code: "ECONNREFUSED", + token: fakeAdeToken, + body: { authorization: `Bearer ${fakeSkToken}` }, + }; + + const formatted = formatDiagnosticError(thrown); + + expect(formatted).not.toContain(fakeAdeToken); + expect(formatted).not.toContain(fakeSkToken); + // The shape still has to be diagnosable: the errno and the key names that + // locate the throw site survive. + expect(formatted).toContain("code=ECONNREFUSED"); + expect(formatted).toContain("token"); + }); + + it("redacts a credential that an Error carried in its own message", () => { + const fakeUrlToken = ["6f1c", "8a2b", "4d9e", "7f30"].join(""); + const formatted = formatDiagnosticError( + new Error(`connect failed: tcp://127.0.0.1:5051?token=${fakeUrlToken}`), + ); + + expect(formatted).not.toContain(fakeUrlToken); + expect(formatted).toContain("token="); + // Loopback is the fact a maintainer needs and identifies nobody. + expect(formatted).toContain("127.0.0.1"); + }); + + it("caps a runaway error so one throw cannot bury the log it is written to", () => { + const formatted = formatDiagnosticError("x".repeat(20_000)); + + expect(formatted.length).toBeLessThan(4_200); + expect(formatted).toMatch(/characters truncated/); + }); + + it("keeps a plain string error and survives a value that throws while described", () => { + expect(formatDiagnosticError("brain socket vanished")).toBe("brain socket vanished"); + + const hostile = new Proxy( + {}, + { + ownKeys: () => { + throw new Error("no keys for you"); + }, + get: () => { + throw new Error("no fields for you"); + }, + }, + ); + expect(() => formatDiagnosticError(hostile)).not.toThrow(); + }); +}); diff --git a/apps/ade-cli/src/cli.ts b/apps/ade-cli/src/cli.ts index 169d12d41..1a8be3329 100644 --- a/apps/ade-cli/src/cli.ts +++ b/apps/ade-cli/src/cli.ts @@ -36,8 +36,11 @@ import { import { buildCliDiagnosticReport, buildReportIssuePayload, + describeDiagnosticUpload, openDiagnosticIssue, + sendDiagnosticReport, } from "./commands/reportIssue"; +import { redactDiagnosticText } from "./services/diagnostics/diagnosticReport"; export { readInstalledDesktopVersion }; import { MAX_STATUS_NOTE_CHARACTERS, @@ -110,6 +113,7 @@ import { startJsonRpcServer, type JsonRpcHandler, type JsonRpcId, + type JsonRpcInternalErrorReport, type JsonRpcRequest, type JsonRpcServerErrorContext, type JsonRpcTransport, @@ -176,6 +180,7 @@ import { PAIRING_REAUTHENTICATION_REQUIRED_MESSAGE, } from "./services/account/accountMachinePublisherService"; import type { MachinePairingRepairResult } from "./services/account/machinePairingRepair"; +import type { MachinePairingAutoRecovery } from "./services/account/machinePairingAutoRecovery"; import type { SyncHostSingletonLease } from "./services/sync/syncHostSingleton"; import type { SyncTunnelClientService } from "./services/sync/syncTunnelClientService"; import type { RelayTunnelAuthorityGate } from "./services/sync/relayTunnelAuthorityGate"; @@ -411,7 +416,7 @@ type CliPlan = | { kind: "setup"; rest: string[] } | { kind: "connect"; rest: string[] } | { kind: "doctor"; online: boolean } - | { kind: "report-issue"; open: boolean } + | { kind: "report-issue"; open: boolean; send: boolean } | { kind: "serve"; rest: string[] } | { kind: "rpc-stdio"; rest: string[] } | { kind: "pty-host-worker" } @@ -710,7 +715,7 @@ const TOP_LEVEL_HELP = `${ADE_BANNER} $ ade sync web [--open] [--no-clipboard] Print (and copy) the web client pairing link + code $ ade sync status | pin generate Manage machine sync and phone pairing $ ade doctor [--online] Inspect installed app and machine-brain health - $ ade report-issue [--open] Print a redacted diagnostic report for a bug report + $ ade report-issue [--open] [--send] Print a redacted diagnostic report; --send hands it to ADE $ ade lanes list | show | create | child Work with lanes and lane stacks $ ade git status | commit | push | stash Run ADE-aware git operations $ ade operations status | wait Poll operation/test/chat/run status @@ -12702,6 +12707,7 @@ function buildCliPlan( return { kind: "report-issue", open: readFlag(args, ["--open"]), + send: readFlag(args, ["--send"]), }; } if (primary === "auth") { @@ -14197,14 +14203,90 @@ function reportContainedJsonRpcError( } } +/** + * The full text behind a redacted `internalError` reply. It lands on stderr, + * which for the installed brain is `launchd.err.log` — one of the logs + * `ade report-issue` tails — so the `ref` the user was shown is searchable. + */ +function reportInternalJsonRpcError(report: JsonRpcInternalErrorReport): void { + try { + process.stderr.write( + `ade jsonrpc internal error ref=${report.errorId} method=${report.method}: ${formatDiagnosticError(report.error)}\n`, + ); + } catch { + // Stderr may be gone during shutdown; contained errors should stay contained. + } +} + +/** + * One thrown value must not be able to bury a log the user is asked to send: + * a rejected fetch can carry a whole response body. + */ +const DIAGNOSTIC_ERROR_MAX_CHARS = 4_000; + +/** Fields worth printing off a thrown non-Error: they describe, never carry. */ +const DIAGNOSTIC_ERROR_SAFE_FIELDS = [ + "name", + "message", + "code", + "errno", + "syscall", + "status", + "statusCode", +] as const; + +/** + * Renders a thrown value for stderr — which for the installed brain is + * `launchd.err.log`, a file `ade report-issue` tails into a report the user is + * told to paste into a public issue. So this is a redaction boundary, not a + * formatter: everything it returns goes through the same pass the report body + * gets, and a thrown non-Error is described rather than serialized. Dumping + * such a value with `JSON.stringify` would print whatever an upstream caller + * attached to it — `sync.connectToBrain` carries `draft.token`, and a rejected + * request object would have shipped it verbatim. + */ function formatDiagnosticError(error: unknown): string { + return capDiagnosticText(redactDiagnosticText(describeDiagnosticError(error))); +} + +function capDiagnosticText(text: string): string { + if (text.length <= DIAGNOSTIC_ERROR_MAX_CHARS) return text; + return `${text.slice(0, DIAGNOSTIC_ERROR_MAX_CHARS)}… (${text.length - DIAGNOSTIC_ERROR_MAX_CHARS} more characters truncated)`; +} + +function describeDiagnosticError(error: unknown): string { if (error instanceof Error) return error.stack || error.message; if (typeof error === "string") return error; + if (error === null || error === undefined) return String(error); + if (typeof error !== "object") return String(error); + + // A thrown object: name the shape and the diagnostic fields, then list the + // remaining keys by name only. Key names locate the throw site; their values + // are exactly what must not reach a log the user may hand over. + const record = error as Record; + const described: string[] = []; + let label = "Object"; try { - return JSON.stringify(error); + for (const field of DIAGNOSTIC_ERROR_SAFE_FIELDS) { + const value = record[field]; + if (value === undefined || value === null) continue; + if (typeof value === "object" || typeof value === "function") continue; + described.push(`${field}=${String(value)}`); + } + const otherKeys = Object.keys(record).filter( + (key) => !(DIAGNOSTIC_ERROR_SAFE_FIELDS as readonly string[]).includes(key), + ); + if (otherKeys.length > 0) { + described.push(`otherKeys=[${otherKeys.slice(0, 20).join(", ")}]`); + } + label = Array.isArray(error) ? "Array" : (record.constructor?.name ?? "Object"); } catch { - return String(error); + // Getters and proxy traps run arbitrary code and can throw; a value we + // cannot describe still must not take the error boundary down with it. } + return described.length > 0 + ? `[thrown ${label}] ${described.join(" ")}` + : `[thrown ${label}]`; } function installRuntimeProcessErrorBoundary(label: string): () => void { @@ -14280,6 +14362,7 @@ function createHeadlessRpcServer( const stop = startJsonRpcServer(handler, transport, { nonFatal: true, onError: reportContainedJsonRpcError, + onInternalError: reportInternalJsonRpcError, }); (handler as NotifiableJsonRpcHandler).setNotifier?.((method, params) => stop.notify(method, params), @@ -16670,6 +16753,7 @@ async function runNativeRpcStdio(options: GlobalOptions): Promise { stop = startJsonRpcServer(handler, createStdioTransport(), { nonFatal: true, onError: reportContainedJsonRpcError, + onInternalError: reportInternalJsonRpcError, }); unsubscribeNotifications = client.onAnyNotification((method, params) => stop?.notify(method, params), @@ -17243,8 +17327,16 @@ async function runServe( const machineCloudRelayFilePath = path.join(layout.secretsDir, "sync-cloud-relay.json"); const machineCloudRelayStore = createSyncCloudRelayStore({ filePath: machineCloudRelayFilePath, + // Every mint, rotation, and backup recovery of this machine's identity is + // logged here. A machine key that changes silently is how a live computer + // became a phantom row its owner deleted. + logger: headlessProjectLogger, }); let accountMachinePublisher: AccountMachinePublisherService | null = null; + // Turns the "Reconnect this computer" button into something the machine can + // press for itself when the directory refuses it. Built below, next to the + // publisher it watches. + let machinePairingAutoRecovery: MachinePairingAutoRecovery | null = null; // Held only while this brain hosts phone sync WITHOUT a project scope (a // scope's sync service owns its own lease). Machine-exclusive subsystems // gate on holding one or the other. @@ -17602,6 +17694,8 @@ async function runServe( const disposeServeResources = async () => { releaseAccountPublisherAuthoritySubscription?.(); releaseAccountPublisherAuthoritySubscription = null; + machinePairingAutoRecovery?.stop(); + machinePairingAutoRecovery = null; accountMachinePublisher?.dispose(); accountMachinePublisher = null; brainRelayTunnelGate?.dispose(); @@ -17883,6 +17977,18 @@ async function runServe( return brainSyncHostLease ? projectlessSyncSnapshot() : null; }, getMachineKey: () => machineCloudRelayStore.getMachineIdentity().machineKey, + // The SAME store instance the machine key above comes from, so a + // `supersededMachineKeys` answer is checked against the keys this brain + // actually retired. The publisher used to build a private second store + // over the same file, which is one identity guarded by two independent + // readers for no benefit at all. + confirmSupersededMachineKeys: (keys) => { + try { + return machineCloudRelayStore.confirmSupersededMachineKeys(keys); + } catch { + return []; + } + }, directoryBaseUrl: () => process.env.ADE_ACCOUNT_DIRECTORY_URL?.trim() || undefined, captureAnalytics: (input) => { brainProductAnalytics.captureInternal(input); @@ -17932,6 +18038,29 @@ async function runServe( reason: "This brain does not hold the machine-wide sync host lease; another ADE process publishes this machine.", }); } + // A directory refusal used to dead-end: heartbeats stopped and the machine + // waited for a human to click "Reconnect this computer", which nobody ever + // sees on a headless box. This runs the identical repair on a slow, budgeted + // schedule. It reads the publisher through `getPublisher` rather than + // capturing it, because the publisher is destroyed and rebuilt whenever the + // sync host lease moves. + const { createMachinePairingAutoRecovery } = await import( + "./services/account/machinePairingAutoRecovery" + ); + machinePairingAutoRecovery = createMachinePairingAutoRecovery({ + getPublisher: () => accountMachinePublisher, + runRepair: () => repairMachinePairing(), + hasAccountSession: () => { + try { + return brainAccountAuthService.getStatus().signedIn === true; + } catch { + return false; + } + }, + budget: machineCloudRelayStore, + logger: headlessProjectLogger, + }); + machinePairingAutoRecovery.start(); } process.stderr.write( @@ -22458,16 +22587,20 @@ async function runCli( // opening it without copying first sends them to a form with nothing to // paste. Both steps are best effort and the report is printed regardless. const openedIssue = plan.open ? await openDiagnosticIssue(built) : null; + // Sending is opt-in and never blocks the printed report: a failed upload + // still leaves the user holding everything they need to file by hand. + const sent = plan.send ? await sendDiagnosticReport(built) : null; if (parsed.options.text) { const clipboardNote = openedIssue?.copied ? "\n(the report is on your clipboard)" : ""; + const sendNote = sent ? `\n${describeDiagnosticUpload(sent)}` : ""; return { - output: `${built.report}\nFile the issue at:\n${built.issueUrl}${clipboardNote}\n`, + output: `${built.report}\nFile the issue at:\n${built.issueUrl}${clipboardNote}${sendNote}\n`, exitCode: 0, }; } return { output: formatOutput( - buildReportIssuePayload(built, openedIssue), + buildReportIssuePayload(built, openedIssue, sent), parsed.options, undefined, ), @@ -22686,6 +22819,7 @@ export { checkLinearReadiness, detectUnmergedLaneCreateNudge, findProjectRoots, + formatDiagnosticError, formatOutput, graphWaitState, inferFormatter, diff --git a/apps/ade-cli/src/commands/doctor.test.ts b/apps/ade-cli/src/commands/doctor.test.ts index 537e72e65..c1ec7529d 100644 --- a/apps/ade-cli/src/commands/doctor.test.ts +++ b/apps/ade-cli/src/commands/doctor.test.ts @@ -8,6 +8,7 @@ import { type DoctorInput, } from "./doctor"; import { createSyncAccountDirectoryHealth } from "../../../desktop/src/shared/types/sync"; +import { PAIRING_REAUTHENTICATION_REQUIRED_MESSAGE } from "../services/account/accountMachinePublisherService"; const NOW = Date.parse("2026-07-23T12:00:00.000Z"); @@ -324,6 +325,22 @@ describe("doctor row evaluation", () => { expect(rows.find((row) => row.key === "publish")?.detail).not.toContain("ade brain restart"); }); + it("keeps the directory refusal sentence on a long-failing publish row", () => { + // A refused machine is terminal, so by the time anyone runs doctor it is + // always in the ≥2min branch — the one that used to print the bare state. + const input = healthyInput(); + input.publishHealth = createSyncAccountDirectoryHealth( + "http_error", + PAIRING_REAUTHENTICATION_REQUIRED_MESSAGE, + { failingSinceMs: NOW - 6 * 60_000, lastHttpStatus: 403 }, + ); + + const publish = evaluateDoctorRows(input).find((row) => row.key === "publish"); + + expect(publish?.status).toBe("fail"); + expect(publish?.detail).toContain(PAIRING_REAUTHENTICATION_REQUIRED_MESSAGE); + }); + it("points an unreadable account session at `ade brain restart`", () => { // Desktop's Connections panel shows a Repair (brain restart) button for // this state; the CLI has to name the same remedy or an agent is stuck. @@ -391,6 +408,41 @@ describe("doctor row evaluation", () => { ); }); + it("reports the needs-reconnect reason over a stale relay self-probe failure", () => { + const input = healthyInput(); + input.relayHealth = { + ...input.relayHealth!, + relayControlConnected: false, + relayBridgeValidated: false, + relayEndToEndVerifiedAt: null, + // Kept across control generations by the tunnel client, so a machine that + // is now capped still reports whatever its last probe said. + relayEndToEndFailure: "Relay self-probe skipped because the control socket is not connected.", + // What the brain ranks into skipReason once the rotation budget is spent. + skipReason: "This computer needs to be reconnected to your ADE account.", + lastControlError: "claim failed (409)", + }; + + const relay = evaluateDoctorRows(input).find((row) => row.key === "relay"); + + expect(relay?.status).toBe("fail"); + expect(relay?.detail).toBe("This computer needs to be reconnected to your ADE account."); + }); + + it("still reports a self-probe failure while relay control is connected", () => { + const input = healthyInput(); + input.relayHealth = { + ...input.relayHealth!, + relayEndToEndVerifiedAt: null, + relayEndToEndFailure: "Relay echo never came back.", + }; + + const relay = evaluateDoctorRows(input).find((row) => row.key === "relay"); + + expect(relay?.status).toBe("fail"); + expect(relay?.detail).toBe("Relay echo never came back."); + }); + it("compares release versions without depending on tag formatting", () => { expect(compareDoctorVersions("v1.2.36", "1.2.35")).toBe(1); expect(compareDoctorVersions("1.2.35", "v1.2.35")).toBe(0); diff --git a/apps/ade-cli/src/commands/doctor.ts b/apps/ade-cli/src/commands/doctor.ts index c0f110d75..b3f13f966 100644 --- a/apps/ade-cli/src/commands/doctor.ts +++ b/apps/ade-cli/src/commands/doctor.ts @@ -698,12 +698,20 @@ function publishRow( const remedy = isBrainAccountSessionFailure(health.state) ? " · run `ade brain restart` so the brain re-reads the account session" : ""; + // The refusals the directory answers with — a removed machine, a pairing that + // needs a fresh sign-in — all arrive as `state: "http_error"`, and the + // sentence the user can act on lives ONLY in `skipReason`. It used to be + // dropped from exactly the branch a refusal ends up in (a refusal is terminal, + // so `failingSince` is always set by the time anyone runs this), leaving the + // row saying "failing for 6m · http_error" about a machine whose repair is one + // sign-in away. + const reasonDetail = health.skipReason ? ` · ${health.skipReason}` : ""; if (failingForMs != null && failingForMs >= PUBLISH_FAILURE_RED_MS) { return { key: "publish", label: "Publish health", status: "fail", - detail: `failing for ${compactDuration(failingForMs)} · ${health.state}${publishLegDetail(health)}${remedy}`, + detail: `failing for ${compactDuration(failingForMs)} · ${health.state}${reasonDetail}${publishLegDetail(health)}${remedy}`, }; } if (health.state === "published") { @@ -724,8 +732,8 @@ function publishRow( label: "Publish health", status: "warn", detail: failingForMs == null - ? `${health.state}${health.skipReason ? ` · ${health.skipReason}` : ""}${remedy}` - : `failing for ${compactDuration(failingForMs)} · ${health.state}${publishLegDetail(health)}${remedy}`, + ? `${health.state}${reasonDetail}${remedy}` + : `failing for ${compactDuration(failingForMs)} · ${health.state}${reasonDetail}${publishLegDetail(health)}${remedy}`, }; } @@ -746,14 +754,25 @@ function relayRow(relay: DoctorInput["relayHealth"]): DoctorRow { detail: relay.skipReason ?? "disabled", }; } + // While relay control is DOWN the route reason outranks the self-probe, and + // while it is up the probe outranks it. `relayEndToEndFailure` is deliberately + // carried across control generations, so a disconnected machine still reports + // the last probe it managed to run — and the brain ranks a spent identity + // rotation budget ("this computer needs to be reconnected to your ADE + // account") into `skipReason` precisely because nothing else names that fix. + // Printing the stale probe line instead is how this row would answer a + // needs-reconnect machine with "self-probe skipped". `ade sync status` shows + // both; a one-line row has to choose the proximate one. + const routeFailure = relay.relayControlConnected === true + ? relay.relayEndToEndFailure ?? relay.skipReason + : relay.skipReason ?? relay.relayEndToEndFailure; // Suppression outranks every other failure: while another ADE process owns // this machine's relay slot, nothing downstream can succeed and no other // reason tells the user what to do about it. const failure = (relay.relayControlSuppressed === true ? relay.relayControlSuppressedReason ?? "Relay control is suppressed." : null) - ?? relay.relayEndToEndFailure - ?? relay.skipReason + ?? routeFailure ?? relay.lastControlError ?? null; const healthy = relay.relayControlConnected === true diff --git a/apps/ade-cli/src/commands/reportIssue.test.ts b/apps/ade-cli/src/commands/reportIssue.test.ts index 422c0ce96..4e447f8e2 100644 --- a/apps/ade-cli/src/commands/reportIssue.test.ts +++ b/apps/ade-cli/src/commands/reportIssue.test.ts @@ -5,7 +5,9 @@ import { afterEach, describe, expect, it } from "vitest"; import { buildCliDiagnosticReport, buildReportIssuePayload, + describeDiagnosticUpload, openDiagnosticIssue, + sendDiagnosticReport, } from "./reportIssue"; const tempDirs: string[] = []; @@ -151,3 +153,79 @@ describe("buildReportIssuePayload", () => { expect(payload.issueUrl).toBe(built.issueUrl); }); }); + +describe("sendDiagnosticReport", () => { + const built = { + report: "REPORT BODY", + installId: "install-abc", + appVersion: "1.2.60", + secretsDir: "/tmp/does-not-need-to-exist/secrets", + }; + + function capture(response: Response) { + const calls: Array<{ url: string; init: RequestInit }> = []; + const fetchImpl = (async (input: RequestInfo | URL, init?: RequestInit) => { + calls.push({ url: String(input), init: init ?? {} }); + return response; + }) as unknown as typeof fetch; + return { calls, fetchImpl }; + } + + it("sends the built report unchanged, with the account token when signed in", async () => { + const { calls, fetchImpl } = capture( + new Response(JSON.stringify({ ok: true, id: "abcdef12-3456-4789-8abc-def012345678" }), { + status: 200, + }), + ); + + const result = await sendDiagnosticReport(built, { + baseUrl: "https://directory.example", + getToken: async () => "clerk-token", + fetchImpl, + }); + + expect(result).toEqual({ + ok: true, + id: "abcdef12-3456-4789-8abc-def012345678", + reference: "abcdef12", + }); + expect(calls[0]!.url).toBe("https://directory.example/diagnostics/upload"); + expect(new Headers(calls[0]!.init.headers).get("authorization")).toBe("Bearer clerk-token"); + // The report is redacted upstream; --send must post those exact bytes. + expect(JSON.parse(String(calls[0]!.init.body))).toEqual({ + report: "REPORT BODY", + installId: "install-abc", + appVersion: "1.2.60", + }); + }); + + it("uploads anonymously when there is no session, and drops the placeholder install id", async () => { + const { calls, fetchImpl } = capture( + new Response(JSON.stringify({ ok: true, id: "0f0f0f0f-1111-4222-8333-444444444444" }), { + status: 200, + }), + ); + + await sendDiagnosticReport( + { ...built, installId: "unknown" }, + { baseUrl: "https://directory.example", getToken: async () => null, fetchImpl }, + ); + + expect(new Headers(calls[0]!.init.headers).get("authorization")).toBeNull(); + // "unknown" is the report's stand-in for "analytics is off"; sending it as + // an install id would attach a metadata value that identifies nothing. + expect(JSON.parse(String(calls[0]!.init.body))).toEqual({ + report: "REPORT BODY", + appVersion: "1.2.60", + }); + }); + + it("describes each failure in one plain sentence", async () => { + expect(describeDiagnosticUpload({ ok: true, id: "abcdef1234", reference: "abcdef12" })) + .toBe("Sent to ADE — reference abcdef12"); + expect(describeDiagnosticUpload({ ok: false, reason: "rate_limited" })) + .toContain("already sent several reports today"); + expect(describeDiagnosticUpload({ ok: false, reason: "network" })) + .toContain("couldn't reach"); + }); +}); diff --git a/apps/ade-cli/src/commands/reportIssue.ts b/apps/ade-cli/src/commands/reportIssue.ts index 8ccde0bfc..65a1e69b4 100644 --- a/apps/ade-cli/src/commands/reportIssue.ts +++ b/apps/ade-cli/src/commands/reportIssue.ts @@ -9,6 +9,16 @@ import { collectMachineDiagnosticSources, readDiagnosticJsonFile, } from "../services/diagnostics/diagnosticSources"; +import { + uploadDiagnosticReport, + type DiagnosticUploadResult, +} from "../../../desktop/src/shared/diagnosticsUpload"; +import { DEFAULT_ADE_ACCOUNT_DIRECTORY_URL } from "../../../desktop/src/shared/accountDirectory"; +import { getSignedInAccountAccessToken } from "../services/account/accountAuthService"; +import { + getSharedAccountAuthService, + getSharedAccountDirectoryBaseUrl, +} from "../services/account/sharedAccountAuthService"; import { copyToClipboard } from "../lib/clipboard"; import { openExternalUrl } from "../lib/externalLinks"; @@ -31,6 +41,10 @@ export type ReportIssueResult = { report: string; issueUrl: string; installId: string; + /** CLI version stamped into the report; sent as upload metadata. */ + appVersion: string | null; + /** Where this machine's account session lives, so `--send` can read a token. */ + secretsDir: string; }; /** @@ -101,6 +115,8 @@ export function buildCliDiagnosticReport(options: ReportIssueOptions = {}): Repo return { report, installId, + appVersion: options.cliVersion ?? null, + secretsDir: sources.layout.secretsDir, issueUrl: buildDiagnosticIssueUrl({ surface, appVersion: options.cliVersion ?? null, @@ -151,6 +167,75 @@ export async function openDiagnosticIssue( return { copied, opened }; } +/** + * `ade report-issue --send`: hand the same redacted report to ADE over HTTPS. + * + * Everything it needs is read from local files — the account session out of the + * machine's own credential store, the directory origin out of the same resolver + * the brain uses — so it still works on the machine where the brain will not + * start, which is the only machine anyone runs this on. A signed-in machine + * sends its Clerk token so support can tie the report to the account; a + * signed-out one uploads anonymously against the install id the report already + * carries. Neither path changes a byte of the report. + */ +export async function sendDiagnosticReport( + built: Pick, + deps: { + env?: NodeJS.ProcessEnv; + baseUrl?: string; + /** Test seam; production resolves the token from the credential store. */ + getToken?: () => Promise; + fetchImpl?: typeof fetch; + } = {}, +): Promise { + const env = deps.env ?? process.env; + // Both lookups touch the machine's own config and credential store, and this + // command exists for machines whose state is damaged. Neither may throw: a + // failed send has to stay a failed send, not take the printed report with it. + const baseUrl = deps.baseUrl ?? (() => { + try { + return getSharedAccountDirectoryBaseUrl({ secretsDir: built.secretsDir, env }); + } catch { + return DEFAULT_ADE_ACCOUNT_DIRECTORY_URL; + } + })(); + const token = await (deps.getToken ?? (async () => { + // An unreadable or absent session simply means an anonymous upload, which + // is exactly what the route accepts them for. + try { + return await getSignedInAccountAccessToken( + getSharedAccountAuthService({ secretsDir: built.secretsDir, env }), + ); + } catch { + return null; + } + }))(); + + return uploadDiagnosticReport({ + baseUrl, + report: built.report, + token, + installId: built.installId === "unknown" ? null : built.installId, + appVersion: built.appVersion, + fetchImpl: deps.fetchImpl, + }); +} + +/** One short line for `--text`, in the same register as the rest of the command. */ +export function describeDiagnosticUpload(result: DiagnosticUploadResult): string { + if (result.ok) return `Sent to ADE — reference ${result.reference}`; + switch (result.reason) { + case "rate_limited": + return "Not sent: you've already sent several reports today. Try again tomorrow."; + case "too_large": + return "Not sent: this report is too big to send. File it on GitHub instead."; + case "unavailable": + return "Not sent: ADE can't take reports right now. File it on GitHub instead."; + default: + return "Not sent: ADE couldn't reach the report service. File it on GitHub instead."; + } +} + /** * The `--json` shape of `ade report-issue`. `copied` is here because `--open` * has two side effects, and a script that asked for machine-readable output @@ -162,12 +247,29 @@ export async function openDiagnosticIssue( export function buildReportIssuePayload( built: Pick, side: { copied: boolean } | null, -): { ok: true; installId: string; issueUrl: string; copied: boolean; report: string } { + sent?: DiagnosticUploadResult | null, +): { + ok: true; + installId: string; + issueUrl: string; + copied: boolean; + report: string; + sent?: { ok: boolean; reference?: string; reason?: string }; +} { return { ok: true, installId: built.installId, issueUrl: built.issueUrl, copied: side?.copied ?? false, report: built.report, + // Omitted entirely without `--send`, so a script can tell "not asked for" + // from "asked for and failed". + ...(sent + ? { + sent: sent.ok + ? { ok: true, reference: sent.reference } + : { ok: false, reason: sent.reason }, + } + : {}), }; } diff --git a/apps/ade-cli/src/headlessLinearServices.test.ts b/apps/ade-cli/src/headlessLinearServices.test.ts index 1436fd3c7..9f33d0c39 100644 --- a/apps/ade-cli/src/headlessLinearServices.test.ts +++ b/apps/ade-cli/src/headlessLinearServices.test.ts @@ -35,6 +35,7 @@ vi.mock("../../desktop/src/main/services/automations/automationSecretService", ( })); import { EncryptedFileCredentialStore } from "./services/credentials/credentialStore"; +import { resolveMachineAdeLayout } from "./services/projects/machineLayout"; import { createHeadlessGitHubService, createHeadlessLinearServices } from "./headlessLinearServices"; import { resetGitHubServiceHealthCache } from "../../desktop/src/main/services/github/githubStatusPage"; import { @@ -372,6 +373,48 @@ describe("headlessLinearServices", () => { } }); + // A PAT saved over an unreadable store re-seals it with a key this process + // holds, so the "ADE can't read your saved sign-in" verdict is stale the + // moment the write lands. Without clearing it there, every later read + // short-circuits on the in-memory override and never re-reads the store, so + // the status kept reporting a broken store forever after the user fixed it. + it("stops reporting an unreadable credential store after a replacement PAT is saved", async () => { + const environment = isolateHeadlessGithubAuth("ade-headless-github-store-recovery-", { + emptyGhConfig: true, + }); + const previousFetch = globalThis.fetch; + globalThis.fetch = vi.fn(async () => new Response(JSON.stringify({ login: "octocat" }), { + status: 200, + headers: { + "content-type": "application/json", + "x-oauth-scopes": "repo, workflow", + }, + })) as unknown as typeof fetch; + try { + const { secretsDir } = resolveMachineAdeLayout(); + fs.mkdirSync(secretsDir, { recursive: true }); + // An undecryptable store returns an EMPTY view instead of throwing, which + // is exactly why the read state has to be tracked separately. + fs.writeFileSync(path.join(secretsDir, "credentials.json.enc"), "not-json", "utf8"); + const githubService = createHeadlessGitHubService( + "/tmp/ade-project", + { debug() {}, info() {}, warn() {}, error() {} } as any, + ); + + await expect(githubService.getStatus({ forceRefresh: true })).resolves.toMatchObject({ + credentialStoreUnreadable: true, + }); + + githubService.setToken("ghp_replacement_token"); + await expect(githubService.getStatus({ forceRefresh: true })).resolves.toMatchObject({ + credentialStoreUnreadable: false, + }); + } finally { + globalThis.fetch = previousFetch; + environment.restore(); + } + }); + it("clears only App health when headless App authorization is removed", () => { const environment = isolateHeadlessGithubAuth("ade-headless-github-app-health-", { emptyGhConfig: true, diff --git a/apps/ade-cli/src/headlessLinearServices.ts b/apps/ade-cli/src/headlessLinearServices.ts index 6811f3c57..2e4cef6e4 100644 --- a/apps/ade-cli/src/headlessLinearServices.ts +++ b/apps/ade-cli/src/headlessLinearServices.ts @@ -25,6 +25,10 @@ import { resolveModelAlias, } from "../../desktop/src/shared/modelRegistry"; import { parseGitHubScopeHeaders } from "../../desktop/src/shared/githubScopes"; +import { + readCredentialWithState, + readCredentialWithStateAsync, +} from "../../desktop/src/main/services/github/credentialReadState"; import type { GitHubAuthFailure, GitHubAppDeviceAuthPollResult, @@ -327,6 +331,8 @@ type HeadlessGitHubCredentialInventory = { patTokenStored: boolean; ghCliPath: string | null; ghAuthError: string | null; + /** Whether the credential file was readable on the read that built this. */ + credentialStoreUnreadable: boolean; }; class HeadlessGithubCredentialAttemptError extends Error { @@ -726,6 +732,10 @@ export function createHeadlessGitHubService( let cachedStatusBinding: string | null = null; let tokenOverride: string | null = null; let tokenDecryptionFailed = false; + // An undecryptable store returns an EMPTY view instead of throwing, so without + // this a remote runtime reports a corrupted store as "GitHub was never + // connected" — identical to a fresh install. See GitHubStatus.credentialStoreUnreadable. + let credentialStoreUnreadable = false; let statusLookupGeneration = 0; let statusLookupInFlight: { generation: number; @@ -744,16 +754,16 @@ export function createHeadlessGitHubService( statusLookupGeneration += 1; }; + const noteCredentialStoreReadState = (unreadable: boolean): boolean => { + credentialStoreUnreadable = unreadable; + return credentialStoreUnreadable; + }; + const readStoredPatToken = (): string | null => { if (tokenOverride != null) return tokenOverride; - try { - const stored = credentialStore.getSync(tokenKey); - tokenDecryptionFailed = false; - if (stored?.trim()) return stored.trim(); - } catch { - tokenDecryptionFailed = true; - } - return null; + const read = readCredentialWithState(credentialStore, tokenKey); + tokenDecryptionFailed = noteCredentialStoreReadState(read.unreadable); + return read.value; }; const readToken = (): HeadlessGitHubTokenLookup => { @@ -787,19 +797,19 @@ export function createHeadlessGitHubService( const readStoredPatTokenAsync = async (): Promise => { if (tokenOverride != null) return tokenOverride; - try { - const stored = await credentialStore.get(tokenKey); - tokenDecryptionFailed = false; - if (stored?.trim()) return stored.trim(); - } catch { - tokenDecryptionFailed = true; - } - return null; + const read = await readCredentialWithStateAsync(credentialStore, tokenKey); + tokenDecryptionFailed = noteCredentialStoreReadState(read.unreadable); + return read.value; }; const readCredentialInventoryAsync = async (): Promise => { const patToken = await readStoredPatTokenAsync(); const patTokenStored = Boolean(patToken); + // Snapshotted next to `patTokenStored`, because the read that just ran is + // the one this verdict belongs to. `credentialStoreUnreadable` is shared + // mutable state: another caller reading the same store during the awaits + // below would otherwise hand this inventory somebody else's outcome. + const storeUnreadableForThisRead = credentialStoreUnreadable; const environmentToken = envToken("ADE_GITHUB_TOKEN", "GITHUB_TOKEN", "GH_TOKEN"); const appStatus = appUserAuth.getAuthStatus(); const [appResult, gh] = await Promise.all([ @@ -872,6 +882,7 @@ export function createHeadlessGitHubService( patTokenStored, ghCliPath: gh.ghCliPath, ghAuthError: gh.ghAuthError, + credentialStoreUnreadable: storeUnreadableForThisRead, }; }; @@ -1724,6 +1735,7 @@ export function createHeadlessGitHubService( repo, hasOrigin, patTokenStored: inventory.patTokenStored, + credentialStoreUnreadable: inventory.credentialStoreUnreadable, ghCliPath: inventory.ghCliPath ?? cachedStatus.ghCliPath, ghAuthError: inventory.ghAuthError, credentialStates: githubCredentialStates({ @@ -1765,6 +1777,7 @@ export function createHeadlessGitHubService( tokenStored: inventory.appTokenStored, patTokenStored: inventory.patTokenStored, tokenDecryptionFailed, + credentialStoreUnreadable: inventory.credentialStoreUnreadable, storageScope: "app", authSource: failure?.source ?? "none", writeAuthSource: "none", @@ -1855,6 +1868,7 @@ export function createHeadlessGitHubService( tokenStored: true, patTokenStored: inventory.patTokenStored, tokenDecryptionFailed: false, + credentialStoreUnreadable: inventory.credentialStoreUnreadable, storageScope: "app", authSource: candidate.source, writeAuthSource: activeWriteSource ?? "none", @@ -1905,6 +1919,7 @@ export function createHeadlessGitHubService( tokenStored: true, patTokenStored: inventory.patTokenStored, tokenDecryptionFailed: false, + credentialStoreUnreadable: inventory.credentialStoreUnreadable, storageScope: "app", authSource: primaryCandidate.source, writeAuthSource: "none", @@ -2117,6 +2132,11 @@ export function createHeadlessGitHubService( credentialStore.deleteSync(tokenKey); } tokenDecryptionFailed = false; + // A write that landed re-sealed the store with a key this process holds, + // so whatever made the previous read unreadable no longer applies. Without + // this the status keeps reporting a broken store forever, because every + // later read short-circuits on `tokenOverride` before re-reading it. + credentialStoreUnreadable = false; if (previousToken) clearGithubCredentialHealth(previousToken); if (clean && clean !== previousToken) clearGithubCredentialHealth(clean); invalidateStatusCache(); @@ -2127,6 +2147,9 @@ export function createHeadlessGitHubService( tokenOverride = null; credentialStore.deleteSync(tokenKey); tokenDecryptionFailed = false; + // Same reasoning as `setToken`: the delete rewrote the store with a key + // this process holds, so the previous unreadable verdict is stale. + credentialStoreUnreadable = false; if (previousToken) clearGithubCredentialHealth(previousToken); invalidateStatusCache(); emitStatusChanged(); diff --git a/apps/ade-cli/src/jsonrpc.test.ts b/apps/ade-cli/src/jsonrpc.test.ts index 13b0b292d..3f6d1c3fc 100644 --- a/apps/ade-cli/src/jsonrpc.test.ts +++ b/apps/ade-cli/src/jsonrpc.test.ts @@ -3,6 +3,7 @@ import { describe, expect, it, vi } from "vitest"; import { startJsonRpcServer, type JsonRpcHandler, + type JsonRpcInternalErrorReport, type JsonRpcTransport, } from "./jsonrpc"; @@ -315,3 +316,113 @@ describe("startJsonRpcServer", () => { expect(transport.closed).toBe(true); }); }); + +describe("internal error replies", () => { + async function failWith(error: unknown, options?: { onInternalError?: (report: JsonRpcInternalErrorReport) => void }) { + const transport = new MemoryTransport(); + const stop = startJsonRpcServer(async () => { + throw error; + }, transport, { + nonFatal: true, + ...(options?.onInternalError ? { onInternalError: options.onInternalError } : {}), + }); + transport.push({ jsonrpc: "2.0", id: 1, method: "ade/actions/call" }); + await waitForDrain(); + stop(); + return jsonlResponses(transport)[0] as { + error: { code: number; message: string; data?: { code?: string; errorId?: string } }; + }; + } + + it("never forwards a raw filesystem errno, and logs it against a reference", async () => { + // The production shape: macOS EDEADLK, which libuv cannot name. + const raw = Object.assign( + new Error("Unknown system error -11: Unknown system error -11, read"), + { errno: -11, code: "EDEADLK", syscall: "read" }, + ); + const reports: JsonRpcInternalErrorReport[] = []; + const response = await failWith(raw, { onInternalError: (report) => reports.push(report) }); + + expect(response.error.code).toBe(-32603); + expect(response.error.message).not.toContain("Unknown system error"); + expect(response.error.message).toMatch(/^Internal error in ade\/actions\/call \(ref [0-9a-f]+\)$/); + expect(response.error.data?.errorId).toMatch(/^[0-9a-f]+$/); + expect(reports).toHaveLength(1); + expect(reports[0]?.method).toBe("ade/actions/call"); + expect(reports[0]?.errorId).toBe(response.error.data?.errorId); + expect(reports[0]?.error).toBe(raw); + }); + + it("redacts path-bearing fs errors and runtime faults", async () => { + const enoent = Object.assign( + new Error("ENOENT: no such file or directory, open '/Users/someone/private/ade.db'"), + { errno: -2, code: "ENOENT", syscall: "open" }, + ); + expect((await failWith(enoent)).error.message).not.toContain("/Users/someone"); + + const fault = new TypeError("Cannot read properties of undefined (reading 'db')"); + expect((await failWith(fault)).error.message).not.toContain("undefined"); + + expect((await failWith("boom")).error.message).not.toContain("boom"); + }); + + it("redacts Node's own internal codes, which quote absolute paths", async () => { + // A packaged build that lost a file reaches the boundary as a plain Error + // whose code is a Node internal one — not an errno — and whose message + // names the path. Both shapes must take the reference path, not the + // "service verdict" path that forwards the message verbatim. + for (const code of ["ERR_MODULE_NOT_FOUND", "MODULE_NOT_FOUND", "ERR_FS_EISDIR"]) { + const reports: JsonRpcInternalErrorReport[] = []; + const error = Object.assign( + new Error("Cannot find module '/Users/someone/ADE/apps/ade-cli/dist/brain.js'"), + { code }, + ); + const response = await failWith(error, { + onInternalError: (report) => reports.push(report), + }); + + expect(response.error.message).not.toContain("/Users/someone"); + expect(response.error.message).toMatch(/^Internal error in ade\/actions\/call \(ref [0-9a-f]+\)$/); + expect(response.error.data?.code).toBeUndefined(); + expect(reports).toHaveLength(1); + expect(reports[0]?.error).toBe(error); + } + }); + + it("forwards a service's coded verdict so the caller can act on it", async () => { + const response = await failWith(Object.assign( + new Error("ADE couldn't read this project's data at /tmp/p/.ade/ade.db."), + { code: "storage_read_failed" }, + )); + + expect(response.error.code).toBe(-32603); + expect(response.error.message).toBe( + "storage_read_failed: ADE couldn't read this project's data at /tmp/p/.ade/ade.db.", + ); + expect(response.error.data?.code).toBe("storage_read_failed"); + }); + + it("keeps a message that merely starts with an identifier and a colon", async () => { + // Only the error's OWN code may be stripped off the front. A message whose + // first word happens to be an identifier followed by a colon — a `gh` + // failure relayed verbatim, a Windows drive letter — would otherwise lose + // its head on the way out and reach the caller decapitated. + for (const [message, expected] of [ + ["gh: not authenticated. Run `gh auth login`.", "gh: not authenticated. Run `gh auth login`."], + ["C:\\Users\\Ada\\project is not a git repository.", "C:\\Users\\Ada\\project is not a git repository."], + ] as const) { + const response = await failWith(Object.assign(new Error(message), { + code: "github_cli_unavailable", + })); + + expect(response.error.message).toBe(`github_cli_unavailable: ${expected}`); + expect(response.error.data?.code).toBe("github_cli_unavailable"); + } + }); + + it("keeps a service-authored refusal readable", async () => { + const response = await failWith(new Error("Project root does not exist: /tmp/gone")); + expect(response.error.message).toBe("Project root does not exist: /tmp/gone"); + expect(response.error.data).toBeUndefined(); + }); +}); diff --git a/apps/ade-cli/src/jsonrpc.ts b/apps/ade-cli/src/jsonrpc.ts index 0ad4bf31e..6e4129835 100644 --- a/apps/ade-cli/src/jsonrpc.ts +++ b/apps/ade-cli/src/jsonrpc.ts @@ -1,4 +1,12 @@ import { Buffer } from "node:buffer"; +import { randomBytes } from "node:crypto"; +import { + encodeCodedErrorMessage, + isErrnoLikeCode, + parseCodedErrorMessage, + stripElectronErrorWrapper, + UNKNOWN_SYSTEM_ERRNO_PATTERN, +} from "../../desktop/src/shared/codedError"; export type JsonRpcId = string | number | null; @@ -73,6 +81,93 @@ type JsonRpcServerErrorReporter = ( context: JsonRpcServerErrorContext, ) => void; +/** + * An unclassified handler failure, reported in full to the process log while + * only `errorId` crosses the wire. Callers correlate the two. + */ +export type JsonRpcInternalErrorReport = { + errorId: string; + method: string; + error: unknown; +}; + +export type JsonRpcInternalErrorReporter = (report: JsonRpcInternalErrorReport) => void; + +function nextErrorId(): string { + return randomBytes(4).toString("hex"); +} + +/** + * Services signal known failures with a string `Error.code` (the same + * convention the desktop's `codedError` uses). Those messages are authored for + * people, so they may cross the boundary; everything else is an internal + * detail — a libuv errno, a Node internal code, a stack-bearing message, a + * path — and must not. + * + * The wire format itself is not this file's to invent: the desktop parses these + * replies with `parseCodedErrorMessage`, so the same module both reads the + * incoming error and writes the outgoing message. + */ +function readCodedErrorShape( + error: unknown, +): { code: string; message: string; rootPath?: string } | null { + if (!(error instanceof Error)) return null; + const rawCode = (error as Error & { code?: unknown }).code; + if (typeof rawCode !== "string" || !rawCode.trim()) return null; + if (isErrnoLikeCode(rawCode)) return null; + const parsed = parseCodedErrorMessage(error); + const code = parsed.code ?? rawCode.trim(); + // `parseCodedErrorMessage` drops ANY leading `identifier:` from the message, + // which is right for one this encoder already wrote ("storage_read_failed: + // …") and wrong for a service sentence that merely begins that way — "gh: + // not authenticated", or a Windows "C:\Users\… is not a repository" — whose + // first word would be silently eaten before the message is re-encoded below. + // So the strip is accepted only when the prefix WAS this error's own code. + const body = messageBody(error, parsed.rootPath); + return { + code, + message: body.startsWith(`${code}:`) ? parsed.message : body, + ...(parsed.rootPath ? { rootPath: parsed.rootPath } : {}), + }; +} + +/** + * Everything `parseCodedErrorMessage` removes EXCEPT its leading-identifier + * rule: the rootPath tail and the transport wrappers. + * + * `parsed.rootPath` is the message's own bytes after the delimiter, preserved + * verbatim, so trimming it back off by length is exact. + */ +function messageBody(error: Error, rootPath: string | undefined): string { + const raw = error.message; + return stripElectronErrorWrapper( + rootPath ? raw.slice(0, raw.length - rootPath.length - 1) : raw, + ); +} + +/** + * Whether the failure came from the runtime or the operating system rather + * than from a service deciding something about the caller's request. + * + * Only these are redacted. RPC handlers throw `JsonRpcError` for protocol + * failures and plain `Error`s carrying sentences meant for the person who + * asked ("Project root does not exist: …"), and blanking those would turn + * every actionable refusal into a reference number. What must never cross is + * the other kind: an errno the platform could not even name, a filesystem + * message quoting a path, or a runtime fault carrying a stack — the class of + * message that produced "Unknown system error -11: … read" on a user's screen. + */ +function isInternalRuntimeError(error: unknown): boolean { + if (!(error instanceof Error)) return true; + const raw = error as Error & { code?: unknown; errno?: unknown; syscall?: unknown }; + if (typeof raw.syscall === "string" && raw.syscall.length > 0) return true; + if (typeof raw.errno === "number") return true; + if (isErrnoLikeCode(raw.code)) return true; + if (UNKNOWN_SYSTEM_ERRNO_PATTERN.test(error.message)) return true; + // Runtime faults (TypeError, RangeError, …) are always programming errors. + return /^(?:Type|Range|Reference|Syntax|Eval|URI)Error$/.test(error.name); +} + function writeMessage( message: JsonRpcResponse | JsonRpcResponse[] | JsonRpcNotification, mode: TransportMode, @@ -87,7 +182,11 @@ function writeMessage( writeFn(framed); } -function toErrorResponse(id: JsonRpcId, error: unknown): JsonRpcFailure { +function toErrorResponse( + id: JsonRpcId, + error: unknown, + context: { method: string; onInternalError?: JsonRpcInternalErrorReporter }, +): JsonRpcFailure { if (error instanceof JsonRpcError) { return { jsonrpc: "2.0", @@ -100,12 +199,52 @@ function toErrorResponse(id: JsonRpcId, error: unknown): JsonRpcFailure { }; } + // A coded failure is a verdict the service already made and worded; forward + // the code so the caller can act on it, in the `code: message` shape the + // desktop already parses. + const coded = readCodedErrorShape(error); + if (coded) { + return { + jsonrpc: "2.0", + id, + error: { + code: JsonRpcErrorCode.internalError, + message: encodeCodedErrorMessage( + coded.code, + coded.message, + coded.rootPath ? { rootPath: coded.rootPath } : undefined, + ), + data: { code: coded.code }, + } + }; + } + + if (isInternalRuntimeError(error)) { + // The message and stack go to the process log — which `ade report-issue` + // collects — and the caller gets only the reference that finds them. + const errorId = nextErrorId(); + try { + context.onInternalError?.({ errorId, method: context.method, error }); + } catch { + // Reporting must not become a second failure path. + } + return { + jsonrpc: "2.0", + id, + error: { + code: JsonRpcErrorCode.internalError, + message: `Internal error in ${context.method} (ref ${errorId})`, + data: { errorId }, + } + }; + } + return { jsonrpc: "2.0", id, error: { code: JsonRpcErrorCode.internalError, - message: error instanceof Error ? error.message : String(error) + message: (error as Error).message, } }; } @@ -118,6 +257,7 @@ async function handleSingleMessage( message: unknown, handler: JsonRpcHandler, onError?: JsonRpcServerErrorReporter, + onInternalError?: JsonRpcInternalErrorReporter, ): Promise { if (!isValidRequest(message)) { return { @@ -172,7 +312,7 @@ async function handleSingleMessage( result: result ?? {} }; } catch (error) { - return toErrorResponse(id, error); + return toErrorResponse(id, error, { method: request.method, onInternalError }); } } @@ -291,8 +431,9 @@ async function dispatchPayload(args: { transport: TransportMode; writeFn: (data: string) => void; onError?: JsonRpcServerErrorReporter; + onInternalError?: JsonRpcInternalErrorReporter; }): Promise { - const { payloadText, handler, transport, writeFn, onError } = args; + const { payloadText, handler, transport, writeFn, onError, onInternalError } = args; const trimmed = payloadText.trim(); if (!trimmed.length) return; @@ -338,7 +479,7 @@ async function dispatchPayload(args: { } const results = ( - await Promise.all(parsed.map((entry) => handleSingleMessage(entry, handler, onError))) + await Promise.all(parsed.map((entry) => handleSingleMessage(entry, handler, onError, onInternalError))) ).filter((entry): entry is JsonRpcResponse => entry != null); if (results.length) { @@ -347,7 +488,7 @@ async function dispatchPayload(args: { return; } - const response = await handleSingleMessage(parsed, handler, onError); + const response = await handleSingleMessage(parsed, handler, onError, onInternalError); if (response) { writeMessage(response, transport, writeFn); } @@ -366,6 +507,11 @@ export interface JsonRpcServerOptions { nonFatal?: boolean; /** Called for transport or notification failures that are contained by the server. */ onError?: JsonRpcServerErrorReporter; + /** + * Called with the full error behind a redacted `internalError` reply. Wire it + * to the process log: the reply carries only the matching `errorId`. + */ + onInternalError?: JsonRpcInternalErrorReporter; } export function startJsonRpcServer(handler: JsonRpcHandler, transport: JsonRpcTransport, options?: JsonRpcServerOptions): JsonRpcServerHandle { @@ -415,6 +561,7 @@ export function startJsonRpcServer(handler: JsonRpcHandler, transport: JsonRpcTr transport: args.transport, writeFn, onError: reportError, + ...(options?.onInternalError ? { onInternalError: options.onInternalError } : {}), }) .catch((error) => { reportError(error, "dispatch"); diff --git a/apps/ade-cli/src/services/account/accountMachinePublisherService.test.ts b/apps/ade-cli/src/services/account/accountMachinePublisherService.test.ts index 53c07d080..72658574f 100644 --- a/apps/ade-cli/src/services/account/accountMachinePublisherService.test.ts +++ b/apps/ade-cli/src/services/account/accountMachinePublisherService.test.ts @@ -202,6 +202,159 @@ describe("account machine publisher health", () => { }); }); + it("sends the account-salted hardware anchor on the heartbeat, and omits it when there is none", async () => { + const bodies: Array> = []; + const fetchImpl = vi.fn(async (_url: unknown, init?: RequestInit) => { + bodies.push(JSON.parse(String(init?.body)) as Record); + return new Response(null, { status: 204 }); + }); + const readAccountHardwareId = vi.fn((userId: string) => `anchor-for-${userId}`); + const options = { + getAccessToken: async () => "account-token", + getAccountStatus: () => ({ + signedIn: true, + userId: "account-user", + sessionReadState: "available" as const, + }), + getSnapshot: async () => snapshot(), + getMachineKey: () => "machine-studio", + directoryBaseUrl: () => "https://directory.example", + fetchImpl: fetchImpl as unknown as typeof fetch, + }; + + await createAccountMachinePublisherService({ ...options, readAccountHardwareId }).publishNow(); + // Salted with the account id, and sent on the plain heartbeat: a row can + // only be matched on a later reinstall if it stored an anchor first. + expect(readAccountHardwareId).toHaveBeenCalledWith("account-user"); + expect(bodies[0]).toMatchObject({ + machineKey: "machine-studio", + deviceId: "device-studio", + hardwareId: "anchor-for-account-user", + }); + expect(bodies[0]).not.toHaveProperty("pairing"); + + // No reader wired, no anchor to read, and a reader that throws are all the + // same ordinary outcome: the field is absent and the publish succeeds. + await createAccountMachinePublisherService(options).publishNow(); + await createAccountMachinePublisherService({ + ...options, + readAccountHardwareId: () => null, + }).publishNow(); + const thrower = createAccountMachinePublisherService({ + ...options, + readAccountHardwareId: () => { + throw new Error("ioreg unavailable"); + }, + }); + await thrower.publishNow(); + + expect(bodies.slice(1).every((body) => !("hardwareId" in body))).toBe(true); + expect(thrower.getPublisherHealth().state).toBe("published"); + }); + + it("sends no anchor when the publisher has no account id to salt with", async () => { + const bodies: Array> = []; + const readAccountHardwareId = vi.fn(() => "anchor"); + const service = createAccountMachinePublisherService({ + getAccessToken: async () => "account-token", + // The env-token publisher: authenticated, but with no account identity. + // Hashing without the salt would produce a value shared across accounts. + getAccountStatus: () => ({ + signedIn: false, + source: "env-token" as const, + sessionReadState: "available" as const, + }), + getSnapshot: async () => snapshot(), + getMachineKey: () => "machine-studio", + directoryBaseUrl: () => "https://directory.example", + readAccountHardwareId, + fetchImpl: (async (_url: unknown, init?: RequestInit) => { + bodies.push(JSON.parse(String(init?.body)) as Record); + return new Response(null, { status: 204 }); + }) as unknown as typeof fetch, + }); + + await service.publishNow(); + + expect(readAccountHardwareId).not.toHaveBeenCalled(); + expect(bodies[0]).not.toHaveProperty("hardwareId"); + expect(service.getPublisherHealth().state).toBe("published"); + }); + + it("keeps the anchor alongside a deliberate pairing publish and its grant", async () => { + const bodies: Array> = []; + const service = createAccountMachinePublisherService({ + getAccessToken: async () => "account-token", + getAccountStatus: () => ({ + signedIn: true, + userId: "account-user", + sessionReadState: "available" as const, + }), + getSnapshot: async () => snapshot(), + getMachineKey: () => "machine-studio", + directoryBaseUrl: () => "https://directory.example", + consumePairingGrant: () => "grant-token", + readAccountHardwareId: () => "anchor-for-account-user", + fetchImpl: (async (_url: unknown, init?: RequestInit) => { + bodies.push(JSON.parse(String(init?.body)) as Record); + return new Response(null, { status: 204 }); + }) as unknown as typeof fetch, + }); + + await service.publishPairing(); + + // The anchor is the identifier a proven re-pair supersedes on; a payload + // that carried the proof but not the anchor would prove nothing about it. + expect(bodies[0]).toMatchObject({ + pairing: true, + pairingGrant: "grant-token", + hardwareId: "anchor-for-account-user", + }); + }); + + it("confirms only its own superseded machine keys, and tolerates a directory that sends none", async () => { + const confirmSupersededMachineKeys = vi.fn((keys: readonly unknown[]) => + (keys as string[]).filter((key) => key === "machine-old")); + const info = vi.fn(); + const warn = vi.fn(); + let body: BodyInit | null = JSON.stringify({ + ok: true, + supersededMachineKeys: ["machine-old", "somebody-elses-machine"], + }); + const service = createAccountMachinePublisherService({ + getAccessToken: async () => "account-token", + getAccountStatus: () => ({ signedIn: true, sessionReadState: "available" as const }), + getSnapshot: async () => snapshot(), + getMachineKey: () => "machine-studio", + directoryBaseUrl: () => "https://directory.example", + confirmSupersededMachineKeys, + fetchImpl: vi.fn(async () => new Response(body, { + status: 200, + headers: body ? { "content-type": "application/json" } : undefined, + })), + logger: { info, warn }, + }); + + await service.publishNow(); + + expect(confirmSupersededMachineKeys).toHaveBeenCalledWith([ + "machine-old", + "somebody-elses-machine", + ]); + expect(info).toHaveBeenCalledWith("account.machine_identity_superseded_confirmed", { + machineKey: "machine-studio", + previousMachineKeys: ["machine-old"], + }); + + // An older directory answers 204 with no body at all; the publish still + // succeeds and nothing is claimed. + confirmSupersededMachineKeys.mockClear(); + body = null; + await service.publishNow(); + expect(confirmSupersededMachineKeys).not.toHaveBeenCalled(); + expect(service.getPublisherHealth().state).toBe("published"); + }); + it("samples successful leg durations and escalates slow legs to warn", async () => { let clock = 0; let tokenDelayMs = 2; @@ -1217,6 +1370,7 @@ describe("brain account machine publisher directory policy", () => { isSyncEnabled: () => true, getSnapshot: async () => snapshot(), getMachineKey: () => "machine-studio", + confirmSupersededMachineKeys: () => [], directoryBaseUrl: () => DEVELOPMENT_ADE_ACCOUNT_DIRECTORY_URL, logger: { info: vi.fn(), warn: vi.fn() }, // This test is about directory routing; no power monitor, so no real @@ -1256,6 +1410,7 @@ describe("brain account machine publisher directory policy", () => { isSyncEnabled: () => true, getSnapshot: async () => snapshot(), getMachineKey: () => "machine-studio", + confirmSupersededMachineKeys: () => [], logger: { info: vi.fn(), warn: vi.fn() }, }); diff --git a/apps/ade-cli/src/services/account/accountMachinePublisherService.ts b/apps/ade-cli/src/services/account/accountMachinePublisherService.ts index 596b78c09..21d84327b 100644 --- a/apps/ade-cli/src/services/account/accountMachinePublisherService.ts +++ b/apps/ade-cli/src/services/account/accountMachinePublisherService.ts @@ -39,6 +39,7 @@ import { } from "../sync/machineIdentitySigningStore"; import { trackBrainLoopWatchdogCommand } from "../runtime/brainLoopWatchdog"; import { createEpisodeAnalytics } from "./episodeAnalytics"; +import { readAccountHardwareId } from "./hardwareAnchor"; export const ACCOUNT_MACHINE_HEARTBEAT_MS = 30_000; export const ACCOUNT_MACHINE_RELAY_STATE_POLL_MS = 2_000; @@ -65,6 +66,23 @@ export const PUBLISH_FAILURE_ANALYTICS_THRESHOLD_MS = 120_000; export type AccountMachineRegistration = { machineKey: string; deviceId: string; + /** + * A per-account hash of this computer's OS-level machine identifier, when one + * can be read. See `hardwareAnchor.ts` for the recipe and for why the raw + * identifier never appears here. + * + * It exists because `machineKey` and `deviceId` BOTH live under `~/.ade`, so + * a user who wipes that directory and signs in again arrives as a machine the + * directory has never seen and leaves their old row behind as a phantom. This + * is the one identifier that survives the wipe. + * + * Optional in every direction: a host that cannot produce an anchor omits it, + * and a directory that predates it ignores it. It is sent on the heartbeat as + * well as on a deliberate pairing, because a row can only be matched later if + * it stored an anchor at some point — but storing one authorizes nothing on + * its own. Superseding still requires the same fresh-authentication proof. + */ + hardwareId?: string; name: string; platform: string; deviceType: string; @@ -271,6 +289,26 @@ function readHttpReasonBounded(response: Response): Promise { return boundedBodyRead(response, readAccountDirectoryHttpReason); } +/** + * Read the keys a compatible directory says it retired for THIS device. + * + * Purely additive: a directory that predates the field, or answers 204, yields + * an empty list and nothing downstream changes. It exists so a rotation this + * machine performed can be confirmed as clean — the alternative is a roster + * where the owner cannot tell a retired key from a live one, which is how a + * working MacBook came to be deleted by hand. + */ +async function readSupersededMachineKeysBounded(response: Response): Promise { + const keys = await boundedBodyRead(response, async (bounded) => { + const body: unknown = await bounded.clone().json(); + if (!body || typeof body !== "object") return null; + const raw = (body as Record).supersededMachineKeys; + if (!Array.isArray(raw)) return null; + return raw.filter((entry): entry is string => typeof entry === "string" && entry.trim() !== ""); + }); + return keys ?? []; +} + function failureLegForState( state: SyncAccountDirectoryHealth["state"], ): AccountMachinePublishLeg { @@ -546,6 +584,21 @@ export function createAccountMachinePublisherService(options: { * user's one proof on a request that has no use for it. */ consumePairingGrant?: () => string | null; + /** + * Hand the directory's `supersededMachineKeys` to the identity store, which + * answers with the subset this machine actually retired. Optional so small + * and test publishers stay unaffected; absent means "do not reconcile". + */ + confirmSupersededMachineKeys?: (keys: readonly unknown[]) => string[]; + /** + * The per-account hardware anchor for the signed-in owner, or null when this + * host has none. Called with the account id because the value is salted with + * it — see `hardwareAnchor.ts`. + * + * Optional so small and test publishers stay unaffected; absent means "send + * no anchor", which is exactly what every client did before this existed. + */ + readAccountHardwareId?: (userId: string) => string | null; /** * This machine's power and sleep state. Optional: a publisher without one * simply omits the fields, which is what an older host does anyway. @@ -1135,13 +1188,29 @@ export function createAccountMachinePublisherService(options: { pairingGrant = null; } } - const registration: AccountMachineRegistration = isPairingPublish - ? { - ...registrationWithRelayHint, - pairing: true, - ...(pairingGrant ? { pairingGrant } : {}), + // Read here rather than in `buildAccountMachineRegistration`, because the + // account id is the salt and the builder has no account context — it also + // runs on the relay-state poll, which only compares route signatures and + // never sends anything. A throwing or absent reader is an ordinary "no + // anchor": this field must never be the reason a machine fails to publish. + let hardwareId: string | null = null; + if (options.readAccountHardwareId && accountOwnerId) { + try { + hardwareId = options.readAccountHardwareId(accountOwnerId)?.trim() || null; + } catch { + hardwareId = null; } - : registrationWithRelayHint; + } + const registration: AccountMachineRegistration = { + ...(isPairingPublish + ? { + ...registrationWithRelayHint, + pairing: true, + ...(pairingGrant ? { pairingGrant } : {}), + } + : registrationWithRelayHint), + ...(hardwareId ? { hardwareId } : {}), + }; const reachableEndpointCount = registration.reachableEndpoints.length; let accessToken: string | null = null; @@ -1258,6 +1327,26 @@ export function createAccountMachinePublisherService(options: { warnOnce("http_error", "http", legDurations, { status: response.status }); return; } + // Read before draining. A directory that reports retired keys is telling + // this machine its rotation landed cleanly; nothing has to act on it, but + // an unexplained rotation is exactly what nobody could explain last time. + if (options.confirmSupersededMachineKeys) { + try { + const superseded = await readSupersededMachineKeysBounded(response); + const confirmed = superseded.length > 0 + ? options.confirmSupersededMachineKeys(superseded) + : []; + if (confirmed.length > 0) { + options.logger?.info?.("account.machine_identity_superseded_confirmed", { + machineKey, + previousMachineKeys: confirmed, + }); + } + } catch { + // An older directory sends no body at all; never fail a good publish + // over bookkeeping. + } + } await response.body?.cancel().catch(() => {}); lastWarning = null; resetPublishCadence(); @@ -1638,6 +1727,15 @@ export function createBrainAccountMachinePublisherService(options: { isSyncEnabled: () => boolean; getSnapshot: () => Promise; getMachineKey: () => string; + /** + * Passed in rather than built here, from the ONE relay store the caller + * already holds. Two instances over one identity file is two of everything + * that file protects — two backup reconcilers, two rotation-lock clients — + * and the machine key this publisher reports comes from the caller's instance + * anyway, so a private second one could confirm a supersession against a + * different read of the same file. + */ + confirmSupersededMachineKeys: (keys: readonly unknown[]) => string[]; directoryBaseUrl?: () => string | null | undefined; logger: BrainAccountMachinePublisherLogger; captureAnalytics?: (input: ProductAnalyticsCapture) => void; @@ -1694,6 +1792,20 @@ export function createBrainAccountMachinePublisherService(options: { // Same shared auth service the access token comes from, so the grant a // device sign-in earned in this brain reaches the publish that needs it. consumePairingGrant: () => accountAuthService.consumePairingGrant(), + confirmSupersededMachineKeys: options.confirmSupersededMachineKeys, + // The only identity input that does not come out of `secretsDir`, which is + // the entire point: everything else in this composition is destroyed by the + // `~/.ade` wipe this anchor exists to survive. Cached for the process, so + // the heartbeat pays for the lookup once. + // + // The ADE home path is what keeps a Beta install from claiming Stable's + // row: the platform UUID underneath is shared by every ADE on the box, and + // the install's own home directory is the thing that differs. It is the + // parent of `secretsDir` by construction (`/secrets`), so the + // publisher anchors the install it actually serves rather than whatever + // `ADE_HOME` this process happened to launch with. + readAccountHardwareId: (userId) => + readAccountHardwareId(userId, path.dirname(options.secretsDir)), directoryBaseUrl: () => { const explicit = options.directoryBaseUrl?.(); if (explicit?.trim()) { diff --git a/apps/ade-cli/src/services/account/hardwareAnchor.test.ts b/apps/ade-cli/src/services/account/hardwareAnchor.test.ts new file mode 100644 index 000000000..3fcd70b67 --- /dev/null +++ b/apps/ade-cli/src/services/account/hardwareAnchor.test.ts @@ -0,0 +1,291 @@ +import * as childProcess from "node:child_process"; +import { createHash } from "node:crypto"; +import fs from "node:fs"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { canonicalWindowsPath } from "../projects/machineLayout"; +import { + canonicalAdeHomePath, + HARDWARE_ANCHOR_DOMAIN, + hardwareAnchorId, + LINUX_MACHINE_ID_PATHS, + normalizeHardwareAnchorUuid, + parseIoregPlatformUuid, + parseWindowsMachineGuid, + probeHardwareAnchorUuid, + readAccountHardwareId, + readHardwareAnchorUuid, + resetHardwareAnchorCacheForTests, +} from "./hardwareAnchor"; + +/** + * The ADE home every install-agnostic assertion below anchors against, already + * in canonical form so the expected digests are the same on Windows (where + * canonicalisation lowercases) as on POSIX. + */ +const ADE_HOME = canonicalAdeHomePath("/home/ada/.ade"); + +/** + * Captured from `ioreg -rd1 -c IOPlatformExpertDevice` on Apple silicon. Kept + * verbatim — the surrounding keys and indentation are exactly what a parser + * written against the format rather than the pair would trip over. + */ +const IOREG_OUTPUT = `+-o J316sAP + { + "IOPolledInterface" = "AppleARMWatchdogTimerHibernateHandler is not serializable" + "IOPlatformSerialNumber" = "H2XYZ1234567" + "IOPlatformUUID" = "8F4E2C1A-9B3D-5E6F-A7B8-C9D0E1F2A3B4" + "platform-name" = <"t6000"> + "target-type" = <"J316s"> + } +`; + +/** Captured from `reg query HKLM\\SOFTWARE\\Microsoft\\Cryptography /v MachineGuid`. */ +const REG_OUTPUT = "\r\n" + + "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Cryptography\r\n" + + " MachineGuid REG_SZ 4a9e1f52-3f4b-4c1d-9a70-1f2e3d4c5b6a\r\n" + + "\r\n"; + +const LINUX_MACHINE_ID = "b7d3f0a1c2e94d5f8a6b0c1d2e3f4a5b\n"; + +/** + * The real `spawnSync`, counted rather than replaced. + * + * `readHardwareAnchorUuid` deliberately has no dependency seam — that is the + * point of it — so the only way to prove its cache is to watch the syscall + * underneath. `vi.spyOn` cannot patch a node builtin's namespace ("Cannot + * redefine property"), so the module is mocked as a pass-through instead. + */ +vi.mock("node:child_process", async () => { + const actual = await vi.importActual("node:child_process"); + const mocked = { ...actual, spawnSync: vi.fn(actual.spawnSync) }; + return { ...mocked, default: mocked }; +}); + +afterEach(() => { + resetHardwareAnchorCacheForTests(); + vi.restoreAllMocks(); +}); + +describe("hardware anchor parsers", () => { + it("reads IOPlatformUUID out of real ioreg output", () => { + expect(parseIoregPlatformUuid(IOREG_OUTPUT)).toBe("8f4e2c1a-9b3d-5e6f-a7b8-c9d0e1f2a3b4"); + }); + + it("reads MachineGuid out of real reg query output", () => { + expect(parseWindowsMachineGuid(REG_OUTPUT)).toBe("4a9e1f52-3f4b-4c1d-9a70-1f2e3d4c5b6a"); + }); + + it("matches the value line by name and type, not by position", () => { + // A `/s`-style answer with siblings, and a decoy line naming the value in + // prose. Taking "the last token of the third line" would pick either. + const noisy = "\r\n" + + "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Cryptography\r\n" + + " MachineGuidBackup REG_SZ 00000000-0000-0000-0000-000000000000\r\n" + + " MachineGuid REG_SZ 4a9e1f52-3f4b-4c1d-9a70-1f2e3d4c5b6a\r\n"; + expect(parseWindowsMachineGuid(noisy)).toBe("4a9e1f52-3f4b-4c1d-9a70-1f2e3d4c5b6a"); + }); + + it.each([ + ["absent property", '"IOPlatformSerialNumber" = "H2XYZ1234567"'], + ["empty property", '"IOPlatformUUID" = ""'], + ["all-zero sentinel", '"IOPlatformUUID" = "00000000-0000-0000-0000-000000000000"'], + ])("returns null for %s", (_label, output) => { + expect(parseIoregPlatformUuid(output)).toBeNull(); + }); + + it.each([ + ["a value the query did not return", "\r\nHKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Cryptography\r\n"], + ["an error page", "ERROR: The system was unable to find the specified registry key or value.\r\n"], + ])("returns null for %s", (_label, output) => { + expect(parseWindowsMachineGuid(output)).toBeNull(); + }); + + it("accepts a bare-hex machine-id and a brace-wrapped guid, rejects junk", () => { + expect(normalizeHardwareAnchorUuid(LINUX_MACHINE_ID)).toBe("b7d3f0a1c2e94d5f8a6b0c1d2e3f4a5b"); + expect(normalizeHardwareAnchorUuid("{4A9E1F52-3F4B-4C1D-9A70-1F2E3D4C5B6A}")) + .toBe("4a9e1f52-3f4b-4c1d-9a70-1f2e3d4c5b6a"); + expect(normalizeHardwareAnchorUuid("uninitialized")).toBeNull(); + expect(normalizeHardwareAnchorUuid("00000000000000000000000000000000")).toBeNull(); + expect(normalizeHardwareAnchorUuid("deadbeef")).toBeNull(); + expect(normalizeHardwareAnchorUuid(null)).toBeNull(); + }); +}); + +describe("hardware anchor lookup", () => { + it("runs ioreg without a shell on macOS", () => { + const runCommand = vi.fn(() => IOREG_OUTPUT); + expect(probeHardwareAnchorUuid({ platform: "darwin", runCommand })) + .toBe("8f4e2c1a-9b3d-5e6f-a7b8-c9d0e1f2a3b4"); + expect(runCommand).toHaveBeenCalledWith("ioreg", ["-rd1", "-c", "IOPlatformExpertDevice"]); + }); + + it("queries the Cryptography key on Windows", () => { + const runCommand = vi.fn(() => REG_OUTPUT); + expect(probeHardwareAnchorUuid({ platform: "win32", runCommand })) + .toBe("4a9e1f52-3f4b-4c1d-9a70-1f2e3d4c5b6a"); + const [command, args] = runCommand.mock.calls[0] as unknown as [string, string[]]; + // Resolved through the trusted-tool path, never a bare "reg" off PATH. + expect(command.toLowerCase()).toContain("reg.exe"); + expect(args).toEqual([ + "query", + "HKLM\\SOFTWARE\\Microsoft\\Cryptography", + "/v", + "MachineGuid", + ]); + }); + + it("falls back to the dbus machine-id on Linux", () => { + const readFile = vi.fn((filePath: string) => + filePath === LINUX_MACHINE_ID_PATHS[1] ? LINUX_MACHINE_ID : null); + expect(probeHardwareAnchorUuid({ platform: "linux", readFile })) + .toBe("b7d3f0a1c2e94d5f8a6b0c1d2e3f4a5b"); + expect(readFile).toHaveBeenCalledWith(LINUX_MACHINE_ID_PATHS[0]); + }); + + it("returns null when the probe fails instead of throwing", () => { + // A sandbox that refuses to spawn, a VM with no platform UUID, a hardened + // image with no machine-id: all of them are "no anchor", never an error. + expect(probeHardwareAnchorUuid({ platform: "darwin", runCommand: () => null })).toBeNull(); + expect(probeHardwareAnchorUuid({ + platform: "darwin", + runCommand: () => { + throw new Error("EPERM"); + }, + })).toBeNull(); + expect(probeHardwareAnchorUuid({ platform: "linux", readFile: () => null })).toBeNull(); + }); +}); + +describe("account hardware id", () => { + /** The raw identifier a macOS probe would return, without touching the cache. */ + const macUuid = (): string | null => + probeHardwareAnchorUuid({ platform: "darwin", runCommand: () => IOREG_OUTPUT }); + + it("hashes with the versioned domain, the account salt, and the install path", () => { + const expected = createHash("sha256") + .update( + `${HARDWARE_ANCHOR_DOMAIN}:user_1:8f4e2c1a-9b3d-5e6f-a7b8-c9d0e1f2a3b4:${ADE_HOME}`, + ) + .digest("hex"); + + expect(hardwareAnchorId("user_1", "8f4e2c1a-9b3d-5e6f-a7b8-c9d0e1f2a3b4", ADE_HOME)) + .toBe(expected); + expect(readAccountHardwareId("user_1", ADE_HOME, macUuid)).toBe(expected); + }); + + it("never puts the raw identifier on the wire", () => { + const id = readAccountHardwareId("user_1", ADE_HOME, macUuid); + expect(id).toMatch(/^[0-9a-f]{64}$/); + expect(id).not.toContain("8f4e2c1a"); + }); + + it("gives two accounts on one machine unrelated values", () => { + // The point of the per-account salt: the directory can dedup within an + // account and cannot join across accounts. + expect(readAccountHardwareId("user_1", ADE_HOME, macUuid)) + .not.toBe(readAccountHardwareId("user_2", ADE_HOME, macUuid)); + }); + + it("separates two ADE installs that share one platform UUID", () => { + // Stable and Beta on one Mac, and a second OS user's `~/.ade`: the probe + // underneath returns the SAME identifier for all three, which is exactly + // why the hash cannot be built from it alone — they were superseding each + // other's directory row. + const stable = readAccountHardwareId("user_1", "/home/ada/.ade", macUuid); + const beta = readAccountHardwareId("user_1", "/home/ada/.ade-beta", macUuid); + const otherUser = readAccountHardwareId("user_1", "/home/bo/.ade", macUuid); + expect(new Set([stable, beta, otherUser]).size).toBe(3); + }); + + it("reproduces one install's anchor across a wipe and reinstall", () => { + // The north star: `~/.ade` is deleted and recreated, every secret in it is + // minted afresh, and the anchor still names the same machine. + expect(readAccountHardwareId("user_1", "/home/ada/.ade/", macUuid)) + .toBe(readAccountHardwareId("user_1", "/home/ada/./.ade", macUuid)); + }); + + it("treats a Windows home path case-insensitively and a POSIX one exactly", () => { + // NTFS is case-insensitive, so one directory must not hash as two installs. + expect(canonicalAdeHomePath("C:\\Users\\Ada\\.ade", "win32")) + .toBe(canonicalAdeHomePath("c:\\users\\ada\\.ade", "win32")); + expect(canonicalAdeHomePath("/home/Ada/.ade", "linux")) + .not.toBe(canonicalAdeHomePath("/home/ada/.ade", "linux")); + }); + + it("folds a Windows home path the same way the runtime pipe identity does", () => { + // Routed through `canonicalWindowsPath`, not a bare `path.resolve`: that is + // what makes `realpath` — and so 8.3 short names and junction casing — + // collapse to one spelling. Forward slashes are the part of that fold this + // test can prove off Windows; `path.resolve` on a POSIX host would not even + // recognise the drive letter and would join both spellings onto its cwd. + expect(canonicalAdeHomePath("C:/Users/Ada/.ade", "win32")) + .toBe(canonicalAdeHomePath("C:\\Users\\Ada\\.ade", "win32")); + expect(canonicalAdeHomePath("C:\\Users\\Ada\\.ade", "win32")) + .toBe(canonicalWindowsPath("C:\\Users\\Ada\\.ade").toLowerCase()); + }); + + it("returns null with no account id and with no anchor", () => { + expect(readAccountHardwareId(null, ADE_HOME, macUuid)).toBeNull(); + expect(readAccountHardwareId(" ", ADE_HOME, macUuid)).toBeNull(); + expect(readAccountHardwareId("user_1", ADE_HOME, () => null)).toBeNull(); + }); +}); + +describe("hardware anchor cache", () => { + it("probes once and answers every later read without touching the host", () => { + // The cache is unconditional now: it used to switch itself off whenever the + // caller passed any argument at all, which made "is this cached?" depend on + // how the call happened to be spelled. + // + // Counted at the real seams rather than compared as return values. There is + // no dependency seam on `readHardwareAnchorUuid`, so two equal answers prove + // nothing — an implementation that re-probed on every call would produce + // them, and on a host with no anchor at all both would simply be null. + resetHardwareAnchorCacheForTests(); + const spawnSpy = vi.mocked(childProcess.spawnSync); + spawnSpy.mockClear(); + const readFileSpy = vi.spyOn(fs, "readFileSync"); + const probeCalls = (): number => spawnSpy.mock.calls.length + readFileSpy.mock.calls.length; + + const first = readHardwareAnchorUuid(); + // Whichever probe this host uses — `ioreg` / `reg` through spawnSync, or + // `/etc/machine-id` through fs — the first read really did reach it. Without + // this the count assertion below would hold for a spy that intercepted + // nothing, which is the vacuous-assertion trap itself. + const afterFirstProbe = probeCalls(); + expect(afterFirstProbe).toBeGreaterThan(0); + + expect(readHardwareAnchorUuid()).toBe(first); + expect(readHardwareAnchorUuid()).toBe(first); + // Including a null first answer: a machine with no anchor must not respawn + // `ioreg` twice a minute forever to keep learning the same thing. + expect(probeCalls()).toBe(afterFirstProbe); + }); + + it("leaves the explicit probe uncached", () => { + const runCommand = vi.fn(() => IOREG_OUTPUT); + probeHardwareAnchorUuid({ platform: "darwin", runCommand }); + probeHardwareAnchorUuid({ platform: "darwin", runCommand }); + expect(runCommand).toHaveBeenCalledTimes(2); + }); + + it("memoizes the canonical home and clears that memo on reset too", () => { + // The Windows fold is the half with real work behind it: a `realpath` walk + // that can BLOCK on a stalled network home, on the 30-second publish path. + const nativeSpy = vi.spyOn(fs.realpathSync, "native"); + const home = "C:\\Users\\Ada\\.ade-memo-probe"; + + const canonical = canonicalAdeHomePath(home, "win32"); + const afterFirst = nativeSpy.mock.calls.length; + expect(afterFirst).toBeGreaterThan(0); + expect(canonicalAdeHomePath(home, "win32")).toBe(canonical); + expect(nativeSpy.mock.calls.length).toBe(afterFirst); + + // The reset seam owns BOTH process-lifetime caches. Clearing only the anchor + // uuid would leave a test reading a canonicalisation computed under the + // previous test's cwd or platform stub. + resetHardwareAnchorCacheForTests(); + expect(canonicalAdeHomePath(home, "win32")).toBe(canonical); + expect(nativeSpy.mock.calls.length).toBeGreaterThan(afterFirst); + }); +}); diff --git a/apps/ade-cli/src/services/account/hardwareAnchor.ts b/apps/ade-cli/src/services/account/hardwareAnchor.ts new file mode 100644 index 000000000..21c3b017f --- /dev/null +++ b/apps/ade-cli/src/services/account/hardwareAnchor.ts @@ -0,0 +1,342 @@ +import { spawnSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import fs from "node:fs"; +import path from "node:path"; +import { resolveTrustedWindowsTool } from "../../lib/trustedWindowsTools"; +import { canonicalWindowsPath } from "../projects/machineLayout"; + +/** + * The one piece of machine identity a reinstall cannot destroy. + * + * Both halves of ADE's machine identity live under `~/.ade/secrets` — the + * machine key in `sync-cloud-relay.json`, the device id in `sync-device-id` — + * so a user who deletes `~/.ade` and signs in again mints BOTH afresh. The + * directory's device-supersede dedup then has nothing to match on, and the + * account keeps a phantom row for a computer the user only owns once. That is + * the whole remaining gap in "reinstall + sign in again heals everything". + * + * The operating system already knows this machine's name and will still know it + * after the wipe: `IOPlatformUUID` on macOS, `MachineGuid` on Windows, + * `/etc/machine-id` on Linux. Anchoring on that closes the gap without asking + * the user for anything. + * + * THREE RULES GOVERN EVERYTHING BELOW. + * + * 1. The raw identifier NEVER leaves this process. What goes on the wire is + * `sha256("ade-machine-anchor-v2:" + userId + ":" + rawUuid + ":" + adeHome)`, + * salted with the account id, so the same computer signed into two accounts + * produces two unrelated values and no server-side join can correlate them. + * A raw platform UUID is a stable, cross-application device fingerprint; a + * per-account hash of one is not. + * 2. An anchor identifies an ADE INSTALL, not a chassis. The platform UUID is + * shared by every ADE on the box — Stable in `~/.ade`, Beta in `~/.ade-beta`, + * a second OS user's `~/.ade` — and hashing it alone made all of them one + * machine that took turns superseding each other's directory row. Folding in + * the ADE home path separates them while preserving the north star: a wipe + * and reinstall lands on the SAME path, so it still reproduces the same + * anchor, which is the entire reason this file exists. + * 3. It is OPTIONAL end to end. VMs with no platform UUID, hardened Linux + * images with no machine-id, a sandbox that refuses to spawn `ioreg` — all + * of them return null and every caller carries on with exactly the behavior + * it had before this existed. An anchor is an improvement to dedup, never a + * precondition for registering a machine. + */ + +/** + * Domain separator baked into every hash. + * + * Versioned because the recipe is a wire contract with the directory: rows + * carry the hash, so changing the salt, the separator, or the digest silently + * orphans every anchor already stored. + * + * `-v2` adds the ADE home path to the recipe. NO MIGRATION IS SHIPPED and none + * is needed: a v1-hashed row simply stops matching anything this client sends, + * so it dedups on `device_id` exactly as a pre-anchor client's row always did, + * and ages out with the device or by the owner removing it. The cost of a + * stale, unmatched anchor column is nil; the cost of two installs claiming one + * row is the bug this bump fixes. + */ +export const HARDWARE_ANCHOR_DOMAIN = "ade-machine-anchor-v2"; + +/** + * Where Linux keeps its stable machine identifier, in preference order. + * + * `/etc/machine-id` is the systemd location and the one that survives an ADE + * reinstall; the dbus path is the older fallback that some images still ship as + * the only populated file. A distro with neither simply has no anchor. + */ +export const LINUX_MACHINE_ID_PATHS = ["/etc/machine-id", "/var/lib/dbus/machine-id"] as const; + +/** + * Hard bound on the identity probe. + * + * The probe runs on the account-publish path, and a `reg` or `ioreg` call that + * hangs must cost the heartbeat a couple of seconds once, not block it. A + * timeout is indistinguishable from "no anchor here" and is treated as such. + */ +const PROBE_TIMEOUT_MS = 2_000; +const PROBE_MAX_OUTPUT_BYTES = 1024 * 1024; + +/** Overridable for tests, which feed captured command output instead of running commands. */ +export type HardwareAnchorDeps = { + platform?: NodeJS.Platform; + /** Returns the command's stdout, or null for any failure at all. */ + runCommand?: (command: string, args: readonly string[]) => string | null; + /** Returns the file's contents, or null when it does not exist or cannot be read. */ + readFile?: (filePath: string) => string | null; +}; + +/** + * Process-lifetime cache, including the negative answer. + * + * The publisher asks for this on every 30-second heartbeat, and the answer + * cannot change while the process runs — the identifier is a property of the + * hardware and the OS install. Caching the null too is deliberate: a machine + * with no anchor must not respawn `ioreg` twice a minute forever to keep + * learning the same thing. + */ +let cachedAnchorUuid: { value: string | null } | null = null; + +/** + * Test seam only. Nothing in the product invalidates these caches. + * + * Both of them, deliberately: the canonical-home memo below is the module's + * other process-lifetime cache, and a reset that cleared one but not the other + * would let a test read a canonicalisation computed under a previous test's + * cwd or platform stub. + */ +export function resetHardwareAnchorCacheForTests(): void { + cachedAnchorUuid = null; + canonicalHomeByInput.clear(); +} + +/** + * Accept only something that actually looks like a machine identifier. + * + * The three sources disagree on shape — a dashed UUID on macOS and Windows, 32 + * bare hex characters on Linux — so the check is a charset and length bound + * rather than a UUID grammar. What it really exists to reject is the two ways + * these lookups "succeed" while telling us nothing: an empty value, and the + * all-zero sentinel that firmware and unprovisioned images report in place of + * an identifier. An all-zero anchor would be shared by every such machine on + * the account and would supersede rows that belong to different computers. + */ +export function normalizeHardwareAnchorUuid(raw: string | null | undefined): string | null { + const trimmed = (raw ?? "").trim().replace(/^\{/, "").replace(/\}$/, "").trim().toLowerCase(); + if (!/^[0-9a-f-]{16,64}$/.test(trimmed)) return null; + const hex = trimmed.replace(/-/g, ""); + if (hex.length < 16 || /^0+$/.test(hex)) return null; + return trimmed; +} + +/** + * Pull `IOPlatformUUID` out of `ioreg -rd1 -c IOPlatformExpertDevice`. + * + * The property sits in a plist-ish dump of one node, one `"key" = "value"` pair + * per line, so a single quoted-value match is the whole parse. Written against + * the text and not the surrounding format on purpose: the block's indentation + * and neighbouring keys differ between Intel and Apple silicon, and the pair + * itself does not. + */ +export function parseIoregPlatformUuid(output: string): string | null { + const match = output.match(/"IOPlatformUUID"\s*=\s*"([^"]*)"/); + return match ? normalizeHardwareAnchorUuid(match[1]) : null; +} + +/** + * Pull `MachineGuid` out of `reg query ... /v MachineGuid`. + * + * `reg` answers with a blank line, the key path, then one indented + * ` ` line per value. Neither the value name nor the type + * is localized, so matching on both is safe on any Windows UI language, and + * anchoring the name means a future `/s` query that returns sibling values + * cannot be misread. The value is taken as the rest of the line rather than a + * token, because splitting on whitespace would truncate anything unexpected + * into a plausible-looking prefix instead of rejecting it. + */ +export function parseWindowsMachineGuid(output: string): string | null { + for (const line of output.split(/\r?\n/)) { + const match = line.match(/^\s*MachineGuid\s+REG_[A-Z_]+\s+(\S.*)$/); + if (match) return normalizeHardwareAnchorUuid(match[1]); + } + return null; +} + +function runProbe(command: string, args: readonly string[]): string | null { + try { + // Argument array, never a shell string: nothing here is user input today, + // and the way that stops being true is someone interpolating a path into a + // command line that a shell then re-parses. + const result = spawnSync(command, [...args], { + encoding: "utf8", + timeout: PROBE_TIMEOUT_MS, + windowsHide: true, + maxBuffer: PROBE_MAX_OUTPUT_BYTES, + stdio: ["ignore", "pipe", "ignore"], + }); + if (result.error || result.status !== 0) return null; + return typeof result.stdout === "string" ? result.stdout : null; + } catch { + return null; + } +} + +function readProbeFile(filePath: string): string | null { + try { + return fs.readFileSync(filePath, "utf8"); + } catch { + return null; + } +} + +function probeAnchorUuid(deps: HardwareAnchorDeps): string | null { + const platform = deps.platform ?? process.platform; + const runCommand = deps.runCommand ?? runProbe; + const readFile = deps.readFile ?? readProbeFile; + + if (platform === "darwin") { + const output = runCommand("ioreg", ["-rd1", "-c", "IOPlatformExpertDevice"]); + return output ? parseIoregPlatformUuid(output) : null; + } + if (platform === "win32") { + let reg: string; + try { + // The GLOBALROOT-resolved reg.exe, not a bare "reg": PATH and cwd are + // caller-controlled in a CLI launch, and a planted reg.exe would get to + // choose this machine's identity. + reg = resolveTrustedWindowsTool("reg"); + } catch { + return null; + } + const output = runCommand(reg, [ + "query", + String.raw`HKLM\SOFTWARE\Microsoft\Cryptography`, + "/v", + "MachineGuid", + ]); + return output ? parseWindowsMachineGuid(output) : null; + } + for (const filePath of LINUX_MACHINE_ID_PATHS) { + const normalized = normalizeHardwareAnchorUuid(readFile(filePath)); + if (normalized) return normalized; + } + return null; +} + +/** + * Run the probe once with explicit dependencies, bypassing the cache. + * + * This is the test and diagnostics seam. It exists so `readHardwareAnchorUuid` + * does not have to guess whether it is being called by a test — the previous + * `Object.keys(deps).length === 0` check made "cached or not" a property of how + * the caller happened to spell the argument, which is exactly the kind of + * implicit mode a cache should never have. + */ +export function probeHardwareAnchorUuid(deps: HardwareAnchorDeps = {}): string | null { + try { + return probeAnchorUuid(deps); + } catch { + // The no-throw guarantee is the module's contract, not a property of the + // individual probes: this sits on the account-publish path, and an anchor + // this host cannot produce must never be the reason a machine stops + // publishing. A throw is simply "no anchor". + return null; + } +} + +/** + * The raw per-machine identifier, or null when this host has none to give. + * + * Always cached, including the null. Product code wants `readAccountHardwareId`, + * which is the only form allowed to leave the process. + */ +export function readHardwareAnchorUuid(): string | null { + if (cachedAnchorUuid) return cachedAnchorUuid.value; + const value = probeHardwareAnchorUuid(); + cachedAnchorUuid = { value }; + return value; +} + +/** + * The ADE home path as it goes into the hash. + * + * Resolved so `~/.ade`, `~/.ade/`, and a relative `ADE_HOME` all agree. On + * Windows it goes through the same `canonicalWindowsPath` the runtime pipe + * identity uses, then is lowercased: NTFS is case-insensitive, so + * `C:\Users\Ada\.ade` and `c:\users\ada\.ade` are one directory, and `realpath` + * additionally folds the two spellings a bare `resolve` would keep apart — an + * 8.3 short name (`C:\Users\ADAOBI~1\.ade`) and a junction reached by a + * different casing. Any of those splitting the anchor would make one install + * look like two machines, which is the exact bug the anchor exists to prevent. + * Case is preserved everywhere else, where two spellings really are two paths. + * + * The result is memoized per input: this sits on the 30-second publish path, + * and `canonicalWindowsPath`'s realpath walk can BLOCK (not throw) on a stalled + * network home. The home cannot change mid-process any more than the UUID can. + * The recipe itself is not frozen by the `-v2` domain — an install whose fold + * changes simply presents an unmatched anchor and dedups on `device_id`, the + * same documented fallback every pre-anchor client uses. + */ +const canonicalHomeByInput = new Map(); + +export function canonicalAdeHomePath( + adeHomePath: string, + platform: NodeJS.Platform = process.platform, +): string { + const cacheKey = `${platform}:${adeHomePath}`; + const cached = canonicalHomeByInput.get(cacheKey); + if (cached !== undefined) return cached; + const canonical = platform === "win32" + ? canonicalWindowsPath(adeHomePath).toLowerCase() + : path.resolve(adeHomePath); + canonicalHomeByInput.set(cacheKey, canonical); + return canonical; +} + +/** + * The wire value: this ADE install's anchor as seen by ONE account. + * + * Salting with the account id is what makes the field safe to send. The + * directory can compare two registrations from the same user and see the same + * install; it cannot compare two users and learn they share a computer, because + * the two hashes have no relationship it can compute. + */ +export function hardwareAnchorId( + userId: string, + rawUuid: string, + adeHomePath: string, + platform: NodeJS.Platform = process.platform, +): string { + return createHash("sha256") + .update( + `${HARDWARE_ANCHOR_DOMAIN}:${userId}:${rawUuid}:${canonicalAdeHomePath(adeHomePath, platform)}`, + ) + .digest("hex"); +} + +/** + * What the publisher sends, or null to send nothing. + * + * Null on two paths that are equally ordinary: no account id (the env-token + * publisher never has one) and no obtainable anchor. Neither is an error and + * neither is logged — the caller simply omits the field, and the directory + * falls back to matching on device id exactly as it does for every client that + * predates this. + * + * `adeHomePath` is the install this publisher speaks for. The caller already + * knows it (it is the parent of the secrets directory every other credential + * comes out of), so it is passed rather than re-derived from the environment — + * a brain launched with a different `ADE_HOME` than the one it is serving would + * otherwise anchor as the wrong install. + */ +export function readAccountHardwareId( + userId: string | null | undefined, + adeHomePath: string, + /** Test seam: supply the raw identifier instead of probing (and caching) it. */ + readUuid: () => string | null = readHardwareAnchorUuid, +): string | null { + const account = userId?.trim(); + if (!account) return null; + const rawUuid = readUuid(); + return rawUuid ? hardwareAnchorId(account, rawUuid, adeHomePath) : null; +} diff --git a/apps/ade-cli/src/services/account/machinePairingAutoRecovery.test.ts b/apps/ade-cli/src/services/account/machinePairingAutoRecovery.test.ts new file mode 100644 index 000000000..03ae9a86b --- /dev/null +++ b/apps/ade-cli/src/services/account/machinePairingAutoRecovery.test.ts @@ -0,0 +1,297 @@ +import { describe, expect, it, vi } from "vitest"; +import { createSyncAccountDirectoryHealth } from "../../../../desktop/src/shared/types"; +import type { SyncAccountDirectoryHealth } from "../../../../desktop/src/shared/types"; +import { + createMachinePairingAutoRecovery, + PAIRING_AUTO_REPAIR_DELAYS_MS, + PAIRING_AUTO_REPAIR_REVOCATION_QUIET_MS, + PAIRING_AUTO_REPAIR_SNAPSHOT_GRACE_MS, + type MachinePairingAutoRecoveryPublisher, +} from "./machinePairingAutoRecovery"; +import type { MachinePairingRepairResult } from "./machinePairingRepair"; + +const REFUSED_HEALTH = ( + code: "machine_revoked" | "pairing_authentication_required", +): SyncAccountDirectoryHealth => + createSyncAccountDirectoryHealth("http_error", "This machine was removed from your ADE account.", { + lastHttpStatus: 403, + lastHttpReason: code, + lastAttemptAt: 1_000, + failingSinceMs: 1_000, + }); + +const REPAIR_FAILED: MachinePairingRepairResult = { + repaired: false, + wasRevoked: true, + published: false, + pushRestored: false, + state: "http_error", + reason: "The account directory did not accept this machine.", + reasonCode: "machine_revoked", +}; + +const REPAIR_OK: MachinePairingRepairResult = { + repaired: true, + wasRevoked: true, + published: true, + pushRestored: true, + state: "published", + reason: null, +}; + +function harness(options: { + health: () => SyncAccountDirectoryHealth; + revoked: () => boolean; + /** ISO removal timestamp, as the directory reports it. Absent by default. */ + revokedAt?: () => string | null; + repair: () => Promise; + hasAccountSession?: () => boolean; + budgetLimit?: number; + now: () => number; +}) { + let spent = 0; + const limit = options.budgetLimit ?? 3; + const publisher: MachinePairingAutoRecoveryPublisher = { + getPublisherHealth: options.health, + getMachineRevocation: () => ({ + revoked: options.revoked(), + revokedAt: options.revokedAt?.() ?? null, + }), + }; + const recovery = createMachinePairingAutoRecovery({ + getPublisher: () => publisher, + runRepair: options.repair, + hasAccountSession: options.hasAccountSession ?? (() => true), + budget: { + tryConsumePairingAutoRepair: () => { + if (spent >= limit) return { allowed: false, countInWindow: spent, limit }; + spent += 1; + return { allowed: true, countInWindow: spent, limit }; + }, + }, + now: options.now, + }); + return { recovery, spentRepairs: () => spent }; +} + +describe("machinePairingAutoRecovery", () => { + it("repairs a machine_revoked refusal without any user action", async () => { + let clock = 0; + let revoked = true; + const repair = vi.fn(async () => { + revoked = false; + return REPAIR_OK; + }); + const { recovery } = harness({ + health: () => revoked ? REFUSED_HEALTH("machine_revoked") : createSyncAccountDirectoryHealth("published", null), + revoked: () => revoked, + repair, + now: () => clock, + }); + + // First tick only opens the episode; the repair is deliberately delayed. + await recovery.tick(); + expect(repair).not.toHaveBeenCalled(); + expect(recovery.getState().trigger).toBe("refusal"); + + await recovery.tick(); + expect(repair).not.toHaveBeenCalled(); + + clock += PAIRING_AUTO_REPAIR_DELAYS_MS[0]; + await recovery.tick(); + expect(repair).toHaveBeenCalledTimes(1); + expect(recovery.getState().trigger).toBe(null); + }); + + it("acts on pairing_authentication_required too, and backs off between attempts", async () => { + let clock = 0; + const repair = vi.fn(async () => REPAIR_FAILED); + const { recovery } = harness({ + health: () => REFUSED_HEALTH("pairing_authentication_required"), + revoked: () => true, + repair, + now: () => clock, + }); + + await recovery.tick(); + clock += PAIRING_AUTO_REPAIR_DELAYS_MS[0]; + await recovery.tick(); + expect(repair).toHaveBeenCalledTimes(1); + + // Still inside the second delay: nothing more may be sent. + clock += PAIRING_AUTO_REPAIR_DELAYS_MS[1] - 1; + await recovery.tick(); + expect(repair).toHaveBeenCalledTimes(1); + + clock += 1; + await recovery.tick(); + expect(repair).toHaveBeenCalledTimes(2); + }); + + it("never spends more than the persisted budget allows", async () => { + let clock = 0; + const repair = vi.fn(async () => REPAIR_FAILED); + const { recovery, spentRepairs } = harness({ + health: () => REFUSED_HEALTH("machine_revoked"), + revoked: () => true, + repair, + budgetLimit: 3, + now: () => clock, + }); + + await recovery.tick(); + for (let attempt = 0; attempt < 8; attempt += 1) { + clock += 24 * 60 * 60 * 1_000; + await recovery.tick(); + } + + expect(repair).toHaveBeenCalledTimes(3); + expect(spentRepairs()).toBe(3); + expect(recovery.getState().settled).toBe(true); + }); + + it("waits for a signed-in session instead of burning the budget proving there is none", async () => { + let clock = 0; + let signedIn = false; + const repair = vi.fn(async () => REPAIR_OK); + const { recovery, spentRepairs } = harness({ + health: () => REFUSED_HEALTH("machine_revoked"), + revoked: () => true, + repair, + hasAccountSession: () => signedIn, + now: () => clock, + }); + + await recovery.tick(); + clock += PAIRING_AUTO_REPAIR_DELAYS_MS[0]; + await recovery.tick(); + expect(repair).not.toHaveBeenCalled(); + expect(spentRepairs()).toBe(0); + + signedIn = true; + clock += PAIRING_AUTO_REPAIR_DELAYS_MS[0]; + await recovery.tick(); + expect(repair).toHaveBeenCalledTimes(1); + }); + + it("runs exactly one repair cycle for a long snapshot_failed episode", async () => { + let clock = PAIRING_AUTO_REPAIR_SNAPSHOT_GRACE_MS * 10; + const repair = vi.fn(async () => REPAIR_FAILED); + const { recovery } = harness({ + health: () => createSyncAccountDirectoryHealth( + "snapshot_failed", + "The active sync snapshot could not be read.", + { failingSinceMs: 0, lastAttemptAt: 0 }, + ), + revoked: () => false, + repair, + now: () => clock, + }); + + await recovery.tick(); + expect(recovery.getState().trigger).toBe("snapshot_failed"); + clock += PAIRING_AUTO_REPAIR_DELAYS_MS[0]; + await recovery.tick(); + expect(repair).toHaveBeenCalledTimes(1); + + // A publish leg that cannot read a snapshot is not a pairing problem; + // repeating the request would only settle into the same failing banner. + clock += 24 * 60 * 60 * 1_000; + await recovery.tick(); + expect(repair).toHaveBeenCalledTimes(1); + expect(recovery.getState().settled).toBe(true); + }); + + it("ignores a snapshot failure that has not outlasted the grace window", async () => { + let clock = PAIRING_AUTO_REPAIR_SNAPSHOT_GRACE_MS - 1; + const repair = vi.fn(async () => REPAIR_OK); + const { recovery } = harness({ + health: () => createSyncAccountDirectoryHealth("snapshot_failed", null, { + failingSinceMs: 0, + }), + revoked: () => false, + repair, + now: () => clock, + }); + + await recovery.tick(); + expect(recovery.getState().trigger).toBe(null); + clock += 1; + await recovery.tick(); + expect(recovery.getState().trigger).toBe("snapshot_failed"); + }); + + it("leaves a removal the user just performed alone", async () => { + // The user clicked "Remove this computer" seconds ago, while the sign-in + // that authorised it is still fresh enough for the directory to accept a + // re-registration. This is the ONE case where an auto-repair would win the + // argument, and it is the one case where it must not have it. + const revokedAt = "2026-08-18T12:00:00.000Z"; + let clock = Date.parse(revokedAt) + 1_000; + const repair = vi.fn(async () => REPAIR_OK); + const { recovery, spentRepairs } = harness({ + health: () => REFUSED_HEALTH("machine_revoked"), + revoked: () => true, + revokedAt: () => revokedAt, + repair, + now: () => clock, + }); + + // No episode even opens, so nothing is scheduled and nothing is spent. + await recovery.tick(); + expect(recovery.getState().trigger).toBe(null); + clock += PAIRING_AUTO_REPAIR_REVOCATION_QUIET_MS - 2_000; + await recovery.tick(); + await recovery.tick(); + expect(repair).not.toHaveBeenCalled(); + expect(spentRepairs()).toBe(0); + expect(recovery.getState().trigger).toBe(null); + }); + + it("starts an episode once the removal has aged past the quiet window", async () => { + const revokedAt = "2026-08-18T12:00:00.000Z"; + let clock = Date.parse(revokedAt) + PAIRING_AUTO_REPAIR_REVOCATION_QUIET_MS; + const repair = vi.fn(async () => REPAIR_FAILED); + const { recovery } = harness({ + health: () => REFUSED_HEALTH("machine_revoked"), + revoked: () => true, + revokedAt: () => revokedAt, + repair, + now: () => clock, + }); + + await recovery.tick(); + expect(recovery.getState().trigger).toBe("refusal"); + clock += PAIRING_AUTO_REPAIR_DELAYS_MS[0]; + await recovery.tick(); + expect(repair).toHaveBeenCalledTimes(1); + }); + + it("pins the quiet window to the directory's pairing-auth freshness bound", () => { + // `PAIRING_AUTH_FRESHNESS_MS` in apps/account-directory/src/callerToken.ts. + // The worker builds separately, so this value is duplicated rather than + // imported; this assertion is what keeps the duplicate honest. + expect(PAIRING_AUTO_REPAIR_REVOCATION_QUIET_MS).toBe(10 * 60_000); + }); + + it("leaves an unrecognised 403 alone", async () => { + let clock = 0; + const repair = vi.fn(async () => REPAIR_OK); + const { recovery } = harness({ + health: () => createSyncAccountDirectoryHealth("http_error", "Forbidden by a proxy.", { + lastHttpStatus: 403, + lastHttpReason: "Forbidden by a proxy.", + failingSinceMs: 0, + }), + revoked: () => true, + repair, + now: () => clock, + }); + + await recovery.tick(); + clock += 24 * 60 * 60 * 1_000; + await recovery.tick(); + expect(repair).not.toHaveBeenCalled(); + expect(recovery.getState().trigger).toBe(null); + }); +}); diff --git a/apps/ade-cli/src/services/account/machinePairingAutoRecovery.ts b/apps/ade-cli/src/services/account/machinePairingAutoRecovery.ts new file mode 100644 index 000000000..c7e9b7ad5 --- /dev/null +++ b/apps/ade-cli/src/services/account/machinePairingAutoRecovery.ts @@ -0,0 +1,363 @@ +import { readAccountRefusalCode } from "../../../../desktop/src/shared/accountMachineRefusal"; +import type { SyncAccountDirectoryHealth } from "../../../../desktop/src/shared/types"; +import type { AccountMachinePairingRefusalCode } from "./accountMachinePublisherService"; +import type { MachinePairingRepairResult } from "./machinePairingRepair"; + +/** + * Automatic recovery from "this computer is not in your account any more". + * + * Both refusals the directory can answer with — `machine_revoked` and + * `pairing_authentication_required` — are terminal for the heartbeat by design, + * and until now terminal for the machine too: the ONLY way back was a human + * finding the "Reconnect this computer" button. A machine whose session is + * perfectly good and whose owner never removed it (a stale row, a key rotation, + * a directory hiccup) therefore stayed silently disconnected forever. + * + * This runs that same repair on the machine's behalf, on a slow schedule, with + * a budget that survives restarts. It deliberately does NOT widen what a repair + * is allowed to do: it calls the identical brain action the button calls, so a + * genuine removal is refused exactly as it is today and the user-facing state + * is left untouched. + * + * One thing it must never do is argue with a removal the user just performed. + * A genuine removal stands for at least the freshness window; recovery + * afterwards requires the user's next interactive sign-in — because that is the + * only thing that mints the proof the directory will accept. + */ + +/** + * Delays from the start of a refusal episode. Slow on purpose: a repair sends a + * deliberate pairing registration, and a machine the owner really did remove + * must not be able to argue about it more than a handful of times a day. + */ +export const PAIRING_AUTO_REPAIR_DELAYS_MS = [60_000, 5 * 60_000, 60 * 60_000] as const; +/** + * How long the publish leg may report `snapshot_failed` before one repair cycle + * runs. Long enough that ordinary startup races (no sync scope yet, listener + * still binding) settle on their own. + */ +export const PAIRING_AUTO_REPAIR_SNAPSHOT_GRACE_MS = 120_000; +/** + * How long a fresh revocation is left alone before an episode may even start. + * + * Mirrors `PAIRING_AUTH_FRESHNESS_MS` in `apps/account-directory/src/callerToken.ts` + * (10 minutes) — deliberately the same number, and it must stay the same + * number. That is the window in which the directory still accepts the sign-in + * this machine authenticated with, so a repair sent inside it is the one repair + * that WOULD succeed: the user clicks "Remove this computer" seconds after + * signing in, and the machine quietly re-registers itself before the page + * finishes reloading. Waiting the window out means the only repair this loop + * can ever land is one the directory would grant on stale-but-valid grounds — + * a stale row, a key rotation, a directory hiccup — never a deliberate removal. + * + * Duplicated rather than imported because the directory is a Cloudflare Worker + * with its own build; the tie is this comment plus the test that pins the value. + */ +export const PAIRING_AUTO_REPAIR_REVOCATION_QUIET_MS = 10 * 60_000; +const DEFAULT_POLL_MS = 15_000; + +/** What put this machine into a recovery episode. */ +export type MachinePairingAutoRecoveryTrigger = "refusal" | "snapshot_failed"; + +type AutoRecoveryLogger = { + info?: (event: string, meta?: Record) => void; + warn?: (event: string, meta?: Record) => void; +}; + +/** The slice of the account publisher this loop reads. Nothing is mutated here. */ +export type MachinePairingAutoRecoveryPublisher = { + getPublisherHealth(): SyncAccountDirectoryHealth; + getMachineRevocation(): { revoked: boolean; revokedAt: string | null }; +}; + +/** The persisted 6-hour allowance, shared with the identity file. */ +export type MachinePairingAutoRecoveryBudget = { + tryConsumePairingAutoRepair(): { + allowed: boolean; + countInWindow: number; + limit: number; + }; +}; + +export type MachinePairingAutoRecoveryArgs = { + /** Null while no publisher runs (this brain holds no sync host lease). */ + getPublisher: () => MachinePairingAutoRecoveryPublisher | null; + runRepair: () => Promise; + /** + * A repair without a session cannot succeed and would burn budget proving it, + * so the loop simply waits for one. + */ + hasAccountSession: () => boolean; + budget: MachinePairingAutoRecoveryBudget; + logger?: AutoRecoveryLogger; + /** Test seams. */ + pollMs?: number; + delaysMs?: readonly number[]; + snapshotGraceMs?: number; + revocationQuietMs?: number; + now?: () => number; +}; + +export type MachinePairingAutoRecoveryState = { + trigger: MachinePairingAutoRecoveryTrigger | null; + attempts: number; + /** Budget exhausted or the episode ran its single allowed cycle. */ + settled: boolean; + nextAttemptAtMs: number | null; +}; + +export type MachinePairingAutoRecovery = { + start(): void; + stop(): void; + /** Run one evaluation immediately; exported for tests and for start(). */ + tick(): Promise; + getState(): MachinePairingAutoRecoveryState; +}; + +/** + * Which refusal, if any, the last publish attempt ended in. + * + * The 403-vs-code decoding is `readAccountRefusalCode`'s job — one decoder for + * every surface that reads a directory refusal, so the desktop banner and this + * loop can never disagree about what a response meant. Narrowed here because a + * repair only knows how to answer the two named codes: `"other"` (an + * unrecognised 403) and `null` are both "no refusal I can act on", and guessing + * at either is how an auto-repair loop starts arguing with a response nobody + * has taught it to read. + */ +function refusalCodeFrom( + health: SyncAccountDirectoryHealth, +): AccountMachinePairingRefusalCode | null { + const code = readAccountRefusalCode(health); + return code === "machine_revoked" || code === "pairing_authentication_required" + ? code + : null; +} + +export function createMachinePairingAutoRecovery( + args: MachinePairingAutoRecoveryArgs, +): MachinePairingAutoRecovery { + const now = args.now ?? Date.now; + const log = args.logger; + const pollMs = Math.max(1_000, Math.floor(args.pollMs ?? DEFAULT_POLL_MS)); + const delays = args.delaysMs?.length ? args.delaysMs : PAIRING_AUTO_REPAIR_DELAYS_MS; + const snapshotGraceMs = Math.max(0, args.snapshotGraceMs ?? PAIRING_AUTO_REPAIR_SNAPSHOT_GRACE_MS); + const revocationQuietMs = Math.max( + 0, + args.revocationQuietMs ?? PAIRING_AUTO_REPAIR_REVOCATION_QUIET_MS, + ); + + let timer: ReturnType | null = null; + let stopped = true; + let repairInFlight = false; + let episode: { + trigger: MachinePairingAutoRecoveryTrigger; + attempts: number; + nextAttemptAtMs: number; + settled: boolean; + } | null = null; + + const delayFor = (attempt: number): number => + delays[Math.min(attempt, delays.length - 1)] ?? delays[delays.length - 1] ?? DEFAULT_POLL_MS; + + const endEpisode = (reason: string): void => { + if (!episode) return; + const previous = episode; + episode = null; + log?.info?.("account.machine_auto_repair_episode_ended", { + trigger: previous.trigger, + attempts: previous.attempts, + reason, + }); + }; + + const currentTrigger = ( + publisher: MachinePairingAutoRecoveryPublisher, + ): { trigger: MachinePairingAutoRecoveryTrigger; code: string } | null => { + const health = publisher.getPublisherHealth(); + const refusal = refusalCodeFrom(health); + if (refusal && publisher.getMachineRevocation().revoked) { + return { trigger: "refusal", code: refusal }; + } + // The publish leg can also fail short of a refusal. A snapshot that cannot + // be read for minutes on a signed-in machine is the same user-visible + // "this computer isn't published yet" banner, and one repair cycle is + // cheaper than leaving it there indefinitely. + if ( + health.state === "snapshot_failed" + && health.failingSinceMs != null + && now() - health.failingSinceMs >= snapshotGraceMs + ) { + return { trigger: "snapshot_failed", code: "snapshot_failed" }; + } + return null; + }; + + /** + * Is the latched revocation recent enough that repairing it would be undoing + * something the user just did? + * + * Only a revocation the directory actually timestamped can be judged. Both + * refusal shapes carry one — `machine_revoked` and + * `pairing_authentication_required` are two answers about the same revocation + * row and both 403s include its `revokedAt` — so both are held quiet while + * the removal is fresh. A missing timestamp means a directory deployed before + * the field existed; that reads as NOT fresh and the loop behaves exactly as + * it did before this gate, because blocking on a missing timestamp would + * disable recovery for precisely the stale-row case this loop was built for. + */ + const revocationIsFresh = (publisher: MachinePairingAutoRecoveryPublisher): boolean => { + const { revoked, revokedAt } = publisher.getMachineRevocation(); + if (!revoked || !revokedAt) return false; + const revokedAtMs = Date.parse(revokedAt); + if (!Number.isFinite(revokedAtMs)) return false; + return now() - revokedAtMs < revocationQuietMs; + }; + + const runOneRepair = async ( + trigger: MachinePairingAutoRecoveryTrigger, + code: string, + ): Promise => { + const spend = args.budget.tryConsumePairingAutoRepair(); + if (!spend.allowed) { + if (episode && !episode.settled) { + episode.settled = true; + // Deliberately quiet from here on. The user-facing state is whatever + // the publisher already reports; the loop simply stops arguing. + log?.warn?.("account.machine_auto_repair_budget_exhausted", { + trigger, + code, + limit: spend.limit, + }); + } + return; + } + repairInFlight = true; + let result: MachinePairingRepairResult | null = null; + try { + log?.info?.("account.machine_auto_repair_started", { + trigger, + code, + attempt: (episode?.attempts ?? 0) + 1, + repairsInWindow: spend.countInWindow, + }); + result = await args.runRepair(); + } catch (error) { + log?.warn?.("account.machine_auto_repair_error", { + trigger, + code, + error: error instanceof Error ? error.message : String(error), + }); + } finally { + repairInFlight = false; + } + if (!episode) return; + episode.attempts += 1; + if (result?.repaired) { + log?.info?.("account.machine_auto_repaired", { + trigger, + code, + attempts: episode.attempts, + pushRestored: result.pushRestored, + }); + endEpisode("repaired"); + return; + } + log?.warn?.("account.machine_auto_repair_failed", { + trigger, + code, + attempts: episode.attempts, + state: result?.state ?? "error", + reasonCode: result?.reasonCode ?? null, + }); + if (trigger === "snapshot_failed") { + // One cycle only: a publish leg that cannot read a snapshot is not a + // pairing problem, and repeating the request will not make it one. + episode.settled = true; + return; + } + episode.nextAttemptAtMs = now() + delayFor(episode.attempts); + }; + + const tick = async (): Promise => { + // Deliberately not gated on `stopped`: the scheduler owns the lifecycle, and + // an explicit tick (tests, a future "check now" action) must still evaluate. + if (repairInFlight) return; + const publisher = args.getPublisher(); + if (!publisher) { + endEpisode("publisher_unavailable"); + return; + } + const observed = currentTrigger(publisher); + if (!observed) { + endEpisode("recovered"); + return; + } + if (observed.trigger === "refusal" && revocationIsFresh(publisher)) { + // The user removed this computer moments ago. Repairing now would undo a + // deliberate action while their sign-in is still fresh enough for the + // directory to accept it — the one case where this loop could actually + // win an argument it has no business having. Stay idle; the poll keeps + // running, and once the removal has aged past the quiet window the + // episode may start on a later tick. + return; + } + if (!episode || episode.trigger !== observed.trigger) { + episode = { + trigger: observed.trigger, + attempts: 0, + nextAttemptAtMs: now() + delayFor(0), + settled: false, + }; + log?.info?.("account.machine_auto_repair_episode_started", { + trigger: observed.trigger, + code: observed.code, + firstAttemptInMs: delayFor(0), + }); + return; + } + if (episode.settled || now() < episode.nextAttemptAtMs) return; + if (!args.hasAccountSession()) { + // Not a failed attempt — nothing was sent — so the budget is untouched + // and the schedule simply slips until a session exists. + episode.nextAttemptAtMs = now() + delayFor(episode.attempts); + return; + } + await runOneRepair(observed.trigger, observed.code); + }; + + const schedule = (): void => { + if (stopped || timer) return; + timer = setTimeout(() => { + timer = null; + if (stopped) return; + void tick().catch(() => { + // A recovery loop must never take the brain down with it. + }).finally(schedule); + }, pollMs); + timer.unref?.(); + }; + + return { + start(): void { + if (!stopped) return; + stopped = false; + schedule(); + }, + stop(): void { + stopped = true; + if (timer) clearTimeout(timer); + timer = null; + episode = null; + }, + tick, + getState(): MachinePairingAutoRecoveryState { + return { + trigger: episode?.trigger ?? null, + attempts: episode?.attempts ?? 0, + settled: episode?.settled ?? false, + nextAttemptAtMs: episode?.nextAttemptAtMs ?? null, + }; + }, + }; +} diff --git a/apps/ade-cli/src/services/credentials/credentialStore.test.ts b/apps/ade-cli/src/services/credentials/credentialStore.test.ts index 10f932b92..ecb1c4dc3 100644 --- a/apps/ade-cli/src/services/credentials/credentialStore.test.ts +++ b/apps/ade-cli/src/services/credentials/credentialStore.test.ts @@ -979,9 +979,12 @@ describe("ElectronSafeStorageCredentialStore", () => { // This process cannot obtain the OS material the ciphertext was sealed with // (locked/denied keychain), so it falls back to the bare machine key, which // does not decrypt either — the exact shape that reads as an empty store. + // `peerMayHoldMaterial` is declared rather than inherited from the test + // host: no peer can open this one either, which is what makes it a plain + // decrypt failure below on macOS and Linux alike. const undecryptableLegacyStore = new EncryptedFileCredentialStore({ secretsDir: tempDir, - keyMaterial: { read: () => null }, + keyMaterial: { read: () => null, peerMayHoldMaterial: false }, }); expect(undecryptableLegacyStore.readAllForMigration()).toEqual({}); expect(undecryptableLegacyStore.getLastReadState()).toBe("unreadable"); @@ -992,6 +995,10 @@ describe("ElectronSafeStorageCredentialStore", () => { }); expect(store.getSync("linear.token.v1")).toBeNull(); + // …and it says so. A `null` here is indistinguishable from "never stored", + // which is how a corrupted store reached the UI as a fresh install. + expect(store.getLastReadState()).toBe("unreadable"); + expect(store.getLastReadFailureReason()).toBe("decrypt_failure"); // Nothing written, nothing deleted: the credentials stay recoverable. expect(fs.existsSync(safePath)).toBe(false); @@ -1004,6 +1011,31 @@ describe("ElectronSafeStorageCredentialStore", () => { expect(recovered.getSync("linear.token.v1")).toBe("lin_secret"); }); + it("keeps the legacy store's own reason for an unreadable migration source", () => { + // The launchd-brain shape: the legacy file is sealed with OS material this + // process cannot obtain, but a PEER process can. That is + // `no_os_key_material`, which is recoverable — reporting it as + // `decrypt_failure` puts the wrong repair in front of a user whose + // credentials are all still there. + sealLegacyOsBoundStore(tempDir, { "linear.token.v1": "lin_secret" }, Buffer.from("os-material-A")); + const legacyStore = new EncryptedFileCredentialStore({ + secretsDir: tempDir, + keyMaterial: { read: () => null, peerMayHoldMaterial: true }, + }); + expect(legacyStore.readAllForMigration()).toEqual({}); + expect(legacyStore.getLastReadFailureReason()).toBe("no_os_key_material"); + + const store = new ElectronSafeStorageCredentialStore({ + secretsDir: tempDir, + safeStorage, + legacyStore, + }); + + expect(store.getSync("linear.token.v1")).toBeNull(); + expect(store.getLastReadState()).toBe("unreadable"); + expect(store.getLastReadFailureReason()).toBe("no_os_key_material"); + }); + it("still moves and removes a legacy file that is already safeStorage-encrypted", () => { // Nothing in an Electron-only file is brain-readable, so retaining it would // only leave an undecryptable file behind for the brain to trip over. diff --git a/apps/ade-cli/src/services/credentials/credentialStore.ts b/apps/ade-cli/src/services/credentials/credentialStore.ts index edde3f538..86d4e50f2 100644 --- a/apps/ade-cli/src/services/credentials/credentialStore.ts +++ b/apps/ade-cli/src/services/credentials/credentialStore.ts @@ -168,6 +168,14 @@ type CredentialStoreMigrationSource = { * unreadable store deletes it. */ getLastReadState(): CredentialStoreReadState; + /** + * Why that read failed, when it did. The legacy store is the only thing that + * knows: a store nothing on this machine can open is `no_os_key_material` + * (a PEER process can still open it, so the credentials are not lost) and a + * broken file is `store_format`. Reporting either as `decrypt_failure` sends + * the user at the wrong repair. + */ + getLastReadFailureReason(): CredentialStoreReadFailureReason | null; /** * Rewrites the legacy file to exactly `values` WITHOUT acquiring the store's * lock: the migration already holds that same lock file, and the file lock is @@ -749,8 +757,20 @@ export class EncryptedFileCredentialStore implements SyncCredentialStore { } async get(key: string): Promise { + return (await this.getWithReadState(key)).value; + } + + /** + * A read paired with the state that read produced. Prefer this over `get()` + * followed by `getLastReadState()`: the latter answers about the store's most + * recent read, which after an await is not necessarily this one. + */ + async getWithReadState( + key: string, + ): Promise<{ value: string | null; state: CredentialStoreReadState }> { const normalized = normalizeKey(key); - return (await this.readAllAsync())[normalized] ?? null; + const { values, state } = await this.readAllAsync(); + return { value: values[normalized] ?? null, state }; } async set(key: string, value: string): Promise { @@ -1124,17 +1144,28 @@ export class EncryptedFileCredentialStore implements SyncCredentialStore { return true; } - private async readAllAsync(): Promise> { + /** + * Returns the decoded values AND the state this read produced, because + * `lastReadState` is a single field every reader overwrites. An async caller + * can only consult it once its own read has resolved, by which point another + * reader — App user authentication shares this store — may have moved it. + * Capturing the verdict here, in the same step that records it, is what keeps + * "no credential" and "a credential ADE cannot read" tellable apart. + */ + private async readAllAsync(): Promise<{ + values: Record; + state: CredentialStoreReadState; + }> { const { value: raw, exists: credentialsExist } = await readJsonObjectAsync(this.credentialsPath); if (!credentialsExist) { this.lastReadState = "missing"; this.lastReadFailureReason = null; - return {}; + return { values: {}, state: "missing" }; } if (!raw || Object.keys(raw).length === 0) { this.lastReadState = "unreadable"; this.lastReadFailureReason = "store_format"; - return {}; + return { values: {}, state: "unreadable" }; } const machineKey = await readOrCreateMachineKeyAsync(this.machineKeyPath); const material = await this.readKeyMaterialAsync(); @@ -1159,12 +1190,12 @@ export class EncryptedFileCredentialStore implements SyncCredentialStore { // The reason is what the caller needs, and it gets it. this.lastReadState = "unreadable"; this.lastReadFailureReason = attempt.reason; - return {}; + return { values: {}, state: "unreadable" }; } this.lastReadState = "available"; this.lastReadFailureReason = null; if (attempt.sealedBinding !== "machine") this.scheduleRebindToMachineKey(material); - return attempt.values; + return { values: attempt.values, state: "available" }; } /** @@ -1259,6 +1290,17 @@ export class ElectronSafeStorageCredentialStore implements SyncCredentialStore { private readonly lockPath: string; private readonly legacyLockPath: string; private readonly legacyStore: CredentialStoreMigrationSource | null; + /** + * Result of the most recent read, for the same reason the file store records + * one: a read that cannot open the ciphertext still has to return SOMETHING, + * and the only branch here that returns `{}` rather than throwing is the + * aborted legacy migration below. Without this, a caller cannot tell that + * empty view from a machine that was never signed in — and telling a user + * "not connected" when the truth is "not readable" invites them to reconnect + * over credentials that are still on disk. + */ + private lastReadState: CredentialStoreReadState = "missing"; + private lastReadFailureReason: CredentialStoreReadFailureReason | null = null; constructor(args: { safeStorage: SafeStorageLike; @@ -1305,6 +1347,14 @@ export class ElectronSafeStorageCredentialStore implements SyncCredentialStore { return this.readAll()[normalized] ?? null; } + getLastReadState(): CredentialStoreReadState { + return this.lastReadState; + } + + getLastReadFailureReason(): CredentialStoreReadFailureReason | null { + return this.lastReadFailureReason; + } + setSync(key: string, value: string): void { const normalized = normalizeKey(key); const nextValue = value.trim(); @@ -1361,22 +1411,47 @@ export class ElectronSafeStorageCredentialStore implements SyncCredentialStore { } catch (error: unknown) { if (isEnoent(error)) { const legacyValues = this.migrateLegacyStore(args.safeLockHeld === true); - return legacyValues ?? {}; + if (legacyValues) return this.recordRead(legacyValues, "available"); + // No safeStorage file AND no migratable legacy values. That is "missing" + // UNLESS the legacy store told us it could not decrypt what is there — + // the migration aborts on exactly that case (readLegacyEncryptedFileStore) + // and returning `{}` without saying so is the masking bug. + // + // The legacy store's OWN reason is carried across rather than assumed: + // an `os`-sealed store this process cannot open is `no_os_key_material`, + // which a peer process can still recover from, and calling that a + // decrypt failure offers a repair that throws the session away. + const legacy = this.legacyStore; + return legacy?.getLastReadState() === "unreadable" + ? this.recordRead({}, "unreadable", legacy.getLastReadFailureReason() ?? "decrypt_failure") + : this.recordRead({}, "missing"); } + this.recordRead({}, "unreadable", "store_format"); throw error; } try { const decrypted = this.safeStorage.decryptString(payload.encrypted); - return normalizeStoredCredentialValues(JSON.parse(decrypted)); + return this.recordRead(normalizeStoredCredentialValues(JSON.parse(decrypted)), "available"); } catch (error: unknown) { if (!payload.hasMagic && isStoredCredentialEnvelopeBuffer(payload.encrypted)) { const legacyValues = this.migrateLegacyStore(args.safeLockHeld === true); - if (legacyValues) return legacyValues; + if (legacyValues) return this.recordRead(legacyValues, "available"); } + this.recordRead({}, "unreadable", "decrypt_failure"); throw error; } } + private recordRead( + values: Record, + state: CredentialStoreReadState, + reason: CredentialStoreReadFailureReason | null = null, + ): Record { + this.lastReadState = state; + this.lastReadFailureReason = state === "unreadable" ? reason : null; + return values; + } + private readLegacySafeStorageFile(): Record | null { if (!fs.existsSync(this.legacyCredentialsPath)) return null; let payload: { encrypted: Buffer; hasMagic: boolean }; diff --git a/apps/ade-cli/src/services/projects/machineLayout.ts b/apps/ade-cli/src/services/projects/machineLayout.ts index f10c92a69..04e9053a8 100644 --- a/apps/ade-cli/src/services/projects/machineLayout.ts +++ b/apps/ade-cli/src/services/projects/machineLayout.ts @@ -80,8 +80,13 @@ function windowsChannelIdentity(adeDir: string, env: NodeJS.ProcessEnv): { /** * The on-disk casing of `value`, for the components that exist. Components that * do not exist yet can only be re-joined as they were given. + * + * Exported because the hardware anchor folds the same ADE home path into its + * hash and must agree with the pipe identity on what one path IS: `realpath` + * also expands 8.3 short names (`C:\Users\ADAOBI~1\.ade`) and resolves junction + * casing, so two spellings of one directory cannot become two machines. */ -function canonicalWindowsPath(value: string): string { +export function canonicalWindowsPath(value: string): string { const original = path.win32.resolve(value).replace(/\//g, "\\"); const missingParts: string[] = []; let cursor = original; diff --git a/apps/ade-cli/src/services/sync/brainMachineSyncStores.ts b/apps/ade-cli/src/services/sync/brainMachineSyncStores.ts index fe38c49c3..ca21be123 100644 --- a/apps/ade-cli/src/services/sync/brainMachineSyncStores.ts +++ b/apps/ade-cli/src/services/sync/brainMachineSyncStores.ts @@ -2,7 +2,8 @@ import path from "node:path"; import { randomInt } from "node:crypto"; import { pathKey } from "../../../../desktop/src/main/services/shared/pathCompare"; import type { ProjectlessSyncControls } from "../../multiProjectRpcServer"; -import { buildSyncCloudRelayStatus, type SyncCloudRelayStore } from "./syncCloudRelayStore"; +import { buildSyncCloudRelayStatus } from "./syncCloudRelayStatus"; +import type { SyncCloudRelayStore } from "./syncCloudRelayStore"; import { createSyncPairingStore } from "./syncPairingStore"; import { createSyncPinStore } from "./syncPinStore"; import { createSyncSecurityStore } from "./syncSecurityStore"; diff --git a/apps/ade-cli/src/services/sync/syncCloudRelayStatus.ts b/apps/ade-cli/src/services/sync/syncCloudRelayStatus.ts new file mode 100644 index 000000000..759544d1a --- /dev/null +++ b/apps/ade-cli/src/services/sync/syncCloudRelayStatus.ts @@ -0,0 +1,37 @@ +import type { SyncCloudRelayStatus } from "../../../../desktop/src/shared/types"; +import type { SyncCloudRelayStore } from "./syncCloudRelayStore"; +import { + RELAY_SIGN_IN_REQUIRED_MESSAGE, + type SyncTunnelClientStatus, +} from "./syncTunnelClientService"; + +/** + * `accountSignedIn` is the gate: without it the live fields collapse to their + * off values and the error becomes the sign-in prompt, so a signed-out machine + * never reports a connection it cannot have. + */ +export function buildSyncCloudRelayStatus(args: { + cloudRelayStore: Pick; + tunnelStatus: SyncTunnelClientStatus | null; + accountSignedIn: boolean; +}): SyncCloudRelayStatus { + const { cloudRelayStore, tunnelStatus, accountSignedIn } = args; + return { + relayWssUrl: cloudRelayStore.getRelayWssUrl(), + machineKey: cloudRelayStore.getConfig().machineKey, + relayUrl: cloudRelayStore.getRelayUrl(), + connected: accountSignedIn && (tunnelStatus?.connected ?? false), + activeTunnels: accountSignedIn ? tunnelStatus?.activeTunnels ?? 0 : 0, + relayBridgeValidated: accountSignedIn && (tunnelStatus?.relayBridgeValidated ?? false), + lastFailureAt: tunnelStatus?.lastFailureAt ?? null, + lastControlOpenAt: tunnelStatus?.lastControlOpenAt ?? null, + lastBridgeValidationAt: tunnelStatus?.lastBridgeValidationAt ?? null, + relayEndToEndVerifiedAt: tunnelStatus?.relayEndToEndVerifiedAt ?? null, + relayEndToEndFailure: tunnelStatus?.relayEndToEndFailure ?? null, + relayEndToEndRoundTripMs: tunnelStatus?.relayEndToEndRoundTripMs ?? null, + lastControlError: tunnelStatus?.lastControlError ?? null, + lastError: accountSignedIn + ? tunnelStatus?.lastError ?? null + : RELAY_SIGN_IN_REQUIRED_MESSAGE, + }; +} diff --git a/apps/ade-cli/src/services/sync/syncCloudRelayStore.test.ts b/apps/ade-cli/src/services/sync/syncCloudRelayStore.test.ts index c60e4de73..2cdf34b99 100644 --- a/apps/ade-cli/src/services/sync/syncCloudRelayStore.test.ts +++ b/apps/ade-cli/src/services/sync/syncCloudRelayStore.test.ts @@ -61,20 +61,92 @@ describe("syncCloudRelayStore", () => { expect(JSON.parse(fs.readFileSync(filePath, "utf8"))).toEqual(config); }); - it("regenerates both halves when either persisted credential is invalid", () => { - const oldMachineKey = "a".repeat(32); + it("keeps the machine key and re-mints only the broken secret", () => { + const existingMachineKey = "a".repeat(32); fs.writeFileSync(filePath, `${JSON.stringify({ - machineKey: oldMachineKey, + machineKey: existingMachineKey, secret: "too-short", relayUrl: "https://relay.example.com", })}\n`); + const events: Array<[string, Record | undefined]> = []; - const repaired = createSyncCloudRelayStore({ filePath }).getConfig(); + const repaired = createSyncCloudRelayStore({ + filePath, + logger: { info: (event, data) => events.push([event, data]) }, + }).getConfig(); - expect(repaired.machineKey).toMatch(/^[a-f0-9]{32}$/); - expect(repaired.machineKey).not.toBe(oldMachineKey); + // The machine key IS the account's primary key for this computer; losing it + // turns a live machine into a phantom row the owner is invited to delete. + expect(repaired.machineKey).toBe(existingMachineKey); expect(repaired.secret).toMatch(/^[a-f0-9]{48}$/); expect(JSON.parse(fs.readFileSync(filePath, "utf8"))).toEqual(repaired); + expect(events).toContainEqual([ + "sync_cloud_relay.identity_rotated", + { + previousMachineKey: existingMachineKey, + machineKey: existingMachineKey, + reason: "secret_remint", + }, + ]); + }); + + it("never mints a new identity out of an unparsable file", () => { + const seeded = createSyncCloudRelayStore({ filePath }); + const first = seeded.getMachineIdentity(); + // Exactly what a truncated write leaves behind — and what used to be read + // as "this machine has no identity", minting a brand new one. + fs.writeFileSync(filePath, '{"machineKey": "'); + + const recovered = createSyncCloudRelayStore({ filePath }).getMachineIdentity(); + + expect(recovered).toEqual(first); + expect(JSON.parse(fs.readFileSync(filePath, "utf8"))).toMatchObject(first); + }); + + it("falls back to the .bak sibling when the primary file is destroyed", () => { + const seeded = createSyncCloudRelayStore({ filePath }); + const first = seeded.getMachineIdentity(); + expect(fs.existsSync(`${filePath}.bak`)).toBe(true); + fs.writeFileSync(filePath, "not json at all"); + const events: Array<[string, Record | undefined]> = []; + + const recovered = createSyncCloudRelayStore({ + filePath, + logger: { info: (event, data) => events.push([event, data]) }, + }).getMachineIdentity(); + + expect(recovered).toEqual(first); + expect(events).toContainEqual([ + "sync_cloud_relay.identity_rotated", + { + previousMachineKey: null, + machineKey: first.machineKey, + reason: "recovered_from_backup", + }, + ]); + }); + + it("mints a fresh identity only when neither file yields one", () => { + const seeded = createSyncCloudRelayStore({ filePath }); + const first = seeded.getMachineIdentity(); + fs.writeFileSync(filePath, "{}"); + fs.writeFileSync(`${filePath}.bak`, "{}"); + const events: Array<[string, Record | undefined]> = []; + + const minted = createSyncCloudRelayStore({ + filePath, + logger: { info: (event, data) => events.push([event, data]) }, + }).getMachineIdentity(); + + expect(minted.machineKey).not.toBe(first.machineKey); + expect(events).toContainEqual([ + "sync_cloud_relay.identity_rotated", + { + previousMachineKey: null, + machineKey: minted.machineKey, + reason: "corrupt_file_remint", + }, + ]); }); it("mints a stable identity and persists the file chmod 600", () => { @@ -108,16 +180,144 @@ describe("syncCloudRelayStore", () => { store.setRelayUrl("https://relay.example.com"); const first = store.getMachineIdentity(); - const rotated = store.rotateMachineIdentity(first.machineKey); - expect(rotated).toMatchObject({ + const rotation = store.rotateMachineIdentity(first.machineKey); + expect(rotation.rotated).toBe(true); + expect(rotation.config).toMatchObject({ machineKey: expect.stringMatching(/^[a-f0-9]{32}$/), secret: expect.stringMatching(/^[a-f0-9]{48}$/), relayUrl: "https://relay.example.com", }); - expect(rotated.machineKey).not.toBe(first.machineKey); - expect(rotated.secret).not.toBe(first.secret); - expect(store.rotateMachineIdentity(first.machineKey)).toEqual(rotated); - expect(createSyncCloudRelayStore({ filePath }).getConfig()).toEqual(rotated); + expect(rotation.config.machineKey).not.toBe(first.machineKey); + expect(rotation.config.secret).not.toBe(first.secret); + const stale = store.rotateMachineIdentity(first.machineKey); + expect(stale.rotated).toBe(false); + expect(stale.config).toEqual(rotation.config); + expect(createSyncCloudRelayStore({ filePath }).getConfig()).toEqual(rotation.config); + }); + + it("keeps the 409 rotation budget across a simulated brain restart", () => { + // A brand-new store instance over the same directory IS a brain restart as + // far as this budget is concerned — the closure counter it replaced reset + // here, so a crash loop could mint one phantom machine per boot. + const rotateOnce = (): ReturnType< + ReturnType["rotateMachineIdentity"] + > => { + const store = createSyncCloudRelayStore({ filePath }); + return store.rotateMachineIdentity(store.getMachineIdentity().machineKey); + }; + + expect(rotateOnce().rotated).toBe(true); + expect(rotateOnce().rotated).toBe(true); + + const third = rotateOnce(); + expect(third.rotated).toBe(false); + expect(third.budgetExhausted).toBe(true); + expect(third.rotationsInWindow).toBe(2); + // And a fourth restart still reads the spent window off disk rather than + // handing the fresh process a fresh allowance. + const fourth = rotateOnce(); + expect(fourth.rotated).toBe(false); + expect(fourth.budgetExhausted).toBe(true); + expect(fourth.rotationsInWindow).toBe(2); + }); + + it("reads an ABSENT rotation budget as a fresh allowance", () => { + // A machine that has never rotated writes no `identityRotations` member at + // all. That must stay the cheap, ordinary case — the fail-closed read below + // is for garbled counters, not for missing ones. + const seeded = createSyncCloudRelayStore({ filePath }); + const { machineKey, secret } = seeded.getMachineIdentity(); + fs.writeFileSync(filePath, `${JSON.stringify({ machineKey, secret })}\n`); + + const store = createSyncCloudRelayStore({ filePath }); + const rotation = store.rotateMachineIdentity(machineKey); + expect(rotation.rotated).toBe(true); + expect(rotation.budgetExhausted).toBe(false); + expect(rotation.rotationsInWindow).toBe(1); + }); + + it.each([ + ["a garbled non-object budget", "truncated"], + ["a budget with no count at all", { windowStartedAt: "2026-08-18T00:00:00.000Z" }], + ["a budget whose count is not a number", { count: "two" }], + ["a budget whose count serialized as null", { count: null }], + ["a budget that garbled into an array", [2]], + ])("reads %s as SPENT rather than as a fresh allowance", (_label, identityRotations) => { + // The failure this closes: a crash loop that truncates the identity file + // used to hand every boot a full rotation allowance, minting one phantom + // machine row per restart — exactly what persisting the budget prevents. + const seeded = createSyncCloudRelayStore({ filePath }); + const { machineKey, secret } = seeded.getMachineIdentity(); + fs.writeFileSync(filePath, `${JSON.stringify({ + machineKey, + secret, + identityRotations, + })}\n`); + + const store = createSyncCloudRelayStore({ filePath }); + const rotation = store.rotateMachineIdentity(machineKey); + expect(rotation.rotated).toBe(false); + expect(rotation.budgetExhausted).toBe(true); + expect(store.getMachineIdentity().machineKey).toBe(machineKey); + // And the same file read on the next boot is still spent: forgiving it once + // per restart is the whole failure mode. + expect(createSyncCloudRelayStore({ filePath }).rotateMachineIdentity(machineKey).rotated) + .toBe(false); + }); + + it("reads an unreadable pairing-repair budget as SPENT", () => { + const seeded = createSyncCloudRelayStore({ filePath }); + const { machineKey, secret } = seeded.getMachineIdentity(); + fs.writeFileSync(filePath, `${JSON.stringify({ + machineKey, + secret, + pairingAutoRepairs: { count: "many" }, + })}\n`); + + expect(createSyncCloudRelayStore({ filePath }).tryConsumePairingAutoRepair()) + .toMatchObject({ allowed: false, limit: 3 }); + }); + + it("reopens the rotation budget once the 24-hour window has passed", () => { + let clock = Date.parse("2026-08-18T00:00:00.000Z"); + const build = () => createSyncCloudRelayStore({ filePath, now: () => clock }); + for (let i = 0; i < 2; i += 1) { + const store = build(); + expect(store.rotateMachineIdentity(store.getMachineIdentity().machineKey).rotated).toBe(true); + } + expect(build().rotateMachineIdentity(build().getMachineIdentity().machineKey).rotated).toBe(false); + + clock += 24 * 60 * 60 * 1_000 + 1; + const store = build(); + expect(store.rotateMachineIdentity(store.getMachineIdentity().machineKey).rotated).toBe(true); + }); + + it("bounds automatic pairing repairs to three per persisted six-hour window", () => { + createSyncCloudRelayStore({ filePath }).getConfig(); + const spend = () => createSyncCloudRelayStore({ filePath }).tryConsumePairingAutoRepair(); + + expect(spend()).toMatchObject({ allowed: true, countInWindow: 1 }); + expect(spend()).toMatchObject({ allowed: true, countInWindow: 2 }); + expect(spend()).toMatchObject({ allowed: true, countInWindow: 3 }); + // Still refused after another simulated restart: the window lives in the + // file, so a brain that reboots every minute cannot buy more repairs. + expect(spend()).toMatchObject({ allowed: false, countInWindow: 3, limit: 3 }); + }); + + it("confirms only the superseded machine keys this machine actually retired", () => { + const store = createSyncCloudRelayStore({ filePath }); + const first = store.getMachineIdentity(); + const rotated = store.rotateMachineIdentity(first.machineKey); + expect(rotated.rotated).toBe(true); + + // A key this machine never held describes somebody else's device. + expect(store.confirmSupersededMachineKeys(["f".repeat(32)])).toEqual([]); + expect(store.confirmSupersededMachineKeys([first.machineKey, "f".repeat(32)])) + .toEqual([first.machineKey]); + // Confirmed once, then forgotten — and an old directory that sends nothing + // is simply an empty answer. + expect(store.confirmSupersededMachineKeys([first.machineKey])).toEqual([]); + expect(store.confirmSupersededMachineKeys([])).toEqual([]); }); it("does not race an identity rotation while another process owns the lock", () => { @@ -126,9 +326,10 @@ describe("syncCloudRelayStore", () => { const lockPath = `${filePath}.rotate.lock`; fs.writeFileSync(lockPath, "", { flag: "wx", mode: 0o600 }); - expect(store.rotateMachineIdentity(first.machineKey)).toMatchObject(first); + expect(store.rotateMachineIdentity(first.machineKey).config).toMatchObject(first); fs.unlinkSync(lockPath); - expect(store.rotateMachineIdentity(first.machineKey).machineKey).not.toBe(first.machineKey); + expect(store.rotateMachineIdentity(first.machineKey).config.machineKey) + .not.toBe(first.machineKey); }); it("does not steal an old lock from a live owner", () => { @@ -145,11 +346,54 @@ describe("syncCloudRelayStore", () => { const old = new Date(Date.now() - 60_000); fs.utimesSync(lockPath, old, old); - expect(store.rotateMachineIdentity(first.machineKey)).toEqual(first); + expect(store.rotateMachineIdentity(first.machineKey).config).toEqual(first); expect(JSON.parse(fs.readFileSync(lockPath, "utf8"))).toEqual(liveOwner); expect(createSyncCloudRelayStore({ filePath }).getConfig()).toEqual(first); }); + it("serves a .bak-recovered identity while another process owns the lock", () => { + const seeded = createSyncCloudRelayStore({ filePath }); + const first = seeded.getConfig(); + fs.writeFileSync(filePath, "not json at all"); + const lockPath = `${filePath}.rotate.lock`; + fs.writeFileSync(lockPath, `${JSON.stringify({ + version: 1, + pid: process.pid, + token: "live-recovery-owner".padEnd(32, "0"), + createdAt: new Date().toISOString(), + })}\n`, { flag: "wx", mode: 0o600 }); + + // Every field of this identity is already on disk in the sibling, so a + // reader that cannot take the lock has nothing to be unsure about — the + // rewrite of the primary simply waits for an uncontended read. + const store = createSyncCloudRelayStore({ filePath, lockWaitMs: 0 }); + expect(store.getConfig()).toEqual(first); + expect(store.getRelayWssUrl()).toContain(first.machineKey); + + fs.unlinkSync(lockPath); + expect(store.getConfig()).toEqual(first); + expect(JSON.parse(fs.readFileSync(filePath, "utf8"))).toEqual(first); + }); + + it("still refuses to hand out an identity that exists on no file", () => { + createSyncCloudRelayStore({ filePath }).getConfig(); + // Neither copy yields a machine key, so the only answer this process has is + // one it minted in memory — and the lock holder may be minting a different + // one right now. + fs.writeFileSync(filePath, "{}"); + fs.writeFileSync(`${filePath}.bak`, "{}"); + const lockPath = `${filePath}.rotate.lock`; + fs.writeFileSync(lockPath, `${JSON.stringify({ + version: 1, + pid: process.pid, + token: "live-mint-owner".padEnd(32, "0"), + createdAt: new Date().toISOString(), + })}\n`, { flag: "wx", mode: 0o600 }); + + expect(() => createSyncCloudRelayStore({ filePath, lockWaitMs: 0 }).getConfig()) + .toThrow("configuration is being updated by another live ADE process"); + }); + it("serializes relay URL writes behind the identity rotation lock", () => { const store = createSyncCloudRelayStore({ filePath, lockWaitMs: 0 }); const first = store.getConfig(); diff --git a/apps/ade-cli/src/services/sync/syncCloudRelayStore.ts b/apps/ade-cli/src/services/sync/syncCloudRelayStore.ts index 083d3cd91..c7555f38c 100644 --- a/apps/ade-cli/src/services/sync/syncCloudRelayStore.ts +++ b/apps/ade-cli/src/services/sync/syncCloudRelayStore.ts @@ -1,20 +1,53 @@ import fs from "node:fs"; import path from "node:path"; import { createHmac, randomBytes } from "node:crypto"; -import { safeJsonParse, writeTextAtomic } from "../../../../desktop/src/main/services/shared/utils"; +import { safeJsonParse } from "../../../../desktop/src/main/services/shared/utils"; +import { writeFileAtomic } from "../../../../desktop/src/main/services/state/durableFile"; import { DEFAULT_ADE_TUNNEL_RELAY_URL } from "../../../../desktop/src/shared/accountDirectory"; -import type { SyncCloudRelayStatus } from "../../../../desktop/src/shared/types"; -import { - RELAY_SIGN_IN_REQUIRED_MESSAGE, - type SyncTunnelClientStatus, -} from "./syncTunnelClientService"; const DEFAULT_RELAY_URL = DEFAULT_ADE_TUNNEL_RELAY_URL; + +/** The identity file's name inside the machine secrets directory. */ +export const SYNC_CLOUD_RELAY_FILE_NAME = "sync-cloud-relay.json"; + const IDENTITY_ROTATION_LOCK_STALE_MS = 30_000; const IDENTITY_ROTATION_LOCK_VERSION = 1; const IDENTITY_CONFIG_LOCK_WAIT_MS = 2_000; const IDENTITY_CONFIG_LOCK_RETRY_MS = 10; +/** + * Sibling copy of the identity file, rewritten after every successful write. + * + * A machine key is not a cache: losing it turns a known computer into a + * stranger the account has never seen, and the row it used to own becomes a + * phantom the owner is invited to delete — which is precisely how a live + * MacBook got locked out of its own account. One truncated write is enough to + * cause that, so the identity always exists in two places. + */ +const IDENTITY_BACKUP_SUFFIX = ".bak"; + +/** + * Relay-claim conflicts may rotate this machine's identity at most twice a day. + * + * The budget used to be a closure variable, so every brain restart handed the + * client a fresh allowance — a crash loop could mint an unbounded number of + * machine rows, each one a phantom on the owner's roster. It is persisted for + * exactly that reason: restarts must not forgive rotations. + */ +export const IDENTITY_ROTATION_WINDOW_MS = 24 * 60 * 60 * 1_000; +export const MAX_IDENTITY_ROTATIONS_PER_WINDOW = 2; + +/** Automatic pairing repairs are bounded the same way, on a 6-hour window. */ +export const PAIRING_AUTO_REPAIR_WINDOW_MS = 6 * 60 * 60 * 1_000; +export const MAX_PAIRING_AUTO_REPAIRS_PER_WINDOW = 3; + +/** + * How many retired machine keys are remembered so a server that reports + * `supersededMachineKeys` can be recognised as confirming OUR rotation rather + * than describing somebody else's device. + */ +const MAX_RETAINED_PREVIOUS_MACHINE_KEYS = 5; + export type SyncCloudRelayConfig = { /** Per-machine identifier phones dial through the relay (32 hex chars). */ machineKey: string; @@ -24,12 +57,90 @@ export type SyncCloudRelayConfig = { relayUrl?: string; }; +/** A count of consumptions inside a rolling window, persisted with the identity. */ +export type SyncCloudRelayBudget = { + count: number; + /** ISO start of the current window; null while the budget is untouched. */ + windowStartedAt: string | null; + /** ISO timestamp of the most recent consumption. */ + lastAt: string | null; +}; + +export type SyncCloudRelayBudgetStatus = SyncCloudRelayBudget & { + limit: number; + windowMs: number; + exhausted: boolean; +}; + +/** + * Why this machine's identity changed. Every one of these is logged with the + * same `identity_rotated` shape, so "where did my machine key go" is always + * answerable from the brain log alone. + */ +export type SyncCloudRelayIdentityReason = + /** No identity file existed — a genuinely new machine. */ + | "first_mint" + /** Both halves were unrecoverable from the file AND its backup. */ + | "corrupt_file_remint" + /** The machine key survived; only the HMAC secret had to be replaced. */ + | "secret_remint" + /** The primary file was unusable and the backup supplied the identity. */ + | "recovered_from_backup" + /** A confirmed relay claim conflict (HTTP 409) rotated the whole credential. */ + | "conflict_rotation"; + +export type SyncCloudRelayIdentityEvent = { + /** The key in force before this event; null when there was none to lose. */ + previousMachineKey: string | null; + machineKey: string; + reason: SyncCloudRelayIdentityReason; +}; + +export type SyncCloudRelayRotationResult = { + config: SyncCloudRelayConfig; + /** False when the expected key had already moved, or the budget is spent. */ + rotated: boolean; + /** The 24h rotation budget is spent: reconnect by hand, never mint again. */ + budgetExhausted: boolean; + rotationsInWindow: number; +}; + type SyncCloudRelayFile = Partial & { + identityRotations?: unknown; + pairingAutoRepairs?: unknown; + previousMachineKeys?: unknown; /** Deprecated kill-switch fields are accepted only so old files are cleaned up. */ enabled?: unknown; enabledSetByUser?: unknown; }; +/** Everything the identity file carries, as this module works with it. */ +type SyncCloudRelayState = { + config: SyncCloudRelayConfig; + identityRotations: SyncCloudRelayBudget; + pairingAutoRepairs: SyncCloudRelayBudget; + previousMachineKeys: string[]; +}; + +type ResolvedState = { + state: SyncCloudRelayState; + /** The identity differs from what the primary file literally held. */ + needsIdentityWrite: boolean; + /** + * Some half of the identity was minted in memory and exists on NO file yet. + * False means every field came off disk, so a reader may use it before the + * primary file has been rewritten. + */ + identityMinted: boolean; + needsWrite: boolean; + event: SyncCloudRelayIdentityEvent | null; +}; + +type SyncCloudRelayStoreLogger = { + info?: (event: string, data?: Record) => void; + warn?: (event: string, data?: Record) => void; +}; + type RotationLockOwner = { version: typeof IDENTITY_ROTATION_LOCK_VERSION; pid: number; @@ -85,34 +196,215 @@ export function signRelayHmacHex(secret: string, base: string): string { export type SyncCloudRelayStore = ReturnType; +function emptyBudget(): SyncCloudRelayBudget { + return { count: 0, windowStartedAt: null, lastAt: null }; +} + +function readIsoTimestamp(value: unknown): string | null { + return typeof value === "string" && value.trim() && !Number.isNaN(Date.parse(value)) + ? value + : null; +} + +/** + * A budget whose whole window is already spent, anchored at now. + * + * This is what an unreadable counter reads as. The limit has to be passed in + * because the same reader serves two different allowances, and "spent" is only + * meaningful against one of them. + */ +function spentBudget(limit: number): SyncCloudRelayBudget { + return { count: Math.max(1, limit), windowStartedAt: new Date().toISOString(), lastAt: null }; +} + +/** + * Read a persisted budget defensively, distinguishing the two ways it can fail + * to produce a number. + * + * ABSENT is the ordinary case — a machine that has never rotated or repaired + * writes no member at all (see `serializeBudget`) — and reads as empty. + * + * PRESENT BUT UNREADABLE is a loss, and reads as SPENT for a full window. + * Forgiving a garbled counter is the same failure mode as not persisting it: + * a crash loop that truncates the identity file on every boot would otherwise + * regain its whole rotation allowance each time, which is precisely the + * phantom-machine failure this budget was made durable to stop. + */ +function readBudget(value: unknown, lastKey: string, limit: number): SyncCloudRelayBudget { + if (value == null) return emptyBudget(); + if (typeof value !== "object" || Array.isArray(value)) return spentBudget(limit); + const record = value as Record; + const rawCount = record.count; + // A literal 0 is readable and means untouched; anything that is not a + // non-negative finite number is a counter this build cannot trust. + if (typeof rawCount !== "number" || !Number.isFinite(rawCount) || rawCount < 0) { + return spentBudget(limit); + } + const count = Math.floor(rawCount); + const windowStartedAt = readIsoTimestamp(record.windowStartedAt); + const lastAt = readIsoTimestamp(record[lastKey]); + if (count === 0) return emptyBudget(); + return { + count, + // A count with no window would never expire; anchor it to the last + // consumption, and failing that treat the window as starting now. + windowStartedAt: windowStartedAt ?? lastAt ?? new Date().toISOString(), + lastAt, + }; +} + +function serializeBudget( + budget: SyncCloudRelayBudget, + lastKey: string, +): Record | null { + if (budget.count <= 0) return null; + return { + count: budget.count, + ...(budget.windowStartedAt ? { windowStartedAt: budget.windowStartedAt } : {}), + ...(budget.lastAt ? { [lastKey]: budget.lastAt } : {}), + }; +} + +function isValidMachineKey(value: unknown): value is string { + return typeof value === "string" && /^[a-f0-9]{32,64}$/i.test(value); +} + +function isValidSecret(value: unknown): value is string { + return typeof value === "string" && value.length >= 32; +} + +function readPreviousMachineKeys(value: unknown): string[] { + if (!Array.isArray(value)) return []; + const keys: string[] = []; + for (const entry of value) { + if (!isValidMachineKey(entry) || keys.includes(entry)) continue; + keys.push(entry); + if (keys.length >= MAX_RETAINED_PREVIOUS_MACHINE_KEYS) break; + } + return keys; +} + /** * Persists the tunnel-relay identity next to the other sync secrets. * machineKey/secret are minted lazily on first read (matching the push-relay * store's randomBytes sizing) and the file is chmod 600. + * + * Two rules govern every path through this store, both bought by a production + * lockout: an identity is never discarded when any copy on disk still holds it, + * and every mint, rotation, or recovery leaves a log line naming what changed + * and why. */ -export function createSyncCloudRelayStore(args: { filePath: string; lockWaitMs?: number }) { +export function createSyncCloudRelayStore(args: { + filePath: string; + lockWaitMs?: number; + logger?: SyncCloudRelayStoreLogger; + /** Test seam for the rotation/auto-repair windows. */ + now?: () => number; +}) { fs.mkdirSync(path.dirname(args.filePath), { recursive: true }); const rotationLockPath = `${args.filePath}.rotate.lock`; + const backupPath = `${args.filePath}${IDENTITY_BACKUP_SUFFIX}`; const lockWaitMs = Math.max(0, args.lockWaitMs ?? IDENTITY_CONFIG_LOCK_WAIT_MS); const lockWaiter = new Int32Array(new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT)); + const now = args.now ?? Date.now; + const log = args.logger; + /** + * The machine key whose backup this process has already reconciled. Purely an + * optimisation: `load()` runs on every identity read, and re-statting the + * backup each time would put a syscall on the relay hot path. + */ + let backupVerifiedFor: string | null = null; + + const logIdentityEvent = (event: SyncCloudRelayIdentityEvent): void => { + // One event name for every identity transition, matching the tunnel + // client's `identity_rotated` payload, so a single grep answers "when did + // this machine's key change, and what changed it". + log?.info?.("sync_cloud_relay.identity_rotated", { + previousMachineKey: event.previousMachineKey, + machineKey: event.machineKey, + reason: event.reason, + }); + }; - const read = (): SyncCloudRelayFile => { - if (!fs.existsSync(args.filePath)) return {}; - return safeJsonParse(fs.readFileSync(args.filePath, "utf8"), {}); + /** + * Read one identity file. Unlike a `safeJsonParse(..., {})` read, a parse + * failure is reported as a FAILURE rather than as an empty object — the + * difference between "this machine has no identity yet" and "this machine's + * identity is temporarily unreadable", and conflating them is what minted a + * whole new machine out of one corrupt file. + */ + const readFileAt = (target: string): { + raw: SyncCloudRelayFile; + present: boolean; + parsed: boolean; + } => { + let text: string; + try { + text = fs.readFileSync(target, "utf8"); + } catch { + return { raw: {}, present: false, parsed: false }; + } + let value: unknown; + try { + value = JSON.parse(text); + } catch { + log?.warn?.("sync_cloud_relay.identity_file_unparsable", { file: path.basename(target) }); + return { raw: {}, present: true, parsed: false }; + } + if (!value || typeof value !== "object" || Array.isArray(value)) { + log?.warn?.("sync_cloud_relay.identity_file_unparsable", { file: path.basename(target) }); + return { raw: {}, present: true, parsed: false }; + } + return { raw: value as SyncCloudRelayFile, present: true, parsed: true }; }; - const write = (value: SyncCloudRelayConfig): void => { - const fileValue: SyncCloudRelayFile = { - machineKey: value.machineKey, - secret: value.secret, - ...(value.relayUrl ? { relayUrl: value.relayUrl } : {}), + const serialize = (state: SyncCloudRelayState): string => { + const identityRotations = serializeBudget(state.identityRotations, "lastRotationAt"); + const pairingAutoRepairs = serializeBudget(state.pairingAutoRepairs, "lastRepairAt"); + // Optional members stay ABSENT while unused so a machine that has never + // rotated serializes byte-for-byte like it always did. + const fileValue = { + machineKey: state.config.machineKey, + secret: state.config.secret, + ...(state.config.relayUrl ? { relayUrl: state.config.relayUrl } : {}), + ...(identityRotations ? { identityRotations } : {}), + ...(pairingAutoRepairs ? { pairingAutoRepairs } : {}), + ...(state.previousMachineKeys.length > 0 + ? { previousMachineKeys: state.previousMachineKeys } + : {}), }; - // 0o600 at temp-file creation so the identity secret is never world-readable. - writeTextAtomic(args.filePath, `${JSON.stringify(fileValue, null, 2)}\n`, { mode: 0o600 }); + return `${JSON.stringify(fileValue, null, 2)}\n`; + }; + + /** + * Atomic AND durable: 0o600 temp file, fsync, rename, then fsync the + * directory so the rename itself survives a power loss. + * + * `writeFileAtomic` is the repo's one durable writer, and using it is the + * whole point: it renames straight onto the target on every platform (libuv's + * `MOVEFILE_REPLACE_EXISTING`), so the identity file is never briefly absent. + * A store that deleted the target first would hand a concurrent reader with a + * damaged primary a MISSING backup — and that reader mints a phantom machine, + * which is the exact failure this module exists to prevent. + */ + const writeIdentityFile = (target: string, text: string): void => { + fs.mkdirSync(path.dirname(target), { recursive: true }); + writeFileAtomic(target, text, { fsync: true, mode: 0o600 }); + }; + + const write = (state: SyncCloudRelayState): void => { + const text = serialize(state); + writeIdentityFile(args.filePath, text); + // The backup follows the primary, never leads it: a reader that falls back + // to the backup must find the last identity that was actually in force. try { - fs.chmodSync(args.filePath, 0o600); - } catch { - // ignore chmod failures on platforms that don't support it + writeIdentityFile(backupPath, text); + backupVerifiedFor = state.config.machineKey; + } catch (error) { + backupVerifiedFor = null; + log?.warn?.("sync_cloud_relay.identity_backup_write_failed", { + error: error instanceof Error ? error.message : String(error), + }); } }; @@ -122,37 +414,106 @@ export function createSyncCloudRelayStore(args: { filePath: string; lockWaitMs?: ...(relayUrl ? { relayUrl } : {}), }); - const normalize = (raw: SyncCloudRelayFile): { - config: SyncCloudRelayConfig; - needsIdentityWrite: boolean; - needsWrite: boolean; - } => { - const validMachineKey = typeof raw.machineKey === "string" - && /^[a-f0-9]{32,64}$/i.test(raw.machineKey) - ? raw.machineKey + const relayUrlFrom = (raw: SyncCloudRelayFile): string | undefined => + typeof raw.relayUrl === "string" && raw.relayUrl.trim() ? raw.relayUrl.trim() : undefined; + + /** + * Build the live state from the primary file, falling back to the backup for + * anything the primary cannot supply. + * + * The machine key is the one field that is preserved at any cost: it is kept + * from whichever file still holds one, and only a file pair that yields none + * at all may mint a new one. A broken secret costs a secret, never a machine. + */ + const resolve = (): ResolvedState => { + const primary = readFileAt(args.filePath); + const primaryMachineKey = isValidMachineKey(primary.raw.machineKey) + ? primary.raw.machineKey : null; - const validSecret = typeof raw.secret === "string" && raw.secret.length >= 32 - ? raw.secret + const primarySecret = isValidSecret(primary.raw.secret) ? primary.raw.secret : null; + + const needsBackup = !primaryMachineKey || !primarySecret; + const backup = needsBackup ? readFileAt(backupPath) : null; + const backupMachineKey = backup && isValidMachineKey(backup.raw.machineKey) + ? backup.raw.machineKey : null; - const generated = validMachineKey && validSecret ? null : mintIdentity(); - // A machine key and secret are one relay credential. If either half is - // invalid, never combine the surviving half with a newly minted value. - const machineKey = generated?.machineKey ?? validMachineKey; - const secret = generated?.secret ?? validSecret; - if (!machineKey || !secret) { - throw new Error("Could not mint the ADE Relay machine identity."); + const backupSecret = backup && isValidSecret(backup.raw.secret) ? backup.raw.secret : null; + + const machineKey = primaryMachineKey ?? backupMachineKey; + // Pair the secret with its own machine key wherever possible: a secret is + // only meaningful for the key it was minted alongside. + const secret = primaryMachineKey && primarySecret + ? primarySecret + : machineKey && machineKey === backupMachineKey && backupSecret + ? backupSecret + : null; + + const source: SyncCloudRelayFile = machineKey && machineKey === primaryMachineKey + ? primary.raw + : backup?.raw ?? primary.raw; + const fallbackSource = source === primary.raw ? backup?.raw ?? {} : primary.raw; + + const relayUrl = relayUrlFrom(source) ?? relayUrlFrom(fallbackSource); + const identityRotations = readBudget( + source.identityRotations ?? fallbackSource.identityRotations, + "lastRotationAt", + MAX_IDENTITY_ROTATIONS_PER_WINDOW, + ); + const pairingAutoRepairs = readBudget( + source.pairingAutoRepairs ?? fallbackSource.pairingAutoRepairs, + "lastRepairAt", + MAX_PAIRING_AUTO_REPAIRS_PER_WINDOW, + ); + let previousMachineKeys = readPreviousMachineKeys( + source.previousMachineKeys ?? fallbackSource.previousMachineKeys, + ); + + let event: SyncCloudRelayIdentityEvent | null = null; + let identityMinted = false; + let config: SyncCloudRelayConfig; + if (machineKey && secret) { + config = { machineKey, secret, ...(relayUrl ? { relayUrl } : {}) }; + if (!primaryMachineKey) { + event = { previousMachineKey: null, machineKey, reason: "recovered_from_backup" }; + } + } else if (machineKey) { + // The machine survives; only its relay secret is replaced. + identityMinted = true; + config = { + machineKey, + secret: mintIdentity().secret, + ...(relayUrl ? { relayUrl } : {}), + }; + event = { + previousMachineKey: machineKey, + machineKey, + reason: primaryMachineKey ? "secret_remint" : "recovered_from_backup", + }; + } else { + const minted = mintIdentity(relayUrl); + identityMinted = true; + config = minted; + // A file that exists but yields nothing is a LOSS, not a new machine, and + // it is logged as such so the roster phantom it may create is explainable. + const lost = primary.present || backup?.present === true; + event = { + previousMachineKey: null, + machineKey: minted.machineKey, + reason: lost ? "corrupt_file_remint" : "first_mint", + }; + previousMachineKeys = []; } - const needsIdentityWrite = raw.machineKey !== machineKey || raw.secret !== secret; - const hasDeprecatedKillSwitchFields = Object.prototype.hasOwnProperty.call(raw, "enabled") - || Object.prototype.hasOwnProperty.call(raw, "enabledSetByUser"); + + const needsIdentityWrite = primary.raw.machineKey !== config.machineKey + || primary.raw.secret !== config.secret; + const hasDeprecatedKillSwitchFields = Object.prototype.hasOwnProperty.call(primary.raw, "enabled") + || Object.prototype.hasOwnProperty.call(primary.raw, "enabledSetByUser"); return { - config: { - machineKey, - secret, - relayUrl: typeof raw.relayUrl === "string" && raw.relayUrl.trim() ? raw.relayUrl.trim() : undefined, - }, + state: { config, identityRotations, pairingAutoRepairs, previousMachineKeys }, needsIdentityWrite, - needsWrite: needsIdentityWrite || hasDeprecatedKillSwitchFields, + identityMinted, + needsWrite: needsIdentityWrite || hasDeprecatedKillSwitchFields || !primary.parsed, + event, }; }; @@ -258,17 +619,88 @@ export function createSyncCloudRelayStore(args: { filePath: string; lockWaitMs?: const busyError = (): Error => new Error("The ADE Relay configuration is being updated by another live ADE process."); - const configWhileLocked = (): SyncCloudRelayConfig => { - const latest = normalize(read()); - if (latest.needsIdentityWrite) throw busyError(); - return latest.config; + /** + * The best answer a reader may have while ANOTHER process owns the rotation + * lock and this one therefore cannot persist anything. + * + * A MINTED identity — a first mint, a remint out of two dead files, or a + * replacement secret — exists nowhere but this process's memory, and the + * process holding the lock may be about to write a different one. Handing + * that out would put a machine key on the wire that never lands on disk, so + * the read fails loudly instead. + * + * An identity RECOVERED from the `.bak` sibling is a different thing: every + * field of it is already on disk, and it is the same identity the lock holder + * itself will resolve. Refusing it turned "the primary file is damaged and + * another ADE is busy" into a hard failure of every status accessor — + * `getConfig`, `getRelayUrl`, `getRelayWssUrl` — for a machine whose identity + * was never actually in doubt. Return it and let the rewrite of the primary + * happen on the next uncontended read. + */ + const stateWhileLocked = (): SyncCloudRelayState => { + const latest = resolve(); + if (latest.needsIdentityWrite && latest.identityMinted) throw busyError(); + return latest.state; + }; + + /** + * Roll a persisted budget forward and try to spend one unit of it. + * `windowStartedAt` is only reset once the window has fully elapsed, so a + * steady drip of attempts cannot keep the window open indefinitely. + */ + const consumeBudget = ( + budget: SyncCloudRelayBudget, + windowMs: number, + limit: number, + ): { budget: SyncCloudRelayBudget; allowed: boolean; countInWindow: number } => { + const nowMs = now(); + const startedAtMs = budget.windowStartedAt ? Date.parse(budget.windowStartedAt) : Number.NaN; + const windowExpired = !Number.isFinite(startedAtMs) || nowMs - startedAtMs >= windowMs; + const countInWindow = windowExpired ? 0 : budget.count; + if (countInWindow >= limit) { + return { budget, allowed: false, countInWindow }; + } + const nowIso = new Date(nowMs).toISOString(); + return { + budget: { + count: countInWindow + 1, + windowStartedAt: windowExpired ? nowIso : budget.windowStartedAt ?? nowIso, + lastAt: nowIso, + }, + allowed: true, + countInWindow: countInWindow + 1, + }; + }; + + const budgetStatus = ( + budget: SyncCloudRelayBudget, + windowMs: number, + limit: number, + ): SyncCloudRelayBudgetStatus => { + const nowMs = now(); + const startedAtMs = budget.windowStartedAt ? Date.parse(budget.windowStartedAt) : Number.NaN; + const windowExpired = !Number.isFinite(startedAtMs) || nowMs - startedAtMs >= windowMs; + const countInWindow = windowExpired ? 0 : budget.count; + return { + count: countInWindow, + windowStartedAt: windowExpired ? null : budget.windowStartedAt, + lastAt: budget.lastAt, + limit, + windowMs, + exhausted: countInWindow >= limit, + }; }; - const updateConfig = ( - update: (current: SyncCloudRelayConfig) => { config: SyncCloudRelayConfig; changed: boolean }, - onLocked?: () => SyncCloudRelayConfig, + const updateState = ( + update: (current: SyncCloudRelayState) => { + state: SyncCloudRelayState; + changed: boolean; + event?: SyncCloudRelayIdentityEvent | null; + result?: T; + }, + onLocked?: () => { state: SyncCloudRelayState; result?: T }, waitMs = lockWaitMs, - ): SyncCloudRelayConfig => { + ): { state: SyncCloudRelayState; result?: T } => { const lease = acquireRotationLockWithWait(waitMs); if (!lease) { if (onLocked) return onLocked(); @@ -276,109 +708,254 @@ export function createSyncCloudRelayStore(args: { filePath: string; lockWaitMs?: } try { // Read only after acquiring the shared lock so every whole-file update - // is based on the latest identity and relay URL. - const current = normalize(read()); - const result = update(current.config); - if (current.needsWrite || result.changed) write(result.config); - return result.config; + // is based on the latest identity, relay URL, and budgets. + const current = resolve(); + const outcome = update(current.state); + if (current.needsWrite || outcome.changed) { + write(outcome.state); + // Logged only after the write lands: an event announcing an identity + // that failed to persist would be worse than silence. + if (current.event) logIdentityEvent(current.event); + if (outcome.event) logIdentityEvent(outcome.event); + } + return { state: outcome.state, result: outcome.result }; } finally { releaseRotationLock(lease); } }; + /** + * Make sure the backup exists and names the identity currently in force. + * Cheap and idempotent; the in-process guard keeps it off the hot path after + * the first read. + */ + const ensureBackup = (state: SyncCloudRelayState): void => { + if (backupVerifiedFor === state.config.machineKey) return; + const existing = readFileAt(backupPath); + if (existing.parsed && existing.raw.machineKey === state.config.machineKey + && existing.raw.secret === state.config.secret) { + backupVerifiedFor = state.config.machineKey; + return; + } + try { + writeIdentityFile(backupPath, serialize(state)); + backupVerifiedFor = state.config.machineKey; + } catch (error) { + log?.warn?.("sync_cloud_relay.identity_backup_write_failed", { + error: error instanceof Error ? error.message : String(error), + }); + } + }; + // Reads the file and fills in a freshly generated identity when absent, // persisting it so the machineKey stays stable across restarts. - const load = (): SyncCloudRelayConfig => { - const current = normalize(read()); - if (!current.needsWrite) return current.config; - return updateConfig( - (latest) => ({ config: latest, changed: false }), - configWhileLocked, - ); + const load = (): SyncCloudRelayState => { + const current = resolve(); + if (!current.needsWrite) { + ensureBackup(current.state); + return current.state; + } + return updateState( + (latest) => ({ state: latest, changed: false }), + () => ({ state: stateWhileLocked() }), + ).state; }; return { getConfig(): SyncCloudRelayConfig { - return load(); + return load().config; }, getMachineIdentity(): { machineKey: string; secret: string } { - const { machineKey, secret } = load(); + const { machineKey, secret } = load().config; return { machineKey, secret }; }, getRelayUrl(): string { - return load().relayUrl ?? defaultRelayUrl(); + return load().config.relayUrl ?? defaultRelayUrl(); }, setRelayUrl(relayUrl: string | null): SyncCloudRelayConfig { - return updateConfig((current) => ({ - config: { ...current, relayUrl: relayUrl?.trim() || undefined }, + return updateState((current) => ({ + state: { + ...current, + config: { ...current.config, relayUrl: relayUrl?.trim() || undefined }, + }, changed: true, - })); + })).state.config; }, /** * Replaces a relay identity only when the caller is still looking at the - * expected machine key. The exclusive sibling lock serializes brain - * processes so exactly one confirmed-conflict recovery wins. + * expected machine key, and only while the persisted 24-hour rotation + * budget allows it. The exclusive sibling lock serializes brain processes + * so exactly one confirmed-conflict recovery wins. + * + * The budget lives in the FILE, not in a closure: the previous in-memory + * counter reset on every brain restart, so a restart loop could mint one + * new machine row per boot and bury the owner's roster in phantoms. */ - rotateMachineIdentity(expectedMachineKey: string): SyncCloudRelayConfig { - return updateConfig((current) => { - if (current.machineKey !== expectedMachineKey) { - return { config: current, changed: false }; - } - const next = mintIdentity(current.relayUrl); - return { config: next, changed: true }; - }, configWhileLocked, 0); + rotateMachineIdentity(expectedMachineKey: string): SyncCloudRelayRotationResult { + const outcome = updateState( + (current) => { + if (current.config.machineKey !== expectedMachineKey) { + return { + state: current, + changed: false, + result: { + config: current.config, + rotated: false, + budgetExhausted: false, + rotationsInWindow: budgetStatus( + current.identityRotations, + IDENTITY_ROTATION_WINDOW_MS, + MAX_IDENTITY_ROTATIONS_PER_WINDOW, + ).count, + }, + }; + } + const spend = consumeBudget( + current.identityRotations, + IDENTITY_ROTATION_WINDOW_MS, + MAX_IDENTITY_ROTATIONS_PER_WINDOW, + ); + if (!spend.allowed) { + return { + state: current, + changed: false, + result: { + config: current.config, + rotated: false, + budgetExhausted: true, + rotationsInWindow: spend.countInWindow, + }, + }; + } + const next = mintIdentity(current.config.relayUrl); + const previousMachineKeys = [ + current.config.machineKey, + ...current.previousMachineKeys.filter((key) => key !== current.config.machineKey), + ].slice(0, MAX_RETAINED_PREVIOUS_MACHINE_KEYS); + return { + state: { + config: next, + identityRotations: spend.budget, + pairingAutoRepairs: current.pairingAutoRepairs, + previousMachineKeys, + }, + changed: true, + event: { + previousMachineKey: current.config.machineKey, + machineKey: next.machineKey, + reason: "conflict_rotation", + }, + result: { + config: next, + rotated: true, + budgetExhausted: spend.countInWindow >= MAX_IDENTITY_ROTATIONS_PER_WINDOW, + rotationsInWindow: spend.countInWindow, + }, + }; + }, + () => { + const state = stateWhileLocked(); + return { + state, + result: { + config: state.config, + rotated: false, + budgetExhausted: false, + rotationsInWindow: budgetStatus( + state.identityRotations, + IDENTITY_ROTATION_WINDOW_MS, + MAX_IDENTITY_ROTATIONS_PER_WINDOW, + ).count, + }, + }; + }, + 0, + ); + return outcome.result ?? { + config: outcome.state.config, + rotated: false, + budgetExhausted: false, + rotationsInWindow: 0, + }; + }, + + /** + * Spend one automatic pairing repair from the persisted 6-hour budget. + * + * Persisted for the same reason the rotation budget is: an auto-repair loop + * whose allowance resets on restart is a machine that hammers the account + * directory forever, and the brain restarts far more often than six hours. + */ + tryConsumePairingAutoRepair(): { allowed: boolean; countInWindow: number; limit: number } { + let allowed = false; + let countInWindow = 0; + try { + const outcome = updateState<{ allowed: boolean; countInWindow: number }>( + (current) => { + const spend = consumeBudget( + current.pairingAutoRepairs, + PAIRING_AUTO_REPAIR_WINDOW_MS, + MAX_PAIRING_AUTO_REPAIRS_PER_WINDOW, + ); + return { + state: spend.allowed + ? { ...current, pairingAutoRepairs: spend.budget } + : current, + changed: spend.allowed, + result: { allowed: spend.allowed, countInWindow: spend.countInWindow }, + }; + }, + () => ({ state: stateWhileLocked(), result: { allowed: false, countInWindow: 0 } }), + ); + allowed = outcome.result?.allowed ?? false; + countInWindow = outcome.result?.countInWindow ?? 0; + } catch { + // A contended or unwritable store must not authorize an unbounded + // repair loop: fail closed and let the next cycle try again. + allowed = false; + } + return { allowed, countInWindow, limit: MAX_PAIRING_AUTO_REPAIRS_PER_WINDOW }; + }, + + /** + * Reconcile the directory's `supersededMachineKeys` against the keys this + * machine actually retired, and forget the ones it confirmed. + * + * Returns only OUR keys. A server naming a key we never held describes some + * other device, and acting on it would be the phantom problem in reverse. + * Older directories send nothing, which is simply an empty answer. + */ + confirmSupersededMachineKeys(serverKeys: readonly unknown[]): string[] { + const claimed = readPreviousMachineKeys(serverKeys); + if (claimed.length === 0) return []; + const current = load(); + const confirmed = current.previousMachineKeys.filter((key) => + claimed.some((candidate) => candidate.toLowerCase() === key.toLowerCase())); + if (confirmed.length === 0) return []; + try { + updateState((state) => ({ + state: { + ...state, + previousMachineKeys: state.previousMachineKeys.filter( + (key) => !confirmed.includes(key), + ), + }, + changed: true, + })); + } catch { + // Bookkeeping only — the confirmation is still true if the prune fails. + } + return confirmed; }, /** `wss:///connect/` — the value the QR integration reads. */ getRelayWssUrl(): string { - const { relayUrl, machineKey } = load(); + const { relayUrl, machineKey } = load().config; return deriveRelayWssConnectUrl(relayUrl ?? defaultRelayUrl(), machineKey); }, }; } - -/** - * The relay status the desktop and the CLI read, built the same way whether a - * project scope owns sync or the brain answers for the bare machine. - * - * Both surfaces had their own copy of this projection and they had already - * drifted apart in whitespace only — one edit away from drifting in meaning, - * which would show two different relay stories for one machine. - * - * `accountSignedIn` is the gate: without it the live fields collapse to their - * off values and the error becomes the sign-in prompt, so a signed-out machine - * never reports a connection it cannot have. - */ -export function buildSyncCloudRelayStatus(args: { - cloudRelayStore: Pick; - tunnelStatus: SyncTunnelClientStatus | null; - accountSignedIn: boolean; -}): SyncCloudRelayStatus { - const { cloudRelayStore, tunnelStatus, accountSignedIn } = args; - // Built as a variable, not returned inline: the relay self-probe fields below - // are not in `SyncCloudRelayStatus` yet and both original copies passed them - // through. Dropping them here would quietly blank the desktop's probe row. - const status = { - relayWssUrl: cloudRelayStore.getRelayWssUrl(), - machineKey: cloudRelayStore.getConfig().machineKey, - relayUrl: cloudRelayStore.getRelayUrl(), - connected: accountSignedIn && (tunnelStatus?.connected ?? false), - activeTunnels: accountSignedIn ? tunnelStatus?.activeTunnels ?? 0 : 0, - relayBridgeValidated: accountSignedIn && (tunnelStatus?.relayBridgeValidated ?? false), - lastFailureAt: tunnelStatus?.lastFailureAt ?? null, - lastControlOpenAt: tunnelStatus?.lastControlOpenAt ?? null, - lastBridgeValidationAt: tunnelStatus?.lastBridgeValidationAt ?? null, - relayEndToEndVerifiedAt: tunnelStatus?.relayEndToEndVerifiedAt ?? null, - relayEndToEndFailure: tunnelStatus?.relayEndToEndFailure ?? null, - relayEndToEndRoundTripMs: tunnelStatus?.relayEndToEndRoundTripMs ?? null, - lastControlError: tunnelStatus?.lastControlError ?? null, - lastError: accountSignedIn - ? tunnelStatus?.lastError ?? null - : RELAY_SIGN_IN_REQUIRED_MESSAGE, - }; - return status; -} diff --git a/apps/ade-cli/src/services/sync/syncRouteHealth.ts b/apps/ade-cli/src/services/sync/syncRouteHealth.ts index 38c7760c2..8bc950235 100644 --- a/apps/ade-cli/src/services/sync/syncRouteHealth.ts +++ b/apps/ade-cli/src/services/sync/syncRouteHealth.ts @@ -1,6 +1,9 @@ import type { SyncRouteHealth } from "../../../../desktop/src/shared/types"; import type { SyncLoopbackValidationStatus } from "./syncLoopbackProbe"; -import type { SyncTunnelClientStatus } from "./syncTunnelClientService"; +import { + RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE, + type SyncTunnelClientStatus, +} from "./syncTunnelClientService"; /** * How a machine describes its own inbound routes. @@ -130,6 +133,12 @@ function resolveRelaySkipReason(args: BuildRelayRouteHealthArgs & { // fault, and its reason is the only one that tells the user what to // actually do — so it outranks the raw close text. return status.controlSuppressedReason + // A spent rotation budget outranks the raw close text for the same + // reason: this machine will not mint another identity, so the only way + // back is reconnecting the computer, and nothing else says that. + ?? (status.identityRotationCapped + ? status.lastError ?? RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE + : null) ?? status.lastControlError ?? status.lastError ?? "Relay control is not connected."; diff --git a/apps/ade-cli/src/services/sync/syncService.ts b/apps/ade-cli/src/services/sync/syncService.ts index 47dd92279..17cbd15f9 100644 --- a/apps/ade-cli/src/services/sync/syncService.ts +++ b/apps/ade-cli/src/services/sync/syncService.ts @@ -65,10 +65,10 @@ import { createSyncPairingStore } from "./syncPairingStore"; import { isValidDpopPublicKey } from "./syncPairingStore"; import { createSyncSecurityStore } from "./syncSecurityStore"; import { - buildSyncCloudRelayStatus, createSyncCloudRelayStore, type SyncCloudRelayStore, } from "./syncCloudRelayStore"; +import { buildSyncCloudRelayStatus } from "./syncCloudRelayStatus"; import { createSyncPeerService } from "./syncPeerService"; import { createSyncPinStore } from "./syncPinStore"; import { createSyncRuntimeNameStore } from "./syncRuntimeNameStore"; @@ -541,6 +541,10 @@ export function createSyncService(args: SyncServiceArgs) { }); const cloudRelayStore = args.cloudRelayStore ?? createSyncCloudRelayStore({ filePath: path.join(pairingStateDir, CLOUD_RELAY_FILE), + // Identity mints, rotations, and backup recoveries are logged wherever the + // store is built: a machine key that changes with no record is how a live + // computer turned into a phantom row its owner deleted. + logger: args.logger, }); const accountAuthService = args.accountAuthService ?? getSharedAccountAuthService({ projectRoots: () => [args.projectRoot], diff --git a/apps/ade-cli/src/services/sync/syncTunnelClientService.test.ts b/apps/ade-cli/src/services/sync/syncTunnelClientService.test.ts index 59a72aa39..56f6224f5 100644 --- a/apps/ade-cli/src/services/sync/syncTunnelClientService.test.ts +++ b/apps/ade-cli/src/services/sync/syncTunnelClientService.test.ts @@ -25,6 +25,7 @@ import { RELAY_CLOSE_FORWARD_FAILED, RELAY_CLOSE_HOST_UNAVAILABLE, RELAY_CONTROL_REPLACED_MESSAGE, + RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE, RELAY_READY_VERSION, RELAY_SELF_PROBE_DEBOUNCE_MS, } from "./syncTunnelClientService"; @@ -48,10 +49,9 @@ function fakeStore(relayUrl = "https://relay.example.com"): SyncCloudRelayStore setRelayUrl: () => identity, getRelayWssUrl: () => `wss://relay.example.com/connect/${identity.machineKey}`, rotateMachineIdentity: (expectedMachineKey: string) => { - if (identity.machineKey === expectedMachineKey) { - identity = { machineKey: "c".repeat(32), secret: "d".repeat(48) }; - } - return identity; + const rotated = identity.machineKey === expectedMachineKey; + if (rotated) identity = { machineKey: "c".repeat(32), secret: "d".repeat(48) }; + return { config: identity, rotated, budgetExhausted: false, rotationsInWindow: rotated ? 1 : 0 }; }, } as unknown as SyncCloudRelayStore; } @@ -1578,6 +1578,45 @@ describe("createSyncTunnelClientService", () => { } }); + it("stops minting identities and asks for a reconnect once the budget is spent", async () => { + const originalFetch = globalThis.fetch; + const fetchMock = vi.fn(async () => new Response(null, { status: 409 })); + globalThis.fetch = fetchMock as unknown as typeof fetch; + const rotateMachineIdentity = vi.fn(() => ({ + config: { machineKey: "a".repeat(32), secret: "b".repeat(48) }, + rotated: false, + budgetExhausted: true, + rotationsInWindow: 2, + })); + const store = { + ...fakeStore("http://127.0.0.1:9"), + rotateMachineIdentity, + } as unknown as SyncCloudRelayStore; + const service = createSyncTunnelClientService({ + getSyncPort: () => 8787, + getExpectedLoopbackNonce: () => "f".repeat(32), + getRelayBridgeProof: () => "e".repeat(43), + configStore: store, + // Keep the retry far away; the capped state, not the schedule, is under test. + reconnectBackoffMs: () => 600_000, + }); + + try { + await service.start(); + await vi.waitFor(() => { + expect(service.getStatus().identityRotationCapped).toBe(true); + }); + // The machine keeps the key the account already knows about instead of + // minting yet another row nobody asked for. + expect(service.getStatus().machineKey).toBe("a".repeat(32)); + expect(service.getStatus().lastError).toBe(RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE); + expect(rotateMachineIdentity).toHaveBeenCalledTimes(1); + } finally { + await service.dispose(); + globalThis.fetch = originalFetch; + } + }); + it("claims again when another process rotates the shared identity", async () => { const relay = new WebSocketServer({ host: "127.0.0.1", port: 0 }); await new Promise((resolve, reject) => { @@ -1594,7 +1633,12 @@ describe("createSyncTunnelClientService", () => { getRelayUrl: () => relayUrl, setRelayUrl: () => ({ ...identity, relayUrl }), getRelayWssUrl: () => `ws://127.0.0.1:${relayPort}/connect/${identity.machineKey}`, - rotateMachineIdentity: () => ({ ...identity, relayUrl }), + rotateMachineIdentity: () => ({ + config: { ...identity, relayUrl }, + rotated: false, + budgetExhausted: false, + rotationsInWindow: 0, + }), } as unknown as SyncCloudRelayStore; const controlPaths: string[] = []; relay.on("connection", (socket, request) => { diff --git a/apps/ade-cli/src/services/sync/syncTunnelClientService.ts b/apps/ade-cli/src/services/sync/syncTunnelClientService.ts index 2f6c0a328..9c60a7abd 100644 --- a/apps/ade-cli/src/services/sync/syncTunnelClientService.ts +++ b/apps/ade-cli/src/services/sync/syncTunnelClientService.ts @@ -48,6 +48,12 @@ export type SyncTunnelClientStatus = { /** True while the client is deliberately not redialing after a 4505 eviction. */ controlSuppressed?: boolean; controlSuppressedReason?: string | null; + /** + * The relay refused this machine's key AND the persisted rotation budget is + * spent, so the client will not mint another identity. Additive and optional: + * older consumers keep reading `lastError`, which carries the same sentence. + */ + identityRotationCapped?: boolean; /** Epoch ms the current uninterrupted control outage began; null when connected. */ controlFailingSinceMs?: number | null; }; @@ -189,12 +195,19 @@ export const MAX_CONTROL_REPLACED_REATTEMPTS = 3; */ export const CONTROL_REPLACED_REARM_MS = 10 * 60_000; export const RELAY_SIGN_IN_REQUIRED_MESSAGE = "Sign in to ADE to use ADE Relay."; +/** + * What a machine says once it has spent its rotation budget on relay claim + * conflicts. It names the only safe next step: minting another identity would + * add another phantom row to the owner's account, and the row this machine + * already owns is the one that has to be repaired. + */ +export const RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE = + "This computer needs to be reconnected to your ADE account."; export const BRIDGE_VALIDATION_LEASE_MS = 2_000; export const CONTROL_READY_STABLE_MS = 5_000; export const RELAY_READY_VERSION = 2; export const RELAY_SELF_PROBE_DEBOUNCE_MS = 2_000; const MAX_UNEXPECTED_RESPONSE_BODY_BYTES = 512; -const MAX_CONFIRMED_CONFLICT_ROTATIONS = 1; class RelayClaimError extends Error { logged = false; @@ -405,7 +418,12 @@ export function createSyncTunnelClientService(args: SyncTunnelClientArgs): SyncT let claimedIdentity: { relayOrigin: string; machineKey: string } | null = null; let accountLeaseExpiresAtMs: number | null = null; let consecutiveAccountLeaseFailures = 0; - let confirmedConflictRotations = 0; + /** + * Latched once the persisted rotation budget refuses another mint. Cleared by + * the next successful claim, so a machine that gets repaired stops saying it + * needs repairing without a restart. + */ + let identityRotationCapped = false; let identityRotationPendingPublish = false; let accountGeneration = 0; let controlGeneration = 0; @@ -763,6 +781,13 @@ export function createSyncTunnelClientService(args: SyncTunnelClientArgs): SyncT } await response.body?.cancel().catch(() => {}); claimedIdentity = { relayOrigin, machineKey: id.machineKey }; + // The relay accepted this identity, so whatever conflict spent the budget + // is over. Reported through the status the desktop reads, not just cleared. + if (identityRotationCapped) { + identityRotationCapped = false; + if (lastError === RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE) lastError = null; + requestPublicationStatePublish("route-state-changed"); + } log.info?.("sync_tunnel.claimed", { machineKey: id.machineKey, status: response.status, @@ -777,7 +802,6 @@ export function createSyncTunnelClientService(args: SyncTunnelClientArgs): SyncT if ( !(error instanceof RelayClaimError) || error.status !== 409 - || confirmedConflictRotations >= MAX_CONFIRMED_CONFLICT_ROTATIONS || args.machineIdentity ) { throw error; @@ -790,17 +814,33 @@ export function createSyncTunnelClientService(args: SyncTunnelClientArgs): SyncT status: error.status, identityRotation: "starting", }); - const rotated = args.configStore.rotateMachineIdentity(initialIdentity.machineKey); - if (rotated.machineKey === initialIdentity.machineKey) { + // The budget lives in the identity file, so a restart cannot forgive a + // rotation. Exhausting it is a product state, not a retry: every extra + // mint is another machine row the owner never asked for. + const rotation = args.configStore.rotateMachineIdentity(initialIdentity.machineKey); + if (!rotation.rotated) { + if (rotation.budgetExhausted && !identityRotationCapped) { + identityRotationCapped = true; + log.warn?.("sync_tunnel.identity_rotation_capped", { + machineKey: initialIdentity.machineKey, + rotationsInWindow: rotation.rotationsInWindow, + reason: RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE, + }); + requestPublicationStatePublish("route-state-changed"); + } throw error; } - confirmedConflictRotations += 1; identityRotationPendingPublish = true; log.info?.("sync_tunnel.identity_rotated", { previousMachineKey: initialIdentity.machineKey, - machineKey: rotated.machineKey, + machineKey: rotation.config.machineKey, triggerStatus: 409, + rotationsInWindow: rotation.rotationsInWindow, }); + const rotated: MachineIdentity = { + machineKey: rotation.config.machineKey, + secret: rotation.config.secret, + }; await claimOnce(rotated); return rotated; } @@ -823,11 +863,19 @@ export function createSyncTunnelClientService(args: SyncTunnelClientArgs): SyncT try { id = await claimWithConflictRecovery(id); } catch (error) { - const reason = error instanceof Error ? error.message : String(error); + const rawReason = error instanceof Error ? error.message : String(error); + // A capped machine reports the action its owner can take, not the + // status code that produced it — `claim failed (409)` tells nobody that + // this computer has to be reconnected to the account. + const reason = identityRotationCapped + && error instanceof RelayClaimError + && error.status === 409 + ? RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE + : rawReason; recordFailure(reason); if (!(error instanceof RelayClaimError) || !error.logged) { log.warn?.("sync_tunnel.claim_failed", { - error: reason, + error: rawReason, machineKey: error instanceof RelayClaimError ? error.machineKey : id.machineKey, status: error instanceof RelayClaimError ? error.status : null, }); @@ -2016,6 +2064,7 @@ export function createSyncTunnelClientService(args: SyncTunnelClientArgs): SyncT machineKey, controlSuppressed: controlSuppressedReason != null, controlSuppressedReason, + identityRotationCapped: eligible && identityRotationCapped, controlFailingSinceMs: connected ? null : controlFailingSinceMs, }; }, diff --git a/apps/ade-cli/src/tuiClient/__tests__/reportIssue.test.ts b/apps/ade-cli/src/tuiClient/__tests__/reportIssue.test.ts index fc49a90de..19572e8ca 100644 --- a/apps/ade-cli/src/tuiClient/__tests__/reportIssue.test.ts +++ b/apps/ade-cli/src/tuiClient/__tests__/reportIssue.test.ts @@ -2,7 +2,7 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; import { afterEach, describe, expect, it } from "vitest"; -import { buildTuiDiagnosticReport } from "../reportIssue"; +import { buildTuiDiagnosticReport, sendTuiDiagnosticReport } from "../reportIssue"; const tempDirs: string[] = []; @@ -61,4 +61,83 @@ describe("buildTuiDiagnosticReport", () => { expect(built.issueUrl).toMatch(/^https:\/\/github\.com\//); expect(built.body).toContain(built.issueUrl); }); + + it("offers the send before anything has left the machine", () => { + const adeHome = tempDir("ade-home-"); + const built = buildTuiDiagnosticReport({ + projectRoot: null, + env: { ADE_HOME: adeHome }, + reportsDir: path.join(adeHome, "diagnostic-reports"), + }); + expect(built.body).toContain("/report-issue send"); + }); +}); + +describe("sendTuiDiagnosticReport", () => { + function build(): ReturnType { + const adeHome = tempDir("ade-home-"); + return buildTuiDiagnosticReport({ + projectRoot: null, + env: { ADE_HOME: adeHome, ADE_CLI_VERSION: "9.9.9" }, + reportsDir: path.join(adeHome, "diagnostic-reports"), + }); + } + + it("posts the exact bytes the pane showed and reports the reference", async () => { + const built = build(); + let posted: { url: string; body: unknown } | null = null; + const sent = await sendTuiDiagnosticReport(built, { + baseUrl: "https://directory.example", + getToken: async () => "token-abc", + fetchImpl: (async (url: string, init: RequestInit) => { + posted = { url, body: JSON.parse(String(init.body)) }; + expect((init.headers as Record).authorization).toBe("Bearer token-abc"); + return new Response(JSON.stringify({ id: "abcdef01-2345-6789-abcd-ef0123456789" }), { + status: 200, + headers: { "content-type": "application/json" }, + }); + }) as unknown as typeof fetch, + }); + + expect(sent.result.ok).toBe(true); + expect(posted!.url).toBe("https://directory.example/diagnostics/upload"); + // The report is redacted once, at build time; sending must not reshape it. + expect((posted!.body as { report: string }).report).toBe(built.sendable.report); + expect((posted!.body as { appVersion?: string }).appVersion).toBe("9.9.9"); + expect(sent.body).toContain("reference"); + // Nothing is left to do, so the pane stops asking for the send. + expect(sent.body).not.toContain("/report-issue send"); + // The local escape hatches survive a successful send. + expect(sent.body).toContain(built.issueUrl); + }); + + it("keeps the manual route when the send fails", async () => { + const built = build(); + const sent = await sendTuiDiagnosticReport(built, { + baseUrl: "https://directory.example", + getToken: async () => null, + fetchImpl: (async () => new Response("", { status: 429 })) as unknown as typeof fetch, + }); + + expect(sent.result).toEqual({ ok: false, reason: "rate_limited" }); + expect(sent.notice).toContain("Not sent"); + expect(sent.body).toContain(built.issueUrl); + expect(sent.body).toContain(built.filePath!); + // Still offered: a rate limit is a "try later", not a dead end. + expect(sent.body).toContain("/report-issue send"); + }); + + it("downgrades an unreachable service to a failure, not a throw", async () => { + const built = build(); + const sent = await sendTuiDiagnosticReport(built, { + baseUrl: "https://directory.example", + getToken: async () => null, + fetchImpl: (async () => { + throw new Error("getaddrinfo ENOTFOUND"); + }) as unknown as typeof fetch, + }); + + expect(sent.result).toEqual({ ok: false, reason: "network" }); + expect(sent.body).toContain(built.issueUrl); + }); }); diff --git a/apps/ade-cli/src/tuiClient/app.tsx b/apps/ade-cli/src/tuiClient/app.tsx index 57d8e7243..48e7558d6 100644 --- a/apps/ade-cli/src/tuiClient/app.tsx +++ b/apps/ade-cli/src/tuiClient/app.tsx @@ -10508,10 +10508,24 @@ export function AdeCodeApp({ project, forceEmbedded, requireSocket, socketPath, // Deliberately above the `!conn` gate: the report reads local files only, // so it still answers while the runtime is unreachable — the state a bug // report is most worth filing from. + // `--send` too: the CLI spelling is the one half these users already know, + // including alongside other flags they may carry over (`--open --send`). + const wantsSend = args + .trim() + .toLowerCase() + .split(/\s+/) + .some((argument) => /^--?send$/.test(argument) || argument === "send"); try { - const { buildTuiDiagnosticReport } = await import("./reportIssue"); + const { buildTuiDiagnosticReport, sendTuiDiagnosticReport } = await import("./reportIssue"); const built = buildTuiDiagnosticReport({ projectRoot: project.projectRoot }); + // Shown before anything is sent, so the file path and the issue URL are + // in hand no matter how the upload goes. setRightPane({ kind: "details", title: "Report issue", body: built.body }); + if (!wantsSend) return; + addNotice("Sending the report to ADE…", "info"); + const sent = await sendTuiDiagnosticReport(built); + setRightPane({ kind: "details", title: "Report issue", body: sent.body }); + addNotice(sent.notice, sent.result.ok ? "success" : "error"); } catch (error) { setRightPane({ kind: "details", @@ -17434,7 +17448,7 @@ export function AdeCodeApp({ project, forceEmbedded, requireSocket, socketPath, Retrying automatically · r retry now · Ctrl+C quit - Run ade report-issue --open in another terminal to prepare a report you can post. Personal information is removed. + Run ade report-issue --open --send in another terminal to send a report to ADE and post it. Personal information is removed. ); diff --git a/apps/ade-cli/src/tuiClient/commands.ts b/apps/ade-cli/src/tuiClient/commands.ts index fad0a9acc..daf51fc44 100644 --- a/apps/ade-cli/src/tuiClient/commands.ts +++ b/apps/ade-cli/src/tuiClient/commands.ts @@ -165,7 +165,7 @@ export const BUILTIN_COMMANDS: BuiltinCommand[] = [ { name: "/doctor", description: "Show ADE Code and Claude-compat diagnostics", placement: "right", category: "System" }, // The terminal counterpart of the desktop "Report issue" button. Local-only, // like `ade report-issue`, so it still works when the brain is the problem. - { name: "/report-issue", description: "Build a redacted diagnostic report for a bug report", placement: "right", category: "System" }, + { name: "/report-issue", description: "Build a redacted diagnostic report for a bug report; send hands it to ADE", placement: "right", argumentHint: "[send]", category: "System" }, { name: "/model", description: "Open the model, reasoning, and permission picker", placement: "right", category: "Model" }, { name: "/effort", description: "Open the reasoning-effort picker", placement: "right", category: "Model" }, { name: "/system", description: "Show system and runtime details", placement: "right", category: "System" }, diff --git a/apps/ade-cli/src/tuiClient/reportIssue.ts b/apps/ade-cli/src/tuiClient/reportIssue.ts index 5f1bfc1c8..56523d1fc 100644 --- a/apps/ade-cli/src/tuiClient/reportIssue.ts +++ b/apps/ade-cli/src/tuiClient/reportIssue.ts @@ -1,5 +1,11 @@ import path from "node:path"; -import { buildCliDiagnosticReport } from "../commands/reportIssue"; +import { + buildCliDiagnosticReport, + describeDiagnosticUpload, + sendDiagnosticReport, + type ReportIssueResult, +} from "../commands/reportIssue"; +import type { DiagnosticUploadResult } from "../../../desktop/src/shared/diagnosticsUpload"; import { diagnosticReportFilePath, writeDiagnosticReportFile, @@ -14,6 +20,12 @@ import { resolveMachineAdeLayout } from "../services/projects/machineLayout"; * anything, so it still answers on a machine where the brain is the problem. * The report is redacted in {@link buildCliDiagnosticReport} (private paths, * account names, emails, addresses and tokens) before it is written anywhere. + * + * `/report-issue send` is the third surface of the same one-way handoff the + * desktop's "Send to ADE" link and `ade report-issue --send` offer. Building + * and sending are separate steps on purpose: the report exists, is written, and + * is shown before a single byte leaves the machine, so a failed send costs the + * user nothing they had already been given. */ declare const __ADE_VERSION__: string | undefined; @@ -37,8 +49,50 @@ export type TuiDiagnosticReport = { filePath: string | null; issueUrl: string; installId: string; + /** + * Exactly what {@link sendTuiDiagnosticReport} posts — the same bytes this + * pane showed and wrote to disk, never a re-derived report. + */ + sendable: Pick; }; +/** + * The pane body, in both of its states: freshly built, or built and then sent. + * Pure so the wording is testable without touching disk or the network. + */ +export function formatTuiDiagnosticBody(input: { + filePath: string | null; + issueUrl: string; + installId: string; + /** Null before `/report-issue send` runs; the outcome after it does. */ + sent: DiagnosticUploadResult | null; +}): string { + const written = input.filePath !== null; + return [ + "A diagnostic report has been prepared.", + "Private paths, account names, emails and tokens are removed before it is written or sent.", + "", + input.sent ? describeDiagnosticUpload(input.sent) : null, + input.sent ? "" : null, + written ? "Saved to:" : "It could not be saved to disk, so paste it from the issue page instead.", + input.filePath, + "", + "File the issue at:", + input.issueUrl, + "", + `Install id: ${input.installId}`, + "", + // Only offered while it is still the thing left to do: repeating it under a + // "Sent to ADE" line reads as though the send did not take. + input.sent?.ok + ? null + : "Run /report-issue send to hand this same report to ADE instead of filing it yourself.", + "If ADE Code will not start at all, run ade report-issue --open --send in any terminal — it reads local files only.", + ] + .filter((line): line is string => line !== null) + .join("\n"); +} + export function buildTuiDiagnosticReport(args: { projectRoot: string | null; env?: NodeJS.ProcessEnv; @@ -60,21 +114,47 @@ export function buildTuiDiagnosticReport(args: { ?? path.join(resolveMachineAdeLayout(env).adeDir, "diagnostic-reports"); const filePath = diagnosticReportFilePath(reportsDir, surface, at); const written = writeDiagnosticReportFile(filePath, built.report); - const body = [ - "A diagnostic report has been prepared.", - "Private paths, account names, emails and tokens are removed before it is written.", - "", - written ? "Saved to:" : "It could not be saved to disk, so paste it from the issue page instead.", - written ? filePath : null, - "", - "File the issue at:", - built.issueUrl, - "", - `Install id: ${built.installId}`, - "", - "If ADE Code will not start at all, run ade report-issue --open in any terminal — it reads local files only.", - ] - .filter((line): line is string => line !== null) - .join("\n"); - return { body, filePath: written ? filePath : null, issueUrl: built.issueUrl, installId: built.installId }; + const resolved = { + filePath: written ? filePath : null, + issueUrl: built.issueUrl, + installId: built.installId, + }; + return { + ...resolved, + body: formatTuiDiagnosticBody({ ...resolved, sent: null }), + sendable: { + report: built.report, + installId: built.installId, + appVersion: built.appVersion, + secretsDir: built.secretsDir, + }, + }; +} + +/** + * `/report-issue send`: hand the already-built report to ADE and re-render the + * pane around the outcome. + * + * The upload itself, the account token, and the directory origin are all + * resolved by {@link sendDiagnosticReport}, so the terminal, the desktop, and + * `ade report-issue --send` post identical bytes to identical places. Failures + * come back as a result, never a throw — the report is already on disk and the + * issue URL is still in the pane, so a dead network downgrades to "file it + * yourself" rather than losing the report. + */ +export async function sendTuiDiagnosticReport( + built: TuiDiagnosticReport, + deps?: Parameters[1], +): Promise<{ result: DiagnosticUploadResult; body: string; notice: string }> { + const result = await sendDiagnosticReport(built.sendable, deps); + return { + result, + body: formatTuiDiagnosticBody({ + filePath: built.filePath, + issueUrl: built.issueUrl, + installId: built.installId, + sent: result, + }), + notice: describeDiagnosticUpload(result), + }; } diff --git a/apps/desktop/src/main/services/account/accountBridge.test.ts b/apps/desktop/src/main/services/account/accountBridge.test.ts index da4e15834..51fd910cb 100644 --- a/apps/desktop/src/main/services/account/accountBridge.test.ts +++ b/apps/desktop/src/main/services/account/accountBridge.test.ts @@ -39,9 +39,18 @@ vi.mock( }), ); +// The bridge builds a directory service per call, so the stub forwards to a +// mutable hook the removal suite can point at a success or a rejection. +const deleteMachine = vi.fn(); vi.mock( "../../../../../ade-cli/src/services/account/accountMachineDirectoryService", - () => ({ AccountMachineDirectoryService: class {} }), + () => ({ + AccountMachineDirectoryService: class { + deleteMachine(machineKey: string) { + return deleteMachine(machineKey); + } + }, + }), ); vi.mock( @@ -227,6 +236,70 @@ const BRAIN_SUCCESS = { statusHints: {}, }; +describe("accountBridge.removeMachine", () => { + afterEach(() => { + deleteMachine.mockReset(); + }); + + it("reports the removal the directory accepted, with no outcome but the pair", async () => { + deleteMachine.mockResolvedValue({ removed: true }); + const recordMachineRemoved = vi.fn(); + const bridge = createAccountBridge({ getProjectRoot: () => null, recordMachineRemoved }); + + await bridge.removeMachine("machine_1"); + + expect(recordMachineRemoved).toHaveBeenCalledTimes(1); + expect(recordMachineRemoved).toHaveBeenCalledWith("completed"); + // Nothing identifying the machine may be handed to the sink. + expect(recordMachineRemoved.mock.calls[0]).toHaveLength(1); + }); + + it("reports a refused removal as failed", async () => { + deleteMachine.mockRejectedValue(new Error("The account directory returned HTTP 403.")); + const recordMachineRemoved = vi.fn(); + const bridge = createAccountBridge({ getProjectRoot: () => null, recordMachineRemoved }); + + await expect(bridge.removeMachine("machine_1")).rejects.toThrow(/403/); + + expect(recordMachineRemoved).toHaveBeenCalledWith("failed"); + }); + + /** + * The reason this fact is recorded here and not at the IPC handler. The + * directory delete is the authoritative membership change; the Activity purge + * that follows rethrows so the user can retry clearing it. The machine IS off + * the account, and telemetry that called that a failed removal would be wrong + * about the only thing it reports. + */ + it("still reports the removal when only the Activity purge fails", async () => { + deleteMachine.mockResolvedValue({ removed: true }); + const recordMachineRemoved = vi.fn(); + const bridge = createAccountBridge({ + getProjectRoot: () => null, + recordMachineRemoved, + purgeMachineActivity: async () => { + throw new Error("activity store is locked"); + }, + }); + + await expect(bridge.removeMachine("machine_1")).rejects.toThrow(/Activity could not be cleared/); + + expect(recordMachineRemoved).toHaveBeenCalledWith("completed"); + }); + + it("never lets the telemetry sink break the removal", async () => { + deleteMachine.mockResolvedValue({ removed: true }); + const bridge = createAccountBridge({ + getProjectRoot: () => null, + recordMachineRemoved: () => { + throw new Error("analytics state file is unwritable"); + }, + }); + + await expect(bridge.removeMachine("machine_1")).resolves.toEqual({ removed: true }); + }); +}); + describe("accountBridge.repairMachinePairing", () => { it("forwards to the brain, because only that process owns the live push gate", async () => { const callBrainAccountAction = vi.fn(async () => BRAIN_SUCCESS); diff --git a/apps/desktop/src/main/services/account/accountBridge.ts b/apps/desktop/src/main/services/account/accountBridge.ts index b169c95da..5e98bbecf 100644 --- a/apps/desktop/src/main/services/account/accountBridge.ts +++ b/apps/desktop/src/main/services/account/accountBridge.ts @@ -72,6 +72,22 @@ type AccountBridgeOptions = { action: string, args?: Record, ) => Promise; + /** + * One coarse membership fact per machine removal: did the account directory + * accept it, or not. + * + * Reported from here rather than from the IPC handler because only this + * function knows WHICH half failed. The directory delete is the authoritative + * membership change; the Activity purge that follows it is a local cleanup + * that rethrows so the user can retry. From outside, that rethrow looks like a + * failed removal — the machine is off the account either way — and telemetry + * that said so would be wrong about the only thing it reports. + * + * Nothing about the machine travels: no key, no name, no account id. The + * caller owns the event vocabulary, so this bridge keeps no analytics + * dependency. + */ + recordMachineRemoved?: (outcome: "completed" | "failed") => void; logger?: { info(message: string, meta?: Record): void; warn(message: string, meta?: Record): void; @@ -693,7 +709,22 @@ export function createAccountBridge(options: AccountBridgeOptions): AccountBridg // Directory removal first: it is the authoritative membership change, and // a purge that landed against a machine still on the roster would just be // refilled by its next publish. - const result = await directoryService().deleteMachine(machineKey); + let result: AdeAccountMachineRemovalResult; + try { + result = await directoryService().deleteMachine(machineKey); + } catch (error) { + try { + options.recordMachineRemoved?.("failed"); + } catch { + // Never let a telemetry sink mask the directory's own failure. + } + throw error; + } + try { + options.recordMachineRemoved?.("completed"); + } catch { + // Same: the removal already happened and the caller must still see it. + } try { await options.purgeMachineActivity?.(machineKey); } catch (error) { diff --git a/apps/desktop/src/main/services/adeActions/domains.ts b/apps/desktop/src/main/services/adeActions/domains.ts new file mode 100644 index 000000000..46e553585 --- /dev/null +++ b/apps/desktop/src/main/services/adeActions/domains.ts @@ -0,0 +1,59 @@ +/** + * The closed list of ADE action domains, and nothing else. + * + * It lives apart from `registry.ts` because that module pulls in the whole + * runtime service graph (auth services, the CLI bootstrap) while several + * consumers need only the names — the analytics policy, loaded by the analytics + * service and its exporters, is the one that used to keep a hand-written copy + * of all 45 entries. A file with zero imports can be read by any of them. + */ +export const ADE_ACTION_DOMAIN_NAMES = [ + "account", + "attention", + "lane", + "git", + "diff", + "conflicts", + "pr", + "tests", + "chat", + "keybindings", + "ai", + "onboarding", + "automation_planner", + "cto_state", + "cto_memory", + "session", + "operation", + "ade_project", + "project_config", + "project_secret", + "linear_credentials", + "linear_oauth", + "linear_issue_tracker", + "github", + "feedback", + "usage", + "analytics", + "storage", + "budget", + "update", + "file", + "pty", + "terminal", + "layout", + "tiling_tree", + "graph_state", + "computer_use_artifacts", + "ios_simulator", + "app_control", + "built_in_browser", + "automations", + "review", + "issue", + "orchestration", + "search", + "external-sessions", +] as const; + +export type AdeActionDomain = (typeof ADE_ACTION_DOMAIN_NAMES)[number]; diff --git a/apps/desktop/src/main/services/adeActions/registry.ts b/apps/desktop/src/main/services/adeActions/registry.ts index 6eb971c5c..a15eb7744 100644 --- a/apps/desktop/src/main/services/adeActions/registry.ts +++ b/apps/desktop/src/main/services/adeActions/registry.ts @@ -148,56 +148,14 @@ import { import { createOrchestrationDomainService } from "../orchestration/orchestrationDomain"; import { createAccountActionDomainService } from "../../../../../ade-cli/src/services/account/accountAuthService"; -export const ADE_ACTION_DOMAIN_NAMES = [ - "account", - "attention", - "lane", - "git", - "diff", - "conflicts", - "pr", - "tests", - "chat", - "keybindings", - "ai", - "onboarding", - "automation_planner", - "cto_state", - "cto_memory", - "session", - "operation", - "ade_project", - "project_config", - "project_secret", - "linear_credentials", - "linear_oauth", - "linear_issue_tracker", - "github", - "feedback", - "usage", - "analytics", - "storage", - "budget", - "update", - "file", - "pty", - "terminal", - "layout", - "tiling_tree", - "graph_state", - "computer_use_artifacts", - "ios_simulator", - "app_control", - "built_in_browser", - "automations", - "review", - "issue", - "orchestration", - "search", - "external-sessions", -] as const; - -export type AdeActionDomain = (typeof ADE_ACTION_DOMAIN_NAMES)[number]; +// The names themselves live in `./domains`, which has no imports, so consumers +// that need only the vocabulary (the analytics policy) do not have to load this +// module's whole service graph to get it. Re-exported here because this is +// where every existing caller looks for them. +import type { AdeActionDomain } from "./domains"; + +export { ADE_ACTION_DOMAIN_NAMES } from "./domains"; +export type { AdeActionDomain } from "./domains"; export type AdeActionRole = "cto" | "orchestrator" | "agent" | "external" | "evaluator"; diff --git a/apps/desktop/src/main/services/analytics/productAnalyticsPolicy.ts b/apps/desktop/src/main/services/analytics/productAnalyticsPolicy.ts index c49b5bf74..c57dc30d5 100644 --- a/apps/desktop/src/main/services/analytics/productAnalyticsPolicy.ts +++ b/apps/desktop/src/main/services/analytics/productAnalyticsPolicy.ts @@ -1,10 +1,12 @@ import { isMeaningfulUsageAction } from "../usage/usageStatsStore"; +import { ACCOUNT_MACHINE_REFUSAL_CODES } from "../../../shared/accountMachineRefusal"; import { AUTO_UPDATE_INSTALL_ABORT_REASONS } from "../../../shared/types"; import { RENDERER_GONE_ANALYTICS_REASONS, RENDERER_GONE_UNKNOWN_REASON, } from "../../rendererCrashRecovery"; import type { ToolErrorKind } from "../../../shared/types"; +import { ADE_ACTION_DOMAIN_NAMES } from "../adeActions/domains"; import type { ProductAnalyticsCapture, ProductAnalyticsEventName, @@ -27,6 +29,7 @@ export const INTERNAL_ONLY_EVENTS = new Set([ "ade_brain_recovered", "ade_renderer_recovered", "ade_publish_failing", "ade_relay_suppressed", "ade_account_session_unreadable", "ade_tool_fetched", + "ade_brain_action_failed", ]); export const EVENT_DAILY_BUDGETS: Record = { @@ -56,6 +59,11 @@ export const EVENT_DAILY_BUDGETS: Record = { ade_account_session_unreadable: 10, // Three pinned tools, plus headroom for a retry apiece after a flaky network. ade_tool_fetched: 12, + // Brain action failures are the loudest un-instrumented class there is, so + // the cap has to survive a genuinely broken machine without becoming the + // whole day's budget. The emitter dedupes per domain+code per hour, so 20 is + // roughly "twenty distinct failure shapes a day", not twenty failures. + ade_brain_action_failed: 20, }; export const EVENT_MINUTE_BUDGETS: Record = { @@ -86,6 +94,9 @@ export const EVENT_MINUTE_BUDGETS: Record = { ade_relay_suppressed: 3, ade_account_session_unreadable: 3, ade_tool_fetched: 3, + // A brain that fails every action fails it in a tight loop; the per-minute + // ceiling is what stops the first minute of an outage from spending the day. + ade_brain_action_failed: 3, }; const STRING_PROPERTIES = new Set([ @@ -94,6 +105,7 @@ const STRING_PROPERTIES = new Set([ "entry_point", "release_channel", "summary_kind", "reason", "last_command", "leg", "code", "escalation_reason", "install_source", "trigger", "from_version", "to_version", "user_action", "tool_error_kind", "crash_reason", "count_bucket", + "action_domain", "error_code", "refusal_code", ]); const NUMBER_PROPERTIES = new Set([ "sent_count", "dropped_count", "interaction_count", "session_count", "chat_session_count", @@ -127,6 +139,12 @@ const ANALYTICS_ONLY_ACTIONS = new Set([ // One coarse fact per "Report issue" press: whether the GitHub issue page // opened. Never the surface it was pressed on, and never the report itself. "issue_report", + // Machine membership. `machine_removed` is the user dropping a computer from + // their account; `machine_register_refused` is the account directory refusing + // to take THIS computer back, which is the state a revoked machine sits in + // and the one that produced no telemetry at all last time. + "machine_removed", + "machine_register_refused", ]); const EVENT_PROPERTY_KEYS: Record> = { @@ -140,6 +158,10 @@ const EVENT_PROPERTY_KEYS: Record ade_feature_used: new Set([ "feature", "action", "outcome", "source", "mode", "provider", "model_family", "duration_bucket", "connection_state", "bytes_freed", "files_compressed", "count_bucket", + // Why the account directory refused to register this machine. Its own key + // rather than `code`, which is unallowlisted free-slug shared with the + // publisher-health events; `refusal_code` is a three-value closed set. + "refusal_code", ]), ade_work_session_started: new Set(["feature", "action", "outcome", "source", "mode", "provider"]), ade_work_session_completed: new Set([ @@ -171,9 +193,47 @@ const EVENT_PROPERTY_KEYS: Record // `provider` already enumerates codex/claude/opencode, so the tool identity // reuses it rather than adding a parallel key. ade_tool_fetched: new Set(["provider", "outcome", "duration_bucket", "tool_error_kind"]), + // Exactly two keys, and deliberately NOT folded into `ade_error`. That event + // is keyed by `action` (a usage-ledger action name) and coarsens its failure + // to `error_kind`'s eight buckets, which is precisely the information this + // event exists to keep: which brain domain, and the structured code the brain + // attached. Reusing it would also put a new high-volume emitter inside + // `ade_error`'s 20/day cap and dilute the one signal that already works. + ade_brain_action_failed: new Set(["action_domain", "error_code"]), }; const SLUG_VALUE = /^[a-z0-9][a-z0-9._+-]*$/i; + +/** + * The ADE action domains, taken from the one list that defines them. + * + * They come from `services/adeActions/domains` rather than from `registry.ts` + * precisely so this policy — loaded by the analytics service, the exporters, and + * their tests — does not drag in the registry's whole runtime service graph + * (auth services, the CLI bootstrap) just to learn 45 names. This file used to + * hand-copy them, which meant a domain added to the registry was silently + * unreportable until somebody noticed. + */ +const ADE_ACTION_DOMAIN_ALLOWLIST: ReadonlySet = new Set(ADE_ACTION_DOMAIN_NAMES); + +/** + * Structured failure codes (`codedError`, `Error.code`, and the `code:` prefix + * the runtime RPC boundary encodes) are an open, code-authored vocabulary — the + * point of collecting them is to see the ones nobody predicted — so they cannot + * be pinned to a literal allowlist the way every other string property is. + * + * They are bounded by SHAPE instead, and deliberately more tightly than + * `SLUG_VALUE`: a lower-case identifier, snake_case or kebab-case, because the + * code base authors both (`storage_read_failed`, `cto-identity-invalid`). No + * dots, slashes, colons, plus signs, spaces, or `@` — which is what makes it + * impossible for a path, hostname, URL, email, or any fragment of an error + * sentence to arrive here. The length cap is well under any realistic code and + * far under a message. Note that a hyphen can only ever reach this from a real + * `Error.code`: `parseCodedErrorMessage` reads a `code:` message prefix with an + * even narrower charset, so nothing hyphenated can be scraped out of prose. + */ +const ERROR_CODE_VALUE = /^[a-z][a-z0-9_-]{0,47}$/; + const SAFE_STRING_VALUES: Partial>> = { screen: new Set([ "project", "hub", "lanes", "files", "work", "graph", "prs", "review", "history", "automations", @@ -267,6 +327,12 @@ const SAFE_STRING_VALUES: Partial>> = { "manifest", "unsupported-target", "network", "integrity", "disk-space", "extract", "lock-timeout", "filesystem", ]), + action_domain: ADE_ACTION_DOMAIN_ALLOWLIST, + // The account directory's own refusals, read off the one list that defines + // them, plus the honest bucket for a refusal a newer directory names and this + // build has never heard of. The brain's user-facing sentence is NOT a + // fallback: it is free text. + refusal_code: new Set([...ACCOUNT_MACHINE_REFUSAL_CODES, "other"]), }; export function safeProductAnalyticsString(value: ProductAnalyticsPropertyValue): string | null { @@ -296,6 +362,11 @@ function safeStringProperty(key: string, value: ProductAnalyticsPropertyValue): return raw === "open" || isMeaningfulUsageAction(raw) || ANALYTICS_ONLY_ACTIONS.has(raw) ? raw : null; } if (key === "error_kind") return coarseErrorKind(value); + if (key === "error_code") { + if (typeof value !== "string" || value.length > 256) return null; + const normalized = value.trim().toLowerCase(); + return ERROR_CODE_VALUE.test(normalized) ? normalized : null; + } const safe = safeProductAnalyticsString(value); if (!safe) return null; const allowlist = SAFE_STRING_VALUES[key]; diff --git a/apps/desktop/src/main/services/analytics/productAnalyticsService.test.ts b/apps/desktop/src/main/services/analytics/productAnalyticsService.test.ts index 01d276776..6dca43c13 100644 --- a/apps/desktop/src/main/services/analytics/productAnalyticsService.test.ts +++ b/apps/desktop/src/main/services/analytics/productAnalyticsService.test.ts @@ -560,6 +560,201 @@ describe("productAnalyticsService", () => { fs.rmSync(harness.root, { recursive: true, force: true }); }); + it("reports a failed brain action as a domain and a code, never as a message", () => { + const harness = makeHarness(); + + // What the local-runtime callAction error path has in scope: the domain the + // renderer asked for, the structured code the brain attached, and the raw + // message — which is the one thing that must not cross the boundary. + expect(harness.service.captureInternal({ + event: "ade_brain_action_failed", + surface: "desktop", + properties: { + action_domain: "lane", + error_code: "storage_read_failed", + message: "storage_read_failed: could not read /Users/alice/secret-project/.ade/ade.db", + error_message: "EACCES: permission denied, open '/Users/alice/secret-project/.ade/ade.db'", + root_path: "/Users/alice/secret-project", + action: "lanes.create", + }, + dedupeKey: "brain-action-failed:lane:storage_read_failed", + minimumIntervalMs: 60 * 60 * 1_000, + })).toEqual({ accepted: true, reason: "accepted" }); + + expect(harness.messages).toHaveLength(1); + expect(harness.messages[0]?.properties).toMatchObject({ + action_domain: "lane", + error_code: "storage_read_failed", + }); + for (const forbidden of ["message", "error_message", "root_path", "action"]) { + expect(harness.messages[0]?.properties).not.toHaveProperty(forbidden); + } + expect(JSON.stringify(harness.messages)).not.toContain("secret-project"); + expect(JSON.stringify(harness.messages)).not.toContain("alice"); + + // The loop that produced no telemetry last time: the same failure, again and + // again. One accepted event per domain+code per hour, not thousands. + for (let attempt = 0; attempt < 50; attempt += 1) { + expect(harness.service.captureInternal({ + event: "ade_brain_action_failed", + surface: "desktop", + properties: { action_domain: "lane", error_code: "storage_read_failed" }, + dedupeKey: "brain-action-failed:lane:storage_read_failed", + minimumIntervalMs: 60 * 60 * 1_000, + })).toEqual({ accepted: false, reason: "duplicate" }); + } + expect(harness.messages).toHaveLength(1); + + // A different code inside the same hour is a different fact and still lands. + expect(harness.service.captureInternal({ + event: "ade_brain_action_failed", + surface: "desktop", + properties: { action_domain: "lane", error_code: "ipc_timeout" }, + dedupeKey: "brain-action-failed:lane:ipc_timeout", + minimumIntervalMs: 60 * 60 * 1_000, + })).toEqual({ accepted: true, reason: "accepted" }); + expect(harness.messages[1]?.properties).toMatchObject({ + action_domain: "lane", + error_code: "ipc_timeout", + }); + fs.rmSync(harness.root, { recursive: true, force: true }); + }); + + it("drops a brain-action domain or code that is not a closed value", () => { + // Each of these is an accepted event with a stripped property, and the + // event's per-minute ceiling is 3 — so the clock advances past the window + // between captures rather than the test asserting on a rate-limited drop. + let clock = Date.parse("2026-08-18T12:00:00.000Z"); + const harness = makeHarness({ now: () => clock }); + const capture = (properties: Record, dedupeKey: string) => { + clock += 61_000; + expect(harness.service.captureInternal({ + event: "ade_brain_action_failed", + surface: "desktop", + properties, + dedupeKey, + })).toEqual({ accepted: true, reason: "accepted" }); + return harness.messages.at(-1)?.properties as Record; + }; + + // A domain outside the registry's closed list is dropped, not widened. + expect(capture({ action_domain: "/Users/alice/secret", error_code: "not_found" }, "d1")) + .not.toHaveProperty("action_domain"); + + // The one registry domain that is not a plain identifier still survives, so + // the allowlist is a real mirror of the registry and not a rounded-off copy. + expect(capture({ action_domain: "external-sessions", error_code: "not_found" }, "d2")) + .toMatchObject({ action_domain: "external-sessions" }); + + // An error "code" that is really a sentence, a path, a URL, or an address + // fails the identifier shape and never reaches PostHog. Each of these is + // something `parseCodedErrorMessage` could conceivably hand back if the + // caller stopped extracting the code and passed the message instead. + // A hyphenated code is real (`cto-identity-invalid`) and must survive. + expect(capture({ action_domain: "chat", error_code: "cto-identity-invalid" }, "d3")) + .toMatchObject({ error_code: "cto-identity-invalid" }); + + for (const [index, leaked] of [ + "could not open the project data store", + "/Users/alice/secret-project/.ade/ade.db", + "https://directory.example.test/account/machines", + "ada@example.com", + "alice-macbook.local", + ].entries()) { + expect(capture({ action_domain: "file", error_code: leaked }, `leak-${index}`)) + .not.toHaveProperty("error_code"); + } + expect(JSON.stringify(harness.messages)).not.toContain("alice"); + expect(JSON.stringify(harness.messages)).not.toContain("example.com"); + fs.rmSync(harness.root, { recursive: true, force: true }); + }); + + it("reports machine membership changes as coarse connections facts", () => { + const harness = makeHarness(); + + // Removing a computer from the account. The roster row this came from has + // the machine's key and display name in it; neither is a product fact. + expect(harness.service.capture({ + event: "ade_feature_used", + surface: "desktop", + properties: { + feature: "connections", + action: "machine_removed", + outcome: "completed", + machine_key: "not-a-real-machine-key", + machine_name: "Example MacBook", + }, + projectId: null, + dedupeKey: "machine_removed:completed", + minimumIntervalMs: 60 * 60 * 1_000, + })).toEqual({ accepted: true, reason: "accepted" }); + expect(harness.messages[0]?.properties).toMatchObject({ + feature: "connections", + action: "machine_removed", + outcome: "completed", + }); + expect(JSON.stringify(harness.messages[0])).not.toContain("MacBook"); + + // The directory refusing to take this computer back. `refusal_code` is the + // whole point of the event; the brain's sentence explaining it is not. + expect(harness.service.capture({ + event: "ade_feature_used", + surface: "desktop", + properties: { + feature: "connections", + action: "machine_register_refused", + outcome: "failed", + refusal_code: "machine_revoked", + reason: "This machine was removed from your ADE account.", + }, + projectId: null, + dedupeKey: "machine_register_refused:machine_revoked", + minimumIntervalMs: 60 * 60 * 1_000, + })).toEqual({ accepted: true, reason: "accepted" }); + expect(harness.messages[1]?.properties).toMatchObject({ + feature: "connections", + action: "machine_register_refused", + outcome: "failed", + refusal_code: "machine_revoked", + }); + expect(harness.messages[1]?.properties).not.toHaveProperty("reason"); + + // A poll loop against a machine that stays refused cannot spend the budget. + expect(harness.service.capture({ + event: "ade_feature_used", + surface: "desktop", + properties: { + feature: "connections", + action: "machine_register_refused", + outcome: "failed", + refusal_code: "machine_revoked", + }, + projectId: null, + dedupeKey: "machine_register_refused:machine_revoked", + minimumIntervalMs: 60 * 60 * 1_000, + })).toEqual({ accepted: false, reason: "duplicate" }); + + // A refusal code outside the closed set is dropped rather than widening it, + // so a directory that starts naming refusals differently cannot turn this + // key into free text. + expect(harness.service.capture({ + event: "ade_feature_used", + surface: "desktop", + properties: { + feature: "connections", + action: "machine_register_refused", + outcome: "failed", + refusal_code: "http 403 from https://directory.example.test", + }, + projectId: null, + dedupeKey: "machine_register_refused:unknown", + minimumIntervalMs: 60 * 60 * 1_000, + })).toEqual({ accepted: true, reason: "accepted" }); + expect(harness.messages[2]?.properties).not.toHaveProperty("refusal_code"); + expect(JSON.stringify(harness.messages)).not.toContain("directory.example.test"); + fs.rmSync(harness.root, { recursive: true, force: true }); + }); + it("keeps a renderer-crash report to the reason and the outcome", () => { const harness = makeHarness(); diff --git a/apps/desktop/src/main/services/analytics/reliabilityTelemetry.test.ts b/apps/desktop/src/main/services/analytics/reliabilityTelemetry.test.ts new file mode 100644 index 000000000..1dd375c41 --- /dev/null +++ b/apps/desktop/src/main/services/analytics/reliabilityTelemetry.test.ts @@ -0,0 +1,165 @@ +import { describe, expect, it, vi } from "vitest"; +import { createSyncAccountDirectoryHealth } from "../../../shared/types"; +import type { SyncAccountDirectoryHealth, SyncRoleSnapshot } from "../../../shared/types"; +import { sanitizeProductAnalyticsProperties } from "./productAnalyticsPolicy"; +import { + brainActionErrorCode, + createMachineRegisterRefusalObserver, + machineRegisterRefusalCode, +} from "./reliabilityTelemetry"; + +/** Only `routeHealth.accountDirectory` is read, so nothing else is built. */ +function snapshotWith(health: SyncAccountDirectoryHealth): SyncRoleSnapshot { + return { routeHealth: { accountDirectory: health } } as unknown as SyncRoleSnapshot; +} + +const refused = ( + status: number | null, + reason: string | null, +): SyncRoleSnapshot => snapshotWith(createSyncAccountDirectoryHealth("http_error", null, { + lastHttpStatus: status, + lastHttpReason: reason, +})); + +describe("brainActionErrorCode", () => { + it("takes the code the brain attached and leaves the message behind", () => { + const error = Object.assign( + new Error("could not read /Users/alice/secret-project/.ade/ade.db"), + { code: "storage_read_failed" }, + ); + expect(brainActionErrorCode(error, false)).toBe("storage_read_failed"); + }); + + it("reads the code back out of the wrapped RPC message", () => { + // What actually arrives at the desktop: the runtime RPC wrapper, then + // Electron's own, around a message the brain prefixed with its code. + const wrapped = new Error( + "Error invoking remote method 'ade.localRuntime.callAction': " + + "Error: Remote ADE service method actions/call failed (code -32000): " + + "storage_read_failed: ADE could not open the project data store.", + ); + expect(brainActionErrorCode(wrapped, false)).toBe("storage_read_failed"); + }); + + it("names a wrapper timeout instead of borrowing an unrelated code", () => { + expect(brainActionErrorCode(new Error("timed out after 30000ms"), true)).toBe("ipc_timeout"); + }); + + it("says the failure was uncoded rather than dropping it", () => { + expect(brainActionErrorCode(new Error("Session 'chat-stale' was not found."), false)) + .toBe("unknown"); + expect(brainActionErrorCode(null, false)).toBe("unknown"); + }); + + it("keeps a platform errno as a code instead of costing the event", () => { + // An unmapped errno — EDEADLK is the one a cloud-evicted file answers with + // on macOS — is exactly the failure nobody predicted, which is what + // `error_code` is collected for. The policy lower-cases before it shape- + // checks, so the uppercase form is normalized rather than refused; and a + // property it did refuse would be stripped from an event that still lands, + // never drop the event. + const evicted = Object.assign( + new Error("Unknown system error -11: Unknown system error -11, read"), + { code: "EDEADLK" }, + ); + expect(brainActionErrorCode(evicted, false)).toBe("EDEADLK"); + expect(sanitizeProductAnalyticsProperties("ade_brain_action_failed", { + action_domain: "lane", + error_code: brainActionErrorCode(evicted, false), + })).toEqual({ action_domain: "lane", error_code: "edeadlk" }); + }); + + it("never returns anything the policy would accept as a message", () => { + // The load-bearing privacy property: whatever this returns, the policy only + // lets a lower-case identifier through, so a message that slipped past the + // parser still cannot reach PostHog. + const leaky = new Error("Local runtime project is not available for this window."); + const properties = sanitizeProductAnalyticsProperties("ade_brain_action_failed", { + error_code: brainActionErrorCode(leaky, false), + }); + expect(properties).toEqual({ error_code: "unknown" }); + expect(sanitizeProductAnalyticsProperties("ade_brain_action_failed", { + error_code: "could not open /Users/alice/secret.db", + })).toEqual({}); + }); +}); + +describe("machineRegisterRefusalCode", () => { + it("recognises both refusals the directory names", () => { + expect(machineRegisterRefusalCode(refused(403, "machine_revoked"))).toBe("machine_revoked"); + expect(machineRegisterRefusalCode(refused(403, "pairing_authentication_required"))) + .toBe("pairing_authentication_required"); + }); + + it("is not fooled by a 401, which is an auth failure and not a refusal", () => { + // The directory answers a register refusal with 403. A 401 means the token + // this machine presented was not accepted — a different failure with a + // different repair, which `ade_publish_failing` already carries. + expect(machineRegisterRefusalCode(refused(401, "pairing_authentication_required"))) + .toBeNull(); + expect(machineRegisterRefusalCode(refused(401, "token expired"))).toBeNull(); + }); + + it("reports an unnamed refusal as `other` instead of the server's prose", () => { + const code = machineRegisterRefusalCode( + refused(403, "your account is over its machine limit, contact support@example.test"), + ); + expect(code).toBe("other"); + }); + + it("is silent for everything that is not a refusal", () => { + expect(machineRegisterRefusalCode(refused(500, "machine_revoked"))).toBeNull(); + expect(machineRegisterRefusalCode(refused(null, null))).toBeNull(); + expect(machineRegisterRefusalCode( + snapshotWith(createSyncAccountDirectoryHealth("published", null)), + )).toBeNull(); + expect(machineRegisterRefusalCode( + snapshotWith(createSyncAccountDirectoryHealth("token_timeout", null, { lastHttpStatus: 403 })), + )).toBeNull(); + expect(machineRegisterRefusalCode(null)).toBeNull(); + expect(machineRegisterRefusalCode({} as SyncRoleSnapshot)).toBeNull(); + }); +}); + +describe("createMachineRegisterRefusalObserver", () => { + it("reports the edge into a refusal, not every poll tick", () => { + const onRefused = vi.fn(); + const observe = createMachineRegisterRefusalObserver(onRefused); + const revoked = refused(403, "machine_revoked"); + + for (let tick = 0; tick < 200; tick += 1) observe(revoked); + + expect(onRefused).toHaveBeenCalledTimes(1); + expect(onRefused).toHaveBeenCalledWith("machine_revoked"); + }); + + it("reports a refusal that changes into a different one", () => { + const onRefused = vi.fn(); + const observe = createMachineRegisterRefusalObserver(onRefused); + + observe(refused(403, "pairing_authentication_required")); + observe(refused(403, "machine_revoked")); + + expect(onRefused.mock.calls.map(([code]) => code)) + .toEqual(["pairing_authentication_required", "machine_revoked"]); + }); + + it("re-arms after a recovery so a second episode still reports", () => { + const onRefused = vi.fn(); + const observe = createMachineRegisterRefusalObserver(onRefused); + const revoked = refused(403, "machine_revoked"); + + observe(revoked); + observe(snapshotWith(createSyncAccountDirectoryHealth("published", null))); + observe(revoked); + + expect(onRefused).toHaveBeenCalledTimes(2); + }); + + it("never lets telemetry break the status read", () => { + const observe = createMachineRegisterRefusalObserver(() => { + throw new Error("analytics state file is unwritable"); + }); + expect(() => observe(refused(403, "machine_revoked"))).not.toThrow(); + }); +}); diff --git a/apps/desktop/src/main/services/analytics/reliabilityTelemetry.ts b/apps/desktop/src/main/services/analytics/reliabilityTelemetry.ts new file mode 100644 index 000000000..211e77bf5 --- /dev/null +++ b/apps/desktop/src/main/services/analytics/reliabilityTelemetry.ts @@ -0,0 +1,92 @@ +// Pure derivations for the two failure classes that produced NO telemetry when +// a machine was revoked and its brain stopped working: brain action failures, +// and the account directory refusing to register this computer. +// +// They live here, apart from the IPC layer that emits them, because each one is +// a small privacy decision — what may be read off an error or a status snapshot, +// and what must be left behind — and those decisions are worth testing on their +// own. Nothing in this file captures; the caller owns the event vocabulary. + +import { readAccountRefusalCode } from "../../../shared/accountMachineRefusal"; +import { parseCodedErrorMessage } from "../../../shared/codedError"; +import type { SyncRoleSnapshot } from "../../../shared/types"; + +/** + * The structured code of a brain action failure, and NEVER its message. + * + * `parseCodedErrorMessage` prefers a real `Error.code` and otherwise reads the + * `code:` prefix the RPC boundary encodes, so this is always a token the code + * base authored — but the policy still revalidates the shape before it can leave + * the machine, and a "code" that is really a sentence is dropped there. + * + * `unknown` is reported rather than dropped: how much of the failure surface is + * still uncoded is itself part of the answer to "why did the last incident + * produce nothing", and omitting it would hide that. A wrapper timeout gets its + * own name because the brain never answered at all, so any code in scope would + * belong to the timeout, not to the failure. + */ +export function brainActionErrorCode(error: unknown, didTimeout: boolean): string { + if (didTimeout) return "ipc_timeout"; + const code = parseCodedErrorMessage(error).code; + return code && code.trim() ? code.trim() : "unknown"; +} + +/** + * Why the account directory refused to register THIS computer, as one of three + * closed values — never the brain's user-facing sentence. + * + * The decode itself belongs to `readAccountRefusalCode`, which the brain's + * auto-recovery loop reads too: what counts as a refusal must not differ + * between the thing that repairs one and the thing that reports it. This + * status snapshot is simply the only place the desktop can see a refusal — it + * never talks to the directory itself. + * + * Everything that is not a refusal — a timeout, a 5xx, a transport failure, and + * a 401, which is an authentication problem rather than the directory turning a + * valid caller away — is left to `ade_publish_failing`, which the brain emits. + */ +export function machineRegisterRefusalCode( + snapshot: SyncRoleSnapshot | null | undefined, +): string | null { + const health = snapshot?.routeHealth?.accountDirectory; + // A snapshot is a state, not an event: only `http_error` means the status + // fields describe the attempt that is currently failing. Reporting a refusal + // off any other state would date-stamp an old rejection as a new incident. + if (!health || health.state !== "http_error") return null; + return readAccountRefusalCode(health); +} + +/** + * Turns the refusal STATE into refusal EVENTS. + * + * Connections and the app shell both poll the local sync status on a timer, and + * a revoked machine stays revoked — so the state is read hundreds of times a day + * and is a product fact exactly twice: when it starts, and when it changes into + * a different refusal. Only the edge calls `onRefused`. Recovery clears the + * latch, so a machine that is refused, repaired, and refused again reports both + * episodes. The caller is still expected to dedupe, which is what bounds the + * case this latch cannot see: a process that restarts inside a refusal. + */ +export function createMachineRegisterRefusalObserver( + onRefused: (code: string) => void, +): (snapshot: SyncRoleSnapshot | null | undefined) => void { + let lastCode: string | null = null; + return (snapshot) => { + let code: string | null = null; + try { + code = machineRegisterRefusalCode(snapshot); + } catch { + // A snapshot from an older or partially-populated brain must never fail + // the status read the whole Connections surface depends on. + return; + } + if (code === lastCode) return; + lastCode = code; + if (!code) return; + try { + onRefused(code); + } catch { + // Analytics is never worth failing a status read over. + } + }; +} diff --git a/apps/desktop/src/main/services/github/credentialReadState.test.ts b/apps/desktop/src/main/services/github/credentialReadState.test.ts new file mode 100644 index 000000000..dea6fe575 --- /dev/null +++ b/apps/desktop/src/main/services/github/credentialReadState.test.ts @@ -0,0 +1,100 @@ +import { describe, expect, it } from "vitest"; +import { + readCredentialWithState, + readCredentialWithStateAsync, +} from "./credentialReadState"; + +/** + * Models the real store: one `lastReadState` field that every read overwrites, + * shared between the GitHub credential path and App user authentication. + */ +function createSharedStore(initial: Record = {}) { + const store = { + values: { ...initial }, + lastReadState: "available", + getSync(key: string): string | null { + store.lastReadState = key in store.values ? "available" : "missing"; + return store.values[key] ?? null; + }, + async get(key: string): Promise { + await Promise.resolve(); + store.lastReadState = key in store.values ? "available" : "missing"; + return store.values[key] ?? null; + }, + async getWithReadState(key: string): Promise<{ value: string | null; state: string }> { + await Promise.resolve(); + // The real store assigns and returns in the same synchronous step; the + // captured state is what makes that pairing survive the await below. + const state = key in store.values ? "available" : "missing"; + store.lastReadState = state; + const value = store.values[key] ?? null; + await Promise.resolve(); + return { value, state }; + }, + getLastReadState(): string { + return store.lastReadState; + }, + }; + return store; +} + +describe("credentialReadState", () => { + it("pairs a sync read with the state that read produced", () => { + const store = createSharedStore({ "github.token.v1": " ghp_token " }); + + expect(readCredentialWithState(store, "github.token.v1")).toEqual({ + value: "ghp_token", + unreadable: false, + }); + }); + + it("reports an unreadable store rather than an absent credential", async () => { + const store = createSharedStore(); + store.getWithReadState = async () => ({ value: null, state: "unreadable" }); + + await expect(readCredentialWithStateAsync(store, "github.token.v1")).resolves.toEqual({ + value: null, + unreadable: true, + }); + }); + + // An async read only gets to ask about the state once it has resolved, and by + // then App user authentication may have read the same store and moved the + // answer. Reporting that other read's verdict is how a working credential got + // called unreadable — and an unreadable one got called merely absent. + it("does not report a concurrent read's state as its own", async () => { + const store = createSharedStore({ "github.token.v1": "ghp_token" }); + let unreadableRead: Promise | null = null; + const original = store.getWithReadState; + store.getWithReadState = async (key: string) => { + const result = await original(key); + // A second reader lands while this read is still settling. + unreadableRead = Promise.resolve().then(() => { + store.lastReadState = "unreadable"; + }); + await unreadableRead; + return result; + }; + + await expect(readCredentialWithStateAsync(store, "github.token.v1")).resolves.toEqual({ + value: "ghp_token", + unreadable: false, + }); + expect(store.getLastReadState()).toBe("unreadable"); + }); + + it("treats a throwing store as unreadable", async () => { + const store = createSharedStore(); + store.getWithReadState = async () => { + throw new Error("decrypt failed"); + }; + const seen: unknown[] = []; + + await expect( + readCredentialWithStateAsync(store, "github.token.v1", { + onError: (error) => seen.push(error), + }), + ).resolves.toEqual({ value: null, unreadable: true }); + expect(seen).toHaveLength(1); + }); +}); diff --git a/apps/desktop/src/main/services/github/credentialReadState.ts b/apps/desktop/src/main/services/github/credentialReadState.ts new file mode 100644 index 000000000..59cfeeff5 --- /dev/null +++ b/apps/desktop/src/main/services/github/credentialReadState.ts @@ -0,0 +1,96 @@ +/** + * Read one credential AND learn whether the store could be read at all, in a + * single call. + * + * The invariant this exists to keep: an undecryptable credential store returns + * an EMPTY view rather than throwing, so "no token" and "a token ADE cannot + * read" are the same answer — and only `getLastReadState()` tells them apart. + * That method describes the store's MOST RECENT read, so it must be consulted + * immediately after the `getSync`/`get` it is being asked about; asking a moment + * later answers about somebody else's read. Pairing the two here is what makes + * the ordering impossible to get wrong, instead of a comment each caller has to + * remember. Getting it wrong is what told users with working credentials that + * they were "not connected", and invited them to reconnect over them. + * + * "Immediately after" is free on the sync path and impossible on the async one: + * resolving `get()` costs at least one tick, and the same store is read by App + * user authentication, so another read can land inside that gap. Async stores + * therefore hand back the state alongside the value (`getWithReadState`) rather + * than being asked for it afterwards. + * + * A store that throws is unreadable too — the Electron safeStorage store + * reports decrypt failures that way rather than by returning `{}`. + */ + +export type CredentialReadStateResult = { + value: string | null; + unreadable: boolean; +}; + +type SyncCredentialReader = { + getSync(key: string): string | null | undefined; + getLastReadState?(): string; +}; + +type AsyncCredentialReader = { + get(key: string): Promise; + getLastReadState?(): string; + /** + * Preferred over `get` + `getLastReadState()`: it returns the state produced + * by THIS read. `getLastReadState()` describes the store's most recent read, + * and an async caller only gets to ask once its own read has resolved — by + * which point a read from elsewhere (the same store backs App user auth) can + * have landed and moved the answer. The sync path has no such gap, so only + * the async one needs this. + */ + getWithReadState?(key: string): Promise<{ + value: string | null | undefined; + state: string; + }>; +}; + +function toResult( + value: string | null | undefined, + store: { getLastReadState?(): string }, +): CredentialReadStateResult { + const trimmed = value?.trim() ?? ""; + return { + value: trimmed || null, + unreadable: store.getLastReadState?.() === "unreadable", + }; +} + +/** `onError` lets a caller log the throw it would otherwise never see. */ +export function readCredentialWithState( + store: SyncCredentialReader, + key: string, + options: { onError?: (error: unknown) => void } = {}, +): CredentialReadStateResult { + try { + return toResult(store.getSync(key), store); + } catch (error) { + options.onError?.(error); + return { value: null, unreadable: true }; + } +} + +export async function readCredentialWithStateAsync( + store: AsyncCredentialReader, + key: string, + options: { onError?: (error: unknown) => void } = {}, +): Promise { + try { + if (store.getWithReadState) { + const read = await store.getWithReadState(key); + const trimmed = read.value?.trim() ?? ""; + return { value: trimmed || null, unreadable: read.state === "unreadable" }; + } + // A store without the paired accessor answers about its most recent read, + // which is the best available answer and stays correct while it is the only + // reader. + return toResult(await store.get(key), store); + } catch (error) { + options.onError?.(error); + return { value: null, unreadable: true }; + } +} diff --git a/apps/desktop/src/main/services/github/githubService.test.ts b/apps/desktop/src/main/services/github/githubService.test.ts index 5be6269c1..e2c3019e5 100644 --- a/apps/desktop/src/main/services/github/githubService.test.ts +++ b/apps/desktop/src/main/services/github/githubService.test.ts @@ -126,6 +126,26 @@ class MemoryCredentialStore { } } +/** + * A store whose ciphertext will not decrypt. It reports the truth the way the + * real file store does — an EMPTY view plus `getLastReadState() === "unreadable"` + * — rather than by throwing, which is precisely how "corrupted" used to reach + * the UI wearing "never connected". + */ +class UnreadableCredentialStore extends MemoryCredentialStore { + override getSync(_key: string): string | null { + return null; + } + + getLastReadState(): "available" | "missing" | "unreadable" { + return "unreadable"; + } + + getLastReadFailureReason(): "decrypt_failure" { + return "decrypt_failure"; + } +} + function makeService(options: { credentialStore?: MemoryCredentialStore; ghAuthTokenProvider?: () => @@ -1362,6 +1382,55 @@ describe("githubService.getStatus", () => { }); } + // Regression: an undecryptable credential store used to emit a status that was + // byte-identical to a fresh install (tokenStored:false, connected:false, + // authFailure:null), so the UI told users GitHub had never been connected and + // invited them to reconnect over credentials that were still on disk. + it("reports an undecryptable credential store instead of a fresh-install status", async () => { + stubOriginRemote(); + const status = await makeService({ + credentialStore: new UnreadableCredentialStore(), + }).getStatus(); + + expect(status.credentialStoreUnreadable).toBe(true); + expect(status.tokenStored).toBe(false); + expect(status.connected).toBe(false); + // The distinct state must not be reachable by inspecting the token fields: + // that is exactly the conflation this field exists to break. + expect(status.tokenDecryptionFailed).toBe(true); + expect(mockFetch).not.toHaveBeenCalled(); + }); + + it("does not claim an unreadable store when the store is merely empty", async () => { + stubOriginRemote(); + const status = await makeService({ + credentialStore: new MemoryCredentialStore(), + }).getStatus(); + + expect(status.credentialStoreUnreadable).toBe(false); + expect(status.tokenStored).toBe(false); + expect(status.connected).toBe(false); + }); + + it("keeps the unreadable flag off a status a working gh credential produced", async () => { + stubOriginRemote(); + delete process.env.ADE_DISABLE_GH_AUTH_FALLBACK; + mockFetch.mockResolvedValueOnce( + jsonResponse(200, { login: "alice" }, { "x-oauth-scopes": "repo, workflow" }), + ); + const status = await makeService({ + credentialStore: new MemoryCredentialStore(), + ghAuthTokenProvider: () => ({ + token: "gho_cli_token", + ghCliPath: "/opt/homebrew/bin/gh", + ghAuthError: null, + }), + }).getStatus(); + + expect(status.connected).toBe(true); + expect(status.credentialStoreUnreadable).toBe(false); + }); + it("keeps repo-capable classic tokens connected while withholding write access", async () => { stubOriginRemote(); process.env.GITHUB_TOKEN = "ghp_classic"; diff --git a/apps/desktop/src/main/services/github/githubService.ts b/apps/desktop/src/main/services/github/githubService.ts index 4f1ce3849..564bccbfc 100644 --- a/apps/desktop/src/main/services/github/githubService.ts +++ b/apps/desktop/src/main/services/github/githubService.ts @@ -43,6 +43,7 @@ import { mergePathEntries, resolveExecutableFromKnownLocations } from "../ai/cli import { fetchGitHubAppInstallationStatus, type GitHubRelaySecretReader } from "./githubRelayConfig"; import { createGitHubAppUserAuthService } from "./githubAppUserAuthService"; import { GITHUB_REST_API_VERSION } from "./githubApiVersion"; +import { readCredentialWithState } from "./credentialReadState"; import { requestGithubRawWithCredentialFallback, type GithubRawRequestArgs, @@ -197,6 +198,13 @@ type GitHubCredentialInventory = { patTokenStored: boolean; ghCliPath: string | null; ghAuthError: string | null; + /** + * Whether the credential file backing `github.token.v1` / + * `github.appUserToken.v1` was readable when this inventory was built. Carried + * on the inventory (rather than read again at status time) so the answer is + * the one that belongs to the read that produced these candidates. + */ + credentialStoreUnreadable: boolean; }; class GithubCredentialAttemptError extends Error { @@ -572,6 +580,9 @@ export function createGithubService({ let tokenDecryptionFailed = false; let machineTokenReadFailed = false; + // Sticky across reads so the status can report it, and so the log line below + // fires on the transition instead of on every cached status refresh. + let credentialStoreUnreadable = false; const ghAuthProvider = ghAuthTokenProvider ?? readGitHubCliAuthToken; const sharedGhAuth = processGithubAuthState(ghAuthProvider); let statusInFlight: Promise | null = null; @@ -608,19 +619,42 @@ export function createGithubService({ invalidateStatusCache(); }; + /** + * Records whether the credential file was readable on the read that just ran, + * warning once per transition into unreadable. + */ + const noteCredentialStoreReadState = (unreadable: boolean): boolean => { + if (unreadable !== credentialStoreUnreadable) { + credentialStoreUnreadable = unreadable; + if (unreadable) { + logger.warn("github.credential_store_unreadable", { + reason: credentialStore?.getLastReadFailureReason?.() ?? null, + }); + } + } + return unreadable; + }; + const readMachineToken = (): string | null => { if (!credentialStore) return null; - try { - const token = credentialStore.getSync(MACHINE_TOKEN_KEY)?.trim() ?? ""; + // The read and the readability verdict come back together: an undecryptable + // store returns an EMPTY view instead of throwing, so an absent token means + // either "never connected" or "connected, but ADE cannot read it", and only + // the store's own read state tells the two apart. + const read = readCredentialWithState(credentialStore, MACHINE_TOKEN_KEY, { + onError: (error) => { + logger.warn("github.machine_token_read_failed", { + error: error instanceof Error ? error.message : String(error), + }); + }, + }); + const unreadable = noteCredentialStoreReadState(read.unreadable); + if (read.value) { machineTokenReadFailed = false; - return token.length > 0 ? token : null; - } catch (error) { - machineTokenReadFailed = true; - logger.warn("github.machine_token_read_failed", { - error: error instanceof Error ? error.message : String(error), - }); - return null; + return read.value; } + machineTokenReadFailed = unreadable; + return null; }; const persistMachineToken = (token: string | null): void => { @@ -633,6 +667,9 @@ export function createGithubService({ credentialStore.deleteSync(MACHINE_TOKEN_KEY); } machineTokenReadFailed = false; + // A write that landed re-sealed the store with a key this process holds, + // so whatever made the previous read unreadable no longer applies. + credentialStoreUnreadable = false; } catch (error) { machineTokenReadFailed = true; logger.warn("github.machine_token_write_failed", { @@ -814,6 +851,11 @@ export function createGithubService({ const buildCredentialInventory = async (): Promise => { const patLookup = readPatAuthToken(); const patTokenStored = Boolean(patLookup); + // Snapshotted here, next to `patTokenStored`, because the read that just ran + // is the one this verdict belongs to. `credentialStoreUnreadable` is shared + // mutable state: any other caller reading the same store during the `await` + // below would otherwise hand this inventory somebody else's outcome. + const storeUnreadableForThisRead = credentialStoreUnreadable; const environment = readEnvironmentAuthToken(); const appStatus = appUserAuth.getAuthStatus(); const [appResult, gh] = await Promise.all([ @@ -883,6 +925,11 @@ export function createGithubService({ patTokenStored, ghCliPath: gh.ghCliPath, ghAuthError: gh.ghAuthError, + // `readPatAuthToken()` above is the read that touched the store, so this + // is that read's outcome. An unreadable store also empties the App user + // token (same file), which is why it is reported once for the whole + // inventory rather than per source. + credentialStoreUnreadable: storeUnreadableForThisRead, }; }; @@ -1650,6 +1697,7 @@ export function createGithubService({ tokenStored: inventory.appTokenStored, patTokenStored: inventory.patTokenStored, tokenDecryptionFailed, + credentialStoreUnreadable: inventory.credentialStoreUnreadable, storageScope: "app", authSource: failure?.source ?? "none", writeAuthSource: "none", @@ -1717,6 +1765,7 @@ export function createGithubService({ repo, hasOrigin, patTokenStored: inventory.patTokenStored, + credentialStoreUnreadable: inventory.credentialStoreUnreadable, ghCliPath: inventory.ghCliPath ?? cachedStatus.ghCliPath, ghAuthError: inventory.ghAuthError, writeAuthSource: activeWriteSource ?? "none", @@ -1807,6 +1856,7 @@ export function createGithubService({ tokenStored: true, patTokenStored: inventory.patTokenStored, tokenDecryptionFailed: false, + credentialStoreUnreadable: inventory.credentialStoreUnreadable, storageScope: "app", authSource: candidate.source, writeAuthSource: activeWriteSource ?? "none", @@ -1861,6 +1911,7 @@ export function createGithubService({ tokenStored: true, patTokenStored: inventory.patTokenStored, tokenDecryptionFailed: false, + credentialStoreUnreadable: inventory.credentialStoreUnreadable, storageScope: "app", authSource: primaryCandidate.source, writeAuthSource: "none", diff --git a/apps/desktop/src/main/services/ipc/ipcTimeouts.test.ts b/apps/desktop/src/main/services/ipc/ipcTimeouts.test.ts index 22efd19f4..75ad639d5 100644 --- a/apps/desktop/src/main/services/ipc/ipcTimeouts.test.ts +++ b/apps/desktop/src/main/services/ipc/ipcTimeouts.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; import { IPC } from "../../../shared/ipc"; -import { ipcInvokeTimeoutMs } from "./ipcTimeouts"; +import { ipcInvokeTimeoutMs, readRuntimeActionRequest } from "./ipcTimeouts"; import { LOCAL_RUNTIME_ACTION_REGISTRY_TIMEOUT_MS, LOCAL_RUNTIME_ACTION_TIMEOUT_MS, @@ -277,3 +277,39 @@ describe("ipcInvokeTimeoutMs", () => { }])).toBe(405_000); }); }); + +describe("readRuntimeActionRequest", () => { + it("reads the domain and action the renderer already sent", () => { + expect(readRuntimeActionRequest([{ + rootPath: "/Users/alice/secret-project", + request: { domain: " lane ", action: " create ", args: {} }, + }])).toEqual({ domain: "lane", action: "create" }); + }); + + it("keeps a domain-only request, which is enough to attribute a failure", () => { + expect(readRuntimeActionRequest([{ request: { domain: "lane" } }])).toEqual({ domain: "lane" }); + expect(readRuntimeActionRequest([{ request: { domain: "lane", action: " " } }])) + .toEqual({ domain: "lane" }); + expect(readRuntimeActionRequest([{ request: { domain: "lane", action: 7 } }])) + .toEqual({ domain: "lane" }); + }); + + it("returns null for anything malformed rather than guessing", () => { + for ( + const args of [ + [], + [null], + ["not-an-object"], + [[{ request: { domain: "lane" } }]], + [{}], + [{ request: null }], + [{ request: [] }], + [{ request: { action: "create" } }], + [{ request: { domain: " " } }], + [{ request: { domain: 7 } }], + ] + ) { + expect(readRuntimeActionRequest(args)).toBeNull(); + } + }); +}); diff --git a/apps/desktop/src/main/services/ipc/ipcTimeouts.ts b/apps/desktop/src/main/services/ipc/ipcTimeouts.ts index f38333890..3fc114c19 100644 --- a/apps/desktop/src/main/services/ipc/ipcTimeouts.ts +++ b/apps/desktop/src/main/services/ipc/ipcTimeouts.ts @@ -15,6 +15,28 @@ function isRecord(value: unknown): value is Record { return Boolean(value && typeof value === "object" && !Array.isArray(value)); } +/** + * The `{ request: { domain, action } }` payload every runtime `callAction` + * invoke carries, read off the raw IPC arguments. + * + * One decoder for every reader of that shape — the timeout policy below and the + * failure telemetry in `registerIpc` — because they all describe the same call + * and a reader that disagreed about what a well-formed request looks like would + * silently attribute a timeout to one action and its failure to another. + * `action` is optional: a payload naming only a domain is still enough to + * attribute a failure, and callers that need to route on the action check it. + */ +export function readRuntimeActionRequest( + args: readonly unknown[], +): { domain: string; action?: string } | null { + const payload = args[0]; + const request = isRecord(payload) && isRecord(payload.request) ? payload.request : null; + const domain = typeof request?.domain === "string" ? request.domain.trim() : ""; + if (!domain) return null; + const action = typeof request?.action === "string" ? request.action.trim() : ""; + return { domain, ...(action ? { action } : {}) }; +} + const RUNTIME_ACTION_CHANNEL: Record> = { ai: { piLoginStart: IPC.aiPiLoginStart, @@ -48,32 +70,26 @@ const REMOTE_RUNTIME_BOOTSTRAP_TIMEOUT_MS = 10 * 60_000; const REMOTE_RUNTIME_RETRYABLE_ACTION_TIMEOUT_MS = 75_000; function runtimeActionTimeoutMs(args: readonly unknown[]): number | null { - const payload = args[0]; - const request = isRecord(payload) && isRecord(payload.request) ? payload.request : null; - if (typeof request?.domain !== "string" || typeof request.action !== "string") return null; + const request = readRuntimeActionRequest(args); + if (!request?.action) return null; const channel = RUNTIME_ACTION_CHANNEL[request.domain]?.[request.action]; return channel ? ipcInvokeTimeoutMs(channel) : null; } function retryableRemoteActionTimeoutMs(args: readonly unknown[]): number | null { - const payload = args[0]; - const request = isRecord(payload) && isRecord(payload.request) ? payload.request : null; - const domain = request?.domain; - const action = request?.action; - if (typeof domain !== "string" || typeof action !== "string") return null; - return isRetryableRemoteAction(domain, action) + const request = readRuntimeActionRequest(args); + if (!request?.action) return null; + return isRetryableRemoteAction(request.domain, request.action) ? REMOTE_RUNTIME_RETRYABLE_ACTION_TIMEOUT_MS : null; } export function ipcInvokeTimeoutMs(channel: string, args: readonly unknown[] = []): number { if (channel === IPC.localRuntimeCallAction) { - const payload = args[0]; - const request = isRecord(payload) && isRecord(payload.request) ? payload.request : null; - if (typeof request?.domain === "string" && typeof request.action === "string") { - return localRuntimeActionIpcTimeoutMs(request.domain, request.action); - } - return LOCAL_RUNTIME_IPC_PROJECT_COMPLETION_TIMEOUT_MS; + const request = readRuntimeActionRequest(args); + return request?.action + ? localRuntimeActionIpcTimeoutMs(request.domain, request.action) + : LOCAL_RUNTIME_IPC_PROJECT_COMPLETION_TIMEOUT_MS; } if (channel === IPC.localRuntimeCallSync) return LOCAL_RUNTIME_IPC_SYNC_TIMEOUT_MS; // Reconnecting forwards to the brain on the same 30s sync budget. On the 30s diff --git a/apps/desktop/src/main/services/ipc/registerIpc.ts b/apps/desktop/src/main/services/ipc/registerIpc.ts index 50f6dda40..880c42fb7 100644 --- a/apps/desktop/src/main/services/ipc/registerIpc.ts +++ b/apps/desktop/src/main/services/ipc/registerIpc.ts @@ -88,6 +88,10 @@ import { type ProductAnalyticsStatus, } from "../../../shared/types/productAnalytics"; import type { ProductAnalyticsService } from "../analytics/productAnalyticsService"; +import { + brainActionErrorCode, + createMachineRegisterRefusalObserver, +} from "../analytics/reliabilityTelemetry"; import type { createProjectSecretService } from "../secrets/projectSecretService"; import { PROJECT_SECRET_ENV_MAX_BYTES } from "../secrets/projectSecretEnv"; import { lookupOpenPrForBranch } from "../git/ghOpenPrLookup"; @@ -731,7 +735,7 @@ import { buildComputerUseOwnerSnapshot } from "../computerUse/controlPlane"; import type { createIosSimulatorService } from "../ios/iosSimulatorService"; import type { createAppControlService } from "../appControl/appControlService"; import type { createBuiltInBrowserService } from "../builtInBrowser/builtInBrowserService"; -import { ipcInvokeTimeoutMs } from "./ipcTimeouts"; +import { ipcInvokeTimeoutMs, readRuntimeActionRequest } from "./ipcTimeouts"; import { readGlobalState, writeGlobalState, reorderRecentProjects, setRecentProjectPinned, recentProjectKey } from "../state/globalState"; import type { RecentProject } from "../state/globalState"; import type { createKeybindingsService } from "../keybindings/keybindingsService"; @@ -2357,6 +2361,37 @@ export function registerIpc({ // Analytics capture must never mask the original IPC error. } } + // Every brain action the app performs goes through this one channel, + // and `usageActionFromIpcChannel` maps it to `localRuntime.callAction` + // — not a meaningful usage action — so the branch above has never + // fired for it. That is why an installation whose brain failed every + // action produced no telemetry at all. It is deliberately NOT fixed by + // adding the channel to MEANINGFUL_ACTIONS: that set is the durable + // `usage_events` mutation ledger, and joining it would write a + // mutation row per brain call and redefine what "interaction" counts. + if (channel === IPC.localRuntimeCallAction) { + try { + // The analytics policy re-checks this against the closed domain + // list, so an unrecognised domain is dropped rather than reported. + const actionDomain = readRuntimeActionRequest(args)?.domain ?? null; + const errorCode = brainActionErrorCode(error, didTimeout); + productAnalyticsService?.captureInternal({ + event: "ade_brain_action_failed", + surface: "desktop", + // Per domain+code per hour: a brain that rejects every call in + // a retry loop costs one accepted event an hour, not one per + // call, and the per-event daily cap bounds the rest. + dedupeKey: `brain-action-failed:${actionDomain ?? "unknown"}:${errorCode}`, + minimumIntervalMs: 60 * 60 * 1_000, + properties: { + ...(actionDomain ? { action_domain: actionDomain } : {}), + error_code: errorCode, + }, + }); + } catch { + // Analytics capture must never mask the original IPC error. + } + } if (traceIpcInvokes || didTimeout) { const logger = traceLogger ?? getTraceLogger(); logger.warn("ipc.invoke.failed", { @@ -5367,7 +5402,7 @@ export function registerIpc({ ); }); - ipcMain.handle(IPC.syncGetLocalStatus, async (_event, arg?: SyncGetStatusArgs): Promise => { + const readLocalSyncStatus = async (arg?: SyncGetStatusArgs): Promise => { const params = { includeTransferReadiness: arg?.includeTransferReadiness === true, forceTransferReadiness: arg?.forceTransferReadiness === true, @@ -5401,6 +5436,34 @@ export function registerIpc({ } catch (error) { return buildMachineOnlySyncSnapshot(error); } + }; + + // The account directory refusing to register THIS computer is the state a + // revoked machine sits in, and the local status read is where the desktop can + // see it. The observer reports only the edge into a refusal, so a machine that + // stays revoked for days costs one event and not one per poll tick; the + // per-code hour below bounds the case the in-process latch cannot see, which + // is an app or brain restarting inside the refusal. + const observeMachineRegisterRefusal = createMachineRegisterRefusalObserver((code) => { + productAnalyticsService?.capture({ + event: "ade_feature_used", + surface: "desktop", + properties: { + feature: "connections", + action: "machine_register_refused", + outcome: "failed", + refusal_code: code, + }, + projectId: null, + dedupeKey: `machine_register_refused:${code}`, + minimumIntervalMs: 60 * 60 * 1_000, + }); + }); + + ipcMain.handle(IPC.syncGetLocalStatus, async (_event, arg?: SyncGetStatusArgs): Promise => { + const snapshot = await readLocalSyncStatus(arg); + observeMachineRegisterRefusal(snapshot); + return snapshot; }); ipcMain.handle(IPC.syncRefreshDiscovery, async (event): Promise => { @@ -9806,6 +9869,19 @@ export function registerIpc({ localRuntimeConnectionPool, LOCAL_RUNTIME_SYNC_TIMEOUT_MS, ), + // Same shape and the same per-outcome hour as the two Connections controls + // below, so removal joins that funnel rather than starting a parallel one. + // The machine key, its display name and the account id all stay here. + recordMachineRemoved: (outcome) => { + productAnalyticsService?.capture({ + event: "ade_feature_used", + surface: "desktop", + properties: { feature: "connections", action: "machine_removed", outcome }, + projectId: null, + dedupeKey: `machine_removed:${outcome}`, + minimumIntervalMs: 60 * 60 * 1_000, + }); + }, logger: { info: (message, meta) => getCtx().logger.info(message, meta), warn: (message, meta) => getCtx().logger.warn(message, meta), diff --git a/apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.test.ts b/apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.test.ts index 22e1fb640..e8433aad8 100644 --- a/apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.test.ts +++ b/apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.test.ts @@ -3857,6 +3857,26 @@ describe("local runtime action retry classification", () => { expect(isLocalRuntimeConnectionDropped(new Error("Local ADE service action failed."))).toBe(false); }); + it("retries socket-level drops but never a failure the daemon itself reported", () => { + // A socket that went away mid-call, named by its errno rather than by text. + expect(isLocalRuntimeConnectionDropped(Object.assign(new Error("socket hang up"), { code: "ECONNRESET" }))) + .toBe(true); + expect(isLocalRuntimeConnectionDropped(Object.assign(new Error("write after end"), { code: "EPIPE" }))) + .toBe(true); + + // The daemon answered — the failure is the brain's, and retrying it would + // re-run a non-idempotent action against a healthy daemon. + expect(isLocalRuntimeConnectionDropped(new Error( + "Remote ADE service method ade/actions/call failed (code -32603): storage_read_failed: ADE couldn't read this project's data.", + ))).toBe(false); + // …even when the brain's own message quotes the transport sentinel. + expect(isLocalRuntimeConnectionDropped(new Error( + "Remote ADE service method ade/actions/call failed (code -32011): Remote ADE service connection closed.", + ))).toBe(false); + // A message that merely names an errno buys no retry either. + expect(isLocalRuntimeConnectionDropped(new Error("Local ADE service action failed: ECONNRESET"))).toBe(false); + }); + it("only retries idempotent read actions, never mutations", () => { // Reads — safe to retry after a connection drop. expect(isRetryableReadAction("lane", "list")).toBe(true); diff --git a/apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.ts b/apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.ts index 0e48052ed..731cf64b6 100644 --- a/apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.ts +++ b/apps/desktop/src/main/services/localRuntime/localRuntimeConnectionPool.ts @@ -830,8 +830,30 @@ function closeRuntimeClient(client: RuntimeRpcClient): void { // messages (see RuntimeRpcClient.failConnection). A drop happens whenever the // daemon restarts or is recycled — e.g. when a desktop rebuild changes the // expected build hash and the running daemon is deemed incompatible. +const LOCAL_RUNTIME_TRANSPORT_DROP_PATTERN = /Remote ADE service connection (closed|failed)/i; + +// A `Remote ADE service method X failed (code …)` error is the daemon's own +// JSON-RPC reply: the socket answered, so whatever went wrong is the brain's, +// not the transport's. This is checked first and wins, because a brain-side +// message can quote a transport sentence verbatim — and retrying a brain-side +// failure would re-run a non-idempotent action against a healthy daemon. +const LOCAL_RUNTIME_RPC_METHOD_FAILURE_PATTERN = /^Remote ADE service method \S+ failed[\s(:]/i; + +// Socket-level codes that mean the connection went away mid-call. Read from +// `Error.code`, never from message text, so a forwarded application message +// that happens to mention ECONNRESET cannot buy itself a retry. +const LOCAL_RUNTIME_TRANSPORT_ERROR_CODES = new Set([ + "ECONNRESET", + "ECONNABORTED", + "EPIPE", + "ENOTCONN", +]); + export function isLocalRuntimeConnectionDropped(error: Error): boolean { - return /Remote ADE service connection (closed|failed)/i.test(error.message); + if (LOCAL_RUNTIME_RPC_METHOD_FAILURE_PATTERN.test(error.message)) return false; + const code = (error as Error & { code?: unknown }).code; + if (typeof code === "string" && LOCAL_RUNTIME_TRANSPORT_ERROR_CODES.has(code)) return true; + return LOCAL_RUNTIME_TRANSPORT_DROP_PATTERN.test(error.message); } // Conservative mirror of the preload's isReadOnlyRuntimeAction. Only these diff --git a/apps/desktop/src/main/services/prs/prService.ts b/apps/desktop/src/main/services/prs/prService.ts index 4458e8458..e9503e975 100644 --- a/apps/desktop/src/main/services/prs/prService.ts +++ b/apps/desktop/src/main/services/prs/prService.ts @@ -139,6 +139,7 @@ import type { ReviewPublicationDestination, ReviewPublicationInlineComment, } from "../../../shared/types"; +import { GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY } from "../../../shared/types"; import type { AdeDb } from "../state/kvDb"; import type { Logger } from "../logging/logger"; import type { createLaneService } from "../lanes/laneService"; @@ -9231,6 +9232,12 @@ export function createPrService({ }; const buildGithubSnapshotAuthError = (githubStatus: GitHubStatus): string => { + // Ahead of `!tokenStored`: an unreadable credential store reports no token, + // so this would otherwise tell someone whose credentials are intact to run + // `gh auth login` and overwrite them. + if (githubStatus.credentialStoreUnreadable === true) { + return `${GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.title}. ${GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.detail}`; + } if (!githubStatus.tokenStored) { return "GitHub auth missing. Run gh auth login or add a PAT in Settings to sync pull requests."; } diff --git a/apps/desktop/src/main/services/runtime/projectRecoveryService.ts b/apps/desktop/src/main/services/runtime/projectRecoveryService.ts index 312852d31..fd0b50e61 100644 --- a/apps/desktop/src/main/services/runtime/projectRecoveryService.ts +++ b/apps/desktop/src/main/services/runtime/projectRecoveryService.ts @@ -161,6 +161,12 @@ function diagnosisCopy(state: ProjectRecoveryDiagnosis["state"]): Pick< body: "Something interrupted ADE while it was saving. Your files and chats are still here.", canAutoRepair: true, }; + case "storage_unreadable": + return { + headline: "ADE couldn't read this project's data.", + body: "The project's files couldn't be read from this computer. If the folder is in iCloud Drive, Dropbox or OneDrive, move it to a folder on this computer and open it again.", + canAutoRepair: false, + }; case "brain_crash_looping": return { headline: "ADE's background service keeps stopping.", diff --git a/apps/desktop/src/main/services/state/durableFile.test.ts b/apps/desktop/src/main/services/state/durableFile.test.ts index 32ad55d8e..04dd9c53b 100644 --- a/apps/desktop/src/main/services/state/durableFile.test.ts +++ b/apps/desktop/src/main/services/state/durableFile.test.ts @@ -53,14 +53,123 @@ describe("durableFile", () => { expect(tempLeftovers()).toEqual([]); }); - it("leaves the destination untouched when rename fails", () => { + const renameError = (code: string): Error => + Object.assign(new Error(`${code}: rename refused`), { code }); + + it.each(["EXDEV", "EBUSY", "EPERM", "EACCES"])( + "falls back to a copy when rename is refused with %s", + (code) => { + // A temp file on another device, or a target an indexer/antivirus is + // holding open on Windows. The link cannot be made; the bytes are fine. + fs.writeFileSync(filePath, "before"); + injectFsFault({ op: "renameSync", error: () => renameError(code) }); + + writeFileAtomic(filePath, "after"); + expect(fs.readFileSync(filePath, "utf8")).toBe("after"); + expect(tempLeftovers()).toEqual([]); + }, + ); + + it("leaves the destination untouched when rename fails on a full disk", () => { + // ENOSPC is NOT retried as a copy: the filesystem just proved it cannot + // take these bytes, and a second attempt would only half-write the + // target. A failed durable write must leave the old file intact. fs.writeFileSync(filePath, "before"); + const copy = vi.spyOn(fs, "copyFileSync"); injectFsFault({ op: "renameSync" }); expect(() => writeFileAtomic(filePath, "after")).toThrow(/no space/i); + expect(copy).not.toHaveBeenCalled(); expect(fs.readFileSync(filePath, "utf8")).toBe("before"); expect(tempLeftovers()).toEqual([]); }); + + it("reports the rename failure when the copy fallback fails too", () => { + fs.writeFileSync(filePath, "before"); + injectFsFault({ op: "renameSync", error: () => renameError("EXDEV") }); + injectFsFault({ op: "copyFileSync" }); + + // Both causes, not one: the copy failure used to be swallowed and the + // rename reported alone, which named the wrong device for a disk that + // filled up under the copy. + let thrown: NodeJS.ErrnoException | null = null; + try { + writeFileAtomic(filePath, "after"); + } catch (error) { + thrown = error as NodeJS.ErrnoException; + } + expect(thrown?.message).toMatch(/EXDEV/); + expect(thrown?.message).toMatch(/no space/i); + expect(thrown?.code).toBe("EXDEV"); + expect((thrown?.cause as NodeJS.ErrnoException | undefined)?.code).toBe("ENOSPC"); + expect(fs.readFileSync(filePath, "utf8")).toBe("before"); + expect(tempLeftovers()).toEqual([]); + }); + + it("keeps the previous file whole when the copy fallback dies half-written", () => { + // The copy fallback is not atomic, so it must never run onto the target. + // Copying straight over it turned a failed write into a truncated + // destination — the old file gone and the new one incomplete. + fs.writeFileSync(filePath, "before"); + injectFsFault({ op: "renameSync", error: () => renameError("EXDEV") }); + vi.spyOn(fs, "copyFileSync").mockImplementation(((_src: unknown, dest: unknown) => { + fs.writeFileSync(String(dest), "aft"); + throw Object.assign(new Error("ENOSPC: no space left on device"), { code: "ENOSPC" }); + }) as never); + + expect(() => writeFileAtomic(filePath, "after")).toThrow(/copy fallback failed/); + expect(fs.readFileSync(filePath, "utf8")).toBe("before"); + expect(tempLeftovers()).toEqual([]); + }); + + it("still lands the write when every rename onto the target is refused", () => { + // A Windows holder that permits writes and denies the replace: the staged + // rename cannot land either, so the direct copy is the only thing left + // between the user and a lost write. + fs.writeFileSync(filePath, "before"); + vi.spyOn(fs, "renameSync").mockImplementation((() => { + throw renameError("EBUSY"); + }) as never); + + writeFileAtomic(filePath, "after"); + expect(fs.readFileSync(filePath, "utf8")).toBe("after"); + expect(tempLeftovers()).toEqual([]); + }); + + it.skipIf(process.platform === "win32")("keeps a secret's mode through the copy fallback", () => { + // The staged file carries the requested mode BEFORE it is exposed under + // the target's name, so a 0o600 secret is never briefly readable by + // anyone else — not even on the fallback path. + fs.writeFileSync(filePath, "old", { mode: 0o644 }); + injectFsFault({ op: "renameSync", error: () => renameError("EXDEV") }); + + writeFileAtomic(filePath, "secret", { mode: 0o600 }); + expect(fs.readFileSync(filePath, "utf8")).toBe("secret"); + expect(fs.statSync(filePath).mode & 0o777).toBe(0o600); + expect(tempLeftovers()).toEqual([]); + }); + + // Windows has no POSIX mode bits to assert on; the mode is still passed and + // is simply ignored by the filesystem there. + it.skipIf(process.platform === "win32")("creates the temp file with the requested mode so a secret is never briefly readable", () => { + writeFileAtomic(filePath, "secret", { fsync: true, mode: 0o600 }); + expect(fs.readFileSync(filePath, "utf8")).toBe("secret"); + // The rename carries the temp file's mode onto the target. + expect(fs.statSync(filePath).mode & 0o777).toBe(0o600); + }); + + it("never removes the destination before the replacement is in place", () => { + // The pre-unlink some writers use as "the Windows fix" opens a window in + // which the file simply does not exist, and a concurrent reader that + // looks during it sees neither version. `rename` replaces in one step on + // every platform, so `unlink` has no business being on this path. + fs.writeFileSync(filePath, "before"); + const unlink = vi.spyOn(fs, "unlinkSync"); + + writeFileAtomic(filePath, "after", { fsync: true }); + expect(unlink.mock.calls.map(([target]) => String(target))) + .not.toContain(filePath); + }); }); describe("writeJsonWithPrevious", () => { diff --git a/apps/desktop/src/main/services/state/durableFile.ts b/apps/desktop/src/main/services/state/durableFile.ts index 66c030cc6..1ce985f0b 100644 --- a/apps/desktop/src/main/services/state/durableFile.ts +++ b/apps/desktop/src/main/services/state/durableFile.ts @@ -2,7 +2,15 @@ import { randomBytes } from "node:crypto"; import fs from "node:fs"; import path from "node:path"; -export type AtomicWriteOptions = { fsync?: boolean }; +export type AtomicWriteOptions = { + fsync?: boolean; + /** + * Permission bits for the temp file, e.g. `0o600`. The rename carries them + * onto the target, so a secret never exists world-readable — not even for the + * instant between create and rename. + */ + mode?: number; +}; function bestEffortUnlink(filePath: string): void { try { @@ -20,26 +28,154 @@ function bestEffortFsync(fd: number): void { } } +/** + * A scratch name beside the target. The shape matches + * `ABANDONED_TEMP_FILE_PATTERN`, so anything left behind by a crash is swept by + * `cleanupAbandonedTempFiles` rather than accumulating forever. + */ +function siblingTempPath(filePath: string): string { + return path.join( + path.dirname(filePath), + `.${path.basename(filePath)}.tmp-${process.pid}-${randomBytes(8).toString("hex")}`, + ); +} + +function describeCause(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +/** + * Both causes, never one. + * + * The fallback used to rethrow the rename error and swallow its own, so a copy + * that failed on a full disk was reported as the `EXDEV` that sent it there — + * the wrong diagnosis for the wrong device. The rename's `code` is carried onto + * the thrown error because callers switch on errnos, and the copy failure + * travels as `cause`. + */ +function replaceFailure(filePath: string, renameError: unknown, copyError: unknown): Error { + const error = new Error( + `Failed to replace '${filePath}': rename failed (${describeCause(renameError)})` + + ` and the copy fallback failed (${describeCause(copyError)}).`, + { cause: copyError }, + ) as NodeJS.ErrnoException; + const code = (renameError as NodeJS.ErrnoException | null)?.code; + if (code) error.code = code; + return error; +} + +/** + * Rename failures a copy can actually fix. + * + * `EXDEV` is a temp file that landed on another device; the rest are the + * Windows shapes of "something else is holding the target open" — an indexer, + * an antivirus scanner, another ADE process reading it. Every one of them is a + * link-level refusal that says nothing about whether the bytes can be written. + * + * Deliberately NOT a catch-all. Retrying `ENOSPC` or `EIO` as a copy writes the + * payload a second time to a filesystem that just proved it cannot take it, and + * turns a clean "the write failed, the old file is intact" into a half-written + * target. A disk-full rename must stay terminal. + */ +const COPY_RECOVERABLE_RENAME_CODES = new Set(["EXDEV", "EPERM", "EACCES", "EBUSY"]); + +/** + * Replace the target with the temp file. + * + * `rename` is the atomic path on every platform this ships to, including + * Windows: libuv implements `fs.rename` with `MoveFileExW(MOVEFILE_REPLACE_EXISTING)`, + * which replaces an existing target in one step. Deleting the target first + * would NOT be "the Windows fix" — it would open a window in which the file + * simply does not exist, and a concurrent reader that looks during that window + * sees a missing file rather than either version of it. + * + * The copy fallback is the last resort and never the first move: it is not + * atomic, so it is reached only for the failures above, where the alternative + * is losing the write outright. It runs in two steps, and the second one only + * when the first proves the target itself is what refuses to be renamed over. + */ +function replaceViaRename(tempPath: string, filePath: string, mode?: number): void { + try { + fs.renameSync(tempPath, filePath); + return; + } catch (renameError) { + const code = (renameError as NodeJS.ErrnoException).code; + if (!code || !COPY_RECOVERABLE_RENAME_CODES.has(code)) throw renameError; + + // Copy BESIDE the target, then rename that into place. The destination is + // still only ever replaced by a rename, so a copy that dies half-written — + // the disk fills, the source turns out to be unreadable — leaves the + // previous file whole instead of truncating it. It is also the whole fix + // for `EXDEV`: the copy lands the bytes on the destination's device, and + // the rename that follows is a local one. + const stagedPath = siblingTempPath(filePath); + let reachedStagedRename = false; + try { + fs.copyFileSync(tempPath, stagedPath); + // The copy creates the staged file with the source's permissions, so a + // 0o600 secret is never briefly world-readable; the chmod states it + // outright for a filesystem that does not carry the mode across, and + // happens BEFORE the file is exposed under the target's name. + if (mode != null) fs.chmodSync(stagedPath, mode); + reachedStagedRename = true; + fs.renameSync(stagedPath, filePath); + bestEffortUnlink(tempPath); + return; + } catch (stagedError) { + bestEffortUnlink(stagedPath); + const stagedCode = (stagedError as NodeJS.ErrnoException).code; + // Only a link-level refusal of the staged RENAME earns the last resort + // below. When the copy is what failed, writing those same bytes straight + // onto the target would produce exactly the half-written destination the + // staging exists to prevent. + if (!reachedStagedRename || !stagedCode || !COPY_RECOVERABLE_RENAME_CODES.has(stagedCode)) { + throw replaceFailure(filePath, renameError, stagedError); + } + } + + // Nothing can be renamed into this name — a Windows holder that permits + // writes and denies the replace, or a directory that refuses links + // outright. Copying onto the target is not atomic and a reader can catch it + // mid-write, which is why it is last: the alternative here is losing the + // write outright. + try { + fs.copyFileSync(tempPath, filePath); + if (mode != null) { + try { + fs.chmodSync(filePath, mode); + } catch { + // Copying onto an existing file keeps that file's permissions; a + // platform without chmod simply keeps whatever it had. + } + } + bestEffortUnlink(tempPath); + } catch (copyError) { + throw replaceFailure(filePath, renameError, copyError); + } + } +} + export function writeFileAtomic( filePath: string, data: string | Buffer, opts: AtomicWriteOptions = {}, ): void { const dir = path.dirname(filePath); - const tempPath = path.join( - dir, - `.${path.basename(filePath)}.tmp-${process.pid}-${randomBytes(8).toString("hex")}`, - ); + const tempPath = siblingTempPath(filePath); let fd: number | null = null; try { - fd = fs.openSync(tempPath, "wx"); + fd = opts.mode != null + ? fs.openSync(tempPath, "wx", opts.mode) + : fs.openSync(tempPath, "wx"); fs.writeFileSync(fd, data); if (opts.fsync) bestEffortFsync(fd); fs.closeSync(fd); fd = null; - fs.renameSync(tempPath, filePath); + replaceViaRename(tempPath, filePath, opts.mode); - if (opts.fsync) { + // Windows has no directory handle to flush; opening one fails outright, so + // the attempt is skipped rather than caught. + if (opts.fsync && process.platform !== "win32") { let dirFd: number | null = null; try { dirFd = fs.openSync(dir, "r"); diff --git a/apps/desktop/src/main/services/state/kvDb.rebuildRecovery.test.ts b/apps/desktop/src/main/services/state/kvDb.rebuildRecovery.test.ts index 04df1f75e..db7a091b7 100644 --- a/apps/desktop/src/main/services/state/kvDb.rebuildRecovery.test.ts +++ b/apps/desktop/src/main/services/state/kvDb.rebuildRecovery.test.ts @@ -390,6 +390,31 @@ describe("kvDb migration backup", () => { expect(classifySqliteOpenError(new Error("database disk image is malformed"))).toBe("db_integrity"); expect(classifySqliteOpenError(new Error("surprise"))).toBe("unknown"); }); + + it("buckets unreadable-storage errnos apart from a damaged database", () => { + // macOS returns EDEADLK for a File-Provider read it cannot satisfy, and + // libuv has no name for it — this exact message reached a user's screen. + expect(classifySqliteOpenError(Object.assign( + new Error("Unknown system error -11: Unknown system error -11, read"), + { errno: -11, syscall: "read" }, + ))).toBe("storage_read_failed"); + expect(classifySqliteOpenError(new Error("Unknown system error -11, read"))).toBe("storage_read_failed"); + expect(classifySqliteOpenError(Object.assign(new Error("read failed"), { code: "EDEADLK" }))) + .toBe("storage_read_failed"); + expect(classifySqliteOpenError(Object.assign(new Error("input/output error"), { code: "EIO" }))) + .toBe("storage_read_failed"); + // A full disk still wins: it has its own repair path. + expect(classifySqliteOpenError(Object.assign(new Error("Unknown system error -28"), { code: "ENOSPC" }))) + .toBe("disk_full"); + // An unreadable file is never reported as a corrupt one — repairing it + // would rewrite data ADE could not read. + expect(classifySqliteOpenError(Object.assign( + new Error("Unknown system error -11, read"), + { code: "EDEADLK" }, + ))).not.toBe("db_integrity"); + expect(classifySqliteOpenError(Object.assign(new Error("permission denied"), { code: "EACCES" }))) + .toBe("unknown"); + }); }); describe("kvDb storage maintenance", () => { diff --git a/apps/desktop/src/main/services/state/kvDb.ts b/apps/desktop/src/main/services/state/kvDb.ts index 3971024bd..0360b3918 100644 --- a/apps/desktop/src/main/services/state/kvDb.ts +++ b/apps/desktop/src/main/services/state/kvDb.ts @@ -4,7 +4,7 @@ import { Buffer } from "node:buffer"; import { randomBytes } from "node:crypto"; import { createRequire } from "node:module"; import type { DatabaseSync as DatabaseSyncType } from "node:sqlite"; -import { codedError } from "../../../shared/codedError"; +import { codedError, UNKNOWN_SYSTEM_ERRNO_PATTERN } from "../../../shared/codedError"; import type { Logger } from "../logging/logger"; import { safeJsonParse } from "../shared/utils"; import { isNoSpaceError, readVolumeSpace } from "../storage/volume"; @@ -59,6 +59,7 @@ export type KvDbOpenErrorCode = | "db_integrity" | "migration_incomplete" | "migration_unknown_state" + | "storage_read_failed" | "unknown"; const KV_DB_OPEN_ERROR_CODES = new Set([ @@ -67,9 +68,39 @@ const KV_DB_OPEN_ERROR_CODES = new Set([ "db_integrity", "migration_incomplete", "migration_unknown_state", + "storage_read_failed", "unknown", ]); +/** + * Filesystem failures that mean "the bytes are not readable here", as opposed + * to "the database is damaged". Reading a File-Provider placeholder (iCloud + * Drive, Dropbox, OneDrive) that the provider cannot materialize is the common + * cause; a failing disk or a dropped network mount produces the same shapes. + */ +const STORAGE_READ_ERRNO_CODES = new Set([ + "EDEADLK", + "EIO", + "ENXIO", + "ENODEV", + "ESTALE", + "EHOSTDOWN", + "EREMOTEIO", +]); + +/** + * An unnameable platform errno (see `UNKNOWN_SYSTEM_ERRNO_PATTERN` in + * `shared/codedError`) is a raw filesystem failure here, never a SQLite-level + * verdict — which is why it is classified ahead of the integrity bucket below. + */ +function isStorageReadError(error: unknown): boolean { + if (!error || typeof error !== "object") return false; + const raw = error as { code?: unknown; errno?: unknown; syscall?: unknown }; + if (typeof raw.code === "string" && STORAGE_READ_ERRNO_CODES.has(raw.code)) return true; + const message = error instanceof Error ? error.message : String(error); + return UNKNOWN_SYSTEM_ERRNO_PATTERN.test(message); +} + export function classifySqliteOpenError(error: unknown): KvDbOpenErrorCode { const explicitCode = error && typeof error === "object" && "code" in error ? String((error as { code?: unknown }).code) @@ -80,6 +111,9 @@ export function classifySqliteOpenError(error: unknown): KvDbOpenErrorCode { const message = error instanceof Error ? error.message : String(error); if (isNoSpaceError(error) || /SQLITE_FULL/i.test(message)) return "disk_full"; + // Ahead of the integrity bucket: an unreadable file is not a corrupt one, and + // offering to "repair" a placeholder would rewrite data ADE cannot even read. + if (isStorageReadError(error)) return "storage_read_failed"; if (/malformed|not a database|integrity/i.test(message)) return "db_integrity"; return "unknown"; } diff --git a/apps/desktop/src/main/services/storage/cloudPlaceholder.test.ts b/apps/desktop/src/main/services/storage/cloudPlaceholder.test.ts new file mode 100644 index 000000000..89b91fa79 --- /dev/null +++ b/apps/desktop/src/main/services/storage/cloudPlaceholder.test.ts @@ -0,0 +1,91 @@ +import { describe, expect, it } from "vitest"; +import { + detectCloudPlaceholderFile, + detectCloudStorageProvider, + isDatalessFileStats, + storageUnreadableMessage, +} from "./cloudPlaceholder"; + +describe("cloud storage provider detection", () => { + it("recognizes the folders each provider actually mounts", () => { + expect(detectCloudStorageProvider("/Users/a/Library/Mobile Documents/com~apple~CloudDocs/proj/.ade/ade.db")) + .toBe("icloud"); + expect(detectCloudStorageProvider("/Users/a/Library/CloudStorage/Box-Personal/proj/.ade/ade.db")) + .toBe("cloud-storage"); + expect(detectCloudStorageProvider("/Users/a/Dropbox/proj/.ade/ade.db")).toBe("dropbox"); + expect(detectCloudStorageProvider("C:\\Users\\a\\OneDrive - Contoso\\proj\\.ade\\ade.db")) + .toBe("onedrive"); + expect(detectCloudStorageProvider("/home/a/Google Drive/proj/.ade/ade.db")).toBe("google-drive"); + }); + + it("leaves ordinary local paths alone", () => { + expect(detectCloudStorageProvider("/Users/a/Projects/proj/.ade/ade.db")).toBeNull(); + expect(detectCloudStorageProvider("/Users/a/dropboxes/proj/.ade/ade.db")).toBeNull(); + expect(detectCloudStorageProvider("")).toBeNull(); + }); +}); + +describe("dataless placeholder detection", () => { + it("treats content with no allocation as evicted", () => { + expect(isDatalessFileStats({ size: 4_194_304, blocks: 0 })).toBe(true); + expect(isDatalessFileStats({ size: 4_194_304, blocks: 8192 })).toBe(false); + expect(isDatalessFileStats({ size: 0, blocks: 0 })).toBe(false); + }); + + it("only refuses when the file is both cloud-hosted and dehydrated", () => { + const cloudPath = "/Users/a/Library/Mobile Documents/com~apple~CloudDocs/proj/.ade/ade.db"; + expect(detectCloudPlaceholderFile(cloudPath, { statSync: () => ({ size: 4_194_304, blocks: 0 }) })) + .toEqual({ path: cloudPath, provider: "icloud" }); + + // Materialized inside iCloud Drive: works today, must keep working. + expect(detectCloudPlaceholderFile(cloudPath, { statSync: () => ({ size: 4_194_304, blocks: 8192 }) })) + .toBeNull(); + + // Sparse-looking but local: far more likely a fresh database than an + // evicted one, so it is never blocked here. + expect(detectCloudPlaceholderFile("/Users/a/Projects/proj/.ade/ade.db", { + statSync: () => ({ size: 4_194_304, blocks: 0 }), + })).toBeNull(); + }); + + it("stays out of the way when the file cannot be stat'd", () => { + expect(detectCloudPlaceholderFile("/Users/a/Dropbox/proj/.ade/ade.db", { + statSync: () => { + throw Object.assign(new Error("ENOENT"), { code: "ENOENT" }); + }, + })).toBeNull(); + }); +}); + +describe("storageUnreadableMessage", () => { + const dbPath = "/Users/a/Google Drive/proj/.ade/ade.db"; + + it("names the provider it actually detected", () => { + // The Google Drive case is the one that used to be told its folder was in + // iCloud Drive, Dropbox or OneDrive — three services it is not in. + const message = storageUnreadableMessage(dbPath, "google-drive"); + expect(message).toContain("stored in Google Drive"); + expect(message).not.toContain("iCloud"); + expect(message).not.toContain("Dropbox"); + expect(message).not.toContain("OneDrive"); + expect(message).toContain("Move the project to a folder on this computer"); + + expect(storageUnreadableMessage(dbPath, "icloud")).toContain("stored in iCloud Drive"); + expect(storageUnreadableMessage(dbPath, "dropbox")).toContain("stored in Dropbox"); + expect(storageUnreadableMessage(dbPath, "onedrive")).toContain("stored in OneDrive"); + }); + + it("says `the cloud` for a provider folder with no brand to read", () => { + // `~/Library/CloudStorage/Box-Personal/…` is a provider root whose brand + // this build cannot name, so it says what it knows and nothing more. + expect(storageUnreadableMessage(dbPath, "cloud-storage")).toContain("stored in the cloud"); + }); + + it("keeps the remedy conditional when no provider was detected", () => { + // The same failure arrives from a failing disk or a dropped network mount, + // so a local folder is never told to move. + const message = storageUnreadableMessage("/Users/a/Projects/proj/.ade/ade.db", null); + expect(message).toContain("If the folder is in iCloud Drive, Dropbox or OneDrive"); + expect(message).toContain("/Users/a/Projects/proj/.ade/ade.db"); + }); +}); diff --git a/apps/desktop/src/main/services/storage/cloudPlaceholder.ts b/apps/desktop/src/main/services/storage/cloudPlaceholder.ts new file mode 100644 index 000000000..0b1764d19 --- /dev/null +++ b/apps/desktop/src/main/services/storage/cloudPlaceholder.ts @@ -0,0 +1,132 @@ +import fs from "node:fs"; + +/** + * Detecting a project (or `~/.ade`) that lives inside a file-syncing cloud + * folder whose contents have been evicted to the cloud. + * + * When ADE opens a database file that the provider has replaced with a + * placeholder, the read fails with an errno the platform never names — on + * macOS it is EDEADLK, which libuv surfaces as "Unknown system error -11". + * That message is what reached users, and it says nothing about what to do. + * + * Two independent signals are used, and a boot is only refused when BOTH + * agree: + * + * - The file has a size but no allocated blocks. This is what a dehydrated + * placeholder looks like through `fs.stat` on both macOS (File Provider) + * and Windows (OneDrive sparse placeholders); Node exposes no access to + * macOS `st_flags`/`UF_DATALESS` or to Windows file attributes, so the + * allocation is the only attribute-free signal available. + * - The path sits under a known provider root. + * + * Requiring both is deliberate. A project inside iCloud Drive whose files are + * materialized works fine today, and a sparse-looking file outside any cloud + * root is far more likely to be an empty or freshly created database than an + * evicted one. Neither signal alone is worth blocking a working setup for — + * and when this preflight declines to fire, the open still fails safe, because + * `classifySqliteOpenError` buckets the raw errno into the same code. + */ +export type CloudStorageProvider = + | "icloud" + | "onedrive" + | "dropbox" + | "google-drive" + | "cloud-storage"; + +/** + * Provider roots, matched on the path text alone so this stays free on every + * platform. macOS mounts modern providers under `~/Library/CloudStorage/` and + * iCloud Drive under `~/Library/Mobile Documents/`; Windows and Linux clients + * use a home-relative folder named after the provider, which is all Node can + * see there (`fs.stat` reports no file attributes, so Windows placeholder flags + * like FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS are not reachable without a native + * addon). + */ +const PROVIDER_PATTERNS: ReadonlyArray<{ provider: CloudStorageProvider; pattern: RegExp }> = [ + { provider: "icloud", pattern: /[\\/]Library[\\/]Mobile Documents[\\/]/i }, + { provider: "onedrive", pattern: /[\\/]OneDrive([\\/]|$)|[\\/]OneDrive[ -][^\\/]*[\\/]/i }, + { provider: "dropbox", pattern: /[\\/]Dropbox([\\/]|$)|[\\/]Dropbox[ (][^\\/]*[\\/]/i }, + { provider: "google-drive", pattern: /[\\/]Google ?Drive([\\/]|$)/i }, + { provider: "cloud-storage", pattern: /[\\/]Library[\\/]CloudStorage[\\/]/i }, +]; + +/** The provider folder `targetPath` sits inside, or null when it is local. */ +export function detectCloudStorageProvider(targetPath: string): CloudStorageProvider | null { + if (!targetPath) return null; + for (const { provider, pattern } of PROVIDER_PATTERNS) { + if (pattern.test(targetPath)) return provider; + } + return null; +} + +/** + * A file with content but no allocated blocks. Materialized databases always + * allocate; only placeholders (and fully sparse files) report zero. + */ +export function isDatalessFileStats(stats: Pick): boolean { + return stats.size > 0 && stats.blocks === 0; +} + +export type CloudPlaceholderFinding = { + path: string; + provider: CloudStorageProvider; +}; + +/** + * One `fs.statSync` on a file ADE is about to open. Returns a finding only when + * the file is both cloud-hosted and dehydrated; a missing file (the normal + * first-run case) and any stat failure return null, because refusing to boot on + * an unreadable stat would be a worse failure than the one this prevents. + */ +export function detectCloudPlaceholderFile( + filePath: string, + deps: { statSync?: (target: string) => Pick } = {}, +): CloudPlaceholderFinding | null { + const provider = detectCloudStorageProvider(filePath); + if (!provider) return null; + const statSync = deps.statSync ?? ((target: string) => fs.statSync(target)); + let stats: Pick; + try { + stats = statSync(filePath); + } catch { + return null; + } + return isDatalessFileStats(stats) ? { path: filePath, provider } : null; +} + +/** + * What each provider is called on the user's own machine. + * + * `cloud-storage` has no name to give: `~/Library/CloudStorage/` hosts Box, + * Egnyte and any other File Provider, including brands this build has never + * heard of, so that one says "the cloud" rather than guessing a brand. + */ +const PROVIDER_LABELS: Record = { + icloud: "iCloud Drive", + onedrive: "OneDrive", + dropbox: "Dropbox", + "google-drive": "Google Drive", + "cloud-storage": null, +}; + +/** + * The one sentence a person needs: where the data is and what to do. + * + * The remedy is stated conditionally unless a provider was actually detected, + * because the same unreadable-file failure also arrives from a failing disk or + * a dropped network mount, and telling someone to move a folder that is + * already local would send them chasing the wrong thing. Brand names rather + * than "File Provider": the brand is how the folder is labelled on their + * machine — and it is the brand that was DETECTED, never a list, because + * naming three services a Google Drive folder is not in only reads as wrong. + */ +export function storageUnreadableMessage( + targetPath: string, + provider?: CloudStorageProvider | null, +): string { + const label = provider ? PROVIDER_LABELS[provider] ?? "the cloud" : null; + const remedy = label + ? `The folder is stored in ${label} and isn't downloaded to this computer. Move the project to a folder on this computer, then open it again.` + : "If the folder is in iCloud Drive, Dropbox or OneDrive, move it to a folder on this computer and open it again."; + return `ADE couldn't read this project's data at ${targetPath}. ${remedy}`; +} diff --git a/apps/desktop/src/renderer/components/app/IntegrationBannerHost.test.tsx b/apps/desktop/src/renderer/components/app/IntegrationBannerHost.test.tsx index 5137fd3a7..0bde21854 100644 --- a/apps/desktop/src/renderer/components/app/IntegrationBannerHost.test.tsx +++ b/apps/desktop/src/renderer/components/app/IntegrationBannerHost.test.tsx @@ -393,6 +393,35 @@ describe("IntegrationBannerHost relay-offline banner", () => { expect(labels.some((label) => /reconnect|connect github|fix github|set up/i.test(label))).toBe(false); }); + it("lets an unreadable credential store pierce the outage suppression", async () => { + // The unreadable store is a local, repairable fact that outlives any GitHub + // incident, and this banner is the discovery surface for the Repair path — + // an outage must not hide the one thing the user can actually fix. + setAdeMock({ + getAppInstallationStatus: vi.fn(async () => makeInstall()), + getAppUserAuthStatus: vi.fn(async () => makeAuth({ tokenStored: false })), + onStatusChanged: vi.fn(() => () => {}), + }); + + await act(async () => { + render( + , + ); + }); + await act(async () => {}); + + // Both facts are on screen: the incident notice AND the repairable one. + expect(screen.getByText("GitHub is down")).toBeTruthy(); + expect(screen.getByText(/can't read your saved sign-in/i)).toBeTruthy(); + }); + it("sends the outage action to the live incident, not ADE settings", async () => { const openExternal = vi.fn(async () => {}); Object.defineProperty(window, "ade", { diff --git a/apps/desktop/src/renderer/components/app/IntegrationBannerHost.tsx b/apps/desktop/src/renderer/components/app/IntegrationBannerHost.tsx index b1afb51fe..1a693f10f 100644 --- a/apps/desktop/src/renderer/components/app/IntegrationBannerHost.tsx +++ b/apps/desktop/src/renderer/components/app/IntegrationBannerHost.tsx @@ -395,8 +395,12 @@ export function IntegrationBannerHost({ // 2) gh CLI / PAT not connected (MIGRATED). A DISTINCT concern from the App // block: this is the token ADE uses for git & PR operations, not webhooks. + // An unreadable credential store pierces the outage suppression: it is a + // local, repairable fact that outlives any GitHub incident, and this banner + // is the discovery surface for the Repair path — an outage must not hide + // the one thing the user can actually fix. if ( - !githubSuppressed + (!githubSuppressed || githubStatus?.credentialStoreUnreadable === true) && currentProjectRoot && githubStatus && (!githubStatus.connected || !githubStatusHasWriteCredential(githubStatus)) @@ -411,7 +415,13 @@ export function IntegrationBannerHost({ { label: cli.action, variant: "primary", - onClick: () => navigate(GITHUB_CONNECTION_SETTINGS_ROUTE), + // The banner states its own destination: an unreadable credential + // store is not fixed on the GitHub card, and its Repair control + // lives in the Connections panel — the same one the relay banner + // opens. + onClick: cli.target === "connections" + ? () => openConnectionsPanel("machines") + : () => navigate(GITHUB_CONNECTION_SETTINGS_ROUTE), }, ], dismiss: { key: `github-cli:${currentProjectRoot}`, fingerprint: cli.subState }, diff --git a/apps/desktop/src/renderer/components/app/ProjectRecoveryScreen.tsx b/apps/desktop/src/renderer/components/app/ProjectRecoveryScreen.tsx index c69ea61c2..df18256f4 100644 --- a/apps/desktop/src/renderer/components/app/ProjectRecoveryScreen.tsx +++ b/apps/desktop/src/renderer/components/app/ProjectRecoveryScreen.tsx @@ -98,6 +98,18 @@ const STATE_COPY: Record< body: "Something interrupted ADE while it was saving. ADE can finish the job and reopen the project — your files and chats stay exactly where they are.", canAutoRepair: true, }, + storage_unreadable: { + headline: "ADE couldn't read this project's data", + // No repair offer: rewriting files ADE can't read would risk the user's + // work, and the same failure also comes from a failing disk — so the + // remedy is stated as a condition, not an accusation. + body: "The project's files couldn't be read from this computer. If the folder is in iCloud Drive, Dropbox or OneDrive, move it to a folder on this computer and open it again.", + canAutoRepair: false, + prerequisites: [ + "Move the project folder out of iCloud Drive, Dropbox or OneDrive.", + "Then choose Try again.", + ], + }, brain_not_installed: { headline: "ADE needs to finish setting up", body: "A background component isn't ready yet. ADE can set it up and reopen the project.", diff --git a/apps/desktop/src/renderer/components/app/ReportIssueButton.test.tsx b/apps/desktop/src/renderer/components/app/ReportIssueButton.test.tsx index 599113a36..2bfcc9ad2 100644 --- a/apps/desktop/src/renderer/components/app/ReportIssueButton.test.tsx +++ b/apps/desktop/src/renderer/components/app/ReportIssueButton.test.tsx @@ -3,8 +3,17 @@ import { cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react"; import { afterEach, describe, expect, it, vi } from "vitest"; import type { DiagnosticReportPayload } from "../../../shared/types/diagnostics"; +import type { DiagnosticUploadResult } from "../../../shared/diagnosticsUpload"; import { ReportIssueButton } from "./ReportIssueButton"; +const uploadDiagnosticReport = vi.hoisted(() => vi.fn()); +vi.mock("../../../shared/diagnosticsUpload", async () => { + const actual = await vi.importActual( + "../../../shared/diagnosticsUpload", + ); + return { ...actual, uploadDiagnosticReport }; +}); + const CONTEXT = { surface: "project_recovery", headline: "ADE couldn't open this project", @@ -33,6 +42,7 @@ function installBridge(openIssue: ReturnType) { afterEach(() => { cleanup(); + uploadDiagnosticReport.mockReset(); delete (window as unknown as { ade?: unknown }).ade; }); @@ -70,6 +80,40 @@ describe("ReportIssueButton", () => { expect(screen.queryByText(/ENOSPC/)).toBeNull(); }); + it("ignores an upload result that belongs to a report the user already replaced", async () => { + // "Report issue" stays enabled while a send is in flight, so the first + // upload's reply can land after a second report exists. Showing its + // reference then would point a maintainer at the wrong report. + const openIssue = vi + .fn() + .mockResolvedValueOnce(payload({ report: "first report" })) + .mockResolvedValueOnce(payload({ report: "second report" })); + installBridge(openIssue); + let settleFirstUpload: (result: DiagnosticUploadResult) => void = () => {}; + uploadDiagnosticReport.mockImplementationOnce( + () => new Promise((resolve) => { settleFirstUpload = resolve; }), + ); + + render(); + fireEvent.click(screen.getByRole("button", { name: "Report issue" })); + await screen.findByRole("button", { name: "Send to ADE" }); + fireEvent.click(screen.getByRole("button", { name: "Send to ADE" })); + await screen.findByRole("button", { name: "Sending…" }); + + // A second report, generated while the first upload is still open. + fireEvent.click(screen.getByRole("button", { name: "Report issue" })); + await waitFor(() => { + expect(openIssue).toHaveBeenCalledTimes(2); + }); + + settleFirstUpload({ ok: true, id: "abc", reference: "ADE-STALE-REF" }); + + // The stale reply frees the Send button again but never claims the newer + // report was sent. + await screen.findByRole("button", { name: "Send to ADE" }); + expect(screen.queryByText(/ADE-STALE-REF/)).toBeNull(); + }); + it("drops the disclosure inside one-line banners, and keeps it when asked", () => { installBridge(vi.fn()); const { rerender } = render(); diff --git a/apps/desktop/src/renderer/components/app/ReportIssueButton.tsx b/apps/desktop/src/renderer/components/app/ReportIssueButton.tsx index 1a36b48ed..2e2e2298d 100644 --- a/apps/desktop/src/renderer/components/app/ReportIssueButton.tsx +++ b/apps/desktop/src/renderer/components/app/ReportIssueButton.tsx @@ -1,8 +1,14 @@ -import { useCallback, useState } from "react"; +import { useCallback, useRef, useState } from "react"; import type { DiagnosticReportPayload, DiagnosticReportRequestPayload, } from "../../../shared/types/diagnostics"; +import { + describeDiagnosticUploadFailure, + resolveDiagnosticsUploadBaseUrl, + uploadDiagnosticReport, + type DiagnosticUploadResult, +} from "../../../shared/diagnosticsUpload"; import { useCopyToClipboard } from "../../hooks/useCopyToClipboard"; import { ERROR_DISCLOSURE_CARET, @@ -12,6 +18,10 @@ import { export type ReportIssueVariant = "primary" | "secondary" | "ghost"; +/** The inline actions inside the result line: text links, not buttons in a row. */ +const REPORT_LINK_BUTTON = + "font-medium text-fg/75 underline decoration-fg/25 underline-offset-2 transition-colors hover:text-fg disabled:no-underline disabled:opacity-60"; + const VARIANT_CLASS: Record = { primary: ERROR_PRIMARY_BUTTON, secondary: ERROR_SECONDARY_BUTTON, @@ -23,7 +33,16 @@ const VARIANT_CLASS: Record = { /** * Every error screen's escape hatch: collect a redacted diagnostic report, - * put it on the clipboard and on disk, and open a prefilled GitHub issue. + * put it on the clipboard and on disk, and open a prefilled GitHub issue — + * then optionally hand that exact report straight to ADE. + * + * The send runs here rather than in the main process because the diagnostics + * preload bridge exposes only `openIssue`; the renderer already holds the + * finished, redacted report that call returns, so it posts those same bytes. + * One consequence, deliberate: the renderer has no access to the account token + * (it lives in the brain's credential store), so a desktop upload is anonymous + * and identified only by the install id the report already carries. + * `ade report-issue --send` reads the store directly and does send a token. * * Deliberately self-contained — one import and one element per host screen — * so the error surfaces can be redesigned without untangling it. @@ -48,14 +67,25 @@ export function ReportIssueButton({ const [pending, setPending] = useState(false); const [result, setResult] = useState(null); const [error, setError] = useState(null); + const [sending, setSending] = useState(false); + const [sent, setSent] = useState(null); const { copy, copied } = useCopyToClipboard(); + /** + * Which report the upload result belongs to. "Report issue" stays live while + * a send is in flight, so a user who reports twice can have the first + * upload's reply land after the second report exists — and a reference that + * points at the older report is worse than none. + */ + const reportGenerationRef = useRef(0); const bridge = typeof window !== "undefined" ? window.ade?.diagnostics : undefined; const run = useCallback(async () => { if (!bridge?.openIssue || pending) return; + reportGenerationRef.current += 1; setPending(true); setError(null); + setSent(null); try { const payload = await bridge.openIssue(context); setResult(payload); @@ -67,6 +97,37 @@ export function ReportIssueButton({ } }, [bridge, context, pending]); + const send = useCallback(async () => { + if (!result || sending) return; + const generation = reportGenerationRef.current; + setSending(true); + try { + const outcome = await uploadDiagnosticReport({ + // The very bytes the clipboard holds. Redaction already happened in + // the main process; nothing here reshapes the report. + report: result.report, + // "unknown" is the report's stand-in for "analytics is switched off"; + // sending it as an install id would attach a value that matches nothing. + installId: result.installId === "unknown" ? null : result.installId, + // Resolved here rather than inside the upload: the CLI resolves its own + // origin the way the brain does, so the client itself takes a base URL + // its caller already decided on. + baseUrl: resolveDiagnosticsUploadBaseUrl( + typeof import.meta.env.VITE_ADE_ACCOUNT_DIRECTORY_URL === "string" + ? import.meta.env.VITE_ADE_ACCOUNT_DIRECTORY_URL + : null, + ), + }); + if (reportGenerationRef.current !== generation) return; + setSent(outcome); + } finally { + // Cleared unconditionally: `sending` is the only thing keeping a second + // upload out, so a stale reply that left it set would strand the newer + // report with a permanently disabled Send. + setSending(false); + } + }, [result, sending]); + // An older preload has no diagnostics bridge; offering a dead button is // worse than offering nothing on a screen that is already failing. if (!bridge?.openIssue) return null; @@ -104,10 +165,33 @@ export function ReportIssueButton({ + {sent?.ok ? null : ( + <> + {" · "} + + + )} + {sent + ? ( + + {" "} + {sent.ok + ? `Sent — reference ${sent.reference}` + : describeDiagnosticUploadFailure(sent.reason)} + + ) + : null} ) : null} @@ -153,7 +237,8 @@ export function ReportIssueButton({ } > File paths, your name, email addresses and any sign-in codes are removed - before the report is created. Nothing is sent until you post the issue. + before the report is created. Nothing leaves this computer unless you post + the issue or choose "Send to ADE".

) : null} diff --git a/apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.test.tsx b/apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.test.tsx new file mode 100644 index 000000000..d16672cc2 --- /dev/null +++ b/apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.test.tsx @@ -0,0 +1,111 @@ +// @vitest-environment jsdom +import React from "react"; +import { cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY } from "../../../shared/types"; +import { subscribeOpenConnectionsPanel } from "../../lib/connectionsPanel"; +import { PublishToGitHubDialog } from "./PublishToGitHubDialog"; + +const getStatus = vi.fn(); +const publishCurrentProject = vi.fn(); +const setToken = vi.fn(); +const openExternal = vi.fn(); + +function status(overrides: Record = {}) { + return { + connected: false, + userLogin: null, + credentialStoreUnreadable: false, + ...overrides, + }; +} + +beforeEach(() => { + getStatus.mockReset(); + publishCurrentProject.mockReset(); + setToken.mockReset(); + openExternal.mockReset(); + getStatus.mockResolvedValue(status()); + (globalThis as any).window.ade = { + github: { getStatus, setToken, publishCurrentProject }, + app: { openExternal }, + }; +}); + +afterEach(() => { + cleanup(); +}); + +function renderDialog() { + return render( + {}} + defaultRepoName="ade" + onPublished={() => {}} + />, + ); +} + +/** Drives the dialog into the connect step the way the main process does. */ +async function reachConnectStep() { + publishCurrentProject.mockRejectedValue( + new Error("github_not_connected: GitHub is not connected."), + ); + await waitFor(() => expect(getStatus).toHaveBeenCalled()); + fireEvent.click(screen.getByRole("button", { name: "Publish" })); +} + +describe("PublishToGitHubDialog", () => { + // An unreadable store is NOT "not connected" — the saved sign-in may still be + // on disk. Offering "Save token" here writes over a credential that repair can + // still recover, so the only route out of this state is Settings → Connections. + it("offers repair instead of token replacement when the credential store is unreadable", async () => { + getStatus.mockResolvedValue(status({ credentialStoreUnreadable: true })); + const openedTabs: string[] = []; + const unsubscribe = subscribeOpenConnectionsPanel((tab) => openedTabs.push(tab)); + try { + renderDialog(); + await reachConnectStep(); + + const repair = await screen.findByRole("button", { + name: GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.action, + }); + expect(screen.queryByRole("button", { name: /save token/i })).toBeNull(); + expect(screen.queryByLabelText(/personal access token/i)).toBeNull(); + + fireEvent.click(repair); + expect(openedTabs).toEqual(["machines"]); + expect(setToken).not.toHaveBeenCalled(); + } finally { + unsubscribe(); + } + }); + + it("still offers token entry when the store is readable and GitHub is simply not connected", async () => { + renderDialog(); + await reachConnectStep(); + + expect(await screen.findByRole("button", { name: /save token/i })).toBeTruthy(); + // Also proves the negative assertion in the unreadable case is not vacuous. + expect(screen.getByLabelText(/personal access token/i)).toBeTruthy(); + expect( + screen.queryByRole("button", { name: GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.action }), + ).toBeNull(); + }); + + // The open-time status is stale by the time a publish fails; a store that went + // unreadable in between would otherwise still be offered a replacement token. + it("re-reads the store verdict before offering a fix", async () => { + getStatus + .mockResolvedValueOnce(status()) + .mockResolvedValue(status({ credentialStoreUnreadable: true })); + renderDialog(); + await reachConnectStep(); + + expect( + await screen.findByRole("button", { name: GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.action }), + ).toBeTruthy(); + expect(screen.queryByRole("button", { name: /save token/i })).toBeNull(); + }); +}); diff --git a/apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.tsx b/apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.tsx index d05afe0e2..c2e15d76c 100644 --- a/apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.tsx +++ b/apps/desktop/src/renderer/components/projects/PublishToGitHubDialog.tsx @@ -19,8 +19,10 @@ import { } from "@phosphor-icons/react"; import type { PublishProjectResult } from "../../../shared/types"; import { extractCodeFromMessage } from "../../lib/codedError"; +import { openConnectionsPanel } from "../../lib/connectionsPanel"; import { extractError } from "../../lib/format"; import { describeGithubPatVerification } from "../../lib/githubIntegrationStatus"; +import { GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY } from "../../../shared/types"; import { fadeScale } from "../../lib/motion"; import { COLORS, @@ -115,6 +117,7 @@ export function PublishToGitHubDialog({ const [tokenDraft, setTokenDraft] = useState(""); const [tokenSaving, setTokenSaving] = useState(false); const [tokenError, setTokenError] = useState(null); + const [storeUnreadable, setStoreUnreadable] = useState(false); // Reset all local state whenever the dialog opens. useEffect(() => { @@ -131,10 +134,15 @@ export function PublishToGitHubDialog({ setTokenDraft(""); setTokenSaving(false); setTokenError(null); + setStoreUnreadable(false); void window.ade.github.getStatus({ forceRefresh: false }).then((status) => { - if (!cancelled && status.connected && status.userLogin) { + if (cancelled) return; + if (status.connected && status.userLogin) { setOwner(status.userLogin); } + // "Not connected" would be a guess here: an unreadable store reports no + // credential whether or not one is saved. + setStoreUnreadable(status.credentialStoreUnreadable === true); }).catch(() => {}); return () => { cancelled = true; @@ -162,6 +170,13 @@ export function PublishToGitHubDialog({ } catch (err) { const code = extractCodeFromMessage(err); if (code === "github_not_connected") { + // Re-read before offering a fix. "Not connected" is also what an + // unreadable store looks like, and the answer to that one is repair, + // not a replacement token. + const status = await window.ade.github + .getStatus({ forceRefresh: true }) + .catch(() => null); + if (status) setStoreUnreadable(status.credentialStoreUnreadable === true); setConnectMode(true); } else if (code === "remote_already_exists") { setError({ @@ -220,6 +235,13 @@ export function PublishToGitHubDialog({ void window.ade.app.openExternal(GITHUB_CLASSIC_TOKEN_NEW_URL); }, []); + // The repair lives in the Connections panel, which is a header popover — this + // modal has to get out of its way first. + const handleOpenConnections = useCallback(() => { + onOpenChange(false); + openConnectionsPanel("machines"); + }, [onOpenChange]); + const headerTitle = success ? "Publish to GitHub" : connectMode @@ -313,6 +335,8 @@ export function PublishToGitHubDialog({ error={tokenError} onOpenTokenLink={handleOpenTokenLink} onCancel={() => setConnectMode(false)} + storeUnreadable={storeUnreadable} + onOpenConnections={handleOpenConnections} /> ) : ( void; @@ -530,28 +556,57 @@ function ConnectBody({ error: string | null; onOpenTokenLink: () => void; onCancel: () => void; + storeUnreadable: boolean; + onOpenConnections: () => void; }) { const detected = tokenDraft.trim() ? detectTokenType(tokenDraft.trim()) : null; + const notice = ( +
+ + + {storeUnreadable + ? `${GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.title}. ${GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.detail}` + : "GitHub is not connected. Run gh auth login with repo and workflow scopes, or paste a personal access token."} + +
+ ); + + // Unreadable is NOT "not connected": the saved sign-in may still be on disk, + // just unopenable here. Saving a token over it would destroy a credential + // that repair can recover, so this state offers only the repair route. + if (storeUnreadable) { + return ( +
+ {notice} +
+ + +
+
+ ); + } + return (
-
- - GitHub is not connected. Run gh auth login with repo and workflow scopes, or paste a personal access token. -
+ {notice} + {credentialStoreUnreadable ? ( +
+
+ {GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.title} +
+
{GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.detail}
+ +
+ ) : null} + {credentialFallback ? (
{ }); describe("describeGithubCliBanner", () => { + // Regression: an unreadable credential store arrives with tokenStored:false — + // identical to a fresh install — and the banner used to say "GitHub CLI or + // token not connected", pointing the user at a Connect flow that would write + // over credentials that are still on disk. + it("says the sign-in is unreadable rather than never connected", () => { + const banner = describeGithubCliBanner(makeCliStatus({ + tokenStored: false, + credentialStoreUnreadable: true, + })); + + expect(banner.subState).toBe("credential-store-unreadable"); + expect(banner.title).toBe(GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY.title); + expect(banner.title).not.toContain("not connected"); + expect(banner.detail).toContain("Settings → Connections"); + // The Repair control lives in Connections, not on the GitHub settings card. + expect(banner.target).toBe("connections"); + }); + + it("outranks a stale auth failure with the unreadable store", () => { + const banner = describeGithubCliBanner(makeCliStatus({ + tokenStored: false, + credentialStoreUnreadable: true, + authFailure: { kind: "invalid_token", message: "Bad credentials", retryAt: null }, + })); + + expect(banner.subState).toBe("credential-store-unreadable"); + }); + + it("still reports a genuinely absent credential as not connected", () => { + const banner = describeGithubCliBanner(makeCliStatus({ + tokenStored: false, + credentialStoreUnreadable: false, + })); + + expect(banner.subState).toBe("no-token"); + expect(banner.title).toBe("GitHub CLI or token not connected"); + // Every banner states its destination; only the unreadable store leaves the + // GitHub card, so a caller never has to re-derive the default. + expect(banner.target).toBe("github-settings"); + }); + it("does not tell a signed-in rate-limited user to reconnect", () => { const banner = describeGithubCliBanner(makeCliStatus({ authFailure: { diff --git a/apps/desktop/src/renderer/lib/githubIntegrationStatus.ts b/apps/desktop/src/renderer/lib/githubIntegrationStatus.ts index c1f408611..fa1a27ba2 100644 --- a/apps/desktop/src/renderer/lib/githubIntegrationStatus.ts +++ b/apps/desktop/src/renderer/lib/githubIntegrationStatus.ts @@ -4,6 +4,7 @@ import type { GitHubSetTokenResult, GitHubStatus, } from "../../shared/types"; +import { GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY } from "../../shared/types"; import { GITHUB_STATUS_PAGE_URL, githubServiceAffectedLabel, @@ -379,18 +380,36 @@ export function describeGithubPatVerification(result: GitHubSetTokenResult): { }; } +/** + * Where the banner's single CTA has to land. Everything about GitHub itself is + * fixed in the GitHub settings card; an unreadable credential store is not a + * GitHub problem at all, and its only repair control lives in Connections. + */ +export type GithubBannerTarget = "github-settings" | "connections"; + export function describeGithubCliBanner(status: GitHubStatus): { subState: string; title: string; detail: string; action: string; + /** Always stated: an omitted target left every caller to re-derive the default. */ + target: GithubBannerTarget; } { + // First, and ahead of `!tokenStored`: an unreadable store returns an EMPTY + // view, so every other conclusion below is drawn from credentials ADE could + // not read. Saying "not connected" here is what invited users to reconnect + // over working credentials. + if (status.credentialStoreUnreadable === true) { + const { subState, title, detail, action } = GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY; + return { subState, title, detail, action, target: "connections" }; + } if (!status.tokenStored) { return { subState: "no-token", title: "GitHub CLI or token not connected", detail: "Connect the GitHub CLI (gh auth login) or add a personal access token so ADE can run git and PR operations.", action: "Connect GitHub", + target: "github-settings", }; } // Below this point every state is inferred from GitHub's answers, so an @@ -402,18 +421,27 @@ export function describeGithubCliBanner(status: GitHubStatus): { // It stays so that any other caller — or a future refactor of that // suppression — cannot silently reintroduce the credential accusation. const outage = describeGithubOutage(status); - if (outage) return outage; + // Targeted at the GitHub card like every other GitHub-blaming state: an + // outage is not a credential-store problem, so Connections has nothing for + // it. (The Repair control there is only for a store ADE cannot read.) The + // action label is rewritten because this shape drops `actionUrl` — a button + // that navigates in-app must not read as a link to githubstatus.com; the + // settings card itself carries the real external incident link. + if (outage) return { ...outage, action: "Open GitHub settings", target: "github-settings" }; if (status.connected && !githubStatusHasWriteCredential(status)) { return { subState: "no-write-credential", title: "GitHub write access isn't connected", detail: "The ADE GitHub App can keep pull request data fresh, but GitHub CLI or a personal access token is needed for create, update, and merge actions.", action: "Connect GitHub", + target: "github-settings", }; } const authFailure = describeGithubAuthFailure(status); + // Every auth failure is fixed on the GitHub card: the credential ADE holds is + // readable, it is the account behind it that GitHub has an objection to. if (authFailure) { - return authFailure; + return { ...authFailure, target: "github-settings" }; } if (status.tokenType === "fine-grained" && status.repoAccessOk === false) { const repoLabel = status.repo ? `${status.repo.owner}/${status.repo.name}` : "this repository"; @@ -422,6 +450,7 @@ export function describeGithubCliBanner(status: GitHubStatus): { title: `GitHub token can't access ${repoLabel}`, detail: "Your fine-grained token is valid but hasn't been granted this repository. Update its repository access.", action: "Fix GitHub auth", + target: "github-settings", }; } return { @@ -429,6 +458,7 @@ export function describeGithubCliBanner(status: GitHubStatus): { title: "GitHub token is missing permissions", detail: "Your GitHub token lacks the scopes ADE needs. Reconnect it with repo and workflow access.", action: "Fix GitHub auth", + target: "github-settings", }; } diff --git a/apps/desktop/src/renderer/state/appStore.test.ts b/apps/desktop/src/renderer/state/appStore.test.ts index 43e7ae744..45518ff38 100644 --- a/apps/desktop/src/renderer/state/appStore.test.ts +++ b/apps/desktop/src/renderer/state/appStore.test.ts @@ -1556,25 +1556,54 @@ describe("appStore", () => { }); }); + it("keeps the brain's own words for a code this screen has nothing better to say about", async () => { + // `storage_read_failed` names the unreadable file and the cloud provider + // holding it. Replacing that with a generic line — and demoting the real + // sentence into the collapsed `detail` fold — was the regression. + const brainMessage = + "ADE couldn't read this project's data at /tmp/project/.ade/ade.db. " + + "Move the project out of iCloud Drive, Dropbox, or OneDrive, then try again."; + (window.ade.project.switchToPath as any).mockRejectedValueOnce( + new Error( + `Error invoking remote method 'ade.project.switchToPath': Error: storage_read_failed: ${brainMessage}`, + ), + ); + + await expect( + useAppStore.getState().switchProjectToPath("/tmp/project"), + ).rejects.toThrow("storage_read_failed"); + + const transitionError = useAppStore.getState().projectTransitionError; + expect(transitionError).toEqual({ + code: "storage_read_failed", + message: brainMessage, + rootPath: "/tmp/project", + }); + // The brain's sentence is now the headline, so it must hold the same + // no-jargon bar the generic copy did. + expectNoJargon(transitionError?.message ?? ""); + }); + it.each([ "provider_thread_missing", "provider_resume_failed", "continuity_reconstruction_required", "optional_mcp_failed", - ] as const)("uses calm copy for the recognized %s recovery code", async (code) => { + ] as const)("falls back to calm copy when %s arrives with nothing to say", async (code) => { + // A bare code and an empty message is the only case a generic line is an + // improvement on — the alternative is a blank recovery screen. (window.ade.project.switchToPath as any).mockRejectedValueOnce( - new Error(`Error invoking remote method 'ade.project.switchToPath': Error: ${code}: raw socket detail`), + Object.assign(new Error(""), { code }), ); await expect( useAppStore.getState().switchProjectToPath("/tmp/project"), - ).rejects.toThrow(code); + ).rejects.toThrow(); const transitionError = useAppStore.getState().projectTransitionError; expect(transitionError).toEqual({ code, message: "ADE ran into a problem with this project.", - detail: "raw socket detail", rootPath: "/tmp/project", }); expectNoJargon(transitionError?.message ?? ""); diff --git a/apps/desktop/src/renderer/state/appStore.ts b/apps/desktop/src/renderer/state/appStore.ts index cf967ddc6..43e78212a 100644 --- a/apps/desktop/src/renderer/state/appStore.ts +++ b/apps/desktop/src/renderer/state/appStore.ts @@ -7,7 +7,7 @@ import { recentProjectStateKey } from "../../shared/projectIdentity"; import { THIS_MACHINE_ID } from "../../shared/machineIdentity"; import { MODEL_REGISTRY, type ModelDescriptor } from "../../shared/modelRegistry"; import { parseCodedErrorMessage } from "../lib/codedError"; -import { toAdeRecoveryErrorCode } from "../../shared/types/recovery"; +import { toAdeRecoveryErrorCode, type AdeRecoveryErrorCode } from "../../shared/types/recovery"; import { isWebClientMode } from "../lib/webClientMode"; import { getAiStatusCached, invalidateAiDiscoveryCache } from "../lib/aiDiscoveryCache"; import { hasConfiguredAiProvider } from "../lib/aiProviderStatus"; @@ -1393,6 +1393,39 @@ function withPreservedLaneStatus( : lane; } +/** + * What to say instead of the brain's own words, per coded failure. + * + * A code is listed here only when this screen can say something the brain's + * message does not — a libuv errno, a socket path, a migration state. Anything + * NOT listed keeps the brain's own sentence as the headline, because the brain + * is the only party that knows the specifics: `storage_read_failed` is built by + * `storageUnreadableMessage`, which names the file that could not be read and + * tells the user to move it out of iCloud Drive/Dropbox/OneDrive, and a generic + * paraphrase here would push that into the collapsed details fold. + * + * So the rule is exactly three cases: + * - listed code → this file's line, brain's message kept as `detail` + * - unlisted code, message → the brain's message, verbatim, with no `detail` + * - unlisted code, NO message → `GENERIC_RECOVERY_MESSAGE`, the only case where + * a bare code would otherwise leave a blank screen + * No code at all falls through to the brain's message as well. + */ +const RECOVERY_MESSAGE_BY_CODE: Partial> = { + disk_full: + "Your computer ran out of storage while ADE was saving project data. Free up space, then try again.", + brain_crash_looping: "ADE's background service needs a repair before this project can open.", + migration_incomplete: "ADE's background service needs a repair before this project can open.", + migration_unknown_state: "ADE's background service needs a repair before this project can open.", + insufficient_headroom: "ADE's background service could not open this project.", + db_integrity: "ADE's background service could not open this project.", + brain_not_installed: "ADE's background service could not open this project.", + socket_stale_no_owner: "ADE's background service could not open this project.", + socket_owned_by_other: "ADE's background service could not open this project.", +}; + +const GENERIC_RECOVERY_MESSAGE = "ADE ran into a problem with this project."; + function formatProjectTransitionError( kind: "opening" | "switching" | "closing", error: unknown, @@ -1421,21 +1454,9 @@ function formatProjectTransitionError( }; } const code = toAdeRecoveryErrorCode(parsed.code); - const recoveryMessage = code === "disk_full" - ? "Your computer ran out of storage while ADE was saving project data. Free up space, then try again." - : code === "brain_crash_looping" || code === "migration_incomplete" || code === "migration_unknown_state" - ? "ADE's background service needs a repair before this project can open." - : code && [ - "insufficient_headroom", - "db_integrity", - "brain_not_installed", - "socket_stale_no_owner", - "socket_owned_by_other", - ].includes(code) - ? "ADE's background service could not open this project." - : code - ? "ADE ran into a problem with this project." - : null; + const recoveryMessage = code + ? RECOVERY_MESSAGE_BY_CODE[code] ?? (raw ? null : GENERIC_RECOVERY_MESSAGE) + : null; const fallback = raw.length > 0 ? raw : "Project action failed."; return { message: recoveryMessage ?? fallback, diff --git a/apps/desktop/src/shared/accountMachineRefusal.test.ts b/apps/desktop/src/shared/accountMachineRefusal.test.ts new file mode 100644 index 000000000..8f22aeedd --- /dev/null +++ b/apps/desktop/src/shared/accountMachineRefusal.test.ts @@ -0,0 +1,51 @@ +import { describe, expect, it } from "vitest"; +import { readAccountRefusalCode } from "./accountMachineRefusal"; +import { createSyncAccountDirectoryHealth } from "./types/sync"; + +describe("readAccountRefusalCode", () => { + it("decodes the named refusal codes off a live 403", () => { + expect(readAccountRefusalCode(createSyncAccountDirectoryHealth("http_error", null, { + lastHttpStatus: 403, + lastHttpReason: "machine_revoked", + }))).toBe("machine_revoked"); + expect(readAccountRefusalCode(createSyncAccountDirectoryHealth("http_error", null, { + lastHttpStatus: 403, + lastHttpReason: "pairing_authentication_required", + }))).toBe("pairing_authentication_required"); + }); + + it("reports an unrecognised 403 as `other` without leaking the prose", () => { + expect(readAccountRefusalCode(createSyncAccountDirectoryHealth("http_error", null, { + lastHttpStatus: 403, + lastHttpReason: "your seat expired on 2026-08-18, contact ada@example.com", + }))).toBe("other"); + }); + + it("is not a refusal for a 401, a 5xx, or no health at all", () => { + // A 401 is an authentication problem with a different repair; counting it + // as a register refusal mis-attributes the incident and hides the real one. + expect(readAccountRefusalCode(createSyncAccountDirectoryHealth("http_error", null, { + lastHttpStatus: 401, + lastHttpReason: "machine_revoked", + }))).toBeNull(); + expect(readAccountRefusalCode(createSyncAccountDirectoryHealth("http_error", null, { + lastHttpStatus: 503, + }))).toBeNull(); + expect(readAccountRefusalCode(null)).toBeNull(); + expect(readAccountRefusalCode(undefined)).toBeNull(); + }); + + it("ignores stale 403 refusal reasons outside http_error", () => { + // Health is a STATE. Once the publisher moves on to a transport or token + // failure, a 403 still sitting in the status fields describes an attempt + // that is no longer the reason this machine is unpublished — and the CLI's + // pairing-recovery loop reads this decoder directly, so a stale read there + // spends real repair budget arguing with nothing. + for (const state of ["token_timeout", "transport_error", "timeout", "published"] as const) { + expect(readAccountRefusalCode(createSyncAccountDirectoryHealth(state, null, { + lastHttpStatus: 403, + lastHttpReason: "machine_revoked", + }))).toBeNull(); + } + }); +}); diff --git a/apps/desktop/src/shared/accountMachineRefusal.ts b/apps/desktop/src/shared/accountMachineRefusal.ts new file mode 100644 index 000000000..994737470 --- /dev/null +++ b/apps/desktop/src/shared/accountMachineRefusal.ts @@ -0,0 +1,55 @@ +import type { SyncAccountDirectoryHealth } from "./types/sync"; + +/** + * Why the account directory refused to register THIS machine, decoded once. + * + * Every surface that reacts to a refusal — the auto-recovery loop in the brain, + * the desktop's reliability telemetry — reads the same two fields off the same + * publisher health snapshot, and each used to decode them itself. One home, so + * "what counts as a refusal" cannot drift between the thing that repairs it and + * the thing that reports it. + */ + +/** The codes the directory answers with; see `apps/account-directory/src/directory.ts`. */ +export const ACCOUNT_MACHINE_REFUSAL_CODES = [ + "machine_revoked", + "pairing_authentication_required", +] as const; + +export type AccountMachineRefusalCode = typeof ACCOUNT_MACHINE_REFUSAL_CODES[number]; + +/** + * The known code, `"other"` for a 403 this build has no name for, or null when + * the last attempt was not a refusal at all. + * + * 403 ONLY, deliberately. A 401 is an authentication problem — the token this + * machine presented was not accepted — which is a different failure with a + * different repair, and counting it as a register refusal both mis-attributes + * the incident and hides the auth failure behind it. A refusal is the + * directory looking at a valid caller and saying no. + * + * `lastHttpReason` on the unrecognised path is server-supplied prose and must + * never travel; `"other"` is all a caller may learn from it. That an + * unrecognised 403 still resolves to something rather than to null is the + * point: "the directory turned this machine away for a reason this build + * cannot name" is exactly the fact the last incident needed and could not get. + * + * Anything else — a timeout, a 5xx, a transport failure — is not a refusal. + * + * The state gate is the other half of that. Health is a STATE, not an event: + * `lastHttpStatus` / `lastHttpReason` describe the attempt currently failing + * only while the state is `http_error`, and every other state either carries + * its own status or leaves the pair behind. Decoding a refusal off one of those + * would date-stamp a rejection that is no longer the reason this machine is + * unpublished — and on the CLI's pairing-recovery path that is not just a + * mislabelled metric, it is a repair episode started for nothing. + */ +export function readAccountRefusalCode( + health: SyncAccountDirectoryHealth | null | undefined, +): AccountMachineRefusalCode | "other" | null { + if (!health || health.state !== "http_error" || health.lastHttpStatus !== 403) return null; + const reason = typeof health.lastHttpReason === "string" ? health.lastHttpReason.trim() : ""; + return (ACCOUNT_MACHINE_REFUSAL_CODES as readonly string[]).includes(reason) + ? reason as AccountMachineRefusalCode + : "other"; +} diff --git a/apps/desktop/src/shared/codedError.test.ts b/apps/desktop/src/shared/codedError.test.ts index 5402c2cb3..fe830b397 100644 --- a/apps/desktop/src/shared/codedError.test.ts +++ b/apps/desktop/src/shared/codedError.test.ts @@ -3,7 +3,9 @@ import { codedError, encodeCodedErrorMessage, extractCodeFromMessage, + isErrnoLikeCode, parseCodedErrorMessage, + UNKNOWN_SYSTEM_ERRNO_PATTERN, } from "./codedError"; describe("codedError wire format", () => { @@ -20,6 +22,28 @@ describe("codedError wire format", () => { expect(parsed.message).toBe("Restart ADE, then run repair again."); }); + it("recognizes a code the brain attached, behind the runtime RPC wrapper", () => { + // The exact chain a daemon-side failure travels: Electron IPC wraps the + // main process's error, which wraps the runtime client's, which wraps the + // brain's coded reply. Without the innermost strip the code is invisible + // and the user gets a generic "couldn't open this project". + const parsed = parseCodedErrorMessage(new Error( + "Error invoking remote method 'ade.localRuntime.callAction': Error: " + + "Remote ADE service method ade/actions/call failed (code -32603): " + + "storage_read_failed: ADE couldn't read this project's data at /tmp/p/.ade/ade.db.", + )); + expect(parsed.code).toBe("storage_read_failed"); + expect(parsed.message).toBe("ADE couldn't read this project's data at /tmp/p/.ade/ade.db."); + }); + + it("leaves an uncoded runtime failure without a code", () => { + const parsed = parseCodedErrorMessage(new Error( + "Remote ADE service method attention.call failed (code -32601): Method not found", + )); + expect(parsed.code).toBeUndefined(); + expect(parsed.message).toBe("Method not found"); + }); + it("carries an encoded rootPath the renderer never saw, without leaking it into the message", () => { const rootPath = "/Users/dev/Projects/My App"; const wire = encodeCodedErrorMessage("disk_full", "Free up space, then try again.", { rootPath }); @@ -52,3 +76,44 @@ describe("codedError wire format", () => { expect(parseCodedErrorMessage(new Error("just a message")).code).toBeUndefined(); }); }); + +describe("isErrnoLikeCode", () => { + it("claims platform codes, including the Node internal ones that quote paths", () => { + for (const code of [ + "ENOENT", + "EDEADLK", + "ECONNRESET", + "ERR_MODULE_NOT_FOUND", + "ERR_FS_EISDIR", + "ERR_INVALID_ARG_TYPE", + "MODULE_NOT_FOUND", + ]) { + expect(isErrnoLikeCode(code)).toBe(true); + } + }); + + it("leaves ADE's own verdicts alone, so they still cross a boundary intact", () => { + for (const code of [ + "storage_read_failed", + "disk_full", + "brain_not_installed", + "migration_incomplete", + "", + " ", + null, + undefined, + 42, + ]) { + expect(isErrnoLikeCode(code)).toBe(false); + } + }); +}); + +describe("UNKNOWN_SYSTEM_ERRNO_PATTERN", () => { + it("matches the errno libuv could not name, in either casing", () => { + expect(UNKNOWN_SYSTEM_ERRNO_PATTERN.test("Unknown system error -11: Unknown system error -11, read")) + .toBe(true); + expect(UNKNOWN_SYSTEM_ERRNO_PATTERN.test("unknown system error 11")).toBe(true); + expect(UNKNOWN_SYSTEM_ERRNO_PATTERN.test("ADE couldn't read this project's data.")).toBe(false); + }); +}); diff --git a/apps/desktop/src/shared/codedError.ts b/apps/desktop/src/shared/codedError.ts index a99511ca6..b1c4667f9 100644 --- a/apps/desktop/src/shared/codedError.ts +++ b/apps/desktop/src/shared/codedError.ts @@ -14,6 +14,44 @@ export function codedError(message: string, code: TCode): return Object.assign(new Error(message), { code }); } +/** + * libuv can only name the errnos it has a mapping for. macOS returns EDEADLK + * (errno 11) for a File-Provider read it cannot satisfy, which libuv does not + * map, so the error reaches us as the uninterpretable "Unknown system error + * -11: Unknown system error -11, read". Anything in that shape is a raw + * platform failure, never a verdict a service authored. + */ +export const UNKNOWN_SYSTEM_ERRNO_PATTERN = /unknown system error\s+-?\d+/i; + +/** libuv/POSIX errno codes: `ENOENT`, `EDEADLK`, `ECONNRESET`, … */ +const ERRNO_CODE_PATTERN = /^E[A-Z0-9]+$/; + +/** + * Node's own internal codes: `ERR_MODULE_NOT_FOUND`, `ERR_FS_EISDIR`, + * `ERR_INVALID_ARG_TYPE`, … plus the one legacy code that predates the prefix. + * + * They belong with the errnos and not with service verdicts: their messages are + * written for whoever is reading a stack trace and routinely quote an absolute + * path ("Cannot find module '/Users/…'"), so a boundary that treats them as + * authored copy forwards a filesystem path to a caller that must never see one. + */ +const NODE_INTERNAL_CODE_PATTERN = /^ERR_[A-Z0-9_]+$/; +const LEGACY_NODE_MODULE_NOT_FOUND_CODE = "MODULE_NOT_FOUND"; + +/** + * Whether an `Error.code` was attached by the platform rather than chosen by + * ADE. ADE's own codes are lowercase snake_case (`storage_read_failed`), so the + * two vocabularies cannot collide. + */ +export function isErrnoLikeCode(code: unknown): boolean { + if (typeof code !== "string") return false; + const trimmed = code.trim(); + if (!trimmed) return false; + return ERRNO_CODE_PATTERN.test(trimmed) + || NODE_INTERNAL_CODE_PATTERN.test(trimmed) + || trimmed === LEGACY_NODE_MODULE_NOT_FOUND_CODE; +} + export function encodeCodedErrorMessage(code: string, message: string, meta?: { rootPath?: string }): string { const base = `${code}: ${message}`; return meta?.rootPath ? `${base}${ROOT_PATH_DELIMITER}${meta.rootPath}` : base; @@ -23,6 +61,12 @@ export function stripElectronErrorWrapper(message: string): string { return message .replace(/^Error invoking remote method '[^']+':\s*/i, "") .replace(/^Error:\s*/i, "") + // The runtime RPC client wraps every daemon-side failure in "Remote ADE + // service method failed (code ): …". A code the brain attached is + // behind that wrapper, and without stripping it no brain-side failure can + // ever be recognised by its code — which is how a project whose data files + // were unreadable reached the user as a raw libuv errno. + .replace(/^Remote ADE service method \S+ failed(?:\s*\(code -?\d+\))?:\s*/i, "") .trim(); } diff --git a/apps/desktop/src/shared/diagnosticsUpload.test.ts b/apps/desktop/src/shared/diagnosticsUpload.test.ts new file mode 100644 index 000000000..3e5c14ef7 --- /dev/null +++ b/apps/desktop/src/shared/diagnosticsUpload.test.ts @@ -0,0 +1,183 @@ +import { describe, expect, it, vi } from "vitest"; +import { DEFAULT_ADE_ACCOUNT_DIRECTORY_URL } from "./accountDirectory"; +import { + describeDiagnosticUploadFailure, + diagnosticsUploadUrl, + MAX_DIAGNOSTIC_UPLOAD_BYTES, + resolveDiagnosticsUploadBaseUrl, + uploadDiagnosticReport, +} from "./diagnosticsUpload"; + +const REPORT = "# ADE diagnostic report\n\n- surface: brain_repair\n- note: already redacted\n"; +const BASE_URL = "https://directory.example"; + +function capture(response: Response) { + const calls: Array<{ url: string; init: RequestInit }> = []; + const fetchImpl = (async (input: RequestInfo | URL, init?: RequestInit) => { + calls.push({ url: String(input), init: init ?? {} }); + return response; + }) as unknown as typeof fetch; + return { calls, fetchImpl }; +} + +function ok(id = "aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee"): Response { + return new Response(JSON.stringify({ ok: true, id }), { status: 200 }); +} + +function sentBody(init: RequestInit): Record { + return JSON.parse(String(init.body)) as Record; +} + +describe("resolveDiagnosticsUploadBaseUrl", () => { + it("defaults to ADE's hosted directory and honours a valid override", () => { + expect(resolveDiagnosticsUploadBaseUrl()).toBe(DEFAULT_ADE_ACCOUNT_DIRECTORY_URL); + expect(resolveDiagnosticsUploadBaseUrl(" ")).toBe(DEFAULT_ADE_ACCOUNT_DIRECTORY_URL); + expect(resolveDiagnosticsUploadBaseUrl("https://self.hosted.example")) + .toBe("https://self.hosted.example"); + }); + + it("falls back rather than posting a report to a malformed destination", () => { + expect(resolveDiagnosticsUploadBaseUrl("not a url")) + .toBe(DEFAULT_ADE_ACCOUNT_DIRECTORY_URL); + expect(resolveDiagnosticsUploadBaseUrl("https://x.dev/?leak=1")) + .toBe(DEFAULT_ADE_ACCOUNT_DIRECTORY_URL); + }); +}); + +describe("uploadDiagnosticReport", () => { + it("posts the exact report the clipboard holds, with the caller's metadata", async () => { + const { calls, fetchImpl } = capture(ok()); + const result = await uploadDiagnosticReport({ + report: REPORT, + installId: "install-5", + appVersion: "1.2.60", + baseUrl: `${BASE_URL}/`, + fetchImpl, + }); + + expect(result).toEqual({ + ok: true, + id: "aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee", + reference: "aaaaaaaa", + }); + expect(calls).toHaveLength(1); + expect(calls[0]!.url).toBe(`${BASE_URL}/diagnostics/upload`); + expect(calls[0]!.init.method).toBe("POST"); + // Redaction is upstream: whatever this module was handed is exactly what + // goes on the wire. If this ever stops matching, the user is being shown + // one thing and sending another. + expect(sentBody(calls[0]!.init)).toEqual({ + report: REPORT, + installId: "install-5", + appVersion: "1.2.60", + }); + }); + + it("sends the account token only when a caller has one", async () => { + // `ade report-issue --send` reads the machine's credential store; the + // renderer cannot, and uploads anonymously against the default directory. + const authed = capture(ok()); + await uploadDiagnosticReport({ + report: REPORT, + baseUrl: BASE_URL, + token: "clerk-token", + fetchImpl: authed.fetchImpl, + }); + expect(new Headers(authed.calls[0]!.init.headers).get("authorization")) + .toBe("Bearer clerk-token"); + + const anonymous = capture(ok()); + await uploadDiagnosticReport({ + report: REPORT, + baseUrl: resolveDiagnosticsUploadBaseUrl(), + fetchImpl: anonymous.fetchImpl, + }); + expect(new Headers(anonymous.calls[0]!.init.headers).get("authorization")).toBeNull(); + expect(anonymous.calls[0]!.url.startsWith(DEFAULT_ADE_ACCOUNT_DIRECTORY_URL)).toBe(true); + expect(sentBody(anonymous.calls[0]!.init)).toEqual({ report: REPORT }); + }); + + it("refuses an oversized report locally instead of burning a daily upload", async () => { + const { calls, fetchImpl } = capture(ok()); + await expect(uploadDiagnosticReport({ + report: "x".repeat(MAX_DIAGNOSTIC_UPLOAD_BYTES + 1), + baseUrl: BASE_URL, + fetchImpl, + })).resolves.toEqual({ ok: false, reason: "too_large" }); + expect(calls).toHaveLength(0); + }); + + it("never puts a body on the wire that the Worker's identical cap would refuse", async () => { + // The Worker bounds the whole request body, so a report sitting exactly at + // the cap is already over it once the JSON envelope is added. Caught here, + // it costs nothing; missed, it is a 413 that spends a request. + const atCap = capture(ok()); + await expect(uploadDiagnosticReport({ + report: "x".repeat(MAX_DIAGNOSTIC_UPLOAD_BYTES), + baseUrl: BASE_URL, + fetchImpl: atCap.fetchImpl, + })).resolves.toEqual({ ok: false, reason: "too_large" }); + expect(atCap.calls).toHaveLength(0); + + // The largest report that does fit is still sent, and what goes out is + // within the cap the Worker applies to the same bytes. + const envelope = new TextEncoder().encode(JSON.stringify({ report: "" })).byteLength; + const fits = capture(ok()); + await expect(uploadDiagnosticReport({ + report: "x".repeat(MAX_DIAGNOSTIC_UPLOAD_BYTES - envelope), + baseUrl: BASE_URL, + fetchImpl: fits.fetchImpl, + })).resolves.toMatchObject({ ok: true }); + expect(new TextEncoder().encode(String(fits.calls[0]!.init.body)).byteLength) + .toBe(MAX_DIAGNOSTIC_UPLOAD_BYTES); + }); + + it("turns every refusal into a reason, never an exception on an error screen", async () => { + for ( + const [status, reason] of [ + [413, "too_large"], + [429, "rate_limited"], + [503, "unavailable"], + [400, "rejected"], + [500, "rejected"], + ] as const + ) { + const { fetchImpl } = capture(new Response("{}", { status })); + await expect(uploadDiagnosticReport({ report: REPORT, baseUrl: BASE_URL, fetchImpl })) + .resolves.toEqual({ ok: false, reason }); + } + + const thrown = vi.fn(async () => { + throw new Error("getaddrinfo ENOTFOUND"); + }) as unknown as typeof fetch; + await expect(uploadDiagnosticReport({ report: REPORT, baseUrl: BASE_URL, fetchImpl: thrown })) + .resolves.toEqual({ ok: false, reason: "network" }); + }); + + it("treats an unusable success body as a failure rather than inventing a reference", async () => { + const { fetchImpl } = capture(new Response("not json", { status: 200 })); + await expect(uploadDiagnosticReport({ report: REPORT, baseUrl: BASE_URL, fetchImpl })) + .resolves.toEqual({ ok: false, reason: "rejected" }); + + const noId = capture(new Response(JSON.stringify({ ok: true }), { status: 200 })); + await expect(uploadDiagnosticReport({ + report: REPORT, + baseUrl: BASE_URL, + fetchImpl: noId.fetchImpl, + })).resolves.toEqual({ ok: false, reason: "rejected" }); + }); + + it("normalizes the base URL", () => { + expect(diagnosticsUploadUrl("https://x.dev")).toBe("https://x.dev/diagnostics/upload"); + expect(diagnosticsUploadUrl("https://x.dev///")).toBe("https://x.dev/diagnostics/upload"); + expect(diagnosticsUploadUrl(" https://x.dev ")).toBe("https://x.dev/diagnostics/upload"); + }); + + it("explains failures without a status code or a file path", () => { + for (const reason of ["too_large", "rate_limited", "unavailable", "rejected", "network"] as const) { + const sentence = describeDiagnosticUploadFailure(reason); + expect(sentence.length).toBeGreaterThan(0); + expect(sentence).not.toMatch(/\d{3}|http|\//); + } + }); +}); diff --git a/apps/desktop/src/shared/diagnosticsUpload.ts b/apps/desktop/src/shared/diagnosticsUpload.ts new file mode 100644 index 000000000..48f8dedf0 --- /dev/null +++ b/apps/desktop/src/shared/diagnosticsUpload.ts @@ -0,0 +1,194 @@ +import { + DEFAULT_ADE_ACCOUNT_DIRECTORY_URL, + resolveTrustedAccountDirectoryBaseUrl, +} from "./accountDirectory"; + +/** + * "Send to ADE": posts an already-redacted diagnostic report to the account + * directory Worker's `POST /diagnostics/upload`. + * + * ONE home for the upload, shared by every surface that offers it — the desktop + * renderer's "Report issue" button, the desktop main process, and + * `ade report-issue --send`. It lives under `apps/desktop/src/shared` because + * that is the only directory all three can import: Vite's dev server refuses to + * serve files outside `apps/desktop`, so the renderer cannot reach the CLI's + * tree, while the CLI already imports from here. + * + * Deliberately free of Node built-ins and of `import.meta`, so the identical + * module loads in the renderer bundle, in the main process, and in the CLI. + * Plain `fetch` with no platform branches: Windows behaves exactly as macOS. + * + * Everything private is stripped by `redactDiagnosticText` before a report + * exists at all, so this module deliberately does nothing to the text: it posts + * the exact bytes the user could have pasted themselves. Any transformation + * here would mean the thing that was sent is not the thing that was shown. + * + * Surface-specific concerns stay with their surface: the CLI reads an account + * token out of the machine's credential store in `sendDiagnosticReport`, and + * the renderer has none (access tokens live in the brain's store and are not + * exposed over the preload bridge), so desktop uploads are anonymous. + */ + +export const DIAGNOSTICS_UPLOAD_PATH = "/diagnostics/upload"; + +/** + * Mirror of `MAX_DIAGNOSTIC_REPORT_BYTES` in `apps/account-directory/src/diagnostics.ts`. + * + * Same number, deliberately measured against a different thing on each side — + * and they still agree. The Worker has to bound the bytes as they arrive, before + * anything is parsed, so its cap is on the WHOLE request body; this side knows + * what it is about to send, so it weighs the serialized body too rather than the + * report inside it. Checking only the report would pass a report sitting exactly + * at the cap and then have the `{"report":...}` envelope and its escaping push + * the request over on the wire — a 413 a round-trip later for something that was + * knowable here. + */ +export const MAX_DIAGNOSTIC_UPLOAD_BYTES = 512 * 1024; + +const DEFAULT_UPLOAD_TIMEOUT_MS = 20_000; + +/** + * Why a send failed, in terms a UI can turn into one short sentence. Never a + * server string: the caller is a user who already hit one failure, and a raw + * status line is not an improvement on "couldn't send". + */ +export type DiagnosticUploadFailure = + | "too_large" + | "rate_limited" + | "unavailable" + | "rejected" + | "network"; + +export type DiagnosticUploadResult = + | { ok: true; id: string; reference: string } + | { ok: false; reason: DiagnosticUploadFailure }; + +/** + * Where DESKTOP uploads go. + * + * The desktop's own account origin is resolved in the main process by the + * account bridge, which the renderer cannot reach, so the base is derived here + * from the same two sources that bridge uses: an explicit override, else ADE's + * hosted directory. A malformed override falls back to the default rather than + * silently posting a report somewhere else. + * + * The CLI does NOT go through this: it already resolves the directory origin + * the way the brain does (project secret, machine override, else the official + * URL for the issuer) and hands the result to `uploadDiagnosticReport`. Passing + * it through here as well would silently redirect a self-hosted machine's + * report — and its account token — to ADE's directory instead. + */ +export function resolveDiagnosticsUploadBaseUrl(override?: string | null): string { + return resolveTrustedAccountDirectoryBaseUrl(override) ?? DEFAULT_ADE_ACCOUNT_DIRECTORY_URL; +} + +export function diagnosticsUploadUrl(baseUrl: string): string { + return `${baseUrl.trim().replace(/\/+$/, "")}${DIAGNOSTICS_UPLOAD_PATH}`; +} + +/** + * The short handle a user reads back to support. Full uuids are unreadable over + * a phone call and the prefix is enough to find the object. + */ +export function diagnosticReference(id: string): string { + return id.trim().slice(0, 8); +} + +export type DiagnosticUploadRequest = { + /** The redacted report, byte-for-byte as the user sees it. */ + report: string; + installId?: string | null; + appVersion?: string | null; + /** + * Clerk access token, when a caller has one. `ade report-issue --send` reads + * the machine's credential store and does send one; the renderer cannot and + * uploads anonymously against the install id the report already carries. + */ + token?: string | null; + /** Already resolved by the caller; see `resolveDiagnosticsUploadBaseUrl`. */ + baseUrl: string; + fetchImpl?: typeof fetch; + timeoutMs?: number; +}; + +export async function uploadDiagnosticReport( + request: DiagnosticUploadRequest, +): Promise { + const report = request.report; + if (!report.trim()) return { ok: false, reason: "rejected" }; + + const body = JSON.stringify({ + report, + ...(request.installId ? { installId: request.installId } : {}), + ...(request.appVersion ? { appVersion: request.appVersion } : {}), + }); + // Checked here as well as on the Worker so an oversized report fails without + // spending one of the user's few daily uploads on a doomed request. The exact + // bytes that would go on the wire, so this side never sends something the + // Worker's identical cap would refuse. + if (new TextEncoder().encode(body).byteLength > MAX_DIAGNOSTIC_UPLOAD_BYTES) { + return { ok: false, reason: "too_large" }; + } + + const send = request.fetchImpl ?? fetch; + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), request.timeoutMs ?? DEFAULT_UPLOAD_TIMEOUT_MS); + let response: Response; + try { + response = await send( + diagnosticsUploadUrl(request.baseUrl), + { + method: "POST", + headers: { + "content-type": "application/json", + ...(request.token ? { authorization: `Bearer ${request.token}` } : {}), + }, + body, + signal: controller.signal, + }, + ); + } catch { + return { ok: false, reason: "network" }; + } finally { + clearTimeout(timer); + } + + if (response.status === 413) return { ok: false, reason: "too_large" }; + if (response.status === 429) return { ok: false, reason: "rate_limited" }; + if (response.status === 503) return { ok: false, reason: "unavailable" }; + if (!response.ok) return { ok: false, reason: "rejected" }; + + let payload: unknown; + try { + payload = await response.json(); + } catch { + return { ok: false, reason: "rejected" }; + } + const id = payload && typeof payload === "object" && !Array.isArray(payload) + ? (payload as Record).id + : null; + if (typeof id !== "string" || !id.trim()) return { ok: false, reason: "rejected" }; + return { ok: true, id: id.trim(), reference: diagnosticReference(id) }; +} + +/** + * One short, non-technical sentence per outcome. The person reading this is + * already looking at an error screen; a status code is not help. + * + * The CLI words its own line differently (`describeDiagnosticUpload` in + * `apps/ade-cli/src/commands/reportIssue.ts`) because it prints a terminal line + * rather than a sentence under a button; the reasons themselves are this + * module's, and there is only one table of them. + */ +export function describeDiagnosticUploadFailure(reason: DiagnosticUploadFailure): string { + switch (reason) { + case "rate_limited": + return "You've already sent a few reports today. Try again tomorrow."; + case "too_large": + return "This report is too big to send. Copy it and post it on GitHub instead."; + case "unavailable": + return "ADE can't take reports right now. Copy it and post it on GitHub instead."; + default: + return "ADE couldn't send the report. Copy it and post it on GitHub instead."; + } +} diff --git a/apps/desktop/src/shared/types/git.ts b/apps/desktop/src/shared/types/git.ts index db1271075..c010431c8 100644 --- a/apps/desktop/src/shared/types/git.ts +++ b/apps/desktop/src/shared/types/git.ts @@ -395,6 +395,18 @@ export type GitHubStatus = { tokenStored: boolean; patTokenStored: boolean; tokenDecryptionFailed: boolean; + /** + * True when ADE's encrypted credential store could not be decrypted on this + * read. Optional for compatibility with older remote runtimes, which simply + * omit it. + * + * An unreadable store returns an EMPTY view instead of throwing, so every + * "no token" conclusion downstream of it is indistinguishable from a fresh + * install. Clients MUST NOT render this as "never connected": the saved + * credentials are still on disk, and inviting the user to reconnect over them + * is how a recoverable read failure turns into real credential loss. + */ + credentialStoreUnreadable?: boolean; storageScope: "app"; authSource: "app" | "pat" | "environment" | "gh" | "none"; tokenType?: GitHubTokenType; @@ -435,6 +447,22 @@ export type GitHubStatus = { connected: boolean; }; +/** + * The one wording for `credentialStoreUnreadable`, shared by every surface that + * has to say it: the PR tab's empty state (built in the main process), the + * integration banner, and the Settings card. Lives beside the field rather than + * in a renderer helper because the main process needs it too, and two hand-kept + * copies of the same sentence is how the "not connected" masking survived in + * more than one place to begin with. + */ +export const GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY = { + subState: "credential-store-unreadable", + statusLabel: "Can't read sign-in", + title: "ADE can't read your saved sign-in on this computer", + detail: "Your GitHub connection may still be there — ADE just can't open it. Repair it in Settings → Connections.", + action: "Open connections", +} as const; + export type GitHubSetTokenResult = GitHubStatus & { credentialVerification: GitHubCredentialVerification; }; diff --git a/apps/desktop/src/shared/types/productAnalytics.ts b/apps/desktop/src/shared/types/productAnalytics.ts index 2ca62622d..510954c7a 100644 --- a/apps/desktop/src/shared/types/productAnalytics.ts +++ b/apps/desktop/src/shared/types/productAnalytics.ts @@ -24,6 +24,7 @@ export const PRODUCT_ANALYTICS_EVENTS = [ "ade_relay_suppressed", "ade_account_session_unreadable", "ade_tool_fetched", + "ade_brain_action_failed", ] as const; export type ProductAnalyticsEventName = (typeof PRODUCT_ANALYTICS_EVENTS)[number]; diff --git a/apps/desktop/src/shared/types/recovery.ts b/apps/desktop/src/shared/types/recovery.ts index d8e00846d..9e1d2e287 100644 --- a/apps/desktop/src/shared/types/recovery.ts +++ b/apps/desktop/src/shared/types/recovery.ts @@ -4,6 +4,8 @@ export const ADE_RECOVERY_ERROR_CODES = [ "db_integrity", "migration_incomplete", "migration_unknown_state", + /** The data files exist but the filesystem refuses to read them. */ + "storage_read_failed", "brain_not_installed", "brain_crash_looping", "socket_stale_no_owner", @@ -42,6 +44,13 @@ export type ProjectRecoveryDiagnosis = { | "disk_full" | "insufficient_headroom" | "db_repair_needed" + /** + * The data files can't be read at all — typically a project or `~/.ade` + * parked in a cloud folder whose contents were evicted. Repair is not + * offered: rewriting files ADE cannot read would risk the user's data, and + * the fix is to move the folder. + */ + | "storage_unreadable" | "brain_crash_looping" | "brain_not_installed" | "socket_stale_no_owner" @@ -135,6 +144,7 @@ export function mapKvDbOpenErrorCode(code: string): AdeRecoveryErrorCode { case "db_integrity": case "migration_incomplete": case "migration_unknown_state": + case "storage_read_failed": return code; default: return "unknown"; @@ -155,6 +165,7 @@ export function stateForCode(code: AdeRecoveryErrorCode): ProjectRecoveryDiagnos case "db_integrity": case "migration_incomplete": case "migration_unknown_state": return "db_repair_needed"; + case "storage_read_failed": return "storage_unreadable"; case "brain_crash_looping": return "brain_crash_looping"; case "brain_not_installed": return "brain_not_installed"; case "socket_stale_no_owner": return "socket_stale_no_owner"; diff --git a/apps/desktop/src/shared/types/sync.ts b/apps/desktop/src/shared/types/sync.ts index 80bfae04f..6b5dca8fd 100644 --- a/apps/desktop/src/shared/types/sync.ts +++ b/apps/desktop/src/shared/types/sync.ts @@ -1195,6 +1195,15 @@ export type SyncCloudRelayStatus = { lastBridgeValidationAt: string | null; lastControlError: string | null; lastError: string | null; + /** + * The relay self-probe: ADE dialing its own relay endpoint and reading the + * echo back. Declared optional because only the brain's status builder fills + * them in and only `ade doctor` reads them; every other producer of this type + * omits all three, and did so while these fields were travelling untyped. + */ + relayEndToEndVerifiedAt?: string | null; + relayEndToEndFailure?: string | null; + relayEndToEndRoundTripMs?: number | null; }; /** diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index d7eb36841..6140b7eb5 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -261,7 +261,7 @@ It installs `ade-win32-x64.exe` as `%ADE_HOME%\bin\ade.exe`, transactionally sta **Health check (`ade doctor [--online] [--text]`).** `apps/ade-cli/src/commands/doctor.ts` connects to the machine brain over the local socket (bounded ~2 s) and prints one status row (`ok` / `warn` / `fail`) per subsystem: **App** (installed desktop version from the `.app` `Info.plist` vs the latest known version — read from disk, or from GitHub with `--online`), **Brain** (running version/pid/uptime plus any build-hash or role mismatch), **Wedge history** (the most recent recovered event-loop wedge, if any), **Sync port** (whether the shared listener bound the default `8787`, and the holders of the base ports when it drifted — with no visible holder reported as exactly that, since a root-owned holder such as `tailscaled` is invisible to a user-level probe and must be checked with `tailscale serve status` / `netstat -an -p tcp`), **Publish health** (the account-directory publisher's last-leg durations and slowest leg), **Relay** (end-to-end verified vs a classified failure — with a deliberate suppression, i.e. another ADE process on this machine owning the relay slot, outranking every other reason, since nothing downstream can succeed while it holds and no other reason tells the user what to do), and **Account** (signed-in state and source). The command exits non-zero when any row is `fail`. The row-evaluation logic (`evaluateDoctorRows`) is pure and dependency-injected so the desktop connection-doctor card and the CLI share one verdict. -**Diagnostic report (`ade report-issue [--open]`).** `apps/ade-cli/src/commands/reportIssue.ts` prints the same redacted Markdown report the desktop's **Report issue** button produces, and `--open` copies the report to the system clipboard and then opens the prefilled GitHub issue in the browser (the template asks the user to paste the report, which is far too big for a query string; `--json` reports whether the copy succeeded as `copied`) (`lib/externalLinks.ts`, which allows only `http(s)`/`mailto:` and falls back to Electron's `shell.openExternal` when the OS opener is unavailable). It reads local files only — it never starts or contacts the brain — so it still works on the machine where ADE will not come up, and on a headless or Windows host with no desktop error screen to press. The builder, the redactor, and the machine source collector (`services/diagnostics/diagnosticReport.ts`, `diagnosticSources.ts`) are shared with the desktop, so a log added for one appears in both. See [features/storage-and-recovery/README.md](./features/storage-and-recovery/README.md#diagnostic-reports-report-issue). +**Diagnostic report (`ade report-issue [--open] [--send]`).** `apps/ade-cli/src/commands/reportIssue.ts` prints the same redacted Markdown report the desktop's **Report issue** button produces, and `--open` copies the report to the system clipboard and then opens the prefilled GitHub issue in the browser (the template asks the user to paste the report, which is far too big for a query string; `--json` reports whether the copy succeeded as `copied`) (`lib/externalLinks.ts`, which allows only `http(s)`/`mailto:` and falls back to Electron's `shell.openExternal` when the OS opener is unavailable). It reads local files only — it never starts or contacts the brain — so it still works on the machine where ADE will not come up, and on a headless or Windows host with no desktop error screen to press. The builder, the redactor, and the machine source collector (`services/diagnostics/diagnosticReport.ts`, `diagnosticSources.ts`) are shared with the desktop, so a log added for one appears in both. `--send` — and the desktop's **Send to ADE** action next to the same button — posts that identical redacted document to `POST /diagnostics/upload` on the account directory Worker and shows a short reference id, so a user who cannot be walked through a terminal still gets the report to us in one action; both go through the single client in `apps/desktop/src/shared/diagnosticsUpload.ts`, with the CLI attaching the machine's account token when it has one and the desktop renderer (which cannot reach the credential store) uploading anonymously. The upload is capped at 512 KB, limited to five a day per user-or-IP, and stored in R2 without ever being echoed back. See [features/storage-and-recovery/README.md](./features/storage-and-recovery/README.md#diagnostic-reports-report-issue). **Install + PATH wiring (when the desktop ships `ade`).** On macOS / Linux the desktop installer drops the launcher at `$HOME/.local/bin/ade`; on Windows it lands at `%LOCALAPPDATA%\ADE\bin\ade.cmd`. After a successful install on Windows, the packaged `.cmd` installer adds the target directory to HKCU `Environment\Path` when needed and broadcasts an environment-change notification. After a successful install on POSIX, `ensureUserBinOnShellPath` appends a marked `export PATH="$HOME/.local/bin:$PATH"` block to the user's shell rc (`.zshrc` for zsh, `.bashrc` for bash, `.profile` otherwise) iff (a) the install dir isn't already on the inherited `PATH` and (b) the file doesn't already contain the marker / line / target dir. The install IPC reply tells the renderer which profile was edited so the Settings/Onboarding UI can prompt the user to open a new terminal or `source` it. @@ -910,7 +910,8 @@ ade.remoteRuntime.* # remote target registry, connect/projects/project- `apps/desktop/src/main/services/ipc/registerIpc.ts` (~6,400 lines) is the single registration point: - `ipcMain.handle(IPC.channelName, async (event, args) => { ... })` for invoke channels. -- Every handler is wrapped with a timeout — 30 seconds by default, with explicit longer budgets for known long operations such as direct lane delete, iOS Simulator launch/control, App Control, and built-in browser actions. Runtime-dispatched actions use the runtime-call channel budget; the timeout wrapper no longer inspects the action payload to give `lane.delete` a special runtime-dispatch override. +- Every handler is wrapped with a timeout — 30 seconds by default, with explicit longer budgets for known long operations such as direct lane delete, iOS Simulator launch/control, App Control, and built-in browser actions. The two runtime `callAction` channels are generic, so `ipcTimeouts.ts` reads `{ request: { domain, action } }` off the raw invoke arguments through the shared `readRuntimeActionRequest` decoder and derives the budget from the action itself (`localRuntimeActionIpcTimeoutMs` for the local pool; the equivalent direct channel's budget, else the retryable-remote budget, for the remote one). A local-pool payload it cannot read falls back to the project-completion budget rather than the 30-second default, because a `callAction` invoke can be waiting on cold project setup. +- That same decoder is what attributes a failed brain call. Every action the app performs reaches the daemon through the one `ade.localRuntime.callAction` channel, so its error path reports `action_domain` plus a structured `error_code` — see [logging.md](./logging.md#desktop-runtime-tui-and-hosted-web-client) for why the channel is deliberately kept out of `MEANINGFUL_ACTIONS`. One decoder rather than two matters here: readers that disagreed about what a well-formed request looks like would attribute a timeout to one action and its failure to another. - Every handler emits structured tracing: `ipc.invoke.begin`, `ipc.invoke.done`, `ipc.invoke.failed` with call ID, channel, window ID, duration, and summarized args/results. - `AppContext` indirection: handlers close over a context pointer that swaps atomically on project switch, so IPC channels remain registered across project transitions. - Lane branch-drift handlers (`ade.lanes.getBranchDrift`, `ade.lanes.resolveBranchDrift`) are registered here with the rest of `ade.lanes.*` and delegate to `laneService`. Preload prefers the `lane` runtime action of the same name, so a remote-bound window resolves drift on the machine that owns the worktree and falls back to these handlers only when no runtime is bound. @@ -972,7 +973,7 @@ Most services described here live under `apps/desktop/src/main/services/ | `github/` | `githubService.ts` | GitHub REST/GraphQL access; PR CRUD; checks; reviewers. | | `history/` | `operationService.ts` | Operation audit records (one row per mutation). | | `ios/` | `iosSimulatorService.ts` | macOS-only iOS Simulator backend: tool readiness probes, simctl device + app discovery, build/install/launch with progress events (hardened with `simctl bootstatus` and `simctl install` timeouts), screenshot + ADEInspector + accessibility hit-test, Simulator.app window live-view status, idb-backed input, and single-owner chat session locking. The macOS Simulator window placement / capture state probe (`getSimulatorWindowState`, `prepareSimulatorWindowForCapture`) lives next to the IPC handlers in `ipc/registerIpc.ts` because it depends on the active `BrowserWindow`. See [features/ios-simulator/README.md](./features/ios-simulator/README.md). | -| `ipc/` | `registerIpc.ts`, `runtimeBridge.ts`, `runtimeEventSubscriptionRegistry.ts`, `knownProjectRoots.ts`, `ipcTimeouts.ts` | Single registration point for all IPC handlers. `runtimeEventSubscriptionRegistry.ts` holds runtime-event subscriptions keyed by (sender, requestKey) with idle expiry and a single removal path (see §5.4). `knownProjectRoots.ts` validates renderer-supplied project roots for the recovery and diagnostics channels — the open project, a local recent-projects entry, or a root main itself recently attempted to open (a bounded, expiring, single-writer registry, which is what keeps a folder whose *first* open failed repairable). `runtimeBridge.ts` owns the runtime-facing channels (remote target registry, remote-runtime connect / project list / project-open / action dispatch / sync dispatch / event stream, per-target `listActionRegistry` lookup against the remote daemon, LAN + Tailscale discovery with diagnostics) and routes runtime calls through `LocalRuntimeConnectionPool` or `RemoteConnectionPool` based on the active window binding. The explicit `ade.sync.getLocalStatus` handler is the exception: it calls machine-level `sync.getStatus` on `LocalRuntimeConnectionPool` (with only the local in-process diagnostics service as fallback) so a remote-bound Connections panel can still identify the physical computer (its This computer card, pairing code, and local Phone/Web device lists). Device and pairing *mutations* still follow the window binding, so the panel presents them read-only while remote-bound rather than routing them to the remote machine. Event-stream subscription init/results preserve replay-gap metadata (`gap`, `oldestCursor`, `eventEpoch`) for both local and remote bindings, and subscription bookkeeping is delegated to `runtimeEventSubscriptionRegistry.ts`; `runtimeBridge.ts` derives the request key (one helper shared by the subscribe and release paths, so a release rebuilds exactly the key subscribe registered) and registers the `ade.runtime.events.release` handler, which resolves the binding from the same descriptor shape the subscribe call used and refuses to act on an unauthorized local root. Remote project opens are generation-guarded per window/webContents before main persists the binding. It also subscribes `powerMonitor` `resume` and `unlock-screen` to `remoteConnectionService.probeSavedConnections()` so a laptop waking up cycles dead SSH sessions before the renderer pokes them. Machine-level sync fallback recognizes only the canonical unavailable-service predicates in `shared/runtimeErrors.ts`, shared with preload and renderer recovery guidance. `ipcTimeouts.ts` carries the default 30-second handler timeout plus named channel-level overrides for long direct IPC operations; it does not inspect runtime action payloads. | +| `ipc/` | `registerIpc.ts`, `runtimeBridge.ts`, `runtimeEventSubscriptionRegistry.ts`, `knownProjectRoots.ts`, `ipcTimeouts.ts` | Single registration point for all IPC handlers. `runtimeEventSubscriptionRegistry.ts` holds runtime-event subscriptions keyed by (sender, requestKey) with idle expiry and a single removal path (see §5.4). `knownProjectRoots.ts` validates renderer-supplied project roots for the recovery and diagnostics channels — the open project, a local recent-projects entry, or a root main itself recently attempted to open (a bounded, expiring, single-writer registry, which is what keeps a folder whose *first* open failed repairable). `runtimeBridge.ts` owns the runtime-facing channels (remote target registry, remote-runtime connect / project list / project-open / action dispatch / sync dispatch / event stream, per-target `listActionRegistry` lookup against the remote daemon, LAN + Tailscale discovery with diagnostics) and routes runtime calls through `LocalRuntimeConnectionPool` or `RemoteConnectionPool` based on the active window binding. The explicit `ade.sync.getLocalStatus` handler is the exception: it calls machine-level `sync.getStatus` on `LocalRuntimeConnectionPool` (with only the local in-process diagnostics service as fallback) so a remote-bound Connections panel can still identify the physical computer (its This computer card, pairing code, and local Phone/Web device lists). Device and pairing *mutations* still follow the window binding, so the panel presents them read-only while remote-bound rather than routing them to the remote machine. Event-stream subscription init/results preserve replay-gap metadata (`gap`, `oldestCursor`, `eventEpoch`) for both local and remote bindings, and subscription bookkeeping is delegated to `runtimeEventSubscriptionRegistry.ts`; `runtimeBridge.ts` derives the request key (one helper shared by the subscribe and release paths, so a release rebuilds exactly the key subscribe registered) and registers the `ade.runtime.events.release` handler, which resolves the binding from the same descriptor shape the subscribe call used and refuses to act on an unauthorized local root. Remote project opens are generation-guarded per window/webContents before main persists the binding. It also subscribes `powerMonitor` `resume` and `unlock-screen` to `remoteConnectionService.probeSavedConnections()` so a laptop waking up cycles dead SSH sessions before the renderer pokes them. Machine-level sync fallback recognizes only the canonical unavailable-service predicates in `shared/runtimeErrors.ts`, shared with preload and renderer recovery guidance. `ipcTimeouts.ts` carries the default 30-second handler timeout plus named channel-level overrides for long direct IPC operations, and exports `readRuntimeActionRequest` — the one decoder for the `{ request: { domain, action } }` payload the generic `callAction` channels carry, used both to pick an action-specific timeout and, in `registerIpc.ts`, to attribute a failed brain call to its action domain. | | `jobs/` | `jobEngine.ts` | Event-driven background scheduler for lane refresh + conflict prediction. Coalesced, debounced. | | `keybindings/` | `keybindingsService.ts` | User keybindings read/write. | | `lanes/` | `laneService.ts`, `laneEnvironmentService.ts`, `laneTemplateService.ts`, `laneProxyService.ts`, `portAllocationService.ts`, `autoRebaseService.ts`, `rebaseSuggestionService.ts`, `laneLaunchContext.ts`, `oauthRedirectService.ts`, `runtimeDiagnosticsService.ts`, `laneUsageTombstone.ts` | Worktree lifecycle, env bootstrap, templates, reverse proxy, port leases, auto-rebase, suggestions, OAuth redirect, diagnostics. `laneUsageTombstone.ts` writes the one aggregate row a deleted lane leaves behind (`lane_usage_tombstones`) so lifetime activity survives lane deletion. | @@ -1249,7 +1250,7 @@ webPreferences: { **CSP** (`rendererCsp.ts`): `default-src 'self'`; `script-src 'self'` (no eval, no inline scripts); `style-src 'self' 'unsafe-inline'` (required for Tailwind); `connect-src 'self'`; `img-src 'self' data:` plus a host-scoped allowlist (no blanket `https:`) for the image origins PR/README surfaces actually load — the GitHub avatar/asset hosts (`*.githubusercontent.com`, `github.githubassets.com`, …) and `www.gravatar.com` / `secure.gravatar.com` (commit-author identicon fallback). `frame-src` stays local/about by default with a narrow external exception for the ADE welcome video hosts (`www.youtube-nocookie.com` and `www.youtube.com`). The Electron header hook applies this policy only to ADE renderer main-frame documents; external subframes keep their own response CSP so embedded players can execute their host-provided scripts. -Every IPC handler **validates** its arguments; invalid args return structured errors, never crash. Every handler has a **30s timeout** by default; `ipcTimeouts.ts` carries per-channel overrides for long-running operations and inspects the payload of `localRuntime.callAction` / `remoteRuntime.callAction` so action-specific timeouts (e.g. `lane.create` / `lane.delete` → 4 min; `ios_simulator.launch` → 10 min) apply even when the channel itself is generic. Every handler emits structured tracing. +Every IPC handler **validates** its arguments; invalid args return structured errors, never crash. Every handler has a **30s timeout** by default; `ipcTimeouts.ts` carries per-channel overrides for long-running operations (`ade.iosSimulator.launch` → 10 min) and reads the `{ domain, action }` payload of `localRuntime.callAction` / `remoteRuntime.callAction` so action-specific timeouts (e.g. `lane.delete` → 4 min) apply even when the channel itself is generic. Every handler emits structured tracing. Most `window.ade.sync.*` preload methods follow the active project binding and therefore target a remote brain when the window is remote-bound. diff --git a/docs/features/ade-code/README.md b/docs/features/ade-code/README.md index 0360240a5..d7f265668 100644 --- a/docs/features/ade-code/README.md +++ b/docs/features/ade-code/README.md @@ -59,7 +59,7 @@ Point Cursor’s browser inspector at the served page for layout debugging. The | `apps/ade-cli/src/tuiClient/workListLayout.ts` | Single source of truth for sessions-pane row geometry: `workListRowHeight` (a card is always 3 lines, matching the desktop SessionCard), `computeWorkListLayout` (scroll window that always contains the selection), `workListMouseHitForLayout`, and `workListHitRects`. Singleton cards split the first line as a `lane-identity` hit so a click on the lane name opens lane details; title and preview still open the chat. The renderer and the mouse handler both consume `layout.placements`, so a click and what is on screen cannot drift. | | `apps/ade-cli/src/tuiClient/newLaneForm.ts` | Pure model for the `/new lane` form: start-from modes (primary / child / import), Linear issue + setup-template fields, per-mode field lists, and `buildNewLaneSubmission` mapping form values onto `lane.create` / `lane.createChild` / `lane.importBranch` payloads. | | `apps/ade-cli/src/tuiClient/eventDedup.ts` | Reserves and syncs chat-event dedupe keys so replayed runtime events do not render twice. | -| `apps/ade-cli/src/tuiClient/reportIssue.ts` | `/report-issue` for the TUI — the terminal counterpart of the desktop **Report issue** button and of `ade report-issue`. Builds the report through the same shared builder/redactor, writes it `0600` under an owner-only directory, and renders a narrow-pane summary with the saved path and the prefilled GitHub issue URL. Local files only: it never asks the brain for anything, so it still answers on a machine where the brain is the problem, and a read-only or full disk degrades to "no file, URL still usable" rather than a second failure. `resolveTuiCliVersion` reads the bundle's `__ADE_VERSION__` define (falling back to `ADE_CLI_VERSION`) so a report filed from `ade code` names the same build as `ade` itself. | +| `apps/ade-cli/src/tuiClient/reportIssue.ts` | `/report-issue` for the TUI — the terminal counterpart of the desktop **Report issue** button and of `ade report-issue`. Builds the report through the same shared builder/redactor, writes it `0600` under an owner-only directory, and renders a narrow-pane summary with the saved path and the prefilled GitHub issue URL. Builds from local files only — it never asks the brain for anything, so it still answers on a machine where the brain is the problem, and a read-only or full disk degrades to "no file, URL still usable" rather than a second failure. `/report-issue send` (or `--send`) then uploads the already-built, already-redacted bytes through the shared `diagnosticsUpload` client — the same path the desktop button and `ade report-issue --send` use — after the pane has shown the saved path and issue URL, so a failed upload costs nothing the user was not already holding. `resolveTuiCliVersion` reads the bundle's `__ADE_VERSION__` define (falling back to `ADE_CLI_VERSION`) so a report filed from `ade code` names the same build as `ade` itself. | | `apps/ade-cli/src/tuiClient/feedback.ts` | Builds the multi-field `/feedback` form. Validates required fields, packs the `FeedbackDraftInput` envelope, and adds project / lane / runtime context before submission. | | `apps/ade-cli/src/tuiClient/heartbeat.ts` | Maintains the `startTuiHeartbeat` loop that tells the runtime the terminal client is still attached. | | `apps/ade-cli/src/tuiClient/highlightCache.ts` | Pre-registers highlight.js languages (TypeScript, JavaScript, Python, Rust, Go, Swift, Bash, JSON, YAML, Markdown, XML, CSS, SQL) and caches token streams so chat code fences render once instead of being re-highlighted on every redraw. | @@ -310,7 +310,7 @@ Right pane (open contextual content): | `/keybindings [open]` | Show Claude-compatible keybinding config diagnostics. Pass `open` to launch the configured editor on `~/.claude/keybindings.json`. | | `/statusline` | Show Claude-compatible status line config. | | `/doctor` | Show ADE Code and Claude-compat diagnostics. | -| `/report-issue` | Build a redacted diagnostic report (saved path + prefilled GitHub issue URL) without contacting the brain. | +| `/report-issue [send]` | Build a redacted diagnostic report (saved path + prefilled GitHub issue URL) without contacting the brain; `send` uploads the same redacted bytes to ADE. | | `/model` | Open the transient model wizard (provider → family → model → settings). | | `/effort` | Open the model wizard directly on its settings step for the active provider. | | `/import` | Import an external CLI session (provider-agnostic). Only actionable while starting a new chat; elsewhere it says so instead of no-opping. | diff --git a/docs/features/onboarding-and-settings/README.md b/docs/features/onboarding-and-settings/README.md index b18ee2822..c711ac1bd 100644 --- a/docs/features/onboarding-and-settings/README.md +++ b/docs/features/onboarding-and-settings/README.md @@ -126,6 +126,25 @@ Main process: into missing permissions. `service_unavailable` gets no credential cooldown, because the credential is not the problem and must stay usable the instant GitHub recovers. + `GitHubStatus.credentialStoreUnreadable` is carried on the credential + inventory (cached with it for 30 s) rather than re-read at status time, so the + readability verdict always belongs to the read that produced that inventory's + candidates. The service holds it sticky across reads and warns + `github.credential_store_unreadable` — with the store's own failure reason — + once per transition into unreadable rather than on every cached refresh; a + successful token write clears it, because a write that landed re-sealed the + store under a key this process holds. +- `apps/desktop/src/main/services/github/credentialReadState.ts` — returns a + credential *and* whether the store was readable in one call + (`readCredentialWithState` / `readCredentialWithStateAsync`). An undecryptable + store returns an empty view instead of throwing, so "no token" and "a token + ADE cannot read" are the same answer, and only the store's + `getLastReadState()` separates them — and that method describes the store's + most recent read, so it has to be consulted immediately after the `getSync` it + is being asked about. Pairing the two in one helper is what makes that + ordering impossible to get wrong. A store that throws counts as unreadable + too: the Electron `safeStorage` store reports decrypt failures that way rather + than by returning `{}`. - `apps/desktop/src/shared/githubServiceHealth.ts` and `apps/desktop/src/main/services/github/githubStatusPage.ts` — telling a GitHub outage apart from a broken credential. The shared module parses @@ -150,9 +169,14 @@ Main process: GitHub request/status path. It applies the same candidate order, cooldowns, GraphQL classification, conditional-request cache isolation, read/write status fields, and githubstatus.com corroboration when a packaged or - remote-bound window uses `ade serve`. The renderer reaches GitHub through - whichever service owns the project, so anything applied to only one of the two - `getStatus` implementations is dead in the shipping runtime-backed build. + remote-bound window uses `ade serve`. It reads its stored tokens through the + same `credentialReadState.ts` helpers the desktop service uses and reports + `credentialStoreUnreadable` on its own status and inventory, because an + undecryptable store returns an empty view rather than throwing — without it a + remote runtime describes a corrupted store exactly as it describes a fresh + install. The renderer reaches GitHub through whichever service owns the + project, so anything applied to only one of the two `getStatus` + implementations is dead in the shipping runtime-backed build. - `apps/desktop/src/main/services/config/projectConfigService.ts` — YAML config read/merge/save, AI mode migration, lane env init, Linear sync resolver. ~3,150 lines, the largest service. @@ -177,9 +201,18 @@ Shared types and IPC: - `apps/desktop/src/shared/types/git.ts` — `GitHubStatus`, `GitHubAuthFailure`, `GitHubRateLimitState`, and the credential source, capability, state, and fallback contracts. `writeAuthSource`, - `credentialStates`, `credentialFallback`, `backgroundRefreshPausedUntil`, and - `serviceHealth` are optional so a newer client remains compatible with an - older remote runtime. + `credentialStates`, `credentialFallback`, `credentialStoreUnreadable`, + `backgroundRefreshPausedUntil`, and `serviceHealth` are optional so a newer + client remains compatible with an older remote runtime, which simply omits + them. The same module owns `GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY` — the + single wording for the unreadable case, shared by the Settings card, the + integration banner, the PR tab's main-process empty state, and the + Publish-to-GitHub dialog — which, in this state, drops the token field + entirely and offers only **Open connections**, because saving a token would + overwrite a sign-in that repair can still recover. It lives beside + the field rather than in a renderer helper because the main process needs it + too, and two hand-kept copies of one sentence is how the "not connected" + masking survived in more than one place. - `apps/desktop/src/shared/ipc.ts` — channels: - `ade.onboarding.*` (status, detectDefaults, applySuggestedConfig, complete, setDismissed) @@ -368,7 +401,13 @@ Renderer — settings: account bucket renders the reset time and explains that background refresh is paused automatically; it never asks the user to re-authenticate. When no fallback remains, a missing, invalid, or genuinely under-scoped credential - shows login/refresh instructions. The App-installation card also classifies relay + shows login/refresh instructions. An unreadable credential store outranks all + of that: the card reads **Can't read sign-in** instead of "Not connected", + renders the shared unreadable notice with an **Open connections** button that + opens the Connections panel's Machines tab, and suppresses the `gh auth login` + setup steps — those steps would answer a question ADE has not actually asked, + and following them writes a new credential over one that is probably intact. + The App-installation card also classifies relay rate-limit responses as a concise cooldown state instead of displaying GitHub's raw request-id / scraping-policy error. Raw network/unknown validation errors stay in Settings rather than the global banner. The shared @@ -432,15 +471,38 @@ Renderer — settings: before a mutation. `describeGithubOutage(status)` is the one presentation entry point for a corroborated GitHub outage — it answers "is there an outage", "what do we say", and "where does the button go" together, and every - GitHub-blaming surface gates on it. `describeGithubAuthFailure` and - `describeGithubCliBanner` consult it first, so an outage outranks every - credential-shaped reading of the same failure. When it returns null ADE says - nothing about GitHub's health and keeps its existing copy. + GitHub-blaming surface gates on it. When it returns null ADE says nothing + about GitHub's health and keeps its existing copy. + `describeGithubCliBanner` resolves three states in a fixed order, and the + order is the whole point: + 1. `credentialStoreUnreadable` — first, ahead of everything. An unreadable + store returns an EMPTY view, so every conclusion below it would be drawn + from credentials ADE could not read. It is also a local, repairable fact + that outlives any incident, so an outage must not mask the one thing the + user can actually fix. + 2. `!tokenStored` — ahead of the outage. A missing token is a purely local + fact and stays true regardless of GitHub's health, so the genuine "connect + GitHub" instruction survives an incident. + 3. the outage — ahead of every remaining state, all of which are inferred + from GitHub's own answers and are therefore unreliable while GitHub is + failing. `describeGithubAuthFailure` consults the outage first for the same + reason. + + Every branch also returns a `target` (`github-settings` | `connections`) + rather than leaving the destination implicit: an unreadable store is not a + GitHub problem and is not fixed on the GitHub card, whereas an outage and + every auth failure are — there the credential ADE holds is readable and it is + the account behind it, or GitHub itself, that has the objection. - `apps/desktop/src/renderer/components/app/IntegrationBannerHost.tsx` and `FeedbackReporterModal.tsx` — consume the shared read/write distinction. The app shell raises a write-access banner for an otherwise connected App-only status, and feedback submission requires a write-capable credential rather - than treating read connectivity as sufficient. + than treating read connectivity as sufficient. The gh-CLI/token banner routes + its single action by the `target` the derivation supplies — the GitHub + connection settings route, or the same Connections panel the relay banner + opens — and keys its dismissal fingerprint on the sub-state, so a store that + becomes unreadable re-raises a banner the user had dismissed for a different + reason. - `apps/desktop/src/renderer/components/settings/LinearIntegrationSection.tsx` and `LinearSection.tsx` — Linear OAuth / API key, workspace status, and GitHub autolink setup. Embedded inside General. @@ -1298,7 +1360,7 @@ and [machine power and sleep in the account directory](../sync-and-multi-device/ | Terminal preferences | `localStorage` under `ade.terminalPreferences.v1` | font size, line height, scrollback, font family | | Work view state | `localStorage` under `ade.workViewState.v1` | per-project and per-lane-project slices | | Keep-awake level | `GlobalState` in `/ade-state.json` under `keepAwakePreferences` | machine-scoped; anything unreadable normalizes to `never` | -| GitHub credentials | Keychain via `safeStorage` | tokens encrypted, banner on decryption failure | +| GitHub credentials | Keychain via `safeStorage` | tokens encrypted; a store ADE cannot decrypt reports `credentialStoreUnreadable` rather than "not connected" | | Linear credentials | Active project's `.ade/secrets` | project-local token/OAuth state, encrypted on disk | ## AI mode and provider behavior @@ -1409,6 +1471,55 @@ margin, because a wait shorter than the lock timeout could expire while the winning peer is still legitimately queued behind the lock — and the loser would then declare a live session dead. +## GitHub connection status has the same third state + +The account session is not the only thing the credential store can hide. The +same file backs `github.token.v1` and `github.appUserToken.v1`, and it fails the +same way: an undecryptable store returns an **empty view** rather than an error, +so a GitHub status built from it is indistinguishable from a fresh install. That +is the whole hazard — "not connected" invites a reconnect, and a reconnect +overwrites credentials that were only unreadable, turning a recoverable read +failure into real credential loss. + +`GitHubStatus` therefore separates three answers, not two: + +| Answer | What is actually true | What the surface offers | +| --- | --- | --- | +| connected | A credential validated against `GET /user` and, where required, the repo probe. | Nothing; `writeAuthSource` still gates mutations separately. | +| not connected | ADE read the store successfully and there is no usable credential. | `gh auth login` steps, or a PAT field. | +| `credentialStoreUnreadable` | ADE could not open the store on this read. Saved credentials may be entirely intact. | Say so, and offer **repair** — never a reconnect as the primary path. | + +Every surface that would otherwise conclude "not connected" resolves the third +state *first*, ahead of `!tokenStored`, because everything downstream of an +unreadable read is a guess: + +- **Settings → Integrations → GitHub** (`GitHubSection.tsx`) shows **Can't read + sign-in** in warning tone with the shared notice and an **Open connections** + button, and hides the gh-CLI setup steps. +- **The app-shell integration banner** (`IntegrationBannerHost.tsx`) points its + action at the Connections panel rather than the GitHub settings route, since + no GitHub-side action repairs a store ADE cannot open. +- **The PR tab's empty state** — the message is built in the main process by + `prService.buildGithubSnapshotAuthError`, which is why the wording is a shared + constant in `shared/types/git.ts` rather than renderer copy. +- **Publish to GitHub** (`PublishToGitHubDialog.tsx`) reads the flag once when + the dialog opens and replaces "GitHub is not connected" with the unreadable + wording. The token field stays, because pasting a token there is a deliberate + act by a user who already came to connect — the dialog only stops asserting a + disconnection it cannot actually observe. + +The Settings button and the banner action both land on the Connections panel's +**Machines** tab, where the account header already carries the sign-in repair +path — the same +`account.repairSession` the account tri-state uses, which converges the shared +credential store's key binding before restarting the brain. One store, one +repair: a GitHub-specific fix-it control would be a second button for the same +file. Writing a new GitHub token also clears the flag, because a successful +write re-seals the store under a key this process holds. + +Because the field is optional, an older remote runtime that omits it degrades to +the previous two-way behaviour instead of reporting a state it cannot compute. + ## Gotchas - **Shared vs local.** Shared config is version-controlled and visible diff --git a/docs/features/pull-requests/README.md b/docs/features/pull-requests/README.md index af36154df..3767f76d7 100644 --- a/docs/features/pull-requests/README.md +++ b/docs/features/pull-requests/README.md @@ -183,8 +183,9 @@ GitHub access and relay dependencies: |------|---------------| | `apps/desktop/src/main/services/github/githubService.ts`, `apps/ade-cli/src/headlessLinearServices.ts` | Desktop-local and runtime-owned GitHub request paths. Both build the environment → App → GitHub CLI → PAT read chain, skip the read-only App for writes, retry compatible credentials after auth/permission/rate failures, and expose the active/fallback sources through `GitHubStatus`. Both also record a **transport** failure — a hang, timeout, DNS/TLS error, or a body that stalls mid-stream, caught at the header read *and* at the body read — before rethrowing it, with a null rate limit so it cannot clobber real quota numbers. Without that record the request budget reported no kind at all for the outage shape it exists to survive. Both expose `getRequestBudget()`; implementing it only on the desktop side would leave the shipping runtime-bound build's poll governor un-gated. | | `apps/desktop/src/main/services/github/githubCredentialHealth.ts`, `githubRateLimit.ts` | Token-digest health keyed by REST/GraphQL resource, five-minute invalid/permission cooldowns, rate-limit reset handling, same-account primary-quota propagation, and the 500-request background reserve. `classifyGitHubAuthFailure` maps GitHub 5xx (and GitHub's own outage bodies) to `service_unavailable` ahead of the transient-network check, and that kind is deliberately given **no** credential cooldown — the credential is not the problem, so parking it would fail the user's next local merge or PR read for the whole window. `githubRequestBudget()` exposes the reserve plus the worst *recent* failure kind as a zero-network `GitHubRequestBudget`, which is how foreground pollers honour the same reserve. The reserve half uses the quota-bucket filter (`core` / `graphql` / a large-limit `unknown`); the kind half deliberately does not, skipping only the independent `search` bucket, and bounds what it reports by `REQUEST_BUDGET_FAILURE_FRESHNESS_MS` (90 s). `REQUEST_BUDGET_FAILURE_SEVERITY` ranks the kinds and is a stated two-way contract with `ladderBaseMs` in the renderer's poll governor. See [Keeping automatic GitHub reads inside the quota](#keeping-automatic-github-reads-inside-the-quota). | +| `apps/desktop/src/main/services/github/credentialReadState.ts` | Reads a credential and the store's readability verdict in one call, so the `getLastReadState()` answer belongs to the `getSync` it is being asked about. An undecryptable store returns an empty view instead of throwing, which is what made "no token" and "a token ADE cannot read" the same answer; a store that throws counts as unreadable too. Feeds `GitHubStatus.credentialStoreUnreadable`. | | `apps/desktop/src/main/services/github/githubStatusPage.ts`, `apps/desktop/src/shared/githubServiceHealth.ts` | GitHub-outage attribution. See [Telling a GitHub outage apart from a broken credential](#telling-a-github-outage-apart-from-a-broken-credential). The shared module is the pure half — Statuspage `summary.json` parsing into `GitHubServiceHealth`, the ADE-relevant component allowlist, and `isGithubServiceUnavailable`; the main-process module owns the failure-triggered lookup, its cache, and `attachGitHubServiceHealth`. | -| `apps/desktop/src/shared/githubOperationCredential.ts`, `apps/desktop/src/shared/types/git.ts` | Capability-aware credential order and the optional status DTOs for source state, fallback, write availability, background-pause time, and corroborated service health. `resolveGithubStatusCredentials` stops walking the chain on `service_unavailable` alongside `network` / `unknown`: a GitHub 5xx says nothing about the credential, so the next candidate would fail identically and only add load to a failing service. | +| `apps/desktop/src/shared/githubOperationCredential.ts`, `apps/desktop/src/shared/types/git.ts` | Capability-aware credential order and the optional status DTOs for source state, fallback, write availability, background-pause time, credential-store readability, and corroborated service health. `resolveGithubStatusCredentials` stops walking the chain on `service_unavailable` alongside `network` / `unknown`: a GitHub 5xx says nothing about the credential, so the next candidate would fail identically and only add load to a failing service. `git.ts` also owns `GITHUB_CREDENTIAL_STORE_UNREADABLE_COPY`, the one wording for the unreadable case — it sits beside the field because the main process builds the PR tab's empty-state message from it. | | `apps/desktop/src/main/services/automations/automationIngressService.ts`, `apps/ade-cli/src/bootstrap.ts`, `apps/desktop/src/main/main.ts` | Relay cursor drain and targeted reconciliation, relay-health tracking, and injection of relay/quota state into the runtime-owned or desktop-local PR poller. | | `apps/webhook-relay/src/relay.ts` | Hosted event/subscription authorization. Signed-in ADE account requests use the installed repository binding in D1 first; legacy clients fall back to a GitHub-token repository-access check. | @@ -926,6 +927,18 @@ Fields: capabilities, active roles, cooldown/failure state, and the active read fallback transition. Different sources that resolve to the same token are attempted once. +- `credentialStoreUnreadable` — optional (older remote runtimes omit it): ADE's + encrypted credential store could not be decrypted on the read that produced + this status. It is not a variant of "no token" — an unreadable store returns + an **empty view** instead of throwing, so `tokenStored: false`, + `patTokenStored: false`, and `authSource: "none"` all become indistinguishable + from a fresh install while the saved credentials are still on disk. Clients + must not render it as "never connected": the reconnect that invitation leads + to overwrites them. The service carries the flag on the 30-second credential + inventory rather than re-reading the store at status time, so the verdict + belongs to the read that produced the inventory's candidates, and clears it on + a successful token write, which re-seals the store under a key the process + holds. - `authFailure` — optional structured validation failure for compatibility with older runtimes: `rate_limited`, `invalid_token`, `permission_denied`, `service_unavailable`, `network`, or `unknown`, with the original message and @@ -1000,6 +1013,22 @@ stays quiet when a fallback keeps reads and writes usable, distinguishes App-only read access from a write-capable connection, and never advertises a reconnect command for an account-level rate-limit pause. +An unreadable credential store is the one case the helper resolves before +anything else — ahead of `!tokenStored` and ahead of the outage check below. +`describeGithubCliBanner` checks `credentialStoreUnreadable` first because an +unreadable store returns an EMPTY view, so every state under it would be read +off credentials ADE never saw; and because it is a local, repairable fact that +outlives any incident, so a GitHub outage must not hide the one thing the user +can actually fix. The helper returns a `target` alongside the copy so the +banner's single action lands where the fix actually is: an outage and every +auth failure are addressed on the GitHub settings card, because there the +credential is readable and it is the account behind it — or GitHub itself — +that has the objection, while an unreadable store is not a GitHub problem at +all and its repair control lives in the Connections panel. +`prService.buildGithubSnapshotAuthError` makes the same check first, for the +same reason — the PR tab's empty state would otherwise tell someone whose +credentials are intact to run `gh auth login` and overwrite them. + ## Telling a GitHub outage apart from a broken credential A failing GitHub request looks the same at the response layer whether the diff --git a/docs/features/storage-and-recovery/README.md b/docs/features/storage-and-recovery/README.md index cc46d5fba..b72978c1e 100644 --- a/docs/features/storage-and-recovery/README.md +++ b/docs/features/storage-and-recovery/README.md @@ -4,9 +4,9 @@ | Path | Role | |---|---| -| `apps/desktop/src/main/services/state/kvDb.ts` | Opens the project database (enabling `journal_mode = WAL` + `synchronous = NORMAL` at open), runs the interrupted-rebuild recovery pass, classifies database-open errors, creates the headroom-gated migration backup, and exports `rebuildTableInTransaction` / `recoverInterruptedTableRebuilds`. Attaches the optional `maintenance` (`DbMaintenanceApi`) handle — the prune / compact / vacuum hooks the storage doctor invokes. The machine-local `local_lane_storage_state` and `local_storage_lifecycle_runs` tables retain reclaim retry/estimate and scan timing state; both are excluded from CRR sync because paths and cleanup results belong only to this checkout. | +| `apps/desktop/src/main/services/state/kvDb.ts` | Opens the project database (enabling `journal_mode = WAL` + `synchronous = NORMAL` at open), runs the interrupted-rebuild recovery pass, classifies database-open errors (`classifySqliteOpenError`, whose `storage_read_failed` bucket is checked *before* the integrity bucket so an unreadable file is never reported as a corrupt one), creates the headroom-gated migration backup, and exports `rebuildTableInTransaction` / `recoverInterruptedTableRebuilds`. Attaches the optional `maintenance` (`DbMaintenanceApi`) handle — the prune / compact / vacuum hooks the storage doctor invokes. The machine-local `local_lane_storage_state` and `local_storage_lifecycle_runs` tables retain reclaim retry/estimate and scan timing state; both are excluded from CRR sync because paths and cleanup results belong only to this checkout. | | `apps/desktop/src/main/services/state/dbMaintenanceApi.ts` | The `DbMaintenanceApi` interface consumed by the storage doctor, plus the single source of truth for the DB retention/count bounds (`INGRESS_EVENT_RETENTION_MS` = 7 days, `INGRESS_EVENT_MAX_ROWS_PER_PROJECT` = 2,000, `REVIEW_ARTIFACT_RETENTION_DAYS` = 30, `PR_SNAPSHOT_RETENTION_DAYS` = 60, `EVENT_LOG_RETENTION_DAYS` = 30) imported by the ingress writer, the kvDb hooks, and the storage ledger so the policy can never drift across enforcement sites. Also exports `pruneRowsInBatches` — the paced `delete … where rowid in (select rowid … limit N)` loop (`MAINTENANCE_DELETE_BATCH_ROWS` = 2,000, `MAINTENANCE_DELETE_MAX_BATCHES` = 200) that every new prune uses. | -| `apps/desktop/src/main/services/state/durableFile.ts` | Atomic temp-write-and-rename persistence, one-generation `.lkg` JSON backup, validation, and primary/previous recovery reads. | +| `apps/desktop/src/main/services/state/durableFile.ts` | Atomic temp-write-and-rename persistence, one-generation `.lkg` JSON backup, validation, and primary/previous recovery reads. `AtomicWriteOptions.mode` creates the temp file with the caller's permission bits so a secret is never briefly world-readable, and a rename refused with `EXDEV` / `EPERM` / `EACCES` / `EBUSY` falls back to a copy — a deliberately closed list that leaves `ENOSPC` and `EIO` terminal. | | `apps/desktop/src/main/services/chat/agentChatService.ts` | Persists chat metadata and transcripts, records provider-pointer transitions to the bounded thread-pointer ledger, reconciles missing pointers from ledger/resume command/transcript, gates new turns on disk pressure (`canPerform("chat_turn")`), and implements explicit `recoverContinuity` modes. | | `apps/desktop/src/main/services/chat/threadPointerLedger.ts` | Standalone append-only continuity ledger (`thread-pointers.jsonl`): typed `ThreadPointerLedgerEntry` records, tolerant parse that drops only a torn tail line, newest-per-session read, and 64 KiB self-compaction (newest records first) via an atomic rewrite. | | `apps/desktop/src/main/services/chat/providerResumeClassifier.ts` | Classifies a provider resume failure as missing thread, provider environment, transient transport, or unknown without treating every provider error as lost continuity. | @@ -22,6 +22,7 @@ | `apps/desktop/src/main/services/runtime/projectRecoveryService.ts` | Brain-independent diagnosis and ordered repair: space, ownership, database validation, migration recovery, service restart, endpoint/project verification, and chat reconciliation. Also owns `restartBrain()` — the machine-scoped restart behind the Connections **Repair** button — which shares one `restartServiceAndWait()` sequence (install → wait ≤90 s for the endpoint → `ping`) with `repair()`'s restart_service/verify_endpoint steps. The two are mutually exclusive: `restartBrain()` rejects while a `repair()` is in flight, because repair stops the service and then does exclusive database work that a reinstall would put a second writer on top of. A forced restart also treats a *skipped* install as a failure ("A newer ADE runtime is already running — quit and reopen ADE instead."), where `repair()` tolerates one, since a protocol-compatible brain that is already running satisfies its step. `main.ts` constructs exactly one of these and shares it with `registerIpc`, so the mutual exclusion actually holds — the post-update transaction's `restart` step (see [desktop auto-update](../onboarding-and-settings/desktop-auto-update.md#applying-an-update-is-one-transaction)) binds to the same instance rather than a second one that could run alongside a repair. | | `apps/desktop/src/main/services/storage/diskPressure.ts` | Samples all ADE storage roots, classifies pressure with recovery hysteresis, and gates write-producing operation classes via `canPerform(kind)`. Exports the `DiskPressureMonitor` type and refusal-message copy. | | `apps/desktop/src/main/services/storage/volume.ts` | `readVolumeSpace(dir)` (statfs free/total bytes) and `isNoSpaceError(err)` (ENOSPC/EDQUOT and disk-full message detection), shared by the pressure monitor and the database-open error classifier. | +| `apps/desktop/src/main/services/storage/cloudPlaceholder.ts` | The cloud-eviction preflight. `detectCloudStorageProvider` matches a path against the provider roots (`Library/Mobile Documents`, `Library/CloudStorage`, and home-relative `OneDrive` / `Dropbox` / `Google Drive` folders) on text alone, so it costs nothing on any platform; `isDatalessFileStats` spots a file with a size but zero allocated blocks, which is what a dehydrated placeholder looks like through `fs.stat`. `detectCloudPlaceholderFile` reports a finding only when both agree, and `storageUnreadableMessage` writes the one sentence a person needs — with the "move it out of the cloud folder" remedy stated conditionally when no provider was matched, because a failing disk or a dropped network mount produces the same unreadable file. | | `apps/desktop/src/main/services/storage/storageInsightsService.ts` | Builds categorized storage snapshots and preview-confirmed cleanup plans without following symlinks or deleting protected state. `proof_attachments` is a manual `review_first` cleanup target for `.ade/artifacts` and `.ade/attachments`; after bytes are removed it invokes the broker's `purgeArtifactRecordsUnder` hook so proof rows cannot outlive their files. It also runs the lane-lifecycle scan at the configured interval: safely archives excess or inactive lanes, marks old archived worktrees for review, and never removes lane files in the background. The **storage doctor** compresses history and maintains the database; filesystem candidates such as staging, backups, DerivedData, and build output remain review-first. Every run is journaled and emits one deduped `ade_feature_used` analytics event. Populates the snapshot's optional `extras` plus lifecycle policy/status and per-item ownership, age, blocked reasons, and reclaim estimates. | | `apps/desktop/src/main/services/lanes/laneService.ts` | Owns the lane-aware `getReclaimRisk`, `archiveAndReclaim`, and restore-aware `unarchive` operations. It proves exact path-and-branch ownership against this project's Git worktree registry, rejects symlinks, rechecks directory identity before removal, shares the database-backed lane worktree lease with PR workflows, and stores retryable reclaim failures locally. | | `apps/desktop/src/main/services/storage/storageLedger.ts` | The **storage ledger** (`STORAGE_LEDGER`): the declared policy for every persistent table and directory ADE writes — its privacy class (`user_data` / `derived` / `operational`) and how it is bounded (`write_time` / `doctor` / `both` / `manual`). `LEDGER_LAYOUT_COVERAGE` maps every `ADE_LAYOUT_DEFINITIONS` directory to a ledger id (or `null` for intentionally-unmanaged config/credentials) so a coverage test fails CI if a new tracked directory ships without a declared policy. `deriveCategoryPolicyChips()` renders the Settings policy chips from the ledger. | @@ -35,8 +36,9 @@ | `apps/desktop/src/main/services/diagnostics/diagnosticReportService.ts` | Desktop half of **Report issue**: shared machine sources plus the desktop's own jsonl logs, local runtime status, the recovery diagnosis for the open project, the typed last-failure store, and an Electron-aware volume reader. Saves the report `0600`, copies it, and opens a prefilled GitHub issue. | | `apps/ade-cli/src/services/diagnostics/diagnosticReport.ts` | The pure report builder, the redactor (`redactDiagnosticText`), and `buildDiagnosticIssueUrl`. No I/O, so both the desktop and the CLI produce byte-identical documents from the same sources. | | `apps/ade-cli/src/services/diagnostics/diagnosticSources.ts` | `collectMachineDiagnosticSources` — the machine-level logs, layout, disk figures and redaction context both surfaces read, so a log added for one appears in both. | -| `apps/ade-cli/src/commands/reportIssue.ts` | `ade report-issue [--open]`, the headless equivalent. Local files only: it never starts or contacts the brain, so it still works where ADE will not come up and on hosts with no error screen to press. | +| `apps/ade-cli/src/commands/reportIssue.ts` | `ade report-issue [--open] [--send]`, the headless equivalent. `--send` posts the same redacted report to ADE (Clerk token when the machine is signed in, anonymous otherwise) and prints a short reference id. Local files only: it never starts or contacts the brain, so it still works where ADE will not come up and on hosts with no error screen to press. | | `apps/ade-cli/src/lib/externalLinks.ts` | `normalizeExternalUrl` / `openExternalUrl` for the CLI: allows only `http(s)` and `mailto:`, opens through the platform helper (`open` / `rundll32` via the trusted-tool resolver / `xdg-open`), and falls back to Electron's `shell.openExternal` only when actually running inside Electron — a static `electron` import crashes headless startup. | +| `apps/desktop/src/shared/diagnosticsUpload.ts` | The one **Send to ADE** client, shared by the renderer button and the CLI: `uploadDiagnosticReport`, the `DiagnosticUploadFailure` vocabulary and its one-sentence copy, `resolveDiagnosticsUploadBaseUrl`, and `diagnosticReference` (the first 8 characters of the returned id — a full uuid is unreadable over a phone call). It lives in `shared/` because that is the only tree the renderer, the main process and the CLI can all import (Vite refuses to serve files outside `apps/desktop`), and it is deliberately free of Node built-ins and `import.meta` so the identical module loads in all three. It posts the report's exact bytes and transforms nothing: the thing that is sent has to be the thing that was shown. | | `apps/desktop/src/shared/types/diagnostics.ts` | The `DiagnosticSurface` / request / payload contract shared by main, preload and renderer. | | `apps/desktop/src/renderer/components/app/ReportIssueButton.tsx` | The button itself, on every error surface. One press assembles, saves, copies, and opens the issue; it reports what actually happened rather than claiming success. | | `apps/desktop/src/renderer/components/app/errorSurfaceKit.tsx` | Shared parts for the full-screen error surfaces — `ErrorSurfaceCard`, `WhatToDo`, `TechnicalDetailsFold`, `ERROR_PRIMARY_BUTTON` — so the recovery screen, the renderer/page boundaries and the CTO wake failure keep the raw text behind a fold and the plain-language account on top. | @@ -49,7 +51,8 @@ | `apps/desktop/src/renderer/components/settings/storage/storageView.ts` | Pure, DOM-free presentation + policy helpers. Category metadata/order/hues, safety labels, and `buildCleanupTarget` / `cleanableEntries` / `groupLaneItems` map a snapshot item to a typed `StorageCleanupTarget`. The overhaul adds the diagnostics/maintenance view-model: `dbBreakdownRows`, `buildSafeCleanupPlan`, journal/db-size-sparkline/trend helpers, `daemonMemoryBytes`, `healthChip`, `formatSlowActions`, and `categoryPolicyChip` — each degrading to a sensible "not available" value so the UI renders against an older daemon that never sends `extras`. | | `apps/desktop/src/shared/types/storage.ts` | Shared storage contracts: disk-pressure types, `StorageCategoryId`, `StorageSafety`, `StorageItem`/`StorageCategorySnapshot`/`StorageSnapshot`, and the `StorageCleanupTarget`/`StorageCleanupPreview`/`StorageCleanupResult` DTOs. `StorageItem` carries ownership, age, blocked reasons, reclaim estimate/state, and lane ownership for the review screen; `StorageLifecycleSnapshot` carries the effective four-rule policy plus last/next scan and review counts. The ledger/maintenance surface includes `StorageLedgerEntry`/`StoragePolicyClass`, `MaintenanceAction`/`MaintenanceRunReport`/`MaintenanceTrigger`, `DbBreakdownEntry`, `StorageSnapshotExtras`, and `RuntimeHealthSnapshot`. | | `apps/desktop/src/shared/types/recovery.ts` | Typed recovery contracts: the `AdeRecoveryErrorCode` union + `toAdeRecoveryErrorCode`, `AdeLastFailureReport`, `ProjectRecoveryDiagnosis`, the ordered `RepairStepId` list + `ProjectRepairReport`, and `mapKvDbOpenErrorCode`. | -| `apps/desktop/src/shared/codedError.ts` | `codedError(message, code)`, `encodeCodedErrorMessage`, and the `parseCodedErrorMessage`/`stripElectronErrorWrapper`/`extractCodeFromMessage` decoders that let the renderer recover a `code` through the Electron IPC error-wrapping. Re-exported to the renderer via `apps/desktop/src/renderer/lib/codedError.ts`. | +| `apps/desktop/src/shared/codedError.ts` | `codedError(message, code)`, `encodeCodedErrorMessage`, and the `parseCodedErrorMessage`/`stripElectronErrorWrapper`/`extractCodeFromMessage` decoders that let the renderer recover a `code` through the Electron IPC error-wrapping — and through the runtime RPC client's own `Remote ADE service method failed (code ):` wrapper, which the strip list has to name or no brain-side code survives the trip. It also draws the line between the two error vocabularies: `isErrnoLikeCode` recognises platform codes (`E…`, `ERR_…`, `MODULE_NOT_FOUND`) and `UNKNOWN_SYSTEM_ERRNO_PATTERN` recognises an errno libuv could not even name, neither of which can collide with ADE's lowercase snake_case codes. Re-exported to the renderer via `apps/desktop/src/renderer/lib/codedError.ts`. | +| `apps/ade-cli/src/jsonrpc.ts` | The brain's JSON-RPC server, and the boundary that decides which failures may be worded to a caller at all. A service verdict — a plain `Error` carrying a non-errno string `code` — is re-encoded as `code: message` (with the rootPath tail intact) in the shape `parseCodedErrorMessage` decodes, and the code is repeated in `error.data.code`. A platform or runtime fault (a `syscall`/`errno`, an errno-like code, an unnameable errno, a `TypeError`-class fault) is replaced on the wire by `Internal error in (ref )` and handed in full to `onInternalError`, which `cli.ts` writes to stderr — `launchd.err.log`, one of the logs `ade report-issue` tails — so the reference the user was shown is searchable. | ## Behavior @@ -83,6 +86,27 @@ renames it over the primary. A failed write does not replace the primary. `readJsonWithRecovery` accepts only payloads that pass the caller's validator and falls back from primary to the one `.lkg` generation. +`writeFileAtomic` never unlinks the destination first. `rename` replaces an +existing target in one step on every platform ADE ships to — libuv implements +`fs.rename` on Windows with `MoveFileExW(MOVEFILE_REPLACE_EXISTING)` — so the +delete-then-rename shape some writers reach for as "the Windows fix" would only +open a window in which the file does not exist at all, and a concurrent reader +looking during it sees neither generation. What Windows genuinely needs is the +other end: a rename refused because something else holds the target open (an +indexer, an antivirus scanner, a second ADE process), plus `EXDEV` for a temp +file that landed on another device. Those four codes — `EXDEV`, `EPERM`, +`EACCES`, `EBUSY` — fall back to a non-atomic `copyFileSync`, and the list is +closed on purpose: retrying `ENOSPC` or `EIO` as a copy would write the payload +a second time to a filesystem that just proved it cannot take it, turning a +clean "the write failed, the old file is intact" into a half-written target. A +caller that passes `mode` — today `0o600` from the machine sync-relay identity +store (`apps/ade-cli/src/services/sync/syncCloudRelayStore.ts`) — gets it on the +temp file, so the rename carries the bits onto the target and the secret is +never world-readable for even an instant; on the copy path the mode is reapplied +with a best-effort `chmod`. Directory fsync is +skipped on Windows rather than attempted and caught, because opening a directory +handle there fails outright. + Chat thread identity has three redundant sources: 1. Version-2 chat metadata in `.ade/cache/chat-sessions/.json`. @@ -118,7 +142,67 @@ service runs. `stateForCode` lives there too, so a screen falling back to the last recorded failure can never offer a different verdict, or a different repair offer, than the service would have given. -One diagnosis deliberately offers no repair: `brain_starting`, when the service +**A code is only worth attaching if it survives every wrapper on the way out.** The brain throws +`codedError(message, code)`; `apps/ade-cli/src/jsonrpc.ts` re-encodes it as +`code: message` (the rootPath, when there is one, rides after a NUL delimiter +that never appears in human-readable text) and repeats the code in +`error.data.code`; the desktop's runtime RPC client prefixes `Remote ADE service +method failed (code ):`; and Electron IPC strips custom `Error` +properties, so `registerIpc`'s `surfaceCodedError` re-encodes once more on the +way to the renderer. `stripElectronErrorWrapper` names all of those prefixes — +missing the runtime-RPC one is how a project whose data files were unreadable +reached the user as a raw libuv errno. The renderer's +`RECOVERY_MESSAGE_BY_CODE` (`renderer/state/appStore.ts`) then overrides the +brain's sentence only for codes where the screen genuinely knows more; a code +that is not listed keeps the brain's own wording as the headline, because for +`storage_read_failed` the brain's message is the one that names the file and the +fix, and a generic paraphrase would push it into the collapsed details fold. + +The same boundary decides what may *not* be worded to a caller. A failure that +came from the platform rather than from a service — one carrying `syscall` / +`errno`, an errno-like `code`, an errno libuv could not name, or a +`TypeError`-class runtime fault — is replaced on the wire with `Internal error +in (ref )` and reported in full through `onInternalError` to the +brain's stderr. Service verdicts are exempt: `JsonRpcError`s and plain sentences +like "Project root does not exist: …" are authored for the person who asked, and +blanking those would turn every actionable refusal into a reference number. The +split is `isErrnoLikeCode` in `shared/codedError.ts`, which is also why ADE's +codes are lowercase snake_case — the two vocabularies cannot collide. + +That same `Remote ADE service method …` prefix also decides what the desktop +retries. `isLocalRuntimeConnectionDropped` in `localRuntimeConnectionPool.ts` is the +predicate behind resetting the connection and re-running an action, so it has to +mean "the socket went away", not "the message mentioned a socket". It reads +`Error.code` for the transport errnos (`ECONNRESET`, `ECONNABORTED`, `EPIPE`, +`ENOTCONN`) rather than matching them in text, and it rejects anything shaped +like `Remote ADE service method failed …` first and unconditionally: that +prefix means the daemon answered, so the failure is the brain's and may quote a +transport sentence verbatim — retrying it would re-run a non-idempotent action +against a healthy daemon. + +**Unreadable storage is not a damaged database.** A project (or `~/.ade`) parked +in iCloud Drive, Dropbox or OneDrive whose contents the provider has evicted +answers a read with an errno the platform never names; on macOS it is `EDEADLK`, +which reaches the app as "Unknown system error -11: … read". `createAdeRuntime` +(`apps/ade-cli/src/bootstrap.ts`) runs `detectCloudPlaceholderFile` on the +database path *before* opening it and throws `storage_read_failed` with +`storageUnreadableMessage`, so the common case fails with the sentence that +names the fix rather than the errno. When the preflight declines to fire — the +file is materialized, or outside any known provider root — the open still fails +safe, because `classifySqliteOpenError` buckets `EDEADLK` / `EIO` / `ENXIO` / +`ENODEV` / `ESTALE` / `EHOSTDOWN` / `EREMOTEIO` and any unnameable errno into +the same code, ahead of the integrity check. Both paths record the same typed +failure and rethrow a coded error carrying the offending `dbPath` and the raw +errno as `detail`, rather than the bare libuv message. + +`storage_read_failed` maps to the `storage_unreadable` diagnosis, which offers +no repair on purpose: rewriting files ADE cannot read would risk the user's +work, and the same failure also arrives from a failing disk or a dropped network +mount. The recovery screen states the remedy as a condition ("if the folder is +in iCloud Drive, Dropbox or OneDrive, move it…"), lists moving the folder as a +prerequisite, and leaves Try again as the only action. + +One further diagnosis offers no repair: `brain_starting`, when the service is registered and its brain is alive but has not bound the socket yet (see [remote runtime](../remote-runtime/README.md)). There is nothing to fix and a repair would only kill a booting brain and restart its clock, so the screen @@ -519,7 +603,8 @@ rides the clipboard. | Desktop-only extras (its own jsonl logs, runtime status, recovery diagnosis, typed last-failure store) | `apps/desktop/src/main/services/diagnostics/diagnosticReportService.ts` | | IPC | `IPC.diagnosticsOpenIssue` | | Saved report | `/diagnostic-reports/-.md`, mode `0600` | -| Headless equivalent | `ade report-issue [--open]` | +| Headless equivalent | `ade report-issue [--open] [--send]` | +| Upload (opt-in) | `POST /diagnostics/upload` on the account directory Worker (`apps/account-directory/src/diagnostics.ts`); one client for both senders — the renderer button and the CLI — in `apps/desktop/src/shared/diagnosticsUpload.ts` | `ade report-issue` and the desktop button read the same machine sources through `collectMachineDiagnosticSources`, so a log added for one appears in both; the @@ -574,6 +659,40 @@ One coarse analytics event is emitted per press: `ade_feature_used { feature: "connections", action: "issue_report", outcome: "opened" | "failed" }`, deduped to one per hour per outcome. +**Send to ADE.** Filing on GitHub asks a user who is already looking at an error +screen to paste a document into a form; the second action on the result line — +and `ade report-issue --send` — posts the same finished report straight to +`POST /diagnostics/upload` on the account directory Worker and shows a short +reference id back. `apps/desktop/src/shared/diagnosticsUpload.ts` is the only +client: it takes an already-redacted string and a base URL its caller resolved, +and changes nothing about the bytes, because redaction happened once in the +builder and any transformation here would mean the thing that was sent is not +the thing that was shown. It also re-checks the 512 KB ceiling locally, so an +oversized report fails without spending one of the user's few daily uploads on a +doomed request. Failures come back as a small closed vocabulary — `too_large`, +`rate_limited`, `unavailable`, `rejected`, `network` — never a server string, +because the person reading it already hit one failure and a status line is not +an improvement on "couldn't send". Each maps to one plain sentence that points +back at GitHub where posting by hand is still the answer, or at tomorrow where +it is not; the CLI words the same reasons for a terminal line in +`describeDiagnosticUpload`. + +The two surfaces differ in exactly one way, and deliberately. The renderer runs +the upload itself (the diagnostics preload bridge exposes only `openIssue`, and +the renderer already holds the report that call returned), and it has no access +to an account token — those live in the brain's credential store — so a desktop +upload is anonymous, identified only by the install id the report already +carries. `ade report-issue --send` reads the machine's own credential store and +directory origin off local files, so it sends a Clerk token when the machine is +signed in and still works on a machine whose brain will not start; resolving the +origin the way the brain does also means a self-hosted machine's report and its +token are not silently redirected to ADE's directory. The Worker treats the body +as opaque: it never parses, indexes or echoes a report, which is what lets it +accept anonymous uploads at all, and it bounds one identity (Clerk user, else a +hash of the caller address) to five uploads a UTC day. The button's disclosure +text says so — nothing leaves the computer unless the user posts the issue or +chooses **Send to ADE**. + ## Gotchas - The interrupted-rebuild recovery pass **must run before `migrate()`**. An @@ -583,6 +702,19 @@ One coarse analytics event is emitted per press: every required unique and secondary index. - `PRAGMA foreign_keys` changes must happen outside a transaction; changing it inside an active transaction is ineffective. +- **A rename failure is not automatically retryable.** Only `EXDEV` / `EPERM` / + `EACCES` / `EBUSY` fall back to a copy in `writeFileAtomic`; widening that set + to `ENOSPC` or `EIO` writes the payload a second time to a filesystem that + just refused it and can leave a half-written target where a clean failure + would have left the previous file intact. +- **A new ADE error code must be lowercase snake_case.** `isErrnoLikeCode` + separates ADE's vocabulary from the platform's by shape alone, and the brain's + JSON-RPC boundary redacts anything it reads as platform-shaped. A code spelled + like an errno reaches the user as `Internal error in (ref …)` instead + of its own message. +- **`storage_read_failed` must stay ahead of the integrity bucket in + `classifySqliteOpenError`.** An unreadable file classified as a corrupt one + offers a repair that would rewrite data ADE could not even read. - launchd holds open descriptors for its stdout/stderr paths. Use copytruncate, not rename, or the service keeps writing to the unbounded old inode. diff --git a/docs/features/sync-and-multi-device/README.md b/docs/features/sync-and-multi-device/README.md index 8e7c577a4..da06b716b 100644 --- a/docs/features/sync-and-multi-device/README.md +++ b/docs/features/sync-and-multi-device/README.md @@ -235,7 +235,7 @@ itself and the snapshot builder may read through a different handle — without it, setting a PIN returned a snapshot claiming there wasn't one. Relay status is likewise one projection: `buildSyncCloudRelayStatus` in -`syncCloudRelayStore.ts`, used by both the scoped and machine paths so the +`syncCloudRelayStatus.ts`, used by both the scoped and machine paths so the desktop and the CLI cannot tell two different relay stories about one machine. Its `accountSignedIn` gate is *ownership*, not usability — see [remote runtime → Account state and reachability](../remote-runtime/README.md#account-state-and-reachability). @@ -563,8 +563,10 @@ Runtime support files outside `services/sync/`: the directory records a revocation before deleting the machine row and then asks the relay to purge that machine's Activity, and a failed purge surfaces as a typed `AccountMachineActivityPurgeError` (`machineRemoved: true`) rather than - a clean success. Getting back on requires `machinePairingRepair.ts` and proof - of a fresh interactive sign-in — see `push-notifications.md`. + a clean success. Getting back on requires `machinePairingRepair.ts` and either + proof of a fresh interactive sign-in or a spent pairing grant — pressed by the + user, or run unattended by `machinePairingAutoRecovery.ts` once the refusal has + outlived the ten-minute quiet window. See `push-notifications.md`. - `apps/desktop/src/renderer/webclient/workspace/WebMachineSessionManager.ts` and `workspace/webWorkspaceModel.ts` — hosted-browser directory/session projection. `mergeWebMachines` merges account rows with browser-saved @@ -632,6 +634,26 @@ Runtime support files outside `services/sync/`: (`decrypt_failure`, `no_os_key_material`, `store_format`, `session_parse`, `read_error`, `unknown`) that `accountAuthService.getSessionReadFailureReason()` supplies. See [logging](../../logging.md). + Every registration also carries `deviceId` and, when this host can produce + one, `hardwareId` — the two identifiers the directory dedups a rotated machine + key on. The anchor is read on the publish path rather than inside + `buildAccountMachineRegistration`, because the account id is its salt and the + builder has no account context (it also runs on the relay-state poll, which + only compares route signatures and sends nothing); a reader that throws or is + absent is an ordinary "no anchor" and never the reason a publication fails. + It is sent on the heartbeat as well as on a deliberate pairing, because a row + can only be matched later if it stored an anchor at some point — but storing + one authorizes nothing, and superseding still demands the same proof + un-revoking does. A directory that answers with `supersededMachineKeys` is + reporting which rows it retired for this device; the publisher hands them to + the identity store (`confirmSupersededMachineKeys`, on the *same* store + instance the machine key comes from, so a confirmation cannot be checked + against a different read of the same file), logs only the subset this machine + actually retired as `account.machine_identity_superseded_confirmed`, and + treats an older directory's empty or absent body as nothing to do. Nothing + downstream acts on it: it exists so an unexplained rotation is explainable, + which is what nobody could do the last time a working MacBook was deleted by + hand. Successful account sign-in also requests an immediate publish; the brain observes both its local auth event and cross-process credential-file changes from desktop sign-in. Separately, a lightweight 2-second observer computes a @@ -672,6 +694,77 @@ Runtime support files outside `services/sync/`: `ADE_ALLOW_DEVELOPMENT_CLERK=1` is the explicit controlled-testing escape hatch. Source-checkout runtimes and non-development custom issuers keep their existing override behavior. +- `apps/ade-cli/src/services/account/hardwareAnchor.ts` — the one piece of + machine identity a reinstall cannot destroy. Both halves of ADE's identity + live under `~/.ade` (the machine key in `sync-cloud-relay.json`, the device id + in `sync-device-id`), so a user who deletes that directory and signs in again + mints both afresh, the directory's device dedup has nothing to match on, and + the account keeps a phantom row for a computer the user owns once. The + operating system still knows this machine after the wipe — `IOPlatformUUID` + via `ioreg`, `MachineGuid` via the GLOBALROOT-resolved `reg.exe` (a bare + `reg` would let a planted binary choose this machine's identity), + `/etc/machine-id` or the dbus fallback on Linux. Three rules govern the + module. **The raw identifier never leaves the process**: what goes on the wire + is `sha256("ade-machine-anchor-v2:" + userId + ":" + rawUuid + ":" + + adeHomePath)`, salted with the account id, so one computer signed into two + accounts produces two unrelated values and no server-side join can correlate + them. **An anchor identifies an ADE install, not a chassis**: the platform + UUID is shared by every ADE on the box (Stable in `~/.ade`, Beta in + `~/.ade-beta`, a second OS user's home), and hashing it alone made all of them + one machine taking turns superseding each other's row, so the canonicalized + ADE home path is folded in — a wipe and reinstall lands on the same path and + still reproduces the same anchor, which is the whole point. On Windows that + path goes through `canonicalWindowsPath` and is lowercased, because an 8.3 + short name or a differently-cased spelling of one NTFS directory would + otherwise split one install into two machines. **It is optional end to end**: + a VM with no platform UUID, a hardened image with no machine-id, a sandbox + that refuses to spawn `ioreg` all yield null, and every caller behaves exactly + as it did before the anchor existed. The probe is bounded at 2 s and cached + for the process lifetime including the negative answer, so a machine with no + anchor does not respawn `ioreg` twice a minute. `normalizeHardwareAnchorUuid` + rejects the two ways these lookups "succeed" while saying nothing — an empty + value and the all-zero firmware sentinel, which would be shared by every + unprovisioned machine on the account. The `-v2` domain ships no migration and + needs none: a v1-hashed row simply stops matching and dedups on `device_id`, + as every pre-anchor client's row always did. +- `apps/ade-cli/src/services/account/machinePairingAutoRecovery.ts` — automatic + recovery from "this computer is not in your account any more". Both refusals + the directory can answer with are terminal for the heartbeat by design, and + were terminal for the machine too: the only way back was a human finding + **Reconnect this computer**, which nobody ever sees on a headless box. This + loop polls the publisher every 15 s and runs the identical brain action the + button runs — it widens nothing, so a genuine removal is refused exactly as it + is today. An episode starts on a latched refusal (`machine_revoked` or + `pairing_authentication_required`, decoded by the shared + `accountMachineRefusal.ts` so the repairer and the reporter cannot disagree + about what a response meant) or on a publish leg stuck in `snapshot_failed` + for two minutes, and attempts run 1 minute, 5 minutes, then hourly. Three + gates keep it from arguing with the user. A revocation younger than + `PAIRING_AUTO_REPAIR_REVOCATION_QUIET_MS` (10 minutes) is left alone: that + window deliberately mirrors `PAIRING_AUTH_FRESHNESS_MS` in the Worker's + `callerToken.ts`, because inside it the directory would still accept the + sign-in this machine authenticated with — so a repair sent then is precisely + the one that would succeed at undoing a removal the user just performed. The + budget is the persisted 6-hour allowance in the identity file, not a closure, + since the brain restarts far more often than six hours. And a repair with no + account session is not attempted at all, so the schedule slips instead of + burning budget proving it. A `snapshot_failed` episode gets exactly one cycle: + a publish leg that cannot read a snapshot is not a pairing problem. Everything + it does is logged (`account.machine_auto_repair_episode_started`, + `_started`, `_failed`, `account.machine_auto_repaired`, + `_budget_exhausted`, `_episode_ended`) and none of it changes user-visible + state — an exhausted budget simply stops arguing and leaves whatever the + publisher already reports. +- `apps/desktop/src/shared/accountMachineRefusal.ts` — `readAccountRefusalCode`, + the single decoder for "why did the directory refuse to register this + machine", read by the auto-recovery loop above and by the desktop's + reliability telemetry. **403 only**: a 401 is an authentication problem with a + different repair, and counting it as a refusal both mis-attributes the + incident and hides the auth failure behind it. A refusal is the directory + looking at a valid caller and saying no. An unrecognised 403 resolves to + `"other"` rather than to null — "turned away for a reason this build cannot + name" is exactly the fact the last incident needed — and the server's prose in + `lastHttpReason` never travels past this function. - `apps/ade-cli/src/services/power/` — the machine's own power and sleep truth, shared by the brain, the desktop main process, and tests. `machinePowerReader.ts` reads battery/wall power per platform (macOS @@ -750,7 +843,15 @@ Runtime support files outside `services/sync/`: `getLastReadState() === "unreadable"` with a coarse `getLastReadFailureReason()` of `decrypt_failure`, `no_os_key_material`, or `store_format`. It never writes an empty store over ciphertext it could not - decrypt. + decrypt. The Electron store records the same two verdicts, because it has one + branch that returns an empty view instead of throwing: an aborted legacy + migration, where there is no safeStorage file *and* the legacy file store + could not decrypt the one that exists. Returning `{}` there without saying so + is what let a machine with credentials on disk render as one that was never + signed in — see [onboarding and settings → GitHub connection + status](../onboarding-and-settings/README.md#github-connection-status-has-the-same-third-state), + where the same distinction drives the desktop's `credentialStoreUnreadable` + state. - `apps/ade-cli/src/services/credentials/osBoundKeyMaterial.ts` — everything about obtaining the machine-local secret the file store's key is derived from: the `security` invocations, the process-wide cache, the negative-cache @@ -772,7 +873,83 @@ Runtime support files outside `services/sync/`: trusted web-client CORS response exposes that header. Every request also receives a validated/generated `X-ADE-Correlation-ID`, echoed on the response and included in one privacy-safe structured completion log; trusted - web CORS exposes the id and allows the request header. + web CORS exposes the id and allows the request header. Registration also + **supersedes phantom duplicates**: because machines are keyed + `(user_id, machine_key)`, a client that rotates its identity file arrives as a + second row for one physical computer, and the owner then deletes whichever row + looks stale — half the time the live one. A register call whose `deviceId` + *or* `hardwareId` matches other rows on the same account deletes them (at most + `MAX_SUPERSEDED_MACHINES`, five, oldest-seen first) and reports them as + `supersededMachineKeys`, a field that is additive and omitted when empty. Both + identifiers are caller-supplied and therefore forgeable, so on a plain token + they authorize nothing: the call must carry the same proof un-revoking needs. + It **folds** rather than merely deleting — the one thing a superseded row + holds that the new one cannot rebuild is `custom_name`, so the most recently + seen superseded name is carried onto the survivor, and only onto a survivor + with no name of its own, since a name set on the new row is the fresher + statement of intent. The carry-forward and the deletes go out as one + `DB.batch()`, because the pairing grant is already spent by the time they run + and a half-finished loop would strand phantoms with no credential left to + clear them. Superseded keys get no `revoked_machines` row: the physical device + holds the new key, and blocking the old one would trapdoor any client that + rolls its identity file back; the relay is not called either, because the + device never left the account and its Activity is still the user's own. +- `apps/account-directory/src/callerToken.ts` — Clerk token verification for + every route that takes a caller bearer, and the definition of *proven-recent + interactive authentication*. `pairing: true` arrives in the request body, so + on its own it is an unauthenticated client boolean — a removed-but-still- + signed-in machine could set it on its next heartbeat and make the Worker a + confused deputy clearing its own removal. Authentication **time** is the + credential a removed machine cannot mint: a background heartbeat carries an + old authentication even after its access token is refreshed (a refresh renews + `exp`/`iat`, never the moment a human authenticated), while a real sign-in + carries a new one. `PAIRING_AUTH_FRESHNESS_MS` (10 minutes) is the bound, read + from `auth_time` or Clerk's `fva` and never from `iat`, and it fails closed — + a token with no such claim proves nothing, which is why a pairing grant exists + as the second path. The module declares the slice of the env it needs rather + than importing `Env`, which is what keeps an import cycle back into + `directory.ts` from forming. +- `apps/account-directory/src/pairingGrants.ts` — minting, reserving, consuming, + releasing, and expiring the single-use grants. A spend is **two phases**, not + one `DELETE`: an atomic `UPDATE ... SET reserved_at` whose `WHERE` still + carries every rule (this user, this machine, inside its TTL, not already + held), proven by `changes === 1`, and then either a scoped `DELETE` once the + relay agrees or `SET reserved_at = null` when it does not. Destroying the + grant before knowing the relay's answer meant a relay outage burned the only + credential a reinstalled machine had — the same lockout the grant exists to + prevent, moved one step later. A release restores the row exactly as it was, + `expires_at` included, so forcing relay failures buys an attacker nothing + beyond the original TTL; a reservation older than + `PAIRING_GRANT_RESERVATION_MS` (60 s) counts as unheld, so a Worker that dies + mid-hand-off strands the grant for a minute rather than until it expires. +- `apps/account-directory/src/activityRelay.ts`, `logging.ts`, + `trustedOrigin.ts` — the relay hand-off (revocation clear and Activity purge), + the structured-log helpers, and the CORS origin rules, split out of + `directory.ts` so each has one owner. Every refusal path emits exactly one + line — `directory.register_refused`, `.remove_refused`, or + `.supersede_refused` — carrying the wire `code` the client received, an + optional finer `reason` (`no_proof` versus `grant_rejected`, or the relay's + own failure text), and the request's correlation id. Every refusal is a user + who cannot get their computer back onto their account, and by the time they + ask for help the request is gone; Workers observability runs at + `head_sampling_rate: 1` so the line is always there. Identifiers appear as + **8-character prefixes only** — a machine key is capability-shaped and a grant + is a live credential. There is deliberately no admin restore route: it would + be a new authentication boundary guarding exactly the tables + `wrangler d1 execute --env production` already reaches, so support recovery is + a direct D1 statement after these logs identify the row. +- `apps/account-directory/src/diagnostics.ts` — `POST /diagnostics/upload`, the + write-only R2 sink behind the desktop's **Send to ADE** action and + `ade report-issue --send`. It is matched in `index.ts` *before* the directory + router, because it is the one route here that is not account-scoped and the + directory's exact-origin CORS rule and 404-on-unknown-`OPTIONS` fit neither an + unauthenticated Electron renderer nor a CLI. Authentication is optional but + never silently downgraded, the body is capped at 512 KB by both + `content-length` and a counted stream, and the quota is five a day per signed- + in user or per `cf-connecting-ip`. See + [storage and recovery → Diagnostic reports](../storage-and-recovery/README.md#diagnostic-reports-report-issue) + for the client half, and `apps/account-directory/README.md` for the full + contract and the R2 bucket + lifecycle setup the deploy does not do for you. - `apps/desktop/src/shared/accountDirectory.ts` — canonical account-directory origin, bounded success/error response decoding, route allowlisting, machine selection, and paired endpoint validation shared by desktop, the brain, ADE @@ -1659,21 +1836,54 @@ Canonical files (`apps/ade-cli/src/services/sync/`): the challenge signature input (`aead` field) so it cannot be downgraded by an on-path attacker. A client that sends no AEAD list, and both sides by default, fall back to `chacha20-poly1305`. -- `syncCloudRelayStore.ts` — persists the cloud tunnel-relay identity, and - exports `buildSyncCloudRelayStatus`, the one projection of relay state that - the desktop and the CLI read whether a project scope owns sync or the brain - answers for the bare machine. Both surfaces had their own copy and they had +- `syncCloudRelayStatus.ts` — `buildSyncCloudRelayStatus`, the one projection of + relay state that the desktop and the CLI read whether a project scope owns + sync (`syncService.ts`) or the brain answers for the bare machine + (`brainMachineSyncStores.ts`). Both surfaces had their own copy and they had already drifted in whitespace, one edit away from telling two different relay stories about one machine. `accountSignedIn` is the gate: without it the live fields collapse to their off values and `lastError` becomes the sign-in - prompt, so a signed-out machine never reports a connection it cannot have. - The identity itself lives at + prompt, so a signed-out machine never reports a connection it cannot have. It + is its own module rather than an export of the identity store because the two + have nothing in common but a name — one reads a file of secrets, the other + reshapes a status object — and the projection is imported by callers that + have no business constructing a store. +- `syncCloudRelayStore.ts` — persists the cloud tunnel-relay identity at `~/.ade/secrets/sync-cloud-relay.json` (lazily-minted 32-hex `machineKey` + - HMAC `secret`, chmod `0600`). The identity is stable in normal operation. + HMAC `secret`, chmod `0600`). Two rules govern every path through it, both + bought by a production lockout in which a live MacBook became a stranger to + its own account: **an identity is never discarded while any copy on disk still + holds it**, and every mint, rotation, or recovery leaves one + `sync_cloud_relay.identity_rotated` line naming what changed and why. So the + file is written through `writeFileAtomic` (0600 temp, fsync, rename, then a + parent-directory fsync everywhere but Windows, which has no directory handle + to flush — and never a pre-unlink, which would leave a concurrent reader + looking at nothing) and mirrored to a `.bak` sibling *after* the primary + lands, so a + reader that falls back finds the last identity actually in force. A parse + failure is reported as a failure rather than as an empty object: conflating + "this machine has no identity yet" with "this machine's identity is + temporarily unreadable" is what minted a whole new machine out of one corrupt + file. Resolution keeps the machine key from whichever copy still has one and + pairs the secret with its own key; only a file pair that yields nothing at all + may mint, and it is logged as `corrupt_file_remint` rather than `first_mint` + so the roster phantom it may create is explainable. + The identity is stable in normal operation. Only a claim endpoint response with the exact HTTP status `409` can trigger - the tunnel client's one-attempt recovery: the store serializes competing + the tunnel client's recovery: the store serializes competing brains with an exclusive sibling lock, compare-and-swaps the expected - `machineKey`, and mints a replacement key + secret. Generic network, auth, + `machineKey`, and mints a replacement key + secret — at most twice per rolling + 24 hours (`MAX_IDENTITY_ROTATIONS_PER_WINDOW`). That budget lives **in the + file**, not in a closure: an in-memory counter reset on every brain restart, + so a crash loop could mint one new machine row per boot and bury the owner's + roster in phantoms. A budget that cannot be parsed reads as spent, because + forgiving an unreadable counter is the same failure mode as not persisting it. + The file carries a second persisted allowance on the same terms — + `pairingAutoRepairs`, three per rolling six hours, spent by + `machinePairingAutoRecovery.ts` — and up to five `previousMachineKeys`, the + keys this machine actually retired, so `confirmSupersededMachineKeys` can tell + a directory confirming *our* rotation from one describing somebody else's + device; confirmed keys are then forgotten. Generic network, auth, upgrade, and bridge failures never rotate identity. Legacy `enabled` / `enabledSetByUser` fields are accepted only long enough to rewrite the file without them; there is no stored enablement or user kill-switch. The store @@ -1795,6 +2005,18 @@ Canonical files (`apps/ade-cli/src/services/sync/`): `.control_suppression_cleared`; one edge-triggered `ade_relay_suppressed` analytics event (coarse attempt count + `control_replaced` code, no URL, `machineKey`, or close reason) is captured per suppression episode. + A `409` claim conflict is the other regime. The client asks the identity store + to rotate, and when the store refuses because the persisted 24-hour budget is + spent it latches `identityRotationCapped` on its status and reports + `RELAY_IDENTITY_ROTATION_CAPPED_MESSAGE` — "This computer needs to be + reconnected to your ADE account." — in place of the raw `claim failed (409)`. + That is not a retry state but a product state: another mint would be another + phantom row on the owner's roster, and the row this machine already owns is + the one that has to be repaired, which is the only thing `claim failed (409)` + never says. `syncRouteHealth.ts` ranks that message above the raw close text + for the same reason it ranks `controlSuppressedReason` above it. The latch is + cleared by the next successful claim and republished, so a repaired machine + stops asking to be repaired without a restart. Control observability preserves the causal failure rather than replacing it with a generic WebSocket error: upgrade rejection captures the HTTP status and at most 512 sanitized response @@ -2327,6 +2549,62 @@ hostname and reachability lease but never overwrites that custom name. Clients preserve both values and use `customName`, then the reported hostname, as the display precedence. +### One computer, one row + +A directory row is keyed `(user_id, machine_key)`, and the machine key lives in +a file. Anything that replaces that file — a reinstall, a wiped config +directory, a restored backup, a relay claim conflict this machine recovered +from — therefore produces a *second row for one physical computer*. The owner +sees two, deletes the one that looks stale, and half the time that is the live +install. Three mechanisms, each in a different layer, keep that from happening: + +- **The key is hard to lose.** `sync-cloud-relay.json` is written durably and + mirrored to a `.bak` sibling, an unreadable file is distinguished from an + absent one, and a machine key is preserved from whichever copy still holds it. + A new identity is minted only when both copies yield nothing. +- **Rotations are budgeted and the budget is persisted.** Two per rolling 24 + hours, counted in the identity file itself so a crash loop cannot mint one row + per boot. A machine that spends the budget stops minting and says so — "This + computer needs to be reconnected to your ADE account" — instead of retrying. +- **The directory dedups what still gets through.** A register call that carries + proof of a fresh interactive sign-in (or spends a pairing grant) and whose + `deviceId` or `hardwareId` matches other rows on the account retires those + rows, folds the most recent user-typed `custom_name` onto the survivor, and + returns the retired keys. `hardwareId` is what covers the case `deviceId` + cannot: both the machine key and the device id live under `~/.ade`, so a full + wipe mints both afresh and matches nothing, while a per-account hash of an + OS-level machine identifier survives it. It is salted with the account id and + folded with the ADE home path, so it can neither correlate two accounts nor + merge a Beta install into Stable's row, and a host that cannot read one simply + omits it. + +### Getting back on after a refusal + +Removal is deliberately durable: the directory records a revocation before +deleting the row, so a removed machine that still holds a valid account token +cannot simply re-register itself. Getting back on needs a credential a removed +machine cannot mint — either an access token whose *interactive* authentication +happened within the last ten minutes, or the single-use pairing grant minted at +the end of a device-flow sign-in. Spending a grant is two-phase (reserve, then +consume or release) so a relay outage during the hand-off no longer burns the +one credential a reinstalled machine had. + +That repair no longer requires a human. `machinePairingAutoRecovery` runs the +same brain action the **Reconnect this computer** button runs, on a slow +budgeted schedule (1 minute, 5 minutes, then hourly; three repairs per rolling +six hours, persisted), for the two refusal codes and for a publish leg wedged in +`snapshot_failed`. It widens nothing — a genuine removal is refused exactly as +it would be interactively — and it holds off entirely while a revocation is less +than ten minutes old, which is the same window in which the directory would +still accept the machine's existing sign-in. Waiting that window out means the +only repair this loop can land is one granted on stale-but-valid grounds: a +stale row, a key rotation, a directory hiccup. A deliberate removal stands, and +recovering from it needs the user's next interactive sign-in. + +Every refusal the Worker issues is also logged with its wire code, a finer +`reason`, the correlation id, and 8-character identifier prefixes, because by +the time a locked-out user asks for help the request itself is long gone. + Account adoption captures the account owner/session generation and rechecks it before and after credential persistence so a late result cannot recreate trust after sign-out or an account switch. Once the host has minted a device-bound diff --git a/docs/features/sync-and-multi-device/push-notifications.md b/docs/features/sync-and-multi-device/push-notifications.md index 3097fc9b6..35564931a 100644 --- a/docs/features/sync-and-multi-device/push-notifications.md +++ b/docs/features/sync-and-multi-device/push-notifications.md @@ -399,7 +399,7 @@ code is typed as a plain string across the version boundary — a newer brain ma name a refusal an older desktop has never heard of, and anything unrecognized (including absence) must read as "unknown", never as "not that code". -Three entry points reach it: +Four entry points reach it: - `ade machines reconnect` (alias `repair`), which takes no machine selector because a brain can only lift its own machine's revocation. When the directory @@ -415,6 +415,13 @@ Three entry points reach it: device-login recovery when the directory demands fresh proof, and reports the honest outcome — including the case where the machine re-joined but push has not resumed. +- `machinePairingAutoRecovery`, the brain's own slow loop, which calls the same + function unattended once a refusal has been latched for a while. A headless + box has no Settings button to press, so without it a stale row or a key + rotation left the machine off the account permanently. It runs on a persisted + 6-hour budget and stays idle for the first ten minutes after a revocation, so + it can never undo a removal the user just performed — see *Getting back on + after a refusal* in [README.md](./README.md#getting-back-on-after-a-refusal). ## Brain publisher diff --git a/docs/logging.md b/docs/logging.md index 7925d100a..2b97110f1 100644 --- a/docs/logging.md +++ b/docs/logging.md @@ -139,6 +139,7 @@ The public contract is `apps/desktop/src/shared/types/productAnalytics.ts`. The - `ade_publish_failing` - `ade_relay_suppressed` - `ade_account_session_unreadable` +- `ade_brain_action_failed` The update and reliability events are low-frequency by construction: the five `ade_update_*` events fire at most once per install attempt or idle-apply cycle (daily caps 10–20, minute caps 3–6). `ade_update_install_did_not_land` is emitted once at startup when a requested install relaunched on the old version, so it is bounded by app launches that follow a failed handoff, and carries only a bounded `attempt` counter; `ade_brain_recovered` fires once per wedge recovery at brain startup; `ade_renderer_recovered` fires once per lost renderer and is bounded by the recovery budget itself (three reload attempts per rolling 60 seconds, after which the window stays down rather than looping), carrying only `crash_reason` — Electron's closed enum, normalized to `unknown` for any future value — and whether the reload was still allowed, never the window URL or title; `ade_publish_failing` is edge-triggered once per sustained failure episode (first crossing of two minutes), never per attempt. @@ -319,10 +320,61 @@ banner copy and in local logs. A per-outcome one-hour deduplication key bounds a click-loop to at most 24 accepted events per outcome — 48 across both — per installation per UTC day, inside the existing `ade_feature_used` and shared ceilings. The Activity feed's polling, rendering, section collapse, filters, and -acknowledgements, notch and iOS widget updates, machine removal, pairing-grant -mint and redeem, and relay control sweeps remain untracked: they are -high-frequency reads and UI mechanics, or they run on the relay and -account-directory surfaces that have no analytics path. +acknowledgements, notch and iOS widget updates, pairing-grant mint and redeem, +and relay control sweeps remain untracked: they are high-frequency reads and UI +mechanics, or they run on the relay and account-directory surfaces that have no +analytics path. + +Machine membership is two more coarse facts on the same `ade_feature_used` +event, added because a production incident — a machine revoked, then a brain +that would not boot — produced no analytics at all. + +Removing a computer from the account records `feature: "connections"`, +`action: "machine_removed"`, and a coarse `outcome`. It is captured in +`accountBridge.removeMachine`, not in the IPC handler, because only that +function knows which half failed: the directory delete is the authoritative +membership change, and the Activity purge that follows it rethrows so the user +can retry clearing it. `completed` therefore means the directory accepted the +removal, and `failed` means it did not. No machine key, display name, or account +identifier travels. + +The account directory refusing to register **this** computer records +`action: "machine_register_refused"`, `outcome: "failed"`, and `refusal_code` — +one of `machine_revoked`, `pairing_authentication_required`, or `other`. The +desktop can see this because the brain's publisher puts the machine-readable +code in `routeHealth.accountDirectory.lastHttpReason` alongside `http_error` and +a 401/403 (`accountMachinePublisherService`); the desktop never talks to the +directory itself. Any other 401/403 is reported as `other` rather than passing +the server's prose through, and non-refusals (timeouts, 5xx, transport failures) +are left to `ade_publish_failing`, which the brain already emits. The refusal is +a **state**, and the Connections pane and app shell both poll it on a timer, so +only the edge into a refusal is captured; a per-code one-hour key bounds the +case the in-process latch cannot see, an app or brain restarting inside the +refusal. Both events reuse the existing `ade_feature_used` 140-per-day / +30-per-minute limits and the shared 200-event ceiling; no ceiling was raised. + +`ade_brain_action_failed` is the one new event. Every brain action the desktop +performs goes through the single `ade.localRuntime.callAction` IPC channel, and +that channel is not a meaningful usage action, so the existing `ade_error` +capture in `registerIpc` has never fired for it — an installation whose brain +rejected every action was silent. The channel is deliberately **not** added to +`MEANINGFUL_ACTIONS`: that set defines the durable `usage_events` mutation +ledger, and joining it would write a mutation row per brain call. Instead the +`callAction` error path emits exactly two properties: `action_domain`, the ADE +action domain, allowlisted against the closed `ADE_ACTION_DOMAIN_NAMES` list in +`services/adeActions/domains.ts` — its own zero-import module, because +`registry.ts` pulls in the whole runtime service graph and the analytics policy +needs only the names, which is why it used to keep a hand-written copy of all +of them; and `error_code`, the structured code from +`codedError`/`Error.code`/the RPC `code:` prefix — the code only, never the +message, never a path, and `ipc_timeout` or `unknown` when there is no code. +Because codes are an open code-authored vocabulary (seeing an unpredicted one is +the point), `error_code` is bounded by shape rather than a literal allowlist: a +lower-case identifier of at most 48 characters, which no path, URL, email, +hostname, or sentence fragment can satisfy. A per-domain-per-code one-hour +deduplication key turns an error loop into one accepted event an hour, and the +event's own 20-per-day / 3-per-minute caps bound the rest without touching +`ade_error`'s budget. The default machine-wide ceiling is 200 accepted events per UTC day, shared across desktop, runtime, TUI, hosted web, and API-originated aggregates. Each event also has a tighter per-day and per-minute ceiling. Capture ingress is capped, noisy events use persisted deduplication windows, the in-memory transport queue is bounded, and the previous day's accepted/drop totals are summarized in at most two budget events per day.