diff --git a/devlog/_plan/260906_a_macos_verification/000_plan.md b/devlog/_plan/260906_a_macos_verification/000_plan.md new file mode 100644 index 0000000000..460d566fec --- /dev/null +++ b/devlog/_plan/260906_a_macos_verification/000_plan.md @@ -0,0 +1,5 @@ +# Final macOS verification repairs + +C4 spec-satisfaction repair of Unix probe cleanup classification. Consume the already reviewed replay-fixture commit7ff811ced to keep caller identity stable in the shared verification baseline. Main owns this new foundation PR below the two remaining A layers. No local suite/typecheck/build; all execution uses isolated remote Bun1.4.0 and CI. Existing GitHub/SSH identities and own branches only; no account/service/release changes. The only live processes exercised are temporary launchers created by the regression fixture. No additional termination signals or widened permissions are authorized. A2h checkpoint reassesses progress; no token/cost cap was specified. Detailed OS traces stay in ignored scratch. + +Goal: initial EPERM during an already-owned probe-group teardown does not prevent bounded observation of that group's disappearance. Success still requires an observed ESRCH. Persistent permission uncertainty or live groups continue to refuse installation and restore the launcher. Keep the existing one-second cleanup bound, one SIGKILL attempt, diagnostic sanitation and rollback guarantees. diff --git a/devlog/_plan/260906_a_macos_verification/010_cleanup_plan.md b/devlog/_plan/260906_a_macos_verification/010_cleanup_plan.md new file mode 100644 index 0000000000..403b1770e4 --- /dev/null +++ b/devlog/_plan/260906_a_macos_verification/010_cleanup_plan.md @@ -0,0 +1,11 @@ +# Diff-level cleanup plan + +1. Carry reviewed commit7ff811ced (test-only replay caller snapshots, forced second boundary and changed-token isolation) onto this dev foundation. Resolve only contextual offsets; do not introduce affinity production code or its cohort matrix. +2. MODIFY src/codex/shim.ts terminateUnixProcessGroup: retain the single initial SIGKILL. Save EPERM rather than immediately throwing it; other non-ESRCH errors still throw. Use the unchanged one-second passive signal-0 observation loop. An observed disappearance succeeds; if the group remains or cannot be observed, rethrow saved EPERM, otherwise retain the existing generic nontermination error. No new signal retry, timeout increase, cache change or test-only production export. +3. MODIFY tests/codex-integration/codex-shim.test.ts timeout rollback fixture. Keep its real native case, exact timeout message, restored launcher/no backup/no marker, native group-missing and child-dead/zombie assertions. Add scoped parent-only process.kill observation for its recorded negative PGID; unrelated calls delegate unchanged and spawned probes have independent native bindings. +4. Deterministic cases: SIGKILL throws sentinel EPERM then signal-0 EPERM→ESRCH must produce ordinary timeout refusal; persistent EPERM and continually-live signal-0 must retain fail-closed EPERM diagnostics. Assert one SIGKILL, actual passive probes, and the existing bounded runtime. Restore spies before native process cleanup proof; never count synthetic ESRCH as real cleanup. Passive bounded joining of the known fixture group is allowed for injected cases; native case retains its original immediate cleanup assertions. Finally restore environment/mocks and clean only fixture-owned paths/processes. +5. Emit bounded pid/state/error-code diagnostics on failure, with no commands, credentials or environment dumps. Actual CI EPERM is observed; the zombie-only-group explanation is a hypothesis, not claimed captured fact. +6. Remote proof: focused shim and replay/cache/security tests plus typecheck. Revert only the EPERM observation correction in remote scratch; the disappearing-group control must fail its exact diagnostic assertion. Candidate must pass transient, persistent and live controls, the native timeout integration and all original rollback checks. Restore source bytes. Independent implementation/security audit then exact-head full CI before admin landing. +7. Cascade verified foundation into affinity then capability, retain source-author commits, update PR bases before auto-deletion and reverify their current heads. No original remaining PR is closed before its change is on dev. Full current-head CI and final dev proof remain mandatory. + +Cleanup completion is not installation approval: existing timeout/recursive/descendant markers and the pre-cleanup group-survival result still refuse the launcher. The change only permits bounded absence proof before choosing the existing refusal diagnostic. No previously unsafe launcher is accepted. diff --git a/devlog/_plan/260906_a_macos_verification/020_direct_transport_watchdog.md b/devlog/_plan/260906_a_macos_verification/020_direct_transport_watchdog.md new file mode 100644 index 0000000000..708a1ce380 --- /dev/null +++ b/devlog/_plan/260906_a_macos_verification/020_direct_transport_watchdog.md @@ -0,0 +1,7 @@ +# Focused verification watchdog correction + +ClassC1: one test file, no production behavior or public API change. WindowsCIjob101361741694 hit the fixture's flat3000ms childwatchdog before routing assertions. The same child performs imports, an unbounded control fetch, two750ms probes and a2000ms read. The log cannot identify which stage consumed time. + +Use the existing CI-watchdog owner for a derived whole-child budget:3000ms startup +2000ms bounded control +750ms identity +750ms readiness +2000ms read +1000ms exit =9500ms. OnCI the existing30s/45s floor applies. Give the test itself the child budget plus1000ms cleanup. Add fixed child phase markers and bounded phase/request-count diagnostics, never capability values. Keep every exact routing/header assertion and existing per-operation budgets. Bound only the previously unbounded control fetch. + +Verify remotely on pinnedBun: originalfilechecks, an explicit3500ms pre-import delay underCI that succeeds withthecorrectbudget and fails withtheold3000ms guard, and an intentional memory-read misroute that fails the unchangedproxy/capability assertions despite valid-looking responses. No local execution. This is a causal verifier fix within the ongoing final landing repair loop, not an unconditional rerun or production timeout increase. diff --git a/devlog/_plan/260906_a_replay_credentials/000_plan.md b/devlog/_plan/260906_a_replay_credentials/000_plan.md new file mode 100644 index 0000000000..01b5f57117 --- /dev/null +++ b/devlog/_plan/260906_a_replay_credentials/000_plan.md @@ -0,0 +1,3 @@ +# Stable replay-fixture caller identity + +C2 spec-satisfaction repair of a concrete macOS control failure. Two logical replay conversations generated a new synthetic credential for each request; a second-boundary change made them different callers. Preserve production credential scope and every existing response/cache assertion. Only tests/server/server-agent-task-recovery-replay.test.ts and this numbered unit change. No local tests/typecheck/build; pinned remote Bun1.4 isolated regressions, deterministic old/new control, typecheck and current-head CI before final landing. Owner-authorized no-verify pushes/admin merge remain scoped to A. No credential or service changes. Same session goal/ledger owns this extra mandatory cycle; no completion criteria removed. diff --git a/devlog/_plan/260906_a_replay_credentials/010_replay_plan.md b/devlog/_plan/260906_a_replay_credentials/010_replay_plan.md new file mode 100644 index 0000000000..abe95054b7 --- /dev/null +++ b/devlog/_plan/260906_a_replay_credentials/010_replay_plan.md @@ -0,0 +1,9 @@ +# Replay fixture diff plan + +MODIFY tests/server/server-agent-task-recovery-replay.test.ts only: + +1. In the two original real-handler tests (cached NEW_TASK continuation and MESSAGE replay), capture one headers object before the first post and reuse it for the second. Keep status200, one recovery, two provider bodies, plaintext-present and ciphertext-absent assertions. +2. Scope a Date.now spy to each test at a real current second plus995ms. Advance controlled time by10ms between posts. Assert a newly constructed unused credential differs across that boundary, while the actual conversation continues with its original headers. Restore the clock in finally. No sleep or timeout increase. +3. Add a changed-token isolation control using the existing fakeChatGptJwt claim override: same account/envelope and two valid tokens differing in exp must not share cached plaintext. Reusing the original request still restores. Assert no extra network recovery and unchanged encrypted input on the miss. +4. Main performs exact-head remote isolated replay/cache/security tests and typecheck. A scratch red control restores per-post codexHeaders() calls while keeping the forced boundary; both conversations must lose the expected plaintext. The changed-token negative remains a pass. Restore candidate bytes after the probe. +5. Independent review checks fixture identity, clock cleanup and unchanged production boundary. Publish the own affinity branch, cascade the capability child and obtain fresh CI after all recorded verification repairs. Original source author commits remain intact. No new production file or test-layout entry. diff --git a/src/codex/shim.ts b/src/codex/shim.ts index f53f778b12..a63c48fb7a 100644 --- a/src/codex/shim.ts +++ b/src/codex/shim.ts @@ -961,14 +961,20 @@ function unixProcessGroupAlive(groupId: number): boolean { } function terminateUnixProcessGroup(groupId: number): void { + let permissionError: unknown; try { process.kill(-groupId, "SIGKILL"); } catch (error) { - if ((error as NodeJS.ErrnoException).code !== "ESRCH") throw error; + const code = (error as NodeJS.ErrnoException).code; + if (code === "EPERM") permissionError = error; + else if (code !== "ESRCH") throw error; } + // A concurrently exiting group can briefly reject a second signal. Only + // observed disappearance clears that uncertainty; never send another signal. const deadline = Date.now() + CODEX_SHIM_INSTALL_PROBE_EXIT_TIMEOUT_MS; while (Date.now() < deadline && unixProcessGroupAlive(groupId)) Bun.sleepSync(10); if (unixProcessGroupAlive(groupId)) { + if (permissionError) throw permissionError; throw new Error(`Codex shim install probe process group ${groupId} did not terminate`); } } diff --git a/tests/codex-integration/codex-shim.test.ts b/tests/codex-integration/codex-shim.test.ts index f413fbdffb..406ed945bd 100644 --- a/tests/codex-integration/codex-shim.test.ts +++ b/tests/codex-integration/codex-shim.test.ts @@ -1,4 +1,4 @@ -import { afterAll, describe, expect, test } from "bun:test"; +import { afterAll, describe, expect, spyOn, test } from "bun:test"; import { spawnSync } from "node:child_process"; import { chmodSync, copyFileSync, existsSync, linkSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, statSync, symlinkSync, utimesSync, writeFileSync } from "node:fs"; import { delimiter, dirname, join } from "node:path"; @@ -712,53 +712,149 @@ os._exit(0) }, ); - test("Unix install rolls back when launcher validation times out", () => { - if (process.platform === "win32") return; - - const binDir = mkdtempSync(join(tmpdir(), "ocx-shim-install-timeout-bin-")); - const home = mkdtempSync(join(tmpdir(), "ocx-shim-install-timeout-home-")); - const oldPath = process.env.PATH; - const oldHome = process.env.OPENCODEX_HOME; - const codexPath = join(binDir, "codex"); - const childPidPath = join(home, "probe-child.pid"); - const groupIdPath = join(home, "probe-group.pid"); - const original = `#!/bin/sh + for (const [name, mode] of [ + ["Unix install rolls back when launcher validation times out", "native"], + ["Unix timeout cleanup observes disappearance after EPERM without another signal", "disappears"], + ["Unix timeout cleanup preserves EPERM when passive probes keep failing", "permission"], + ["Unix timeout cleanup preserves EPERM when passive probes report a live group", "live"], + ] as const) { + test(name, () => { + if (process.platform === "win32") return; + + const binDir = mkdtempSync(join(tmpdir(), "ocx-shim-install-timeout-bin-")); + const home = mkdtempSync(join(tmpdir(), "ocx-shim-install-timeout-home-")); + const oldPath = process.env.PATH; + const oldHome = process.env.OPENCODEX_HOME; + const codexPath = join(binDir, "codex"); + const childPidPath = join(home, "probe-child.pid"); + const groupIdPath = join(home, "probe-group.pid"); + const original = `#!/bin/sh /bin/sleep 30 & child=$! printf '%s\\n' "$child" > "${childPidPath}" printf '%s\\n' "$$" > "${groupIdPath}" wait "$child" `; - try { - process.env.PATH = prependPath(binDir, oldPath); - process.env.OPENCODEX_HOME = home; - writeFileSync(codexPath, original, "utf8"); - chmodSync(codexPath, 0o755); + const nativeKill = process.kill.bind(process); + const permissionError = Object.assign(new Error("fixture termination denied"), { code: "EPERM" }); + let restoreKill: (() => void) | undefined; + let killCalls = 0; + let passiveProbes = 0; + let terminationStartedAt = 0; + let terminationElapsedMs = 0; + let childPid = 0; + let groupId = 0; + try { + process.env.PATH = prependPath(binDir, oldPath); + process.env.OPENCODEX_HOME = home; + writeFileSync(codexPath, original, "utf8"); + chmodSync(codexPath, 0o755); - const installed = installCodexShim(); + if (mode !== "native") { + const killSpy = spyOn(process, "kill").mockImplementation((pid, signal) => { + // The child writes its own group identity before the parent resumes from spawnSync. + if (groupId === 0 && existsSync(groupIdPath)) { + const recorded = Number.parseInt(readFileSync(groupIdPath, "utf8").trim(), 10); + if (Number.isInteger(recorded) && recorded > 1) groupId = recorded; + } + if (groupId <= 1 || pid !== -groupId) return nativeKill(pid, signal); + if (signal === "SIGKILL") { + killCalls += 1; + if (killCalls === 1) terminationStartedAt = Date.now(); + throw permissionError; + } + if (signal === 0 && killCalls > 0) { + passiveProbes += 1; + if (mode === "permission" || (mode === "disappears" && passiveProbes === 1)) { + throw permissionError; + } + if (mode === "disappears") { + throw Object.assign(new Error("fixture group disappeared"), { code: "ESRCH" }); + } + return true; + } + return nativeKill(pid, signal); + }); + restoreKill = () => { killSpy.mockRestore(); }; + } + let installed: ReturnType; + try { + installed = installCodexShim(); + terminationElapsedMs = Date.now() - terminationStartedAt; + } finally { + restoreKill?.(); + restoreKill = undefined; + if (mode !== "native" && existsSync(groupIdPath)) { + groupId = Number.parseInt(readFileSync(groupIdPath, "utf8").trim(), 10); + if (Number.isInteger(groupId) && groupId > 1) { + // Join only this fixture's real group, even when a later assertion fails. + // Synthetic ESRCH never proves cleanup; these observations use the native binding. + const deadline = Date.now() + 1_000; + while (Date.now() < deadline) { + try { nativeKill(-groupId, 0); } + catch (error) { + if ((error as NodeJS.ErrnoException).code === "ESRCH") break; + } + Bun.sleepSync(10); + } + } + } + } + childPid = Number.parseInt(readFileSync(childPidPath, "utf8").trim(), 10); + groupId = Number.parseInt(readFileSync(groupIdPath, "utf8").trim(), 10); - expect(installed.installed).toBe(false); - expect(installed.message).toContain("did not finish --version within 5000ms"); - expect(installed.message).toContain("original launcher was restored"); - expect(readFileSync(codexPath, "utf8")).toBe(original); - expect(existsSync(`${codexPath}.opencodex-real`)).toBe(false); - expect(existsSync(join(home, "codex-shim.json"))).toBe(false); - const childPid = Number.parseInt(readFileSync(childPidPath, "utf8").trim(), 10); - const groupId = Number.parseInt(readFileSync(groupIdPath, "utf8").trim(), 10); - expect(Number.isInteger(childPid)).toBe(true); - expect(Number.isInteger(groupId)).toBe(true); - expectProcessGroupMissing(groupId); - const childState = processState(childPid); - expect(childState === "" || childState.startsWith("Z")).toBe(true); - } finally { - if (oldPath === undefined) delete process.env.PATH; - else process.env.PATH = oldPath; - if (oldHome === undefined) delete process.env.OPENCODEX_HOME; - else process.env.OPENCODEX_HOME = oldHome; - removeTreeWithRetry(binDir); - removeTreeWithRetry(home); - } - }, 10_000); + expect(installed.installed).toBe(false); + if (mode === "native" || mode === "disappears") { + expect(installed.message).toContain("did not finish --version within 5000ms"); + } else { + expect(installed.message).toContain("[phase=termination; code=EPERM; status=124; signal=none]"); + expect(installed.message).not.toContain("did not finish --version within 5000ms"); + expect(terminationElapsedMs).toBeGreaterThanOrEqual(1_000); + } + if (mode !== "native") { + expect(killCalls).toBe(1); + expect(passiveProbes).toBeGreaterThanOrEqual(2); + } + expect(installed.message).toContain("original launcher was restored"); + expect(readFileSync(codexPath, "utf8")).toBe(original); + expect(existsSync(`${codexPath}.opencodex-real`)).toBe(false); + expect(existsSync(join(home, "codex-shim.json"))).toBe(false); + expect(Number.isInteger(childPid)).toBe(true); + expect(Number.isInteger(groupId)).toBe(true); + expect(childPid).toBeGreaterThan(1); + expect(groupId).toBeGreaterThan(1); + expectProcessGroupMissing(groupId); + const childState = processState(childPid); + expect(childState === "" || childState.startsWith("Z")).toBe(true); + } catch (error) { + restoreKill?.(); + restoreKill = undefined; + let groupState = "unrecorded"; + if (groupId > 1) { + try { nativeKill(-groupId, 0); groupState = "present"; } + catch (probeError) { + const code = (probeError as NodeJS.ErrnoException).code; + groupState = code === "ESRCH" || code === "EPERM" ? code : "other-error"; + } + } + let childState = "unrecorded"; + if (childPid > 1) { + try { childState = processState(childPid).replace(/[^A-Za-z+<>N]/g, "").slice(0, 16) || "absent"; } + catch { childState = "unavailable"; } + } + console.error("[shim-timeout-fixture]", { mode, groupId, childPid, groupState, childState, killCalls, passiveProbes }); + throw error; + } finally { + restoreKill?.(); + if (oldPath === undefined) delete process.env.PATH; + else process.env.PATH = oldPath; + if (oldHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = oldHome; + removeTreeWithRetry(binDir); + removeTreeWithRetry(home); + } + }, 10_000); + } test("Unix install preserves an existing backup without probing or mutation", () => { if (process.platform === "win32") return; diff --git a/tests/server/local-management-direct-transport.test.ts b/tests/server/local-management-direct-transport.test.ts index 2fda9cbcb3..c8c74ce4fa 100644 --- a/tests/server/local-management-direct-transport.test.ts +++ b/tests/server/local-management-direct-transport.test.ts @@ -6,9 +6,13 @@ import { join } from "node:path"; import { pathToFileURL } from "node:url"; import { directLocalHttpFetch } from "../../src/server/direct-local-http"; import { repoPath, repoRoot } from "../helpers/repo-root"; +import { watchdogMs } from "../helpers/ci-watchdog"; const PID = 4242; const SECRET = "A".repeat(43); +const CONTROL_TIMEOUT_MS = 2_000; +// Startup/imports + control + two liveness probes + capability read + process exit. +const DIRECT_CHILD_BUDGET_MS = watchdogMs(3_000 + CONTROL_TIMEOUT_MS + 750 + 750 + 2_000 + 1_000); async function listen(server: Server, hostname = "127.0.0.1"): Promise { return await new Promise((resolve, reject) => { @@ -299,14 +303,22 @@ describe("local management direct transport", () => { const localClientUrl = pathToFileURL(repoPath("src", "server", "local-management-read-client.ts")).href; const capabilityUrl = pathToFileURL(repoPath("src", "lib", "local-management-capability.ts")).href; const childSource = ` + const phase = name => console.error("DIRECT_PHASE:" + name); + phase("imports"); const liveness = await import(${JSON.stringify(proxyLivenessUrl)}); const client = await import(${JSON.stringify(localClientUrl)}); const capability = await import(${JSON.stringify(capabilityUrl)}); const port = ${targetPort}; const pid = ${PID}; - const control = await fetch(\`http://127.0.0.1:\${port}/__proxy-control\`).then(response => response.json()); + phase("control"); + const control = await fetch(\`http://127.0.0.1:\${port}/__proxy-control\`, { + signal: AbortSignal.timeout(${CONTROL_TIMEOUT_MS}), + }).then(response => response.json()); + phase("identity"); const identity = await liveness.proxyIdentityAt(port, { hostname: "127.0.0.1", expectedPid: pid }); + phase("readiness"); const readiness = await liveness.probeReadiness(port, { hostname: "127.0.0.1", expectedPid: pid }); + phase("memory"); const read = await client.fetchBoundLocalManagementRead( { hostname: "127.0.0.1", port, pid, source: "runtime" }, capability.LOCAL_MANAGEMENT_READ_PATHS.systemMemory, @@ -319,6 +331,7 @@ describe("local management direct transport", () => { const memory = read.kind === "response" ? await read.response.json() : null; const result = { control, identity, readiness, readKind: read.kind, memory }; console.log(JSON.stringify(result)); + phase("complete"); if (control?.via !== "proxy" || identity?.pid !== pid || readiness?.ready !== true || read.kind !== "response" || memory?.pid !== pid) { process.exitCode = 2; } @@ -341,13 +354,14 @@ describe("local management direct transport", () => { const childWatchdog = setTimeout(() => { childTimedOut = true; child.kill(); - }, 3_000); + }, DIRECT_CHILD_BUDGET_MS); const [exitCode, stdout, stderr] = await Promise.all([ child.exited, new Response(child.stdout).text(), new Response(child.stderr).text(), ]).finally(() => clearTimeout(childWatchdog)); - if (childTimedOut) throw new Error("direct-transport child timed out"); + const phase = [...stderr.matchAll(/DIRECT_PHASE:(imports|control|identity|readiness|memory|complete)/g)].at(-1)?.[1] ?? "startup"; + if (childTimedOut) throw new Error(`direct-transport child timed out (phase=${phase}; targetRequests=${targetPaths.length}; proxyRequests=${proxyPaths.length})`); if (exitCode !== 0) { throw new Error(`direct-transport child failed (${exitCode}): ${stderr.trim()}\n${stdout.trim()}`); } @@ -375,5 +389,5 @@ describe("local management direct transport", () => { if (proxyPort !== 0) await close(proxy); if (targetPort !== 0) await close(target); } - }); + }, DIRECT_CHILD_BUDGET_MS + 1_000); }); diff --git a/tests/server/server-agent-task-recovery-replay.test.ts b/tests/server/server-agent-task-recovery-replay.test.ts index fd572e9fd8..42821ac30f 100644 --- a/tests/server/server-agent-task-recovery-replay.test.ts +++ b/tests/server/server-agent-task-recovery-replay.test.ts @@ -6,7 +6,7 @@ import { bindTurnTerminationScope, rememberDeliveredFinalAnswer } from "../../sr import { conversationIdFromResponsesRequest } from "../../src/server/request-log-conversation"; import type { OcxParsedRequest } from "../../src/types"; import { recoverEncryptedAgentTask, resetAgentTaskRecoveryState, restoreCachedEncryptedAgentTasks } from "../../src/server/responses/agent-task-recovery"; -import { codexHeaders, encryptedInput, FERNET_TASK, SECOND_FERNET_TASK, originalFetch, recoverySse, routedConfig } from "../helpers/agent-task-recovery"; +import { codexHeaders, encryptedInput, fakeChatGptJwt, FERNET_TASK, SECOND_FERNET_TASK, originalFetch, recoverySse, routedConfig } from "../helpers/agent-task-recovery"; afterEach(() => { globalThis.fetch = originalFetch; resetAgentTaskRecoveryState(); }); test("replay reuses admitted recovery after a tool result without another network call", async () => { @@ -51,12 +51,22 @@ test("Responses handler restores a cached task in a continued child turn", async return providerResponse(); }) as typeof fetch; const config = routedConfig({ enabled: true }); - expect((await post(config, "xai/grok-4.5", encryptedInput(), codexHeaders())).status).toBe(200); - expect((await post(config, "xai/grok-4.5", [...encryptedInput(), { type: "message", role: "user", content: "Continue the original task." }], codexHeaders())).status).toBe(200); - expect(recoveries).toBe(1); - expect(bodies).toHaveLength(2); - expect(bodies[1]).toContain("Read nonce.txt exactly."); - expect(bodies[1]).not.toContain(FERNET_TASK); + let now = Math.floor(Date.now() / 1_000) * 1_000 + 995; + const clock = spyOn(Date, "now").mockImplementation(() => now); + try { + const headers = codexHeaders(); + expect((await post(config, "xai/grok-4.5", encryptedInput(), headers)).status).toBe(200); + now += 10; + // A freshly generated fixture JWT would be a different caller across this boundary. + expect(codexHeaders().get("authorization")).not.toBe(headers.get("authorization")); + expect((await post(config, "xai/grok-4.5", [...encryptedInput(), { type: "message", role: "user", content: "Continue the original task." }], headers)).status).toBe(200); + expect(recoveries).toBe(1); + expect(bodies).toHaveLength(2); + expect(bodies[1]).toContain("Read nonce.txt exactly."); + expect(bodies[1]).not.toContain(FERNET_TASK); + } finally { + clock.mockRestore(); + } }); function encryptedMessage(): unknown[] { @@ -77,18 +87,57 @@ test("MESSAGE recovery reaches the provider and survives tool-result replay", as return providerResponse(); }) as typeof fetch; const config = routedConfig({ enabled: true }); - expect((await post(config, "xai/grok-4.5", encryptedMessage(), codexHeaders())).status).toBe(200); - expect((await post(config, "xai/grok-4.5", [...encryptedMessage(), { - type: "message", role: "user", content: "Continue after the tool result.", - }], codexHeaders())).status).toBe(200); - expect(recoveries).toBe(1); - expect(bodies).toHaveLength(2); - for (const body of bodies) { - expect(body).toContain("Stop waiting and report your result."); - expect(body).not.toContain(FERNET_TASK); + let now = Math.floor(Date.now() / 1_000) * 1_000 + 995; + const clock = spyOn(Date, "now").mockImplementation(() => now); + try { + const headers = codexHeaders(); + expect((await post(config, "xai/grok-4.5", encryptedMessage(), headers)).status).toBe(200); + now += 10; + expect(codexHeaders().get("authorization")).not.toBe(headers.get("authorization")); + expect((await post(config, "xai/grok-4.5", [...encryptedMessage(), { + type: "message", role: "user", content: "Continue after the tool result.", + }], headers)).status).toBe(200); + expect(recoveries).toBe(1); + expect(bodies).toHaveLength(2); + for (const body of bodies) { + expect(body).toContain("Stop waiting and report your result."); + expect(body).not.toContain(FERNET_TASK); + } + } finally { + clock.mockRestore(); } }); +test("a changed valid token cannot read another credential snapshot's recovery", async () => { + let recoveries = 0; + globalThis.fetch = (async () => { + recoveries++; + return new Response(recoverySse("Original caller assignment.")); + }) as typeof fetch; + const config = routedConfig({ enabled: true }); + const exp = Math.floor(Date.now() / 1_000) + 3_600; + const headers = codexHeaders("acct-caller"); + headers.set("authorization", `Bearer ${fakeChatGptJwt("acct-caller", { exp })}`); + const req = new Request("http://localhost/v1/responses", { headers }); + expect(await recoverEncryptedAgentTask(req, encryptedInput(), {}, config)).toBe(true); + + const changedHeaders = new Headers(headers); + changedHeaders.set("authorization", `Bearer ${fakeChatGptJwt("acct-caller", { exp: exp + 1 })}`); + expect(changedHeaders.get("authorization")).not.toBe(headers.get("authorization")); + const changedCallerInput = encryptedInput(); + expect(restoreCachedEncryptedAgentTasks(new Request("http://localhost/v1/responses", { + headers: changedHeaders, + }), changedCallerInput, config)).toBe(0); + expect(JSON.stringify(changedCallerInput)).toContain(FERNET_TASK); + expect(JSON.stringify(changedCallerInput)).not.toContain("Original caller assignment."); + + const sameCallerInput = encryptedInput(); + expect(restoreCachedEncryptedAgentTasks(req, sameCallerInput, config)).toBe(1); + expect(JSON.stringify(sameCallerInput)).toContain("Original caller assignment."); + expect(JSON.stringify(sameCallerInput)).not.toContain(FERNET_TASK); + expect(recoveries).toBe(1); +}); + test("MESSAGE cache remains isolated by message type, account, parent and sender", async () => { let calls = 0; globalThis.fetch = (async () => { calls++; return new Response(recoverySse("Private message.")); }) as typeof fetch;