diff --git a/.github/pr-assets/3863-storage-skip-referenced.png b/.github/pr-assets/3863-storage-skip-referenced.png new file mode 100644 index 0000000000..0fcc6370a4 Binary files /dev/null and b/.github/pr-assets/3863-storage-skip-referenced.png differ diff --git a/.github/pr-assets/muse-spark-meta-search-content-types-400.jpg b/.github/pr-assets/muse-spark-meta-search-content-types-400.jpg new file mode 100644 index 0000000000..d18dd98dab Binary files /dev/null and b/.github/pr-assets/muse-spark-meta-search-content-types-400.jpg differ diff --git a/docs-site/src/content/docs/fr/reference/cli/lifecycle.md b/docs-site/src/content/docs/fr/reference/cli/lifecycle.md index 59652bbaef..d87aa4895a 100644 --- a/docs-site/src/content/docs/fr/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/fr/reference/cli/lifecycle.md @@ -53,6 +53,10 @@ Récupération explicite destinée aux anciennes versions de développement qui Il s'agit d'un réétiquetage large et destructif : chaque fil contenant un message utilisateur et actuellement marqué `opencodex` passe à `openai`, `exec` est normalisé en `cli` et l'indicateur d'événement est activé. L'historique légitime d'un fournisseur dédié est également concerné. Sauvegardez l'état et n'exécutez la commande que si vous souhaitez cette portée complète. +### `ocx recover-history --ocx-compaction --yes` + +Réparez l'historique d'une tâche compactée par un fournisseur routé avant de la reprendre avec Codex natif. La commande sélectionne exactement une tâche par UUID, enregistre d'abord une sauvegarde privée octet par octet, puis convertit uniquement l'état de compaction `ocx1:` propre à OpenCodeX en résumé ordinaire relisible par Codex natif. Le contenu chiffré natif et les autres tâches restent inchangés. Fermez la tâche sélectionnée avant d'exécuter la commande ; toute modification simultanée du rollout interrompt la récupération sans remplacer le fichier. + ### `ocx uninstall` · `ocx remove` Arrête le service et le proxy, supprime le service et le shim Codex, rétablit le fonctionnement natif de Codex, puis supprime la configuration locale d’opencodex uniquement si toutes les étapes de restauration ont réussi. `remove` est un alias de `uninstall`. Le nettoyage de la configuration exige les métadonnées de propriété créées par une installation récente ; les répertoires anciens ou partagés sont conservés. diff --git a/docs-site/src/content/docs/guides/codex-app-models.md b/docs-site/src/content/docs/guides/codex-app-models.md index bcda44080b..6f9b756789 100644 --- a/docs-site/src/content/docs/guides/codex-app-models.md +++ b/docs-site/src/content/docs/guides/codex-app-models.md @@ -319,3 +319,5 @@ ocx sync opencodex rewrites `models_cache.json` with a deliberately stale cache wrapper whenever catalog visibility, priority, or metadata changes, so the next Codex model refresh reads the new catalog. + +After a catalog or model-cache write, OpenCodex invalidates its cached app-server observation so the next request checks process freshness again. A configuration sync also invalidates the observation when catalog contents are unchanged. This refresh does not restart Codex processes. diff --git a/docs-site/src/content/docs/ja/reference/cli/lifecycle.md b/docs-site/src/content/docs/ja/reference/cli/lifecycle.md index b7952b5c28..9f8612a2c6 100644 --- a/docs-site/src/content/docs/ja/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/ja/reference/cli/lifecycle.md @@ -53,6 +53,10 @@ ocx eject back これは広範囲で破壊的な再ラベル付けです。ユーザーメッセージを持ち、現在 `opencodex` とタグ付けされているすべてのスレッドを `openai` に変更し、`exec` を `cli` に正規化してイベントマーカーを設定します。正当な専用プロバイダー履歴も対象です。状態をバックアップし、この全範囲を意図する場合にのみ実行してください。 +### `ocx recover-history --ocx-compaction --yes` + +ルーティングされたプロバイダーで圧縮されたタスクをネイティブ Codex で再開する前に、その履歴を修復します。このコマンドは UUID で 1 つのタスクだけを選択し、非公開のバイト単位バックアップを保存してから、OpenCodeX 所有の `ocx1:` 圧縮状態だけをネイティブ Codex が再生できる通常の要約に変換します。ネイティブの暗号化コンテンツと他のタスクは変更しません。実行前に対象タスクを閉じてください。処理中に rollout が変更された場合、ファイルを置き換えずに修復を中止します。 + ### `ocx uninstall`・`ocx remove` すべての復元手順が成功した場合にのみ、サービスとプロキシを停止し、サービスと Codex シムを削除し、ネイティブ Codex を復元してから、opencodex ローカル設定を削除します。 `remove` は `uninstall` の別名です。設定のクリーンアップには、新規インストールによって作成された所有権メタデータが必要です。従来のディレクトリまたは共有ディレクトリはそのまま残ります。 diff --git a/docs-site/src/content/docs/ko/reference/cli/lifecycle.md b/docs-site/src/content/docs/ko/reference/cli/lifecycle.md index 796a3d7a12..15a8bfee78 100644 --- a/docs-site/src/content/docs/ko/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/ko/reference/cli/lifecycle.md @@ -75,6 +75,10 @@ ocx eject back thread를 `openai`로 바꾸고, `exec`를 `cli`로 정규화하며 event marker를 설정합니다. 정상적인 dedicated-provider history도 포함됩니다. 상태를 백업하고 이 전체 범위를 의도한 경우에만 실행하세요. +### `ocx recover-history --ocx-compaction --yes` + +라우팅된 provider를 통해 압축된 작업을 native Codex에서 다시 열기 전에 해당 기록을 복구합니다. 이 명령은 UUID로 정확히 하나의 작업을 선택하고 비공개 바이트 단위 백업을 저장한 뒤, OpenCodeX가 소유한 `ocx1:` 압축 상태만 native Codex가 재생할 수 있는 일반 요약으로 변환합니다. native 암호화 콘텐츠와 다른 작업은 변경하지 않습니다. 실행 전에 선택한 작업을 닫으십시오. 처리 중 rollout이 변경되면 파일을 교체하지 않고 복구를 중단합니다. + ### `ocx uninstall` · `ocx remove` 서비스와 프록시를 중지하고, 서비스와 Codex shim을 제거한 뒤, 기본 Codex를 복원합니다. 그 다음 diff --git a/docs-site/src/content/docs/reference/cli/lifecycle.md b/docs-site/src/content/docs/reference/cli/lifecycle.md index 92253e41f8..13a5b17616 100644 --- a/docs-site/src/content/docs/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/reference/cli/lifecycle.md @@ -81,6 +81,15 @@ changed to `openai`, `exec` is normalized to `cli`, and the event marker is set. legitimate dedicated-provider history. Back up the state and run it only when that full scope is intended. +### `ocx recover-history --ocx-compaction --yes` + +Repair one thread that was compacted through a routed provider before resuming it through native +Codex. The command reads the exact thread selected by UUID, saves a private byte-for-byte backup, +then converts only OpenCodeX-owned `ocx1:` compaction state into a plain summary that native Codex +can replay. Native encrypted content and other threads are left unchanged. Close the selected +thread before running the command; a concurrent rollout change makes recovery stop without +replacing the file. + ### `ocx uninstall` · `ocx remove` Stop the service and proxy, remove the service and Codex shim, restore native Codex, then remove diff --git a/docs-site/src/content/docs/ru/reference/cli/lifecycle.md b/docs-site/src/content/docs/ru/reference/cli/lifecycle.md index f743f5f468..e72d956bcd 100644 --- a/docs-site/src/content/docs/ru/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/ru/reference/cli/lifecycle.md @@ -78,6 +78,10 @@ ocx eject back устанавливается. Корректная история выделенного провайдера тоже входит в охват. Сначала сделайте резервную копию и запускайте команду только если нужен весь этот охват. +### `ocx recover-history --ocx-compaction --yes` + +Исправьте историю одной задачи, сжатой через маршрутизируемого провайдера, перед её возобновлением в нативном Codex. Команда выбирает ровно одну задачу по UUID, сначала сохраняет приватную побайтовую резервную копию, а затем преобразует только принадлежащее OpenCodeX состояние сжатия `ocx1:` в обычную сводку, которую может воспроизвести нативный Codex. Нативное зашифрованное содержимое и другие задачи не изменяются. Перед запуском закройте выбранную задачу; если rollout изменится во время обработки, восстановление остановится без замены файла. + ### `ocx uninstall` · `ocx remove` Остановить службу и прокси, удалить службу и Codex shim, восстановить native Codex, а затем diff --git a/docs-site/src/content/docs/tr/reference/cli/lifecycle.md b/docs-site/src/content/docs/tr/reference/cli/lifecycle.md index 6a7a565139..0aa50bfebf 100644 --- a/docs-site/src/content/docs/tr/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/tr/reference/cli/lifecycle.md @@ -84,6 +84,10 @@ Bu, geniş kapsamlı ve yıkıcı bir yeniden etiketlemedir: kullanıcı iletisi olarak normalleştirilir ve event marker ayarlanır. Geçerli dedicated-provider geçmişi de kapsama dahildir. Durumu yedekleyin ve yalnızca bu kapsamın tamamını istiyorsanız çalıştırın. +### `ocx recover-history --ocx-compaction --yes` + +Yönlendirilmiş bir sağlayıcı üzerinden sıkıştırılmış bir görevi yerel Codex ile sürdürmeden önce geçmişini onarın. Komut UUID ile yalnızca bir görevi seçer, önce özel ve bayt bayt bir yedek kaydeder, ardından yalnızca OpenCodeX'e ait `ocx1:` sıkıştırma durumunu yerel Codex'in yeniden oynatabileceği düz bir özete dönüştürür. Yerel şifreli içerik ve diğer görevler değişmeden kalır. Komutu çalıştırmadan önce seçili görevi kapatın; işlem sırasında rollout değişirse kurtarma dosyayı değiştirmeden durur. + ### `ocx uninstall` · `ocx remove` Servisi ve proxy'yi durdurun, servisi ve Codex dolgusunu kaldırın, yerel Codex'i diff --git a/docs-site/src/content/docs/zh-cn/reference/cli/lifecycle.md b/docs-site/src/content/docs/zh-cn/reference/cli/lifecycle.md index 5977c734de..541485dbf4 100644 --- a/docs-site/src/content/docs/zh-cn/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/zh-cn/reference/cli/lifecycle.md @@ -53,6 +53,10 @@ ocx eject back 这是范围很广且具有破坏性的重标记：所有包含用户消息且当前标记为 `opencodex` 的线程都会改标为 `openai`，`exec` 会规范化为 `cli`，并设置事件标记。正常的专用提供方历史记录也在范围内。请先备份状态，并且仅在确实需要这一完整范围时执行。 +### `ocx recover-history --ocx-compaction --yes` + +在通过原生 Codex 恢复某个曾由路由提供方压缩的任务前，修复该任务的历史记录。此命令按 UUID 精确选择一个任务，先保存私有的逐字节备份，然后仅将 OpenCodeX 自有的 `ocx1:` 压缩状态转换为原生 Codex 可重放的普通摘要。原生加密内容和其他任务保持不变。运行前请关闭所选任务；如果 rollout 在处理期间发生变化，恢复会停止且不会替换原文件。 + ### `ocx uninstall` · `ocx remove` 停止服务和代理，移除服务和 Codex shim，恢复原生 Codex，然后仅在所有恢复步骤都成功时才删除 opencodex 本地配置。`remove` 是 `uninstall` 的别名。配置清理需要由全新安装创建的所有权元数据；旧版或共享目录会保留原样。 diff --git a/docs-site/src/content/docs/zh-tw/reference/cli/lifecycle.md b/docs-site/src/content/docs/zh-tw/reference/cli/lifecycle.md index 71d575e774..0eaf6c75f9 100644 --- a/docs-site/src/content/docs/zh-tw/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/zh-tw/reference/cli/lifecycle.md @@ -52,6 +52,10 @@ ocx eject back 這是範圍很廣且具破壞性的重新標記：所有含有使用者訊息且目前標記為 `opencodex` 的 thread 都會改標為 `openai`，`exec` 會正規化為 `cli`，並設定 event marker。正常的專用 provider 歷史也包含在內。請先備份狀態，而且只有在確實需要這個完整範圍時才執行。 +### `ocx recover-history --ocx-compaction --yes` + +在透過原生 Codex 恢復曾由路由提供方壓縮的工作前，修復該工作的歷史記錄。此命令依 UUID 精確選取一個工作，先儲存私有的逐位元組備份，然後只把 OpenCodeX 自有的 `ocx1:` 壓縮狀態轉換成原生 Codex 可重播的普通摘要。原生加密內容與其他工作不會變更。執行前請關閉所選工作；若 rollout 在處理期間發生變化，復原會停止且不會取代原始檔案。 + ### `ocx uninstall` · `ocx remove` 停止服務與代理、移除服務與 Codex shim、還原原生 Codex，然後僅在所有還原步驟成功時移除 opencodex 本機設定。`remove` 是 `uninstall` 的別名。設定清理需要由全新安裝建立的擁有權中繼資料；舊版或共享目錄會被原樣保留。 diff --git a/gui/src/i18n/de.ts b/gui/src/i18n/de.ts index 40ee6ea36f..2a889b8f2b 100644 --- a/gui/src/i18n/de.ts +++ b/gui/src/i18n/de.ts @@ -1777,6 +1777,7 @@ export const de: Record = { "storage.cleanup.confirmPermanent": "Dauerhaft löschen", "storage.cleanup.doneQuarantine": "{count} Datei(en) in Quarantäne ({size}).", "storage.cleanup.donePermanent": "{count} Datei(en) dauerhaft gelöscht ({size}).", + "storage.cleanup.skippedReferenced": "{count} referenzierte Datei(en) übersprungen.", "storage.cleanup.previewFailed": "Vorschau fehlgeschlagen.", "storage.cleanup.cleanupFailed": "Bereinigung fehlgeschlagen.", "storage.cleanup.err.codex_busy": "Codex verwendet state.sqlite — beende Codex und versuche es erneut.", diff --git a/gui/src/i18n/en.ts b/gui/src/i18n/en.ts index 53d940d7e6..0cf7469f56 100644 --- a/gui/src/i18n/en.ts +++ b/gui/src/i18n/en.ts @@ -1010,6 +1010,7 @@ export const en = { "storage.cleanup.confirmPermanent": "Delete permanently", "storage.cleanup.doneQuarantine": "Quarantined {count} file(s) ({size}).", "storage.cleanup.donePermanent": "Permanently deleted {count} file(s) ({size}).", + "storage.cleanup.skippedReferenced": "Skipped {count} referenced file(s).", "storage.cleanup.previewFailed": "Preview failed.", "storage.cleanup.cleanupFailed": "Cleanup failed.", "storage.cleanup.err.codex_busy": "Codex is using state.sqlite — try again after quitting Codex.", diff --git a/gui/src/i18n/fr.ts b/gui/src/i18n/fr.ts index 7afb44ab2c..576c3b7f23 100644 --- a/gui/src/i18n/fr.ts +++ b/gui/src/i18n/fr.ts @@ -986,6 +986,7 @@ export const fr: Record = { "storage.cleanup.confirmPermanent": "Supprimer définitivement", "storage.cleanup.doneQuarantine": "{count} fichier(s) mis en quarantaine ({size}).", "storage.cleanup.donePermanent": "{count} fichier(s) supprimé(s) définitivement ({size}).", + "storage.cleanup.skippedReferenced": "{count} fichier(s) référencé(s) ignoré(s).", "storage.cleanup.previewFailed": "Échec de l’aperçu.", "storage.cleanup.cleanupFailed": "Échec du nettoyage.", "storage.cleanup.err.codex_busy": "Codex utilise state.sqlite — réessayez après avoir quitté Codex.", diff --git a/gui/src/i18n/ja.ts b/gui/src/i18n/ja.ts index 75095da332..1e01aea545 100644 --- a/gui/src/i18n/ja.ts +++ b/gui/src/i18n/ja.ts @@ -925,6 +925,7 @@ export const ja: Record = { "storage.cleanup.confirmPermanent": "完全に削除", "storage.cleanup.doneQuarantine": "{count} 件を隔離しました（{size}）。", "storage.cleanup.donePermanent": "{count} 件を完全削除しました（{size}）。", + "storage.cleanup.skippedReferenced": "参照されている {count} 件をスキップしました。", "storage.cleanup.previewFailed": "プレビューに失敗しました。", "storage.cleanup.cleanupFailed": "クリーンアップに失敗しました。", "storage.cleanup.err.codex_busy": "Codex が state.sqlite を使用中です — Codex を終了して再試行してください。", diff --git a/gui/src/i18n/ko.ts b/gui/src/i18n/ko.ts index 5ae812e9e8..f1d20bf65e 100644 --- a/gui/src/i18n/ko.ts +++ b/gui/src/i18n/ko.ts @@ -1816,6 +1816,7 @@ export const ko: Record = { "storage.cleanup.confirmPermanent": "영구 삭제", "storage.cleanup.doneQuarantine": "파일 {count}개를 격리했습니다({size}).", "storage.cleanup.donePermanent": "파일 {count}개를 영구 삭제했습니다({size}).", + "storage.cleanup.skippedReferenced": "참조된 파일 {count}개를 건너뛰었습니다.", "storage.cleanup.previewFailed": "미리보기에 실패했습니다.", "storage.cleanup.cleanupFailed": "정리에 실패했습니다.", "storage.cleanup.err.codex_busy": "Codex가 state.sqlite를 사용 중입니다 — Codex를 종료한 뒤 다시 시도하세요.", diff --git a/gui/src/i18n/ru.ts b/gui/src/i18n/ru.ts index 2900abb6f3..c583bccb99 100644 --- a/gui/src/i18n/ru.ts +++ b/gui/src/i18n/ru.ts @@ -980,6 +980,7 @@ export const ru: Record = { "storage.cleanup.confirmPermanent": "Удалить навсегда", "storage.cleanup.doneQuarantine": "В карантин: {count} файл(ов) ({size}).", "storage.cleanup.donePermanent": "Удалено навсегда: {count} файл(ов) ({size}).", + "storage.cleanup.skippedReferenced": "Пропущено файлов, на которые ссылается история: {count}.", "storage.cleanup.previewFailed": "Не удалось выполнить предпросмотр.", "storage.cleanup.cleanupFailed": "Не удалось выполнить очистку.", "storage.cleanup.err.codex_busy": "Codex использует state.sqlite — закройте Codex и повторите попытку.", diff --git a/gui/src/i18n/tr.ts b/gui/src/i18n/tr.ts index f551a97d43..ca39260677 100644 --- a/gui/src/i18n/tr.ts +++ b/gui/src/i18n/tr.ts @@ -999,6 +999,7 @@ export const tr: Record = { "storage.cleanup.confirmPermanent": "Kalıcı Olarak Sil", "storage.cleanup.doneQuarantine": "{count} dosya karantinaya alındı ({size}).", "storage.cleanup.donePermanent": "{count} dosya kalıcı olarak silindi ({size}).", + "storage.cleanup.skippedReferenced": "Başvurulan {count} dosya atlandı.", "storage.cleanup.previewFailed": "Önizleme başarısız oldu.", "storage.cleanup.cleanupFailed": "Temizleme başarısız oldu.", "storage.cleanup.err.codex_busy": "Codex state.sqlite dosyasını kullanıyor.", diff --git a/gui/src/i18n/zh-TW.ts b/gui/src/i18n/zh-TW.ts index ae0ee0517b..6462f6c4b5 100644 --- a/gui/src/i18n/zh-TW.ts +++ b/gui/src/i18n/zh-TW.ts @@ -788,6 +788,7 @@ export const zhTW: Record = { "storage.cleanup.confirmPermanent": "永久刪除", "storage.cleanup.doneQuarantine": "已隔離 {count} 個檔案（{size}）。", "storage.cleanup.donePermanent": "已永久刪除 {count} 個檔案（{size}）。", + "storage.cleanup.skippedReferenced": "已略過 {count} 個被參照的檔案。", "storage.cleanup.previewFailed": "預覽失敗。", "storage.cleanup.cleanupFailed": "清理失敗。", "storage.cleanup.err.codex_busy": "Codex 正在使用 state.sqlite — 請退出 Codex 後重試。", diff --git a/gui/src/i18n/zh.ts b/gui/src/i18n/zh.ts index ca92aaeafc..685227abc0 100644 --- a/gui/src/i18n/zh.ts +++ b/gui/src/i18n/zh.ts @@ -1797,6 +1797,7 @@ export const zh: Record = { "storage.cleanup.confirmPermanent": "永久删除", "storage.cleanup.doneQuarantine": "已隔离 {count} 个文件（{size}）。", "storage.cleanup.donePermanent": "已永久删除 {count} 个文件（{size}）。", + "storage.cleanup.skippedReferenced": "已跳过 {count} 个被引用的文件。", "storage.cleanup.previewFailed": "预览失败。", "storage.cleanup.cleanupFailed": "清理失败。", "storage.cleanup.err.codex_busy": "Codex 正在使用 state.sqlite — 请退出 Codex 后重试。", diff --git a/gui/src/pages/Storage.tsx b/gui/src/pages/Storage.tsx index fceb45fe04..751e609de1 100644 --- a/gui/src/pages/Storage.tsx +++ b/gui/src/pages/Storage.tsx @@ -29,6 +29,7 @@ interface CleanupResult { trashDir?: string; error?: string; message?: string; + skippedReferencedPaths?: string[]; } interface TrashEntry { @@ -222,11 +223,12 @@ function ArchivedCleanupPanel({ throw new Error(mapCleanupError(json.error, json.message, json.trashDir)); } closeConfirm(true); - setStatus( - permanent + const complete = permanent ? t("storage.cleanup.donePermanent", { count: String(json.count), size: formatBytes(json.bytes, locale) }) - : t("storage.cleanup.doneQuarantine", { count: String(json.count), size: formatBytes(json.bytes, locale) }), - ); + : t("storage.cleanup.doneQuarantine", { count: String(json.count), size: formatBytes(json.bytes, locale) }); + setStatus(json.skippedReferencedPaths?.length + ? `${complete} ${t("storage.cleanup.skippedReferenced", { count: String(json.skippedReferencedPaths.length) })}` + : complete); onDone(); } catch (e) { // Keep the dialog open (except stale_preview) so the failure is visible. diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 0ff7d4861c..d0eee3c739 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -689,6 +689,7 @@ "gui-static.test.ts": "gui", "health-scoring.test.ts": "server", "history-migration-guardian.test.ts": "codex-integration", + "history-ocx-compaction-recovery.test.ts": "codex-integration", "hyperbolic-provider.test.ts": "providers", "identity-neutralize.test.ts": "adapters", "init-backup-cleanup.test.ts": "service", diff --git a/src/adapters/openai-responses.ts b/src/adapters/openai-responses.ts index 2900fd58b2..d60bd6ce1f 100644 --- a/src/adapters/openai-responses.ts +++ b/src/adapters/openai-responses.ts @@ -2134,6 +2134,7 @@ const MUSE_SPARK_WEB_SEARCH_STRICT_MODELS = new Set([ const MUSE_SPARK_WEB_SEARCH_STRICT_RESPONSE_URLS = new Set([ "https://opencode.ai/zen/v1/responses", "https://opencode.ai/zen/go/v1/responses", + "https://api.meta.ai/v1/responses", ]); const MUSE_SPARK_UNSUPPORTED_WEB_SEARCH_FIELDS = [ @@ -2142,12 +2143,13 @@ const MUSE_SPARK_UNSUPPORTED_WEB_SEARCH_FIELDS = [ ] as const; /** - * OpenCode Zen / Go Muse Spark Responses gateway refuses a short list of Codex - * `web_search` fields. `web_search_preview` keeps its accepted shape, and Luna - * remains untouched. Match the exact effective request URL; malformed, credentialed, - * or parameterized destinations keep their original body instead of assuming this - * gateway contract. Keep the rejected names together so a newly identified field is - * a one-line compatibility update rather than another bespoke rewrite. + * OpenCode Zen / Go and the direct Meta Muse Spark Responses gateways refuse a + * short list of Codex `web_search` fields. `web_search_preview` keeps its accepted + * shape, and Luna remains untouched. Match the exact effective request URL; + * malformed, credentialed, or parameterized destinations keep their original body + * instead of assuming this gateway contract. Keep the rejected names together so a + * newly identified field is a one-line compatibility update rather than another + * bespoke rewrite. */ function stripMuseSparkUnsupportedWebSearchFields( body: unknown, diff --git a/src/cli/dispatch.ts b/src/cli/dispatch.ts index c884bb2e5d..e85de1c05f 100644 --- a/src/cli/dispatch.ts +++ b/src/cli/dispatch.ts @@ -25,6 +25,7 @@ import { afterCatalogWriteHandleAppServers } from "../codex/app-server-processes import { normalizeUpdateChannel, runGuiUpdateWorker } from "../update/job"; import { isJsonOption, takeFlag } from "./runtime-api"; import type { ClientConnectionState } from "../client/state"; +import { OCX_NATIVE_REPLAY_RECOVERY_NOTE } from "../responses/compaction"; export interface CliDispatchDeps { args: string[]; @@ -199,6 +200,7 @@ const commandRunners: Record = { } if (r.success) { console.log("Codex integration is OFF and plain `codex` now runs natively. Switch back with: ocx restore back"); + console.log(`Note: ${OCX_NATIVE_REPLAY_RECOVERY_NOTE}`); } else { console.error("Plain `codex` was not fully restored. Inspect $CODEX_HOME/config.toml before using native Codex."); } diff --git a/src/cli/help.ts b/src/cli/help.ts index 0cd6bec4dc..43916695b6 100644 --- a/src/cli/help.ts +++ b/src/cli/help.ts @@ -33,6 +33,8 @@ Usage: ocx restore back Re-point codex at the running proxy (undo restore) ocx recover-history --legacy-openai --yes Force all user-message opencodex rows to OpenAI (legacy recovery) + ocx recover-history --ocx-compaction --yes + Back up and make one ocx1-compacted thread replayable by native Codex ocx uninstall Remove service/shim/config and restore native Codex (alias: remove) ocx service [sub] Run as a background service (default: install/update/start) ocx codex-shim Auto-start proxy when \`codex\` launches (install|status|uninstall|remove) diff --git a/src/cli/index.ts b/src/cli/index.ts index 663514a120..309eea763a 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -1442,8 +1442,31 @@ async function handleStatus() { } async function handleRecoverHistory() { + if (args[1] === "--ocx-compaction") { + const threadId = args[2]; + if (args.length !== 4 || !threadId || args[3] !== "--yes") { + console.error("Usage: ocx recover-history --ocx-compaction --yes"); + console.error("This rewrites one rollout after saving a private byte-for-byte backup. Close that Codex thread before retrying."); + process.exit(1); + } + console.error("WARNING: this converts OpenCodeX-owned ocx1 compaction state into a plain summary for native Codex replay."); + try { + const { recoverOcxCompactionHistory } = await import("../codex/ocx-compaction-history"); + const result = recoverOcxCompactionHistory({ threadId }); + if (result.replaced === 0) { + console.log(`Thread ${threadId} has no repairable ocx1 compaction history; no files changed.`); + return; + } + console.log(`Recovered ${result.replaced} ocx1 compaction item(s) in thread ${threadId}.`); + console.log(`Backup: ${result.backupPath}`); + return; + } catch (error) { + console.error(`Recovery failed: ${error instanceof Error ? error.message : String(error)}`); + process.exit(1); + } + } if (args[1] !== "--legacy-openai") { - console.error("Usage: ocx recover-history --legacy-openai --yes"); + console.error("Usage: ocx recover-history (--legacy-openai | --ocx-compaction ) --yes"); console.error("This force-relabels every user-message opencodex row to OpenAI, including legitimate dedicated-provider history. Back up first and use it only for pre-backup legacy recovery."); process.exit(1); } diff --git a/src/cli/registry.ts b/src/cli/registry.ts index 467a0f7971..73bbd68e31 100644 --- a/src/cli/registry.ts +++ b/src/cli/registry.ts @@ -36,8 +36,8 @@ export const CLI_COMMANDS: CliCommandEntry[] = [ }, { name: "recover-history", - usage: "ocx recover-history --legacy-openai --yes", - summary: "Force all user-message opencodex rows to OpenAI for legacy recovery.", + usage: "ocx recover-history (--legacy-openai | --ocx-compaction ) --yes", + summary: "Recover legacy provider metadata or one OpenCodeX-compacted thread for native replay.", }, { name: "uninstall", diff --git a/src/codex/auth-api.ts b/src/codex/auth-api.ts index 2e2a775867..1a1e66d88b 100644 --- a/src/codex/auth-api.ts +++ b/src/codex/auth-api.ts @@ -241,12 +241,15 @@ function codexAccountPersistenceConflict( } /** - * The exact label `parseUsageQuota` emits for the Codex Spark window (quota.ts). + * The exact labels `parseUsageQuota` emits for the Codex Spark windows (quota.ts). * Matching on the label rather than on "is a custom window" is load-bearing: the same array * carries Cursor's First-party models / API usage, Anthropic's Fable / Opus / Sonnet, * Antigravity's Gem / Cla, Kimi's subscription credits and a dozen dynamic provider meters. */ -const CODEX_SPARK_WINDOW_LABEL = "GPT-5.3-Codex-Spark Weekly"; +const CODEX_SPARK_WINDOW_LABELS = new Set([ + "GPT-5.3-Codex-Spark 5h", + "GPT-5.3-Codex-Spark Weekly", +]); /** * Drop the Spark window unless the operator asked for it (default hidden). @@ -264,7 +267,7 @@ export function withSparkVisibility window.label !== CODEX_SPARK_WINDOW_LABEL); + const kept = quota.customWindows.filter(window => !CODEX_SPARK_WINDOW_LABELS.has(window.label)); if (kept.length === quota.customWindows.length) return quota; // An empty list is dropped rather than serialized: an absent field and an empty array should // not be two different ways of saying "no custom windows" on the wire. diff --git a/src/codex/catalog/provider-fetch.ts b/src/codex/catalog/provider-fetch.ts index dab45af38e..1e361ad7bc 100644 --- a/src/codex/catalog/provider-fetch.ts +++ b/src/codex/catalog/provider-fetch.ts @@ -954,16 +954,21 @@ function comboMemberVendorMetadata(provider: string, modelId: string): ModelMeta */ function vendorMetadataComboFallback(target: { provider: string; model: string }): ComboCatalogMemberFallback | undefined { const metadataProvider = resolveMetadataProvider(target.provider); - const metadata = metadataProvider ? comboMemberVendorMetadata(metadataProvider, target.model) : undefined; + // Custom OpenAI-compatible routes commonly retain the canonical OpenAI model id + // while using a provider name that has no metadata alias. Reuse only its effort + // ladder below; context/modality rows remain provider-owned. + const metadata = metadataProvider + ? comboMemberVendorMetadata(metadataProvider, target.model) + : comboMemberVendorMetadata("openai", target.model); if (!metadata) return undefined; return { - ...(typeof metadata.contextWindow === "number" && metadata.contextWindow > 0 + ...(metadataProvider && typeof metadata.contextWindow === "number" && metadata.contextWindow > 0 ? { contextWindow: metadata.contextWindow } : {}), - ...(typeof metadata.maxTokens === "number" && metadata.maxTokens > 0 + ...(metadataProvider && typeof metadata.maxTokens === "number" && metadata.maxTokens > 0 ? { maxOutputTokens: metadata.maxTokens } : {}), - ...(Array.isArray(metadata.input) && metadata.input.length > 0 + ...(metadataProvider && Array.isArray(metadata.input) && metadata.input.length > 0 ? { inputModalities: [...metadata.input] } : {}), ...(metadata.reasoning === true ? { reasoningEfforts: [...ROUTED_COMBO_MEMBER_REASONING_EFFORTS] } : {}), @@ -1040,15 +1045,21 @@ export function resolveComboCatalogMember( && typeof existing.contextWindow === "number" && existing.contextWindow > 0 ) { - const capped = applyProviderContextCap(existing.contextWindow, contextCap); + // Live discovery can explicitly say text-only even when configured routing + // supplies a vision sidecar. Apply the same provider hints used for thin + // rows before deriving a combo from this complete row. + const hinted = prov && isModelVisionSidecarConsumer(prov, existing.id) + ? applyProviderConfigHints(target.provider, prov, existing, contextCap, metadataModelIdCaseFold) + : existing; + const capped = applyProviderContextCap(hinted.contextWindow, contextCap); if (capped === undefined || capped === existing.contextWindow) { - return withFallbackMetadata(existing); + return withFallbackMetadata(hinted); } - const maxInput = typeof existing.maxInputTokens === "number" && existing.maxInputTokens > 0 - ? Math.min(existing.maxInputTokens, capped) + const maxInput = typeof hinted.maxInputTokens === "number" && hinted.maxInputTokens > 0 + ? Math.min(hinted.maxInputTokens, capped) : Math.min(fallback?.maxInputTokens ?? capped, capped); return withFallbackMetadata({ - ...existing, + ...hinted, contextWindow: capped, maxInputTokens: maxInput, contextCap, diff --git a/src/codex/internal/catalog-writer.ts b/src/codex/internal/catalog-writer.ts index 370bbda95e..0d9bee79ef 100644 --- a/src/codex/internal/catalog-writer.ts +++ b/src/codex/internal/catalog-writer.ts @@ -16,6 +16,7 @@ import { forgetEphemeralSecretPath, hardenSecretPath, } from "../../lib/windows-secret-acl"; +import { resetCodexAppServerCatalogStateCache } from "../app-server-processes"; export interface PreparedCatalogFileWrite { readonly path: string; @@ -167,6 +168,7 @@ export function replaceActiveCodexCatalog( ): void { assertCatalogWritePermit(permit, owningCodexHome); atomicWriteFile(prepared.path, prepared.content, io); + resetCodexAppServerCatalogStateCache(); } /** Atomically publish the catalog-path-keyed immutable backup without clobbering. */ @@ -200,4 +202,5 @@ export function replaceCodexModelsCache( ): void { assertCatalogWritePermit(permit, owningCodexHome); atomicWriteFile(prepared.path, prepared.content, io); + resetCodexAppServerCatalogStateCache(); } diff --git a/src/codex/ocx-compaction-history.ts b/src/codex/ocx-compaction-history.ts new file mode 100644 index 0000000000..7cab765c4f --- /dev/null +++ b/src/codex/ocx-compaction-history.ts @@ -0,0 +1,226 @@ +import { createHash } from "node:crypto"; +import { + chmodSync, + closeSync, + constants, + existsSync, + fsyncSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + realpathSync, + truncateSync, + unlinkSync, + writeFileSync, +} from "node:fs"; +import { basename, isAbsolute, join, relative, resolve } from "node:path"; + +import { Database } from "bun:sqlite"; + +import { getConfigDir } from "../config"; +import { hardenSecretPath } from "../lib/windows-secret-acl"; +import { renameAtomicFile } from "../lib/windows-atomic-replace"; +import { + decodeCompactionSummary, + isCompactionItemType, + SUMMARY_PREFIX, +} from "../responses/compaction"; +import { resolveCodexHomeDir } from "./home"; +import { resolveCodexStateDbPath } from "./paths"; + +export interface OcxCompactionRewriteResult { + content: string; + replaced: number; +} + +export interface OcxCompactionHistoryRecoveryResult { + rolloutPath: string; + backupPath: string | null; + replaced: number; +} + +export interface OcxCompactionHistoryRecoveryOptions { + threadId: string; + codexHome?: string; + stateDbPath?: string; + backupRoot?: string; + now?: () => Date; +} + +const THREAD_ID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; + +function digest(content: string | Buffer): string { + return createHash("sha256").update(content).digest("hex"); +} + +function pathInside(root: string, candidate: string): boolean { + const rel = relative(root, candidate); + return rel === "" || (!rel.startsWith("..") && !isAbsolute(rel)); +} + +function resolveOwnedRolloutPath(codexHome: string, rawPath: string): string { + const candidate = resolve(isAbsolute(rawPath) ? rawPath : join(codexHome, rawPath)); + const roots = [join(codexHome, "sessions"), join(codexHome, "archived_sessions")] + .filter(existsSync) + .map(root => realpathSync.native(root)); + const entry = lstatSync(candidate); + if (!entry.isFile() || entry.isSymbolicLink()) { + throw new Error("the referenced rollout is not a regular file"); + } + const canonical = realpathSync.native(candidate); + if (!roots.some(root => pathInside(root, canonical))) { + throw new Error("the referenced rollout is outside Codex session storage"); + } + return canonical; +} + +function writePrivateFile(path: string, content: string): void { + const fd = openSync(path, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL, 0o600); + try { + if (process.platform !== "win32") chmodSync(path, 0o600); + else hardenSecretPath(path, { required: true, timeoutMemoKey: path }); + writeFileSync(fd, content, "utf8"); + fsyncSync(fd); + } finally { + closeSync(fd); + } +} + +function safeRemovePrivateFile(path: string): void { + try { truncateSync(path, 0); } catch { /* best effort before unlink */ } + try { unlinkSync(path); } catch { /* caller reports the original failure */ } +} + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +function lowerCompactionItem(item: unknown): { item: unknown; changed: boolean } { + if (!isRecord(item) || !isCompactionItemType(item.type)) { + return { item, changed: false }; + } + if (typeof item.encrypted_content !== "string") { + return { item, changed: false }; + } + const summary = decodeCompactionSummary(item.encrypted_content); + if (summary === null) return { item, changed: false }; + return { + item: { + type: "message", + role: "user", + content: [{ type: "input_text", text: `${SUMMARY_PREFIX}\n${summary}` }], + }, + changed: true, + }; +} + +function rewriteJsonlLine(line: string): { line: string; replaced: number } { + let record: unknown; + try { + record = JSON.parse(line); + } catch { + return { line, replaced: 0 }; + } + if (!isRecord(record) || record.type !== "compacted" || !isRecord(record.payload)) { + return { line, replaced: 0 }; + } + const history = record.payload.replacement_history; + if (!Array.isArray(history)) return { line, replaced: 0 }; + + let replaced = 0; + const replacementHistory = history.map(item => { + const lowered = lowerCompactionItem(item); + if (lowered.changed) replaced += 1; + return lowered.item; + }); + if (replaced === 0) return { line, replaced: 0 }; + + return { + line: JSON.stringify({ + ...record, + payload: { ...record.payload, replacement_history: replacementHistory }, + }), + replaced, + }; +} + +/** + * Convert OpenCodeX-owned `ocx1:` compaction items into ordinary replayable user messages. + * + * Only the authoritative `compacted.payload.replacement_history` snapshot is changed. Earlier + * response-item events are historical output and are deliberately preserved byte-for-byte. + * Native opaque compactions are also untouched because OpenCodeX cannot decode them safely. + */ +export function rewriteOcxCompactionsForNativeReplay(content: string): OcxCompactionRewriteResult { + const parts = content.split(/(\r?\n)/); + let replaced = 0; + for (let index = 0; index < parts.length; index += 2) { + const line = parts[index]; + if (!line) continue; + const rewritten = rewriteJsonlLine(line); + if (rewritten.replaced === 0) continue; + parts[index] = rewritten.line; + replaced += rewritten.replaced; + } + return replaced === 0 + ? { content, replaced: 0 } + : { content: parts.join(""), replaced }; +} + +/** + * Repair one explicitly selected Codex rollout for direct native replay. + * + * The original bytes are copied to an owner-private backup before the rollout is atomically + * replaced. A last-moment digest check refuses a concurrent Codex append instead of losing it. + */ +export function recoverOcxCompactionHistory( + options: OcxCompactionHistoryRecoveryOptions, +): OcxCompactionHistoryRecoveryResult { + if (!THREAD_ID_RE.test(options.threadId)) throw new Error("thread id must be a UUID"); + const codexHome = realpathSync.native(options.codexHome ?? resolveCodexHomeDir()); + const stateDbPath = options.stateDbPath ?? resolveCodexStateDbPath({ codexHome }); + if (!existsSync(stateDbPath)) throw new Error("Codex state database was not found"); + + const db = new Database(stateDbPath, { readonly: true }); + let rawRolloutPath: string | undefined; + try { + db.exec("PRAGMA busy_timeout = 1000"); + rawRolloutPath = db.query<{ rollout_path: string }, [string]>( + "SELECT rollout_path FROM threads WHERE id = ? LIMIT 1", + ).get(options.threadId)?.rollout_path; + } finally { + db.close(); + } + if (!rawRolloutPath) throw new Error("thread was not found in the Codex state database"); + + const rolloutPath = resolveOwnedRolloutPath(codexHome, rawRolloutPath); + const originalBytes = readFileSync(rolloutPath); + const original = originalBytes.toString("utf8"); + if (!Buffer.from(original, "utf8").equals(originalBytes)) { + throw new Error("the rollout is not valid UTF-8 and cannot be repaired safely"); + } + const rewritten = rewriteOcxCompactionsForNativeReplay(original); + if (rewritten.replaced === 0) { + return { rolloutPath, backupPath: null, replaced: 0 }; + } + + const stamp = (options.now ?? (() => new Date()))().toISOString().replace(/[:.]/g, "-"); + const backupDir = resolve(options.backupRoot ?? join(getConfigDir(), "history-recovery-backups", options.threadId)); + mkdirSync(backupDir, { recursive: true, mode: 0o700 }); + const backupPath = join(backupDir, `${basename(rolloutPath)}.${stamp}.bak`); + writePrivateFile(backupPath, original); + + const tempPath = `${rolloutPath}.ocx-repair-${process.pid}-${crypto.randomUUID()}.tmp`; + try { + writePrivateFile(tempPath, rewritten.content); + if (digest(readFileSync(rolloutPath)) !== digest(originalBytes)) { + throw new Error("the rollout changed while it was being repaired; close Codex and retry"); + } + renameAtomicFile(tempPath, rolloutPath); + } catch (error) { + safeRemovePrivateFile(tempPath); + throw error; + } + return { rolloutPath, backupPath, replaced: rewritten.replaced }; +} diff --git a/src/codex/quota.ts b/src/codex/quota.ts index 0648e4a717..351f5fb994 100644 --- a/src/codex/quota.ts +++ b/src/codex/quota.ts @@ -336,6 +336,9 @@ function mergeAccountQuota( } if (snapshotHasCustom(quota)) next.customWindows = quota.customWindows; + // Ordinary response headers omit model-specific windows reported by WHAM. + // Absence is a partial update; an explicit list (including []) still replaces it. + else if (existing?.customWindows !== undefined) next.customWindows = existing.customWindows; if (quota.resetCredits !== undefined) next.resetCredits = quota.resetCredits; else if (existing?.resetCredits !== undefined) next.resetCredits = existing.resetCredits; @@ -795,6 +798,10 @@ export function parseUsageQuota(data: WhamUsageResponse): Omit !!window); + const sparkShort = sparkWindows.find(window => { + const percent = normalizeUsagePercent(window.used_percent); + return percent !== undefined && isExplicitShortWindow(window); + }); const sparkWeekly = sparkWindows.find(window => { const percent = normalizeUsagePercent(window.used_percent); const seconds = window.limit_window_seconds; @@ -803,16 +810,19 @@ export function parseUsageQuota(data: WhamUsageResponse): Omit= WEEKLY_WINDOW_MIN_SECONDS); }); - const sparkPercent = normalizeUsagePercent(sparkWeekly?.used_percent); - if (sparkPercent !== undefined) { - const sparkWindow: { label: string; percent: number; resetAt?: number } = { - label: "GPT-5.3-Codex-Spark Weekly", - percent: sparkPercent, - }; - const resetAt = normalizeResetAt(sparkWeekly?.reset_at); + const sparkCustomWindows: Array<{ label: string; percent: number; resetAt?: number }> = []; + for (const [label, window] of [ + ["GPT-5.3-Codex-Spark 5h", sparkShort], + ["GPT-5.3-Codex-Spark Weekly", sparkWeekly], + ] as const) { + const percent = normalizeUsagePercent(window?.used_percent); + if (percent === undefined) continue; + const sparkWindow: { label: string; percent: number; resetAt?: number } = { label, percent }; + const resetAt = normalizeResetAt(window?.reset_at); if (resetAt !== undefined) sparkWindow.resetAt = resetAt; - quota.customWindows = [sparkWindow]; + sparkCustomWindows.push(sparkWindow); } + if (sparkCustomWindows.length > 0) quota.customWindows = sparkCustomWindows; if (resetCredits !== undefined) quota.resetCredits = resetCredits; return hasKnownQuotaValue(quota) || resetCredits !== undefined ? quota : null; diff --git a/src/codex/sync.ts b/src/codex/sync.ts index 4c10068b74..6d7008a01a 100644 --- a/src/codex/sync.ts +++ b/src/codex/sync.ts @@ -8,6 +8,7 @@ import { summarizeComboCatalogOmissions, type ComboCatalogOmission } from "./cat import { shouldSyncCodexOnStart } from "./desired-state"; import { admitCodexWrite, type CodexAdmission } from "./admission"; import type { CodexCatalogSyncOptions } from "./catalog/sync"; +import { resetCodexAppServerCatalogStateCache } from "./app-server-processes"; export interface CodexSyncResult { /** @@ -116,6 +117,10 @@ export async function syncModelsToCodex( message: admission.message, }; } + // Config injection is a relevant Codex write even when the catalog bytes are unchanged. + // Drop cached process evidence before async discovery so a process that appeared since the + // last read cannot make native-default guidance report active after this sync. + resetCodexAppServerCatalogStateCache(); const p = port ?? config.port ?? 10100; const externalProvider = (deps.currentExternalCodexModelProvider ?? currentExternalCodexModelProvider)(); diff --git a/src/responses/compaction.ts b/src/responses/compaction.ts index f3fba7a033..b067e7c858 100644 --- a/src/responses/compaction.ts +++ b/src/responses/compaction.ts @@ -17,6 +17,10 @@ export const OCX_COMPACTION_PREFIX = "ocx1:"; +export const OCX_NATIVE_REPLAY_RECOVERY_NOTE = + "Threads compacted through a routed provider can contain OpenCodeX-owned ocx1 state. " + + "Before resuming one through native Codex, run `ocx recover-history --ocx-compaction --yes`."; + /** Mirrors codex-rs core/templates/compact/prompt.md (the local-compaction instruction). */ export const COMPACT_PROMPT = `You are performing a CONTEXT CHECKPOINT COMPACTION. Create a handoff summary for another LLM that will resume the task. diff --git a/src/server/management/logs-usage-routes.ts b/src/server/management/logs-usage-routes.ts index 0177e56776..a66ff4b79c 100644 --- a/src/server/management/logs-usage-routes.ts +++ b/src/server/management/logs-usage-routes.ts @@ -425,6 +425,7 @@ export async function handleLogsUsageRoutes(ctx: ManagementContext): Promise { } } const { restoreNativeCodexAsync } = await import("../../codex/inject"); + const { OCX_NATIVE_REPLAY_RECOVERY_NOTE } = await import("../../responses/compaction"); const restored = await restoreNativeCodexAsync({ revalidateDesiredState: true }); return jsonResponse({ ok: true, clientId: "codex", changed: durable && persisted.status === "committed", state: restored.success ? "absent" : "unsafe", desiredEnabled: enabled, message: restored.success - ? "Codex restored to its native path; the proxy is still serving other clients" + ? `Codex restored to its native path; the proxy is still serving other clients. ${OCX_NATIVE_REPLAY_RECOVERY_NOTE}` : `Codex intent saved, but restoring the native path did not complete: ${restored.message}`, ...(restored.success ? (durable ? {} : { reason: "not_durable" }) diff --git a/src/storage/cleanup.ts b/src/storage/cleanup.ts index e44f7d7b5c..adefc25443 100644 --- a/src/storage/cleanup.ts +++ b/src/storage/cleanup.ts @@ -96,6 +96,7 @@ export interface CleanupResult { trashDir?: string; error?: CleanupErrorCode; removedPaths: string[]; + skippedReferencedPaths?: string[]; } const STATE_DB_FILE = /^state_(\d+)\.sqlite$/; @@ -684,52 +685,59 @@ function loadMatchingThreads(db: Database, candidates: ArchivedCandidate[], code } /** - * True when any matched thread is still linked to a thread outside the delete set - * (spawn edges) or uses paginated history that other live threads may depend on via fork. - * Throws real DB errors (busy/corruption) so callers can refuse cleanup. + * Partition matched threads into deletable and referenced snapshots. Linked spawn/fork + * history and paginated histories stay in the skipped set. Throws real DB errors. */ -function findReferencedHistory( +function filterReferencedHistory( db: Database, threads: ThreadSnapshot[], -): boolean { - if (threads.length === 0) return false; - const ids = threads.map(t => t.id); - const idSet = new Set(ids); - - // Paginated history keeps durable projections tied to the rollout — refuse cleanup. - if (threads.some(t => (t.history_mode ?? "").toLowerCase() === "paginated")) { - return true; - } - - // Spawn edges that cross the delete boundary keep history reachable. - if (tableExists(db, "thread_spawn_edges")) { - for (const chunk of chunkIds(ids, SQLITE_ID_CHUNK)) { +): { safe: ThreadSnapshot[]; skipped: ThreadSnapshot[] } { + let safe = threads.filter(t => (t.history_mode ?? "").toLowerCase() !== "paginated"); + const skipped = new Map(threads + .filter(t => (t.history_mode ?? "").toLowerCase() === "paginated") + .map(t => [t.id, t])); + + while (safe.length > 0) { + const idSet = new Set(safe.map(t => t.id)); + const unsafeIds = new Set(); + + // Spawn edges that cross the delete boundary keep history reachable. + if (tableExists(db, "thread_spawn_edges")) { + for (const chunk of chunkIds([...idSet], SQLITE_ID_CHUNK)) { const placeholders = chunk.map(() => "?").join(","); const edges = db.query<{ parent_thread_id: string; child_thread_id: string }, string[]>( `SELECT parent_thread_id, child_thread_id FROM thread_spawn_edges WHERE parent_thread_id IN (${placeholders}) OR child_thread_id IN (${placeholders})`, ).all(...chunk, ...chunk); for (const edge of edges) { - if (!idSet.has(edge.parent_thread_id) || !idSet.has(edge.child_thread_id)) { - return true; + if (!idSet.has(edge.parent_thread_id)) unsafeIds.add(edge.child_thread_id); + if (!idSet.has(edge.child_thread_id)) unsafeIds.add(edge.parent_thread_id); + } + } + } + + // Other threads that list one of ours as forked_from / parent (when columns exist). + for (const column of ["forked_from_id", "parent_thread_id", "source_thread_id"] as const) { + if (!columnExists(db, "threads", column)) continue; + for (const chunk of chunkIds([...idSet], SQLITE_ID_CHUNK * 2)) { + const placeholders = chunk.map(() => "?").join(","); + const rows = db.query<{ id: string; ref: string }, string[]>( + `SELECT id, ${column} AS ref FROM threads WHERE ${column} IN (${placeholders})`, + ).all(...chunk); + for (const row of rows) { + if (!idSet.has(row.id)) unsafeIds.add(row.ref); } } } - } - // Other threads that list one of ours as forked_from / parent (when columns exist). - for (const column of ["forked_from_id", "parent_thread_id", "source_thread_id"] as const) { - if (!columnExists(db, "threads", column)) continue; - for (const chunk of chunkIds(ids, SQLITE_ID_CHUNK * 2)) { - const placeholders = chunk.map(() => "?").join(","); - const rows = db.query<{ id: string }, string[]>( - `SELECT id FROM threads WHERE ${column} IN (${placeholders})`, - ).all(...chunk); - if (rows.some(r => !idSet.has(r.id))) return true; + if (unsafeIds.size === 0) break; + for (const thread of safe) { + if (unsafeIds.has(thread.id)) skipped.set(thread.id, thread); } + safe = safe.filter(thread => !unsafeIds.has(thread.id)); } - return false; + return { safe, skipped: [...skipped.values()] }; } function tableExists(db: Database, name: string): boolean { @@ -778,6 +786,7 @@ function deleteThreadsAndDependents(db: Database, threadIds: string[]): void { interface ReconcileOk { ok: true; threads: ThreadSnapshot[]; + skipped: ThreadSnapshot[]; } interface ReconcileErr { ok: false; @@ -1463,14 +1472,14 @@ function withWritableDb( } } -/** Load matching archived threads and refuse referenced history — no deletes yet. */ +/** Load matching archived threads and retain referenced history — no deletes yet. */ function loadThreadsForCleanup( stateDbPath: string, candidates: ArchivedCandidate[], codexHome: string, busyTimeoutMs: number, ): ReconcileOk | ReconcileErr { - if (!stateDbPath || !existsSync(stateDbPath)) return { ok: true, threads: [] }; + if (!stateDbPath || !existsSync(stateDbPath)) return { ok: true, threads: [], skipped: [] }; let db: Database | undefined; try { db = openDbWritable(stateDbPath, busyTimeoutMs); @@ -1478,10 +1487,8 @@ function loadThreadsForCleanup( if (threads.some(t => Number(t.is_pinned ?? 0) === 1)) { return { ok: false, error: "pinned_thread" }; } - if (findReferencedHistory(db, threads)) { - return { ok: false, error: "referenced_history" }; - } - return { ok: true, threads }; + const filtered = filterReferencedHistory(db, threads); + return { ok: true, threads: filtered.safe, skipped: filtered.skipped }; } catch (error) { return { ok: false, error: mapDbError(error) }; } finally { @@ -1504,7 +1511,7 @@ function reconcileDeletedThreads( stageDir: string, hooks?: ReconcileTestHooks, ): ReconcileOk | ReconcileErr { - if (!paths.state || !existsSync(paths.state)) return { ok: true, threads: [] }; + if (!paths.state || !existsSync(paths.state)) return { ok: true, threads: [], skipped: [] }; if (hooks?.beforeReconcileLock) hooks.beforeReconcileLock(); @@ -1546,7 +1553,7 @@ function reconcileDeletedThreads( stateDb.exec("ROLLBACK"); return { ok: false, error: "pinned_thread" }; } - if (findReferencedHistory(stateDb, threads)) { + if (filterReferencedHistory(stateDb, threads).safe.length !== threads.length) { stateDb.exec("ROLLBACK"); return { ok: false, error: "referenced_history" }; } @@ -1582,7 +1589,7 @@ function reconcileDeletedThreads( if (hooks?.afterSatelliteMutations) hooks.afterSatelliteMutations(); // Re-check under the same lock before committing state deletes. - if (findReferencedHistory(stateDb, threads)) { + if (filterReferencedHistory(stateDb, threads).safe.length !== threads.length) { stateDb.exec("ROLLBACK"); return failWithRestore("referenced_history"); } @@ -1590,7 +1597,7 @@ function reconcileDeletedThreads( if (hooks?.failBeforeStateCommit) throw new Error("test_fail_before_state_commit"); stateDb.exec("COMMIT"); // Keep satellite-backup.json for quarantine restore; permanent purge removes the stage. - return { ok: true, threads }; + return { ok: true, threads, skipped: [] }; } catch (error) { if (satelliteLocks) rollbackAllSatelliteLocks(satelliteLocks); throw error; @@ -1895,7 +1902,30 @@ export function executeArchivedCleanup(options: ExecuteCleanupOptions): CleanupR const normalized = normalizeArchivedRolloutPath(thread.rollout_path, codexHome); if (normalized) threadByRelPath.set(normalized, thread); } - const manifestEntries: CleanupManifestEntry[] = preview.candidates.map(candidate => { + const skippedReferencedPaths = loaded.skipped + .map(thread => normalizeArchivedRolloutPath(thread.rollout_path, codexHome)) + .filter((path): path is string => path !== null); + const matchedPaths = new Set([ + ...threadByRelPath.keys(), + ...skippedReferencedPaths, + ]); + const candidates = preview.candidates.filter(candidate => { + return !matchedPaths.has(candidate.relPath) || threadByRelPath.has(candidate.relPath); + }); + if (candidates.length === 0) { + removeStageIfEmpty(stageDir, []); + removeEmptyTrashRoot(codexHome); + return { + ok: true, + mode, + percent, + count: 0, + bytes: 0, + removedPaths: [], + ...(skippedReferencedPaths.length ? { skippedReferencedPaths } : {}), + }; + } + const manifestEntries: CleanupManifestEntry[] = candidates.map(candidate => { const thread = threadByRelPath.get(candidate.relPath); return { relPath: candidate.relPath, @@ -1936,7 +1966,7 @@ export function executeArchivedCleanup(options: ExecuteCleanupOptions): CleanupR return fail(mode, percent, "fs_failed"); } - const stageResult = stageCandidates(codexHome, preview.candidates, stageDir, { + const stageResult = stageCandidates(codexHome, candidates, stageDir, { blockDestBasenames: blockStageDest.size > 0 ? blockStageDest : undefined, }); if (!stageResult.ok) { @@ -1956,7 +1986,7 @@ export function executeArchivedCleanup(options: ExecuteCleanupOptions): CleanupR const deleted = reconcileDeletedThreads( paths, - preview.candidates, + candidates, codexHome, busyTimeoutMs, stageDir, @@ -1973,8 +2003,8 @@ export function executeArchivedCleanup(options: ExecuteCleanupOptions): CleanupR return fail(mode, percent, deleted.error, keepTrash ? { trashDir } : undefined); } - const removedPaths = preview.candidates.map(c => c.relPath); - const bytes = preview.candidates.reduce((sum, c) => sum + c.bytes, 0); + const removedPaths = candidates.map(c => c.relPath); + const bytes = candidates.reduce((sum, c) => sum + c.bytes, 0); if (mode === "quarantine") { return { @@ -1985,6 +2015,7 @@ export function executeArchivedCleanup(options: ExecuteCleanupOptions): CleanupR bytes, trashDir, removedPaths, + ...(skippedReferencedPaths.length ? { skippedReferencedPaths } : {}), }; } @@ -2038,6 +2069,7 @@ export function executeArchivedCleanup(options: ExecuteCleanupOptions): CleanupR count: removedPaths.length, bytes, removedPaths, + ...(skippedReferencedPaths.length ? { skippedReferencedPaths } : {}), }; } diff --git a/src/types/config.ts b/src/types/config.ts index 0b0a2b2b98..fc9a55a8fa 100644 --- a/src/types/config.ts +++ b/src/types/config.ts @@ -753,7 +753,7 @@ export interface OcxConfig { */ codexAccountPickerEnabled?: boolean; /** - * Show the GPT-5.3-Codex-Spark weekly window on Codex quota surfaces. Default false. + * Show the GPT-5.3-Codex-Spark 5-hour and weekly windows on Codex quota surfaces. Default false. * * Spark is a single-model window that reads 0% for most operators, and on a multi-account * pool it doubles the bar count for information almost nobody acts on. Hidden by default and diff --git a/src/web-search/progress-stream.ts b/src/web-search/progress-stream.ts index f51a55efc6..e3889eb5bc 100644 --- a/src/web-search/progress-stream.ts +++ b/src/web-search/progress-stream.ts @@ -303,6 +303,10 @@ export async function* parseStreamWithProgress( } if (event.type === "done" || event.type === "incomplete") { heldTerminal = event; + // Response-byte inactivity ends once the adapter has produced a terminal event. + // From here the separate post-terminal drain guard owns the bounded wait for + // iterator cleanup, so leaving the inactivity timer armed creates a false timeout. + clearInactivity(); continue; } await handoff.deliver(event); diff --git a/tests/cli/cli-help.test.ts b/tests/cli/cli-help.test.ts index d101b75bc8..1020439e49 100644 --- a/tests/cli/cli-help.test.ts +++ b/tests/cli/cli-help.test.ts @@ -1,6 +1,6 @@ import { describe, expect, setDefaultTimeout, test } from "bun:test"; import { spawnSync } from "node:child_process"; -import { chmodSync, existsSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs"; +import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; @@ -288,8 +288,8 @@ describe("CLI subcommand help", () => { expectSpawnFinished(result, "ocx recover-history --help"); expect(result.status).toBe(0); - expect(result.stdout).toContain("Usage: ocx recover-history --legacy-openai --yes"); - expect(result.stdout).toContain("Force all user-message opencodex rows to OpenAI"); + expect(result.stdout).toContain("Usage: ocx recover-history (--legacy-openai | --ocx-compaction ) --yes"); + expect(result.stdout).toContain("Recover legacy provider metadata or one OpenCodeX-compacted thread"); expect(result.stdout).not.toContain("Recovered"); expect(result.stderr).toBe(""); expect(existsSync(statePath)).toBe(false); @@ -345,6 +345,48 @@ describe("CLI subcommand help", () => { } }); + test("recover-history repairs one explicitly selected ocx1-compacted thread", () => { + const codexHome = mkdtempSync(join(tmpdir(), "ocx-recover-compaction-")); + const opencodexHome = mkdtempSync(join(tmpdir(), "ocx-recover-compaction-state-")); + try { + writeFileSync(join(codexHome, "config.toml"), 'model = "gpt-5"\n', "utf8"); + const threadId = "01a018e6-242f-7801-81b8-ffc0a5c6d589"; + const rolloutDir = join(codexHome, "sessions", "2026", "09", "07"); + mkdirSync(rolloutDir, { recursive: true }); + const rollout = join(rolloutDir, `rollout-fixture-${threadId}.jsonl`); + const summary = `ocx1:${Buffer.from("portable summary", "utf8").toString("base64")}`; + writeFileSync(rollout, `${JSON.stringify({ + type: "compacted", + payload: { + replacement_history: [{ type: "compaction", id: "cmp_fixture", encrypted_content: summary }], + }, + })}\n`, "utf8"); + const statePath = join(codexHome, "state_5.sqlite"); + const db = new Database(statePath, { create: true }); + db.exec("CREATE TABLE threads (id TEXT PRIMARY KEY, rollout_path TEXT NOT NULL)"); + db.query("INSERT INTO threads (id, rollout_path) VALUES (?, ?)").run(threadId, rollout); + db.close(); + + const result = runCli( + ["recover-history", "--ocx-compaction", threadId, "--yes"], + { CODEX_HOME: codexHome, OPENCODEX_HOME: opencodexHome, CI: "1" }, + ); + + expectSpawnFinished(result, "ocx recover-history --ocx-compaction"); + expect(result.status).toBe(0); + expect(result.stdout).toContain("Recovered 1 ocx1 compaction item(s)"); + expect(readFileSync(rollout, "utf8")).toContain("portable summary"); + expect(readFileSync(rollout, "utf8")).not.toContain("ocx1:"); + const backupDir = join(opencodexHome, "history-recovery-backups", threadId); + const backups = readdirSync(backupDir); + expect(backups).toHaveLength(1); + expect(readFileSync(join(backupDir, backups[0]), "utf8")).toContain("ocx1:"); + } finally { + removeTreeWithRetry(opencodexHome); + removeTreeWithRetry(codexHome); + } + }); + test("start rejects unknown and partially numeric port arguments", () => { const cases = [ { args: ["start", "--port", "123abc"], expected: "Invalid port number" }, diff --git a/tests/cli/cli-restore-back.test.ts b/tests/cli/cli-restore-back.test.ts index 04051a4b10..a750ae8ee9 100644 --- a/tests/cli/cli-restore-back.test.ts +++ b/tests/cli/cli-restore-back.test.ts @@ -45,6 +45,7 @@ describe("ocx restore back", () => { expect(result.status).toBe(0); expect(JSON.parse(readFileSync(join(ocxHome, "config.json"), "utf8")).clientIntegrations.codex).toBe(false); expect(`${result.stdout}\n${result.stderr}`).toContain("Codex integration is OFF and plain `codex` now runs natively."); + expect(result.stdout).toContain("ocx recover-history --ocx-compaction --yes"); } finally { removeTreeWithRetry(codexHome); removeTreeWithRetry(ocxHome); diff --git a/tests/codex-integration/codex-catalog.test.ts b/tests/codex-integration/codex-catalog.test.ts index bb3c8a880f..38f362c81b 100644 --- a/tests/codex-integration/codex-catalog.test.ts +++ b/tests/codex-integration/codex-catalog.test.ts @@ -1640,6 +1640,43 @@ describe("combo catalog capability intersection", () => { )).not.toHaveProperty("reasoningEfforts"); }); + test("resolveComboCatalogMember restores canonical OpenAI effort levels through generic routes", () => { + const providers = new Map([["azu-lab2", { + adapter: "openai-chat" as const, + baseUrl: "https://azu-lab2.example/v1", + }]]); + const member = resolveComboCatalogMember( + { provider: "azu-lab2", model: "gpt-5.6-terra" }, + new Map([["azu-lab2/gpt-5.6-terra", { + provider: "azu-lab2", + id: "gpt-5.6-terra", + contextWindow: 373_000, + inputModalities: ["text", "image"], + }]]), + providers, + ); + expect(member?.reasoningEfforts).toEqual(["low", "medium", "high", "xhigh", "max"]); + }); + + test("resolveComboCatalogMember applies sidecar hints to complete discovery rows", () => { + const providers = new Map([["sidecar", { + adapter: "openai-chat" as const, + baseUrl: "https://sidecar.example/v1", + modelInputModalities: { planner: ["text"] }, + }]]); + const member = resolveComboCatalogMember( + { provider: "sidecar", model: "planner" }, + new Map([["sidecar/planner", { + provider: "sidecar", + id: "planner", + contextWindow: 200_000, + inputModalities: ["text"], + }]]), + providers, + ); + expect(member?.inputModalities).toEqual(["text", "image"]); + }); + // Sniper for the OUTPUT-vs-INPUT mapping defect carried over from PR #3332. The test // above uses toMatchObject, which only inspects the keys it names, so without this a // regression that puts the OUTPUT ceiling into the INPUT slot passes green. @@ -7085,6 +7122,21 @@ describe("Codex reasoning-effort capability clamp", () => { }); }); +test("provider-configured cap applies to discovered window and does not get overwritten by discovery", () => { + const resolved = applyProviderConfigHints("prov", { + adapter: "openai-chat", + baseUrl: "https://prov.test/v1", + modelContextWindows: { "disco-model": 100_000 }, + }, { + provider: "prov", + id: "disco-model", + contextWindow: 200_000, + }, 150_000); + + expect(resolved.contextWindow).toBe(100_000); + expect(resolved.contextCap).toBe(150_000); +}); + describe("auto_review_model configuration (#1225)", () => { test("applyAutoReviewModelOverride sets auto_review_model_override across all entries", () => { const { applyAutoReviewModelOverride } = require("../../src/codex/catalog/sync"); diff --git a/tests/codex-integration/codex-composed-acceptance.test.ts b/tests/codex-integration/codex-composed-acceptance.test.ts index b333fa3c65..44730e5453 100644 --- a/tests/codex-integration/codex-composed-acceptance.test.ts +++ b/tests/codex-integration/codex-composed-acceptance.test.ts @@ -484,9 +484,11 @@ describe("WP13 composed toggle acceptance", () => { // The CLI's own output is the assertion message: a bare "expected 0, got 1" sent two // Windows CI rounds chasing a timeout that was never the cause. expect(`exit=${back.exitCode}\nstderr: ${back.stderr}\nstdout: ${back.stdout}`).toContain("exit=0"); - expect((await fx.request(server.runtime, "/api/native-integrations/codex", { + const disabledAgain = await fx.request(server.runtime, "/api/native-integrations/codex", { method: "PUT", body: JSON.stringify({ enabled: false }), - })).body).toMatchObject({ desiredEnabled: false }); + }); + expect(disabledAgain.body).toMatchObject({ desiredEnabled: false }); + expect(String(disabledAgain.body.message)).toContain("ocx recover-history --ocx-compaction --yes"); } finally { await fx.stop(server); } diff --git a/tests/codex-integration/codex-models-cache-invalidate.test.ts b/tests/codex-integration/codex-models-cache-invalidate.test.ts index 3efb316511..b644740e40 100644 --- a/tests/codex-integration/codex-models-cache-invalidate.test.ts +++ b/tests/codex-integration/codex-models-cache-invalidate.test.ts @@ -5,9 +5,14 @@ import { join } from "node:path"; import { invalidateCodexModelsCache } from "../../src/codex/catalog"; import { invalidateCodexModelsCacheWithPermit } from "../../src/codex/catalog/sync"; import { withCatalogWriteSerialization } from "../../src/codex/catalog-write-serialization"; -import { afterCatalogWriteHandleAppServers } from "../../src/codex/app-server-processes"; +import { + collectCodexAppServerCatalogStateForRequest, + resetCodexAppServerCatalogStateCache, + afterCatalogWriteHandleAppServers, +} from "../../src/codex/app-server-processes"; import { refreshCodexModelCatalog } from "../../src/codex/refresh"; import { syncModelsToCodex } from "../../src/codex/sync"; +import { flushConfigDirHardening } from "../../src/config/paths"; import type { OcxConfig } from "../../src/types"; import { removeTreeWithRetry } from "../helpers/remove-tree"; @@ -32,7 +37,9 @@ describe("invalidateCodexModelsCache write gate (#476 / #518)", () => { process.env.OPENCODEX_HOME = opencodexHome; }); - afterEach(() => { + afterEach(async () => { + await flushConfigDirHardening(opencodexHome); + resetCodexAppServerCatalogStateCache(); if (previousCodexHome === undefined) delete process.env.CODEX_HOME; else process.env.CODEX_HOME = previousCodexHome; if (previousOpenCodexHome === undefined) delete process.env.OPENCODEX_HOME; @@ -298,4 +305,55 @@ describe("invalidateCodexModelsCache write gate (#476 / #518)", () => { expect(errors).toEqual([]); expect(logs).toEqual([]); }); + test("sync invalidates a cached not-running observation before a catalog write", async () => { + let snapshots: Array<{ pid: number; commandLine: string }> = []; + const io = { + platform: "win32" as const, + now: () => 3_000, + listSnapshotsAsync: async () => snapshots, + readStartMsBatchAsync: async (pids: readonly number[]) => new Map(pids.map(pid => [pid, 1_000])), + catalogMtimeMs: () => 2_000, + }; + resetCodexAppServerCatalogStateCache(); + expect((await collectCodexAppServerCatalogStateForRequest(io)).state).toBe("not_running"); + snapshots = [{ pid: 42, commandLine: "codex app-server" }]; + // Prove the real request cache is warm; injected synchronous IO bypasses it. + expect((await collectCodexAppServerCatalogStateForRequest(io)).state).toBe("not_running"); + + writeFileSync(join(codexHome, "opencodex-catalog.json"), JSON.stringify({ models: [{ slug: "gpt-5.5" }] })); + expect(invalidateCodexModelsCache({ allowWhenDesiredDisabled: true })).toBe(true); + + expect((await collectCodexAppServerCatalogStateForRequest(io)).state).toBe("stale"); + }); + + test("sync invalidates cached process state even when catalog refresh is a no-op", async () => { + let snapshots: Array<{ pid: number; commandLine: string }> = []; + const io = { + platform: "win32" as const, + now: () => 3_000, + listSnapshotsAsync: async () => snapshots, + readStartMsBatchAsync: async (pids: readonly number[]) => new Map(pids.map(pid => [pid, 1_000])), + catalogMtimeMs: () => 2_000, + }; + resetCodexAppServerCatalogStateCache(); + expect((await collectCodexAppServerCatalogStateForRequest(io)).state).toBe("not_running"); + snapshots = [{ pid: 42, commandLine: "codex app-server" }]; + // Prove the real request cache is warm; injected synchronous IO bypasses it. + expect((await collectCodexAppServerCatalogStateForRequest(io)).state).toBe("not_running"); + + await syncModelsToCodex(19107, emptyConfig, null, { + refreshCodexModelCatalog: async () => ({ + added: 0, + path: join(codexHome, "opencodex-catalog.json"), + catalogExists: false, + catalogWritten: false, + cacheSynced: false, + comboOmissions: [], + }), + injectCodexConfig: async () => ({ success: true, message: "injected" }), + currentExternalCodexModelProvider: () => null, + }); + + expect((await collectCodexAppServerCatalogStateForRequest(io)).state).toBe("stale"); + }); }); diff --git a/tests/codex-integration/codex-quota-parser-parity.test.ts b/tests/codex-integration/codex-quota-parser-parity.test.ts index 172168ada8..a698525648 100644 --- a/tests/codex-integration/codex-quota-parser-parity.test.ts +++ b/tests/codex-integration/codex-quota-parser-parity.test.ts @@ -1,12 +1,56 @@ import { describe, expect, it } from "bun:test"; import { clearAccountQuota, + applyAccountQuotaFromUpstreamHeaders, + getAccountQuota, parseUpstreamQuotaHeaders, parseUsageQuota, setAccountQuotaFromParsed, } from "../../src/codex/quota"; import { codexPoolQuotaEvidence } from "../../src/routing/quota"; +describe("Spark quota survives partial header updates", () => { + it("keeps the WHAM Spark window when an ordinary response updates standard quota", () => { + clearAccountQuota(); + const refreshed = parseUsageQuota({ + rate_limit: { primary_window: { used_percent: 20, limit_window_seconds: 604_800 } }, + additional_rate_limits: [{ + limit_name: "GPT-5.3-Codex-Spark", + rate_limit: { primary_window: { used_percent: 30, reset_at: 2_000_000_000, limit_window_seconds: 604_800 } }, + }], + }); + setAccountQuotaFromParsed("spark-partial", refreshed); + applyAccountQuotaFromUpstreamHeaders("spark-partial", new Headers({ + "x-codex-primary-used-percent": "21", + "x-codex-primary-window-minutes": "10080", + })); + expect(getAccountQuota("spark-partial")?.weeklyPercent).toBe(21); + expect(getAccountQuota("spark-partial")?.customWindows).toEqual(refreshed?.customWindows); + }); + + it("replaces custom windows when supplied, including an explicit empty list", () => { + clearAccountQuota(); + setAccountQuotaFromParsed("spark-replace", { + customWindows: [{ label: "GPT-5.3-Codex-Spark Weekly", percent: 30 }], + }); + const replacement = [{ label: "GPT-5.3-Codex-Spark Weekly", percent: 0, resetAt: 2_000_000_000 }]; + setAccountQuotaFromParsed("spark-replace", { customWindows: replacement }); + expect(getAccountQuota("spark-replace")?.customWindows).toEqual(replacement); + setAccountQuotaFromParsed("spark-replace", { weeklyPercent: 21, customWindows: [] }); + expect(getAccountQuota("spark-replace")?.customWindows).toEqual([]); + }); + + it("does not carry custom windows across an account cache clear", () => { + clearAccountQuota(); + setAccountQuotaFromParsed("spark-clear", { + customWindows: [{ label: "GPT-5.3-Codex-Spark Weekly", percent: 30 }], + }); + clearAccountQuota("spark-clear"); + setAccountQuotaFromParsed("spark-clear", { weeklyPercent: 21 }); + expect(getAccountQuota("spark-clear")?.customWindows).toBeUndefined(); + }); +}); + /** * The two quota parsers, pinned against each other. * @@ -109,4 +153,3 @@ describe("routing headroom accounts for the burst window", () => { expect(evidence.headroom).toBeLessThanOrEqual(0.05); }); }); - diff --git a/tests/codex-integration/codex-routing.test.ts b/tests/codex-integration/codex-routing.test.ts index 11774ef474..85ecb3a3fd 100644 --- a/tests/codex-integration/codex-routing.test.ts +++ b/tests/codex-integration/codex-routing.test.ts @@ -1746,8 +1746,9 @@ describe("codex routing", () => { }); }); - test("WHAM preserves the 5h, weekly, and Spark weekly windows", () => { + test("WHAM keeps general and Spark windows separate", () => { expect(parseUsageQuota({ + plan_type: "pro", rate_limit: { primary_window: { used_percent: 11, reset_at: 1, limit_window_seconds: 5 * 60 * 60 }, secondary_window: { used_percent: 22, reset_at: 2, limit_window_seconds: 7 * 24 * 60 * 60 }, @@ -1756,7 +1757,8 @@ describe("codex routing", () => { limit_name: "GPT-5.3-Codex-Spark", metered_feature: "codex_bengalfox", rate_limit: { - primary_window: { used_percent: 33, reset_at: 3, limit_window_seconds: 7 * 24 * 60 * 60 }, + primary_window: { used_percent: 33, reset_at: 3, limit_window_seconds: 5 * 60 * 60 }, + secondary_window: { used_percent: 44, reset_at: 4, limit_window_seconds: 7 * 24 * 60 * 60 }, }, }], })).toEqual({ @@ -1765,7 +1767,10 @@ describe("codex routing", () => { shortWindowSeconds: 5 * 60 * 60, weeklyPercent: 22, weeklyResetAt: 2, - customWindows: [{ label: "GPT-5.3-Codex-Spark Weekly", percent: 33, resetAt: 3 }], + customWindows: [ + { label: "GPT-5.3-Codex-Spark 5h", percent: 33, resetAt: 3 }, + { label: "GPT-5.3-Codex-Spark Weekly", percent: 44, resetAt: 4 }, + ], }); }); diff --git a/tests/codex-integration/codex-spark-visibility.test.ts b/tests/codex-integration/codex-spark-visibility.test.ts index c5dfe1c8a7..f97073b142 100644 --- a/tests/codex-integration/codex-spark-visibility.test.ts +++ b/tests/codex-integration/codex-spark-visibility.test.ts @@ -9,6 +9,7 @@ import type { OcxConfig } from "../../src/types"; import { removeTreeWithRetry } from "../helpers/remove-tree"; const SPARK = "GPT-5.3-Codex-Spark Weekly"; +const SPARK_SHORT = "GPT-5.3-Codex-Spark 5h"; const originalHome = process.env.OPENCODEX_HOME; let home = ""; @@ -33,7 +34,7 @@ afterEach(() => { }); /** - * Codex Spark is a single-model weekly window. It reads 0% for most operators and, on a + * Codex Spark is a single-model quota with 5-hour and weekly windows. It reads 0% for most operators and, on a * multi-account pool, doubles the bar count on every card for information almost nobody acts * on — so it is hidden unless the operator asks for it. * @@ -46,7 +47,10 @@ describe("Codex Spark quota visibility", () => { saveConfig(baseConfig()); const stored = { weeklyPercent: 11, - customWindows: [{ label: SPARK, percent: 33, resetAt: 3 }], + customWindows: [ + { label: SPARK_SHORT, percent: 33, resetAt: 2 }, + { label: SPARK, percent: 34, resetAt: 3 }, + ], updatedAt: Date.now(), }; const projected = withSparkVisibility(stored); @@ -54,7 +58,7 @@ describe("Codex Spark quota visibility", () => { // saying the same thing on the wire. expect(projected.customWindows).toBeUndefined(); // The source object is untouched — routing and capacity still see the window. - expect(stored.customWindows).toHaveLength(1); + expect(stored.customWindows).toHaveLength(2); expect(projected.weeklyPercent).toBe(11); }); @@ -62,17 +66,26 @@ describe("Codex Spark quota visibility", () => { saveConfig(baseConfig(true)); loadConfig(); const projected = withSparkVisibility({ - customWindows: [{ label: SPARK, percent: 33, resetAt: 3 }], + customWindows: [ + { label: SPARK_SHORT, percent: 33, resetAt: 2 }, + { label: SPARK, percent: 34, resetAt: 3 }, + ], updatedAt: Date.now(), }); - expect(projected.customWindows).toEqual([{ label: SPARK, percent: 33, resetAt: 3 }]); + expect(projected.customWindows).toEqual([ + { label: SPARK_SHORT, percent: 33, resetAt: 2 }, + { label: SPARK, percent: 34, resetAt: 3 }, + ]); }); test("an explicit false hides it", () => { saveConfig(baseConfig(false)); loadConfig(); const projected = withSparkVisibility({ - customWindows: [{ label: SPARK, percent: 33 }], + customWindows: [ + { label: SPARK_SHORT, percent: 33 }, + { label: SPARK, percent: 34 }, + ], updatedAt: Date.now(), }); expect(projected.customWindows).toBeUndefined(); @@ -87,6 +100,7 @@ describe("Codex Spark quota visibility", () => { customWindows: [ { label: "First-party models", percent: 40 }, { label: "API usage", percent: 12 }, + { label: SPARK_SHORT, percent: 32 }, { label: SPARK, percent: 33 }, { label: "Fable", percent: 7 }, { label: "Total subscription credits", percent: 90 }, @@ -112,4 +126,3 @@ describe("Codex Spark quota visibility", () => { expect(withSparkVisibility(null)).toBeNull(); }); }); - diff --git a/tests/codex-integration/history-ocx-compaction-recovery.test.ts b/tests/codex-integration/history-ocx-compaction-recovery.test.ts new file mode 100644 index 0000000000..325a83c93d --- /dev/null +++ b/tests/codex-integration/history-ocx-compaction-recovery.test.ts @@ -0,0 +1,108 @@ +import { describe, expect, test } from "bun:test"; +import { Database } from "bun:sqlite"; +import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { + recoverOcxCompactionHistory, + rewriteOcxCompactionsForNativeReplay, +} from "../../src/codex/ocx-compaction-history"; +import { encodeCompactionSummary, SUMMARY_PREFIX } from "../../src/responses/compaction"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +describe("OpenCodeX compaction history recovery", () => { + test("lowers only proxy-owned compactions in compacted replacement history", () => { + const source = [ + JSON.stringify({ type: "session_meta", payload: { id: "thread-fixture" } }), + JSON.stringify({ + type: "compacted", + payload: { + replacement_history: [ + { type: "message", role: "user", content: [{ type: "input_text", text: "keep" }] }, + { + type: "compaction", + id: "cmp_fixture", + encrypted_content: encodeCompactionSummary("fixture summary"), + }, + { type: "compaction", id: "cmp_native", encrypted_content: "native-opaque" }, + ], + }, + }), + JSON.stringify({ + type: "response_item", + payload: { type: "compaction", encrypted_content: encodeCompactionSummary("historical output") }, + }), + "", + ].join("\n"); + + const result = rewriteOcxCompactionsForNativeReplay(source); + + expect(result.replaced).toBe(1); + const lines = result.content.trimEnd().split("\n").map(line => JSON.parse(line)); + expect(lines[1].payload.replacement_history).toEqual([ + { type: "message", role: "user", content: [{ type: "input_text", text: "keep" }] }, + { + type: "message", + role: "user", + content: [{ type: "input_text", text: `${SUMMARY_PREFIX}\nfixture summary` }], + }, + { type: "compaction", id: "cmp_native", encrypted_content: "native-opaque" }, + ]); + expect(lines[2].payload.encrypted_content).toStartWith("ocx1:"); + expect(result.content.endsWith("\n")).toBe(true); + }); + + test("is byte-stable when no repairable compaction exists", () => { + const source = `${JSON.stringify({ + type: "compacted", + payload: { replacement_history: [{ type: "compaction", encrypted_content: "native-opaque" }] }, + })}\nnot-json\n`; + + expect(rewriteOcxCompactionsForNativeReplay(source)).toEqual({ content: source, replaced: 0 }); + }); + + test("backs up and atomically repairs one database-selected rollout", () => { + const root = mkdtempSync(join(tmpdir(), "ocx-compaction-recovery-")); + try { + const codexHome = join(root, "codex"); + const rolloutDir = join(codexHome, "sessions", "2026", "09", "07"); + const backupRoot = join(root, "backups"); + mkdirSync(rolloutDir, { recursive: true }); + const threadId = "01a018e6-242f-7801-81b8-ffc0a5c6d589"; + const rolloutPath = join(rolloutDir, `rollout-fixture-${threadId}.jsonl`); + const original = `${JSON.stringify({ + type: "compacted", + payload: { + replacement_history: [{ + type: "compaction", + id: "cmp_fixture", + encrypted_content: encodeCompactionSummary("recover me"), + }], + }, + })}\n`; + writeFileSync(rolloutPath, original, "utf8"); + const stateDbPath = join(codexHome, "state_5.sqlite"); + const db = new Database(stateDbPath, { create: true }); + db.exec("CREATE TABLE threads (id TEXT PRIMARY KEY, rollout_path TEXT NOT NULL)"); + db.query("INSERT INTO threads (id, rollout_path) VALUES (?, ?)").run(threadId, rolloutPath); + db.close(); + + const result = recoverOcxCompactionHistory({ + threadId, + codexHome, + stateDbPath, + backupRoot, + now: () => new Date("2026-09-07T00:00:00.000Z"), + }); + + expect(result.replaced).toBe(1); + expect(result.backupPath).not.toBeNull(); + expect(readFileSync(result.backupPath!, "utf8")).toBe(original); + expect(readFileSync(rolloutPath, "utf8")).toContain(`${SUMMARY_PREFIX}\\nrecover me`); + expect(readFileSync(rolloutPath, "utf8")).not.toContain("ocx1:"); + } finally { + removeTreeWithRetry(root); + } + }); +}); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 98907bd26a..568fd6f6ee 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -524,6 +524,7 @@ "gui-static.test.ts": "gui", "health-scoring.test.ts": "server", "history-migration-guardian.test.ts": "codex-integration", + "history-ocx-compaction-recovery.test.ts": "codex-integration", "hyperbolic-provider.test.ts": "providers", "identity-neutralize.test.ts": "adapters", "init-backup-cleanup.test.ts": "service", diff --git a/tests/providers/muse-spark-web-search-compat.test.ts b/tests/providers/muse-spark-web-search-compat.test.ts index 062a42ebcd..5254c7163d 100644 --- a/tests/providers/muse-spark-web-search-compat.test.ts +++ b/tests/providers/muse-spark-web-search-compat.test.ts @@ -35,6 +35,12 @@ const META_PROVIDER = { baseUrl: "https://api.meta.ai/v1", }; +const META_PATH_PROVIDER = { + ...ZEN_PROVIDER, + baseUrl: "https://api.meta.ai", + responsesPath: "/v1/responses", +}; + /** A Codex web_search declaration exactly as `hosted_spec.rs` emits it for TextAndImage. */ function webSearchTool(): Record { return { @@ -219,7 +225,14 @@ describe("#2617/#3378 Muse Spark web_search compatibility", () => { } }); - test("direct Meta preserves its web_search fields at both tool positions", () => { + /** + * #3456 scoped the guard to OpenCode Zen/Go URLs on the assumption that + * `https://api.meta.ai/v1` accepted Codex's extra web_search fields. Direct + * Meta still 400s `tools[].search_content_types` on ordinary `web_search` + * (live 2026-09-07 against muse-spark-1.3-contributor). Same model-id set, + * same field drop, same preview preservation. + */ + test("direct Meta strips rejected web_search fields at both tool positions", () => { const body = buildForProvider(META_PROVIDER, "muse-spark-1.3-contributor", { tools: [webSearchTool()], input: [{ type: "additional_tools", tools: [webSearchTool()] }], @@ -228,8 +241,29 @@ describe("#2617/#3378 Muse Spark web_search compatibility", () => { const item = (body.input as Array>)[0]!; const nested = (item.tools as Array>)[0]!; for (const declaration of [tool, nested]) { - expect(declaration.search_content_types).toEqual(["text", "image"]); - expect(declaration.indexed_web_access).toBe(true); + expect(declaration.type).toBe("web_search"); + expect(declaration.search_context_size).toBe("medium"); + expect(Object.hasOwn(declaration, "search_content_types")).toBe(false); + expect(Object.hasOwn(declaration, "indexed_web_access")).toBe(false); } }); + + test("direct Meta keeps the field on web_search_preview", () => { + const body = buildForProvider(META_PROVIDER, "muse-spark-1.3-contributor", { + tools: [{ ...webSearchTool(), type: "web_search_preview" }], + }); + const tool = toolsOf(body)[0]!; + expect(tool.type).toBe("web_search_preview"); + expect(tool.search_content_types).toEqual(["text", "image"]); + expect(tool.indexed_web_access).toBe(true); + }); + + test("split Meta baseUrl and responsesPath derives the same strict destination", () => { + const body = buildForProvider(META_PATH_PROVIDER, "muse-spark-1.3-contributor", { + tools: [webSearchTool()], + }); + const tool = toolsOf(body)[0]!; + expect(Object.hasOwn(tool, "search_content_types")).toBe(false); + expect(Object.hasOwn(tool, "indexed_web_access")).toBe(false); + }); }); diff --git a/tests/storage/storage-cleanup.test.ts b/tests/storage/storage-cleanup.test.ts index 31cbcd6d21..8e4564e0ef 100644 --- a/tests/storage/storage-cleanup.test.ts +++ b/tests/storage/storage-cleanup.test.ts @@ -586,23 +586,60 @@ describe("executeArchivedCleanup", () => { db.close(); }, { timeout: STORE_BUDGET_MS }); - test("rejects candidates still referenced by a live spawn edge", () => { + test("skips candidates still referenced by a live spawn edge", () => { home = buildHome({ withSpawnEdges: true }); // Edge told→tmid; deleting only oldest (told) leaves tmid outside the set. const result = runWithDigest(34, "quarantine", home); - expect(result.ok).toBe(false); - expect(result.error).toBe("referenced_history"); + expect(result.ok).toBe(true); + expect(result.count).toBe(0); + expect(result.skippedReferencedPaths).toEqual(["archived_sessions/rollout-old.jsonl"]); expect(existsSync(join(home, "archived_sessions", "rollout-old.jsonl"))).toBe(true); + // Stage dir should not remain when no candidates are selected. + expect(existsSync(join(home, ".trash"))).toBe(false); }); - test("rejects paginated history_mode threads", () => { + test("skips paginated history_mode threads", () => { home = buildHome(); const db = new Database(join(home, "state_5.sqlite")); db.exec(`UPDATE threads SET history_mode='paginated' WHERE id='told'`); db.close(); const result = runWithDigest(50, "quarantine", home); - expect(result.ok).toBe(false); - expect(result.error).toBe("referenced_history"); + expect(result.ok).toBe(true); + expect(result.count).toBe(0); + expect(result.skippedReferencedPaths).toEqual(["archived_sessions/rollout-old.jsonl"]); + // Ensure the trash root has been removed when nothing was staged. + expect(existsSync(join(home, ".trash"))).toBe(false); + }); + + test("deletes safe candidates while skipping referenced history", () => { + home = buildHome({ withSpawnEdges: true }); + const exactPaths = [ + "archived_sessions/rollout-old.jsonl", + "archived_sessions/rollout-new.jsonl", + ]; + const preview = previewExactArchivedCleanup( + listArchivedCandidates(home).filter(candidate => exactPaths.includes(candidate.relPath)), + home, + ); + const result = executeArchivedCleanup({ + percent: 0, + mode: "quarantine", + digest: preview.digest, + candidateRelPaths: [ + "archived_sessions/rollout-old.jsonl", + "archived_sessions/rollout-new.jsonl", + ], + codexHome: home, + }); + expect(result.ok).toBe(true); + expect(result.removedPaths).toEqual(["archived_sessions/rollout-new.jsonl"]); + expect(result.skippedReferencedPaths).toEqual(["archived_sessions/rollout-old.jsonl"]); + expect(existsSync(join(home, "archived_sessions", "rollout-old.jsonl"))).toBe(true); + expect(existsSync(join(home, "archived_sessions", "rollout-new.jsonl"))).toBe(false); + const db = new Database(join(home, "state_5.sqlite"), { readonly: true }); + expect(db.query("SELECT id FROM threads WHERE id='told'").get()).toBeTruthy(); + expect(db.query("SELECT id FROM threads WHERE id='tnew'").get()).toBeNull(); + db.close(); }); test("quarantine removes both plain and compressed physical files", () => { diff --git a/tests/web-search/web-search-progress-stream.test.ts b/tests/web-search/web-search-progress-stream.test.ts index ddb325b2fb..d36bb42594 100644 --- a/tests/web-search/web-search-progress-stream.test.ts +++ b/tests/web-search/web-search-progress-stream.test.ts @@ -215,10 +215,13 @@ describe("web-search streamed-body progress collector", () => { let returned = false; const parser: ParseStream = async function* () { yield { type: "done", usage: { inputTokens: 1, outputTokens: 2 } }; - await sleep(20); + await sleep(30); returned = true; }; - const iterator = parseStreamWithProgress(new Response(chunkStream([])), parser, { inactivityTimeoutMs: 100 }); + const iterator = parseStreamWithProgress(new Response(chunkStream([])), parser, { + inactivityTimeoutMs: 10, + postTerminalDrainTimeoutMs: 100, + }); const next = await iterator.next(); expect(returned).toBe(true); expect(next.value).toEqual({ type: "done", usage: { inputTokens: 1, outputTokens: 2 } });