fix(opencode): capture every assistant turn and stop losing concurrent extraction records

.agents/atlas/handoffs/2026-07-25-opencode-phase4-runtime-verification.md

@@ -0,0 +1,136 @@
+# Recall OpenCode Phase 4 — Runtime Verification Handoff
+
+Status: Complete — branch `fm/recall-opencode-phase4-o1` ready for PR
+
+Phase: OpenCode integration — Phase 4 acceptance verification (real runtime)
+
+Branch: `fm/recall-opencode-phase4-o1`
+
+Supersedes as the current handoff: `.agents/atlas/handoffs/2026-07-22-opencode-phase4-testing-polish.md`
+(that handoff is accurate about what PR #248 shipped; it is not the authority for
+the runtime claims this pass re-tests).
+
+## Why this pass exists
+
+PR #248 and #249 landed the Phase 4 test surface and marked
+`docs/OPENCODE_INTEGRATION.md` Phase 4 "complete". Two of its recorded claims were
+not actually established by the code that claims them:
+
+1. `scripts/e2e-opencode.ts:260-266` reaches the adapter with
+   `await import(...opencode/RecallExtract.ts)` and then **synthesizes** the event
+   payload. It proves Recall's handler *accepts* `{type:'session.idle',
+   properties:{sessionID}}`; it does not prove OpenCode *emits* it, and it does not
+   prove OpenCode ever *loads* the plugin.
+2. `docs/OPENCODE_INTEGRATION.md:133-137` deferred the `session.idle` frequency
+   question ("add time-based debouncing only if future runtime evidence shows
+   repeated idle events"). The frequency was never measured, and the hedge points
+   at the wrong remedy.
+
+## Measured runtime facts (OpenCode 1.18.5, macOS)
+
+| Question | Answer | How |
+|---|---|---|
+| Does OpenCode load Recall's plugin from the installer's path? | **Yes** — `$XDG_CONFIG_HOME/opencode/plugins/` (and `plugin/` singular also works) | real `opencode serve`, plugin factory side effect observed |
+| `session.idle` frequency | **1.00 per assistant turn** (3 turns -> 3 events, delta 1 each) | one session, one long-lived server process, `GET /event` SSE |
+| `session.idle` payload | `properties.sessionID` | captured from the live stream |
+| Does `opencode export <id>` return the whole conversation? | **Yes** — all 6 messages after 3 turns | direct `opencode export` |
+
+## Defect this measurement exposed
+
+`opencode/RecallExtract.ts:140` gates on a **permanent** tracker
+(`tracker.has(sessionId)`). Because idle fires once per turn, the first idle wins
+and every later turn is discarded.
+
+Reproduced end to end against a real 3-turn session with Recall's real plugin
+loaded by a real OpenCode server: the drop file froze at 154 bytes containing only
+turn 1; `BETAMARKERTWO` and `GAMMAMARKERTHREE` never reached it; `.extracted.json`
+held the session id, permanently suppressing re-export.
+
+This is silent data loss for every multi-turn OpenCode session — the normal case.
+
+## Scope
+
+In scope: the four Phase 4 acceptance items exercised for real, the tracker defect
+the measurement exposed, #243 characterisation, and reconciling
+`docs/OPENCODE_INTEGRATION.md` + `README.md` with what is proven.
+
+Out of scope (unchanged from #248): installation reconciliation, the semantic
+#240/#241/#226 wave, release/version bump, and #236/#237/#238/#174.
+
+## Phase 4 acceptance — outcome
+
+| Item | Outcome |
+|---|---|
+| E2E session → export → drop → extract → search | Passes. `bun run test:e2e:opencode` green against OpenCode 1.18.5. |
+| Concurrent Claude + OpenCode against one store | **Was broken.** Found, fixed, regression-tested (see below). |
+| Installer rollback and restore | `install.sh restore` had zero coverage; now covered, including the collision backup and the non-interactive no-op. |
+| `session.idle` frequency, measured | **1.00 per assistant turn** (3 turns → 3 events, OpenCode 1.18.5). Asserted, not just recorded. |
+
+## Defects found and fixed
+
+1. **Multi-turn data loss** (`opencode/RecallExtract.ts`). Permanent tracker +
+   per-turn idle meant only turn 1 was ever dropped. Reproduced against a real
+   three-turn session: 154-byte drop, 2 of 3 markers lost.
+2. **Plugin load error on every launch.** OpenCode calls every export of a
+   top-level `plugins/*.ts` as a plugin factory; the exported test helpers made
+   it log `failed to load plugin ... "Object is not a function"`. Helpers moved
+   to `opencode/lib/session-export.ts`.
+3. **Concurrent extraction lost records** (`hooks/lib/sqlite-writers.ts`). The
+   duplicate probe made these DEFERRED transactions read-then-write, and SQLite
+   fails that upgrade with `SQLITE_BUSY` instantly, never consulting
+   `busy_timeout`. Measured 1ms hard failure vs the full timeout honoured under
+   `IMMEDIATE`.
+
+Defects 1 and 2 were invisible to the existing e2e by construction: it imports
+the adapter and supplies its own `$` and event payload.
+
+## #243 — closed
+
+All three acceptance items met:
+
+1. Fresh worktree matches a normal checkout — verified in a genuinely fresh
+   worktree: 1298 pass, 0 fail. Requires `bun install` inside it, which is #165's
+   gap, so #165 stays open and the CI job asserts the empty-`node_modules`
+   starting condition rather than hiding it.
+2. `recall_configure_opencode_mcp` and `recall_configure_pi_mcp` both exit 1 on a
+   malformed config and leave it byte-identical.
+3. CI now runs lint plus the full suite inside a git worktree.
+
+## Known gaps, deliberately not closed
+
+- `opencode/RecallPreCompact.ts` compaction injection is unit-tested only; no
+  test drives a real OpenCode compaction. Stated in the README row and in
+  `docs/OPENCODE_INTEGRATION.md`.
+- The full-suite failure that went unidentified is diagnosed. It was never one
+  flake but two, both of them tests inheriting host state rather than anything
+  hanging: `bun test --timeout 30000` is 1298 pass / 0 fail, and every affected
+  test passes in isolation.
+  - **Fixed here.** The three Pi-path tests in `tests/install/uninstall.test.ts`
+    let `remove_pi` reach the host's real `pi`, and `pi remove` costs ~10s on a
+    machine that actually has Pi installed - while on CI, where `pi` is absent,
+    it never runs at all. That is the entire 1s -> 12s spread, and the exact
+    three tests that failed the last pre-fix run. They now shadow `pi` with a
+    no-op stub on `PATH`, the way the file already stubs `bun unlink`; nothing
+    asserted on that call, so no coverage is lost. Now a stable 0.7-1.0s each.
+  - **Left to [#251](https://github.com/edheltzel/Recall/issues/251).** Both of
+    its named files still trip the 5s **default** on a cold network, and they
+    need different fixes: the `packFileList()` `beforeAll` in
+    `tests/install/npm-pack.test.ts` runs `npm pack --dry-run` with no timeout
+    argument, while the sibling `beforeAll` 18 lines above already carries an
+    explicit `180_000` for exactly this reason; `update.sh > --check prints
+    current + latest` in `tests/install/update.test.ts` is a plain `test` that
+    reaches the GitHub release API. Deliberately not taken here. A full suite is
+    1298 pass / 0 fail whenever these two happen not to trip, so a single green
+    run is not evidence #251 is closed.
+  Neither is this branch. An interleaved A/B of `uninstall.test.ts` against the
+  base commit's `uninstall.sh` + `lib/install-lib.sh` reproduced the outlier on
+  base, and 15 targeted runs of `sqlite-writers-concurrency`, `restore`, and
+  `opencode-integration` were clean. The concurrency test was hardened anyway -
+  it no longer asserts a wall-clock floor, only a happens-before against a stamp
+  the peer prints while it still holds the lock.
+- Out of scope and untouched: installation reconciliation, the semantic
+  #240/#241/#226 wave, release/version bump, #236/#237/#238/#174, #165.
+
+## Holds
+
+No captain product decision outstanding.

.github/workflows/ci.yml

@@ -43,6 +43,44 @@ jobs:
       - name: OpenCode integration e2e
         run: bun run test:e2e:opencode
 
+      # The pipeline e2e above supplies its own `$` and event payload, so it
+      # cannot prove OpenCode loads the plugin or emits session.idle. This leg
+      # drives a live server and asserts the measured once-per-turn frequency,
+      # so a runtime change fails CI instead of silently invalidating the
+      # contract recorded in docs/OPENCODE_INTEGRATION.md.
+      - name: OpenCode runtime contract e2e
+        run: bun run test:e2e:opencode:runtime
+
+  worktree:
+    name: Worktree baseline
+    runs-on: macos-latest
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v7
+
+      - name: Set up Bun
+        uses: oven-sh/setup-bun@v2
+        with:
+          bun-version-file: package.json
+
+      # AGENTS.md mandates that workers run in an isolated git worktree, so a
+      # worktree is the DEFAULT environment for automated work — not an edge
+      # case. Ten lifecycle/config tests once failed there while passing in a
+      # normal checkout (#243), which trains agents to read a red suite as
+      # normal. This leg makes worktree-hostility fail loudly instead.
+      #
+      # A fresh worktree has no node_modules (#165), so dependencies are
+      # installed inside it rather than inherited from the primary checkout.
+      - name: Run the suite inside a git worktree
+        run: |
+          set -euo pipefail
+          git worktree add ../recall-worktree HEAD
+          cd ../recall-worktree
+          test ! -d node_modules
+          bun install --frozen-lockfile
+          bun run lint
+          bun test
+
   ubuntu-smoke:
     name: Ubuntu compatibility smoke
     runs-on: ubuntu-latest

AGENTS.md

@@ -176,8 +176,9 @@ Child AGENTS.md files own domain-specific local rules. Read the applicable one b
 - [`docs/AGENTS.md`](docs/AGENTS.md) — user-facing published docs, ADRs, agent skill docs (never plans/specs)
 - [`agent-skills/AGENTS.md`](agent-skills/AGENTS.md) — `recall-*` Agent Skill definitions
 - [`plugins/AGENTS.md`](plugins/AGENTS.md) — per-host native plugin manifests, MCP registration, and generated skill payloads
+- [`opencode/AGENTS.md`](opencode/AGENTS.md) — OpenCode adapter plugins, their shared helpers, and the runtime contract they must satisfy
 
-Owned at root (no child doc): lifecycle scripts (`install.sh`, `update.sh`, `uninstall.sh`) + their shared `lib/install-lib.sh` and `lib/jsonc-mcp.ts`; platform guides (`FOR_CLAUDE.md`, `FOR_OPENCODE.md`, `FOR_PI.md`); `CONTEXT.md`; and `assets/` (banner + VHS demo tapes/gifs).
+Owned at root (no child doc): lifecycle scripts (`install.sh`, `update.sh`, `uninstall.sh`) + their shared `lib/install-lib.sh` and `lib/jsonc-mcp.ts`; `scripts/` (dev/CI helpers + the per-host isolated e2e harnesses); platform guides (`FOR_CLAUDE.md`, `FOR_OPENCODE.md`, `FOR_PI.md`); `CONTEXT.md`; and `assets/` (banner + VHS demo tapes/gifs).
 
 ## Maintaining this file
 

CHANGELOG.md

@@ -15,9 +15,46 @@ note in the 0.9.0 entry.
 ### Added
 
 - **OpenCode Phase 4 validation** — an isolated end-to-end harness provisions
-  OpenCode 1.18.4, verifies the current `session.idle` event and JSON export
+  OpenCode 1.18.5, verifies the current `session.idle` event and JSON export
   contract, exercises export retry, concurrent WAL writers, installer/update
   idempotence, JSONC-preserving uninstall, and searchable Recall retrieval.
+- **OpenCode live-server verification** (`bun run test:e2e:opencode:runtime`) —
+  drives a real `opencode serve`, proving what the pipeline e2e structurally
+  cannot: that OpenCode loads Recall's plugin from the path the installer writes,
+  that it loads with zero plugin errors, and that OpenCode itself emits
+  `session.idle`. **Measured: `session.idle` fires once per assistant turn**
+  (three turns produced three events, OpenCode 1.18.5), counted from OpenCode's
+  own event stream in one long-lived process. The frequency is asserted, so a
+  runtime change fails the suite instead of silently invalidating the design.
+- **Installer rollback coverage** — the documented `./install.sh restore` path
+  had no tests. Restoring the latest snapshot, restoring a specific timestamp,
+  the pre-restore snapshot, and non-zero exits for unknown/absent backups are now
+  covered, along with the collision backup that preserves a user's hand-edited
+  file before Recall symlinks over it. Restore's default-No confirm means it is a
+  deliberate no-op in non-interactive contexts; that is now asserted rather than
+  surprising.
+- **CI runs the suite inside a git worktree** — the environment `AGENTS.md`
+  mandates for workers, so worktree-hostility fails loudly instead of training
+  agents to accept a red baseline (#243).
+
+### Fixed
+
+- **OpenCode multi-turn sessions no longer lose every turn after the first.**
+  Because `session.idle` fires per turn, the adapter's permanent dedup tracker
+  let the first idle win and discarded the rest: a real three-turn session left a
+  154-byte drop holding only turn 1. The tracker now records a content digest per
+  session, so a grown session overwrites its earlier, shorter drop.
+- **OpenCode no longer logs a plugin load error on every launch.** OpenCode calls
+  every export of a top-level `plugins/*.ts` as a plugin factory, so the
+  extraction plugin's exported test helpers were invoked with the plugin context
+  and failed with `Object is not a function`. Helpers moved to
+  `opencode/lib/session-export.ts`, which OpenCode does not glob.
+- **Concurrent Claude and OpenCode extraction no longer loses records.** The batch
+  writers probe for duplicates before inserting, and SQLite fails that
+  read-to-write upgrade inside a DEFERRED transaction with `SQLITE_BUSY`
+  immediately — never consulting the `busy_timeout` the code relied on. Measured
+  1ms hard failure versus the full timeout honoured once the write lock is taken
+  up front; the transactions are now `IMMEDIATE`.
 
 ### Changed
 

FOR_OPENCODE.md

@@ -79,6 +79,33 @@ Get database statistics (record counts, database size).
 
 Show a full Library of Alexandria entry with its extracted wisdom.
 
+### recall-memory_memory_dump
+
+Persist a session explicitly, mid-conversation. Automatic extraction already captures
+OpenCode sessions (see [How Extraction Works](#how-extraction-works)), so reach for this
+only when the user asks for an immediate capture.
+
+Always pass the visible `messages` and `source: "opencode"`. With `messages` omitted,
+Recall falls back to native transcript discovery, which checks Claude Code first and so
+dumps the wrong host's session on a machine that has both installed.
+
+```
+recall-memory_memory_dump({
+  title: "Auth middleware refactor",
+  source: "opencode",
+  messages: [{ role: "user", content: "..." }, { role: "assistant", content: "..." }]
+})
+```
+
+### recall-memory_decision_update
+
+Mark a stored decision superseded (a newer decision replaced it) or reverted (it was
+wrong and was rolled back), by ID.
+
+```
+recall-memory_decision_update({ id: 42, action: "supersede" })
+```
+
 ## The CLI
 
 You can also use the `recall` CLI directly via Bash tool:
@@ -133,7 +160,7 @@ OpenCode first.
 2. **Record decisions** — When architectural decisions are made, use `recall-memory_memory_add` to record them
 3. **Context for agents** — Before spawning subagents via `@agent`, call `recall-memory_context_for_agent`
 4. **Onboarding check** — At session start, if the L0 identity tier is empty (no `~/.claude/MEMORY/identity.md` or the file is missing), suggest `recall onboard` once. Do not nag on subsequent turns.
-5. **Never store secrets** — `recall-memory_memory_add` persists content verbatim into `recall.db` (and automatic extraction captures session text), and stored records can resurface in future sessions' L0/L1 context. Redact API keys, tokens, passwords, and credential-bearing snippets before recording (e.g. `[REDACTED:api-key]`). When dumping a session that touched credentials, say so and confirm with the user first.
+5. **Never store secrets** — `recall-memory_memory_add` and `recall-memory_memory_dump` persist content verbatim into `recall.db` (and automatic extraction captures session text), and stored records can resurface in future sessions' L0/L1 context. Redact API keys, tokens, passwords, and credential-bearing snippets before recording (e.g. `[REDACTED:api-key]`). When dumping a session that touched credentials, say so and confirm with the user first.
 6. **Record corrections** — When the user corrects you ("no, actually…", "that's wrong, use X"), record it immediately: `recall-memory_memory_add({ type: "learning", content: "<what was wrong → what is right>", confidence: "high", importance: 7 })`. Corrections are the highest-signal and most perishable memory; do not wait for session end.
 
 ## How Extraction Works
@@ -145,6 +172,11 @@ A Recall plugin (`RecallExtract.ts`) runs inside OpenCode:
 3. Drops the file into `$RECALL_HOME/MEMORY/opencode-sessions/`
 4. A batch extraction job quality-gates these files into the shared SQLite database
 
+`session.idle` fires once per assistant turn (measured against OpenCode 1.18.5),
+so the drop is rewritten as the conversation grows and always reflects the whole
+session rather than only its first turn. An unchanged session costs no write —
+the plugin compares a digest of what it last wrote.
+
 The plugin records a session only after the export and drop write succeed, so
 failed exports remain retryable. OpenCode must be available on `PATH`, and the
 plugin runtime requires Bun. Verify both with `opencode --version` and `bun --version`.

README.md

@@ -18,7 +18,7 @@ All coding agents forget when a session ends. Recall doesn't — it extracts, in
 
 Built on the [Model Context Protocol](https://modelcontextprotocol.io). One SQLite file. No phone-home. No vendor lock-in.
 
-> Stable on [Claude Code](https://claude.com/claude-code). Beta on [Pi](https://pi.dev/) and Alpha for [OpenCode](https://opencode.ai/) (MCP works; lifecycle extensions are early). [Codex CLI](https://github.com/openai/codex) uses a native plugin for MCP and skills; lifecycle auto-capture is not yet supported. [Gemini CLI](https://github.com/google-gemini/gemini-cli) remains on the roadmap. See [Roadmap](#roadmap).
+> Stable on [Claude Code](https://claude.com/claude-code). Beta on [Pi](https://pi.dev/) and Beta for [OpenCode](https://opencode.ai/) (capture is verified against a live server; compaction injection is not). [Codex CLI](https://github.com/openai/codex) uses a native plugin for MCP and skills; lifecycle auto-capture is not yet supported. [Gemini CLI](https://github.com/google-gemini/gemini-cli) remains on the roadmap. See [Roadmap](#roadmap).
 
 ---
 
@@ -321,7 +321,7 @@ Recall is built around two integration surfaces: **MCP** (memory search and add,
 | ------------------------------------------------------------- | :-: | :--------------------------------------------------------: | ------------------------------------- |
 | [**Claude Code**](https://claude.com/claude-code)             | ✅  |            ✅ Stop · SessionStart · PreCompact             | **Stable** — reference implementation; native plugin ships skills + MCP |
 | [**Pi**](https://pi.dev/)                                     | ✅  | ⚠ Beta — native package with memory-injection + shutdown-capture extensions | Package + separate MCP adapter/config |
-| [**OpenCode**](https://opencode.ai/)                          | ✅  |             ⚠ Alpha — `recall-extract` plugin              | In progress                           |
+| [**OpenCode**](https://opencode.ai/)                          | ✅  | ⚠ Beta — `recall-extract` plugin; `session.idle` capture verified against a live OpenCode 1.18.5 server | Capture runtime-verified; compaction injection not yet |
 | [**Codex CLI**](https://github.com/openai/codex)              | ✅  |               — native plugin, explicit dump only          | MCP + skills available                |
 | [**Gemini CLI**](https://github.com/google-gemini/gemini-cli) |  —  |                             —                              | Coming soon                           |