Skip to content

fix(scripts): add recover-dev-db.sql for fork-branch migration mismatch#16

Merged
42tg merged 2 commits into
mainfrom
fix/recover-dev-db-from-fork-migrations
Apr 15, 2026
Merged

fix(scripts): add recover-dev-db.sql for fork-branch migration mismatch#16
42tg merged 2 commits into
mainfrom
fix/recover-dev-db-from-fork-migrations

Conversation

@Benniphx

Copy link
Copy Markdown
Collaborator

Problem

Local dev DBs that were grown on a fork branch can fail to start bun run dev after switching back to main:

SQLiteError: no such column: default_model_selection_json
ProviderSessionDirectoryPersistenceError: Unknown persisted provider 'claudeCode'.

The Effect-SQL migrator tracks migrations by numeric ID only. If a DB has IDs 14–18 recorded from a fork timeline that used different migrations under those IDs, main's 14–18 are silently skipped — leaving the schema half-converted.

Concrete divergence I hit (Tobias' fork → main):

ID Fork (recorded in DB) main (never runs)
14 ProjectionThreadActivityParentAndItemId ProjectionThreadProposedPlanImplementation
15 ProjectionThreadsJiraTicket ProjectionTurnsSourceProposedPlan
16 ReviewComments CanonicalizeModelSelections (the painful one)
17 ReviewRequests ProjectionThreadsArchivedAt
18 ReviewRequestsPrMeta ProjectionThreadsArchivedAtIndex

Plus the claudeCodeclaudeAgent provider rename was code-only, so legacy rows still reference claudeCode.

What this PR does

Adds scripts/recover-dev-db.sql — a single-transaction recovery script that:

  1. Renames claudeCodeclaudeAgent in provider_session_runtime (provider_name + adapter_key), projection_thread_sessions.provider_name, and orchestration_events.payload_json.
  2. Applies the missing schema deltas for main migrations 14–18:
    • 14 projection_thread_proposed_plans: implemented_at, implementation_thread_id
    • 15 projection_turns: source_proposed_plan_thread_id, source_proposed_plan_id
    • 16 CanonicalizeModelSelections — adds default_model_selection_json (projects), model_selection_json (threads), drops default_model/model, and ports orchestration_events payloads (project.created, project.meta-updated, thread.created, thread.meta-updated, thread.turn-start-requested) to the new modelSelection shape.
    • 17 projection_threads.archived_at
    • 18 idx_projection_threads_project_archived_at
  3. Does not touch effect_sql_migrations — IDs 14–21 are already recorded as applied, so no migration will be re-run.

Recommended path: keep your data (Option B)

This is the path teammates should default to — wiping the DB loses all local sessions.

# 1. Stop the dev server
# 2. Backup
cp ~/.t3/dev/state.sqlite ~/.t3/dev/state.sqlite.bak-$(date +%Y%m%d-%H%M%S)
# 3. Apply
sqlite3 ~/.t3/dev/state.sqlite < scripts/recover-dev-db.sql
# 4. Start
bun run dev

Fallback only: nuke and re-create (Option A)

Use this only if Option B somehow doesn't work for you. You will lose all local dev sessions.

TS=$(date +%Y%m%d-%H%M%S)
mv ~/.t3/dev/state.sqlite     ~/.t3/dev/state.sqlite.bak-$TS
mv ~/.t3/dev/state.sqlite-shm ~/.t3/dev/state.sqlite-shm.bak-$TS 2>/dev/null
mv ~/.t3/dev/state.sqlite-wal ~/.t3/dev/state.sqlite-wal.bak-$TS 2>/dev/null
bun run dev

Verification

After running the script:

sqlite3 ~/.t3/dev/state.sqlite "
  SELECT 'default_model_selection_json:', COUNT(*) FROM pragma_table_info('projection_projects') WHERE name='default_model_selection_json'
  UNION ALL SELECT 'model_selection_json:', COUNT(*) FROM pragma_table_info('projection_threads')  WHERE name='model_selection_json'
  UNION ALL SELECT 'archived_at:',          COUNT(*) FROM pragma_table_info('projection_threads')  WHERE name='archived_at'
  UNION ALL SELECT 'claudeCode rest:',      COUNT(*) FROM provider_session_runtime WHERE provider_name='claudeCode';
"

First three should be 1, last should be 0. Tested locally on a real fork-grown DB (~3.7 MB, ~200 events with claudeCode references) — all checks green, dev server starts cleanly, sessions intact.

Out of scope (but worth a follow-up)

The root cause — migrator keying only on ID — is what makes this class of break possible. A future change could verify recorded migration names match expected ones at startup and warn or fail fast.

Local dev DBs that were built up on a fork branch can have migration
IDs 14-18 recorded that map to different migrations than the ones on
main. The Effect-SQL migrator only tracks numeric IDs, so main's
migrations 14-18 never run on those DBs, producing errors like:

  SQLiteError: no such column: default_model_selection_json
  ProviderSessionDirectoryPersistenceError: Unknown persisted provider
    'claudeCode'.

This script applies the missing schema deltas (migrations 14-18) and
renames the legacy 'claudeCode' provider to 'claudeAgent', without
touching effect_sql_migrations.

Usage:
  cp ~/.t3/dev/state.sqlite ~/.t3/dev/state.sqlite.bak
  sqlite3 ~/.t3/dev/state.sqlite < scripts/recover-dev-db.sql
@github-actions github-actions Bot added size:L vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. labels Apr 13, 2026
@Benniphx Benniphx requested a review from 42tg April 13, 2026 08:21

@42tg 42tg left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: scripts/recover-dev-db.sql

Verified SQL against actual migrations 014–018 in apps/server/src/persistence/Migrations/. Line-by-line match is faithful. Migration 16 (CanonicalizeModelSelections) event payload transforms are complex but correctly transcribed from the Effect SQL template literals.

Approve-worthy

  • Correctness: Steps B–F match their corresponding migrations exactly.
  • Provider rename (Step A): Covers the three right tables plus the REPLACE in payload_json.
  • Transaction wrapping: Good — all-or-nothing.
  • Clear docs: Backup instructions, verification queries, both recovery paths documented.

Issues to address

  1. Missing 019_ProjectionSnapshotLookupIndexes coverage. The migration registry (Migrations.ts:67) assigns ID 19 to ProjectionSnapshotLookupIndexes, ID 20 to ReviewComments, ID 21 to ReviewRequests. The script only patches 14–18. If the fork DB recorded IDs beyond 18 (PR description says "14–21 already applied"), then 19–21 ran on the real main definitions — fine, because ReviewComments/ReviewRequests use CREATE TABLE IF NOT EXISTS and the indexes use CREATE INDEX IF NOT EXISTS. But: if someone's fork had 19+ as different migrations, ProjectionSnapshotLookupIndexes would be silently skipped. Worth documenting this assumption explicitly.

  2. No idempotency guards on ALTER TABLE ADD COLUMN. The actual migration 017 checks PRAGMA table_info before adding archived_at. The script doesn't. If someone runs it on a partially-recovered DB (maybe they manually added some columns), ALTER fails and the transaction rolls back. Suggest adding -- NOTE: not idempotent to the header or, better, guarding with a PRAGMA check like the real migration does:

    -- Only add if missing (mirrors migration 017 guard)
    SELECT CASE WHEN COUNT(*) = 0 THEN 1 ELSE 0 END
    FROM pragma_table_info('projection_threads') WHERE name = 'archived_at';

    (SQLite doesn't support IF NOT EXISTS for ALTER TABLE ADD COLUMN, so this is the only option.)

  3. Orphaned fork schema stays. The fork's migrations 14–18 created columns/tables that don't exist in main's schema (e.g., whatever ProjectionThreadActivityParentAndItemId, ProjectionThreadsJiraTicket, ReviewRequestsPrMeta added). These remain as dead weight after recovery. Not a blocker — worth a one-liner in the header noting they're harmless leftovers.

Nit (pre-existing, not from this PR)

019_ReviewComments.ts filename has prefix 019 but is registered as ID 20 in Migrations.ts:69. Confusing — the file numbering and registry IDs diverge starting here.

Verdict

Solid recovery tool. Issues #1 and #3 are documentation-only fixes. Issue #2 is a nice-to-have. Happy to approve once the assumptions are documented.

… schema notes

Document assumptions about migrations 19-21, add prominent
non-idempotency warning, include ProjectionSnapshotLookupIndexes
(Step G), and note that fork-specific orphaned columns are harmless.
@42tg 42tg merged commit 333070c into main Apr 15, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants