Skip to content

Speed up preview seeding: seed locally, then bulk-import to Turso #1143

Description

@busbyk

This was generated by AI during triage.

Context / problem

The preview deploy workflow seeds each PR's fresh Turso (libSQL) preview database by running pnpm seed:standalone with DATABASE_URI pointed at the remote Turso database. Payload's seed performs thousands of individual payload.create() operations, and against a remote libSQL database each one becomes a network round-trip. The step is therefore latency-bound, not compute-bound:

  • ~289s seeding against remote Turso (GitHub-hosted ubuntu-latest)
  • ~16s seeding the same data against a local SQLite file (measured in the CI build job)

That ~18× gap is pure network round-trip overhead. On Blacksmith runners (us-west) the higher round-trip latency to Turso stretched the seed to 12–22 minutes and then reproducibly killed the runner mid-seed (2/2 attempts; abrupt runner death, empty logs). We reverted the preview workflow to ubuntu-latest in #1139 to unblock moving CI onto Blacksmith; this issue is the durable fix so preview seeding is fast and runner-agnostic.

Prior art: the development deploy workflow already provisions its database with a single server-side bulk clone (turso db create <name> --from-db payloadcms-prod) instead of per-row writes — the same philosophy this issue applies to the seed path.

Agent Brief

Category: enhancement
Summary: Seed the preview database locally to a SQLite file, then bulk-import that file into the fresh Turso preview DB in one operation, instead of running seed:standalone directly against remote Turso.

Current behavior:
The preview deploy seeds by running the standalone seed with DATABASE_URI set to the remote Turso preview database. Every seeded row is a separate network round-trip to Turso, so the seed step takes ~5 minutes on GitHub-hosted runners — and long enough on higher-latency runners that the connection/runner is lost before it finishes.

Desired behavior:
The preview deploy seeds into a local SQLite database file (the same fast path the build/test jobs use, ~16s), then loads the resulting database into the freshly-created Turso preview database in a single bulk transfer (e.g. turso db import <file> / create-from-file, or equivalent). The end state is identical — a fully seeded Turso preview DB — but the seed step drops from ~5 minutes to seconds and no longer depends on sustained low-latency round-trips, so it works on any runner (including Blacksmith).

Key interfaces / behavior:

  • Seeding must continue to run through the existing standalone seed entrypoint (seed:standalone) so seeded content stays identical; only the target changes from remote Turso to a local SQLite file.
  • After a successful local seed, the local DB file is loaded into the PR's Turso preview database. The workflow already creates that DB and has its name + auth token available; verify the chosen Turso mechanism populates/replaces an existing empty DB, or reorder create/import so the imported DB is the one the deployment points at.
  • Migrations: today the workflow runs pnpm migrate against Turso before seeding. Decide whether migrations run against the local file before import (preferred — keeps schema + data in one imported snapshot) or still against Turso; the resulting Turso DB must match what the deployed preview expects.
  • The Vercel preview deployment must still receive the correct DATABASE_URI / DATABASE_AUTH_TOKEN for the populated Turso DB (unchanged from today).

Acceptance criteria:

  • The preview deploy no longer runs seed:standalone with DATABASE_URI pointed at remote Turso; seeding writes to a local SQLite file.
  • The seeded local database is bulk-loaded into the PR's Turso preview database in a single operation.
  • A preview deployment for a PR comes up fully seeded (tenants, users, pages, etc.) — content equivalent to the current approach.
  • The seed/import portion of the preview job completes in well under a minute (vs ~5 min today) on ubuntu-latest.
  • The preview workflow runs green on a Blacksmith runner (blacksmith-2vcpu-ubuntu-2404) — i.e. once this lands, preview no longer suffers the mid-seed runner death and can be moved back onto Blacksmith.

Out of scope:

  • The production and development deploy workflows (they don't run seed:standalone — production migrates only, development clones from prod). Don't change their DB provisioning.
  • Changing what the seed produces (collections, fixtures, data volume).
  • Actually re-enabling Blacksmith on the preview workflow — that's a trivial follow-up once seeding is fast and reliable.
  • The CI-runner swap in ci: run GitHub Actions on Blacksmith runners #1139.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified; an AFK agent can pick this up

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions