Generate a flattened schema.sql snapshot for humans and agents#94
Merged
Conversation
Reading the current shape of the index meant replaying the whole forward-only migration chain in migrations.go (v5 baseline + 8 steps). Add a generated, checked-in store/schema.sql so humans and agents can see the v13 shape at a glance without that. DumpSchema reads sqlite_master (tables, indexes, triggers) into a deterministic, table-grouped script; canonicalSchemaSQL migrates a throwaway in-memory DB through the real migrate path to SchemaVersion and prepends a generated-file header. TestSchemaSnapshot compares the result against the checked-in file and fails on drift; `go test ./store -update-schema` rewrites it. The test runs in the normal suite, so CI catches a stale snapshot with no extra wiring. The snapshot is documentation only — databases are still created and migrated by applyV5 plus the migration registry, never from this file.
Dumps the schema (tables, indexes, triggers) of whichever DB the usual --db/config resolution opens, so an operator or agent can inspect a real index — including one sitting at an older schema version — without reading migrations.go or trusting the checked-in store/schema.sql to match. Joins the existing db backup/check/restore cluster as an inspection primitive.
Add a Schema & migrations section to AGENTS.md: the Go migration registry is the source of truth, store/schema.sql is a generated snapshot for reading the current shape, and `go test ./store -update-schema` refreshes it after a schema change (the golden test fails on drift).
There was a problem hiding this comment.
Pull request overview
This PR introduces a checked-in, generated SQLite schema snapshot (store/schema.sql) and supporting code so contributors (humans and agents) can quickly inspect the current schema shape without replaying the full forward-only migration chain in store/migrations.go.
Changes:
- Add
(*Store).DumpSchemaplus an internal canonical snapshot generator used to produce a deterministic flattened DDL script fromsqlite_master. - Add a golden test (
TestSchemaSnapshot) to detect drift and a-update-schemaflag to regeneratestore/schema.sql. - Add
squirrel db schemaplus documentation inAGENTS.mddescribing the schema source-of-truth and regeneration workflow.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| store/schema.sql | Generated flattened snapshot of the schema at version 13. |
| store/schema.go | Implements schema dumping from sqlite_master and canonical snapshot generation. |
| store/schema_test.go | Golden test to detect schema snapshot drift + regeneration flag. |
| cmd/squirrel/db.go | Registers the new squirrel db schema subcommand. |
| cmd/squirrel/db_schema.go | Implements squirrel db schema to print DDL for an opened database. |
| AGENTS.md | Documents migrations as source-of-truth and how to regenerate the snapshot. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // (header + DDL) that store/schema.sql must match. It is the single source | ||
| // of truth shared by the golden test and its -update-schema rewrite path. | ||
| func canonicalSchemaSQL(ctx context.Context) (string, error) { | ||
| db, err := openSQLite(":memory:") |
Owner
Author
There was a problem hiding this comment.
Good call. Routed the in-memory generation through buildDSN (925b910) so it carries the same pragmas as a production Open (foreign_keys, _txlock, …). The dumped DDL is unaffected by pragmas, so the snapshot stayed byte-identical and the golden test still passes.
Comment on lines
+11
to
+14
| // reads the shape of whichever DB the usual --db/config resolution opens, | ||
| // so an operator or agent can see what a real index actually looks like — | ||
| // including its current schema version — without replaying migrations.go | ||
| // or trusting the checked-in store/schema.sql to match an older file. |
Owner
Author
There was a problem hiding this comment.
Correct — I over-claimed. Open runs the migration chain, so the DB is at the binary's SchemaVersion and the output is DDL-only (no version line). Reworded the comment in 7aa4a1c to describe the actual behavior.
Comment on lines
+30
to
+32
| `go test ./store -update-schema`; the `TestSchemaSnapshot` golden test fails | ||
| on drift, so CI catches a stale snapshot. `squirrel db schema` prints the DDL | ||
| of a live database (which may sit at an older version). |
Owner
Author
There was a problem hiding this comment.
Fixed in 7aa4a1c — reworded so it no longer implies a pre-migration / older-version inspector; opening runs migrations first.
canonicalSchemaSQL opened the throwaway in-memory DB with a bare ":memory:" DSN, so the snapshot was generated without the pragmas a real Open applies (foreign_keys, _txlock, …). Route it through buildDSN so generation happens under the same constraints databases actually migrate under. The dumped DDL is unchanged (pragmas don't affect sqlite_master), so the golden snapshot is byte-identical. Addresses Copilot review feedback on PR #94.
The doc comment and AGENTS.md claimed `db schema` shows a database "at an older version" and "its current schema version", but openStore runs the migration chain on open (bringing the DB to the binary's SchemaVersion) and the output is DDL-only with no version line. Reword both to describe the actual behavior: DDL of the opened database, after the normal migration-on-open. Addresses Copilot review feedback on PR #94.
The other db subcommands (backup/check/restore) have CLI tests; db schema didn't. Add one that runs it against a fixture DB and asserts the output carries the schema's core invariants — the volumes table, the blake3-immutability trigger, and the live-row-per-path unique index.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Reading the current shape of the index meant mentally replaying the whole forward-only migration chain in
store/migrations.go(the v5 baseline plus eight steps to v13). This adds a generated, checked-instore/schema.sqlso humans and agents can see the v13 shape at a glance.This mirrors the "flattened schema snapshot" pattern from another project (there:
SHOW CREATE ALL TABLESon CockroachDB), adapted to squirrel's stack. The SQLite analogue issqlite_master, generated in Go through squirrel's actual migrate path — nosqlite3CLI dependency, and the snapshot is provably what the code produces.What
store/schema.go—(*Store).DumpSchemareadssqlite_master(tables, indexes, triggers) into a deterministic, table-grouped script; implicit PK/UNIQUE autoindexes are omitted (already implied by their table).canonicalSchemaSQLmigrates a throwaway in-memory DB toSchemaVersionand prepends a generated-file header. Pure library — returns strings, no stdout.store/schema.sql— the generated v13 snapshot. Shows the rebuiltfilesPK(folder_id, name, blake3), theALTER … ADD COLUMNadditions, thefiles_blake3_immutabletrigger, and theuniq_files_live_per_pathpartial unique index.store/schema_test.go—TestSchemaSnapshotgolden test; fails on drift,go test ./store -update-schemarewrites the file. Runs in the normal suite, so CI catches a stale snapshot with no extra wiring.squirrel db schema— prints the DDL of a live database (which may sit at an older version), for inspecting a real index without trusting the checked-in file to match.AGENTS.md— new "Schema & migrations" section documenting the source-of-truth chain, the snapshot, and the regenerate command.Deliberately not changed
The migration engine keeps its current design — the snapshot is documentation only; databases are still created and migrated by
applyV5plus the registry, never from this file. squirrel already has the version-skew guard (store.gorejects a DB newer than the binary) and already exercises the full migration chain on every testOpen, so no.sqlfiles, down-migrations, or snapshot-seeded tests were introduced.