feat(core): Model DDL semantics, structured indexes, and lossless schema export#220
Merged
Merged
Conversation
Columns previously carried only name, type, nullability, primary-key membership, comment and inline enum values, so the parser discarded DEFAULT, CHECK, generated, identity, collation, character set, auto-increment and ON UPDATE. A change to any of them (e.g. DEFAULT false -> true) was reported as "no change" by diff/review. - Add ColumnSemantics to Column and check_constraints to Table, plus CheckConstraint / GeneratedColumn / IdentitySpec supporting types. - Parse these attributes in CREATE TABLE, ALTER TABLE ADD/MODIFY/CHANGE COLUMN, ALTER COLUMN SET/DROP DEFAULT, ADD GENERATED, and table-level and column-level CHECK constraints (previously explicit no-ops). - Compare column semantics in diff and add table-level CHECK constraint diffs (added/removed/modified) with a summary counter. - Carry semantics through ColumnExport so column diffs show what changed. Adds regression tests covering DEFAULT and CHECK add/remove/change and updates parser snapshots that now capture the recovered attributes.
…e support Indexes carried only a flat list of column-name strings, so the parser and DB introspection either dropped functional/expression indexes or, worse, kept only their plain columns and asserted index coverage or uniqueness the DDL never declared. Partial-index predicates, INCLUDE columns and access methods were lost entirely. - Replace Index.columns with ordered key_parts (IndexKey::Column / IndexKey::Expression), and add predicate, included_columns and method. IndexColumn also records sort order, nulls order and prefix length. - Add Index helpers (from_columns, column_names, key_slots, plain_columns, has_expression, key_labels) so coverage/uniqueness checks stay conservative about expression parts and never assert false coverage. - CREATE INDEX now retains expression key parts and captures predicate, INCLUDE columns and method instead of dropping the whole index. - PostgreSQL, MySQL and SQLite catalog queries recover expression key parts, partial predicates, INCLUDE columns, prefix lengths and access method rather than filtering expression indexes out. - Evolve IndexExport additively (structured key_parts alongside the 1.0.0 columns field) and update diff, lint, review and rendering. Adds parser tests for expression indexes and a PostgreSQL integration test covering expression, partial and INCLUDE indexes; updates snapshots to the structured index shape.
The SchemaExport format claimed long-term stability but silently dropped table comments, column comments, the primary-key constraint name, inline enum values and every attribute added since 1.0.0, and its version field was never checked, so a "999.0.0" document imported without complaint. - Add comment, enum_values, primary_key_name and table-level check_constraints to the export types, and carry column semantics and the structured index shape through export/import so nothing is lost. - Bump the format to 1.1.0 (additive) and validate the version on import: reject a different major version and malformed strings, accept unknown minors from newer additive writers. Adds a round-trip regression test covering comments, enum values, constraint names, table CHECK constraints, column semantics and the structured index, plus version-validation tests.
…rsion
Addresses review findings on the P0 work:
- SQLite CREATE INDEX parsing skipped quoted identifiers/strings, so a
quoted index name containing '(' (e.g. `"idx(foo)"`) or a `WHERE`
substring inside a string literal could misplace the key list or
predicate. Parse it quote-aware (', ", `, [ ]) with whole-word WHERE
detection.
- Schema export version validation only parsed the leading major, so
malformed strings like "1", "1.foo" or "1.2.3.extra" passed while the
error still claimed MAJOR.MINOR.PATCH. Require three numeric components
and keep major-only compatibility gating.
Adds SQLite parser unit tests (quoted name, expression + predicate,
WHERE-in-literal, commas in expressions) and extends the malformed
version test.
…databases Live-DB introspection built the schema with empty column semantics and no CHECK constraints, so diff/review of a DB-sourced schema still missed DEFAULT, generated, identity, auto-increment, ON UPDATE and CHECK changes even though the model, parser, diff and export now carry them. - Extend RawColumn with default, generated (expr + stored), identity, auto-increment and ON UPDATE, and add RawCheckConstraint / RawSchema checks threaded through map_schema onto Column semantics and Table::check_constraints. - PostgreSQL: read pg_attrdef defaults, attgenerated generation expressions, attidentity, and pg_constraint CHECK definitions. - MySQL: read COLUMN_DEFAULT, EXTRA (auto_increment / ON UPDATE / generated), GENERATION_EXPRESSION, and information_schema.CHECK_CONSTRAINTS. - SQLite: read PRAGMA default values and parse table-level CHECK constraints from the CREATE TABLE text (quote/depth-aware). Adds PostgreSQL/MySQL/SQLite integration tests asserting the recovered semantics and CHECK constraints, plus unit tests for the SQLite CHECK parser.
…ames Follow-up to the introspection semantics work, addressing review findings: - MySQL `information_schema.COLUMN_DEFAULT` returns literal string defaults unquoted (e.g. `active` for `DEFAULT 'active'`). Restore the quoting for literal string-typed defaults (skipping expression defaults flagged by `EXTRA=DEFAULT_GENERATED`) so the value round-trips as SQL. - The SQLite CHECK parser kept a `CONSTRAINT <name>` pending across intervening clauses, so `x INT CONSTRAINT x_nn NOT NULL CHECK (x > 0)` mis-assigned `x_nn` to the unnamed CHECK. Only keep the name when a CHECK directly follows the named-constraint clause. Adds a MySQL string-default integration assertion and a SQLite parser unit test for the constraint-name scoping.
Schema reviewTip ✅ No risk findings — schema changes look safe to merge. |
This comment has been minimized.
This comment has been minimized.
Harden the DDL-semantics and lossless-export handling: - diff: report table CHECK constraint changes in text, Markdown and overlay output and include them in the change counts and summary - diff: compare index sort order, nulls ordering and prefix length, normalizing an unspecified value to its SQL default so an implicit ASC matches an explicit one while a real ASC/DESC change is still detected - introspect: capture index sort order (MySQL/MariaDB STATISTICS.COLLATION, PostgreSQL pg_index.indoption, SQLite index_xinfo) plus MySQL collation and character set - lint/review: a partial, prefix or expression index no longer counts as full uniqueness or foreign-key index coverage - review: flag uniqueness loss when a full UNIQUE is dropped, degraded to a partial/expression/prefix index, or widened to a larger column set, and only suppress add/drop warnings via a genuine whole-column replacement - parser: keep column-level CHECK constraint names and remove them via ALTER TABLE DROP CONSTRAINT; drop or rename an index's INCLUDE columns when the underlying column is dropped or renamed - export: make key_parts the sole index representation (drop the legacy flattened columns field), bump the format to 2.0.0 and reject an index that carries no key parts - introspect(mysql): detect MariaDB and adjust the STATISTICS index query (no EXPRESSION column) and COLUMN_DEFAULT quoting accordingly
Code Metrics Report
Details | | main (7281f1f) | #220 (4cb01bd) | +/- |
|---------------------|----------------|----------------|--------|
+ | Coverage | 94.5% | 94.6% | +0.0% |
| Files | 101 | 101 | 0 |
| Lines | 37056 | 39097 | +2041 |
+ | Covered | 35050 | 37003 | +1953 |
+ | Test Execution Time | 3m31s | 1m42s | -1m49s |Code coverage of files in pull request scope (93.9% → 94.1%)Reported by octocov |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
diff/reviewno longer report "no change" when only one of them is altered.key_partsas the sole index representation (the flattenedcolumnsfield is gone) and version validation that rejects an incompatible major while accepting additive minors.diff/review: table CHECK changes appear in text, Markdown and overlay output; index sort-order/nulls/prefix changes are detected; and partial, prefix or expression indexes no longer imply false uniqueness or foreign-key coverage.Changes
ColumnSemanticstoColumnandcheck_constraintstoTablewithCheckConstraint/GeneratedColumn/IdentitySpectypes, and parse them fromCREATE TABLE,ALTER TABLE ADD/MODIFY/CHANGE COLUMN,ALTER COLUMN SET/DROP DEFAULT,ADD GENERATED, and column/tableCHECK.diffand add table-level CHECK constraint diffs with a summary counter, carrying semantics throughColumnExportso column diffs show what changed.Index.columnswith orderedkey_parts(IndexKey::Column/Expression) pluspredicate,included_columnsandmethod, with helpers that keep coverage/uniqueness checks conservative about expression parts.CREATE INDEXand make the PostgreSQL/MySQL/SQLite catalogs recover expression parts, partial predicates, INCLUDE columns, prefix length and access method instead of dropping them.CREATE INDEXtext quote-aware so a quoted index name containing a paren or aWHEREsubstring inside a string literal no longer misplaces the key list or predicate.RawColumn/RawSchemaand thread defaults, generated expressions, identity, auto-increment, ON UPDATE and CHECK constraints onto the mappedColumn/Tableso DB-sourced schemas detect these changes in diff.pg_attrdef,attgenerated,attidentity,pg_constraint), MySQL (COLUMN_DEFAULT,EXTRA,GENERATION_EXPRESSION,CHECK_CONSTRAINTS), SQLite (PRAGMA defaults and CHECK parsed from the CREATE TABLE text).information_schemareturns unquoted, skipping expression defaults flagged byEXTRA=DEFAULT_GENERATED.CONSTRAINT <name>to a SQLite CHECK when the CHECK directly follows the named-constraint clause, so a name in front of another constraint type does not leak onto a later CHECK.STATISTICS.COLLATION, PostgreSQLpg_index.indoption, SQLiteindex_xinfo) and comparing against the SQL default so an implicitASCmatches an explicit one.ALTER TABLE DROP CONSTRAINT), track an index's INCLUDE columns onDROP/RENAME COLUMN, makekey_partsthe sole index representation at export format 2.0.0 (rejecting an index with no key parts), and split MySQL/MariaDB introspection so the MariaDBSTATISTICSquery andCOLUMN_DEFAULTquoting are correct.