Add canonical in-place transformation layer

CHANGELOG.md

@@ -2,8 +2,22 @@
 
 ## [Unreleased]
 
+## [0.4.0] - 2026-07-31
+
+### Added
+
+- Added a source-neutral canonical transformation plan, validation, provenance,
+  and in-place execution layer for recodes, variable labels, and value labels.
+- Added an SPSS syntax frontend and documented extension points for future
+  statistical-language frontends, with explicit SAS and Stata placeholders.
+
 ### Changed
 
+- Transformations now mutate the existing logical dataset and physical wide
+  table without creating copied datasets, persistent staging tables, or an
+  OpenStatSpec-managed undo/version history; Dolt identity can be recorded for
+  audit without making Dolt mandatory for other supported connections.
+
 - Distinguished conservative MySQL 8.4.x/9.7.x, MariaDB
   11.4.x/11.8.x/12.3.x, and PostgreSQL 17.x/18.x runtime claims from exact CI
   evidence at MySQL 8.4.11/9.7.2, MariaDB 11.4.12/11.8.8/12.3.2, and
@@ -17,7 +31,8 @@
 - Pinned active conformance fixtures and capabilities to released OpenStatSpec
   specification v0.1.0 at commit `d287c2cde9ade71f04e27dd012caec876901aed5`.
 
-[Unreleased]: https://github.com/OpenStatSpec/php/compare/v0.3.0...HEAD
+[Unreleased]: https://github.com/OpenStatSpec/php/compare/v0.4.0...HEAD
+[0.4.0]: https://github.com/OpenStatSpec/php/compare/v0.3.0...v0.4.0
 
 ## [0.3.0] - 2026-07-30
 

README.md

@@ -119,6 +119,9 @@ Pass only loss codes consciously accepted for that conversion. `operation_catalo
 - `src/Spss` - SAV/ZSAV gating, typed V3 engine bridge and public adapter API.
 
 See [docs/architecture.md](docs/architecture.md) for the complete relational contract.
+See [docs/transformations.md](docs/transformations.md) for the canonical
+transformation plan, frontend boundaries, in-place guarantees, supported
+syntax, and development commands.
 
 ## Upgrading an existing catalogue
 

docs/architecture.md

@@ -82,6 +82,26 @@ Encrypted files, Portable (`.por`) files and arbitrary external-engine formats a
 
 `src/Spss` owns SAV/ZSAV extension gating, external-engine normalization, typed php-spss V3 bridging and the public `SpssAdapter` API.
 
+### Transformations
+
+The src/Transformation package owns the statistics-package-neutral canonical
+plan, validation, deterministic plan identity, and PDO in-place executor. An
+apply preserves the canonical dataset UUID and registered wide-table identity.
+It does not publish a derived dataset or create a persistent data copy,
+snapshot, rollback table, or OpenStatSpec-managed version.
+
+The src/Frontend/Spss package is a separate language boundary that turns the
+documented SPSS subset into a canonical plan. The executor accepts that plan
+and does not import or invoke the SPSS frontend. The Stata and SAS directories
+are documented placeholders only; they contain no implementation and make no
+support claim.
+
+All implemented PDO profiles remain eligible transformation targets. Dolt adds
+active-branch, HEAD, and clean-working-set guards; it is not a gateway for the
+feature. Dolt owns history and rollback. See the
+[transformation manual](transformations.md) for the complete contract and
+operational guidance.
+
 ## External engine
 
 The selected engine is [TonisOrmisson/php-spss](https://github.com/TonisOrmisson/php-spss), Composer dependency `tiamo/spss` 3.x. It is external. If a compatible reader or writer is unavailable, the adapter produces an explicit `external_engine_unavailable` diagnostic rather than pretending conversion succeeded.

docs/transformations.md

@@ -0,0 +1,188 @@
+# Transformations
+
+## Purpose and boundary
+
+The transformation layer applies small, deterministic edits to an existing
+OpenStatSpec dataset. Its canonical `TransformationPlan`, validation, and SQL
+executor do not depend on SPSS, Stata, SAS, or another statistics package.
+Language-specific syntax belongs to a frontend that compiles into the same
+canonical plan.
+
+The initial frontend implements a documented subset of SPSS transformation
+syntax. The Stata and SAS directories are placeholders only. Their presence is
+an architectural reservation, not a support claim.
+
+## In-place contract
+
+Every successful apply preserves both identities:
+
+- the existing `dataset.dataset_id`; and
+- the existing `dataset.physical_table_schema` plus
+  `dataset.physical_table_name`.
+
+The executor updates that wide table and its existing normative metadata
+catalog in place. It does not create a derived dataset, persistent output or
+staging table, full-table copy, snapshot table, hidden rollback table, or a
+parallel OpenStatSpec version. A successful edit therefore does not increase
+the persistent dataset count or physical data-table count.
+
+The database engine's native transaction is used where it can make the
+operation atomic. OpenStatSpec does not add a durable undo or recovery-version
+layer around engines whose DDL commits implicitly. Dolt remains the history,
+diff, branch, and rollback layer when Dolt is the selected SQL server.
+
+## Architecture
+
+The package separates four responsibilities:
+
+1. `OpenStatSpec\Transformation\Model` defines the canonical, typed plan and
+   operations.
+2. `OpenStatSpec\Transformation\Validation` validates plans without knowing
+   their source language or SQL dialect.
+3. `OpenStatSpec\Frontend\Spss` lexes, parses, binds, and compiles the supported
+   SPSS subset into a canonical plan.
+4. `OpenStatSpec\Transformation\Execution` resolves catalog identities and
+   applies a validated plan through the active PDO profile.
+
+The SQL executor accepts a completed plan. It never invokes the SPSS parser.
+Likewise, the SPSS frontend does not issue SQL or select a database profile.
+This is the package boundary a future real frontend must use.
+
+Canonical serialization is deterministic. The plan hash identifies the exact
+validated operation sequence; source text and language provenance stay outside
+the source-neutral plan.
+
+## Supported operations
+
+The canonical layer supports:
+
+- ordered recode rules with exact values, numeric ranges, missing values, and
+  exactly one explicit final else rule;
+- assigning a scalar value, copying the source value, or assigning system
+  missing;
+- variable-label replacement; and
+- complete value-label replacement for one variable.
+
+Recode rules use first-match semantics. Validation rejects overlapping,
+duplicate, or ill-typed rules before SQL mutation. An SPSS frontend plan always
+meets the canonical explicit-else contract: when source syntax omits `ELSE`,
+the compiler adds SPSS's context-appropriate default action. Variables are
+resolved through the normative `variable` catalog and physical identifiers
+are quoted by the active PDO SQL profile; callers cannot supply raw table or
+column SQL.
+
+## SPSS frontend scope
+
+The SPSS frontend recognizes the documented transformation subset:
+
+- `RECODE ... INTO ...` with exact values, `THRU` ranges,
+  `LOWEST`, `HIGHEST`, `SYSMIS`, `ELSE`, `COPY`, and `SYSMIS`
+  outputs;
+- `VARIABLE LABELS`; and
+- `VALUE LABELS`.
+
+Keywords are case-insensitive. Dataset variable references currently must
+match the normative `variable.source_name` spelling exactly; this documented
+subset does not claim SPSS's case-insensitive symbol binding. The `MISSING`
+selector fails closed because SPSS user-missing semantics require binding the
+dataset's `missing_rule` metadata; use `SYSMIS` for system missing or list
+supported explicit values. Statements end with a period. Unsupported SPSS
+commands fail closed with a frontend diagnostic; they are not silently skipped
+or passed to an external statistics engine. This package does not claim full
+SPSS syntax compatibility.
+
+A multi-variable RECODE with INTO targets is expanded into ordered canonical
+operations only when an earlier target does not overwrite a source needed by a
+later pair in the same statement. Dependency-overlapping lists fail closed
+because preserving SPSS simultaneous-input semantics would otherwise require
+hidden row snapshots.
+
+## SQL profiles and Dolt
+
+Transformations are not restricted to Dolt. The executor uses every SQL
+connection profile implemented by this package: SQLite, PostgreSQL,
+MySQL/MariaDB, and Dolt.
+
+Dolt adds safety evidence rather than acting as a gateway. Before mutation the
+executor checks the active branch, resolves `HEAD`, and requires a clean Dolt
+working set. After mutation it verifies that branch and `HEAD` did not change
+under the operation. The executor does not switch branches or create a Dolt
+commit. The caller owns the later review and commit policy.
+
+MySQL-family DDL commits implicitly. A recode into a new physical target column
+can only be part of one native atomic apply on a profile with transactional DDL.
+On MySQL, MariaDB, and Dolt, create and catalog the intended target variable in
+the deployment workflow before applying a recode to it. Existing-column
+recodes and metadata edits remain supported. This capability boundary avoids
+pretending that a compensating copy or OpenStatSpec rollback layer is atomic.
+
+SQLite and PostgreSQL may create a new numeric target column inside their
+native transaction. A new string target must be registered on every profile
+before execution so its normative `declared_string_width` is explicit.
+
+The machine-readable capability declaration reports in-place transformations
+as supported for every implemented SQL profile and states these target-creation
+boundaries. Dolt additionally reports its clean-working-set and stable
+branch/HEAD guard.
+
+## Minimal PHP flow
+
+```php
+use OpenStatSpec\Frontend\Spss\SpssCompiler;
+use OpenStatSpec\Sql\Connection;
+use OpenStatSpec\Transformation\Execution\InPlaceTransformationExecutor;
+
+$datasetId = '018f47a2-4c10-7d34-8f11-93b1c3efc321';
+$syntax = 'RECODE score (1=10) (ELSE=COPY).';
+
+$plan = (new SpssCompiler())->compile($syntax, $datasetId);
+$result = (new InPlaceTransformationExecutor(new Connection($pdo)))->execute($plan);
+```
+
+`$pdo` must already point to the dedicated OpenStatSpec catalog namespace.
+The executor verifies the catalog ownership marker and resolves the same
+`dataset_id` and physical wide table before mutation.
+
+## Development commands
+
+Install dependencies and run the complete local gate from the PHP repository:
+
+```bash
+composer install
+composer check
+```
+
+Run only transformation tests while developing the layer:
+
+```bash
+vendor/bin/phpunit tests/Transformation tests/Frontend/Spss
+```
+
+Apply the formatter, then rerun the complete gate:
+
+```bash
+composer fix
+composer check
+```
+
+Database integration checks require the corresponding PDO driver and server.
+They must use a dedicated OpenStatSpec namespace, as described in the
+[architecture guide](architecture.md#deployment-namespace-and-connection-isolation).
+
+## Operational checklist
+
+Before applying a plan:
+
+1. verify that the connection uses the intended dedicated OpenStatSpec
+   namespace;
+2. select the existing dataset by its canonical UUID;
+3. compile source syntax explicitly with the intended frontend, or construct a
+   canonical plan directly;
+4. validate the plan before any mutation;
+5. on Dolt, start from the expected branch and a clean working set; and
+6. after success, inspect the data and metadata diff and decide separately
+   whether to create a Dolt commit.
+
+OpenStatSpec stores only compact operation evidence such as the plan identity
+and relevant Dolt state. It never stores copied row state as transformation
+audit data.

src/Core/CapabilityDeclaration.php

@@ -195,7 +195,18 @@ private function profile(string $name, PdoSqlProfile $profile, bool $active): ar
                     'unit' => 'bytes',
                 ],
             ] : null,
-            'transformation_workflow' => $name === 'dolt' ? 'unsupported' : null,
+            'transformation_workflow' => 'supported',
+            'in_place_transformations' => [
+                'status' => 'supported',
+                'existing_target' => 'supported',
+                'new_numeric_target' => $profile->ddlAtomic()
+                    ? 'supported_in_native_transaction'
+                    : 'preexisting_target_required',
+                'dolt_repository_guard' => $name === 'dolt'
+                    ? 'clean_working_set_and_stable_branch_head'
+                    : null,
+                'persistent_rollback_artifacts' => false,
+            ],
             'physical_table_mapping' => 'dataset.physical_table_schema + dataset.physical_table_name',
             'identifier_policy' => 'deterministic ASCII mapping; source name remains authoritative',
         ];

src/Frontend/Sas/README.md

@@ -0,0 +1,6 @@
+# SAS transformation frontend
+
+This namespace is reserved for a future SAS frontend. No SAS lexer, parser,
+binder, compiler, conformance claim, or runtime support is currently provided.
+
+Do not treat this placeholder as an accepted-syntax list or roadmap commitment.

src/Frontend/Spss/Ast/ElseInput.php

@@ -0,0 +1,7 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+final readonly class ElseInput implements RecodeInput {}

src/Frontend/Spss/Ast/ExecuteStatement.php

@@ -0,0 +1,15 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+final readonly class ExecuteStatement implements Statement
+{
+    public function __construct(public int $sourceLine) {}
+
+    public function line(): int
+    {
+        return $this->sourceLine;
+    }
+}

src/Frontend/Spss/Ast/MissingInput.php

@@ -0,0 +1,7 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+final readonly class MissingInput implements RecodeInput {}

src/Frontend/Spss/Ast/Program.php

@@ -0,0 +1,11 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+final readonly class Program
+{
+    /** @param list<Statement> $statements */
+    public function __construct(public array $statements) {}
+}

src/Frontend/Spss/Ast/RangeInput.php

@@ -0,0 +1,13 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+final readonly class RangeInput implements RecodeInput
+{
+    public function __construct(
+        public ?ScalarValue $lower,
+        public ?ScalarValue $upper,
+    ) {}
+}

src/Frontend/Spss/Ast/RecodeInput.php

@@ -0,0 +1,7 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+interface RecodeInput {}

src/Frontend/Spss/Ast/RecodeOutput.php

@@ -0,0 +1,13 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+final readonly class RecodeOutput
+{
+    public function __construct(
+        public RecodeOutputKind $kind,
+        public ?ScalarValue $value = null,
+    ) {}
+}

src/Frontend/Spss/Ast/RecodeOutputKind.php

@@ -0,0 +1,12 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+enum RecodeOutputKind
+{
+    case Value;
+    case Copy;
+    case SystemMissing;
+}

src/Frontend/Spss/Ast/RecodeRule.php

@@ -0,0 +1,13 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenStatSpec\Frontend\Spss\Ast;
+
+final readonly class RecodeRule
+{
+    public function __construct(
+        public RecodeInput $input,
+        public RecodeOutput $output,
+    ) {}
+}