Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 50 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
uses: actions/checkout@v7
with:
repository: OpenStatSpec/specification
ref: 6b9d1fc38f2f083c0ac5cf1c64874a6d07b95045
ref: 34141dda023d9e0217c37c232e39f436edfb0746
path: openstatspec-specification
- uses: shivammathur/setup-php@v2
with:
Expand Down Expand Up @@ -62,7 +62,7 @@ jobs:
uses: actions/checkout@v7
with:
repository: OpenStatSpec/specification
ref: 6b9d1fc38f2f083c0ac5cf1c64874a6d07b95045
ref: 34141dda023d9e0217c37c232e39f436edfb0746
path: openstatspec-specification
- uses: shivammathur/setup-php@v2
with:
Expand Down Expand Up @@ -105,7 +105,7 @@ jobs:
uses: actions/checkout@v7
with:
repository: OpenStatSpec/specification
ref: 6b9d1fc38f2f083c0ac5cf1c64874a6d07b95045
ref: 34141dda023d9e0217c37c232e39f436edfb0746
path: openstatspec-specification
- uses: shivammathur/setup-php@v2
with:
Expand Down Expand Up @@ -148,7 +148,7 @@ jobs:
uses: actions/checkout@v7
with:
repository: OpenStatSpec/specification
ref: 6b9d1fc38f2f083c0ac5cf1c64874a6d07b95045
ref: 34141dda023d9e0217c37c232e39f436edfb0746
path: openstatspec-specification
- uses: shivammathur/setup-php@v2
with:
Expand All @@ -157,3 +157,49 @@ jobs:
extensions: pdo_mysql, pdo_sqlite, bcmath, mbstring, zlib
- run: composer install --no-interaction --prefer-dist
- run: vendor/bin/phpunit --filter "MariaDbSpssRoundTripTest|ServerCatalogMigrationTest|StrictSpssDictionaryConformanceTest|OfficialSpssConformanceManifestTest"

dolt-integration:
name: Dolt 2.2.2 integration / PHP ${{ matrix.php }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: ['8.4', '8.5']
services:
dolt:
image: dolthub/dolt-sql-server:2.2.2@sha256:eb59cbb2bd5e1860f192a88f68c89697578256216ff848acf4bdc60ab5fc7ffc
env:
DOLT_DATABASE: openstatspec
DOLT_USER: openstatspec
DOLT_PASSWORD: openstatspec
DOLT_USER_HOST: "%"
DOLT_ROOT_PASSWORD: root
DOLT_ROOT_HOST: "%"
ports:
- 3306:3306
env:
OPENSTATSPEC_DOLT_DSN: mysql:host=127.0.0.1;port=3306;dbname=openstatspec;charset=utf8mb4
OPENSTATSPEC_DOLT_USER: openstatspec
OPENSTATSPEC_DOLT_PASSWORD: openstatspec
steps:
- uses: actions/checkout@v7
- name: Checkout OpenStatSpec specification fixtures
uses: actions/checkout@v7
with:
repository: OpenStatSpec/specification
ref: 34141dda023d9e0217c37c232e39f436edfb0746
path: openstatspec-specification
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
coverage: none
extensions: pdo_mysql, pdo_sqlite, bcmath, mbstring, zlib
- name: Wait for Dolt 2.2.2
run: |
for attempt in {1..120}; do
php -r 'try { $pdo = new PDO(getenv("OPENSTATSPEC_DOLT_DSN"), getenv("OPENSTATSPEC_DOLT_USER"), getenv("OPENSTATSPEC_DOLT_PASSWORD")); exit($pdo->query("SELECT DOLT_VERSION()")->fetchColumn() === "2.2.2" ? 0 : 1); } catch (Throwable) { exit(1); }' && exit 0
sleep 1
done
exit 1
- run: composer install --no-interaction --prefer-dist
- run: vendor/bin/phpunit --filter "DoltSpssRoundTripTest|ServerCatalogMigrationTest|StrictSpssDictionaryConformanceTest|OfficialSpssConformanceManifestTest"
36 changes: 33 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ It imports an unencrypted SPSS `.sav` or `.zsav` dataset into a relational datab

This is an early reference implementation. Its round-trip contract is **semantic**, not byte-identical: supported cases, order, variables, values, dictionary metadata and technical metadata are preserved; compression layout, timestamps and other writer-specific bytes are not promised.

SQLite, PostgreSQL 17/18, MySQL 8.4/9.7 and MariaDB 11.4/11.8/12.3 are implemented PDO profiles. Each follows one strict-wide contract:
SQLite, PostgreSQL 17/18, MySQL 8.4/9.7, MariaDB 11.4/11.8/12.3 and Dolt 2.2.2 are implemented PDO profiles. Each follows one strict-wide contract:

1. One source dataset becomes one dedicated SQL data table.
2. One SPSS case becomes one SQL row.
Expand All @@ -32,6 +32,7 @@ Composer resolves dependencies against PHP 8.4.1, the package minimum.
## API

```php
use OpenStatSpec\Spss\GuardedImportSpssEngine;
use OpenStatSpec\Spss\SpssAdapter;

$pdo = new PDO('pgsql:host=localhost;dbname=statistics', $user, $password);
Expand All @@ -44,6 +45,35 @@ $export = $adapter->export('survey_2026', '/data/survey-export.sav');
// SpssExportResult: operationId, datasetName, caseCount, diagnostics, allowLoss
```

Use `GuardedImportSpssEngine` when an engine must read from an ephemeral
descriptor while the adapter and database receive only a logical source path:

```php
$engine = new GuardedImportSpssEngine($innerEngine, $procFdPath, 'sav');
$adapter = new SpssAdapter($pdo, $engine);
$import = $adapter->import(
$engine->logicalPath(),
'survey_2026',
verifiedSourceSha256: $verifiedSourceSha256,
);
```

`verifiedSourceSha256` must be exactly 64 lowercase hexadecimal characters.
The adapter persists it as `dataset.source_hash`, but validates only its shape:
the caller is responsible for proving that it hashes the exact bytes read by
the engine. Keep any physical guarded path, such as `/proc/self/fd/...`,
internal to the engine; `SpssAdapter::import()` rejects exact Linux
descriptor paths under `/proc/*/fd/` and `/dev/fd/` before any database
mutation. `GuardedImportSpssEngine` also recursively rejects descriptor
paths in inner-engine identity keys or values and replaces every inner read
exception with a neutral logical-source error. Sanitized errors do not chain
the original exception, so descriptor paths cannot enter operation or fidelity
journals through identity metadata or read failures. The adapter and catalogue
need only the logical `.sav`/`.zsav` path
and the verified hash. Omitting the argument preserves the
existing behavior: a readable source file is hashed by pathname, otherwise
`dataset.source_hash` is `NULL`.

### Fidelity policy

Export is fail-closed. If an exporter reports a known fidelity diagnostic, it does **not** write a file until the caller explicitly accepts its code:
Expand Down Expand Up @@ -86,7 +116,7 @@ unrelated application code while an adapter operation is running:

- PostgreSQL: create a dedicated schema and use a dedicated connection with a
fixed `search_path` containing that schema only.
- MySQL/MariaDB: select a dedicated database in the adapter DSN.
- MySQL/MariaDB/Dolt: select a dedicated database in the adapter DSN.
- SQLite: use a dedicated database file and connection.

The machine-readable capability declaration must expose the active namespace
Expand Down Expand Up @@ -134,7 +164,7 @@ composer check

`composer check` validates Composer configuration, lints PHP, checks style, runs PHPStan and runs PHPUnit. Use `composer fix` for safe style fixes, then rerun `composer check`.

GitHub Actions runs the regular suite on PHP 8.4 and 8.5. It also runs real SAV and ZSAV integration round trips against PostgreSQL 17 and 18, MySQL 8.4 and 9.7, and MariaDB 11.4, 11.8 and 12.3. Those profile checks use their PDO drivers and php-spss V3 read/write paths, not only DDL snapshots.
GitHub Actions runs the regular suite on PHP 8.4 and 8.5. It also runs real SAV and ZSAV integration round trips against PostgreSQL 17 and 18, MySQL 8.4 and 9.7, MariaDB 11.4, 11.8 and 12.3, and Dolt 2.2.2. Those profile checks use their PDO drivers and php-spss V3 read/write paths, not only DDL snapshots.

## Contributing

Expand Down
5 changes: 3 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ The public `SpssAdapter` chooses a profile from the PDO driver:
| `sqlite` | SQLite | in-memory unit and round-trip suite |
| `pgsql` | PostgreSQL | live PostgreSQL 17 and 18 SAV/ZSAV CI round trips |
| `mysql` | MySQL/MariaDB | live MySQL 8.4/9.7 and MariaDB 11.4/11.8/12.3 SAV/ZSAV CI round trips |
| `mysql` | Dolt | live Dolt 2.2.2 SAV/ZSAV CI round trips; detected by `@@version_comment` plus `DOLT_VERSION()` |

Every profile creates the same logical strict-wide layout and metadata catalogue. Physical SQL types, identifier limits and capability preflight are profile-specific. A source that cannot be represented must be rejected before an incomplete substitute is created.

Expand Down Expand Up @@ -101,7 +102,7 @@ $adapter->migrateCatalog();

The command creates and versions the canonical catalogue through
`openstatspec_schema_migration`; it also applies the write-format migration to
SQLite, MySQL/MariaDB and PostgreSQL compatibility catalogues, then backfills
SQLite, MySQL/MariaDB/Dolt and PostgreSQL compatibility catalogues, then backfills
each exportable legacy dataset into the singular standard tables.

A completely empty dedicated namespace is initialized automatically on its
Expand Down Expand Up @@ -146,7 +147,7 @@ Verify `current_schema()` and `current_schemas(false)` on the exact connection
passed to `SpssAdapter`. Do not change `search_path` while an import, export, or
catalogue migration is running.

### MySQL and MariaDB
### MySQL, MariaDB and Dolt

Create and select a dedicated database in the PDO DSN, for example
`dbname=openstatspec`. Grant the adapter principal only the required privileges
Expand Down
Loading