Add production Hybrid OPFS PWA integration

.github/workflows/ci.yml

@@ -79,6 +79,72 @@ jobs:
           test -f pkg/absurder_sql_bg.wasm
           echo "✓ WASM build successful"
 
+  # Exercise the worker-only Hybrid backend, recovery paths, and formal
+  # IndexedDB-vs-Hybrid workload sweep in a real Chromium process.
+  test-browser-storage:
+    runs-on: ubuntu-latest
+    needs: test-wasm
+    env:
+      BENCHMARK_SWEEP: '1'
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+        with:
+          targets: wasm32-unknown-unknown
+
+      - name: Install wasm-pack
+        run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
+
+      - name: Build WASM
+        run: wasm-pack build --target web --out-dir pkg
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+
+      - name: Install browser test dependencies
+        run: |
+          npm install
+          npx playwright install --with-deps chromium
+
+      - name: Run Hybrid storage and benchmark coverage
+        run: >-
+          npx playwright test
+          tests/e2e/worker-hybrid-opfs.spec.js
+          tests/e2e/backend-auto-fallback.spec.js
+          tests/e2e/benchmark-page.spec.js
+          --config=playwright.storage.config.js
+          --reporter=line
+
+  # The PWA suite deliberately runs a production build serially: service-worker
+  # caching, coordinated dedicated-worker ownership, OPFS persistence, and
+  # offline reload are not faithfully tested by the Next development server.
+  test-pwa-storage:
+    runs-on: ubuntu-latest
+    needs: test-wasm
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+
+      - name: Install PWA dependencies
+        working-directory: pwa
+        run: npm install
+
+      - name: Install Chromium
+        working-directory: pwa
+        run: npx playwright install --with-deps chromium
+
+      - name: Run production PWA storage tests
+        working-directory: pwa
+        run: npm run test:pwa
+
   # Check version consistency (informational)
   check-versions:
     runs-on: ubuntu-latest

.gitignore

@@ -2,6 +2,8 @@ target/
 node_modules/
 test_env/
 target
+.cosynt/
+.vectordb/
 
 # Test artifact directories (native fs_persist)
 .absurdersql_fs/

Cargo.toml

@@ -113,6 +113,8 @@ opentelemetry-prometheus = "0.14"
 [features]
 default = ["console_error_panic_hook", "console_log", "bundled-sqlite"]
 fs_persist = []
+opfs = []
+hybrid = ["opfs"]
 telemetry = ["prometheus", "opentelemetry", "opentelemetry_sdk", "opentelemetry-prometheus"]
 bundled-sqlite = ["rusqlite", "rusqlite/bundled"]
 encryption = ["rusqlite", "rusqlite/sqlcipher"]  # Android: links pre-built SQLCipher in jniLibs

README.md

@@ -1,7 +1,7 @@
 <div align="center">
   <img src="absurder-sql.png" alt="AbsurderSQL Logo" width="200"/>
   <h1>AbsurderSQL</h1>
-  <p><strong>Rust + WASM + SQLite + IndexedDB</strong></p>
+  <p><strong>Rust + WASM + SQLite + IndexedDB/OPFS</strong></p>
 
   [![npm](https://img.shields.io/npm/v/@npiesco/absurder-sql)](https://www.npmjs.com/package/@npiesco/absurder-sql)
   [![npm downloads](https://img.shields.io/npm/dm/@npiesco/absurder-sql)](https://www.npmjs.com/package/@npiesco/absurder-sql)
@@ -14,6 +14,7 @@
 [![WASM](https://img.shields.io/badge/wasm-supported-blue)](https://webassembly.org/)
 [![SQLite](https://img.shields.io/badge/sqlite-embedded-blue)](https://www.sqlite.org/)
 [![IndexedDB](https://img.shields.io/badge/indexeddb-browser-green)](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API)
+[![OPFS](https://img.shields.io/badge/opfs-worker%20hybrid-teal)](docs/HYBRID_OPFS_PLAN.md)
 
 **Capabilities:**
 [![Dual Mode](https://img.shields.io/badge/mode-Browser%20%2B%20Native-purple)](docs/DUAL_MODE.md)
@@ -24,15 +25,15 @@
 [![Grafana](https://img.shields.io/badge/grafana-dashboards-orange)](monitoring/grafana/)
 [![DevTools](https://img.shields.io/badge/devtools-chrome%2Ffirefox-blue)](browser-extension/)
 
-> *SQLite + IndexedDB + Custom VFS that's absurdly absurder than absurd-sql*
+> *SQLite + IndexedDB/OPFS + Custom VFS that's absurdly absurder than absurd-sql*
 
 ## This is an absurd*er* project.
 
-It implements a custom SQLite Virtual File System (VFS) backend that treats **IndexedDB like a disk** and stores data in blocks there. Your database lives permanently in browser storage with **intelligent block-level I/O**—reading and writing 4KB chunks with LRU caching—avoiding the performance nightmare of serializing entire database files on every operation.
+It implements a custom SQLite Virtual File System (VFS) backend that treats **browser storage like a disk**. In browsers, AbsurderSQL can use **IndexedDB directly** or a **Hybrid OPFS + IndexedDB backend** that keeps blocks in OPFS and metadata in IndexedDB. Your database lives permanently in browser storage with **intelligent block-level I/O**—reading and writing 4KB chunks with LRU caching—avoiding the performance nightmare of serializing entire database files on every operation.
 
 **It basically stores a whole database into another database using a custom VFS. Which is absurd*er*.**
 
-But AbsurderSQL takes it further: it's **absurdly better**. Unlike absurd-sql, your data isn't locked in IndexedDB forever—you can **[export and import](docs/EXPORT_IMPORT.md)** standard SQLite files. Need to query from both browser and CLI? Use **[dual-mode persistence](docs/DUAL_MODE.md)**—same database structure, IndexedDB in the browser and real files on the server. Multiple tabs? **[Multi-tab coordination](docs/MULTI_TAB_GUIDE.md)** with automatic leader election prevents conflicts. Want production observability? Optional **[Prometheus + Grafana monitoring](monitoring/)** with **[DevTools extension](browser-extension/)** for debugging WASM telemetry.
+But AbsurderSQL takes it further: it's **absurdly better**. Unlike absurd-sql, your data isn't locked in browser storage forever—you can **[export and import](docs/EXPORT_IMPORT.md)** standard SQLite files. Need to query from both browser and CLI? Use **[dual-mode persistence](docs/DUAL_MODE.md)**—same database structure, browser persistence on IndexedDB or Hybrid OPFS+IndexedDB, and real files on the server. Multiple tabs? **[Multi-tab coordination](docs/MULTI_TAB_GUIDE.md)** with automatic leader election prevents conflicts. Need faster worker-backed browser persistence? Use the Hybrid backend with OPFS block I/O and IndexedDB metadata fallback. Want production observability? Optional **[Prometheus + Grafana monitoring](monitoring/)** with **[DevTools extension](browser-extension/)** for debugging WASM telemetry.
 
 **Read the [blog post](https://iscopesolutions.net/) that explains the absurdity in detail.**
 
@@ -42,18 +43,41 @@ But AbsurderSQL takes it further: it's **absurdly better**. Unlike absurd-sql, y
 
 A high-performance **tri-mode** Rust library that brings full SQLite functionality to **browsers, native applications, and mobile devices**:
 
-- **Browser (WASM)**: SQLite → IndexedDB with multi-tab coordination, Web Worker support, and full export/import
+- **Browser (WASM)**: SQLite → IndexedDB or Hybrid OPFS+IndexedDB, with multi-tab coordination, Web Worker support, and full export/import
 - **Native/CLI**: SQLite → Real filesystem with traditional `.db` files
 - **Mobile (React Native)**: SQLite → Device filesystem via UniFFI with SQLCipher encryption for iOS and Android
 
 **Unique Advantages:** 
 
-Export/import databases as standard SQLite files (absurd-sql has no export/import—data is permanently locked in IndexedDB). Build web apps that store data in IndexedDB, then query the same database structure from CLI/server using standard SQLite tools. Multi-tab coordination with automatic leader election prevents conflicts. Perfect for offline-first applications with backup/restore, data migration, and optional server synchronization.
+Export/import databases as standard SQLite files (absurd-sql has no export/import—data is permanently locked in browser storage). Build web apps that store data with IndexedDB by default or Hybrid OPFS+IndexedDB in worker contexts, then query the same database structure from CLI/server using standard SQLite tools. Multi-tab coordination with automatic leader election prevents conflicts. Perfect for offline-first applications with backup/restore, data migration, and optional server synchronization.
 
 **Production Observability (Optional):** When enabled with `--features telemetry`, includes complete monitoring stack: Prometheus metrics, OpenTelemetry tracing, pre-built Grafana dashboards, production-ready alert rules with runbooks, and a Chrome/Firefox DevTools extension for debugging WASM telemetry. All telemetry features are opt-in—default builds include zero monitoring overhead.
 
 Enabling production-ready SQL operations with crash consistency, multi-tab coordination, complete data portability, optional observability, and the flexibility to run anywhere from web apps to server applications.
 
+## Browser Storage Backends
+
+AbsurderSQL now exposes three browser constructors so you can choose persistence behavior explicitly:
+
+- `Database.newDatabase(name)` uses the default IndexedDB-backed browser path and works on the main thread.
+- `Database.newDatabaseAuto(name)` probes for OPFS `SyncAccessHandle` support and selects `Hybrid` in supported worker contexts, otherwise falls back to IndexedDB.
+- `Database.newDatabaseWithBackend(name, 'IndexedDB' | 'OPFS' | 'Hybrid')` forces a specific backend.
+- `db.getStorageBackend()` returns the backend actually in use.
+
+`Hybrid` is the recommended worker backend: blocks live in OPFS for fast block I/O, while IndexedDB stores metadata and provides fallback durability. Main-thread browser code should continue using IndexedDB or rely on `newDatabaseAuto()` to fall back automatically.
+
+```javascript
+import init, { Database } from '@npiesco/absurder-sql';
+
+await init();
+
+const mainThreadDb = await Database.newDatabase('app-main');
+const autoDb = await Database.newDatabaseAuto('app-auto');
+const workerHybridDb = await Database.newDatabaseWithBackend('app-worker', 'Hybrid');
+
+console.log(await autoDb.getStorageBackend());
+```
+
 ## Tri-Mode Architecture
 
 AbsurderSQL runs in **three modes** - Browser (WASM), Native (Rust CLI/Server), and Mobile (React Native):
@@ -102,8 +126,9 @@ graph TB
     end
 
     subgraph "Browser Persistence"
-        INDEXEDDB["IndexedDB<br/>(Browser Storage)"]
-        LOCALSTORAGE["localStorage<br/>(Coordination)"]
+      INDEXEDDB["IndexedDB<br/>(Browser Storage)"]
+      OPFS["OPFS<br/>(Worker SyncAccessHandle)"]
+      LOCALSTORAGE["localStorage<br/>(Coordination)"]
     end
 
     subgraph "Native Persistence"
@@ -141,7 +166,8 @@ graph TB
     SYNC -->|metadata| META
     EXPORT -->|read blocks| BS
     IMPORT -->|write blocks| BS
-    BS -->|"WASM mode"| INDEXEDDB
+    BS -->|"WASM IndexedDB"| INDEXEDDB
+    BS -->|"WASM Hybrid/OPFS"| OPFS
     BS -->|"Native mode"| FILESYSTEM
     NATIVE_DB -->|"fs_persist"| BLOCKS
     UNIFFI -->|"Mobile mode"| DEVICE_FS
@@ -162,6 +188,7 @@ graph TB
     style EXPORT fill:#ec4899,stroke:#333,color:#fff
     style IMPORT fill:#8b5cf6,stroke:#333,color:#fff
     style INDEXEDDB fill:#22c55e,stroke:#333,color:#000
+    style OPFS fill:#14b8a6,stroke:#333,color:#fff
     style QUEUE fill:#ef4444,stroke:#333,color:#fff
     style OBS fill:#92400e,stroke:#333,color:#fff
     style PROM fill:#1c1c1c,stroke:#333,color:#fff
@@ -220,7 +247,10 @@ absurder-sql/
 │   │   ├── import.rs                 # Database import from SQLite files
 │   │   ├── retry_logic.rs            # Retry logic for transient failures
 │   │   ├── fs_persist.rs             # Native filesystem persistence
+│   │   ├── backend_detect.rs         # Browser backend auto-detection
 │   │   ├── wasm_indexeddb.rs         # WASM IndexedDB integration
+│   │   ├── wasm_opfs.rs              # WASM OPFS block storage bridge
+│   │   ├── hybrid_store.rs           # Hybrid OPFS + IndexedDB orchestration
 │   │   ├── wasm_vfs_sync.rs          # WASM VFS sync coordination
 │   │   ├── recovery.rs               # Crash recovery logic
 │   │   ├── auto_sync.rs              # Native auto-sync
@@ -247,10 +277,19 @@ absurder-sql/
 │   ├── lru_cache_tests.rs            # Cache tests
 │   ├── e2e/                          # Playwright E2E tests
 │   │   ├── dual_mode_persistence.spec.js  # Browser + CLI validation
+│   │   ├── worker-hybrid-opfs.spec.js     # Worker Hybrid/OPFS durability coverage
+│   │   ├── benchmark-page.spec.js         # Benchmark page smoke coverage
 │   │   ├── advanced-features.spec.js
 │   │   └── multi-tab-vite.spec.js
 │   └── ...                           # 65+ test files total
 │
+├── pwa/                    # Production Next.js PWA
+│   ├── app/                # Application routes, including the database UI
+│   ├── lib/db/             # SharedWorker coordinator, runtime worker, typed proxy
+│   ├── public/sw.js        # Offline service worker and production asset cache
+│   ├── e2e/                # Production PWA Playwright coverage
+│   └── playwright.pwa.config.ts # Serial next build/start test configuration
+│
 ├── examples/               # Browser demos and documentation
 │   ├── vite-app/           # Production Vite application
 │   ├── export_import_demo.html        # Export/import 4-step wizard demo
@@ -259,6 +298,7 @@ absurder-sql/
 │   ├── sql_demo.html       # Interactive SQL demo page
 │   ├── web_demo.html       # Full-featured web interface
 │   ├── benchmark.html      # Performance comparison tool
+│   ├── absurder-benchmark-worker.js # Worker benchmark runner for Hybrid backend
 │   ├── multi-tab-demo.html # Multi-tab coordination demo
 │   ├── worker-example.html # Web Worker demo
 │   ├── worker-db.js        # Web Worker implementation
@@ -269,6 +309,7 @@ absurder-sql/
 │   ├── EXPORT_IMPORT.md    # Export/import guide (DATABASE PORTABILITY)
 │   ├── DUAL_MODE.md        # Tri-mode persistence guide
 │   ├── MULTI_TAB_GUIDE.md  # Multi-tab coordination
+│   ├── HYBRID_OPFS_PLAN.md # Hybrid OPFS implementation status and validation
 │   ├── TRANSACTION_SUPPORT.md # Transaction handling
 │   ├── BENCHMARK.md        # Performance benchmarks
 │   ├── ENCRYPTION.md       # SQLCipher encryption guide
@@ -769,6 +810,38 @@ worker.postMessage({ type: 'processData' });
 
 **Example:** See `examples/worker-example.html` for a complete working demo
 
+### Production PWA (Hybrid OPFS + IndexedDB)
+
+The checked-in [`pwa/`](pwa/) is a production Next.js PWA backed by AbsurderSQL's local WASM build. A `SharedWorker` coordinates tabs while one elected dedicated Worker owns and serializes the physical database runtime. The Hybrid backend stores blocks in OPFS, keeps coordination metadata and fallback data in IndexedDB, and reopens logical database handles after runtime handoff. The service worker precaches the emitted application, worker, WASM, JavaScript, and CSS assets for offline reload.
+
+Build the local WASM package first, then start the PWA:
+
+```bash
+# Repository root: build WASM and sync it into pwa/lib
+npm install
+npm run build
+
+cd pwa
+npm install
+npm run dev
+```
+
+These commands use your configured npm registry; do not add a direct public-registry override.
+
+Run the production PWA storage suite:
+
+```bash
+cd pwa
+npx playwright install chromium  # First run only
+npm run test:pwa
+```
+
+`test:pwa` runs serially against `next build` and `next start`. It verifies Hybrid selection and reopen, two-tab shared ownership with continued writes after one tab closes, and offline reload from persisted OPFS data.
+
+**Verified 2026-07-26:** all 3 production PWA Playwright scenarios passed in Chrome. Installation through the configured Microsoft CFS feed completed, `npm audit --json` reported 0 vulnerabilities, dependency resolution passed, and the production build completed.
+
+**Implementation provenance:** the PWA is an AbsurderSQL implementation by `npiesco`; it does not import Onyx source or packages. Onyx commits `d039d03`, `a4ecdea`, `417e23d`, `8a18a7c`, and `1d90c3c`, also by `npiesco`, supplied behavioral and production-test references. AbsurderSQL's Hybrid recovery, multi-tab coordination, and benchmark commits remain the implementation authority. See [**docs/HYBRID_OPFS_PLAN.md**](docs/HYBRID_OPFS_PLAN.md#pwa-integration-update-2026-07-25) for the per-section code and author references.
+
 ### Native/CLI Usage (Filesystem)
 
 ```bash
@@ -848,6 +921,8 @@ await db.close();
 - Full export/import for backup/restore
 - Type-safe from Rust to TypeScript via UniFFI
 
+**Verified 2026-07-26:** the locked `uniffi-bindings` host matrix passes all 67 Rust integration tests. That includes all 9 real export/import cases: new-database restore, same-vault restore, encrypted round-trip, BLOB preservation, invalid handles, and missing files. The same matrix passes `cargo fmt` and `cargo clippy --all-targets -- -D warnings`.
+
 **Setup:** See [**absurder-sql-mobile/README.md**](absurder-sql-mobile/README.md) for build instructions.
 
 #### Mobile Development Environment Setup
@@ -1405,4 +1480,4 @@ This is a strong copyleft license that requires:
 
 See [LICENSE.md](LICENSE.md) for the full license text.
 
-**Why AGPL-3.0?** This license ensures that improvements to AbsurderSQL remain open source and benefit the entire community, even when used in cloud/SaaS environments.
+**Why AGPL-3.0?** This license ensures that improvements to AbsurderSQL remain open source and benefit the entire community, even when used in cloud/SaaS environments.