Skip to content

feat(core): flat models[] config + internal field migration (PR 1 of 3)#776

Merged
loopingz merged 18 commits into
mainfrom
feat/repository-refactor-pr1
Jun 14, 2026
Merged

feat(core): flat models[] config + internal field migration (PR 1 of 3)#776
loopingz merged 18 commits into
mainfrom
feat/repository-refactor-pr1

Conversation

@loopingz

@loopingz loopingz commented May 9, 2026

Copy link
Copy Markdown
Owner

Summary

First of three PRs for the Repository concept refactor. This PR introduces the flat models: string[] parameter on StoreParameters, migrates Store internals from _model/_modelMetadata/_modelType to _models[]/_modelMetadatas Map, and updates downstream stores (fs, postgres, runtime) to match.

Spec: docs/superpowers/specs/2026-05-09-repository-design.md (local-only — gitignored).
Plan: docs/superpowers/plans/2026-05-09-repository-design.md (local-only — gitignored).

What's in this PR (Tasks 1–5 of 8)

  • Task 1StoreParameters accepts models: string[] alongside legacy model + additionalModels; deprecation WARN log fires on legacy use; ambiguity guard throws when both shapes are set.
  • Task 2Store class migrated to _models: ModelClass[] and _modelMetadatas: Map<string, ModelMetadata>. _modelType dropped. New getModels() accessor; getModel() deprecated. computeParameters() rewritten to walk parameters.models[]. Latent bug fixed in recursive() subclass walker (the runtime type from Application.setModelMetadata is (string | ModelClass)[], not string[] as the type declared).
  • Task 3FileStore._get() strict-mode __type check rewritten to use _modelsHierarchy (replaces dropped _modelType reference).
  • Task 4PostgresStore.resolveTable() migrated off _modelMetadata; parameters.table now treated as a single-model shortcut, with WARN log when set with multiple models.
  • Task 5 — Final _model/_modelType references in fs/filestore.ts:205, fs/filestore-unit.spec.ts:299, runtime/invitationservice.spec.ts:100 migrated.

What's not in this PR (deferred)

  • Task 6 (migrate spec configs from model: to models:) — cosmetic; deprecation shim keeps existing configs working.
  • Task 7 (regenerate webda.module.json schemas) — surfaced a load-bearing regression where the legacy model: { default: "Webda/CoreModel" } schema default is essential for framework-loaded stores to claim every model. Reverted in commit ce6f4112. Needs a follow-up architectural fix.
  • Task 8 (final verification) — pending Task 7.

PRs 2 (Repository event re-emission) and 3 (API positioning / docs) follow this one.

Backward compatibility

  • All existing { model: "X", additionalModels: [...] } configs keep working via the deprecation shim. A WARN log fires once per service noting the migration path.
  • getModel() still returns the first model for back-compat (marked @deprecated).
  • useStore() is unchanged in this PR.

Test plan

  • pnpm --filter @webda/core test — 554 passed, 1 todo (was 550 before this PR; 4 new tests added).
  • pnpm --filter @webda/fs test — 123 passed (was 122; 1 new strict-mode test).
  • pnpm --filter @webda/postgres test — 3 new resolveTable tests pass; pre-existing PostgresStoreSmokeTest failures unrelated (DB connectivity).
  • CI build green (TBD on push).

How this was built

Each task: TDD (failing test → implementation → passing test → commit). Two-stage subagent review per task (spec compliance, then code quality), with fix loops where reviewers found issues. Two follow-up commits for code-review feedback land on top of the original task commits where applicable.

🤖 Generated with Claude Code


Update — PR 1 completed (Tasks 6–8)

The three deferred tasks are now done; PR 1 is feature-complete.

  • Task 6 (spec config migration) — migrated the canonical Store configs in core (memory, coremodel, oauth), fs (filestore) and postgres (smoke) specs from { model: "X" } to { models: ["X"] }. Deprecated-shape coverage is retained: memory.spec additionalModels(), filestore-unit loadParameters(), and the StoreParameters.load() compat-matrix unit tests still exercise the legacy model + additionalModels path. Added MemoryLogger-based assertions that the deprecation WARN fires on legacy input and stays silent on models[].

  • Task 7 (schema regeneration) — the "load-bearing regression" was a phantom. Investigated the feared model: { default: "Webda/CoreModel" } default: the compiler-generated loadParameters() builds params via new StoreParameters().load(data) with no JSON-schema default injection, and service creation does no boot-time schema validation — so that default was documentation-only and never claimed models at runtime. Confirmed empirically (core was already regenerated and all tests pass). Force-regenerated the stale fs/postgres/runtime schemas (webdac skips regen when a package's own source digest is unchanged, so packages that merely inline StoreParameters never picked up the change). Legacy { model: "X" } configs keep working through the load() shim regardless.

    • aws/gcp/mongodb are out of scope: untouched by this branch and currently non-compiling for pre-existing/unrelated reasons (type drift, incomplete worktree deps). Their schemas remain stale as before this PR.
  • Task 8 (verification)@webda/core 559 passed / 1 todo (store.ts coverage up to 84.8% stmts, 71.4% fns, 89.2% lines), @webda/fs 123 passed, @webda/postgres resolveTable/non-DB tests pass (the 21 failures are pre-existing ECONNREFUSED — no local Postgres). Lint and prettier clean on touched packages; in-scope packages build clean. Also removed the now-obsolete parameters.models = [...] test workarounds that only existed to compensate for the pre-Task-7 stale schema.

Note on backward compatibility

Commit 082ca41b removed the deprecated model / additionalModels typed fields and getModel() from StoreParameters/Store, keeping only the load() input shim. Runtime back-compat for legacy configs is preserved by that shim. Because store schemas use additionalProperties: false, the regenerated schema no longer lists model/additionalModels, so the config UI / tooling would flag a legacy model: config — runtime is unaffected. This matches the direction set by 082ca41b.

loopingz and others added 13 commits May 9, 2026 07:36
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New `models: string[]` parameter
- `model` and `additionalModels` mapped to `models[]` with WARN log
- Throws if both old and new shapes are set
- Deprecation tests cover the five compat cases
…ll comment

Code review follow-ups for Task 1:
- Conflict between models[] and additionalModels detected even when
  additionalModels is empty (presence check, not length)
- Backfill comment explains why ??= is a no-op in the legacy branch
- Two new tests: empty-additionalModels conflict and additionalModels-only
  fallback to Webda/RegistryEntry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Drop _model, _modelMetadata, _modelType from Store class
- Add _models: ModelClass[] (index 0 = legacy primary)
- Add _modelMetadatas: Map<string, ModelMetadata>
- computeParameters() walks parameters.models[] (canonical list from Task 1)
- recursive() normalises ModelClass refs to string IDs as _modelsHierarchy keys
- getModel() returns _models[0] (deprecated, kept for back-compat)
- getModels() added as new public method
- setModelDefinitionHelper() sets _models[0] for test-harness compat
- Two new @test methods verify _models/_modelMetadatas and getModel() back-compat

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- populatesModelsArrayAndMetadatas now uses models: [...] constructor arg
  and sets getParameters().models directly before resolve() to exercise
  computeParameters() via the canonical models[] path (not the legacy shim)
- Removed dead-code test stubs from memory.spec.ts (file is excluded from vitest)
- Dropped extra useLog TRACE noise from computeParameters()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- walksSubclassHierarchyForNonStrictStore exercises recursive() ModelClass branch
  via Webda/User → Webda/SimpleUser depth-1 traversal
- strictStoreSkipsSubclassWalk verifies strict-mode short-circuit
- populatesModelsArrayAndMetadatas now also asserts getModels().length

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces dropped _modelType field. In strict mode, accept files whose
__type matches a depth-0 model in this store's models[]; subclasses
(depth > 0) remain excluded. Two new tests cover accept and reject paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ters.table for multi-model

- One model in models[]: parameters.table maps to that model (compat shortcut)
- Multiple models: parameters.table is ignored with WARN log at init()
- parameters.tables[id] continues as the canonical per-model override
- Default derivation (id.lower.replace('/', '_')) unchanged
- 3 new tests cover single, multi, and explicit-map paths in a separate
  PostgresStoreResolveTableTest suite (no DB connection required)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- fs/filestore.ts:205: this._model → this._models[0]
- fs/filestore-unit.spec.ts: getWithStrictMode now sets _modelsHierarchy
  directly (matching what computeParameters does at runtime)
- runtime/invitationservice.spec.ts: this.store._model = X → setModelDefinitionHelper(X)

Closes the migration started in Task 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After Tasks 1-5, StoreParameters exposes a flat models: string[] alongside
deprecated model + additionalModels. Regenerated schemas accept both shapes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@loopingz

loopingz commented May 9, 2026

Copy link
Copy Markdown
Owner Author

Task 7 (schema regen) — investigation update

After ~30 min of debugging the regression, found that running pnpm exec webdac build --force in packages/core regenerates webda.module.json such that Store.init() is no longer invoked for the Registry MemoryStore in test mode. As a result Store.computeStores() never runs and no Repository is registered for RegistryEntry, breaking 8 tests (CoreTest > registry, CoreTest > cov, CryptoServiceTest > *).

The schema diff itself looks superficially benign:

  • additionalModels/model lose default: ... (instead get deprecated: true)
  • A new models field is added with default: []

Things ruled out:

  • The models field's default value is not the trigger — restoring Webda/CoreModel as the model default did not fix it.
  • required array is unchanged for MemoryStore ([type] only).
  • MemoryStore's Configuration / Import paths in moddas are correct after regen.
  • Direct pnpm exec vitest run shows the same failure with intact schema (3843 lines), so no test-time corruption is at play.

Best guess: when both legacy model and new models fields are deprecated/added in the schema, something downstream in service registration / Modda lookup silently drops the Registry binding. Needs deeper trace through unpackedapplication.loadWebdaModule and Core.createService to confirm.

For now, Task 7 is reverted via fa27845b and the branch is clean (554/554 core tests pass). Tasks 1–5 are independently shippable thanks to the deprecation shim — schema regen + Tasks 6/8 will follow in a separate PR once the root cause is identified.

@codecov

codecov Bot commented Jun 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.02%. Comparing base (9f0155f) to head (30f9baf).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
packages/core/src/stores/store.ts 78.72% 4 Missing and 6 partials ⚠️
packages/postgres/src/postgresstore.ts 33.33% 1 Missing and 1 partial ⚠️
packages/fs/src/filestore.ts 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #776      +/-   ##
==========================================
+ Coverage   73.91%   74.02%   +0.11%     
==========================================
  Files         229      229              
  Lines       15603    15610       +7     
  Branches     3788     3790       +2     
==========================================
+ Hits        11533    11556      +23     
+ Misses       2982     2967      -15     
+ Partials     1088     1087       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

loopingz and others added 4 commits June 13, 2026 15:19
Force-regenerated the StoreParameters-derived schemas that went stale after
the Task 1-2 field migration (webdac skips regen when a package's own source
digest is unchanged, so packages that only inline core's StoreParameters
never picked up the change). Schemas now expose models[] and drop the
deprecated model/additionalModels properties, matching the regenerated core
schema.

Confirmed runtime-safe: loadParameters() builds params via
new StoreParameters().load(data) with no schema-default injection and no
boot-time schema validation, so the dropped `model: { default: Webda/CoreModel }`
was documentation-only. Legacy { model: "X" } configs keep working via the
load() deprecation shim.

aws/gcp/mongodb left as-is: untouched by this branch and currently
non-compiling for pre-existing/unrelated reasons (type drift, incomplete
worktree deps).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Migrate canonical Store configs in core (memory, coremodel, oauth),
  fs (filestore) and postgres (smoke) specs from { model: "X" } to
  { models: ["X"] }.
- Keep the deprecated-shape coverage: memory.spec additionalModels() and
  filestore-unit loadParameters() still exercise model + additionalModels;
  StoreParameters.load() unit tests still cover the full compat matrix.
- Add MemoryLogger-based assertions that the deprecation WARN fires on
  legacy { model } / { additionalModels } input and stays silent on
  { models }, covering the load() deprecation branch directly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…edge paths

Adds focused tests for previously-uncovered store.ts branches:
- load() throws on the removed `expose` parameter
- computeParameters() skips an unknown models[] entry without throwing
- setModelDefinitionHelper() overrides the primary model class

store.ts coverage: statements 79.8->84.8%, functions 64.3->71.4%,
lines 83.9->89.2%.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task 7 regenerated the StoreParameters schema, so the 'set
parameters.models directly' workarounds (added when filterParameters
stripped the then-unknown models[] field) are no longer needed:
direct-construction specs now receive models[] through load() normally.
Removed the redundant assignments and their now-stale comments in the
core StoreFieldsMigration tests and the postgres resolveTable tests, and
applied prettier formatting.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@loopingz
loopingz merged commit 56d4b01 into main Jun 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant