Skip to content

bug(config): validation misses router.stats.baselineModel #249

Description

@MicroMilo

UI-related note: this touches the Settings/UI path and may be lower priority during the ongoing frontend refactor.

Summary

UI-side config validation does not check router.stats.baselineModel, even though the core router parser treats it as a strict provider/model ref.

Why it matters

If a provider is deleted or renamed, router.stats.baselineModel can still point to the old provider. Settings validation can pass and save the config, but core load fails with a fatal router diagnostic. This is another save-success/load-failure config path.

Evidence

  • src/router/config/schema.ts:61 defines baselineModel as the provider/model ref used for saved-cost baseline calculation.
  • src/router/config/parseRouterConfig.ts:76 parses router stats.
  • src/router/config/parseRouterConfig.ts:491 enters parseStats.
  • src/router/config/parseRouterConfig.ts:532 validates raw.baselineModel through optionalRef.
  • src/router/config/parseRouterConfig.ts:566 turns ref resolution issues into fatal diagnostics.
  • src/router/config/schema.ts:229 looks up the referenced provider.
  • src/router/config/schema.ts:232 emits ROUTER_REF_PROVIDER_NOT_FOUND for an unknown provider.
  • ui/server/services/pilotdeckConfig.js:191 starts validateRouterModelRefs, but only checks scenarios, fallback, and tokenSaver refs.
  • ui/server/services/pilotdeckConfig.js:248 calls that incomplete validator.
  • ui/server/services/pilotdeckConfig.js:511 starts provider deletion/orphan cleanup.
  • ui/server/services/pilotdeckConfig.js:525 repairs scenario refs.
  • ui/server/services/pilotdeckConfig.js:530 repairs fallback refs.
  • ui/server/services/pilotdeckConfig.js:535 repairs tokenSaver refs.
  • ui/server/services/pilotdeckConfig.js:548 returns without repairing router.stats.baselineModel.
  • ui/src/components/settings/view/tabs/PilotDeckConfigTab.tsx:176 models router.stats in the Settings config type.
  • ui/src/components/settings/view/tabs/PilotDeckConfigTab.tsx:2202 edits router.stats.modelPricing.
  • ui/src/components/settings/view/tabs/PilotDeckConfigTab.tsx:3021 toggles router.stats.enabled.
  • ui/server/routes/config.js:120 and ui/server/routes/config.js:124 route UI saves through the config writer.

Validation

Validation level: dynamic reproduction plus source-control-flow confirmation.

Minimal reproduction:

  1. Create a config with a valid agent.model.
  2. Set router.stats.baselineModel: old/m.
  3. Ensure model.providers.old does not exist.
  4. Run UI validation and then core router parsing on the same config.

Key output:

  • validatePilotDeckConfig(...) returns valid: true.
  • parseRouterConfig(...) emits fatal diagnostic:
    • router.stats.baselineModel references unknown provider old.

Boundary: this validates the validator/parser mismatch directly. It does not require the stats collector to write data. It does not depend on the frontend rendering path beyond the save route using the same validation service.

Expected behavior

Every model ref accepted by Settings should be validated against configured providers before save. router.stats.baselineModel should also participate in provider rename and provider deletion cleanup, matching the strict parser behavior.

Existing coverage checked

No matching fix was found.

Checked adjacent work:

Search terms checked included router.stats.baselineModel, validateRouterModelRefs stats, and baselineModel references unknown provider.

Suggested fix

Add router.stats.baselineModel to:

  • UI/server save-time router ref validation.
  • Provider rename rewrite logic.
  • Provider deletion/orphan cleanup logic.

Prefer using the same model-ref validation and rewrite helpers already used for router.scenarios, router.fallback, and router.tokenSaver.

Suggested tests

  • UI/server validation rejects router.stats.baselineModel: old/m when old is not configured.
  • Provider rename rewrites router.stats.baselineModel.
  • Provider deletion/orphan cleanup repairs or removes stale router.stats.baselineModel.
  • A config accepted by UI validation should not produce fatal ROUTER_REF_PROVIDER_NOT_FOUND diagnostics for router.stats.baselineModel in the core parser.

Submitted with Codex.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions