Feat: Add models.dev pricing metadata pipeline#863
Conversation
8020d71 to
28756e3
Compare
|
Current CI build failure seems to be caused by increased pressure from the new tests introduced for the feature. |
28756e3 to
df87d1c
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df87d1cd99
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| func containsProviderModels(from cachedCatalog: ModelsDevCatalog) -> Bool { | ||
| cachedCatalog.providers.allSatisfy { providerID, cachedProvider in | ||
| guard let provider = self.providers[ModelsDevProvider.normalizeProviderID(providerID)] else { return false } | ||
| return cachedProvider.models.keys.allSatisfy { provider.models.keys.contains($0) } |
There was a problem hiding this comment.
Validate model continuity using model IDs, not map keys
refreshIfNeeded treats a fetched catalog as invalid when any cached model key is missing, but lookup itself supports key/ID divergence by matching ModelsDevModel.id in a second pass. Because containsProviderModels compares only cachedProvider.models.keys to provider.models.keys, a legitimate upstream key rename (same id, new map key) will be misclassified as a dropped model and block cache updates indefinitely, leaving pricing stale even though the model still exists.
Useful? React with 👍 / 👎.
steipete
left a comment
There was a problem hiding this comment.
Reviewed model pricing pipeline changes and local validation; CI is green. Thanks @iam-brain!
Model pricing metadata
CodexBar has an additive models.dev pricing pipeline for future cost lookup work. Existing hardcoded pricing remains unchanged for now.
Source and cache
https://models.dev/api.json~/Library/Caches/CodexBar/model-pricing/models-dev-v1.jsonThe pipeline lets future scanner code read the last valid cache synchronously with
ModelsDevPricingPipeline.lookupand refresh stale metadata separately withModelsDevPricingPipeline.refreshIfNeeded. If a refresh fails, the last valid cache remains usable.Lookup rules
Pricing is scoped by provider id and model id. This prevents two providers with the same model id or display name from sharing pricing accidentally.
Planned local source mapping:
openaianthropicgoogle-vertex-anthropicThe first integration PR only adds the parser, client, cache, provider-scoped lookup, and tests. It does not route live cost calculations through models.dev yet.
Units
models.dev publishes costs as USD per 1M tokens. CodexBar converts those to USD per token in the metadata layer:
When models.dev includes
cost.context_over_200k, CodexBar parses those values as the above-200k-token pricing lane and converts them with the same per-1M-token rule.Summary
Reasoning
Scope
References
https://models.dev/api.jsonValidation
swift test --filter ModelsDevPricingTestspnpm check./Scripts/compile_and_run.shlint-build-testcurrently fails in existing Codex RPC fallback/baseline tests withtimeout(method: "initialize"); the models.dev pricing suite passes locally.swift testwas also run earlier; it still fails on the pre-existingMistralUsageParserTests.parses dates from responseexpectation (expected month 11, got 10).