Skip to content

feat(openrouter): native vendor selection for OpenRouter models (config and/or /model Vendor pane) #6007

Description

@7jrxt42BxFZo4iAnN4CX

Problem

OpenRouter routes each model across multiple vendors (upstream providers). Sometimes you need to pin a specific one (quality, latency, family-specific behavior), but codewhale has no first-class way to do that today:

  • Vendor pinning works only through the model string — OpenRouter passes model ids verbatim (aggregator passthrough, crates/config/src/route/resolver.rs:445), so @preset/... (account presets) and OpenRouter suffixes :vendor/:floor/:ceil all function inside [providers.openrouter] model. This is undocumented: no mention of @preset/ or :vendor in docs/CONFIGURATION.md, docs/design/, or docs/rfcs/.
  • The request body carries no OpenRouter vendor controls: crates/tui/src/client/chat.rs:1048 sends only model/messages/max_tokens/stream (+ temperature/tools). OpenRouter's native provider.order/provider.allow_fallbacks fields are never emitted.
  • /models/{id}/endpoints (the only OpenRouter API that lists a model's vendors, with per-vendor pricing) is never fetched; the current parse (crates/tui/src/client.rs:3264) keeps only top_provider context/max_output. Nothing exists to feed a vendor picker.

Proposed solution

Three options, increasing in scope — open for maintainers to pick the one that fits codewhale best:

Option A — Document the existing string-based pinning (docs only). In docs/CONFIGURATION.md (OpenRouter section), show model = "@preset/..." and model = "deepseek/deepseek-chat:deepinfra" (plus :floor/:ceil), and state that the id is forwarded verbatim.

  • Effort: small (docs). Risk: none. UX: unchanged — user types the suffix by hand.

Option B — Config field → body-level vendor pinning. Add vendor (name TBD: vendor/route/provider) to [providers.openrouter], e.g. vendor = "deepinfra". When set, emit "provider": { "order": ["deepinfra"], "allow_fallbacks": false } in the body builder (crates/tui/src/client/chat.rs). Optionally show the pinned vendor in the status line next to the model.

  • Effort: small–medium. Risk: low — empty vendor keeps current behavior. UX: survives restarts, visible in status.

Option C — Full Vendor pane in the /model picker. Add a third pane alongside Model | Effort (Pane enum {Model, Effort}, crates/tui/src/tui/model_picker.rs:173, Tab cycle :719-720 — a Vendor pane slots in identically). Fetch GET {base}/models/{id}/endpoints when a model is selected. Persist (model, vendor) in provider_models/settings; when set, emit provider.order and show the vendor in the status line and picker hints (model · vendor).

  • Effort: several days. Risk: medium. UX: discoverable, one Tab away from model selection.

Use case

I keep a specific OpenRouter preset (@preset/v4-flash-stream-lake) to pin one vendor (StreamLake/fp8). It works today only because the model string passes through verbatim — undocumented, no UI, and invisible in the /model picker except as a manually-typed Configured row. A native vendor control would make the pin explicit, visible next to the model, and selectable.

Alternatives considered

  • Account presets in OpenRouter settings (@preset/...): works, but managed outside codewhale and undocumented here.
  • Model-string suffix (deepseek/deepseek-chat:deepinfra): works via passthrough, but invisible to the picker and unknown to the catalog (no pricing/context).
  • Polling /models/{id}/endpoints from a sidecar: overkill; belongs in the app.

Impact

Anyone using OpenRouter for models with multiple upstream vendors (quality/latency-sensitive work). Documenting A is nearly free; C adds discoverability. Frequency: every time a user needs a specific vendor rather than OpenRouter's default routing.

Additional context

Open questions for the maintainers / AI to decide:

  1. Body vs string: provider.order (per-request, OpenRouter-native) vs :vendor suffix (survives proxies that strip body fields)?
  2. Catalog visibility: should :vendor-suffixed rows appear in discovery views (Coding/Cheap/LongContext), or only Configured?
  3. Per-vendor pricing: show the per-vendor prices from /models/{id}/endpoints (only viable in C), or keep single top_provider price?
  4. @preset/... as the blessed path: is documenting account presets (A) enough?

References: crates/tui/src/client/chat.rs:1048, crates/tui/src/tui/model_picker.rs:173-230/:719-720, crates/tui/src/client.rs:3264-3340, crates/config/src/route/resolver.rs:445-493.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    • Status
      Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions