fix(llm): replace retired ollama_cloud default gemma3:27b with gemma4:31b-cloud#23
Open
yuzushi-dev wants to merge 2 commits into
Open
fix(llm): replace retired ollama_cloud default gemma3:27b with gemma4:31b-cloud#23yuzushi-dev wants to merge 2 commits into
yuzushi-dev wants to merge 2 commits into
Conversation
…:31b-cloud gemma3:27b was retired upstream on Ollama Cloud (returns 410 Gone on every key). It was the `ollama_cloud` entry in both DEFAULT_LLM_MODEL and the model catalog. No active step of the default tenant hits this default today (all LLM steps resolve their model via tenant config overrides), so this is a dormant last-resort fallback — but it is a landmine for any future tenant or step that falls through to the provider default. - model_registry.py: DEFAULT_LLM_MODEL["ollama_cloud"] and the catalog entry now point at gemma4:31b-cloud (verified reachable on the ollama_cloud pool). - test_model_registry_defaults.py: guard that every provider default exists in the catalog, and that the ollama_cloud default is not the retired model. Note: the runtime last-resort fallback is the env var DEFAULT_LLM_MODEL (.env / docker-compose), which is a separate prod-config change requiring an api/worker restart — not included here.
PR left the retired gemma3:27b in three places that override or contradict the new registry default, risking a split-brain where prod still points at the retired model: - .env.example: cutover template DEFAULT_LLM_MODEL example -> gemma4:31b-cloud. - docs/changelogs/v1.4.0.md: registry-default note and the cutover runbook line (an operator following it verbatim would set the retired model via env, which overrides the code default). - tests/integration/test_usage_metrics.py: fixture default model. Note (not addressed here): a wrong/retired model id still surfaces as a generic ProviderUnavailableError and trips the circuit breaker like a transient outage (retry storm). Classifying "model not found" as permanent is a separate hardening; gemma4:31b-cloud is confirmed present in the live Ollama Cloud catalog, so this is latent, not active.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
gemma3:27bwas retired upstream on Ollama Cloud — it now returns 410 Gone on every key. It was theollama_clouddefault in bothDEFAULT_LLM_MODELand the model catalog.Blast radius (verified on prod, read-only)
No active LLM step of the
defaulttenant hits this default today: HyDE,chat.generationand the 10 configuredllm_stepsall resolve their model via tenant-config overrides (glm-5.2:cloud/ per-step models), so the retired constant is a dormant last-resort fallback. It is still a landmine for any future tenant or step that falls through to the provider default.gemma4:31b-cloudwas tested live on theollama_cloudpool (clean completion) and is already used by the tenant's per-step overrides.Change
model_registry.py:DEFAULT_LLM_MODEL["ollama_cloud"]and the catalog entry →gemma4:31b-cloud.tests/unit/test_model_registry_defaults.py: guard that every provider default exists in the catalog and that the ollama_cloud default is not the retired model (catches this drift class going forward).Not included (separate, needs restart)
The runtime last-resort fallback is the env var
DEFAULT_LLM_MODELin.env/docker-compose.yml, stillgemma3:27b. Updating it requires anapi+workerrestart (prod config change) and is tracked separately.Correction to earlier triage
An earlier diagnostic reported "HyDE broken in prod" — that was a test artifact (empty tenant_config bypassed the DB override and fell to this broken env default). With real tenant config, HyDE resolves to
glm-5.2:cloudand works. This PR only removes the dormant landmine.