An OpenAI fallback when the primary provider dies mid-demo - #67
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
A hackathon deploy runs on borrowed or personal API keys with nobody on call. A Bedrock bearer token expiring overnight, or an OpenRouter rate limit while a room full of people all try the prototype at once, is the ordinary failure mode here, not the rare one. generateStructured/streamStructured now retry once against an OpenAI model if OPENAI_FALLBACK_API_KEY is set, on top of whatever LLM_PROVIDER already names. Deliberately a different vendor from every existing option (Anthropic direct, OpenRouter, Bedrock) rather than a fourth way to reach Claude — an OpenRouter outage and an Anthropic outage are correlated with each other in a way neither is with OpenAI's. Absent the env var, this is a total no-op: existing deployments are unaffected. streamStructured only retries before the first partial has been yielded — falling back mid-stream would mean un-yielding output a caller already rendered, and that window is exactly where a dead credential or a 429 fails anyway. "With limits" is a dashboard setting, not something this code can enforce — documented in .env.example: pair the key with a hard monthly budget in the OpenAI dashboard (Settings -> Limits). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A hackathon deploy runs on borrowed or personal API keys with nobody on call. A Bedrock bearer token expiring overnight, or an OpenRouter rate limit while a room full of people all try the prototype at once, is the ordinary failure mode here, not the rare one.
What it does
generateStructured/streamStructurednow retry once against an OpenAI model ifOPENAI_FALLBACK_API_KEYis set, on top of whateverLLM_PROVIDERalready names. Deliberately a different vendor from every existing option (Anthropic direct, OpenRouter, Bedrock) rather than a fourth way to reach Claude — an OpenRouter outage and an Anthropic outage are correlated with each other in a way neither is with OpenAI's.Absent the env var, this is a total no-op — existing deployments are unaffected.
streamStructuredonly retries before the first partial has been yielded. Falling back mid-stream would mean un-yielding output a caller already rendered, and that window is exactly where a dead credential or a 429 fails anyway, so nothing real is given up.Config
"With limits" is a dashboard setting, not something this code can enforce — documented in
.env.example: pair the key with a hard monthly budget in the OpenAI dashboard (Settings → Limits).Verified: typecheck clean, lint clean (0 errors, pre-existing warnings only), production build succeeds.
🤖 Generated with Claude Code