fix: pass model and llm_provider to ContextWindowExceededError constructor#275
Open
paulxg12 wants to merge 1 commit into
Open
fix: pass model and llm_provider to ContextWindowExceededError constructor#275paulxg12 wants to merge 1 commit into
paulxg12 wants to merge 1 commit into
Conversation
…uctor When litellm raises a non-ContextWindowExceededError that indicates context overflow (e.g. BadRequestError with 'prompt is too long'), _call_llm_streaming and _call_llm_non_streaming try to re-raise it as ContextWindowExceededError but omit the required model and llm_provider arguments, causing: TypeError: ContextWindowExceededError.__init__() missing 2 required positional arguments: 'model' and 'llm_provider' This prevents context compaction from triggering and the session is permanently stuck in an error loop. Fixes huggingface#270
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fix a
TypeErrorin_call_llm_streamingand_call_llm_non_streamingthat prevents context compaction from triggering when the LLM responds with a context overflow error.Why
Fixes #270. When litellm raises a
BadRequestError(or similar) indicating the prompt exceeds the model context window, the exception handlers attempt to re-raise it asContextWindowExceededErrorbut omit the requiredmodelandllm_providerpositional arguments:This prevents the
except ContextWindowExceededError:handler in the main agent loop (line ~1692) from ever catching the error, so context compaction never fires. The session is permanently stuck throwing the sameTypeErroron every iteration.How
Both
_call_llm_streaming(line ~876) and_call_llm_non_streaming(line ~1033) now pass:model— fromllm_params.get("model", "")llm_provider— extracted as the prefix before/in the model string (e.g."openai","anthropic","bedrock")These values are already available in
llm_paramsat the call site.Testing
except ContextWindowExceededError:handler inrun_agentat line ~1692 can now correctly catch the re-raised error and trigger context compaction.Checklist