remove auto disable default thinking - #179
Open
jgieringer wants to merge 3 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes the special-casing that forcibly disabled Claude “adaptive thinking” when thinking_effort was not explicitly provided, so that model defaults remain in effect (notably for Sonnet 5–style models). It also extends the quirk registry to cover opus-5 and updates documentation/tests to reflect the new behavior.
Changes:
- Removed the
defaults_thinking_onquirk and stopped auto-injectingthinking={"type":"disabled"}whenthinking_effortis unset. - Added
opus-5to the Claude quirk map and ensuredmax_tokensis defaulted appropriately where LangChain lacks a model profile. - Updated unit tests and docs for the adjusted “thinking” defaults and new model marker.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
llm_clients/claude_llm.py |
Removes defaults_thinking_on, adds opus-5 quirks, and adjusts init logging/extra-params handling around thinking. |
tests/unit/llm_clients/test_claude_llm.py |
Updates expectations for Sonnet 5 when effort is unset and adds coverage for opus-5 max-tokens quirk. |
docs/evaluating.md |
Removes the deprecated quirk from the documentation table and documents sparse max-tokens behavior for opus-5/sonnet-5. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
luca-belli
approved these changes
Jul 28, 2026
| | `adaptive_thinking` | `opus-4-7`, `opus-4-8`, `fable-5`, `sonnet-5` | Uses `thinking={"type": "adaptive"}` + `effort=<level>` instead of manual `budget_tokens` (which 400s on these models); rejects `temperature`/`top_p`/`top_k` at any non-default value. | | ||
| | `defaults_thinking_on` | `sonnet-5` | Runs adaptive thinking by default when `thinking` is omitted, so `thinking_effort` must be explicitly unset to disable it. Not needed for `fable-5`, where adaptive thinking is always on and can't be disabled at all. | | ||
| | `sparse_max_tokens_profile` | `sonnet-5` | The installed `langchain-anthropic` has no model-profile entry for this model, so it silently falls back to `max_tokens=4096` instead of the usual 64k–128k auto-set — too tight for structured output. The client sets an explicit default instead. | | ||
| | `adaptive_thinking` | `opus-4-7`, `opus-4-8`, `opus-5`, `fable-5`, `sonnet-5` | Uses `thinking={"type": "adaptive"}` + `effort=<level>` instead of manual `budget_tokens` (which 400s on these models); rejects `temperature`/`top_p`/`top_k` at any non-default value. | |
Collaborator
There was a problem hiding this comment.
I wonder if there is a way of auto populating those by looking at the code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
v1.2 had a flag for Anthropic models that utilized thinking by default, even if thinking wasn't specified.
In favor of letting defaults be defaults, this PR removes
defaults_thinking_onwhich would disable thinking if model parameters didn't explicitly specify to use thinking.