Skip to content

[BUG]: ModelConfig Bedrock Thinking not working. #1870

@marcofranssen

Description

@marcofranssen

I have been testing this, and it doesn't seem to work.

LLM error: API_ERROR operation error Bedrock Runtime: Converse, https response error StatusCode: 400, RequestID: 7dbf6c3a-d526-43ef-a9b9-b48228b1821f, ValidationException: The value at messages.1.content.2.toolUse.input is empty.

https://docs.aws.amazon.com/bedrock/latest/userguide/claude-messages-extended-thinking.html

According to Claude it thinks following:

Now I have the full picture. The Bedrock SDK has ContentBlockMemberReasoningContent with ReasoningTextBlock{Text, Signature}
  and RedactedContent []byte. The genai Part struct has Thought bool and ThoughtSignature []byte.

  The fix requires two things in bedrock.go:

  1. generateNonStreaming: When extracting response content, handle ContentBlockMemberReasoningContent and emit
  genai.Part{Thought: true, Text: ..., ThoughtSignature: ...}
  2. generateStreaming: Track reasoning content deltas (text + signature) and emit thinking parts
  3. convertGenaiContentsToBedrockMessages: When building messages from conversation history, handle part.Thought == true parts
  by emitting ContentBlockMemberReasoningContent

Originally posted by @marcofranssen in #1732 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions