Wrap Claude 3.7 thinking tokens in <thinking> tags
#11486
Conversation
| @@ -1127,6 +1127,14 @@ impl AnthropicLLM { | |||
| StreamContent::AnthropicStreamThinking( | |||
| thinking, | |||
| ) => { | |||
| // Send <thinking> tag at the start of a thinking block | |||
| let _ = event_sender.send(json!({ | |||
There was a problem hiding this comment.
aren't you sending one at each event?
There was a problem hiding this comment.
at each new thinking block only it would seem (side note: trying out an AI agent tool here, launched the agent, tested, opened the PR)
There was a problem hiding this comment.
if i get multiple thinking blocks I separate them nicely in <thinking> elements I think
There was a problem hiding this comment.
aren't we getting such events per token? or token group?
There was a problem hiding this comment.
here it's a group of token with content_block_start as the event_type
There was a problem hiding this comment.
The question is can we get multiple in a row? If not which is your argument here we should just wrap here instead of the extra logic?
There was a problem hiding this comment.
I think we can't, we're supposed to get a block start then deltas then a block stop
There was a problem hiding this comment.
Ok I hadn't seen the name of the event up top. 👍
Description
<thinking>tags in order to fallback on front's delimiter detection. It thus aligns the behavior of 3.5 and 3.7.Before/After
Tests
Risk
Deploy Plan