Skip to content

Conversation

jordibeen
Copy link
Contributor

Summary

This PR implements Anthropic's Extended Thinking feature in compliance with their official documentation. The feature enables Claude models (Sonnet 4, Opus 4.1, Haiku 3.5) to display their reasoning process wrapped in <think> tags before providing the final response.

image

Changes

  • Full support for Anthropic's thinking feature with configurable token budgets
  • Proper <think> tag wrapping for clear visual separation
  • Tracks processing state between thinking and regular content blocks
  • Prevents thinking blocks in chat topic generation
  • Removed legacy line.delta.text fallback (superseded by proper text_delta handling)
  • Removed legacy anthropic-beta: messages-2023-12-15 header
  • Isolated to Anthropic models without affecting other providers

Technical Implementation

1. Payload Preparation (dispatcher.lua)

  • Added thinking_budget parameter support for Anthropic models
  • When thinking_budget is present in model configuration, adds thinking object to API payload:
    payload.thinking = {
      type = "enabled",
      budget_tokens = model.thinking_budget
    }

2. Response Stream Parsing (dispatcher.lua)

Enhanced Anthropic response parser to handle new thinking block events:

Event Type Action
content_block_start with type: "thinking" Outputs <think> tag
content_block_delta with type: "thinking_delta" Outputs thinking content
content_block_start with type: "text" Closes thinking block with </think>
content_block_delta with type: "text_delta" Outputs regular response content

Usage Example

Example configuration of using Sonnet 4 with thinking budget tokens:

{
	provider = "anthropic",
	name = "ChatClaude-Sonnet-4-Thinking",
	chat = true,
	command = false,
	model = {model = "claude-sonnet-4-20250514", thinking_budget = 1024 },
	system_prompt = require("gp.defaults").chat_system_prompt,
}

@Robitx Robitx merged commit 8e2d86a into Robitx:main Aug 11, 2025
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants