-
Notifications
You must be signed in to change notification settings - Fork 2.7k
feat(deepseek): implement interleaved thinking mode for deepseek-reasoner #9969
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
hannesrudolph
wants to merge
6
commits into
main
Choose a base branch
from
feat/deepseek-interleaved-thinking
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+768
−74
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Re-review complete. No new issues were identified for the latest commit.
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
|
so needed too for kimi-k2-thinking here is some more details on that tech # Your tool implementation
def get_weather(city: str) -> dict:
return {"weather": "Sunny"}
# Tool schema definition
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Retrieve current weather information. Call this when the user asks about the weather.",
"parameters": {
"type": "object",
"required": ["city"],
"properties": {
"city": {
"type": "string",
"description": "Name of the city"
}
}
}
}
}]
# Map tool names to their implementations
tool_map = {
"get_weather": get_weather
}
def tool_call_with_client(client: OpenAI, model_name: str):
messages = [
{"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
{"role": "user", "content": "What's the weather like in Beijing today? Use the tool to check."}
]
finish_reason = None
while finish_reason is None or finish_reason == "tool_calls":
completion = client.chat.completions.create(
model=model_name,
messages=messages,
temperature=1.0,
tools=tools, # tool list defined above
tool_choice="auto"
)
choice = completion.choices[0]
finish_reason = choice.finish_reason
if finish_reason == "tool_calls":
messages.append(choice.message)
for tool_call in choice.message.tool_calls:
tool_call_name = tool_call.function.name
tool_call_arguments = json.loads(tool_call.function.arguments)
tool_function = tool_map[tool_call_name]
tool_result = tool_function(**tool_call_arguments)
print("tool_result:", tool_result)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"name": tool_call_name,
"content": json.dumps(tool_result)
})
print("-" * 100)
print(choice.message.content) |
8 tasks
…oner - Add thinking parameter support for deepseek-reasoner model - Handle streaming reasoning_content from DeepSeek API - Add tool call conversion (tool_use -> tool_calls) for thinking mode - Add tool result conversion (tool_result -> tool messages) - Extract reasoning from content blocks for API continuations - Add getReasoningContent() method for accumulated reasoning - Add comprehensive tests for interleaved thinking mode
Enable reasoning_content to be passed back to DeepSeek API during tool call continuations within the same turn. This is required for DeepSeek's interleaved thinking mode to work correctly with native tool calls. See: https://api-docs.deepseek.com/guides/thinking_mode - Add preserveReasoning: true to deepseek-reasoner model - Add tests verifying preserveReasoning flag
…g merge - Change OpenAiHandler.client from private to protected to allow DeepSeekHandler to access it properly without @ts-ignore - Remove unused getClient() method from DeepSeekHandler - Fix r1-format.ts to use finalReasoning (includes extracted reasoning from content blocks) instead of reasoningContent (top-level only) when merging assistant messages
09cdadf to
200b1b4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Enhancement
New feature or request
PR - Draft / In Progress
size:XL
This PR changes 500-999 lines, ignoring generated files.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Implements proper interleaved thinking support for DeepSeek V3's thinking mode (deepseek-reasoner model) following the DeepSeek API documentation at https://api-docs.deepseek.com/guides/thinking_mode
Changes
thinking: { type: "enabled" }parameter fordeepseek-reasonermodelreasoning_contentfrom DeepSeek API that yields reasoning chunkstool_use→ OpenAItool_callsformat) for thinking modetool_result→ OpenAItoolmessages)getReasoningContent()method to accumulate and expose reasoning contentHow It Works
Per DeepSeek's API documentation:
thinking: { type: "enabled" }fordeepseek-reasonermodelreasoning_contentis yielded as reasoning chunks and accumulatedreasoning_contentis preserved and passed back to the API for continuationImportant
Implements interleaved thinking mode for DeepSeek reasoner model, handling reasoning content and tool calls, with comprehensive tests added.
deepseek-reasonermodel withthinking: { type: "enabled" }parameter.reasoning_contentand tool calls inDeepSeekHandler.getReasoningContent()inDeepSeekHandlerto expose accumulated reasoning content.convertToR1Format()to handlereasoning_contentand tool calls.deepseek.spec.ts.r1-format.spec.ts.This description was created by
for f03a5b6. You can customize this summary. It will automatically update as commits are pushed.