Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

参照文档搭建支持function calling的qwen2 openai server时报错openai.BadRequestError: Error code: 400 #517

Open
niykx opened this issue Jul 4, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@niykx
Copy link

niykx commented Jul 4, 2024

根据文档,运行支持工具调用的openai api服务测试时,modelscope-agent向vllm发出请求:
message: call openai api, model: Qwen2-7B-Instruct, messages: [{'role': 'system', 'content': '\n# 指令\n\nNone\n\n请注意:你具有图像和视频的展示能力,也具有运行代码的能力,不要在回复中说你做不到。\n'}, {'role': 'user', 'content': '北京有多少个叫张三的人?', 'tool_calls': None}, {'role': 'assistant', 'content': 'Action: locationNameFunction\nAction Input: {"location": "北京", "name": "张三"}\n', 'tool_calls': [{'type': 'function', 'function': {'name': 'locationNameFunction', 'arguments': '{"location": "北京", "name": "张三"}'}}]}, {'role': 'user', 'content': '(你可以使用工具:[locationNameFunction])Response[message=10个]'}],
"stop": ["", "Observation:", "Observation:\n"], stream: True, args: {'tools': None, 'tool_choice': None}

vllm响应:openai.BadRequestError: Error code: 400
向vllm搭建的openai_api_server请求时:需要去掉stop和args才能正常响应。

从openai doc查询到,
1、不支持args属性。
2、tools不支持为none,需要为[]
3、stop不能为"",需要修改为 "stop": ["Observation:", "Observation:\n"],

调整后测试正常

@niykx niykx added the bug Something isn't working label Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants