We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
根据文档,运行支持工具调用的openai api服务测试时,modelscope-agent向vllm发出请求: message: call openai api, model: Qwen2-7B-Instruct, messages: [{'role': 'system', 'content': '\n# 指令\n\nNone\n\n请注意:你具有图像和视频的展示能力,也具有运行代码的能力,不要在回复中说你做不到。\n'}, {'role': 'user', 'content': '北京有多少个叫张三的人?', 'tool_calls': None}, {'role': 'assistant', 'content': 'Action: locationNameFunction\nAction Input: {"location": "北京", "name": "张三"}\n', 'tool_calls': [{'type': 'function', 'function': {'name': 'locationNameFunction', 'arguments': '{"location": "北京", "name": "张三"}'}}]}, {'role': 'user', 'content': '(你可以使用工具:[locationNameFunction])Response[message=10个]'}], "stop": ["", "Observation:", "Observation:\n"], stream: True, args: {'tools': None, 'tool_choice': None}
vllm响应:openai.BadRequestError: Error code: 400 向vllm搭建的openai_api_server请求时:需要去掉stop和args才能正常响应。
从openai doc查询到, 1、不支持args属性。 2、tools不支持为none,需要为[] 3、stop不能为"",需要修改为 "stop": ["Observation:", "Observation:\n"],
调整后测试正常
The text was updated successfully, but these errors were encountered:
zzhangpurdue
suluyana
No branches or pull requests
根据文档,运行支持工具调用的openai api服务测试时,modelscope-agent向vllm发出请求:
message: call openai api, model: Qwen2-7B-Instruct, messages: [{'role': 'system', 'content': '\n# 指令\n\nNone\n\n请注意:你具有图像和视频的展示能力,也具有运行代码的能力,不要在回复中说你做不到。\n'}, {'role': 'user', 'content': '北京有多少个叫张三的人?', 'tool_calls': None}, {'role': 'assistant', 'content': 'Action: locationNameFunction\nAction Input: {"location": "北京", "name": "张三"}\n', 'tool_calls': [{'type': 'function', 'function': {'name': 'locationNameFunction', 'arguments': '{"location": "北京", "name": "张三"}'}}]}, {'role': 'user', 'content': '(你可以使用工具:[locationNameFunction])Response[message=10个]'}],
"stop": ["", "Observation:", "Observation:\n"], stream: True, args: {'tools': None, 'tool_choice': None}
vllm响应:openai.BadRequestError: Error code: 400
向vllm搭建的openai_api_server请求时:需要去掉stop和args才能正常响应。
从openai doc查询到,
1、不支持args属性。
2、tools不支持为none,需要为[]
3、stop不能为"",需要修改为 "stop": ["Observation:", "Observation:\n"],
调整后测试正常
The text was updated successfully, but these errors were encountered: