Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Qwen 模型无法正确适配 & 修复同步模式下 openai 流式输出问题 #287

Closed
wants to merge 3 commits into from

Conversation

wojiaoyishang
Copy link
Contributor

此 PR 解决 issue #286

Qwen 模型无法正确适配

通义千问大语言模型有兼容 OpenAI 数据输出格式的 API 的接口(除 通义千问Audio 外),原 Qwen 模型适配代码误将官方提供的 DashScope 格式混淆了。此处进行修改,将 gpt 与 qwen 的接口调用部分并列。

修复同步模式下 openai 流式输出问题

model_cli_demo.py 文件中展示了本地大语言模型流式输出的示例,但是对于 OpenAI 形式的远程调用的大语言模型无法正确处理同步模式下的 streaming chat (流式聊天),经过排查是在同步模式请求下的 requests.post 忘记添加 stream=True 导致的。

杂项

  • 注释了未知的解码内容的打印(这项打印应该属于调试用处,不应该在出现在发布的代码中)
  • 添加了 .idea 文件夹的忽略,PyCharm 的配置文件不会被提交

其余说明

对于项目 MindSearch 中的 Qwen 模型 API 地址需要修改,修改后的版本在测试时通过。

image

晚些提交 MindSearch 的 PR。

@wojiaoyishang
Copy link
Contributor Author

可用代码测试:

from lagent.agents import Agent
from lagent.schema import AgentMessage
from lagent.llms import GPTAPI

# qwen 智能体配置来自项目 MindSearch (修改后)
url = "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"
qwen = dict(
    type=GPTAPI,
    model_type="qwen-max-longcontext",
    key="sk-apikey",
    api_base=url,
    meta_template=[
        dict(role="system", api_role="system"),
        dict(role="user", api_role="user"),
        dict(role="assistant", api_role="assistant"),
        dict(role="environment", api_role="system"),
    ],
    top_p=0.8,
    temperature=0,
    max_new_tokens=4096,
    repetition_penalty=1.02,
    stop_words=["<|im_end|>"],
)

system_prompt = '你的回答只能从“典”、“孝”、“急”三个字中选一个。'
agent = Agent(qwen, system_prompt)

user_msg = AgentMessage(sender='user', content='今天天气情况')
bot_msg = agent(user_msg)
print(bot_msg)

@Harold-lkk
Copy link
Member

可以把一些注释的代码清理一下么,同时不要破坏BC原始的与qwen无关的代码

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants