Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meta-Llama-3.1-8B模型mmlu和ceval测试集评分为0 #271

Open
yosure opened this issue Jan 3, 2025 · 0 comments
Open

Meta-Llama-3.1-8B模型mmlu和ceval测试集评分为0 #271

yosure opened this issue Jan 3, 2025 · 0 comments
Assignees
Labels
bug Something isn't working opencompass

Comments

@yosure
Copy link

yosure commented Jan 3, 2025

问题描述 / Issue Description

请简要描述您遇到的问题。 / Please briefly describe the issue you encountered.

使用的工具 / Tools Used

1.Opencompass backend
2.VLLM推理服务启动Meta-Llama-3.1-8B模型
3.使用贵司官方python脚本进行评分

  • 操作系统 / Operating System:
    Ubuntu

  • Python版本 / Python Version:3.10

结果:
评分基本为0,使用数据集问题单独进行request请求返回结果和评测一致基本为空,是测试集不适用该模型吗
1735896087267

@Yunnglin Yunnglin added bug Something isn't working opencompass labels Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working opencompass
Projects
None yet
Development

No branches or pull requests

3 participants