Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eval 模块使用时 C-Eval 进行测试时,让人不确定日志输出 #905

Open
ShuMengZ opened this issue May 10, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@ShuMengZ
Copy link

Describe the bug
Eval 模块使用时 C-Eval 进行测试时,会大量重复出现
[INFO:modelscope] dataset_type: 1
[INFO:modelscope] dataset_type: 1
[INFO:modelscope] dataset_type: 1
image
这让我怀疑程序是否卡死,直到想起C-Eval拥有大量推理科目时才明白程序在做什么

希望能优化这个问题

@tastelikefeet tastelikefeet added the bug Something isn't working label May 14, 2024
@wangxingjun778
Copy link
Collaborator

正常情况只log一次dataset_type; 但是如果像mmlu、ceval这类有几十个subset的情况,每个subset都会打印一次。最新的modelscope master分支已删除该log;后续小版本1.14.1会自动更新。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants