Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different evaluation results #35

Open
zjintheroom opened this issue Dec 15, 2023 · 5 comments
Open

Different evaluation results #35

zjintheroom opened this issue Dec 15, 2023 · 5 comments
Labels
disc-law-eval-benchmark Related to DISC-Law-Eval-Benchmark evaluation framework

Comments

@zjintheroom
Copy link

zjintheroom commented Dec 15, 2023

你好,问您一下,为什么在评测分支下,用您这边提供的DISC-lawLLM测评 评测集的结果与仓库底下评测的结果略有差距呢?

是跟解码方式有关么,另外,咱们的外挂知识库,代码里有用到么。谢谢

@Charlie-XIAO Charlie-XIAO changed the title 你好,问您一下,为什么在评测分支下,用您这边提供的DISC-lawLLM测评 评测集的结果与仓库底下评测的结果略有差距呢? Different evaluation results Dec 15, 2023
@Charlie-XIAO
Copy link
Collaborator

The eval branch is not finalized yet, and our evaluation does not include the retrieval module.

@Charlie-XIAO Charlie-XIAO added the disc-law-eval-benchmark Related to DISC-Law-Eval-Benchmark evaluation framework label Dec 15, 2023
@zjintheroom
Copy link
Author

The eval branch is not finalized yet, and our evaluation does not include the retrieval module.

Thank you for your reply, and I'd like to ask you when our final review branch will be available

@Charlie-XIAO
Copy link
Collaborator

I'm sorry but this is not something I can decide. Perhaps @yueshengbin would be able to give an approximate release date.

@zjintheroom
Copy link
Author

I'm sorry but this is not something I can decide. Perhaps @yueshengbin would be able to give an approximate release date.

谢谢您的回复。如果您@yueshengbin 这边后续有完善评价分支的话,可以回复告诉我一下么。谢谢您

@zjintheroom
Copy link
Author

还有个问题,想问一下。咱们在评测的过程中,generate的decode配置参数是什么,可以提供么。是用的贪心策略么还是其他的。@yueshengbin @Charlie-XIAO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
disc-law-eval-benchmark Related to DISC-Law-Eval-Benchmark evaluation framework
Projects
None yet
Development

No branches or pull requests

2 participants