Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

当模型推理结果为空时会报错 #228

Closed
ChavesLiu opened this issue Dec 3, 2024 · 6 comments
Closed

当模型推理结果为空时会报错 #228

ChavesLiu opened this issue Dec 3, 2024 · 6 comments
Assignees
@Yunnglin
Labels
native

Comments

@ChavesLiu
Copy link

ChavesLiu commented Dec 3, 2024 •
edited
Loading

image

evalscope/metrics/rouge_metric.py:56

score = zh_scorer.get_scores(p, r)[0]

可以增加忽略输出为空的

score = zh_scorer.get_scores(p, r, ignore_empty=True)[0]

image
但外部调用的调用操作没看懂，为什么这里用了for循环，输入的应该是1个样本，如果是一个样本，调用get_scores,只有一个还为空，则内部又报错~
@ChavesLiu ChavesLiu changed the title 当推理结果为空时，会报错，可修复 当模型推理结果为空时会报错 Dec 3, 2024
@Yunnglin
Copy link
Collaborator

Yunnglin commented Dec 3, 2024

模型推理结果什么情况下会为空呢

@Yunnglin Yunnglin added the native label Dec 3, 2024
@ChavesLiu
Copy link
Author

模型可能透出的是一个special token，非中文。这里会过滤掉，所以调用计算分逻辑报错。

目前最粗暴的解决办法就是调用模型返回特殊时，给赋一个非常见的中文字符。

@Yunnglin
Copy link
Collaborator

Yunnglin commented Dec 4, 2024

请问执行的代码是什么，我们复现一下

@ChavesLiu
Copy link
Author

这个是模型的输出，复现的话可以把predict复制为空字符串或者空格符号就可以了

@Yunnglin Yunnglin mentioned this issue Dec 10, 2024
Closed
@Yunnglin Yunnglin self-assigned this Dec 16, 2024
@Yunnglin
Copy link
Collaborator

Yunnglin commented Jan 3, 2025

EvalScope v0.9.0 已发布，可以使用新版本，该问题已修复

@Yunnglin
Copy link
Collaborator

由于长时间未收到活动，我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Yunnglin Yunnglin

Labels
native
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ChavesLiu @Yunnglin