测评llama2-7b bbh出错

[Haruka1307]

报错：
Traceback (most recent call last):
File "/home/anaconda3/envs/evalscope/bin/evalscope", line 33, in
sys.exit(load_entry_point('evalscope', 'console_scripts', 'evalscope')())
File "/home/evalscope/evalscope/cli/cli.py", line 23, in run_cmd
cmd.execute()
File "/home//evalscope/evalscope/cli/start_eval.py", line 31, in execute
run_task(self.args)
File "/home//evalscope/evalscope/run.py", line 36, in run_task
return run_single_task(task_cfg, run_time)
File "/home//evalscope/evalscope/run.py", line 51, in run_single_task
return evaluate_model(task_cfg, outputs)
File "/home//evalscope/evalscope/run.py", line 109, in evaluate_model
res_dict = evaluator.eval(infer_cfg=task_cfg.generation_config, debug=task_cfg.debug, limit=task_cfg.limit)
File "/home//evalscope/evalscope/evaluator/evaluator.py", line 387, in eval
answers_list: list = self.get_answers(
File "/home///evalscope/evalscope/evaluator/evaluator.py", line 188, in get_answers
answer_d: dict = self._pred_answer(
File "/home//evalscope/evalscope/evaluator/evaluator.py", line 95, in _pred_answer
ans: dict = self.model_adapter.predict(inputs=input_d, infer_cfg=infer_cfg)
File "/home/anaconda3/envs/evalscope/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/home//evalscope/evalscope/models/model_adapter.py", line 450, in predict
response = self._model_generate(query, infer_cfg)
File "/home//evalscope/evalscope/models/model_adapter.py", line 411, in _model_generate
formatted_prompt = self.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
File "/home//anaconda3/envs/evalscope/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1617, in apply_chat_template
chat_template = self.get_chat_template(chat_template, tools)
File "/home/anaconda3/envs/evalscope/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1785, in get_chat_template
raise ValueError(
ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating

[Yunnglin]

请使用 Llama-2-7b-chat 模型来评测；或者自定义提供template，例如

--chat-template "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% set system_message = false %}{% endif %}{% for message in loop_messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if loop.index0 == 0 and system_message != false %}{% set content = '<<SYS>>\\n' + system_message + '\\n<</SYS>>\\n\\n' + message['content'] %}{% else %}{% set content = message['content'] %}{% endif %}{% if message['role'] == 'user' %}{{ bos_token + '[INST] ' + content.strip() + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' '  + content.strip() + ' ' + eos_token }}{% endif %}{% endfor %}"

[Yunnglin]

我们再修改一下这个部分，对于没有chat template的base model去掉apply_chat_template

[Yunnglin]

EvalScope v0.9.0 已发布，可以看看这个问题解决了吗

[Haruka1307]

bbh是否支持使用vllm推理加速呢，现在测评的时间超过2h

[Yunnglin]

可以使用vllm拉起服务来评测，参考：https://evalscope.readthedocs.io/zh-cn/latest/get_started/basic_usage.html#api