-
Notifications
You must be signed in to change notification settings - Fork 26
Open
Description
Hi Lolcats Authors,
I'm trying to run the code, especially the LM-Eval benchmark, but I'm getting an error. I attached the command and the error for reproducing the issue.
Could you help me with the solution? appreciate your time.
python3 lm_eval_harness/eval_lm_harness.py --model_type lolcats_ckpt --attn_mlp_checkpoint_path './checkpoints/distill_llama3_1_8b_lk_smd_wtk64_fd64_w01/dl-d=distill_alpaca_clean_xent0_mse1000_lr1e-2-m=distill_llama3_1_8b_lk_smd_wtk64_fd64_w01-f=finetune_lora_qkvo_alpaca_clean-s=0-se=0-re=420-lzi=1_distill.pt' --finetune_checkpoint_path './checkpoints/distill_llama3_1_8b_lk_smd_wtk64_fd64_w01/dl-d=distill_alpaca_clean_xent0_mse1000_lr1e-2-m=distill_llama3_1_8b_lk_smd_wtk64_fd64_w01-f=finetune_lora_qkvo_alpaca_clean-s=0-se=0-re=420-lzi=1-bs=1-gas=8-nte=2-ms=-1-se=0-re=420_ft.pt' --task piqa --num_shots 0 --no_cache --verbos
Traceback (most recent call last):
File "/home/vimal/phd_research/lolcats/lm_eval_harness/eval_lm_harness.py", line 256, in <module>
main()
File "/home/vimal/phd_research/lolcats/lm_eval_harness/eval_lm_harness.py", line 164, in main
model, model_config, tokenizer = load_model_from_checkpoint(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vimal/phd_research/lolcats/src/model/load_model_for_eval.py", line 197, in load_model_from_checkpoint
lm = get_lm_eval_model(model_loader.loading_kwargs, path_to_lm_eval_harness,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vimal/phd_research/lolcats/src/model/load_model_for_eval.py", line 101, in get_lm_eval_model
lm = ModelClass.create_from_arg_string('', lm_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vimal/phd_research/lm-evaluation-harness/lm_eval/base.py", line 115, in create_from_arg_string
return cls(**args, **args2)
^^^^^^^^^^^^^^^^^^^^
TypeError: HuggingFaceAutoLM.__init__() got an unexpected keyword argument 'pretrained_model_name_or_path'
Thanks,
Vimal William
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels