Llama2 Error while converting model weights to run with Hugging Face

[neerajg5]

Before submitting a bug, please make sure the issue hasn't been already addressed by searching through the FAQs and existing/past issues

Describe the bug

I'm following steps listed here https://ai.meta.com/blog/5-steps-to-getting-started-with-llama-2/ I've been able to complete couple of steps from this. However, while trying to follow "convert the model weights to run with Hugging Face" step, getting the following error.

Command:
pip install protobuf && python3 $TRANSFORM --input_dir ./llama-2-7b-chat --model_size 7B --output_dir ./llama-2-7b-chat-hf --llama_version 2

Output

Traceback (most recent call last):
  File "/home/neeraj/.local/lib/python3.10/site-packages/transformers/models/llama/convert_llama_weights_to_hf.py", line 339, in <module>
    main()
  File "/home/neeraj/.local/lib/python3.10/site-packages/transformers/models/llama/convert_llama_weights_to_hf.py", line 326, in main
    write_model(
  File "/home/neeraj/.local/lib/python3.10/site-packages/transformers/models/llama/convert_llama_weights_to_hf.py", line 94, in write_model
    params = read_json(os.path.join(input_base_path, "params.json"))
  File "/home/neeraj/.local/lib/python3.10/site-packages/transformers/models/llama/convert_llama_weights_to_hf.py", line 75, in read_json
    return json.load(f)
  File "/usr/lib/python3.10/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Runtime Environment

• Model: llama-2-7b-chat
• Using via huggingface?: no
• OS: Ubuntu 22.04.3 LTS
• GPU VRAM:
• Number of GPUs:
• GPU Make: Intel Iris Xe Graphics Family

[fbnav]

Could you try directly running the command without the " --llama_version 2" as that may not be a valid argument:

pip install protobuf && python $TRANSFORM --input_dir ./llama-2-7b-chat --model_size 7B --output_dir ./llama-2-7b-chat-hf

[neerajg5]

Thank you for sharing your input. However, there is no difference in the output. I had given this parameter after checking the source code in convert_llama_weights_to_hf.py. The reason of providing this parameter was: "I thought that by giving the model version the script may work and the JSON error may go away"

[fbnav]

I am not able to reproduce the issue on my side. Could you please provide the exact steps you followed and the entire stack trace? Thanks!

[neerajg5]

I had followed the exact steps listed here
https://ai.meta.com/blog/5-steps-to-getting-started-with-llama-2/

pip install transformers
pip install accelerate
git clone https://github.com/facebookresearch/llama
TRANSFORM=`python -c "import transformers;print('/'.join(transformers.__file__.split('/')[:-1])+'/models/llama/convert_llama_weights_to_hf.py')"`
pip install protobuf && python $TRANSFORM --input_dir ./llama-2-7b-chat --model_size 7B --output_dir ./llama-2-7b-chat-hf

Getting error in last step. traceback is shared above.
Hope, this helps. not sure what exactly you are looking for.

Thank you for prompt response.

[fbnav]

Thank you. Could you check if the params.json file is in the same directory as the model you are trying to convert, and that it's called exactly params.json? Also, could you double check if the json is a valid json file and not malformed?