Getting error while loading meta-llama/Llama-2-7b-chat-hf

[Shrijeeth]

Hello All,

I have fine tuned meta-llama/Llama-2-7b-chat-hf model using Ludwig and saved it. When I try to load the model again, I am getting following error.
Specified base model `meta-llama/Llama-2-7b-chat-hf` is not a valid pretrained CausalLM listed on huggingface. Please see: https://huggingface.co/models?pipeline_tag=text-generation&sort=downloads. I am also having access for Llama 2 model. Can someone please help me.

The code used for loading model:

def load_model():
    model = LudwigModel.load('<Pretrained Model Path>')
    return model

[arnavgarg1]

Hi @Shrijeeth, are you able to check if your HUGGING_FACE_HUB_TOKEN environment variable is set?

This error usually happens when the token isn't set in the environment, which is required to get access to the Llama-2 suite of models from Meta.

In case it isn't set, you can either do

export HUGGING_FACE_HUB_TOKEN=<token>

in your shell/environment, or do something like:

import os
os.environ['HUGGING_FACE_HUB_TOKEN'] = "token"

And then try to load your model.

Let me know if this fixes the issue!

[Shrijeeth]

Yeah it worked, previously had the wrong token key HUGGING_FACE_TOKEN. Once I changed it to HUGGING_FACE_HUB_TOKEN it works fine. Thank you @arnavgarg1

[arnavgarg1]

Glad to hear!