Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After adding tokens, the model doubles in size. #1095

Open
supech opened this issue Apr 10, 2024 · 0 comments
Open

After adding tokens, the model doubles in size. #1095

supech opened this issue Apr 10, 2024 · 0 comments

Comments

@supech
Copy link

supech commented Apr 10, 2024

l use the code below to add about 100 tokens to model and tokenizer,and the model doubles in size.

from transformers import  AutoTokenizer,AutoModel
tokenizer = AutoTokenizer.from_pretrained("llama-7b-model")
model = AutoModel.from_pretrained("llama-7b-model")
characters = [
  #100tokens
]
num_added_toks = tokenizer.add_tokens(characters)
tokenizer.save_pretrained("new-llama-7b-model")
model.save_pretrained("new-llama-7b-model")

In contrast, other language models such as BERT, BART, GPT-2, etc., that I have used previously do not exhibit such a significant increase in size.
I have also tried fine-tuning LLAMA1.3b, and after adding tokens, although the size nearly doubles, the entire fine-tuned model ends up slightly smaller than the original 1.3b model.
I am puzzled by this situation and unsure if it is normal. Do you have any insights into the reasons behind this?
Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant