Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ollama Model is configured wrong: Double BOS #50

Open
Snowman-25 opened this issue Sep 19, 2024 · 1 comment
Open

ollama Model is configured wrong: Double BOS #50

Snowman-25 opened this issue Sep 19, 2024 · 1 comment

Comments

@Snowman-25
Copy link

Snowman-25 commented Sep 19, 2024

Whenever I run deepseek-coder-v2:latest through ollama, the following Error pops up in the log for each prompt:

llm_tokenizer_bpe::check_double_bos_eos: Added a BOS token to the prompt as specified by the model but the prompt also starts with a BOS token. So now the final prompt starts with 2 BOS tokens. Are you sure this is what you want?

I believe that this is due to tokenizer.ggml.add_bos_token being set to true but the template also already having a <|begin▁of▁sentence|> token.

I'm not sure how this affects hallucinations of the model

@Arvin2focus
Copy link

same issue for me, any update?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@Snowman-25 @Arvin2focus and others