Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pixtral throws an error when using system role in chat. #174

Open
chigkim opened this issue Jan 8, 2025 · 1 comment
Open

Pixtral throws an error when using system role in chat. #174

chigkim opened this issue Jan 8, 2025 · 1 comment

Comments

@chigkim
Copy link
Contributor

chigkim commented Jan 8, 2025

Llama vision works with --system message, but not Pixtral for some reason.

% mlx_vlm.generate --model mlx-community/pixtral-12b-8bit --image ../imgs/boat-256.png --system "You are a helpful assistant." --chat
Fetching 11 files: 100%|█████████████████████| 11/11 [00:00<00:00, 91907.06it/s]
User:Describe the image.
Traceback (most recent call last):
  File "/Users/cgk/Desktop/code/llm/mlx/.venv/bin/mlx_vlm.generate", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/cgk/Desktop/code/llm/mlx/mlx-vlm/mlx_vlm/generate.py", line 116, in main
    prompt = apply_chat_template(
             ^^^^^^^^^^^^^^^^^^^^
  File "/Users/cgk/Desktop/code/llm/mlx/mlx-vlm/mlx_vlm/prompt_utils.py", line 158, in apply_chat_template
    return processor.apply_chat_template(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cgk/Desktop/code/llm/mlx/.venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1131, in apply_chat_template
    return self.tokenizer.apply_chat_template(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cgk/Desktop/code/llm/mlx/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1683, in apply_chat_template
    rendered_chat = compiled_template.render(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cgk/Desktop/code/llm/mlx/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/Users/cgk/Desktop/code/llm/mlx/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 15, in top-level template code
TypeError: can only concatenate str (not "list") to str
@Blaizzy
Copy link
Owner

Blaizzy commented Jan 8, 2025

Perhaps we need to update the prompt utils.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants