You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I attempted to run benchmarks for the llama-3-8b-instruct and llama-3.1-8b-instruct models using both CPU and GPU, but the process failed. (I successfully tested the llama2-7b-chatbot model)
I followed the instructions in openvino_notebooks/llm-chatbot.ipynb to download the models and ensured that all necessary files (including the required tokenizer.model) were included. I am using the latest version of OpenVINO (2024.3.0) and have also upgraded the transformers library.
The command I executed is:
python benchmark.py -m {path}/openvino_notebooks/notebooks/llm-chatbot/llama-3-8b-instruct/INT4_compressed_weights -n 2 -d CPU -p "What is large language model (LLM)?"
And I received the following error output:
The text was updated successfully, but these errors were encountered:
I attempted to run benchmarks for the llama-3-8b-instruct and llama-3.1-8b-instruct models using both CPU and GPU, but the process failed. (I successfully tested the llama2-7b-chatbot model)
I followed the instructions in openvino_notebooks/llm-chatbot.ipynb to download the models and ensured that all necessary files (including the required tokenizer.model) were included. I am using the latest version of OpenVINO (2024.3.0) and have also upgraded the transformers library.
The command I executed is:
python benchmark.py -m {path}/openvino_notebooks/notebooks/llm-chatbot/llama-3-8b-instruct/INT4_compressed_weights -n 2 -d CPU -p "What is large language model (LLM)?"
And I received the following error output:
The text was updated successfully, but these errors were encountered: