-
Notifications
You must be signed in to change notification settings - Fork 322
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docker运行镜像cpu模式下,cpu的利用率最大只有1600%,如何提升? #281
Comments
环境变量num_threads可以设置线程数 |
还请指教,num_threads 这个参数具体在哪里设置,感谢! |
docker run -it --rm -v $PWD:/opt -p 8000:8000 -e MODEL=/opt/chatglm-ggml.bin \
+ -e num_threads=64 \
chatglm.cpp \
uvicorn chatglm_cpp.openai_api:app --host 0.0.0.0 --port 8000 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
我有一台虚机测试运行,32个cpus,我观察到在cpu模式运行下,通过docker stats查看cpu利用率最多只有1600%(如果满负荷,理论上应该达到3200%)。
调整虚机到64个cpus,观察cpu利用率依然只有1600%(如果满负荷,理论上应该达到6400%)。
说明文档中有写到,cpu模式测试性能时是跑的16个线程,是否跟这个有关系?能否有参数进行调整让cpu利用率提高一些?
The text was updated successfully, but these errors were encountered: