New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

predict_sft.sh 推理速度好慢 #236

Open

wangyongshuai88 opened this issue Feb 14, 2024 · 0 comments

wangyongshuai88 commented Feb 14, 2024

相比train_sft.sh 的Qlora微调速度predict_sft.sh 的推理速度好慢，我设置的训练集的大小是验证集的5倍，但是验证集推理的需求时间是训练的数十倍。。。
请问大佬有没有办法加快推理速度？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment