-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
如何设置Adan学习率 #48
Comments
保守一点可以设置为adamW的两倍,也可以更大,可以适当调整beta3和beta2。 |
谢谢,我先试试2倍 |
如果一开始下降的很慢,可以试着调整beta2,调到大一点的值,例如0.95,或者0.98。 beta3也可以调整,0.95可以尝试一下。最后可以试着把no_prox设置为True试一试。基本上就可以找到一个稳定好用AdamW |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
您好请问您是否有研究过将Adan用于Diffusion模型训练,其学习率应该如何设置,可否与使用AdamW的学习率一样?
The text was updated successfully, but these errors were encountered: