Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The training details about Unet parameters frozen #129

Open
hda-xian opened this issue Jun 9, 2024 · 2 comments
Open

The training details about Unet parameters frozen #129

hda-xian opened this issue Jun 9, 2024 · 2 comments

Comments

@hda-xian
Copy link

hda-xian commented Jun 9, 2024

Why are the UNet parameters frozen during training for SD1v5, but not for SDXL? the haggingface training sdxl script sets " Unet.train() "

***** huggingface train sdxl *********
vae.requires_grad_(False)
text_encoder_one.requires_grad_(False)
text_encoder_two.requires_grad_(False)
t2iadapter.train()
unet.train()

*******Tencent ARC train sd1v5 **************
model.cuda()
model.eval() # model is contain all models vae ,cliptext
return model

*******Tencent ARC train sdxl **************
vae.requires_grad_(False)
text_encoder_one.requires_grad_(False)
text_encoder_two.requires_grad_(False) -> the Unet does not set no grad means Unet need grad

@bonlime
Copy link

bonlime commented Jul 2, 2024

because they have params_to_optimize = adapter.parameters(), and only optimise the adapter's params. But disabling unet grads reduces the memory consumption, they just forgot it

@hda-xian
Copy link
Author

hda-xian commented Jul 2, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants