Problems with the Weight of resolution 256 stage2

[LOSKIw]

The released stage 2 weight of resolution 256 seems to be incomplete, the error log is shown below.

File "/home/user/data/PT/PCDMs/stage2_batchtest_inpaint_model.py", line 126, in inference pose_proj.load_state_dict(pose_proj_dict)

RuntimeError: Error(s) in loading state_dict for ControlNetConditioningEmbedding: Missing key(s) in state_dict: "conv_in.weight", "conv_in.bias", "blocks.0.weight", "blocks.0.bias", "blocks.1.weight", "blocks.1.bias", "blocks.2.weight", "blocks.2.bias", "blocks.3.weight", "blocks.3.bias", "blocks.4.weight", "blocks.4.bias", "blocks.5.weight", "blocks.5.bias", "conv_out.weight", "conv_out.bias".

I also find the stage 2 .pt file is quite different between resolution 256 and 512.
Maybe its actually the stage 1 weight of 256?
Could you please help check this file or I wrongly process this step? thank you very much.

[muzishen]

I did not make this mistake.

[muzishen]

I did not make this mistake.

[LOSKIw]

I did not make this mistake.

Besides the error log, apparently the two ckpts of stage 2 have different files size under this link
stage2_checkpoints/256/s2_256.pt -> 1.91GB
stage2_checkpoints/512/s2_512.pt -> 1.62GB

While the size of s2_256 is same with the stage 1 file
stage1_checkpoints/256/s1_256.pt -> 1.91GB

And I have check the inner content, s2_256.pt contains numbers of transformer blocks, which is the stage 1 structure in the paper.
I doubt this ckpt file is not correct.

s2_256 state_dict['module']

positional_embedding
prd_embedding
pose_encoder.net.0.weight
pose_encoder.net.0.bias
pose_encoder.net.3.weight
pose_encoder.net.3.bias
pose_encoder.net.4.weight
pose_encoder.net.4.bias
pose_encoder.net.6.weight
pose_encoder.net.6.bias
pose_encoder1.net.0.weight
pose_encoder1.net.0.bias
pose_encoder1.net.3.weight
pose_encoder1.net.3.bias
pose_encoder1.net.4.weight
pose_encoder1.net.4.bias
pose_encoder1.net.6.weight
pose_encoder1.net.6.bias
time_embedding.linear_1.weight
time_embedding.linear_1.bias
time_embedding.linear_2.weight
time_embedding.linear_2.bias
proj_in.weight
proj_in.bias
embedding_proj.weight
embedding_proj.bias
encoder_hidden_states_proj.weight
encoder_hidden_states_proj.bias
encoder_hidden_states_proj1.weight
encoder_hidden_states_proj1.bias
transformer_blocks.0.norm1.weight
transformer_blocks.0.norm1.bias
transformer_blocks.0.attn1.to_q.weight
transformer_blocks.0.attn1.to_q.bias
transformer_blocks.0.attn1.to_k.weight
transformer_blocks.0.attn1.to_k.bias
transformer_blocks.0.attn1.to_v.weight
transformer_blocks.0.attn1.to_v.bias
transformer_blocks.0.attn1.to_out.0.weight
transformer_blocks.0.attn1.to_out.0.bias
transformer_blocks.0.norm3.weight
transformer_blocks.0.norm3.bias
transformer_blocks.0.ff.net.0.proj.weight
transformer_blocks.0.ff.net.0.proj.bias
transformer_blocks.0.ff.net.2.weight
transformer_blocks.0.ff.net.2.bias
transformer_blocks.1.norm1.weight
transformer_blocks.1.norm1.bias
transformer_blocks.1.attn1.to_q.weight
transformer_blocks.1.attn1.to_q.bias
transformer_blocks.1.attn1.to_k.weight
transformer_blocks.1.attn1.to_k.bias
transformer_blocks.1.attn1.to_v.weight
transformer_blocks.1.attn1.to_v.bias
transformer_blocks.1.attn1.to_out.0.weight
transformer_blocks.1.attn1.to_out.0.bias
transformer_blocks.1.norm3.weight
transformer_blocks.1.norm3.bias
transformer_blocks.1.ff.net.0.proj.weight
transformer_blocks.1.ff.net.0.proj.bias
transformer_blocks.1.ff.net.2.weight
transformer_blocks.1.ff.net.2.bias
transformer_blocks.2.norm1.weight
transformer_blocks.2.norm1.bias
transformer_blocks.2.attn1.to_q.weight
transformer_blocks.2.attn1.to_q.bias
transformer_blocks.2.attn1.to_k.weight
transformer_blocks.2.attn1.to_k.bias
transformer_blocks.2.attn1.to_v.weight
transformer_blocks.2.attn1.to_v.bias
transformer_blocks.2.attn1.to_out.0.weight
transformer_blocks.2.attn1.to_out.0.bias
transformer_blocks.2.norm3.weight
transformer_blocks.2.norm3.bias
transformer_blocks.2.ff.net.0.proj.weight
transformer_blocks.2.ff.net.0.proj.bias
transformer_blocks.2.ff.net.2.weight
transformer_blocks.2.ff.net.2.bias
transformer_blocks.3.norm1.weight
transformer_blocks.3.norm1.bias
transformer_blocks.3.attn1.to_q.weight
transformer_blocks.3.attn1.to_q.bias
transformer_blocks.3.attn1.to_k.weight
transformer_blocks.3.attn1.to_k.bias
transformer_blocks.3.attn1.to_v.weight
transformer_blocks.3.attn1.to_v.bias
transformer_blocks.3.attn1.to_out.0.weight
transformer_blocks.3.attn1.to_out.0.bias
transformer_blocks.3.norm3.weight
transformer_blocks.3.norm3.bias
transformer_blocks.3.ff.net.0.proj.weight
transformer_blocks.3.ff.net.0.proj.bias
transformer_blocks.3.ff.net.2.weight
transformer_blocks.3.ff.net.2.bias
transformer_blocks.4.norm1.weight
transformer_blocks.4.norm1.bias
transformer_blocks.4.attn1.to_q.weight
transformer_blocks.4.attn1.to_q.bias
transformer_blocks.4.attn1.to_k.weight
transformer_blocks.4.attn1.to_k.bias
transformer_blocks.4.attn1.to_v.weight
transformer_blocks.4.attn1.to_v.bias
transformer_blocks.4.attn1.to_out.0.weight
transformer_blocks.4.attn1.to_out.0.bias
transformer_blocks.4.norm3.weight
transformer_blocks.4.norm3.bias
transformer_blocks.4.ff.net.0.proj.weight
transformer_blocks.4.ff.net.0.proj.bias
transformer_blocks.4.ff.net.2.weight
transformer_blocks.4.ff.net.2.bias
transformer_blocks.5.norm1.weight
transformer_blocks.5.norm1.bias
transformer_blocks.5.attn1.to_q.weight
transformer_blocks.5.attn1.to_q.bias
transformer_blocks.5.attn1.to_k.weight
transformer_blocks.5.attn1.to_k.bias
transformer_blocks.5.attn1.to_v.weight
transformer_blocks.5.attn1.to_v.bias
transformer_blocks.5.attn1.to_out.0.weight
transformer_blocks.5.attn1.to_out.0.bias
transformer_blocks.5.norm3.weight
transformer_blocks.5.norm3.bias
transformer_blocks.5.ff.net.0.proj.weight
transformer_blocks.5.ff.net.0.proj.bias
transformer_blocks.5.ff.net.2.weight
transformer_blocks.5.ff.net.2.bias
transformer_blocks.6.norm1.weight
transformer_blocks.6.norm1.bias
transformer_blocks.6.attn1.to_q.weight
transformer_blocks.6.attn1.to_q.bias
transformer_blocks.6.attn1.to_k.weight
transformer_blocks.6.attn1.to_k.bias
transformer_blocks.6.attn1.to_v.weight
transformer_blocks.6.attn1.to_v.bias
transformer_blocks.6.attn1.to_out.0.weight
transformer_blocks.6.attn1.to_out.0.bias
transformer_blocks.6.norm3.weight
transformer_blocks.6.norm3.bias
transformer_blocks.6.ff.net.0.proj.weight
transformer_blocks.6.ff.net.0.proj.bias
transformer_blocks.6.ff.net.2.weight
transformer_blocks.6.ff.net.2.bias
transformer_blocks.7.norm1.weight
transformer_blocks.7.norm1.bias
transformer_blocks.7.attn1.to_q.weight
transformer_blocks.7.attn1.to_q.bias
transformer_blocks.7.attn1.to_k.weight
transformer_blocks.7.attn1.to_k.bias
transformer_blocks.7.attn1.to_v.weight
transformer_blocks.7.attn1.to_v.bias
transformer_blocks.7.attn1.to_out.0.weight
transformer_blocks.7.attn1.to_out.0.bias
transformer_blocks.7.norm3.weight
transformer_blocks.7.norm3.bias
transformer_blocks.7.ff.net.0.proj.weight
transformer_blocks.7.ff.net.0.proj.bias
transformer_blocks.7.ff.net.2.weight
transformer_blocks.7.ff.net.2.bias
transformer_blocks.8.norm1.weight
transformer_blocks.8.norm1.bias
transformer_blocks.8.attn1.to_q.weight
transformer_blocks.8.attn1.to_q.bias
transformer_blocks.8.attn1.to_k.weight
transformer_blocks.8.attn1.to_k.bias
transformer_blocks.8.attn1.to_v.weight
transformer_blocks.8.attn1.to_v.bias
transformer_blocks.8.attn1.to_out.0.weight
transformer_blocks.8.attn1.to_out.0.bias
transformer_blocks.8.norm3.weight
transformer_blocks.8.norm3.bias
transformer_blocks.8.ff.net.0.proj.weight
transformer_blocks.8.ff.net.0.proj.bias
transformer_blocks.8.ff.net.2.weight
transformer_blocks.8.ff.net.2.bias
transformer_blocks.9.norm1.weight
transformer_blocks.9.norm1.bias
transformer_blocks.9.attn1.to_q.weight
transformer_blocks.9.attn1.to_q.bias
transformer_blocks.9.attn1.to_k.weight
transformer_blocks.9.attn1.to_k.bias
transformer_blocks.9.attn1.to_v.weight
transformer_blocks.9.attn1.to_v.bias
transformer_blocks.9.attn1.to_out.0.weight
transformer_blocks.9.attn1.to_out.0.bias
transformer_blocks.9.norm3.weight
transformer_blocks.9.norm3.bias
transformer_blocks.9.ff.net.0.proj.weight
transformer_blocks.9.ff.net.0.proj.bias
transformer_blocks.9.ff.net.2.weight
transformer_blocks.9.ff.net.2.bias
transformer_blocks.10.norm1.weight
transformer_blocks.10.norm1.bias
transformer_blocks.10.attn1.to_q.weight
transformer_blocks.10.attn1.to_q.bias
transformer_blocks.10.attn1.to_k.weight
transformer_blocks.10.attn1.to_k.bias
transformer_blocks.10.attn1.to_v.weight
transformer_blocks.10.attn1.to_v.bias
transformer_blocks.10.attn1.to_out.0.weight
transformer_blocks.10.attn1.to_out.0.bias
transformer_blocks.10.norm3.weight
transformer_blocks.10.norm3.bias
transformer_blocks.10.ff.net.0.proj.weight
transformer_blocks.10.ff.net.0.proj.bias
transformer_blocks.10.ff.net.2.weight
transformer_blocks.10.ff.net.2.bias
transformer_blocks.11.norm1.weight
transformer_blocks.11.norm1.bias
transformer_blocks.11.attn1.to_q.weight
transformer_blocks.11.attn1.to_q.bias
transformer_blocks.11.attn1.to_k.weight
transformer_blocks.11.attn1.to_k.bias
transformer_blocks.11.attn1.to_v.weight
transformer_blocks.11.attn1.to_v.bias
transformer_blocks.11.attn1.to_out.0.weight
transformer_blocks.11.attn1.to_out.0.bias
transformer_blocks.11.norm3.weight
transformer_blocks.11.norm3.bias
transformer_blocks.11.ff.net.0.proj.weight
transformer_blocks.11.ff.net.0.proj.bias
transformer_blocks.11.ff.net.2.weight
transformer_blocks.11.ff.net.2.bias
transformer_blocks.12.norm1.weight
transformer_blocks.12.norm1.bias
transformer_blocks.12.attn1.to_q.weight
transformer_blocks.12.attn1.to_q.bias
transformer_blocks.12.attn1.to_k.weight
transformer_blocks.12.attn1.to_k.bias
transformer_blocks.12.attn1.to_v.weight
transformer_blocks.12.attn1.to_v.bias
transformer_blocks.12.attn1.to_out.0.weight
transformer_blocks.12.attn1.to_out.0.bias
transformer_blocks.12.norm3.weight
transformer_blocks.12.norm3.bias
transformer_blocks.12.ff.net.0.proj.weight
transformer_blocks.12.ff.net.0.proj.bias
transformer_blocks.12.ff.net.2.weight
transformer_blocks.12.ff.net.2.bias
transformer_blocks.13.norm1.weight
transformer_blocks.13.norm1.bias
transformer_blocks.13.attn1.to_q.weight
transformer_blocks.13.attn1.to_q.bias
transformer_blocks.13.attn1.to_k.weight
transformer_blocks.13.attn1.to_k.bias
transformer_blocks.13.attn1.to_v.weight
transformer_blocks.13.attn1.to_v.bias
transformer_blocks.13.attn1.to_out.0.weight
transformer_blocks.13.attn1.to_out.0.bias
transformer_blocks.13.norm3.weight
transformer_blocks.13.norm3.bias
transformer_blocks.13.ff.net.0.proj.weight
transformer_blocks.13.ff.net.0.proj.bias
transformer_blocks.13.ff.net.2.weight
transformer_blocks.13.ff.net.2.bias
transformer_blocks.14.norm1.weight
transformer_blocks.14.norm1.bias
transformer_blocks.14.attn1.to_q.weight
transformer_blocks.14.attn1.to_q.bias
transformer_blocks.14.attn1.to_k.weight
transformer_blocks.14.attn1.to_k.bias
transformer_blocks.14.attn1.to_v.weight
transformer_blocks.14.attn1.to_v.bias
transformer_blocks.14.attn1.to_out.0.weight
transformer_blocks.14.attn1.to_out.0.bias
transformer_blocks.14.norm3.weight
transformer_blocks.14.norm3.bias
transformer_blocks.14.ff.net.0.proj.weight
transformer_blocks.14.ff.net.0.proj.bias
transformer_blocks.14.ff.net.2.weight
transformer_blocks.14.ff.net.2.bias
transformer_blocks.15.norm1.weight
transformer_blocks.15.norm1.bias
transformer_blocks.15.attn1.to_q.weight
transformer_blocks.15.attn1.to_q.bias
transformer_blocks.15.attn1.to_k.weight
transformer_blocks.15.attn1.to_k.bias
transformer_blocks.15.attn1.to_v.weight
transformer_blocks.15.attn1.to_v.bias
transformer_blocks.15.attn1.to_out.0.weight
transformer_blocks.15.attn1.to_out.0.bias
transformer_blocks.15.norm3.weight
transformer_blocks.15.norm3.bias
transformer_blocks.15.ff.net.0.proj.weight
transformer_blocks.15.ff.net.0.proj.bias
transformer_blocks.15.ff.net.2.weight
transformer_blocks.15.ff.net.2.bias
transformer_blocks.16.norm1.weight
transformer_blocks.16.norm1.bias
transformer_blocks.16.attn1.to_q.weight
transformer_blocks.16.attn1.to_q.bias
transformer_blocks.16.attn1.to_k.weight
transformer_blocks.16.attn1.to_k.bias
transformer_blocks.16.attn1.to_v.weight
transformer_blocks.16.attn1.to_v.bias
transformer_blocks.16.attn1.to_out.0.weight
transformer_blocks.16.attn1.to_out.0.bias
transformer_blocks.16.norm3.weight
transformer_blocks.16.norm3.bias
transformer_blocks.16.ff.net.0.proj.weight
transformer_blocks.16.ff.net.0.proj.bias
transformer_blocks.16.ff.net.2.weight
transformer_blocks.16.ff.net.2.bias
transformer_blocks.17.norm1.weight
transformer_blocks.17.norm1.bias
transformer_blocks.17.attn1.to_q.weight
transformer_blocks.17.attn1.to_q.bias
transformer_blocks.17.attn1.to_k.weight
transformer_blocks.17.attn1.to_k.bias
transformer_blocks.17.attn1.to_v.weight
transformer_blocks.17.attn1.to_v.bias
transformer_blocks.17.attn1.to_out.0.weight
transformer_blocks.17.attn1.to_out.0.bias
transformer_blocks.17.norm3.weight
transformer_blocks.17.norm3.bias
transformer_blocks.17.ff.net.0.proj.weight
transformer_blocks.17.ff.net.0.proj.bias
transformer_blocks.17.ff.net.2.weight
transformer_blocks.17.ff.net.2.bias
transformer_blocks.18.norm1.weight
transformer_blocks.18.norm1.bias
transformer_blocks.18.attn1.to_q.weight
transformer_blocks.18.attn1.to_q.bias
transformer_blocks.18.attn1.to_k.weight
transformer_blocks.18.attn1.to_k.bias
transformer_blocks.18.attn1.to_v.weight
transformer_blocks.18.attn1.to_v.bias
transformer_blocks.18.attn1.to_out.0.weight
transformer_blocks.18.attn1.to_out.0.bias
transformer_blocks.18.norm3.weight
transformer_blocks.18.norm3.bias
transformer_blocks.18.ff.net.0.proj.weight
transformer_blocks.18.ff.net.0.proj.bias
transformer_blocks.18.ff.net.2.weight
transformer_blocks.18.ff.net.2.bias
transformer_blocks.19.norm1.weight
transformer_blocks.19.norm1.bias
transformer_blocks.19.attn1.to_q.weight
transformer_blocks.19.attn1.to_q.bias
transformer_blocks.19.attn1.to_k.weight
transformer_blocks.19.attn1.to_k.bias
transformer_blocks.19.attn1.to_v.weight
transformer_blocks.19.attn1.to_v.bias
transformer_blocks.19.attn1.to_out.0.weight
transformer_blocks.19.attn1.to_out.0.bias
transformer_blocks.19.norm3.weight
transformer_blocks.19.norm3.bias
transformer_blocks.19.ff.net.0.proj.weight
transformer_blocks.19.ff.net.0.proj.bias
transformer_blocks.19.ff.net.2.weight
transformer_blocks.19.ff.net.2.bias
norm_out.weight
norm_out.bias
proj_to_clip_embeddings.weight
proj_to_clip_embeddings.bias

[muzishen]

Could you try to see if the weight of 512 has this problem?

[LOSKIw]

Could you try to see if the weight of 512 has this problem?

The weight of 512 is fine. All other stages could be complete using the given ckpts except stage 2 of 256.

[muzishen]

Maybe it was the wrong version when I uploaded it, and now it seems there is no way to fix it, because I have left my job. You can retrain according to the training script. I'm sorry.