Why is the generated picture deformed? Why can't I generate a face picture that is the same as the original picture?

[dizhenx]

Describe the bug

When I put in 10 face pictures and trained them for 1200 steps with dreambooth, most of the faces generated by reasoning are deformed. Why? Why can't I generate a face picture that is the same as the original picture?

Reproduction

conda activate diffusers
cd /mnt/data/creative/diffusers/examples/dreambooth
export MODEL_NAME="/mnt/data/creative/diffusers/examples/dreambooth/hub/models--runwayml--stable-diffusion-v1-5/snapshots/c9ab35ff5f2c362e9e22fbafe278077e196057f0"
export INSTANCE_DIR="/mnt/data/creative/diffusers/instance-images/liudehua"
export CLASS_DIR="/mnt/data/creative/diffusers/class-images/liudehua"
export OUTPUT_DIR="/mnt/data/creative/diffusers/class-images/model/liudehua"

accelerate launch --config_file /mnt/data/huggingface/accelerate/default_config1.yaml --mixed_precision="fp16" train_dreambooth.py
--pretrained_model_name_or_path=$MODEL_NAME
--instance_data_dir=$INSTANCE_DIR
--class_data_dir=$CLASS_DIR
--output_dir=$OUTPUT_DIR
--with_prior_preservation --prior_loss_weight=1.0
--instance_prompt="a photo of ldh man"
--class_prompt="a photo of man"
--resolution=512
--train_batch_size=1
--sample_batch_size=1
--gradient_accumulation_steps=1 --gradient_checkpointing
--learning_rate=1e-6
--lr_scheduler="constant"
--lr_warmup_steps=0
--num_class_images=200
--max_train_steps=1200

Logs

No response

System Info

• diffusers version: 0.15.0.dev0
• Platform: Linux-3.10.0-1160.92.1.el7.x86_64-x86_64-with-glibc2.17
• Python version: 3.9.2
• PyTorch version (GPU?): 2.0.1+cu117 (True)
• Huggingface_hub version: 0.16.2
• Transformers version: 4.27.1
• Accelerate version: 0.20.3
• xFormers version: not installed
• Using GPU in script?: yes
• Using distributed or parallel set-up in script?: yes