Skip to content

TMElyralab/Comfyui-MusePose

Folders and files

NameName
Last commit message
Last commit date

Latest commit

f1f15c5 · Jun 8, 2024

History

21 Commits
Jun 3, 2024
May 31, 2024
May 31, 2024
Jun 1, 2024
Jun 3, 2024
May 31, 2024
May 31, 2024
Jun 8, 2024
May 31, 2024
May 31, 2024
May 31, 2024
May 31, 2024
Jun 6, 2024
Jun 3, 2024
Jun 5, 2024

Repository files navigation

MusePose

MusePose is an image-to-video generation framework for virtual human under control signal such as pose.

MusePose is the last building block of the Muse opensource serie. Together with MuseV and MuseTalk, we hope the community can join us and march towards the vision where a virtual human can be generated end2end with native ability of full body movement and interaction. Please stay tuned for our next milestone!

Comfyui-MusePose

If you're running on Linux, or non-admin account on windows you'll want to ensure /ComfyUI/custom_nodes and Comfyui-MusePose has write permissions.

Followed ComfyUI's manual installation steps and do the following:

  • Navigate to your /ComfyUI/custom_nodes/ folder
  • Run git clone https://github.com/TMElyralab/Comfyui-MusePose.git
  • Navigate to your /ComfyUI/custom_nodes/Comfyui-MusePose folder and run
 pip install -r requirements.txt

 pip install --no-cache-dir -U openmim 
 mim install mmengine 
 mim install "mmcv>=2.0.1" 
 mim install "mmdet>=3.1.0" 
 mim install "mmpose>=1.1.0" 
  • Start ComfyUI

Updates

  • requirements.txt: diffusers 0.27.2 is now supported

Download weights

You can download weights manually as follows:

  1. Download our trained weights.

  2. Download the weights of other components:

Finally, these weights should be organized in pretrained_weights as follows:

./pretrained_weights/
|-- MusePose
|   |-- denoising_unet.pth
|   |-- motion_module.pth
|   |-- pose_guider.pth
|   └── reference_unet.pth
|-- dwpose
|   |-- dw-ll_ucoco_384.pth
|   └── yolox_l_8x8_300e_coco.pth
|-- sd-image-variations-diffusers
|   └── unet
|       |-- config.json
|       └── diffusion_pytorch_model.bin
|-- image_encoder
|   |-- config.json
|   └── pytorch_model.bin
└── sd-vae-ft-mse
    |-- config.json
    └── diffusion_pytorch_model.bin

workflow demo

https://github.com/TMElyralab/Comfyui-MusePose/blob/main/musepose-workflow-demo.json

musepose-workflow-demo.mov

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages