Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis

📝 Meissonic Updates and Family Papers

• Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis [ICLR 2025]
• Bag of Design Choices for Inference of High-Resolution Masked Generative Transformer
• Di[𝙼]O: Distilling Masked Diffusion Models into One-step Generator
• More papers are coming soon!

🚀 Introduction

Meissonic is a non-autoregressive mask image modeling text-to-image synthesis model that can generate high-resolution images. It is designed to run on consumer graphics cards.

Key Features:

• 🖼️ High-resolution image generation (up to 1024x1024)
• 💻 Designed to run on consumer GPUs
• 🎨 Versatile applications: text-to-image, image-to-image

🛠️ Prerequisites

Step 1: Clone the repository

git clone https://github.com/viiika/Meissonic/
cd Meissonic

Step 2: Create virtual environment

conda create --name meissonic python
conda activate meissonic
pip install -r requirements.txt

Step 3: Install diffusers

git clone https://github.com/huggingface/diffusers.git
cd diffusers
pip install -e .

💡 Inference Usage

Gradio Web UI

python app.py

Command-line Interface

Text-to-Image Generation

python inference.py --prompt "Your creative prompt here"

Inpainting and Outpainting

python inpaint.py --mode inpaint --input_image path/to/image.jpg
python inpaint.py --mode outpaint --input_image path/to/image.jpg

Advanced: FP8 Quantization

Optimize performance with FP8 quantization:

Requirements:

• CUDA 12.4
• PyTorch 2.4.1
• TorchAO

Note: Windows users install TorchAO using

pip install --pre torchao --index-url https://download.pytorch.org/whl/nightly/cpu

Command-line inference

python inference_fp8.py --quantization fp8

Gradio for FP8 (Select Quantization Method in Advanced settings)

python app_fp8.py

Performance Benchmarks

Precision (Steps=64, Resolution=1024x1024)	Batch Size=1 (Avg. Time)	Memory Usage
FP32	13.32s	12GB
FP16	12.35s	9.5GB
FP8	12.93s	8.7GB

🎨 Showcase

"A pillow with a picture of a Husky on it."

"A white coffee mug, a solid black background"

🎓 Training

To train Meissonic, follow these steps:

1. Install dependencies:

   cd train
   pip install -r requirements.txt

2. Download the Meissonic base model from Hugging Face.

3. Prepare your dataset:

   • Use the sample dataset: MeissonFlow/splash
   • Or prepare your own dataset and dataset class following the format in line 100 in dataset_utils.py and line 656-680 in train_meissonic.py
   • Modify train.sh with your dataset path

4. Start training:

   bash train/train.sh

Note: For custom datasets, you'll likely need to implement your own dataset class.

📚 Citation

If you find this work helpful, please consider citing:

@article{bai2024meissonic,
  title={Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis},
  author={Bai, Jinbin and Ye, Tian and Chow, Wei and Song, Enxin and Chen, Qing-Guo and Li, Xiangtai and Dong, Zhen and Zhu, Lei and Yan, Shuicheng},
  journal={arXiv preprint arXiv:2410.08261},
  year={2024}
}

🙏 Acknowledgements

We thank the community and contributors for their invaluable support in developing Meissonic. We thank apolinario@multimodal.art for making Meissonic Demo. We thank @NewGenAI and @飛鷹しずか@自称文系プログラマの勉強 for making YouTube tutorials. We thank @pprp for making fp8 and int4 quantization. We thank @camenduru for making jupyter tutorial. We thank @chenxwh for making Replicate demo and api. We thank Collov Labs for reproducing Monetico. We thank Shitong et al. for identifying effective design choices for enhancing visual quality.

---

Made with ❤️ by the MeissonFlow Research