This is the official code repo for our paper: Enhancing Meme Emotion Understanding with Multi-Level Modality Enhancement and Dual-Stage Modal Fusion.
Please follow the below instructions to prepare the required environment.
conda create -n meme python=3.12
conda activate meme
pip install -r requirements.txt
We have already given some examples for the data format in data/MET-MEME. These json files contain our four-step enhancement text for each meme. Complete json files for MET-MEME and MOOD can be downloaded here.
In addition, meme image files are not included in our repo. They can be downloaded in official data repo for MET-MEME and MOOD. Please download these meme images and place them in following paths: data/MET-MEME/image and data/mood/mood_images.
You can train MemoDetector under different settings using corresponding command line instructions. For example, if you want to train full setting MemoDetector mentioned in our paper, you can run following instructions:
python train.py --pre_fuse --fuse bi_cross_attn --dataset metmeme --device 0
If you want to run other ablations, please change cmd options.
Final test results can be found in Logs folder, where the average metrics of 5 runnings are recorded.