From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos

🔥 [Recognized as a Highly Cited Paper by Web of Science (Top 1%)]

From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos
Yin Chen$^{†}$, Jia Li$^{†∗}$, Shiguang Shan, Meng Wang, and Richang Hong

📰 News

[2025.9.15] Our new paper S4D has been accepted by IEEE Transactions on Affective Computing.

[2025.9.15] S2D has been recognized as a Highly Cited Paper by Clarivate.

[2024.9.5] The fine-tuned checkpoints are available.

[2024.9.2] The code and pre-trained models are available.

[2024.8.28] The paper is accepted by IEEE Transactions on Affective Computing.

[2023.12.5] Code and pre-trained models will be released here.

🚀 Main Results

Dynamic Facial Expression Recognition

Static Facial Expression Recognition

Visualization

Fine-tune with pre-trained weights

1、 Download the pre-trained weights from baidu drive or google drive or Huggingface, and move it to the ckpts directory.

2、 Run the following command to fine-tune the model on the target dataset.

conda create -n s2d python=3.9
conda activate s2d
pip install -r requirements.txt
bash run.sh

📋 Reported Results and Fine-tuned Weights

The fine-tuned checkpoints can be downloaded from Baidu Drive or Huggingface.

Datasets	w/o oversampling		w/ oversampling
	UAR	WAR	UAR	WAR
FERV39K
FERV39K	41.28	52.56	43.97	46.21
DFEW
DFEW01	61.56	76.16	64.80	75.35
DFEW02	59.93	73.99	62.54	72.53
DFEW03	61.33	76.41	66.47	75.87
DFEW04	62.75	76.31	66.03	74.48
DFEW05	63.51	77.27	67.43	76.80
DFEW	61.82	76.03	65.45	74.81
MAFW
MAFW01	32.78	46.76	36.16	44.21
MAFW02	40.43	55.96	41.94	51.22
MAFW03	47.01	62.08	48.08	61.48
MAFW04	45.66	62.61	47.67	60.64
MAFW05	43.45	59.42	43.16	58.55
MAFW	41.86	57.37	43.40	55.22

✏️ Citation

If you find this work helpful, please consider citing:

@ARTICLE{10663980,
  author={Chen, Yin and Li, Jia and Shan, Shiguang and Wang, Meng and Hong, Richang},
  journal={IEEE Transactions on Affective Computing}, 
  title={From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos}, 
  year={2024},
  volume={},
  number={},
  pages={1-15},
  keywords={Adaptation models;Videos;Computational modeling;Feature extraction;Transformers;Task analysis;Face recognition;Dynamic facial expression recognition;emotion ambiguity;model adaptation;transfer learning},
  doi={10.1109/TAFFC.2024.3453443}}

@ARTICLE{11207542,
  author={Chen, Yin and Li, Jia and Zhang, Yu and Hu, Zhenzhen and Shan, Shiguang and Wang, Meng and Hong, Richang},
  journal={IEEE Transactions on Affective Computing}, 
  title={Static for Dynamic: Towards a Deeper Understanding of Dynamic Facial Expressions Using Static Expression Data}, 
  year={2025},
  volume={},
  number={},
  pages={1-15},
  keywords={Videos;Adaptation models;Face recognition;Transformers;Semantics;Multitasking;Computer vision;Spatiotemporal phenomena;Correlation;Emotion recognition;Dynamic facial expression recognition;mixture of experts;self-supervised learning;vision transformer},
  doi={10.1109/TAFFC.2025.3623135}}

Star History