Fitness-AQA (Fitness Action Quality Assessment) [ECCV'22]

Full paper: Domain Knowledge-Informed Self-Supervised Representations for Workout Form Assessment

Contents

1. Introduction
2. Our Self-Supervised Pose Contrastive Learning Approach for Fine-grained Action Assessment
3. Our Self-Supervised Motion Disentangling Approach for Fine-grained Action Assessment
4. Our Self-Supervised Pose and Appearance Disentangling Approach
5. Our Method for Synchronizing In-the-Wild Videos
6. Fitness-AQA Dataset for Fine-grained Exercise Action Quality Assessment

Introduction

Analyzing a person's posture during exercising is necessary to prevent injuries and maximizing muscle mass gains. In this work, we present an AI-based approach to detect errors in workout form. Our approach is particularly applicable/useful in real world gym scenarios, where off-the-shelf pose estimators fail to effectively capture person's pose due to challenging factors like camera recording angles, clothing styles, occlusions from gym equipment, etc. We applied our system to detect posture errors in three exercises: 1) BackSquat; 2) OverheadPress; and 3) BarbellRow. For this we collected the largest fine-grained exercise action quality assessment dataset, Fitness-AQA. Details on our self-supervised representation learning approaches and dataset are as follows:

Our Self-Supervised Pose Contrastive Learning Approach for Fine-grained Action Assessment

Our Self-Supervised Motion Disentangling Approach for Fine-grained Action Assessment

Our Self-Supervised Pose and Appearance Disentangling Approach

Our Method for Synchronizing In-the-Wild Videos

Fitness-AQA Dataset for Fine-grained Exercise Action Quality Assessment

Dataset available from: https://forms.gle/PbPTX1eVxGpa3QG88. Please send us the request to access the dataset using this form. By requesting the dataset, you agree to the terms and conditions of usage. This dataset shall only be used for non-commercial purposes. Please check your spam folder in case you seem to not have received the access after requesting it; or please contact me if you are still have not received the access. Thank you!

Please feel free to reach out to me if you have any questions or face any problems.

If you find our work useful, please consider citing:

@inproceedings{parmar2022domain,
  title={Domain Knowledge-Informed Self-supervised Representations for Workout Form Assessment},
  author={Parmar, Paritosh and Gharat, Amol and Rhodin, Helge},
  booktitle={Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXXVIII},
  pages={105--123},
  year={2022},
  organization={Springer}
}