The task of Dense Video Captioning consists in generating engaging caption describing soccer actions and localizing each caption with a timestamp. For that, we provide 471 videos from soccer broadcast games available at two resolutions (720p and 224p) with captions. We also provide extracted features at 2 frames per second for lighter models, including the feature provided by the 2021 action spotting challenge winners, Baidu Research. The provided data also includes original comments and versions where referees, coaches, players, and teams have been anonymized or identified, as well as team lineups. The challenge set is composed of 42 separate games.
Participate in the 2024 edition of our Dense Video Captioning Challenge on our Evaluation Server.
This repository provides a first Baseline method and code for participants to get started.
| Team | METEOR | Bleu_1 | Bleu_2 | Bleu_3 | Bleu_4 | ROUGE_L | CIDEr | Recall | Precision |
|---|---|---|---|---|---|---|---|---|---|
| OPPO - VideoText | 26.66 | 35.55 | 31.03 | 28.13 | 25.65 | 33.23 | 69.73 | 24.59 | 68.59 |
| HZC | 21.3 | 29.73 | 24.52 | 21.44 | 19.13 | 24.56 | 24.76 | 98.68 | 51.19 |
| Baseline 2 | 21.25 | 30.01 | 24.80 | 21.74 | 19.44 | 24.65 | 25.68 | 98.68 | 51.21 |
| justplay | 21.2 | 29.83 | 24.68 | 21.66 | 19.38 | 24.34 | 25.89 | 98.68 | 50.99 |
| aisoccer | 21.02 | 29.53 | 24.42 | 21.42 | 19.15 | 24.31 | 23.72 | 98.63 | 50.83 |
| Baseline 1 | 15.24 | 11.91 | 9.97 | 8.83 | 7.97 | 10.69 | 16.33 | 98.97 | 23.92 |
A SoccerNet pip package to easily download the data and the annotations is available.
To install the pip package simply run:
pip install SoccerNet
Then use the API to downlaod the data of interest including annotations and features at 2fps:
from SoccerNet.Downloader import SoccerNetDownloader as SNdl
mySNdl = SNdl(LocalDirectory="path/to/SoccerNet")
mySNdl.downloadDataTask(task="caption-2023", split=["train","valid", "test","challenge"]) # SN challenge 2023
mySNdl.downloadDataTask(task="caption-2024", split=["train","valid", "test","challenge"]) # SN challenge 2024
If you want to download the videos, you will need to fill a NDA to get the password.
mySNdl.password = input("Password for videos?:\n")
mySNdl.downloadGames(files=["1_224p.mkv", "2_224p.mkv"], split=["train","valid","test","challenge"])
mySNdl.downloadGames(files=["1_720p.mkv", "2_720p.mkv", "video.ini"], split=["train","valid","test","challenge"])
Check out our other challenges related to SoccerNet!
Please cite our work if you use our dataset:
@inproceedings{mkhallati2023soccernet,
title={SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries},
author={Mkhallati, Hassan and Cioppa, Anthony and Giancola, Silvio and Ghanem, Bernard and Van Droogenbroeck, Marc},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={5073--5084},
year={2023}
}