👀 Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardware 👀

💻 Blog |📜 Paper | 🗂️ Data

Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardware
🧑🏻‍🚀 Pietro Bonazzi ¹, Sizhen Bian ¹, Giovanni Lippolis ², Yawei Li¹, Sadique Sheik ², Michele Magno¹

¹ ETH Zurich, Switzerland
² SynSense AG, Switzerland

Quick Comparaison: Ground Truth (GT) vs. Prediction

In the following GIFs, Yellow represents the Ground Truth (GT), and Green represents the Prediction. These images are taken from the validation set.

✉️ Citation ❤️

Leave a star to support our open source initiative!⭐️

@InProceedings{Bonazzi_2024_CVPR,
    author    = {Bonazzi, Pietro and Bian, Sizhen and Lippolis, Giovanni and Li, Yawei and Sheik, Sadique and Magno, Michele},
    title     = {Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardware},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2024},
    pages     = {5684-5692}
}

🚀 TL;DR quickstart 🚀

Clone the repo

git clone https://gitlab.ethz.ch/pbonazzi/retina.git
cd retina

Create the environment

conda create -n retina python=3.10 numpy=1.8.1
conda activate retina
pip install -r requirements.txt
pip install git+https://gitlab.com/inivation/dv/dv-processing.git

Downloads

Click here to download the dataset.

Verify the structure:

.
├── name
│   ├── annotations.csv
│   └── events.aedat4
├── ...
├── silver.csv

Click here to download a pretrained model.

Training

See the list of arguments in the launch_fire function. The run-name has the format version-name.

python train.py --run-name="1-train"