FastV2C-HandNet : Fast Voxel to Coordinate Hand Pose Estimation with 3D Convolutional Neural Networks

Introduction

This is the project repository for the paper, FastV2C-HandNet : Fast Voxel to Coordinate Hand Pose Estimation with 3D Convolutional Neural Networks (Springer).

Please refer to our paper for details.

If you find our work useful in your research or publication, please cite our work:

[1] Rohan Lekhwani, Bhupendra Singh. "FastV2C-HandNet : Fast Voxel to Coordinate Hand Pose Estimation with 3D Convolutional Neural Networks"[Springer]

Lekhwani, Rohan, and Bhupendra Singh. 
"FastV2C-HandNet: Fast Voxel to Coordinate Hand Pose Estimation with 3D Convolutional Neural Networks." 
International Conference on Innovative Computing and Communications. 
Springer, Singapore, 2019.

In this repository, we provide

• Our model architecture description (FastV2C-HandNet)
• Comparison with the previous state-of-the-art methods
• Training code
• Dataset we used (MSRA)
• Trained models and estimated results

Model Architecture

Comparison with the previous state-of-the-art methods

About our code

Dependencies

• Keras
• Tensorflow
• CUDA
• cuDNN

The code is tested under Ubuntu 18.04, Windows 10 environment with Nvidia P100 GPU (16GB VRAM).

Code

Clone this repository into any place you want. You may follow the example below.

makeReposit = [/the/directory/as/you/wish]
mkdir -p $makeReposit/; cd $makeReposit/
git clone https://github.com/RonLek/FastV2C-HandNet.git

• src folder contains python script files for data loader, trainer, tester and other utilities.
• data folder should contain an 'MSRA' folder with binary image files.

To train our model, please run the following command in the src directory:

python train.py

Dataset

We trained and tested our model on the MSRA Hand Pose Dataset.

• MSRA Hand Pose Dataset [link] [paper]

Results

Here we provide the precomputed centers, estimated 3D coordinates and pre-trained models of MSRA dataset. You can download precomputed centers and 3D hand pose results in here and pre-trained models in here

The precomputed centers are obtained by training the hand center estimation network from DeepPrior++ . Each line represents 3D world coordinate of each frame. In case depth map does not exist or not contain hand, that frame is considered as invalid. All test images are considered as valid.

We used awesome-hand-pose-estimation to evaluate the accuracy of the FastV2C-HandNet on the MSRA dataset.

Belows are qualitative results.