Automatic_Signal_Detector

Abstract

In this project, there are two parts:

• Create hands dataset
• Train model to recognize hands (Here, implemented three models: MLP, CNN, Transfer Learning)

In order to create hands dataset, we need to find a way to detect our hands. There are some pattern detecting faces and eyes. Given that patterns, we use Camshift algorithm to detect hands by setting face region probability to zero, and then it's likely that hands is the most likely region to be detected.

Getting Started

Clone the repository.

git clone https://github.com/Zhijie-He/Automatic_Signal_Detector.git

Goto the cloned folder.

cd Automatic_Signal_Detector

Steps to run Code

### For Linux Users
python3 -m venv psestenv
source psestenv/bin/activate

### For Window Users
python3 -m venv psestenv
cd psestenv
cd Scripts
activate
cd ..
cd ..

Install requirements with mentioned command below.

pip install -r requirements.txt

Create hands dataset

python create_hands_dataset [A-Z]

• --[A-Z] specifies the signal name.

Select the face capture by type "y"

After decide the face bounding, type "y" to create hand dataset according to the hands gesture(Signal_Language).

Train model to recoginze signal language, see model performance below

python main --model_name MLP --device cpu --wandb

• model_name specifies the model name, can select from MLP, CNN and TF.
• device cpu/0,1,2,3(gpu)
• wandb Open wandb

Hands dataset example

Different ways to detect face

Common way to detect faces using Haar Cascades	Optimization by seraching subregion	MeanShift	CamShift

Model Performance

Wandb workspace

MLP

CNN

Transfer Learning

Other

Colab Notebook Tutorial