Clear Face
Clear Face is python project with C++/Python libraries for facial landmark detection(in 2d and 3d), tracking faces and multiple models detection from faces such as:
- Gender, Expressions, Illumination, Pose, Occlusion, Age, and Makeup.
Content Tabel
Requerments
Download and setup
Download Repo
git clone https://github.com/DiaaZiada/ClearFace.git
cd ClearFace
Virtual Environment
# create virtual environment
python -m venv ./venv
# activate virtual environment
source venv/bin/activate
Install Requirements
pip install -r requirements.txt
Download libtorch for c++ (CPU)
./download.sh
note: this command will download and extract the libarary
Build all c++ files
./build.sh
Running Options
to use ClearFace execute run.py
file with various options
usage: run.py [-h] [--show] [--_2d] [--_3d] [--_2d3d] [--tracking]
[--delay DELAY] [--inputs_path INPUTS_PATH]
[--video_name VIDEO_NAME] [--outputs_path OUTPUTS_PATH]
[--models_path MODELS_PATH] [--cam_num CAM_NUM]
[--models MODELS [MODELS ...]]
Clear Face is python project with C++/Python libraries for facial landmark detection(in 2d and 3d), tracking faces and multiple models detection from faces such as:
- Gender, Expressions, Illumination, Pose, Occlusion, Age, and Makeup.
optional arguments:
-h, --help show this help message and exit
--show set this parameter to True value if you want display
images/videos while processing, default is False
--_2d set this parameter to True value if you 2d landmarks
from from the 2d model, default is False
--_3d set this parameter to True value if you 3d landmarks
from from the 3d model, default is False
--_2d3d set this parameter to True value if you 2d landmarks
from from the 3d model, default is False
--tracking set this parameter to True value if you want tracking
faces in images/videos, default is False
--delay DELAY amount of seconds to wait to switch between images
while show the precess
--inputs_path INPUTS_PATH
path for directory contains images/videos to process,
if you don't use it webcam will open to start the
record
--video_name VIDEO_NAME
name of recorded video from camera
--outputs_path OUTPUTS_PATH
path for directory to add the precesses images/videos
on it, if you don't use it output directory will
created and add the precesses images/videos on it
--models_path MODELS_PATH
path for directory contains pytorch model
--cam_num CAM_NUM number of camera to use it
--models MODELS [MODELS ...]
first index refers to gender model, second index
refers to expression model, and third index refers to
multiple models
Examples
for process images/videos in directory
$python run.py --inputs_path /path/to/inputs_dir --tracking --show --_2d3d --_3d
to use webcam
$python run.py ---tracking --show --_2d3d --_3d
Gender Expressions and other models
- Gender : Male, Female
- Expressions : Anger, Happiness, Sadness, Surprise, Fear, Disgust
- Illumination : Bad, Medium, High
- Pose : Frontal, Left, Right, Up, Down
- Occlusion : Glasses, Beard, Ornaments, Hair, Hand, None, Others
- Age : Child, Young, Middle and Old
- Makeup : Partial makeup, Over-makeup
Datasets
Models
models in this project are based on Real-time Convolutional Neural Networks for Emotion and Gender Classification paper model architecture:
this part of the project consist of 3 models:
- Gender
- Expression
- Multiple
Gender Model
trained done by using IMDB , WIKI, IMFDB datasets. consist of ~ 600 K image, and by using tencrop data augmentation dataset increased to be ~ 6 M image Accuracy ~ 78% using only 6 epochs and it will reach higher accuracy expected to be ~ 96 %
Expression Model
trained done by using FER , IMFDB, JAFFE image datasets. consist of ~ 40 K image, and by using tencrop data augmentation dataset increased to be ~ 400 K image Accuracy ~ 60% using 15 epochs and it will reach higher accuracy expected to be ~ 66 %
Multiple Models
this model is little bit different form Gender & Expression models in this model we use one feature extractor model and 5 different classifiers each classifier predict specific features form faces and they are:
- Illumination : Bad, Medium, High
- Pose : Frontal, Left, Right, Up, Down
- Occlusion : Glasses, Beard, Ornaments, Hair, Hand, None, Others
- Age : Child, Young, Middle and Old
- Makeup : Partial makeup, Over-makeup
trained done by using IMFDB image datasets consist of ~ 3 K image, and by using tencrop data augmentation dataset increased to be ~ 30 K image Accuracy ~ 77% using 15 epochs
Train
all training process done on notebook using Google Colab cloud
Convert to TorchScript
we convert ower models to torchscript to load them in c++ and use them as extention of ClearFace all convertions about this part will found in this notebook
Face Tracking
we used the centroid tracking algorithm
Facial Landmark Detection
DataSet
kaggle youtube faces with facial keypoints ~ 150K image
Train
all training process for the 2d and 3d model done in this notebook using Google Colab cloud
Pre-Trained Models
You can email me on [email protected] to get the pre-trained models
Credits
- Real-time Convolutional Neural Networks for Emotion and Gender Classification paper
- Simple object tracking with OpenCV by pyimagesearch
- Extending TorchScript with Custom C++ Operators
- CMake Tutorial
- Ryan Reynolds and Jake Gyllenhaal interview for LIFE, DEADPOOL - UNCENSORED Youtube
- 1adrianb face alignment repo