hhj1897 / face_parsing Goto Github PK

View Code? Open in Web Editor NEW

207.0 6.0 36.0 180 KB

Official Pytorch implementation of 'RoI Tanh-polar Transformer Network for Face Parsing in the Wild.'

License: MIT License

Python 100.00%

pytorch face-parsing

face_parsing's Introduction

RoI Tanh-polar Transformer Network for Face Parsing in the Wild

Recent Updates

2022-04-02 Update: If you could not download the weights with LFS, check out issue #7 (comment) for alternative downloading links.

2022-03-04 Update: We have released the FP-Age model which can perform face parsing and age estimation simultaneously, please visit https://github.com/ibug-group/fpage for details.

Official repo for our paper RoI Tanh-polar transformer network for face parsing in the wild.

Note: If you use this repository in your research, we kindly rquest you to cite the following paper:

@article{lin2021roi,
title = {RoI Tanh-polar transformer network for face parsing in the wild},
journal = {Image and Vision Computing},
volume = {112},
pages = {104190},
year = {2021},
issn = {0262-8856},
doi = {https://doi.org/10.1016/j.imavis.2021.104190},
url = {https://www.sciencedirect.com/science/article/pii/S0262885621000950},
author = {Yiming Lin and Jie Shen and Yujiang Wang and Maja Pantic},
keywords = {Face parsing, In-the-wild dataset, Head pose augmentation, Tanh-polar representation},
}

Dependencies

git-lfs
Numpy: $pip3 install numpy
OpenCV: $pip3 install opencv-python
PyTorch: $pip3 install torch torchvision
ibug.roi_tanh_warping: See this repository for details: https://github.com/ibug-group/roi_tanh_warping.
ibug.face_detection (only needed by the test script): See this repository for details: https://github.com/hhj1897/face_detection.

How to Install

git clone https://github.com/hhj1897/face_parsing
cd face_parsing
git lfs pull
pip install -e .

How to Test

python face_warping_test.py -i 0 -e rtnet50 --decoder fcn -n 11 -d cuda:0

Command-line arguments:

-i VIDEO: Index of the webcam to use (start from 0) or
          path of the input video file
-d: Device to be used by PyTorch (default=cuda:0)
-e: Encoder (default=rtnet50)
--decoder: Decoder (default=fcn)
-n: Number of facial classes, can be 11 or 14 for now (default=11)

iBugMask Dataset

The training and testing images, bounding boxes, landmarks, and parsing maps can be found in the following:

Label Maps

Label map for 11 classes:

0 : background
1 : skin (including face and scalp)
2 : left_eyebrow
3 : right_eyebrow
4 : left_eye
5 : right_eye
6 : nose
7 : upper_lip
8 : inner_mouth
9 : lower_lip
10 : hair

Label map for 14 classes:

0 : background
1 : skin (including face and scalp)
2 : left_eyebrow
3 : right_eyebrow
4 : left_eye
5 : right_eye
6 : nose
7 : upper_lip
8 : inner_mouth
9 : lower_lip
10 : hair
11 : left_ear
12 : right_ear
13 : glasses

Visualisation

face_parsing's People

Contributors

Stargazers

Watchers

face_parsing's Issues

Dataset license

Hi, what's the license of the dataset? Does the MIT license also cover the dataset? Thanks

git lfs pull is too slow

could you please put the model on google driver for download?
The "git lfs pull" is too slow, just 20kb/s

Hello, thanks for sharing this great study.
I'm researching face parsing and i'm trying to port to Tflite and compare the performance, but I can't since this is using special ops - it uses "grid sample"

What do you suggest I can do in order to test on Tflite/CoreML?

Will training on Lapa dataset improve the accuracy? if not why?

Where can I find the weights of face parsing network

The weights of face parsing network is missing.
Where can I find them?

face parsing label

It seems that the dataset released contains only the annotation of 11 facial parts. However, the repository also provide the model trained with dataset containing labels of 14 facial parts. Thus, we wonder how can we get the labels of 14 facial parts. Can you provide the download link? Thanks!

training model

hello，How can we train this model?

training code

Thank you very much for your work. Have you considered releasing the training code?

face_warping_test.py, where the file?

python face_warping_test.py -i 0 -e rtnet50 --decoder fcn -n 11 -d cuda:0

Inference without face detector

Hi, thanks for sharing codes, it shows impressive results on my data. Is it possible to work without face detector?

iBugmask

Hello, how can l download the iBugmask dataset?

Issues with installation and running

As can be seen in the above repo, I am having a hard time getting the code demo to run. Can you help?

_pickle.UnpicklingError: invalid load key, 'v'

How to fix it

Traceback (most recent call last):
File "face_parsing_test.py", line 141, in
main()
File "face_parsing_test.py", line 50, in main
face_parser = RTNetPredictor(
File "/home/ml/radishevskii/face_parsing/ibug/face_parsing/parser.py", line 81, in init
ckpt = torch.load(ckpt, 'cpu')
File "/home/ml/radishevskii/anaconda3/envs/inga_vlad/lib/python3.8/site-packages/torch/serialization.py", line 593, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/ml/radishevskii/anaconda3/envs/inga_vlad/lib/python3.8/site-packages/torch/serialization.py", line 762, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.