PyTorch implementation of the training and inference procedures described in the papers:
- On Generalizing Driver Gaze Zone Estimation using Convolutional Neural Networks," IEEE Intelligent Vehicles Symposium, 2017
- Driver Gaze Zone Estimation using Convolutional Neural Networks: A General Framework and Ablative Analysis," IEEE Transactions on Intelligent Vehicles, 2018
- Driver Gaze Estimation in the Real World: Overcoming the Eyeglass Challenge, IEEE Intelligent Vehicles Symposium, 2020.
- Gaze Preserving CycleGANs for Eyeglass Removal & Persistent Gaze Estimation, arXiv:2002.02077, 2020.
Parts of the CycleGAN code have been adapted from the PyTorch-CycleGAN respository.
- Clone this repository
- Install Pipenv:
pip3 install pipenv- Install all requirements and dependencies in a new virtual environment using Pipenv:
cd GPCycleGAN
pipenv install- Get link for desired PyTorch and Torchvision wheel from here and install it in the Pipenv virtual environment as follows:
pipenv install https://download.pytorch.org/whl/cu100/torch-1.2.0-cp36-cp36m-manylinux1_x86_64.whl
pipenv install https://download.pytorch.org/whl/cu100/torchvision-0.3.0-cp36-cp36m-linux_x86_64.whlThis is the dataset introduced in the papers On Generalizing Driver Gaze Zone Estimation using Convolutional Neural Networks and Driver Gaze Zone Estimation using Convolutional Neural Networks: A General Framework and Ablative Analysis. To use this dataset, do the following:
- Download the complete RGB dataset for driver gaze classification using this link.
- Unzip the file.
This is the second dataset introduced in the paper Driver Gaze Zone Estimation using Convolutional Neural Networks: A General Framework and Ablative Analysis. To use this dataset, do the following:
- Download the complete RGB dataset for driver gaze classification using this link.
- Unzip the file.
This is the dataset introduced in the paper Driver Gaze Estimation in the Real World: Overcoming the Eyeglass Challenge. To use this dataset, do the following:
- Download the complete IR+RGB dataset for driver gaze classification using this link.
- Unzip the file.
- Prepare the train, val and test splits as follows:
python prepare_gaze_data.py --dataset-dir=/path/to/lisat_gaze_data_v2The best performing SqueezeNet gaze classifier can be trained using the following command:
pipenv shell # activate virtual environment
python gazenet.py --dataset-root-path=/path/to/lisat_gaze_data_v0/ --version=1_1 --snapshot=./weights/squeezenet1_1_imagenet.pth --random-transformsThe best performing SqueezeNet gaze classifier can be trained using the following command:
pipenv shell # activate virtual environment
python gazenet.py --dataset-root-path=/path/to/lisat_gaze_data_v1/ --version=1_1 --snapshot=./weights/squeezenet1_1_imagenet.pth --random-transformsThe prescribed three-step training procedure can be carried out as follows:
pipenv shell # activate virtual environment
python gazenet.py --dataset-root-path=/path/to/lisat_gaze_data_v2/ir_no_glasses/ --version=1_1 --snapshot=./weights/squeezenet1_1_imagenet.pth --random-transformspython gpcyclegan.py --dataset-root-path=/path/to/lisat_gaze_data_v2/ --data-type=ir --version=1_1 --snapshot-dir=/path/to/trained/gaze-classifier/directory/ --random-transformspython create_fake_images.py --dataset-root-path=/path/to/lisat_gaze_data_v2/ir_all_data/ --snapshot-dir=/path/to/trained/gpcyclegan/directory/
cp /path/to/lisat_gaze_data_v2/ir_all_data/mean_std.mat /path/to/lisat_gaze_data_v2/ir_all_data_fake/mean_std.mat # copy over dataset mean/std information to fake data folderpython gazenet-ft.py --dataset-root-path=/path/to/lisat_gaze_data_v2/ir_all_data_fake/ --version=1_1 --snapshot-dir=/path/to/trained/gpcyclegan/directory/ --random-transforms
exit # exit virtual environmentThe prescribed three-step training procedure can be carried out as follows:
pipenv shell # activate virtual environment
python gazenet.py --dataset-root-path=/path/to/lisat_gaze_data_v2/rgb_no_glasses/ --version=1_1 --snapshot=./weights/squeezenet1_1_imagenet.pth --random-transformspython gpcyclegan.py --dataset-root-path=/path/to/lisat_gaze_data_v2/ --data-type=rgb --version=1_1 --snapshot-dir=/path/to/trained/gaze-classifier/directory/ --random-transformspython create_fake_images.py --dataset-root-path=/path/to/lisat_gaze_data_v2/rgb_all_data/ --snapshot-dir=/path/to/trained/gpcyclegan/directory/
cp /path/to/lisat_gaze_data_v2/rgb_all_data/mean_std.mat /path/to/lisat_gaze_data_v2/rgb_all_data_fake/mean_std.mat # copy over dataset mean/std information to fake data folderpython gazenet-ft.py --dataset-root-path=/path/to/lisat_gaze_data_v2/rgb_all_data_fake/ --version=1_1 --snapshot-dir=/path/to/trained/gpcyclegan/directory/ --random-transforms
exit # exit virtual environmentInference can be carried out using this script as follows:
pipenv shell # activate virtual environment
python infer.py --dataset-root-path=/path/to/lisat_gaze_data_v0/ --split=val --version=1_1 --snapshot-dir=/path/to/trained/rgb-model/directory/ --save-viz
exit # exit virtual environmentInference can be carried out using this script as follows:
pipenv shell # activate virtual environment
python infer.py --dataset-root-path=/path/to/lisat_gaze_data_v0/ --split=val --version=1_1 --snapshot-dir=/path/to/trained/rgb-model/directory/ --save-viz
exit # exit virtual environmentInference can be carried out using this script as follows:
pipenv shell # activate virtual environment
python infer.py --dataset-root-path=/path/to/lisat_gaze_data_v2/ir_all_data/ --split=test --version=1_1 --snapshot-dir=/path/to/trained/ir-models/directory/ --save-viz
exit # exit virtual environmentInference can be carried out using this script as follows:
pipenv shell # activate virtual environment
python infer.py --dataset-root-path=/path/to/lisat_gaze_data_v2/rgb_all_data/ --split=val --version=1_1 --snapshot-dir=/path/to/trained/rgb-models/directory/ --save-viz
exit # exit virtual environmentYou can download our pre-trained model weights using this link.
Config files, logs, results, snapshots, and visualizations from running the above scripts will be stored in the GPCycleGAN/experiments folder by default.
If you find our data, code, and/or models useful in your research, please consider citing the following papers:
@inproceedings{vora2017generalizing,
title={On generalizing driver gaze zone estimation using convolutional neural networks},
author={Vora, Sourabh and Rangesh, Akshay and Trivedi, Mohan M},
booktitle={2017 IEEE Intelligent Vehicles Symposium (IV)},
pages={849--854},
year={2017},
organization={IEEE}
}
@article{vora2018driver,
title={Driver gaze zone estimation using convolutional neural networks: A general framework and ablative analysis},
author={Vora, Sourabh and Rangesh, Akshay and Trivedi, Mohan Manubhai},
journal={IEEE Transactions on Intelligent Vehicles},
volume={3},
number={3},
pages={254--265},
year={2018},
publisher={IEEE}
}
@article{rangesh2020driver,
title={Gaze Preserving CycleGANs for Eyeglass Removal & Persistent Gaze Estimation},
author={Rangesh, Akshay and Zhang, Bowen and Trivedi, Mohan M},
journal={arXiv preprint arXiv:2002.02077},
year={2020}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent