Giter Club home page Giter Club logo

gaze_redirection's Introduction

Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks

License: MIT

[Paper] [Video]

Authors: Zhe He, Adrian Spurr, Xucong Zhang, Otmar Hilliges

Contact: [email protected]

The following gifs are made of images generated by our method. For each GIF, the input is a still image.

Our method is also capable of handling different head poses.

Note

The code here is the development version. It can be used for training, but there might be some redundant code and compatiblity issues. The final version will be released soon.

Dependencies

tensorflow == 1.7
numpy == 1.13.1
scipy == 0.19.1

Dataset

The dataset contains eye patch images parsed from Columbia Gaze Dataset. It can be downloaded via this link.

tar -xvf dataset.tar

The dataset contains six subfolders, N30P/, N15P/, 0P/, P15P/, P30P/ and all/. Prefix 'N' means negative head pose, and 'P' means positive head pose. Folder all/ contains all eye patch images with different head poses.

VGG-16 pretrained weights

wget http://download.tensorflow.org/models/vgg_16_2016_08_28.tar.gz .
tar -xvf vgg_16_2016_08_28.tar.gz

Train

python main.py --mode train --data_path ./dataset/all/ --log_dir ./log/ --batch_size 32 --vgg_path ./vgg_16.ckpt

Test

To test the model on frontal faces, run the following command.

python main.py --mode eval --data_path ./dataset/0P/ --log_dir ./log/ --batch_size 21

Then, a folder named eval will be generated in folder ./log/. Generated images, input images and target images will be stored in eval/.

gaze_redirection's People

Contributors

hzdms avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gaze_redirection's Issues

modern tensorflow equivalent

is it possible to run this with a more recent version of tensorflow? the resources that I use don't have such an old version of CUDA for tensorflow 1.7

How to restore checkpoint and only use generator to generate image

function eval in model.py
I can successfully generate targets, genes and reals by my saved checkpoint model.
However, I want to use only generator to generate the eye with target angle.
That is, input: x_test_r, angles_test_g, output: 64 x 64 x 3 eye image.
So, how to used the restored checkpoint and only use generator to generate image?

Thanks

how to find minimal enclosed circle

hi,i am very appreciate for your excellent work,the method of solving minimal enclosed circle has confused me a lot,can your explain it with more detail。

Test with in-the-wild images

Hi, have you ever test your model with some in-the-wild images like celebA. I think this would enhance the experiment results in your paper as well as this repo.

how to hand with the whitening face processed by whitening technology of phone?

hello, this project has a good performance handing with yellow-skinned face, but poor performance for whitening face.
As we all know, now many people like to use whitening technology of the phone or camera to process image to whiten the face skin. And I find that the Columbia Gaze Dataset has slightly white-skinned face which is still different the whitening face processed by whitening technology of the phone or camera. So there are some larger gaze dataset including whitening face processed by whitening technology? If no, how should I hand with this whitening face test image to get a good performance?
Anyone can give some advises? @HzDmS

the color of generated pic is not consistent with orginal

thanks a lot,recently i train a model with your provided code and data,i find the color of generated eyes is not always consistent with the original input(test on other pic which is not in columbia dataset),when i cancel all instance normalization layer in generator, the color of generated eyes become approximate consitent with the original input? did you pay attention on this phenomenon.

Config environment

Hi,

I tried configuring the environment via Anaconda however, quite a few conflicts showed up.

I wonder could you provide a more detailed dependency list, including python version and tensorflow-gpu version as well?

Thanks a lot.

Is it wrong to pass img in value range [-1, 1] to pretrained VGG-16?

the pretrained VGG-16 should accept the input image in range [0, 255] minus the mean value of RGB channels [123.6800, 116.7790, 103.9390]. if give the input image in range [-1,1], do that mean give an input image around the mean value ? so the pretrained network can not well extract the feature difference of source image and target image?

training 128 px generator

while setting the image size to 128 px, the generated images are mostly exactly similar to the input image with almost zero gaze redirection. I tried increasing the loss component for gaze redirection reg_g_loss by a factor of 200, still the actual redirections is almost zero. Any insights on how to get this to work?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.