Neural network-based programs can do better about pychubby HOT 3 CLOSED

You are not able to provide your own input image. You can just sample some latent z and then pass it through the (pretrained) generator. Changing the expressions/appearance is then just simply done via perturbing the latent z in the correct direction.

Isn' that net a half of an autoencoder, so if we had an encoder, it should have been possible to create a latent vector corresponding to the image? Anyway, even if it isn't, restoring the latent vector corresponding to the image with backprop should work, though not as efficient, as if we had an encoder.

from pychubby.

So the StyleGAN only contains the "decoder" path - the generator. Assuming that one wants to use their pretrained model then the only possibility left is to find a z such that G(z) ~ x. As you pointed out it can be done through backprop and in general it is not clear whether suchz even exists. Sure, one can just hope that we are close enough but I wonder what artifacts it is going to create.

I actually did some research and found one project that tries to achieve something very similar https://github.com/Puzer/stylegan-encoder. It's great and I totally commend the author, however, let me point out a few problems:

• Requires gpus
• Not very user friendly (no setup.py -> not on PyPI + requires some background in ML). It's a fork of the official StyleGAN so I understand there are inherent limitations
• Only provides 3 latent directions (smile, age and gender)
• Aligns the original face and then only works with the aligned version of the image. That means that additional logic for changing the initial image would have to be implemented + logic for multiple faces
• I haven't tested it but I would guess that in some cases it creates artifacts that are either due to imperfections in the pretrained model, encoding approximation (x -> z) or the latent space interpolations. For all of these there is no quick fix.

I am actually really interested in extending some of the ideas from that project. However, for now I just wanted to list some things the "Neural network-based programs" might struggle with.

Some links

• reddit - stylegan_but_in_reverse

• youtube video

• Fork of the Puzer/StyleGAN code - best implementation out there

from pychubby.

Hey @KOLANICH, thank you for your interest.

pychubby is purely based on geometric transformations (warping) of the input image. It cannot create new textures, objects, etc. So just from this point of view it is fundamentally different from pix2pix GANs and similar.

See below some comments on the two links:

Transparent Latent GAN

• You are not able to provide your own input image. You can just sample some latent z and then pass it through the (pretrained) generator. Changing the expressions/appearance is then just simply done via perturbing the latent z in the correct direction.

• The number of actions is limited by the attributes the feature extractor network was trained on Dataset Celebrities

Style GAN
This is a brilliant paper however again I would like to point out that the goal of it is fundamentally different from pychubby.

• Same as for the first link, you are not able to provide your own input image.

I would be more than happy to discuss further if you want to!
Cheers

from pychubby.