<<<<<<< HEAD
This repository contains a pytorch implementation for the paper: SofGAN: A Portrait Image Generator with Dynamic Styling.
We propose a SofGAN image generator to decouple the latent space of portraits into two subspaces: a geometry space and a texture space.
Experiments on SofGAN show that our system can generate high quality portrait images with independently controllable geometry and texture attributes.
Install environment:
git clone https://github.com/apchenstu/sofgan.git --recursive
conda install pytorch==1.7.1 torchvision==0.8.2 cudatoolkit=10.2 -c pytorch
pip install tqdm argparse scikit-image lmdb config-argparse dlib
Please see each subsection for training on different datasets. Available training datasets:
- FFHQ
- CelebA
- Your own data (portrait images or segmaps)
We also provide our pre-process ffhq and celeba segmaps (in our classes labels). You may also want to retrain the SOF model base on your own multi-view segmaps.
Run
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 --master_port=9999 train.py \
--num_worker 4 --resolution 1024
--name $exp_name
--iter 10000000
--batch 1 --mixing 0.9 \
path/to/your/image/folders \
--condition_path path/to/your/segmap/folders
We strongly suggest you to use at least 4 GPUs, 4GPUs would take ~20 days for 10000k iterations. You can also reduce the resolution or iterations on your own dataset, you should have pleasure results after 1000k iterations.
Important: training on none pair-wise data (image/segmap) is encouraged, this is one of the key features of our sofgan.
We provide a rendering script in renderer.ipynb, you can restyle your own photos, videos, generating free-viewpoing portrait images.
Just to download our checkpoints and unzip to the root folder.
A Painter is in included in Painter, you can pull down and drawing on-the-fly.
Before that, you need to install the enviroment with pip install -r ./Painter/requirements.txt
You could download and try the Wand, an ipad app developed by Deemos.
On the way ...
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (TOG 2021)
Rameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka
SEAN: Image Synthesis With Semantic Region-Adaptive Normalization (CVPR 2020)
Peihao Zhu, Rameen Abdal, Yipeng Qin, Peter Wonka
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images (CVPR 2020)
A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H.P. Seidel, P. Pérez, M. Zollhöfer, Ch. Theobalt
StyleGAN2: Analyzing and Improving the Image Quality of {StyleGAN} (CVPR 2020)
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila
SPADE: Semantic Image Synthesis with Spatially-Adaptive Normalization (CVPR 2019)
Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu
If you find our code or paper helps, please consider citing:
@article{sofgan,
author = {Chen, Anpei and Liu, Ruiyang and Xie, Ling and Chen, Zhang and Su, Hao and Yu Jingyi},
title = {SofGAN: A Portrait Image Generator with Dynamic Styling},
year = {2021},
issue_date = {Jul 2021},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {41},
number = {1},
url = {https://doi.org/10.1145/3470848},
doi = {10.1145/3470848},
journal = {ACM Trans. Graph.},
month = July,
articleno = {1},
numpages = {26},
keywords = {image editing, Generative adversarial networks}
}
=======
## A Portrait Image Generator with Dynamic Styling
### [Anpei Chen*](https://scholar.google.com/citations?user=fuR1FBwAAAAJ&hl=en), [Ruiyang Liu*](http://vic.shanghaitech.edu.cn/vrvc/en/people/ruiyang-liu/), [Ling Xie](https://scholar.google.com/citations?user=o5AJIyMAAAAJ&hl=en), [Zhang Chen](https://scholar.google.com/citations?user=4MIbSrAAAAAJ&hl=en), [Hao Su](https://cseweb.ucsd.edu/~haosu/), [Jingyi Yu](http://vic.shanghaitech.edu.cn/vrvc/en/people/jingyi-yu/)
Paper: [http://arxiv.org/abs/2007.03780](http://arxiv.org/abs/2007.03780)
Project page: [https://apchenstu.github.io/sofgan/](https://apchenstu.github.io/sofgan/)
Recently, GANs have been widely used for portrait generation. However, in the latent space learned by GANs, different attributes, such as pose, shape, and texture style, are generally entangled, making the explicit control of specific attributes difficult. To address this issue, we propose a SOFGAN image generator to decouple the latent space of portraits into two subspaces: a geometry space and a texture space. The latent codes sampled from the two subspaces are fed to two network branches separately, one to generate the 3D geometry of portraits with canonical pose, and the other to generate textures. The aligned 3D geometries also come with semantic part segmentation, encoded as a semantic occupancy field (SOF). The SOF allows the rendering of consistent 2D semantic segmentation maps at arbitrary views, which are then fused with the generated texture maps and stylized to a portrait photo using our semantic instance-wise (SIW) Module. Through extensive experiments, we show that our system can generate high quality portrait images with independently controllable geometry and texture attributes. The method also generalizes well in various applications such as appearance-consistent facial animation and dynamic styling.
>>>>>>> 7fdcd95945b4cc2be706293c01a8c154fb1dfb72
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent