dSRVAE (Generative Variational AutoEncoder for Real Image Super-Resolution)

By Zhi-Song Liu, Li-Wen Wang, Chu-Tak Li, Marie-Paule Cani and Wan-Chi Siu

This repo only provides simple testing codes, pretrained models and the network strategy demo.

We propose a joint image denoising and Super-Resolution model by using generative Variational AutoEncoder (dSRVAE)

We participate CVPRW NTIRE2020 Real Image Super-Resolution Challenge

Please check our paper

BibTex

    @InProceedings{Liu2020dsrvae,
        author = {Zhi-Song Liu, Wan-Chi Siu and Li-Wen Wang and Chu-Tak Li and Marie-Paule Cani and Yui-Lam Chan},
        title = {Unsupervised Real Image Super-Resolution via Generative Variational AutoEncoder},
        booktitle = {IEEE International Conference on Computer Vision and Pattern Recognition Workshop(CVPRW)},
        month = {June},
        year = {2020}
    }

For proposed dSRVAE model, we claim the following points:

• First working on using Variational AutoEncoder for image denoising.

• Then the Super-Resolution Sub-Network (SRSN) is attached as a small overhead to the DAE which forms the proposed dSRVAE to output super-resolved images.

Dependencies

Python > 3.0
OpenCV library
Pytorch > 1.0
NVIDIA GPU + CUDA
pytorch-gan-metrics

Complete Architecture

The complete architecture is shown as follows,

Implementation

1. Quick testing

Download pre-trained models from

https://drive.google.com/open?id=1SUZGE04vw5_yDYiw6PJ4sbHAOIEV6TJ7

and copy them to the folder "models"

Copy your image to folder "Test" and run

$ python test.py

The SR images will be in folder "Result" 3. For self-ensemble, run

$ python test_enhance.py

GAN feature evaluation

# download statistics.npz from http://bioinf.jku.at/research/ttur/
from pytorch_gan_metrics import get_inception_score, get_fid

images = ... # [N, 3, H, W] normalized to [0, 1]
IS, IS_std = get_inception_score(images)        # Inception Score
FID = get_fid(images, 'path/to/statistics.npz') # Frechet Inception Distance