Delving deep into Generative Adversarial Networks (GANs)

A curated list of state-of-the-art publications and resources about Generative Adversarial Networks (GANs) and their applications.

Overview

Generative models are models that can learn to create data that is similar to data that we give them. One of the most promising approaches of those models are Generative Adversarial Networks (GANs), a branch of unsupervised machine learning implemented by a system of two neural networks competing against each other in a zero-sum game framework. They were first introduced by Ian Goodfellow et al. in 2014. This repository aims at presenting an elaborate list of the state-of-the-art works on the field of Generative Adversarial Networks since their introduction in 2014.

Image taken from http://multithreaded.stitchfix.com/blog/2016/02/02/a-fontastic-voyage/

This is going to be an evolving repo and I will keep updating it (at least twice monthly) so make sure you have starred and forked this repository before moving on !

👥 Contributing

Contributions are welcome !! If you have any suggestions (missing or new papers, missing repos or typos) you can pull a request or start a discussion.

📌 Opening Publication

Generative Adversarial Nets (GANs) (2014) [pdf] [presentation] [code] [video]

📋 State-of-the-art papers (Descending order based on Google Scholar Citations - number in brackets denotes the current number of citations)

1. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (DCGANs) (2015) [pdf] [326]
2. Explaining and Harnessing Adversarial Examples (2014) [pdf] [240]
3. 🔼 Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks (LAPGAN) (2015) [pdf] [227]
4. 🔼 Semi-Supervised Learning with Deep Generative Models (2014) [pdf] [217]
5. 🔼 Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks (MGAN) (2016) [pdf] [100]
6. Improved Techniques for Training GANs (2016) [pdf] [99]
7. Conditional Generative Adversarial Nets (CGAN) (2014) [pdf] [99]
8. 🔼 Context Encoders: Feature Learning by Inpainting (2016) [pdf] [75]
9. 🔼 Deep multi-scale video prediction beyond mean square error (2015) [pdf] [72]
10. 🔼 Generative Adversarial Text to Image Synthesis (2016) [pdf] [69]
11. Autoencoding beyond pixels using a learned similarity metric (VAE-GAN) (2015) [pdf] [65]
12. Adversarial Autoencoders (2015) [pdf] [65]
13. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (2016) [pdf] [65]
14. 🔼 Generative Moment Matching Networks (2015) [pdf] [61]
15. Energy-based Generative Adversarial Network (EBGAN) (2016) [pdf] [51]
16. Conditional Image Generation with PixelCNN Decoders (2015) [pdf] [50]
17. Generating Images with Perceptual Similarity Metrics based on Deep Networks (2016) [pdf] [45]
18. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (SRGAN) (2016) [pdf] [44]
19. 🔼 Adversarial Feature Learning (BiGAN) (2016) [pdf] [42]
20. 🔼 Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples (2016) [pdf] [39]
21. 🔼 Generative Visual Manipulation on the Natural Image Manifold (iGAN) (2016) [pdf] [39]
22. Improving Variational Inference with Inverse Autoregressive Flow (2016) [pdf] [37]
23. 🔼 Wasserstein GAN (WGAN) (2017) [pdf] [36]
24. Generative Image Modeling using Style and Structure Adversarial Networks (S^2GAN) (2016) [pdf] [35]
25. Image-to-Image Translation with Conditional Adversarial Networks (pix2pix) (2016) [pdf] [35]
26. 🔼 Adversarially Learned Inference (ALI) (2016) [pdf] [35]
27. 🆕 Conditional generative adversarial nets for convolutional face generation(2014) [pdf] [33]
28. Unsupervised Learning for Physical Interaction through Video Prediction (2016) [pdf] [32]
29. f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization (2016) [pdf] [32]
30. Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks (CatGAN) (2015) [pdf] [31]
31. Generating images with recurrent adversarial networks (2016) [pdf] [31]
32. Attend, infer, repeat: Fast scene understanding with generative models (2016) [pdf] [30]
33. Training generative neural networks via Maximum Mean Discrepancy optimization (2015) [pdf] [29]
34. Generating Videos with Scene Dynamics (VGAN) (2016) [pdf] [29]
35. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks (2016) [pdf] [22]
36. 🔼 Coupled Generative Adversarial Networks (CoGAN) (2016) [pdf] [21]
37. StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks (2016) [pdf] [19]
38. 🔼 SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient (2016) [pdf] [18]
39. Semantic Image Inpainting with Perceptual and Contextual Losses (2016) [pdf] [17]
40. Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space (PPGN) (2016) [pdf] [17]
41. Generative Adversarial Imitation Learning (2016) [pdf] [17]
42. Unsupervised Cross-Domain Image Generation (DTN) (2016) [pdf] [16]
43. Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling (3D-GAN) (2016) [pdf] [14]
44. Pixel-Level Domain Transfer (2016) [pdf] [13]
45. Learning What and Where to Draw (GAWWN) (2016) [pdf] [10]
46. Conditional Image Synthesis with Auxiliary Classifier GANs (AC-GAN) (2016) [pdf] [10]
47. Amortised MAP Inference for Image Super-resolution (AffGAN) (2016) [pdf] [10]
48. 🆕 Full Resolution Image Compression with Recurrent Neural Networks(2016) [pdf] [10]
49. Learning in Implicit Generative Models (2016) [pdf] [9]
50. VIME: Variational Information Maximizing Exploration (2016) [pdf] [9]
51. Unrolled Generative Adversarial Networks (Unrolled GAN) (2016) [pdf] [9]
52. Towards Principled Methods for Training Generative Adversarial Networks (2017) [pdf] [9]
53. Semantic Segmentation using Adversarial Networks (2016) [pdf] [9]
54. Neural Photo Editing with Introspective Adversarial Networks (IAN) (2016) [pdf] [8]
55. 🆕 Mode Regularized Generative Adversarial Networks (2016) [pdf] [8]
56. 🆕 Learning a Driving Simulator(2016) [pdf] [7]
57. 🆕 Learning to Protect Communications with Adversarial Neural Cryptography(2016) [pdf] [7]
58. On the Quantitative Analysis of Decoder-Based Generative Models (2016) [pdf] [6]
59. 🆕 Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro(2017) [pdf] [6]
60. 🆕 Cooperative Training of Descriptor and Generator Network (2016) [pdf] [5]
61. Connecting Generative Adversarial Networks and Actor-Critic Methods (2016) [pdf] [4]
62. Learning from Simulated and Unsupervised Images through Adversarial Training (SimGAN) by Apple (2016) [pdf] [4]
63. Stacked Generative Adversarial Networks (SGAN) (2016) [pdf] [4]
64. ArtGAN: Artwork Synthesis with Conditional Categorial GANs (2017) [pdf] [4]
65. GP-GAN: Towards Realistic High-Resolution Image Blending (2017) [pdf] [4]
66. LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation (2017) [pdf] [3]
67. Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery (AnoGAN) (2017) [pdf] [3]
68. Temporal Generative Adversarial Nets (TGAN) (2016) [pdf] [3]
69. Invertible Conditional GANs for image editing (IcGAN) (2016) [pdf] [3]
70. Contextual RNN-GANs for Abstract Reasoning Diagram Generation (Context-RNN-GAN) (2016) [pdf] [3]
71. Generative Adversarial Nets with Labeled Data by Activation Maximization (AMGAN) (2017) [pdf] [3]
72. 🆕 Imitating Driver Behavior with Generative Adversarial Networks(2017) [pdf] [3]
73. MAGAN: Margin Adaptation for Generative Adversarial Networks (2017) [pdf] [2]
74. CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training (2017) [pdf] [2]
75. Multi-Agent Diverse Generative Adversarial Networks (MAD-GAN) (2017) [pdf] [2]
76. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (CycleGAN) (2017) [pdf] [2]
77. Learning to Discover Cross-Domain Relations with Generative Adversarial Networks (DiscoGAN)(2017) [pdf] [2]
78. DualGAN: Unsupervised Dual Learning for Image-to-Image Translation (2017) [pdf] [2]
79. Image De-raining Using a Conditional Generative Adversarial Network (ID-CGAN) (2017) [pdf] [2]
80. C-RNN-GAN: Continuous recurrent neural networks with adversarial training (2016) [pdf] [2]
81. Generative Multi-Adversarial Networks (2016) [pdf] [2]
82. Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts (AL-CGAN) (2016) [pdf] [2]
83. BEGAN: Boundary Equilibrium Generative Adversarial Networks (2017) [pdf] [2]
84. Boundary-Seeking Generative Adversarial Networks (BS-GAN) (2017) [pdf] [2]
85. SEGAN: Speech Enhancement Generative Adversarial Network (2017) [pdf] [2]
86. SeGAN: Segmenting and Generating the Invisible (2017) [pdf] [2]
87. Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities (LS-GAN) (2017) [pdf] [2]
88. AdaGAN: Boosting Generative Models (2017) [pdf] [2]
89. Unsupervised Image-to-Image Translation with Generative Adversarial Networks (2017) [pdf] [2]
90. Robust LSTM-Autoencoders for Face De-Occlusion in the Wild (2016) [pdf] [2]
91. Disentangled Representation Learning GAN for Pose-Invariant Face Recognition (2017) [pdf] [2]
92. 🆕 Adversarial Discriminative Domain Adaptation (2017) [pdf] [2]
93. 🆕 Generalization and Equilibrium in Generative Adversarial Nets (GANs) (2017) [pdf] [2]
94. 🆕 Inverting The Generator Of A Generative Adversarial Network (2016) [pdf] [2]
95. Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN (MalGAN) (2016) [pdf] [1]
96. Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks (SSL-GAN) (2016) [pdf] [1]
97. Ensembles of Generative Adversarial Networks (2016) [pdf] [1]
98. Improved generator objectives for GANs (2017) [pdf] [1]
99. Precise Recovery of Latent Vectors from Generative Adversarial Networks (2016) [pdf] [1]
100. Least Squares Generative Adversarial Networks (LSGAN) (2017) [pdf] [1]
101. McGan: Mean and Covariance Feature Matching GAN (2017) [pdf] [1]
102. Generalization and Equilibrium in Generative Adversarial Nets (MIX+GAN) (2016) [pdf] [1]
103. 3D Shape Induction from 2D Views of Multiple Objects (PrGAN) (2016) [pdf] [1]
104. Adversarial Training For Sketch Retrieval (SketchGAN) (2016) [pdf] [1]
105. RenderGAN: Generating Realistic Labeled Data (2016) [pdf] [1]
106. Texture Synthesis with Spatial Generative Adversarial Networks (SGAN) (2016) [pdf] [1]
107. SAD-GAN: Synthetic Autonomous Driving using Generative Adversarial Networks (2017) [pdf] [1]
108. Message Passing Multi-Agent GANs (MPM-GAN) (2017) [pdf] [1]
109. Improved Training of Wasserstein GANs (WGAN-GP) (2017) [pdf] [1]
110. Deep and Hierarchical Implicit Models (Bayesian GAN) (2017) [pdf] [1]
111. A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection (2017) [pdf] [1]
112. 🆕 Maximum-Likelihood Augmented Discrete Generative Adversarial Networks (2017) [pdf] [1]
113. 🆕 Simple Black-Box Adversarial Perturbations for Deep Networks (2016) [pdf] [1]
114. Generative Mixture of Networks (2017) [pdf] [0]
115. Generative Temporal Models with Memory (2017) [pdf] [0]
116. Stopping GAN Violence: Generative Unadversarial Networks (2016) [pdf] [0]
117. Gang of GANs: Generative Adversarial Networks with Maximum Margin Ranking (GoGAN) (2017) [pdf] [0]
118. Deep Unsupervised Representation Learning for Remote Sensing Images (MARTA-GAN) (2017) [pdf] [0]
119. Generating Multi-label Discrete Electronic Health Records using Generative Adversarial Networks (MedGAN) (2017) [pdf] [0]
120. Semi-Latent GAN: Learning to generate and modify facial images from attributes (SL-GAN) (2017) [pdf] [0]
121. TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network (2017) [pdf] [0]
122. Triple Generative Adversarial Nets (Triple-GAN) (2017) [pdf] [0]
123. Image Generation and Editing with Variational Info Generative Adversarial Networks (ViGAN) (2016) [pdf] [0]
124. Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis (TP-GAN) (2017) [pdf] [0]
125. Generative Adversarial Networks as Variational Training of Energy Based Models (VGAN) (2017) [pdf] [0]
126. SalGAN: Visual Saliency Prediction with Generative Adversarial Networks (2016) [pdf] [0]
127. WaterGAN: Unsupervised Generative Network to Enable Real-time Color Correction of Monocular Underwater Images (2017) [pdf] [0]
128. Multi-view Generative Adversarial Networks (MV-BiGAN) (2017) [pdf] [0]
129. Recurrent Topic-Transition GAN for Visual Paragraph Generation (RTT-GAN) (2017) [pdf] [0]
130. Generative face completion (2016) [pdf] [0]
131. MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation using 1D and 2D Conditions (2016) [pdf] [0]
132. Multi-View Image Generation from a Single-View (2016) [pdf] [0]
133. Towards Large-Pose Face Frontalization in the Wild (2016) [pdf] [0]
134. Adversarial Training Methods for Semi-Supervised Text Classification (2016) [pdf] [0]
135. 🆕 An Adversarial Regularisation for Semi-Supervised Training of Structured Output Neural Networks (2017) [pdf] [0]
136. 🆕 Associative Adversarial Networks (2017) [pdf] [0]
137. 🆕 Generative Adversarial Parallelization (2015) [pdf] [0]
138. 🆕 Generative Adversarial Residual Pairwise Networks for One Shot Learning (2017) [pdf] [0]
139. 🆕 Generative Adversarial Structured Networks (2017) [pdf] [0]
140. 🆕 On the effect of Batch Normalization and Weight Normalization in Generative Adversarial Networks(2017) [pdf] [0]
141. 🆕 Softmax GAN(2017) [pdf] [0]
142. 🆕 Adversarial Networks for the Detection of Aggressive Prostate Cancer(2017) [pdf] [0]
143. 🆕 Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation(2017) [pdf] [0]
144. 🆕 Age Progression / Regression by Conditional Adversarial Autoencoder(2017) [pdf] [0]
145. 🆕 Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks(2017) [pdf] [0]
146. 🆕 Generate To Adapt: Aligning Domains using Generative Adversarial Networks(2017) [pdf] [0]
147. 🆕 Outline Colorization through Tandem Adversarial Networks(2017) [pdf] [0]
148. 🆕 Supervised Adversarial Networks for Image Saliency Detection(2017) [pdf] [0]
149. 🆕 Towards Diverse and Natural Image Descriptions via a Conditional GAN(2017) [pdf] [0]
150. 🆕 Reconstruction of three-dimensional porous media using generative adversarial neural networks(2017) [pdf] [0]
151. 🆕 Steganographic Generative Adversarial Networks(2017) [pdf] [0]

📔 Theory

• Improved Techniques for Training GANs [pdf]
• Energy-Based GANs & other Adversarial things by Yann Le Cun [pdf]
• Mode RegularizedGenerative Adversarial Networks [pdf]

🔩 Presentations

• Generative Adversarial Networks (GANs) by Ian Goodfellow [pdf]
• Learning Deep Generative Models by Russ Salakhutdinov [pdf]

📚 Courses / Tutorials / Blogs (Webpages unless other is stated)

• NIPS 2016 Tutorial: Generative Adversarial Networks (2016) [pdf]
• How to Train a GAN? Tips and tricks to make GANs work
• Generative Models by OpenAI
• MNIST Generative Adversarial Model in Keras
• Image Completion with Deep Learning in TensorFlow
• Attacking machine learning with adversarial examples by OpenAI
• On the intuition behind deep learning & GANs?—?towards a fundamental understanding
• SimGANs - a game changer in unsupervised learning, self driving cars, and more

📦 Resources / Models (Descending order based on GitHub stars)

• Image super-resolution through deep learning
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks (CycleGAN)
• Image-to-image translation with conditional adversarial nets (pix2pix)
• Deep Convolutional Generative Adversarial Networks (DCGAN)
• Generative Visual Manipulation on the Natural Image Manifold (iGAN)
• Tensorflow implementation of Deep Convolutional Generative Adversarial Networks (DCGAN)
• Neural Photo Editing with Introspective Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Wasserstein GAN
• Improved Techniques for Training GANs
• cleverhans: A library for benchmarking vulnerability to adversarial examples"
• StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks
• Semantic Image Inpainting with Perceptual and Contextual Losses (2016)
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks (The Eyescream Project)
• Unsupervised Cross-Domain Image Generation
• Generating Videos with Scene Dynamics
• Improved Training of Wasserstein GANs
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• HyperGAN
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (KERAS-DCGAN)
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
• Image-to-image translation using conditional adversarial nets
• Deep multi-scale video prediction beyond mean square error
• Learning from Simulated and Unsupervised Images through Adversarial Training
• Learning What and Where to Draw
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Precomputed real-time texture synthesis with markovian generative adversarial networks
• Unrolled Generative Adversarial Networks
• Adversarially Learned Inference (2016) (ALI)
• Energy-based generative adversarial network
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Generating images with recurrent adversarial networks (sequence_gan)
• A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection
• Pixel-Level Domain Transfer
• Autoencoding beyond pixels using a learned similarity metric
• Sampling Generative Networks
• Invertible Conditional GANs for image editing
• Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
• Generative Image Modeling using Style and Structure Adversarial Networks (ss-gan)
• Adversarial Autoencoders
• SalGAN: Visual Saliency Prediction with Generative Adversarial Networks
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Coupled Generative Adversarial Networks
• Context Encoders: Feature Learning by Inpainting (2016)
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• Generative face completion (2017)
• Conditional Generative Adversarial Nets
• Least Squares Generative Adversarial Networks
• Improving Generative Adversarial Networks with Denoising Feature Matching

🔌 Frameworks & Libraries (Descending order based on GitHub stars)

• Tensorflow by Google [C++ and CUDA]: [homepage] [github]
• Caffe by Berkeley Vision and Learning Center (BVLC) [C++]: [homepage] [github] [Installation Instructions]
• Keras by François Chollet [Python]: [homepage] [github]
• Microsoft Cognitive Toolkit - CNTK [C++]: [homepage] [github]
• MXNet adapted by Amazon [C++]: [homepage] [github]
• Torch by Collobert, Kavukcuoglu & Clement Farabet, widely used by Facebook [Lua]: [homepage] [github]
• Convnetjs by Andrej Karpathy [JavaScript]: [homepage] [github]
• Theano by Université de Montréal [Python]: [homepage] [github]
• Deeplearning4j by startup Skymind [Java]: [homepage] [github]
• Paddle by Baidu [C++]: [homepage] [github]
• 🆕 Caffe2 by Facebook Open Source [C++ & Python]: [github] [web]
• Deep Scalable Sparse Tensor Network Engine (DSSTNE) by Amazon [C++]: [github]
• Neon by Nervana Systems [Python & Sass]: [homepage] [github]
• Chainer [Python]: [homepage] [github]
• h2o [Java]: [homepage] [github]
• Brainstorm by Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA) [Python]: [github]
• Matconvnet by Andrea Vedaldi [Matlab]: [homepage] [github]

License

MIT