This repository collects the studies on 3D generation, including both 3D shape generation and 3D-aware image generation. Different from 3D reconstruction, which focuses on per-instance recovery (i.e., the data already exists in the real world), 3D generation targets learning the real distribution and hence allows sampling new data.

Overall, the paper collection is organized as follows. If you find some work is missing, feel free to raise an issue or create a pull request. We appreciate contributions in any form.

• 3D Shape Generation
  • Point Cloud
  • Voxel
  • Mesh
  • Implicit Function
  • Parametric Surface
  • Primitive Shape
  • Hybrid Representation
  • Program
• 3D-aware Image Generation
  • Point Cloud
  • Voxel
  • Depth
  • Implicit Function
  • Hybrid Representation
• 3D Control of 2D Generative Models

We categorize the studies on 3D shape generation according to the representation used.

• Learning Representations and Generative Models for 3D Point Clouds
  ICML 2018 / Code
• Multiresolution Tree Networks for 3D Point Cloud Processing
  ECCV 2018 / Code / Project Page
• 3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions
  ICCV 2019 / Code
• Point Cloud GAN
  ICLR 2019 / Code
• Learning Localized Generative Models for 3D Point Clouds via Graph Convolution
  ICLR 2019 / Code
• PointFlow : 3D Point Cloud Generation with Continuous Normalizing Flows
  ICCV 2019 / Code
• Spectral-GANs for High-Resolution 3D Point-Cloud Generation
  IROS 2020 / Code
• Progressive Point Cloud Deconvolution Generation Network
  ECCV 2020 / Code
• A Progressive Conditional Generative Adversarial Network for Generating Dense and Colored 3D Point Clouds
  3DV 2020 / Code
• SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation
  SIGGRAPH 2021 / Code
• Adversarial Autoencoders for Generating 3D Point Clouds
  ICLR 2020 / Code
• Learning Gradient Fields for Shape Generation
  ECCV 2020 / Code / Project Page
• SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds
  NeurIPS 2020 / Code
• Discrete Point Flow Networks for Efficient Point Cloud Generation
  ECCV 2020 / Code
• Pointgrow: Autoregressively Learned Point Cloud Generation with Self-Attention
  WACV 2020 / Code / Project Page
• MRGAN: MultiRooted 3D Shape Generation with Unsupervised Part Disentanglement
  ICCVW 2021
• Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification
  CVPR 2021 / Code / Project Page

• 3d Shapenets: A Deep Representation for Volumetric Shapes
  CVPR 2015 / Code / Project Page
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
  NeurIPS 2016 / Code / Project Page
• Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
  arXiv 2016 / Code
• Octree Generating Networks: Efficient Convolutional Architectures for High-Resolution 3D Outputs
  ICCV 2017 / Code
• SAGNet: Structure-aware Generative Network for 3D-Shape Modeling
  SIGGRAPH 2019 / Code / Project Page
• Generalized Autoencoder for Volumetric Shape Generation
  CVPRW 2020 / Code
• PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes
  CVPR 2020 / Code
• Learning Part Generation and Assembly for Structure-Aware Shape Synthesis
  AAAI 2020
• Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis
  TPAMI 2020 / Code / Project Page
• DLGAN: Depth-Preserving Latent Generative Adversarial Network for 3D Reconstruction
  TMM 2020
• DECOR-GAN: 3D Shape Detailization by Conditional Refinement
  CVPR 2021 / Code
• Octree Transformer: Autoregressive 3D Shape Generation on Hierarchically Structured Sequences
  arXiv 2021

• SDM-Net: Deep Generative Network for Structured Deformable Mesh
  SIGGRAPH Asia 2019 / Project Page
• PolyGen: An Autoregressive Generative Model of 3D Meshes
  ICML 2020 / Code
• TM-NET: Deep Generative Networks for Textured Meshes
  TOG 2021 / Code / Project Page

• Learning Implicit Fields for Generative Shape Modeling
  CVPR 2019 / Code / Project Page
• Adversarial Generation of Continuous Implicit Shape Representations
  arXiv 2020 / Code
• DualSDF: Semantic Shape Manipulation using a Two-Level Representation
  CVPR 2020 / Code / Project Page
• SurfGen: Adversarial 3D Shape Synthesis with Explicit Surface Discriminators
  ICCV 2021
• 3D Shape Generation with Grid-Based Implicit Functions
  CVPR 2021
• gDNA: Towards Generative Detailed Neural Avatars
  CVPR 2022 / Code / Project Page
• Deformed Implicit Field: Modeling 3D shapes with Learned Dense Correspondence
  CVPR 2021 / Code

• Multi-Chart Generative Surface Modeling
  SIGGRAPH Asia 2018 / Code

• Physically-Aware Generative Network for 3D Shape Modeling
  CVPR 2021

• Coupling Explicit and Implicit Surface Representations for Generative 3D Modeling
  ECCV 2020
• Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis
  NeurIPS 2021 / Project Page

• ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis
  SIGGRAPH Asia 2020 / Code / Project Page

We categorize the studies on 3D-aware image generation according to the representation used.

• Points2Pix: 3D Point-Cloud to Image Translation using conditional Generative Adversarial Networks
  arXiv 2019

• Visual Object Networks: Image Generation with Disentangled 3D Representation
  NeurIPS 2018 / Code / Project Page
• HoloGAN: Unsupervised Learning of 3D representations from Natural Images
  ICCV 2019 / Code
• BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images
  NeurIPS 2020 / Code
• Towards a Neural Graphics Pipeline for Controllable Image Generation
  Computer Graphics Forum 2021 / Project Page

• Generative Image Modeling using Style and Structure Adversarial Networks
  ECCV 2016 / Code
• Geometric Image Synthesis
  ACCV 2018
• RGBD-GAN: Unsupervised 3D Representation Learning From Natural Image Datasets via RGBD Image Synthesis
  ICLR 2020 / Code
• 3D-Aware Indoor Scene Synthesis with Depth Priors
  arXiv 2022 / Code / Project Page

• GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
  NeurIPS 2020 / Code / Project Page
• GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields
  CVPR 2021 / Code / Project Page
• pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
  CVPR 2021 / Code / Project Page
• Unconstrained Scene Generation with Locally Conditioned Radiance Fields
  ICCV 2021 / Code / Project Page
• A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis
  NeurIPS 2021 / Code / Project Page
• Campari: Camera-aware Decomposed Generative Neural Radiance Fields
  3DV 2021 / Code
• CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
  arXiv 2021 / Code
• GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds
  ICCV 2021 / Code / Project Page
• Generative Occupancy Fields for 3D Surface-Aware Image Synthesis
  NeurIPS 2021 / Code / Project Page
• 3D-Aware Semantic-Guided Generative Model for Human Synthesis
  arXiv 2021
• FENeRF: Face Editing in Neural Radiance Fields
  CVPR 2022
• StyleNeRF: A Style-Based 3D-Aware Generator for High-resolution Image Synthesis
  ICLR 2022 / Code / Project Page
• StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation
  CVPR 2022 / Code / Project Page
• GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation
  CVPR 2022 / Project Page
• A Generative Model for 3D Face Synthesis with HDRI Relighting
  arXiv 2022
• Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation
  CVPR 2022
• 3D-GIF: 3D-Controllable Object Generation via Implicit Factorized Representations
  arXiv 2022

• 3D-aware Image Synthesis via Learning Structural and Textural Representations
  CVPR 2022 / Code / Project Page
• Efficient Geometry-aware 3D Generative Adversarial Networks
  CVPR 2022 / Code / Project Page

Besides explicitly learning a 3D generative model, there are also some attempts working on the 3D controllability of 2D models.

• Interpreting the latent space of gans for semantic face editing
  CVPR 2020 / Code / Project Page
• StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
  CVPR 2020 / Project Page
• Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
  CVPR 2020 / Code
• FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold
  SIGGRAPH Asia 2021 / Code / Project Page
• Do 2D GANs Know 3D Shape? Unsupervised 3D Shape Reconstruction from 2D Image GANs
  ICLR 2021 / Code / Project Page
• Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
  IJCV 2021 / Code / Project Page
• Cross-Domain and Disentangled Face Manipulation with 3D Guidance
  TVCG 2021 / Code / Project Page
• Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
  ICLR 2021 / Project Page
• Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
  TOG 2021 / Code / Project Page