Implicit Functions in Feature Space for Shape Reconstruction and Completion
Julian Chibane, Thiemo Alldieck, Gerard Pons-Moll

Paper - Supplementaty - Project Website - Arxiv - Video - Slides - Published in CVPR 2020.

@inproceedings{chibane20ifnet,
    title = {Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion},
    author = {Chibane, Julian and Alldieck, Thiemo and Pons-Moll, Gerard},
    booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {jun},
    organization = {{IEEE}},
    year = {2020},
}

A linux system with cuda 9.0 is required for the project.

The if-net_env.yml file contains all necessary python dependencies for the project. To conveniently install them automatically with anaconda you can use:

conda env create -f if-net_env.yml
conda activate if-net

Please clone the repository and navigate into it in your terminal, its location is assumed for all subsequent commands.

This project uses libraries for Occupancy Networks by [Mescheder et. al. CVPR'19] and the ShapeNet data preprocessed for DISN by [Xu et. al. NeurIPS'19], please also cite them if you use our code.

Install the needed libraries with:

cd data_processing/libmesh/
python setup.py build_ext --inplace
cd ../libvoxelize/
python setup.py build_ext --inplace
cd ../..

Download the ShapeNet data preprocessed by [Xu et. al. NeurIPS'19] from here into the shapenet folder.

Now extract the files into shapenet\data with:

ls shapenet/*.tar.gz |xargs -n1 -i tar -xf {} -C shapenet/data/

Next, the inputs and training point samples for IF-Nets are created. The following three commands can be run in parallel on multiple machines to significantly increase speed. First, the data is converted to the .off-format and scaled using

python data_processing/convert_to_scaled_off.py

The input data for Voxel Super-Resolution of voxels is created with

python data_processing/voxelize.py -res 32

using -res 32 for 32³ and -res 128 for 128³ resolution.

The input data for Point Cloud Completion is created with

python data_processing/voxelized_pointcloud_sampling.py -res 128 -num_points 300

using -num_points 300 for point clouds with 300 points and -num_points 3000 for 3000 points.

Training input points and the corresponding ground truth occupancy values are generated with

python data_processing/boundary_sampling.py -sigma 0.1
python data_processing/boundary_sampling.py -sigma 0.01

where -sigma specifies the standard deviation of the normally distributed displacements added onto surface samples.

In order to remove meshes that could not be preprocessed (should not be more than around 15 meshes) you should run

python data_processing/filter_corrupted.py -file 'voxelization_32.npy' -delete

The input data can be visualized by converting them to .off-format using

python data_processing/create_voxel_off.py -res 32

for voxel input and

python data_processing/create_pc_off.py -res 128 -num_points 300

where -res and -num_points matches the values from the previous steps.

The training of IF-Nets is started running

python train.py -std_dev 0.1 0.01 -res 32 -m ShapeNet32Vox -batch_size 6

where -std_dev indicates the sigmas to use, -res the input resolution (32³ or 128³), -m the IF-Net model setup

• ShapeNet32Vox for 32³ voxel Super-Resolution experiment
• ShapeNet128Vox for 128³ voxel Super-Resolution experiment
• ShapeNetPoints for Point Cloud Completion experiments
• SVR for 3D Single-View Reconstruction

and -batch_size the number of different meshes inputted in a batch, each with 50.000 point samples (=6 for small GPU's). Consider using the highest possible batch_size in order to speed up training.

In the experiments/ folder you can find an experiment folder containing the model checkpoints, the checkpoint of validation minimum, and a folder containing a tensorboard summary, which can be started at with

tensorboard --logdir experiments/YOUR_EXPERIMENT/summary/ --host 0.0.0.0

The command

python generate.py -std_dev 0.1 0.01 -res 32 -m ShapeNet32Vox -checkpoint 10 -batch_points 400000

generates the reconstructions of the, during training unseen, test examples from ShapeNet into the folder experiments/YOUR_EXPERIMENT/evaluation_CHECKPOINT_@256/generation. With -checkpoint you can choose the IF-Net model checkpoint. Use the model with minimum validation error for this, -batch_points indicates the number of points that fit into GPU memory at once (400k for small GPU's), the other parameters are set as during training.

The generation script can be run on multiple machines in parallel in order to increase generation speed significantly. Also, consider using the maximal batch size possible for your GPU.

Please run

python data_processing/evaluate.py -reconst -generation_path experiments/iVoxels_dist-0.5_0.5_sigmas-0.1_0.01_v32_mShapeNet32Vox/evaluation_10_@256/generation/

to evaluate each reconstruction, where -generation_path is the path to the reconstructed objects generated in the previous step.

The above evaluation script can be run on multiple machines in parallel in order to increase generation speed significantly.

Then run

python data_processing/evaluate.py -voxels -res 32

to evaluate the quality of the input. For voxel girds use '-voxels' with '-res' to specify the input resolution and for point clouds use '-pc' with '-points' to specify the number of points.

The quantitative evaluation of all reconstructions and inputs are gathered and put into experiment/YOUR_EXPERIMENT/evaluation_CHECKPOINT_@256 using

python data_processing/evaluate_gather.py -voxel_input -res 32 -generation_path experiments/iVoxels_dist-0.5_0.5_sigmas-0.1_0.01_v32_mShapeNet32Vox/evaluation_10_@256/generation/

where you should use -voxel_input for Voxel Super-Resolution experiments, with -res specifying the input resolution or -pc_input for Point Cloud Completion, with -points specifying the number of points used.

For questions and comments regarding the code please contact Julian Chibane via mail. (See Paper)