MINER: Multiscale Implicit Neural Representation
Abstract: We present a novel implicit representation framework called MINER that is well suited for tasks such as fitting very high resolution point clouds ovre multiple levels of detail (LoD). This figure demonstrates fitting of the Lucy 3D mesh over five spatial scales. MINER takes less than half an hour to achieve an intersection over union (IoU) of 0.999 when trained over more than a billion 3D points.
Paper: https://arxiv.org/abs/2202.03532
Instructions
Data
Download image and occupancy examples from here. Place it in path/to/this/folder/data/
.
Requirements
check requirements.txt
. This code was tested on python 3.8 in both Windows and Linux environments. Requirements file generated in Linux environment but should work similarly in a Windows environment. Memory consumption statistics work only in Linux environment.
Running MINER on images
Please check miner_image.py
for fitting an image. We have included an example image in data/pluto.png
which is a modified image from the wiki page on Pluto (zero-padded to 8192x8192).
Running MINER on volumes
Please check miner_pointcloud.py
for fitting a 3D shape with occupancy information. For simplicity, we have included occupancy volume of Lucy with regular sampling over 512x512x512 cube.
Information about individual files
-
miner_image.py
: Runs training script for a single image -
miner_pointcloud.py
: Runs training script for a uniformly sampled occupancy of a 3D shape -
requirements.txt
: All requirements for running scripts in this folder. -
modules
: Contains functions to run the training scripts:a.
folding_utils.py
: Fast script for converting image/pointcloud into batches. This version assumes no overlap of patches and hence is much faster than default torch foldb.
losses.py
: Loss functions for trainingc.
miner.py
: Functions required to train images and occupancies with MINER.d.
utils.py
: Miscellaneous utilitiese.
volutils.py
: Utilities for handling volume signals
Cite
@inproceedings{saragadam2022miner,
title={MINER: Multiscale Implicit Neural Representations},
author={Saragadam, Vishwanath and Tan, Jasper and Balakrishnan, Guha and Baraniuk, Richard G and Veeraraghavan, Ashok},
booktitle={European Conf. Computer Vision (ECCV)}
year={2022}
}