Zhang Chen¹, Zhong Li¹, Liangchen Song², Lele Chen¹, Jingyi Yu³, Junsong Yuan², Yi Xu^1
1OPPO US Research Center, ²University at Buffalo, ³ShanghaiTech University

This repo is an official PyTorch implementation for the ICCV 2023 paper "NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functions". Our work presents a novel type of neural fields with high representation accuracy and model compactness. The repo contains the codes for image fitting, SDF fitting and neural radiance fields.

conda create -n neurbf python=3.9 -y
conda activate neurbf

pip install cupy-cuda117
python -m cupyx.tools.install_library --cuda 11.7 --library cutensor
python -m cupyx.tools.install_library --cuda 11.7 --library cudnn
python -m cupyx.tools.install_library --cuda 11.7 --library nccl

pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu117

pip install einops matplotlib kornia imageio imageio-ffmpeg opencv-python pysdf PyMCubes trimesh plotly scipy GPUtil scikit-image scikit-learn pykdtree commentjson tqdm configargparse lpips tensorboard torch-ema ninja tensorboardX numpy pandas rich packaging scipy torchmetrics jax pillow plyfile omegaconf

cd thirdparty/torch_ngp/gridencoder
pip install .
cd ../../../

• Ubuntu 20.04 with PyTorch 1.13.0 & CUDA 11.6 on RTX A6000.
• Ubuntu 20.04 with PyTorch 2.0.1 & CUDA 11.7 on RTX A6000.

Download an example pluto image. Put it in data/img and rename to pluto.png. Then run

python main.py --config configs/img.py --path ./data/img/pluto.png --alias pluto

The result and tensorboard log will be located in log/img.

Download the validation set of the DIV2K dataset and put it in data/img/div2k. The path to each image should be data/img/div2k/DIV2K_valid_HR/xxxx.png. Then run

python main_img_div2k.py

The script will fit each image separately and the results will be stored in log/img_div2k.

Download an example armadillo mesh of the Stanford 3D Scanning Repository. Unzip it, put it in data/sdf, and rename to armadillo.ply.

Run the following preprocessing script, which normalizes the mesh and sample additional evaluation points.

python preproc_mesh.py --path ./data/sdf/armadillo.ply

Then run

python main.py --config configs/sdf.py --path ./data/sdf/armadillo_nrml.obj --alias armadillo

The result and tensorboard log will be located in log/sdf.

• Synthetic NeRF

Download the dataset and unzip to data. For example, the path to the lego scene should be data/nerf_synthetic/lego.

For the Synthetic NeRF dataset, use the following command

python main_nerf.py --config_init configs/nerf_tensorf/nerf_synthetic_init.py --config configs/nerf_tensorf/nerf_synthetic.py --data_name lego

It will first distill scene information to initialize RBF position and shape parameters, and then train the full model. The result and tensorboard log will be located in log/nerf_synthetic. Change --data_name to run on other scenes.

If you find our work useful, please consider citing:

@inproceedings{chen2023neurbf,
    title={NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functions},
    author={Chen, Zhang and Li, Zhong and Song, Liangchen and Chen, Lele and Yu, Jingyi and Yuan, Junsong and Xu, Yi},
    booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
    year={2023}
}