NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
- Install basic required packages.
git clone https://github.com/liuyuan-pal/NeRO.git
cd NeRO
pip install -r requirements.txt- Install
nvdiffrast. Please follow instructions here https://nvlabs.github.io/nvdiffrast/#installation. - Install
raytracing. Please follow instructions here https://github.com/ashawkey/raytracing.
Models and datasets all can be found here.
- In the
NeROdirectory, ensure that you have the following data:
NeRO
|-- data
|-- GlossyReal
|-- bear
...
|-- GlossySynthetic
|-- bell
...
- Run the training script
# reconstructing the "bell" of the Glossy Synthetic dataset
python run_training.py --cfg configs/shape/syn/bell.yaml
# reconstructing the "bear" of the Glossy Real dataset
python run_training.py --cfg configs/shape/real/bear.yamlIntermediate results will be saved at data/train_vis. Models will be saved at data/model.
- Extract mesh from the model.
python extract_mesh.py --cfg configs/shape/syn/bell.yaml
python extract_mesh.py --cfg configs/shape/real/bear.yamlThe extracted meshes will be saved at data/meshes.
- In the
NeROdirectory, ensure that you have the following data:
NeRO
|-- data
|-- GlossyReal
|-- bear
...
|-- GlossySynthetic
|-- bell
...
|-- meshes
| -- bell_shape-300000.ply
| -- bear_shape-300000.ply
...
- Run the training script:
# estimate BRDF of the "bell" of the Glossy Synthetic dataset
python run_training.py --cfg configs/material/syn/bell.yaml
# estimate BRDF of the "bear" of the Glossy Real dataset
python run_training.py --cfg configs/material/real/bear.yamlIntermediate results will be saved at data/train_vis. Models will be saved at data/model.
- Extract materials from the model.
python extract_materials.py --cfg configs/material/syn/bell.yaml
python extract_materials.py --cfg configs/material/real/bear.yamlThe extracted materials will be saved at data/materials.
- In the
NeROdirectory, ensure that you have the following data:
NeRO
|-- data
|-- GlossyReal
|-- bear
...
|-- GlossySynthetic
|-- bell
...
|-- meshes
| -- bell_shape-300000.ply
| -- bear_shape-300000.ply
...
|-- materials
| -- bell_material-100000
| -- albedo.npy
| -- metallic.npy
| -- roughness.npy
| -- bear_material-100000
| -- albedo.npy
| -- metallic.npy
| -- roughness.npy
|-- hdr
| -- neon_photostudio_4k.exr
- Run relighting script
python relight.py --blender <path-to-your-blender> \
--name bell-neon \
--mesh data/meshes/bell_shape-300000.ply \
--material data/materials/bell_material-100000 \
--hdr data/hdr/neon_photostudio_4k.exr \
--trans
python relight.py --blender <path-to-your-blender> \
--name bear-neon \
--mesh data/meshes/bear_shape-300000.ply \
--material data/materials/bear_material-100000 \
--hdr data/hdr/neon_photostudio_4k.exrThe relighting results will be saved at data/relight with the directory name of bell-neon or bear-neon. This command means that we use neon_photostudio_4k.exr to relight the object.
Refer to custom_object.md.
Todo: Under construction.
In this repository, we have used codes from the following repositories. We thank all the authors for sharing great codes.
@inproceedings{liu2023nero,
title={NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images},
author={Liu, Yuan and Wang, Peng and Lin, Cheng and Long, Xiaoxiao and Wang, Jiepeng and Liu, Lingjie and Komura, Taku and Wang, Wenping},
booktitle={SIGGRAPH},
year={2023}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent