Giter Club home page Giter Club logo

plenoctrees_nerf-sh's Introduction

PlenOctrees_NeRF-SH

This is an implementation of the Paper PlenOctrees for Real-time Rendering of Neural Radiance Fields. Not only the code provides the implementation of the NeRF-SH,but also provides the conversion code from NeRF-SH to PlenOctree. You can use the code to generate the .npz file so as to run the C++ renderer by the PlenOctrees for Real-time Rendering of Neural Radiance Fields. And the conversion code is in the tools/PlenOctrees.ipynb. The results by our code is shown in the Screenshot But before using the code, you must train the NeRF-SH model. If you don't want to train the model, please contact the mail:[email protected].

Quick Start

The implementation of dataloader is from the Multi-view Neural Human Rendering (NHR). So the datasets format should be the same as theNHR.
To train the code:

    
cd tools && python train_net.py <gpu id>     

And you can run the tools/PlenOctrees.ipynb to generate the .npz file which can run the C++ renderer by the PlenOctrees for Real-time Rendering of Neural Radiance Fields.

trained model

We provide a trained model about NeRF-SH. And The trained model about scene lego is stored in here. And you can also download the lego3.npz generated by the tools/PlenOctrees.ipynb.

Requirements

  • yacs (Yet Another Configuration System)
  • PyTorch (An open source deep learning platform)
  • ignite (High-level library to help with training neural networks in PyTorch)

If you have any questions, you can contact [email protected].

Citation

@inproceedings{yu2021plenoctrees,
      title={PlenOctrees for Real-time Rendering of Neural Radiance Fields},
      author={Alex Yu and Ruilong Li and Matthew Tancik and Hao Li and Ren Ng and Angjoo Kanazawa},
      year={2021},
      booktitle={arXiv},
}

plenoctrees_nerf-sh's People

Contributors

suoxinkey avatar walnut-ree avatar wuminye avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

plenoctrees_nerf-sh's Issues

Training Problem

I made the Lego dataset from MetaShape,and it has the same structure as requested.

but When extracting the .npz file from the trained results using PlenOctree.ipynb, it appears as a black mess.
I would like to know if there is a way to solve this problem

image

Problem with mask and background

Hi,

I tried to train with sport_1 dataset, with below config, but it seems "USE_MASK" option doesn't work and I still see the background. Any idea what could be the problem?

SOLVER:
OPTIMIZER_NAME: "Adam"
BASE_LR: 0.0005
WEIGHT_DECAY: 0.0000000
IMS_PER_BATCH: 2
START_ITERS: 3000
END_ITERS: 60000
LR_SCALE: 0.1
WARMUP_ITERS: 0

MAX_EPOCHS: 100
CHECKPOINT_PERIOD: 3000
LOG_PERIOD: 30
BUNCH: 1000
COARSE_STAGE: 0

INPUT:
SIZE_TRAIN: [256,192]
SIZE_TEST: [256,192]

DATASETS:
TRAIN: "/home/Downloads/sport_1"
NUM_FRAME: 1
SHIFT: 0.0
MAXRATION: 0.0
ROTATION: 0.0
USE_MASK: True

DATALOADER:
NUM_WORKERS: 8

MODEL:
COARSE_RAY_SAMPLING: 64
FINE_RAY_SAMPLING: 128
SAMPLE_METHOD: "NEAR_FAR" # "NEAR_FAR" "BBOX"
BOARDER_WEIGHT: 1e10
SAME_SPACENET: True
TKERNEL_INC_RAW: True
USE_SH: True
DEVICE_IDS: [0]

TEST:
IMS_PER_BATCH: 1

OUTPUT_DIR: "/media/Data_DL_HD/projects/PlenOctrees_NeRF-SH/output/sport_1"

Screenshot-20210416220214-1149x880

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.