I want to test on the KITTI dataset.
I have real data, how do I generate a file like the following：
./f2-nerf/data/example/ngp_fox/cams_meta.npy

How do I get dist_params and bounds ? thanks
Dataset.cpp:

cam_data = cam_data.reshape({n_images_, 27});
Tensor poses = cam_data.slice(1, 0, 12).reshape({-1, 3, 4}).contiguous();

Tensor intri = cam_data.slice(1, 12, 21).reshape({-1, 3, 3}).contiguous();
intri.index_put_({Slc(), Slc(0, 2), Slc(0, 3)}, intri.index({Slc(), Slc(0, 2), Slc(0, 3)}) / factor);

Tensor dist_params = cam_data.slice(1, 21, 25).reshape({-1, 4}).contiguous();   // [k1, k2, p1, p2]
Tensor bounds = cam_data.slice(1, 25, 27).reshape({-1, 2}).contiguous();

Hi,
thanks for publishing code for this great work!
I see that you normalize the camera poses in the init of dataset.
If I want to retrieve point cloud from depth maps from trained scenes is it possible to get it in the original pose before normalization ? so for example the point cloud will be to scale to the original calibration.

use my custom data
python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=outdoor mode=train +work_dir=$(pwd)
Working directory is /cephfs/yuanweizhong/project/f2-nerf
register Hash3DAnchoredInfo
register TCNNWPInfo
register volume render info
Aoligei!
[Dataset::Dataset] begin
Load render poses
[LoadImages] begin
[LoadImages] end in 17.7646 seconds
Number of train/test/val images: 1822/261/0
[Dataset::Dataset] end in 35.4331 seconds
[PersSampler::PersSampler] begin
[PersOctree::PersOctree] begin

[PersOctree::ConstructEdgePool] begin
edge_pool_.size() is value 9702
[PersOctree::ConstructEdgePool] end in 0.0749403 seconds
[PersOctree::PersOctree] end in 371.741 seconds
[PersSampler::PersSampler] end in 377.338 seconds
[Hash3DAnchored::Hash3DAnchored] begin
Warning: FullyFusedMLP is not supported for the selected architecture 70. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
Warning: FullyFusedMLP is not supported for the selected architecture 70. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
[Hash3DAnchored::Hash3DAnchored] end in 13.7505 seconds
Warning: FullyFusedMLP is not supported for the selected architecture 70. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
Warning: FullyFusedMLP is not supported for the selected architecture 70. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
n_nodes_before is value 3169
n_nodes_compacted is value 3117
tree_nodes_.size() is value 3117
[F Renderer.cpp:206] Check failed: std::isfinite((colors).mean().item())
Aborted (core dumped)

CUDA=11.2
libtorch=1.9

f2-nerf/src/Dataset/Dataset.cpp:182:45: error: too many arguments to function ‘std::vectorat::Tensor at::meshgrid(at::TensorList)’ 182 | auto ij = torch::meshgrid({ ii, jj }, "ij");
f2-nerf/src/Dataset/Dataset.cpp:203:45: error: too many arguments to function ‘std::vectorat::Tensor at::meshgrid(at::TensorList)’ 203 | auto ij = torch::meshgrid({ ii, jj }, "ij");

f2-nerf/External/libtorch/include/ATen/Functions.h:1317:35: note: declared here
1317 | TORCH_API std::vectorat::Tensor meshgrid(at::TensorList tensors);
| ^~~~~~~~
make[3]: *** [CMakeFiles/main.dir/build.make:147: CMakeFiles/main.dir/src/Dataset/Dataset.cpp.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:187: CMakeFiles/main.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:194: CMakeFiles/main.dir/rule] Error 2
make: *** [Makefile:125: main] Error 2

If I want to train on my own dataset with known internal and external parameters of the camera, how can I generate the corresponding cams_meta.npy？
Can you describe the meaning of each parameter in the file of cams_meta.npy?
Thanks!!

when training, i used config:
dataset:
factor: 2
It will be sampled for training at the (original resolution) / 2 of the dataset

but when test,i used same config:
dataset:
factor: 1
will wrong

how should I solve it?

When following the instructions, I get:

$ python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=train +work_dir=$(pwd)
Working directory is ...
register Hash3DAnchoredInfo
register TCNNWPInfo
register volume render info
Aoligei!
[Dataset::Dataset] begin
Load render poses
[LoadImages] begin
[LoadImages] end in 2.3389 seconds
Number of train/test/val images: 43/7/0
[Dataset::Dataset] end in 6.70828 seconds
[PersSampler::PersSampler] begin
[PersOctree::PersOctree] begin
[PersOctree::ConstructEdgePool] begin
edge_pool_.size() is value 1131
[PersOctree::ConstructEdgePool] end in 0.00132046 seconds
[PersOctree::PersOctree] end in 12.9086 seconds
[PersSampler::PersSampler] end in 12.934 seconds
[Hash3DAnchored::Hash3DAnchored] begin
Warning: FullyFusedMLP is not supported for the selected architecture 61. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
Warning: FullyFusedMLP is not supported for the selected architecture 61. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
[Hash3DAnchored::Hash3DAnchored] end in 1.74168 seconds
Warning: FullyFusedMLP is not supported for the selected architecture 61. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
Warning: FullyFusedMLP is not supported for the selected architecture 61. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
[F TCNNWP.cpp:130] Check failed: (params.scalar_type()) == (torch_type(tcnnwp->module_->param_precision())) 
Aborted (core dumped)

(on Linux, with an NVIDIA 1080Ti)

f2-nerf/src/Field/Hash3DAnchored.cu(151): error: no instance of overloaded function "atomicAdd" matches the argument list
argument types are: (__half2 *, __half2)

Hi,
I have some photos of a building taken by a drone, and can be reconstructed successfully by instant-ngp:

I tried to reconstruct it by F2-NeRF, but found the result full of artifacts (both free.yaml and wanjinyou.yaml):

RGB	DEPTH	OCT_DEPTH

Is there any hyperparameters need to be changed for scenes like this?
Thanks.

python scripts/run.py --config-name=wanjiyou dataset_name=example case_name=ngp_fox mode=render_path is_continue=true

it should be --config-name=wanjinyou

Are you planning to create Python bindings for the Anchored Hash Grid and Pers Sampler?

Thanks.

thanks

Dear author,
I got the following error when executing the cmake . -B build command. Do you have any solution. My cuda version is 11.6.

In the function of render in render module, after first density calculation(tcnn forward), pts_sampler_->UpdateOctNodes get run. I don't understand the function of UpdateOctNodes, specially set alpha and weight as input. Is it for accelerating the training process, if nothing in the sub grid , we can't need to do cal caution? And why MarkVistNode and invalid node inside it? thank u very much!

Not mentioned in the paper

Why is the angle of render limited to within 0~50 degrees, how to extend the angle range of rendering?

When I train on the KITTI dataset again, the algorithm is unable to reconstruct dynamic objects, such as cars. What is the reason for this?
For instance: Cars in motion are automatically removed。

Hi thank you for your great work, I got compilation error on ubuntu 22.04, it's work well on ubuntu 20.04
I got the following error, when running cmake --build build --target main --config RelWithDebInfo -j

eigen error fatal error: Core: No such file or directory

Hello,

I would like to express my sincere appreciation for your amazing work.
I am curious as to which dataset these images have been sourced from.
If it happens to be a collection that you have personally compiled, might there be any plans for making these datasets publicly available in the future?
Thank you.

I followed all data preprocessing required and then ran f2-nerf on a short snippet (i.e. 50 frames with a 10 frame interval) of a ScanNet sequence and run into nan. Could you look into this please? The images are attached for your reference. The processed data is unfortunately too large to upload.
images.zip

Do you have any script that can visualize something like figure 4, or any detailed explanations?

I wrote something that follows your formula, but get totally different results.
I use a toy example where there are two 1D cameras, and I use F(x) = (C1(x), C2(x)) as warping function. The warping looks roughly correct for theta=30

but totally different for theta=180

Should I increase the number of cameras and really do PCA? I feel like it should work with only two cameras.

Any plans on supporting windows 11?

Pleaseeeeeeeeeeeeee

Hi thank you for your great work, I saw that the demo is generating sequence of images by interpolating image pose, I wonder if it is possible to generate the 3D object (maybe .ply file) for entire scene instead so we can open it using software like blender.

I get the camera pose from slam, but the truncation (near, far) is missing, and I uniformly set: near: 0.01 far: 20. How much does this affect the outcome

The code is feasible! May I ask how many iterations have been made for the training time mentioned in the author's paper (12 minutes)? For the dataset Fox, my training seems to be slow, and my environment is as follows:
RTX 6000
NVIDIA 510.108.03
CUDA 11.3
CUDNN 8.7.0
gcc 9.4.0

camera free, faster construction-train/render-infer, dynamic-object/scene, memory-reduction, ....
all these nerf optimization points, fast and camera-free, F2 has done. (CVPR best paper, must!!!)

but, implmented in libtorch and cuda, needs half-precision, ...
too many compiling errors, too many running errors, ....

hi, our great authors, help me, I want to optimze these issues and submit. more widely spread. ...

Generate cameras file:

When I run the command of :python scripts/colmap2poses.py --data_dir ./data//
The issues as follows：
Traceback (most recent call last):
File "/home/xxx/anaconda3/envs/F2-NERF/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/xxx/anaconda3/envs/F2-NERF/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/xxx/anaconda3/envs/F2-NERF/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/xxx/f2-nerf/scripts/colmap2poses.py", line 218, in main
dataset = Dataset(data_dir)
File "/home/xxx/f2-nerf/scripts/colmap2poses.py", line 137, in init
image_id_to_image_idx[name_to_ids[image_name]] = sorted_image_names.index(image_name)
IndexError: index 372 is out of bounds for axis 0 with size 29
python-BaseException

Hi, and thanks for your excellent work.
When i train the model using python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=train +work_dir=$(pwd),

The error occurs:
Traceback (most recent call last): File "/home/xuxumiao/new_disk/MA/f2-nerf/scripts/run.py", line 38, in <module> @hydra.main(version_base=None, config_path='../confs', config_name='default') TypeError: main() got an unexpected keyword argument 'version_base'

Hopes for your answer ^ ^

python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=train
Error executing job with overrides: ['dataset_name=example', 'case_name=ngp_fox', 'mode=train']
Traceback (most recent call last):
File "scripts/run.py", line 59, in main
make_image_list(data_path, conf['dataset']['factor'])
File "scripts/run.py", line 35, in make_image_list
f = open(os.path.join(data_path, 'image_list.txt'), 'w')
FileNotFoundError: [Errno 2] No such file or directory: '/cephfs/yuanweizhong/project/f2-nerf/outputs/2023-04-23/17-50-45/data/example/ngp_fox/image_list.txt'

Are free datasets images taken with different internal parameters

Really appreciate for your important job and share of the project ! I use data captured by Dji sport camera to reconstruct the scene, which are high quality!

But recently I move a fly camera around a unbounded scene, the rendered images have float blur!

I checked the camera poses (as below). Looks like no problem.

All the data of images and camera poses are available:
https://drive.google.com/file/d/1Zf36hdqAk1NI5u9ucJ7rQs2HBc3i1sIA/view?usp=sharing
Looking forward to paying attention to this issue

Run cmake --build build --target main --config RelWithDebInfo -j, there have some errors:

xxx/f2-nerf/src/Field/TCNNWP.h(11): warning: overloaded virtual function "Field::Query" is only partially overridden in class "TCNNWP"
xxx/f2-nerf/src/Field/TCNNWP.h(40): error: qualified name is not allowed
xxx/f2-nerf/src/Field/TCNNWP.h(42): error: Function is not a template
xxx/f2-nerf/src/Field/TCNNWP.h(42): error: not a class or struct name
xxx/f2-nerf/src/Field/TCNNWP.h(44): error: identifier "variable_list" is undefined
xxx/f2-nerf/src/Field/TCNNWP.h(44): error: identifier "AutogradContext" is undefined

How to solve?

python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=render_path is_continue=true
Error executing job with overrides: ['dataset_name=example', 'case_name=ngp_fox', 'mode=render_path', 'is_continue=true']
Traceback (most recent call last):
File "scripts/run.py", line 74, in main
assert False, 'Can not find executable file'
AssertionError: Can not find executable file

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Fixing these would help better formula understanding:

1. In Definition 1. in the last sentence, the summation is from i to n_c not n
2. In Problem 1. in the last sentence, the summation is from j to n_p not K
3. What is Figure. 11? It is not described in text.

Thank you.

Thanks to the author for great work, but I encountered an error in the compilation process, Hope you can give us some information about modifying cmakelist

-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The CUDA compiler identification is NVIDIA 11.6.124
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- No release type specified. Setting to 'Release'.
-- Obtained CUDA architectures automatically from installed GPUs
-- Targeting CUDA architectures: 70
CMake Warning at External/tiny-cuda-nn/CMakeLists.txt:204 (message):
Fully fused MLPs do not support GPU architectures of 70 or less. Falling
back to CUTLASS MLPs. Remove GPU architectures 70 and lower to allow
maximum performance

-- Module support is disabled.
-- Version: 9.1.1
-- Build type: Release
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "11.6")
-- Caffe2: CUDA detected: 11.6
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 11.6
-- Found CUDNN: /usr/local/cuda/lib64/libcudnn.so
-- Found cuDNN: v8.6.0 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so)
-- /usr/local/cuda/lib64/libnvrtc.so shorthash is 4dd39364
-- Autodetected CUDA architecture(s): 7.0 7.0 7.0 7.0 7.0 7.0 7.0 7.0
-- Added CUDA NVCC flags for: -gencode;arch=compute_70,code=sm_70
-- Found Torch: /workspace/nerf/f2-nerf/External/libtorch/lib/libtorch.so
-- Configuring done
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
_CMAKE_CUDA_WHOLE_FLAG
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_OBJECT
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
_CMAKE_CUDA_WHOLE_FLAG
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_OBJECT
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
_CMAKE_CUDA_WHOLE_FLAG
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_OBJECT
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
_CMAKE_CUDA_WHOLE_FLAG
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_OBJECT
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
_CMAKE_CUDA_WHOLE_FLAG
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_OBJECT
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
_CMAKE_CUDA_WHOLE_FLAG
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_OBJECT
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
_CMAKE_CUDA_WHOLE_FLAG
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_OBJECT
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
_CMAKE_CUDA_WHOLE_FLAG
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_OBJECT
-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.

Hey,

I just get confused.

Do you mean: python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=test is_continue=true ?

something like this?

Can you give more details about using custome-data :) ?

thanks

I encountered an issue while running the command

python scripts/run.py --config-name=wanjinyou \
dataset_name=example case_name=ngp_fox mode=train \
+work_dir=$(pwd)

terminate called after throwing an instance of 'c10::Error'
  what():  Trying to create tensor with negative dimension -2036932816: [-2036932816, 10249059, 32766]
Exception raised from check_size_nonnegative at ../aten/src/ATen/EmptyTensor.h:15 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x7f52f04e83cb in /home/ubuntu/f2-nerf/External/libtorch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0xce (0x7f52f04e3d9e in /home/ubuntu/f2-nerf/External/libtorch/lib/libc10.so)
frame #2: at::detail::empty_generic(c10::ArrayRef<long>, c10::Allocator*, c10::DispatchKeySet, c10::ScalarType, c10::optional<c10::MemoryFormat>) + 0x8ec (0x7f52c03c3dcc in /home/ubuntu/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #3: at::detail::empty_cpu(c10::ArrayRef<long>, c10::ScalarType, bool, c10::optional<c10::MemoryFormat>) + 0x55 (0x7f52c03c4e75 in /home/ubuntu/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #4: at::detail::empty_cpu(c10::ArrayRef<long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, c10::optional<c10::MemoryFormat>) + 0x49 (0x7f52c03c4ef9 in /home/ubuntu/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #5: at::native::empty_cpu(c10::ArrayRef<long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, c10::optional<c10::MemoryFormat>) + 0x33 (0x7f52c0ab1f63 in /home/ubuntu/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x249be31 (0x7f52c152ee31 in /home/ubuntu/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #7: at::_ops::empty_memory_format::redispatch(c10::DispatchKeySet, c10::ArrayRef<c10::SymInt>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, c10::optional<c10::MemoryFormat>) + 0x1b7 (0x7f52c11ec107 in /home/ubuntu/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #8: <unknown function> + 0x2473f98 (0x7f52c1506f98 in /home/ubuntu/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #9: at::_ops::empty_memory_format::call(c10::ArrayRef<c10::SymInt>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, c10::optional<c10::MemoryFormat>) + 0x15f (0x7f52c12274cf in /home/ubuntu/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #10: <unknown function> + 0xff531 (0x55f69fb79531 in /home/ubuntu/f2-nerf/build/main)
frame #11: <unknown function> + 0xfef81 (0x55f69fb78f81 in /home/ubuntu/f2-nerf/build/main)
frame #12: <unknown function> + 0x97b8f (0x55f69fb11b8f in /home/ubuntu/f2-nerf/build/main)
frame #13: <unknown function> + 0x1357f1 (0x55f69fbaf7f1 in /home/ubuntu/f2-nerf/build/main)
frame #14: <unknown function> + 0x51c01 (0x55f69facbc01 in /home/ubuntu/f2-nerf/build/main)
frame #15: __libc_start_main + 0xf3 (0x7f529182f083 in /lib/x86_64-linux-gnu/libc.so.6)
frame #16: <unknown function> + 0x5f5be (0x55f69fad95be in /home/ubuntu/f2-nerf/build/main)

Aborted (core dumped)

It looks like an incorrect CUDA setup or something...

My environment:
Ubuntu 20.04
A100 GPU
native CUDA 11.6 (the one nvcc link to, does this matter? I don't quite understand if libtorch is using its own built-in cuda)
CUDNN 8.0
torch.version '1.13.1+cu117'

I compiled successfully, but got an error when run python scripts/run.py --config-name=wanjinyou dataset_name=example case_name=ngp_fox mode=test is_continue=true +work_dir=$(pwd)
this is the middle process

Working directory is /data01/pot/exp/f2-nerf
register Hash3DAnchoredInfo
register TCNNWPInfo
register volume render info
Aoligei!
[Dataset::Dataset] begin
Load render poses
[LoadImages] begin
[LoadImages] end in 0.398194 seconds
Number of train/test/val images: 43/7/0
[Dataset::Dataset] end in 2.81254 seconds
[PersSampler::PersSampler] begin
[PersOctree::PersOctree] begin
[PersOctree::ConstructEdgePool] begin
edge_pool_.size() is value 1131
[PersOctree::ConstructEdgePool] end in 0.00137949 seconds
[PersOctree::PersOctree] end in 3.6452 seconds
[PersSampler::PersSampler] end in 3.66801 seconds
[Hash3DAnchored::Hash3DAnchored] begin
[Hash3DAnchored::Hash3DAnchored] end in 1.33869 seconds
terminate called after throwing an instance of 'c10::Error'
  what():  open file failed because of errno 2 on fopen: , file path: /data01/pot/exp/f2-nerf/exp/ngp_fox/test/checkpoints/latest/scalars.pt
Exception raised from RAIIFile at ../caffe2/serialize/file_adapter.cc:27 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x151f312613cb in /data01/pot/exp/f2-nerf/External/libtorch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0xce (0x151f3125cd9e in /data01/pot/exp/f2-nerf/External/libtorch/lib/libc10.so)
frame #2: caffe2::serialize::FileAdapter::RAIIFile::RAIIFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x134 (0x151f034e5e94 in /data01/pot/exp/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #3: caffe2::serialize::FileAdapter::FileAdapter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x41 (0x151f034e6511 in /data01/pot/exp/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #4: caffe2::serialize::PyTorchStreamReader::PyTorchStreamReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x7f (0x151f034e396f in /data01/pot/exp/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #5: torch::jit::import_ir_module(std::shared_ptr<torch::jit::CompilationUnit>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, c10::optional<c10::Device>, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&) + 0x294 (0x151f04634bd4 in /data01/pot/exp/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #6: torch::jit::import_ir_module(std::shared_ptr<torch::jit::CompilationUnit>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, c10::optional<c10::Device>) + 0x94 (0x151f046354a4 in /data01/pot/exp/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #7: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, c10::optional<c10::Device>) + 0x76 (0x151f04635576 in /data01/pot/exp/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #8: torch::serialize::InputArchive::load_from(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, c10::optional<c10::Device>) + 0x2a (0x151f04d522fa in /data01/pot/exp/f2-nerf/External/libtorch/lib/libtorch_cpu.so)
frame #9: <unknown function> + 0x13c466 (0x55e405470466 in /data01/pot/exp/f2-nerf/build/main)
frame #10: <unknown function> + 0x13196d (0x55e40546596d in /data01/pot/exp/f2-nerf/build/main)
frame #11: <unknown function> + 0x135b21 (0x55e405469b21 in /data01/pot/exp/f2-nerf/build/main)
frame #12: <unknown function> + 0x51c01 (0x55e405385c01 in /data01/pot/exp/f2-nerf/build/main)
frame #13: __libc_start_main + 0xf3 (0x151ed1eb7083 in /lib/x86_64-linux-gnu/libc.so.6)
frame #14: <unknown function> + 0x5f5be (0x55e4053935be in /data01/pot/exp/f2-nerf/build/main)

In PersSampler module of the code, oct_idx_start_end appears quite often, wonder the physical meaning of oct_idx_start_end? Is it the volume point in sub grid? And what's the meaning of trans_idx in tree node? to mark a node whether to visited or not? thank u very much~~

!cmake . -B build
!cmake --build build --target main --config RelWithDebInfo -j

Killed
make[3]: *** [CMakeFiles/main.dir/build.make:234: CMakeFiles/main.dir/src/Renderer/Renderer.cu.o] Error 137

Killed
make[3]: *** [CMakeFiles/main.dir/build.make:433: CMakeFiles/main.dir/src/Utils/CustomOps/FlexOps.cu.o] Error 137
Killed
make[3]: *** [CMakeFiles/main.dir/build.make:404: CMakeFiles/main.dir/src/Utils/CustomOps/CustomOps.cu.o] Error 137
Killed
make[3]: *** [CMakeFiles/main.dir/build.make:119: CMakeFiles/main.dir/src/Field/Hash3DAnchored.cu.o] Error 137
/content/f2-nerf/src/PtsSampler/PtsSamplerFactory.cpp: In function ‘std::unique_ptr<PtsSampler> ConstructPtsSampler(GlobalDataPool*)’:
/content/f2-nerf/src/PtsSampler/PtsSamplerFactory.cpp:8:80: warning: control reaches end of non-void function [-Wreturn-type]
    8 |   auto type = global_data_pool->config_["pts_sampler"]["type"].as<std::string>();
      |                                                                                ^
/content/f2-nerf/src/Shader/../Field/TCNNWP.h(11): warning #611-D: overloaded virtual function "Field::Query" is only partially overridden in class "TCNNWP"

make[2]: *** [CMakeFiles/Makefile2:187: CMakeFiles/main.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:194: CMakeFiles/main.dir/rule] Error 2
make: *** [Makefile:125: main] Error 2

hi, thanks for your amazing work. when I trained a model using my custom dataset, the psnr of training dataset is 37, but the psnr of test/validation is 10. I have checked and not found any problems. Can you help me analyze the problem?

After render_path mode, I only see ./exp/ngp_fox/test/novel_images

run cmake --build build --target main --config RelWithDebInfo -j got following error:

/usr/bin/ld: CMakeFiles/main.dir/src/Field/TCNNWP.cpp.o: in function `TCNNWP::TCNNWP(GlobalDataPool*, int, int, int, int)':
TCNNWP.cpp:(.text+0x37c3): undefined reference to `tcnn::cpp::create_network(unsigned int, unsigned int, nlohmann::basic_json<std::map, std::vector, std::string, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > > const&)'
collect2: error: ld returned 1 exit status
make[3]: *** [CMakeFiles/main.dir/build.make:571: main] Error 1
make[2]: *** [CMakeFiles/Makefile2:188: CMakeFiles/main.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:195: CMakeFiles/main.dir/rule] Error 2
make: *** [Makefile:125: main] Error 2

Any idea?

Hello, I am wondering if I use f2-NeRF on a large scale dataset, which parameters should be tuned and how to(eg. increase, or reduce).

Also, in my result, I found thin stuff often missing, like image below, is it caused by aliasing?

In your future work part, I see you plan to add anti-aliasing, how do you plan to add it? like ZipNeRF?

Hi, and thank you for sharing this code.
Is there any functionality to export a textured mesh after training?

Thanks!

Hi @Totoro97,

Thank you for your great work.
I tried to train your code with my own dataset.

python scripts/run.py --config-name=wanjinyou
dataset_name= case_name= mode=train
+work_dir=$(pwd)

But I got Nan!.
Can you help me? How to modify the configuration in wanjinyou to overcome the Nan?

Thank you

In the demo video you gave, I found that there will be blocky blurred areas in some frames. This problem was even more pronounced when testing on my own dataset collected outdoors. What could be the reason for this? Thank you so much for your work!

On my datasets: