r00tman / nerf-osr Goto Github PK
View Code? Open in Web Editor NEWNeRF for Outdoor Scene Relighting [ECCV 2022]
Home Page: https://4dqv.mpi-inf.mpg.de/NeRF-OSR/
NeRF for Outdoor Scene Relighting [ECCV 2022]
Home Page: https://4dqv.mpi-inf.mpg.de/NeRF-OSR/
Hello. Thank you for sharing your excellent research and code.
I have a question about the dataset. Your paper mentions the following: "Furthermore, even though NeRF-OSR can synthesise the sky and vegetation (e.g., trees), it is not possible to evaluate their predictions due to their highly varying appearance, especially when recordings sessions span different weather seasons. Hence, we also estimate the masks of these regions and exclude them from our evaluation."
However, I noticed that there are no masks available under the 'scene_name/final/test' directory in the dataset, and the masks found in the 'scene_name/final/mask' directory do not seem to include masks for trees and the sky. In order to conduct a fair evaluation, could you please release the masks that you used for evaluation?
Hi there,
This is a very promising project, thanks for open-sourcing it.
We are trying to evaluate the project, but the model and data download link is really slow (< 100 kbps).
In the best interest of the project, can the data and models please be uploaded to some other cloud storage?
Thanks
Thank you for the open source, but the width and height of the trevi dataset images in the dataset you provided are not fixed, can you please provide a version with fixed width and height?
no envmap found for validation/rgb/21-08_16_00_IMG_4592-JPG
Hello, I have a question about iterations.
In your paper, you train the model for 5 * 10^5 iterations. However, N_iters in config.txt is set to 5000001. What is the correct number of parameters?
Hi!
Thanks for sharing your code and I think this is a wonderful work to solve the problems in relighting outdoor scenes by a NeRF framework.
I have an issue with the env params consisting of 9*3 variables. I clearly know the network how to process the training images and optimize the env params from default values. However, I find that the network also takes the default values as testing env params and do not optimize them. if i dont misunderstand. Is this reasonable? How do u get the default values?
Looking forward to your answer,
Thanks again!
Hi @r00tman ,
Thanks for sharing the code and the wonderful work.
I 've downloaded all the models and data. But when I try to run the tesh *sh scripts, there is an error message like:
Traceback (most recent call last):
File "/home/mi/anaconda3/envs/nerf/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
fn(i, *args)
File "/home/mi/PycharmProjects/NeRF-OSR/ddp_test_nerf.py", line 48, in ddp_test_nerf
start, models = create_nerf(rank, args)
File "/home/mi/PycharmProjects/NeRF-OSR/ddp_train_nerf.py", line 348, in create_nerf
with open(f) as file:
FileNotFoundError: [Errno 2] No such file or directory: 'logs/trevi_final_masked_flipxzinitenv/train_images.json'
I want to know how can I get these json file or how to generate them?
Excellent work! How can I use your code to train on my own dataset?
Hi~ Thanks for your awesome work!
I am using your dataset to train my own nerf. I see min_depth and max_depth in your dataloder. But I have not seen these in your provided dataset. Are min_depth and max_depth nesseary? Will it cause error in your code when min_depth is None?
Hope for your reply, Thank you very much.
Hi Viktor et al - very excited about this code, but in all the test*.sh files there's a parameter --test_env e.g.:
python ddp_test_nerf.py --config configs/europa/final.txt --render_splits static_path1_blend --test_env data/europa/final/static_path1_blend/envmaps
...but "--test-env" doesn't seem to be a supported parameter - I can't see it added anywhere in the code, and ddp_test_nerf.py never gets past the fact it doesn't recognise one or more parameters. Am I missing something obvious?
Hi, thank you for the great dataset.
Provided test (siggraph) data contains only 1 image for LK2 site although in your paper you mention 5 images from 5 different recording sessions.
https://nextcloud.mpi-klsb.mpg.de/index.php/s/mGXYKpD8raQ8nMk?path=%2FData%2Flk2%2Ffinal%2Fsiggraph1200
I see all 5 images for st and lwp sites. Could you please update the dataset?
I Use
python ddp_test_nerf.py --config configs/lk2/final.txt --render_splits static_path1_blend --test_env data/lk2/final/static_end_blend/envmaps
Obtain output
2023-07-22 19:33:57,538 [INFO] root: Reloading from: logs/lk2_final_masked_flipxzinitenv/model_1165000.pth
2023-07-22 19:33:57,657 [INFO] root: raw intrinsics_files: 0
2023-07-22 19:33:57,659 [INFO] root: raw pose_files: 0
2023-07-22 19:33:57,659 [INFO] root: Split static_path1_blend, # views: 0
The algorithm does not have an ideal effect on the indoor dataset.
As shown in the figure below, the effect of relighting is not very good. There is a "hole" on the surface of the chair.
I guss this is due to some inaccuracies in geometry estimation.
I would appreciate it if receiving your reply.
@r00tman
The following error is generated when running python ddp_mesh_nerf.py --config configs/lk2/final.txt
. Any ideas how to resolve it?
2023-02-12 15:56:18,287 [INFO] root: Found ckpts: ['logs/lk2_final_masked_flipxzinitenv/model_420000.pth']
2023-02-12 15:56:18,287 [INFO] root: Reloading from: logs/lk2_final_masked_flipxzinitenv/model_420000.pth
Traceback (most recent call last):
File "ddp_mesh_nerf.py", line 102, in <module>
mesh()
File "ddp_mesh_nerf.py", line 97, in mesh
join=True)
File "/projappl/project_2007011/miniconda3/envs/nerfosr/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 200, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/projappl/project_2007011/miniconda3/envs/nerfosr/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
while not context.join():
File "/projappl/project_2007011/miniconda3/envs/nerfosr/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 119, in join
raise Exception(msg)
Exception:
-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "/projappl/project_2007011/miniconda3/envs/nerfosr/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 20, in _wrap
fn(i, *args)
File "/scratch/project_2007011/NeRF-OSR/ddp_mesh_nerf.py", line 32, in ddp_mesh_nerf
start, models = create_nerf(rank, args)
File "/scratch/project_2007011/NeRF-OSR/ddp_train_nerf.py", line 420, in create_nerf
models[name].load_state_dict(to_load[name])
File "/projappl/project_2007011/miniconda3/envs/nerfosr/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1045, in load_state_dict
self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for DistributedDataParallel:
Missing key(s) in state_dict: "module.env_params.01-08_07_30_IMG_6645-JPG", "module.env_params.01-08_07_30_IMG_6650-JPG",
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.