pix2nerf's People
Forkers
haifengzeng whatcell bruinxiong gs01md david20080125 cylonspace memoryrobber yumianhuli2 ofirkris acgnnsj yijunwu wei-baldwin-zeng artorixx tianhaofu kkodoo hongbo-sun 41xu landian60 zivzone atlasgooo2 xg-chu hslim11 yangqing-20 yurkar2333 jackzhousz bluedream1121 dijkstra14 chikara-n-ellipse iwtbam tanglespace mmarking dungeonmassster keep-learning-cmd mmarei1pix2nerf's Issues
The question about training in celebA
Would there is possible to release the train code?
Hi, Dr. Cai, would you like first release the train code of the Pixel2NeRF?Thank you.
The new view is terrible, although reconstruction is good
wrong author list
Hi,
The author list in your page has a small mistake, Noah Snavely should be replaced by luc Van Gool? The link is right.
Use the pretrain model to producing multi-view images of pubilc datasets
how can I apply your model to my dataset
input an image and I want to output my own 3D model
Use custom image as input, but result has nothing to do with input?
as mentioned
Test set for celeba
Thanks for your great work!
I'm trying to test the pre-trained model on celeba dataset and cannot find how you split the dataset in the provided code or csv file. How could I get the test set for celeba as mentioned in the paper ( "We hold out 8k images as the test set. ").
Thanks again for you work and help.
Question about the training environment.
Hi, First of all, I was impressed with your paper.
I read your paper in CVPR 2022 , but it doesn't mention about training environment and time. Can i get information about it?
if you are reluctant to disclose, plz contact me email or other connection
Pre-trained model for Shapenet Car
Thanks for your excellent work!
Could you please provide checkpoints for shapenet car?
Thanks for your help!
How to know the relationship between pose d
I means that if i know the image of 30°, i could easily get the rendered image of 40° by given a absolute angles of 40°.
However, i donnot know how the pose d which is a latent code achieve this.
That is, could i modify the latent pose d to get the clear direction?
The question about the re-train the pix2nerf.
Hi, I have re-train the pix2nerf with a single 3090 GPU on synchairs, but I have not achieve the view synthesis under the condition of a single view. If I utilize a single 3090 GPU to train the pix2nerf, would I change the batch size from 9 to 36? Thank you very much.
The question about how to train the Pixel2nerf?
There is no 'train.py'. Would you like to tell me how to train the pixel2nerf? Thank you.
when use 2 Tesla V100 GPU cards, it gives this below error which said No locks available, what do you think the issue is ?
(Pix2NeRF) bash-4.4$ CUDA_VISIBLE_DEVICES=0,1 python3 train_con.py --curriculum=celeba --output_dir='/data/yshan/data/Img/pix2NerfOutput' --dataset_dir='/data/yshan/data/Img/img_align_celeba' --encoder_type='CCS' --recon_lambda=5 --ssim_lambda=1 --vgg_lambda=1 --pos_lambda_gen=15 --lambda_e_latent=1 --lambda_e_pos=1 --cond_lambda=1 --load_encoder=1
Namespace(n_epochs=3000, sample_interval=1000, output_dir='/data/yshan/data/Img/pix2NerfOutput', load_dir='/data/yshan/data/Img/pix2NerfOutput', curriculum='celeba', eval_freq=5000, port='12354', set_step=None, model_save_interval=200, pretrained_dir='', wandb_name='', wandb_entity='', wandb_project='', recon_lambda=5.0, ssim_lambda=1.0, vgg_lambda=1.0, dataset_dir='/data/yshan/data/Img/img_align_celeba', pos_lambda_gen=15.0, sn=0, lambda_e_latent=1.0, lambda_e_pos=1.0, encoder_type='CCS', cond_lambda=1.0, ema=1, load_encoder=1)
Lock not found
Traceback (most recent call last):
File "/data/yshan/Pix2NeRF/train_con.py", line 684, in
mp.spawn(train, args=(num_gpus, opt), nprocs=num_gpus, join=True)
File "/home/yshan/anaconda3/envs/Pix2NeRF/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 239, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/home/yshan/anaconda3/envs/Pix2NeRF/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 197, in start_processes
while not context.join():
File "/home/yshan/anaconda3/envs/Pix2NeRF/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 160, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:
-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "/home/yshan/anaconda3/envs/Pix2NeRF/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
fn(i, *args)
File "/data/yshan/Pix2NeRF/train_con.py", line 85, in train
setup(rank, world_size, opt.port, opt.output_dir)
File "/data/yshan/Pix2NeRF/train_con.py", line 46, in setup
dist.init_process_group('gloo', init_method=file_lock, rank=rank, world_size=world_size)
File "/home/yshan/anaconda3/envs/Pix2NeRF/lib/python3.10/site-packages/torch/distributed/distributed_c10d.py", line 907, in init_process_group
default_pg = _new_process_group_helper(
File "/home/yshan/anaconda3/envs/Pix2NeRF/lib/python3.10/site-packages/torch/distributed/distributed_c10d.py", line 1009, in _new_process_group_helper
backend_class = ProcessGroupGloo(backend_prefix_store, group_rank, group_size, timeout=timeout)
RuntimeError: flock: No locks available
Question on the open sourced time of the 'train' script
Hi, thanks for your contributions on "Single Image to Neural Radiance Fields Translation". I want to know when you will open sourced the "train" script about the pix2nerf? Thank you.
About the CelebA_pos.csv?
Hi, many thanks for your great work!
May I ask you a question?
- What is the 'CelebA_pos.csv' used for in the training phase?
- Is it compulsory in training? If not, how to remove it?
RuntimeError: open(/process_group_sync.lock): Permission denied
First of all, thank you for your work!
When I used code to train my data, such problems occurred, which could not be solved. I hope you can make valuable suggestions.
Specific errors are as follows:
CUDA_VISIBLE_DEVICES=2 python train_con.py --curriculum=carla --output_dir=PATH_TO_OUTPUT --dataset_dir=dataset/exp0/train/.png --encoder_type='CCS' --recon_lambda=5 --ssim_lambda=1 --vgg_lambda=1 --pos_lambda_gen=15 --lambda_e_latent=1 --lambda_e_pos=1 --cond_lambda=1 --load_encoder=1
Namespace(cond_lambda=1.0, curriculum='carla', dataset_dir='dataset/exp0/train/.png', ema=1, encoder_type='CCS', eval_freq=5000, lambda_e_latent=1.0, lambda_e_pos=1.0, load_dir='', load_encoder=1, model_save_interval=200, n_epochs=3000, output_dir='PATH_TO_OUTPUT', port='12354', pos_lambda_gen=15.0, pretrained_dir='', recon_lambda=5.0, sample_interval=1000, set_step=None, sn=0, ssim_lambda=1.0, vgg_lambda=1.0, wandb_entity='', wandb_name='', wandb_project='')
Lock not found
terminate called after throwing an instance of 'std::system_error'
what(): open(/process_group_sync.lock): Permission denied
Traceback (most recent call last):
File "train_con.py", line 686, in
mp.spawn(train, args=(num_gpus, opt), nprocs=num_gpus, join=True)
File "/home/nn/anaconda3/envs/nerf/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 199, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/home/nn/anaconda3/envs/nerf/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 157, in start_processes
while not context.join():
File "/home/nn/anaconda3/envs/nerf/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 118, in join
raise Exception(msg)
Exception:
-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "/home/nn/anaconda3/envs/nerf/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
fn(i, *args)
File "/home/nn/code/Pix2NeRF/train_con.py", line 87, in train
setup(rank, world_size, opt.port, opt.output_dir)
File "/home/nn/code/Pix2NeRF/train_con.py", line 48, in setup
dist.init_process_group('gloo', init_method=file_lock, rank=rank, world_size=world_size)
File "/home/nn/anaconda3/envs/nerf/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py", line 433, in init_process_group
timeout=timeout)
File "/home/nn/anaconda3/envs/nerf/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py", line 508, in _new_process_group_helper
timeout=timeout)
RuntimeError: open(/process_group_sync.lock): Permission denied
Compatibility Issue with torch==1.7.0 and torchvision==0.8.1
Hello,
I got some problems when I was building the environment, and find something strange in the requirement.txt.
in requirement.txt:
- torch==1.7.0
- torchvision==0.8.1
However, the official PyTorch website only provides combinations of torch 1.7.0 with torchvision 0.8.0, or torch 1.7.1 with torchvision 0.8.1. The required combination of torch 1.7.0 with torchvision 0.8.1 does not seem to be available.
Now I am trying to use the combination of torch 1.7.1 with torchvision 0.8.1 to build the environment, but I still hope you can help me solve this problem
How to calculate IS and FID score
IS there anyone who can share the code for calculating IS 、 FID scores?Thank you!
OOM in hybrid_optimization
RuntimeError: CUDA out of memory. Tried to allocate 768.00 MiB (GPU 0; 6.00 GiB total capacity; 3.51 GiB already allocated; 391.91 MiB free; 3.57 GiB reserved in total by PyTorch)
Code request
When will the code be released? There are long time after the acceptance of the CVPR 2022.
Could the Pix2nerf extract high quality mesh?
Hi, the results of the pix2nerf is interesting. Please release the code of the pix2nerf as soon as possible.
The question about the re-trained results in celebA
Hi, I have re-trained the pix2NeRF in a single 3090. I got similar results of IS and FID as your paper but a low result of KID. Would you like to tell me why?
The re-trained results of Pix2NeRF:
{'inception_score_mean': 2.207920169906786, 'inception_score_std': 0.04608231534322652, 'frechet_inception_distance': 28.55671719303598, 'kernel_inception_distance_mean': 0.022525952379879886, 'kernel_inception_distance_std': 0.001017455628829365}
Thank you very much.
Implementation Details
Hi there,
I have used 2 T4 GPU to retrain the model using the train_con.py script. The issue(cleanup function) persists across 2 different datasets, CARLA and CelebA. I would like to ask if this happened to you while training the model or if it's some error while executing the script.
Lots of thanks!
Bugs when I run on 3090 GPU
ModuleNotFoundError: No module named 'moviepy'
pip install moviepy
RuntimeError: CUDA error: no kernel image is available for execution on the device
pip install torch==1.12.0+cu113 torchvision==0.13.0+cu113 torchaudio==0.12.0 --extra-index-url https://download.pytorch.org/whl/cu113
AssertionError: Cannot find installation of real FFmpeg (which comes with ffprobe).
sudo apt-get install ffmpeg
hi bro,the project is amazing,plz release it as soon as possible
How to re-train the pix2nerf?
Hi, would you like to tell me how to re-train the pix2nerf. Thank you.
some question about the PSNR evaluation
Thank you for your work! I'd like to ask how the PSNR of the srnchairs part of the paper compares. Because the pose angle defined in the code is different from the actual distribution in the data set, how do you compare the PSNR values of the rendered new view images? If it's convenient, could you provide code or some suggestions to me. Thanks!
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.