Comments (7)
- I agree with you that the error caused by running the colmap script is due to low overlap, which can also be found here.
- Our method uses MVS to predict depth. When the overlap area of the input image is small, especially when there are only two images as inputs, the depth estimated by MVS is inaccurate due to the inability to find the corresponding point, resulting in poor reconstruction and rendering quality.
- I think this is due to inaccurate depth prediction, here are some things that might be worth trying:
i) Modify the value of scale_factor. Since our model is trained on DTU data set (depth range is 425~905), when testing on other new datasets, we use scale_factor to adjust the depth of the new scenario to be close to the depth of the DTU dataset. For example, for the nerf synthetic dataset with a depth range of 2.5 to 5.5, we set scale_factor to 200, and for the LLFF dataset with a depth range of about 20 to 100, we set it to 12. You would try modifying scale_factor here;
ii) You can also modify the number of sampling points here, commonly used settings are [64,8], [48,8] and [16,8];
from mvsgaussian.
P/s : 2 images of input and target below :
2 input views:
3 target views:
from mvsgaussian.
Hello, depth_ranges include two values, namely the minimum depth of the scene (depth_min) and the maximum depth (depth_max). For different data sets or scenes, this value is completely different. For DTU data sets, it's 425 and 905, for your own data, you need to run colmap to get depth_ranges.
You can use the demo we just uploaded, that only requires multi-view images as input. The first step is to run colmap to get the camera parameters and depth_ranges. More details are available here .
from mvsgaussian.
Thank you for your help, but now I have a problem that my host is headless, so I can not use colmap after install, can you have any solution for this problem ? Thank you so much
Error :
Need to run COLMAP
qt.qpa.xcb: could not connect to display :0
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
*** Aborted at 1721114100 (unix time) try "date -d @1721114100" if you are using GNU date ***
PC: @ 0x7f5ad03cd00b gsignal
*** SIGABRT (@0x3fd00004a4a) received by PID 19018 (TID 0x7f5acbbc7900) from PID 19018; stack trace: ***
@ 0x7f5ad1fbb631 (unknown)
@ 0x7f5ad14fe420 (unknown)
@ 0x7f5ad03cd00b gsignal
@ 0x7f5ad03ac859 abort
@ 0x7f5ad0991aad QMessageLogger::fatal()
@ 0x7f5ad0f737ae QGuiApplicationPrivate::createPlatformIntegration()
@ 0x7f5ad0f74708 QGuiApplicationPrivate::createEventDispatcher()
@ 0x7f5ad0b98f55 QCoreApplicationPrivate::init()
@ 0x7f5ad0f76543 QGuiApplicationPrivate::init()
@ 0x7f5ad16803bd QApplicationPrivate::init()
@ 0x559014f0f602 RunFeatureExtractor()
@ 0x559014efbeaf main
@ 0x7f5ad03ae083 __libc_start_main
@ 0x559014efff6e _start
Traceback (most recent call last):
File "lib/colmap/imgs2poses.py", line 17, in
gen_poses(args.scenedir, args.match_type)
File "/data/Phat/MVSGaussian/lib/colmap/poses/pose_utils.py", line 268, in gen_poses
run_colmap(basedir, match_type)
File "/data/Phat/MVSGaussian/lib/colmap/poses/colmap_wrapper.py", line 35, in run_colmap
feat_output = ( subprocess.check_output(feature_extractor_args, universal_newlines=True) )
File "/home/vinai/.conda/envs/mvsgs/lib/python3.7/subprocess.py", line 411, in check_output
**kwargs).stdout
File "/home/vinai/.conda/envs/mvsgs/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['colmap', 'feature_extractor', '--database_path', '/data/Phat/MVSGaussian/examples/scene1/database.db', '--image_path', '/data/Phat/MVSGaussian/examples/scene1/images', '--ImageReader.single_camera', '1']' died with <Signals.SIGABRT: 6>.
from mvsgaussian.
Kindly refer to colmap/colmap#570 (comment), NVlabs/instant-ngp#329 (comment), NVlabs/instant-ngp#329 (comment).
from mvsgaussian.
Thank you for your help, your model is very good when predict the inward-facing images. But now I'm try to fit 2 outward-facing images which low overlapping area and try to predict 3 target views which are 3 views between 2 outward-facing inputs. I got the issue that my dataset can not use for run colmap by your script, it will error when I run code ( I guess my images are low overlapping between each image because those are outward-facing).
So I want to ask you that could your model config to reconstruct the novel view with multi outward-facing input views (the overlapping area is low) ? Thank you
P/s : I try to calculate depth range by colmap and then run the old version code to predict target view, the result is not good. The results are as below, can you tell me the reason for the phenomenon of two distinct light and dark areas? Thank you
from mvsgaussian.
Thank you for your help.
from mvsgaussian.
Related Issues (20)
- Weird results on a custom Blender-synthetic dataset HOT 4
- What are the differences between run.py and lib/render.py? HOT 1
- Why the metric get lower after gaussian optimization satge? HOT 6
- About module of decoding gaussian parameters HOT 2
- About depth estimation HOT 2
- about rot HOT 2
- 关于所使用的坐标系方向定义提问 HOT 2
- Questions about how to build feature volumes HOT 4
- ValueError: too many values to unpack (expected 2) HOT 2
- About the .ply file from the generalizable model HOT 14
- About depth estimation HOT 4
- About the .ply file HOT 7
- 关于点云的问题 HOT 1
- In the paper, why FPS in Table 1 is only 24.5? HOT 24
- 代码运行时遇到问题! HOT 3
- question about rendering of nerf_llff datasets! HOT 6
- About the ply file HOT 8
- 对训练指标的一些疑问 HOT 3
- How can I do inference using just 4 images? HOT 3
- OSError: [Errno 24] Too many open files HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mvsgaussian.