Comments (5)
I checked issue_9, but that doesn't explains why colmap output not matching the metadata in nerfsemble. π
from nersemble.
Hi,
Can you share a visualization of your colmap poses?
You can use the dreifus library for quickly visualizing camera poses. It is expected that you won't get the same world space since you are not using a calibration checkerboard with metrical distances. But that shouldn't matter for the method as long as the scale of the world space is roughly metrical.
from nersemble.
I loaded custom data successfully with my own multi-view camera setup with the coordinate system plotted here:
X increases to the right
Y increases downwards
Z increases forward
from nersemble.
Hi @tobias-kirschstein and @KaneLindsay , this is colmap output in colmap.
This is colmap visual in dreifus. Really small and sparse cameras.
This is meta NeRSemble in dreifus.
Colmap visualization in colmap looks same with meta NeRSemble in dreifus. But Colmap visualization in dreifus seems weird.
The colmap ouput is OpenCV, so there must be some convertion L missed.
What's the possible reason account for thisοΌ
Thank you for the reply!
from nersemble.
Hi @jeb0813 ,
thanks for the visualizations. To me, that just looks like COLMAP gives you a different world-space scale. Note the positions of the cameras from NeRSemble are all within 1 unit distance of the origin, while the output of COLMAP seems to be giving you camera positions with 5 or more units distance to the origin.
This is somewhat expected since COLMAP cannot know what 1 unit should mean in your scene.
You could just multiply the translation part of the cam2world matrices that you get from COLMAP with a small factor (say 0.2, effectively shrinking the world space by 5x) and that should bring the cameras closer to the origin where the NeRSemble cameras are.
In terms of 3D reconstructions it does not really matter what scale the world space has. The only point where it becomes relevant is if you have a 3D bounding box where everything outside is assumed to be empty. In the case of NeRSemble, there is a tight 3D bounding box around the reconstructed head to avoid having too many floaters. Therefore, if your world-space has a vastly different scale (as it seems to be the case for your COLMAP poses), the 3D reconstruction will fail since most of the reconstruction would be outside the bounding box. Instead of ensuring your world space has roughly metric scale, you can of course also adjust the size of the bounding box in the train script.
from nersemble.
Related Issues (13)
- Pretrained Model HOT 1
- Extract .ply or .obj file HOT 1
- Expressions Transfer HOT 1
- Hardware Requirements HOT 1
- Training Requirement HOT 6
- Matting masks for free part? HOT 2
- how to get the depth map HOT 2
- How to do color correction in other sequences? HOT 1
- Coordinate System of Camera Matrix HOT 2
- Resume Training from Checkpoint HOT 2
- How to train on custom dataset?
- DataManager compatible Extraction script
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nersemble.