Comments (14)
Discuss this with Attila from the VRVis. One option is AliceVision.
Another, more experimental, solution is to try a NeRF with surface extraction.
from pix2model.
AliceVision was not that good in my tests. Worse reconstructions, similar speed as COLMAP.
The way forward seems to be with NeRFs. The most promising being:
- NVDiffRec: https://github.com/NVlabs/nvdiffrec
- Nerfstudio (Nerfacto): https://github.com/nerfstudio-project/nerfstudio
from pix2model.
i would prefer Nerfstudio because it has a nicer license
from pix2model.
Nerfstudio just released a new version with NeuS and NeuSFacto model based on this paper: https://arxiv.org/pdf/2106.10689.pdf
Could be worth a try.
from pix2model.
interesting paper by a colleague: https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
maybe wait and see if it will be included in nerfstudio
from pix2model.
another interesting work that could replace COLMAP: https://openaccess.thecvf.com/content/CVPR2023/papers/Li_Neuralangelo_High-Fidelity_Neural_Surface_Reconstruction_CVPR_2023_paper.pdf
from pix2model.
and another: https://liuyuan-pal.github.io/NeRO/
from pix2model.
the experiments so far were not convincing. all nerf methods have either too strict requirements to the inputs (accurate camera poses, images from all sides etc.) or worse quality than COLMAP (although the paper results look better)
from pix2model.
had a longer discussion with a visitor... NeuS and derivates are probably the way to go. we care about the geometry, which typical NeRFs can't represent properly.
note that we need a version without segmentation masks that also works on unbounded scenes. apparently, they all depend on COLMAP for camera poses.
NeuS (rather slow): https://github.com/Totoro97/NeuS
NeuDA (slow?): https://github.com/3D-FRONT-FUTURE/NeuDA
NeuS++ (faster, check branch for unmasked): https://github.com/19reborn/NeuS2/tree/neuspp
Instant Neural Surface Reconstruction(fast): https://github.com/bennyguo/instant-nsr-pl
NeRO (slow?, no unbounded scenes?): https://github.com/liuyuan-pal/NeRO
Neuralangelo (need license): https://research.nvidia.com/labs/dir/neuralangelo/
SDFStudio (like NeRFStudio but for reconstruction, check license for angelo): https://github.com/autonomousvision/sdfstudio
most interesting seem SDFStudio and NeuS++
from pix2model.
@ErlerPhilipp Hi Philipp. I have recently released a derivative version of Neus and Neuralangelo called Instant-angelo. This new project has a friendly open-source license, higher efficiency, and is fully compatible with Colmap. Additionally, while our current reconstruction pipeline does not use masking, it focuses on reconstructing just the central area within the camera poses' field of view. I hope this new project will be useful to the research community. We provide some visualization of current results. Please let me know if you have any other questions!
from pix2model.
Hi @hugoycj ! Thanks, your work looks indeed very interesting.
I have several questions:
- There is no official publication, right? Do you plan to publish it?
- Are the implementation details described or documented somewhere or should I just look at the code?
- Can I download some results as textured meshes? I see only GIFs in your repo.
- In case you have the time, I would really appreciate if you could run this example with your method: https://drive.google.com/drive/folders/1AgSzlI884Hezh4ebPBe5dITLAI8fX4He?usp=sharing
from pix2model.
Apologies for the delayed response as I am currently occupied with the deadline for a conference.
Instant-Angelo is still in the developmental phase. Its functionality heavily relies on the results obtained through sfm (structure from motion) and may encounter issues if the sfm points are noisy or sparse. Our main objective for developing Instant-Angelo is to provide a useful tool for the community, and we have not yet a plan for publication. However, we intend to create a blog and detailed documentation once I complete the deadline in mid-November.
Currently, only geometry mesh export is supported in Instant-Angelo, but we are working tirelessly to incorporate a new API for exporting textured mesh.
For reference, I have run the example on your provided data. The process takes 2 minutes for structure-from-motion and 20 minutes on 3090 for dense reconstruction. You could download the geometry mesh from this link. Some quick look up results are shown below:
from pix2model.
Sorry for the delay on my side, this time. Good luck with your submission!
Thank you so much for running it on our example. This result looks indeed pretty good. Interesting that it creates a hole in the bottom plate while it fills in the space between the feet.
We're currently switching from COLMAP to AliceVision, which is way more modular. Once this is done, we can replace e.g. the SfM/MVS part with a NeRF/NeuS variant. Your method is definitely a hot candidate for that. Thanks!
from pix2model.
switching to AliceVision or other methods is too much work for now
from pix2model.
Related Issues (20)
- Readme update HOT 4
- Check and resolve deploy warnings
- Editor cursor position HOT 5
- New Dev Environment HOT 1
- Share Test Data HOT 3
- Optimize Parameters HOT 2
- Default Meshroom Pipeline HOT 11
- Improved Pipeline HOT 5
- Check skipping MVS HOT 2
- More public test data HOT 2
- Setup Meshroom Docker HOT 1
- Clean-Up HOT 1
- Improved UI HOT 5
- Error on reconstructing an uploaded point cloud HOT 1
- unable to start after deployment HOT 7
- Better Reconstruction HOT 1
- Clean-up COLMAP exception handling
- Clean-up COLMAP update UI aftermath HOT 1
- Upload Area too large
- About point cloud visualization HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pix2model.