WP 5.2: better colors about pix2model HOT 14 CLOSED

Hinweis von Stefan: Bei Texture mapping auf depth images gibt's das und ein Nachfolge Paper, aber sicher auch was Neueres:
https://www.cg.tuwien.ac.at/research/publications/2014/arikan-2014-pcvis/

from pix2model.

Hinweis von Michi: Fürs Texture Mapping hatte damals Michael Birsak was implementiert mit einem paper:
https://www.cg.tuwien.ac.at/research/publications/2013/birsak-2013-sta/
da gabs ein ähnliches Paper auch von Michael Goesele irgendwo...

from pix2model.

@joschi1212
very interesting paper, combines surface reconstruction and texturing. "Deep Hybrid Self-Prior for Full 3D Mesh Generation": https://arxiv.org/pdf/2108.08017.pdf
Repo: https://github.com/weixk2015/DHSP3D

from pix2model.

@joschi1212
in the NeRF direction: https://github.com/NVlabs/instant-ngp
they are using COLMAP for camera poses

many other related papers: https://github.com/yenchenlin/awesome-NeRF

from pix2model.

COLMAP writes the camera poses to binary or text files: https://colmap.github.io/format.html#sparse-reconstruction
I see that the binary version is active in our project, since I have a "cameras.bin" in my local test dir .\upload\4c5c6d38-255e-4708-b5f8-41e029e36340\step1\v000\sparse\0\

so, we have the camera poses already available for further processing.
see this https://github.com/Fyusion/LLFF/blob/c6e27b1ee59cb18f054ccb0f87a90214dbe70482/llff/poses/pose_utils.py#L259
and this https://github.com/Fyusion/LLFF/blob/c6e27b1ee59cb18f054ccb0f87a90214dbe70482/llff/poses/pose_utils.py#L11
for how to read the bin files:

from pix2model.

one more related paper: https://arxiv.org/pdf/2101.10734.pdf

from pix2model.

Thanks, I will look into it!

from pix2model.

Maybe, we can take some parts from Alicevision/Meshroom: https://github.com/alicevision/meshroom
Here is some more info from their docu: https://meshroom-manual.readthedocs.io/en/bibtex1/node-reference/nodes/Texturing.html
and https://meshroom-manual.readthedocs.io/en/latest/feature-documentation/nodes/Texturing.html

Their most interesting reference is Seamless Image-Based Texture Atlases using Multi-band Blending: http://imagine.enpc.fr/publications/papers/ICPR08a.pdf

I don't see a public repo but you can check on Google Scholar which papers cite this paper. Those might have public code which we can use.

from pix2model.

Ok, I will look into it

from pix2model.

Nerfstudio also has a texturing based on xatlas, see export_textured_mesh(): https://github.com/nerfstudio-project/nerfstudio/blob/4545dcb6b30538013e27ddcdcbcecfc0402c5007/nerfstudio/exporter/texture_utils.py

Not sure if this is better than the current attempt but it combines color extraction and unwrapping. If the NeRF-Variant works, we might switch to Nerfstudio anyway.

from pix2model.

we want reconstruction and coloring independent of each other to get the best method for each. we checked several methods and packages but didn't find an easy solution yet.
openmvs: how to replace reconstruction?
xatlas: how to project images on mesh (with texture)?
nerfstudio: how to replace nerf with photo?
TEXTure: how to use?
alicevision: how to use texturing? use pipeline except reconstruction?

from pix2model.

found a repo based on a 2014 paper that solves exactly our problem: mesh+ registered images -> texture
https://github.com/nmoehrle/mvs-texturing
https://www.youtube.com/watch?v=Ie-qLJdmlLI

also this improvement, but sadly without public source: https://cseweb.ucsd.edu/~viscomp/projects/SIG17TextureMapping/
there is this unofficial repo but the it's unclear whether it does what we want: https://github.com/OneEyedEagle/EAGLE-TextureMapping

from pix2model.

still not solved. no available library exactly matches our problem. a seminar paper shows that AliceVision provides the best quality, then OpenMVG. easiest to implement is probably https://github.com/nmoehrle/mvs-texturing

from pix2model.

Texturing with OpenMVG is implemented and running

from pix2model.