Comments (6)
Hi!
I have been trying to extract an OBJ file with marching cubes from your Voxel NeRF implementation.
My initial idea was to make your code work similar to the Stable-DreamFusion's mesh extraction implemented here: https://github.com/ashawkey/stable-dreamfusion/blob/53d4cef2055234e242d0b9b247c6f41bee7bbf46/nerf/renderer.py#L148L328
I like the utilization of xatlas here since the generated UV becomes very easily editable.
However, I got confused at one point: Here at line 170, https://github.com/ashawkey/stable-dreamfusion/blob/53d4cef2055234e242d0b9b247c6f41bee7bbf46/nerf/renderer.py#L170, the density function retrieves the albedo information as well as the sigmas. The same density function is utilized later at 244.
Furthermore, as far as I understand, they retrieve the mask information from the nvdiffrast after rasterization.
I have read your implementation, and I saw that you generate your own alpha masks and feats/colors.
I would like to utilize your implementation to export mesh in a similar manner and then create a PR for you to review and utilize. However, I was unsure at the points below, could you correct me if I am wrong?
- I believe I should utilize the sigmas generated by compute_density_feats function within Vox_RF class.
- Albedo channel/colors/feats (I am unsure about the difference here as well) should be retrieved from compute_app_feats function within Vox_RF class.
- Mask should be retrieved by running make_alpha_mask function and then utilizing Vox_RF.alphaMask
- Is there another way of retrieving sigmas, feats and masks in a single dictionary? I can write a function to retrieve them all from a single function as well. Does state_dict do that?
- I am unsure about the binary dilation and erosions within the export_mesh function of Stable-DreamFusion implementation and the nearest neighbors search for anti-aliasing part. Would you like me to implement that part as well?
from sjc.
Same here! It would be a great addition for this demo!
Congratulations for the amazing work by the way!
from sjc.
@caseybasichis thanks for ur interest we are doing a few other things but it's on our mind.
from sjc.
Did you make any progress exporting an obj from NeRF?
from sjc.
Looks like Sparsefusion just released some of their code! Could be interesting to integrate!
from sjc.
Any update on this
from sjc.
Related Issues (20)
- Different angles in SD HOT 1
- The next SD version could speed up the generation of 3d models ? "Stable Diffusion will generate 30 images per second instead of one image in 5.6 seconds" HOT 1
- Drop the duplicate running process in the queue on Hugging Face
- Another viewpoint to interpret the score HOT 5
- Details about the eq19 HOT 2
- About bs=1 and the use of the Monte-Carlo estimate HOT 4
- question of the provement of the formula17 HOT 3
- Question about train_eye_with_prompts HOT 3
- scripts for figure 4 HOT 2
- Can you provide the rendered videos for better comparison?
- About OOD issues. HOT 2
- Questions about center depth loss in the paper HOT 5
- Where is the jacobian implementation for the backpropogation?
- Bibtex not up to date HOT 1
- noisy_x scaling before diffusion models HOT 1
- Query : 4 channels in nerf output HOT 5
- Question about paper
- How to train the model with a new dataset? HOT 1
- Confused about sampled poses
- question about formula HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sjc.