Comments (2)
Did you mean view-dependent RGB values? That would be a good idea I think, but I am struggling with other things recently:(
from tensorf.
Hi again - just wanted to give an update and get feedback if possible:
I added this new function in tensorBase.py
- alot of the code is the same as in the forward()
method, so I'll only show a few lines and TODOs where it's a little unfinished:
def forward_based_on_ray_points(
self,
rays_pts, # [n, 3] tensor
white_bg=True,
is_train=False,
ndc_ray=False,
):
"""
Returns only the RGB and alpha values for each point in a dense grid,
based on only the XYZ locations of the grid cell centers.
"""
if ndc_ray:
# TODO - figure this out later
else:
# pretending like the each ray point is a ray origin
xyz_sampled = rays_pts.reshape((
rays_pts.shape[0], 1, rays_pts.shape[1]
))
# we only take 1 sample (and we let viewsdirs = 0, so as not to move from the origin)
z_vals = torch.ones((xyz_sampled.shape[0], 1)).to(xyz_sampled.device)
mask_out_bbox = (
(xyz_sampled < self.aabb[0]) | (xyz_sampled > self.aabb[1])
).any(dim=-1)
ray_valid = ~mask_out_bbox
dists = torch.cat(
(z_vals[:, 1:] - z_vals[:, :-1], torch.zeros_like(z_vals[:, :1])),
dim=-1,
)
...
# pass in None for the viewdirs - b/c none the render modules need it
if app_mask.any():
app_features = self.compute_appfeature(xyz_sampled[app_mask])
valid_rgbs = self.renderModule(
xyz_sampled[app_mask], None, app_features
)
rgb[app_mask] = valid_rgbs
...
# TODO: unsure about how to get depth maps, as it requires the viewdir?
# with torch.no_grad():
# depth_map = torch.sum(weight * z_vals, -1)
# # depth_map = depth_map + (1.0 - acc_map) * rays_chunk[..., -1]
# depth_map = (
# depth_map + (1.0 - acc_map) * torch.ones_like(xyz_sampled)[..., -1]
# ) # just a guess
return rgb_map, None, alpha
Thank you Anpei for confirming this would be at least possible!
from tensorf.
Related Issues (20)
- Use on unbounded scenes? HOT 1
- Problem training on own blender dataset HOT 2
- Problem on replicating LLFF results HOT 1
- Question aboout training with depth supervision
- question about distance_scale HOT 1
- About hyperparameter near_far HOT 1
- May I ask which result data did you use in papers?
- Test PSNR is higher than train PSNR.
- custom data results suffer from floaters HOT 2
- Results with Ignatius not reproducible due to problematic intrinsics file in data HOT 2
- Different logic to compute nSamples in two different places
- Question about weights
- activation function problem (math.)
- Question regarding depth map computation in volume rendering
- Shape of AlphaGridMask is transposed
- Some question about the outer product between the vector and the plane
- Issues in indoor scene
- Coordinate system used by TensoRF as opposed to that of NeRF and instant-ngp
- implement the "render_path" operation on the Blender dataset
- the link of pretrained checkpoints and results failed
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorf.