Curved Depth Map about noncuboidroom HOT 14 CLOSED

@bertjiazheng and @marcomiglionico94
Could you please share with us the code you implemented to visualise the 3D model?

from noncuboidroom.

Hi @marcomiglionico94

I think this may be due to the incorrect depth distance we used. You may try two different depths: i) distance from the point to the image plane or ii) distance from point to the camera center.

Best,
Jia

from noncuboidroom.

Hi @marcomiglionico94

I think this may be due to the incorrect depth distance we used. You may try two different depths: i) distance from the point to the image plane or ii) distance from point to the camera center.

Best, Jia

Hi thanks for the quick answer, where exactly I can find that distance parameter that you used? Can you refer to the file or line of code where I can find it

from noncuboidroom.

For a point (x,y,z) in 3D space, the depth map extracted by network is the z coordinate, not the np.sqrt(x^2+y^2+z^2).

from noncuboidroom.

Hi @marcomiglionico94

I think this may be due to the incorrect depth distance we used. You may try two different depths: i) distance from the point to the image plane or ii) distance from point to the camera center.

Best, Jia

Is it possible to know at least what kind of visualization you use for the paper? Did you use Open3D or something else?

from noncuboidroom.

For a point (x,y,z) in 3D space, the depth map extracted by network is the z coordinate, not the np.sqrt(x^2+y^2+z^2).

I know that, but in the code the depth map is basically created in ConvertLayout function, in particular in this part of code:

n_d = param[:3] / np.clip(param[3], 1e-8, 1e8) # meter n/d n_d = n_d[np.newaxis, np.newaxis, :] inverdepth = -1 * np.sum(np.dot(n_d, K_inv) * ixy1map, axis=2) depth[mask] = inverdepth[mask]

My question is which parameters can I change so that the depthmap is not curved? I imagine that in the reconstruction you show in the paper you used a depth map, to make the 3d visualization. But not a curved one otherswise your results will look like the image I posted at the beginning of this issue.

from noncuboidroom.

Hi @marcomiglionico94
I think this may be due to the incorrect depth distance we used. You may try two different depths: i) distance from the point to the image plane or ii) distance from point to the camera center.
Best, Jia

Is it possible to know at least what kind of visualization you use for the paper? Did you use Open3D or something else?

We use Open3D to visualize the reconstructed 3D model.

from noncuboidroom.

Hi, @marcomiglionico94
There is a mistake about the variable name. The returned depth in code is the inverse of depth. So the actually depth is by 1/depth.

from noncuboidroom.

Hi, @marcomiglionico94 There is a mistake about the variable name. The returned depth in code is the inverse of depth. So the actually depth is by 1/depth.

Hi thanks for your reply. I am not very sure about this. When i return 1/depth the results of the depthmap are very strange since only a small part of the scene is included, the rest is missing. In fact before the values of the depthmap are between 0 and 1, but after i do 1/depth all the values are around 2 or 3, which I don't think is correct. Maybe you mean depth = np.ones_like(depth) - depth ??
Trying to figure out what is the problem

from noncuboidroom.

We add an example on NYU303 dataset. The actually depth is 1/depth, not depth = np.ones_like(depth) - depth.

from noncuboidroom.

Using depth = np.ones_like(depth) - depth I was able to obtain the following visualization, which seems correct.

Using 1/depth instead, this is what i was getting for the same image:

from noncuboidroom.

We add an example on NYU303 dataset. The actually depth is 1/depth, not depth = np.ones_like(depth) - depth.

Hello, I realized that if I use 1/depth it only works if then I divide the values of the depthmap by the max value of it, only to scale the values between 0 and 1 so that I can visualize them. Do you know if that is the correct approach? or is there another way to scale the values of the depthmap between 0 and 1 so that I can visualize it?

from noncuboidroom.

@marcomiglionico94 I wonder if it could be possible for you to briefly discuss how you solve the problem?

from noncuboidroom.

hey @marcomiglionico94, can you share the code you have implemented for 3D visualization. Also, I am trying to implement this function DisplayMeshLayout in reconstruction.py for my custom images in test.py

It needs these as input def DisplayMeshLayout(ups, downs, attribution, pwalls, pceiling, pfloor, ixy1map, K_inv)

This is I have implemented it in test.py

DisplayMeshLayout(ups, downs, attribution, pwalls=params_layout, pceiling=pceiling, pfloor=pfloor, ixy1map=inputs['ixy1map'][i].cpu().numpy(), K_inv = np.linalg.inv(inputs['intri'][i].cpu().numpy()))

I am getting this error:

Traceback (most recent call last):
  File "/workspace/hardik/NonCuboidRoom/test.py", line 508, in <module>
    test_custom(model, criterion, dataloader, device, cfg)
  File "/workspace/hardik/NonCuboidRoom/test.py", line 370, in test_custom
    DisplayMeshLayout(ups, downs, attribution, pwalls=params_layout, pceiling=pceiling, pfloor=pfloor, ixy1map=inputs['ixy1map'][i].cpu().numpy(), K_inv = np.linalg.inv(inputs['intri'][i].cpu().numpy()))
  File "/workspace/hardik/NonCuboidRoom/models/reconstruction.py", line 669, in DisplayMeshLayout
    poly = empty_polygon.intersection(poly)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/layout/lib/python3.12/site-packages/shapely/geometry/base.py", line 599, in intersection
    return shapely.intersection(self, other, grid_size=grid_size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/layout/lib/python3.12/site-packages/shapely/decorators.py", line 77, in wrapped
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/layout/lib/python3.12/site-packages/shapely/set_operations.py", line 131, in intersection
    return lib.intersection(a, b, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
shapely.errors.GEOSException: TopologyException: side location conflict at 297 72.826979472140764. This can occur if the input geometry is invalid.

from noncuboidroom.