Hi,

I can see the correct mesh+trajectory will run the visualizer.py, but when I check output/tmp_rendering the jpg frames are all black and vis.mp4 is also all black, do know why this is happening?

I also have this [mjpeg @ 0x5566f358ae00] EOI missing, emulating waring will running the visualizer.py

Thanks in advance.

Hello! I encountered the following error while running your code,

I used the nvidia-smi command to find out that my GPU's memory is sufficient.

I suspect this is due to an error with torch.multiprocessing because the use of shared memory causes the device to be unable to continue running once there are many other tasks running on the host, butit seems that the file-sharing strategy is irreplaceable. Can you tell me how to solve this problem? Thank you!

When I visualized the results of the dataset, the resulting view consisted only of the camera's trajectory and no scene mesh.

There is some warning when i run python visualizer.py configs/Replica/room0.yaml --output output/Replica/room0 --top_view --save_rendering:

[mjpeg @ 0x562cad8edf00] overread 8 7424kB time=00:06:44.10 bitrate= 150.5kbits/s speed=3.91x [mjpeg @ 0x562cad8edf00] EOI missing, emulating

Thank you so much for your awesome work!

I installed your repo on WSL2 and docker, and my GPU is 3090. I haven't seen any errors during training.

After running the code, I was so confused why the reconstructed results on TUM (freiburg1_desk) and ScanNet (Scene0000_00) is not satisfactory. I was using the provided config files without making any changes.

Here is the culled mesh of Freiburg1_desk:

As you can see, it appears like a tetrahedron without no details inside.

The same issue is observed with Scannet:

Could you please provide guidance on how to troubleshoot the issue?

I understand from the thesis that feature_plane takes part in the training. But when I debugging,I find that though the reconstruction result is good, the value of feature_plane seems to be not changed,. Is there something wrong with my operation

I do not have any errors or working in the previews steps, but after I run python visualizer.py configs/Replica/room0.yaml --output output/Replica/room0 --top_view --save_rendering

I only got this red line and green line moving video, the background, which is supposed to be the room0, is just white color.

Is this the same for everyone?

Hi Mohammad,

thank you every much for your great job! About 4.2 Runtime Analysis in the paper I have some questions.
When you analysis avarage frame processing time(FPT), do you count mapping and tracking time togother? I mean, do you count FPT after a whole sequence (while blocking visualization part and mesh part)?

In addition, in nice-slam and eslam, you use different strategies for tracking and maping frequency. So you calculate speed FPT based on same completion/completion ratio/accuracy or?

Thanks a lot in advance!
Jonas

Hallo,

thanks for your great job!
I want to compare different reconstructed mesh at same view for Replica Dataset. But actually, at Replica official website they provide the mesh without texture. They provide also ReplicaViewer to check rendered mesh. However, it is hard to adjust recontructed mesh in meshlab with ReplicaViewe.

Is it possible to get textured mesh from ReplicaViewer? How can you check them like Figure 3. in paper?

I really appreciate any help you can provide!

Dakai

Ours : GTX TitanX Ubuntu 20.04
RuntimeError: CUDA out of memory. Tried to allocate 62.00 MiB (GPU 0; 11.92 GiB total capacity; 8.81 GiB already allocated; 50.31 MiB free; 9.09 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

We got this Error while running Replica or TumRGBD.
In the Nice-SLAM model, this error only appeared in the dataset we constructed ourselves, and not in TumRGBD or Replica.
How can we fix it?

H idiap,

Thanks for releasing your code, it really helps me. I'm testing your code on my own dataset, but caught the CUDA Memory Out error. So I follow this issue and change the meshing setting below:

meshing:
  level_set: 0
  resolution: 0.01 #cm # Increase this number to speed up meshing algorithm
  eval_rec: False
  mesh_bound_scale: 1.02

# change to:

meshing:
  level_set: 0
  resolution: 32 #cm # Increase this number to speed up meshing algorithm
  eval_rec: False
  mesh_bound_scale: 1.00

The Memory Out error disappears bug I got another problem:

Traceback (most recent call last):
  File "/home/yanchi/miniconda3/envs/eslam/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/yanchi/miniconda3/envs/eslam/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/yanchi/qudelin/ESLAM/src/ESLAM.py", line 259, in mapping
    self.mapper.run()
  File "/home/yanchi/qudelin/ESLAM/src/Mapper.py", line 473, in run
    self.mesher.get_mesh(mesh_out_file, all_planes, self.decoders, self.keyframe_dict, self.device)
  File "/home/yanchi/qudelin/ESLAM/src/utils/Mesher.py", line 234, in get_mesh
    for i, pnts in enumerate(torch.split(points, self.points_batch_size, dim=0)):
  File "/home/yanchi/qudelin/ESLAM/src/utils/Mesher.py", line 170, in eval_points
    ret = decoders(pi, all_planes=all_planes)
  File "/home/yanchi/miniconda3/envs/eslam/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/yanchi/qudelin/ESLAM/src/networks/decoders.py", line 165, in forward
    raw = raw.reshape(*p_shape[:-1], -1)
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous

It seems that they decoder reshape an empty tensor of shape (0, 4) at the forward step.

I have no idea how to solve this problem, could you provide me something about the setting rule of mesh_bound_scale, resolution, and bound? BTW, would the 3D reconstruction evaluation stay safe if I do not render a 3D ply file?

I'm looking forward to your reply and thanks in advance.

Dose render a image mush be use gt_depth as arg? how can I render a image with Novel View?

Hi Mohammad,

Thank you every much for your great job! About Table 4 in the paper I have some questions.
How did you calculate the number of parameters in the table? Is it the memory of the decoder plus the size of the entire scene representation?

I also tried to calculate the memory of scene room0 of Replica in the same way as in table 4, but found that it is much smaller than the size given in table 4 of the paper (6.79MB), and my calculation is only about 4.587MB. Please point out if I have missed something.

Here is how I calculated it and the results.

For decoder:

        model = config.get_model(cfg)
        model.to("cuda:0")
        memory_usage = torch.cuda.memory_allocated()
        print(f"Memory Usage: {memory_usage} bytes",f"{memory_usage/1024**2}MB")

Memory Usage: 14848 bytes 0.014MB

For Embedding if scene:

        memory = 0
        for i in range(len(planes_xy)):
            memory=+planes_xy[i].element_size()* planes_xy[i].nelement()
        for i in range(len(planes_xz)):
            memory=+planes_xz[i].element_size()* planes_xz[i].nelement()
        for i in range(len(planes_yz)):
            memory=+planes_yz[i].element_size()* planes_yz[i].nelement()
            
        for i in range(len(c_planes_xy)):
            memory=+c_planes_xy[i].element_size()* c_planes_xy[i].nelement()
        for i in range(len(planes_xz)):
            memory=+c_planes_xz[i].element_size()* c_planes_xz[i].nelement()
        for i in range(len(planes_yz)):
            memory=+c_planes_yz[i].element_size()* c_planes_yz[i].nelement()

        print(f"embedding size {memory/1024**2}MB")

embedding size 4.5732421875MB

Thanks a lot in advance!
Cancan

Hi @MohammadJohari ,

Thanks for your excellent work! I find that the depth mask is not used for the mapper color loss https://github.com/idiap/ESLAM/blob/main/src/Mapper.py#L362. Is it intended since I find the depth_mask for tracking color loss can make a lot of difference to the system? Thanks in advance.

Bests,
Weining

Congrats on the impressive work!

Is it possible to test ESLAM on data obtained from a commercial RGBD camera like Intel Realsense or Kinect?

I tried using the config file for TUM RGBD, after I have prepared the rgb.txt and depth.txt files, and the corresponding depth and rgb folders, and I also changed the width and hight and the intrinsic parameters as well, but I got the following error:

Traceback (most recent call last):
File "run.py", line 65, in
main()
File "run.py", line 60, in main
eslam = ESLAM(cfg, args)
File "/home/desk/my_WorkingDirectory/Neural Implicit-Based SLAM/Depth-Based/ESLAM/src/ESLAM.py", line 105, in init
self.frame_reader = get_dataset(cfg, args, self.scale)
File "/home/desk/my_WorkingDirectory/Neural Implicit-Based SLAM/Depth-Based/ESLAM/src/utils/datasets.py", line 71, in get_dataset
return dataset_dict[cfg['dataset']](cfg, args, scale, device=device)
File "/home/desk/my_WorkingDirectory/Neural Implicit-Based SLAM/Depth-Based/ESLAM/src/utils/datasets.py", line 193, in init
self.input_folder, frame_rate=32)
File "/home/desk/my_WorkingDirectory/Neural Implicit-Based SLAM/Depth-Based/ESLAM/src/utils/datasets.py", line 233, in loadtum
pose_data = self.parse_list(pose_list, skiprows=1)
UnboundLocalError: local variable 'pose_list' referenced before assignment

It seems like I should also pass the ground-truth poses (?)

Any advice would be highly appreciated!
Thank you in advance for your time

Hi，
Thank you for your excellent work。For the bound and marching in the configuration file_ Cubes_ I have some questions about bound and I need your help. May I ask what is the reason for setting it here? I think this is a hyperparameter. If I want to use my own dataset to run ESLAM, how should I configure bound and marching_ Cubes_bound。
Thanks in advance.

Hi,
Thanks for making this project open source!

I am trying to run ESLAM on the ScanNet scenes 233, 465 and 54. I created the configs with the bounds borrowed from GO-SLAM.

However, when I run on these three scenes I get the following runtime errors which I have not been able to figure out myself yet.

Scenes 465 and 54 I get the error:

On scene 233 I get the error:

Would be very grateful for ideas on what I can do to fix this. I currently suspect that it might be the bounds of the scenes that is causing this, but thought I'd check with you if I can rule that out as an issue.

Cheers,
Erik

Thank you for your wonderful work,

and have you ever tried to control random number seeds to make the results reproducible? But I was trying to add random seeds and found that there was no guarantee that the output would be consistent every time. And I found that as long as the network performs a backpropagation, the output results at this time are already inconsistent. Do you know how to solve it?

Thank you!

def setup_seed(seed):
    torch.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)
    np.random.seed(seed)
    random.seed(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False