After searching for a while, I found a website that had that answer, basically you need to replace https://github.com/zang09/ORB_SLAM3_ROS2/blob/00c54335ccc010d74c1e24e336aa817604124947/src/rgbd/rgbd-slam-node.cpp#L11C1-L12C122

Replace with the following lines:

rgb_sub = std::make_shared<message_filters::Subscriber<ImageMsg> >(this, "/camera/rgb");
depth_sub = std::make_shared<message_filters::Subscriber<ImageMsg> >(this, "/camera/depth");

Now it should work.

Originally posted by @RodrigofrBastos in #24 (comment)

what's wrong with this error?

I have seen that there is someone else who has had the same error, but I cant find any solution. OrbSlam3 works fine in the euroc demo. Here is the full error:

nikolai@NikUbuntu22:~/colcon_ws$ ros2 run orbslam3 mono /home/nikolai/colcon_ws/src/orbslam3/vocabulary/ORBvoc.txt /home/nikolai/colcon_ws/src/orbslam3/config/monocular/TUM1.yaml

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Initialization of Atlas from scratch
Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:

Camera Parameters:

• Camera: Pinhole
• Image scale: 1
• fx: 517.306
• fy: 516.469
• cx: 318.643
• cy: 255.314
• k1: 0.262383
• k2: -0.953104
• p1: -0.005358
• p2: 0.002628
• k3: 1.16331
• fps: 30
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 1000
• Scale Levels: 8
• Scale Factor: 1.2
• Initial Fast Threshold: 20
• Minimum Fast Threshold: 7
  There are 1 cameras in the atlas
  Camera 0 is pinhole
  slam changed
  ============================
  Starting the Viewer
  [ros2run]: Segmentation fault

ROS2 Humble
Ubuntu 22.04
I tried this already. Not working.

rgb_sub = std::make_shared<message_filters::Subscriber<ImageMsg> >(this, "/camera/rgb");
depth_sub = std::make_shared<message_filters::Subscriber<ImageMsg> >(this, "/camera/depth");

The program crashes without showing output.
ros2 run orbslam3 stereo vocabulary/ORBvoc.txt BFS.yaml False

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Stereo

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Initialization of Atlas from scratch
Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:

Camera Parameters:

Camera: Pinhole
Image scale: 0.25
fx: 1749.73
fy: 1764.11
cx: 255.644
cy: 114.834
k1: -0.00412
k2: 0.723251
p1: -0.007554
p2: 0.001609
k3: 0
fps: 30
color order: BGR (ignored if grayscale)
Depth Threshold (Close/Far Points): 50.0339

ORB Extractor Parameters:

Number of Features: 1250
Scale Levels: 8
Scale Factor: 1.2
Initial Fast Threshold: 20
Minimum Fast Threshold: 7
There are 1 cameras in the atlas
Camera 0 is pinhole
Starting the Viewer
Shutdown
Saving keyframe trajectory to KeyFrameTrajectory.txt ...
double free or corruption (out)
[ros2run]: Aborted

I tried to test orb-slam3 using ros2 run orbslam3 mono /home/zjp/ORB/ORB_SLAM3/Vocabulary/ORBvoc.txt /home/zjp/ORB/ORB_SLAM3/Examples/Monocular/TUM1.yaml.
I modified the image topic in ORB_SLAM3_ROS2/src/monocular/monocular-slam-node.cpp in advance to the topic published by my oak-d camera.
I also checked it with rqt_graph, but it still shows a black screen and WAITING FOR IMAGES. It is also blank in the map viewer.

I saw someone say it is because of the camera resolution, but I actually only use 320×180 images. I checked the bandwidth and it is only 2M. Someone else said that you can try it a little longer. He mentioned that he went to the bathroom to wash his hands and came back to find that it was running normally (here). Now, my hands are as clean as they can be, but I am still WAITING FOR IMAGES.

Does anyone have any thoughts on this issue?

Hello, I operated according to README .
I executes instructions ros2 run orbslam3 mono/vocabulary/ORBvoc. TXT . / config/monocular/TUM1 yaml, with output " Input sensor was set to: Monocular", however vocabulary may not be loaded. Execution ended without the word "Loading ORB Vocabulary. This could take a while...Vocabulary loaded!" appearing.
Can anyone help me?

Hi guy i try to colcon build ORB_SLAM3_ROS2 after i run this line "colcon build --symlink-install --packages-select orbslam3" this error happens.

i use
ros2 foxy
opencv 4.2.0
ubuntu 20.04
anyone know how to fix thankyou!

Hi, people.
I modified the original codebase so that I could use it inside navigation.
https://github.com/jnskkmhr/orbslam3/tree/main

Here is what I did:

• Modified the node to provide topics (tf2, tracked image, pose topics)
• The stereo-inertial node did not work because of QOS setting in IMU subscriber, so I fixed this.
• SLAM pose was an optical frame pose expressed in OpenCV frame, but I needed camera frame pose in ROS FLU coordinate.
• The node provide map to odom (localization) transform.
• Docker for easy development.

Currently, unfortunately, the stereo-inertial node does not work properly.
I see IMU initialization takes a lot of time, and the pose sometimes jumps.
This might be because of the wrong calibration configuration.
Please send PR if you know how to solve issues.

System Setup:

• ROS2 Humble
• Ubuntu 22.04
• Simulated RGB-D camera (gazebo Ignition)

When setting everything to run ORBSLAM3, trying to run $ ros2 run orbslam3 rgbd PATH_TO_VOCABULARY PATH_TO_YAML_CONFIG_FILE and receiving the following output:

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Initialization of Atlas from scratch 
Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 
There are 1 cameras in the atlas
Camera 0 is pinhole
Shutdown

Saving keyframe trajectory to KeyFrameTrajectory.txt ...
double free or corruption (out)
[ros2run]: Aborted

For solving this error what is needed to do?

Hi, i think what you are doing is interesting and you did great job. But i facing some problem with compile
when i run $ colcon build --symlink-install --packages-select orbslam3
I got
li@li:~/colcon_ws$ colcon build --symlink-install --packages-select orbslam3
Starting >>> orbslam3
--- stderr: orbslam3
CMake Error at ament_cmake_symlink_install/ament_cmake_symlink_install.cmake:100 (message):
ament_cmake_symlink_install_directory() can't find
'/home/li/colcon_ws/src/orbslam3_ros2/launch'
Call Stack (most recent call first):
ament_cmake_symlink_install/ament_cmake_symlink_install.cmake:317 (ament_cmake_symlink_install_directory)
cmake_install.cmake:41 (include)

make: *** [install] Error 1

Failed <<< orbslam3 [1.39s, exited with code 2]
Do you how to deal with it ?

Hi!
I use ROS2 Humble on WSL Ubuntu witn Intel i5-10600K processor.
tried several times to build with colcon build --symlink-install --packages-select orbslam3
last time was waiting for 1 hour, but nothing happaned, it stuck on 66%
I tried to verbose the output with
colcon build --symlink-install --packages-select orbslam3 --event-handlers console_direct+ --cmake-args -DCMAKE_VERBOSE_MAKEFILE=ON

And after 2 hours i got "c++: fatal error: Killed signal terminated program cc1plus"

**
....
....
/home/dan/ros2_ws/src/orbslam3_ros2/src/stereo-inertial/stereo-inertial.cpp: In function ‘int main(int, char**)’:
/home/dan/ros2_ws/src/orbslam3_ros2/src/stereo-inertial/stereo-inertial.cpp:22:19: warning: ISO C++ forbids converting a string constant to ‘char*’ [-Wwrite-strings]
22 | argv[4] = "false";
| ^~~~~~~
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
gmake[2]: *** [CMakeFiles/stereo-inertial.dir/build.make:93: CMakeFiles/stereo-inertial.dir/src/stereo-inertial/stereo-inertial-node.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:224: CMakeFiles/stereo-inertial.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
/usr/bin/ld: CMakeFiles/stereo.dir/src/stereo/stereo-slam-node.cpp.o: undefined reference to symbol 'ZN2cv23initUndistortRectifyMapERKNS_11_InputArrayES2_S2_S2_NS_5Size_IiEEiRKNS_12_OutputArrayES7'
/usr/bin/ld: /lib/x86_64-linux-gnu/libopencv_calib3d.so.4.5d: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/stereo.dir/build.make:221: stereo] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:198: CMakeFiles/stereo.dir/all] Error 2
gmake: *** [Makefile:149: all] Error 2

Failed <<< orbslam3 [2h 0min 27s, exited with code 2]
Summary: 0 packages finished [2h 0min 36s]

**
and when trying to bult it again I get error

usr/bin/ld: CMakeFiles/stereo.dir/src/stereo/stereo-slam-node.cpp.o: undefined reference to symbol 'ZN2cv23initUndistortRectifyMapERKNS_11_InputArrayES2_S2_S2_NS_5Size_IiEEiRKNS_12_OutputArrayES7'
/usr/bin/ld: /lib/x86_64-linux-gnu/libopencv_calib3d.so.4.5d: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

What could be wrong?
Thanks!

ros2 run orbslam3 rgbd /home/wang/ros2_ws_d435/src/orbslam3_ros2/vocabulary/ORBvoc.txt /home/wang/ros2_ws_d435/src/orbslam3_ros2/config/rgb-d/TUM1.yaml
ros2 bag play taiKong/MH_01_easyros2（将ros1的bag转化成的ros2的）
两个命令分别如上所示。

但是结果如图所示，一直等待一直没有显示

/usr/local/include/sophus/common.hpp:36:10: fatal error: fmt/format.h: No such file or directory
36 | #include <fmt/format.h>

hi, i got this problem when compiling the project, i do install Sophus, but it seems like something wrong with Sophus,
how should i deal with it ?

When running a USB monocular camera, there is image output, but there are always indirect errors:
ERROR: Frame with a timestamp older than previous frame detected!

Thanks a lot for this beautiful work. I am publishing the sensor.image/Image type message from my one ROS2 node and trying to run ORB-Slam3 for the published topics. I have also edited the topic name in the monocular-slam-node.cpp. Even though I can see the stream of images as a video(continous frames) from the subscriber, the ORBSLAM3-for-ROS2 stops working as soon as it receives one frame. Any idea what I am doing wrong?

v3@v3:~/ORB_SLAM3$ ros2 run orbslam3 mono /home/v3/colcon_ws/src/orbslam3_ros2/vocabulary/ORBvoc.txt /home/v3/colcon_ws/src/orbslam3_ros2/config/rgb-d/TUM1.yaml

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Initialization of Atlas from scratch
Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:

Camera Parameters:

• Camera: Pinhole
• Image scale: 1
• fx: 517.306
• fy: 516.469
• cx: 318.643
• cy: 255.314
• k1: 0.262383
• k2: -0.953104
• p1: -0.005358
• p2: 0.002628
• k3: 1.16331
• fps: 30
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 1000
• Scale Levels: 8
• Scale Factor: 1.2
• Initial Fast Threshold: 20
• Minimum Fast Threshold: 7
  There are 1 cameras in the atlas
  Camera 0 is pinhole
  Starting the Viewer
  [ros2run]: Segmentation fault

Hardware: Realsense D435i

OS: Ubuntu 22.04

ROS2 Humble

I have changed the node file as here

1. launch the realsense ros2 wrapper: ros2 launch realsense2_camera rs_launch.py enable_infra1:=true enable_infra2:=true enable_accel:=true enable_gyro:=true unite_imu_method:=2 infra_width:=640 infra_height:=480
2. topic list:

/camera/camera/accel/imu_info
/camera/camera/accel/metadata
/camera/camera/accel/sample
/camera/camera/color/camera_info
/camera/camera/color/image_raw
/camera/camera/color/image_raw/compressed
/camera/camera/color/image_raw/compressedDepth
/camera/camera/color/image_raw/theora
/camera/camera/color/metadata
/camera/camera/depth/camera_info
/camera/camera/depth/image_rect_raw
/camera/camera/depth/image_rect_raw/compressed
/camera/camera/depth/image_rect_raw/compressedDepth
/camera/camera/depth/image_rect_raw/theora
/camera/camera/depth/metadata
/camera/camera/extrinsics/depth_to_accel
/camera/camera/extrinsics/depth_to_color
/camera/camera/extrinsics/depth_to_gyro
/camera/camera/extrinsics/depth_to_infra1
/camera/camera/extrinsics/depth_to_infra2
/camera/camera/gyro/imu_info
/camera/camera/gyro/metadata
/camera/camera/gyro/sample
/camera/camera/imu
/camera/camera/infra1/camera_info
/camera/camera/infra1/image_rect_raw
/camera/camera/infra1/image_rect_raw/compressed
/camera/camera/infra1/image_rect_raw/compressedDepth
/camera/camera/infra1/image_rect_raw/theora
/camera/camera/infra1/metadata
/camera/camera/infra2/camera_info
/camera/camera/infra2/image_rect_raw
/camera/camera/infra2/image_rect_raw/compressed
/camera/camera/infra2/image_rect_raw/compressedDepth
/camera/camera/infra2/image_rect_raw/theora
/camera/camera/infra2/metadata
/parameter_events
/rosout
/tf_static

3. launch the rgbd node with ros2 run orbslam3 rgbd <my_path>/vocabulary/ORBvoc.txt <my_path>/config/rgb-d/RealSense_D435i.yaml

4. Error:
   

Hi @zang09,

i have a question regarding the installation.
You mentioned in the Prerequisites, that you tested the humble-branch with:

Ubuntu 22.04
ROS2 Humble
OpenCV 4.5.4
Build ORB_SLAM3

How did you installed than ORB_SLAM3 ? Which OpenCV version did you used? 4.0 or your tested 4.5.4?

Tanks in advance!

i have opencv 4.2.0 installed and i think it supports 4.0.9 if needed , so how can i fix this issue?
im using ubuntu 20.04lts + ros2 foxy+ orbslam3

Hello,
Getting below error while running
lib/orbslam3_ros2/rgbd: error while loading shared libraries: libORB_SLAM3.so: cannot open shared object file: No such file or directory

Have below configuration and steps
Ros2 Galactic
opencv 4.6.0
Installed ORB_SLAM. build is success and libORB_SLAM3.so created
Created ROS 2 workspace from repository
Changed path on 2 files and colcon build

Please let me know if anything I am missing
Thanks

Hi there,

I have trouble running the ROS 2 nodes.

Issue

The rgbd and the stereo node both crash after I run them.

Setup

I have set up a Dockerfile to create an environment with all the necessary dependencies for ORB-SLAM3. After compiling ORB-SLAM3, I proceed to clone the ORB_SLAM3_ROS2 repo from Github and then I switch to the humble branch since my Docker environment is running ROS 2 Humble.

I configure the path to my ORB-SLAM3 installation and I use colcon build to create the orbslam3 package. I then source the workspace.

Expected behaviour

When I run ros2 run orbslam3 rgbd <PATH_TO_VOC_FILE> <PATH_TO_CAMERA_CONFIG> --ros-args <topic remappings> I expect that the node should initiate the parameters from the indicated config file and then start applying SLAM.

Actual behaviour

When I run the following command, I can see that it is loading the config file. I put everything in a launch file, but the result is the same when I run it with ros2 run and all the parameters.

The node shuts down with the following terminal output:

root@adminuser-Precision-5560:/home/orb/ros2_ws# ros2 launch leorover_orbslam3 orbslam3_rgbd.launch.py 
[INFO] [launch]: All log files can be found below /root/.ros/log/2023-08-04-12-52-17-895346-adminuser-Precision-5560-537
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [rgbd-1]: process started with pid [538]
[rgbd-1] 
[rgbd-1] ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
[rgbd-1] ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
[rgbd-1] This program comes with ABSOLUTELY NO WARRANTY;
[rgbd-1] This is free software, and you are welcome to redistribute it
[rgbd-1] under certain conditions. See LICENSE.txt.
[rgbd-1] 
[rgbd-1] Input sensor was set to: RGB-D
[rgbd-1] 
[rgbd-1] Loading ORB Vocabulary. This could take a while...
[rgbd-1] Vocabulary loaded!
[rgbd-1] 
[rgbd-1] Initialization of Atlas from scratch 
[rgbd-1] Creation of new map with id: 0
[rgbd-1] Creation of new map with last KF id: 0
[rgbd-1] Seq. Name: 
[rgbd-1] 
[rgbd-1] Camera Parameters: 
[rgbd-1] - Camera: Pinhole
[rgbd-1] - Image scale: 0.5
[rgbd-1] - fx: 308.6
[rgbd-1] - fy: 308.681
[rgbd-1] - cx: 162.318
[rgbd-1] - cy: 121.231
[rgbd-1] - k1: 0
[rgbd-1] - k2: 0
[rgbd-1] - p1: 0
[rgbd-1] - p2: 0
[rgbd-1] - fps: 30
[rgbd-1] - color order: RGB (ignored if grayscale)
[rgbd-1] 
[rgbd-1] Depth Threshold (Close/Far Points): 2.98185
[rgbd-1] 
[rgbd-1] ORB Extractor Parameters: 
[rgbd-1] - Number of Features: 1250
[rgbd-1] - Scale Levels: 8
[rgbd-1] - Scale Factor: 1.2
[rgbd-1] - Initial Fast Threshold: 20
[rgbd-1] - Minimum Fast Threshold: 7
[rgbd-1] There are 1 cameras in the atlas
[rgbd-1] Camera 0 is pinhole
[rgbd-1] Shutdown
[rgbd-1] 
[rgbd-1] Saving keyframe trajectory to KeyFrameTrajectory.txt ...
[rgbd-1] double free or corruption (out)
[ERROR] [rgbd-1]: process has died [pid 538, exit code -6, cmd '/home/orb/ros2_ws/install/orbslam3/lib/orbslam3/rgbd /home/orb/ros2_ws/install/leorover_orbslam3/share/leorover_orbslam3/vocabulary/ORBvoc.txt /home/orb/ros2_ws/install/leorover_orbslam3/share/leorover_orbslam3/config/RealSense_D455.yaml --ros-args -r __node:=leorover_orbslam3_rgbd -r camera/rgb:=/leo02/realsense_camera_leo02/color/image_raw -r camera/depth:=/leo02/realsense_camera_leo02/aligned_depth_to_color/image_raw'].

Question

Can you help me figure out why the node shuts down after launching it? Am I missing something? Is there something wrong in how I run the node? Am I missing a configuration?

With kind regards,
Dave

I see that there are folders that have scripts for inertial, rgbd, sterio and sterio inertial. I believe that it should be possible for obrslam3 to handle monocular inertial data. I was wondring if any of you have the three C++ scripts for this implementation?

i want to combine gazebo and ros2_orbslam3.
on the photo,the image should be represented on orb slam3,but it doesn't work,i don't know how to solve this problem.

when I build the ros2 workspace, error occurs:

system: Ubuntu 20.04 + ros galactic

Hey zang, have you successfully run stereo-inertial mode in realtime?
By my side it also output IMU not initialzation and Empty IMU Measurement Vector!!
Do you have any idea how to solve this?

when I run with momo,it seems like that it initialized successfully,but i got this error

New Map created with 91 points
one frame has been sent
Fail to track local map!
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.5.4) ./modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<1>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = cv::impl::<unnamed>::NONE; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 2

Aborted (core dumped)

I'm using a simulate stereo cam, on a simulated robot with IMU, to use stereo-inertial mode, but my map results was horrible, anyone can help-me to calibrate my sensors?