jdgalviss / jetbot-ros2 Goto Github PK

ROS 2 implementation of a Teleoperated robot with live video feed using webrtc and SLAM using realsense's stereocameras. Running on a Jetson Nano

License: MIT License

Python 59.70% JavaScript 27.99% HTML 6.95% Shell 5.35%

jetbot rtabmap ros2 cartographer teleoperation webrtc

jetbot-ros2's Introduction

Jetbot-ros2

explore_lite.mp4

This is an implementation of a mobile robot in ros2. The software includes the following functionalities:

Teleoperation through websockets with live video feed using webrtc (aiortc).
Integration of Intel realsense d435 and t265 cameras for depth estimation and localization respectively.
Autonomous Navigation using Nav2
Autonomous Exploration using m-explore
2D SLAM with slam-toolbox.
3D SLAM using rtabmap.

I used the Xiaor Geek Jetbot as a base platform and modified it to include a wide-angle camera, as well as the Intel Realsense d435 and t265.

Requirements

ROS2 eloquent.
librealsense2.
Motor Drivers - In this case, installed from the jetbot's repository.

Installation

Clone this repo and its submodules.

git clone --recurse-submodules https://github.com/jdgalviss/jetbot-ros2.git

Clone additional thirdparty packages

SLAM-TOOLBOX

cd jetbot-ros2/dev_ws/src
git clone -b eloquent-devel [email protected]:stevemacenski/slam_toolbox.git

Navigation2

git clone https://github.com/ros-planning/navigation2.git --branch eloquent-devel

m-explore

git clone -b eloquent https://github.com/robo-friends/m-explore-ros2.git

BehaviorTree.CPP

git clone https://github.com/BehaviorTree/BehaviorTree.CPP.git
cd BehaviorTree.CPP
git checkout 3.5.1
cd ../..

Copy our modified files

cd thirdparty_files
source copy_files.sh
cd ../..

Teleoperation support

Install aiortc for webrtc support.

pip3 install crc32c==2.0
pip3 install aiortc==0.9.28
pip3 install aiohttp==3.6.2

Install pyfakewebcam so that camera frames can be modified inside a ROS2 node and then shared through webrtc:
```
apt-get install v4l2loopback-utils
pip3 install pyfakewebcam==0.1.0
```
Write a service that creates fake webcam devices that can be used to share camera frames.
```
gedit /etc/rc.local
```
Copy and paste in file:
```
#!/bin/sh -e
modprobe v4l2loopback devices=2 # will create two fake webcam devices
exit 0
```
Save the file and make it executable with this command:
```
chmod +x /etc/rc.local
```

SLAM Support

Install rtabmap and rtabmap_ros following these instructions in the branch ros2.

Build ros 2 workspace

cd dev_ws
rosdep install -y -r -q --from-paths src --ignore-src --rosdistro eloquent
colcon build --symlink-install

Run

Teleoperation

Run local teleoperation server.
```
python3 local_server/webcam.py
```
In a browser, open the teleoperation interface by going to: <jetson_nano's ip-address>:8080
In a new terminal, run the motion control launchfile to start streaming video and receiving motion commands.
```
ros2 launch motion_control jetbot_launch.py
```

Navigation

In a new terminal, run nav2

ros2 launch nav2_bringup nav2_navigation_launch.py

SLAM

To Run SLAM.
- For 2D-SLAM, in another terminal:
```
ros2 launch realsense_ros2 realsense_launch.py
```
  In another terminal:
```
ros2 launch slam_toolbox online_async_launch.py
```
- For 3D-SLAM, in another terminal:
```
ros2 launch realsense_ros2 slam_rtabmap_launch.py
```
3D Dense SLAM is too resource consuming for the Jetson Nano, in this case it is recommended to run it on a remote host. For this, simply set the same DOMAIN_ID on both the Jetson Nano and the remote host. (e.g. export DOMAIN_ID=0) and run the cameras in the Jetson Nano:
```
ros2 launch realsense_ros2 realsense_launch.py
```
Comment the nodes corresponding to the cameras on the host and run the rtabmap launch:
```
ros2 launch realsense_ros2 slam_rtabmap_launch.py
```

Exploration

In a new terminal, run explore_lite

ros2 run explore_lite explore --ros-args -p costmap_topic:=/map -p visualize:=true -p use_sim_time:=false -p min_frontier_size:=0.4 -p planner_frequency:=0.5

Some Results

Video

explore_lite.mp4

jetbot-ros2's People

Contributors

Stargazers

Watchers

jetbot-ros2's Issues

about the librealsense2

Hello, I would like to ask whether you installed librealsense2 using source code or binary installation?

How can I change the speed of scan??

Hi!

I would like to change the speed of scan.
I guess I should change fps rate, but not sure....

Is it possible to do that??

Thank you

Launch RViz GUI

What scripts would be necessary to execute RViz to navigate setting pose and set a 2D Nav Goal and move to Waypoints?

Installing ROS2 on JetBot - what OS were you using?

ROS2 normally runs on Ubuntu 20.04, but it isn't clear here if you were running an upgraded Ubuntu 18.04 to 20.04 or using some other method. Can you please update here or in your readme which base OS and/or JetPack version you were booting from please.

Support Foxy

Hi,

great project !
I whish I would be there already !

Would it work with ros2-foxy ?
What os Version/Image did you use ?

No eloquent devel for m_explore_ros package

Hi,

Great work for the repo! I am trying to build the m_explore ros2 on a jetson nano 4gb. I can not find eloquent devel branch in the original repo of m_explore_ros2 so I tried installing it from the main branch. however, when I was building the package, an error shows that there is no file named asserts.hpp in rcpputils of ros2 eloquent. So I also went to the official repo of rcpputils does not have asserts.hpp until ros2 foxy distro, which is not compatible with jetson nano. Can anyone help me with this? Thank you!

Version of the Python Packages

I'm trying to rebuild your project with a Jetson Xavier NX.

During the installation of the python package:
pip3 install aiortc
pip tries to install the newest version available which results in some errors (gcc compiler errors).
Afterwards I tested it with an older version of aiortc and it worked instantly.

I really would appreciate if you could tell me the specific versions for the following packages on your rover system:

aiortc
aiohttp
pyfakewebcam

Teleop server as part of the launch script

Hello there,
great project! Is there any particular reason why you need to start local_server/webcam.py manually? Wouldn't it be more convient to somehow make it part of the launch script (although it is not a ROS node)?

“colcon build --symlink-install" Failed

How can I solve this problem? Thanks!!

root@davinci-mini:/usr/repo/jetbot-ros2/dev_ws# colcon build --symlink-install
Starting >>> nav2_common
Starting >>> nav_2d_msgs
Starting >>> behaviortree_cpp_v3
Starting >>> motion_control
--- stderr: nav2_common
Traceback (most recent call last):
File "/opt/ros/eloquent/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py", line 21, in
from catkin_pkg.package import parse_package_string
ModuleNotFoundError: No module named 'catkin_pkg'
CMake Error at /opt/ros/eloquent/share/ament_cmake_core/cmake/core/ament_package_xml.cmake:94 (message):
execute_process(/usr/local/python3.7.5/bin/python3
/opt/ros/eloquent/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py
/usr/repo/jetbot-ros2/dev_ws/src/navigation2/nav2_common/package.xml
/usr/repo/jetbot-ros2/dev_ws/build/nav2_common/ament_cmake_core/package.cmake)
returned error code 1
Call Stack (most recent call first):
/opt/ros/eloquent/share/ament_cmake_core/cmake/core/ament_package_xml.cmake:49 (_ament_package_xml)
/opt/ros/eloquent/share/ament_cmake_core/cmake/core/ament_package.cmake:63 (ament_package_xml)
CMakeLists.txt:10 (ament_package)

Failed <<< nav2_common [2.06s, exited with code 1]
--- stderr: behaviortree_cpp_v3
Traceback (most recent call last):
File "/opt/ros/eloquent/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py", line 21, in
from catkin_pkg.package import parse_package_string
ModuleNotFoundError: No module named 'catkin_pkg'
CMake Error at /opt/ros/eloquent/share/ament_cmake_core/cmake/core/ament_package_xml.cmake:94 (message):
execute_process(/usr/local/python3.7.5/bin/python3
/opt/ros/eloquent/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py
/usr/repo/jetbot-ros2/dev_ws/src/BehaviorTree.CPP/package.xml
/usr/repo/jetbot-ros2/dev_ws/build/behaviortree_cpp_v3/ament_cmake_core/package.cmake)
returned error code 1
Call Stack (most recent call first):
/opt/ros/eloquent/share/ament_cmake_core/cmake/core/ament_package_xml.cmake:49 (_ament_package_xml)
/opt/ros/eloquent/share/ament_cmake_core/cmake/core/ament_package.cmake:63 (ament_package_xml)
CMakeLists.txt:123 (ament_package)

Failed <<< behaviortree_cpp_v3 [2.01s, exited with code 1]
Aborted <<< nav_2d_msgs [2.60s]
Aborted <<< motion_control [4.72s]

Summary: 0 packages finished [7.55s]
2 packages failed: behaviortree_cpp_v3 nav2_common
2 packages aborted: motion_control nav_2d_msgs
3 packages had stderr output: behaviortree_cpp_v3 nav2_common nav_2d_msgs
31 packages not processed

tf setup

Hi, thanks for sharing your work. I want to ask do I need any tf information (e.g. map, base_link, Odom, sensor) if I want to implement it on another UGV (mine is scout-mini). Second is it possible to use it with a ZED2i camera instead of realsense camera?

Nav2 not working

Hi, I am getting this error at runtime:

[ERROR] [gzserver-1]: process has died [pid 21874, exit code -2, cmd 'gzserver -s libgazebo_ros_init.so /home/jetbot/nav2_ws/install/nav2_bringup/share/nav2_bringup/worlds/waffle.model'].

WebRTC without STUN server (local network)

Hey:)
I tried to start a local webRTC connection without a STUN server.
I just started the webcam.py file and connected to the webserver (without ROS and everything).

With the STUN server everything works flawlessly and I get the following messages in the console:

ICE connection state is checking
ICE connection state is complete

But when I disable the STUN Server I only get:

ICE connection state is checking

and it is constantly in this state.
Both PCs are in the same network.
Any idea what could go be wrong?:c