Scenario 1: Detecting multiple objects in the scene Scenario 2: Accurately retaining detection and track ids

Scenario 3: Detecting objects in the reverse direction

Scenario 4: Detecting a virtual vehicle on point-cloud data

Scene: Pointpillar + Regnet Scene: Pointpillar (TensorRT)

1. Create virtual environment

conda create --name rosdeploy python=3.8
conda activate rosdeploy

2. Install pytorch conda install pytorch from https://pytorch.org/ according to cuda version or lower for eg.

conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge

3. Clone this repository

git clone [email protected]:ksm26/ROS-based-3D-detection-Tracking.git

4. Install steps

pip install -U openmim
mim install mmcv-full
mim install mmdet
mim install mmsegmentation

5. Install mmdetection3d

cd mmdetection3d
pip install -e .

6. Install mmdeploy

cd mmdeploy 
pip install -e .

7. Install tf and jsk tf tutorial Bounding box visualization ROS jsk_visualization jsk_recognition_msgs

sudo apt-get install ros-melodic-jsk-recognition-msgs 
sudo apt-get install ros-melodic-jsk-rviz-plugins
sudo apt-get install ros-melodic-jsk-rqt-plugins
sudo apt-get install ros-melodic-jsk-visualization

8. Install tensorrt

pip install nvidia-tensorrt
python3 -m pip install --upgrade tensorrt

Error with rospy.init_node(): An error with ros packages installation into conda environment. Run the above ros_python_environment.yml using:

pip install scikit-build
conda env update --file ros_python_environment.yml

import tf: Error
Import tf does not work with python 3 as it is compiled for python 2. Solution: Recompile with python3 in catkin_workspace
To be able to work with python interpreter (pycharm,visual studio) add the compile path as below:

sys.path.append("~/Desktop/ROS-tracker/catkin_ws/src:/opt/ros/melodic/share")

Source catkin envrionment after tf installation and creation of new catkin workspace

source ~/catkin_ws/devel/setup.bash

Download the checkpoint for 3D-detection model-zoo. This repository evaluates the performance of Pointpillars and Regnet models pretrained on nuScences dataset.
Test by running a rosbag and subscribe to appropriate rostopics

rosbag play rosbag_name.bag
python3 mmdetection3d/demo/pcd_demo_class.py

Convert the checkpoint into ONNX file and than to tensorRT .engine file

python3 mmdeploy/tools/deploy.py

ONNX file can be visualized using NETRON.

This project is released under the Apache 2.0 license.

• MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.
• MMDeploy: OpenMMLab model deployment framework.
• 3D-Multi-Object-Tracker: Tracking multiple objects in 3D scene.