christopherdoer / rio Goto Github PK

RIO - EKF-based Radar Inertial Odometry using 4D mmWave radar sensors

License: GNU General Public License v3.0

CMake 1.43% Python 18.79% C++ 79.50% GLSL 0.28%

rio's Introduction

RIO - Radar Inertial Odometry and Radar based Ego Velocity Estimation

Navigation in GNSS denied and visually degraded environments is still very challenging. Approaches based on visual sensors tend to fail in conditions such as darkness, direct sunlight, fog or smoke. Therefore, we are using 4D mmWave FMCW radar sensors and inertial sensor data as these are not affected by such conditions.

Highlights:

Robust and accurate navigation even in Degraded Visual and GNSS denied Environments
Super fast: x_rio achieves runtimes ~125x faster than realtime on an Intel NUC i7 and ~21x on an Up Core embedded computer
Demonstrated for online navigation of drones even in confined indoor environments

News

08/2022: gnss_x_rio is released adding global information from GNSS-measurements to x_rio and is based on our paper.
03/2022: x_rio is released generalizing ekf_rio and ekf_yrio for multi radar sensor setups and providing a faster implementation using approximated radar clones as described in our paper. The paper datasets are also released and can be evaluated with a single script.
06/2021: The radar inertial datasets with pseudo ground truth used in our Yaw aided Radar Inertial Odometry paper are released: radar_inertial_datasets_icins_2021. Both ekf_rio and ekf_yrio can be evaluated on the whole dataset with a single script.
05/2021: Initial release of RIO - Radar Inertial Odometry and Radar based ego velocity estimation.

Introduction

RIO is a toolbox for EKF-based Radar Inertial Odometry. RIO features the following packages:

gnss_x_rio (recommended): Adds global information from GNSS-measurements to x_rio.
x_rio (recommended): An EKF-based Multi-Radar Inertial Odometry Pipeline with online calibration of the radar sensor extrinsics and yaw aiding using Manhattan world assumptions. Can be used with a single or multi radar setups.
ekf_rio (deprecated): An EKF-based Radar Inertial Odometry Pipeline with online calibration of the radar sensor extrinsics
ekf_yrio (deprecated): An extension of ekf_rio featuring yaw aiding based on Manhattan world assumptions

Checkout the README files of the individual packages for more details.

Demos

Autonomous Radar Inertial Drone Navigation even in Dense Fog (x_rio)

Autonomous Indoor Drone Flight using Yaw aided Radar Inertial Odometry (ekf_yrio)

Indoor Demo and Evaluation of Yaw aided Radar Inertial Odometry (ekf_yrio)

Autonomous UAV Flights using Radar Inertial Odometry (ekf_rio)

References

If you use our implementation for your academic research, please cite the related paper:

gnss_x_rio:

@INPROCEEDINGS{DoerAeroConf2022,
    author = {Doer, Christopher and Atman, Jamal and Trommer, Gert F.},
    title = { GNSS aided Radar Inertial Odometry for UAS Flights in Challenging Conditions },
    booktitle={2022 IEEE Aerospace Conference (AeroConf}, 
    year={2022}
}

x_rio:

@INPROCEEDINGS{DoerJGN2022,
    author = {Doer, Christopher and Trommer, Gert F.},
    year = {2022},
    month = {02},
    pages = {329-339},
    title = {x-RIO: Radar Inertial Odometry with Multiple Radar Sensors and Yaw Aiding},
    volume = {12},
    journal = {Gyroscopy and Navigation}}

ekf_yrio:

@INPROCEEDINGS{DoerICINS2021,
  author={Doer, Christopher and Trommer, Gert F.},
  booktitle={2021 28th Saint Petersburg International Conference on Integrated Navigation Systems (ICINS)}, 
  title={Yaw aided Radar Inertial Odometry uisng Manhattan World Assumptions}, 
  year={2021},
  pages={1-10}}

ekf_rio:

@INPROCEEDINGS{DoerENC2020,
  author={Doer, Christopher and Trommer, Gert F.},
  booktitle={2020 European Navigation Conference (ENC)}, 
  title={Radar Inertial Odometry with Online Calibration}, 
  year={2020},
  pages={1-10},
  doi={10.23919/ENC48637.2020.9317343}}

@INPROCEEDINGS{DoerMFI2020,
  author={Doer, Christopher and Trommer, Gert F.},
  booktitle={2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI)}, 
  title={An EKF Based Approach to Radar Inertial Odometry}, 
  year={2020},
  pages={152-159},
  doi={10.1109/MFI49285.2020.9235254}}

Getting Started

RIO supports:

Ubuntu 16.04 and ROS Kinetic
Ubuntu 18.04 and ROS Melodic
Ubuntu 20.04 and ROS Noetic

RIO depends on:

catkin_simple
catkin_tools (for convenience)
Pull dependencies via submodules, run once: git submodule update --init --recursive. This will setup the following two submodules:
- reve
- rpg_trajectory_evaluation (optional, for comprehensive evaluation) To use the evaluation scripts, the following dependencies are required:
  - sudo apt-get install texlive-latex-extra texlive-fonts-recommended dvipng cm-super
  - pip2 install -U PyYAML colorama ruamel.yaml==0.15.0

Build in Release is highly recommended:

catkin build --cmake-args -DCMAKE_BUILD_TYPE=Release

We provide some demo datasets which can be run using the demo launch files of each package. Check out the Getting Started section of the READMEs in the individual packages for more details.

License

The source code is released under the GPLv3 license.

rio's People

Contributors

Stargazers

Watchers

rio's Issues

issues with consistent pointcloud

Hi,

I am still trying to generate data similar to your datasets, to contribute.

I managed to modify the ti_mmwave_ros_pkg to match your message formats. Also the external triggering of the IWR6843AOPEVM is done via a microcontroller, which also collects IMU mesaurements with accurate timestamps.

But there is one problem:
In your datasets, it seems like the pointcloud and other data is always publishing with a consistent rate of 10hz. Even though I am using exernal triggering, if there is no movement of the sensor, no data will publish. How did you solve this problem? I can see in the pointcloud, that when you are standing still, a lot of points still get published over and over again at 10hz.

It would be awesome if you could help out again!

Thanks

question about TI sensor config and resolution

Hi again,

we are still trying to generate usable datasets.
We have been trying a lot to find suitable parameters for the ti mmwave config but were unable to do so.
Would it be possible for you to post your config for the IWR6843AOPEVM (or add it to the readme or docs)?

For now we are using relatively cheap IMUs (MPU6050, ICM-20689, BMI055). When moving very slowly (like a drone hovering), it always detects zero velocity which is inaccurate and would not be useful to control a drone in hover for example.
We already tried lowering the zero velocity threshold but that didn't seem to have any great effect. (We also adjusted the noise parameters for more robust tracking.)

Does this behaviour come from the radar's velocity resolution or the IMU? Which one is the influence in this scenario? I think the IWR6843AOP only has a velocity resolution of 0.06m/s
But the IMU should fill the gaps right?

We really appreciate your help again and hope to contribute and upload datasets soon.

Thanks,
Tom

Question about imu attitude

Hi, this is a very interesting project. I found x_rio give a very fast downward trajectory in rviz in our own data even I set n_radar:=0
I suspect it's because in our setting the gravity acts on the y axis of imu instead of z axis in your demo dataset. Is this a reasonable explanation? If it is, how can I run it successfully?

Many thanks.

Support ti_mmwave_rospkg

Hi,

this issue was created for the upcoming support of the ti_mmwave_rospkg, which was mentioned in IWR6843AOPEVM external trigger.

Once you released it, it would be great if you could share which parameters changed etc..
Thanks.

Extrinsic Transformation

Hi,

thank you for this great project! Just one question: where can I find/change the extrinsic configuration between radar and IMU?

I know that you perform online calibration, but is there an initial guess or something like that required? And where do I find that?

Thanks,
Tom

IWR6843AOPEVM external trigger

Hi again,

sorry to bother you, but I saw that you used a IWR6843AOPEVM Rev. F for the dataset recording. How did you synchronise the Radar and IMU via a microcontroller? Particularly, I am interested in how you triggered the IWR6843AOPEVM externally, since there is very little information out there about this.
In the schematics, I could only find the AR_SYNC_IN line, but I am not sure if this is how it is triggered. Did you trigger it via a pin or via USB?

Thanks again.

Error: Ignoring transform for child_frame_id - when running demo datasets

Hi,

when I try to run the demos, I always get the following error:

Ignoring transform for child_frame_id "base_link" from authority "unknown_publisher" because of a nan value in the transform (-nan -nan -nan) (0,000000 0,000000 0,000000 1,000000)
Ignoring transform for child_frame_id "radar" from authority "unknown_publisher" because of a nan value in the transform (-nan -nan -nan) (0,000000 0,000000 0,000000 1,000000)

Can you help me please?

CMake errors

Is it possible to catkin_make all the packages? I get the following cmake errors (for all the packages) when trying to do so.

CMake Error at rio/x_rio/CMakeLists.txt:38 (add_custom_target):
  add_custom_target cannot create target "headers" because another target
  with the same name already exists.  The existing target is a custom target
  created in source directory
  "/home/user/catkin_ws/src/rio/reve/radar_ego_velocity_estimator".  See
  documentation for policy CMP0002 for more details.

Is it possible to build only one package, x_rio? Assuming that reve is to be necessarily built for all the variants.