uzh-rpg / agile_autonomy Goto Github PK

Repository Containing the Code associated with the Paper: "Learning High-Speed Flight in the Wild"

License: GNU General Public License v3.0

CMake 1.09% C++ 63.94% Cuda 5.84% Python 29.10% Shell 0.03%

agile_autonomy's Introduction

Learning High-Speed Flight in the Wild

This repo contains the code associated with the paper Learning Agile Flight in the Wild. For more information, please check the project webpage.

Paper, Video, and Datasets

If you use this code in an academic context, please cite the following publication:

Paper: Learning High-Speed Flight in the Wild

Video (Narrated): YouTube

Datasets: Zenodo

Science Paper: DOI

@inproceedings{Loquercio2021Science,
  title={Learning High-Speed Flight in the Wild},
    author={Loquercio, Antonio and Kaufmann, Elia and Ranftl, Ren{\'e} and M{\"u}ller, Matthias and Koltun, Vladlen and Scaramuzza, Davide},
      booktitle={Science Robotics}, 
      year={2021}, 
      month={October}, 
}

Installation

Requirements

The code was tested with Ubuntu 20.04, ROS Noetic, Anaconda v4.8.3., and gcc/g++ 7.5.0. Different OS and ROS versions are possible but not supported.

Before you start, make sure that your compiler versions match gcc/g++ 7.5.0. To do so, use the following commands:

sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 100
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 100

Step-by-Step Procedure

Use the following commands to create a new catkin workspace and a virtual environment with all the required dependencies.

export ROS_VERSION=noetic
mkdir agile_autonomy_ws
cd agile_autonomy_ws
export CATKIN_WS=./catkin_aa
mkdir -p $CATKIN_WS/src
cd $CATKIN_WS
catkin init
catkin config --extend /opt/ros/$ROS_VERSION
catkin config --merge-devel
catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fdiagnostics-color
cd src

git clone [email protected]:uzh-rpg/agile_autonomy.git
vcs-import < agile_autonomy/dependencies.yaml
cd rpg_mpl_ros
git submodule update --init --recursive

#install extra dependencies (might need more depending on your OS)
sudo apt-get install libqglviewer-dev-qt5

# Install external libraries for rpg_flightmare
sudo apt install -y libzmqpp-dev libeigen3-dev libglfw3-dev libglm-dev

# Install dependencies for rpg_flightmare renderer
sudo apt install -y libvulkan1 vulkan-utils gdb

# Add environment variables (Careful! Modify path according to your local setup)
echo 'export RPGQ_PARAM_DIR=/home/<path/to/>catkin_aa/src/rpg_flightmare' >> ~/.bashrc

Now open a new terminal and type the following commands.

# Build and re-source the workspace
catkin build
. ../devel/setup.bash

# Create your learning environment
roscd planner_learning
conda create --name tf_24 python=3.7
conda activate tf_24
pip install tensorflow-gpu==2.4
pip install rospkg==1.2.3 pyquaternion open3d opencv-python

Now download the flightmare standalone available at this link, extract it and put in the flightrender folder.

Let's Fly!

Once you have installed the dependencies, you will be able to fly in simulation with our pre-trained checkpoint. You don't need necessarily need a GPU for execution. Note that if the network can't run at least at 15Hz, you won't be able to fly successfully.

Launch the simulation! Open a terminal and type:

cd agile_autonomy_ws
source catkin_aa/devel/setup.bash
roslaunch agile_autonomy simulation.launch

Run the Network in an other terminal:

cd agile_autonomy_ws
source catkin_aa/devel/setup.bash
conda activate tf_24
python test_trajectories.py --settings_file=config/test_settings.yaml

Change execution speed or environment

You can change the average speed at which the policy will fly as well as the environment type by changing the following files.

Environment Change:

rosed agile_autonomy flightmare.yaml

Set either the spawn_trees or spawn_objects to true. Doing both at the same time is possible but would make the environment too dense for navigation. Also adapt the spacings parameter in test_settings.yaml to the environment.

Speed Change:

rosed agile_autonomy default.yaml

Edit the test_time_velocity and maneuver_velocity to the required speed. Note that the ckpt we provide will work for all speeds in the range [1,10] m/s. However, to reach the best performance at a specific speed, please consider finetuning the ckpt at the desired speed (see code below).

Train your own navigation policy

There are two ways in which you can train your own policy. One easy and one more involved. The trained checkpoint can then be used to control a physical platform (if you have one!).

Use pre-collected dataset

The first method, requiring the least effort, is to use a dataset that we pre-collected. The dataset can be found at this link. This dataset was used to train the model we provide and collected at an average speed of 7 m/s. To do this, adapt the file train_settings.yaml to point to the train and test folder and run:

cd agile_autonomy_ws
source catkin_aa/devel/setup.bash
conda activate tf_24
python train.py --settings_file=config/train_settings.yaml

Feel free to ablate the impact of each parameter!

Collect your own dataset

You can use the following commands to generate data in simulation and train your model on it. Note that training a policy from scratch could require a lot of data, and depending on the speed of your machine this could take several days. Therefore, we always recommend finetuning the provided checkpoint to your use case. As a general rule of thumb, you need a dataset with comparable size to ours to train a policy from scratch, but only 1/10th of it to finetune.

Generate data

To train or finetune a policy, use the following commands: Launch the simulation in one terminal

cd agile_autonomy_ws
source catkin_aa/devel/setup.bash
roslaunch agile_autonomy simulation.launch

Launch data collection (with dagger) in an other terminal

cd agile_autonomy_ws
source catkin_aa/devel/setup.bash
conda activate tf_24
python dagger_training.py --settings_file=config/dagger_settings.yaml

It is possible to change parameters (number of rollouts, dagger constants, tracking a global trajectory, etc. ) in the file dagger_settings.yaml. Keep in mind that if you change the network or input, you will need to adapt the file test_settings.yaml for compatibility.

When training from scratch, follow a pre-computed global trajectory to give consistent labels. To activate this, you need to put to true the flag perform_global_planning in default.yaml and label_generation.yaml. Note that this will make the simulation slower (a global plan has to be computed at each iteration). The network will not have access to this global plan, but only to the straight (possibly in collision) reference.

Visualize the Data

You can visualize the generated trajectories in open3d using the visualize_trajectories.py script.

python visualize_trajectories.py --data_dir /PATH/TO/rollout_21-09-21-xxxx --start_idx 0 --time_steps 100 --pc_cutoff_z 2.0 --max_traj_to_plot 100

The result should more or less look as the following:

Test the Network

To test the network you trained, adapt the test_settings.yaml with the new checkpoint path. You might consider putting back the flag perform_global_planning in default.yaml to false to make the simulation faster. Then follow the instructions in the above section (Let's Fly!) to test.

Ackowledgements

We would like to thank Yunlong Song and Selim Naji for their help with the implementations of the simulation environment. The code for global planning is strongly inspired by the one of Search-based Motion Planning for Aggressive Flight in SE(3).

agile_autonomy's People

Contributors

Stargazers

Watchers

Forkers

zhefan-xu mfkiwl bailiping silegc-xguard mengchaoheng belal-ibrahim sd196821 snowytoxa land007 evolution99 liyang-whu winterchaufr metavai ecmnet norci xepost maxtianyy tne-ai shubham-shahh stjordanis coatz volvo007 cdg-cell nitish-01 chandanpanda den250400 fdoperezi peterzhousz uavworks raistlinsp wenhao12111 zolalee dlyshare 911wb sunxiongfeng uncleyoung geographerwang hwb0314 dmfengh xiaojake jlchulilla coolkesword atlasgooo2 deyu001 dawnchen123 slam-box manucalop freeclouds maohaili amoslive enmin-c logosqiao kevinlibaba akashleena msathler breederbai nihonges swxgithub ggsonic liming-zheng savnani5 fangzheng81 abecadel fenglaijun hcque wjmessi1 yuwei-wu alex-bazhl tuskaw 122880428 racheraven zymale noticeable eqqlyz hzfengfengxia howardcl teryzh evandanrao dreamwest gilfoyle-code angelog98 liangyongjue yanrui89 waikit-nus coffear aridwanw hang-shi tmddn833 filippsam ding-guo jiayi113 mengxingshifen1218 kielerrr denniswangcw m4d-sc1entist stampliu yucnet sidtalia bttung-2020 michaelchoum

agile_autonomy's Issues

Error occurs when catkin build `mpl_test_node`

I was running catkin b and some strange events happened on mpl_test_node. As you can see in the following error message, in my opinion it seems like something to do with C/C++ compiler. The versions of gcc/g++ are both 7.5.0

Errors     << mpl_test_node:make /home/m/agile_autonomy_ws/catkin_aa/logs/mpl_test_node/build.make.010.log
In file included from /usr/local/include/open3d/geometry/PointCloud.h:36:0,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner/include/mpl_external_planner/ellipsoid_planner/ellipsoid_util.h:11,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner/include/mpl_external_planner/ellipsoid_planner/env_cloud.h:8,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner/include/mpl_external_planner/ellipsoid_planner/ellipsoid_planner.h:9,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:3:
/usr/local/include/open3d/utility/Optional.h:308:18: error: ‘constexpr T& open3d::utility::optional<T>::contained_val() const &’ cannot be overloaded
     constexpr T& contained_val() & { return OptionalBase<T>::storage_.value_; }
                  ^~~~~~~~~~~~~
/usr/local/include/open3d/utility/Optional.h:302:24: error: with ‘constexpr const T& open3d::utility::optional<T>::contained_val() const &’
     constexpr const T& contained_val() const& {
                        ^~~~~~~~~~~~~
/usr/local/include/open3d/utility/Optional.h:478:36: error: ‘constexpr T* open3d::utility::optional<T>::operator->() const’ cannot be overloaded
     TR2_OPTIONAL_HOST_CONSTEXPR T* operator->() {
                                    ^~~~~~~~
/usr/local/include/open3d/utility/Optional.h:474:42: error: with ‘constexpr const T* open3d::utility::optional<T>::operator->() const’
     TR2_OPTIONAL_HOST_CONSTEXPR T const* operator->() const {
                                          ^~~~~~~~
/usr/local/include/open3d/utility/Optional.h:487:36: error: ‘constexpr T& open3d::utility::optional<T>::operator*() const &’ cannot be overloaded
     TR2_OPTIONAL_HOST_CONSTEXPR T& operator*() & {
                                    ^~~~~~~~
/usr/local/include/open3d/utility/Optional.h:483:42: error: with ‘constexpr const T& open3d::utility::optional<T>::operator*() const &’
     TR2_OPTIONAL_HOST_CONSTEXPR T const& operator*() const& {
                                          ^~~~~~~~
/usr/local/include/open3d/utility/Optional.h:504:36: error: ‘constexpr T& open3d::utility::optional<T>::value() const &’ cannot be overloaded
     TR2_OPTIONAL_HOST_CONSTEXPR T& value() & {
                                    ^~~~~
/usr/local/include/open3d/utility/Optional.h:497:42: error: with ‘constexpr const T& open3d::utility::optional<T>::value() const &’
     TR2_OPTIONAL_HOST_CONSTEXPR T const& value() const& {
                                          ^~~~~
In file included from /usr/local/include/open3d/core/Tensor.h:43:0,
                 from /usr/local/include/open3d/core/EigenConverter.h:34,
                 from /usr/local/include/open3d/Open3D.h:38,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:19:
/usr/local/include/open3d/core/TensorInit.h:95:49: error: ‘std::index_sequence’ has not been declared
 SizeVector InitializerShape(const L& list, std::index_sequence<D...>) {
                                                 ^~~~~~~~~~~~~~
/usr/local/include/open3d/core/TensorInit.h:95:63: error: expected ‘,’ or ‘...’ before ‘<’ token
 SizeVector InitializerShape(const L& list, std::index_sequence<D...>) {
                                                               ^
/usr/local/include/open3d/core/TensorInit.h: In function ‘open3d::core::SizeVector open3d::core::tensor_init::InferShape(const L&)’:
/usr/local/include/open3d/core/TensorInit.h:103:24: error: ‘make_index_sequence’ is not a member of ‘std’
             list, std::make_index_sequence<InitializerDim<L>::value>());
                        ^~~~~~~~~~~~~~~~~~~
/usr/local/include/open3d/core/TensorInit.h:103:24: note: suggested alternatives:
In file included from /usr/include/boost/mp11/algorithm.hpp:25:0,
                 from /usr/include/boost/mp11/bind.hpp:11,
                 from /usr/include/boost/parameter/aux_/is_placeholder.hpp:46,
                 from /usr/include/boost/parameter/value_type.hpp:101,
                 from /usr/include/boost/parameter/aux_/name.hpp:31,
                 from /usr/include/boost/parameter/name.hpp:10,
                 from /usr/include/boost/heap/policies.hpp:13,
                 from /usr/include/boost/heap/detail/stable_heap.hpp:20,
                 from /usr/include/boost/heap/d_ary_heap.hpp:21,
                 from /home/m/agile_autonomy_ws/catkin_aa/devel/share/motion_primitive_library/cmake/../../../include/mpl_planner/common/state_space.h:10,
                 from /home/m/agile_autonomy_ws/catkin_aa/devel/share/motion_primitive_library/cmake/../../../include/mpl_planner/common/graph_search.h:10,
                 from /home/m/agile_autonomy_ws/catkin_aa/devel/share/motion_primitive_library/cmake/../../../include/mpl_planner/common/planner_base.h:12,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner/include/mpl_external_planner/ellipsoid_planner/ellipsoid_planner.h:10,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:3:
/usr/include/boost/mp11/integer_sequence.hpp:104:90: note:   ‘boost::mp11::make_index_sequence’
 template<std::size_t N> using make_index_sequence = make_integer_sequence<std::size_t, N>;
                                                                                          ^
In file included from /usr/local/include/open3d/utility/Logging.h:42:0,
                 from /usr/local/include/open3d/core/Device.h:32,
                 from /usr/local/include/open3d/core/Blob.h:33,
                 from /usr/local/include/open3d/Open3D.h:34,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:19:
/usr/local/include/open3d/3rdparty/fmt/ranges.h:141:66: note:   ‘fmt::v6::internal::make_index_sequence’
 using make_index_sequence = make_integer_sequence<std::size_t, N>;
                                                                  ^
In file included from /usr/local/include/open3d/core/Tensor.h:43:0,
                 from /usr/local/include/open3d/core/EigenConverter.h:34,
                 from /usr/local/include/open3d/Open3D.h:38,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:19:
/usr/local/include/open3d/core/TensorInit.h:103:70: error: expected primary-expression before ‘)’ token
             list, std::make_index_sequence<InitializerDim<L>::value>());
                                                                      ^
In file included from /usr/local/include/open3d/visualization/gui/UIImage.h:30:0,
                 from /usr/local/include/open3d/visualization/gui/Button.h:33,
                 from /usr/local/include/open3d/Open3D.h:119,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19,
                 from /home/m/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:19:
/usr/local/include/open3d/visualization/rendering/RendererHandle.h: At global scope:
/usr/local/include/open3d/visualization/rendering/RendererHandle.h:181:35: error: ‘format’ function uses ‘auto’ type specifier without trailing return type
                 FormatContext& ctx) {
                                   ^
/usr/local/include/open3d/visualization/rendering/RendererHandle.h:181:35: note: deduced return type only available with -std=c++14 or -std=gnu++14
/usr/local/include/open3d/visualization/rendering/RendererHandle.h:189:43: error: ‘parse’ function uses ‘auto’ type specifier without trailing return type
     constexpr auto parse(ParseContext& ctx) {
                                           ^
/usr/local/include/open3d/visualization/rendering/RendererHandle.h:189:43: note: deduced return type only available with -std=c++14 or -std=gnu++14
make[2]: *** [CMakeFiles/ellipsoid_planner_node.dir/build.make:63：CMakeFiles/ellipsoid_planner_node.dir/src/ellipsoid_planner_node.cpp.o] 错误 1
make[1]: *** [CMakeFiles/Makefile2:4364：CMakeFiles/ellipsoid_planner_node.dir/all] 错误 2
make: *** [Makefile:141：all] 错误 2
cd /home/m/agile_autonomy_ws/catkin_aa/build/mpl_test_node; catkin build --get-env mpl_test_node | catkin env -si  /usr/bin/make --jobserver-auth=3,4; cd -

....................................................................................................
Failed     << mpl_test_node:make                          [ Exited with code 2 ]                    
Failed    <<< mpl_test_node                               [ 27.0 seconds ]

Implementation

Hello I would like to find in your project the part where you detect objects and their distance, can you guide me ?

Failed to reach repository

vcs-import < agile_autonomy/dependencies.yaml

EEEEEEEEEEEEEEEEEEEE
=== ./assimp_catkin (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./catkin_boost_python_buildtool (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./catkin_simple (git) ===
Could not determine ref type of version: Warning: Permanently added the RSA host key for IP address '13.234.176.102' to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./eigen_catkin (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./eigen_checks (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./gflags_catkin (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./glog_catkin (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./mav_comm (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./minimum_jerk_trajectories (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./minkindr (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./minkindr_ros (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./numpy_eigen (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rotors_simulator (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_common (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_flightmare (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_mpc (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_mpl_ros (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_quadrotor_common (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_quadrotor_control (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_single_board_io (git) ===
Could not determine ref type of version: [email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

How the MAV with high speed localize accurately by T265 in wild ?

Thanks for your team's brilliant work! It's an unbelievable work for the MAVs flying in the wild environment with such a high speed. However, I am facing with some localization problems with the Intel T265 odometry.
In our configuration, it often has a huge localization error in the z-axis. Does the T265 odometry always perform perfectly in your setup? If not, how about the success rate? btw, it's there some methods to deal with that problem, such as calibration?
Thanks for any response!

physical platform test

Hello!
Can you privide the tutorial to fly in physical platform @antonilo.
Thank you!

Flightmare crashes on roslaunch agile_autonomy simulation.launch in docker

I have gotten all of the packages to build in an Ubuntu 20.04 docker container. The container has X11 forwarding setup (on /tmp/.X11-unix) and is run with the --privileged flag.
When I launch agile_autonomy flightmare starts for about a second, then crashes. The rviz and rqt GUIs appear and seem to function correctly. It is only flightmare that crashes.

Has anyone else experienced this issue, or have ideas on how to proceed?

I have attached the relevant terminal output below.

QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Ready ? : 0                                                         
[ INFO] [1642693599.801859432]: Finished loading Gazebo ROS API Plugin.
[ INFO] [1642693599.803162953]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting...
Ready ? : 0                                                         
RosPluginProvider.load(qt_gui_cpp/CppPluginProvider) exception raised in __builtin__.__import__(qt_gui_cpp.cpp_plugin_provider, [CppPlug
inProvider]):                                                       
Traceback (most recent call last): 
  File "/opt/ros/noetic/lib/python3/dist-packages/rqt_gui/ros_plugin_provider.py", line 80, in load
    attributes['module_name'], fromlist=[attributes['class_from_class_type']], level=0)
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui_cpp/cpp_plugin_provider.py", line 33, in <module>
    from .cpp_binding_helper import qt_gui_cpp
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui_cpp/cpp_binding_helper.py", line 43, in <module>
    from . import libqt_gui_cpp_sip
ValueError: PyCapsule_GetPointer called with incorrect name

RecursivePluginProvider.discover() loading plugin "qt_gui_cpp/CppPluginProvider" failed:
Traceback (most recent call last): 
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui/recursive_plugin_provider.py", line 60, in discover
    instance = self._plugin_provider.load(plugin_descriptor.plugin_id(), None)
  File "/opt/ros/noetic/lib/python3/dist-packages/rqt_gui/ros_plugin_provider.py", line 90, in load
    raise e
  File "/opt/ros/noetic/lib/python3/dist-packages/rqt_gui/ros_plugin_provider.py", line 80, in load
    attributes['module_name'], fromlist=[attributes['class_from_class_type']], level=0)
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui_cpp/cpp_plugin_provider.py", line 33, in <module>
    from .cpp_binding_helper import qt_gui_cpp
  File "/opt/ros/noetic/lib/python3/dist-packages/qt_gui_cpp/cpp_binding_helper.py", line 43, in <module>
    from . import libqt_gui_cpp_sip
ValueError: PyCapsule_GetPointer called with incorrect name

Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
[flight_render-20] process has died [pid 654, exit code -11, cmd /home/usr/agile_autonomy_ws/catkin_aa/src/rpg_flightmare/flightrender/f
lightmare.x86_64 __name:=flight_render __log:=/root/.ros/log/278c32a4-7a08-11ec-a9b3-0242ac110002/flight_render-20.log].
log file: /root/.ros/log/278c32a4-7a08-11ec-a9b3-0242ac110002/flight_render-20*.log

Flightmare unable to generate "pointcloud-unity.ply"

theFlightmare unable to generate "pointcloud-unity.ply" or too slow,and the simulation is always failed....
My computer have Intel i9-12th CPU & RTX 3090 GPU,but i find the flightmare do not use GPU,it run in the CPU,you can see from the next picture.
can you help me?Any suggestion will be appreciated！@antonilo@kelia@shubham-shahh

‘class rviz::PointCloudCommon’ has no member named ‘getCallbackQueue’

Environment: ubuntu20.04, gcc7.5, cmake3.22.2
I enter the "catkin make" command but it says that
"/home/<path_to>/catkin_aa/src/rpg_mpl_ros/planning_ros_utils/src/planning_rviz_plugins/map_display.cpp:39:52: error: ‘class rviz::PointCloudCommon’ has no member named ‘getCallbackQueue’
update_nh_.setCallbackQueue(point_cloud_common_->getCallbackQueue());"

How to use my own planner

Hi, I am trying to use my own planner with obstacle avoidance to train the network; my planner return me just one trajectory rather than top 3 as you do; so I was wondering how can I integrate my own planner in your project? If anyone has already tried, please let me know how you did. Thank you.

link error with generate_labels in traj_sampler

Hi, I'm trying to build this project in docker with ubuntu 20.04, gcc/g++7.5 installed.
I struggle to build this project a lot, but still I failed to build generate_labels node in traj_sampler pkg.
Below is the error I got:

root@tw:~/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/traj_sampler# catkin build traj_sampler 
---------------------------------------------------------------------------------------------
Profile:                     default
Extending:          [cached] /opt/ros/noetic
Workspace:                   /root/agile_autonomy_ws/catkin_aa
---------------------------------------------------------------------------------------------
Build Space:        [exists] /root/agile_autonomy_ws/catkin_aa/build
Devel Space:        [exists] /root/agile_autonomy_ws/catkin_aa/devel
Install Space:      [unused] /root/agile_autonomy_ws/catkin_aa/install
Log Space:          [exists] /root/agile_autonomy_ws/catkin_aa/logs
Source Space:       [exists] /root/agile_autonomy_ws/catkin_aa/src
DESTDIR:            [unused] None
---------------------------------------------------------------------------------------------
Devel Space Layout:          merged
Install Space Layout:        None
---------------------------------------------------------------------------------------------
Additional CMake Args:       -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fdiagnostics-color
Additional Make Args:        None
Additional catkin Make Args: None
Internal Make Job Server:    True
Cache Job Environments:      False
---------------------------------------------------------------------------------------------
Whitelisted Packages:        None
Blacklisted Packages:        None
---------------------------------------------------------------------------------------------
Workspace configuration appears valid.
---------------------------------------------------------------------------------------------
[build] Found '70' packages in 0.0 seconds.                                                             
[build] Package table is up to date.                                                                    
Starting  >>> catkin_simple                                                                             
Finished  <<< catkin_simple                               [ 0.1 seconds ]                               
Starting  >>> eigen_catkin                                                                              
Starting  >>> gflags_catkin                                                                             
Starting  >>> minimum_jerk_trajectories                                                                 
Starting  >>> quadrotor_msgs                                                                            
Finished  <<< minimum_jerk_trajectories                   [ 0.1 seconds ]                               
Finished  <<< gflags_catkin                               [ 0.1 seconds ]                               
Starting  >>> glog_catkin                                                                               
Finished  <<< eigen_catkin                                [ 0.1 seconds ]                               
Finished  <<< quadrotor_msgs                              [ 0.1 seconds ]                               
Starting  >>> quadrotor_common                                                                          
Finished  <<< glog_catkin                                 [ 0.1 seconds ]                               
Starting  >>> eigen_checks                                                                              
Finished  <<< quadrotor_common                            [ 0.1 seconds ]                               
Starting  >>> polynomial_trajectories                                                                   
Starting  >>> position_controller                                                                       
Starting  >>> state_predictor                                                                           
Finished  <<< eigen_checks                                [ 0.1 seconds ]                               
Starting  >>> minkindr                                                                                  
Finished  <<< polynomial_trajectories                     [ 0.1 seconds ]                               
Finished  <<< state_predictor                             [ 0.1 seconds ]                               
Finished  <<< position_controller                         [ 0.1 seconds ]                               
Finished  <<< minkindr                                    [ 0.1 seconds ]                               
Starting  >>> rpg_common                                                                                
Finished  <<< rpg_common                                  [ 0.1 seconds ]                               
Starting  >>> trajectory_generation_helper                                                              
Finished  <<< trajectory_generation_helper                [ 0.1 seconds ]                               
Starting  >>> autopilot                                                                                 
Finished  <<< autopilot                                   [ 0.1 seconds ]                               
Starting  >>> agile_autonomy_utils                                                                      
Starting  >>> rpg_mpc                                                                                   
Finished  <<< rpg_mpc                                     [ 0.2 seconds ]                               
Finished  <<< agile_autonomy_utils                        [ 0.1 seconds ]                               
Starting  >>> traj_sampler                                                                              
________________________________________________________________________________________________________
Warnings   << traj_sampler:check /root/agile_autonomy_ws/catkin_aa/logs/traj_sampler/build.check.014.log
CMake Warning (dev) at CMakeLists.txt:1 (project):
  Policy CMP0048 is not set: project() command manages VERSION variables.
  Run "cmake --help-policy CMP0048" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  The following variable(s) would be set to empty:

    CMAKE_PROJECT_VERSION
    CMAKE_PROJECT_VERSION_MAJOR
    CMAKE_PROJECT_VERSION_MINOR
    CMAKE_PROJECT_VERSION_PATCH
This warning is for project developers.  Use -Wno-dev to suppress it.

Detected cmake version 3.22.0-rc2
CMake Deprecation Warning at /usr/src/googletest/CMakeLists.txt:4 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Deprecation Warning at /usr/src/googletest/googlemock/CMakeLists.txt:45 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Deprecation Warning at /usr/src/googletest/googletest/CMakeLists.txt:56 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


Searching open3D in /usr/local/lib/cmake/
CMake Deprecation Warning at /root/lib/Open3D/lib/cmake/Open3D/Open3DConfig.cmake:18 (cmake_policy):
  The OLD behavior for policy CMP0072 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
  CMakeLists.txt:12 (find_package)


Found OpenMP TRUE   4 0.13.0 OpenMP::OpenMP_CXX
Found Open3D 0.13.0
cd /root/agile_autonomy_ws/catkin_aa/build/traj_sampler; catkin build --get-env traj_sampler | catkin env -si  /usr/bin/make cmake_check_build_system; cd -

........................................................................................................
________________________________________________________________________________________________________
Errors     << traj_sampler:make /root/agile_autonomy_ws/catkin_aa/logs/traj_sampler/build.make.012.log  
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `generate_label::GenerateLabel::completedThreadCallback(boost::shared_ptr<std_msgs::Int8_<std::allocator<void> > const> const&)':
generate_label.cpp:(.text+0x449): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `generate_label::GenerateLabel::loadParameters()':
generate_label.cpp:(.text+0x51c): undefined reference to `ros::NodeHandle::getParam(std::string const&, std::string&) const'
/usr/bin/ld: generate_label.cpp:(.text+0x5d8): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x6cf): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x7ae): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x89d): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x990): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o:generate_label.cpp:(.text+0xa83): more undefined references to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)' follow
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `generate_label::GenerateLabel::loadParameters()':
generate_label.cpp:(.text+0x157d): undefined reference to `ros::NodeHandle::getParam(std::string const&, int&) const'
/usr/bin/ld: generate_label.cpp:(.text+0x15c9): undefined reference to `ros::NodeHandle::getParam(std::string const&, int&) const'
/usr/bin/ld: generate_label.cpp:(.text+0x164a): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x172e): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x180f): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x18f3): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `computeReferenceTrajectoryPosBased(Eigen::Matrix<double, 3, 1, 0, 3, 1> const&, quadrotor_common::Trajectory const&, double, quadrotor_common::Trajectory*, int*)':
generate_label.cpp:(.text+0x2e25): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `generate_label::GenerateLabel::GenerateLabel(ros::NodeHandle const&, ros::NodeHandle const&)':
generate_label.cpp:(.text+0x5294): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x530d): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x532a): undefined reference to `rpg_mpc::MpcController<double>::MpcController(ros::NodeHandle const&, ros::NodeHandle const&, std::string const&)'
/usr/bin/ld: generate_label.cpp:(.text+0x608a): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: generate_label.cpp:(.text+0x60d9): undefined reference to `ros::this_node::getName()'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `generate_label::GenerateLabel::generateInitialGuesses(boost::shared_ptr<std_msgs::Bool_<std::allocator<void> > const> const&)':
generate_label.cpp:(.text+0x8ac9): undefined reference to `logging::Logging::save_rollout_to_csv(std::vector<TrajectoryExt, std::allocator<TrajectoryExt> > const&, int, int, double, std::string, std::string, std::string, std::string, bool)'
/usr/bin/ld: generate_label.cpp:(.text+0x90b3): undefined reference to `logging::Logging::save_rollout_to_csv(std::vector<TrajectoryExt, std::allocator<TrajectoryExt> > const&, int, int, double, std::string, std::string, std::string, std::string, bool)'
/usr/bin/ld: generate_label.cpp:(.text+0x986c): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `generate_label::GenerateLabel::generateLabelCallback(boost::shared_ptr<std_msgs::Bool_<std::allocator<void> > const> const&)':
generate_label.cpp:(.text+0xb58c): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: generate_label.cpp:(.text+0xb827): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `bool quadrotor_common::getParam<int>(std::string const&, int&, int const&, ros::NodeHandle const&)':
generate_label.cpp:(.text._ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x37): undefined reference to `ros::NodeHandle::getParam(std::string const&, int&) const'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x1c2): undefined reference to `ros::console::print(ros::console::FilterBase*, void*, ros::console::levels::Level, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int, char const*)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x215): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x333): undefined reference to `ros::console::print(ros::console::FilterBase*, void*, ros::console::levels::Level, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int, char const*)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIiEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x37d): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `bool quadrotor_common::getParam<double>(std::string const&, double&, double const&, ros::NodeHandle const&)':
generate_label.cpp:(.text._ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x37): undefined reference to `ros::NodeHandle::getParam(std::string const&, double&) const'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x1c8): undefined reference to `ros::console::print(ros::console::FilterBase*, void*, ros::console::levels::Level, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int, char const*)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x21d): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x345): undefined reference to `ros::console::print(ros::console::FilterBase*, void*, ros::console::levels::Level, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int, char const*)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIdEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x38d): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `bool quadrotor_common::getParam<bool>(std::string const&, bool&, bool const&, ros::NodeHandle const&)':
generate_label.cpp:(.text._ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x37): undefined reference to `ros::NodeHandle::getParam(std::string const&, bool&) const'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x1c4): undefined reference to `ros::console::print(ros::console::FilterBase*, void*, ros::console::levels::Level, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int, char const*)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x21d): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x344): undefined reference to `ros::console::print(ros::console::FilterBase*, void*, ros::console::levels::Level, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int, char const*)'
/usr/bin/ld: generate_label.cpp:(.text._ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE[_ZN16quadrotor_common8getParamIbEEbRKSsRT_RKS3_RKN3ros10NodeHandleE]+0x38d): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `void ros::Publisher::publish<std_msgs::Bool_<std::allocator<void> > >(std_msgs::Bool_<std::allocator<void> > const&) const':
generate_label.cpp:(.text._ZNK3ros9Publisher7publishIN8std_msgs5Bool_ISaIvEEEEEvRKT_[_ZNK3ros9Publisher7publishIN8std_msgs5Bool_ISaIvEEEEEvRKT_]+0x2ca): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `void ros::Publisher::publish<std_msgs::Int8_<std::allocator<void> > >(std_msgs::Int8_<std::allocator<void> > const&) const':
generate_label.cpp:(.text._ZNK3ros9Publisher7publishIN8std_msgs5Int8_ISaIvEEEEEvRKT_[_ZNK3ros9Publisher7publishIN8std_msgs5Int8_ISaIvEEEEEvRKT_]+0x2ca): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `generate_label::GenerateLabel::GenerateLabel()':
generate_label.cpp:(.text._ZN14generate_label13GenerateLabelC2Ev[_ZN14generate_label13GenerateLabelC5Ev]+0x86): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: generate_label.cpp:(.text._ZN14generate_label13GenerateLabelC2Ev[_ZN14generate_label13GenerateLabelC5Ev]+0xe1): undefined reference to `ros::NodeHandle::NodeHandle(std::string const&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `main':
generate_label.cpp:(.text.startup+0x4a): undefined reference to `ros::init(int&, char**, std::string const&, unsigned int)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `ros::SubscriptionCallbackHelperT<boost::shared_ptr<std_msgs::Int8_<std::allocator<void> > const> const&, void>::deserialize(ros::SubscriptionCallbackHelperDeserializeParams const&)':
generate_label.cpp:(.text._ZN3ros27SubscriptionCallbackHelperTIRKN5boost10shared_ptrIKN8std_msgs5Int8_ISaIvEEEEEvE11deserializeERKNS_43SubscriptionCallbackHelperDeserializeParamsE[_ZN3ros27SubscriptionCallbackHelperTIRKN5boost10shared_ptrIKN8std_msgs5Int8_ISaIvEEEEEvE11deserializeERKNS_43SubscriptionCallbackHelperDeserializeParamsE]+0x257): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: CMakeFiles/generate_label.dir/src/generate_label.cpp.o: in function `ros::SubscriptionCallbackHelperT<boost::shared_ptr<std_msgs::Bool_<std::allocator<void> > const> const&, void>::deserialize(ros::SubscriptionCallbackHelperDeserializeParams const&)':
generate_label.cpp:(.text._ZN3ros27SubscriptionCallbackHelperTIRKN5boost10shared_ptrIKN8std_msgs5Bool_ISaIvEEEEEvE11deserializeERKNS_43SubscriptionCallbackHelperDeserializeParamsE[_ZN3ros27SubscriptionCallbackHelperTIRKN5boost10shared_ptrIKN8std_msgs5Bool_ISaIvEEEEEvE11deserializeERKNS_43SubscriptionCallbackHelperDeserializeParamsE]+0x257): undefined reference to `ros::console::initializeLogLocation(ros::console::LogLocation*, std::string const&, ros::console::levels::Level)'
/usr/bin/ld: /root/agile_autonomy_ws/catkin_aa/devel/lib/libtraj_sampler.so: undefined reference to `google::base::CheckOpMessageBuilder::NewString()'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/generate_label.dir/build.make:410: /root/agile_autonomy_ws/catkin_aa/devel/lib/traj_sampler/generate_label] Error 1
make[1]: *** [CMakeFiles/Makefile2:1730: CMakeFiles/generate_label.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
cd /root/agile_autonomy_ws/catkin_aa/build/traj_sampler; catkin build --get-env traj_sampler | catkin env -si  /usr/bin/make --jobserver-auth=3,4; cd -

........................................................................................................
Failed     << traj_sampler:make                           [ Exited with code 2 ]                        
Failed    <<< traj_sampler                                [ 1.4 seconds ]                               
[build] Summary: 17 of 18 packages succeeded.                                                           
[build]   Ignored:   52 packages were skipped or are blacklisted.                                       
[build]   Warnings:  1 packages succeeded with warnings.                                                
[build]   Abandoned: None.                                                                              
[build]   Failed:    1 packages failed.                                                                 
[build] Runtime: 2.8 seconds total.                                                                     
root@tw:~/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/traj_sampler#

I checked or tried below

I checked gcc/g++ version, which were 7.5 each.
I checked ${catkin_LIBRARIES} variable in CMakeLists.txt, and confirmed if necessary library(libroscpp.so) is included.
I tried not using catkin_simple and tried ordinary CMakelists.txt format

Did anyone give me hinks about this?

Thanks in advance.

Simulation Drone Constantly Crashing into Obstacles and Ground

@antonilo @kelia @shubham-shahh
Thanks for your project! It is a great project! The idea behind is very innovative. I appreciate it a lot!

I have successfully run the entire project, and this project is amazing! I had never been flying a drone at 7m/s or more in forest before .

However, I had found that the performance of the simulated drone sometimes is not as good as the paper described, if I used the default settings in test_settings.yaml. The drone constantly crashing into trees if I fly the drone at 7m/s. In 10 rollouts, it crashed 4 times.

If I reduce the speed of drone to 3.0 by changing the maneuver_velocity and test_time_velocity in default.yaml to 3.0, and increase the flight distance (i.e. length_straight in agile_autonomy.cpp) to 100, the drone will crash into ground when the ground level goes hight. (see the visualization result below). There are clearly available paths if the drone just fly higher a bit.

The black lines are the planned trajectories. I had limited the pc_cutoff_z to 4.5. It can be seen that the traj is going into the ground and the drone crashed there.

This is how the scene looks like where the drone crashed.

I am deeply confused by this phenomenon. I have checked the depth graph input and network output frequency. They are all about 15Hz, as is suggested in the ReadMe file. I wonder whether this is caused by the imperfect model parameters in the models/ckpt-50. Or is it the problem caused by limited generalization ability of network itself? Or have I made some mistake in the project settings so that it is not working properly?

Thanks for your reply!

Tensorflow OOM when trying to launch test_trajectory.py

After 1 month of trials and errors, I was finally able to build agile_autonomy. The only ROS package I was unable to build is mpl_test_node, though I think that it's not crucial part of the system.

roslaunch agile_autonomy simulation.launch launched nice - I was even able to make the copter hover via GUI interface. However, when I tried to fly using the network's predictions (python test_trajectories.py --settings_file=config/test_settings.yaml), I encountered the following error:

2021-11-24` 18:58:17.380102: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-11-24 18:58:19.828541: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-11-24 18:58:19.829417: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2021-11-24 18:58:19.855816: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:19.856260: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1650 Ti computeCapability: 7.5
coreClock: 1.485GHz coreCount: 16 deviceMemorySize: 3.82GiB deviceMemoryBandwidth: 178.84GiB/s
2021-11-24 18:58:19.856316: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-11-24 18:58:19.858212: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-11-24 18:58:19.858527: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.10
2021-11-24 18:58:19.860495: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-11-24 18:58:19.861046: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-11-24 18:58:19.863336: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-11-24 18:58:19.864947: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10
2021-11-24 18:58:19.869907: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.7
2021-11-24 18:58:19.870182: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:19.870822: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:19.871087: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-11-24 18:58:19.872465: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-11-24 18:58:19.873089: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:19.873410: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1650 Ti computeCapability: 7.5
coreClock: 1.485GHz coreCount: 16 deviceMemorySize: 3.82GiB deviceMemoryBandwidth: 178.84GiB/s
2021-11-24 18:58:19.873488: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-11-24 18:58:19.873558: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-11-24 18:58:19.873608: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.10
2021-11-24 18:58:19.873641: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-11-24 18:58:19.873671: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-11-24 18:58:19.873701: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-11-24 18:58:19.873731: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10
2021-11-24 18:58:19.873761: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.7
2021-11-24 18:58:19.873862: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:19.874198: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:19.874564: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-11-24 18:58:19.874651: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-11-24 18:58:20.494457: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-11-24 18:58:20.494495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2021-11-24 18:58:20.494503: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2021-11-24 18:58:20.494743: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:20.495361: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:20.495849: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-24 18:58:20.496326: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2922 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1650 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5)
2021-11-24 18:58:20.496799: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
------------------------------------------
Restored from models/ckpt-50
------------------------------------------
2021-11-24 18:58:23.119284: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2021-11-24 18:58:23.183628: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2499950000 Hz
2021-11-24 18:58:23.684142: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.7
2021-11-24 18:58:24.938844: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2021-11-24 18:58:24.990377: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2021-11-24 18:58:25.396965: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-11-24 18:58:31.038641: W tensorflow/core/common_runtime/bfc_allocator.cc:314] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
2021-11-24 18:58:31.619634: W tensorflow/core/common_runtime/bfc_allocator.cc:314] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
2021-11-24 18:58:31.685442: I tensorflow/stream_executor/cuda/cuda_driver.cc:789] failed to allocate 2.73G (2931228672 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
Net initialized
Traceback (most recent call last):
  File "test_trajectories.py", line 19, in <module>
    main()
  File "test_trajectories.py", line 15, in main
    trainer.perform_testing()
  File "/home/denis/agile_autonomy_ws/catkin_aa/src/agile_autonomy/planner_learning/dagger_training.py", line 125, in perform_testing
    removable_rollout_folders = os.listdir(self.settings.expert_folder)
FileNotFoundError: [Errno 2] No such file or directory: '../data_generation/data/'
--- Logging error ---
Traceback (most recent call last):
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/logging/handlers.py", line 69, in emit
    if self.shouldRollover(record):
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/logging/handlers.py", line 183, in shouldRollover
    self.stream = self._open()
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/logging/__init__.py", line 1116, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
NameError: name 'open' is not defined
Call stack:
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/site-packages/tensorflow/python/training/tracking/util.py", line 161, in __del__
    .format(pretty_printer.node_names[node_id]))
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/site-packages/tensorflow/python/platform/tf_logging.py", line 178, in warning
    get_logger().warning(msg, *args, **kwargs)
Message: 'Unresolved object in checkpoint: (root).optimizer.iter'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/logging/handlers.py", line 69, in emit
    if self.shouldRollover(record):
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/logging/handlers.py", line 183, in shouldRollover
    self.stream = self._open()
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/logging/__init__.py", line 1116, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
NameError: name 'open' is not defined
Call stack:
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/site-packages/tensorflow/python/training/tracking/util.py", line 161, in __del__
    .format(pretty_printer.node_names[node_id]))
  File "/home/denis/anaconda3/envs/tf_24/lib/python3.7/site-packages/tensorflow/python/platform/tf_logging.py", line 178, in warning
    get_logger().warning(msg, *args, **kwargs)
Message: 'Unresolved object in checkpoint: (root).optimizer.beta_1'
Arguments: ()

I don't know what exactly is the source of the problem, but I have two suggestions:

My GPU really doesn't have enough memory (my GPU is Geforce GTX1650 Ti and has 4 Gb of memory)
Checkpoint files were saved using older tensorflow version, and the newer one fails to read them. @antonilo @kelia Can you please provide your tensorflow version?

FoundError: conda activate tf_24; python test_trajectories.py --settings_file=config/test_settings.yaml

conda activate tf_24
CommandNotFoundError: 'activate is not a conda command.
Did you mean 'source activate'?

ThinkPad-P15v-Gen-1: conda info --envs

conda environments:
tf_24 /home/linzgood/anaconda3/envs/tf_24
root * /home/linzgood/anaconda3

ThinkPad-P15v-Gen-1: source activate
ThinkPad-P15v-Gen-1: source deactivate
But the error still remain，anybody met the same error？Thanks

Could you please provide the GPU model and driver information when you are doing the simulation experiment?

Cannot load simulation environment

When I was running the command roslaunch agile_autonomy simulation.launch, something wrong happened as the following image shows:

It kept printing Ready ? : 0, the standalone has been extracted to the 'flightrenderfolder. Is it necessary to build thestandalone` in advance?

Add LICENSE file

to clarify reusability.

Ideally, this would distinguish between

software
data
other media files (e.g. the images and videos)

Why python test_trajectories.py --settings_file=config/test_settings.yaml is so slow?

When I run python test_trajectories.py --settings_file=config/test_settings.yaml, it is so slow to upload the model. I need to wait for several minutes. Can someone tell me the reason?

Which flight controller software is used for testing

Hi, Which flight controller software is used for testing? can we use PX4 or Ardupilot? @antonilo

Thanks

an illegal memory access was encountered 700

[ERROR] [1637285665.411913589, 3.173000000]: out of memory 2

[ERROR] [1637285665.566221042, 3.246000000]: an illegal memory access was encountered 700

I meet the problem when roslaunch agile_autonomy simulation.launch, and it is likely the problem of GPU? If there is somebody who resolves the problem, can you tell me how can I do it? I will thank you appreciately.

Build is failing!

I am having error in catkin build. It shows message below without any error message.

[build] Summary: 12 of 71 packages succeeded.
[build] Warnings: 1 packages succeeded with warnings.
[build] Abandoned: 58 packages were abandoned.
[build] Failed: 1 packages failed.

1 Are there some people achieve the reappearance of the project?

Are there some people achieve the reappearance of the project? Why are there so many problems? If somebody is successful with the project, can you share the process?

'PlanLearning' object has no attribute 'XXX' when running test_trajectories.py

I have successfully compiled, but when running test_trajectories.py I get the following error (Occasionally it works successfully, but the drone doesn't avoid obstacles and flies straight to the destination):

  File "/home/lu/RPG/agile_autonomy/src/agile_autonomy/planner_learning/src/PlannerLearning/PlannerBase.py", line 316, in update_input_queues
    self.odom_rot_input
AttributeError: 'PlanLearning' object has no attribute 'odom_rot_input'

and

  File "test_trajectories.py", line 19, in <module>
    main()
  File "test_trajectories.py", line 15, in main
    trainer.perform_testing()
  File "/home/lu/RPG/agile_autonomy/src/agile_autonomy/planner_learning/dagger_training.py", line 144, in perform_testing
    self.start_experiment(rollout_idx)
  File "/home/lu/RPG/agile_autonomy/src/agile_autonomy/planner_learning/dagger_training.py", line 48, in start_experiment
    self.learner.expert_pub.publish(msg)
AttributeError: 'PlanLearning' object has no attribute 'expert_pub'
...
...
  File "/home/lu/miniconda3/envs/agile/lib/python3.7/logging/__init__.py", line 1116, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
NameError: name 'open' is not defined

My conda environment is python3.7 + tensorflow-gpu2.4 + CUDA11.0 + cudnn 8.0.

I did not follow the readme to run test_trajectories.py as:

cd agile_autonomy_ws
source catkin_aa/devel/setup.bash
conda activate tf_24
python test_trajectories.py --settings_file=config/test_settings.yaml

because the test_trajectories.py file could not be found. I run as following instead:

cd agile_autonomy_ws
source devel/setup.bash
conda activate tf_24
cd catkin_aa/src/agile_autonomy/planner_learning/
python test_trajectories.py --settings_file=config/test_settings.yaml

I don't know if this error is related to the run path.
Thank you for your reply!

Open3d not found while building catkin workspace

Environment: Ubuntu 20.04, ROS Noetic, CMake 3.16.3

I have installed ROS according to instructions from the official website (http://wiki.ros.org/noetic/Installation/Ubuntu) and followed your instructions line-by-line. However, the following error occurred when I tried catkin build step.

Edit: tried to update cmake to 3.21 and build Open3D from sources, but it didn't help.

Would appreciate any help.

denis@denis-HP-ENVY-Laptop-15-ep0xxx:~/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros$ catkin build
---------------------------------------------------------------------------------------------
Profile:                     default
Extending:        [explicit] /opt/ros/noetic
Workspace:                   /home/denis/agile_autonomy_ws/catkin_aa
---------------------------------------------------------------------------------------------
Build Space:        [exists] /home/denis/agile_autonomy_ws/catkin_aa/build
Devel Space:        [exists] /home/denis/agile_autonomy_ws/catkin_aa/devel
Install Space:      [unused] /home/denis/agile_autonomy_ws/catkin_aa/install
Log Space:          [exists] /home/denis/agile_autonomy_ws/catkin_aa/logs
Source Space:       [exists] /home/denis/agile_autonomy_ws/catkin_aa/src
DESTDIR:            [unused] None
---------------------------------------------------------------------------------------------
Devel Space Layout:          merged
Install Space Layout:        None
---------------------------------------------------------------------------------------------
Additional CMake Args:       -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fdiagnostics-color
Additional Make Args:        None
Additional catkin Make Args: None
Internal Make Job Server:    True
Cache Job Environments:      False
---------------------------------------------------------------------------------------------
Whitelisted Packages:        None
Blacklisted Packages:        None
---------------------------------------------------------------------------------------------
Workspace configuration appears valid.
---------------------------------------------------------------------------------------------
[build] Found '71' packages in 0.0 seconds.                                                                                                                                                               
[build] Package table is up to date.                                                                                                                                                                      
Starting  >>> catkin_boost_python_buildtool                                                                                                                                                               
Starting  >>> catkin_simple                                                                                                                                                                               
Starting  >>> decomp_util                                                                                                                                                                                 
Starting  >>> mav_msgs                                                                                                                                                                                    
Starting  >>> mav_state_machine_msgs                                                                                                                                                                      
Starting  >>> mav_system_msgs                                                                                                                                                                             
Starting  >>> motion_primitive_library                                                                                                                                                                    
Starting  >>> open3d_conversions                                                                                                                                                                          
Finished  <<< catkin_boost_python_buildtool                     [ 0.1 seconds ]                                                                                                                           
Finished  <<< mav_state_machine_msgs                            [ 0.4 seconds ]                                                                                                                           
Starting  >>> rotors_comm                                                                                                                                                                                 
Finished  <<< catkin_simple                                     [ 0.2 seconds ]                                                                                                                           
Finished  <<< mav_system_msgs                                   [ 0.5 seconds ]                                                                                                                           
Finished  <<< mav_msgs                                          [ 0.7 seconds ]                                                                                                                           
Starting  >>> assimp_catkin                                                                                                                                                                               
Starting  >>> catkin_boost_python_buildtool_test                                                                                                                                                          
__________________________________________________________________________________________________________________________________________________________________________________________________________
Errors     << open3d_conversions:make /home/denis/agile_autonomy_ws/catkin_aa/logs/open3d_conversions/build.make.006.log                                                                                  
In file included from /home/denis/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/src/open3d_conversions.cpp:15:0:
/home/denis/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19:10: fatal error: Open3D/Open3D.h: No such file or directory

 #include <Open3D/Open3D.h>
          ^~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/open3d_conversions.dir/build.make:63: CMakeFiles/open3d_conversions.dir/src/open3d_conversions.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:305: CMakeFiles/open3d_conversions.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
cd /home/denis/agile_autonomy_ws/catkin_aa/build/open3d_conversions; catkin build --get-env open3d_conversions | catkin env -si  /usr/bin/make --jobserver-auth=3,4; cd -

Why start with expert model

Hi, I noticed that you mentioned at the beginning always use expert in the python side. What problems will happen if always using network?

Facing error while running roslaunch agile_autonomy simulation.launch

when i am running this facing this error

started core service [/rosout]
process[gazebo-2]: started with pid [12282]
process[hummingbird/spawn_hummingbird-3]: started with pid [12287]
ERROR: cannot launch node of type [rpg_rotors_interface/rpg_rotors_interface]: Cannot locate node of type [rpg_rotors_interface] in package [rpg_rotors_interface]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [rpg_mpc/autopilot_mpc_instance]: Cannot locate node of type [autopilot_mpc_instance] in package [rpg_mpc]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [joy/joy_node]: joy
ROS path [0]=/opt/ros/melodic/share/ros
ROS path [1]=/home/enord/agile_autonomy_ws/catkin_aa/src/catkin_boost_python_buildtool/catkin_boost_python_buildtool
ROS path [2]=/home/enord/agile_autonomy_ws/catkin_aa/src/catkin_simple
ROS path [3]=/home/enord/agile_autonomy_ws/catkin_aa/src/assimp_catkin
ROS path [4]=/home/enord/agile_autonomy_ws/catkin_aa/src/catkin_boost_python_buildtool/catkin_boost_python_buildtool_test
ROS path [5]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/DecompROS/decomp_ros_msgs
ROS path [6]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/DecompROS/DecompUtil
ROS path [7]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/DecompROS/decomp_ros_utils
ROS path [8]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/DecompROS/decomp_test_node
ROS path [9]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_calibration_gui
ROS path [10]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_msgs
ROS path [11]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/davis_ros_driver
ROS path [12]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_calibration
ROS path [13]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_file_writer
ROS path [14]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_renderer
ROS path [15]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_ros_driver
ROS path [16]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvxplorer_ros_driver
ROS path [17]=/home/enord/agile_autonomy_ws/catkin_aa/src/eigen_catkin
ROS path [18]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/rpg_flightmare/example_vision
ROS path [19]=/home/enord/agile_autonomy_ws/catkin_aa/src/gflags_catkin
ROS path [20]=/home/enord/agile_autonomy_ws/catkin_aa/src/glog_catkin
ROS path [21]=/home/enord/agile_autonomy_ws/catkin_aa/src/eigen_checks
ROS path [22]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_comm
ROS path [23]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_msgs
ROS path [24]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_planning_msgs
ROS path [25]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_state_machine_msgs
ROS path [26]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_system_msgs
ROS path [27]=/home/enord/agile_autonomy_ws/catkin_aa/src/minimum_jerk_trajectories
ROS path [28]=/home/enord/agile_autonomy_ws/catkin_aa/src/minkindr/minkindr
ROS path [29]=/home/enord/agile_autonomy_ws/catkin_aa/src/minkindr_ros/minkindr_conversions
ROS path [30]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/motion_primitive_library
ROS path [31]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner
ROS path [32]=/home/enord/agile_autonomy_ws/catkin_aa/src/numpy_eigen
ROS path [33]=/home/enord/agile_autonomy_ws/catkin_aa/src/minkindr/minkindr_python
ROS path [34]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions
ROS path [35]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/planning_ros_msgs
ROS path [36]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/planning_ros_utils
ROS path [37]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/quad_launch_files
ROS path [38]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_common/quadrotor_msgs
ROS path [39]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy_msgs
ROS path [40]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_common/quadrotor_common
ROS path [41]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/trajectory_planning/polynomial_trajectories
ROS path [42]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/control/position_controller
ROS path [43]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_comm
ROS path [44]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_control
ROS path [45]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_description
ROS path [46]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_evaluation
ROS path [47]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_gazebo_plugins
ROS path [48]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_gazebo
ROS path [49]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_hil_interface
ROS path [50]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_joy_interface
ROS path [51]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_simulator
ROS path [52]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/simulation/rpg_rotors_interface
ROS path [53]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_single_board_io
ROS path [54]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/gui/rqt_quad_gui
ROS path [55]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rqt_rotors
ROS path [56]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/bridges/sbus_bridge
ROS path [57]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/utils/manual_flight_assistant
ROS path [58]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_common/state_predictor
ROS path [59]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/trajectory_planning/trajectory_generation_helper
ROS path [60]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/control/autopilot
ROS path [61]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy_utils
ROS path [62]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node
ROS path [63]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/planner_learning
ROS path [64]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpc
ROS path [65]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy
ROS path [66]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/test/rpg_quadrotor_integration_test
ROS path [67]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/traj_sampler
ROS path [68]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/utils/vbat_thrust_calibration
ROS path [69]=/opt/ros/melodic/share
ERROR: cannot launch node of type [manual_flight_assistant/manual_flight_assistant]: Cannot locate node of type [manual_flight_assistant] in package [manual_flight_assistant]. Make sure file exists in package path and permission is set to executable (chmod +x)
process[hummingbird/rqt_quad_gui-8]: started with pid [12288]
ERROR: cannot launch node of type [agile_autonomy/agile_autonomy]: Cannot locate node of type [agile_autonomy] in package [agile_autonomy]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [mpl_test_node/ellipsoid_planner_node]: Cannot locate node of type [ellipsoid_planner_node] in package [mpl_test_node]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [traj_sampler/generate_label]: Cannot locate node of type [generate_label] in package [traj_sampler]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [traj_sampler/generate_label]: Cannot locate node of type [generate_label] in package [traj_sampler]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [traj_sampler/generate_label]: Cannot locate node of type [generate_label] in package [traj_sampler]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [traj_sampler/generate_label]: Cannot locate node of type [generate_label] in package [traj_sampler]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [traj_sampler/generate_label]: Cannot locate node of type [generate_label] in package [traj_sampler]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [traj_sampler/generate_label]: Cannot locate node of type [generate_label] in package [traj_sampler]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [traj_sampler/generate_label]: Cannot locate node of type [generate_label] in package [traj_sampler]. Make sure file exists in package path and permission is set to executable (chmod +x)
ERROR: cannot launch node of type [traj_sampler/generate_label]: Cannot locate node of type [generate_label] in package [traj_sampler]. Make sure file exists in package path and permission is set to executable (chmod +x)
process[viz_face-19]: started with pid [12294]
ERROR: cannot launch node of type [flightrender/flightmare.x86_64]: flightrender
ROS path [0]=/opt/ros/melodic/share/ros
ROS path [1]=/home/enord/agile_autonomy_ws/catkin_aa/src/catkin_boost_python_buildtool/catkin_boost_python_buildtool
ROS path [2]=/home/enord/agile_autonomy_ws/catkin_aa/src/catkin_simple
ROS path [3]=/home/enord/agile_autonomy_ws/catkin_aa/src/assimp_catkin
ROS path [4]=/home/enord/agile_autonomy_ws/catkin_aa/src/catkin_boost_python_buildtool/catkin_boost_python_buildtool_test
ROS path [5]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/DecompROS/decomp_ros_msgs
ROS path [6]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/DecompROS/DecompUtil
ROS path [7]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/DecompROS/decomp_ros_utils
ROS path [8]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/DecompROS/decomp_test_node
ROS path [9]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_calibration_gui
ROS path [10]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_msgs
ROS path [11]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/davis_ros_driver
ROS path [12]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_calibration
ROS path [13]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_file_writer
ROS path [14]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_renderer
ROS path [15]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvs_ros_driver
ROS path [16]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_dvs_ros/dvxplorer_ros_driver
ROS path [17]=/home/enord/agile_autonomy_ws/catkin_aa/src/eigen_catkin
ROS path [18]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/rpg_flightmare/example_vision
ROS path [19]=/home/enord/agile_autonomy_ws/catkin_aa/src/gflags_catkin
ROS path [20]=/home/enord/agile_autonomy_ws/catkin_aa/src/glog_catkin
ROS path [21]=/home/enord/agile_autonomy_ws/catkin_aa/src/eigen_checks
ROS path [22]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_comm
ROS path [23]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_msgs
ROS path [24]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_planning_msgs
ROS path [25]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_state_machine_msgs
ROS path [26]=/home/enord/agile_autonomy_ws/catkin_aa/src/mav_comm/mav_system_msgs
ROS path [27]=/home/enord/agile_autonomy_ws/catkin_aa/src/minimum_jerk_trajectories
ROS path [28]=/home/enord/agile_autonomy_ws/catkin_aa/src/minkindr/minkindr
ROS path [29]=/home/enord/agile_autonomy_ws/catkin_aa/src/minkindr_ros/minkindr_conversions
ROS path [30]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/motion_primitive_library
ROS path [31]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner
ROS path [32]=/home/enord/agile_autonomy_ws/catkin_aa/src/numpy_eigen
ROS path [33]=/home/enord/agile_autonomy_ws/catkin_aa/src/minkindr/minkindr_python
ROS path [34]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions
ROS path [35]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/planning_ros_msgs
ROS path [36]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/planning_ros_utils
ROS path [37]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/quad_launch_files
ROS path [38]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_common/quadrotor_msgs
ROS path [39]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy_msgs
ROS path [40]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_common/quadrotor_common
ROS path [41]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/trajectory_planning/polynomial_trajectories
ROS path [42]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/control/position_controller
ROS path [43]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_comm
ROS path [44]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_control
ROS path [45]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_description
ROS path [46]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_evaluation
ROS path [47]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_gazebo_plugins
ROS path [48]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_gazebo
ROS path [49]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_hil_interface
ROS path [50]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_joy_interface
ROS path [51]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rotors_simulator
ROS path [52]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/simulation/rpg_rotors_interface
ROS path [53]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_single_board_io
ROS path [54]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/gui/rqt_quad_gui
ROS path [55]=/home/enord/agile_autonomy_ws/catkin_aa/src/rotors_simulator/rqt_rotors
ROS path [56]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/bridges/sbus_bridge
ROS path [57]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/utils/manual_flight_assistant
ROS path [58]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_common/state_predictor
ROS path [59]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/trajectory_planning/trajectory_generation_helper
ROS path [60]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/control/autopilot
ROS path [61]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy_utils
ROS path [62]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node
ROS path [63]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/planner_learning
ROS path [64]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_mpc
ROS path [65]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy
ROS path [66]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/test/rpg_quadrotor_integration_test
ROS path [67]=/home/enord/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/traj_sampler
ROS path [68]=/home/enord/agile_autonomy_ws/catkin_aa/src/rpg_quadrotor_control/utils/vbat_thrust_calibration
ROS path [69]=/opt/ros/melodic/share

implement this package on a real quad

Hi !
I would really love to implement this package on a real quad
Could you give any suggestion?

Thank you !

can't fly when run test_trajectories.py

When run "python test_trajectories.py --settings_file=config/test_settings.yaml", it will loop as follows:

In rviz, the RGM is NO Image

Check the rostopic, I find many topics received no message.

What's wrong with this? It's the fightmare can't communicate with ROS ? Thanks a lot !

Further instruction needed for someone new to this.

After running roslaunch agile_autonomy simulation.launch

I am not sure what I should expect or what I should do next in order to see this demo. In the Unity window, the environment is being rendered in real time, and you can observe the grass is moving. The view angle is fixed.

The partial console output is shown as below.

roslaunch agile_autonomy simulation.launch

[ INFO] [1649622819.898319698]: [/hummingbird/agile_autonomy] camera/baseline = 0.1
[ INFO] [1649622819.898993112]: [/hummingbird/agile_autonomy] camera/pitch_angle_deg = 0
Vertical Fov is 74.7021
Initializing ZMQ connections...
Initializing ZMQ done...
[ INFO] [1649622819.901140818]: Connecting to unity...
Ready ? : 0
[ INFO] [1649622820.083847360]: Finished loading Gazebo ROS API Plugin.
[ INFO] [1649622820.084971604]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting...
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
[ INFO] [1649622821.892760684]: waitForService: Service [/gazebo/set_physics_properties] is now available.
Ready ? : 0
[ INFO] [1649622821.910235945]: Physics dynamic reconfigure ready.
[INFO] [1649622821.992587, 0.000000]: Calling service /gazebo/spawn_urdf_model
[INFO] [1649622822.084647, 0.000000]: Spawn status: SpawnModel: Successfully spawned entity
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
[hummingbird/spawn_hummingbird-3] process has finished cleanly
log file: /home/psun/.ros/log/808eb694-b90d-11ec-b32c-11a93648a3ed/hummingbird-spawn_hummingbird-3*.log
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 1
[ INFO] [1649622827.802552360, 2.876000000]: Unity is ready!
[ERROR] [1649622828.212042755, 3.080000000]: no kernel image is available for execution on the device 209

[ERROR] [1649622828.321543409, 3.135000000]: no kernel image is available for execution on the device 209

[ERROR] [1649622828.459425264, 3.203000000]: no kernel image is available for execution on the device 209

[ERROR] [1649622828.582797969, 3.265000000]: no kernel image is available for execution on the device 209

[ERROR] [1649622828.721430274, 3.334000000]: no kernel image is available for execution on the device 209

^C[ERROR] [1649622828.859060628, 3.403000000]: no kernel image is available for execution on the device 209

open3d_conversions error when catkin build

[build] Found '71' packages in 0.0 seconds.
[build] Package table is up to date.
Starting >>> catkin_boost_python_buildtool
Starting >>> catkin_simple
Starting >>> decomp_util
Starting >>> mav_msgs
Starting >>> mav_state_machine_msgs
Starting >>> mav_system_msgs
Starting >>> motion_primitive_library
Starting >>> open3d_conversions
Starting >>> rotors_comm
Starting >>> rotors_description
Starting >>> rotors_evaluation
Starting >>> rqt_rotors
Finished <<< mav_state_machine_msgs [ 0.1 seconds ]
Finished <<< mav_msgs [ 0.1 seconds ]
Starting >>> mav_planning_msgs
Starting >>> rotors_control
Starting >>> rotors_hil_interface
Starting >>> rotors_joy_interface
Finished <<< rotors_evaluation [ 0.1 seconds ]
Finished <<< catkin_boost_python_buildtool [ 0.0 seconds ]
Finished <<< mav_system_msgs [ 0.1 seconds ]
Finished <<< rqt_rotors [ 0.1 seconds ]
Finished <<< catkin_simple [ 0.1 seconds ]
Starting >>> assimp_catkin
Starting >>> catkin_boost_python_buildtool_test
Starting >>> decomp_ros_msgs
Starting >>> eigen_catkin
Starting >>> flightrender
Starting >>> gflags_catkin
Starting >>> minimum_jerk_trajectories
Starting >>> planning_ros_msgs
Starting >>> quad_launch_files
Starting >>> quadrotor_msgs
Starting >>> rpg_single_board_io
Starting >>> rpgq_msgs
Finished <<< rotors_comm [ 0.1 seconds ]
Finished <<< rotors_description [ 0.1 seconds ]
Finished <<< rotors_control [ 0.1 seconds ]
Finished <<< rotors_hil_interface [ 0.1 seconds ]
Starting >>> rotors_gazebo_plugins
Finished <<< mav_planning_msgs [ 0.1 seconds ]
Starting >>> mav_comm
Finished <<< rotors_joy_interface [ 0.1 seconds ]
Finished <<< decomp_ros_msgs [ 0.1 seconds ]
Finished <<< quad_launch_files [ 0.1 seconds ]
Finished <<< gflags_catkin [ 0.1 seconds ]
Starting >>> glog_catkin

Warnings << open3d_conversions:check /home/pc205/agile_autonomy_ws/catkin_aa/logs/open3d_conversions/build.check.022.log
Searching open3D in /usr/local/bin/cmake/
Found OpenMP TRUE 0.9.0.0 OpenMP::OpenMP_CXX
Found Open3D 0.9.0.0
cd /home/pc205/agile_autonomy_ws/catkin_aa/build/open3d_conversions; catkin build --get-env open3d_conversions | catkin env -si /usr/bin/make cmake_check_build_system; cd -

...............................................................................
Finished <<< open3d_conversions [ 0.5 seconds ]
Finished <<< eigen_catkin [ 0.1 seconds ]
Starting >>> numpy_eigen
Finished <<< catkin_boost_python_buildtool_test [ 0.1 seconds ]
Finished <<< rpgq_msgs [ 0.1 seconds ]
Starting >>> rpgq_common
Finished <<< planning_ros_msgs [ 0.1 seconds ]
Finished <<< rpg_single_board_io [ 0.1 seconds ]
Finished <<< assimp_catkin [ 0.1 seconds ]
Finished <<< minimum_jerk_trajectories [ 0.1 seconds ]
Finished <<< flightrender [ 0.1 seconds ]
Finished <<< quadrotor_msgs [ 0.1 seconds ]
Starting >>> agile_autonomy_msgs
Starting >>> quadrotor_common
Starting >>> rqt_quad_gui
Starting >>> vbat_thrust_calibration
Finished <<< motion_primitive_library [ 0.2 seconds ]
Starting >>> planning_ros_utils
Finished <<< decomp_util [ 0.1 seconds ]
Starting >>> decomp_ros_utils
Starting >>> mpl_external_planner
Finished <<< mav_comm [ 0.1 seconds ]
Finished <<< rotors_gazebo_plugins [ 0.2 seconds ]
Starting >>> rotors_gazebo
Finished <<< glog_catkin [ 0.1 seconds ]
Starting >>> eigen_checks
Finished <<< numpy_eigen [ 0.1 seconds ]
Finished <<< rpgq_common [ 0.1 seconds ]
Starting >>> rpgq_command_bridge
Starting >>> rpgq_components
Finished <<< quadrotor_common [ 0.1 seconds ]
Starting >>> polynomial_trajectories
Starting >>> position_controller
Starting >>> sbus_bridge
Starting >>> state_predictor
Starting >>> trajectory_visualizer
Finished <<< vbat_thrust_calibration [ 0.1 seconds ]
Finished <<< agile_autonomy_msgs [ 0.1 seconds ]
Finished <<< rqt_quad_gui [ 0.1 seconds ]
Finished <<< planning_ros_utils [ 0.1 seconds ]
Finished <<< decomp_ros_utils [ 0.1 seconds ]
Starting >>> decomp_test_node
Finished <<< mpl_external_planner [ 0.1 seconds ]
Finished <<< rotors_gazebo [ 0.1 seconds ]
Starting >>> rotors_simulator
Finished <<< eigen_checks [ 0.1 seconds ]
Starting >>> minkindr
Finished <<< rpgq_components [ 0.1 seconds ]
Finished <<< rpgq_command_bridge [ 0.1 seconds ]
Starting >>> rpgq_simulator
Finished <<< polynomial_trajectories [ 0.1 seconds ]
Finished <<< position_controller [ 0.1 seconds ]
Finished <<< trajectory_visualizer [ 0.1 seconds ]
Finished <<< sbus_bridge [ 0.1 seconds ]
Starting >>> manual_flight_assistant
Finished <<< state_predictor [ 0.1 seconds ]
Finished <<< decomp_test_node [ 0.1 seconds ]
Finished <<< rotors_simulator [ 0.1 seconds ]
Finished <<< minkindr [ 0.1 seconds ]
Starting >>> minkindr_conversions
Starting >>> minkindr_python
Starting >>> rpg_common
Finished <<< rpgq_simulator [ 0.1 seconds ]
Starting >>> example_pcd
Starting >>> example_vision
Starting >>> example_vison
Finished <<< manual_flight_assistant [ 0.1 seconds ]
Finished <<< rpg_common [ 0.1 seconds ]
Starting >>> rpg_rotors_interface
Starting >>> trajectory_generation_helper
Finished <<< minkindr_conversions [ 0.1 seconds ]
Finished <<< minkindr_python [ 0.1 seconds ]
Finished <<< example_vison [ 0.1 seconds ]
Finished <<< example_vision [ 0.1 seconds ]
Finished <<< example_pcd [ 0.1 seconds ]
Finished <<< trajectory_generation_helper [ 0.1 seconds ]
Starting >>> autopilot
Finished <<< rpg_rotors_interface [ 0.1 seconds ]
Finished <<< autopilot [ 0.1 seconds ]
Starting >>> agile_autonomy_utils
Starting >>> example_control
Starting >>> planner_learning
Starting >>> rpg_mpc
Starting >>> rpg_quadrotor_integration_test
Finished <<< rpg_quadrotor_integration_test [ 0.1 seconds ]
Finished <<< rpg_mpc [ 0.1 seconds ]
Finished <<< example_control [ 0.1 seconds ]
Finished <<< agile_autonomy_utils [ 0.1 seconds ]
Starting >>> mpl_test_node
Starting >>> agile_autonomy
Starting >>> traj_sampler
Finished <<< planner_learning [ 0.1 seconds ]
Finished <<< agile_autonomy [ 0.1 seconds ]
Finished <<< traj_sampler [ 0.1 seconds ]

Errors << mpl_test_node:cmake /home/pc205/agile_autonomy_ws/catkin_aa/logs/mpl_test_node/build.cmake.013.log
CMake Warning (dev) at CMakeLists.txt:2 (project):
Policy CMP0048 is not set: project() command manages VERSION variables.
Run "cmake --help-policy CMP0048" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.

The following variable(s) would be set to empty:

CMAKE_PROJECT_VERSION
CMAKE_PROJECT_VERSION_MAJOR
CMAKE_PROJECT_VERSION_MINOR
CMAKE_PROJECT_VERSION_PATCH

This warning is for project developers. Use -Wno-dev to suppress it.

CMake Error at /home/pc205/agile_autonomy_ws/catkin_aa/devel/share/open3d_conversions/cmake/open3d_conversionsConfig.cmake:173 (message):
Project 'mpl_test_node' tried to find library 'Open3D'. The library is
neither a target nor built/installed properly. Did you compile project
'open3d_conversions'? Did you find_package() it before the subdirectory
containing its code is included?
Call Stack (most recent call first):
CMakeLists.txt:27 (find_package)

cd /home/pc205/agile_autonomy_ws/catkin_aa/build/mpl_test_node; catkin build --get-env mpl_test_node | catkin env -si /usr/bin/cmake /home/pc205/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node --no-warn-unused-cli -DCATKIN_DEVEL_PREFIX=/home/pc205/agile_autonomy_ws/catkin_aa/devel -DCMAKE_INSTALL_PREFIX=/home/pc205/agile_autonomy_ws/catkin_aa/install -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fdiagnostics-color -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3; cd -

...............................................................................
Failed << mpl_test_node:cmake [ Exited with code 1 ]
Failed <<< mpl_test_node [ 0.4 seconds ]
[build] Summary: 70 of 71 packages succeeded.
[build] Ignored: None.
[build] Warnings: 1 packages succeeded with warnings.
[build] Abandoned: None.
[build] Failed: 1 packages failed.
[build] Runtime: 4.2 seconds total.

Failed to reach some repository

vcs-import < agile_autonomy/dependencies.yaml
results some errors that failed to reach repositories.

=== ./rpg_common (git) ===
Could not determine ref type of version: ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_flightmare (git) ===
Could not determine ref type of version: ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
=== ./rpg_mpl_ros (git) ===
Could not determine ref type of version: ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Facing error when building open3d_conversions

Errors     << open3d_conversions:make /home/reimu/agile_autonomy_ws/catkin_aa/logs/open3d_conversions/build.make.018.log
/usr/bin/ld: cannot find -lJPEG::JPEG
collect2: error: ld returned 1 exit status
make[2]: *** [/home/reimu/agile_autonomy_ws/catkin_aa/devel/lib/libopen3d_conversions.so] Error 1
make[1]: *** [CMakeFiles/open3d_conversions.dir/all] Error 2
make: *** [all] Error 2
cd /home/reimu/agile_autonomy_ws/catkin_aa/build/open3d_conversions; catkin build --get-env open3d_conversions | catkin env -si  /usr/bin/make --jobserver-fds=6,7 -j; cd -

Environment:Ubuntu 18.04, ROS:Melodic, CMake:3.22.0
I have built Open3D from sources, but the above error occurred when I tried catkin build step.
Would appreciate any help!

Questions about training with pre-collected datasets

error while run python test_trajectories.py

2021-12-14 16:24:17.789855: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-12-14 16:24:18.717899: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-12-14 16:24:18.717983: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2021-12-14 16:24:18.718069: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:24:18.718719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 23.69GiB deviceMemoryBandwidth: 871.81GiB/s
2021-12-14 16:24:18.718752: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-12-14 16:24:18.719952: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-12-14 16:24:18.720016: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.10
2021-12-14 16:24:18.721169: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-12-14 16:24:18.721397: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-12-14 16:24:18.722677: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-12-14 16:24:18.723358: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10
2021-12-14 16:24:18.725769: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.7
2021-12-14 16:24:18.725855: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:24:18.726511: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:24:18.727095: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-12-14 16:24:18.728012: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-12-14 16:24:18.728749: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:24:18.729386: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 23.69GiB deviceMemoryBandwidth: 871.81GiB/s
2021-12-14 16:24:18.729449: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-12-14 16:24:18.729514: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-12-14 16:24:18.729536: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.10
2021-12-14 16:24:18.729555: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-12-14 16:24:18.729573: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-12-14 16:24:18.729591: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-12-14 16:24:18.729608: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10
2021-12-14 16:24:18.729627: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.7
2021-12-14 16:24:18.729672: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:24:18.730287: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:24:18.730868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-12-14 16:24:18.730925: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-12-14 16:30:48.839481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-12-14 16:30:48.839510: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2021-12-14 16:30:48.839519: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2021-12-14 16:30:48.839654: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:30:48.840286: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:30:48.840902: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-12-14 16:30:48.841501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21697 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:01:00.0, compute capability: 8.6)
2021-12-14 16:30:48.841690: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set

Restored from models/ckpt-50

2021-12-14 16:30:49.790317: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2021-12-14 16:30:49.825588: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 3187200000 Hz
2021-12-14 16:30:50.065749: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.7
2021-12-14 16:41:34.065744: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
Net initialized
Setting Tree Spacing to 5
Setting Object Spacing to 5

 RESET SIMULATION

Unpausing Physics...

Placing quadrotor...
success: True
status_message: "SetModelState: set model state done"
Received call to Clear Buffer and Restart Experiment

/home/pc205/anaconda3/envs/rpg_uav/lib/python3.8/site-packages/tensorflow/python/keras/backend.py:434: UserWarning: tf.keras.backend.set_learning_phase is deprecated and will be removed after 2020-10-11. To update it, simply pass a True/False value to the training argument of the __call__ method of your layer or model.
warnings.warn('tf.keras.backend.set_learning_phase is deprecated and '
Resetting experiment

Done Reset
Doing experiment 0
Current experiment failed. Will try again

 RESET SIMULATION

Unpausing Physics...

Placing quadrotor...
success: True
status_message: "SetModelState: set model state done"
Received call to Clear Buffer and Restart Experiment

Resetting experiment

Done Reset
Doing experiment 0
Current experiment failed. Will try again
Traceback (most recent call last):
File "test_trajectories.py", line 19, in
main()
File "test_trajectories.py", line 15, in main
trainer.perform_testing()
File "/home/pc205/agile_autonomy_ws/catkin_aa/src/agile_autonomy/planner_learning/dagger_training.py", line 186, in perform_testing
sorted(os.listdir(self.settings.expert_folder))[-1])
IndexError: list index out of range
--- Logging error ---
Traceback (most recent call last):
File "/home/pc205/anaconda3/envs/rpg_uav/lib/python3.8/logging/handlers.py", line 69, in emit
if self.shouldRollover(record):
File "/home/pc205/anaconda3/envs/rpg_uav/lib/python3.8/logging/handlers.py", line 183, in shouldRollover
self.stream = self._open()
File "/home/pc205/anaconda3/envs/rpg_uav/lib/python3.8/logging/init.py", line 1176, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
NameError: name 'open' is not defined
Call stack:
File "/home/pc205/anaconda3/envs/rpg_uav/lib/python3.8/site-packages/tensorflow/python/training/tracking/util.py", line 160, in del
log_fn("Unresolved object in checkpoint: {}"
File "/home/pc205/anaconda3/envs/rpg_uav/lib/python3.8/site-packages/tensorflow/python/platform/tf_logging.py", line 178, in warning
get_logger().warning(msg, *args, **kwargs)
Message: 'Unresolved object in checkpoint: (root).optimizer.iter'
Arguments: ()

How I can test this in real time physical environment?

I have a physical drone platform with Nvidia Xavier agx and depth cameras. Will it be possible to fly a drone with these tests?

is this a scalable approach?

Hi,
I went through the paper and didn't find any mentions related to swarm, so I was curious if one can leverage this approach for a swarm of drones?
are there any plans to support the swarm based architecture?

thanks

valdation dataset

Hi, Where is the validation dataset? @antonilo When I train use you pre-collected dataset, I met the problem:

Thanks

How you are calculating depthmap

How can i replace intel realsense with my own custom Stereo cmera

RuntimeError: Multiple packages found with the same name "example_control"

When I run "catkin build", I received this error:

Profile:                     default
Extending:        [explicit] /opt/ros/melodic
Workspace:                   /home/zlt/agile_autonomy_ws/catkin_aa
--------------------------------------------------------------------------------
Build Space:        [exists] /home/zlt/agile_autonomy_ws/catkin_aa/build
Devel Space:        [exists] /home/zlt/agile_autonomy_ws/catkin_aa/devel
Install Space:      [unused] /home/zlt/agile_autonomy_ws/catkin_aa/install
Log Space:          [exists] /home/zlt/agile_autonomy_ws/catkin_aa/logs
Source Space:       [exists] /home/zlt/agile_autonomy_ws/catkin_aa/src
DESTDIR:            [unused] None
--------------------------------------------------------------------------------
Devel Space Layout:          merged
Install Space Layout:        None
--------------------------------------------------------------------------------
Additional CMake Args:       -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fdiagnostics-color
Additional Make Args:        None
Additional catkin Make Args: None
Internal Make Job Server:    True
Cache Job Environments:      False
--------------------------------------------------------------------------------
Whitelisted Packages:        None
Blacklisted Packages:        None
--------------------------------------------------------------------------------
Workspace configuration appears valid.
--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/bin/catkin", line 11, in <module>
    load_entry_point('catkin-tools==0.6.1', 'console_scripts', 'catkin')()
  File "/usr/lib/python2.7/dist-packages/catkin_tools/commands/catkin.py", line 272, in main
    catkin_main(sysargs)
  File "/usr/lib/python2.7/dist-packages/catkin_tools/commands/catkin.py", line 267, in catkin_main
    sys.exit(args.main(args) or 0)
  File "/usr/lib/python2.7/dist-packages/catkin_tools/verbs/catkin_build/cli.py", line 422, in main
    summarize_build=opts.summarize  # Can be True, False, or None
  File "/usr/lib/python2.7/dist-packages/catkin_tools/verbs/catkin_build/build.py", line 283, in build_isolated_workspace
    workspace_packages = find_packages(context.source_space_abs, exclude_subspaces=True, warnings=[])
  File "/usr/lib/python2.7/dist-packages/catkin_pkg/packages.py", line 96, in find_packages
    raise RuntimeError('\n'.join(duplicates))
**RuntimeError: Multiple packages found with the same name "example_control":**
- flightmare_agile_autonomy/examples/example_control
- rpg_flightmare/examples/example_control
Multiple packages found with the same name "example_pcd":
- flightmare_agile_autonomy/examples/example_pcd
- rpg_flightmare/examples/example_pcd
Multiple packages found with the same name "example_vison":
- flightmare_agile_autonomy/examples/example_vision
- rpg_flightmare/examples/example_vision
Multiple packages found with the same name "flightrender":
- flightmare_agile_autonomy/flightrender
- rpg_flightmare/flightrender
Multiple packages found with the same name "rpgq_command_bridge":
- flightmare_agile_autonomy/rpgq_command_bridge
- rpg_flightmare/rpgq_command_bridge
Multiple packages found with the same name "rpgq_common":
- flightmare_agile_autonomy/rpgq_common
- rpg_flightmare/rpgq_common
Multiple packages found with the same name "rpgq_components":
- flightmare_agile_autonomy/rpgq_components
- rpg_flightmare/rpgq_components
Multiple packages found with the same name "rpgq_msgs":
- flightmare_agile_autonomy/rpgq_msgs
- rpg_flightmare/rpgq_msgs
Multiple packages found with the same name "rpgq_simulator":
- flightmare_agile_autonomy/rpgq_simulator
- rpg_flightmare/rpgq_simulator

Error while 'catkin build'

Environment: ubuntu 18.04, ROS Melodic, cmake version 3.22.1, gcc/g++7.5. open3d v0.14 installed.

The following errors occurred when I tried catkin build step.

I have tried to set set(CMAKE_CXX_FLAGS "-std=c++14 ") in CMakeList.txt, but it didn't work for that error.

Thanks for any helps!

Errors     << mpl_test_node:make /home/rui/agile_autonomy_ws/catkin_aa/logs/mpl_test_node/build.make.001.log
In file included from /usr/local/include/open3d/geometry/PointCloud.h:36:0,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner/include/mpl_external_planner/ellipsoid_planner/ellipsoid_util.h:11,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner/include/mpl_external_planner/ellipsoid_planner/env_cloud.h:8,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_external_planner/include/mpl_external_planner/ellipsoid_planner/ellipsoid_planner.h:9,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:3:
/usr/local/include/open3d/utility/Optional.h:308:18: error: ‘constexpr T& open3d::utility::optional<T>::contained_val() const &’ cannot be overloaded
     constexpr T& contained_val() & { return OptionalBase<T>::storage_.value_; }
                  ^~~~~~~~~~~~~
/usr/local/include/open3d/utility/Optional.h:302:24: error: with ‘constexpr const T& open3d::utility::optional<T>::contained_val() const &’
     constexpr const T& contained_val() const& {
                        ^~~~~~~~~~~~~
/usr/local/include/open3d/utility/Optional.h:478:36: error: ‘constexpr T* open3d::utility::optional<T>::operator->() const’ cannot be overloaded
     TR2_OPTIONAL_HOST_CONSTEXPR T* operator->() {
                                    ^~~~~~~~
/usr/local/include/open3d/utility/Optional.h:474:42: error: with ‘constexpr const T* open3d::utility::optional<T>::operator->() const’
     TR2_OPTIONAL_HOST_CONSTEXPR T const* operator->() const {
                                          ^~~~~~~~
/usr/local/include/open3d/utility/Optional.h:487:36: error: ‘constexpr T& open3d::utility::optional<T>::operator*() const &’ cannot be overloaded
     TR2_OPTIONAL_HOST_CONSTEXPR T& operator*() & {
                                    ^~~~~~~~
/usr/local/include/open3d/utility/Optional.h:483:42: error: with ‘constexpr const T& open3d::utility::optional<T>::operator*() const &’
     TR2_OPTIONAL_HOST_CONSTEXPR T const& operator*() const& {
                                          ^~~~~~~~
/usr/local/include/open3d/utility/Optional.h:504:36: error: ‘constexpr T& open3d::utility::optional<T>::value() const &’ cannot be overloaded
     TR2_OPTIONAL_HOST_CONSTEXPR T& value() & {
                                    ^~~~~
/usr/local/include/open3d/utility/Optional.h:497:42: error: with ‘constexpr const T& open3d::utility::optional<T>::value() const &’
     TR2_OPTIONAL_HOST_CONSTEXPR T const& value() const& {
                                          ^~~~~
In file included from /usr/local/include/open3d/core/Tensor.h:42:0,
                 from /usr/local/include/open3d/core/EigenConverter.h:34,
                 from /usr/local/include/open3d/Open3D.h:38,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:19:
/usr/local/include/open3d/core/TensorInit.h:95:49: error: ‘std::index_sequence’ has not been declared
 SizeVector InitializerShape(const L& list, std::index_sequence<D...>) {
                                                 ^~~~~~~~~~~~~~
/usr/local/include/open3d/core/TensorInit.h:95:63: error: expected ‘,’ or ‘...’ before ‘<’ token
 SizeVector InitializerShape(const L& list, std::index_sequence<D...>) {
                                                               ^
/usr/local/include/open3d/core/TensorInit.h: In function ‘open3d::core::SizeVector open3d::core::tensor_init::InferShape(const L&)’:
/usr/local/include/open3d/core/TensorInit.h:103:24: error: ‘make_index_sequence’ is not a member of ‘std’
             list, std::make_index_sequence<InitializerDim<L>::value>());
                        ^~~~~~~~~~~~~~~~~~~
/usr/local/include/open3d/core/TensorInit.h:103:24: note: suggested alternative:
In file included from /usr/local/include/open3d/utility/Logging.h:47:0,
                 from /usr/local/include/open3d/core/Device.h:32,
                 from /usr/local/include/open3d/core/Blob.h:33,
                 from /usr/local/include/open3d/Open3D.h:34,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:19:
/usr/local/include/open3d/3rdparty/fmt/ranges.h:141:66: note:   ‘fmt::v6::internal::make_index_sequence’
 using make_index_sequence = make_integer_sequence<std::size_t, N>;
                                                                  ^
In file included from /usr/local/include/open3d/core/Tensor.h:42:0,
                 from /usr/local/include/open3d/core/EigenConverter.h:34,
                 from /usr/local/include/open3d/Open3D.h:38,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:19:
/usr/local/include/open3d/core/TensorInit.h:103:70: error: expected primary-expression before ‘)’ token
             list, std::make_index_sequence<InitializerDim<L>::value>());
                                                                      ^
In file included from /usr/local/include/open3d/visualization/gui/UIImage.h:30:0,
                 from /usr/local/include/open3d/visualization/gui/Button.h:33,
                 from /usr/local/include/open3d/Open3D.h:119,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/open3d_conversions/include/open3d_conversions/open3d_conversions.h:19,
                 from /home/rui/agile_autonomy_ws/catkin_aa/src/rpg_mpl_ros/mpl_test_node/src/ellipsoid_planner_node.cpp:19:
/usr/local/include/open3d/visualization/rendering/RendererHandle.h: At global scope:
/usr/local/include/open3d/visualization/rendering/RendererHandle.h:189:35: error: ‘format’ function uses ‘auto’ type specifier without trailing return type
                 FormatContext& ctx) {
                                   ^
/usr/local/include/open3d/visualization/rendering/RendererHandle.h:189:35: note: deduced return type only available with -std=c++14 or -std=gnu++14
/usr/local/include/open3d/visualization/rendering/RendererHandle.h:197:43: error: ‘parse’ function uses ‘auto’ type specifier without trailing return type
     constexpr auto parse(ParseContext& ctx) {
                                           ^
/usr/local/include/open3d/visualization/rendering/RendererHandle.h:197:43: note: deduced return type only available with -std=c++14 or -std=gnu++14
make[2]: *** [CMakeFiles/ellipsoid_planner_node.dir/src/ellipsoid_planner_node.cpp.o] Error 1
make[1]: *** [CMakeFiles/ellipsoid_planner_node.dir/all] Error 2
make: *** [all] Error 2

train problem

hi, did you only train on the forest dataset? Can the strategy trained under the forest dataset be directly used in other scenarios?

You use the CPU to run the simulation?

I saw your paper that said the simulation was ran with the CPU. So did that mean you used the CPU to run the simulation? I use your code, and it uses the GPU to run the simulation and gives me the error "out of memory 2".

Where can I find simulation map of Fig4 in the paper(Forest, Narrow gap, Disaster, and City street).

Hi there,

I'm currently trying to make some comparisons with your algorithm. Is there any way for me to test my planner in the same scenario of Fig. 4 in your paper?

I only see the forest in flightmare, what about the other three maps?

Thanks for the help.

Number of Features extracted from image/state

Hello!

I have a question on your features extraction architecture (PlaNet network), especially on layer resize_op_3:
self.resize_op_3 = [Conv1D(self.config.modes, kernel_size=3, strides=1, padding='valid', dilation_rate=1,use_bias=has_bias)]. (line 88)
Its input is a vector size (batch_size, 32, seq_len). According to the convolution parameters, the output should be size [batch_size, 30, modes] instead of [batch_size, 32, modes] (drop of the two last values of the input: kernel_size=3 and padding=0). Therefore, are you using a 30-dimensional states features vector to predict the actions or a 32-dim? Or should I change the way I am using this layer?

The same goes for the image extraction part with layer resize_op_2.

Thank you in advance!

Environment not rendering after roslaunch agile_autonomy simulation.launch

When I run roslaunch agile_autonomy simulation.launch, I get the following error:

...
[ INFO] [1638835491.189653098]: Connecting to unity...
[ INFO] [1638835491.191211253]: [/hummingbird/agile_autonomy] unity/spawn_trees = 1
[ INFO] [1638835491.193514743]: [/hummingbird/agile_autonomy] unity/avg_tree_spacing = 4
[ INFO] [1638835491.194996597]: [/hummingbird/agile_autonomy] unity/perform_sgm = 1
[ INFO] [1638835491.197263122]: [/hummingbird/agile_autonomy] unity/spawn_objects = 0
[ INFO] [1638835491.200555849]: [/hummingbird/agile_autonomy] unity/avg_object_spacing = 5
[ INFO] [1638835491.203184788]: [/hummingbird/agile_autonomy] unity/rand_width = 5
[ INFO] [1638835491.206424910]: [/hummingbird/agile_autonomy] unity/seed = 69
[ INFO] [1638835491.209098221]: [/hummingbird/agile_autonomy] unity/env_idx = 4
[ INFO] [1638835491.224689504]: [/hummingbird/agile_autonomy] unity/pointcloud_resolution = 0.2
[ INFO] [1638835491.227539812]: [/hummingbird/agile_autonomy] camera/fov = 91
[ INFO] [1638835491.229623364]: [/hummingbird/agile_autonomy] camera/width = 640
[ INFO] [1638835491.233344206]: [/hummingbird/agile_autonomy] camera/height = 480
[ INFO] [1638835491.234598109]: [/hummingbird/agile_autonomy] camera/baseline = 0.1
[ INFO] [1638835491.235841083]: [/hummingbird/agile_autonomy] camera/pitch_angle_deg = 0
terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
[hummingbird/agile_autonomy-9] process has died [pid 296, exit code -6, cmd /home/improbable/agile_autonomy_ws/catkin_aa/devel/lib/agile_autonomy/agile_autonomy completed_global_plan:=/test_primitive/completed_planning __name:=agile_autonomy __log:=/home/improbable/.ros/log/4ae3846c-56f1-11ec-bc8d-0c9d92c3307d/hummingbird-agile_autonomy-9.log].
log file: /home/improbable/.ros/log/4ae3846c-56f1-11ec-bc8d-0c9d92c3307d/hummingbird-agile_autonomy-9*.log
[flight_render-20] process has died [pid 500, exit code -11, cmd /home/improbable/agile_autonomy_ws/catkin_aa/src/agile_autonomy/flightmare_agile_autonomy/flightrender/20201127/flightmare.x86_64 __name:=flight_render __log:=/home/improbable/.ros/log/4ae3846c-56f1-11ec-bc8d-0c9d92c3307d/flight_render-20.log].
log file: /home/improbable/.ros/log/4ae3846c-56f1-11ec-bc8d-0c9d92c3307d/flight_render-20*.log
[ INFO] [1638835492.436102818]: waitForService: Service [/gazebo/set_physics_properties] is now available.
[INFO] [1638835492.584719, 0.000000]: Calling service /gazebo/spawn_urdf_model
[ INFO] [1638835492.596939197, 0.013000000]: Physics dynamic reconfigure ready.
[INFO] [1638835492.854186, 0.000000]: Spawn status: SpawnModel: Successfully spawned entity
[hummingbird/spawn_hummingbird-3] process has finished cleanly
log file: /home/improbable/.ros/log/4ae3846c-56f1-11ec-bc8d-0c9d92c3307d/hummingbird-spawn_hummingbird-3*.log

This is causing the environment to not show up in Rviz (and probably something more important isn't working).

Cannot download dataset

I am trying to download the dataset, but it shows server error after downloading 37.6 GB and stops the download after that. Can you please look into this.

Thanks.

Reference trajectory in train dataset

Hello! When I downloaded the training data set you provided, I found that all the reference trajectories in the file ’reference_trajectory.csv‘ were a straight line. Did you not use the With global planning method proposed in the Supplementary Material S6 of the paper in the training data set? When I run the command python test_trajectories.py, the drone’s trajectory is also a straight line....... This error may be that my compilation was not successful。

Diagram of the network architecture

Is there a diagram of the neural network architecture (specifically the branches and heads) used in this paper?

Flightmare is not responding after roslaunch agile_autonomy simulation.launch

Hello! I have a trouble with Flightmare simulator.
Here is my roslaunch agile_autonomy simulation.launch result
simulation_log.txt
As far as I know, this message have no error or ploblem to excute the all nodes in launch file.

Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
�]2;/home/seungwooubuntu/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy/launch/simulation.launch�
�[93mredefining global symbol: pi�[0m
when processing file: /home/seungwooubuntu/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy/resources/urdf/component_snippets.xacro
included from: /home/seungwooubuntu/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy/resources/urdf/hummingbird_base.xacro
included from: /home/seungwooubuntu/agile_autonomy_ws/catkin_aa/src/agile_autonomy/data_generation/agile_autonomy/resources/urdf/mav_generic_odometry_sensor_rgb.gazebo
�[1mstarted roslaunch server http://seungwooubuntu:41435/�[0m

SUMMARY
========

PARAMETERS
 * /gazebo/enable_ros_network: True
 * /hummingbird/agile_autonomy/Q_attitude: 200
 * /hummingbird/agile_autonomy/Q_perception: 0
 * /hummingbird/agile_autonomy/Q_pos_xy: 100
 * /hummingbird/agile_autonomy/Q_pos_z: 100
 * /hummingbird/agile_autonomy/Q_velocity: 10
 * /hummingbird/agile_autonomy/R_pitchroll: 1.0
 * /hummingbird/agile_autonomy/R_thrust: 1.0
 * /hummingbird/agile_autonomy/R_yaw: 1.0
 * /hummingbird/agile_autonomy/camera/baseline: 0.1
 * /hummingbird/agile_autonomy/camera/fov: 91.0
 * /hummingbird/agile_autonomy/camera/height: 480
 * /hummingbird/agile_autonomy/camera/pitch_angle_deg: 0.0
 * /hummingbird/agile_autonomy/camera/width: 640
 * /hummingbird/agile_autonomy/data_dir: /home/seungwooubu...
 * /hummingbird/agile_autonomy/data_generation/save_freq: 15.0
 * /hummingbird/agile_autonomy/general/control_command_delay: 0.02
 * /hummingbird/agile_autonomy/general/num_traj_viz: 1000
 * /hummingbird/agile_autonomy/general/nw_predictions_frame_id: 1
 * /hummingbird/agile_autonomy/general/perform_global_planning: False
 * /hummingbird/agile_autonomy/general/process_every_nth_odometry: 20
 * /hummingbird/agile_autonomy/general/real_world_exp: False
 * /hummingbird/agile_autonomy/general/test_time_max_z: 30.0
 * /hummingbird/agile_autonomy/general/test_time_min_z: 1.0
 * /hummingbird/agile_autonomy/general/test_time_velocity: 7.0
......
(skip)
......
�[0m[ INFO] [1652756385.966290367]: [/hummingbird/agile_autonomy] camera/height = 480�[0m
�[0m[ INFO] [1652756385.966524840]: [/hummingbird/agile_autonomy] camera/baseline = 0.1�[0m
�[0m[ INFO] [1652756385.966882249]: [/hummingbird/agile_autonomy] camera/pitch_angle_deg = 0�[0m
Vertical Fov is 74.7021
Initializing ZMQ connections...
Initializing ZMQ done...
�[0m[ INFO] [1652756385.968852129]: Connecting to unity...�[0m
Ready ? : 0
[INFO] [1652756385.995516, 0.000000]: Loading model XML from ros parameter robot_description
[INFO] [1652756385.997963, 0.000000]: Waiting for service /gazebo/spawn_urdf_model
Ready ? : 0
�[0m[ INFO] [1652756386.242429703]: Finished loading Gazebo ROS API Plugin.�[0m
�[0m[ INFO] [1652756386.243175695]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting...�[0m
Ready ? : 0
Ready ? : 0
�[0m[ INFO] [1652756386.572179509, 0.005000000]: waitForService: Service [/gazebo/set_physics_properties] is now available.�[0m
�[0m[ INFO] [1652756386.578769584, 0.008000000]: Physics dynamic reconfigure ready.�[0m
[INFO] [1652756386.601708, 0.000000]: Calling service /gazebo/spawn_urdf_model
Ready ? : 0
[INFO] [1652756386.792584, 0.091000]: Spawn status: SpawnModel: Successfully spawned entity
Ready ? : 0
Ready ? : 0
Ready ? : 0
�[1m[hummingbird/spawn_hummingbird-3] process has finished cleanly
log file: /home/seungwooubuntu/.ros/log/67b1578e-d58d-11ec-9ee8-ffb260b24679/hummingbird-spawn_hummingbird-3*.log�[0m
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 0
Ready ? : 1
�[0m[ INFO] [1652756393.909530532, 3.647000000]: Unity is ready!�[0m

With the message "Unity is ready!", the flightmare change to other envirenment(like desert) and stop.
After a few second, Flightmare shows this message

I installed Open3D(v0.9.0) and all ros packages following your step by step instruction. and I built every packages
I also installed.

My cuda version is 11.2, nvidia gtx 2060, 32G RAM, intel i7-12700 cpu.
I don't thins this is because of computer spec.

Please check my issue and reply.
Thank you!

[ INFO] [1638364145.246536777, 2.618000000]: Unity is ready! [ERROR] [1638364145.596009985, 2.793000000]: no kernel image is available for execution on the device 209

[ INFO] [1638364145.246536777, 2.618000000]: Unity is ready!
[ERROR] [1638364145.596009985, 2.793000000]: no kernel image is available for execution on the device 209

Has anyone meet the problem? My environment is RTX3080ti, CUDA11.3. The error happened when I use unity. I don't know the reason. How can I resolve the problem?