Giter Club home page Giter Club logo

autonomous_mobile_robot's Introduction

autonomous_mobile_robot

[Paper] [BibTex]

01_results

SemanticV4.mp4

This is the implementation of an navigation system for an autonomous mobile robot using only front-facing RGB Camera. The proposed approach uses semantic segmentation to detect drivable areas in an image and object detection to emphasize objects of interest such as people and cars using yolov5. These detections are then transformed into a Bird's-Eye view semantic map that also contains spatial information about the distance towards the edges of the drivable area and the objects around the robot. Then, a multi-objective cost function is computed from the semantic map and used to generate a safe path for the robot to follow.

The code was tested on both simulation and a real robot (clearpath robotics' jackal).

The simulation is implemented in gazebo and uses dolly and citysim.

Semantic segmentation is strongly based on PSPNet and FCHardNet.

Install

  1. Install ROS 2.

  2. Install Docker following the instructions on the link and nvidia-docker (for gpu support). Semantic segmentation will be run inside docker container, however it could be run on the host with the proper configuration of pytorch.

  3. Clone this repo and its submodules

    git clone --recursive -j8 https://github.com/jdgalviss/autonomous_mobile_robot.git
    cd autonomous_mobile_robot
  4. (Optional if you want tu use PSPNet, FCHardNet are already included) Download Semantic Segmentation pretrained models for PSPNet from the following link: Google Drive. This is the required folder structure for these models:

    autonomous_mobile_robot
    |   ...
    └───pretrained_models
        |   ...
        └───exp
            └───ade20k
            |   |   ...
            |
            └───cityscapes
            |   |   ...
            |
            └───voc2012
                |   ...
    
  5. Build Dockerfile.

    cd semantic_nav
    docker build . -t amr

Run

  1. Run docker container using provided script
    source run_docker.sh

Run Simulation

  1. Inside the docker container, run ros2/gazebo simulation using the provided scripts (The first time, it might take a few minutes for gazebo to load all the models)
    source run.sh

Test Semantic Segmentation and calculate perspective transformation matrix

  1. Run docker container and jupyterlab

    source run_docker.sh
    
    jupyter lab --ip=0.0.0.0 --port=8888 --allow-root --no-browser
  2. Follow the instructions in the Jupytenotebook located inside the container in: /usr/src/app/dev_ws/src/vision/vision/_calculate_perspective_transform.ipynb

Additional notebooks are provided in /usr/src/app/dev_ws/src/vision/vision/ to explain some of the concepts used in this work.

Note: Old Implementation using DWAin citysim: dwa

Citation

If you find this project useful for your research, please consider citing:

@inproceedings{galvis2023autonomous,
  title={An Autonomous Navigation Approach based on Bird’s-Eye View Semantic Maps},
  author={Galvis, Juan and Pediaditis, Dimitrios and Almazrouei, Khawla Saif and Aspragathos, Nikos},
  booktitle={2023 27th International Conference on Methods and Models in Automation and Robotics (MMAR)},
  pages={81--86},
  year={2023},
  organization={IEEE}
}

autonomous_mobile_robot's People

Contributors

dependabot[bot] avatar jdgalviss avatar juan-galvis avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.