This is the versatile robot platform. I've gave it to possible usecases: a surveillence robot for home and a object tracker robot.
This is a research project fun to build and fun to explore and we'll take on the following concepts and technologies:
-
programming: Computer vision, Python, Arduino (C++), Java for Android
-
miscelanious: MQTT, Docker, Docker compose, UV4l, linux services
-
electronics: Raspberry pi, Arduino, H-Bridge, DC motors, sensors, soldering, building a robot etc
###############################################################################################
Full tutorial on instructables
A video demo is available on youtube
How does it work
a. The android app shows the uv4l streaming inside a webview. The uv4l process runs on the raspberry pi, captures video input from the camera and streams it. It's an awesome tool with many features
b. Using controls inside the android app lights and engines commands are issued to the MQTT server
c. The python server inside the docker container on the raspberry pi listens to MQTT commands and passes them using serial interface to the arduino board. The arduino board controlls the motors and the lights.
d. The arduino board senses distances in front and back of the robot and sends the data through the serial interface to the python server, the python forwards them to the MQTT and they get picked up by the android interface and shown to the user
Why does an intermediary arduino layer has to exist and not directly the Pi ?
- it's more modular, you can reuse the arduino robot in another project without the PI
- for safety, it's cheaper to replace a 3$ arduino pro mini than to replace a Pi (35$)
- an arduino it's not intrerupted by the operating system like the pi is, so it's more efficient to implement PWM controlls for the mottors, polling the front and back sensors a few times per second
- if an error might occur in the python script the robot might run forever draining the batteries and probably damaging it or catching fire if not supervised, in an arduino sketch a safeguard it's more reliable because it does not depends on an operating system
Extra
The robot will stream the video using UV4l
The android application is located in this repository
Installation
Install Uv4l streamming:
chmod +x uv4l/install.sh
chmod +x uv4l/start.sh
sh ./uv4l/install.sh
A complete tutorial about uv4l is found here: https://www.instructables.com/id/Raspberry-Pi-Video-Streaming/?ALLSTEPS
Clone the project in the home folder:
git clone https://github.com/danionescu0/robot-camera-platform
The folder location it's important because in docker-compose.yml the location is hardcoded as: /home/pi/robot-camera-platform:/root/debug If you need to change the location, please change the value in docker-compose too
Configuration:
- by editing uv4l/start.sh you can configure the following aspects of the video streaming: password, port, framerate, with, height, rotation and some other minor aspects
- edit config.py and replace password with your own password that you've set on the mosquitto server
- edit docker-container/mosquitto/Dockerfile and replace this line
RUN mosquitto_passwd -b /etc/mosquitto/pwfile user your_password
with your own user and password for mosquitto
- optional you can change the baud rate (default 9600) and don't forget to edit that on the arduino-sketch too
Test uv4l installation
a. Start it:
sh ./uv4l/start.sh
b. Test it in the browser at the address: http://your_ip:9090/stream
c. Stop it
sudo pkill uv4l
Install docker and docker-compose
About docker installation: https://www.raspberrypi.org/blog/docker-comes-to-raspberry-pi/ About docker-compose installation: https://www.berthon.eu/2017/getting-docker-compose-on-raspberry-pi-arm-the-easy-way/
Auto starting services on reboot/startup
a. Copy the files from systemctl folder in systemctl folder to /etc/systemd/system/
b. Enable services:
sudo systemctl enable robot-camera.service
sudo systemctl enable robot-camera-video.service
c. Reboot
d. Optional, check status:
sudo systemctl status robot-camera.service
sudo systemctl status robot-camera-video.service
The robot is able to follow
-
objects of a specific color A demo video is available on youtube
-
a known face
First install dependencies using pip, the installation process will be slow. You may need to manually compile opencv to the required version 3.4.2. For this purpose see this tutorial https://www.pyimagesearch.com/2017/09/04/raspbian-stretch-install-opencv-3-python-on-your-raspberry-pi/ and be sure to replace 3.3.0 with 3.4.2
sudo pip3 install -r /home/pi/robot-camera-platform/navigation/requirements.txt
Unit tests
Unit tests are using nose2
In console run with:
nose2
Troubleshooting:
-
If the camera module is not working please check this official Raspberry pi tutorial first
-
I've used opencv to capture the camera image from /dev/video0 so ensure it exists, if not you can try to activate it using:
sudo modprobe bcm2835-v4l2
Configuration [optional]
In navigation/config_navigation.py you'll find:
# minimum and maximum HSV touples for color object detector
# the color below is green
hsv_bounds = (
(24, 86, 6),
(77, 255, 255)
)
# minimum and maximum object size in percent of image width to be considered a valid detection
object_size_threshold = (4, 60)
#image is resized by width before processing to increase performance (speed)
resize_image_by_width = 300
# angle to rotate camera in degreeds
rotate_camera_by = 90
If you want to modify "hsv_bounds" Here is a python helper for HSV range detection
Increasing "resize_image_by_width" will result in more accurate detection but slow processing times.
Running the project:
Running the object tracking script in VNC graphical interface in a terminal: More information of how to install VNC โhere.
python3 object_tracking.py colored-object --show-video
This will enable you to view the video, with a circle drawn over it. The circle means that the object has been detected.
Running the object tracking script with no video output:
python3 object_tracking.py colored-object
The face follower it's in beta state right now, it seems to be quyte slow, it only processes about between one and two frames per second on a Raspberr pi 3.
If you want to give it a try use this:
python3 object_tracking.py specific-face --extra_cfg /path_to_a_picture_containing_a_face --show-video
How does the image detection works ?
** 1. colored object detector **
- First the image is converted to HSV
- Using the function "inRange" HSV ranges are applied to the image (the ranges are defined in the the config)
- Erode and Dilate opencv functions are applied to make the interest zones more clear
- FindContours functon is applied and we select the largest contour (the largest colored object)
- From our selected contour we apply "minEnclosingCircle" and "moments" to get the coordonates of a circle that will best enclose our largest colored object
For the code you ca view "ColoredObjectDetector.py" file
2. face recognition
For the face recognition we're using "face_recognition" library to extract our specific face of interst from the image.
The problem is this library is quite slow on a development board even if we scale the image to 300 x 300 it takes more than a second for a detection. We'll use a new adition to Opencv 3.4 object trackers. These trackers are quite fast but far less accurate than "face_detection" technique
So i'm combining the two detectors to build a compromise between the two. First the "face_recognition" library runs in a different async process, and when a face is found, it communicates to the faster library the face coordonates. The faster method is called "TrackerCSRT_create" from the opencv library and it's able to run syncronious on the pi, processing frame by frame and help guide the robot.
For the code you can start with "SpecificFaceDetector.py" file
The arduino sketch can be found in arduino-sketck folder.
Components
Fritzing schematic:
Checklist:
-
Plexiglass sheet
-
Plastic sheet ( you can also use a plexiglass sheet here )
-
Glue
-
Tyre + DC motor with gearbox + bracket (eBay) 13$
-
Small nuts and bolts
-
2 x any directional wheel tyre
-
Small LED flashlight (it will be transformed into a headlight)
-
Arduino pro mini 328p (eBay) 2 $
-
2 x infrared obstacle sensor
-
PCB
-
NPN tranzistor (to drive the flashlight)
-
L7805CV 5V regulator
-
L298 H-bridge
-
Rezistor 220 Ohms
-
Male usb connector
-
Male micro usb connector
-
Various wires
-
3 v regulator (for communication between arduino and raspberry pi)
-
Male & female PCB connectors
-
On / off switch
-
XT-60 female LiPo connector (eBay) 1.2$
-
2S 1300 mAh LiPo battery with XT-60 connector
-
5v battery pack
-
Raspberry Pi 3
-
Raspberry Pi card
-
Raspberry Pi case
-
Raspberry Pi camera
Pinout:
Led flashlight: D3
Left motor: PWM (D5), EN1, EN2(A4, A5)
Right motor: PWM (D6), EN1, EN2(A3, A2)
Infrared sensors: Front (A0), Back(A1)
Tx: D11, Rx: D10