Temple University Computer Science Capstone Project team:

Sean Digirolamo
Sean Reddington
Michael Savitski
Jose Silva
Kwamina Thompson

BinBot is an autonomous robot designed to identify and collect littered waste. The robot is built from a Raspberry Pi robot kit, which includes a camera, proximity sensor, motorized treads, and a mechanical arm. This robot is powered by an embedded Raspberry Pi 3 B+. BinBot’s on-board camera captures live-feed images which are sent to a processing server. To outsource the heavy data processing from the on-site robot; this Linux server will process the images sent by BinBot using data representations created from a trained machine learning model. Using TensorFlow and OpenCV libraries, the machine learning model will identify any pieces of waste in the image. The server will then compute properties about the waste, including the location from the robot, and generate instructions for the robot to navigate to the waste and collect it. The application will also display of the images from the processing the Pi camera, which include bounding boxes surrounding any waste identified via the machine learning model. BinBot also features an Android mobile app which provides the ability to start and stop BinBot’s waste collection process.

• Search for, identify, navigate to, and pick up waste
• LED lights on side of robot indicate current state
  • Blue: Capturing picture, sending and receiving to server
  • Red: Executing movement instructions from server
  • Yellow: Executing patrol instructions
  • Green: Retrieving waste
• Mobile app to toggle operation of robot
• Server displays individual pictures received from BinBot

• Drops waste frequently because claw is weak and too small
• Occasionally mistakes objects for waste when they are not
• Pi camera hangs when trying to manually adjust gain

This is the code for that actual robot that connects to the server, navigates to waste, and retrieves it. It runs directly on a Raspbian OS on a Raspberry PI 3 B+. It is written in Python.

The BinBot Robot does not have build instructions because it is written in Python, which is interpreted rather than compiled. But, ensure that it is downloaded onto the raspberry Pi/Robot rather than the server or anywhere else.

To run, execute main.py with the IP address of the server configured in the file. By default, the connection is on port 7001:

$ sudo python3 main.py

This is the server which the BinBot Robot connects to. Images are sent from the BinBot Robot to the server, and the server returns instructions to BinBot on how it should proceed in order to locate and retrieve waste. It runs on Windows and is written in Java.

The easiest way to build and run is through the IntelliJ Idea IDE, set to use JDK 11. Create a new artifact under Project Structure -> Artifacts. Click the plus button in the top left and create a new JAR from module with dependencies. Select DemoClient as the main class and hit okay. From the main menu, build the project with Build -> Build artifacts. The built jar can be found in <project_folder>/out/artifacts/BinBot_Processing_jar

Required libraries: com.google.code.gson 2.8.6, opencv 411, org.apache.commons-lang3 3.3.2, org.json 20190722, org.junit.jupiter 5.0.2, org.tensorflow 1.12.0-rs2

NOTE: Due to the large file size, the .zip of the processing server does not include the two model folders. In Please download the folders separately:

001
faster_rcnn_inception_v2_coco_2018_01_28

Build using Javac compiler, JDK version 11.

Ensure that OpenCV is installed on the system and that the appropriate DLL for your system is in your PATH environment variable.

Run the jar with java 11, as such:

<path to java11> -jar <path to jar>

Runs on Android

Build using Android studio compiler, SDK version 29 , minSDKversion 23, targetSDKversion 29

IP address needs to be identical to all the host servers IP, to allow communication.