Giter Club home page Giter Club logo

chrisedel / vehicle-classification Goto Github PK

View Code? Open in Web Editor NEW
3.0 2.0 0.0 75.19 MB

The aim of this project is to use the LiDAR sensor of the new Apple devices in combination with the camera sensors in order to classify cars (based on the make and model) using deep learning. This repository contains the corresponding code consisting of the iOS App, a preprocessing and augmentation script, and our custom neural network.

Python 24.21% CMake 0.32% C++ 20.81% Shell 0.52% Swift 48.59% C 0.69% Objective-C 0.10% Metal 4.76%
deep-learning neural-network machine-learning ios-app preprocessing data-augmentation tensorflow gpu

vehicle-classification's Introduction

Vehicle Classification Using Commercial Off-The-Shelf LiDAR and Camera Sensors (Master's Project 2021)

Project Summary

The aim of this project is to use the LiDAR sensor of the new Apple devices in combination with the camera sensors in order to classify cars (based on the make and model) using deep learning. We used the iPhone 12 Pro for gathering our data (i.e. labelled 3D scans of cars). Overall, we collected about 400 different scans. Since these scans tend to get quite big (in our case about 500 MB each), preprocessing is necessary. Furthermore, in order increase the size of the dataset, we implemented and used several data augmentation methods. The preprocessed and augmented data is then given as input to our custom neural network, the Color3DNet, which can classify the car makes and even the car models. More information on all these topics can be found below, along with specific instructions on how to run this project.

Usage Instructions

Here you can find the instructions on how to run the different parts of the projects.

iOS App

  1. Clone this repository.
  2. Open the ios_app directory with Xcode.
  3. Connect a compatible Apple device to your machine running Xcode. Note: Since the app uses the LiDAR scanner, it only works on Apple devices with a LiDAR scanner. It is not possible to run this app in the simulator or on an Apple device without a LiDAR scanner.
  4. Follow this guide to run the app: Running Your App in the Simulator or on a Device Note: As mentioned before, running the app in the simulator will not work.

Preprocessing and Data Augmentation

Here you can find the install instructions for the preprocessing/data augmentation program called vis_c on Ubuntu.

  1. Run pcl_vis_c_setup.sh to install PCL and vis_c dependencies.

  2. Create a directory build in the same directory as the files main.cpp and CMakeLists.txt.

  3. Go into the build directory with cd build and execute cmake .. then, execute make.

  4. Run export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib.

  5. Execute ./vis_c to check if it is working.

    Optional:

  6. To make vis_c available on the entire machine execute sudo cp vis_c /usr/local/bin.

  7. To make step 4 permanent add LD_LIBRARY_PATH=/usr/local/lib to the file /etc/environment or extend the entry, if LD_LIBRARY_PATH is already in there; you need to log out and back in for the change to take effect.

Neural Network

  1. Clone this repository.
  2. If you have access to a GPU and want to use that GPU for training the neural network, you need to have CUDA installed (tested with CUDA 11.3 (on Linux) and CUDA 11.2 (on Windows 10)). Please refer to the corresponding installation guide:
  1. You need to have Python 3.8 64-bit installed.
  2. Go into the corresponding directory:
    cd color3Dnet
  3. Install all of the necessary requirements. In order to do that, you can use the following command:
    pip3 install -r requirements.txt
  4. Run the script:
    python3 color3Dnet.py [data_path]
  5. For further information, run:
    python3 color3Dnet.py -h

The data for training and testing the neural network can be downloaded here. Note that this data is already preprocessed with:

  • Floor filter: 0.3
  • Distance filter: 0.05
  • Smoothening: 0.01
  • Subsampling: 10 000

Results

Here you can find the summary of the most important results obtained in both theses.

Infrastructure

Different options were used in order to examine the corresponding training time of the neural network. The most important results compare the training time of the neural network when using different CPUs and GPUs, as well as the epochs and time needed in order to reach an accuracy of over 90%.

Average runtimes of an epoch with a batch size of 6.
Average runtimes of an epoch with a batch size of 6.
Number of epochs needed to reach an accuracy of over 90%.
Number of epochs needed to reach an accuracy of over 90%.
Training time needed to reach an accuracy of over 90%.
Training time needed to reach an accuracy of over 90%.

Color3DNet

The following graphs show the train and test accuracies of the Color3DNet during training for different configurations. The train/test split used was 80/20 and the batch size used was 20.

Training with point clouds of size 10 000 classifying car makes.
Training with point clouds of size 10 000 classifying car makes.
Training with point clouds of size 25 000 classifying car makes.
Training with point clouds of size 25 000 classifying car makes.
Training with point clouds of size 10 000 classifying car models.
Training with point clouds of size 10 000 classifying car models.


More information can be found in the corresponding theses.

vehicle-classification's People

Contributors

chrisedel avatar sebastian-landl avatar

Stargazers

Pradel Francis avatar  avatar  avatar

Watchers

 avatar  avatar

vehicle-classification's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.