Giter Club home page Giter Club logo

analog_gauge_reader's Introduction

analog_gauge_reader

This is the code for the paper Under Pressure: Learning-Based Analog Gauge Reading In The Wild by Maurits Reitsma, Julian Keller, Kenneth Blomqvist and Roland Siegwart.

Setup installation (Poetry, automatic)

Install Poetry

curl -sSL https://install.python-poetry.org | python3 -

Install the project dependencies

poetry install

Enter Poetry shell

poetry shell

Setup installation (manual)

To setup the conda environment to run all scripts follow the following instruction:

Install miniconda

mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh

Activate conda environment

conda create --name gauge_reader python=3.8 -y
conda activate gauge_reader

install pytorch

We use torch version 2.0.0.

conda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 -c pytorch -c nvidia

install mmocr

Refer to this page for installation https://mmocr.readthedocs.io/en/dev-1.x/get_started/install.html We use the version dev-1.x

pip install -U openmim
mim install mmengine==0.7.2
mim install mmcv==2.0.0
mim install mmdet==3.0.0
mim install mmocr==1.0.0

We use the following versions: mmocr 1.0.0, mmdet 3.0.0, mmcv 2.0.0, mmengine 0.7.2. If for some reason the installation fails refer to open-mmlab/mmcv#2938. We found that it is essential that we have Pytorch version 2.0.0

install yolov8

We use ultralytics version 8.0.66

pip install ultralytics

install sklearn

We use scikit-learn version 1.2.2

pip install -U scikit-learn

Run pipeline script

The pipeline script can be run with the following command:

python pipeline.py --detection_model path/to/detection_model --segmentation_model /path/to/segmentation_model --key_point_model path/to/key_point_model --base_path path/to/results --input path/to/test_image_folder/images --debug --eval

For the input you can either choose an entire folder of images or a single image. Both times the result will be saved to a new run folder created in the base_path folder. For each image in the input folder a separate folder will be created.

In each such folder the reading is stored inside the result.json file. If there is no such reading, one of the pipeline stages failed before a reading could be computed. Best check the log file which is saved inside the run folder, to see where the error came up. There will also be a error.json file saved to the image folder, which computes some metrics to check without any labels how good our estimate is.

Additionally if the debug flag is set then the plots of all pipeline stages will be added to this folder. If the eval flag is set then there will also be a result_full.json file created. This file contains the data of the individual stages of the pipeline, which is used when evaluating in the script full_evaluation.py.

Run experiments

I prepared two scripts to automatically run the pipeline and evaluations on multiple folders with one command. This allows us to easily conduct experiments for images that we group by their characteristics in different folders.

If they want to be used, make sure to modify the paths inside the scripts, to match with your data.

analog_gauge_reader's People

Contributors

jk-ethz avatar kekeblom avatar mauritsreitsma avatar victorreijgwart avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

analog_gauge_reader's Issues

Can not get it to run

I have Anaconda installed on my windows machine, so I only followed the instructions below Activate conda environment. So create the new environment, activate it and install the dependencies via copy&paste of the given commands. I then copied the github repository into a local folder C:/Users/me/.spyder-py3/analog_gauge_reader-main to get the actual script etc. (this is missing in the installation procedure?).

When I now try to run the pipeline script, I get this error (line breaks and spaces added for readability)

(gauge_reader) C:\Users\me\.spyder-py3\analog_gauge_reader-main>python pipeline.py 
--detection_model        C:/Users/me/.spyder-py3/analog_gauge_reader-main/gauge_detection/detection_inference.py 
--segmentation_model     C:/Users/me/.spyder-py3/analog_gauge_reader-main/segmentation/segmenation_inference.py 
--key_point_model        C:/Users/me/.spyder-py3/analog_gauge_reader-main/key_point_detection/key_point_inference.py 
--base_path              C:/Users/me/.spyder-py3/analog_gauge_reader-main/results 
--input                  C:/Users/me/.spyder-py3/analog_gauge_reader-main/input 
--debug --eval
-------------------
Gauge Detection
WARNING ⚠️ Unable to automatically guess model task, assuming 'task=detect'. Explicitly define task for your model, i.e. 'task=detect', 'segment', 'classify','pose' or 'obb'.
Unexpected err=TypeError("model='C:\\Users\\me\\.spyder-py3\\analog_gauge_reader-main\\gauge_detection\\detection_inference.py' is not a supported model format. See https://docs.ultralytics.com/modes/predict for help.\n\n                   Format     Argument           Suffix    CPU    GPU\n0                 PyTorch            -              .pt   True   True\n1             TorchScript  torchscript     .torchscript   True   True\n2                    ONNX         onnx            .onnx   True   True\n3                OpenVINO     openvino  _openvino_model   True  False\n4                TensorRT       engine          .engine  False   True\n5                  CoreML       coreml       .mlpackage   True  False\n6   TensorFlow SavedModel  saved_model     _saved_model   True   True\n7     TensorFlow GraphDef           pb              .pb   True   True\n8         TensorFlow Lite       tflite          .tflite   True  False\n9     TensorFlow Edge TPU      edgetpu  _edgetpu.tflite   True  False\n10          TensorFlow.js         tfjs       _web_model   True  False\n11           PaddlePaddle       paddle    _paddle_model   True   True\n12                   NCNN         ncnn      _ncnn_model   True   True"), type(err)=<class 'TypeError'>

In the "results" folder, I can find the original picture, a JSON file with this content:

{
    "input": "C:/Users/me/.spyder-py3/analog_gauge_reader-main/input",
    "detection_model": "C:/Users/me/.spyder-py3/analog_gauge_reader-main/gauge_detection/detection_inference.py",
    "key_point_model": "C:/Users/me/.spyder-py3/analog_gauge_reader-main/key_point_detection/key_point_inference.py",
    "segmentation_model": "C:/Users/me/.spyder-py3/analog_gauge_reader-main/segmentation/segmenation_inference.py",
    "base_path": "C:/Users/me/.spyder-py3/analog_gauge_reader-main/results",
    "debug": true,
    "eval": true
}

And run.log with this content:

root - INFO - Start processing image at path C:/Users/me/.spyder-py3/analog_gauge_reader-main/input\IMG_20240606_130929_028.jpg
root - INFO - Start Gauge Detection
root - ERROR - Unexpected err=TypeError("model='C:\\Users\\me\\.spyder-py3\\analog_gauge_reader-main\\gauge_detection\\detection_inference.py' is not a supported model format. See https://docs.ultralytics.com/modes/predict for help.\n\n                   Format     Argument           Suffix    CPU    GPU\n0                 PyTorch            -              .pt   True   True\n1             TorchScript  torchscript     .torchscript   True   True\n2                    ONNX         onnx            .onnx   True   True\n3                OpenVINO     openvino  _openvino_model   True  False\n4                TensorRT       engine          .engine  False   True\n5                  CoreML       coreml       .mlpackage   True  False\n6   TensorFlow SavedModel  saved_model     _saved_model   True   True\n7     TensorFlow GraphDef           pb              .pb   True   True\n8         TensorFlow Lite       tflite          .tflite   True  False\n9     TensorFlow Edge TPU      edgetpu  _edgetpu.tflite   True  False\n10          TensorFlow.js         tfjs       _web_model   True  False\n11           PaddlePaddle       paddle    _paddle_model   True   True\n12                   NCNN         ncnn      _ncnn_model   True   True"), type(err)=<class 'TypeError'>

I also tried to use the gauge_detection_model etc. that are in the "models" folder like this:

python pipeline.py 
--detection_model        C:/Users/me/.spyder-py3/analog_gauge_reader-main/models/gauge_detection_model.pt 
--segmentation_model     C:/Users/me/.spyder-py3/analog_gauge_reader-main/models/segmentation_model.pt 
--key_point_model        C:/Users/me/.spyder-py3/analog_gauge_reader-main/models/key_point_model.pt 
--base_path              C:/Users/me/.spyder-py3/analog_gauge_reader-main/results 
--input                  C:/Users/me/.spyder-py3/analog_gauge_reader-main/input 
--debug --eval

And get this error:

-------------------
Gauge Detection
Unexpected err=UnpicklingError("invalid load key, 'v'."), type(err)=<class '_pickle.UnpicklingError'>

Any ideas what I am doing wrong?

Request for Gauge Dataset

Hi Team,

I am trying to implement your project to detect multiple pressure gauges. I want to request for access to your dataset. So I can add images of my gauge images and re-train the model further.
Also, I would like to know how to train the network to detect certain Data using OCR. The current model does not pick certain OCR text like units of the gauge. I want to re-train the model to get better accuracy.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.