This project aims to use machine learning algorithms to detect seizure from ECG data, in a MLOps environment. Here is an overview of the automated ML pipeline :

This pipeline runs inside a dockerised environment, represented below :

The pipeline requires these packages to run :

• Python >= 3.7
• pandas == 1.1.5
• numpy == 1.19.5
• pyEDFlib == 0.1.22
• click == 8.0.1
• py-ecg-detectors == 1.0.2
• wfdb == 3.4.0
• biosppy == 0.7.3
• hrv-analysis == 1.0.4
• ecg-qc == 1.0b5
• great-expectations == 0.13.25
• airflow-provider-great-expectations == 0.0.7
• psycopg2-binary == 2.8.6

You can install them in a virtual environment on your machine via the command :

    $ pip install -r requirements.txt

You need to have docker and docker-compose installed on your machine to set the environment up.

After cloning this repository, replace the value of the environment variable DATA_PATH in the env.sh file with the absolute path of the data you are working with.

You can now run these commands :

    $ source setup_env.sh
    $ docker-compose build
    $ docker-compose up airflow-init
    $ docker-compose up -d

Warning: Here are the default ports used by the different services. If one of them is already in use on your machine, change the value of the corresponding environment variables in the env.sh file before running the commands above.

Once the services are up, you can interact with their UI :

• Airflow : http://localhost:8080
• Grafana : http://localhost:3000
• MLFlow : http://localhost:5000
• Great expectations : http://localhost:8082
• Flower : http://localhost:5555

When required, usernames and passwords are admin.

First export the python path to access the scripts :

    $ export PYTHONPATH=$(pwd)

You can now execute each Python script separately by running :

    $ python3 <path-to-Python-script> [OPTIONS]

The required options are shown by running the --help option.

You can stop all services by running :

    $ docker-compose down 

If you add the -v option, all services' persistent data will be erased.

This project is licensed under the GNU GENERAL PUBLIC License.