LeDoc

1. Introduction

The basic use case of this app is in the field of medical image analysis. With the help of various advances in the field of deep learning we wish to assist in the medical field.
Right now we are able to tackle some of the fatal diseases like Skin Cancer, Brain Tumor and Brain Cancer. Through our app, we wish to assist doctors in the preliminary stages of treatment of these diseases. For this, we use clustering for segmentation purposes, and for predicting the severity of the disease we use various computer vision techniques.

2. Motivation

The name of this project has been inspired from the famous Image Classification model 'LeNet' by Yann LeCun.The major motivation to take up this project came from Enlitic, a startup by Jeremy Howard. Their basic aim is to assist doctors by using latest advancements in the field of deep learning and hence easening out the burden faced by doctors due to increasing number of patients. For more information you can see here.

3. Methodology

A. Skin Cancer

The whole dataset for this was obtained from ISIC challenge 2018.
For this task, we ask the user to input a 224*224 skin lesion image. For the segmentation purposes we used simple K-Means clustering techniques(k=2). The classification of the skin lesions was on these 7 categories: Melanoma, Melanocytic Nevus, Basal Cell Carcinoma, Acinic Keratosis, Benign Keratosis, Dermatofibroma, Vascular Lesion. More information regarding these types can be collected from here
The classification into these 7 categories was done by training a ConvNet. For training we used a pretrained VGG_16 network and used transfer learning by removing all the fully connected layers and then attaching our own 5 fc layers and then training accordingly on the dataset. The whole ConvNet model is made using Pytorch.

B. Brain Tumor

The whole dataset for this was obtained from ISLES challenge 2015.
For the processing of this task, the user is made to upload a zip file cotaining two kinds of MRI scans, namely t1c and flair. These two different kinds of MRI sequences are basically essential to get different kind of information about the patient's brain state. For the segmentation purpose we used K-Means clustering techniques(k=3). We apply the clustering process on both the types of sequences separately and then combine them by using color coding.

• The red part represents the major tumor area
• The green part represents the area that is the most affected by the tumor region and hence most likely to be the next affected

Apart from this, we also show the respective percentage area of these two regions w.r.t the brain area so as to help doctors in better analysis of the affected region. We also further classify the severity of the disease into HGG or LGG which is basically high grade or low grade tumor, done using simple thresholding by analyzing the area affected

C. Brain Stroke

The whole dataset for this was obtained from BraTS challenge 2016.
For the processing of this task, the user is made to upload a zip file containing two kinds of MRI scans, namely dwi and flair. These two different kinds of MRI sequences are basically essential to get different kind of information about the patient's brain state. For the segmentation purpose we used K-Means clustering techniques(k=3). We apply the clustering process on both the types of sequences separately and the combine them by using color codind.

• The red part shows the transient stroke area
• The green part shows the main stroke area

Apart from this, we also show the respective percentage area of these two regions w.r.t the brain area so as to help doctors in better analysis of the affected region. This can certainly help doctors determine whether the condition of the patient is worsening or improving

4. Testing

Firslty download the pretrained model for classification from here. Now place this file in the directory le_doc/skin_cancer

Clone both the repositories containing the backend and frontend for the app.
Opening a terminal session from the home directory.

cd le_doc_fronted

npm start

Opening another terminal session from the home directory

cd le_doc

source .env/bin/activate

pip install -r requirements.txt

python manage.py runserver

Also to test the results you can download the refined dataset with all the preprocessing already done: Skin Cancer, Brain Tumor, Brain Stroke.

5. Results

A. Skin Cancer

Through our techniques for segmentation we were able to achieve a Jaccard index of about 0.78, averaged on a dataset of 100 images. Also the top-3 accuracy of our classification model is of about 85%.

B. Brain Tumor

Through our techniques for segmentation we were able to achieve a Jaccard index of about 0.75, averaged on a dataset of 100 zip files.

C. Brain Stroke

Through our techniques for segmentation we were able to achieve a Jaccard index of about 0.75, averaged on a dataset of 100 zip files.

6. Further development of the project

1. To further extend the project, we will also be incorporating lung cancer detection and segmentation and some other diseases which require preliminary stages of treatment so as to assist the doctors and hence lessen up their work load.
2. We will also be further imporving the accuracy of the classification model, by trying out better models and even increasing our dataset size.
3. Also we will also be implementing more advanced segmentation techniques like Mask RCNN so as to improve the results.
4. During the development of the project, we also tested genetic algorithms and swarm optimization, which did improve the segmentation results for all three cases but took a heavy toll on the processing time and hence were not incorporated in the final version of the project. So we would also be further investigating on the feasibility of their incoporation in the project by optimizing the algorithm.

Contributors