1 model on 1 fold

I ran this project, but I got the following error and tried to install OpenSlide, but apparently this package does not install properly
Given this error, where is the problem with installing an open slide?
Thank you for your help

File "D:\python\DigitalHistoPath-master\code_cm17\trainer\trainer.py", line 24, in
from dataloader.training_data_loader import DataGeneratorCoordFly

File "D:\python\DigitalHistoPath-master\code_cm17\trainer/..\dataloader\training_data_loader.py", line 14, in
import openslide

File "C:\Users\micro_lab\anaconda3\envs\tf\lib\site-packages\openslide_init_.py", line 35, in
from openslide import lowlevel

File "C:\Users\micro_lab\anaconda3\envs\tf\lib\site-packages\openslide\lowlevel.py", line 43, in
_lib = cdll.LoadLibrary('libopenslide-0.dll')

File "C:\Users\micro_lab\anaconda3\envs\tf\lib\ctypes_init_.py", line 434, in LoadLibrary
return self._dlltype(name)

File "C:\Users\micro_lab\anaconda3\envs\tf\lib\ctypes_init.py", line 356, in _init
self._handle = _dlopen(self._name, mode)

OSError: [WinError 126] The specified module could not be found

It is viable to be used for the software

Hi,
I would like to thank you for sharing this interesting work. However, I am trying to apply this experiment on the same data that you use, it shows me that tumor type and mode required. Can you kindly tell me how can I fix this issue please?. Your reply and help would be highly appreciated.

Kind regards,

Experiments to run

• 9-way ensemble
• Color-jitter vs Stain normalisation
• Interpretability

Originally posted by @haranrk in #7 (comment)

Dears, thank you for your continuous support. When I was exploring the matching levels between the slides and corresponding masks, I noticed that the number of levels in the ground masks is always 1, shouldn't it contain more levels? especially that we convert the tiff files to pyramidal prior to starting the experiments?

So can you help how can I found the ground truth mask of level 1 and level 2 of WSI?

your help is really apperciated.

First of all thank you for sharing such an excellent project. I am planning to run your scripts on the PAIP dataset 2019, I uncompressed the training files then I started from your "points_extractor.py" file to prepare the patches but I faced some difficulty in reading the binary masks (tif files), their path returns "none" while the path exists and it give the following error (OpenSlideUnsupportedFormatError) . Any suggestion how to solve this?

Also your help in stating the order of the files that must be executed to reach the training phase is really appreciated.

Hi,

Thank you for the wonderful repository. Do you plan to release the weights of the models?
It will be helpful to start with reproducing the results and extend beyond.

Thanks,
Shubham

Kori

• Graphic abstract
• Abstract

Haran

• Introduction section 0
• PAIP comparison with other participants
• Put larger images for PAIP
• Expand viable tumor burden
• Contributions

Mahendra

• Camelyon classification
• Conclusion

Undesignated

• Related work part 1: Generalr histopathology work(unsupervised supervised, weakly supervised)
• Related work part 2: Supervised Lagos

When I am converting images to pyramid format, I need to use the lock.json file. What is the content of this file

1. Machine Learning Methods for Histopathological Image Analysis - Review paper covering various applications and problems surrounding histopathological analysis
2. Advanced Deep Convolutional Neural Network Approaches for Digital Pathology Image Analysis: a comprehensive evaluation with different use cases
   https://arxiv.org/pdf/1904.09075.pdf
3. High-throughput adaptive sampling for whole-slide histopathology image analysis (HASHI) via convolutional neural networks: Application to invasive breast cancer detection
   https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0196828
4. Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features
   https://www.microsoft.com/en-us/research/uploads/prod/2018/06/2017SCIBMClarge-scale-tissue-histopathology-image-classification-segmentation-and-visualization-via-deep-convolutional-activation-features.pdf
5. Imitating Pathologist Based Assessment With Interpretable and Context Based Neural Network Modeling of Histology Images
   https://journals.sagepub.com/doi/full/10.1177/1178222618807481
6. Segmenting Potentially Cancerous Areas in Prostate Biopsies using Semi-Automatically Annotated Data
   https://arxiv.org/pdf/1904.06969.pdf
   http://proceedings.mlr.press/v102/burlutskiy19a.html
7. Deep Learning Based Analysis of Histopathological Images of Breast Cancer
   https://www.frontiersin.org/articles/10.3389/fgene.2019.00080/full
8. Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases
   http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2016;volume=7;issue=1;spage=29;epage=29;aulast=Janowczyk;t=6
9. Classification of colorectal cancer based on correlation of clinical, morphological and molecular features
   http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.476.4786&rep=rep1&type=pdf

https://github.com/openslide/openslide-python/blob/master/examples/deepzoom/deepzoom_server.py

Dear Sir, I have a quick question, I calculated the Jaccard index between the original ground truth mask (I read the mask image at level 0 using ReadWholeSlideImage function) and the generated mask after thresholding (prd_im_fll_dict) but I got low values, therefore I reinvestigated the min and max values of each of them and I am surprised that the max value in the original ground truth is 3 instead of 1. Can you explain why?

Originally posted by @codeskings in #19 (comment)

Hello, I need your help and experience, I would like to ask about something strange occurred when I tried a variant of Unet on your same framework, as the epochs increased the performance in terms of Jaccard index become worse (not improving as it supposed). please see below some results:

epoch 3
Jaccard for Training_phase_1_001 = 0.278841
Jaccard for Training_phase_1_002 = 0.060954
epoch 2
Jaccard for Training_phase_1_001 = 0.286575
Jaccard for Training_phase_1_002 = 0.084818
epoch 1
Jaccard for Training_phase_1_001 = 0.350196
Jaccard for Training_phase_1_002 = 0.170044

Is my learning rate low? or does I miss something somewhere?

one more thing, is it normal that training one epoch take 3.5 hours to complete using GPU?

Thank you for your help and code as well.

After converting the binary mask images into the pyramidal format, and start running the script of "points_extractor", I am facing a memory error when using deep copy. Before throwing this error, it only save the coordinates of the patches extracted from the first training svs file. Can you please advise me if the problem is with my machine or not? I am using 32GB RAM. Thank you in advance.

Originally posted by @heba9004 in #16 (comment)

Paper skeleton

Abstract

• Model performance on datasets (Liver, Colon, Prostrate(maybe))
• Main content of the paper

Introduction

1. Motivation
   1. Include why
   2. Pathologists aren’t accurate and slow
2. Previous work
   1. Papers (at least 10)
3. Our contribution in brief

Methodology

Data

• Explain all 3 datasets

Sampling strategies

Currently, doing equal tumor and normal tissue sampling

• Some paper - prostrate cancer - deepmind/googleai - use a different ratio investigate

• Sampling randomly? Or visit some papers on sampling,

Data pre-processing

Stain normalisation

• Augmentations

Network architectures

• Explain 3 architectures in detail

Training strategies

Inference strategies

Interpretability

• Data flow analysis - gradcam
• Uncertainty analysis -

Results & Discussion

Implementation details

Conclusion

Future work

Acknowledgements

The documentation advises to use vips for production application. To be investigated.

Thanks for sharing this complete detailed project with us. However, there are many scripts in it that I got confused with the order of their execution, can you please confirm my order below:

1. convert the ground truth images to pyramidal format.
2. run (cross-validation-splitter) twice to create the different folds; once for the viable tumor and once for the whole tumor.
3. run (point_extractor) four times; train/whole, train/viable, valid/whole, valid/viable
4. run (patch_coords_cv_splitter) twice for the whole and viable tumors.
5. run (trainer) for the 3 deep models independently for each fold.

Questions -

• after that step, I got lost, what should be the next script to run? (predict) from the (inference)? but it require a (infer_*) directory that I couldn't find where to create it?
• also, I have a question regarding the 2000 normal patch that we extract from the image; they are extracted from level 0 only?
• what is the difference between img_sampling_level and mask_sampling_level ?
• why they aren't zeros as in the image level?
• Also, in the comments, you stated that (Img downsamples are pows of 4, mask downsamples are pows of 2), I read your paper fully but I couldn't find an explanation for this?

Sorry for the many questions but I want to fully understand each step.
Thank you very much