Giter Club home page Giter Club logo

opm's Introduction

OPM

THIS CODE IS NOW DEPRECATED IN FAVOR OF github.com/mlcommons/GaNDLF

Open Patch Manager: parallel reading/writing of patches.

Installation:

For Usage Only

conda create -p ./venv python=3.8.12 -y
conda activate ./venv
pip install OpenPatchMiner
## alternate
# conda install -c conda-forge openpatchminer

For OPM Development

git clone https://github.com/CBICA/OPM.git
cd OPM/
conda create -p ./venv python=3.8.12 -y
conda activate ./venv
pip install .

Usage

To try an example:

# continue from virtual environment shell
# Extract 10 tissue patches from images/example_slide.tiff
python patch_miner.py -i images/example_slide.tiff -lm images/example_lm.tiff -o example -c opm/config.yml
# Use the output manifest from last run (example/list.csv) to re-extract the same patches for a different replicate
# This is useful if you have multiple slides/label maps and you want corresponding coordinates from each of them
python patch_miner.py -i images/example_slide.tiff -lm images/example_lm.tiff -c opm/config.yml -o example_replicate --input_csv example/list.csv

By default it detects tissue and extracts 1000 random patches from the included .svs file. Play with this number as well as the number of parallel threads in example.py (default patches=1000, default threads=100)

Options

There are also a handful of useful options:

  • READ_TYPE: either 'sequential' or 'random'. If sequential, it repeatedly takes the top-leftmost valid index until quota is met or the slide is saturated. If random, it randomly samples a patch from the valid indices until saturated or the quota is hit.

... and various other parameters such as patch size, thumbnail/valid mask scale, and masking thresholds.

Workflow

OPM follows the following workflow:

Workflow for Open Patch Miner

Project Structure

.
├── images
│   ├── example_lm.tiff
│   ├── example_slide.tiff
│   └── opm_flowchart.png
├── LICENSE.txt
├── opm
│   ├── config.py
│   ├── __init__.py
│   ├── patch_manager.py
│   ├── patch.py
│   └── utils.py
├── patch_miner.py
├── README.md
├── setup.cfg
└── setup.py

Changelog

Dec. 14, 2021:

  • Replaced slide library with tifffile
  • Made barebones OpenSlide mimic with tifffile, named SlideObject
  • Updated requirements, bumped minimum python from 3.6 --> 3.8 (due to zarr requirements)
  • Updated README

Dec. 9, 2021:

  • Changed backend to Pandas.
  • Updated README

Nov. 17, 2020:

  • Added support for mining patches from label map along with the slide.
  • Updated README

Jul. 31, 2020:

  • Changed ALLOW_OVERLAP to OVERLAP_FACTOR. OVERLAP_FACTOR is a float from 0 to 1 that is the portion of patches that are allowed to overlap. If 0, there is no overlap allowed. If 1, they are totally allowed to overlap (except for the origin pixel).

opm's People

Contributors

dependabot[bot] avatar geeks-sid avatar grenkoca avatar joseagraz avatar sarthakpati avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

opm's Issues

Fix output csv writing in patch_manager

If you have several input images and use function mine_patches, the header will be added to csv file for each one. Check if file is empty or header is already there.

Support extraction of patches from different magnification levels

When processing histology images, it would sense to be able to extract X x Y patches in terms of microns rather than pixels. This would require the following:

  • Reading the magnification level(s) in the image
  • Define X and Y from above in terms of M and N pixels
  • Proceed extracting patches in terms of M x N pixels as usual

For example, if a WSI has MPP definition (these are defined as headers "tiffslide.mpp-x" and "tiffslide.mpp-y" from TiffSlide) as A_x along the x-axis and A_y along the y-axis, the patch size (in terms of pixels) that would need to get extracted would be: [M , N] ==[X / A_x , Y / A_y]

In cohesive downstream analyses 2 solutions can be done to ensure that models only process patches in the same shape:

  • Resize extract patches of size [X / A_x , Y / A_y] to a single standard shape (such as [300, 300])
  • Specify a single patch size

Publish on PyPi

Is your feature request related to a problem? Please describe.
Currently, the only way to run OPM is to do a git clone and follow setup steps.

Describe the solution you'd like
It would make it much easier to work with if this is presented in PyPI.

Describe alternatives you've considered
Publish on conda is also a solution.

Additional context
A few examples:

[Warning] From skimage

There is a warning coming from opm.utils:

(/workspaces/OPM/venv) @sarthakpati ➜ /workspaces/OPM (pr/grenkoca/19 ✗) $ python patch_miner.py -h
/workspaces/OPM/opm/utils.py:4: FutureWarning: The `skimage.morphology.selem` module is deprecated and will be removed in scikit-image 1.0 (`skimage.morphology.selem` has been moved to `skimage.morphology.footprints`).
  from skimage.morphology.selem import disk

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.