lee-gihun / mediar Goto Github PK

Thank you for the tool!

Performance is very impressive on my data.

Have you thought about creating a MEDIAR package that people can use to segment their images?

I think this would make it easier to include this into segmentation pipelines without putting sys.path.insert everywhere

Hello, thank you for sharing the code. I used two images from the image/examples folder and ran the predict.py code, but the segmentation results I obtained are two completely black images. I replaced the images with my own cell images, but still obtained completely black images as results. Could you please advise on how to troubleshoot this issue?

Hi, there are no codes given to process the public datasets, and what are the discarded data from each of the public dataset.

Hi, I am tring to intall MEDIAR into Anaconda3 using:

conda create --name Mediar python=3.8
conda activate Mediar
(base) C:\Users\rmd>conda activate Mediar
 (Mediar) C:\Users\rmd>python -m pip install MEDIAR

ERROR: Could not find a version that satisfies the requirement MEDIAR (from vers
ions: none)
ERROR: No matching distribution found for MEDIAR

How can this be solved?
Thank you.

Hi
What does "classes" parameter in configs do? Is it about label classes like cell, background, and ?!
classes in config

Hello team, thanks for this repo.
I am wondering if the "finetuned" model can be further finetuned by training it on a custom dataset.

Thanks!
Ajinkya

Hi, I test the colab demo with no peroblem, however, when I run it as script, error take place:
ModuleNotFoundError: No module named 'train_tools'
I noticed that train_tools is localized inside Anaconda3\envs\Mediar\Lib\site-packages\MEDIAR, thus that's why the module is not detected.

To solve the problem,I add three lines in the beginning of demo script,:

import os
print("============ getcwd 1=======================")
print(os.getcwd())
aMEDIARroot = "E:\\BackupVIP\\RMD\\GreenSoft\\Anaconda3\\envs\Mediar\\Lib\\site-packages\\MEDIAR"
os.chdir(aMEDIARroot)
print("============ getcwd 2=======================")
print(os.getcwd())

However, the error still persist.

I use a batch file to launch the Anaconda3 and Medira env:

call E:\BackupVIP\RMD\GreenSoft\Anaconda3\Scripts\activate.bat E:\BackupVIP\RMD\GreenSoft\Anaconda3
call conda activate Mediar
call python mediar_demo.py
echo "good"
pause

I also tried add 2 lines the beginning, but it still not work.

E:
cd E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\Lib\site-packages\MEDIAR

I confirmed that both os.getcwd() print the correct path:
============ getcwd 1=======================
E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\Lib\site-packages\MEDIAR
============ getcwd 2=======================
E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\Lib\site-packages\MEDIAR

but the same error still appears.

Finally, I found that it only works when I place the bat file and mediar_demo.py together inside folder "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\Lib\site-packages\MEDIAR"

Is there a way to specify the working directory to solve the mising "train_tools" problem?
Thanks

Congratulations for winning the challenge!
Amazing work!

I tried out MEDIAR in the inteference Colab, but it didnt really work:

Input:

Output:

I converted my fluorescence image to grayscale and set it to uint8 (2048, 2048) (0, 255).
Do I need change my image or is the performanced caused by the divergence of my image from MEDIAR's training data?

Thank you!

First and foremost, I wish to express my profound respect for the research presented in your paper titled "[MEDIAR: Harmony of Data-Centric and Model-Centric for Multi-Modality Microscopy]". The approach you introduced about using model embeddings for KNN classification has greatly intrigued me.

As I attempt to understand and apply this method, I have encountered some challenges in its practical implementation. To gain a deeper understanding and practical experience of this concept, I am reaching out to inquire if it would be possible for you to share the related code implementation, be it a GitHub link or a zip file, either would be immensely helpful to me.

I assure you that, if granted access to your code, it will strictly be used for academic research purposes, adhering to all relevant research ethics and copyright policies. Your assistance will be invaluable in the completion of my study.

Thank you for considering my request. I look forward to your response and once again, thank you for your valuable contribution to the academic community.

Hi! Looking forward to trying your software out on some data!

I am trying to install your codebase and noticed that the package version for scikit-image is 0.0, which seems incorrect? It's also called skimage in the requirements folder, which is the incorrect name of the package (should be scikit-image).

Thanks

Thank you for uploading this wonderful segmentation code. Can this be run on large tif or jp2 WSI images? How can we run this for large WSI files?

Hi, I have managed to install the MEDIAR and run the colab demo script in-line code line-by-line.
In the line
_ = predictor.conduct_prediction()
it shows the error no NVIDIA driver.
My laptop has no GPU, could MEDIAR run without GPU?
In the code below, does map_location="cpu" already specify using CPU? Or is there other ways to specify use CPU instead of GPU?
Thanks.

weights1 = torch.load(model_path1, map_location="cpu")
weights2 = torch.load(model_path2, map_location="cpu")
model_args = {
    "classes": 3,
    "decoder_channels": [1024, 512, 256, 128, 64],
    "decoder_pab_channels": 256,
    "encoder_name": 'mit_b5',
    "in_channels": 3
}
model1 = MEDIARFormer(**model_args)
model1.load_state_dict(weights1, strict=False)
model2 = MEDIARFormer(**model_args)
model2.load_state_dict(weights2, strict=False)

predictor = Predictor(model1, "cuda:0", input_path, output_path, algo_params={"use_tta": False})
_ = predictor.conduct_prediction()

Traceback (most recent call last):
File "", line 1, in
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\lib\site-packages\torch
\autograd\grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\Lib\site-packages\MEDIA
R\core\BasePredictor.py", line 40, in conduct_prediction
self.model.to(self.device)
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\lib\site-packages\torch
\nn\modules\module.py", line 927, in to
return self._apply(convert)
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\lib\site-packages\torch
\nn\modules\module.py", line 579, in _apply
module._apply(fn)
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\lib\site-packages\torch
\nn\modules\module.py", line 579, in _apply
module._apply(fn)
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\lib\site-packages\torch
\nn\modules\module.py", line 579, in _apply
module._apply(fn)
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\lib\site-packages\torch
\nn\modules\module.py", line 602, in apply
param_applied = fn(param)
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\lib\site-packages\torch
\nn\modules\module.py", line 925, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else No
ne, non_blocking)
File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\lib\site-packages\torch
\cuda_init.py", line 217, in _lazy_init
torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have
an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.a
spx

Hi, I am tring to run a test in Anaconda3 using the command:

(Mediar) E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\Lib\site-packages\MEDI
AR>python predict.py --config_path=E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Med
iar\Lib\site-packages\MEDIAR\config\pred\pred_mediar.json

I download several models and specify the "finetuned\from_phase1.pth" in json file MEDIAR\config\pred\pred_mediar.json as below:
{
"pred_setups":{
"name": "medair",
"input_path":"E:\temp2\Shieh\nucleusInput",
"output_path": "E:\temp2\Shieh\nucleusOutput",
"make_submission": true,
"model_path": "E:\Software\CellSegmentation\MEDIAR\finetuned\from_phase1.pth",
"device": "cuda:0",
"model":{
"name": "mediar-former",
"params": {},
"pretrained":{
"enabled": false
}
}
}
}
The E:\temp2\Shieh\nucleusInput is a folder having several tif images of DAPI-stained nucleus to segment.

but error appears below.
How to solve this problem?
Thanks .

================================================== Configuration ===============
===================================
{'pred_setups': {'device': 'cuda:0',
                 'input_path': 'E:\\temp2\\Shieh\\nucleusInput',
                 'make_submission': True,
                 'model': {'name': 'mediar-former',
                           'params': {},
                           'pretrained': {'enabled': False}},
                 'model_path': 'E:\\Software\\CellSegmentation\\MEDIAR\\finetune
d\\from_phase1.pth',
                 'name': 'medair',
                 'output_path': 'E:\\temp2\\Shieh\\nucleusOutput'}}
================================================================================
========================================
Traceback (most recent call last):
  File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\Lib\site-packages\MEDIA
R\predict.py", line 74, in <module>
    main(opt)
  File "E:\BackupVIP\RMD\GreenSoft\Anaconda3\envs\Mediar\Lib\site-packages\MEDIA
R\predict.py", line 40, in main
    predictor = PREDICTOR[args.pred_setups.name](
KeyError: 'medair'

Thank you for this amazing model.

We would like to use MEDIAR in our lab for analysis of our mIF images. We have quite specific cell types with very diverse morphologies, so we would like to fine-tune the model. After some minor modifications to the code base (also for the later ease of incorporation into the pipeline), I was able to run the data on our custom dataset. However, there is something weird happening as the training progresses: the performance degrades until model starts to output no segmentation masks at all. Below are examples of the same image from the validation test taken at epochs 1/3/9. During validation step, F1 values are 0, and no segmentation results found usually after epoch 5.

And here is slightly modified config file used for the training:

{
    "data": {
        "root": "./",
        "mapping_file": "./config/mapping_labeled.json",
        "amplified": false,
        "batch_size": 4,
        "valid_portion": 0.1
    },
    "train_setups": {
        "model": {
            "name": "mediar-former",
            "params": {
            },
            "pretrained": {
                "enabled": true,
                "weights": "./weights/finetuned/from_phase2.pth",
                "strict": false
            }
        },
        "trainer": {
            "name": "mediar",
            "params": {            
                "num_epochs": 60,
                "valid_frequency": 1,
                "device": "coda:0",
                "amp": false
            }
        },
        "optimizer": {
            "name": "adamw",
            "ft_rate": 1.0,
            "params": {
                "lr": 5e-5
            }
        },
        "scheduler": {
            "enabled": true,
            "name": "cosine",
            "params": {
                "T_max": 60,
                "eta_min": 1e-6
            }
        },
        "seed": 19940817
    },
    "pred_setups": {
        "input_path": "",
        "output_path": "",
        "algo_params": {
            "use_tta": false
        }
    },
    "wandb_setups": {
        "project": "Akoya Data",
        "group": "Test",
        "name": "Test Configs"
    }
}

We've also tried to run fine-tuning on other available pre-trained weights, on the subset of data from the datasets that was part of the competition, baseline model, different parameters, and so on. It all results in same performance. However, training and fine-tuning Cellpose model on our data doesn't cost same

Can you help to identify the issues?

This is a small thing, but I was wondering how the current Mediar weights differ from the ones submitted to the challenge, if at all. We ran Mediar on the tuning set of the challenge using the evaluation code, and obtained a median F1 score of 0.9126 with TTA and ensembling (this json). Without TTA and ensembling, the median F1 score was 0.9091 (this json).

In the Mediar paper you report 0.9067, which nearly matches the largest score on the leaderboard of 0.9070.

Thanks.

The Cellpose license (BSD-3) is very permissive. However, it requires you to retain the copyright notice here when you copy portions of the code.

All of the contents of this library in your repository are copied over from the Cellpose dynamics library here (primarily) or from the Cellpose utils and metrics libraries. The copyright notice must be maintained and distributed with every copy of your codebase (including in the napari plugin and in the docker image).

In addition, we ask that you include this copyright statement at the top of your file:
"""
Copyright © 2022 Howard Hughes Medical Institute, Authored by Carsen Stringer and Marius Pachitariu.
"""

Thank you.

Hello,

I'm interested in using your model to segment cells in my data. However, I'd like to just run the prediction on your testing dataset. When I try to use the predict.py, it throws an error on not being able to find the data. Could you advise how to get access to the data and/or how I can use what you created for my own data? Thank you.

After I download the code and configure the environment, I can use pretrained and finetuned models to predict successfully, but the predicted label image is rotated 90 degrees clockwise and flipped symmetrically. But when I was about to run python main.py, I found that the parameter name in baseline.json was "tuning_mapping_file", but the parameter name in datasetter.py was "mapping_file_tuning", and there was no such parameter in mediar_example.json , which seems to be somewhat conflicting. Then when I unified the parameter names, the following error occurred in the IntensityDiversification of CellWare.py when running python main.py.

Original Traceback (most recent call last):
  File "/home/yss/anaconda3/envs/python39/lib/python3.9/site-packages/monai/transforms/transform.py", line 89, in apply_transform
    return _apply_transform(transform, data, unpack_items)
  File "/home/yss/anaconda3/envs/python39/lib/python3.9/site-packages/monai/transforms/transform.py", line 53, in _apply_transform
    return transform(parameters)
  File "/home/yss/MEDIAR/train_tools/data_utils/custom/CellAware.py", line 81, in __call__
    img_original = copy.deepcopy((1 - mask) * data["img"])
ValueError: operands could not be broadcast together with shapes (1,512,27) (3,512,512)

I checked my data set, and found the first problem occurring in the two groups of 00009 and 00010 of the official training set. My mapping_labeled.json is in the following format:

{
     "official": [
         {
             "img": "/home/yss/NeurIPS-CellSeg-main-remote/data/Official/Train_Labeled/images/cell_00001.bmp",
             "label": "/home/yss/NeurIPS-CellSeg-main-remote/data/Official/Train_Labeled/labels/cell_00001_label.tiff"
         },
         {
             "img": "/home/yss/NeurIPS-CellSeg-main-remote/data/Official/Train_Labeled/images/cell_00002.bmp",
             "label": "/home/yss/NeurIPS-CellSeg-main-remote/data/Official/Train_Labeled/labels/cell_00002_label.tiff"
         },
         {
             "img": "/home/yss/NeurIPS-CellSeg-main-remote/data/Official/Train_Labeled/images/cell_00003.bmp",
             "label": "/home/yss/NeurIPS-CellSeg-main-remote/data/Official/Train_Labeled/labels/cell_00003_label.tiff"
         },
         {
             "img": "/home/yss/NeurIPS-CellSeg-main-remote/data/Official/Train_Labeled/images/cell_00004.bmp",
             "label": "/home/yss/NeurIPS-CellSeg-main-remote/data/Official/Train_Labeled/labels/cell_00004_label.tiff"
         },
         ...
}

Is there any need for additional processing of the data before the dataloader? Does MEDIAR have the latest code version? I would greatly appreciate it if I could receive help.