mhubai / models Goto Github PK

As discussed earlier, I think this should be added to model.json and also propagate into the web page of the model. I suggest the most straightforward would be to include SeriesInstanceUID(s) for the suitable DICOM series that were confirmed to produce acceptable results with the model.

Per @vkt1414, the version in mhub base image is from 2018!

We should get the latest stable release from https://github.com/rordenlab/dcm2niix/releases instead.

Ursprünglich gepostet von @fedorov in MHubAI/mhubio#2

@LennyN95 @denbonte - we would like to add Prostate158 zonal segmentation model. @ccosmin97 is happy to work on this, although he would definitely appreciate help from the original developer @kbressem (perhaps this is anyway consistent with @hugoaerts desire to have more AIM-Harvard models in MHub).

Can we discuss steps/timeline/missing features (hopefully none!) here?

• The model is currently crashing when more than one GPU is available and this requires explicit GPU selection on machines with >1 GPUs installed on the host machine. This needs to be addressed, as all MHub models are expected to run with the --gpus all option.

Originally posted by @LennyN95 in #42 (comment)

The issue

When running the MHub docker container with the lobe segmentation code from this repository with the --gpus all flags enabled and having 2 or more GPUs we run into the following error:

--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.8/logging/__init__.py", line 1085, in emit
    msg = self.format(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 929, in format
    return fmt.format(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 668, in format
    record.message = record.getMessage()
  File "/usr/lib/python3.8/logging/__init__.py", line 373, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/app/models/gc_lunglobes/utils/LobeSegmentationRunner.py", line 50, in task
    handle = segment_lobe_init()
  File "/app/src/test.py", line 1830, in segment_lobe_init
    lobe_seg_instance = LobeSegmentationTSTestCOVID(settings)
  File "/app/src/test.py", line 1524, in __init__
    self.init()
  File "/app/src/test.py", line 700, in init
    self.logger.info("Let's use", torch.cuda.device_count(), "GPUs!")
Message: "Let's use"
Arguments: (2, 'GPUs!')

This error appears to be related to incorrect formatting of the logger.info call. Furthermore, if the logging line is removed we get the following error:

  File "/app/models/gc_lunglobes/utils/LobeSegmentationRunner.py", line 51, in task
    seg_result_np = segment_lobe(handle, img_np, meta_dict)
  File "/app/src/test.py", line 1861, in segment_lobe
    pred = handle.run(transformed_data_dict)
  File "/app/src/test.py", line 1572, in run
    scan_level_inf = self.model.scan_level_inference(pad_scan).cpu().squeeze(0)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'DataParallel' object has no attribute 'scan_level_inference'

Upon inspecting the latter issue it appears that getting the multi-GPU feature to work properly isn't as simple as wrapping the model with torch.nn.DataParallel because the wrapped model uses custom methods (i.e. scan_level_inference) for inference, which are not picked up by the DataParallel mechanism of PyTorch.

Suggested fix

As fixing the multi-GPU feature properly would be quite some work, the broken multi-GPU feature could be disabled entirely by removal of the following lines:

https://github.com/DIAGNijmegen/bodyct-pulmonary-lobe-segmentation/blob/5a64b70504d46c042c30851a69cec370f1202e67/test.py#L699-L702