SMP example: https://github.com/qubvel/segmentation_models.pytorch/blob/master/examples/cars%20segmentation%20(camvid).ipynb

2018 Data Science Bowl example: https://github.com/selimsef/dsb2018_topcoders/

Given the cell area is used at quite a few places throughout the workflow, compute this for each cell, store it in the csv files for the cross-validation folds. So, re-generate the cv folds.

Deep watershed transform paper: https://arxiv.org/pdf/1611.08303.pdf
Authors' implementation: https://github.com/min2209/dwt/tree/master/DN

This occurs when running pl.LightningModule.to_torchscript, when, in the forward method of the model, there is an instantiation of a nn class, like nn.Upsample. Perhaps, the instantiation needs to happen in the model's __init__ method.

FrontendError: Cannot instantiate class 'Upsample' in a script function:
  File "/kaggle/sartorius_cell_instance_segmentation_kaggle/seggit/models/watershed_transform_net.py", line 47
        x = self.fcn(x)
    
        x = nn.Upsample(input_size)(x)
            ~~~~~~~~~~~ <--- HERE
    
        return x

Authors' Github: https://github.com/min2209/dwt/tree/master/WTN

Using original image and semg size, forward propagation raises error. Likely caused by the sizes not being divisible by 32.

• uvec -- sardata_uvec
• wngy -- sardata_watershed_energy

Whole-image maps of:

• semantic segmentation (semg)
• distance transform (dtfm)
• normalised gradient distance transform (uvec)
• discrete watershed energy (wngy)

are built up by computing them for individual cells, pasting them into the image frame, in order of descending of cell area.

There would be fewer computations if the whole-image uvec map was computed directly from the whole-map dtfm, but the resulting uvec map doesn't appear to mark out the cell boundaries well.

In addition, for rotational transforms during data augmentation, the uvec needs to be computed from the distance transform after the distance transform has been rotated. Its components cannot be simply treated like scalar maps that can be rotated the usual way using albumentations.

Because of these, it seems there are 2 options:

1. During data augmentation, build all the needed maps from individual cell annotations. Pass individual cells' dtfm through the rotation, compute their uvec, then build up the whole-image uvec map from them. This might be too time-consuming.
2. Rotate the whole-image uvec map using own rotation implementation, rotate all other maps using albumentations. This might restrict the variety of rotational transformations that can be used though.

End-to-end = DN + WTN

https://github.com/min2209/dwt/tree/master/E2E

• From image to instance mask in Notebook.

How to cut the discrete watershed energy map to obtain instance segmentation?

https://github.com/min2209/dwt/blob/master/E2E/post_process.py

Maybe consider trying this after seeing how well current semantic segmentation masks work with the deep watershed transform.

Have tried:

1. 1-channel target, cells with overlap removed.
2. 2-channel target. Channel 0 contains cells but with annotation overlap and touching borders removed. Channel 1 contains those regions removed in channel 0. See example.

It seems that the above target choices are geared towards separating the cells.

In the deep watershed transform approach, the semantic segmentation stage seems to be only about separating the different classes. The separation of instances begins with the Direction Net.

In Fig.4 of the deep watershed transform paper, it's seen that the semantic segmentation makes no attempt at separating the two cars. They are merged together under one mask. It's only in the next stage that the direction net is required to predict the direction vectors, which do mark out the boundary between the two cars. The direction loss is maximum when vectors on either side of a boundary point in the same direction.

It therefore looks like the training target should simply be just a 1-channel boolean mask, where it's True if the pixel comes under at least one cell, and False if it comes under no cells.