suggest your papers here! about x-unet HOT 19 OPEN

it will also come complete with conditioning, so it can be used straightforwardly in DDPMs

from x-unet.

Hi Phil,

yeah, Mask2Former is a little bit different from the standard UNet setup, which very likely makes it tricky to implement it in one abstraction (but I wanted to bring it up, because of the good results they achieved).

This is another interesting approach:
"CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation", https://arxiv.org/abs/2103.03024, https://github.com/YtongXie/CoTr

I'm pretty excited of mixing convs + attn for UNets due to https://github.com/xxxnell/how-do-vits-work

I also recently rewrote your UNet code from https://github.com/lucidrains/denoising-diffusion-pytorch/blob/master/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py#L180 to use it as "normal" UNet, but then had to postpone testing it due to other work coming up, but this is why I'm looking forward to your x-unet even more. :-)

Cheers,
Michael

from x-unet.

Yeah, I would say UNets are still the standard there.
If I remember it correctly one of the nnUnet authors (https://github.com/MIC-DKFZ/nnUNet) once mentioned that a well trained UNet is hard to beat. (But training them properly can be of course tricky. ;-) )

from x-unet.

@MicPie yea, even the transformers side seems to converge on the same structure (https://arxiv.org/abs/2110.13711 and https://arxiv.org/abs/2202.10890)

i'll see what i can do to push things forward :)

edit: you are so well read, as usual!

from x-unet.

U-Net_and_Its_Variants_for_Medical_Image_Segmentation_A_Review_of_Theory_and_Applications.pdf

👀

from x-unet.

"iUNets - Fully invertible U-Nets with Learnable Up- and Downsampling", https://arxiv.org/abs/2005.05220, https://github.com/cetmann/iunets

“Mask2Former - Masked-attention Mask Transformer for Universal Image Segmentation", https://bowenc0221.github.io/mask2former/

Looking forward to the code! :-)

from x-unet.

@MicPie ok, let me figure out the main inductive bias behind maskformer and get back to you :)

from x-unet.

Thank you for sharing, I didn't know the "Hierarchical Perceiver" paper and the one about UNet variants. 🙏

UNet++ looks really fancy, the corresponding paper has also some nice figures with the setup: https://arxiv.org/abs/1807.10165
(There is also a PyTorch version out, but with a lot of hard coded connections in the forward: https://github.com/hellopipu/unet_plus/blob/master/model.py)

from x-unet.

Hi @lucidrains, please check out our [CVPR2022-Oral] "MAXIM: Multi-Axis MLP for Image Processing", a multi-stage UNet with (standalone) multi-axis MLP layers. It is concurrent work with Restormer with similar performances on multiple low-level tasks.

We provide Jax code here: https://github.com/google-research/maxim
Paper: https://arxiv.org/abs/2201.02973

from x-unet.

UNet++: A Nested U-Net Architecture for Medical Image Segmentation
Attention U-Net: Learning Where to Look for the Pancreas
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
UNETR: Transformers for 3D Medical Image Segmentation
Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

from x-unet.

@vvvm23 i'll see if i can port over the logic in a clean manner

from x-unet.

Hi Micpie! 👋

Is Maskformer considered a unet?? I guess it does have downsamples and then upsample layers

from x-unet.

@MicPie are you using any u-nets for your AI medical imaging work?

from x-unet.

https://arxiv.org/abs/1906.06148

from x-unet.

Imagen (needs no introduction) proposes some interesting improvements as a so-called "Efficient U-Net". Might be worth checking out Appendix B.1 for a summary.

from x-unet.

Imagen (needs no introduction) proposes some interesting improvements as a so-called "Efficient U-Net". Might be worth checking out Appendix B.1 for a summary.

yup, i have it built over at https://github.com/lucidrains/imagen-pytorch and @nousr has successfully used it for his medical segmentation school work

from x-unet.

UNet++: A Nested U-Net Architecture for Medical Image Segmentation Attention U-Net: Learning Where to Look for the Pancreas TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation UNETR: Transformers for 3D Medical Image Segmentation Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

great list! 🙏

from x-unet.

I think this paper is worth looking at

• UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-Wise
  Perspective with Transformer

from x-unet.

https://github.com/3dim-paper/website
Talks about X-Unet, based on imagen

from x-unet.