Segment anything with our Napari integration of Meta AI's new Segment Anything Model (SAM)!

SAM is the new segmentation system from Meta AI capable of one-click segmentation of any object, and now, our plugin neatly integrates this into Napari.

We have already extended SAM's click-based foreground separation to full click-based semantic segmentation and instance segmentation!

At last, our SAM integration supports both 2D and 3D images!

Everything mode	Click-based semantic segmentation mode	Click-based instance segmentation mode

Click to play the video

The plugin requires python>=3.8, as well as pytorch>=1.7 and torchvision>=0.8. Please follow the instructions here to install both PyTorch and TorchVision dependencies. Installing both PyTorch and TorchVision with CUDA support is strongly recommended.

Install Napari via pip:

pip install napari[all]

You can install napari-sam via pip:

pip install git+https://github.com/facebookresearch/segment-anything.git
pip install napari-sam

To install latest development version :

pip install git+https://github.com/MIC-DKFZ/napari-sam.git

Start Napari from the console with:

napari

Then navigate to Plugins -> Segment Anything (napari-sam) and drag & drop an image into Napari. At last create, a labels layer that will be used for the SAM predictions, by clicking in the layer list on the third button.

You can then auto-download one of the available SAM models (this can take 1-2 minutes), activate one of the annotations & segmentation modes, and you are ready to go!

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

Distributed under the terms of the Apache Software License 2.0 license, "napari-sam" is free and open source software

If you encounter any problems, please file an issue along with a detailed description.

napari-sam is developed and maintained by the Applied Computer Vision Lab (ACVL) of Helmholtz Imaging and the Division of Medical Image Computing at the German Cancer Research Center (DKFZ).