vibbits / nis2pyr Goto Github PK

Python utility and library for converting Nikon .nd2 files to pyramidal OME TIFF.

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

microscopy nikon nis-elements nd2 ome tiff python

nis2pyr's Introduction

nis2pyr

nis2pyr converts Nikon .nd2 files to tiled pyramidal OME TIFF files. The conversion is straightforward and can be performed via the nis2pyr command line tool, or via a simple one-liner in Python using the nis2pyr package.

System Requirements

nis2pyr was tested on these platforms:

Windows 10
Windows Server 2019
Ubuntu 18.04
Ubuntu 20.04
macOS 10.15
macOS 11

Installation

Installation is straightforward:

pip install nis2pyr

This installs the nis2pyr Python package which provides an easy to use function to convert ND2 files to OME TIFF. It also installs the nis2pyr program to convert ND2 files to OME TIFF from the command line.

Using nis2pyr in a Python program

Converting an ND2 file to pyramidal OME TIFF in Python is easy:

from nis2pyr.convertor import convert_nd2_to_pyramidal_ome_tiff

convert_nd2_to_pyramidal_ome_tiff('original.nd2', 'pyramidal.ome.tif', compression='zlib')

The number of pyramid levels and the tile size can be specified as well (see the convert_nd2_to_pyramidal_ome_tiff docstring) but there is typically no need to do so.

Running nis2pyr on the command line

To generate an uncompressed pyramidal file with the default options, run nis2pyr on the command line, specifying the .nd2 input image file and the name of the pyramidal OME TIFF to which it needs to be converted:

nis2pyr input.nd2 pyramid.ome.tif

It is also possible to specify the compression algorithm, number of pyramid levels and the tile size of the output pyramidal OME TIFF.

usage: nis2pyr [-h] [--version] [--compression COMPRESSION] 
               [--pyramid-levels PYRAMID_LEVELS] [--tile-size TILE_SIZE]
               input_file_pattern [pyramid_filename]

Convert Nikon ND2 image files to tiled pyramidal OME TIFF files.

positional arguments:
  input_file_pattern    either the filename of a single ND2 file that needs to 
                        be converted, or a filename pattern ('glob') to convert 
                        multiple files. This filename pattern can be used for
                        example to convert all ND2 files in a given directory.
  pyramid_filename      if only a single filename is specified as input file 
                        pattern, pyramid_filename is the full filename of the
                        resulting pyramidal OME TIFF file; if no pyramid filename
                        is provided, or if multiple files are converted, the 
                        pyramidal OME TIFF(s) will be written to the same directory 
                        as the original ND2 and with the same filename but 
                        with an .ome.tif extension

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --compression COMPRESSION
                        the algorithm used for compressing the image data; currently
                        'zlib' is the only supported compression algorithm
                        (default: no compression)
  --pyramid-levels PYRAMID_LEVELS
                        the maximum number of resolution levels in the pyramidal OME TIFF,
                        including the full resolution image; successive pyramid
                        levels are downsampled by a factor 2
                        (default: 6)
  --tile-size TILE_SIZE
                        width in pixels of the tiles in the pyramidal OME TIFF;
                        the tiles are square; tile size must be a multiple of 16
                        (default: 256)

Limitations

Known limitations of the current version of nis2pyr:

ND2 metadata is lost, except for pixel size and channel names and colors.

nis2pyr's People

Contributors

Stargazers

Watchers

Forkers

bpavie

nis2pyr's Issues

Crash in nd2reader while reading 8-bit RGB image

python nis2pyr.py "g:\NISMakePyramidal\Testfiles\SlideExpressJaneDoe\Slide2-17-1_ChannelBrightfield_Seq0079.nd2" e:\temp\

gives

E:\Anaconda3\envs\nis\lib\site-packages\pims\base_frames.py:472: UserWarning: Please call FramesSequenceND.__init__() at the start of thethe reader initialization.
  warn("Please call FramesSequenceND.__init__() at the start of the"
ND2 file: #frames=1, #components=3, #planes=1, bits per component=8, is RGB=True, pixel size=0.48 um, frames.sizes={'x': 79200, 'y': 26829, 'c': 3}
Reading image
Traceback (most recent call last):
  File "nis2pyr.py", line 100, in <module>
    image, metadata = read_nd2(args.nd2_filename)
  File "E:\xx\xx\xx\xx\reader.py", line 34, in read_nd2
    image = frames[0]
  File "E:\Anaconda3\envs\nis\lib\site-packages\slicerator\__init__.py", line 188, in __getitem__
    return self._get(indices)
  File "E:\Anaconda3\envs\nis\lib\site-packages\pims\base_frames.py", line 98, in __getitem__
    return self.get_frame(key)
  File "E:\Anaconda3\envs\nis\lib\site-packages\pims\base_frames.py", line 592, in get_frame
    result = self._get_frame_wrapped(**coords)
  File "E:\Anaconda3\envs\nis\lib\site-packages\pims\base_frames.py", line 252, in get_frame_T
    return get_frame(**ind).transpose(transposition)
  File "E:\Anaconda3\envs\nis\lib\site-packages\pims\base_frames.py", line 265, in get_frame_bundled
    frame = get_frame(**ind)
  File "E:\Anaconda3\envs\nis\lib\site-packages\pims\base_frames.py", line 303, in get_frame_dropped
    result = get_frame(**ind)
  File "E:\Anaconda3\envs\nis\lib\site-packages\pims_nd2\nd2reader.py", line 194, in get_frame_2D
    h.Lim_FileGetImageData(self._handle, i, self._buf_p, self._buf_md)
OSError: exception: access violation writing 0x00000271BF0DB000

Incorrect output with diagonal pattern and wrong color

Reproduce with
python nis2pyr.py "g:\xx\Testfiles\BF\RGB-8bit\Slide2-3-1_ChannelBrightfield_Seq0001.nd2" e:\temp\bug.ome.tif

Part of the nd2 metadata:

{'width': 12730, 'width_bytes': 38192, 'height': 5039, 'components': 3, 'bitsize_memory': 8, 
'bitsize_significant': 8, 'sequence_count': 1, 'tile_width': 12730, 'tile_height': 5039,  ...}

Note that width_bytes != components * width so perhaps the bug is related to nd2reader not handling padding bytes correctly?

Cython.Compiler.Errors.CompileError: nd2/_sdk/picture.pyx

this happened both:

with pip install nis2pyx
with pip install -e .[dev] after successfully cloning the git repository

  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [38 lines of output]
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      
          cdef np.ndarray to_ndarray(self):
              cdef np.ndarray ndarray = np.array(self, copy=False)
      
              # Assign our object to the 'base' of the ndarray object
              ndarray.base = <PyObject*> self
                     ^
      ------------------------------------------------------------
      
      nd2/_sdk/picture.pyx:41:15: Assignment to a read-only property
      Compiling nd2/_sdk/latest.pyx because it changed.
      Compiling nd2/_sdk/v9.pyx because it changed.
      Compiling nd2/_sdk/picture.pyx because it changed.
      [1/3] Cythonizing nd2/_sdk/latest.pyx
      [2/3] Cythonizing nd2/_sdk/picture.pyx
      Traceback (most recent call last):
        File "/home/sugliano/my-envs/nis2pyr/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/sugliano/my-envs/nis2pyr/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/sugliano/my-envs/nis2pyr/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-4yxxw_lb/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-4yxxw_lb/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-4yxxw_lb/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 54, in <module>
        File "/tmp/pip-build-env-4yxxw_lb/overlay/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
          cythonize_one(*args)
        File "/tmp/pip-build-env-4yxxw_lb/overlay/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: nd2/_sdk/picture.pyx
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.

Use logger class instead of hard-coded print statements to stdout

Perhaps the logger should be configurable with different verbosity levels.

Pip install fails due to issue with metadata generation

Attempting to install within a virtual environment running python 3.9.11 with pip install nis2pyr leads to the following error:

Collecting nis2pyr
  Using cached nis2pyr-0.6.2-py3-none-any.whl (11 kB)
Collecting nd2==0.1.4
  Using cached nd2-0.1.4-cp39-cp39-macosx_11_0_arm64.whl (132 kB)
Collecting opencv-python==4.5.3.56
  Using cached opencv_python-4.5.3.56-cp39-cp39-macosx_11_0_arm64.whl (10.7 MB)
Collecting tifffile==2021.8.30
  Using cached tifffile-2021.8.30-py3-none-any.whl (171 kB)
Collecting ome-types==0.2.9
  Using cached ome_types-0.2.9-py3-none-any.whl (137 kB)
Collecting imagecodecs==2021.8.26
  Using cached imagecodecs-2021.8.26.tar.gz (10.4 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      running egg_info
      creating /private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-pip-egg-info-u3dftrg8/imagecodecs.egg-info
      writing /private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-pip-egg-info-u3dftrg8/imagecodecs.egg-info/PKG-INFO
      writing dependency_links to /private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-pip-egg-info-u3dftrg8/imagecodecs.egg-info/dependency_links.txt
      writing entry points to /private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-pip-egg-info-u3dftrg8/imagecodecs.egg-info/entry_points.txt
      writing requirements to /private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-pip-egg-info-u3dftrg8/imagecodecs.egg-info/requires.txt
      writing top-level names to /private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-pip-egg-info-u3dftrg8/imagecodecs.egg-info/top_level.txt
      writing manifest file '/private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-pip-egg-info-u3dftrg8/imagecodecs.egg-info/SOURCES.txt'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-install-h1tf5g4f/imagecodecs_20b8e6a946b64134bb551a3a87b00a95/setup.py", line 613, in <module>
          setup(
        File "/Users/rogangrant/Documents/GitHub/image_analysis/image_analysis_venv/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/Users/rogangrant/.pyenv/versions/3.9.11/lib/python3.9/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/Users/rogangrant/.pyenv/versions/3.9.11/lib/python3.9/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/Users/rogangrant/.pyenv/versions/3.9.11/lib/python3.9/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/Users/rogangrant/Documents/GitHub/image_analysis/image_analysis_venv/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 299, in run
          self.find_sources()
        File "/Users/rogangrant/Documents/GitHub/image_analysis/image_analysis_venv/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
          mm.run()
        File "/Users/rogangrant/Documents/GitHub/image_analysis/image_analysis_venv/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 541, in run
          self.add_defaults()
        File "/Users/rogangrant/Documents/GitHub/image_analysis/image_analysis_venv/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 578, in add_defaults
          sdist.add_defaults(self)
        File "/Users/rogangrant/.pyenv/versions/3.9.11/lib/python3.9/distutils/command/sdist.py", line 228, in add_defaults
          self._add_defaults_ext()
        File "/Users/rogangrant/.pyenv/versions/3.9.11/lib/python3.9/distutils/command/sdist.py", line 311, in _add_defaults_ext
          build_ext = self.get_finalized_command('build_ext')
        File "/Users/rogangrant/.pyenv/versions/3.9.11/lib/python3.9/distutils/cmd.py", line 299, in get_finalized_command
          cmd_obj.ensure_finalized()
        File "/Users/rogangrant/.pyenv/versions/3.9.11/lib/python3.9/distutils/cmd.py", line 107, in ensure_finalized
          self.finalize_options()
        File "/private/var/folders/3m/ms22yqxx2dq38gxl025800nh0000gn/T/pip-install-h1tf5g4f/imagecodecs_20b8e6a946b64134bb551a3a87b00a95/setup.py", line 582, in finalize_options
          import numpy
      ModuleNotFoundError: No module named 'numpy'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

This error persists if I specify commit fae5229. If you have any insight, I would very much appreciate it. Thank you for your help!

Automatically create Release with stand-alone nis2pyr executables

Add a GitHub action to automatically create a GitHub Release with stand-alone nis2pyr executables for the different platforms whenever a version tag is pushed.

Support .nd2 images with multiple z

Testdata:

G:\Datasets\JohnDoe\OriginalData\20191009_1_OK371_nucLacZ_spinning_disk_40x_tile_scan.nd2 (xyz)
G:\Datasets\OntoData\Input\Cell\G3_79_60x_overview.nd2 (xycz)

cannot import name 'ModelMetaclass' from 'pydantic.main

I found I cannot import he nis2pyr to python.
I use the following command line and received a import error.
from nis2pyr.convertor import convert_nd2_to_pyramidal_ome_tiff
The issue seems to be the authors of pydantic made ModelMetaclass private in recent version.

File ~/anaconda3/envs/nis2pyr/lib/python3.8/site-packages/nis2pyr/convertor.py:3
1 import nd2
2 from typing import Optional
----> 3 from nis2pyr.writer import write_pyramidal_ome_tiff
6 def convert_nd2_to_pyramidal_ome_tiff(nd2_filename: str,
7 pyramid_filename: str,
8 compression: Optional[str] = None,
9 tile_size: int = 256,
10 max_levels: int = 6) -> None:
11 """Convert an ND2 file to a tiled pyramidal OME TIFF file.
12
13 Args:
(...)
23 including the full resolution original.
24 """

File ~/anaconda3/envs/nis2pyr/lib/python3.8/site-packages/nis2pyr/writer.py:8
4 import tifffile # https://github.com/cgohlke/tifffile
6 from typing import Optional, Tuple
----> 8 from nis2pyr.metadata import update_channels_info, get_ome_voxelsize
9 from nis2pyr.reader import read_nd2file
10 from nis2pyr import version

File ~/anaconda3/envs/nis2pyr/lib/python3.8/site-packages/nis2pyr/metadata.py:1
----> 1 import ome_types
2 import tifffile
3 from typing import Any, Dict, List, Optional, Tuple

File ~/anaconda3/envs/nis2pyr/lib/python3.8/site-packages/ome_types/init.py:9
6 version = "unknown"
8 try:
----> 9 from . import model
10 from .model import OME
11 except ModuleNotFoundError as e:

File ~/anaconda3/envs/nis2pyr/lib/python3.8/site-packages/ome_types/model/init.py:1
----> 1 from .affine_transform import AffineTransform
2 from .annotation import Annotation
3 from .annotation_ref import AnnotationRef

File ~/anaconda3/envs/nis2pyr/lib/python3.8/site-packages/ome_types/model/affine_transform.py:1
----> 1 from ome_types._base_type import OMEType
4 class AffineTransform(OMEType):
5 """A matrix used to transform the shape.
6
7 ⎡ A00, A01, A02 ⎤ ⎢ A10, A11, A12 ⎥ ⎣ 0, 0, 1 ⎦
(...)
16 a12 : float
17 """

File ~/anaconda3/envs/nis2pyr/lib/python3.8/site-packages/ome_types/_base_type.py:13
1 from typing import (
2 TYPE_CHECKING,
3 Any,
(...)
9 no_type_check,
10 )
12 from pydantic import BaseModel, validator
---> 13 from pydantic.main import ModelMetaclass
15 if TYPE_CHECKING:
16 import pint

ImportError: cannot import name 'ModelMetaclass' from 'pydantic.main' (/home/kuanwei/anaconda3/envs/nis2pyr/lib/python3.8/site-packages/pydantic/main.py)

Support .nd2 images with multiple t