Giter Club home page Giter Club logo

medimage.jl's Introduction

MedImage.jl

The filesystem and choice of metadata is loosly based on BIDS format [1]

This project was created to standardize data handling of 3D and 4D medical imaging. It is currently subject to change and I am open to suggestions that would improve the library in construction. I've included below in categories what needs to be done and some basic ideas on How to approach it. I will post it for consultations with the community and then process to create test cases where Python SimpleItk (or other) methods will be treated as a reference.

  1. Designing data structure - requirements (need to explicitly specify the most important and the rest will be just in an additional dictionary)
    • hold voxel data as a multidimensional array
    • keep spatial metadata - origin, orientation, spacing
    • type of the image - label/CT/MRI/PET (need to construct enum for this) - frequently will be needed to be supplied manually
    • subtype of the image for example if MRI ADC/DWI/T2 etc. (need to construct enum for this)
    • type of the voxel data (for example Float32)
    • Date of saving
    • Acquisition time
    • Patient ID if present
    • current device - (CPU, GPU)
    • Study UID
    • Patient UID
    • Series UID
    • Study Description
    • Original file name
    • Display data - set of colors for the labels; window value for CT scan - will provide set of defaults based on image type and I will modify MedEye3D for convenient visualizations.
    • Clinical data dictionary - age, gender ...
    • Is contrast administered
    • The rest of the metadata loaded from a file to store in a dictionary
  2. Data loading
    • Nifti - start with Nifti.jl
    • Dicom - Dicom.jl
    • Mha - ?
  3. Modifying voxel data together with spacing data
  • Modifying orientation of all images to single orientation - for example, RAS (we should select some default orientation) - TODO with AxisArrays.jl plus keep track of origin
  • Changing spacing to a given spacing - TODO with AxisArrays.jl with Interpolations.jl plus keep track of origin; use nearest neighbor interpolator for the label; and for example b spline for others
  • Resampling to another grid - for example, resample PET image to CT image based just on spatial metadata keeping track of changed metadata of the moving image TODO with AxisArrays.jl and Interpolations.jl
  • Cropping and dilatating with adjusting of origin offset TODO with AxisArrays.jl, storing also positions of original image in metadata (some artifacts happen on the image edges)
  1. Adding persistency
    • store the data as array in HDF5 and metadata as its attributes
    • design efficient loading and saving irrespective of the device the array is on
    • add the possibility to save back as nifty and dicom files for exporting

Point 3 is the trickiest one to do as there are a lot of corner cases

to look for https://github.com/haberdashPI/MetaArrays.jl

References

1)https://www.nature.com/articles/sdata201644

medimage.jl's People

Contributors

jakubmitura14 avatar janzubik avatar divital-coder avatar

Stargazers

 avatar Jacob S. Zelko avatar

Watchers

Guillermo Sahonero Alvarez avatar  avatar  avatar

medimage.jl's Issues

Non isovolumetric voxels in case of rotations may lead to issues

Becouse rotations.jl seem to have no support of rotating the image for the case when non isovolumetric voxels are present in some cases the resulting image can lose correct representation in real world coordinates. To understand it let's give example let's say we have very "high voxels" so for example voxel has 5 mm in z axis and only 1 mm in x and y axis. We know that the more distant from the center of rotation is the point the bigger the arc it need to travel. Hence in our example the point that will be 10 voxels in z direction should move in far greater arc than the voxel that is 10 voxels from the center in x direction. Hence, we need to work in world coordinate system and take it into account.

image
On the image on the left it is how rotation would look like if we would ignore spacing in green distance from point to center of rotation; and in yellow arc that a point moves during rotation; on the right when we take the spacing into account. Image is a rough schamtic lenths and proportions are only approximate.

Proposed solution can look somewhat like this:

  1. manually using sine and cosines or using rotations.jl obtain rotation matrix
  2. get cartesian indices of all points of the voxel array we want to rotate using "get_base_indicies_arr" function from Utils.jl
  3. transform cartesian indicies (let's name them points_to_interpolate) into real world coordinates by taking into account spacing and origin sth like below
    points_to_interpolate = get_base_indicies_arr(new_size)
    points_to_interpolate=points_to_interpolate.*new_spacing
     points_to_interpolate = points_to_interpolate.+origin
  1. using tensor product apply rotation matrix on coordinates of each point from points_to_interpolate
  2. now we have new coordinates of each point we should test it against Rotations.jl (in case of spacing equal (1,1,1) - so isovolumetric results should be identical)
  3. we should concatenate the value of each point with its coordinate so we will have vector of length for each voxel in original image
  4. now we need to do diffrently depending on weather we want to have the same shape/size of the array as input image (default case) or diffrent. We need to consider a different bigger shape as it can be necessary in order to fit all new points.
    a) in case we want to have the same shape as input image we ignore all points that get outside of physical space of the original image (we can establish it in each axis by mutliplying number of voxels in each axis times thie spacing in this axis and taking into account origin; so for example given origin 10,11,13 and spacing 0.5,0.6,0.8 and image size 90,100,110 the y axis will start at 11 and end at 71).
    b) in case we do not want to have the same shape as original image we need to increase the size of the resulting array and if needed move the origin (so for example as above if the lowest value of y in all points will be 9 we need to set the origin value of y to 9 so 10,9,13 and increase the array size by 4 {11-9=2; ceil(2/0.6)=4 } , analogously for all axes, in case of points getting out down the axis we also need to increase the size of the resulting image but we do not need to change the position of the origin)
  5. having now empty result array; points values and their coordinates we need to interpolate them with the grid associated with output image. In default case where origin position do not change we can reuse the points_to_interpolate we had before applying rotation matrix; for case we want to change result array shape we can get output_points in analogous way as we obtained points_to_interpolate
  6. We need now to interpolate the values we have in points_to_interpolate onto the output_points via interpolator chosen by the user.
    generally one should adapt my_interpolate from Utils.jl - add necessery arguments in it if any modification of this function is required look into griddeed interpolation Gridded as it will probably show what modifications are needed . In case it would not work We can have 3 strategies how to get this problem solved - 1) check how it is performed in Rotations.jl; 2) use Interpolations.jl via griddeed interpolation Gridded if possible 3) use ScatteredInterpolation.jl
  7. test against the simple itk in the test that are present in the test file; visualize the results of simple itk and medimage rotation as nifti files and compare them manually in 3dSlicer

Notes:
We need also to establish is image orientation affecting those calculations, test on diffrent orientations that can be set by change_orientation function from spatial_metadata_change file. in spatial_metadata_change test file there is simple itk function also for testing if needed.

As the inspiration look into resample_to_spacing function implementation

Nearly identical procedure but instead of applying the rotation matrix we would add the translation vector can be used in a variant of translation (other variant is just change the origin; both variants are usefull)

Saving nifti files with MedImage Object data from dicom files

within load_and_save.jl in src, for the save function it seems as if it expects a nifti file to manipulate.
with the MedImage object formulated with data from a dicom file, does a path for an equivalent nifti file is expected by the save function?
Is dicom to nifti file conversion prominent here,
or
we can get away with loading spatial metadata into the MedImage object, from both the formats, and
[]for Nifti File
--manipulating the origin nifti file
[]for Dicom File
--creating and manipulating an empty nifti file

Supporting references and further reads for better understanding of the NIfTI and DICOM functionality.

Gist of dicom data in python
https://towardsdatascience.com/dealing-with-dicom-using-imageio-python-package-117f1212ab82

Gist of NIfTI data in julia and python
https://brainder.org/2012/09/23/the-nifti-file-format/
https://peerherholz.github.io/workshop_weizmann/data/image_manipulation_nibabel.html#

^ above are some good starting points for newbies, who might want to consider reading more about the ecosystem and capabilities.
Feel free to include them in references as the documentation for the project grows!

Add documentation

Add documentation to

MedImage_data_struct.jl file

  1. @divital-coder Describe shortly first enums what they mean; secondly the MedImage struct together with information that it is immutable
  2. @divital-coder show how to change fields using accessors.jl on the basis of the function "update_voxel_and_spatial_data" from Utils.jl

Utils.jl file

  1. @JanZubik describe get_base_indicies_arr as function for convienient cartesian indicies extraction ; give examples of execution and show shapes of the resulting array

  2. @JanZubik cast_to_array_b_type give example that we will change the array of one type to the array of other also example when array a is float and array b is int and describe that the function prevent from getting inexact error message when casting float to Int

  3. @JanZubik interpolate_point - explain convention @JanZubik that all negative indiciec will be casted to extrapolated value and the we are interpolating points here - show examples to get values from negative indicies, indicies in range of the image and indicies way futher than image physical space (which will be extrapolated)

  4. @JanZubik interpolate_my explain the function particularly stressing what happens at extrapolation, negative indicies, that we deal with non isovolumetric voxels and what are the diffrences between diffrent interpolators - mainly that nearest neighbour is usefull for discrete lebels; linear is fast, b spline give good resolution

  5. @divital-coder TransformIndexToPhysicalPoint_julia show examples how we can get a point in physical space from indexes and how it will change for images with diffrent spatial metaata values - move origin show how it will affect, change spacin show how it affect, change direction ...

  6. @divital-coder ensure_tuple - just utility function needed becouse we expect in medimage struct tuples not lists show example of usage with list and with tuple

##Spatial_metadata_change.jl
9) @divital-coder change_orientation function - explain what is orientation show how to get orientation code (using string_to_orientation_enum from orientation_dicts.jl and vice versa using orientation_enum_to_string ), show example that will make it possible to see that changing direction can change order of spacing can change origin and show that it flips image and permute axis

  1. @JanZubik explain resample_to_spacing - show examples how the resolution of the image changes with diffrent spacing (for example using printscreens from slicer) explain what is the goal and copy here also what interpolators to use in what cases

Resample_to_target.jl

  1. @JanZubik resample_to_image explain what is the goal of the function give some example how it will look like and how changing spatial metadata like changing origin or spacing ... will affect change in the resulting image; consider adding simple schematic you used to understand the concept

##Load_and_save
12) @divital-coder you know all about topic make descriptions of the functions explain that both dicom and nifti will load into the same format; explain arguments and why setting all files orientation to rAS at the begining may be good idea (standarization)

HDF5_manag.jl

  1. @divital-coder simple save and load of the MedImage struct describe functions tell why hdf5 format is good (it is explained for example in julia con when hdf5.jl had their talk - it is fast, convinient for multidimensional arrays handling , can load in parts ... )

##Basic_transformations
14) @JanZubik Jan Zubik you know the issue from bottom up; so just describe all functions and make tests work

tests

  1. @JanZubik resample_to_image, change_orientation and resample_to_spacing has their tests ready but they need to be integrated together with other tests from test_spatial_metadata_change.jl into single test set, look into docs of tests in order to make it work https://docs.julialang.org/en/v1/stdlib/Test/ ; the test set combining all of the tests should be in a separate file, also the path to the test file should be relative - hence automatic testing in future could be run.
  2. @divital-coder if possible change simple itk function invocation from tests into wrapped C++ lib version and get rid of all Pycall conda etc from the package :)

Loading Dicom Directory with multiple dicom files

within the load_and_save.jl, loading into the current dicom directory using DICOM.jl.
do we need to return MedImage objects for each individual file and save them to nifti as well?
since a function might end on "return" ,
is it wise to return an array of objects instead , and create new nifti files for each object?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.