Giter Club home page Giter Club logo

dicomp10-to-dicomweb-js's Introduction

dicomp10-to-dicomweb-js

Status: Beta - in development (as of Oct 25, 2021)

The scope of this project is to convert DICOM P10 into a format into a DICOMweb compatible format - specifically JSON metadata, frames and bulkdata. This library will enable the following:

  • Building of DICOMweb compliant services
    • Ability to stream the data for "on the fly" use cases
  • Ability to pregenerate DICOMweb responses and store them so they can be served up with a standard HTTP server to implement a subset of DICOMweb WADO-RS, specifically:
    • Retrieve instance metadata
    • Retrieve series level metadata
    • Retrieve frames in an instance
    • Retrieve bulk data
    • Retrieve study, series or instance query
  • To explore an alternative archive format that is web friendly (no need for a DICOM parser)
    • Should be able to recreate DICOM P10 file from this data (semantic equivalence or possibly bit for bit equivalent)
    • Will make updates more efficient (e.g. updating patient name just requires updating metadata)
    • More efficient image access (no need to scan through DICOM P10 to access frames, or store the offsets of each frame separately and seek to them)

The scope of this library is limited to:

  • Taking one or more DICOM files as input, writing Bulkdata and Pixeldata
  • Creating either deduplicated representations or instance level metadata
  • Combining deduplicated representations into sets of deduplicated representations
  • Using the deduplicated representations to generate study/series/instance query results and series metadata

View the design rationale for more information on the scope

View frequently asked questions

View specification

Pre-requisites

  • NodeJS (v14.18.1 used but earlier/later versions should also)
  • NPM

Running Unit Tests

npm run test

A visual studio code build task is also included so you can run it from there with "Terminal->Run Build Task" (Shift+Command+B)

Running the CLI

Install the CLI with either npm install -g . or npm install -g dicomp10-to-dicomweb

Api

There are 5 different commands: mkdicomweb, mkdicomwebinstances, mkdicomwebstudy, mkdicomwebdeduplicated, mkdicomwebdeduplicatedgroup that can be used (see usage bellow). It can be run as:

commnadName [options] [<input...>]

Arguments are:

input : List of directories/files separated by space. Required for commands: mkdicomweb, mkdicomwebdeduplicated, mkdicomwebinstances. Optional for the others. This argument should be placed at end of command invoking.

The options are:

-c, --clean : Clean the study output directory for these instances (default: false)

-C, --remove-deduplicated-instances : Remove single instance deduplicated files after writing group files (default: false)

-d, --deduplicate : Write deduplicate instance level data (default: false)

-g, --group : Write combined deduplicate data (default: true)

-i, --instances : Write instance metadata (default: false)

-m, --maximum-inline-public-length : Maximum length of public binary data (default: 131074)

-M, --maximum-inline-private-length : Maximum length of private binary data (default: 64)

--no-recompress : Force no recompression see

-o, --dir : Set output directory (default: "~/dicomweb")

--path-deduplicated : Set the deduplicate data directory path (relative to dir) (default: "deduplicated")

--path-instances : Set the instances directory path (relative to dir) (default: "instances")

-r, --recompress <listvalue...> : List of types to recompress separated by comma (choices: "uncompressed", "jp2", "jpeglossless", "rle", default: "uncompressed jp2") see

-s, --study : Write study metadata - on provided instances only (TO FIX}, (default: true)

-t, --content-type : Content type (default: null)

-T, --colour-content-type : Colour content type (default: null)

-v, --verbose : Write verbose output (default: false)

-V, --version : output the version number

Help command

You can output help information by:

commandName help
commandName -h
commandName --help

To create instances

Run the tool:

mkdicomwebinstances <directoryOfP10Files>

To create a full DICOMweb output structure

Run the tool:

mkdicomweb <directoryOfP10Files>

To run separated stages

The mkdicomweb tool runs the three stages all together, on just the studies references. This can instead be done on separate files by running:

mkdicomwebdeduplicated  <directoryOfP10Files>
mkdicomwebdeduplicatedgroup
mkdicomwebstudy

which creates a full study directory. The first stage writes to ~/dicomweb/instances// data about each instance as it is read in. The second stage then groups these files for more efficient compression into the ~/dicomweb/deduplicated// The last stage then creates the actual DICOMweb files.

There is currently no notification of what studies have been updated between stages. The intent is to write notifications to ~/dicomweb/notifications/ of what operations need to be applied/updated.

To Serve Instances As a Web Server

cd ~/dicomweb
npx http-server -p 5000 -g --cors

The -g option serves up compressed files ending in .gz as compressed http streams.

To Create DICOM part 10 from DICOMweb files

TODO

Run the tool mkdicomwebpart10 on the studyUID, and optionally on the series/instance UID's of interest to generate a local set of part 10 files.

To Update DICOM Metadata

TODO

Run the tool

mkdicomwebupdate -<delete/anonymize/patient/study/series/instance> <studyInstanceUID> (tag=newValue)* 

to delete the given item or to update the specified attribute contained in the given level. Multiple mkdicomwebupdate commands may be run to perform updates on different attribute sets, or they may be grouped into a single file for bulk application.

To recompress

It allows commands to recompress data/metadata prior writing it to local. Currently, if recompress is activate the designed data types will be transcoded see.

By default the recompression occurs for incoming types: uncompressed, jp2

mkdicomweb ./folderName

It will recompress any existing data that transfers syntaxes are 1.2.840.10008.1.2.4.90, 1.2.840.10008.1.2.4.91, 1.2.840.10008.1.2, 1.2.840.10008.1.2.1 and 1.2.840.10008.1.2.2.

Define incoming types for recompression of: uncompressed,jp2,rle

mkdicomweb  -r uncompressed jp2 rle ./folderName

It will recompress any existing data that transfers syntaxes: 1.2.840.10008.1.2.4.90, 1.2.840.10008.1.2.4.91, 1.2.840.10008.1.2, 1.2.840.10008.1.2.1, 1.2.840.10008.1.2.2 and 1.2.840.10008.1.2.5 to transfer syntax 1.2.840.10008.1.2.4.80.

Force no recompression

mkdicomweb  -r uncompressed jp2 rle --no-recompress ./folderName

It will NOT recompress any existing data but instead it will keep them in the original encoding.

Force no recompression

mkdicomweb --no-recompress ./folderName

It will NOT recompress any existing data but instead it will keep them in the original encoding.

Obs: --no-recompress forces no compression at all. Use it if you want to disable even the default behavior.

Recompress aliases and input transfer syntaxes

The table below shows the support for transfer syntax recompression(i.e use "... -r jp2" if you want to recompress from 1.2.840.10008.1.2.4.90 to 1.2.840.10008.1.2.4.80)

TransferSyntaxUID Name Recompress alias Target TransferSyntaxUID
1.2.840.10008.1.2 Implicit VR Endian uncompressed 1.2.840.10008.1.2
1.2.840.10008.1.2.1 Explicit VR Little Endian uncompressed 1.2.840.10008.1.2.1
1.2.840.10008.1.2.2 Explicit VR Big Endian uncompressed 1.2.840.10008.1.2.1
1.2.840.10008.1.2.4.57 JPEG Lossless, Nonhierarchical (Processes 14) jpeglossless 1.2.840.10008.1.2.4.80
1.2.840.10008.1.2.4.70 JPEG Lossless, Nonhierarchical, First- Order Prediction jpeglossless 1.2.840.10008.1.2.4.80
1.2.840.10008.1.2.4.90 JPEG 2000 Image Compression (Lossless Only) jp2 1.2.840.10008.1.2.4.80
1.2.840.10008.1.2.4.91 JPEG 2000 Image Compression jp2 1.2.840.10008.1.2.4.80
1.2.840.10008.1.2.5 RLE Lossless rle 1.2.840.10008.1.2.4.80

TODO (Looking for help here!!)

  • Create docker container to encapsulate build environment
  • Create test data (DICOM P10 and expected DICOMweb results) - Jordan working on this
  • Document API
  • Create CI + Publish to NPM
  • Fix Bugs
    • Get bulkdata refs written properly
  • Enhance cli
    • Add support for specifying bulkDataMinSize
    • Add support for writing out DICOM P10 file
  • Enhance library
    • Write out "info" file
      • P10 Header
      • Data needed to recreate original P10 instance
  • Create DICOMweb -> DICOM P10 tool

dicomp10-to-dicomweb-js's People

Contributors

chafey avatar ladeirarodolfo avatar wayfarer3130 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

dicomp10-to-dicomweb-js's Issues

DICOM dimse C-Store SCP service

A dimse service to handle storing a objects directly into DICOMweb static storage allows accessing the data via standard PACS services. Thus, this issue is to add a dimse SCP service support C-Store. The command should be dicomwebscp, and the first version of it just needs to support C-Store, without any support for other operations. Subsequent enhancements can add C-Get and C-Move.

mkdicomp10 command to convert to binary DICOM

There should be a mkdicomp10 command which takes a study UID (or study/series/sop) and convert the data back into DICOM part 10. This allows the web service to respond to requests for binary DICOM.
The java implementation of this relies on the URL for the DICOM object to contain position/offset values, so that the raw image data can be read without needing to parse it, although gzip compressed files still require a decompression step. Those indexes also allow streaming non-multipart responses if desired.

Clean up image frame/video handling

The image frame/video handling is kind of ugly because there are multiple components and naming dealing with writing that data. This should probably be rewritten to not use side affects for frame numbers and to directly generate the paths etc in a way that can be re-used but also intercepted to allow modifications to the data to be written.

Concurrent eventually consistent service

In order to ensure that the DICOMweb data is eventually consistent when there is more than one concurrent write, a consistency check needs to be performed until the study is consistent. This service needs to be created and used.
The logic is basically:

  1. Do some sort of study update
  2. Write the DICOMweb data, using the most recent deduplicated data
  3. Schedule a hash check for time + T

Time check is just: Compare the hash values of the currently written deduplicated data against the DICOMweb data. If the hash values don't match, then goto #2 above, otherwise study is consistent.

C-Find and DICOMweb studies query

There should be a way to query the DICOMweb server for what studies are available. The suggestion is to use MongoDB, either locally or in a cloud deployment to store the study query JSON objects, and then allow them to be queried in order to respond to the query objects. MAYBE the patient level query should also be done. Then, use this to add the dimse C-Find and a lambda service to perform a C-Find. The lambda service can be called by http-server running locally, and by deploying to a lambda service for cloud deployments.

Study QC Operations

When a DICOM QC operation arrives (split/segment/delete), the change should be applied to the study metadata object, and then new deduplicated data written to agree with the QC change.

Allow converting transfer syntax UID from LEI or JPEG2000 to JPEG-LS

When the images are being read from DICOM files which are encoded either in LEI or in JPEG-2000, allow converting them automatically to JPEG-LS depending on a flag being set. If it is easy to add, also allow conversion to JPEG Lossless.

Success criteria:

  1. Run mkdicomweb --convert lei ...dicom-files-in-lei...
  2. Output files ..../frames/1 should have image/jls as the content type
  3. Output files ..../instances.gz and .../metadata.gz should list just JPEG-LS as the transfer syntax (1.2.840.1.2.4.80)

Lambda service to query PACS C-Find for study/series query

There should be a service to allow querying a PACS in order to get study/series data. For the series level data, the query should auto-initiate a C-Move if the data isn't local, or isn't up to date. This C-Move should also be initiated for retrieves when the data isn't locally available or up to date.

Add write support to AWS S3 buckets

Allow the mkdicomweb etc tools to read/write AWS S3.
Files names .gz should be written without the .gz, and should have the compressed attribute added.
Files known to be json should have content type application/json added

Patient/Study query update

Sometimes the patient or studies get updated. This requires modifying the deduplicated data for affected studies and re-writing any indices. Also, sometimes studies come in with slightly different data for patient/query. The data for such studies should get recorded with the original information as stored, but in the end a single study object should be created that is relevant for all instances (last one in wins is probably the right solution here, but needs to be configurable).

Add service for image fetch from remote service

Instead of storing images locally, they can come from some remote image service. This vastly reduces the data storage time, and prevents needing to recompress image data. Then, on a request, both the http-server and the lambda services can just fetch from a remote system.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.