• Github actions or CircleCI?
• What is the process for deploying the docs @ https://dash.plotly.com?

The slicer needs to be able to overlay segmentation data (e.g. masks). Two suggestions:

slicer = VolumeSlice(volume, mask, ...)

# Suggestion 1: provide a mask shaped LxMxNx4 (it includes an alpha channel)
slicer.setOverlay(mask)

# Suggestion 2: provide a mask shaped LxMxN plus a colormap. 
slicer.setOverlay(mask, colormap=[(0, 0, 0, 0), (255, 0, 0, 100)])

With 1) the user has somewhat more flexibility, but the user would have to compose the mask, and the memory cost is high. With 2) we can still support multiple "layers", by allowing the mask to be integer, we can provide a default colormap or allow providing a single color to make single-layer masks easier.

Maybe there are other ideas?

We use the spread syntax (...) in a few places in this code. This means it's incompatible with all browsers except IE (https://caniuse.com/?search=spread%20operator). I personally think this is ok, since IE is being faced out anyway.

When you zoom into a plot, a large part of the image is cut off, even though these is plenty of space. Can we fix that?

Issue to track support to render the images with bilinear interpolation. Upstream: issue plotly/plotly.js#5353 and pr plotly/plotly.js#5353

Once fixed upstream, and the appropriate libs have a new release, we have (or can add) support for bilinear interpolation.

Human brain MRI volumes are generally organized in the R-A-S orientation in voxel space, i.e. moving along the volume axes moves through the "real world" image in Right-Left, Anterior-Posterior, Superior-Inferior directions respectively. And if the data were not oriented as expected, nilearn offers a way to try to get them there.

To display slices from these volumes in the expected display conventions (that differ somewhat between radiologists and neurologists): , we need to flip them by 90 degrees. Here is a short example to show this:

import numpy as np
from nilearn import datasets, plotting
import nibabel as nib
import matplotlib.pyplot as plt

# Get the commonly used MNI152 brain template
mni152 = datasets.fetch_icbm152_2009()
print(mni152.description.decode())

# Load the Nibabel image object
# And the volume as a numpy array
mni152_t1_img = nib.load(mni152.t1)
mni152_t1 = mni152_t1_img.get_fdata()

# Check that the data are in RAS orientation
nib.aff2axcodes(mni152_t1_img.affine)

# Display a slice
a_slice = mni152_t1[:, :, 90]
fig, axes = plt.subplots(1, 2)
axes[0].imshow(a_slice, cmap="gray")
axes[1].imshow(np.rot90(a_slice), cmap="gray")

This looks like so:

This "flip the slice by 90 degrees before displaying" is also what nilearn does in it's plotting tools:

# Note that nilearn displays slice indices in MNI/real-world coordinates
plotting.plot_anat(mni152_t1_img)

Since we aren't displaying static images of single slices, I think it'd be best to allow the user to ask us to flip the slices before displaying on the dash-slicer side of things.

Hi,
I am strugling to make dash-slicer work in the app.callback. Here is my code:

import dash
import dash_html_components as html
import imageio
from dash_slicer import VolumeSlicer

app = dash.Dash(__name__, update_title=None)

app.layout = html.Div(
    children=[
        html.Button('Randomize', id='random_button', n_clicks=0),
        html.Div(id='series_viewer')
    ]
)


@app.callback(
    dash.dependencies.Output('series_viewer', 'children'),
    dash.dependencies.Input('random_button', 'n_clicks')
)
def show(n_clicks):
    vol = imageio.volread("imageio:stent.npz")
    slicer = VolumeSlicer(app, vol)
    slicer.graph.config["scrollZoom"] = False
    return slicer.graph, slicer.slider, *slicer.stores


if __name__ == "__main__":
    app.run_server(debug=True, dev_tools_props_check=False)

What I am getting is:

Where there should be an image over the slider. Is there any way to make it work?

This code is based on the example from: Introduction to Dash Slicer

The Image trace is quite efficient when a base64-encoded png is used as the source. However, when the current implementation (of the Image trace in Plotly) needs extra steps (including per-pixel blitting) when the image is not aligned with the screen.

Possible solutions:

• Improve the canvas rendering itself (if possible): https://github.com/plotly/plotly.js/blob/master/src/traces/image/plot.js
• Maybe we can use glheatmap if our datatransfer can be made fast enough to not need base64-png. -> Nope, see #23

Rate limiting is introduced in #24 to improve the performance. Once plotly/dash-core-components#884 has landed and DCC has a new release, that approach could be considered instead. The UX differs a bit, but the code of this project would become simpler.

The spacing and origin arguments are used to allow the slices to be drawn in "scene space" rather than voxel space. This is a really useful feature. For example when displaying a brain volume, there is typically an affine matrix that translates voxel coordinates into some standard reference space (see e.g. here: https://nipy.org/nibabel/coordinate_systems.html).

One use case here would be to have a negative stepsize / spacing. I.e. walking along one volume axis would decrease the corresponding position in the standard / scene space. This is the case for example for some volume organizations of data in the MNI stereotaxic space, where the origin of the coordinate system (0, 0, 0) is by convention in the center of the brain (anterior commissure). Setting negative values for the origin works well, but at the moment, a negative spacing leads to odd behaviour of the slicers (outline traces are flipped outside, cross-linking positions doesn't seem to work correctly).
edit: cross-linking positions is working with the sliders but does not work correctly when clicking.

import dash
from jupyter_dash import JupyterDash
import dash_html_components as html
import dash_core_components as dcc
from dash.dependencies import Input, Output, State, ALL
from dash_slicer import VolumeSlicer

import imageio

app = JupyterDash(__name__)
server = app.server

vol = imageio.volread("imageio:stent.npz")
origin = [1000, -1000, 0]
spacing = [-1, 1, 1]

saggital_slicer = VolumeSlicer(app, vol, spacing=spacing, origin=origin, axis=0, scene_id="brain")
coronal_slicer = VolumeSlicer(app, vol, spacing=spacing, origin=origin, axis=1, scene_id="brain")
axial_slicer = VolumeSlicer(app, vol, spacing=spacing, origin=origin, axis=2, scene_id="brain")

slicer_list = []
for slicer in [saggital_slicer, coronal_slicer, axial_slicer]:
    slicer.graph.figure.update_layout(dragmode=False, plot_bgcolor="rgb(0, 0, 0)")
    slicer_list.append(html.Div([
        slicer.graph,
        html.Div(slicer.slider),#, style={"display": "none"}),
        *slicer.stores,
    ]))
    

style={
        "display": "grid",
        "gridTemplateColumns": "33% 33% 33%",
    }
app.layout = html.Div(style=style, children=slicer_list)

app.run_server(debug=True, dev_tools_props_check=False, mode="jupyterlab")

This is probably not a high priority feature but could be nice to have.

At the moment it's possible to clear the overlay data of a slicer by passing an array or zeros but it would be faster to return a list of None. This occurred to me when updating the chest X-ray app where I had to use this mask of zeros to clear the overlay data.

The image data is now greyscale. Being able to show it in e.g. Viridis would be nice. We should consider making it possible to create a colorbar in this case.

There are different ways to implement this. Ranging from leveraging numb, scikit-image, or Pillow. One very interesting approach is to produce paletted PNG's, with the palette being the colormap. This can be done via e.g. Pillow or pypng.

There are two parts where we deal with color:

• The slicers color property, used for the indicators. Supports anything compatible with CSS.
• The colormap for overlays. Currently supports RGB tuples, RGBA tuples, and hex colors.

Hey,

I am trying to have a simple app where I show the current slicer index positions in a datatable or similar element and allow for a user to change the slicer values from this datatable (this could also just be Input fields or something like NumericInput from dash-daq). My goal is something like this:

Here is the general strategy I am using:
Suppose I have three slicers:

slicer0 = VolumeSlicer(app, vol, axis=0)
slicer1 = VolumeSlicer(app, vol, axis=1)
slicer2 = VolumeSlicer(app, vol, axis=2)

then to

Write to slicer position:

1. Make store for slicer positions like described in the examples:

setpos_store = dcc.Store(
    id={"context": "app", "scene": slicer0.scene_id, "name": "setpos"}
)

2. Write new index positions to the store like so:

return x, y, z 
...
Output(setpos_store.id, "data")

Read from slicer position

This is more difficult. I cannot just listen to Input({"scene": slicer0.scene_id, "context": ALL, "name": "state"}, "data"), as described in the examples, because that gives me circular dependencies. As per the new devnotes, I tried listening to the slider values instead like so:

[
        Input(slicer0.slider.id, "value"),
        Input(slicer1.slider.id, "value"),
        Input(slicer2.slider.id, "value"),
    ],

However, this also results in a circular dependency problem (i.e. the values in the datatable both depend on and affect the slider positions).

A nice workaround proposed by @emmanuelle is to listen to the clickData events from the slicer.graph objects as the trigger, and have the slicer position info as state. This is possible. For example like so:

[
        Input(slicer0.graph.id, "clickData"),
        Input(slicer1.graph.id, "clickData"),
        Input(slicer2.graph.id, "clickData"),
    ],
    [
        State({"scene": slicer0.scene_id, "context": ALL, "name": "state"}, "data"),
    ],

This works, but the slicer position reported by the State here lags one user click event behind the true position. In other words: the coordinates I get in this way are from the position before the current click event. So also not ideal, particularly not if I want to use the coordinates as starting points to let the user make changes. Here is a brief example to show the latter problem:

import dash
import dash_html_components as html
import dash_core_components as dcc
from dash_slicer import VolumeSlicer
from dash.dependencies import Input, Output, State, ALL
import imageio
import dash_daq as daq

app = dash.Dash(__name__, update_title=None)
server = app.server

# Read volume
vol = imageio.volread("imageio:stent.npz")
vol = vol[::3, :, :]
spacing = 1, 1, 1
ori = 0, 0, 0

# Create slicer objects
slicer0 = VolumeSlicer(app, vol, spacing=spacing, origin=ori, axis=0, thumbnail=False)
slicer1 = VolumeSlicer(
    app, vol, spacing=spacing, origin=ori, axis=1, thumbnail=8, reverse_y=False
)
slicer2 = VolumeSlicer(app, vol, spacing=spacing, origin=ori, axis=2, color="#00ff99")

setpos_store = dcc.Store(
    id={"context": "app", "scene": slicer0.scene_id, "name": "setpos"}
)

app.layout = html.Div(style={
    "display": "grid",
    "gridTemplateColumns": "33% 33% 33%",
}, children=[
    html.Div(
        [
            slicer0.graph,
            html.Div(slicer0.slider, style={"display": "none"}),
            *slicer0.stores
        ]
    ),
    html.Div(
        [
            slicer1.graph,
            html.Div(slicer1.slider, style={"display": "none"}),
            *slicer1.stores
        ]
    ),
    html.Div(
        [
            slicer2.graph,
            html.Div(slicer2.slider, style={"display": "none"}),
            *slicer2.stores
        ]
    ),
    html.Pre(id="slicer_click_output"),
    html.Pre(id="slicer_direct_output"),
]
)


# Show the current slicer coordinates in the numeric input fields
@app.callback(
    Output("slicer_click_output", "children"),
    [
        Input(slicer0.graph.id, "clickData"),
        Input(slicer1.graph.id, "clickData"),
        Input(slicer2.graph.id, "clickData"),
    ],
    [
        State({"scene": slicer0.scene_id, "context": ALL, "name": "state"}, "data"),
    ],
    prevent_initial_call=True
)
def read_slicer_pos(fire1, fire2, fire3, slicer_index):
    if slicer_index is None or any([i is None for i in slicer_index]):
        return dash.no_update
    z_pos = slicer_index[0]["index"]
    y_pos = slicer_index[1]["index"]
    x_pos = slicer_index[2]["index"]
    return f"ClickEvent output from slicers\nx: {x_pos}\ny: {y_pos}\nz: {z_pos}"


@app.callback(
    Output("slicer_direct_output", "children"),
    Input({"scene": slicer0.scene_id, "context": ALL, "name": "state"}, "data"),
    prevent_initial_call=True
)
def write_coords(slicer_index):
    if slicer_index is None or any([i is None for i in slicer_index]):
        return dash.no_update
    z_pos = slicer_index[0]["index"]
    y_pos = slicer_index[1]["index"]
    x_pos = slicer_index[2]["index"]
    return f"Direct output from slicers\nx: {x_pos}\ny: {y_pos}\nz: {z_pos}"


if __name__ == "__main__":
    app.run_server(debug=True, dev_tools_props_check=False)

Summary

It's not clear (to me) how to both listen to and change through user input the slicer positions with the same interactive component (e.g. datatable or input field) in an app. It would probably be good to add an example that illustrates this.

To keep track of progress:

• id's defined must be made unique so one can use multiple slicers in one app.
• Figure out how to "connect" multiple views on the same data. #4
• Add indicators for other slices. #4
• Use image traces, see #2 (comment) #3
• Support anisotropic data and flipping axis'. #8
• callbacks are now defined before the layout, which is not supposed to work? -> I read it the wrong way.
• Make it possible to show low-res data as a placeholder. #3
• Look into that React error / warning. -> It seems to be gone :)
• Different behavior for slider drag and slider release:
  • Rate limiting is implemented in #24
  • plotly/dash-core-components#884
• Release
• Docs
• Allow indicators to be different colors? #29
• API to show extra traces in the figure. #40
• Support for setting contrast limits. #47
• Make it configurable what data is uploaded at the start (none, low-res, full-res). #34
• Colormaps. #49

Ideas maybe worth considering:

• More ways to control the slice index -> #11
• Define slice using a plane instead of an axis + index (needs interpolation with e.g. numba). -> probably out of scope - if going this way, keep the views orthogonal.

Tracking plotly/dash-core-components#879

In all the given examples, the volume and the slicer are global variables. It would be useful to be able to declare these variables in a callback. It would allow the user to select the file to vizualize during the session. The code below doesn't work: the slicer are well designed but the images remain completely blank.

  def layout(slicer0, slicer1, slicer2):

    layout = html.Div([
        html.Div(
            [
                html.Center(html.H3("Axial")),
                slicer0.graph,
                html.Br(),
                slicer0.slider,
                *slicer0.stores,
            ]
        ),
        html.Div(
            [
                html.Center(html.H3("Coronal")),
                slicer1.graph,
                html.Br(),
                slicer1.slider,
                *slicer1.stores,
            ]
        ),
        html.Div(
            [
                html.Center(html.H3("Sagittal")),
                slicer2.graph,
                html.Br(),
                slicer2.slider,
                *slicer2.stores,
            ]
        ),
    ],     style={
        "display": "grid",
        "gridTemplateColumns": "50% 50%",
    },
    )
    return layout


app = dash.Dash(
    __name__,
    meta_tags=[{
        "name": "viewport",
        "content": "width=device-width, initial-scale=1"
    }]
)

app.layout = html.Div([
        dbc.Button("Apply", id="display_ct"),
        html.Div(id="frame_ct_display"),
])

@app.callback(
    [Output("frame_ct_display", "children")],
    [Input("display_ct","n_clicks"),]
)
def display_ct(n_clicks):

    if n_clicks is None:
         raise PreventUpdate

    vol = imageio.volread("imageio:stent.npz")
    slicer0 = VolumeSlicer(app, vol, axis=0)
    slicer1 = VolumeSlicer(app, vol, axis=1)
    slicer2 = VolumeSlicer(app, vol, axis=2)

    return [layout(slicer0, slicer1, slicer2)]


if __name__ == '__main__':
    app.run_server(debug=True, dev_tools_props_check=False)

If you use dash-slicer with the current plotly.version, you get the following warning:

UserWarning: 
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html

Therefore, it would be important to adjust the imports accordingly.

Possible ways to control the slice index:

• A slider.
• Clicking in one view sets position of other slicers of the same scene.
• Scrolling with modifier?
• Keyboard shortcuts?
• Next/Previous buttons? -> Can be added by user if needed. We have an example that does this.
• Numeric input? -> tricky because of recursion. Not so much needed?

Is it possible to interactively annotate slices of a volume in dash-slicer?

In #24 I removed the (very rudimentary) caching mechanism from the code. To not lose the knowledge, here's some background and code.

Reasoning for removing the code (for now)

For caching to become useful it would have to be smarter than what we have now. Further, because we have the low-res slices, the need of caching is lower.

After we've implemented more features that affect the managing of data (like contrast limits), we should revisit whether the additional complexity of a caching mechanism is worthwhile.

Outline

To perform caching, we'd keep a per-slicer client-side dictionary (e.g. on the slicer_state object that #24 introduced). At the moment the server callback that serves slices uses the index.data as an input. We'd need an extra store req-index that will copy over the index.data, except when the cache has data for that index. Something like this:

        self._clientside_callback(
            """
        function update_req_index(index) {
            let slicer_state;  // filled in
            slicer_state.cache = slicer_state.cache || {};
            return slicer_state.cache[index] ? dash_clientside.no_update : index;
        }
        """,
            Output(self._req_index.id, "data"),
            [Input(self._index.id, "data")],
        )

Then when a new slice is received from the server, store the incoming slice:

        self._clientside_callback(
            """
        function update_image_traces(index, server_data, overlays, lowres, info, current_traces) {
            let slicer_state;  // filled in
            slicer_state.cache = slicer_state.cache || {};
            for (let trigger of dash_clientside.callback_context.triggered) {
                if (trigger.prop_id.indexOf('server-data') >= 0) {
                    slicer_state.cache[server_data.index] = server_data;
                    break;
                }
            }
    ...

This will restore more or less the caching mechanism that we had earlier. But more features should be added to make it useful, e.g. obtaining neighboring slices, and purging slices from the cache if needed to preserve memory.

Clicking on a point in one slicer (Ctrl-click or something?) would then update the index of all slicers with the same scene-id.

It would be nice to have a working example in the dash docs

This additional parameter would update the Graph.config for the Graph of the DashVolumeSlicer, which is a way to pass shape drawing tools to be added to the modebar.

Hi,
I am trying to apply a colormap to already segmented images, i.e. I want to be able to color the labels when I change the slice index but I dont know how to incorporate input argument "value" in my function or do I even need that since I based my code on the example threshold_overlay.py but I don't need the contrast limit.
This is the code snippet:

@app.callback(
    [
        Output(slicer1.overlay_data.id, "data")
    ],
    [Input("main-slider", "value")]
)
def apply_colors(value):
    print(value)
    mask = mask.astype(np.uint8)

    return slicer1.create_overlay_data(mask, colormap_gt)

I have three Slicer objects and I want on the two of those three to color the labels.
The thing is my mask is already defined, i.e. my segmented images with 8 possible values (from 1 through 8).

I would really appreciate if someone could help me with this.

When I run the app in the browser, it displays the error:

dash.exceptions.InvalidCallbackReturnValue: Invalid number of output values for ..slicer2-overlay.data...
Expected 1, got 139

But my mask is already the same shape as the volume.

[...] with the long id names, it's quite painful to look at the callback graph in the developer tools (which you can see with debug=True). This is not specific to this project, but it might be worth submitting a PR to Dash to represent long ids in a different format (see for example plotly/dash#1179 for the last modifications of the code generating the callback graph).

In #4, the id's where changed to be dict id's, but these are very long e.g. when shown in the callback graph debugger tool. Some options (both are probably a good idea):

• Somehow display such id's more efficiently in the callback graph.
• Use only dict-id's for objects that need it.