bcda-aps / gemviz Goto Github PK

The DateTimeRangeSlider is limited to maximum possible date of: 2038-01-18 21:14:07 (UTC).

Attempts to set the maximum to later dates result in this exception:

    self.slider.setMaximum(self._slider_units(self._maximum))
OverflowError: argument 1 overflowed: value must be in the range -2147483648 to 2147483647

This limitation is imposed by using integer seconds with the slider (from the underlying Qt C++ library): https://github.com/BCDA-APS/tiled-viz2023/blob/fd52533d5aed91d29fd9c9e2b651f74477986255/gemviz23/demo/date_time_range_slider.py#L129

The QSlider uses an integer. The largest integer represents a timestamp resulting in the maximum possible date shown above.

Increase the range of the DateTimeRangeSlider widget by converting the slider to use integer days rather than integer seconds. A pair of conversion methods will help.

• step 1: table displays all the runs (dataset) for the selected catalog (inputBox1) from the filter gui
• step 2: table display a single run for the selected catalog (inputBox1) and selected scan_id (inputBox2) from the filter gui

          The new `_getKey()` method ...

can lead to some generalized specification of each of the data table items. A few items (time, detectors, motors) would need special handling. There's a pattern here...

Originally posted by @prjemian in #52 (comment)

In the resultswindow, when a cell of the table is clicked, the GUI should select the entire row. There is a setting that can be changed in the designer for the QTableView widget. Also, choose the setting for alternating colors for each row.

The present code uses the complete catalog from the server. Instead, return a catalog with the filter terms applied.

• return a filtered catalog (from the filterpanel.py module)
• disable status widget for now
• allow for multiple motors to be named
• configure the DateTimeSlider with info from the catalog

Following PEP8, apply naming conventions to the widgets built in the .ui files and also to the Python code we use.

There seems to be some question about where best to save the server object and the catalog object. Should it be saved in the resultswindow or the filterpanel?

Keeping an eye on the tiled releases, note that there are several (at least 5) new releases in the last week.

At least one of them requires a minimum server and client pair that are newer than we use now. Might cause us to change our code, either on the server side or the client side.

• BCDA-APS/tiled-template#2

Add some handling for app-specific settings (based on QtCore.QSettings). There is an example in the AssignGUP app. (Ugh, it's Py2 and PyQt4 so the code needs minor changes to work here.)

On example from listruns() function:

• replace spin box with QDateTimeEdit for dates
• replace drop down menus with input boxes
• explore tooltips to display hints for input boxes

From our discussion, this is what I understood:

This issue gathers feature requests from other repos which are related.

Browsing a catalog and got an error where there were no runs but at least one was expected:

catalogSelected: args = ('30id-nrs30',)  kwargs = {}
Traceback (most recent call last):
  File "/home/beams/JEMIAN/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/tiled/iterviews.py", line 60, in __getitem__
    (key,) = keys
ValueError: not enough values to unpack (expected 1, got 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/beams1/JEMIAN/Documents/projects/BCDA-APS/gemviz23/gemviz23/demo/filterpanel.py", line 53, in catalogSelected
    getStartTime(cat.keys().first()),
  File "/home/beams/JEMIAN/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/tiled/iterviews.py", line 18, in first
    return self[0]
  File "/home/beams/JEMIAN/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/tiled/iterviews.py", line 62, in __getitem__
    raise IndexError("Index out of range")
IndexError: Index out of range

The application quit completely.

Create a simple sketch of what we think the GUI should look like.

One reference is the dview software, for visualizing MDA file data. PyMCA & NeXpy are other examples but add additional complexity.

Try the UI client provided by tiled server. It's capable of a lot, it shows many features of the tiled server, but the human interface features are extremely limited. We can develop a list of improvements.

When a run is selected in the resultswindow, display a representation of its metadata in a tab of the visualization panel.

Needs a visualizationpanel ui/py pair of files. The panel needs to have tabs, at least for plot, data, and metadata.

Loaded a catalog with 88k runs. The first 20 runs displayed in the results table. After pressing the last pager button, the app crashed with this exception trace:

(bluesky_2023_2) jemian@otz ~/.../gemviz23/demo $ catalogSelected: args = ('developer',)  kwargs = {}
Displaying catalog: developer
catalogSelected: args = ('20idb_usaxs',)  kwargs = {}
Displaying catalog: 20idb_usaxs
action='next' kwargs={}
doPager action ='next', value =None
self.pageOffset() =20 self.pageSize() =20
action='last' kwargs={}
doPager action ='last', value =None
Traceback (most recent call last):
  File "/home/beams1/JEMIAN/Documents/projects/BCDA-APS/gemviz23/gemviz23/demo/resultwindow.py", line 186, in doPagerButtons
    model.doPager(action)
  File "/home/beams1/JEMIAN/Documents/projects/BCDA-APS/gemviz23/gemviz23/demo/resultwindow.py", line 83, in doPager
    self.setUidList(self._get_uidList())
  File "/home/beams1/JEMIAN/Documents/projects/BCDA-APS/gemviz23/gemviz23/demo/resultwindow.py", line 97, in _get_uidList
    return list(gen)
  File "/home/beams/JEMIAN/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/tiled/client/node.py", line 352, in _keys_slice
    content = self.context.get_json(
  File "/home/beams/JEMIAN/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/tiled/client/context.py", line 595, in get_json
    self.get_content(
  File "/home/beams/JEMIAN/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/tiled/client/context.py", line 557, in get_content
    handle_error(response)
  File "/home/beams/JEMIAN/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/tiled/client/utils.py", line 29, in handle_error
    response.raise_for_status()
  File "/home/beams/JEMIAN/.conda/envs/bluesky_2023_2/lib/python3.10/site-packages/httpx/_models.py", line 749, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'http://otz.xray.aps.anl.gov:8000/api/v1/node/search/20idb_usaxs?page%5Boffset%5D=88196&fields=&sort=time'
For more information check: https://httpstatuses.com/500

... so it loads as a QWidget, just like the resultswindow: https://github.com/BCDA-APS/tiled-viz2023/blob/99dd2c8ff18a441288ad4a37771d79a7a966d957/gemviz23/demo/mainwindow.py#L27-L33

• remove quotation and brackets from detector
• make scan_id text centered
• adjust column size
• add positionner to table content

Similar to Positioner search

When a catalog is first viewed in the table (resultswindow), the number of runs displayed is inconsistent with the QComboBox.

Move QAbstractTable Model into demo code

Examine the code implemented by @canismarko for APS 25ID. Is this a good starting point?

Review data visualization software from @strempfer. Will it be a starting point for the project here?

QtCore.QSplitter allows for two panels to be resized. Can be used for runs|visualization views

Ollivier does not appears to have the permission to create a project!

To view the results of the catalog filter (search) terms, we need to create a model (such as QtCore.QAbstractTableModel) of the search results. The QtWidgets.QTableView widget displays the table in the GUI.

Create a Python Qt5 application as a starting point.

Since #26, when the License window is shown, it cannot be closed directly since the About window is modal. Closing the About window closes the license.

Adjust the main window design to better match the storyboard design. Include consideration of a QtCore.QSplitter (as mentioned in #23) which allows the user to adjust the size balance of two adjacent panels.

Integrate the filter UI QWidget into the demo application

We'll want at least one tiled server for this work. Might help to have a couple. At least one catalog should provide area detector images.

Integrate the results table UI widget into the demo application

While browsing a catalog, the app crashed with this exception trace:

Traceback (most recent call last):
  File "/home/beams1/JEMIAN/Documents/projects/BCDA-APS/gemviz23/gemviz23/demo/resultwindow.py", line 51, in data
    return run.metadata.get("stop", {}).get("exit_status", "")
AttributeError: 'NoneType' object has no attribute 'get'

This particular run did not have a stop document.

Add a date/time slider and calendar pop-ups to the filter panel. Replace the existing date/time widget in the filter panel.

You can use Pete's function from the utils.py file. Here is an example:

import datetime
def iso2time(isotime):
    return datetime.datetime.timestamp(datetime.datetime.fromisoformat(isotime))

start_time_ISO8601 = "2023-03-17 00:30"
end_time_ISO8601 = "2023-05-19 15:15"

start_timestamp= iso2time(start_time)
end_timestamp=iso2time(end_time)

print(f"{start_time_ISO8601 = }")
print(f"{start_timestamp = }")

print(f"{end_time_ISO8601 = }")
print(f"{end_timestamp = }")

With #30 & #31 complete, it is time to integrate the steps to

1. choose a tiled server (already done in File -> Open)
2. try to connect with the server
3. find out what catalogs are available
4. handle user selection of a catalog

Anything to save in the settings?

The about box is not yet modal (it allows interaction with other parts of the application before the dialog is dismissed).

look at https://github.com/BCDA-APS/tiled-viz2023/blob/ce06789b0d824dd3c0244b6cd63e08ee5dc42b80/gemviz23/demo/resultwindow.py#L154-L162

The demo application is in the tiled-viz2023/gemviz23/demo folder. Re-arrange the repository from

tiled-viz2023/
    gemviz23/
        demo/
        test/
        ui_files/

to

tiled-viz2023/
    developer/
        test/
        ui_files/
    gemviz23/

where the demo application is in the tiled-viz2023/gemviz23 folder and the other two directories are relocated to the tiled-viz2023/developer folder