pcdshub / hutch-python Goto Github PK

Expected Behavior

hutch-python should handle the complete setup of the IPython environment. This includes setting the proper backend for matplotlib and setting it to interactive mode.

Current Behavior

Currently users must follow the instructions below in each IPython shell.
http://nsls-ii.github.io/bluesky/callbacks.html#aside-making-plots-update-live

Possible Solution

A combination of shell.enable_matplotlib and plt.ion works. This is obnoxious as it requires an import of matplotlib.

• provide a script to check out a development module and override a package

• Read from the cfg.yaml file to determine which path to load

Example entry:

lightpath:
  - MFX

My suggestion, after seeing that safe_load got stuffed into a size one namespace named safe with attribute load

• How to launch
• How to bug report
• What's new?
• Links

• Catch up to the current state of the module in the docs. After this, require all PRs to have doc updates pushed with them.
• Set up doctr for autobuilding and deploying the docs

Expected Behavior

with safe_load('dodgy_device', timeout=2.0):
    dodgy_device = MightFreeze('dodgy', name='wiggler')

Extension of safe_load for operations that may hang

At setup.py we need to specify a launch script that only takes a path to a config file. This will run the show.

• cli.py
• log_setup.py

For things like https://nsls-ii.github.io/bluesky/tutorial.html#save-some-typing-with-partial

• badges for Travis, codecov, landscape
• generally need to fix the text here

Expected Behavior

Allow users to use an existing OAuth token to post issues to GitHub.

def post_to_github(report, token='my_token')

This precludes the need for a CRON job, instead hutch_python will be able to post without knowledge of the GitHub user password.

Current Behavior

Possible Solution

Requires a username and password

Implement a way from a user to record what they are doing and save partial sessions to a file. This can be useful for designing new procedures and for reporting issues.

Provide a method to specify namespace groupings of devices in the cfg.yaml. This should be flexible enough to allow us to group objects by class, beamline location, and source file. This does not need to encompass small groups, but it should be easy to specify these in a hutch.py file.

User should have an easy way to pick objects from earlier load steps without accidentally creating an import loop.

The user should be able to easily specify additions to the environment without setting custom PYTHONPATH variables.

Expected Behavior

Add an elog object to the global namespace. Possible now that we have elog v0.1.0`

Current Behavior

No elog capabilities.

Expected Behavior

Tests should minimize their requirements.

Current Behavior

Tests needlessly import pcdsdevices. This is because pcdsdevices is included for a more macro-level package management reasons.

Possible Solution

Switch tests to group devices from arbitrary modules... literally any built-in. Come up with some other module to do the macro-level package management.

Expected Behavior

• Upon starting a session, we should be alerted if a newer version of the pcds env is available.
• During a session, if someone modifies the dev folder we should be alerted.

If two versions of a plugin get loaded, the second overwrites the first in the hutch_python.__init__ loaded namespaces.

• Select current experiment by specifying automatic in cfg.yaml
• Requires some work porting things to python 3. May be easy, may be hard, has not been evaluated.

Example entry:

experiment:
  - automatic

Expected Behavior

We often write an ELog post about the devices we have loaded in the Experiment folder as well as a brief description of any special scans and how to use them. It would be nice if all this could simply be written in the experiment file itself then published or at least built somewhere convenient.

Suggested by @koglin

Random idea I had. Would this see any use?

I imagine you could add something like

my_entry: my_path

Then from mfx.db import conf to get the full dictionary, or something like that

Port the scan PVs feature from the old hutch python in a sane way. This was a feature where at each step of the scan we would put the values and the step number to dummy EPICS pvs so that it is clear in the DAQ what the scan state is. This has been requested several times as a crucial feature.

This needs to be optional and configurable on the RunEngine independent of any individual plan, so that means it needs to either be a preprocessor or a callback. It's unclear to me which module this should live in.

Expected Behavior

There are components in the AreaDetector module that are not lazy. When we create the namespaces and call getattr on these they attempt to connect. This causes issues including a large start-up time while we wait for connections.

Current Behavior

2018-04-09 10:51:48 - PID 4477           utils.py: 47  safe_load          INFO     Loading default groups...
2018-04-09 10:51:48 - PID 4477       namespace.py: 36  class_namespace    DEBUG    Create class_namespace cls=EpicsMotor, scope=hutch_python.db
2018-04-09 10:52:03 - PID 4477           utils.py: 52  safe_load          ERROR    Failed to load default groups
2018-04-09 10:52:03 - PID 4477           utils.py: 53  safe_load          DEBUG    Failed to connect to MFX:DG2:P6740:IMAGE2:BayerPattern_RBV
Traceback (most recent call last):
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/hutch_python/utils.py", line 49, in safe_load
    yield
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/hutch_python/load_conf.py", line 258, in load_conf
    default_class_namespace('EpicsMotor', 'motors', cache)
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/hutch_python/load_conf.py", line 316, in default_class_namespace
    objs = class_namespace(cls, scope='hutch_python.db')
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/hutch_python/namespace.py", line 65, in class_namespace
    accumulate(obj, scope_objs, cache)
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/hutch_python/namespace.py", line 58, in accumulate
    accumulate(sub_obj, scope_objs, cache)
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/hutch_python/namespace.py", line 58, in accumulate
    accumulate(sub_obj, scope_objs, cache)
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/hutch_python/namespace.py", line 57, in accumulate
    sub_obj = getattr(obj, comp_name)
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/ophyd/device.py", line 176, in __get__
    instance._signals[self.attr] = self.create_component(instance)
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/ophyd/device.py", line 134, in create_component
    cpt_inst.wait_for_connection()
  File "/reg/g/pcds/pyps/conda/py36/envs/pcds-1.0.0/lib/python3.6/site-packages/ophyd/signal.py", line 474, in wait_for_connection
    self._read_pv.pvname)
TimeoutError: Failed to connect to MFX:DG2:P6740:IMAGE2:BayerPattern_RBV

Possible Solution

We should skip all non-lazy components while creating namespaces.

Expected Behavior

I am an idiot.

There are two hurdles to overcome in automated bug reporting to GitHub.

1.) The operator may not have a GitHub account
This solved by using pcdsbot and setting the password in the config file

2.) There is no access to outside internet on hutch machines
Automated bug reporting does not work if you can not get to GitHub. Therefore attempting to post an issue from a beamline IPython session will get you nowhere quickly.

I think this means we go back to the CRON job setup. 😢

We need to set this up to upload to the pcds-tag and pcds-dev channels.

Simple ways to decide whether or not to use the DAQ in a scan, quick ways to enable/disable/modify beam drop suspender.

Expected Behavior

ALL HANDS ON DECK! CRITICAL BUG! In reality this is slightly obnoxious.

Current Behavior

INFO     Selected default mfx daq platform: 4
            __      _____       _   _                 
           / _|    |  __ \     | | | |                
 _ __ ___ | |___  _| |__) |   _| |_| |__   ___  _ __  
| '_ ` _ \|  _\ \/ /  ___/ | | | __| '_ \ / _ \| '_ \ 
| | | | | | |  >  <| |   | |_| | |_| | | | (_) | | | |
|_| |_| |_|_| /_/\_\_|    \__, |\__|_| |_|\___/|_| |_|
                           __/ |                      
                          |___/                       

INFO     Loading daq...
SUCCESS  Successfully loaded daq

• Interface with happi to load devices
• Read from the cfg.yaml file to determine which beamlines, stands, etc. to load
• Start with simple key matching

Example entry:

happi:
  beamline:
    - MFX
    - HXD
    - FEE
  name:
    - CXI PulsePicker

Expected Behavior

Helper functions to report bugs store information including:

• environment
• logs
• command sequence
• operator notes

We need to be able to enable/disable suspenders on a per-plan basis.

• How to move a motor
• Lightpath demo
• DAQ and scans

As in our previous hutch python, we'd like a method to automatically log all exceptions that bubble up to the ipython layer, with context. This might be easy to adapt, but it should probably be improved.

Expected Behavior

• Class namespace should check subdevices and include them e.g. the default motors group should notice that your device has a motor on it, and include that.

See title

Expected Behavior

daq should work for all the hutches.

Current Behavior

daq will not work outside of mfx because we haven't defined the platform args

Possible Solution

Need to dig through the cnf files and pick out the platforms. We also need to think about what to do for the multidaq in cxi

• Load from hutch.py in the same directory as cfg.yaml, if specified in the config.
• Do not enforce any particular structure

Example entry:

hutch:
    - hutch.py

• chmod a+w so multi users are guaranteed to be able to rewrite it
• switch to db.txt and a virtual hutch.db module to remove possibility of corruption

Current Behavior

Currently if there is no experiment file an error is printed at startup.

  INFO     Loading experiment ls0816
  ERROR    Plugin experiment failed to load, skipping

Requested Behavior

Many experiments (at least thus far in MFX) do not need experiment files. They are content with just the devices within the happi database and the Questionnaire. This error message appears to freak scientists out. I think we should downgrade the not-exists error message, and only report a full exception if there is a file that exists that can not be imported properly.

Expected Behavior

Now that lightpath has a hardcoded configuration we can tell all the devices that need to be loaded quite easily. It would be nice if downstream beamlines are included as well, maybe contained within a namespace i.e

mfx_dg1_pim
xrt.hxx_valve

Current Behavior

Only devices on the beamline that match the hutch are included.

Password for GitHub Account %s:

• cfg.yaml
• ___python

Expected Behavior

There may be use cases where there is information that is needed by multiple Plugins i.e experiment number. In the current schema, this info would have to be duplicated in the YAML file.

Possible Solution

Add an extra class attribute to the BasePlugin called extraneous. This would point to other information in the YAML file that may not be underneath the Plugin explicitly. Then in load_conf.py the extraneous list is checked any additional information needed is added to the info dictionary directly.
For example

YAML file

experiment_no : LJ001

happi:
         ...

Happi Plugin

class Plugin
     name = 'happi'
     extraneous = ['experiment_no']

For MFX, I'm loading 111116.py instead of 11116.py

Perhaps include an https://github.com/slaclab/archapp object

• Load the experiment file associated with the current experiment
• Experiment number can be specified in cfg.yaml
• Do not specify a file structure

Example entry:

experiment:
  - mfx12345
  - mfx25423

Example experiment file:

from ophyd import Device
from cool_module import do_scan

class SpecialClass(Device):
    def cool_scan():
        return do_scan()

special_object = SpecialClass()
__all__ = ['special_object']

Current Behavior

In the event where no devices are found by happi, None is returned not a list. This returns a non-sensical error message. This should be caught by the Plugin and generate a more friendly log message.

Expected Behavior

I think it would be a neat feature to search the namespace by class. This could help operators find the devices they need / have available.

find_my(pcdsdevices.device_types.PIM)