Command + stack trace:

In [17]: dscan([[theta], [delta]], [[-.1], [-.1]], [[.1], [.1]], [2, 4])
  File "build/bdist.linux-x86_64/egg/ophyd/userapi/scan_api.py", line 285, in run
    self.check_paths()
  File "build/bdist.linux-x86_64/egg/ophyd/userapi/scan_api.py", line 196, in check_paths
    pos.name, p.low_limit, p.high_limit))


Moving positioners back to start positions....... Done.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-17-8c61004e6277> in <module>()
----> 1 dscan([[theta], [delta]], [[-.1], [-.1]], [[.1], [.1]], [2, 4])

/home/xf23id1/anaconda/envs/ophyd/lib/python2.7/site-packages/ophyd-0.0.2-py2.7.egg/ophyd/userapi/scan_api.pyc in __call__(self, positioners, start, stop, npts, **kwargs)
    535         """
    536         self.setup_scan(positioners, start, stop, npts, **kwargs)
--> 537         self.run(**kwargs)
    538 
    539     def setup_scan(self, positioners, start, stop, npts, **kwargs):

/home/xf23id1/anaconda/envs/ophyd/lib/python2.7/site-packages/ophyd-0.0.2-py2.7.egg/ophyd/userapi/scan_api.pyc in run(self, **kwargs)
    283 
    284         with self:
--> 285             self.check_paths()
    286             self.setup_detectors(self.detectors)
    287             self.setup_triggers(self.triggers)

/home/xf23id1/anaconda/envs/ophyd/lib/python2.7/site-packages/ophyd-0.0.2-py2.7.egg/ophyd/userapi/scan_api.pyc in check_paths(self)
    194                     raise ValueError('Scan moves positioner {} \
    195                                      out of limits {},{}'.format(
--> 196                                      pos.name, p.low_limit, p.high_limit))
    197 
    198     def __enter__(self):

AttributeError: 'numpy.float64' object has no attribute 'low_limit'

It would be useful to start storing the data values that should be used to normalize detector columns during the ophyd scan. Something like normalize would work. I would assume that, for beamlines who need detector normalization, this should probably start getting dumped into the events automatically?

Here and here

... @stuwilkins ...

I think that there's a semantic problem with the detectors property in that it is intrinsically read-only. A user will be likely to attempt to add a new detector with, e.g., Count.detectors.append([my_detector]), rather than adding it to the user_detectors list. I offer up a few possible changes (in order of reversed personal preference):

1. move this property name to _detectors so that a user has some warning that you don't intend to have this changed outside of the class methods,
2. provide a setter that raises "AttributeError", indicating that appending to this list is not supported, or
3. provide a setter that transparently adds the provided detector list to the user_detectors list.

Dump all configuration in painful detail to the screen. The main purpose of this is for bug reporting/post mortem. I am thinking wh_pos + PV mapping would be enough.

data on this line https://github.com/NSLS-II/ophyd/blob/master/ophyd/userapi/scan_api.py#L307 is being returned as a dictionary of lists of value, timestamp lists. the stats estimation is pretty badly broken because of this

Non-blocking moves of Positioners (EpicsMotors, in particular) incorrectly report status as 'not-moving' immediately after the move is initiated.

Test (using sim motor m1):
m1.move(5, wait=False)
print m1.moving # prints false

Contributions to this problem include network (CA) delays, threading delays (context switching to CA monitor-handler thread), and EPICS device support reporting delays (i.e. poll rate of motor controller).

If an exception is encountered in the scan thread then this thread stops but it does not signal the parent thread to stop .... data collection sits in infinite loop.

S

When running user scripts, Ctrl-C does not abort and return to ipython prompt.

git repository: next branch

Need some experts' opinions on how this should be implemented properly.

Using pcaspy.

Need to come up with a clean interface to the above.

Moved channel access server functionality to a separate package (https://github.com/klauer/pypvserver).

Ok, putting this as an issue so that we can discuss look at it. Attn. @dchabot @gjwillms @tacaswell .

After some discussion with Mark Rivers and playing with the AD Driver for the FastCCD I have managed to work out the issue with timestamps, and blocking. So, Firstly if you need to use the stats plugins (for example) you need to set them to blocking $(P)$(R)BlockingCallbacks. This will ensure that the put complete doesn't come back until the stats have processed.

Now in areaDetector 2.0 there is support for the epics records having the detector timestamp by setting the "TSE" field of those records. I tested this with the fastCCD (I take the timestamp of the first packet arriving at the detector electronics) and it looks good, even at 100Hz. Also all stats plugins have the same timestamp.

Also, the new HDF5 file plugin stores this info (timestamps) in the HDF5 file.

The info is here:

areaDetector timestamps

Hi, @dchabot, @klauer,

I have been looking at 2 issues. 1 Time-stamping of data and secondly triggering detectors. Currently in the run_engine it is only possible to trigger a single EpicsSignal (it uses the put method). This doesn't allow the nice new detector classes to be used such as EpicsScaler or the area detector zoo.

I wonder if it is time to start thinking about adding a trigger() method to these type of detectors which could be called by the run_engine. I would also suggest at this time having a trigger_timestamp property which is the timestamp of the trigger. Even EpicsSignals could have this which would just do a self.put(1). Thoughts?

I am also starting to think about timestamps. It appears that we have 2 different type of detectors. Those that require triggering and have been done so in a step scan. We also have things that get added to the run which are asynchronous. It would appear that we need to distinguish between these two. I think that an event should always have a timestamp and contain only data which was at that timestamp, with the caveat that software triggers still group them together in one event descriptor.

I would propose the following:

We have 3 types of event:

1. In a step scan there is always a trigger event, taken from the system clock which is executed just before ophyd starts triggering and at the end of reading detectors. This event would look like:

trigger {
    start : (float of start from time.time())
    stop : (float of stop)
}

This is useful in data collection because it is the execution window of the step scan.

For detectors that are triggered, they are grouped into a single event descriptor which would look like:

detector_group_1 {
    pos1 : (value of motor)
    sclr_1 : (scalar channel 1)
    ...
    timestamp : (earliest trigger value)
}

The timestamp would be taken from the minimum value of the trigger timestamps (there is no good way to do this.

Lastly, EpicsSignals which are not triggered are recorded as separate events with there own timestamp:

temp_a {
    temp_a : (value)
    timestamp : (earliest trigger value)
}

I think this could be achieved with a little config. This pushes some work onto the data side but provides the truest representation of what actually happened without having each detector have a single event. I don't know if @tacaswell, @ericdill or @danielballan think this is what we discussed at length yesterday.

Also, this could be configurable, with the option to just record each detector as its own event as an advanced option.

While this may not be the long term goal of the run_engine. I wonder if this could be tested (I would be happy to implement a test branch) so that we can see how this runs and what the pitfalls on the data collection side are. Thoughts?

ascan does not stop or give an error when scanned motor hits a hard limit switch.

See: TerminalInteractiveShell.ask_exit

When a scan is running during an exit request, should it query to stop the motors involved? Halt the scan automatically? Do nothing?

Add procedural api for motion, including:

def mov(positioner, position)
''' Blocking, absolute move with position reporting while in progress. '''

def movr(positioner, delta)
''' Blocking, relative move with position reporting while in progress. '''

def set_limits(positioner, plim, mlim)
''' Set +ve/-ve soft-limits on a EpicsMotor object (using caput, etc?). '''

def set_position(positioner, position, offset=None)
''' Adjust EpicsMotor position (so-called User position). '''

etc, etc

DScan.pre_scan( ) method adds the relative positions to the absolute positions after Scan.check_paths( ) is called. Thus, check_paths( ) is actually trying to verify if the deltas are within the (low_limit, high_limit) of a Positioner.

Clear as mud...?

Demonstrated (safely!!!) at xf23id1 with:

 dscan(pgm_energy, -15, 30, 40)

Provide a means to systematically configure logging throughout Ophyd. The implementation should be independent of the SessionManager.

Can you please add one more layer of directories so the top-level directory of the repository can include things like setup.py, the conda recipe, and the sphinx files?

Something along the line of https://github.com/phobson/pytemplate

The mechanism for producing scan_ids needs to be investigated. As of now, the scan_id seems to reset to an old number on CSX. The behavior should be that the scan_id defaults to something like

   session_manager.scan_id = int(DataBroker[-1].scan_id) + 1

On ophyd startup go grab the last scan_id from DataBroker. This scan_id can be overridden by the user at run time.

Reported by Claudio at CSX

See Google style docs

(*) Well, groups of PVs that mimic motor record functionality.

ECLI's pseudomotor / soft motor implementation may be a good reference.

I think our objects should be only held in the session manager as weak references, otherwise we're holding onto objects that should be garbage collected.

For example, if you create a temporary EpicsSignal() while in a function, the signal will be registered with the session and then consequently never deleted (even after leaving the scope of the function).

I suggest that all positioners have the concept of limit. This could be the limits of a Motor Record or the DRVH/DRVL fields of an ao for example. Then when setting the tradj. this should raise an exception that we can catch in the scanning API. This stops starting a scan which will go outside positioner limits.

Personally, i would prefer doing this in the positioner and throwing an exception than checking in the scan. This way we can catch it on moves etc. also.

Thoughts?

S

Looking through the current master branch there is a lot of debug using print() I think this should all be removed. We should use either the logging (after all that is what it is for!) or when a PR is created to master this should be stripped.

@stuwilkins @dchabot @klauer
Gentlemen, I find myself in the midst of an existential crisis. Normally, this would not affect you--unless you were unfortunate enough to find yourselves in my immediate vicinity; however, I was, rather unfortunately, thinking about ophyd during its onset. As a result. I have a design question.

I believe that there is merit in considering beam line control the problem of interfacing a small number of discrete systems: e.g., the white beam slit, the horizontal offset mirror, the monochrometer. These are, of course, comprised of many individual axes of control, but I believe that we do ourselves a disservice by attempting to control the beam line this way; doing so is micromanagement of the worst sort. Said differently, we seem to be building, in ophyd, a rigidly functional approach to control on top of a flamboyantly object-oriented environment.

As an example, suppose I wish to know the horizontal position and size of a slit. In ophyd, I could accomplish this with "wh_pos([slt_pb_hcen,slt_pb_hsize])." Would it not be better to encapsulate these properties of a slit in the object, calling them forth as "slt_pb.hpos.wh()"?

Opinions?

Refer to:

Scan ID : 126

Detector | Value

em_ch1 [ 0.00915527]
em_ch2 [ 0.00944138]
em_ch3 [ 0.01058578]
em_ch4 [ 0.01058578]
pe1_acquire_period [ 5.]
pe1_acquire_time [ 0.2]
pe1_image_lightfield ['c9281f96-091c-4122-8860-d5af4a2ac887']
shctl1 [ 1.]

Could positioners and signals use PREC or make this available to the object? This would help a lot in formatting output

Yet another avenue for searching available to users.
Request/suggestion from Pavol.

Reminder that scalar data from SRX was coming in with the shape [2] where it should be None. I have not looked into why that is happening yet.

Also, Ascan and Dscan docstrings use scan in their examples, which is confusing.

Hi,

I think that @danielballan, @dchabot, @klauer and I should have a discussion on how to handle this both from a hardware point of view and also a data muxer / data portal point. I will try to set something up.

S

For doing things like set on a record in the user api the name of the record is needed.

Singleton impl and _registry handling (ties into issue #10, as well).

When doing long exposure "put complete" times out before end of frame (when counting 120s)
Should this be configurable?

For EpicsMotor when the object is created the property position is None while the RBV is contained in the value. After a move the position property correctly returns a position.

A missive from the trenches at XPD with @dchabot:

With PVPositioner instances, if a done PV is not provided, there is problem with iterating through a trajectory. We have a workaround in this instance, but we must revisit.

def insert_beamline_config(config_params, time, uid=None) will be the exposed API.