Event Descriptors#

In the Data Model section, we gave an overview of the document types. Here we present a more detailed discussion of the Event Descriptor.

Recall our example ‘event’ document.

# 'event' document
{'data':
    {'temperature': 5.0,
      'x_setpoint': 3.0,
      'x_readback': 3.05},
 'timestamps':
    {'temperature': 1442521007.9258342,
     'x_setpoint': 1442521007.5029348,
     'x_readback': 1442521007.5029348},
 'time': 1442521007.3438923,
 'seq_num': 1,
 'uid': '<randomly-generated unique ID>',
 'descriptor': '<reference to a descriptor document>'}

Typically, an experiment generates multiple event documents with the same data keys. For example, there might be ten sequential readings, generating ten event documents like the one above — with different readings and timestamps but identical data keys. All these events refer back to a ‘descriptor’ with metadata about the data keys and the configuration of the devices involved.

Note

We got the term “data keys” from event['data'].keys(). Again, in our example, the data keys are ['temperature', 'x_setpoint', 'x_readback']

Data Keys#

First, the descriptor provides metadata about each data key.

  • dtype — ‘number’, ‘string’, ‘array’, or ‘object’ (dict)

  • shape — None or a list of dimensions like [5, 5] for a 5x5 array

  • source — a description of the hardware that uniquely identifies it, such as an EPICS Process Variable

  • (optional) external — a string specifying where external data, such as a large image array, is stored

Arbitrary additional fields are allowed, such as precision or units. The RunEngine obtains this information from each device it sees by calling device.describe().

# excerpt of a 'descriptor' document
{'data_keys':
    {'temperature':
        {'dtype': 'number',
         'source': '<descriptive string>',
         'shape': [],
         'units': 'K',
         'precision': 3},
     'x_setpoint':
        {'dtype': 'number',
         'source': '<descriptive string>',
         'shape': [],
         'units': 'mm',
         'precision': 2},
     'x_readback':
        {'dtype': 'number',
         'source': '<descriptive string>',
         'shape': [],
         'units': 'mm',
         'precision': 2}},
 ...}

Object Keys#

The object_keys provide an association between each device and its data keys.

This is needed because a given device can produce multiple data keys. For example, suppose the x_readback and x_setpoint data keys in our example came from the same device, a motor named 'x'.

# excerpt of a 'descriptor' document
{'object_keys':
    {'x': ['x_setpoint', 'x_readback'],
     'temp_ctrl': ['temperature']},
 ...}

Specifically, it maps device.name to list(device.describe()).

Configuration#

Complex devices often have many parameters that do not need to be read anew with every data point. They are “configuration,” by which we mean they don’t typically change in the middle of a run. A detector’s exposure time is usually (but not always) in this category.

Devices delineate between the two by providing two different methods that the RunEngine can call: device.read() returns normals readings that are not considered configuration; device.read_configuration() returns the readings that are considered configuration.

The first time during a run that the RunEngine is told to read a device, it reads the device’s configuration also. The return value of device.describe_configuration() is recorded in configuration[device.name]['data_keys']. The return value of device.read_configuration() is collated into configuration[device.name]['data'] and configuration[device.name]['timestamps'].

In this example, x has one configuration data key, and temp_ctrl happens to provide no configuration information.

# excerpt of a 'descriptor' document
{'configuration':
    {'x':
       {'data': {'offset': 0.1},
        'timestamps': {'offset': 1442521007.534918},
        'data_keys':
           {'offset':
               {'dtype': 'number',
                'source': '<descriptive string>',
                'shape': [],
                'units': 'mm',
                'precision': 2}}},
     'temp_ctrl':
        {'data': {},
         'timestamps': {}
         'data_keys': {}}}
 ...}

Hints#

This is an experimental feature. Devices can provide information via a hints attribute that is stored here. See :ref:”hints”.

# excerpt of a 'descriptor' document
 {'hints':
    {'x' {'fields': ['x_readback']},
     'temp_ctrl': {'fields': ['temperature']}}
  ...}

Complete Sample#

Taken together, our example ‘descriptor’ document looks like this.

# complete 'descriptor' document
{'data_keys':
    {'temperature':
        {'dtype': 'number',
         'source': '<descriptive string>',
         'shape': [],
         'units': 'K',
         'precision': 3},
     'x_setpoint':
        {'dtype': 'number',
         'source': '<descriptive string>',
         'shape': [],
         'units': 'mm',
         'precision': 2}},
     'x_readback':
        {'dtype': 'number',
         'source': '<descriptive string>',
         'shape': [],
         'units': 'mm',
         'precision': 2}},

 'object_keys':
    {'x': ['x_setpoint', 'x_readback'],
     'temp_ctrl': ['temperature']},

 'configuration':
     {'x':
        {'data': {'offset': 0.1},
         'timestamps': {'offset': 1442521007.534918},
         'data_keys':
            {'offset':
                {'dtype': 'number',
                 'source': '<descriptive string>',
                 'shape': [],
                 'units': 'mm',
                 'precision': 2}
      'temp_ctrl':
        {'data': {},
         'timestamps': {}
         'data_keys': {}}}
     }

 'hints':
    {'x' {'fields': ['x_readback']},
     'temp_ctrl': {'fields': ['temperature']}}

 'time': 1442521007.3438923,
 'uid': '<randomly-generated unique ID>',
 'run_start': '<reference to the start document>'}