Annotating Bluesky Plans¶
Introduction¶
Queue Server is using information on existing plans and devices stored in
the file existing_plans_and_devices.yaml
for validating submitted plans.
The file can be generated using qserver-list-plans-devices CLI tool or
automatically created/updated by RE Manager (see Updating the List of Existing Plans and Devices for
available options).
Representations of plans generated by qserver-list-plans-devices
contain items
such as text descriptions of plans and plan parameters, parameter type annotations,
default values and ranges for numerical values. The representations are sufficient
to perform validation of parameters of submitted plans without access to startup
scripts or RE Worker namespace. Plan representations can also be downloaded by
client applications (‘plans_allowed’ 0MQ API) and used to validate
plan parameters before the plans are submitted to the server. For details on plan
parameter validation see Validation of Plans.
Client applications may also use items such as text descriptions, type annotations,
default values and ranges for generating or annotating user interfaces, such as
GUI forms for plan parameters.
In this manual, the elements of a plan (Python function) header, docstring and
the parameters of the optional parameter_annotation_decorator
that are used
for generating plan representations are referred as plan annotation.
All elements in plan annotations are optional. But properly annotating plans may be beneficial if features such as parameter validation or automated user interface generation are needed. For example in cases when users are manually entering plan parameter values in Qt or web forms, it is preferable to detect errors at the time when the plans are submitted to the queue and reject plans as opposed to waiting for plans to fail when they are sent for execution.
Note
Validation of plan parameters is performed each time a new or modified plan is submitted to Queue Server. Validation can be also performed on the client side before the plan is submitted. To run validation, the client must download the lists of allowed plans and devices (‘plans_allowed’ and :ref:`method_devices_allowed 0MQ API) and call validate_plan() (API for Plan Validation).
Plan annotation may contain the following (optional) elements:
Description of the plan: multiline text that describes the plan. The plan description may be displayed to users by client applications.
Descriptions for each parameter of the plan: multiline text that describes a plan parameter. Separate description is provided for each parameter. Parameter descriptions may be displayed to users by client applications.
Type annotations for each parameter. Parameter types are used for validation of plan parameters. The types may also be used by client applications for generating user interfaces.
Default values for each parameter. The parameters with defined default values are optional (following Python rules). The default values are used for parameter validation. The default values may also be used by client applications for generating user interfaces.
Minimum, maximum and step values for each numerical parameter. The optional minimum and maximum values define allowed range for numerical parameter values that is used in parameter validation. Step size is passed to client application and may be useful in generating user interfaces.
The elements of plan annotations are defined in the plan header (type hints and
default values) and the docstring (parameter descriptions). In addition, Queue Server
supports parameter_annotation_decorator
(Parameter Annotation Decorator),
which allows to define or override any annotation item. The decorator is optional
and should be used only when necessary.
Note
When a plan is executed in IPython environment, it behaves as a regular Python
generator function. Only the default values defined in the plan header are used.
Any elements defined in parameter_annotation_decorator
are ignored.
Plans Without Annotation (Default Behavior)¶
All elements of parameter annotations are optional. Plans without annotations can be successfully managed by Queue Server. Some of the elements, such as text descriptions of plans and plan parameters or step values for numerical parameters are not used by Queue Server, but may be downloaded and used by client applications. The other elements, such as parameter types and default values are used for plan parameter validation in Queue Server. All the elements may be downloaded and used by client applications.
Depending on whether plan annotation contains a default value for a parameter, the parameter is considered required or optional. Plans submitted to the queue must contain values for all required parameters. The default values are used for missing optional parameters.
For each plan parameter, annotation may contain optional type specification. All submitted parameter values undergo type validation. For parameter with type annotation, validation includes verification of the type of the submitted value based on specified parameter type. The parameters with no type annotations are treated according to the default rules:
Type checking always succeeds, i.e. any submitted value is accepted and passed to the plan. Plan execution may fail due to incorrect parameter type.
All strings found in the submitted parameter value (elements of lists, values of dictionaries, etc.) are matched against the lists of plans and devices allowed for the user submitting the plan. The matching strings are replaced by references to respective objects (plans or devices) from RE Worker namespace, all the other strings are passed as strings.
The validation algorithm is processing each parameter independently. Type validation is applied to the parameters with specified type annotation and default rules to the parameters without specified type.
The examples of the plans with no annotation:
def plan_demo1a(npts, delay):
# Parameters 'npts' and 'delay' accept values of any type.
# No type validation is performed on the parameter values.
# The plan may fail during execution if value is not accepted by the plan.
<code implementing the plan>
def plan_demo1b(npts, delay=1.0):
# Same as 'plan_demo1' except the default value for parameter 'delay' is
# specified, which makes the parameter 'delay' optional.
# No type validation is performed for any parameter.
<code implementing the plan>
Queue Server supports plans with parameters accepting references to devices
or other plans. The devices or plans passed as parameters must be defined
in startup scripts, loaded in RE Worker namespace and represented in the list
of existing devices (existing_plans_and_devices.yaml
). When submitting
plans to the queue, the devices and plans must be represented by their names
(type str
). The names are replaced by references to objects in RE Worker
namespace before the parameter values are passed to the plans for execution.
All submitted parameter values are parsed and each string found in the tree
formed by lists, tuples and dictionaries is replaced with a reference to
the matching object. If there is no object with the matching name found or
the name is not in the list of allowed plans or devices for the user
submitting the plan, then the string is not modified and passed directly
to the plan. If the parameter value contains dictionaries, the dictionary
keys are never modified by the algorithm.
The operation of replacing plan and device names with references to objects from RE Worker namespace is performed for each parameter with no type annotation. This means that every string that matches a name of a device, subdevice or a plan from the list of allowed devices or the list of allowed plans is replaced by the reference to the respective object from RE Worker namespace.
Let’s consider an example of a plan with parameter detectors
that is expected to
receive a list of detectors:
from ophyd.sim import det1, det2, det3
# Assume that the detectors 'det1', 'det2', 'det3' are included in the list
# of allowed devices for the user submitting the plan.
def plan_demo1c(detectors, npts):
# The parameter 'detectors' is expected to receive a list of detectors.
# There is no type annotation, so the type is not validated.
<code implementing the plan>
If the plan parameters submitted to the queue contain "detectors": ["det1", "det3"]
,
then the strings "det1"
and "det3"
are replaced with references to objects
det1
and det3
and the plan is executed as if it was called from IPython
using
RE(plan_demo1c([det1, det3], <value of npts>))
The default behavior, when Queue Server blindly attempts to convert each string found in each parameter to an object reference may works well in simple cases (especially in demos). In some applications it may be important to guarantee that strings are passed as strings regardless on whether the match is found. In those cases the conversion may be disabled for a given parameter by specifying the parameter type, e.g. using type hints in the plan header. For example, one may need to pass plan or device names to the plan:
import typing
from ophyd.sim import det1, det2, det3
# Assume that the detectors 'det1', 'det2', 'det3' are in the list
# of allowed devices for the user submitting the plan.
def plan_demo1d(detector_names, npts):
# The parameter 'detector_names' is expected to receive a list of detector names.
# DOES NOT WORK: references to objects are passed to the plan
<code implementing the plan>
def plan_demo1e(detector_names: typing.List[str], npts):
# The parameter 'detector_names' is expected to receive a list of detector names.
# WORKS: names of detectors are passed without change
<code implementing the plan>
If the value "detector_names": ["det1", "det3"]
is passed to the plan plan_demo1d
,
then the detector names are converted to references. Adding type hint for the parameter
detector_names
(see plan_demo1e
) disables string conversion and names are passed
to the plan unchanged. Adding type hint also enables type validation for parameter
detector_names
and the plan is going to be rejected by Queue Server if the submitted
value is not a list of strings. Type hint may be as restrictive as needed. For example,
type hint typing.Any
will still disable conversion of strings, but the server will accept
value of any type.
The operation of converting strings to objects never fails. If the device name is
incorrectly spelled or not in the list of allowed plans or devices, then the plan will
be added to the queue and sent for execution. Since the name is passed will be passed
to the plan as a string, the plan will likely fail and the queue is going to be stopped.
For example, assume that "detectors": ["det1", "det4"]
is passed to
plan_demo1c
. There is no device named det4
in the RE Worker namespace, so it will
not be converted to a reference. As a result, the plan will receive the value of
detectors=[det1, "det4"]
and fail during execution. Queue Server provides
parameter_annotation_decorator
(Parameter Annotation Decorator), which can be
used to define custom types for advanced parameter validation. In particular, the decorator
allows to define custom enums based on lists of device or plan names and
thus restrict sets of object names that that are accepted by the parameter. Setting
up custom enums with specified lists of plans or devices enables the string conversion,
but only the listed names will be converted to references:
from ophyd.sim import det1, det2, det3
# Assume that the detectors 'det1', 'det2', 'det3' are in the list
# of allowed devices for the user submitting the plan.
from bluesky_queueserver import parameter_annotation_decorator
@parameter_annotation_decorator({
"parameters": {
"detectors": {
"annotation": "typing.List[DevicesType1]",
"devices": {"DevicesType1": ["det1", "det2", "det3"]}
}
}
})
def plan_demo1f(detectors, npts):
# The parameter 'detector_names' is expected to receive a list of detector names.
<code implementing the plan>
The type annotation in the decorator overrides the type annotation in the function header.
Custom enums based on name lists are also used in type validation to guarantee that
only the device/plan names from the defined in the enum are accepted. For example,
if the submitted plan contains "detectors": ["det1", "det4"]
, then the plan
is rejected, because there is no detector det4
in the enum type DeviceType1
.
Note
Value of any type that is serializable to JSON can be passed to the plan if
the respective parameter type is not defined or defined as typing.Any
.
In the latter case the server does not attempt to convert strings to object
references.
Supported Types¶
Queue Server can process limited number of types used in type annotations and default values.
If a plan header contains parameter unsupported type hint, Queue Server ignores the hint
and the plan is processed as if the parameter contained no type annotation. If unsupported
type annotation is defined in parameter_annotation_decorator
, then processing of the plan
fails and existing_plans_and_devices.yaml
can not be generated. The processing also fails
if the default value defined in the plan header or in the decorator has unsupported type.
Note
Type annotations and default values defined in parameter_annotation_decorator
override type
annotations and default values defined in the plan header. If type or default value is defined
in the decorator, the respective type and default value from the header are not analyzed.
If it is necessary to define a plan parameter with unsupported type hint or default value
in the header, use parameter_annotation_decorator
to override the type or the default
value in order for the plan processing to work.
Supported types for type annotations. Type annotations may be native Python types
(such as int
, float
, str
, etc.), NoneType
, or generic types that are based
on native Python types (such as list[int]
, typing.List[typing.Union[int, str]]
).
Technically the type will be accepted if the operation of recreating the type object
from its string representation using eval
function is successful with the namespace
that contains imported typing
and collections.abc
modules and NoneType
type.
The server can recognize and properly handle the following types used in the plan headers
(see Defining Types in Plan Header and Parameter Types):
bluesky.protocols.Readable
(replaced by__READABLE__
built-in type);bluesky.protocols.Movable
(replaced by__MOVABLE__
built-in type);bluesky.protocols.Flyable
(replaced by__FLYABLE__
built-in type);bluesky.protocols.Configurable
(replaced by__DEVICE__
built-in type);bluesky.protocols.Triggerable
(replaced by__DEVICE__
built-in type);bluesky.protocols.Locatable
(replaced by__DEVICE__
built-in type);bluesky.protocols.Stageable
(replaced by__DEVICE__
built-in type);bluesky.protocols.Pausable
(replaced by__DEVICE__
built-in type);bluesky.protocols.Stoppable
(replaced by__DEVICE__
built-in type);bluesky.protocols.Subscribable
(replaced by__DEVICE__
built-in type);bluesky.protocols.Checkable
(replaced by__DEVICE__
built-in type);collections.abc.Callable
(replaced by__CALLABLE__
built-in type);typing.Callable
(replaced by__CALLABLE__
built-in type).
Note
Note, that typing.Iterable
can be used with the types listed above with certain restrictions.
If a parameter is annotated as typing.Iterable[bluesky.protocols.Readable]
, then the validation
will succeed for a list of devices (names of devices), but fails if a single device name is
passed to a plan. If a parameter is expected to accept a single device or a list (iterable) of devices,
the parameter should be annotated as
typing.Union[bluesky.protocols.Readable, typing.Iterable[bluesky.protocols.Readable]]
.
Validation will fail for a single device if the order of types in the union is reversed.
Supported types of default values. The default values can be objects of native Python
types and literal expressions with objects of native Python types. The default value should
be reconstructable with ast.literal_eval()
, i.e. for the default value vdefault
,
the operation ast.literal_eval(f"{vdefault!r}")
should complete successfully.
The following is an example of a plan with type annotation that discarded by Queue Server.
The type annotation is defined in the plan header, so it is ignored and parameter detector
is viewed as having no type annotation.
from ophyd import Device
def plan_demo2a(detector: Device, npts=10):
# Type 'Device' is not recognized by Queue Server, because it is imported
# from an external module. Type annotation is ignored by Queue Server.
# Use 'parameter_annotation_decorator' to override annotation for
# 'detector' if type validation is needed.
<code implementing the plan>
In the following example, the type of the default value of the detector
parameter is not
supported and processing of the plan fails. The issue can be fixed by overriding the default value
using parameter_annotation_decorator
(Parameter Annotation Decorator).
from ophyd.sim import det1
def plan_demo2b(detector=det1, npts=10):
# Default value 'det1' can not be used with the Queue Server.
# Fix: use 'parameter annotation decorator to override the default value.
<code implementing the plan>
Defining Types in Plan Header¶
Signatures of plans from RE Worker namespace are analyzed each time the list of existing plans
is generated (e.g. by qserver-list-plans-devices
tool). If a plan signature contains type
hints, the processing algorithm verify if the types are supported and saves their string
representations. Unsupported types are ignored and the respective parameters are treated
as having no type hints (unless type annotations for those parameters are defined in
parameter_annotation_decorator
).
Note
Queue Server ignores type hints defined in the plan signature for parameters that have
type annotations defined in parameter_annotation_decorator
.
The acceptable types include Python base types, NoneType
and imports from typing
and collections.abc
modules (see Supported Types). Following are the examples
of plans with type hints:
import typing
from typing import List, Optional
def plan_demo3a(detector, name: str, npts: int, delay: float=1.0):
# Type of 'detector' is not defined, therefore Queue Server will find and attempt to
# replace all strings passed to this parameter by references to objects in
# RE Worker namespace. Specifying a type hint for the ``detector`` parameter
# would disable the automatic string conversion.
<code implementing the plan>
def plan_demo3b(positions: typing.Union[typing.List[float], None]=None):
# Generic type using the 'typing' module. Setting default value to 'None'.
<code implementing the plan>
def plan_demo3c(positions: Optional[List[float]]=None):
# This example is precisely identical to the previous example. Both hints are
# converted to 'typing.Union[typing.List[float], NoneType]' and
# correctly processed by the Queue Server.
<code implementing the plan>
The server can process the annotations containing Bluesky protocols such as
bluesky.protocols.Readable
, `bluesky.protocols.Movable
and bluesky.protocols.Flyable
and callable types collections.abc.Callable
and typing.Callable
with or without type parameters.
Those types are replaced with __READABLE__
, __MOVABLE__
, __FLYABLE__
and __CALLABLE__
built-in types respectively. See the details on built-in types in
Parameter Types.
Defining Default Values in Plan Header¶
Follow Python syntax guidelines for defining default values. The type of
the default value must be supported by the Queue Server (see Supported Types).
If the default value in the plan header must have unsupported type, override it by
specifying the default value of supported type in parameter_annotation_decorator
.
Note
If the default value is defined in the parameter_annotation_decorator
,
Queue Server ignores the default value defined in the header. Processing of the plan
fails if the default value for a parameter is defined in the decorator, but
missing in the function header. (A default value in the header is required
if the default value is defined in the decorator.)
Parameter Descriptions in Docstring¶
Queue Server collects text descriptions of the plan and parameters from NumPy-style docstrings. Type information specified in docstrings is ignored. The example below shows a plan with a docstring:
def plan_demo4a(detector, name, npts, delay=1.0):
"""
This is the description of the plan that could be passed
to the client and displayed to users.
Parameters
----------
detector : ophyd.Device
The detector (Ophyd device). Space is REQUIRED before
and after ':' that separates the parameter name and
type. Type information is ignored.
name
Name of the experiment. Type is optional. Queue Server
will still successfully process the docstring.
Documenting types of all parameters is recommended
practice.
npts : int
Number of experimental points.
delay : float
Dwell time.
"""
<code implementing the plan>
Parameter Annotation Decorator¶
The parameter_annotation_decorator
(Plan Annotation API) allows to override
any annotation item of the plan, including text descriptions of the plan and parameters,
parameter type annotations and default values. The decorator can be used to define all
annotation items of a plan, but it is generally advised that it is used only when
absolutely necessary.
Note
If the default value of a parameter is defined in the decorator, the parameter must have a default value defined in the header. The default values in the decorator and the header do not have to match. See the use case in notes.
Plan and Parameter Descriptions¶
Text descriptions of plans and parameters are not used by Queue Server and do not affect processing of plans. In some applications it may be desirable to have different versions of text descriptions for documentation (e.g. technical description) and for user interface (e.g. instructions on how to use plans remotely). The decorator allows to override plan and/or parameter descriptions extracted from the docstring. In this case the descriptions defined in the decorator are displayed to the user.
All parameters in parameter_annotation_decorator are optional. In the following example, the description for the parameter npts is not overridden in the decorator:
from bluesky_queueserver import parameter_annotation_decorator
@parameter_annotation_decorator({
"description": "Plan description displayed to users.",
"parameters": {
"detector": {
"description":
"Description of the parameter 'detector'\n" \
"displayed to Queue Server users",
}
"name": {
"description":
"Description of the parameter 'name'\n" \
"displayed to Queue Server users",
}
}
})
def plan_demo4a(detector, name, npts):
"""
Plan description, which is part of documentation.
It is not visible to Queue Server users.
Parameters
----------
detector : ophyd.Device
The detector. Technical description,
not visible to Queue Server users.
name
Name of the experiment. Technical description,
not visible to Queue Server users.
npts : int
Number of experimental points.
Description remains visible to Queue Server users,
because it is not overridden by the decorator.
"""
<code implementing the plan>
Parameter Types¶
Parameter type hints defined in a plan header can be overridden in
parameter_annotation_decorator
. The type annotations defined in the decorator
do not influence execution of plans in Python. Overriding types should be avoided
whenever possible.
Note
Types in the decorator must be represented as string literals. E.g. "str"
represents string type, "typing.List[int]"
represents an array
of integers, etc. Module name typing
must be explictly used when
defining generic types in the decorator.
Type annotations defined in the decorator may be used to override unsupported type hints in plan headers. But the main application of the decorator is to define custom enum types based on lists of names of plans and devices or string literals. Support for custom enum types is integrated in functionality of Queue Server, including the functionality such as type validation and string conversion. If a parameter type defined in the annotation decorator is using on a custom enum types, which are based on lists of plans or devices, then all strings passed to the parameter that match the names of plans and devices in enum definition are converted to references to plans and devices in RE Worker namespace. The lists of names of plans and devices or string literals may also be used by client applications to generate user interfaces (e.g. populate combo boxes for selecting device names).
from typing import List
from ophyd import Device
from ophyd.sim import det1, det2, det3, det4, det5
from bluesky_queueserver import parameter_annotation_decorator
@parameter_annotation_decorator({
"parameters": {
"detector": {
# 'DetectorType1' is the type name (should be a valid Python name)
"annotation": "DetectorType1",
# 'DetectorType1' is defined as custom enum with string values
# 'det1', 'det2' and 'det3'
"devices": {"DetectorType1": ["det1", "det2", "det3"]},
}
}
})
def plan_demo5a(detector, npts: int, delay: float=1.0):
# Type hint for the parameter 'detector' in the header is not required.
# Queue Server accepts the plan if 'detector' parameter value is
# a string with values 'det1', 'det2' or 'det3'. The string is
# replaced with the respective reference before the plan is executed.
# Plan validation fails if the parameter value is not in the set.
<code implementing the plan>
@parameter_annotation_decorator({
"parameters": {
"detectors": {
# Note that type definition is a string !!!
# Type names 'DetectorType1' and 'DetectorType2' are defined
# only for this parameter. The types with the same names
# may be defined differently for the other parameters
# of the plan if necessary, but doing so is not recommended.
"annotation": "typing.Union[typing.List[DetectorType1]" \
"typing.List[DetectorType2]]",
"devices": {"DetectorType1": ["det1", "det2", "det3"],
"DetectorType2": ["det1", "det4", "det5"]},
}
}
})
def plan_demo5b(detectors: List[Device], npts: int, delay: float=1.0):
# Type hint contains correct Python type that will be passed to the parameter
# before execution.
# Queue Server accepta the plan if 'detectors' is a list of strings
# from any of the two sets. E.g. ['det1', 'det3'] or ['det4', 'det5']
# are accepted but ['det2', 'det4'] is rejected (because the
# detectors belong to different lists).
<code implementing the plan>
Similar syntax may be used to define custom enum types for plans (use "plans"
dictionary key
instead of "devices"
) or string literals (use "enums"
dictionary key). The strings listed
as "devices"
are converted to references to devices and the strings listed as "plans"
are converted to references to plans before plan execution. Strings listed under "enums"
are not converted to references, but are still used for plan parameter validation.
Mixing devices, plans and enums in one type definition is possible (Queue Server will handle
the types correctly), but not recommended.
The lists of plan and device may contain a mix of explicitly listed plan/device names and regular expressions used to select plans and devices. See Lists of Device and Plan Names for detailed reference to writing lists of devices and plans.
The decorator supports the following built-in types: __PLAN__
, __DEVICE__
,
__READABLE__
, __MOVABLE__
, __FLYABLE__
, __PLAN_OR_DEVICE__
and
__CALLABLE__
. The types __DEVICES__
, __READABLE__
, __MOVABLE__
and __FLYABLE__
are treated identically by the server, but additional type checks could be added in the future.
The built-in types are replaced by str
for type validation and
conversion of plan and/or device names enabled for this parameter. No plan/device lists
are generated and plan/device name is not validated. The built-in types should not be
defined in devices
, plans
or enum
sections of the parameter annotation, since
it is going to be treated as a regular custom enum type. The __CALLABLE__
type is
treated similary to the other built-in types during parameter validation. The strings
passed as the parameter values and representing names of python variables or class object
attributes are converted to references to the respective objects from the worker namespace.
from ophyd.sim import det1, det2, det3, det4
from bluesky_queueserver import parameter_annotation_decorator
@parameter_annotation_decorator({
"parameters": {
"detectors": {
# '__DEVICE__' is the built-in type. The plan will accept a list of
# object names (strings), validate the parameter type and attempt to
# convert all string to device objects (not to plans).
"annotation": "typing.List[__DEVICE__]",
# If '__DEVICE__' is explicitly defined in the 'devices' section,
# it will be treated as a custom enum type(only for this parameter).
}
}
})
def plan_demo5c(detectors, npts: int, delay: float=1.0):
<code implementing the plan>
The lists of custom enum types for devices or plans may include any device or plan names defined in startup scripts and loaded into RE Worker namespace. The type definitions are saved as part of plan representations in the list of existing plans. If built-in enum types are used, the definitions will contain full lists of devices from the namespace. When lists of allowed plans are generated for user groups, custom type definitions are filtered based on user group permissions, so that only the devices and plans that are allowed for the user group remain. This allows to use entries from downloaded lists of allowed plans for validation of plans and for generation of user interfaces directly, without verification user permissions, since it is guaranteed, that the type definitions contain only devices and plans that the current user is allowed to use. Filtering type definitions may cause some lists to become empty in case current user does not have permission to use any devices or plans that are listed in type definition.
Explicitly Enabling/Disabling Conversion of Plan and Device Names¶
Parameter annotation allows to specify explicitly whether strings passed to this parameter
are converted to plan or device objects. Optional boolean parameters convert_device_names
and convert_plan_names
override any default behavior. If those parameters are not
specified, then Queue Server determines whether to convert names to objects based on
parameter annotation defined in plan header and parameter_annotation_decorator
:
from ophyd.sim import det1, det2, det3, det4
from bluesky_queueserver import parameter_annotation_decorator
@parameter_annotation_decorator({
"parameters": {
"dets_1": {
"annotation": "typing.List[str]",
# Queue Server attempts to convert each string to a device
# or subdevice from the list of allowed devices.
"convert_device_names": True,
}
"dets_2": {
"annotation": "typing.List[__DEVICE__]",
# The device names are not converted to device objects and
# passed to the plan as strings.
"convert_device_names": False,
}
"dets_3": {
# Device names are going to be converted to device objects.
"annotation": "typing.List[__DEVICE__]",
}
}
})
def plan_demo5d(detectors, npts: int, delay: float=1.0):
<code implementing the plan>
Note
Parameters convert_plan_names
and convert_device_names
control only
conversion of plan and device names to objects from the worker namespace
and have no effect on the process of validation of plan parameters.
Default Values¶
Using decorator to override default values defined in plan header with different values is possible, but generally not recommended unless absolutely necessary. Overriding the default value is justified when the type of the default value defined in the header is not supported, and a different default value of supported type can be defined in the decorator so that the plan will behave identically when it is executed in Queue Server or IPython environment.
Note
The default value defined in the decorator must be a Python expression resulting in the value that satisfy requirements in Supported Types (same requirements as for the default values defined in plan header). For the custom enumerated types, the default must be one of the valid strings values.
The following example illustrates the use case which requires overriding the default value.
In this example, the default value for the parameter detector
is a reference to det1
,
which has unsupported type (ophyd.Device
). When submitting the plan to the queue,
the default parameter value must be the string literal "det1"
, which is then substituted
by reference to det1
. The decorator contains the definition of custom enum type based
on the list of supported device names and sets the default value as a string representing
the name of one of the supported devices.
from ophyd.sim import det1, det2, det3
from bluesky_queueserver import parameter_annotation_decorator
@parameter_annotation_decorator({
"parameters": {
"detector": {
"annotation": "DetectorType1",
"devices": {"DetectorType1": ["det1", "det2", "det3"]},
"default": "det1",
}
}
})
def plan_demo6a(detector=det1, npts: int, delay: float=1.0):
# The default value for the parameter 'detector' is a reference to 'det1'
# when the plan is started from IPython. If the plan is submitted to
# the queue and no value is provided for the parameter 'detector', then
# the parameter is going to be set to string literal value '"det1"',
# which is then substituted with the reference to the detector 'det1'
# before the plan is executed.
<code implementing the plan>
Minimum, Maximum and Step Values¶
The decorator allows to define optional parameters for numeric values passed to plans, including minimum and maxumum values and step size. The minimum and maximum values determine the allowed range of numerical values used in parameter validation. Step size is not used by Queue Server and intended for generating user interfaces in client applications (e.g. combination of minimum, maximum values and step size may be used to set up a spin box). If maximum and/or minimum values are defined for a parameter, validation includes checking if each numerical value in the data structure passed to the parameter is within this range. The algorithm is searching the data structure for numerical values by iterating through list elements and dictionary values. Non-numeric values are ignored. Dictionary keys are not validated.
Setting both minimum and maximum values defines closed range for the parameter value
(including the range boundaries). If only maximum or minimum boundary is set, the range
is limited only from above or below (assuming the missing maximum or minimum
is -Inf
or Inf
respectively). If no minimum or maximum value is specified,
then the range is not validated.
Note
Minimum, maximum values and step size must be integer or floating point numbers.
@parameter_annotation_decorator({
"parameters": {
"v": {
"default": 50,
"min": 20,
"max": 99.9,
"step": 0.1,
}
}
})
def plan_demo7a(v):
<code implementing the plan>
This example defines the range [20, 99.9]
for the parameter v
. The plan is accepted
by Queue Server in the following cases:
{"v": 30}
{"v": [20, 20.001, 20.002]}
{"v": {"a": 30, "b": [50.5, 90.4]}}
The plan will be rejected if
{"v": 10} # Value 10 is out of range
{"v": [20, 100.5, 90]} # Value 100.5 is out of range
{"v": {"a": -2, "b": 80}} # Value -2 is out of range
{"v": {"a": 30, "b": [50.5, 190.4]}} # Value 190.4 is out of range
Lists of Device and Plan Names¶
Annotation of a parameter may contain optional devices
and/or plans
sections, which contains lists of device or plan names that could be passed
with the parameter or patterns based on regular expressions used to pick
matching devices or plans from the list of existing devices or plans.
The listed or automatically picked names are converted to objects exisiting
in Run Engine Worker namespace before being passed to the plan. In the
following example there are two lists (Type1
and Type2
) defined in
the devices
section. Names (keys) of the lists are used as types in
the string that defines the parameter type:
@parameter_annotation_decorator({
"parameters": {
"dets": {
"annotation": "typing.Union[typing.List[Type1]" \
"typing.List[Type2]]",
# "det1", "det2", "det4" are explicitly stated names
# ":det3:?val$", ":^det5$" are patterns
"devices": {"Type1": ["det1", "det2", ":det3:?val$"],
"Type2": ["det1", "det4", ":^det5$"]},
}
}
})
def plan_demo8a(dets, npts: int, delay: float=1.0):
<code implementing the plan>
If a plan accepts reference to other plans defined in the worker
namespace, one of the plan parameters may have a similar section
plans
with lists of plan names.
Each list may contain a mix of device name and patterns. The processing algorithm is searching for names of existing plans and devices based on the patterns and adds them to the lists. After removing Duplicate names, the lists are sorted in alphabetical order and saved as part of plan representation in the list of existing plans.
Lists of Device Names¶
The elements of lists of device names may include:
names of devices defined in the global scope of the startup script, e.g.
"det1"
,sim_stage_A
, etc.names of subdevices of devices defined in the global scope of startup script, e.g.
det1.val
,sim_stageA.mtrs.y
, etc.patterns based on regular expressions for picking devices and subdevices from the list of existing devices.
The explicitly listed device and subdevice names are added to the parameter annotation in the list of existing plans even if such device or plan does not exist in the worker environment. When plan represenations are preprocessed before being added to a list of allowed plans, the devices/subdevices/plans that are not in the list of allowed devices/plans are removed.
# The following devices will be added to the list:
# 'det1'
# 'det1.val'
# devices such as 'd3', 'det3', 'detector3' etc. matching reg. expression 'd.*3'
"Type3": ["det1", "det1.val", ":d.*3"]
Note, that the semicolon :
is not part of regular expressions and used to
tell the processing algorithm that the string contains a pattern as opposed to
a device or subdevice name. For example, "det"
is an explicitly listed name
of the detector det
, while ":det"
is a regular expression that matches
any device name that contains the sequence of characters "d", "e", "t"
,
such as mydetector
.
A device name pattern may consist of multiple regular expressions separated by
:
. The expressions are processed from left to right. The leftmost expression
is applied to the name of each device in global worker namespace. If the device
name matches the expression, the device name is added to the list and each
subdevice is checked using the second expression and matching name is added to
the list. The process continues until the end of the pattern or ‘leaf’ devices
(that have no subdevices) are reached. On each step only the subdevices of
matching devices are searched. The maximum depth of search is defined by
the number of expressions in the pattern.
# The item contains three regular expressions: '^sim' is applied to device
# names (selects all devices starting with 'sim'), '^mt' applies to
# subdevices of matching devices (all subdevices starting with 'mt')
# and '^x$' is applied to subdevices of matching subdevices (all subdevices
# named 'x'). As a result, the list may contain the following
# devices:
# 'sim_stage_A'
# 'sim_stage_A.mtrs',
# 'sim_stage_A.mtrs.x',
# 'sim_stage_B'.
# 'sim_stage_B.mtrs'.
# 'sim_stage_B.mtrs.x'.
"Type3": [":^sim:^mt:^x$"]
The default behavior of the list generation algorithm is to include in the list
all matching devices and subdevices found at each level. The algorithm can be
explicitly instructed to do so by inserting +
(plus sign) before the regular
expression, i.e. the patterns ":^det.val$
” and "":+^det.val$"
would
produce identical results. The default behavior is not always desirable. In order
to skip adding to the list matching devices found at a given depth, add -
(minus sign) before the regular expression (:-
):
# The list will contain devices such as
# 'sim_stage_A.mtrs',
# 'sim_stage_A.mtrs.x',
# 'sim_stage_B.mtrs',
# 'sim_stage_B.mtrs.x'.
"Type4": [":-^sim:^mt:^x$"]
# The list will contain devices such as
# 'sim_stage_A.mtrs.x',
# 'sim_stage_B.mtrs.x'.
"Type5": [":-^sim:-^mt:-^x$"]
Note, that even though -
is allowed in the last expression of the pattern,
it does not influence processing of the pattern. The devices matching the last
expression in the pattern are always inserted in the list.
Regular expressions could be applied to the full device name (such as
"sim_stage_A.mtrs.x"
) or right part a full device name (such as "mtrs.x"
).
An expression may be labelled as a full-name expression by putting ?
(question mark) before the regular expression. The full name regular expression
may only be the last component of the pattern. :
# The list may contain devices with names such as
# 'simval',
# 'sim_stage_A.val',
# 'sim_stage_A.det1.val',
# 'sim_stage_A.detectors.det1.val',
"Type6": [":?^sim.*val$"]
# The list will contain devices such as
# 'sim_stage_A',
# 'sim_stage_A.val',
# 'sim_stage_A.det1_val',
# 'sim_stage_A.det1.val',
# 'sim_stage_A.detectors.det1.val',
"Type7": [":^sim_stage_A$:?.*val$"]
Note, that +
and -
can not be used in conjunction with ?
.
Using full-name expressions is less efficient, since it the algorithm
is forced to searching the full tree of subdevices for matching names.
The depth of search may be explicitly limited by adding depth
parameter (:?<regex>:depth=N
):
# The list will contain devices such as
# 'sim_stage_A',
# 'sim_stage_A.val',
# 'sim_stage_A.det1_val',
# 'sim_stage_A.det1.val',
# The list will not contain 'sim_stage_A.detectors.det1.val',
# since the depth of search is limited to 2 levels.
"Type8": [":+^sim_stage_A$:?.*val$:depth=2"]
A set of devices matching a pattern may be restricted to devices of certain
types by placing one of the type keywords before the patter. The supported
keywords are __DETECTOR__
(readable, not movable), __MOTOR__
(readable and movable), __READABLE__
, __FLYABLE__
before the expression:
# Select only detectors with names matching the regular expression:
"Type9": ["__DETECTORS__:^sim_stage_A$:?.*:depth=3"]
# Select only motors:
"Type10": ["__MOTORS__:^sim_stage_A$:?.*:depth=3"]
Lists of Plan Names¶
Similarly to lists of device names, the lists of plan names may include patterns used to pick matching names of the existing plans:
# The list contains the following names: explicitly listed name of the plan
# ``count`` and regular expression that selects all the plans ending
# with ``_count``, such as ``_count``, ``my_count`` etc.
"Type11": ["count", ":_count$"]
A plan name pattern may contain only a single regular expression (i.e.
a pattern may contain only one :
character). The modifiers +
, -
and ?
may still be used, but the do not influence processing
of the plan name patterns (they are allowed to make patterns for plans
and devices look uniform, but since plan patterns contain only one component
and this component is the last, the matching plan names are always added
to the list and the regular expression is always applied to the full name).
Device type keywords can not be used in plan name patterns.
Plan Annotation API¶
The decorator allows to attach a custom description to a function or generator function. |