Implementing File Writing Detectors#

In Implementing Devices we learned how to create Devices that talk to a control system to implement a particular behavior in bluesky plans. This behavior was based around the verbs from the bluesky.protocols.Movable and bluesky.protocols.Readable protocols, allowing us to use these Devices in a typical scan: moving them to a position, then acquiring data via the control system. We will now explore the bluesky.protocols.WritesExternalAssets protocol, and how it would be implemented for a File Writing Detector.

Run the demo#

We will return to our ophyd_async.sim devices we saw in Using Devices for this tutorial, and dig a little deeper into what Event Model Documents they produce. Let’s run up our ipython shell again:

$ ipython --matplotlib=qt6 -i -m ophyd_async.sim
Python 3.11.11 (main, Dec  4 2024, 20:38:25) [GCC 12.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.30.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: 

Run a grid scan and investigate the documents#

Now let’s run a grid scan on the point detector, and pass a callback to the RunEngine so it prints the documents that are emitted:

In [1]: RE(bp.grid_scan([pdet], stage.x, 1, 2, 2, stage.y, 2, 3, 2), print)


Transient Scan ID: 1     Time: 2025-04-25 09:45:13
Persistent Unique Scan ID: 'fe0cc080-ca61-4f9a-b5cd-a0d438412cae'
start {'uid': 'fe0cc080-ca61-4f9a-b5cd-a0d438412cae', 'time': 1745574313.022047, 'versions': {'ophyd': '1.10.5', 'bluesky': '1.13.1'}, 'scan_id': 1, 'plan_type': 'generator', 'plan_name': 'grid_scan', 'detectors': ['pdet'], 'motors': ('stage-x', 'stage-y'), 'num_points': 4, 'num_intervals': 3, 'plan_args': {'detectors': ['<ophyd_async.sim._point_detector.SimPointDetector object at 0x7f2ff413d050>'], 'args': ['<ophyd_async.sim._motor.SimMotor object at 0x7f2feedd5210>', 1, 2, 2, '<ophyd_async.sim._motor.SimMotor object at 0x7f2feed07050>', 2, 3, 2, False], 'per_step': 'None'}, 'hints': {'gridding': 'rectilinear', 'dimensions': [(['stage-x'], 'primary'), (['stage-y'], 'primary')]}, 'shape': (2, 2), 'extents': ([1, 2], [2, 3]), 'snaking': (False, False), 'plan_pattern': 'outer_product', 'plan_pattern_args': {'args': ['<ophyd_async.sim._motor.SimMotor object at 0x7f2feedd5210>', 1, 2, 2, '<ophyd_async.sim._motor.SimMotor object at 0x7f2feed07050>', 2, 3, 2, False]}, 'plan_pattern_module': 'bluesky.plan_patterns'}
New stream: 'primary'
+-----------+------------+------------+------------+----------------------+----------------------+----------------------+
|   seq_num |       time |    stage-x |    stage-y | pdet-channel-1-value | pdet-channel-2-value | pdet-channel-3-value |
+-----------+------------+------------+------------+----------------------+----------------------+----------------------+
descriptor {'configuration': {'stage-x': {'data': {'stage-x-acceleration_time': 0.5, 'stage-x-velocity': 1000.0, 'stage-x-units': 'mm'}, 'timestamps': {'stage-x-acceleration_time': 197.330747436, 'stage-x-velocity': 197.337519719, 'stage-x-units': 197.330767414}, 'data_keys': {'stage-x-acceleration_time': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x-acceleration_time'}, 'stage-x-velocity': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x-velocity'}, 'stage-x-units': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://stage-x-units'}}}, 'stage-y': {'data': {'stage-y-acceleration_time': 0.5, 'stage-y-velocity': 1000.0, 'stage-y-units': 'mm'}, 'timestamps': {'stage-y-acceleration_time': 197.331059301, 'stage-y-velocity': 197.337624876, 'stage-y-units': 197.331067807}, 'data_keys': {'stage-y-acceleration_time': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y-acceleration_time'}, 'stage-y-velocity': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y-velocity'}, 'stage-y-units': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://stage-y-units'}}}, 'pdet': {'data': {'pdet-channel-1-mode': <EnergyMode.LOW: 'Low Energy'>, 'pdet-channel-2-mode': <EnergyMode.LOW: 'Low Energy'>, 'pdet-channel-3-mode': <EnergyMode.LOW: 'Low Energy'>}, 'timestamps': {'pdet-channel-1-mode': 197.331794019, 'pdet-channel-2-mode': 197.331848982, 'pdet-channel-3-mode': 197.331893024}, 'data_keys': {'pdet-channel-1-mode': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://pdet-channel-1-mode', 'choices': ['Low Energy', 'High Energy']}, 'pdet-channel-2-mode': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://pdet-channel-2-mode', 'choices': ['Low Energy', 'High Energy']}, 'pdet-channel-3-mode': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://pdet-channel-3-mode', 'choices': ['Low Energy', 'High Energy']}}}}, 'data_keys': {'stage-x': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x', 'object_name': 'stage-x'}, 'stage-y': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y', 'object_name': 'stage-y'}, 'pdet-channel-1-value': {'dtype': 'integer', 'shape': [], 'dtype_numpy': '<i8', 'source': 'soft://pdet-channel-1-value', 'object_name': 'pdet'}, 'pdet-channel-2-value': {'dtype': 'integer', 'shape': [], 'dtype_numpy': '<i8', 'source': 'soft://pdet-channel-2-value', 'object_name': 'pdet'}, 'pdet-channel-3-value': {'dtype': 'integer', 'shape': [], 'dtype_numpy': '<i8', 'source': 'soft://pdet-channel-3-value', 'object_name': 'pdet'}}, 'name': 'primary', 'object_keys': {'stage-x': ['stage-x'], 'stage-y': ['stage-y'], 'pdet': ['pdet-channel-1-value', 'pdet-channel-2-value', 'pdet-channel-3-value']}, 'run_start': 'fe0cc080-ca61-4f9a-b5cd-a0d438412cae', 'time': 1745574313.1954243, 'uid': '087ad25a-3d26-48ce-9f78-3916384366c1', 'hints': {'stage-x': {'fields': ['stage-x']}, 'stage-y': {'fields': ['stage-y']}, 'pdet': {'fields': ['pdet-channel-1-value', 'pdet-channel-2-value', 'pdet-channel-3-value']}}}
|         1 | 09:45:13.2 |      1.000 |      2.000 |                  921 |                  887 |                  859 |
event {'uid': '48dd3761-e561-4746-b59a-d64576a18f23', 'time': 1745574313.275056, 'data': {'pdet-channel-1-value': 921, 'pdet-channel-2-value': 887, 'pdet-channel-3-value': 859, 'stage-x': 1.0, 'stage-y': 2.0}, 'timestamps': {'pdet-channel-1-value': 197.562351254, 'pdet-channel-2-value': 197.562393773, 'pdet-channel-3-value': 197.562410395, 'stage-x': 197.441026739, 'stage-y': 197.460449577}, 'seq_num': 1, 'filled': {}, 'descriptor': '087ad25a-3d26-48ce-9f78-3916384366c1'}
|         2 | 09:45:14.3 |      1.000 |      3.000 |                  937 |                  903 |                  875 |
event {'uid': '17f344aa-ee77-4a1f-90f1-e3899840bbf4', 'time': 1745574314.3621502, 'data': {'pdet-channel-1-value': 937, 'pdet-channel-2-value': 903, 'pdet-channel-3-value': 875, 'stage-x': 1.0, 'stage-y': 3.0}, 'timestamps': {'pdet-channel-1-value': 198.730402986, 'pdet-channel-2-value': 198.730445626, 'pdet-channel-3-value': 198.730462247, 'stage-x': 197.441026739, 'stage-y': 198.628651692}, 'seq_num': 2, 'filled': {}, 'descriptor': '087ad25a-3d26-48ce-9f78-3916384366c1'}
|         3 | 09:45:15.3 |      2.000 |      2.000 |                  761 |                  740 |                  722 |
event {'uid': '33849c28-eeda-4249-b234-6500fab32d9b', 'time': 1745574315.3279505, 'data': {'pdet-channel-1-value': 761, 'pdet-channel-2-value': 740, 'pdet-channel-3-value': 722, 'stage-x': 2.0, 'stage-y': 2.0}, 'timestamps': {'pdet-channel-1-value': 199.696313395, 'pdet-channel-2-value': 199.696357027, 'pdet-channel-3-value': 199.696373247, 'stage-x': 199.594479756, 'stage-y': 199.594534569}, 'seq_num': 3, 'filled': {}, 'descriptor': '087ad25a-3d26-48ce-9f78-3916384366c1'}
|         4 | 09:45:16.2 |      2.000 |      3.000 |                  487 |                  467 |                  448 |
event {'uid': '0f240417-628b-4565-ba6d-0deff6715fff', 'time': 1745574316.2772133, 'data': {'pdet-channel-1-value': 487, 'pdet-channel-2-value': 467, 'pdet-channel-3-value': 448, 'stage-x': 2.0, 'stage-y': 3.0}, 'timestamps': {'pdet-channel-1-value': 200.64563446, 'pdet-channel-2-value': 200.645671168, 'pdet-channel-3-value': 200.645685626, 'stage-x': 199.594479756, 'stage-y': 200.544032034}, 'seq_num': 4, 'filled': {}, 'descriptor': '087ad25a-3d26-48ce-9f78-3916384366c1'}
+-----------+------------+------------+------------+----------------------+----------------------+----------------------+
generator grid_scan ['fe0cc080'] (scan num: 1)



stop {'uid': 'fd0b709b-1dd5-4bba-aaac-c85b2a4e3116', 'time': 1745574317.1280544, 'run_start': 'fe0cc080-ca61-4f9a-b5cd-a0d438412cae', 'exit_status': 'success', 'reason': '', 'num_events': {'primary': 4}}
Out[1]: RunEngineResult(run_start_uids=('fe0cc080-ca61-4f9a-b5cd-a0d438412cae',), plan_result='fe0cc080-ca61-4f9a-b5cd-a0d438412cae', exit_status='success', interrupted=False, reason='', exception=None)

We see a series of documents being emitted:

A event_model.RunStart document that tells us a scan is starting and what sort of scan it is, along with the names of the motors that will be moved.
An event_model.EventDescriptor document that tells us that the motor readbacks and detector channels will be all be read together in a single stream. It is used to make the column headings, but it contains more metadata about the Devices too, like their configuration.
For each point in the scan:
- An event_model.Event document, containing the motor readbacks and detector channels with their timestamps. It is used to make each row of the table.
A event_model.RunStop document that tells us the scan has stopped, and gives us its status.

Now let’s try the same thing, but this time with the blob detector:

In [2]: RE(bp.grid_scan([bdet], stage.x, 1, 2, 2, stage.y, 2, 3, 2), print)


Transient Scan ID: 2     Time: 2025-04-25 09:45:17
Persistent Unique Scan ID: '508432da-1651-4d1a-a380-edbcc6f34372'
start {'uid': '508432da-1651-4d1a-a380-edbcc6f34372', 'time': 1745574317.2398782, 'versions': {'ophyd': '1.10.5', 'bluesky': '1.13.1'}, 'scan_id': 2, 'plan_type': 'generator', 'plan_name': 'grid_scan', 'detectors': ['bdet'], 'motors': ('stage-x', 'stage-y'), 'num_points': 4, 'num_intervals': 3, 'plan_args': {'detectors': ['<ophyd_async.sim._blob_detector.SimBlobDetector object at 0x7f2ff4131c90>'], 'args': ['<ophyd_async.sim._motor.SimMotor object at 0x7f2feedd5210>', 1, 2, 2, '<ophyd_async.sim._motor.SimMotor object at 0x7f2feed07050>', 2, 3, 2, False], 'per_step': 'None'}, 'hints': {'gridding': 'rectilinear', 'dimensions': [(['stage-x'], 'primary'), (['stage-y'], 'primary')]}, 'shape': (2, 2), 'extents': ([1, 2], [2, 3]), 'snaking': (False, False), 'plan_pattern': 'outer_product', 'plan_pattern_args': {'args': ['<ophyd_async.sim._motor.SimMotor object at 0x7f2feedd5210>', 1, 2, 2, '<ophyd_async.sim._motor.SimMotor object at 0x7f2feed07050>', 2, 3, 2, False]}, 'plan_pattern_module': 'bluesky.plan_patterns'}
New stream: 'primary'
+-----------+------------+------------+------------+
|   seq_num |       time |    stage-x |    stage-y |
+-----------+------------+------------+------------+
descriptor {'configuration': {'bdet': {'data': {}, 'timestamps': {}, 'data_keys': {}}, 'stage-y': {'data': {'stage-y-acceleration_time': 0.5, 'stage-y-velocity': 1000.0, 'stage-y-units': 'mm'}, 'timestamps': {'stage-y-acceleration_time': 197.331059301, 'stage-y-velocity': 197.337624876, 'stage-y-units': 197.331067807}, 'data_keys': {'stage-y-acceleration_time': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y-acceleration_time'}, 'stage-y-velocity': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y-velocity'}, 'stage-y-units': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://stage-y-units'}}}, 'stage-x': {'data': {'stage-x-acceleration_time': 0.5, 'stage-x-velocity': 1000.0, 'stage-x-units': 'mm'}, 'timestamps': {'stage-x-acceleration_time': 197.330747436, 'stage-x-velocity': 197.337519719, 'stage-x-units': 197.330767414}, 'data_keys': {'stage-x-acceleration_time': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x-acceleration_time'}, 'stage-x-velocity': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x-velocity'}, 'stage-x-units': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://stage-x-units'}}}}, 'data_keys': {'bdet': {'source': 'sim://pattern-generator-hdf-file', 'shape': [1, 240, 320], 'dtype': 'array', 'dtype_numpy': '|u1', 'external': 'STREAM:', 'object_name': 'bdet'}, 'bdet-sum': {'source': 'sim://pattern-generator-hdf-file', 'shape': [], 'dtype': 'number', 'dtype_numpy': '<i8', 'external': 'STREAM:', 'object_name': 'bdet'}, 'stage-y': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y', 'object_name': 'stage-y'}, 'stage-x': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x', 'object_name': 'stage-x'}}, 'name': 'primary', 'object_keys': {'bdet': ['bdet', 'bdet-sum'], 'stage-y': ['stage-y'], 'stage-x': ['stage-x']}, 'run_start': '508432da-1651-4d1a-a380-edbcc6f34372', 'time': 1745574317.4545758, 'uid': '08b9951b-f7a3-4538-9a09-370037436b81', 'hints': {'bdet': {'fields': ['bdet']}, 'stage-y': {'fields': ['stage-y']}, 'stage-x': {'fields': ['stage-x']}}}
stream_resource {'uid': '2dff6a82-ac04-421a-907a-cab6f22fd7d6', 'data_key': 'bdet', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/ee0b20ac-4916-44fa-b245-3cf85df34646.h5', 'parameters': {'dataset': '/entry/data/data', 'chunk_shape': (1024,)}, 'run_start': '508432da-1651-4d1a-a380-edbcc6f34372'}
stream_resource {'uid': '002f1839-c269-4a8f-bfcb-55fb327a83d1', 'data_key': 'bdet-sum', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/ee0b20ac-4916-44fa-b245-3cf85df34646.h5', 'parameters': {'dataset': '/entry/sum', 'chunk_shape': (1024,)}, 'run_start': '508432da-1651-4d1a-a380-edbcc6f34372'}
stream_datum {'stream_resource': '2dff6a82-ac04-421a-907a-cab6f22fd7d6', 'uid': '2dff6a82-ac04-421a-907a-cab6f22fd7d6/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
stream_datum {'stream_resource': '002f1839-c269-4a8f-bfcb-55fb327a83d1', 'uid': '002f1839-c269-4a8f-bfcb-55fb327a83d1/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
|         1 | 09:45:17.4 |      1.000 |      2.000 |
event {'uid': '2ecd47b0-7aac-4dfa-aa92-366067a5c279', 'time': 1745574317.482348, 'data': {'stage-x': 1.0, 'stage-y': 2.0}, 'timestamps': {'stage-x': 201.65750203, 'stage-y': 201.657556282}, 'seq_num': 1, 'filled': {}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
stream_datum {'stream_resource': '2dff6a82-ac04-421a-907a-cab6f22fd7d6', 'uid': '2dff6a82-ac04-421a-907a-cab6f22fd7d6/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
stream_datum {'stream_resource': '002f1839-c269-4a8f-bfcb-55fb327a83d1', 'uid': '002f1839-c269-4a8f-bfcb-55fb327a83d1/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
|         2 | 09:45:17.6 |      1.000 |      3.000 |
event {'uid': 'e064eb8a-5244-4f53-af4b-e5e12a88b6dd', 'time': 1745574317.6341863, 'data': {'stage-x': 1.0, 'stage-y': 3.0}, 'timestamps': {'stage-x': 201.65750203, 'stage-y': 201.898905622}, 'seq_num': 2, 'filled': {}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
stream_datum {'stream_resource': '2dff6a82-ac04-421a-907a-cab6f22fd7d6', 'uid': '2dff6a82-ac04-421a-907a-cab6f22fd7d6/2', 'seq_nums': {'start': 3, 'stop': 4}, 'indices': {'start': 2, 'stop': 3}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
stream_datum {'stream_resource': '002f1839-c269-4a8f-bfcb-55fb327a83d1', 'uid': '002f1839-c269-4a8f-bfcb-55fb327a83d1/2', 'seq_nums': {'start': 3, 'stop': 4}, 'indices': {'start': 2, 'stop': 3}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
|         3 | 09:45:17.7 |      2.000 |      2.000 |
event {'uid': 'a8c25df5-da87-4ec7-8a91-b3c7b9e4af11', 'time': 1745574317.7858746, 'data': {'stage-x': 2.0, 'stage-y': 2.0}, 'timestamps': {'stage-x': 202.050829055, 'stage-y': 202.050872636}, 'seq_num': 3, 'filled': {}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
stream_datum {'stream_resource': '2dff6a82-ac04-421a-907a-cab6f22fd7d6', 'uid': '2dff6a82-ac04-421a-907a-cab6f22fd7d6/3', 'seq_nums': {'start': 4, 'stop': 5}, 'indices': {'start': 3, 'stop': 4}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
stream_datum {'stream_resource': '002f1839-c269-4a8f-bfcb-55fb327a83d1', 'uid': '002f1839-c269-4a8f-bfcb-55fb327a83d1/3', 'seq_nums': {'start': 4, 'stop': 5}, 'indices': {'start': 3, 'stop': 4}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
|         4 | 09:45:17.9 |      2.000 |      3.000 |
event {'uid': 'b29caf0b-7de5-4ebc-90ff-f7ff25cf3bbb', 'time': 1745574317.9374146, 'data': {'stage-x': 2.0, 'stage-y': 3.0}, 'timestamps': {'stage-x': 202.050829055, 'stage-y': 202.202286153}, 'seq_num': 4, 'filled': {}, 'descriptor': '08b9951b-f7a3-4538-9a09-370037436b81'}
+-----------+------------+------------+------------+
generator grid_scan ['508432da'] (scan num: 2)



stop {'uid': 'ef1c533e-c6d6-43b0-96ac-8f06d5788709', 'time': 1745574317.9377592, 'run_start': '508432da-1651-4d1a-a380-edbcc6f34372', 'exit_status': 'success', 'reason': '', 'num_events': {'primary': 4}}
Out[2]: RunEngineResult(run_start_uids=('508432da-1651-4d1a-a380-edbcc6f34372',), plan_result='508432da-1651-4d1a-a380-edbcc6f34372', exit_status='success', interrupted=False, reason='', exception=None)

This time we see some different documents:

The same event_model.RunStart document
A similar event_model.EventDescriptor document, but with 'external': 'STREAM:' on the detector column headings.
A couple of event_model.StreamResource documents for each of those detector column headings giving an HDF file name and dataset within it where data will be written.
For each point in the scan:
- A couple of event_model.StreamDatum documents with a range of indices that have been written to an HDF dataset referenced in the StreamResource document.
- An event_model.Event document, containing the motor readbacks and timestamps. It is used to make each row of the table.
The same event_model.RunStop document

And we can run the plan with both detectors to see a document stream that combines both the previous example:

In [3]: RE(bp.grid_scan([bdet, pdet], stage.x, 1, 2, 2, stage.y, 2, 3, 2), print)


Transient Scan ID: 3     Time: 2025-04-25 09:45:18
Persistent Unique Scan ID: 'de54e58a-daaa-45a3-a15e-2711df0c4fa6'
start {'uid': 'de54e58a-daaa-45a3-a15e-2711df0c4fa6', 'time': 1745574318.0490847, 'versions': {'ophyd': '1.10.5', 'bluesky': '1.13.1'}, 'scan_id': 3, 'plan_type': 'generator', 'plan_name': 'grid_scan', 'detectors': ['bdet', 'pdet'], 'motors': ('stage-x', 'stage-y'), 'num_points': 4, 'num_intervals': 3, 'plan_args': {'detectors': ['<ophyd_async.sim._blob_detector.SimBlobDetector object at 0x7f2ff4131c90>', '<ophyd_async.sim._point_detector.SimPointDetector object at 0x7f2ff413d050>'], 'args': ['<ophyd_async.sim._motor.SimMotor object at 0x7f2feedd5210>', 1, 2, 2, '<ophyd_async.sim._motor.SimMotor object at 0x7f2feed07050>', 2, 3, 2, False], 'per_step': 'None'}, 'hints': {'gridding': 'rectilinear', 'dimensions': [(['stage-x'], 'primary'), (['stage-y'], 'primary')]}, 'shape': (2, 2), 'extents': ([1, 2], [2, 3]), 'snaking': (False, False), 'plan_pattern': 'outer_product', 'plan_pattern_args': {'args': ['<ophyd_async.sim._motor.SimMotor object at 0x7f2feedd5210>', 1, 2, 2, '<ophyd_async.sim._motor.SimMotor object at 0x7f2feed07050>', 2, 3, 2, False]}, 'plan_pattern_module': 'bluesky.plan_patterns'}
New stream: 'primary'
+-----------+------------+------------+------------+----------------------+----------------------+----------------------+
|   seq_num |       time |    stage-x |    stage-y | pdet-channel-1-value | pdet-channel-2-value | pdet-channel-3-value |
+-----------+------------+------------+------------+----------------------+----------------------+----------------------+
descriptor {'configuration': {'bdet': {'data': {}, 'timestamps': {}, 'data_keys': {}}, 'stage-y': {'data': {'stage-y-acceleration_time': 0.5, 'stage-y-velocity': 1000.0, 'stage-y-units': 'mm'}, 'timestamps': {'stage-y-acceleration_time': 197.331059301, 'stage-y-velocity': 197.337624876, 'stage-y-units': 197.331067807}, 'data_keys': {'stage-y-acceleration_time': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y-acceleration_time'}, 'stage-y-velocity': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y-velocity'}, 'stage-y-units': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://stage-y-units'}}}, 'pdet': {'data': {'pdet-channel-1-mode': <EnergyMode.LOW: 'Low Energy'>, 'pdet-channel-2-mode': <EnergyMode.LOW: 'Low Energy'>, 'pdet-channel-3-mode': <EnergyMode.LOW: 'Low Energy'>}, 'timestamps': {'pdet-channel-1-mode': 197.331794019, 'pdet-channel-2-mode': 197.331848982, 'pdet-channel-3-mode': 197.331893024}, 'data_keys': {'pdet-channel-1-mode': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://pdet-channel-1-mode', 'choices': ['Low Energy', 'High Energy']}, 'pdet-channel-2-mode': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://pdet-channel-2-mode', 'choices': ['Low Energy', 'High Energy']}, 'pdet-channel-3-mode': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://pdet-channel-3-mode', 'choices': ['Low Energy', 'High Energy']}}}, 'stage-x': {'data': {'stage-x-acceleration_time': 0.5, 'stage-x-velocity': 1000.0, 'stage-x-units': 'mm'}, 'timestamps': {'stage-x-acceleration_time': 197.330747436, 'stage-x-velocity': 197.337519719, 'stage-x-units': 197.330767414}, 'data_keys': {'stage-x-acceleration_time': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x-acceleration_time'}, 'stage-x-velocity': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x-velocity'}, 'stage-x-units': {'dtype': 'string', 'shape': [], 'dtype_numpy': '|S40', 'source': 'soft://stage-x-units'}}}}, 'data_keys': {'bdet': {'source': 'sim://pattern-generator-hdf-file', 'shape': [1, 240, 320], 'dtype': 'array', 'dtype_numpy': '|u1', 'external': 'STREAM:', 'object_name': 'bdet'}, 'bdet-sum': {'source': 'sim://pattern-generator-hdf-file', 'shape': [], 'dtype': 'number', 'dtype_numpy': '<i8', 'external': 'STREAM:', 'object_name': 'bdet'}, 'stage-y': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-y', 'object_name': 'stage-y'}, 'pdet-channel-1-value': {'dtype': 'integer', 'shape': [], 'dtype_numpy': '<i8', 'source': 'soft://pdet-channel-1-value', 'object_name': 'pdet'}, 'pdet-channel-2-value': {'dtype': 'integer', 'shape': [], 'dtype_numpy': '<i8', 'source': 'soft://pdet-channel-2-value', 'object_name': 'pdet'}, 'pdet-channel-3-value': {'dtype': 'integer', 'shape': [], 'dtype_numpy': '<i8', 'source': 'soft://pdet-channel-3-value', 'object_name': 'pdet'}, 'stage-x': {'dtype': 'number', 'shape': [], 'dtype_numpy': '<f8', 'source': 'soft://stage-x', 'object_name': 'stage-x'}}, 'name': 'primary', 'object_keys': {'bdet': ['bdet', 'bdet-sum'], 'stage-y': ['stage-y'], 'pdet': ['pdet-channel-1-value', 'pdet-channel-2-value', 'pdet-channel-3-value'], 'stage-x': ['stage-x']}, 'run_start': 'de54e58a-daaa-45a3-a15e-2711df0c4fa6', 'time': 1745574318.2648413, 'uid': '033556c3-9975-4576-b022-99213e47d183', 'hints': {'bdet': {'fields': ['bdet']}, 'stage-y': {'fields': ['stage-y']}, 'pdet': {'fields': ['pdet-channel-1-value', 'pdet-channel-2-value', 'pdet-channel-3-value']}, 'stage-x': {'fields': ['stage-x']}}}
stream_resource {'uid': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8', 'data_key': 'bdet', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/8baeeb68-c1e1-4707-bc69-33d126255b0c.h5', 'parameters': {'dataset': '/entry/data/data', 'chunk_shape': (1024,)}, 'run_start': 'de54e58a-daaa-45a3-a15e-2711df0c4fa6'}
stream_resource {'uid': '6e702e46-185e-4a99-aa8b-d91551ed61c6', 'data_key': 'bdet-sum', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/8baeeb68-c1e1-4707-bc69-33d126255b0c.h5', 'parameters': {'dataset': '/entry/sum', 'chunk_shape': (1024,)}, 'run_start': 'de54e58a-daaa-45a3-a15e-2711df0c4fa6'}
stream_datum {'stream_resource': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8', 'uid': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
stream_datum {'stream_resource': '6e702e46-185e-4a99-aa8b-d91551ed61c6', 'uid': '6e702e46-185e-4a99-aa8b-d91551ed61c6/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
|         1 | 09:45:18.3 |      1.000 |      2.000 |                  921 |                  887 |                  859 |
event {'uid': '06ba79ba-ac95-4e03-8cfb-a0969ff26dc1', 'time': 1745574318.3663583, 'data': {'pdet-channel-1-value': 921, 'pdet-channel-2-value': 887, 'pdet-channel-3-value': 859, 'stage-x': 1.0, 'stage-y': 2.0}, 'timestamps': {'pdet-channel-1-value': 202.571073135, 'pdet-channel-2-value': 202.571113, 'pdet-channel-3-value': 202.571129732, 'stage-x': 202.466600988, 'stage-y': 202.466654448}, 'seq_num': 1, 'filled': {}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
stream_datum {'stream_resource': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8', 'uid': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
stream_datum {'stream_resource': '6e702e46-185e-4a99-aa8b-d91551ed61c6', 'uid': '6e702e46-185e-4a99-aa8b-d91551ed61c6/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
|         2 | 09:45:19.7 |      1.000 |      3.000 |                  937 |                  903 |                  875 |
event {'uid': '81d8dee1-29c4-4e29-996d-5ea4c2949548', 'time': 1745574319.7190042, 'data': {'pdet-channel-1-value': 937, 'pdet-channel-2-value': 903, 'pdet-channel-3-value': 875, 'stage-x': 1.0, 'stage-y': 3.0}, 'timestamps': {'pdet-channel-1-value': 204.084724817, 'pdet-channel-2-value': 204.084761866, 'pdet-channel-3-value': 204.084777295, 'stage-x': 202.466600988, 'stage-y': 203.98277666}, 'seq_num': 2, 'filled': {}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
stream_datum {'stream_resource': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8', 'uid': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8/2', 'seq_nums': {'start': 3, 'stop': 4}, 'indices': {'start': 2, 'stop': 3}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
stream_datum {'stream_resource': '6e702e46-185e-4a99-aa8b-d91551ed61c6', 'uid': '6e702e46-185e-4a99-aa8b-d91551ed61c6/2', 'seq_nums': {'start': 3, 'stop': 4}, 'indices': {'start': 2, 'stop': 3}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
|         3 | 09:45:20.9 |      2.000 |      2.000 |                  761 |                  740 |                  722 |
event {'uid': 'f3ebf5b3-931a-486a-a531-a21c6578a48b', 'time': 1745574320.9613032, 'data': {'pdet-channel-1-value': 761, 'pdet-channel-2-value': 740, 'pdet-channel-3-value': 722, 'stage-x': 2.0, 'stage-y': 2.0}, 'timestamps': {'pdet-channel-1-value': 205.32705979, 'pdet-channel-2-value': 205.327101729, 'pdet-channel-3-value': 205.327117138, 'stage-x': 205.225056801, 'stage-y': 205.225114278}, 'seq_num': 3, 'filled': {}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
stream_datum {'stream_resource': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8', 'uid': 'b95aaf14-ce01-45ac-b542-50c7107ab6b8/3', 'seq_nums': {'start': 4, 'stop': 5}, 'indices': {'start': 3, 'stop': 4}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
stream_datum {'stream_resource': '6e702e46-185e-4a99-aa8b-d91551ed61c6', 'uid': '6e702e46-185e-4a99-aa8b-d91551ed61c6/3', 'seq_nums': {'start': 4, 'stop': 5}, 'indices': {'start': 3, 'stop': 4}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
|         4 | 09:45:22.1 |      2.000 |      3.000 |                  487 |                  467 |                  448 |
event {'uid': '50d894bf-11db-4109-b7e8-fe2409f6bea1', 'time': 1745574322.1867225, 'data': {'pdet-channel-1-value': 487, 'pdet-channel-2-value': 467, 'pdet-channel-3-value': 448, 'stage-x': 2.0, 'stage-y': 3.0}, 'timestamps': {'pdet-channel-1-value': 206.552492441, 'pdet-channel-2-value': 206.552530953, 'pdet-channel-3-value': 206.552545541, 'stage-x': 205.225056801, 'stage-y': 206.450562336}, 'seq_num': 4, 'filled': {}, 'descriptor': '033556c3-9975-4576-b022-99213e47d183'}
+-----------+------------+------------+------------+----------------------+----------------------+----------------------+
generator grid_scan ['de54e58a'] (scan num: 3)



stop {'uid': 'b16852a7-1ce3-4898-b252-8168187a6a1e', 'time': 1745574323.3287578, 'run_start': 'de54e58a-daaa-45a3-a15e-2711df0c4fa6', 'exit_status': 'success', 'reason': '', 'num_events': {'primary': 4}}
Out[3]: RunEngineResult(run_start_uids=('de54e58a-daaa-45a3-a15e-2711df0c4fa6',), plan_result='de54e58a-daaa-45a3-a15e-2711df0c4fa6', exit_status='success', interrupted=False, reason='', exception=None)

Simplify the plan to just use the detector#

The above examples show what happens if you trigger() a detector at each point of a scan, in this case taking a single frame each time. Let’s write our own simple plan that only triggers and reads from the detector, using the utility bps (bluesky.plan_stubs) and bpp (bluesky.preprocessors):

In [4]: @bpp.stage_decorator([bdet])
   ...: @bpp.run_decorator()
   ...: def my_count_plan():
   ...:     for i in range(2):
   ...:         yield from bps.trigger_and_read([bdet])
   ...: 

In [5]: RE(my_count_plan(), print)


Transient Scan ID: 4     Time: 2025-04-25 09:45:23
Persistent Unique Scan ID: '8ffb5f7f-700d-4498-a35f-dbf5b1e5b762'
start {'uid': '8ffb5f7f-700d-4498-a35f-dbf5b1e5b762', 'time': 1745574323.4445992, 'versions': {'ophyd': '1.10.5', 'bluesky': '1.13.1'}, 'scan_id': 4, 'plan_type': 'generator', 'plan_name': 'my_count_plan'}
New stream: 'primary'
+-----------+------------+
|   seq_num |       time |
+-----------+------------+
descriptor {'configuration': {'bdet': {'data': {}, 'timestamps': {}, 'data_keys': {}}}, 'data_keys': {'bdet': {'source': 'sim://pattern-generator-hdf-file', 'shape': [1, 240, 320], 'dtype': 'array', 'dtype_numpy': '|u1', 'external': 'STREAM:', 'object_name': 'bdet'}, 'bdet-sum': {'source': 'sim://pattern-generator-hdf-file', 'shape': [], 'dtype': 'number', 'dtype_numpy': '<i8', 'external': 'STREAM:', 'object_name': 'bdet'}}, 'name': 'primary', 'object_keys': {'bdet': ['bdet', 'bdet-sum']}, 'run_start': '8ffb5f7f-700d-4498-a35f-dbf5b1e5b762', 'time': 1745574323.6093905, 'uid': '4b5d0f8c-ccd9-4c9c-97eb-445d17c21eca', 'hints': {'bdet': {'fields': ['bdet']}}}
stream_resource {'uid': 'fcc2c3e3-6459-44d4-93c3-fc0dda734c5d', 'data_key': 'bdet', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/e5e335fe-a443-4826-9524-a3085fdbeb27.h5', 'parameters': {'dataset': '/entry/data/data', 'chunk_shape': (1024,)}, 'run_start': '8ffb5f7f-700d-4498-a35f-dbf5b1e5b762'}
stream_resource {'uid': 'd8ace36d-a6f4-4442-af91-c9284f0d356e', 'data_key': 'bdet-sum', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/e5e335fe-a443-4826-9524-a3085fdbeb27.h5', 'parameters': {'dataset': '/entry/sum', 'chunk_shape': (1024,)}, 'run_start': '8ffb5f7f-700d-4498-a35f-dbf5b1e5b762'}
stream_datum {'stream_resource': 'fcc2c3e3-6459-44d4-93c3-fc0dda734c5d', 'uid': 'fcc2c3e3-6459-44d4-93c3-fc0dda734c5d/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': '4b5d0f8c-ccd9-4c9c-97eb-445d17c21eca'}
stream_datum {'stream_resource': 'd8ace36d-a6f4-4442-af91-c9284f0d356e', 'uid': 'd8ace36d-a6f4-4442-af91-c9284f0d356e/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': '4b5d0f8c-ccd9-4c9c-97eb-445d17c21eca'}
|         1 | 09:45:23.6 |
event {'uid': 'c53b3a7c-c104-45cf-9bc0-2ce1b21e9283', 'time': 1745574323.6181278, 'data': {}, 'timestamps': {}, 'seq_num': 1, 'filled': {}, 'descriptor': '4b5d0f8c-ccd9-4c9c-97eb-445d17c21eca'}
stream_datum {'stream_resource': 'fcc2c3e3-6459-44d4-93c3-fc0dda734c5d', 'uid': 'fcc2c3e3-6459-44d4-93c3-fc0dda734c5d/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': '4b5d0f8c-ccd9-4c9c-97eb-445d17c21eca'}
stream_datum {'stream_resource': 'd8ace36d-a6f4-4442-af91-c9284f0d356e', 'uid': 'd8ace36d-a6f4-4442-af91-c9284f0d356e/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': '4b5d0f8c-ccd9-4c9c-97eb-445d17c21eca'}
|         2 | 09:45:23.7 |
event {'uid': '03083350-b74c-425c-b31e-6bd97d238005', 'time': 1745574323.7226176, 'data': {}, 'timestamps': {}, 'seq_num': 2, 'filled': {}, 'descriptor': '4b5d0f8c-ccd9-4c9c-97eb-445d17c21eca'}
+-----------+------------+
generator my_count_plan ['8ffb5f7f'] (scan num: 4)



stop {'uid': '7f0ec951-f0ed-4708-b98f-482c7e3f7439', 'time': 1745574323.7229693, 'run_start': '8ffb5f7f-700d-4498-a35f-dbf5b1e5b762', 'exit_status': 'success', 'reason': '', 'num_events': {'primary': 2}}
Out[5]: RunEngineResult(run_start_uids=('8ffb5f7f-700d-4498-a35f-dbf5b1e5b762',), plan_result='8ffb5f7f-700d-4498-a35f-dbf5b1e5b762', exit_status='success', interrupted=False, reason='', exception=None)

Here we see the same sort of documents as above, but with only detector information in it.

Note that on each trigger, only a single image is taken, at the default exposure of 0.1s. If we would like a different exposure time, we can specify with a TriggerInfo:

In [6]: from ophyd_async.core import TriggerInfo

In [7]: @bpp.stage_decorator([bdet])
   ...: @bpp.run_decorator()
   ...: def my_count_plan_with_prepare():
   ...:     yield from bps.prepare(bdet, TriggerInfo(livetime=0.001), wait=True)
   ...:     for i in range(2):
   ...:         yield from bps.trigger_and_read([bdet])
   ...: 

In [8]: RE(my_count_plan_with_prepare(), print)


Transient Scan ID: 5     Time: 2025-04-25 09:45:23
Persistent Unique Scan ID: '0b8a6639-1879-4d4c-a661-fdc47e3c892a'
start {'uid': '0b8a6639-1879-4d4c-a661-fdc47e3c892a', 'time': 1745574323.835759, 'versions': {'ophyd': '1.10.5', 'bluesky': '1.13.1'}, 'scan_id': 5, 'plan_type': 'generator', 'plan_name': 'my_count_plan_with_prepare'}
New stream: 'primary'
+-----------+------------+
|   seq_num |       time |
+-----------+------------+
descriptor {'configuration': {'bdet': {'data': {}, 'timestamps': {}, 'data_keys': {}}}, 'data_keys': {'bdet': {'source': 'sim://pattern-generator-hdf-file', 'shape': [1, 240, 320], 'dtype': 'array', 'dtype_numpy': '|u1', 'external': 'STREAM:', 'object_name': 'bdet'}, 'bdet-sum': {'source': 'sim://pattern-generator-hdf-file', 'shape': [], 'dtype': 'number', 'dtype_numpy': '<i8', 'external': 'STREAM:', 'object_name': 'bdet'}}, 'name': 'primary', 'object_keys': {'bdet': ['bdet', 'bdet-sum']}, 'run_start': '0b8a6639-1879-4d4c-a661-fdc47e3c892a', 'time': 1745574323.9012275, 'uid': 'a58a9ec7-7eed-4369-8f90-be708b7a622b', 'hints': {'bdet': {'fields': ['bdet']}}}
stream_resource {'uid': '77e76917-282a-4287-b11b-b44e92cae75d', 'data_key': 'bdet', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/74a87546-9d91-4f1c-b4dd-ba97f8e437b3.h5', 'parameters': {'dataset': '/entry/data/data', 'chunk_shape': (1024,)}, 'run_start': '0b8a6639-1879-4d4c-a661-fdc47e3c892a'}
stream_resource {'uid': '6549238f-eb0d-4ded-9222-ad0e92414b32', 'data_key': 'bdet-sum', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/74a87546-9d91-4f1c-b4dd-ba97f8e437b3.h5', 'parameters': {'dataset': '/entry/sum', 'chunk_shape': (1024,)}, 'run_start': '0b8a6639-1879-4d4c-a661-fdc47e3c892a'}
stream_datum {'stream_resource': '77e76917-282a-4287-b11b-b44e92cae75d', 'uid': '77e76917-282a-4287-b11b-b44e92cae75d/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': 'a58a9ec7-7eed-4369-8f90-be708b7a622b'}
stream_datum {'stream_resource': '6549238f-eb0d-4ded-9222-ad0e92414b32', 'uid': '6549238f-eb0d-4ded-9222-ad0e92414b32/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': 'a58a9ec7-7eed-4369-8f90-be708b7a622b'}
|         1 | 09:45:23.9 |
event {'uid': '621e0a5f-4f14-4372-8959-65625bd2b15b', 'time': 1745574323.9026084, 'data': {}, 'timestamps': {}, 'seq_num': 1, 'filled': {}, 'descriptor': 'a58a9ec7-7eed-4369-8f90-be708b7a622b'}
stream_datum {'stream_resource': '77e76917-282a-4287-b11b-b44e92cae75d', 'uid': '77e76917-282a-4287-b11b-b44e92cae75d/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': 'a58a9ec7-7eed-4369-8f90-be708b7a622b'}
stream_datum {'stream_resource': '6549238f-eb0d-4ded-9222-ad0e92414b32', 'uid': '6549238f-eb0d-4ded-9222-ad0e92414b32/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': 'a58a9ec7-7eed-4369-8f90-be708b7a622b'}
|         2 | 09:45:23.9 |
event {'uid': '51d537b8-6fc5-495d-a53f-dfd061eea209', 'time': 1745574323.9073179, 'data': {}, 'timestamps': {}, 'seq_num': 2, 'filled': {}, 'descriptor': 'a58a9ec7-7eed-4369-8f90-be708b7a622b'}
+-----------+------------+
generator my_count_plan_with_prepare ['0b8a6639'] (scan num: 5)



stop {'uid': '445f0f07-6fac-4db8-b216-62675197e65f', 'time': 1745574323.9076803, 'run_start': '0b8a6639-1879-4d4c-a661-fdc47e3c892a', 'exit_status': 'success', 'reason': '', 'num_events': {'primary': 2}}
Out[8]: RunEngineResult(run_start_uids=('0b8a6639-1879-4d4c-a661-fdc47e3c892a',), plan_result='0b8a6639-1879-4d4c-a661-fdc47e3c892a', exit_status='success', interrupted=False, reason='', exception=None)

This also moves the work of setting up the detector from the first call of trigger() to the prepare() call. We can also move the creation of the descriptor earlier, so there is no extra work to do on the first call to trigger():

In [9]: from ophyd_async.core import TriggerInfo

In [10]: @bpp.stage_decorator([bdet])
   ....: @bpp.run_decorator()
   ....: def my_count_plan_with_prepare():
   ....:     yield from bps.prepare(bdet, TriggerInfo(), wait=True)
   ....:     yield from bps.declare_stream(bdet, name="primary")
   ....:     for i in range(2):
   ....:         yield from bps.trigger_and_read([bdet])
   ....: 

In [11]: RE(my_count_plan_with_prepare(), print)


Transient Scan ID: 6     Time: 2025-04-25 09:45:24
Persistent Unique Scan ID: '817d4a50-489b-418f-97c7-67744d67c165'
start {'uid': '817d4a50-489b-418f-97c7-67744d67c165', 'time': 1745574324.0202327, 'versions': {'ophyd': '1.10.5', 'bluesky': '1.13.1'}, 'scan_id': 6, 'plan_type': 'generator', 'plan_name': 'my_count_plan_with_prepare'}
New stream: 'primary'
+-----------+------------+
|   seq_num |       time |
+-----------+------------+
descriptor {'configuration': {'bdet': {'data': {}, 'timestamps': {}, 'data_keys': {}}}, 'data_keys': {'bdet': {'source': 'sim://pattern-generator-hdf-file', 'shape': [1, 240, 320], 'dtype': 'array', 'dtype_numpy': '|u1', 'external': 'STREAM:', 'object_name': 'bdet'}, 'bdet-sum': {'source': 'sim://pattern-generator-hdf-file', 'shape': [], 'dtype': 'number', 'dtype_numpy': '<i8', 'external': 'STREAM:', 'object_name': 'bdet'}}, 'name': 'primary', 'object_keys': {'bdet': ['bdet', 'bdet-sum']}, 'run_start': '817d4a50-489b-418f-97c7-67744d67c165', 'time': 1745574324.02358, 'uid': 'b3ded388-8114-4f87-842d-b46396535cbe', 'hints': {'bdet': {'fields': ['bdet']}}}
stream_resource {'uid': 'd61986c8-1113-4445-b31a-1813e8fc561e', 'data_key': 'bdet', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/65d162f7-153c-4b04-85d5-049270c60cb1.h5', 'parameters': {'dataset': '/entry/data/data', 'chunk_shape': (1024,)}, 'run_start': '817d4a50-489b-418f-97c7-67744d67c165'}
stream_resource {'uid': 'e162107c-59b6-4fc1-9504-61201130fd4f', 'data_key': 'bdet-sum', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/65d162f7-153c-4b04-85d5-049270c60cb1.h5', 'parameters': {'dataset': '/entry/sum', 'chunk_shape': (1024,)}, 'run_start': '817d4a50-489b-418f-97c7-67744d67c165'}
stream_datum {'stream_resource': 'd61986c8-1113-4445-b31a-1813e8fc561e', 'uid': 'd61986c8-1113-4445-b31a-1813e8fc561e/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': 'b3ded388-8114-4f87-842d-b46396535cbe'}
stream_datum {'stream_resource': 'e162107c-59b6-4fc1-9504-61201130fd4f', 'uid': 'e162107c-59b6-4fc1-9504-61201130fd4f/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': 'b3ded388-8114-4f87-842d-b46396535cbe'}
|         1 | 09:45:24.1 |
event {'uid': '170b8433-e1b8-4c83-a5e5-6cb59e7c2623', 'time': 1745574324.186884, 'data': {}, 'timestamps': {}, 'seq_num': 1, 'filled': {}, 'descriptor': 'b3ded388-8114-4f87-842d-b46396535cbe'}
stream_datum {'stream_resource': 'd61986c8-1113-4445-b31a-1813e8fc561e', 'uid': 'd61986c8-1113-4445-b31a-1813e8fc561e/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': 'b3ded388-8114-4f87-842d-b46396535cbe'}
stream_datum {'stream_resource': 'e162107c-59b6-4fc1-9504-61201130fd4f', 'uid': 'e162107c-59b6-4fc1-9504-61201130fd4f/1', 'seq_nums': {'start': 2, 'stop': 3}, 'indices': {'start': 1, 'stop': 2}, 'descriptor': 'b3ded388-8114-4f87-842d-b46396535cbe'}
|         2 | 09:45:24.2 |
event {'uid': '015dbc5d-847d-4046-9c26-962ccf5e6943', 'time': 1745574324.2910306, 'data': {}, 'timestamps': {}, 'seq_num': 2, 'filled': {}, 'descriptor': 'b3ded388-8114-4f87-842d-b46396535cbe'}
+-----------+------------+
generator my_count_plan_with_prepare ['817d4a50'] (scan num: 6)



stop {'uid': 'f232e2b2-f871-4f05-9b03-18c336195e7a', 'time': 1745574324.2913404, 'run_start': '817d4a50-489b-418f-97c7-67744d67c165', 'exit_status': 'success', 'reason': '', 'num_events': {'primary': 2}}
Out[11]: RunEngineResult(run_start_uids=('817d4a50-489b-418f-97c7-67744d67c165',), plan_result='817d4a50-489b-418f-97c7-67744d67c165', exit_status='success', interrupted=False, reason='', exception=None)

Run a fly scan and investigate the documents#

The above demonstrates the detector portion of a step scan, letting the things you want to scan settle before taking data from the detector, and doing this at every point of the scan. Our filewriting detector also supports the ability to fly scan it, taking data while you are scanning other things. To do this, it implements the bluesky.protocols.Flyable protocol, which allows us to kickoff() a series of images, then wait until it is complete():

In [12]: @bpp.stage_decorator([bdet])
   ....: @bpp.run_decorator()
   ....: def fly_plan():
   ....:     yield from bps.prepare(bdet, TriggerInfo(number_of_triggers=7), wait=True)
   ....:     yield from bps.declare_stream(bdet, name="primary")
   ....:     yield from bps.kickoff(bdet, wait=True)
   ....:     yield from bps.collect_while_completing(flyers=[bdet], dets=[bdet], flush_period=0.5)
   ....: 

In [13]: RE(fly_plan(), print)


Transient Scan ID: 7     Time: 2025-04-25 09:45:24
Persistent Unique Scan ID: '3962734c-0056-4833-9c74-7c756194a75c'
start {'uid': '3962734c-0056-4833-9c74-7c756194a75c', 'time': 1745574324.4041095, 'versions': {'ophyd': '1.10.5', 'bluesky': '1.13.1'}, 'scan_id': 7, 'plan_type': 'generator', 'plan_name': 'fly_plan'}
New stream: 'primary'
+-----------+------------+
|   seq_num |       time |
+-----------+------------+
descriptor {'configuration': {'bdet': {'data': {}, 'timestamps': {}, 'data_keys': {}}}, 'data_keys': {'bdet': {'source': 'sim://pattern-generator-hdf-file', 'shape': [1, 240, 320], 'dtype': 'array', 'dtype_numpy': '|u1', 'external': 'STREAM:', 'object_name': 'bdet'}, 'bdet-sum': {'source': 'sim://pattern-generator-hdf-file', 'shape': [], 'dtype': 'number', 'dtype_numpy': '<i8', 'external': 'STREAM:', 'object_name': 'bdet'}}, 'name': 'primary', 'object_keys': {'bdet': ['bdet', 'bdet-sum']}, 'run_start': '3962734c-0056-4833-9c74-7c756194a75c', 'time': 1745574324.407799, 'uid': 'fa0e0056-a488-4755-9f93-ed0f4df390d3', 'hints': {'bdet': {'fields': ['bdet']}}}
stream_resource {'uid': '972f6b41-c0d7-4d7e-8ae9-e0faa7d814a1', 'data_key': 'bdet', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/b76c564c-4921-44ac-beb6-f141c25a7752.h5', 'parameters': {'dataset': '/entry/data/data', 'chunk_shape': (1024,)}, 'run_start': '3962734c-0056-4833-9c74-7c756194a75c'}
stream_resource {'uid': 'f0c23074-c25a-444f-85b5-2e5c415123b6', 'data_key': 'bdet-sum', 'mimetype': 'application/x-hdf5', 'uri': 'file://localhost/tmp/tmpmfq_rlyt/b76c564c-4921-44ac-beb6-f141c25a7752.h5', 'parameters': {'dataset': '/entry/sum', 'chunk_shape': (1024,)}, 'run_start': '3962734c-0056-4833-9c74-7c756194a75c'}
stream_datum {'stream_resource': '972f6b41-c0d7-4d7e-8ae9-e0faa7d814a1', 'uid': '972f6b41-c0d7-4d7e-8ae9-e0faa7d814a1/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': 'fa0e0056-a488-4755-9f93-ed0f4df390d3'}
stream_datum {'stream_resource': 'f0c23074-c25a-444f-85b5-2e5c415123b6', 'uid': 'f0c23074-c25a-444f-85b5-2e5c415123b6/0', 'seq_nums': {'start': 1, 'stop': 2}, 'indices': {'start': 0, 'stop': 1}, 'descriptor': 'fa0e0056-a488-4755-9f93-ed0f4df390d3'}
+-----------+------------+
generator fly_plan ['3962734c'] (scan num: 7)



stop {'uid': 'b8b956ff-512c-41f5-b113-42d654779069', 'time': 1745574324.5717409, 'run_start': '3962734c-0056-4833-9c74-7c756194a75c', 'exit_status': 'success', 'reason': '', 'num_events': {'primary': 1}}
Out[13]: RunEngineResult(run_start_uids=('3962734c-0056-4833-9c74-7c756194a75c',), plan_result='3962734c-0056-4833-9c74-7c756194a75c', exit_status='success', interrupted=False, reason='', exception=None)

As before, we see the start, descriptor, and pair of stream resources, but this time we don’t see any event documents. Also, even though we asked for 7 frames from each of the 2 streams, we only got 2 stream datums for each stream.

What is happening is that instead of triggering, waiting, and publishing a single frame, we are setting up the detector to take 7 frames without stopping, then at the flush_period of 0.5s emitting a stream datum with the frames that have been captured. If we inspect the stream datum documents for each stream we see that:

The first has 'indices': {'start': 0, 'stop': 4}
The second has 'indices': {'start': 4, 'stop': 7}

This behavior allows us to scale up the framerate of the detector without scaling up the number of documents emitted: whether the detector goes at 10Hz or 10MHz it still only emits one stream datum per stream per flush period, just with different numbers in the indices field.

Look at the Device implementations#

Now we’ll have a look at the code to see how we implement one of these detectors:

`SimBlobDetector`#

from collections.abc import Sequence

from ophyd_async.core import PathProvider, SignalR, StandardDetector

from ._blob_detector_controller import BlobDetectorController
from ._blob_detector_writer import BlobDetectorWriter
from ._pattern_generator import PatternGenerator


class SimBlobDetector(StandardDetector):
    """Simulates a detector and writes Blobs to file."""

    def __init__(
        self,
        path_provider: PathProvider,
        pattern_generator: PatternGenerator | None = None,
        config_sigs: Sequence[SignalR] = (),
        name: str = "",
    ) -> None:
        self.pattern_generator = pattern_generator or PatternGenerator()

        super().__init__(
            controller=BlobDetectorController(
                pattern_generator=self.pattern_generator,
            ),
            writer=BlobDetectorWriter(
                pattern_generator=self.pattern_generator,
                path_provider=path_provider,
            ),
            config_sigs=config_sigs,
            name=name,
        )

It derives from StandardDetector which is a utility baseclass that implements the protocols we have mentioned so far in this tutorial. It uses a pair of logic classes to provide behavior for each protocol verb:

DetectorController to setup the exposure and trigger mode of the detector, arm it, and wait for it to complete
DetectorWriter to tell the detector to open a file, describe the datasets it will write, and wait for it to have written a given number of frames

In this case, we have a Controller and Writer class written just for this simulation, both taking a reference to the pattern generator that provides methods for both detector control and file writing. In other cases the detector control and filewriting may be handled by different sub-devices that talk to different parts of the control system. The job of the top level detector class is to take the arguments that the controller and writer need and distribute them, passing the instances to the superclass init.

Now let’s look at the underlying classes that define the detector behavior:

`BlobDetectorControl`#

First we have BlobDetectorController, a DetectorController subclass:

import asyncio
from contextlib import suppress

from ophyd_async.core import DetectorController, DetectorTrigger, TriggerInfo

from ._pattern_generator import PatternGenerator


class BlobDetectorController(DetectorController):
    def __init__(self, pattern_generator: PatternGenerator):
        self.pattern_generator = pattern_generator
        self.trigger_info: TriggerInfo | None = None
        self.task: asyncio.Task | None = None

    def get_deadtime(self, exposure):
        return 0.001

    async def prepare(self, trigger_info: TriggerInfo):
        # This is a simulation, so only support intenal triggering
        if trigger_info.trigger != DetectorTrigger.INTERNAL:
            raise RuntimeError(f"{trigger_info.trigger} not supported by {self}")
        # Just hold onto the trigger info until we need it
        self.trigger_info = trigger_info

    async def arm(self):
        if self.trigger_info is None:
            raise RuntimeError(f"prepare() not called on {self}")
        livetime = self.trigger_info.livetime or 0.1
        # Start a background process off writing the images to file
        coro = self.pattern_generator.write_images_to_file(
            exposure=livetime,
            period=livetime + self.trigger_info.deadtime,
            number_of_frames=self.trigger_info.total_number_of_exposures,
        )
        self.task = asyncio.create_task(coro)

    async def wait_for_idle(self):
        # Wait for the background task to complete
        if self.task:
            await self.task

    async def disarm(self):
        # Stop the background task and wait for it to finish
        if self.task:
            self.task.cancel()
            with suppress(asyncio.CancelledError):
                await self.task
            self.task = None

It’s job is to control the acquisition process on the detector, starting and stopping the collection of data:

prepare() takes a TriggerInfo which details everything the detector needs to know about the upcoming acquisition. In this case we just store it for later use.
arm() starts the acquisition process that has been prepared. In this case we create a background task that will write our simulation images to file.
wait_for_idle() waits for that acquisition process to be complete.
disarm() interrupts the acquisition process, and then waits for it to complete.

`BlobDetectorWriter`#

Then we have BlobDetectorWriter, a DetectorWriter subclass:

from collections.abc import AsyncGenerator, AsyncIterator
from pathlib import Path

import numpy as np
from bluesky.protocols import Hints, StreamAsset
from event_model import DataKey

from ophyd_async.core import (
    DetectorWriter,
    HDFDatasetDescription,
    HDFDocumentComposer,
    PathProvider,
)

from ._pattern_generator import DATA_PATH, SUM_PATH, PatternGenerator

WIDTH = 320
HEIGHT = 240


class BlobDetectorWriter(DetectorWriter):
    def __init__(
        self,
        pattern_generator: PatternGenerator,
        path_provider: PathProvider,
    ) -> None:
        self.pattern_generator = pattern_generator
        self.path_provider = path_provider
        self.path: Path | None = None
        self.composer: HDFDocumentComposer | None = None
        self.datasets: list[HDFDatasetDescription] = []

    async def open(self, name: str, exposures_per_event: int = 1) -> dict[str, DataKey]:
        path_info = self.path_provider(name)
        self.path = path_info.directory_path / f"{path_info.filename}.h5"
        self.pattern_generator.open_file(self.path, WIDTH, HEIGHT)
        self.exposures_per_event = exposures_per_event
        # We know it will write data and sum, so emit those
        self.datasets = [
            HDFDatasetDescription(
                data_key=name,
                dataset=DATA_PATH,
                shape=(exposures_per_event, HEIGHT, WIDTH),
                dtype_numpy=np.dtype(np.uint8).str,
                chunk_shape=(1024,),
            ),
            HDFDatasetDescription(
                data_key=f"{name}-sum",
                dataset=SUM_PATH,
                shape=(exposures_per_event,) if exposures_per_event > 1 else (),
                dtype_numpy=np.dtype(np.int64).str,
                chunk_shape=(1024,),
            ),
        ]
        self.composer = None
        describe = {
            ds.data_key: DataKey(
                source="sim://pattern-generator-hdf-file",
                shape=list(ds.shape),
                dtype="array"
                if exposures_per_event > 1 or len(ds.shape) > 1
                else "number",
                dtype_numpy=ds.dtype_numpy,
                external="STREAM:",
            )
            for ds in self.datasets
        }
        return describe

    def get_hints(self, name: str) -> Hints:
        """The hints to be used for the detector."""
        return {"fields": [name]}

    async def get_indices_written(self) -> int:
        return self.pattern_generator.get_last_index() // self.exposures_per_event

    async def observe_indices_written(
        self, timeout: float
    ) -> AsyncGenerator[int, None]:
        while True:
            yield self.pattern_generator.get_last_index() // self.exposures_per_event
            await self.pattern_generator.wait_for_next_index(timeout)

    async def collect_stream_docs(
        self, name: str, indices_written: int
    ) -> AsyncIterator[StreamAsset]:
        # When we have written something to the file
        if indices_written:
            # Only emit stream resource the first time we see frames in
            # the file
            if not self.composer:
                if not self.path:
                    raise RuntimeError(f"open() not called on {self}")
                self.composer = HDFDocumentComposer(self.path, self.datasets)
                for doc in self.composer.stream_resources():
                    yield "stream_resource", doc
            for doc in self.composer.stream_data(indices_written):
                yield "stream_datum", doc

    async def close(self) -> None:
        self.pattern_generator.close_file()

Its job is to control the file writing process on the detector, which may or may not be coupled to the acquisition process:

open() tells the detector to open a file, and returns information about the datasets that it will write
hints gives a list of the datasets that are interesting to plot
get_indices_written() returns the last index written
observe_indices_written() repeatedly yields the last index written then waits for the next one to be written
collect_stream_docs() uses the HDFDocumentComposer to publish a document per dataset that says which frames have been written since the last call
close() tells the detector to close the file

Conclusion#

We have seen how to make a StandardDetector and how the DetectorController and DetectorWriter allow us to customise its behavior.