Representation of Bluesky Runs in Tiled#

Tiled is a data management system that allows for the storage and retrieval of structured data along with the associated metadata.

The TiledWriter callback is designed specifically for converting Bluesky run documents into a format suitable for storage in a Tiled database.

It implicitly distinguishes between “internal” and “external” data. The internal data are associated with the Event documents generated during a run; typically this represents scalar measurements from sensors, motor positions, etc., which are stored in a table with columns corresponding to different data keys and each row representing a measurement at a single timestamp.

On the other hand, the external data are written by detectors directly to disk and usually take the form of images or multidimensional arrays. The references to the external files are provided in StreamResource (Resource in legacy implementations) documents, which register the corresponding array-like DataSources in Tiled. StreamDatum (or Datum) documents are processed via the mechanism of Consolidators and determine the correspondence between the indexing within these external arrays and the physically-meaningful sequence of timestamps.

The time dimension (that is, the sequence of measurements) is usually shared between internal and external data. Tiled handles this by writing all data from the same Bluesky stream into a container with a dedicated "composite" spec, which tells the Tiled client how the data are aligned. Each stream node’s metadata includes the specifications for the related data keys as well as the configuration parameters provided in the EventDescriptor document.

Finally, nodes for multiple streams are grouped together and placed into a container for the entire run; its metadata contains the Start and Stop documents. The Run container created by TiledWriter is designated with the BlueskyRun version 3.0 spec to enable its back-compatibility with legacy code via bluesky-tiled-plugins.

An example of the Tiled catalog structure for a Bluesky run might look like this:

BlueskyRun <Container ("BlueskyRun_v3")>
│
├─ baseline <Container ("composite")>
│       ├─ internal <Table>   <-- data from Event documents
│       ├─ image_1 <Array>    <-- external (registered) data
│       │
│       │      ...
│       └─ image_n <Array>
│
├─ primary <Container ("composite")>
│       ├─ internal <Table>
│       ├─ image_1 <Array>
│       │
│       │      ...
│       └─ image_n <Array>
│
└─ third_stream <Container ("composite")>

Note

To be able to use TiledWriter, the Tiled server must be configured with an SQL catalog and an SQL-backed storage database for tabular data.

This Page