Usage

Data Model

A typical bluesky Run has an Event Stream named 'primary', an Event Stream named 'baseline`', and potentially other Event Streams for signals that are monitored asynchronously during the Run. The names of these Event Streams are just convention, encoded by the built-in bluesky plans. Plans can define any Event Streams that they like.

A natural way to include dark frames with a Run is to add a 'dark' Event Stream. Because Events are timestamped, the 'dark' Events can be associated with 'primary' Events to produce dark-subtracted images. Each Run should have at least one 'dark' Event, and it may have more than one if a fresh dark frame is needed mid-run. The most direct way to achieve this is to write trigger_and_read(..., name='dark') into a custom plan:

import bluesky.plan_stubs as bps

def count_with_darkframe(detector, md=None):
    yield from bps.stage(detector)
    yield from bps.open_run(md=md)
    yield from bps.mv(shutter, 'closed')
    yield from bps.trigger_and_read([detector], name='dark')
    yield from bps.mv(shutter, 'open')
    yield from bps.trigger_and_read([detector])  # name='primary' by default
    yield from bps.close_run()
    yield from bps.unstage(detector)

This direct solution is best one for some circumstances. However, if you find yourself looking at the prospect of rewriting a large number of plans just to add this dark frame logic, it may be simpler to use a bluesky preprocessor. A preprocessor can augment or modify the steps in a plan. The DarkFramePreprocessor watches for a given detector to be triggered and inserts steps in the plan to acquire and/or record a dark frame when needed. Depending on how you configure it, it can reuse a given dark frame multiple times. Thus, it will not necessarily acquire a dark frame for every Run, but it will ensure that at least one ‘dark’ Event is recorded in every Run.

The preprocessor can be applied to specific plans, using Python’s decorator syntax

from bluesky.preprocessors import make_decorator

# Do this just once.
dark_frame_preprocessor = ... # See next section.
do_dark_frames = make_decorator(dark_frame_preprocessor)()

# And apply it to as many plans as you like.
@do_dark_frames
def my_custom_plan(...):
    ...

@do_dark_frames
def another_custom_plan(...):
    ...

or it can be applied to all plans.

# Do this just once.
dark_frame_preprocessor = ... # See next section.
RE.preprocessors.append(dark_frame_preprocessor)

This enables the user to use any built-in or user-defined plan and know that dark frames will automatically be included in the logic of the plan. Note that preprocessor will only have an effect is the detector of interest is used during the plan.

Initial Configuration

We need to know:

  1. How do you take a dark frame? Specifically…. What’s the relevant shutter? How do you close it? (Some think “0” is closed; others think “1” is closed; still others need a multi-step dance to open or close.) What’s the relevant detector?

  2. What are the rules for when to take a fresh dark frame and when to reuse one that has already been taken?

  3. If you would like to compute subtracted frames on the fly, where should the results go?

To address (1) define a bluesky plan that closes the shutter, takes an acquistion, and reopens the shutter. The last two lines in this example use a special mechanism, SnapshotDevice, to stash the acquisition where it can potentially be reused. (Later on we’ll set the rules for whether/how dark frames can be reused.)

import bluesky.plan_stubs as bps
import bluesky_darkframes

# This is some simulated hardware for demo purposes.
from bluesky_darkframes.sim import Shutter, DiffractionDetector
det = DiffractionDetector(name='det')
shutter = Shutter(name='shutter', value='open')

def dark_plan(detector):
    # Restage to ensure that dark frames goes into a separate file.
    yield from bps.unstage(detector)
    yield from bps.stage(detector)
    yield from bps.mv(shutter, 'closed')
    # The `group` parameter passed to trigger MUST start with
    # bluesky-darkframes-trigger.
    yield from bps.trigger(detector, group='bluesky-darkframes-trigger')
    yield from bps.wait('bluesky-darkframes-trigger')
    snapshot = bluesky_darkframes.SnapshotDevice(detector)
    yield from bps.mv(shutter, 'open')
    # Restage.
    yield from bps.unstage(detector)
    yield from bps.stage(detector)
    return snapshot

This is boilerplate bluesky and databroker setup not specificially related to dark-frames.

from bluesky import RunEngine
from databroker import Broker
from ophyd.sim import NumpySeqHandler

db = Broker.named('temp')
db.reg.register_handler('NPY_SEQ', NumpySeqHandler)
RE = RunEngine()
RE.subscribe(db.insert);

Here we set the rules for when to take fresh dark frames, (2). Examples:

# Always take a fresh dark frame at the beginning of each run.
dark_frame_preprocessor = bluesky_darkframes.DarkFramePreprocessor(
    dark_plan=dark_plan, detector=det, max_age=0)

# Take a dark frame if the last one we took is more than 30 seconds old.
dark_frame_preprocessor = bluesky_darkframes.DarkFramePreprocessor(
    dark_plan=dark_plan, detector=det, max_age=30)

# Take a fresh dark frame if the last one we took *with this exposure time*
# is more than 30 seconds old.
dark_frame_preprocessor = bluesky_darkframes.DarkFramePreprocessor(
    dark_plan=dark_plan, detector=det, max_age=30,
    locked_signals=[det.exposure_time])

# Always take a new dark frame if the exposure time was changed from the
# previous run, even if we took one with this exposure time on some earlier
# run. Also, re-take if the settings haven't changed but the last dark
# frame is older than 30 seconds.
dark_frame_preprocessor = bluesky_darkframes.DarkFramePreprocessor(
    dark_plan=dark_plan, detector=det, max_age=30,
    locked_signals=[det.exposure_time], limit=1)

We’ll pick one example and configure the RunEngine to apply it to all plans. This means that any plan, including user-defined ones, will automatically have dark frames included.

dark_frame_preprocessor = bluesky_darkframes.DarkFramePreprocessor(
    dark_plan=dark_plan, detector=det, max_age=30)
RE.preprocessors.append(dark_frame_preprocessor)

Acquire and Access Data

Let’s take some data.

from bluesky.plans import count

RE(count([det]))

('52aef1d8-e225-4c4d-a9e1-28c4469db514',)

And now let’s access the data and plot the raw “light” frame, the dark frame, and the difference between the two.

import matplotlib.pyplot as plt

light = list(db[-1].data('det_image'))[0]
dark = list(db[-1].data('det_image', stream_name='dark'))[0]
fig, axes = plt.subplots(1, 3)
titles = ('Light', 'Dark', 'Subtracted')
for image, ax, title in zip((light, dark, light - dark), axes, titles):
   ax.imshow(image);
   ax.set_title(title);
_images/usage_4_0.png

Export Subtracted Images

In this example we’ll export the data to a TIFF series, but it could equally well be written to any other storage format.

Export saved data

First we’ll define a convenience function.

from bluesky_darkframes import DarkSubtraction
from suitcase.tiff_series import Serializer

def export_subtracted_tiff_series(header, *args, **kwargs):
    subtractor = DarkSubtraction('det_image')
    with Serializer(*args, **kwargs) as serializer:
        for name, doc in header.documents(fill=True):
            name, doc = subtractor(name, doc)
            serializer(name, doc)

And now apply it to the data we just took.

export_subtracted_tiff_series(db[-1], 'exported_files/')

This exports the subtracted images (with ‘primary’ in the name) and the dark frames (with ‘dark’) in the name, which makes it possible to reconstruct the original if desired.

!ls exported_files

52aef1d8-e225-4c4d-a9e1-28c4469db514-dark-det_image-00000.tiff
52aef1d8-e225-4c4d-a9e1-28c4469db514-primary-det_image-00000.tiff

To customize the file name and other output options, see Serializer.

Export data during acquisition (streaming)

Here we use a event_model.RunRouter.

from bluesky_darkframes import DarkSubtraction
from event_model import RunRouter
from suitcase.tiff_series import Serializer

def factory(name, doc):
    # The problem this is solving is to store documents from this run long
    # enough to cross-reference them (e.g. light frames and dark frames),
    # and then tearing it down when we're done with this run.
    subtractor = DarkSubtraction('det_image')
    serializer = Serializer('live_exported_files/')

    # And by returning this function below, we are routing all other
    # documents *for this run* through here.
    def subtract_and_serialize(name, doc):
        name, doc = subtractor(name, doc)
        serializer(name, doc)

    return [subtract_and_serialize], []

rr = RunRouter([factory], db.reg.handler_reg)
RE.subscribe(rr);

Now take some data.

RE(count([det]))

('3bfe9214-b980-4f39-8ccd-048871a8e63c',)

And see that files have been generated.

!ls live_exported_files

3bfe9214-b980-4f39-8ccd-048871a8e63c-dark-det_image-00000.tiff
3bfe9214-b980-4f39-8ccd-048871a8e63c-primary-det_image-00000.tiff