toolbox_scs.routines.boz

Beam splitting Off-axis Zone plate analysis routines.

Copyright (2021, 2022, 2023, 2024) SCS Team.

Module Contents

Classes

parameters

Parameters contains all input parameters for the BOZ corrections.

Functions

get_roi_pixel_pos(roi, params)	Compute fake or real pixel position of an roi from roi center.
bad_pixel_map(params)	Compute the bad pixels map.
inspect_dark(arr[, mean_th, std_th])	Inspect dark run data and plot diagnostic.
histogram_module(arr[, mask])	Compute a histogram of the 9 bits raw pixel values over a module.
inspect_histogram(arr[, arr_dark, mask, extra_lines])	Compute and plot a histogram of the 9 bits raw pixel values.
find_rois(data_mean, threshold[, extended])	Find rois from 3 beams configuration.
find_rois_from_params(params)	Find rois from 3 beams configuration.
inspect_rois(data_mean, rois[, threshold, allrois])	Find rois from 3 beams configuration from mean module image.
compute_flat_field_correction(rois, params[, plot])
inspect_flat_field_domain(avg, rois, prod_th, ratio_th)	Extract beams roi from average image and compute the ratio.
inspect_plane_fitting(avg, rois[, domain, vmin, vmax])
plane_fitting_domain(avg, rois, prod_th, ratio_th)	Extract beams roi, compute their ratio and the domain.
plane_fitting(params)	Fit the plane flat-field normalization.
ff_refine_crit(p, alpha, params, arr_dark, arr, tid, ...)	Criteria for the ff_refine_fit.
ff_refine_fit(params[, crit])	Refine the flat-field fit by minimizing data spread.
nl_domain(N, low, high)	Create the input domain where the non-linear correction defined.
nl_lut(domain, dy)	Compute the non-linear correction.
nl_crit(p, domain, alpha, arr_dark, arr, tid, rois, ...)	Criteria for the non linear correction.
nl_crit_sk(p, domain, alpha, arr_dark, arr, tid, rois, ...)	Non linear correction criteria, combining 'n' and 'p' as reference.
nl_fit(params, domain[, ff, crit])	Fit non linearities correction function.
inspect_nl_fit(res_fit)	Plot the progress of the fit.
snr(sig, ref[, methods, verbose])	Compute mean, std and SNR from transmitted and I0 signals.
inspect_Fnl(Fnl)	Plot the correction function Fnl.
inspect_correction(params[, gain])	Comparison plot of the different corrections.
inspect_correction_sk(params, ff[, gain])	Comparison plot of the different corrections, combining 'n' and 'p'.
load_dssc_module(proposalNB, runNB[, moduleNB, ...])	Load single module dssc data as dask array.
average_module(arr[, dark, ret, mask, sat_roi, ...])	Compute the average or std over a module.
process_module(arr, tid, dark, rois[, mask, ...])	Process one module and extract roi intensity.
process(Fmodel, arr_dark, arr, tid, rois, mask, flat_field)	Process dark and run data with corrections.
inspect_saturation(data, gain[, Nbins])	Plot roi integrated histogram of the data with saturation

class toolbox_scs.routines.boz.parameters(proposal, darkrun, run, module, gain, drop_intra_darks=True)

Parameters contains all input parameters for the BOZ corrections.

This is used in beam splitting off-axis zone plate spectrocopy analysis as well as the during the determination of correction parameters themselves to ensure they can be reproduced.

Inputs

proposal: int, proposal number darkrun: int, run number for the dark run run: int, run number for the data run module: int, DSSC module number gain: float, number of ph per bin drop_intra_darks: drop every second DSSC frame

dask_load_persistently(dark_data_size_Gb=None, data_size_Gb=None)

Load dask data array in memory.

Inputs

dark_data_size_Gb: float, optional size of dark to load in memory,

in Gb

data_size_Gb: float, optional size of data to load in memory, in Gb

use_gpu()

set_mask(arr)

Set mask of bad pixels.

Inputs

arr: either a boolean array of a DSSC module image or a list of bad

pixel indices

get_mask()

Get the boolean array bad pixel of a DSSC module.

get_mask_idx()

Get the list of bad pixel indices.

flat_field_guess(guess=None)

Set the flat-field guess parameter for the fit and returns it.

Inputs

guess: a list of 8 floats, the 4 first to define the plane

ax+by+cz+d=0 for ‘n’ beam and the 4 last for the ‘p’ beam in case mirror symmetry is disbaled

set_flat_field(ff_params, ff_type='plane', prod_th=None, ratio_th=None)

Set the flat-field plane definition.

Inputs

ff_params: list of parameters ff_type: string identifying the type of flat field normalization,

default is ‘plane’.

get_flat_field()

Get the flat-field plane definition.

set_Fnl(Fnl)

Set the non-linear correction function.

get_Fnl()

Get the non-linear correction function.

save(path='./')

Save the parameters as a JSON file.

Inputs

path: str, where to save the file, default to ‘./’

classmethod load(fname)

Load parameters from a JSON file.

Inputs

fname: string, name a the JSON file to load

__str__()

Return str(self).

toolbox_scs.routines.boz.get_roi_pixel_pos(roi, params)

Compute fake or real pixel position of an roi from roi center.

Inputs:

roi: dictionnary params: parameters

Returns:

X, Y: 1-d array of pixel position.

toolbox_scs.routines.boz.bad_pixel_map(params)

Compute the bad pixels map.

Inputs

params: parameters

rtype:

bad pixel map

toolbox_scs.routines.boz.inspect_dark(arr, mean_th=(None, None), std_th=(None, None))

Inspect dark run data and plot diagnostic.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) mean_th: tuple of threshold (low, high), default (None, None), to compute

a mask of good pixels for which the mean dark value lie inside this range

std_th: tuple of threshold (low, high), default (None, None), to compute a

mask of bad pixels for which the dark std value lie inside this range

returns:

fig

rtype:

matplotlib figure

toolbox_scs.routines.boz.histogram_module(arr, mask=None)

Compute a histogram of the 9 bits raw pixel values over a module.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) mask: optional bad pixel mask

rtype:

histogram

toolbox_scs.routines.boz.inspect_histogram(arr, arr_dark=None, mask=None, extra_lines=False)

Compute and plot a histogram of the 9 bits raw pixel values.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) arr: dask array of reshaped dssc dark data (trainId, pulseId, x, y) mask: optional bad pixel mask extra_lines: boolean, default False, plot extra lines at period values

returns:

• (h, hd) (histogram of arr, arr_dark)

• figure

toolbox_scs.routines.boz.find_rois(data_mean, threshold, extended=False)

Find rois from 3 beams configuration.

Inputs

data_mean: dark corrected average image threshold: threshold value to find beams extended: boolean, True to define additional ASICS based rois

returns:

rois

rtype:

dictionnary of rois

toolbox_scs.routines.boz.find_rois_from_params(params)

Find rois from 3 beams configuration.

Inputs

params: parameters

returns:

rois

rtype:

dictionnary of rois

toolbox_scs.routines.boz.inspect_rois(data_mean, rois, threshold=None, allrois=False)

Find rois from 3 beams configuration from mean module image.

Inputs

data_mean: mean module image threshold: float, default None, threshold value used to detect beams

boundaries

allrois: boolean, default False, plot all rois defined in rois or only the

main ones ([‘n’, ‘0’, ‘p’])

rtype:

matplotlib figure

toolbox_scs.routines.boz.compute_flat_field_correction(rois, params, plot=False)

toolbox_scs.routines.boz.inspect_flat_field_domain(avg, rois, prod_th, ratio_th, vmin=None, vmax=None)

Extract beams roi from average image and compute the ratio.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary or ROIs prod_th, ratio_th: tuple of floats for low and high threshold on

product and ratio

vmin: imshow vmin level, default None will use 5 percentile value vmax: imshow vmax level, default None will use 99.8 percentile value

returns:

• fig (matplotlib figure plotted)

• domain (a tuple (n_m, p_m) of domain for the ‘n’ and ‘p’ order)

toolbox_scs.routines.boz.inspect_plane_fitting(avg, rois, domain=None, vmin=None, vmax=None)

toolbox_scs.routines.boz.plane_fitting_domain(avg, rois, prod_th, ratio_th)

Extract beams roi, compute their ratio and the domain.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary or rois containing the 3 beams [‘n’, ‘0’, ‘p’] with ‘0’

as the reference beam in the middle

prod_th: float tuple, low and hight threshold level to determine the plane

fitting domain on the product image of the orders

ratio_th: float tuple, low and high threshold level to determine the plane

fitting domain on the ratio image of the orders

returns:

• n (img ratio ‘n’/’0’)

• n_m (mask where the the product ‘n’’0’ is higher than 5 indicting that the*) – img ratio ‘n’/’0’ is defined

• p (img ratio ‘p’/’0’)

• p_m (mask where the the product ‘p’’0’ is higher than 5 indicting that the*) – img ratio ‘p’/’0’ is defined

toolbox_scs.routines.boz.plane_fitting(params)

Fit the plane flat-field normalization.

Inputs

params: parameters

returns:

res – defines the plane as a*x + b*y + c*z + d = 0

rtype:

the minimization result. The fitted vector res.x = [a, b, c, d]

toolbox_scs.routines.boz.ff_refine_crit(p, alpha, params, arr_dark, arr, tid, rois, mask, sat_level=511)

Criteria for the ff_refine_fit.

Inputs

p: ff plane params: parameters arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels sat_level: integer, default 511, at which level pixel begin to saturate

rtype:

sum of standard deviation on binned 0th order intensity

toolbox_scs.routines.boz.ff_refine_fit(params, crit=ff_refine_crit)

Refine the flat-field fit by minimizing data spread.

Inputs

params: parameters

returns:

• res (scipy minimize result. res.x is the optimized parameters)

• fitrres (iteration index arrays of criteria results for) – [alpha=0, alpha, alpha=1]

toolbox_scs.routines.boz.nl_domain(N, low, high)

Create the input domain where the non-linear correction defined.

Inputs

N: integer, number of control points or intervals low: input values below or equal to low will not be corrected high: input values higher or equal to high will not be corrected

rtype:

array of 2**9 integer values with N segments

toolbox_scs.routines.boz.nl_lut(domain, dy)

Compute the non-linear correction.

Inputs

domain: input domain where dy is defined. For zero no correction is

defined. For non-zero value x, dy[x] is applied.

dy: a vector of deviation from linearity on control point homogeneously

dispersed over 9 bits.

returns:

F_INL – lookup table with 9 bits integer input

rtype:

default None, non linear correction function given as a

toolbox_scs.routines.boz.nl_crit(p, domain, alpha, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)

Criteria for the non linear correction.

Inputs

p: vector of dy non linear correction domain: domain over which the non linear correction is defined alpha: float, coefficient scaling the cost of the correction function

in the criterion

arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

returns:

• (1.0 - alpha)*err1 + alpha*err2, where err1 is the 1e8 times the mean of

• error squared from a transmission of 1.0 and err2 is the sum of the square

• of the deviation from the ideal detector response.

toolbox_scs.routines.boz.nl_crit_sk(p, domain, alpha, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)

Non linear correction criteria, combining ‘n’ and ‘p’ as reference.

Inputs

p: vector of dy non linear correction domain: domain over which the non linear correction is defined alpha: float, coefficient scaling the cost of the correction function

in the criterion

arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

returns:

• (1.0 - alpha)*err1 + alpha*err2, where err1 is the 1e8 times the mean of

• error squared from a transmission of 1.0 and err2 is the sum of the square

• of the deviation from the ideal detector response.

toolbox_scs.routines.boz.nl_fit(params, domain, ff=None, crit=None)

Fit non linearities correction function.

Inputs

params: parameters domain: array of index ff: array, flat field correction crit: function, criteria function

returns:

• res (scipy minimize result. res.x is the optimized parameters)

• fitrres (iteration index arrays of criteria results for) – [alpha=0, alpha, alpha=1]

toolbox_scs.routines.boz.inspect_nl_fit(res_fit)

Plot the progress of the fit.

Inputs

res_fit:

rtype:

matplotlib figure

toolbox_scs.routines.boz.snr(sig, ref, methods=None, verbose=False)

Compute mean, std and SNR from transmitted and I0 signals.

Inputs

sig: 1D signal samples ref: 1D reference samples methods: None by default or list of strings to select which methods to use.

Possible values are ‘direct’, ‘weighted’, ‘diff’. In case of None, all methods will be calculated.

verbose: booleand, if True prints calculated values

returns:

• dictionnary of [methods][value] where value is ‘mu’ for mean and ‘s’ for

• standard deviation.

toolbox_scs.routines.boz.inspect_Fnl(Fnl)

Plot the correction function Fnl.

Inputs

Fnl: non linear correction function lookup table

rtype:

matplotlib figure

toolbox_scs.routines.boz.inspect_correction(params, gain=None)

Comparison plot of the different corrections.

Inputs

params: parameters gain: float, default None, DSSC gain in ph/bin

rtype:

matplotlib figure

toolbox_scs.routines.boz.inspect_correction_sk(params, ff, gain=None)

Comparison plot of the different corrections, combining ‘n’ and ‘p’.

Inputs

params: parameters gain: float, default None, DSSC gain in ph/bin

rtype:

matplotlib figure

toolbox_scs.routines.boz.load_dssc_module(proposalNB, runNB, moduleNB=15, subset=slice(None), drop_intra_darks=True, persist=False, data_size_Gb=None)

Load single module dssc data as dask array.

Inputs

proposalNB: proposal number runNB: run number moduleNB: default 15, module number subset: default slice(None), subset of trains to load drop_intra_darks: boolean, default True, remove intra darks from the data persist: default False, load all data persistently in memory data_size_Gb: float, if persist is True, can optionaly restrict

the amount of data loaded for dark data and run data in Gb

returns:

• arr (dask array of reshaped dssc data (trainId, pulseId, x, y))

• tid (array of train id number)

toolbox_scs.routines.boz.average_module(arr, dark=None, ret='mean', mask=None, sat_roi=None, sat_level=300, F_INL=None)

Compute the average or std over a module.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) dark: default None, dark to be substracted ret: string, either ‘mean’ to compute the mean or ‘std’ to compute the

standard deviation

mask: default None, mask of bad pixels to ignore sat_roi: roi over which to check for pixel with values larger than

sat_level to drop the image from the average or std

sat_level: int, minimum pixel value for a pixel to be considered saturated F_INL: default None, non linear correction function given as a

lookup table with 9 bits integer input

rtype:

average or standard deviation image

toolbox_scs.routines.boz.process_module(arr, tid, dark, rois, mask=None, sat_level=511, flat_field=None, F_INL=None, use_gpu=False)

Process one module and extract roi intensity.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) tid: array of train id number dark: pulse resolved dark image to remove rois: dictionnary of rois mask: default None, mask of ignored pixels sat_level: integer, default 511, at which level pixel begin to saturate flat_field: default None, flat-field correction F_INL: default None, non-linear correction function given as a

lookup table with 9 bits integer input

rtype:

dataset of extracted pulse and train resolved roi intensities.

toolbox_scs.routines.boz.process(Fmodel, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)

Process dark and run data with corrections.

Inputs

Fmodel: correction lookup table arr_dark: dark data arr: data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask of good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

rtype:

roi extracted intensities

toolbox_scs.routines.boz.inspect_saturation(data, gain, Nbins=200)

Plot roi integrated histogram of the data with saturation

Inputs

data: xarray of roi integrated DSSC data gain: nominal DSSC gain in ph/bin Nbins: number of bins for the histogram, by default 200

returns:

• f (handle to the matplotlib figure)

• h (xarray of the histogram data)