toolbox_scs.routines

Submodules

• toolbox_scs.routines.Reflectivity
• toolbox_scs.routines.XAS
• toolbox_scs.routines.boz
• toolbox_scs.routines.knife_edge

Package Contents

Classes

parameters

Parameters contains all input parameters for the BOZ corrections.

Functions

xas(nrun[, bins, Iokey, Itkey, nrjkey, Iooffset, ...])	Compute the XAS spectra from a xarray nrun.
xasxmcd(dataP, dataN)	Compute XAS and XMCD from data with both magnetic field direction
get_roi_pixel_pos(roi, params)	Compute fake or real pixel position of an roi from roi center.
bad_pixel_map(params)	Compute the bad pixels map.
inspect_dark(arr[, mean_th, std_th])	Inspect dark run data and plot diagnostic.
histogram_module(arr[, mask])	Compute a histogram of the 9 bits raw pixel values over a module.
inspect_histogram(arr[, arr_dark, mask, extra_lines])	Compute and plot a histogram of the 9 bits raw pixel values.
find_rois(data_mean, threshold[, extended])	Find rois from 3 beams configuration.
find_rois_from_params(params)	Find rois from 3 beams configuration.
inspect_rois(data_mean, rois[, threshold, allrois])	Find rois from 3 beams configuration from mean module image.
compute_flat_field_correction(rois, params[, plot])
inspect_flat_field_domain(avg, rois, prod_th, ratio_th)	Extract beams roi from average image and compute the ratio.
inspect_plane_fitting(avg, rois[, domain, vmin, vmax])
plane_fitting_domain(avg, rois, prod_th, ratio_th)	Extract beams roi, compute their ratio and the domain.
plane_fitting(params)	Fit the plane flat-field normalization.
ff_refine_crit(p, alpha, params, arr_dark, arr, tid, ...)	Criteria for the ff_refine_fit.
ff_refine_fit(params[, crit])	Refine the flat-field fit by minimizing data spread.
nl_domain(N, low, high)	Create the input domain where the non-linear correction defined.
nl_lut(domain, dy)	Compute the non-linear correction.
nl_crit(p, domain, alpha, arr_dark, arr, tid, rois, ...)	Criteria for the non linear correction.
nl_crit_sk(p, domain, alpha, arr_dark, arr, tid, rois, ...)	Non linear correction criteria, combining 'n' and 'p' as reference.
nl_fit(params, domain[, ff, crit])	Fit non linearities correction function.
inspect_nl_fit(res_fit)	Plot the progress of the fit.
snr(sig, ref[, methods, verbose])	Compute mean, std and SNR from transmitted and I0 signals.
inspect_Fnl(Fnl)	Plot the correction function Fnl.
inspect_correction(params[, gain])	Comparison plot of the different corrections.
inspect_correction_sk(params, ff[, gain])	Comparison plot of the different corrections, combining 'n' and 'p'.
load_dssc_module(proposalNB, runNB[, moduleNB, ...])	Load single module dssc data as dask array.
average_module(arr[, dark, ret, mask, sat_roi, ...])	Compute the average or std over a module.
process_module(arr, tid, dark, rois[, mask, ...])	Process one module and extract roi intensity.
process(Fmodel, arr_dark, arr, tid, rois, mask, flat_field)	Process dark and run data with corrections.
inspect_saturation(data, gain[, Nbins])	Plot roi integrated histogram of the data with saturation
reflectivity(data[, Iokey, Irkey, delaykey, binWidth, ...])	Computes the reflectivity R = 100*(Ir/Io[pumped] / Ir/Io[unpumped] - 1)
knife_edge	Toolbox for SCS.

Attributes

__all__

toolbox_scs.routines.xas(nrun, bins=None, Iokey='SCS_SA3', Itkey='MCP3peaks', nrjkey='nrj', Iooffset=0, plot=False, fluorescence=False)

Compute the XAS spectra from a xarray nrun.

Inputs:

nrun: xarray of SCS data bins: an array of bin-edges or an integer number of

desired bins or a float for the desired bin width.

Iokey: string for the Io fields, typically ‘SCS_XGM’ Itkey: string for the It fields, typically ‘MCP3apd’ nrjkey: string for the nrj fields, typically ‘nrj’ Iooffset: offset to apply on Io plot: boolean, displays a XAS spectrum if True fluorescence: boolean, if True, absorption is the ratio,

if False, absorption is negative log

Outputs:

a dictionnary containing:

nrj: the bin centers muA: the absorption sigmaA: standard deviation on the absorption sterrA: standard error on the absorption muIo: the mean of the Io counts: the number of events in each bin

toolbox_scs.routines.xasxmcd(dataP, dataN)

Compute XAS and XMCD from data with both magnetic field direction Inputs:

dataP: structured array for positive field dataN: structured array for negative field

Outputs:

xas: structured array for the sum xmcd: structured array for the difference

class toolbox_scs.routines.parameters(proposal, darkrun, run, module, gain, drop_intra_darks=True)

Parameters contains all input parameters for the BOZ corrections.

This is used in beam splitting off-axis zone plate spectrocopy analysis as well as the during the determination of correction parameters themselves to ensure they can be reproduced.

Inputs

proposal: int, proposal number darkrun: int, run number for the dark run run: int, run number for the data run module: int, DSSC module number gain: float, number of ph per bin drop_intra_darks: drop every second DSSC frame

dask_load_persistently(dark_data_size_Gb=None, data_size_Gb=None)

Load dask data array in memory.

Inputs

dark_data_size_Gb: float, optional size of dark to load in memory,

in Gb

data_size_Gb: float, optional size of data to load in memory, in Gb

use_gpu()

set_mask(arr)

Set mask of bad pixels.

Inputs

arr: either a boolean array of a DSSC module image or a list of bad

pixel indices

get_mask()

Get the boolean array bad pixel of a DSSC module.

get_mask_idx()

Get the list of bad pixel indices.

flat_field_guess(guess=None)

Set the flat-field guess parameter for the fit and returns it.

Inputs

guess: a list of 8 floats, the 4 first to define the plane

ax+by+cz+d=0 for ‘n’ beam and the 4 last for the ‘p’ beam in case mirror symmetry is disbaled

set_flat_field(ff_params, ff_type='plane', prod_th=None, ratio_th=None)

Set the flat-field plane definition.

Inputs

ff_params: list of parameters ff_type: string identifying the type of flat field normalization,

default is ‘plane’.

get_flat_field()

Get the flat-field plane definition.

set_Fnl(Fnl)

Set the non-linear correction function.

get_Fnl()

Get the non-linear correction function.

save(path='./')

Save the parameters as a JSON file.

Inputs

path: str, where to save the file, default to ‘./’

classmethod load(fname)

Load parameters from a JSON file.

Inputs

fname: string, name a the JSON file to load

__str__()

Return str(self).

toolbox_scs.routines.get_roi_pixel_pos(roi, params)

Compute fake or real pixel position of an roi from roi center.

Inputs:

roi: dictionnary params: parameters

Returns:

X, Y: 1-d array of pixel position.

toolbox_scs.routines.bad_pixel_map(params)

Compute the bad pixels map.

Inputs

params: parameters

rtype:

bad pixel map

toolbox_scs.routines.inspect_dark(arr, mean_th=(None, None), std_th=(None, None))

Inspect dark run data and plot diagnostic.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) mean_th: tuple of threshold (low, high), default (None, None), to compute

a mask of good pixels for which the mean dark value lie inside this range

std_th: tuple of threshold (low, high), default (None, None), to compute a

mask of bad pixels for which the dark std value lie inside this range

returns:

fig

rtype:

matplotlib figure

toolbox_scs.routines.histogram_module(arr, mask=None)

Compute a histogram of the 9 bits raw pixel values over a module.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) mask: optional bad pixel mask

rtype:

histogram

toolbox_scs.routines.inspect_histogram(arr, arr_dark=None, mask=None, extra_lines=False)

Compute and plot a histogram of the 9 bits raw pixel values.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) arr: dask array of reshaped dssc dark data (trainId, pulseId, x, y) mask: optional bad pixel mask extra_lines: boolean, default False, plot extra lines at period values

returns:

• (h, hd) (histogram of arr, arr_dark)

• figure

toolbox_scs.routines.find_rois(data_mean, threshold, extended=False)

Find rois from 3 beams configuration.

Inputs

data_mean: dark corrected average image threshold: threshold value to find beams extended: boolean, True to define additional ASICS based rois

returns:

rois

rtype:

dictionnary of rois

toolbox_scs.routines.find_rois_from_params(params)

Find rois from 3 beams configuration.

Inputs

params: parameters

returns:

rois

rtype:

dictionnary of rois

toolbox_scs.routines.inspect_rois(data_mean, rois, threshold=None, allrois=False)

Find rois from 3 beams configuration from mean module image.

Inputs

data_mean: mean module image threshold: float, default None, threshold value used to detect beams

boundaries

allrois: boolean, default False, plot all rois defined in rois or only the

main ones ([‘n’, ‘0’, ‘p’])

rtype:

matplotlib figure

toolbox_scs.routines.compute_flat_field_correction(rois, params, plot=False)

toolbox_scs.routines.inspect_flat_field_domain(avg, rois, prod_th, ratio_th, vmin=None, vmax=None)

Extract beams roi from average image and compute the ratio.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary or ROIs prod_th, ratio_th: tuple of floats for low and high threshold on

product and ratio

vmin: imshow vmin level, default None will use 5 percentile value vmax: imshow vmax level, default None will use 99.8 percentile value

returns:

• fig (matplotlib figure plotted)

• domain (a tuple (n_m, p_m) of domain for the ‘n’ and ‘p’ order)

toolbox_scs.routines.inspect_plane_fitting(avg, rois, domain=None, vmin=None, vmax=None)

toolbox_scs.routines.plane_fitting_domain(avg, rois, prod_th, ratio_th)

Extract beams roi, compute their ratio and the domain.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary or rois containing the 3 beams [‘n’, ‘0’, ‘p’] with ‘0’

as the reference beam in the middle

prod_th: float tuple, low and hight threshold level to determine the plane

fitting domain on the product image of the orders

ratio_th: float tuple, low and high threshold level to determine the plane

fitting domain on the ratio image of the orders

returns:

• n (img ratio ‘n’/’0’)

• n_m (mask where the the product ‘n’’0’ is higher than 5 indicting that the*) – img ratio ‘n’/’0’ is defined

• p (img ratio ‘p’/’0’)

• p_m (mask where the the product ‘p’’0’ is higher than 5 indicting that the*) – img ratio ‘p’/’0’ is defined

toolbox_scs.routines.plane_fitting(params)

Fit the plane flat-field normalization.

Inputs

params: parameters

returns:

res – defines the plane as a*x + b*y + c*z + d = 0

rtype:

the minimization result. The fitted vector res.x = [a, b, c, d]

toolbox_scs.routines.ff_refine_crit(p, alpha, params, arr_dark, arr, tid, rois, mask, sat_level=511)

Criteria for the ff_refine_fit.

Inputs

p: ff plane params: parameters arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels sat_level: integer, default 511, at which level pixel begin to saturate

rtype:

sum of standard deviation on binned 0th order intensity

toolbox_scs.routines.ff_refine_fit(params, crit=ff_refine_crit)

Refine the flat-field fit by minimizing data spread.

Inputs

params: parameters

returns:

• res (scipy minimize result. res.x is the optimized parameters)

• fitrres (iteration index arrays of criteria results for) – [alpha=0, alpha, alpha=1]

toolbox_scs.routines.nl_domain(N, low, high)

Create the input domain where the non-linear correction defined.

Inputs

N: integer, number of control points or intervals low: input values below or equal to low will not be corrected high: input values higher or equal to high will not be corrected

rtype:

array of 2**9 integer values with N segments

toolbox_scs.routines.nl_lut(domain, dy)

Compute the non-linear correction.

Inputs

domain: input domain where dy is defined. For zero no correction is

defined. For non-zero value x, dy[x] is applied.

dy: a vector of deviation from linearity on control point homogeneously

dispersed over 9 bits.

returns:

F_INL – lookup table with 9 bits integer input

rtype:

default None, non linear correction function given as a

toolbox_scs.routines.nl_crit(p, domain, alpha, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)

Criteria for the non linear correction.

Inputs

p: vector of dy non linear correction domain: domain over which the non linear correction is defined alpha: float, coefficient scaling the cost of the correction function

in the criterion

arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

returns:

• (1.0 - alpha)*err1 + alpha*err2, where err1 is the 1e8 times the mean of

• error squared from a transmission of 1.0 and err2 is the sum of the square

• of the deviation from the ideal detector response.

toolbox_scs.routines.nl_crit_sk(p, domain, alpha, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)

Non linear correction criteria, combining ‘n’ and ‘p’ as reference.

Inputs

p: vector of dy non linear correction domain: domain over which the non linear correction is defined alpha: float, coefficient scaling the cost of the correction function

in the criterion

arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

returns:

• (1.0 - alpha)*err1 + alpha*err2, where err1 is the 1e8 times the mean of

• error squared from a transmission of 1.0 and err2 is the sum of the square

• of the deviation from the ideal detector response.

toolbox_scs.routines.nl_fit(params, domain, ff=None, crit=None)

Fit non linearities correction function.

Inputs

params: parameters domain: array of index ff: array, flat field correction crit: function, criteria function

returns:

• res (scipy minimize result. res.x is the optimized parameters)

• fitrres (iteration index arrays of criteria results for) – [alpha=0, alpha, alpha=1]

toolbox_scs.routines.inspect_nl_fit(res_fit)

Plot the progress of the fit.

Inputs

res_fit:

rtype:

matplotlib figure

toolbox_scs.routines.snr(sig, ref, methods=None, verbose=False)

Compute mean, std and SNR from transmitted and I0 signals.

Inputs

sig: 1D signal samples ref: 1D reference samples methods: None by default or list of strings to select which methods to use.

Possible values are ‘direct’, ‘weighted’, ‘diff’. In case of None, all methods will be calculated.

verbose: booleand, if True prints calculated values

returns:

• dictionnary of [methods][value] where value is ‘mu’ for mean and ‘s’ for

• standard deviation.

toolbox_scs.routines.inspect_Fnl(Fnl)

Plot the correction function Fnl.

Inputs

Fnl: non linear correction function lookup table

rtype:

matplotlib figure

toolbox_scs.routines.inspect_correction(params, gain=None)

Comparison plot of the different corrections.

Inputs

params: parameters gain: float, default None, DSSC gain in ph/bin

rtype:

matplotlib figure

toolbox_scs.routines.inspect_correction_sk(params, ff, gain=None)

Comparison plot of the different corrections, combining ‘n’ and ‘p’.

Inputs

params: parameters gain: float, default None, DSSC gain in ph/bin

rtype:

matplotlib figure

toolbox_scs.routines.load_dssc_module(proposalNB, runNB, moduleNB=15, subset=slice(None), drop_intra_darks=True, persist=False, data_size_Gb=None)

Load single module dssc data as dask array.

Inputs

proposalNB: proposal number runNB: run number moduleNB: default 15, module number subset: default slice(None), subset of trains to load drop_intra_darks: boolean, default True, remove intra darks from the data persist: default False, load all data persistently in memory data_size_Gb: float, if persist is True, can optionaly restrict

the amount of data loaded for dark data and run data in Gb

returns:

• arr (dask array of reshaped dssc data (trainId, pulseId, x, y))

• tid (array of train id number)

toolbox_scs.routines.average_module(arr, dark=None, ret='mean', mask=None, sat_roi=None, sat_level=300, F_INL=None)

Compute the average or std over a module.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) dark: default None, dark to be substracted ret: string, either ‘mean’ to compute the mean or ‘std’ to compute the

standard deviation

mask: default None, mask of bad pixels to ignore sat_roi: roi over which to check for pixel with values larger than

sat_level to drop the image from the average or std

sat_level: int, minimum pixel value for a pixel to be considered saturated F_INL: default None, non linear correction function given as a

lookup table with 9 bits integer input

rtype:

average or standard deviation image

toolbox_scs.routines.process_module(arr, tid, dark, rois, mask=None, sat_level=511, flat_field=None, F_INL=None, use_gpu=False)

Process one module and extract roi intensity.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) tid: array of train id number dark: pulse resolved dark image to remove rois: dictionnary of rois mask: default None, mask of ignored pixels sat_level: integer, default 511, at which level pixel begin to saturate flat_field: default None, flat-field correction F_INL: default None, non-linear correction function given as a

lookup table with 9 bits integer input

rtype:

dataset of extracted pulse and train resolved roi intensities.

toolbox_scs.routines.process(Fmodel, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)

Process dark and run data with corrections.

Inputs

Fmodel: correction lookup table arr_dark: dark data arr: data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask of good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

rtype:

roi extracted intensities

toolbox_scs.routines.inspect_saturation(data, gain, Nbins=200)

Plot roi integrated histogram of the data with saturation

Inputs

data: xarray of roi integrated DSSC data gain: nominal DSSC gain in ph/bin Nbins: number of bins for the histogram, by default 200

returns:

• f (handle to the matplotlib figure)

• h (xarray of the histogram data)

toolbox_scs.routines.reflectivity(data, Iokey='FastADC5peaks', Irkey='FastADC3peaks', delaykey='PP800_DelayLine', binWidth=0.05, positionToDelay=True, origin=None, invert=False, pumpedOnly=False, alternateTrains=False, pumpOnEven=True, Ioweights=False, plot=True, plotErrors=True, units='mm')

Computes the reflectivity R = 100*(Ir/Io[pumped] / Ir/Io[unpumped] - 1) as a function of delay. Delay can be a motor position in mm or an optical delay in ps, with possibility to convert from position to delay. The default scheme is alternating pulses pumped/unpumped/… in each train, also possible are alternating trains and pumped only. If fitting is enabled, attempts a double exponential (default) or step function fit.

Parameters:

• data (xarray Dataset) – Dataset containing the Io, Ir and delay data

• Iokey (str) – Name of the Io variable

• Irkey (str) – Name of the Ir variable

• delaykey (str) – Name of the delay variable (motor position in mm or optical delay in ps)

• binWidth (float) – width of bin in units of delay variable

• positionToDelay (bool) – If True, adds a time axis converted from position axis according to origin and invert parameters. Ignored if origin is None.

• origin (float) – Position of time overlap, shown as a vertical line. Used if positionToDelay is True to convert position to time axis.

• invert (bool) – Used if positionToDelay is True to convert position to time axis.

• pumpedOnly (bool) – Assumes that all trains and pulses are pumped. In this case, Delta R is defined as Ir/Io.

• alternateTrains (bool) – If True, assumes that trains alternate between pumped and unpumped data.

• pumpOnEven (bool) – Only used if alternateTrains=True. If True, even trains are pumped, if False, odd trains are pumped.

• Ioweights (bool) – If True, computes the ratio of the means instead of the mean of the ratios Irkey/Iokey. Useful when dealing with large intensity variations.

• plot (bool) – If True, plots the results.

• plotErrors (bool) – If True, plots the 95% confidence interval.

• Output –

• ------ – xarray Dataset containing the binned Delta R, standard deviation, standard error, counts and delays, and the fitting results if full is True.

toolbox_scs.routines.knife_edge(ds, axisKey='scannerX', signalKey='FastADC4peaks', axisRange=None, p0=None, full=False, plot=False, display=False)

Calculates the beam radius at 1/e^2 from a knife-edge scan by fitting with erfc function: f(x, x0, w0, a, b) = a*erfc(np.sqrt(2)*(x-x0)/w0) + b with w0 the beam radius at 1/e^2 and x0 the beam center.

Parameters:

• ds (xarray Dataset) – dataset containing the detector signal and the motor position.

• axisKey (str) – key of the axis against which the knife-edge is performed.

• signalKey (str) – key of the detector signal.

• axisRange (list of floats) – edges of the scanning axis between which to apply the fit.

• p0 (list of floats, numpy 1D array) – initial parameters used for the fit: x0, w0, a, b. If None, a beam radius of 100 micrometers is assumed.

• full (bool) – If False, returns the beam radius and standard error. If True, returns the popt, pcov list of parameters and covariance matrix from scipy.optimize.curve_fit.

• plot (bool) – If True, plots the data and the result of the fit. Default is False.

• display (bool) – If True, displays info on the fit. True when plot is True, default is False.

Returns:

error from the fit in mm. If full is True, returns parameters and covariance matrix from scipy.optimize.curve_fit function.

Return type:

If full is False, tuple with beam radius at 1/e^2 in mm and standard

toolbox_scs.routines.__all__