toolbox_scs.routines

Submodules

• toolbox_scs.routines.Reflectivity
• toolbox_scs.routines.XAS
• toolbox_scs.routines.boz
• toolbox_scs.routines.knife_edge

Package Contents

Classes

parameters

Parameters contains all input parameters for the BOZ corrections.

Functions

xas(nrun[, bins, Iokey, Itkey, nrjkey, Iooffset, ...])	Compute the XAS spectra from a xarray nrun.
xasxmcd(dataP, dataN)	Compute XAS and XMCD from data with both magnetic field direction
get_roi_pixel_pos(roi, params)	Compute fake or real pixel position of an roi from roi center.
bad_pixel_map(params)	Compute the bad pixels map.
inspect_dark(arr[, mean_th, std_th])	Inspect dark run data and plot diagnostic.
histogram_module(arr[, mask])	Compute a histogram of the 9 bits raw pixel values over a module.
inspect_histogram(arr[, arr_dark, mask, extra_lines])	Compute and plot a histogram of the 9 bits raw pixel values.
find_rois(data_mean, threshold[, extended])	Find rois from 3 beams configuration.
find_rois_from_params(params)	Find rois from 3 beams configuration.
inspect_rois(data_mean, rois[, threshold, allrois])	Find rois from 3 beams configuration from mean module image.
compute_flat_field_correction(rois, params[, plot])	Compute the plane-field correction on beam rois.
inspect_flat_field_domain(avg, rois, prod_th, ratio_th)	Extract beams roi from average image and compute the ratio.
inspect_plane_fitting(avg, rois[, domain, vmin, vmax])	Extract beams roi from average image and compute the ratio.
plane_fitting_domain(avg, rois, prod_th, ratio_th)	Extract beams roi, compute their ratio and the domain.
plane_fitting(params)	Fit the plane flat-field normalization.
ff_refine_crit(p, alpha, params, arr_dark, arr, tid, ...)	Criteria for the ff_refine_fit.
ff_refine_fit(params)	Refine the flat-field fit by minimizing data spread.
nl_domain(N, low, high)	Create the input domain where the non-linear correction defined.
nl_lut(domain, dy)	Compute the non-linear correction.
nl_crit(p, domain, alpha, arr_dark, arr, tid, rois, ...)	Criteria for the non linear correction.
nl_fit(params, domain)	Fit non linearities correction function.
inspect_nl_fit(res_fit)	Plot the progress of the fit.
snr(sig, ref[, methods, verbose])	Compute mean, std and SNR from transmitted signal sig and I0 signal ref.
inspect_Fnl(Fnl)	Plot the correction function Fnl.
inspect_correction(params[, gain])	Criteria for the non linear correction.
load_dssc_module(proposalNB, runNB[, moduleNB, ...])	Load single module dssc data as dask array.
average_module(arr[, dark, ret, mask, sat_roi, ...])	Compute the average or std over a module.
process_module(arr, tid, dark, rois[, mask, ...])	Process one module and extract roi intensity.
process(Fmodel, arr_dark, arr, tid, rois, mask, flat_field)	Process dark and run data with corrections.
inspect_saturation(data, gain[, Nbins])	Plot roi integrated histogram of the data with saturation
reflectivity(data[, Iokey, Irkey, delaykey, binWidth, ...])	Computes the reflectivity R = 100*(Ir/Io[pumped] / Ir/Io[unpumped] - 1)
knife_edge	Toolbox for SCS.

Attributes

__all__

toolbox_scs.routines.xas(nrun, bins=None, Iokey='SCS_SA3', Itkey='MCP3peaks', nrjkey='nrj', Iooffset=0, plot=False, fluorescence=False)

Compute the XAS spectra from a xarray nrun.

Inputs:

nrun: xarray of SCS data bins: an array of bin-edges or an integer number of

desired bins or a float for the desired bin width.

Iokey: string for the Io fields, typically ‘SCS_XGM’ Itkey: string for the It fields, typically ‘MCP3apd’ nrjkey: string for the nrj fields, typically ‘nrj’ Iooffset: offset to apply on Io plot: boolean, displays a XAS spectrum if True fluorescence: boolean, if True, absorption is the ratio,

if False, absorption is negative log

Outputs:

a dictionnary containing:

nrj: the bin centers muA: the absorption sigmaA: standard deviation on the absorption sterrA: standard error on the absorption muIo: the mean of the Io counts: the number of events in each bin

toolbox_scs.routines.xasxmcd(dataP, dataN)

Compute XAS and XMCD from data with both magnetic field direction Inputs:

dataP: structured array for positive field dataN: structured array for negative field

Outputs:

xas: structured array for the sum xmcd: structured array for the difference

class toolbox_scs.routines.parameters(proposal, darkrun, run, module, gain, drop_intra_darks=True)

Parameters contains all input parameters for the BOZ corrections.

This is used in beam splitting off-axis zone plate spectrocopy analysis as well as the during the determination of correction parameters themselves to ensure they can be reproduced.

Inputs

proposal: int, proposal number darkrun: int, run number for the dark run run: int, run number for the data run module: int, DSSC module number gain: float, number of ph per bin drop_intra_darks: drop every second DSSC frame

dask_load_persistently(dark_data_size_Gb=None, data_size_Gb=None)

Load dask data array in memory.

Inputs

dark_data_size_Gb: float, optional size of dark to load in memory, in Gb data_size_Gb: float, optional size of data to load in memory, in Gb

use_gpu()

set_mask(arr)

Set mask of bad pixels.

Inputs

arr: either a boolean array of a DSSC module image or a list of bad

pixel indices

get_mask()

Get the boolean array bad pixel of a DSSC module.

get_mask_idx()

Get the list of bad pixel indices.

flat_field_guess(guess=None)

Set the flat-field guess parameter for the fit and returns it.

Inputs

guess: a list of 8 floats, the 4 first to define the plane

ax+by+cz+d=0 for ‘n’ beam and the 4 last for the ‘p’ beam in case mirror symmetry is disbaled

set_flat_field(plane, prod_th=None, ratio_th=None)

Set the flat-field plane definition.

get_flat_field()

Get the flat-field plane definition.

set_Fnl(Fnl)

Set the non-linear correction function.

get_Fnl()

Get the non-linear correction function.

save(path='./')

Save the parameters as a JSON file.

Inputs

path: str, where to save the file, default to ‘./’

classmethod load(fname)

Load parameters from a JSON file.

Inputs

fname: string, name a the JSON file to load

__str__()

Return str(self).

toolbox_scs.routines.get_roi_pixel_pos(roi, params)

Compute fake or real pixel position of an roi from roi center.

Inputs:

roi: dictionnary params: parameters

Returns:

X, Y: 1-d array of pixel position.

toolbox_scs.routines.bad_pixel_map(params)

Compute the bad pixels map.

Inputs

params: parameters

rtype

bad pixel map

toolbox_scs.routines.inspect_dark(arr, mean_th=(None, None), std_th=(None, None))

Inspect dark run data and plot diagnostic.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) mean_th: tuple of threshold (low, high), default (None, None), to compute

a mask of good pixels for which the mean dark value lie inside this range

std_th: tuple of threshold (low, high), default (None, None), to compute a

mask of bad pixels for which the dark std value lie inside this range

returns

fig

rtype

matplotlib figure

toolbox_scs.routines.histogram_module(arr, mask=None)

Compute a histogram of the 9 bits raw pixel values over a module.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) mask: optional bad pixel mask

rtype

histogram

toolbox_scs.routines.inspect_histogram(arr, arr_dark=None, mask=None, extra_lines=False)

Compute and plot a histogram of the 9 bits raw pixel values.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) arr: dask array of reshaped dssc dark data (trainId, pulseId, x, y) mask: optional bad pixel mask extra_lines: boolean, default False, plot extra lines at period values

returns

• (h, hd) (histogram of arr, arr_dark)

• figure

toolbox_scs.routines.find_rois(data_mean, threshold, extended=False)

Find rois from 3 beams configuration.

Inputs

data_mean: dark corrected average image threshold: threshold value to find beams extended: boolean, True to define additional ASICS based rois

returns

rois

rtype

dictionnary of rois

toolbox_scs.routines.find_rois_from_params(params)

Find rois from 3 beams configuration.

Inputs

params: parameters

returns

rois

rtype

dictionnary of rois

toolbox_scs.routines.inspect_rois(data_mean, rois, threshold=None, allrois=False)

Find rois from 3 beams configuration from mean module image.

Inputs

data_mean: mean module image threshold: float, default None, threshold value used to detect beams

boundaries

allrois: boolean, default False, plot all rois defined in rois or only the

main ones ([‘n’, ‘0’, ‘p’])

rtype

matplotlib figure

toolbox_scs.routines.compute_flat_field_correction(rois, params, plot=False)

Compute the plane-field correction on beam rois.

Inputs

rois: dictionnary of beam rois[‘n’, ‘0’, ‘p’] params: parameters plot: boolean, True by default, diagnostic plot

returns

• numpy 2D array of the flat-field correction evaluated over one DSSC ladder

• (2 sensors)

toolbox_scs.routines.inspect_flat_field_domain(avg, rois, prod_th, ratio_th, vmin=None, vmax=None)

Extract beams roi from average image and compute the ratio.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary or ROIs prod_th, ratio_th: tuple of floats for low and high threshold on

product and ratio

vmin: imshow vmin level, default None will use 5 percentile value vmax: imshow vmax level, default None will use 99.8 percentile value

returns

• fig (matplotlib figure plotted)

• domain (a tuple (n_m, p_m) of domain for the ‘n’ and ‘p’ order)

toolbox_scs.routines.inspect_plane_fitting(avg, rois, domain=None, vmin=None, vmax=None)

Extract beams roi from average image and compute the ratio.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary of rois domain: list of domain mask for the -1st and +1st order vmin: imshow vmin level, default None will use 5 percentile value vmax: imshow vmax level, default None will use 99.8 percentile value

returns

fig

rtype

matplotlib figure plotted

toolbox_scs.routines.plane_fitting_domain(avg, rois, prod_th, ratio_th)

Extract beams roi, compute their ratio and the domain.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary or rois containing the 3 beams [‘n’, ‘0’, ‘p’] with ‘0’

as the reference beam in the middle

prod_th: float tuple, low and hight threshold level to determine the plane

fitting domain on the product image of the orders

ratio_th: float tuple, low and high threshold level to determine the plane

fitting domain on the ratio image of the orders

returns

• n (img ratio ‘n’/‘0’)

• n_m (mask where the the product ‘n’‘0’ is higher than 5 indicting that the*) – img ratio ‘n’/‘0’ is defined

• p (img ratio ‘p’/‘0’)

• p_m (mask where the the product ‘p’‘0’ is higher than 5 indicting that the*) – img ratio ‘p’/‘0’ is defined

toolbox_scs.routines.plane_fitting(params)

Fit the plane flat-field normalization.

Inputs

params: parameters

returns

res – defines the plane as a*x + b*y + c*z + d = 0

rtype

the minimization result. The fitted vector res.x = [a, b, c, d]

toolbox_scs.routines.ff_refine_crit(p, alpha, params, arr_dark, arr, tid, rois, mask, sat_level=511)

Criteria for the ff_refine_fit.

Inputs

p: ff plane params: parameters arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels sat_level: integer, default 511, at which level pixel begin to saturate

rtype

sum of standard deviation on binned 0th order intensity

toolbox_scs.routines.ff_refine_fit(params)

Refine the flat-field fit by minimizing data spread.

Inputs

params: parameters

returns

• res (scipy minimize result. res.x is the optimized parameters)

• fitrres (iteration index arrays of criteria results for) – [alpha=0, alpha, alpha=1]

toolbox_scs.routines.nl_domain(N, low, high)

Create the input domain where the non-linear correction defined.

Inputs

N: integer, number of control points or intervals low: input values below or equal to low will not be corrected high: input values higher or equal to high will not be corrected

rtype

array of 2**9 integer values with N segments

toolbox_scs.routines.nl_lut(domain, dy)

Compute the non-linear correction.

Inputs

domain: input domain where dy is defined. For zero no correction is

defined. For non-zero value x, dy[x] is applied.

dy: a vector of deviation from linearity on control point homogeneously

dispersed over 9 bits.

returns

F_INL – lookup table with 9 bits integer input

rtype

default None, non linear correction function given as a

toolbox_scs.routines.nl_crit(p, domain, alpha, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)

Criteria for the non linear correction.

Inputs

p: vector of dy non linear correction domain: domain over which the non linear correction is defined alpha: float, coefficient scaling the cost of the correction function

in the criterion

arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

returns

• (1.0 - alpha)*err1 + alpha*err2, where err1 is the 1e8 times the mean of

• error squared from a transmission of 1.0 and err2 is the sum of the square

• of the deviation from the ideal detector response.

toolbox_scs.routines.nl_fit(params, domain)

Fit non linearities correction function.

Inputs

params: parameters domain: array of index

returns

• res (scipy minimize result. res.x is the optimized parameters)

• fitrres (iteration index arrays of criteria results for) – [alpha=0, alpha, alpha=1]

toolbox_scs.routines.inspect_nl_fit(res_fit)

Plot the progress of the fit.

Inputs

res_fit:

rtype

matplotlib figure

toolbox_scs.routines.snr(sig, ref, methods=None, verbose=False)

Compute mean, std and SNR from transmitted signal sig and I0 signal ref.

Inputs

sig: 1D signal samples ref: 1D reference samples methods: None by default or list of strings to select which methods to use.

Possible values are ‘direct’, ‘weighted’, ‘diff’. In case of None, all methods will be calculated.

verbose: booleand, if True prints calculated values

returns

• dictionnary of [methods][value] where value is ‘mu’ for mean and ‘s’ for

• standard deviation.

toolbox_scs.routines.inspect_Fnl(Fnl)

Plot the correction function Fnl.

Inputs

Fnl: non linear correction function lookup table

rtype

matplotlib figure

toolbox_scs.routines.inspect_correction(params, gain=None)

Criteria for the non linear correction.

Inputs

params: parameters gain: float, default None, DSSC gain in ph/bin

rtype

matplotlib figure

toolbox_scs.routines.load_dssc_module(proposalNB, runNB, moduleNB=15, subset=slice(None), drop_intra_darks=True, persist=False, data_size_Gb=None)

Load single module dssc data as dask array.

Inputs

proposalNB: proposal number runNB: run number moduleNB: default 15, module number subset: default slice(None), subset of trains to load drop_intra_darks: boolean, default True, remove intra darks from the data persist: default False, load all data persistently in memory data_size_Gb: float, if persist is True, can optionaly restrict

the amount of data loaded for dark data and run data in Gb

returns

• arr (dask array of reshaped dssc data (trainId, pulseId, x, y))

• tid (array of train id number)

toolbox_scs.routines.average_module(arr, dark=None, ret='mean', mask=None, sat_roi=None, sat_level=300, F_INL=None)

Compute the average or std over a module.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) dark: default None, dark to be substracted ret: string, either ‘mean’ to compute the mean or ‘std’ to compute the

standard deviation

mask: default None, mask of bad pixels to ignore sat_roi: roi over which to check for pixel with values larger than

sat_level to drop the image from the average or std

sat_level: int, minimum pixel value for a pixel to be considered saturated F_INL: default None, non linear correction function given as a

lookup table with 9 bits integer input

rtype

average or standard deviation image

toolbox_scs.routines.process_module(arr, tid, dark, rois, mask=None, sat_level=511, flat_field=None, F_INL=None, use_gpu=False)

Process one module and extract roi intensity.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) tid: array of train id number dark: pulse resolved dark image to remove rois: dictionnary of rois mask: default None, mask of ignored pixels sat_level: integer, default 511, at which level pixel begin to saturate flat_field: default None, flat-field correction F_INL: default None, non-linear correction function given as a

lookup table with 9 bits integer input

rtype

dataset of extracted pulse and train resolved roi intensities.

toolbox_scs.routines.process(Fmodel, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)

Process dark and run data with corrections.

Inputs

Fmodel: correction lookup table arr_dark: dark data arr: data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask of good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

rtype

roi extracted intensities

toolbox_scs.routines.inspect_saturation(data, gain, Nbins=200)

Plot roi integrated histogram of the data with saturation

Inputs

data: xarray of roi integrated DSSC data gain: nominal DSSC gain in ph/bin Nbins: number of bins for the histogram, by default 200

returns

• f (handle to the matplotlib figure)

• h (xarray of the histogram data)

toolbox_scs.routines.reflectivity(data, Iokey='FastADC5peaks', Irkey='FastADC3peaks', delaykey='PP800_DelayLine', binWidth=0.05, positionToDelay=True, origin=None, invert=False, pumpedOnly=False, alternateTrains=False, pumpOnEven=True, Ioweights=False, plot=True, plotErrors=True, units='mm')

Computes the reflectivity R = 100*(Ir/Io[pumped] / Ir/Io[unpumped] - 1) as a function of delay. Delay can be a motor position in mm or an optical delay in ps, with possibility to convert from position to delay. The default scheme is alternating pulses pumped/unpumped/… in each train, also possible are alternating trains and pumped only. If fitting is enabled, attempts a double exponential (default) or step function fit.

Parameters

• data (xarray Dataset) – Dataset containing the Io, Ir and delay data

• Iokey (str) – Name of the Io variable

• Irkey (str) – Name of the Ir variable

• delaykey (str) – Name of the delay variable (motor position in mm or optical delay in ps)

• binWidth (float) – width of bin in units of delay variable

• positionToDelay (bool) – If True, adds a time axis converted from position axis according to origin and invert parameters. Ignored if origin is None.

• origin (float) – Position of time overlap, shown as a vertical line. Used if positionToDelay is True to convert position to time axis.

• invert (bool) – Used if positionToDelay is True to convert position to time axis.

• pumpedOnly (bool) – Assumes that all trains and pulses are pumped. In this case, Delta R is defined as Ir/Io.

• alternateTrains (bool) – If True, assumes that trains alternate between pumped and unpumped data.

• pumpOnEven (bool) – Only used if alternateTrains=True. If True, even trains are pumped, if False, odd trains are pumped.

• Ioweights (bool) – If True, computes the ratio of the means instead of the mean of the ratios Irkey/Iokey. Useful when dealing with large intensity variations.

• plot (bool) – If True, plots the results.

• plotErrors (bool) – If True, plots the 95% confidence interval.

• Output –

• ------ – xarray Dataset containing the binned Delta R, standard deviation, standard error, counts and delays, and the fitting results if full is True.

toolbox_scs.routines.knife_edge(ds, axisKey='scannerX', signalKey='FastADC4peaks', axisRange=None, p0=None, full=False, plot=False, display=False)

Calculates the beam radius at 1/e^2 from a knife-edge scan by fitting with erfc function: f(x, x0, w0, a, b) = a*erfc(np.sqrt(2)*(x-x0)/w0) + b with w0 the beam radius at 1/e^2 and x0 the beam center.

Parameters

• ds (xarray Dataset) – dataset containing the detector signal and the motor position.

• axisKey (str) – key of the axis against which the knife-edge is performed.

• signalKey (str) – key of the detector signal.

• axisRange (list of floats) – edges of the scanning axis between which to apply the fit.

• p0 (list of floats, numpy 1D array) – initial parameters used for the fit: x0, w0, a, b. If None, a beam radius of 100 micrometers is assumed.

• full (bool) – If False, returns the beam radius and standard error. If True, returns the popt, pcov list of parameters and covariance matrix from scipy.optimize.curve_fit.

• plot (bool) – If True, plots the data and the result of the fit. Default is False.

• display (bool) – If True, displays info on the fit. True when plot is True, default is False.

Returns

error from the fit in mm. If full is True, returns parameters and covariance matrix from scipy.optimize.curve_fit function.

Return type

If full is False, tuple with beam radius at 1/e^2 in mm and standard

toolbox_scs.routines.__all__