toolbox_scs

Package Contents

Classes

AzimuthalIntegrator

AzimuthalIntegratorDSSC

DSSCBinner

DSSCFormatter

hRIXS

The hRIXS analysis, especially curvature correction

Viking

The Viking analysis (spectrometer used in combination with Andor Newton

parameters

Parameters contains all input parameters for the BOZ corrections.

Functions

get_bam(run[, mnemonics, merge_with, bunchPattern, ...])

Load beam arrival monitor (BAM) data and align their pulse ID

get_bam_params(run[, mnemo_or_source])

Extract the run values of bamStatus[1-3] and bamError.

check_peak_params(run, mnemonic[, raw_trace, ntrains, ...])

Checks and plots the peak parameters (pulse window and baseline window

get_digitizer_peaks(run[, mnemonics, merge_with, ...])

Automatically computes digitizer peaks. Sources can be loaded on the

get_laser_peaks(run[, mnemonics, merge_with, ...])

Extracts laser photodiode signal (peak intensity) from Fast ADC

get_peaks(run[, data, source, key, digitizer, useRaw, ...])

Extract peaks from one source (channel) of a digitizer.

get_tim_peaks(run[, mnemonics, merge_with, ...])

Automatically computes TIM peaks. Sources can be loaded on the

digitizer_signal_description(run[, digitizer])

Check for the existence of signal description and return all corresponding

get_dig_avg_trace(run, mnemonic[, ntrains])

Compute the average over ntrains evenly spaced accross all trains

get_data_formatted([filenames, data_list])

Combines the given data into one dataset. For any of extra_data's data

load_xarray(fname[, group, form])

Load stored xarray Dataset.

save_attributes_h5(fname[, data])

Adding attributes to a hdf5 file. This function is intended to be used to

save_xarray(fname, data[, group, mode])

Store xarray Dataset in the specified location

create_dssc_bins(name, coordinates, bins)

Creates a single entry for the dssc binner dictionary. The produced xarray

get_xgm_formatted(run_obj, xgm_name, dssc_frame_coords)

Load the xgm data and define coordinates along the pulse dimension.

load_dssc_info(proposal, run_nr)

Loads the first data file for DSSC module 0 (this is hardcoded)

load_mask(fname, dssc_mask)

Load a DSSC mask file.

quickmask_DSSC_ASIC(poslist)

Returns a mask for the given DSSC geometry with ASICs given in poslist

process_dssc_data(proposal, run_nr, module, chunksize, ...)

Collects and reduces DSSC data for a single module.

get_pes_params(run)

Extract PES parameters for a given extra_data DataCollection.

get_pes_tof(run[, mnemonics, merge_with, start, ...])

Extracts time-of-flight spectra from raw digitizer traces. The

calibrate_xgm(run, data[, xgm, plot])

Calculates the calibration factor F between the photon flux (slow signal)

get_xgm(run[, mnemonics, merge_with, indices])

Load and/or computes XGM data. Sources can be loaded on the

concatenateRuns(runs)

Sorts and concatenate a list of runs with identical data variables

find_run_path(proposalNB, runNB[, data])

Return the run path given the specified proposal and run numbers.

get_array([run, mnemonic, stepsize, subset, ...])

Loads one data array for the specified mnemonic and rounds its values to

load

Toolbox for SCS.

open_run(proposalNB, runNB[, subset])

Get extra_data.DataCollection in a given proposal.

run_by_path(path)

Return specified run

load_run_values(prop_or_run[, runNB, which])

Load the run value for each mnemonic whose source is a CONTORL

check_data_rate(run[, fields])

Calculates the fraction of train ids that contain data in a run.

extractBunchPattern([bp_table, key, runDir])

generate the bunch pattern and number of pulses of a source directly from the

get_sase_pId(run[, loc, run_mnemonics, bpt, merge_with])

Returns the pulse Ids of the specified loc during a run.

npulses_has_changed(run[, loc, run_mnemonics])

Checks if the number of pulses has changed during the run for

pulsePatternInfo(data[, plot])

display general information on the pulse patterns operated by SASE1 and SASE3.

repRate([data, runNB, proposalNB, key])

Calculates the pulse repetition rate (in kHz) in sase

is_sase_3(bpt)

Check for prescence of a SASE3 pulse.

is_sase_1(bpt)

Check for prescence of a SASE1 pulse.

is_ppl(bpt)

Check for prescence of pp-laser pulse.

is_pulse_at(bpt, loc)

Check for prescence of a pulse at the location provided.

degToRelPower(x[, theta0])

converts a half-wave plate position in degrees into relative power

positionToDelay(pos[, origin, invert, reflections])

converts a motor position in mm into optical delay in picosecond

delayToPosition(delay[, origin, invert, reflections])

converts an optical delay in picosecond into a motor position in mm

fluenceCalibration(hwp, power_mW, npulses, w0x[, w0y, ...])

Given a measurement of relative powers or half wave plate angles

align_ol_to_fel_pId(ds[, ol_dim, fel_dim, offset, ...])

Aligns the optical laser (OL) pulse Ids to the FEL pulse Ids.

get_undulator_config(run[, park_pos, plot])

Extract the undulator cells configuration from a given run.

mnemonics_for_run(prop_or_run[, runNB])

Returns the availble ToolBox mnemonics for a give extra_data

xas(nrun[, bins, Iokey, Itkey, nrjkey, Iooffset, ...])

Compute the XAS spectra from a xarray nrun.

xasxmcd(dataP, dataN)

Compute XAS and XMCD from data with both magnetic field direction

get_roi_pixel_pos(roi, params)

Compute fake or real pixel position of an roi from roi center.

bad_pixel_map(params)

Compute the bad pixels map.

inspect_dark(arr[, mean_th, std_th])

Inspect dark run data and plot diagnostic.

histogram_module(arr[, mask])

Compute a histogram of the 9 bits raw pixel values over a module.

inspect_histogram(arr[, arr_dark, mask, extra_lines])

Compute and plot a histogram of the 9 bits raw pixel values.

find_rois(data_mean, threshold[, extended])

Find rois from 3 beams configuration.

find_rois_from_params(params)

Find rois from 3 beams configuration.

inspect_rois(data_mean, rois[, threshold, allrois])

Find rois from 3 beams configuration from mean module image.

compute_flat_field_correction(rois, params[, plot])

Compute the plane-field correction on beam rois.

inspect_flat_field_domain(avg, rois, prod_th, ratio_th)

Extract beams roi from average image and compute the ratio.

inspect_plane_fitting(avg, rois[, domain, vmin, vmax])

Extract beams roi from average image and compute the ratio.

plane_fitting_domain(avg, rois, prod_th, ratio_th)

Extract beams roi, compute their ratio and the domain.

plane_fitting(params)

Fit the plane flat-field normalization.

ff_refine_crit(p, alpha, params, arr_dark, arr, tid, ...)

Criteria for the ff_refine_fit.

ff_refine_fit(params)

Refine the flat-field fit by minimizing data spread.

nl_domain(N, low, high)

Create the input domain where the non-linear correction defined.

nl_lut(domain, dy)

Compute the non-linear correction.

nl_crit(p, domain, alpha, arr_dark, arr, tid, rois, ...)

Criteria for the non linear correction.

nl_fit(params, domain)

Fit non linearities correction function.

inspect_nl_fit(res_fit)

Plot the progress of the fit.

snr(sig, ref[, methods, verbose])

Compute mean, std and SNR from transmitted signal sig and I0 signal ref.

inspect_Fnl(Fnl)

Plot the correction function Fnl.

inspect_correction(params[, gain])

Criteria for the non linear correction.

load_dssc_module(proposalNB, runNB[, moduleNB, ...])

Load single module dssc data as dask array.

average_module(arr[, dark, ret, mask, sat_roi, ...])

Compute the average or std over a module.

process_module(arr, tid, dark, rois[, mask, ...])

Process one module and extract roi intensity.

process(Fmodel, arr_dark, arr, tid, rois, mask, flat_field)

Process dark and run data with corrections.

inspect_saturation(data, gain[, Nbins])

Plot roi integrated histogram of the data with saturation

reflectivity(data[, Iokey, Irkey, delaykey, binWidth, ...])

Computes the reflectivity R = 100*(Ir/Io[pumped] / Ir/Io[unpumped] - 1)

knife_edge(ds[, axisKey, signalKey, axisRange, p0, ...])

Calculates the beam radius at 1/e^2 from a knife-edge scan by

Attributes

mnemonics

__all__

__all__

__all__

__all__

toolbox_scs.mnemonics[source]
toolbox_scs.__all__[source]
class toolbox_scs.AzimuthalIntegrator(imageshape, center, polar_range, aspect=204 / 236, **kwargs)[source]

Bases: object

_calc_dist_array(shape, center, aspect)[source]

Calculate pixel coordinates for the given shape.

_calc_indices(**kwargs)[source]

Calculates the list of indices for the flattened image array.

_calc_polar_mask(polar_range)[source]
calc_q(distance, wavelength)[source]

Calculate momentum transfer coordinate.

Parameters
  • distance (float) – Sample - detector distance in meter

  • wavelength (float) – wavelength of scattered light in meter

Returns

deltaq – Momentum transfer coordinate in 1/m

Return type

np.ndarray

__call__(image)[source]
class toolbox_scs.AzimuthalIntegratorDSSC(geom, polar_range, dxdy=(0, 0), **kwargs)[source]

Bases: AzimuthalIntegrator

_calc_dist_array(geom, dxdy)[source]

Calculate pixel coordinates for the given shape.

toolbox_scs.get_bam(run, mnemonics=None, merge_with=None, bunchPattern='sase3', pulseIds=None)[source]

Load beam arrival monitor (BAM) data and align their pulse ID according to the bunch pattern. Sources can be loaded on the fly via the mnemonics argument, or processed from an existing data set (merge_with).

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the bam data.

  • mnemonics (str or list of str) – mnemonics for BAM, e.g. “BAM1932M” or [“BAM414”, “BAM1932M”]. If None, defaults to “BAM1932M” in case no merge_with dataset is provided.

  • merge_with (xarray Dataset) – If provided, the resulting Dataset will be merged with this one. The BAM variables of merge_with (if any) will also be selected, aligned and merged.

  • bunchPattern (str) – ‘sase1’ or ‘sase3’ or ‘scs_ppl’, bunch pattern used to extract peaks. The pulse ID dimension will be named ‘sa1_pId’, ‘sa3_pId’ or ‘ol_pId’, respectively.

  • pulseIds (list, 1D array) – Pulse Ids. If None, they are automatically loaded.

Returns

merged with Dataset merge_with if provided.

Return type

xarray Dataset with pulse-resolved BAM variables aligned,

Example

>>> import toolbox_scs as tb
>>> run = tb.open_run(2711, 303)
>>> bam = tb.get_bam(run, 'BAM1932S')
toolbox_scs.get_bam_params(run, mnemo_or_source='BAM1932S')[source]

Extract the run values of bamStatus[1-3] and bamError.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the bam data.

  • mnemo_or_source (str) – mnemonic of the BAM, e.g. ‘BAM414’, or source name, e.g. ‘SCS_ILH_LAS/DOOCS/BAM_414_B2.

Returns

params – dictionnary containing the extracted parameters.

Return type

dict

Note

The extracted parameters are run values, they do not reflect any possible change during the run.

toolbox_scs.check_peak_params(run, mnemonic, raw_trace=None, ntrains=200, params=None, plot=True, show_all=False, bunchPattern='sase3')[source]

Checks and plots the peak parameters (pulse window and baseline window of a raw digitizer trace) used to compute the peak integration. These parameters are either set by the digitizer peak-integration settings, or are determined by a peak finding algorithm (used in get_tim_peaks or get_fast_adc_peaks) when the inputs are raw traces. The parameters can also be provided manually for visual inspection. The plot either shows the first and last pulse of the trace or the entire trace.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data.

  • mnemonic (str) – ToolBox mnemonic of the digitizer data, e.g. ‘MCP2apd’.

  • raw_trace (optional, 1D numpy array or xarray DataArray) – Raw trace to display. If None, the average raw trace over ntrains of the corresponding channel is loaded (this can be time-consuming).

  • ntrains (optional, int) – Only used if raw_trace is None. Number of trains used to calculate the average raw trace of the corresponding channel.

  • plot (bool) – If True, displays the raw trace and peak integration regions.

  • show_all (bool) – If True, displays the entire raw trace and all peak integration regions (this can be time-consuming). If False, shows the first and last pulse according to the bunchPattern.

  • bunchPattern (optional, str) – Only used if plot is True. Checks the bunch pattern against the digitizer peak parameters and shows potential mismatch.

Return type

dictionnary of peak integration parameters

toolbox_scs.get_digitizer_peaks(run, mnemonics=None, merge_with=None, bunchPattern='None', integParams=None, digitizer=None, keepAllSase=False)[source]

Automatically computes digitizer peaks. Sources can be loaded on the fly via the mnemonics argument, or processed from an existing data set (merge_with). The bunch pattern table is used to assign the pulse id coordinates.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data.

  • mnemonics (str or list of str) – mnemonics for FastADC or TIM, e.g. “FastADC2raw” or [“MCP2raw”, “MCP3apd”]. If None and no merge_with dataset is provided, defaults to “MCP2apd” if digitizer is ADQ412 or “FastADC5raw” if digitizer is FastADC.

  • merge_with (xarray Dataset) – If provided, the resulting Dataset will be merged with this one. The FastADC variables of merge_with (if any) will also be computed and merged.

  • bunchPattern (str) – ‘sase1’ or ‘sase3’ or ‘scs_ppl’, ‘None’: bunch pattern used to extract peaks.

  • integParams (dict) – dictionnary for raw trace integration, e.g. {‘pulseStart’:100, ‘pulsestop’:200, ‘baseStart’:50, ‘baseStop’:99, ‘period’:24, ‘npulses’:500}. If None, integration parameters are computed automatically.

  • keepAllSase (bool) – Only relevant in case of sase-dedicated trains. If True, all trains are kept, else only those of the bunchPattern are kept.

Returns

  • xarray Dataset with all Fast ADC variables substituted by

  • the peak caclulated values (e.g. “FastADC2raw” becomes

  • ”FastADC2peaks”).

toolbox_scs.get_laser_peaks(run, mnemonics=None, merge_with=None, bunchPattern='scs_ppl', integParams=None)[source]

Extracts laser photodiode signal (peak intensity) from Fast ADC digitizer. Sources can be loaded on the fly via the mnemonics argument, and/or processed from an existing data set (merge_with). The PP laser bunch pattern is used to assign the pulse idcoordinates.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data.

  • mnemonics (str or list of str) – mnemonics for FastADC corresponding to laser signal, e.g. “FastADC2peaks” or [“FastADC2raw”, “FastADC3peaks”]. If None, defaults to “MCP2apd” in case no merge_with dataset is provided.

  • merge_with (xarray Dataset) – If provided, the resulting Dataset will be merged with this one. The FastADC variables of merge_with (if any) will also be computed and merged.

  • bunchPattern (str) – ‘sase1’ or ‘sase3’ or ‘scs_ppl’, bunch pattern used to extract peaks.

  • integParams (dict) – dictionnary for raw trace integration, e.g. {‘pulseStart’:100, ‘pulsestop’:200, ‘baseStart’:50, ‘baseStop’:99, ‘period’:24, ‘npulses’:500}. If None, integration parameters are computed automatically.

Returns

  • xarray Dataset with all Fast ADC variables substituted by

  • the peak caclulated values (e.g. “FastADC2raw” becomes

  • ”FastADC2peaks”).

toolbox_scs.get_peaks(run, data=None, source=None, key=None, digitizer='ADQ412', useRaw=True, autoFind=True, integParams=None, bunchPattern='sase3', bpt=None, extra_dim=None, indices=None)[source]

Extract peaks from one source (channel) of a digitizer.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data

  • data (xarray DataArray or str) – array containing the raw traces or peak-integrated values from the digitizer. If str, must be one of the ToolBox mnemonics. If None, the data is loaded via the source and key arguments.

  • source (str) – Name of digitizer source, e.g. ‘SCS_UTC1_ADQ/ADC/1:network’. Only required if data is a DataArray or None.

  • key (str) – Key for digitizer data, e.g. ‘digitizers.channel_1_A.raw.samples’. Only required if data is a DataArray or is None.

  • digitizer (string) – name of digitizer, e.g. ‘FastADC’ or ‘ADQ412’. Used to determine the sampling rate.

  • useRaw (bool) – If True, extract peaks from raw traces. If False, uses the APD (or peaks) data from the digitizer.

  • autoFind (bool) – If True, finds integration parameters by inspecting the average raw trace. Only valid if useRaw is True.

  • integParams (dict) – dictionnary containing the integration parameters for raw trace integration: ‘pulseStart’, ‘pulseStop’, ‘baseStart’, ‘baseStop’, ‘period’, ‘npulses’. Not used if autoFind is True. All keys are required when bunch pattern is missing.

  • bunchPattern (string or dict) – match the peaks to the bunch pattern: ‘sase1’, ‘sase3’, ‘scs_ppl’. This will dictate the name of the pulse ID coordinates: ‘sa1_pId’, ‘sa3_pId’ or ‘scs_ppl’. Alternatively, a dict with source, key and pattern can be provided, e.g. {‘source’:’SCS_RR_UTC/TSYS/TIMESERVER’, ‘key’:’bunchPatternTable.value’, ‘pattern’:’sase3’}

  • bpt (xarray DataArray) – bunch pattern table

  • extra_dim (str) – Name given to the dimension along the peaks. If None, the name is given according to the bunchPattern.

  • indices (array, slice) – indices from the peak-integrated data to retrieve. Only required when bunch pattern is missing and useRaw is False.

Return type

xarray.DataArray containing digitizer peaks with pulse coordinates

toolbox_scs.get_tim_peaks(run, mnemonics=None, merge_with=None, bunchPattern='sase3', integParams=None, keepAllSase=False)[source]

Automatically computes TIM peaks. Sources can be loaded on the fly via the mnemonics argument, or processed from an existing data set (merge_with). The bunch pattern table is used to assign the pulse id coordinates.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data.

  • mnemonics (str or list of str) – mnemonics for TIM, e.g. “MCP2apd” or [“MCP2apd”, “MCP3raw”]. If None, defaults to “MCP2apd” in case no merge_with dataset is provided.

  • merge_with (xarray Dataset) – If provided, the resulting Dataset will be merged with this one. The TIM variables of merge_with (if any) will also be computed and merged.

  • bunchPattern (str) – ‘sase1’ or ‘sase3’ or ‘scs_ppl’, bunch pattern used to extract peaks. The pulse ID dimension will be named ‘sa1_pId’, ‘sa3_pId’ or ‘ol_pId’, respectively.

  • integParams (dict) – dictionnary for raw trace integration, e.g. {‘pulseStart’:100, ‘pulsestop’:200, ‘baseStart’:50, ‘baseStop’:99, ‘period’:24, ‘npulses’:500}. If None, integration parameters are computed automatically.

  • keepAllSase (bool) – Only relevant in case of sase-dedicated trains. If True, all trains are kept, else only those of the bunchPattern are kept.

Returns

  • xarray Dataset with all TIM variables substituted by

  • the peak caclulated values (e.g. “MCP2raw” becomes

  • ”MCP2peaks”), merged with Dataset *merge_with if provided.*

toolbox_scs.digitizer_signal_description(run, digitizer=None)[source]

Check for the existence of signal description and return all corresponding channels in a dictionnary.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data.

  • digitizer (str or list of str (default=None)) – Name of the digitizer: one in [‘FastADC’, FastADC2’, ‘ADQ412’, ‘ADQ412_2] If None, all digitizers are used

Returns

signal_description – the digitizer channels.

Return type

dictionnary containing the signal description of

Example

import toolbox_scs as tb run = tb.open_run(3481, 100) signals = tb.digitizer_signal_description(run) signals_fadc2 = tb.digitizer_signal_description(run, ‘FastADC2’)

toolbox_scs.get_dig_avg_trace(run, mnemonic, ntrains=None)[source]

Compute the average over ntrains evenly spaced accross all trains of a digitizer trace.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data.

  • mnemonic (str) – ToolBox mnemonic of the digitizer data, e.g. ‘MCP2apd’.

  • ntrains (int) – Number of trains used to calculate the average raw trace. If None, all trains are used.

Returns

trace – The average digitizer trace

Return type

DataArray

class toolbox_scs.DSSCBinner(proposal_nr, run_nr, binners={}, xgm_name='SCS_SA3', tim_names=['MCP1apd', 'MCP2apd', 'MCP3apd'], dssc_coords_stride=2)[source]
__del__()[source]
add_binner(name, binner)[source]

Add additional binner to internal dictionary

Parameters
  • name (str) – name of binner to be created

  • binner (xarray.DataArray) – An array that represents a map how the respective coordinate should be binned.

Raises

ToolBoxValueError – Exception: Raises exception in case the name does not correspond to a valid binner name. To be generalized.

load_xgm()[source]

load xgm data and construct coordinate array according to corresponding dssc frame number.

load_tim()[source]

load tim data and construct coordinate array according to corresponding dssc frame number.

create_pulsemask(use_data='xgm', threshold=(0, np.inf))[source]

creates a mask for dssc frames according to measured xgm intensity. Once such a mask has been constructed, it will be used in the data reduction process to drop out-of-bounds pulses.

get_info()[source]

Returns the expected shape of the binned dataset, in case binners have been defined.

_bin_metadata(data)[source]
get_xgm_binned()[source]

Bin the xgm data according to the binners of the dssc data. The result can eventually be merged into the final dataset by the DSSCFormatter.

Returns

xgm_data – xarray dataset containing the binned xgm data

Return type

xarray.DataSet

get_tim_binned()[source]

Bin the tim data according to the binners of the dssc data. The result can eventually be merged into the final dataset by the DSSCFormatter.

Returns

tim_data – xarray dataset containing the binned tim data

Return type

xarray.DataSet

process_data(modules=[], filepath='./', chunksize=512, backend='loky', n_jobs=None, dark_image=None, xgm_normalization=False, normevery=1)[source]

Load and bin dssc data according to self.bins. No data is returned by this method. The condensed data is written to file by the worker processes directly.

Parameters
  • modules (list of ints) – a list containing the module numbers that should be processed. If empty, all modules are processed.

  • filepath (str) – the path where the files containing the reduced data should be stored.

  • chunksize (int) – The number of trains that should be read in one iterative step.

  • backend (str) – joblib multiprocessing backend to be used. At the moment it can be any of joblibs standard backends: ‘loky’ (default), ‘multiprocessing’, ‘threading’. Anything else than the default is experimental and not appropriately implemented in the dbdet member function ‘bin_data’.

  • n_jobs (int) – inversely proportional of the number of cpu’s available for one job. Tasks within one job can grab a maximum of n_CPU_tot/n_jobs of cpu’s. Note that when using the default backend there is no need to adjust this parameter with the current implementation.

  • dark_image (xarray.DataArray) – DataArray with dimensions compatible with the loaded dssc data. If given, it will be subtracted from the dssc data before the binning. The dark image needs to be of dimension module, trainId, pulse, x and y.

  • xgm_normalization (boolean) – if true, the dssc data is normalized by the xgm data before the binning.

  • normevery (int) – integer indicating which out of normevery frame will be normalized.

class toolbox_scs.DSSCFormatter(filepath)[source]
combine_files(filenames=[])[source]

Read the files given in filenames, and store the data in the class variable ‘data’. If no filenames are given, it tries to read the files stored in the class-internal variable ‘_filenames’.

Parameters

filenames (list) – list of strings containing the names of the files to be combined.

add_dataArray(groups=[])[source]

Reads addional xarray-data from the first file given in the list of filenames. This assumes that all the files in the folder contain the same additional data. To be generalized.

Parameters

groups (list) – list of strings with the names of the groups in the h5 file, containing additional xarray data.

add_attributes(attributes={})[source]

Add additional information, such as run-type, as attributes to the formatted .h5 file.

Parameters

attributes (dictionary) – a dictionary, containing information or data of any kind, that will be added to the formatted .h5 file as attributes.

save_formatted_data(filename)[source]

Create a .h5 file containing the main dataset in the group called ‘data’. Additional groups will be created for the content of the variable ‘data_array’. Metadata about the file is added in the form of attributes.

Parameters

filename (str) – the name of the file to be created

toolbox_scs.get_data_formatted(filenames=[], data_list=[])[source]

Combines the given data into one dataset. For any of extra_data’s data types, an xarray.Dataset is returned. The data is sorted along the ‘module’ dimension. The array dimension have the order ‘trainId’, ‘pulse’, ‘module’, ‘x’, ‘y’. This order is required by the extra_geometry package.

Parameters
  • filenames (list of str) – files to be combined as a list of names. Calls ‘_data_from_list’ to actually load the data.

  • data_list (list) – list containing the already loaded data

Returns

data – A xarray.Dataset containing the combined data.

Return type

xarray.Dataset

toolbox_scs.load_xarray(fname, group='data', form='dataset')[source]

Load stored xarray Dataset. Comment: This function exists because of a problem with the standard netcdf engine that is malfunctioning due to related software installed in the exfel-python environment. May be dropped at some point.

Parameters
  • fname (str) – filename as string

  • group (str) – the name of the xarray dataset (group in h5 file).

  • form (str) – specify whether the data to be loaded is a ‘dataset’ or a ‘array’.

toolbox_scs.save_attributes_h5(fname, data={})[source]

Adding attributes to a hdf5 file. This function is intended to be used to attach metadata to a processed run.

Parameters
  • fname (str) – filename as string

  • data (dictionary) – the data that should be added to the file in form of a dictionary.

toolbox_scs.save_xarray(fname, data, group='data', mode='a')[source]

Store xarray Dataset in the specified location

Parameters
  • data (xarray.DataSet) – The data to be stored

  • fname (str, int) – filename

  • overwrite (bool) – overwrite existing data

Raises

ToolBoxFileError – Exception: File existed, but overwrite was set to False.

toolbox_scs.create_dssc_bins(name, coordinates, bins)[source]

Creates a single entry for the dssc binner dictionary. The produced xarray data-array will later be used to perform grouping operations according to the given bins.

Parameters
  • name (str) – name of the coordinate to be binned.

  • coordinates (numpy.ndarray) – the original coordinate values (1D)

  • bins (numpy.ndarray) – the bins according to which the corresponding dimension should be grouped.

Returns

da – A pre-formatted xarray.DataArray relating the specified dimension with its bins.

Return type

xarray.DataArray

Examples

>>> import toolbox_scs as tb
>>> run = tb.open_run(2212, 235, include='*DA*')

1.) binner along ‘pulse’ dimension. Group data into two bins. >>> bins_pulse = [‘pumped’, ‘unpumped’] * 10 >>> binner_pulse = tb.create_dssc_bins(“pulse”,

np.linspace(0,19,20, dtype=int), bins_pulse)

2.) binner along ‘train’ dimension. Group data into bins corresponding

to the positions of a delay stage for instance.

>>> bins_trainId = tb.get_array(run, 'PP800_PhaseShifter', 0.04)
>>> binner_train = tb.create_dssc_bins("trainId",
                            run.trainIds,
                            bins_trainId.values)
toolbox_scs.get_xgm_formatted(run_obj, xgm_name, dssc_frame_coords)[source]

Load the xgm data and define coordinates along the pulse dimension.

Parameters
  • run_obj (extra_data.DataCollection) – DataCollection object providing access to the xgm data to be loaded

  • xgm_name (str) – valid mnemonic of a xgm source

  • dssc_frame_coords (int, list) – defines which dssc frames should be normalized using data from the xgm.

Returns

xgm – xgm data with coordinate ‘pulse’.

Return type

xarray.DataArray

toolbox_scs.load_dssc_info(proposal, run_nr)[source]

Loads the first data file for DSSC module 0 (this is hardcoded) and returns the detector_info dictionary

Parameters
  • proposal (str, int) – number of proposal

  • run_nr (str, int) – number of run

Returns

info – {‘dims’: tuple, ‘frames_per_train’: int, ‘total_frames’: int}

Return type

dictionary

toolbox_scs.load_mask(fname, dssc_mask)[source]

Load a DSSC mask file.

Copyright (c) 2019, Michael Schneider Copyright (c) 2020, SCS-team license: BSD 3-Clause License (see LICENSE_BSD for more info)

Parameters

fname (str) – string of the filename of the mask file

Return type

dssc_mask

toolbox_scs.quickmask_DSSC_ASIC(poslist)[source]

Returns a mask for the given DSSC geometry with ASICs given in poslist blanked. poslist is a list of (module, row, column) tuples. Each module consists of 2 rows and 8 columns of individual ASICS.

Copyright (c) 2019, Michael Schneider Copyright (c) 2020, SCS-team license: BSD 3-Clause License (see LICENSE_BSD for more info)

toolbox_scs.process_dssc_data(proposal, run_nr, module, chunksize, info, dssc_binners, path='./', pulsemask=None, dark_image=None, xgm_mnemonic='SCS_SA3', xgm_normalization=False, normevery=1)[source]

Collects and reduces DSSC data for a single module.

Copyright (c) 2020, SCS-team

Parameters
  • proposal (int) – proposal number

  • run_nr (int) – run number

  • module (int) – DSSC module to process

  • chunksize (int) – number of trains to load simultaneously

  • info (dictionary) – dictionary containing keys ‘dims’, ‘frames_per_train’, ‘total_frames’, ‘trainIds’, ‘number_of_trains’.

  • dssc_binners (dictionary) – a dictionary containing binner objects created by the ToolBox member function “create_binner()”

  • path (str) – location in which the .h5 files, containing the binned data, should be stored.

  • pulsemask (numpy.ndarray) – array of booleans to be used to mask dssc data according to xgm data.

  • dark_image (xarray.DataArray) – an xarray dataarray with matching coordinates with the loaded data. If dark_image is not None it will be subtracted from each individual dssc frame.

  • xgm_normalization (bool) – true if the data should be divided by the corresponding xgm value.

  • xgm_mnemonic (str) – Mnemonic of the xgm data to be used for normalization.

  • normevery (int) – One out of normevery dssc frames will be normalized.

Returns

module_data – xarray datastructure containing data binned according to bins.

Return type

xarray.Dataset

class toolbox_scs.hRIXS(proposalNB, detector='MaranaX')[source]

The hRIXS analysis, especially curvature correction

The objects of this class contain the meta-information about the settings of the spectrometer, not the actual data, except possibly a dark image for background subtraction.

The actual data is loaded into `xarray`s, and stays there.

PROPOSAL

the number of the proposal

Type

int

DETECTOR

the detector to be used. Can be [‘hRIXS_det’, ‘MaranaX’] defaults to ‘hRIXS_det’ for backward-compatibility.

Type

str

X_RANGE

the slice to take in the dispersive direction, in pixels. Defaults to the entire width.

Type

slice

Y_RANGE

the slice to take in the energy direction

Type

slice

THRESHOLD

pixel counts above which a hit candidate is assumed, for centroiding. use None if you want to give it in standard deviations instead.

Type

float

STD_THRESHOLD

same as THRESHOLD, in standard deviations.

DBL_THRESHOLD

threshold controling whether a detected hit is considered to be a double hit.

BINS

the number of bins used in centroiding

Type

int

CURVE_A, CURVE_B

the coefficients of the parabola for the curvature correction

Type

float

USE_DARK

whether to do dark subtraction. Is initially False, magically switches to True if a dark has been loaded, but may be reset.

Type

bool

ENERGY_INTERCEPT, ENERGY_SLOPE

The calibration from pixel to energy

FIELDS

the fields to be loaded from the data. Add additional fields if so desired.

Example

proposal = 3145 h = hRIXS(proposal) h.Y_RANGE = slice(700, 900) h.CURVE_B = -3.695346575286939e-07 h.CURVE_A = 0.024084479232443695 h.ENERGY_SLOPE = 0.018387 h.ENERGY_INTERCEPT = 498.27 h.STD_THRESHOLD = 3.5

DETECTOR_FIELDS
aggregators
set_params(**params)[source]
get_params(*params)[source]
from_run(runNB, proposal=None, extra_fields=(), drop_first=False, subset=None)[source]

load a run

Load the run runNB. A thin wrapper around toolbox.load. :param drop_first: if True, the first image in the run is removed from the dataset. :type drop_first: bool

Example

data = h.from_run(145) # load run 145

data1 = h.from_run(145) # load run 145 data2 = h.from_run(155) # load run 155 data = xarray.concat([data1, data2], ‘trainId’) # combine both

load_dark(runNB, proposal=None)[source]

load a dark run

Load the dark run runNB from proposal. The latter defaults to the current proposal. The dark is stored in this hRIXS object, and subsequent analyses use it for background subtraction.

Example

h.load_dark(166) # load dark run 166

find_curvature(runNB, proposal=None, plot=True, args=None, **kwargs)[source]

find the curvature correction coefficients

The hRIXS has some abberations which leads to the spectroscopic lines being curved on the detector. We approximate these abberations with a parabola for later correction.

Load a run and determine the curvature. The curvature is set in self, and returned as a pair of floats.

Parameters
  • runNB (int) – the run number to use

  • proposal (int) – the proposal to use, default to the current proposal

  • plot (bool) – whether to plot the found curvature onto the data

  • args (pair of float, optional) – a starting value to prime the fitting routine

Example

h.find_curvature(155) # use run 155 to fit the curvature

centroid_one(image)[source]

find the position of photons with sub-pixel precision

A photon is supposed to have hit the detector if the intensity within a 2-by-2 square exceeds a threshold. In this case the position of the photon is calculated as the center-of-mass in a 4-by-4 square.

Return the list of x, y coordinate pairs, corrected by the curvature.

centroid_two(image, energy)[source]

determine position of photon hits on detector

The algrothm is taken from the ESRF RIXS toolbox. The thresholds for determining photon hits are given by the incident photon energy

The function returns arrays containing the single and double hits as x and y coordinates

centroid(data, bins=None, method='auto')[source]

calculate a spectrum by finding the centroid of individual photons

This takes the xarray.Dataset data and returns a copy of it, with a new xarray.DataArray named spectrum added, which contains the energy spectrum calculated for each hRIXS image.

Added a key for switching between algorithims choices are “auto” and “manual” which selects for method for determining whether thresholds there is a photon hit. It changes whether centroid_one or centroid_two is used.

Example

h.centroid(data) # find photons in all images of the run data.spectrum[0, :].plot() # plot the spectrum of the first image

parabola(x)[source]
integrate(data)[source]

calculate a spectrum by integration

This takes the xarray data and returns a copy of it, with a new dataarray named spectrum added, which contains the energy spectrum calculated for each hRIXS image.

First the energy that corresponds to each pixel is calculated. Then all pixels within an energy range are summed, where the intensity of one pixel is distributed among the two energy ranges the pixel spans, proportionally to the overlap between the pixel and bin energy ranges.

The resulting data is normalized to one pixel, so the average intensity that arrived on one pixel.

Example

h.integrate(data) # create spectrum by summing pixels data.spectrum[0, :].plot() # plot the spectrum of the first image

aggregator(da, dim)[source]
aggregate(ds, var=None, dim='trainId')[source]

aggregate (i.e. mostly sum) all data within one dataset

take all images in a dataset and aggregate them and their metadata. For images, spectra and normalizations that means adding them, for others (e.g. delays) adding would not make sense, so we treat them properly. The aggregation functions of each variable are defined in the aggregators attribute of the class. If var is specified, group the dataset by var prior to aggregation. A new variable “counts” gives the number of frames aggregated in each group.

Parameters
  • ds (xarray Dataset) – the dataset containing RIXS data

  • var (string) – One of the variables in the dataset. If var is specified, the dataset is grouped by var prior to aggregation. This is useful for sorting e.g. a dataset that contains multiple delays.

  • dim (string) – the dimension over which to aggregate the data

Example

h.centroid(data) # create spectra from finding photons agg = h.aggregate(data) # sum all spectra agg.spectrum.plot() # plot the resulting spectrum

agg2 = h.aggregate(data, ‘hRIXS_delay’) # group data by delay agg2.spectrum[0, :].plot() # plot the spectrum for first value

aggregate_ds(ds, dim='trainId')[source]
normalize(data, which='hRIXS_norm')[source]

Adds a ‘normalized’ variable to the dataset defined as the ration between ‘spectrum’ and ‘which’

Parameters
  • data (xarray Dataset) – the dataset containing hRIXS data

  • which (string, default="hRIXS_norm") – one of the variables of the dataset, usually “hRIXS_norm” or “counts”

toolbox_scs.get_pes_params(run)[source]

Extract PES parameters for a given extra_data DataCollection. Parameters are gas, binding energy, voltages of the MPOD.

Parameters

run (extra_data.DataCollection) – DataCollection containing the digitizer data

Returns

params – dictionnary of PES parameters

Return type

dict

toolbox_scs.get_pes_tof(run, mnemonics=None, merge_with=None, start=31390, width=300, origin=None, width_ns=None, subtract_baseline=True, baseStart=None, baseWidth=80, sample_rate=2000000000.0)[source]

Extracts time-of-flight spectra from raw digitizer traces. The tracesvare either loaded via ToolBox mnemonics or those in the optionally provided merge_with dataset. The spectra are aligned by pulse Id using the SASE 3 bunch pattern, and have time coordinates in nanoseconds.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data

  • mnemonics (str or list of str) – mnemonics for PES, e.g. “PES_W_raw” or [“PES_W_raw”, “PES_ENE_raw”]. If None and no merge_with dataset is provided, defaults to “PES_W_raw”.

  • merge_with (xarray Dataset) – If provided, the resulting Dataset will be merged with this one. The PES variables of merge_with (if any) will also be computed and merged.

  • start (int) – starting sample of the first spectrum in the raw trace.

  • width (int) – number of samples per spectra.

  • origin (int) – sample of the raw trace that corresponds to time-of-flight origin. If None, origin is equal to start.

  • width_ns (float) – time window for one spectrum. If None, the time window is defined by width / sample rate.

  • subtract_baseline (bool) – If True, subtract baseline defined by baseStart and baseWidth to each spectrum.

  • baseStart (int) – starting sample of the baseline.

  • baseWidth (int) – number of samples to average (starting from baseStart) for baseline calculation.

  • sample_rate (float) – sample rate of the digitizer.

Returns

pes – Dataset containing the PES time-of-flight spectra (e.g. “PES_W_tof”), merged with optionally provided merg_with dataset.

Return type

xarray Dataset

Example

>>> import toolbox_scs as tb
>>> import toolbox_scs.detectors as tbdet
>>> run, ds = tb.load(2927, 100, "PES_W_raw")
>>> pes = tbdet.get_pes_tof(run, merge_with=ds)
class toolbox_scs.Viking(proposalNB)[source]

The Viking analysis (spectrometer used in combination with Andor Newton camera)

The objects of this class contain the meta-information about the settings of the spectrometer, not the actual data, except possibly a dark image for background subtraction.

The actual data is loaded into xarray`s via the method `from_run(), and stays there.

PROPOSAL

the number of the proposal

Type

int

X_RANGE

the slice to take in the non-dispersive direction, in pixels. Defaults to the entire width.

Type

slice

Y_RANGE

the slice to take in the energy dispersive direction

Type

slice

USE_DARK

whether to do dark subtraction. Is initially False, magically switches to True if a dark has been loaded, but may be reset.

Type

bool

ENERGY_CALIB

The 2nd degree polynomial coefficients for calibration from pixel to energy. Defaults to [0, 1, 0] (no calibration applied).

Type

1D array (len=3)

BL_POLY_DEG

the dgree of the polynomial used for baseline subtraction. Defaults to 1.

Type

int

BL_SIGNAL_RANGE

the dispersive-axis range, defined by an interval [min, max], to avoid when fitting a polynomial for baseline subtraction. Multiple ranges can be provided in the form [[min1, max1], [min2, max2], …].

Type

list

FIELDS

the fields to be loaded from the data. Add additional fields if so desired.

Type

list of str

Example

proposal = 2953 v = Viking(proposal) v.X_RANGE = slice(0, 1900) v.Y_RANGE = slice(38, 80) v.ENERGY_CALIB = [1.47802667e-06, 2.30600328e-02, 5.15884589e+02] v.BL_SIGNAL_RANGE = [500, 545]

set_params(**params)[source]
get_params(*params)[source]
from_run(runNB, add_attrs=True)[source]

load a run

Load the run runNB. A thin wrapper around toolbox_scs.load.

Parameters
  • runNB (int) – the run number

  • add_attrs (bool) – if True, adds the camera parameters as attributes to the dataset (see get_camera_params())

  • Output

  • ------

  • ds (xarray Dataset) – the dataset containing the camera images

Example

data = v.from_run(145) # load run 145

data1 = v.from_run(145) # load run 145 data2 = v.from_run(155) # load run 155 data = xarray.concat([data1, data2], ‘trainId’) # combine both

load_dark(runNB=None)[source]
integrate(data)[source]

This function calculates the mean over the non-dispersive dimension to create a spectrum. If the camera parameters are known, the spectrum is multiplied by the number of photoelectrons per ADC count. A new variable “spectrum” is added to the data.

get_camera_gain(run)[source]

Get the preamp gain of the camera in the Viking spectrometer for a specified run.

Parameters
  • run (extra_data DataCollection) – information on the run

  • Output

  • ------

  • gain (int) –

e_per_counts(run, gain=None)[source]

Conversion factor from camera digital counts to photoelectrons per count. The values can be found in the camera datasheet (Andor Newton) but they have been slightly corrected for High Sensitivity mode after analysis of runs 1204, 1207 and 1208, proposal 2937.

Parameters
  • run (extra_data DataCollection) – information on the run

  • gain (int) – the camera preamp gain

  • Output

  • ------

  • ret (float) – photoelectrons per count

get_camera_params(run)[source]
removePolyBaseline(data)[source]

Removes a polynomial baseline to a spectrum, assuming a fixed position for the signal.

Parameters
  • data (xarray Dataset) – The Viking data containing the variable “spectrum”

  • Output

  • ------

  • data – the original dataset with the added variable “spectrum_nobl” containing the baseline subtracted spectra.

xas(data, data_ref, thickness=1, plot=False, plot_errors=True, xas_ylim=(-1, 3))[source]

Given two independent datasets (one with sample and one reference), this calculates the average XAS spectrum (absorption coefficient), associated standard deviation and standard error. The absorption coefficient is defined as -log(It/I0)/thickness.

Parameters
  • data (xarray Dataset) – the dataset containing the spectra with sample

  • data_ref (xarray Dataset) – the dataset containing the spectra without sample

  • thickness (float) – the thickness used for the calculation of the absorption coefficient

  • plot (bool) – If True, plot the resulting average spectra.

  • plot_errors (bool) – If True, adds the 95% confidence interval on the spectra.

  • xas_ylim (tuple or list of float) – the y limits for the XAS plot.

  • Output

  • ------

  • xas (xarray Dataset) – the dataset containing the computed XAS quantities: I0, It, absorptionCoef and their associated errors.

calibrate(runList, plot=True)[source]

This routine determines the calibration coefficients to translate the camera pixels into energy in eV. The Viking spectrometer is calibrated using the beamline monochromator: runs with various monochromatized photon energy are recorded and their peak position on the detector are determined by Gaussian fitting. The energy vs. position data is then fitted to a second degree polynomial.

Parameters
  • runList (list of int) – the list of runs containing the monochromatized spectra

  • plot (bool) – if True, the spectra, their Gaussian fits and the calibration curve are plotted.

  • Output

  • ------

  • energy_calib (np.array) – the calibration coefficients (2nd degree polynomial)

toolbox_scs.calibrate_xgm(run, data, xgm='SCS', plot=False)[source]

Calculates the calibration factor F between the photon flux (slow signal) and the fast signal (pulse-resolved) of the sase 3 pulses. The calibrated fast signal is equal to the uncalibrated one multiplied by F.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the digitizer data.

  • data (xarray Dataset) – dataset containing the pulse-resolved sase 3 signal, e.g. ‘SCS_SA3’

  • xgm (str) – one in {‘XTD10’, ‘SCS’}

  • plot (bool) – If True, shows a plot of the photon flux, averaged fast signal and calibrated fast signal.

Returns

F – calibration factor F defined as: calibrated XGM [microJ] = F * fast XGM array (‘SCS_SA3’ or ‘XTD10_SA3’)

Return type

float

Example

>>> import toolbox_scs as tb
>>> import toolbox_scs.detectors as tbdet
>>> run, data = tb.load(900074, 69, ['SCS_XGM'])
>>> ds = tbdet.get_xgm(run, merge_with=data)
>>> F = tbdet.calibrate_xgm(run, ds, plot=True)
>>> # Add calibrated XGM to the dataset:
>>> ds['SCS_SA3_uJ'] = F * ds['SCS_SA3']
toolbox_scs.get_xgm(run, mnemonics=None, merge_with=None, indices=slice(0, None))[source]

Load and/or computes XGM data. Sources can be loaded on the fly via the mnemonics argument, or processed from an existing dataset (merge_with). The bunch pattern table is used to assign the pulse id coordinates if the number of pulses has changed during the run.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the xgm data.

  • mnemonics (str or list of str) – mnemonics for XGM, e.g. “SCS_SA3” or [“XTD10_XGM”, “SCS_XGM”]. If None, defaults to “SCS_SA3” in case no merge_with dataset is provided.

  • merge_with (xarray Dataset) – If provided, the resulting Dataset will be merged with this one. The XGM variables of merge_with (if any) will also be computed and merged.

  • indices (slice, list, 1D array) – Pulse indices of the XGM array in case bunch pattern is missing.

Returns

merged with Dataset merge_with if provided.

Return type

xarray Dataset with pulse-resolved XGM variables aligned,

Example

>>> import toolbox_scs as tb
>>> run, ds = tb.load(2212, 213, 'SCS_SA3')
>>> ds['SCS_SA3']
toolbox_scs.concatenateRuns(runs)[source]

Sorts and concatenate a list of runs with identical data variables along the trainId dimension.

Input:

runs: (list) the xarray Datasets to concatenate

Output:

a concatenated xarray Dataset

toolbox_scs.find_run_path(proposalNB, runNB, data='raw')[source]

Return the run path given the specified proposal and run numbers.

Parameters
  • proposalNB ((str, int)) – proposal number e.g. ‘p002252’ or 2252

  • runNB ((str, int)) – run number as integer

  • data (str) – ‘raw’, ‘proc’ (processed) or ‘all’ (both ‘raw’ and ‘proc’) to access data from either or both of those folders. If ‘all’ is used, sources present in ‘proc’ overwrite those in ‘raw’. The default is ‘raw’.

Returns

path – The run path.

Return type

str

toolbox_scs.get_array(run=None, mnemonic=None, stepsize=None, subset=None, subFolder='raw', proposalNB=None, runNB=None)[source]

Loads one data array for the specified mnemonic and rounds its values to integer multiples of stepsize for consistent grouping (except for stepsize=None). Returns a 1D array of ones if mnemonic is set to None.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the data. Used if proposalNB and runNB are None.

  • mnemonic (str) – Identifier of a single item in the mnemonic collection. None creates a dummy 1D array of ones with length equal to the number of trains.

  • stepsize (float) – nominal stepsize of the array data - values will be rounded to integer multiples of this value.

  • subset (slice or extra_data.by_index or numpy.s_) – a subset of train that can be loaded with extra_data.by_index[:5] for the first 5 trains. If None, all trains are retrieved.

  • subFolder ((str)) – sub-folder from which to load the data. Use ‘raw’ for raw data or ‘proc’ for processed data.

  • proposalNB ((str, int)) – proposal number e.g. ‘p002252’ or 2252.

  • runNB ((str, int)) – run number e.g. 17 or ‘r0017’.

Returns

data – xarray DataArray containing rounded array values using the trainId as coordinate.

Return type

xarray.DataArray

Raises

ToolBoxValueError – Exception: Toolbox specific exception, indicating a non-valid mnemonic entry

Example

>>> import toolbox_scs as tb
>>> run = tb.open_run(2212, 235)
>>> mnemonic = 'PP800_PhaseShifter'
>>> data_PhaseShifter = tb.get_array(run, mnemonic, 0.5)
toolbox_scs.load(proposalNB=None, runNB=None, fields=None, subFolder='raw', display=False, validate=False, subset=None, rois={}, extract_digitizers=True, extract_xgm=True, extract_bam=True, bunchPattern='sase3', extract_tim=None, extract_laser=None, tim_bp=None, laser_bp=None)

Load a run and extract the data. Output is an xarray with aligned trainIds.

Parameters
  • proposalNB (str, int) – proposal number e.g. ‘p002252’ or 2252

  • runNB (str, int) – run number as integer

  • fields (str, list of str, list of dict) –

    list of mnemonics to load specific data such as “fastccd”, “SCS_XGM”, or dictionnaries defining a custom mnemonic such as {“extra”: {‘source’: ‘SCS_CDIFFT_MAG/SUPPLY/CURRENT’,

    ’key’: ‘actual_current.value’, ‘dim’: None}}

  • subFolder (str) – ‘raw’, ‘proc’ (processed) or ‘all’ (both ‘raw’ and ‘proc’) to access data from either or both of those folders. If ‘all’ is used, sources present in ‘proc’ overwrite those in ‘raw’. The default is ‘raw’.

  • display (bool) – whether to show the run.info or not

  • validate (bool) – whether to run extra-data-validate or not

  • subset (slice or extra_data.by_index or numpy.s_) – a subset of train that can be loaded with extra_data.by_index[:5] for the first 5 trains. If None, all trains are retrieved.

  • rois (dict) –

    a dictionnary of mnemonics with a list of rois definition and the desired names, for example: {‘fastccd’: {‘ref’: {‘roi’: by_index[730:890, 535:720],

    ’dim’: [‘ref_x’, ‘ref_y’]},

    ’sam’: {‘roi’:by_index[1050:1210, 535:720],

    ’dim’: [‘sam_x’, ‘sam_y’]}}}

  • extract_digitizers (bool) – If True, extracts the peaks from digitizer variables and aligns the pulse Id according to the fadc_bp bunch pattern.

  • extract_xgm (bool) – If True, extracts the values from XGM variables (e.g. ‘SCS_SA3’, ‘XTD10_XGM’) and aligns the pulse Id with the sase1 / sase3 bunch pattern.

  • extract_bam (bool) – If True, extracts the values from BAM variables (e.g. ‘BAM1932M’) and aligns the pulse Id with the sase3 bunch pattern.

  • bunchpattern (str) –

    bunch pattern used to extract the Fast ADC pulses. A string or a dict as in:

    {'FFT_PD2': 'sase3', 'ILH_I0': 'scs_ppl'}
    

    Ignored if extract_digitizers=False.

  • ARGUMENTS (DEPRECATED) –

  • extract_tim (DEPRECATED. Use extract_adq412 instead.) –

  • extract_laser (DEPRECATED. Use extract fadc instead.) –

  • tim_bp (DEPRECATED. Use adq412_bp instead.) –

  • laser_bp (DEPRECATED. Use fadc_bp instead.) –

Returns

run, data – extra_data DataCollection of the proposal and run number and an xarray Dataset with aligned trainIds and pulseIds

Return type

DataCollection, xarray.DataArray

Example

>>> import toolbox_scs as tb
>>> run, data = tb.load(2212, 208, ['SCS_SA3', 'MCP2apd', 'nrj'])
toolbox_scs.open_run(proposalNB, runNB, subset=None, **kwargs)[source]

Get extra_data.DataCollection in a given proposal. Wraps the extra_data open_run routine and adds subset selection, out of convenience for the toolbox user. More information can be found in the extra_data documentation.

Parameters
  • proposalNB ((str, int)) – proposal number e.g. ‘p002252’ or 2252

  • runNB ((str, int)) – run number e.g. 17 or ‘r0017’

  • subset (slice or extra_data.by_index or numpy.s_) – a subset of train that can be loaded with extra_data.by_index[:5] for the first 5 trains. If None, all trains are retrieved.

  • **kwargs

  • --------

  • data (str) – default -> ‘raw’

  • include (str) – default -> ‘*’

Returns

run – DataCollection object containing information about the specified run. Data can be loaded using built-in class methods.

Return type

extra_data.DataCollection

toolbox_scs.run_by_path(path)[source]

Return specified run

Wraps the extra_data RunDirectory routine, to ease its use for the scs-toolbox user.

Parameters

path (str) – path to the run directory

Returns

run – DataCollection object containing information about the specified run. Data can be loaded using built-in class methods.

Return type

extra_data.DataCollection

toolbox_scs.load_run_values(prop_or_run, runNB=None, which='mnemonics')[source]

Load the run value for each mnemonic whose source is a CONTORL source (see extra-data DataCollection.get_run_value() for details)

Parameters
  • prop_or_run (extra_data DataCollection or int) – The run (DataCollection) to check for mnemonics. Alternatively, the proposal number (int), for which the runNB is also required.

  • runNB (int) – The run number. Only used if the first argument is the proposal number.

  • which (str) – ‘mnemonics’ or ‘all’. If ‘mnemonics’, only the run values for the ToolBox mnemonics are retrieved. If ‘all’, a compiled dictionnary of all control sources run values is returned.

  • Output

  • ------

  • run_values (a dictionnary containing the mnemonic or all run values.) –

toolbox_scs.check_data_rate(run, fields=None)[source]

Calculates the fraction of train ids that contain data in a run.

Parameters
  • run (extra_data DataCollection) – the DataCollection associated to the data.

  • fields (str, list of str or dict) – mnemonics to check. If None, all mnemonics in the run are checked. A custom mnemonic can be defined with a dictionnary: {‘extra’: {‘source’: ‘SCS_CDIFFT_MAG/SUPPLY/CURRENT’, ‘key’: ‘actual_current.value’}}

  • Output

  • ------ – ret: dictionnary dictionnary with mnemonic as keys and fraction of train ids that contain data as values.

toolbox_scs.__all__[source]
toolbox_scs.extractBunchPattern(bp_table=None, key='sase3', runDir=None)[source]

generate the bunch pattern and number of pulses of a source directly from the bunch pattern table and not using the MDL device BUNCH_DECODER. This is inspired by the euxfel_bunch_pattern package, https://git.xfel.eu/gitlab/karaboDevices/euxfel_bunch_pattern Inputs:

bp_table: DataArray corresponding to the mnemonics “bunchPatternTable”.

If None, the bunch pattern table is loaded using runDir.

key: str, [‘sase1’, ‘sase2’, ‘sase3’, ‘scs_ppl’] runDir: extra-data DataCollection. Required only if bp_table is None.

Outputs:

bunchPattern: DataArray containing indices of the sase/laser pulses for each train npulses: DataArray containing the number of pulses for each train matched: 2-D DataArray mask (trainId x 2700), True where ‘key’ has pulses

toolbox_scs.get_sase_pId(run, loc='sase3', run_mnemonics=None, bpt=None, merge_with=None)[source]

Returns the pulse Ids of the specified loc during a run. If the number of pulses has changed during the run, it loads the bunch pattern table and extract all pulse Ids used.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the data.

  • loc (str) – The location where to check: {‘sase1’, ‘sase3’, ‘scs_ppl’}

  • run_mnemonics (dict) – the mnemonics for the run (see menonics_for_run)

  • bpt (2D-array) – The bunch pattern table. Used only if the number of pulses has changed. If None, it is loaded on the fly.

  • merge_with (xarray.Dataset) – dataset that may contain the bunch pattern table to use in case the number of pulses has changed. If merge_with does not contain the bunch pattern table, it is loaded and added as a variable ‘bunchPatternTable’ to merge_with.

Returns

pulseIds – the pulse ids at the specified location. Returns None if the mnemonic is not in the run.

Return type

np.array

toolbox_scs.npulses_has_changed(run, loc='sase3', run_mnemonics=None)[source]

Checks if the number of pulses has changed during the run for a specific location loc (=’sase1’, ‘sase3’, ‘scs_ppl’ or ‘laser’) If the source is not found in the run, returns True.

Parameters
  • run (extra_data.DataCollection) – DataCollection containing the data.

  • loc (str) – The location where to check: {‘sase1’, ‘sase3’, ‘scs_ppl’}

  • run_mnemonics (dict) – the mnemonics for the run (see menonics_for_run)

Returns

ret – True if the number of pulses has changed or the source was not found, False if the number of pulses did not change.

Return type

bool

toolbox_scs.pulsePatternInfo(data, plot=False)[source]

display general information on the pulse patterns operated by SASE1 and SASE3. This is useful to track changes of number of pulses or mode of operation of SASE1 and SASE3. It also determines which SASE comes first in the train and the minimum separation between the two SASE sub-trains.

Inputs:

data: xarray Dataset containing pulse pattern info from the bunch decoder MDL: {‘sase1, sase3’, ‘npulses_sase1’, ‘npulses_sase3’} plot: bool enabling/disabling the plotting of the pulse patterns

Outputs:

print of pulse pattern info. If plot==True, plot of the pulse pattern.

toolbox_scs.repRate(data=None, runNB=None, proposalNB=None, key='sase3')[source]

Calculates the pulse repetition rate (in kHz) in sase according to the bunch pattern and assuming a grid of 4.5 MHz.

Inputs:

data: xarray Dataset containing pulse pattern, needed if runNB is none runNB: int or str, run number. Needed if data is None proposal: int or str, proposal where to find the run. Needed if data is None key: str in [sase1, sase2, sase3, scs_ppl], source for which the

repetition rate is calculated

Output:

f: repetition rate in kHz

toolbox_scs.is_sase_3(bpt)[source]

Check for prescence of a SASE3 pulse.

Parameters

bpt (numpy array, xarray DataArray) – The bunch pattern data.

Returns

boolean – true if SASE3 pulse is present.

Return type

numpy array, xarray DataArray

toolbox_scs.is_sase_1(bpt)[source]

Check for prescence of a SASE1 pulse.

Parameters

bpt (numpy array, xarray DataArray) – The bunch pattern data.

Returns

boolean – true if SASE1 pulse is present.

Return type

numpy array, xarray DataArray

toolbox_scs.is_ppl(bpt)[source]

Check for prescence of pp-laser pulse.

Parameters

bpt (numpy array, xarray DataArray) – The bunch pattern data.

Returns

boolean – true if pp-laser pulse is present.

Return type

numpy array, xarray DataArray

toolbox_scs.is_pulse_at(bpt, loc)[source]

Check for prescence of a pulse at the location provided.

Parameters
  • bpt (numpy array, xarray DataArray) – The bunch pattern data.

  • loc (str) – The location where to check: {‘sase1’, ‘sase3’, ‘scs_ppl’}

Returns

boolean – true if a pulse is present at loc.

Return type

numpy array, xarray DataArray

toolbox_scs.degToRelPower(x, theta0=0)[source]

converts a half-wave plate position in degrees into relative power between 0 and 1. Inputs:

x: array-like positions of half-wave plate, in degrees theta0: position for which relative power is zero

Output:

array-like relative power

toolbox_scs.positionToDelay(pos, origin=0, invert=True, reflections=1)[source]

converts a motor position in mm into optical delay in picosecond Inputs:

pos: array-like delay stage motor position origin: motor position of time zero in mm invert: bool, inverts the sign of delay if True reflections: number of bounces in the delay stage

Output:

delay in picosecond

toolbox_scs.delayToPosition(delay, origin=0, invert=True, reflections=1)[source]

converts an optical delay in picosecond into a motor position in mm Inputs:

delay: array-like delay in ps origin: motor position of time zero in mm invert: bool, inverts the sign of delay if True reflections: number of bounces in the delay stage

Output:

delay in picosecond

toolbox_scs.fluenceCalibration(hwp, power_mW, npulses, w0x, w0y=None, train_rep_rate=10, fit_order=1, plot=True, xlabel='HWP [%]')[source]

Given a measurement of relative powers or half wave plate angles and averaged powers in mW, this routine calculates the corresponding fluence and fits a polynomial to the data.

Parameters
  • hwp (array-like (N)) – angle or relative power from the half wave plate

  • power_mW (array-like (N)) – measured power in mW by powermeter

  • npulses (int) – number of pulses per train during power measurement

  • w0x (float) – radius at 1/e^2 in x-axis in meter

  • w0y (float, optional) – radius at 1/e^2 in y-axis in meter. If None, w0y=w0x is assumed.

  • train_rep_rate (float) – repetition rate of the FEL, by default equals to 10 Hz.

  • fit_order (int) – order of the polynomial fit

  • plot (bool) – Plot the results if True

  • xlabel (str) – xlabel for the plot

  • Output

  • ------

  • F (ndarray (N)) – fluence in mJ/cm^2

  • fit_F (ndarray) – coefficients of the fluence polynomial fit

  • E (ndarray (N)) – pulse energy in microJ

  • fit_E (ndarray) – coefficients of the fluence polynomial fit

toolbox_scs.align_ol_to_fel_pId(ds, ol_dim='ol_pId', fel_dim='sa3_pId', offset=0, fill_value=np.nan)[source]

Aligns the optical laser (OL) pulse Ids to the FEL pulse Ids. The new OL coordinates are calculated as ds[ol_dim] + ds[fel_dim][0] + offset. The ol_dim is then removed, and if the number of OL and FEL pulses are different, the missing values are replaced by fill_value (NaN by default).

Parameters
  • ds (xarray.Dataset) – Dataset containing both OL and FEL dimensions

  • ol_dim (str) – name of the OL dimension

  • fel_dim (str) – name of the FEL dimension

  • offset (int) – offset added to the OL pulse Ids.

  • fill_value ((scalar or dict-like, optional)) – Value to use for newly missing values. If a dict-like, maps variable names to fill values. Use a data array’s name to refer to its values.

  • Output

  • ------

  • ds – The newly aligned dataset

toolbox_scs.get_undulator_config(run, park_pos=62.0, plot=True)[source]

Extract the undulator cells configuration from a given run. The gap size and K factor as well as the magnetic chicane delay and photon energy of colors 1, 2 and 3 are compiled into an xarray Dataset.

Note: This function looks at run control values, it does not reflect any change of values during the run. Do not use to extract configuration when scanning the undulator.

Parameters
  • run (EXtra-Data DataCollection) – The run containing the undulator information

  • park_pos (float, optional) – The parked position of a cell (i.e. when fully opened)

  • plot (bool, optional) – If True, plot the undulator cells configuration

Returns

cells – The resulting dataset of the undulator configuration

Return type

xarray Dataset

toolbox_scs.mnemonics_for_run(prop_or_run, runNB=None)[source]

Returns the availble ToolBox mnemonics for a give extra_data DataCollection, or a given proposal + run number.

Parameters
  • prop_or_run (extra_data DataCollection or int) – The run (DataCollection) to check for mnemonics. Alternatively, the proposal number (int), for which the runNB is also required.

  • runNB (int) – The run number. Only used if the first argument is the proposal number.

Returns

mnemonics – The dictionnary of mnemonics that are available in the run.

Return type

dict

Example

>>> import toolbox_scs as tb
>>> tb.mnemonics_for_run(2212, 213)
toolbox_scs.__all__[source]
toolbox_scs.xas(nrun, bins=None, Iokey='SCS_SA3', Itkey='MCP3peaks', nrjkey='nrj', Iooffset=0, plot=False, fluorescence=False)[source]

Compute the XAS spectra from a xarray nrun.

Inputs:

nrun: xarray of SCS data bins: an array of bin-edges or an integer number of

desired bins or a float for the desired bin width.

Iokey: string for the Io fields, typically ‘SCS_XGM’ Itkey: string for the It fields, typically ‘MCP3apd’ nrjkey: string for the nrj fields, typically ‘nrj’ Iooffset: offset to apply on Io plot: boolean, displays a XAS spectrum if True fluorescence: boolean, if True, absorption is the ratio,

if False, absorption is negative log

Outputs:
a dictionnary containing:

nrj: the bin centers muA: the absorption sigmaA: standard deviation on the absorption sterrA: standard error on the absorption muIo: the mean of the Io counts: the number of events in each bin

toolbox_scs.xasxmcd(dataP, dataN)[source]

Compute XAS and XMCD from data with both magnetic field direction Inputs:

dataP: structured array for positive field dataN: structured array for negative field

Outputs:

xas: structured array for the sum xmcd: structured array for the difference

class toolbox_scs.parameters(proposal, darkrun, run, module, gain, drop_intra_darks=True)[source]

Parameters contains all input parameters for the BOZ corrections.

This is used in beam splitting off-axis zone plate spectrocopy analysis as well as the during the determination of correction parameters themselves to ensure they can be reproduced.

Inputs

proposal: int, proposal number darkrun: int, run number for the dark run run: int, run number for the data run module: int, DSSC module number gain: float, number of ph per bin drop_intra_darks: drop every second DSSC frame

dask_load_persistently(dark_data_size_Gb=None, data_size_Gb=None)[source]

Load dask data array in memory.

Inputs

dark_data_size_Gb: float, optional size of dark to load in memory, in Gb data_size_Gb: float, optional size of data to load in memory, in Gb

use_gpu()[source]
set_mask(arr)[source]

Set mask of bad pixels.

Inputs
arr: either a boolean array of a DSSC module image or a list of bad

pixel indices

get_mask()[source]

Get the boolean array bad pixel of a DSSC module.

get_mask_idx()[source]

Get the list of bad pixel indices.

flat_field_guess(guess=None)[source]

Set the flat-field guess parameter for the fit and returns it.

Inputs
guess: a list of 8 floats, the 4 first to define the plane

ax+by+cz+d=0 for ‘n’ beam and the 4 last for the ‘p’ beam in case mirror symmetry is disbaled

set_flat_field(plane, prod_th=None, ratio_th=None)[source]

Set the flat-field plane definition.

get_flat_field()[source]

Get the flat-field plane definition.

set_Fnl(Fnl)[source]

Set the non-linear correction function.

get_Fnl()[source]

Get the non-linear correction function.

save(path='./')[source]

Save the parameters as a JSON file.

Inputs

path: str, where to save the file, default to ‘./’

classmethod load(fname)[source]

Load parameters from a JSON file.

Inputs

fname: string, name a the JSON file to load

__str__()[source]

Return str(self).

toolbox_scs.get_roi_pixel_pos(roi, params)[source]

Compute fake or real pixel position of an roi from roi center.

Inputs:

roi: dictionnary params: parameters

Returns:

X, Y: 1-d array of pixel position.

toolbox_scs.bad_pixel_map(params)[source]

Compute the bad pixels map.

Inputs

params: parameters

rtype

bad pixel map

toolbox_scs.inspect_dark(arr, mean_th=(None, None), std_th=(None, None))[source]

Inspect dark run data and plot diagnostic.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) mean_th: tuple of threshold (low, high), default (None, None), to compute

a mask of good pixels for which the mean dark value lie inside this range

std_th: tuple of threshold (low, high), default (None, None), to compute a

mask of bad pixels for which the dark std value lie inside this range

returns

fig

rtype

matplotlib figure

toolbox_scs.histogram_module(arr, mask=None)[source]

Compute a histogram of the 9 bits raw pixel values over a module.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) mask: optional bad pixel mask

rtype

histogram

toolbox_scs.inspect_histogram(arr, arr_dark=None, mask=None, extra_lines=False)[source]

Compute and plot a histogram of the 9 bits raw pixel values.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) arr: dask array of reshaped dssc dark data (trainId, pulseId, x, y) mask: optional bad pixel mask extra_lines: boolean, default False, plot extra lines at period values

returns
  • (h, hd) (histogram of arr, arr_dark)

  • figure

toolbox_scs.find_rois(data_mean, threshold, extended=False)[source]

Find rois from 3 beams configuration.

Inputs

data_mean: dark corrected average image threshold: threshold value to find beams extended: boolean, True to define additional ASICS based rois

returns

rois

rtype

dictionnary of rois

toolbox_scs.find_rois_from_params(params)[source]

Find rois from 3 beams configuration.

Inputs

params: parameters

returns

rois

rtype

dictionnary of rois

toolbox_scs.inspect_rois(data_mean, rois, threshold=None, allrois=False)[source]

Find rois from 3 beams configuration from mean module image.

Inputs

data_mean: mean module image threshold: float, default None, threshold value used to detect beams

boundaries

allrois: boolean, default False, plot all rois defined in rois or only the

main ones ([‘n’, ‘0’, ‘p’])

rtype

matplotlib figure

toolbox_scs.compute_flat_field_correction(rois, params, plot=False)[source]

Compute the plane-field correction on beam rois.

Inputs

rois: dictionnary of beam rois[‘n’, ‘0’, ‘p’] params: parameters plot: boolean, True by default, diagnostic plot

returns
  • numpy 2D array of the flat-field correction evaluated over one DSSC ladder

  • (2 sensors)

toolbox_scs.inspect_flat_field_domain(avg, rois, prod_th, ratio_th, vmin=None, vmax=None)[source]

Extract beams roi from average image and compute the ratio.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary or ROIs prod_th, ratio_th: tuple of floats for low and high threshold on

product and ratio

vmin: imshow vmin level, default None will use 5 percentile value vmax: imshow vmax level, default None will use 99.8 percentile value

returns
  • fig (matplotlib figure plotted)

  • domain (a tuple (n_m, p_m) of domain for the ‘n’ and ‘p’ order)

toolbox_scs.inspect_plane_fitting(avg, rois, domain=None, vmin=None, vmax=None)[source]

Extract beams roi from average image and compute the ratio.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary of rois domain: list of domain mask for the -1st and +1st order vmin: imshow vmin level, default None will use 5 percentile value vmax: imshow vmax level, default None will use 99.8 percentile value

returns

fig

rtype

matplotlib figure plotted

toolbox_scs.plane_fitting_domain(avg, rois, prod_th, ratio_th)[source]

Extract beams roi, compute their ratio and the domain.

Inputs

avg: module average image with no saturated shots for the flat-field

determination

rois: dictionnary or rois containing the 3 beams [‘n’, ‘0’, ‘p’] with ‘0’

as the reference beam in the middle

prod_th: float tuple, low and hight threshold level to determine the plane

fitting domain on the product image of the orders

ratio_th: float tuple, low and high threshold level to determine the plane

fitting domain on the ratio image of the orders

returns
  • n (img ratio ‘n’/‘0’)

  • n_m (mask where the the product ‘n’‘0’ is higher than 5 indicting that the*) – img ratio ‘n’/‘0’ is defined

  • p (img ratio ‘p’/‘0’)

  • p_m (mask where the the product ‘p’‘0’ is higher than 5 indicting that the*) – img ratio ‘p’/‘0’ is defined

toolbox_scs.plane_fitting(params)[source]

Fit the plane flat-field normalization.

Inputs

params: parameters

returns

res – defines the plane as a*x + b*y + c*z + d = 0

rtype

the minimization result. The fitted vector res.x = [a, b, c, d]

toolbox_scs.ff_refine_crit(p, alpha, params, arr_dark, arr, tid, rois, mask, sat_level=511)[source]

Criteria for the ff_refine_fit.

Inputs

p: ff plane params: parameters arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels sat_level: integer, default 511, at which level pixel begin to saturate

rtype

sum of standard deviation on binned 0th order intensity

toolbox_scs.ff_refine_fit(params)[source]

Refine the flat-field fit by minimizing data spread.

Inputs

params: parameters

returns
  • res (scipy minimize result. res.x is the optimized parameters)

  • fitrres (iteration index arrays of criteria results for) – [alpha=0, alpha, alpha=1]

toolbox_scs.nl_domain(N, low, high)[source]

Create the input domain where the non-linear correction defined.

Inputs

N: integer, number of control points or intervals low: input values below or equal to low will not be corrected high: input values higher or equal to high will not be corrected

rtype

array of 2**9 integer values with N segments

toolbox_scs.nl_lut(domain, dy)[source]

Compute the non-linear correction.

Inputs

domain: input domain where dy is defined. For zero no correction is

defined. For non-zero value x, dy[x] is applied.

dy: a vector of deviation from linearity on control point homogeneously

dispersed over 9 bits.

returns

F_INL – lookup table with 9 bits integer input

rtype

default None, non linear correction function given as a

toolbox_scs.nl_crit(p, domain, alpha, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)[source]

Criteria for the non linear correction.

Inputs

p: vector of dy non linear correction domain: domain over which the non linear correction is defined alpha: float, coefficient scaling the cost of the correction function

in the criterion

arr_dark: dark data arr: data tid: train id of arr data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask fo good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

returns
  • (1.0 - alpha)*err1 + alpha*err2, where err1 is the 1e8 times the mean of

  • error squared from a transmission of 1.0 and err2 is the sum of the square

  • of the deviation from the ideal detector response.

toolbox_scs.nl_fit(params, domain)[source]

Fit non linearities correction function.

Inputs

params: parameters domain: array of index

returns
  • res (scipy minimize result. res.x is the optimized parameters)

  • fitrres (iteration index arrays of criteria results for) – [alpha=0, alpha, alpha=1]

toolbox_scs.inspect_nl_fit(res_fit)[source]

Plot the progress of the fit.

Inputs

res_fit:

rtype

matplotlib figure

toolbox_scs.snr(sig, ref, methods=None, verbose=False)[source]

Compute mean, std and SNR from transmitted signal sig and I0 signal ref.

Inputs

sig: 1D signal samples ref: 1D reference samples methods: None by default or list of strings to select which methods to use.

Possible values are ‘direct’, ‘weighted’, ‘diff’. In case of None, all methods will be calculated.

verbose: booleand, if True prints calculated values

returns
  • dictionnary of [methods][value] where value is ‘mu’ for mean and ‘s’ for

  • standard deviation.

toolbox_scs.inspect_Fnl(Fnl)[source]

Plot the correction function Fnl.

Inputs

Fnl: non linear correction function lookup table

rtype

matplotlib figure

toolbox_scs.inspect_correction(params, gain=None)[source]

Criteria for the non linear correction.

Inputs

params: parameters gain: float, default None, DSSC gain in ph/bin

rtype

matplotlib figure

toolbox_scs.load_dssc_module(proposalNB, runNB, moduleNB=15, subset=slice(None), drop_intra_darks=True, persist=False, data_size_Gb=None)[source]

Load single module dssc data as dask array.

Inputs

proposalNB: proposal number runNB: run number moduleNB: default 15, module number subset: default slice(None), subset of trains to load drop_intra_darks: boolean, default True, remove intra darks from the data persist: default False, load all data persistently in memory data_size_Gb: float, if persist is True, can optionaly restrict

the amount of data loaded for dark data and run data in Gb

returns
  • arr (dask array of reshaped dssc data (trainId, pulseId, x, y))

  • tid (array of train id number)

toolbox_scs.average_module(arr, dark=None, ret='mean', mask=None, sat_roi=None, sat_level=300, F_INL=None)[source]

Compute the average or std over a module.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) dark: default None, dark to be substracted ret: string, either ‘mean’ to compute the mean or ‘std’ to compute the

standard deviation

mask: default None, mask of bad pixels to ignore sat_roi: roi over which to check for pixel with values larger than

sat_level to drop the image from the average or std

sat_level: int, minimum pixel value for a pixel to be considered saturated F_INL: default None, non linear correction function given as a

lookup table with 9 bits integer input

rtype

average or standard deviation image

toolbox_scs.process_module(arr, tid, dark, rois, mask=None, sat_level=511, flat_field=None, F_INL=None, use_gpu=False)[source]

Process one module and extract roi intensity.

Inputs

arr: dask array of reshaped dssc data (trainId, pulseId, x, y) tid: array of train id number dark: pulse resolved dark image to remove rois: dictionnary of rois mask: default None, mask of ignored pixels sat_level: integer, default 511, at which level pixel begin to saturate flat_field: default None, flat-field correction F_INL: default None, non-linear correction function given as a

lookup table with 9 bits integer input

rtype

dataset of extracted pulse and train resolved roi intensities.

toolbox_scs.process(Fmodel, arr_dark, arr, tid, rois, mask, flat_field, sat_level=511, use_gpu=False)[source]

Process dark and run data with corrections.

Inputs

Fmodel: correction lookup table arr_dark: dark data arr: data rois: [‘n’, ‘0’, ‘p’, ‘sat’] rois mask: mask of good pixels flat_field: zone plate flat-field correction sat_level: integer, default 511, at which level pixel begin to saturate

rtype

roi extracted intensities

toolbox_scs.inspect_saturation(data, gain, Nbins=200)[source]

Plot roi integrated histogram of the data with saturation

Inputs

data: xarray of roi integrated DSSC data gain: nominal DSSC gain in ph/bin Nbins: number of bins for the histogram, by default 200

returns
  • f (handle to the matplotlib figure)

  • h (xarray of the histogram data)

toolbox_scs.reflectivity(data, Iokey='FastADC5peaks', Irkey='FastADC3peaks', delaykey='PP800_DelayLine', binWidth=0.05, positionToDelay=True, origin=None, invert=False, pumpedOnly=False, alternateTrains=False, pumpOnEven=True, Ioweights=False, plot=True, plotErrors=True, units='mm')[source]

Computes the reflectivity R = 100*(Ir/Io[pumped] / Ir/Io[unpumped] - 1) as a function of delay. Delay can be a motor position in mm or an optical delay in ps, with possibility to convert from position to delay. The default scheme is alternating pulses pumped/unpumped/… in each train, also possible are alternating trains and pumped only. If fitting is enabled, attempts a double exponential (default) or step function fit.

Parameters
  • data (xarray Dataset) – Dataset containing the Io, Ir and delay data

  • Iokey (str) – Name of the Io variable

  • Irkey (str) – Name of the Ir variable

  • delaykey (str) – Name of the delay variable (motor position in mm or optical delay in ps)

  • binWidth (float) – width of bin in units of delay variable

  • positionToDelay (bool) – If True, adds a time axis converted from position axis according to origin and invert parameters. Ignored if origin is None.

  • origin (float) – Position of time overlap, shown as a vertical line. Used if positionToDelay is True to convert position to time axis.

  • invert (bool) – Used if positionToDelay is True to convert position to time axis.

  • pumpedOnly (bool) – Assumes that all trains and pulses are pumped. In this case, Delta R is defined as Ir/Io.

  • alternateTrains (bool) – If True, assumes that trains alternate between pumped and unpumped data.

  • pumpOnEven (bool) – Only used if alternateTrains=True. If True, even trains are pumped, if False, odd trains are pumped.

  • Ioweights (bool) – If True, computes the ratio of the means instead of the mean of the ratios Irkey/Iokey. Useful when dealing with large intensity variations.

  • plot (bool) – If True, plots the results.

  • plotErrors (bool) – If True, plots the 95% confidence interval.

  • Output

  • ------ – xarray Dataset containing the binned Delta R, standard deviation, standard error, counts and delays, and the fitting results if full is True.

toolbox_scs.knife_edge(ds, axisKey='scannerX', signalKey='FastADC4peaks', axisRange=None, p0=None, full=False, plot=False, display=False)

Calculates the beam radius at 1/e^2 from a knife-edge scan by fitting with erfc function: f(x, x0, w0, a, b) = a*erfc(np.sqrt(2)*(x-x0)/w0) + b with w0 the beam radius at 1/e^2 and x0 the beam center.

Parameters
  • ds (xarray Dataset) – dataset containing the detector signal and the motor position.

  • axisKey (str) – key of the axis against which the knife-edge is performed.

  • signalKey (str) – key of the detector signal.

  • axisRange (list of floats) – edges of the scanning axis between which to apply the fit.

  • p0 (list of floats, numpy 1D array) – initial parameters used for the fit: x0, w0, a, b. If None, a beam radius of 100 micrometers is assumed.

  • full (bool) – If False, returns the beam radius and standard error. If True, returns the popt, pcov list of parameters and covariance matrix from scipy.optimize.curve_fit.

  • plot (bool) – If True, plots the data and the result of the fit. Default is False.

  • display (bool) – If True, displays info on the fit. True when plot is True, default is False.

Returns

error from the fit in mm. If full is True, returns parameters and covariance matrix from scipy.optimize.curve_fit function.

Return type

If full is False, tuple with beam radius at 1/e^2 in mm and standard

toolbox_scs.__all__[source]