Release Notes¶
3.5.4¶
13-06-2022
- [AGIPD] Convert bias_voltage parameter condition to integer in cal_tools.
- [LPD] Fix correcting a single pulse.
- [LPD] VCXI require 4 modules.
3.5.3¶
19-05-2022
- [LPD][CORRECT] Optionally create virtual CXI files
- [LPD][CORRECT] Expose max-nodes parameter
- [AGIPD] Replace gain_choose_int by fused types
- Fix missing install of restful_config.yaml
- Fix use of xfel-calibrate –skip-report
3.5.2¶
16.05.2022
- [LPD][CORRECT] New correction notebook for LPD
- New files module to write European XFEL HDF5 corrected data files.
3.5.1¶
05-04-2022
- Calibration Constant version’s new Variant file attribute. To indicate method of handling the constant post retrieval. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/619
- Epix100 dark Badpixels Map. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/637
- skip-plots flag to finish correction before plotting. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/635
- First trainId’s timestamp as RAW data creation_time, if there is myMDC connection. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/630
- AGIPD correction can correct one cellId without plotting errors. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/642
- Fixed mode relative gain constants in Jungfrau can be retrieved. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/639
- Only instrument source is selected to check number of trains to dark process. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/636
- AGIPD trains for dark processing is selected for each module individually. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/653
- Produce report after trying to correct AGIPD run with no images. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/641
- AGIPD’s bias voltage for AGIPD1M is read from slow data. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/647
- Removed psutil dependency. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/653
- Update Pasha to 0.1.1 https://git.xfel.eu/detectors/pycalibration/-/merge_requests/638
3.5.0¶
01-03-2022
- Updating Correction and dark notebooks for JUNGFRAU: https://git.xfel.eu/detectors/pycalibration/-/merge_requests/518
- Updating Correction and dark notebooks for AGIPD: https://git.xfel.eu/detectors/pycalibration/-/merge_requests/535
- Updating Correction and dark notebooks for PnCCD: https://git.xfel.eu/detectors/pycalibration/-/merge_requests/559
- Updating Correction and dark notebooks for ePix100: https://git.xfel.eu/detectors/pycalibration/-/merge_requests/500
- EXtra-data is integrated to read files in pycalibration for AGIPD, JUNGFRAU, ePix100, and PnCCD. Dark and Correction notebooks.
- Pasha is now used for processing data for JUNGFRAU, ePix100 and PnCCD.
- pyDetLib correction functions were removed (except for common-mode correction).
- db-module is useless now for JUNGFRAU, ePix100 and PnCCD. Some parameters were updated in dark and correction notebooks for the mentioned detectors.
- gain_mode and burst mode are now available for JUNGFRAU. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/591
- JUNGFRAU has now a new badpixel value, WRONG_GAIN_VALUE. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/591
- Pass through available for testing in-progress ORCA service. https://git.xfel.eu/detectors/pycalibration/-/merge_requests?scope=all&state=merged&search=orca
- Non-calibrated RAW h5files are no longer copied.
- High priority partitions (upex-high`and `upex-middle) are used for runs from ACTIVE and READY proposals, only. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/628
- Supporting to disable LPD Correction through the webservice. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/629
- Compatibility for old DAQ files for REMI is added. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/607
- server-overview refactors. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/593 https://git.xfel.eu/detectors/pycalibration/-/merge_requests/589
- AGIPD correction notebook support AgipdLitFrameFinder device. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/603
- Parsing code arguments in xfel-calibrate is refactored. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/575
- skip-plots option for AGIPD. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/581
- Native implementation for transposition of constants AGIPD. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/580
- Trains for AGIPD can be selected for correction. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/574
- Skip report flag in xfel-calibrate. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/579
- Fix ReadTheDocs. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/448
- Fix error reporting for re-injecting the same CCV. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/627
- Fix AGIPD for legacy runs without gain_mode. https://git.xfel.eu/detectors/pycalibration/-/merge_requests/617 https://git.xfel.eu/detectors/pycalibration/-/merge_requests/626
- Pinning markupsafe version 2.0.1 https://git.xfel.eu/detectors/pycalibration/-/merge_requests/631
- Pinning psutil 5.9.0 https://git.xfel.eu/detectors/pycalibration/-/merge_requests/535
- Updating Extra-data to 1.9.1 https://git.xfel.eu/detectors/pycalibration/-/merge_requests/535
- Updating h5py to 3.5.0 https://git.xfel.eu/detectors/pycalibration/-/merge_requests/602
3.4.3¶
20-10-2021
- Update pyDetLib tag.
- Add explicit dependencies on matplotlib, scipy.
- Remove outdated matplotlib rcParams setting.
- Update EXtra-geom to 1.6.
- Remove cluster_profile parameter from notebooks which don’t use it.
- Fix checking availability for the concurrency parameter.
- Fix launching work directly (not via Slurm).
- Fix sphinx-rep temp folder recreation, if sphinx-rep already existed.
- Fix missing string conversion for slurm-scheduling argument.
- Fix title reports for multiple detectors per run folder.
- Append to .out files for preemptable finalize job.
- [AGIPD] [Correct] Reuse previously found constants.
- [AGIPD] Fix missing memory cell index in SlopesPC constant sanitization.
- [AGIPD] Only use bad pixels from darks in agipdutils.baseline_correct_via_stripes.
- [AGIPD] [Dark] Use function to get list of karabo_da from run for making Slurm jobs.
- [EPIX100][CORRECT] Set absolute_gain to false if relative gain was not retrieved.
- [JUNGFRAU] Fix running for multiple modules and flip logic for do_relative_gain.
- [JUNGFRAU] Style changes for Dark and Correct notebooks.
- [REMI] Add notebook to reconstruct detector hits from raw data.
- [webservice] Check run migration status using MyMDC.
- Resolve “Skip ZMQ tests if zmq connection for calibration DB not available”.
- Reproducibility, step 1.
3.4.2¶
17-09-2021
- Remove driver=core from all notebook
- [webservice] Make use of Dynaconf for managing secrets.
- [webservice] Make use of dedicated slurm partitions.
- [webservice] Handle missing migration information (missing user.status fattr).
- [webservice] Implement, raise, and catch, migration errors to send mdc messages.
- [webservice] Simplify handling of user notebook paths.
- [webservice] Update princess to 0.4 (use Unix sockets).
- [webservice] Update MyMDC with begin and end times.
- [webservice] create output folder before copying slow data.
- [AGIPD] [CORRECT] read acq_rate from slow data.
- [AGIPD][CORRECT] Set default memory cells to 352.
- [AGIPD] [CORRECT] Set maximum pulses to correct based on file content.
- [AGIPD] [FF] Correctly label legends in figures.
- [AGIPD] [FF] Add HIBEF AGIPD500K and fix some issue with retrieval of conditions.
- [Jungfrau] Add Gain setting to Jungfrau notebooks.
- [Jungfrau] Fix max gain plot in LPD correct notebook
- [JUNGFRAU] [DARK] Clearer error message for Jungfrau Dark notebooks no suitable files are found
- [LPD] [CORRECT] Fix max gain plot.
- [EPIX100] [CORRECT] Solve conflict between gain correction and clustering
3.4.1¶
16-07-2021
- Update h5py to 3.3
- Stop execution on notebook errors
- [AGIPD] Add integration time as operating condition to all notebooks
- [webservice] Add blocklist pattern when copying untouched files in webservice.
- [webservice] Expose dark configurations in update_config.py
- Fix MetadataClient.get_proposal_runs arguments call.
- Fix Use snapshot for injecting constants for old PDU mappings
- Fix the old time-summary (creation time for retrieved constants)
- Update documentation notes on venv installation
- Ignore all .so files in gitignore
3.4.0¶
28-06-2021
- Update to Python 3.8.
- Bump numpy to 1.20.3 and remove fabio.
- remove PyQT dependency.
- Disable dark requests from serve overview.
- Update report upload parameter key.
- Override locale to always use UTF-8.
- Assorted cleanup of xfel-calibrate.
- Fix pre-commit.
- Use argparse only if name is main, call main with args dict.
- [webservice] Use full hostname for webservice overview.
- [webservice] Show clearer messages when running webservice in sim mode.
- [webservice] Fix filename lineno and typos in webservice logs.
- [webservice] Fix creating an extra run folder in run output folder.
- [AGIPD] Parallelize gain/mask compression for writing corrected AGIPD files.
- [AGIPD][DARK] Fix processing empty sequence files.
- [AGIPD][PC][FF] Update notebooks with new CALCAT mapping.
- [AGIPD][JUNGFRAU] Use all available sequences for processing darks for AGIPD and Jungfrau.
- [AGIPD][LPD][DSSC] Fix retrieve old constants for comparison for modular detectors.
- [LPD] Fix data paths in LPD notebook.
- [REMI] Fix user notebook path for REMI correct notebook provisionally.
- [EPIX][CORRECT] Add Common mode correction.
- Fix plotting-related warnings.
- Test update config.
- Test get_from_db and send_to_db.