HDF5 Virtualise¶
How To Make Virtual CXI data files¶
These commands are available in our Anaconda installation on Maxwell. To use them, first run:
module load exfel exfel-python
extra-data-make-virtual-cxi creates a file in CXI format with virtual datasets for the detector data. This can be used as input for CrystFEL, for example. Run it with a run directory::
extra-data-make-virtual-cxi /gpfs/exfel/exp/SPB/201830/p900022/proc/r0034
The file with the virtual dataset will be written in the scratch directory for the proposal. You can also use the -o option to specify an output file.
More Information
Virtual datasets are a feature introduced with HDF5 1.10. If you are getting HDF5 errors trying to read these files, it may mean that you are using an older version of HDF5.
Checking Access¶
Reading data from a virtual dataset doesn't fail if it can't access the source data files; it reads 0s (the fill value) instead. Use hdf5-vds-check to check that the source data is accessible::
hdf5-vds-check /gpfs/exfel/exp/SPB/201830/p900022/scratch/r0034_detectors_virt.cxi
Installing Tools Elsewhere¶
These commands are available in the module on Maxwell. If you need to install them elsewhere, they are available as Python packages::
pip install extra-data hdf5-vds-check