How To Make Virtual CXI data files
==================================

These commands are available in our Anaconda installation on Maxwell.
To use them, first run::

    module load exfel exfel_anaconda3

``extra-data-make-virtual-cxi`` creates a file in CXI format with virtual datasets
for the detector data. This can be used as input for CrystFEL, for example.
Run it with a run directory::

    extra-data-make-virtual-cxi /gpfs/exfel/exp/SPB/201830/p900022/proc/r0034

The file with the virtual dataset will be written in the ``scratch`` directory
for the proposal. You can also use the ``-o`` option to specify an output file.

.. seealso::

   `extra-data-make-virtual-cxi reference docs
   <https://extra-data.readthedocs.io/en/latest/cli.html#extra-data-make-virtual-cxi>`_

Virtual datasets are a feature introduced with HDF5 1.10. If you are getting
HDF5 errors trying to read these files, it may mean that you are using an older
version of HDF5.

Checking access
---------------

Reading data from a virtual dataset doesn't fail if it can't access the
source data files; it reads 0s (the fill value) instead.
Use ``hdf5-vds-check`` to check that the source data is accessible::

    hdf5-vds-check /gpfs/exfel/exp/SPB/201830/p900022/scratch/r0034_detectors_virt.cxi


Installing tools elsewhere
--------------------------

These commands are available in the module on Maxwell. If you need to
install them elsewhere, they are available as Python packages::

    pip install extra-data hdf5-vds-check