Specific use cases & deployments
================================

SCS toolbox
-----------

This section provides guidelines for data analysis softwares / toolchains
which are deployed for the SCS instrument.

.. seealso::

   `SCS Instrument documentation <https://rtd.xfel.eu/docs/scs-documentation/en/latest/>`_

   `SCS analysis toolbox <https://rtd.xfel.eu/docs/scs-toolbox/en/latest/index.html>`_

.. _EXtra-foam_FastCCD_SCS:

EXtra-foam (FastCCD)
++++++++++++++++++++

Toolchain for real-time data analysis and visualization of FastCCD related
data analysis. In this tool chain, FastCCDProcessor_ correlates and processes
data from FastCCD and other data sources (XGM, digitizer, motors, etc.).
The processed data is then sent out via the karabo bridge to :ref:`extra_foam`,
which provides a rich interface for complicated data analysis and
visualization.

.. _FastCCDProcessor: https://git.xfel.eu/gitlab/karaboDevices/fastCCDProcessorPy


**Usage**

1. Open the project *ZMQ_BRIDGE* within the **SCS** topic;

2. Instantiate two devices: `SCS_XAS_FASTCCD/DA/PROC` and
`SCS_XAS_FASTCCD/DA/BRIDGE`;

3. Open the following scene in Karabo:

.. figure:: _images/FastCCDCorrelatorScene.png

.. note::
    Please strictly follow the following steps since the data flows from
    upstream to downstream (processor -> bridge -> EXtra-foam). It also applies
    for troubleshooting!

4. Click the **Start** button in the scene. If `N processed trains` is
   increasing, it indicates that the processor is working. Otherwise,
   please check whether the other data source devices are running.
   To understand which data source is missing, one can check `Latest sources`;

.. note::
    You may not need all the other data sources other than `FastCCD` in your
    experiment. Please feel free to "Ignore" the unwanted sources by ticking
    the checkbox in the scene.

5. Click the **Activate** button to activate the bridge. If `Input updated` and
   `Data received` are increasing, it indicates that the bridge is working.
   Otherwise, try re-instantiating the bridge device to re-establish the
   connection between the processor and the bridge;

.. note::
    If the bridge is still not working, please re-start the device server and
    repeat the step 2-5;

6. ssh to the online cluster **sa3-br-kc-comp-1** (**exflonc13**) and following
   the instructions about EXtra-foam in the Karabo scene.

.. figure:: _images/EXtra-foam_FastCCD_SCS.png

.. _XasTim-SCS:

XasTim
++++++

Toolchain for real-time data analysis and visualization of XAS
(Xray absorption spectroscopy) experiments using TIM (transmission intensity
monitor). In this tool chain, XasTimProcessor_ correlates and processes data
from XGM, digitizer and softmono. The processed data is then sent out via the
Karabo bridge to xas-tim-view_, which visualizes the data in a web browser.

.. _XasTimProcessor: https://git.xfel.eu/gitlab/karaboDevices/xasTimProcessor
.. _xas-tim-view: https://git.xfel.eu/gitlab/dataAnalysis/xasVisualization

**Usage**

1. Open the project *ZMQ_BRIDGE* within the **SCS** topic;

2. Instantiate two devices: `SCS_XAS_TIM/DA/PROC` and `SCS_XAS_TIM/DA/BRIDGE`;

3. Open the following scene in Karabo;

.. figure:: _images/XasTimProcessorScene.png

.. note::
    Please strictly follow the following steps since the data flows from
    upstream to downstream (processor -> bridge -> xas-tim-view -> browser).
    It also applies for troubleshooting!

4. Click the **Start** button in the scene. If `N processed trains` is
increasing, it indicates that the processor is working. Otherwise, please check
whether the `XGM`/`digitizer`/`softmono` devices are running. To understand
which data source is missing, one can check `Latest sources`;

5. Click the **Activate** button to activate the bridge. If `Input updated` and
`Data received` are increasing, it indicates that the bridge is working.
Otherwise, try re-instantiating the bridge device to re-establish the
connection between the processor and the bridge;

.. note::
    If the bridge is still not working, please re-start the device server and repeat the step 2-5;

6. ssh to the online cluster **sa3-br-kc-comp-1** (**exflonc13**) and following
the instructions about `xas-tim-view` in the Karabo scene.

.. figure:: _images/xas-tim-view.png


FXE toolbox
-----------

This section provides the workflow of data analysis specific tools at FXE
instrument

.. _EXtra-foam_FXE:

Correlations in EXtra-foam
++++++++++++++++++++++++++

.. figure:: _images/correlations.png

To optimize the experimental conditions, users need to observe the
dependence of sample/jet position with respect to the Figure of Merit (FOM)
evaluated from the diffraction signal in real time. `EXtra-foam <https://extra-foam.readthedocs.io/en/latest/>`__
provides several ways of evaluating FOM from the images (fast data) obtained
live during the experiment through **KaraboBridge**. As of now JUNGFRAU and LPD
are the two detectors that are commonly used at FXE. **EXtra-foam** provides
functionality to evaluate FOMs from data obtained from these two detectors.


**To observe correlation between FOM and motor positions (any slow data)** one
needs to follow the workflow::

 DataCorrelator (karabo device) -> KaraboBridge (karabo device) -> EXtra-foam (online analysis tool)


HED toolbox
-----------

This section provides the workflow of data analysis specific tools at HED
instrument.

Karabo bridge
+++++++++++++

:ref:`Karabo bridge<karabo-bridge-overview>` is available at the HED instrument
and can be used to stream in realtime any data source available in Karabo during
beamtime. Streamed data sources can be configured by instrument scientists upon
request.

Dioptas
+++++++

`Dioptas <http://www.clemensprescher.com/programs/dioptas>` is distributed
in our software environment available on the :ref:`online cluster<Online
cluster>` and the :ref:`Maxwell cluster<Offline cluster>`::

    module load exfel dioptas