.. raw:: html

    <style> .red {color:#aa0060; font-weight:bold; font-size:16px} </style>

.. role:: red

.. _RunControlChapter:

pnCCD Data Acquisition 
======================

.. _RunControl:

Run Control
-----------

The SQS DAQ Karabo subproject is called :code:`SQS_DAQ_RUN_MGMT`, which is found under the :code:`AMAIN` main project. Click on the little triangle beside the :code:`SQS_DAQ_RUN_MGMT` subproject to expand it. Open the :code:`SQS_RUN_CONTROL_GLOBAL_STATE` scene (:numref:`figure14`). 

.. _figure14:
.. figure:: Figures/Run_Controller.png
   :width: 100 %

   The :code:`SQS_RUN_CONTROL_GLOBAL_STATE` scene with which one can set up the run control. Here, only the data sources related to the pnCCD devices are selected.

To set up the DAQ in order to take a data run with pnCCD:

1. Make sure that detector is in acquiring mode. If so, the :file:`ADC 1` and :file:`ADC 2` states are both in :file:`Acquiring` mode on the :code:`pnCCD_Main` scene (see :numref:`figure12`).

#. If a run is not currently in progress, click on the :file:`Ignore data` button (see :numref:`figure14`). This step is only necessary if you want to change the proposal number and/or the data sources. With this step, the DAQ will go into the :code:`Ignore` state.

#. From the :code:`Data Source Groups` field (see :numref:`figure14`), select, by clicking on the little square on their left hand side, the pnCCD data aggregators, which are:
   
    - :file:`SQS_NQS_PNCCD1M-CAL`
    - :file:`SQS_NQS_PNCCD1M-META` 
    - :file:`SQS_DIGITIZER-UTC1`: This is required only if you need to use Karabo bridge. 
    - The three abovementioned data sources are sufficient for pnCCD. If the experiment requires some other data sources from SQS devices other than pnCCD, make sure their corresponding data sources are also selected. The SQS beamline scientists should be able to assist you in identifying these latter data sources if you are unsure which ones to select.
    
#. Click on the green check mark (Apply All Changes) on the top left corner of the :code:`SQS_RUN_CONTROL_GLOBAL_STATE` scene to apply the changes. At this point, the :file:`Data source` boolean state indicator (resembels a small LED on the bottom middle of :numref:`figure14`) should be green. 

#. Write the desired proposal number in the corresponding field in the :file:`SQS_RUN_CONTROL_GLOBAL_STATE` scene (see :numref:`figure14`) and hit :file:`Enter` (keyboard return). Click on :file:`Retrieve Proposal` button. The :code:`Proposal` boolean state indicator (resembels a small LED on the bottom middle of :numref:`figure14`) should now also be green. 
    
#. Click on the :file:`Push to DAQ` button on the :file:`SQS_RUN_CONTROL_GLOBAL_STATE` scene.  Wait a few seconds. The :code:`Assign` boolean state indicator should now be green. 

#. Click on :file:`Apply configuration` button and wait until the status message bar (on the middle bottom of :numref:`figure14`) confirms that the application was successful. The :code:`Configure` boolean state indicator should now be green. 

#. Click on :file:`Monitor data` button on the :code:`SQS_RUN_CONTROL_GLOBAL_STATE` scene and ensure the :code:`Monitor` boolean state indicator also turns green.

#. At this point, the :file:`Progress` bar should be green and should show 100 percent. if everything has worked like it should, the big boolean state LED indicator called :file:`Gloabal State` should be green.

#. If you want to take a dark run, choose "Dark Run for Gain xxx" (specify which gain setting it is for) as the :file:`Run Type` and choose "No Sample" for the :file:`Sample` field (see :numref:`figure14`). If these are not available, please ask an SQS beamline scientist to make them using the `Meta Data Catalog <https://in.xfel.eu/metadata/>`_ so that the dark runs can be labeled correctly. **Everytime you make a change, click on the keyboard return key to ensure that your change is saved in Karabo.** If the run is not a dark one, choose appropriate names for :file:`Run Type` and :file:`Sample` fields. 

#. Follow the instructions given in :numref:`DarkRuns` to start a dark run.

#. After enough data are saved (for example, 500 trains are enough for a dark run), click on :file:`Stop run` button (see :numref:`figure14`) to stop the run.

Offline Data 
------------

Following the instructions given in the previous section, raw data are saved on disk. 

Accessing the Raw Data
~~~~~~~~~~~~~~~~~~~~~~

To access the raw data, follow these steps:

1. Make sure you have permission to access the proposal under study. If not, ask the SQS beamline scientists to add your username to the desired proposal if possible. 
#. Connect to the Maxwell cluster (you may need to first connect to :code:`bastion.desy.de`):

     .. code-block:: bash

        ssh -XY username@max-exfl

#. Go to the directory where the current raw data are being stored:

     .. code-block:: bash

        cd /gpfs/exfel/exp/SQS/201921/p002430/raw/r0010

#. Here, the syntax is as follows:

   - :file:`SQS` is the instrument.
   - :file:`201921` is the cycle number.
   - :file:`p002430` is the proposal number.
   - :file:`r0010` means the desired run number is 10.

#. If the DAQ is setup correctly and the connection between the DAQ device and the data aggregators is established correctly, for run number :code:`XXX`, there should be a few files in the :file:`r0XXX` directory. Depending on the selected data sources, these files are of three types:

    - Files like :file:`RAW-R0XXX-DA##-S000##.h5`
    - Files like :file:`RAW-R0XXX-PNCCD01-S000##.h5`
    - Files like :file:`RAW-R0XXX-PNCCD02-S000##.h5`

where :file:`DA##` or :file:`PNCCD##` refers to the data aggregators, and :file:`S000##` refers to the file sequence. Roughly, every 500 trains consists of one sequence. Therefore, if the duration of the run corresponds to 1500 trains, you will end up with 3 raw data files per each selected data aggregator. Note that there will be one image frame saved per train. Also, the slow data are saved in the :code:`PNCCD02` data files, while the fast data are saved in the :code:`PNCCD01` data files. The slow data refer to the pnCCD gain, bias voltages (HV values for top and bottom pnCCD modules) and the sensors' temperatures. The fast data are the pnCCD image arrays. 

.. note:: 

     - Only those files whose names contain :file:`PNCCD01` or :file:`PNCCD02` will have the data collected by the pnCCD detector because only these files are generated by the correct pnCCD data aggregators.
   
6. To open a raw data file, which contains the pnCCD fast data, on Maxwell cluster, do the following:

     .. code-block:: bash
        :linenos:

        module load xray
        hdfview3 RAW-R0XXX-PNCCD01-S000##.h5

#. Go to :file:`Open` menu (looks like an open book with a red arrow pointing downwards) and click on the menu button found on the top looking like a box with a black square and a ring inside. Browse to :file:`gpfs` folder and then browse to your desired raw data file.

#. A window opens, which should have the complete path of your desired raw data file in :file:`Recent Files` box. Browse to the following path to have access to the raw image data:

     .. code-block:: bash

        /INSTRUMENT/SQS_NQS_PNCCD1MP/CAL/PNCCD_FMT-0:output/data/image/ 

#. To ensure data is being properly written on the disk, make sure that the :file:`image` is non empty and non zero. To do this, one has to display the data and ensure the size of the data array is :math:`x \times 1024 \times 1024`, where :math:`x` is the number of frames and :math:`0 < x \leq 500`. To display the raw data:

    - Double click on the :file:`image` to open the raw data image as an array in a spreadsheet.
    - To visualize the data as 2D images (see :numref:`HDFView` and :numref:`DataView`), right click on the :file:`image` in the above mentioned path and click on :file:`Open As`. A window pops up. Select :file:`Image` as display type. Select the desired color scale from the :file:`Select palette` drop-down menu. If desired, choose a valid range by setting the desired minimum and maximum ADU values. Set :file:`Height` as dimension 1 (1024 pixels) and :file:`Width` as dimension 2 (1024 pixels). :file:`Depth` would be dimension 0 which is the number of available frames. Click on :file:`Reset` to set the :file:`Depth-End` to the maximum available number of frames. Now, you can also change the frame to be displayed (default is frame 0). Finally, press :file:`OK` to display the image. Once the image is shown, one can choose to display various frames by using the forward and backward arrows.
    
.. _HDFView:
.. figure:: Figures/hdfview3.png
   :width: 75 %

   Setting up :code:`hdfview3` to display the raw image data.

.. _DataView:
.. figure:: Figures/RawData.png
   :width: 75 %

   The raw image data using the :code:`hdfview3` tool. The image that is shown here is only part of the actual image due to the limitation of the size of my remote desktop display. What is plotted is sequence 3 of run 76 from proposal :file:`p002720` from cycle :file:`202022`. The displayed image is frame 49 out of 500 available images and shows the scattered X-rays from a membrane with a lot of small holes.