Welcome to DSSC’s documentation!¶
Revisions¶
| Version | Date | Description | Authors |
| 2.0 | 08/08/2021 | Updated and adaprot for SQS | David Lomidze |
| 1.0 | 09/08/2019 | Updated with e.g., automatic powering | David Lomidze, Monica Turcato |
| 0.5 | 15/07/2019 | First version (preliminary) | David Lomidze, Monica Turcato |
Abstract¶
This is the instruction manual for the operation of the DSSC detector. Only trained personnel are allowed to operate this detector. Training is given by the personnel of the Detector Operations Group. In case you need training, please contact david.lomidze@xfel.eu. Trained personnel need to sign an appropriate list provided by David.
The latest approved version of this document can always be found on https://rtd.xfel.eu/.
PREFACE
The DSSC detector has been deployed in the SCS hutch since May of 2019. The detector has to be operated in vacuum and at low temperatures (-20 degC nominal temperature). Although an interlock system is implemented, the detector must be handled with care and can be operated only by the trained personnel. The trainings are provided by the detector experts. At the end of the training, a form has to be signed to certify that the training has been attended. For information about the trainings, please contact the Detector Operations Group of the European XFEL.
| ASIC | Application-Specific Integrated Circuit |
| DAQ | Data AcQuisition |
| FPGA | Field Programmable Gate Array |
| GUI | Graphical User Interface |
| HV | High Voltage |
| Mpix | Mega Pixel |
| MPOD | multi-channel low & high voltage computer controlled power supply |
| PPT | Patch Panel Transceiver |
| PLC | Programmable Logic Controller |
| SIB | Safety Interlock Board |
| SCS | Spectroscopy and Coherent Scattering |
| UHV | Ultra High Vacuum |
Vacuum and cooling¶
The DSSC detector is operated in vacuum and at a temperature of -20 degC. This section assumes that the detector is already pumped down, as the latter operation must be done by an expert. The procedure to bring the detector to -20 degC is described below.
Checking if the detector is in vacuum and at room temperature¶
Open a Karabo GUI and connect to the project SCS(SQS)_DSSC. There are other subprojects inside.
The first thing to check is if the detector is in vacuum.
To check the detector vacuum level, open the DSSC_Main scene (see Fig. 3.1) under the SCS(SQS)_DSSC project. The vacuum scene shows the status of the pressure gauges PCC16 and PCC17. PCC16 is used to measure the pressure in the pipe placed between the DSSC and the sample chamber, while PPC17 measures the pressure inside the DSSC chamber. Since a thin filter is placed between the two, a pressure difference of roughly one order to magnitude is expected at room temperature. The usual vacuum level measured by PCC17 should be lower than 9e-06 mbar (units in the view are in mbar). If this is not the case, please contact DET OCD.
If the vacuum level in the detector is lower than 9e-06 mbar, the detector can be cooled down. Before doing that, please check the temperature of the chiller bath in the DSSC_Main scene. In normal conditions, with the detector warmed up, this temperature shall be around 20 degC, but it can go up to 35 degC in case the chiller is off for more than a few hours.
DSSC_Main scene, showing the two UHV pressure gauges and the cooling bath temperature.
Cooling down the detector¶
If the vacuum level in the detector is better than 9e-06 mbar, the detector can be cooled down.
On the DSSC_Main scene find Safety Interlock Boards panel and click on See more, wthis will open the scene SIB (see Fig. 3.2) and check that ‘SIB State’ indicator is equal to 5 for all the Safety Interlock Boards (SIBs). An alternative the :code:`SIB` scene can be accessed from the :code:`SCS_CDIDET_DSSC` or from :code:`SQS_NQS_DSSC` subprojects
Warning
If this is not the case, please contact DOC 98089.
Normally, if the chiller is OFF, the valves on the coolant lines close to the chiller are closed. They are opened automatically when the chiller goes on. Please also check the valves in the hutch, on the wall separating it from the control room, they must also be open. If they are closed, they must be opened by hand.
Check that the chiller device is not in ERROR state in the vacuum scene. The status of the device there should be green. If this is the case, set the bath temperature (Final chiller T setpoint) to -20 degC. The temperature in the scene shall start going down. With a ramp rate of 1 degC/min, roughly 40 minutes will be needed to reach -20 degC.
Note
The temperature of the bath is the temperature of the coolant inside the chiller. Therefore, it takes some time until the cooling blocks reach the same temperature. The temperature of the cooling blocks inside vacuum can be checked in the SIB scene under the same project.
SIB scene.
Powering ON the DSSC detector and sending data to DAQ¶
The DSSC detector can be powered ON if in vacuum and cold. The temperature of the cooling blocks of the DSSC detector can be seen in the SIB scene. In the same scene, at the top, a set of indicators turn from blue to green. The DSSC detector can be switched ON only if all the four Crate En. indicators are green and the status is Operation. In Fig. 4.1, a detail of the SIB scene is shown, where the SIB signals are visible. In the figure, the status is ‘Ready’ (SIB state 5), meaning that the DSSC detector is ready to be cooled down (vacuum level is OK). When the temperature of the cooling blocks goes below -10 degC, the Crate En. signal becomes also high and the status goes to Operation (SIB state 6). In this configuration the detector is ready to be powered up.
Details of the SIB state in the SIB scene.
DSSC power status scene.
The DSSC detector is powered by four MPOD power supplies (one for each of the quadrants) each having low voltage and high voltage channels. Different scenes are available to monitor the powering status of the detector, they are shown in the pictures. Fig. 4.2 shows the status of the low voltage system of DSSC once powered on. The status of the four power supplies (MPOD_1 to MPOD_4) is shown. The channels are grouped corresponding to the ordering of the powering on procedure (channels turn ON from top to bottom at different stages). The names of the channels refer to the different tags used also in the power procedure. In the picture, all the quadrants low voltages are ON and the ASIC are correctly programmed.
Fig. 4.3 shows the status of the high voltage system of DSSC once powered on. The status of the four power supplies (MPOD_1 to MPOD_4) is shown, two columns per MPOD are needed.
DSSC high voltage channels scene.
The MPOD power supplies feature a web interface which updates faster than the Karabo GUI and it also allows seeing the channel settings. It can be opened by typing the MPOD IP in a browser window (on the control network). The IP addresses of the MPODs are the following:
| Quadrant | SCS DSSC MPODs | SQS DSSC MPODs |
|---|---|---|
| Q1 | 192.168.177.91 | 192.168.158.133 |
| Q2 | 192.168.177.92 | 192.168.158.134 |
| Q3 | 192.168.177.93 | 192.168.158.135 |
| Q4 | 192.168.177.94 | 192.168.158.136 |
DSSC setup for data taking¶
The procedure to switch ON the detector proceeds in three general steps which are explained in detail later on in this chapter. These steps are:
- Switching ON the in-vacuum electronics up to the ASICs (see Section 4.2).
- Configuring the detector electronics and verifying this by sending data to the XFEL DAQ (see Section 4.3).
- Switching ON the rest of the in-vacuum channels (source voltages and sensor voltages, the latter are high voltage channels). (see Section 4.4).
All that is required to set up the DSSC detector for data taking is available in the DSSC_Main scene:
SCS(SQS)_DSSC \(\rightarrow\) DSSC_Main
The powering of the detector is handled by the Automatic Power Panel in the DSSC_Main scene, the configuration by the PPT/ASICS Configuration Panel in the same scene.
Switching ON the in-vacuum electronics up to ASICs¶
Open power system monitgorng scenes from the DSSC_Main scene, go to Automatic Power Procedure panel and click on LV Channels Staus and then HV Chanels Status links, that will open LV and HV voltages monitoring scenes.
OR
In navigate to the DSSC powering Karabo subproject: SCS(SQS)_DSSC \(\rightarrow\) Subprojects \(\rightarrow\) DSSC_POWER and open the scenes:
DSSC_Power_Status(monitoring low voltages and currents)DSSC_hv_status(monitoring high voltages and currents)
The quadrants are switched on using the Automatic Power Panel in the DSSC_Main scene. It must be first checked that the channels AON in the DSSC_Power_Status scene are all ON (see Fig. 4.4).
AON channels (auto-ON) in the DSSC_Power_Status scene.
The auto_power scene used to power up the DSSC detector.
The detector needs to be first powered up to the ASICs. To do that, press the ASICS ON button in the auto_power scene (Fig. 4.5). Wait until all the channels in the DSSC_Power_Status up to IOBA become green. SOURCE channels shall still be blue (OFF).
When all the channels up to IOBA are green, the detector is powered up to the ASICs and the ASICs need to be programmed at this stage.
Configuring the detector ASICs¶
In this state, having a look at the log file of the PPT FPGA can be very useful. However, this requires access to the DSSC control server, exflcon145 for SCS and exflcon202 for SQS, which are NOT accessible to everybody with an XFEL account. If you do not know if you have access to that server, you probably do not.
- To open the interesting log file, the procedure is the following:
- Log in to
exflcon145orexflcon202asxctrl. source karabo/activatekarabo-xterm cppserver_scs_dssc_det
- Log in to
An xterm will appear which will log the changes in the FPGA.
The PPT/ASICS Configuration Panel in the DSSC_Main panel.
To go on programming the ASICs, with only the ASICs ON, move to the PPT/ASICS Configuration Panel in the DSSC_Main panel (see Fig. 4.6).
For each quadrant, the proper configuration files have to be loaded. The configuration files are different for each quadrant, while the settings for gain are the same. A list of config files is available in a drop-down menu under each SCS_CDIDET_DSSC(SQS_NQS_DSSC)/FPGSA/FPGA_Qi device.
To load the config files, the devices must be in shutdown state. Make sure that the proper configurations are uploaded for all quadrants! Check the config file name in the FPGA_Qi device under ‘Full Config File’. Check that the ‘Quadrant ID’ is correct for all the quadrants.
When the config files are loaded, instantiate the FPGA_Qi devices by right clicking on the cppServers/scs_dssc_det (which you can see also above) and selecting ‘Instantiate all devices’. Wait until they all become green.
Then press Start PPT devices in the PPT/ASICS Configuration Panel.
In this phase, the currents of the IOB channels (500, 504, 600, 604) shall go from ~300 mA to 850 mA. This means that the IOB are being programmed. This step can take several minutes, as the IOBs are programmed one after the other. Monitor the currents in the DSSC power system status scene. The currents of the RB channels (501, 505, 601, 605) will go to zero. If the IOB channels do not go to 850 mA, something has gone wrong. Press the ‘Start PPT devices’ button again.
When the ‘Start PPT devices button becomes active again and the IOB currents are fine, press Init PPT devices in the PPT/ASICS Configuration Panel.
The currents of the RB channels will go between 400 and 600 mA. The system is programmed when the ‘Init PPT devices’ button becomes active again. If any of the RB channels do not show current, repeat the procedure. Press Start Data Sending in the PPT/ASICS Configuration Panel, and DSSC will start sending data. To monitor the data sent, open the PREVIEW scene in the DSSC_ONLINE_CAL project. In order for the scene to show the DSSC data, the DAQ shall be configured properly (see Section 5). The currents of channels 500, 504, 600 and 604 should go up to to ~900 mA. If the currents show these values, the system is properly programmed. If this is not the case, repeat once again ‘Init PPT devices’ and then ‘Start Data Sending’.
DSSC online preview (PREVIEW scene in DSSC_ONLINE_CAL).
At this point, many ASICs can still be flickering, namely, they will rapidly change color and pattern in the PREVIEW. This means that they are not sending data properly. To repair this, press the button ‘ASICs/Reset on PPT’ in the DSSC_CONTROL scene. You might need to repeat this step more than once.
It might be that single ASICs are not recovered with this procedure. In that case, select the single ASIC on the proper ASICreset scene by selecting the proper module and ASIC number according to the map (see Fig. 4.8). By pressing the appropriate ASIC, the ASIC is reprogrammed and the flickering should disappear, however, it might be that this needs to be done more than once. If needed, press again ‘ASICs/Reset on PPT’.
Map of module and ASICs used for reset of ASICs.
Positioning of quadrants, modules and ASIC in the DSSC 1 Mpixel detector.
Powering up the source voltages and the high voltage channels¶
Go back to the Automatic Power Panel scene. Switch on the source voltages by pushing the SOURCE On button. All the channels in the DSSC_Power_Status shall now go green. The online preview also changes.
To be overcautious, check from the MPOD web interface (see Table 4.1) channels 100-103, 112-115, 212-215, 200-203. All the voltages must be set to 20V. If this is not the case, call DET OCD.
Switch ON the high voltage by pushing the HV On button. All the channels in the DSSC_HV_Status shall go green, but the procedure ends only when the auto_power scene reports that DSSC is ON and the HV_Off button becomes active. Indeed, channels 100-103, 112-115, 212-215, 200-203 are switched ON to -20 V and will eventually go to -25 V in the final step.
Warning
DSSC is ON when the GUI indicates so, and when the HV Off button is active again. Error messages are reported in the GUI, please pay attention as this is a critical step for the detector.
DAQ configuration and data taking¶
In the main GUI open the project SCS(SQS)_DAQ_RUN_MGMT. In the project, open the following scene:
- SQS run controller:
SQS_RUN_CONTROL_GLOBAL_STATE- SQS run controller:
SCS_RUN_CONTROL_GLOBAL_STATE
- In
SCS(SQS)_RUN_CONTROL(see Fig. 5.1), Choose Proposal number: 900210. This is the DSSC commissioning proposal.
Press on ‘Retrieve proposal’.
- Include the correct data sources, in agreement with the beamline personnel. are in addition following slow data source groupes have to be added to the DAQ:
DSSC1M-1 XTDF`- The fast data group for the 16 ladders of the DSSC.DSSC_CFG1- slow data group for FPGA/PPT_Q1,DSSC_CFG2- slow data source for FPGA/PPT_Q2,DSSC_CFG3- slow data group for FPGA/PPT_Q3,DSSC_CFG4- slow data group for FPGA/PPT_Q4,DSSC_INF- General configuration slow data.
Be aware that you have to acknowledge the choice with pressing enter when selecting and deselecting.
Press ‘Push to DAQ’.
Press ‘Apply Configuration’.
Press ‘Monitor data’ and check
SCS(SQS)_DAQ_RUN_CONTROLscene.
Run controller scene SQS example, the similar scene has SCS instrument.
The online preview works only if ‘Monitor Data’ is active and the proper data sources are included.
- To start an acquisition (writing to disk), one can proceed in two different ways:
- Press ‘Start Run’ in the
SCS(SQS)_RUN_CONTROLscene, and then ‘Stop run’ when you want the run to be over. In this method, one cannot record a selected number of frames. - Use the
DSSC_Mainscene. There atPPT/ASICS Configuration Panel(see Fig. 5.2), one can set the number of trains to be sent (number of iterations) as well as the number of pulses per train (frames to be sent). The ‘Preburst Vetos’ parameter shall not be changed by non-experts. - When the parameters are set, press
Run burst acquisitionif you want exactly the set number of trains to be recorded. At the end of the run, press ‘Start data sending’, the detector will start sending data continuously again.
- Press ‘Start Run’ in the
The runs can be migrated to the Maxwell cluster logging into the myMDC system. In a web browser type https://in.xfel.eu/metadata/ and log in with your credentials (see Fig. 5.3). Once the proposal has been selected, you can see the run list, but only managers can migrate runs to Maxwell. If you are not a manager, ask beamline scientists to see if they can make you a manager.
DSSC PPT and ASICs configuration pannel. The panel also is used to configure DSSC # of frame per train.
Taking dark data runs¶
At the beginning of the data taking, when the detector is properly configured and the ASICs correctly programmed as much as possible, a dark run has to be taken. Proceed as explained above. 20 trains are sufficient.
When the run is taken, in order to enable pedestal subtraction in the corrected online preview, migrate it to Maxwell and proceed as follows:
- Afte run is migrated click on the “Calibration Constants on under proposal menu on metadata catalog”
- In
Detectorfield selectSCS(SQS)_DSSC1M1-1- mark all
Detector unitsi.e. all 16 modules of DSSC.Operation ModeselectStandard Operation- from
Run numbersdwop down menu select just migrated run mumber
- click on
Request- In the sable bellow
Requestbutton you will see status of run proccessing. Refresh the broser to monitor Status. When status changes toFinished, it meands you are ready to load new constants forCorrrected online preview
XFEL meta data catalog, recently it has feature to submit dark run and monitor dark run proccessing status.
- This is a old method which is still valid and could be used if you to proccess data manually:
Login to
max-exfl.Type
module load anaconda/3- Run the script which evaluates dark image subtraction
python /gpfs/exfel/data/scratch/xcal/request_darks.py --instrument SCS --cycle 201931 --proposal 900095 --run xxx [xxx: dark run number]
Wait until the script is finished (it will return
DONEon the script, but this can take on the order of 10 minutes).
- Loading new constants
- Open the
MANAGERscene in theDSSC_ONLINEsub-project. - Click on
Reset. - Click on
Initand wait until all the files are updated (OFFSET columns in theMANAGERscene will change from green to blue). - Click on
Resetagain. - Now offset corrections are applied to the online preview.
- Open the
If the results are not good (most likely due to some ASICs still flickering), repeat the procedure.
DSSC Motion System¶
The DSSC Motion Mechanics¶
In order to operate the quadrants, it has to be understood, how the axes of EACH quadrant are defined, which is shown in the Fig. 6.1
coordinate systems for all axes the origin of each coordinate system ( = machine zero) is at the outer limit switches
Understanding Opening and Closing the aperture¶
Every positional change from one position into another is done using two opening and two closing sequences, as shown in Fig. 6.2.
Allowed motions are shown with green arrows and prohibited motions with red arrows. The black clockwise and counterclockwise arrows indicate the overall motion direction of all quadrants (but NOT the order of their movement in case of the closing motion).
The order of the two opening sequences is chosen such that:
- The first opening motion (with probably a tight gap <<1mm) is only of 1mm length, to minimize the risk of a quadrant collision (and to avoid the collision switch snapping of its counterpart)
- Consecutive motions are always perpendicular to the each other (so the bigger gap size ≥1mm between the last and the current quadrant has already been established and the current quadrant is moved “along” this gap)
- The last motion finishes the first motion (which was only 1mm and now goes to target)
In the same way the two closing sequences are chosen such that:
- The first closing motion does not fully go to target but leaves a gap that is 1mm wider than the final gap
- Consecutive motions are always perpendicular to the each other and the current quadrant moves towards the quadrants that were just moved into their final position (and thus are not moved anymore in the sequence)
- The last closing motion closes the gap created by the first motion of the sequence
By this we make sure that we close gaps to quadrants only if they are not moved anymore. The only movement along a tight gap (<<1mm) is 1mm long.
Opening and closing motions, see Fig. 6.1 for quadrant coordinate systems
Now we have four sequences in total. If we execute two of these sequences one after another we can get any hole size and gap size we want, as shown in some examples in the following sections.
Operation Instruction¶
Powering and restartting encoders and motors¶
- Use the following combination of commands to prepare the encoders:
- Power ON Encoders
- Reset Encoder
- All encoders should now go to green state. If one or more encoders remain in redstate:
- Check the encoder cable connections
- Verify that non of the encoders are giving error (in case of ERROR: a LED will be flashing in RED on the encoder)
- Read the log file (see Section 6.2.2)
Note
If all cables are connected and RED light stil remains do not move any quadrant and contact detector respinsible person!
The interlock conditions of the motors should all be removed by now. If not, read the section about interlock conditions in the troubleshooting chapter.
- As a last step, if no interlocks are active:
- Reset motors
- Power OFF motors (should be OFF anyway, but just in case)
All motors should be now in greenstate and OFF.
Opening motion system log file¶
The DSSC motion system middle layer device creates a log file. In the log file each step of motion is written in. It is highly recomended that log file is displayed using one of mossible ways bellow and actively monitored motion steps. In addition log file information is needed to be used to undastand/verify current postion of quandants.
- Method 1: Monitoring log in a bash terminal
- connect to the DSSC Karabo control server as xctrl and do:
# cd karabo/var/log/middlelayerserver_sqs_dssc_mov/ # tail -25f current
this will allow you to have live monitoring of the log file.
- Method 2: Monitoring log using firefox from contol netwok
- simply open firefox
- cottect to URL:
exflcon202:8080if it does not work - connect to
192.168.158.55:8080 - In the brower you will see all log files of DSSC, click on
liveamong themiddlelayerserver_sqs_dssc_moventry.
Method 3: Monitoring log using firefox from office netwok
Open an ssh tunnel this way: .. code-block:
# ssh -L 8321:localhost:28222 yourusername@bastion.desy.de -t ssh -L 28222:exflcon202:8080 exflgatewayRemove backslesh if copying the command!
Open a firefox and then connect to
localhost:8321Click on
liveamong themiddlelayerserver_sqs_dssc_moventry.
Checking positional state of the quadrants¶
Before we can move the quadrants, we have to make sure, that they are in a DEFINED position. With respect to the positions file, this means that the quadrants have to be in one of the 7 positions, defined in the positions file (see Figure 1). In order to do so, proceed as following:
- Check, if the positions file is loaded correctly:
- Make sure that encoderConfig.Read/writeModeisset to
read_from_file - Click the button
read/write quadrant positions - See Motion system log update on the Karabo control server (). You should see the positions as shown in
- Make sure that encoderConfig.Read/writeModeisset to
Check if all axes are in a defined state:
- Click the button show encoder values
- see log file in the rerminal window
- You should seea Boolean matrix with one line of all True values
- You should see a statement i.e. “Current Position is Position 7”
If the last 2 points are not shown as described (i.e. if the statement displayed is “Current Position is Position -1”), not all axes are in a defined position.
Note
if you see Current Position is Position -1 contact DSSC DOC or DSSC responsible person.
If all went well, you are now readyfor quadrant motion.
In theory, quadrant motion is straight forward. It consists solely of the following 3 steps:
- Choose the desired position, go to the
DSSC_Motionscene and select one of the 7 posiions- Click on Go To Position absolute
- Monitor the process in the log terminal
The motion middle layer will now move all 4 quadrants, performing 10 motions:-4 quadrants, 2 directional motions per quadrant plus 2 “gap” movements
Unfortunately, due to the close proximity of the quadrants to the limit switches, it might occur that one of the quadrants hits one of the limit switches and system will be interlocked.
Warning
If motion system gets interlocked, do not try to solve issue by yourself. Contact DOC or DSSC responsible and report what happened, it is very important to know last execudet commands.
Powering OFF the DSSC detector¶
Go to the auto_power scene (Fig. 4.5 and/or Fig. 7.1). Press the All Off button. Pay attention that the procedure is executed correctly and that no error appears on the log window. The procedure is finished when the ASICs ON button becomes active again and on the log window it is written: ‘DSSC is off.’ All the channels in the DSSC_HV_status window shall be OFF (blue) and in the DSSC_Power_Status scene only the AON channels (603, 704, 705, 706) shall be on.
DSSC automatic power procedure scene.
At this point, all the in-vacuum electronics of DSSC are OFF and the detector does not need to be cooled any more.
Leave the AON channels ON so that the PPT and the SIB stay ON.
Troubleshooting¶
This chapter presents a list of the most common problems encountered in running the DSSC detector.
What to do when¶
Here is a list of what to check during detector operation to ensure everything is working fine:
Detector status: up and running (sending data)
| Scene | Issue | Action |
DSSC_Power_Status |
Channel red (ERROR) | Call DET OCD immediately. Switch OFF power with All Off button in the MAIN scene |
DSSC_Power_Status |
Channel yellow (UNKNOWN) | Call DET OCD immediately. MPOD has lost connection with Karabo and can no longer be controlled This is however not an emergency situation, but the detector cannot be switched OFF as control is disconnected. |
DSSC_HV_Status |
Channel red (ERROR) | Call DET OCD immediately. Switch OFF power with All Off button in the MAIN scene |
DSSC_HV_Status |
Channel yellow (UNKNOWN) | Call DET OCD immediately. MPOD has lost connection with Karabo and cannot be controlled anymore. This is however not an emergency situation, but the detector cannot be switched OFF as control is disconnected. |
SIB |
Crate En. lamps
turn from green to blue
or red during operation |
Call DET OCD immediately. If you cannot reach an expert, switch OFF power with All OFF button in the MAIN scene. |
SIB |
ASIC_T status flickering red (value 2) in one or more quadrants | Flickering from time to time is normal if the flickering channel goes back to normal at some point |
SIB |
ASIC_T status stably red (value 2) in one or more quadrants | Open the SIB_ASICTempChecks scene and check
the values. During operation at -20 degC, they should
be at -12 degC (within 2 degC). The threshold is at
-10 degC. If some value is obviously wrong (e.g.,
-50 degC) do not do anything but note down the ASIC
and quadrant number in elog. If some value is between
-10 degC and 10 degC, call DET OCD during the day,
and note down and monitor during the night. If the
value seems to increase, call DET OCD. If some value
is above zero degC, call DET OCD. |
SIB |
ASIC_T status stably red (value 3) in one or more quadrants | Check that the quadrant where the value is 3 is sending data. If data are not sent, this value is normally 3. It should go back to zero when data are sent. |
SIB |
IOB T Status red (status 3) in one or more quadrants | Check that ASICs are ON in the DSSC MAIN panel. If ASICS are OFF, a status 3 there is normal. If ASICS are ON, call DET OCD. |
SIB |
IOB T Status red (status 2) in one or more quadrants | Call DET OCD immediately. Switch OFF power with ALL OFF button in the MAIN scene |
List of NaN in the SIB_ASICTempScene:
For the following ASICs, it is normal to have a NaN readout in the SIB_ASICTempScene:
Q1, Q2 and Q3: No ASIC should have a NaN readout.
Q4: ASIC 9 and 38
Online preview not updating¶
If the online preview is not updating anymore, there might be different problems.
Check that the detector is correctly sending data. Go to the individual SCS_CDIDET_DSSC/FPGA/PPT_Qi devices and in the configuration editor, look for ‘SFP output rate’. This should be different from zero, normally a number like 4197 Mbit/s, but it also depends on the VETO configuration. If the number is zero for one of the quadrants, then the quadrant is not sending data. Check the currents, in case press on Stop acquisition and then on Init system.
You can also check the network traffic to the DAQ nodes via the remcom interface. Type https://remcom.desy.de/ on a network browser, log in as SCSDAQ or as SQSDAQ. Click on ‘SCSDAQ(SQSDAQ)’ and then on ‘DAQ data aggregators network traffic’. Select the required data sources (all the scs(sqs)-br-sys-daq-XX, where XX goes from 00 to 15). Then click on ‘Run Job Now’ on the top right side of the screen. The system will give you back the amount of (possible) failures.
Check that DSSC-XTDF is included in the DAQ data sources in the SCS(SQS) run control. If this is not the case, the DSSC data are not sent to the DAQ and therefore cannot be displayed in the online preview.
- If all this looks OK, you might need to restart the calibration pipeline. The procedure is as follows:
- Stop acquisition on all the scences ‘Init_Qi’.
- On the scene
Subprojects\(\rightarrow\)DSSC_ONLINE_CAL\(\rightarrow\)MANAGER, push the button ‘Restart Servers’, and when the process is finished, push ‘Restart Pipeline’. - Check on the
PREVIEWscene that all the modules are displaying data correctly.
Frames getting IDs up to 810¶
The problem was solved switching on and off the quadrants. A soft reset of the FPGA on the PPT did not help.
Restarting the DAQ¶
SCS(SQS)_DAQ_DSSC: in case the DAQ seems not working, restart the servers under this device (scs(sqs)_daq_0 … scs(sqs)_daq_15). After that, one should restart also online pipeline preview. Follow the procedure below.
Restart DAQ servers (e.g. online preview not updating some module, after restarting online preview)
- Stop sending data from the detector (very important). Use the
Init_Qiscenes.- From https://remcom.desy.de/ , go to reset DAQ.
- Select all nodes (xtdf data nodes 0-15).
- Select NONE of slow data nodes.
- Then, press run job now \(\rightarrow\) Aggregators are then killed.
- Then, go to subproject
SCS(SQS)_DAQ_MACRO.- Choose macro
SCS(SQS)_AUTO_INSTANTIATE.- Run the macro by pressing the green button, macro comes out in the line below.
- In the macro, press start.
- Run controller servers and database devices need to be reinstaintiated by hand (
pythonServers/scs(sqs)_daq_rmdc_1,cppServers/scs(sqs)-daq_rc_1,cppServers/scs(sqs)_daq_conf_1).- At the end, restart the online preview pipeline (scene
ManagerinDSSC_ONLINE_CAL, see manual above).- Select proper proposal and data sources
DSSC configuration files for SCS instrument¶
The tables below list the DSSC configuration files deployed at the SCS instrument.
2.25 MHz rate¶
4.5 MHz repetition rate, 50 ns integration time¶
| File name | Energy | Gain | Current double | Dynamic range | ||
|---|---|---|---|---|---|---|
| (GenConf_*_trimmed.conf) | (eV) | (ph/bin) | (ph) | |||
| TG1.8832_nG24 | 531 | 1 | 256 | |||
| TG0.62775_nG25 | 531 | 3 | 768 | |||
| TG0.18832_nG179 | 531 | 10 | 2560 | |||
| TG4.2437_nG12 | 707 | 0.33 | 85 | |||
| TG2.8289_nG23 | 707 | 0.5 | 128 | |||
| TG1.4144_nG23 | 707 | 1 | 256 | |||
| TG0.70721_nG100 | 707 | 2 | 512 | |||
| TG0.35361_nG178 | 707 | 4 | 1024 | |||
| MixedGain | 707 | [1] | ||||
| TG3.8564_nG12 | 778 | 0.33 | 85 | |||
| TG2.5707_nG13 | 778 | 0.5 | 128 | |||
| TG1.2853_nG34 | 778 | 1 | 256 | |||
| TG0.64267_nG100 | 778 | 2 | 512 | |||
| TG0.32134_nG178 | 778 | 4 | 1024 | |||
| TG2.3447_nG13 | 853 | 0.5 | 128 | |||
| TG1.1723_nG13 | 853 | 1 | 256 | |||
| TG0.58617_nG25 | 853 | 2 | 512 | |||
| TG0.39078_nG112 | 853 | 3 | 768 | |||
| TG0.11723_nG333 | 853 | 10 | 2560 | |||
| TG3.2362_nG23_trimm_f4.5_intgr50 | 927 | 0.33 | 0 | 85 | ||
| TG1.0787_nG35_trimm_f4.5_intgr50 | 927 | 1 | 1 | 128 | ||
| TG0.3596_nG92_trimm_f4.5_intgr50 | 927 | 3 | 1 | 768 | ||
| TG0.1079_nG401_trimm_f4.5_intgr50 | 927 | 10 | 1 | 2560 | ||
| TG2.4218_nG1 | 1240 | 0.33 | 85 | |||
| TG1.6129_nG12 | 1240 | 0.5 | 128 | |||
| TG0.80645_nG35 | 1240 | 1 | 256 | |||
| TG0.26882_nG92 | 1240 | 3 | 651 (ADU 217) [2] | |||
| TG0.080645_nG421 | 1240 | 10 | 1960 (ADU 196) [2] | |||
| TG1_nG24 | 2000 | 0.5 | 128 | |||
| TG1.5015_nG23 | 2000 | 0.33 | 85 | |||
| TG0.5_nG101 | 2000 | 1 | 256 | |||
| TG0.47125_nG101 | 2122 | 1 | 256 | |||
| TG0.94251_nG24 | 2122 | 0.5 | 128 | |||
| TG1.4152_nG23 | 2122 | 0.33 | 85 | |||
| TG1.2012_nG13 | 2500 | 0.33 | 85 | |||
| TG0.8_nG35 | 2500 | 0.5 | 128 | |||
| TG0.4_nG112 | 2500 | 1 | 256 | |||
| TG1.1354_nG13 | 2645 | 0.33 | 85 | |||
| TG0.75614_nG35 | 2645 | 0.5 | 128 | |||
| TG0.37807_nG112 | 2645 | 1 | 256 | |||
| TG1.0725_nG13 | 2800 | 0.33 | 85 | |||
| TG0.71429_nG14 | 2800 | 0.5 | 128 | |||
| TG0.35714_nG91 | 2800 | 1 | 256 | |||
| TG1.0088_nG24 | 3004 | 0.33 | 85 | |||
| TG0.66578_nG46 | 3004 | 0.5 | 128 | |||
| TG0.33289_nG123 | 3004 | 1 | 256 |
| [1] | Central modules Q1M1, Q2M4, Q3M1, Q4M4 have a gain of 4 photons/bin (1024 photons of input range). All the other modules have a gain of 1 photon/bin (256 photons dynamic range). These files were generated manually and are not yet verified. |
| [2] | (1, 2) Linear up to indicated value, after that non-linearity/saturation effects are expected. |
DSSC configuration files for the SQS instrument¶
The table bellow summarizes DSSC configuration files deployed at the SQS instrument
| Energy (keV) | Sampling Frequency (MHz) | Gain (ADU/ph) | Dinamic range (ph) | File name | CD,Int.(ns) | Time seq. | |
|---|---|---|---|---|---|---|---|
| 3 | 4.5 | 7 | 37 | GenConf_TG2.3333_nG24_trimm_f4.5_intgr50_OldSeq | 0 | 50 | OLD |
| 3 | 4.5 | 7 | 37 | GenConf_TG2.3333_nG24_trimm_f4.5_intgr50 | 0 | 50 | NEW2021 |
| 3 | 2.25 | 7 | 37 | GenConf_TG2.3333_nG13_trimm_f2.25_intgr50 | 0 | 50 | NEW2021 |
| 3 | 2.25 | 7 | 37 | GenConf_TG2.3333_nG26_trimm_f2.25_intgr100 | 0 | 100 | NEW2021 |
| 3 | 2.25 | 7 | 37 | GenConf_TG2.3333_nG24_trimm_f2.25_intgr100 | 1 | 100 | NEW2021 |
| 3 | 2.25 | 14 | 18 | GenConf_TG4.6667_nG13_trimm_f2.25_intgr100 | 0 | 100 | NEW2021 |
| 3 | 2.25 | 7 | 37 | GenConf_TG2.3333_nG18_trimm_f2.25_intgr150 | 0 | 150 | NEW2021 |
| 3 | 2.25 | 7 | 37 | GenConf_TG2.3333_nG25_trimm_f2.25_intgr150 | 1 | 150 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG13_trimm_f1.125_intgr50 | 0 | 50 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG26_trimm_f1.125_intgr100 | 0 | 100 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG24_trimm_f1.125_intgr100 | 1 | 100 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG18_trimm_f1.125_intgr150 | 0 | 150 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG25_trimm_f1.125_intgr150 | 1 | 150 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG19_trimm_f1.125_intgr200 | 0 | 200 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG26_trimm_f1.125_intgr200 | 1 | 200 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG32_trimm_f1.125_intgr300 | 0 | 300 | NEW2021 |
| 3 | 1.125 | 7 | 37 | GenConf_TG2.3333_nG18_trimm_f1.125_intgr300 | 1 | 300 | NEW2021 |
| 1.2 | 4.5 | 5 | 51 | GenConf_TG4.1667_nG1_trimm_f4.5_intgr50_OldSeq | 0 | 50 | OLD |
| 1.2 | 4.5 | 0.25 | 1024 | GenConf_TG0.2083_nG190_trimm_f4.5_intgr50_OldSeq | 1 | 50 | OLD |
| 1.7 | 4.5 | 3 | 85 | GenConf_TG1.7647_nG12_trimm_f4.5_intgr50_OldSeq | 1 | 50 | OLD |
| 1.7 | 4.5 | 0.25 | 1024 | GenConf_TG0.1471_nG344_trimm_f4.5_intgr50_OldSeq | 1 | 50 | OLD |
| 2 | 4.5 | 3 | 85 | GenConf_TG1.5_nG23_trimm_f4.5_intgr50_OldSeq | 1 | 50 | OLD |
| 2 | 4.5 | 0.25 | 1024 | GenConf_TG0.125_nG334_trimm_f4.5_intgr50_OldSeq | 1 | 50 | OLD |
DSSC Configuraiton Files Generation¶
Environment Setup¶
Create a folder in your office PC mkdir dsscConfigFileGenerator go into the folder cd dsscConfigFileGenerator and clone the following libraries:
git clone https://git.xfel.eu/samartse/dsscdetlib
git clone https://git.xfel.eu/samartse/dssccaldata
git clone https://git.xfel.eu/samartse/dssccalibratio
go to the dsscdetlib and run
pip install . --upgrade -r requirements.txt
once the installation is done, go back to the head folder (dssfConfigGenerator) and launch jupyter by:
jupyter notebook
in case if you work remotely open ssh tunnel from your home office pc to your office pc to access jupyter:
ssh -L XXXX:localhost:XXXX exflqr****
where exflqr*** hostname of your office PC.
- now on your office pc run jupyer with port forwarding:
jupyter notebook --port=XXXX- Open this file by the jupyter notebook:
dssccalibration/notebooks/GeneratePixelGainConfiguration.ipynb
In the first cell, the main input parameters are defined, inline comments are explaining the meaninng of the variables:
PhotonEnergy = 2200.0 # this is photon energy in electron volts photons_per_bin = (1.0/5.0) # this is gain constant, 1/5 means that a single \ # photon with 2200 electron volt energy will produce 5 bins. op_frequency = 2.25 # options 2.25 or 1.125 (MHz) integration_time = 105 # Integration time in terms 700 MHz clock ticks, # i.e. price of one thick is by 1.44 ns # options at 4.5 MHz: 35 # options at 2.25 MHz: 35, 70, 80, 105 # options at 1.125 MHz: 35, 70, 105, 140, 210 Trimmed = True # All pixels will be trimmed according to the gain obtained by Pulxar # callibration data. In case of False flat gain, configuration will be # produced, i.e. setting of all pixels will be equal. fine_trim_targ = 12 # Iramp parameter, possible range is (0-63), but should not exceed 20 due to ADC limits RmpCurrDouble_settings = [0] # current dounle parameteres, possible lalues are 0 or 1. # 1 works for low gain and 0 for high gain CSA_Resistor_settings = [3,5] #[6] # all settings: [1,2,3,4,5,6,7] # Rest of the variables bellow should not be changed: proc_channels = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] FCF_EnCap_settings = [1,2,3,4,5,6,7,8,9,11,15] # all settings: [1,2,3,4,5,6,7,8,9,11,15] CSA_FbCap_settings = [0,1,4,5,6,7] # all settings: [0,1,4,5,6,7] bins_dynamic_range = {4.5:250, 2.25:500, 1.125:500} #required for CSA max charge estimation default_integr_time = {4.5: 35, 2.25:80, 1.125:80} CSA_Vlim = 0.3 FCF_Vlim = 1.6
Config file generation¶
After adopting the above parameters, run the notebook. It takes up to 2 minutes. A new folder will be created pyGenConfigData/ under the main folder dsscConfigFileGenerator.
The pyGenConfigData/ will contain any newly generated configuration file under a specific folder. These specific folder names contain photon energy, gain contact and integration time. For example: GenConf_TG2.2727_nG29_trimm_f2.25_intgr150/. Under the gain folder, pixel registry and sequencer files are organized by quadrants. The folder now can be copied on the instrument Karabo server and deployed for every PPT device.
SCS Expert Startup¶
This section is for experts only and details the instruction to set the DSSC in operation after it was resinstalled or after a long period of shutdown.
Chiller¶
Bypass¶
The silicone oil absorbs moisture in the air over time. When cooling this leads to the formation of ice crystal in the pipe and inside the DSSC cooling blocks which can prevent DSSC cooling operation.
To prevent this the first thing is to bypass the DSSC and operate the chiller at +45C overnight.
To do this, put the bypass in place, configure the chiller locally, increase the hard limit and run the chiller with valve opened.
Normal operation settings¶
After the bypass operation, the original settings have to be put back. They can be found in: https://rtd.xfel.eu/docs/eee-plc-framework/en/latest/softdevices/SD_CHILLERJULABO.html with the exception of the setting for E-PROG H=55C which should be H=34.40C instead for this particular chiller.
Also the hard limit as to be put back to 40C.
When the chiller is interlocked, the ramp goes to 0 C/min which is not the smallest ramp as one would expect but the highest ramp speed possible. It has to be put back to 1 C/min before the chiller is started.
MPOD¶
Check in the rack room that the power enable green switch button on the 4 MPOD power supply are on.
AON¶
From the DSSC_Main scene, in the Automatic Power Panel frame, open the
Manual Mode scene. In that scene, type aon and press on to start the automatic on channels.
This takes about 3 to 5 min to complete. U704 channel powering the PPT device should reach ~1.5 A.
SIB¶
The next step is to look at the SIB status. Switch the Experiment OK to on. More information on the SIB are accessible on the See more scene.
SIB state of 5 means ready for cooling. Set the chiller setpoint to -18 C.