Getting Started in the Data Analysis Group
Welcome to the EuXFEL Data Analysis (DA) group! This page provides some basic information and important links to start working here.
Checklist
Please go through the following steps. Where required, hyperlinks provide you with the respective information or item:
Introduce yourself to human resources
- HR will most likely create a REXX account for you.
- You will receive a welcome package with some goodies.
Office Key
Get an office key at XHWS, from Frederike Wittmaack.
Staff Card
Get a XFEL/DACHS card at the gate house or at the user office.
Accounts
- Get a global staff member account (with a centralized password for all services).
- This first IT-step is best taken by going to the ITDM office XHQ 1.104 in person.
- While you're there, ask them for access to the Maxwell cluster, if not yet included to the account privileges. Once you have Maxwell access, you should be able to do
ssh max-exfl.desy.de
and login from your office PC. - Separately, you will have to ask ITDM for access to the instrument data paths on GPFS, by means of sending an e-mail to it-support@xfel.eu with CC to the group leader.
- Set up a Zoom account with your work email, this will automatically be linked to the XFEL organisation, giving you access to the contact list.
- Set up a Zulip account with your work email.
Safety Training
Perform online safety trainings at https://in.xfel.eu/safety_training/.
- Select:
- "EuXFEL Safety training online Basic Staff".
- "EuXFEL Safety training online Experiment Staff".
- After completion, a link is provided to the respective Safety Training Certificate that needs to be signed and mailed training.certificate@xfel.eu.
- Now the safety group will register you in the DACHS system.
Info
Basic safety training is required for your DACHS card to be able to open any doors, including the main gate and XHQ side entrances.
Experimental Hall Access
Access to the Experimental Hall requires a Transponder, your DACHS card, and you to have completed and registered your safety training
- Based on successful online training and DACHS registration, send an e-mail to Süleyman Arslan (CC group leader), where you briefly state what tasks you have in the experimental hall, e. g. "discussion of data analysis with instrument staff at the experimental hutch PCs".
- Pick up a Transponder in office E2.121 (Mon, Tue, Thu 8:30h - 12h, Wed 8:30h-10:30h) and keep it for the length of your stay at European XFEL.
Network Access
Get access to the Control Network and Online Cluster:
- Open an ITDM ticket by sending an email to it-support@xfel.eu (with group leader in CC) where you request access to the Control Network.
- DA has some functional accounts which you may require access to depending on your role, you should check with your mentor which (if any) you need access to, and request access from ITDM.
Work Computer Privileges
Your desktop/laptop will by default be 'green', meaning that it is managed by ITDM and that you do not have sudo rights on it. If needed you can request a 'yellow' desktop where you have sudo rights on your computer (but minimal support), or 'red' where you completely manage the OS yourself and can install whatever you want (with zero support).
- Open a ticket to unix@desy.de asking for a yellow desktop to have sudo permissions on your machine.
- Fill the form link with the ticket number from the unix@desy.de, your PC-ID (e.g.
pcx-30345
) and the rest of data required. - Print the document, obtain the signature of Bartosz Poljancewicz or another system administrator (ITDM) and send the form to DESY UCO (see address in the form).
Familiarise yourself with EuXFEL
- XFEL Organizational Structure.
- XFEL Operation.
Familiarise yourself with DA
- DA overview:
- Major software currently developed at DA:
- Extra-data: https://github.com/European-XFEL/EXtra-data
- Extra-geom: https://github.com/European-XFEL/EXtra-geom
- Extra-foam: https://github.com/European-XFEL/EXtra-foam
- Karabo Overview:
- Essential information about the EuXFEL control software ecosystem (SCADA), which has some relevance for online data analysis.
- Optionally: install Karabo in a virtual machine and follow the tutorial within.
- Verify access to Max-Jhub for remote Jupyter notebooks. This should work if you have Maxwell access.
Tools used at EuXFEL
- Alfresco: platform for document exchange.
- Gitlab (incl. CI): software repository for internal project (public projects are hosted on github).
- Redmine: project management and issue tracking tool.
- eLog: electronic logbook, used for OCD reports and experiment logs.
- All EuXFEL IT services can be accessed at https://in.xfel.eu/ (internal services are accessible from the top right corner of the page).
- Zulip: our online chat platform.
Software development practices
- TODO: Merge requests, Unit Tests, etc...
On Call Duty (OCD)
- TODO planning and execution (in preparation)
Set-up Travel Account
XFEL uses Atlatos to manage business travel requests and reimbursements, setting up the account and learning how to use it can take some time, so even if you don't plan on having any business trips in the near future it is worth doing now to avoid having to rush later.
- Send an e-mail to Halimah Shie, asking for the account.
- It's recommended to make an appointment with Halimah for a personal introduction to the travel system.
Remote Work Tips
I was going to make this neutral sounding but that'll probably be a bit too abstract so instead I'll describe my opinionated setup (Robert Rosca), feel free to take this and make modifications as you see fit.
SSH Connections
Thanks to Thomas Kluyver and Philipp Schmidt for these tips. Here's my local (home desktop) ssh config file:
# Use modern ciphers by default, marginally improves performance on modern hardware
Ciphers aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
# Optionally enable SSH connection sharing, this has a few implications, read up
# on it in the ssh config man page under 'ControlMaster' (etc...)
# https://linux.die.net/man/5/ssh_config
#ControlMaster auto
#ControlPersist 600
#ControlPath ~/.ssh/sockets/socket-%r@%h:%p
Host github.com
Hostname github.com
User git
IdentityFile ~/.ssh/github/id_ed25519
# Enable hostname canonicalization
CanonicalizeHostname yes
# Use canonicalization if hostname contains no dot
CanonicalizeMaxDots 0
# Disable system DNS if canonicalization fails
CanonicalizeFallbackLocal no
# Domains for canonicalization
CanonicalDomains desy.de
# Special flags for gitlab over SSH
Host git.xfel.eu
User git
Port 10022
ForwardX11 no
IdentityFile ~/.ssh/gitlab/id_ed25519
# Generic configuration for all DESY/EuXFEL hosts
Host *.desy.de *.xfel.eu
User roscar
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
PasswordAuthentication no
ForwardX11 yes
# Aliases
Host max-exfl
User roscar
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
PasswordAuthentication no
ForwardX11 yes
Hostname max-exfl.desy.de
ProxyJump max-exfl-display.desy.de
Host max-exfl-display
User roscar
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
PasswordAuthentication no
ForwardX11 yes
Hostname max-exfl-display.desy.de
# Control system hosts.
Host exflonc*.desy.de exflong*.desy.de *-onc-*.desy.de *-ong-*.desy.de
ProxyJump exflgateway.desy.de
# Office network jump, either bastion, max-display or max-exfl-display
Host max-*.desy.de *.xfel.eu exfl*.desy.de max-exfl !max-display.desy.de !max-exfl-display.desy.de
ProxyJump max-exfl-display.desy.de
First I set up basic authentication for GitHub since our open-source projects are hosted on there.
Following that are configurations to enable some canonicalization which allows you to ssh to just the host names without the .desy.de
or .xfel.eu
domain names, with this you can type in ssh max-exfl-display
instead of the full max-exfl-display.desy.de
that you would usually have to type from outside the network.
Then there are some specific settings for our GitLab, where I point it to my GitLab ssh keys. You can look up how to create and add ssh keys for GitHub/GitLab online.
Aliases are set since sometimes (for me) the canonicalization is a bit flaky, so I put the two most used ones max-exfl
and max-exfl-display
as explicit aliases.
Most of the hosts will have ProxyJump max-exfl-display.desy.de
, this is required as those hosts are on the internal network and you must first jump through one of the externally accessible nodes. ProxyJump
is equivalent to manually ssh'ing to max-exfl-display, then ssh'ing to the desired node again.
Kerberos
All the DESY IT infrastructure uses Kerberos for authentication (if you're curious Computerphile has a neat video on this: Taming Kerberos) which means that, even if you have ssh keys set up locally for connection to maxwell, you must authenticate every 24 hours to be able to connect without entering your password.
This can be done by installing the local kerberos client (krb5
), running kinit -A -f $USERNAME@DESY.DE
, personally I set an alias alias kinitd="kinit -A -f roscar@DESY.DE"
, so I just type kinitd
every morning and can connect without password requests for the rest of the day.
Visual Studio Code
For remote work I find vscode to be extremely useful since it has built-in ssh remote functionality: Remote Development using SSH.
This means that if you set up your ssh config as described above, and then authenticate with kerberos, you can open vscode and start up a remote connection to maxwell. This will install a remote vscode server on maxwell, and you can then edit and execute files directly on the nodes which can be extremely useful.
Supplementary EuXFEL Information
Scientific environment
- Scientific instruments overview: https://www.xfel.eu/facility/instruments/index_eng.html
- Describes concept and design of all instruments at XFEL.
Other important links
- Webmail client: Zimbra.
- Human resources portal: REXX.
- XFEL and DESY Campus maps.
Documents
- The DAQ & Control page in Alfresco is a page shared between several groups: Central Instrument Engineering (CIE), Electronics (EEE), IT and Data Management (ITDM), Controls and Data Analysis (DA). It contains a lot of useful information, especially in the Document Library tab.
- Naming convention for Karabo devices.
References
- Data analysis in Karabo.
- ICALEPCS-19 article on Jupyter Notebooks for DA, slides (paper to follow).
- Karabo design concept: "An integrated software framework combining control,data management and scientific computing tasks", as well as the Karabo design concepts presentation.
- Karabo GUI "Karabo GUI: Multi-purpose graphical front-end for the karabo framework".