Software infrastructure at euXFEL¶
Overview¶
At euXFEL we use a variety of modern It-solutions for file/software management and for documentation. Below you find a non-exclusive list together with a short explanation of its purpose. The list includes items that are not XFEL specific.
Version control, development¶
- git: A tool for decentralized version control. It is running and available on all common operating systems. It has become the de-facto standard for software development projects. Another common tool is subversion (svn), which is predominantly used for projects with large binary files. Git is fully open and freely available.
- gitlab: A tool that helps with the administration of git repositories. It provides a easy to use web interface. It helps with issue tracking and developments in teams. Gitlab is fully open and freely available. Therefore, anybody can setup a gitlab server on a private host.
- gitHub: A proprietary, partially commercial online service owned by Microsoft. It is an external host for git repositories useful when no private server infrastructure is available.
Control¶
High/software level¶
karabo: A distributed system intended for control and data acquisition purposes, and as such part of the SCADA control system family. Its central entity is the device. It has a standardized interface to the outside world, meaning that each device has device properties, commands (slots) and is uniquely identified by a deviceId. Users can write device classes using two APIs (bound, middlelayer). More specific explanations can be found in:
doocs: Another distributed control system developed at DESY.
Low/hardware level¶
- Beckhoff: A industrial automatization solution developed by the company Beckhoff Automation. The main purpose of such a system is stability, such that it can be used for critical applications (interlock) with certain requirements regarding the probability of failure. Additionally, Beckhoff provides one of the few real time implementations of the ethernet standard, called EtherCAT. The term real time is related to the output of a result within a certain time (not to be confused with fast). A nice illustration of EtherCat’s operation principle (and why we call one Beckhoff installation a “loop”) can be found here: https://en.wikipedia.org/wiki/File:EthercatOperatingPrinciple.webm
Other¶
- Elog: A weblog application developed at PSI which is widely used in research facilities. It is fully open and freely available. The logbook server can be accessed via a web interface or the standalone elog client.
- Alfresco: A free and partially open software tool intended for document management.
- ReadtheDocs: An open and freely available tool for hosting software documentation.
- Jupyter Notebook: The successor up of the “ipython notebook”, basically a combination of a mathematica-like notebook with the interactive ipython console. It now supports a variety of different programming languages.
1. Working with git¶
Comment: Git is a professional software versioning tool. It requires a certain amount of training to get used to it. The manual below is in no way complete. There are various scenarios and policies on how to use this tool. The complete documentation, including manuals, can be found here: https://git-scm.com/doc
What is a repository¶
As previously mentioned, git is a decentralized versioning tool. This means that all clones of the same repository have the exact same role. The definition of a “master” repository is left to the user. To understand the working principle of git it helps to look at the following figure:
A git repository consists of a workspace with the actual files and a (hidden) version database containing the history of each file registered to be tracked. Each clone of a git repository contains the database. The clones just differ with respect to the number of versions (called commits) and to which commit the current workspace (checkout) points.
You can create a new local git repository
git init
Then register files to be tracked for version control:
git add <filename>
Finally once you are reasonably satisfied with a working version you can commit to it by typing:
git commit -a -m "description of commit"
This will create a new entry (commit) in your version database.
Remote repositories¶
When working with remote repositories we basically synchronize the version database between the local and the remote repository. Use:
git pull <remote-name> <branch-name>
This will download the latest commits from the remote and merge the history locally (if neccessary). In the end the workspace points to the latest commit.
If you don’t want to merge immediately use:
git fetch <remote-name> <branch-name>
This will only update your local version database. Your local workspace still points to your latest commit. Now you will need to merge the local and remote history and create a new commit from that:
git merge <branch name>
In case you want to create a new local repository from an existing remote repository use:
git clone <remote-address>
This will download the entire version database and create a workspace pointing to the latest commit. It will also setup an alias for the remote address called origin.
Since git is a decentralized version control system you might have more than one remote repository, illustrated in the following figure.
In this scenario ComputerA could define several remotes:
git remote add origin <server-address>
git remote add ComputerB <computerB-address>
Computer A can now fetch the newest version history from the server by typing:
git fetch origin <branch-name>
or from Computer B using:
git fetch ComputerB <branch-name>
In order to push your commits to another host you can use:
git push <remote-alias> <branch-name>
You can only push to a repository without a workspace, a so called bare repository (In the previous figure only to Server).
2. Working with git/gitlab¶
More specific instructions about the workflow at SCS using git can be found on the main page. There is also an introduction into the use of Gitlab at SCS.
3. Building documentation¶
Example: SCS-documentation at
https://in.xfel.eu/readthedocs/projects/scs-documentation/
In summary: readthedocs uses sphynx to create html documentation from restructured text. The repository in this case is the scs documentation on gitlab. The result is linked on our alfresco main page.