Clients

The output protocol is currently under development and not yet finalized. There will be a full-fledged client API to receive the pipeline results as well as integration into Karabo. Until then, there is only a single client solution called EXtra-metro provided on the European XFEL computing infrastructure.

EXtra-metro

The software is available through the module system. You may start this software on any node on the online cluster. Please find the nodes belonging to each SASE branch and/or instrument and information on their intended usage here.

% module load exfel EXtra-metro
% extra-metro

Once started, the main window shown below will appear. It allows to connect to the output address of any running MetroProcessor device, displays statistics to its pipeline data flow and visualize the results. The address for a number of deployed instances are provided by default, but any custom location may be entered. Please contact the data analysis team (da-support@xfel.eu) if you would like to add a permanent device instance to this list.

Screenshot of EXtra-metro

Once connected successfully, the next index received from the pipeline should fill the tree view below, with at least trains always appearing. Any further entries depend on the loaded context. When the pipeline begins processing, the counts and rate should update correspondingly.

Only the result of views is actually forwarded through the pipeline, the entries for other path types such as karabo# are only for statistics. If a raw signal from Karabo is desired, a corresponding view must be added to the pipeline context which simply returns said value. Views are organized into the tree structure further by slashes / in their name.

Visualizing results

If an entry provides a result, a checkbox is present next to its name and it may be visualized in several ways:

  • Double-clicking a view will try to choose a suitable kind of display and configuration automatically. In most cases, this is the recommended method.
  • Right-clicking a view and selecting Display will choose the same kind of display as above, but allows to change its initial configuration. For most arguments, a tooltip is provided.
  • Right-clicking a view and selecting Display by allows to change both the kind of display as well as customize its configuration. Please note that their use may not be fully documented yet. The available modules, which may be of general use, are:
    • fast_plot: Plots 1D vectors or 2D arrays as list of vectors of up to 1M samples each.
    • hist1d: Bins a stream of scalars to a 1D histogram.
    • hist2d: Bins a stream of points (X,Y coordinates) into an image with extensive support for scaling/translating the projection via mouse.
    • image: Plots 2D arrays as images.
    • polar_plot: Plots a 1D vector in polar coordinates.
    • sorted: Displays a bar plot of a 2D array sorted by its second column and labeled by its first.
    • value: Prints any value as text.
    • waveform: Plots a scalar value over time.

Profiles

The current geometry of all open windows and their configuration state may be saved to a profile in order to restore it at a later time. A new profile is created by selecting Save current configuration in the Profiles menu. If a profile has already been loaded or saved recently, you may also simply choose Overwrite last profile. The top part of the entries in this menu will allow to load any of the available profiles, which may be refreshed manually by selecting Rescan profiles

The profile is saved as a JSON file either in your own home directory, which is shared across all nodes belonging to a particular SASE branch, or the usr/ directory of a proposal. You may switch between these locations by selecting Local profiles or Profiles in proposal, respectively. The list of available profiles will change accordingly and this setting is preserved across closing an instance. Finally, you may load a file from any location using Choose external file.

A profile may be loaded immediately when opening a new instance of EXtra-metro by specifying it on command-line:

% extra-metro --profile <name>

If the profile location differs from your currently selected one or you want to select a location just for this execution, you may also specify a proposal:

% extra-metro --proposal <number>

Please note that only proposals available on the chosen online cluster node are available and you must have the corresponding access rights.

Result topics

The underlying protocol to the data pipeline uses ZeroMQs PUB/SUB pattern to only send those results to a client in which he is interested in. Each view result is represented as a topic to which a client may subscribe in order to receive the data.

If the checkbox Handle subscriptions automatically is selected, this will be done transparently by EXtra-metro whenever data is required for visualization or ceases to be required. You may change this subscription for each view result by the corresponding checkbox in its tree entry. However, this will stop any open visualization for this particular view from updating! You may override this automatic handling completely or just for particular views at any time, e.g. to preserve a particular result.