BeckhoffSim Implementation Spec¶

Karabo Team

Introduction¶

BeckhoffSim is a TCP server that simulates a Beckhoff PLC on TCP protocol level. It is used to test BeckhoffCom and BeckhoffDevices without the need to have actual PLC hardware up and running.

BeckhoffSim is a Python Karabo device. It is composed of several building blocks, which are implemented as separate classes: the Manager, the TcpServer, a TimeSource, and the various soft devices inheriting from BaseDevice (e.g. DigitalInput).

The Manager maintains a set of soft devices, generates the self description, and decodes/encodes TCP messages. The TcpServer opens a TCP port, to which a BeckhoffCom instance can connect, and sends and receives messages. The TimeSource generates train ID and timestamp information. Finally, the soft devices simulate the behaviour of their PLC counterpart.

The association between the building blocks in unidirectional, i.e. that method calls are defined only in one direction. Information flow into the opposite direction is realized by events. Blocks interested in a specific event can subscribe to it by passing in a method, which is called if the event is fired.

Manager¶

The manager implements the core logic of the BeckhoffSim device. Main task is to maintain a set of simulated soft devices and handle the message transfer between these devices and the TCP server. Each soft device has a set of properties and hardware I/Os, which are exposed by the manager as expected parameters of the BeckhoffSim device. Soft device commands are not exposed, so that they can only be executed via the TCP server.

The manager also generates the self description when a BeckhoffCom connects to the TCP server, and sends the heartbeat in regular intervals.

The manager is implemented in the class Manager from module core.manager. It has several dependencies, namely

Framework
TCP server
Time source

The dependencies are injected as constructor arguments. Using dependencies is essential for unit testing, since then and only then they can be mocked[#]_. Dependencies are usually defined as interfaces[#]_. Since Python does not need a formal interface contract as e.g. Java or C#, the interface and it’s implementation are usually not separated into different classes.

[1]	A mock replaces a concrete class with an object that has the same methods and properties, but no behaviour by itself. Mocks can be configured by specifying return values, and one can make assertions about how they have been used.

[2]	Python does not have the notion of an interface like Java. Therefore an interface is implemented as an abstract base class (see also `abc`).

Method Interface¶

The Manager class provides the following public methods:

initialize()¶: subscribe to events from the TcpServer and by starting the PLC uptime counter

shutdown()¶: stops the TcpServer and properly shuts down the PLC uptime counter

start(port)¶: starts a TCP server on the given port. If no port is given, the default 1234 is used.

stop()¶: stops the TCP server

add_device(type, name)¶: adds a soft device of the specified type with the specified name to the list of simluated soft devices, also injects exp. parameters for the device into the schema of BeckhoffSim

set_property(device_name, prop_name, value)¶: sets a named PLC property on the specified device

set_io(device_name, io_name, value)¶: sets a named hardware I/O property on the specified device

Features¶

This section describes the features, provided by the Manager class, in more detail.

Adding Devices¶

Simulated soft devices are created using a factory. The factory method returns a device class depending on the specified type. The device type is a string that is also used in the related expected parameter of BeckhoffSim. The instance of the device class is stored into a dictionary using the device name as the key.

Once the device is added to the dictionary, the Manager queries the list of properties and hardware I/Os from the device class instance. It then calls Framework methods to inject the properties and I/Os as expected parameters into the schema of BeckhoffSim.

Note: PLC commands are intentionally not injected as expected parameters of BeckhoffSim.

Networking¶

When Manager.start() is called, the TCP server is started, and when Manager.stop() is called or if the Manager is destructed, the TCP server is stopped.

When the TCP server is running, it fires events to indicate the occurence of a certain condition. The Manager therfore subscribes to the following events from the TCP server:

error event: is fired in case of errors on the TCP socket
connect event: is fired, when a client has connected to the TCP server
disconnect event: is fired, when a client has disconnected from the TCP server

In case of a connect event, the Manager starts a thread to generate heartbeat messages each 10s. It also generates the self description message from it’s set of soft devices, and passes it to the TCP server for sending the message to the network.

In case of a disconnect event, the Manager stops the heartbeat thread.

Time Source¶

The Time Source generates following time-dependent information that is used in the messages:

Time Stamp. Current time in seconds since the epoch as an integer value.
Time Fraction. Fractional part of the time as an integer with 100ns as a unit.
Train ID. A unique ID of the current XFEL shot.
Delta Time. The time difference between the start of message generation and

arrival of a response from a device. An integer value with 100ns as a unit.

Method Interface¶

A TimeSource class provides the following public methods:

initialize(): starts a thread generating a new Train ID every 100ms

shutdown(): stops the thread generating a new Train ID every 100ms

get_time_stamp()¶: returns the Time Stamp

get_time_frac()¶: returns the Time Fraction

get_train_id()¶: returns the Train ID

generate_time_delta():: increases the current Time Delta by 1ms +- 0.1ms and returns this value

reset_time_delta()¶: sets the current Time Delta to 0, the function should be called at the beginning of message generation

TCP Server¶

The TCP Server opens a TCP socket and binds it to a port. It then starts to listen on that socket for client messages. If needed, it sends server messages on that socket.

Messages are simply stream of bytes. There is no further processing of the message itself within the TCP server.

The TCP server is implemented in the class TcpServer from module core.TcpServer. It has no further dependencies on other classes.

Method Interface¶

The TcpServer class provides the following public methods:

start(port): starts a TCP server on the given port

stop(): stops the TCP server

send(message)¶: sends a message over the network

subscribe_errorEvent(handler)¶: subscribes to the error event by adding the specified handler

subscribe_connectEvent(handler)¶: subscribes to the connect event by adding the specified handler

subscribe_disconnectEvent(handler)¶: subscribes to the disconnect event by adding the specified handler

Features¶

The features implemented within the TcpServer class are described in more detail in the following sections.

Events¶

The TcpServer defines a set of events, which are fired asynchronously:

connect event: is fired, when a client has connected to the TCP server
disconnect event: is fired, when a client has disconnected from the TCP server
error event: is fired in case of errors on the TCP socket

Socket Thread¶

The socket related code is run in a separate thread. Data exchange with this thread is via 2 queues: an input queue for data received on the socket, and an output queue for data to be sent on the socket.

The main loop of the thread listens on the socket for client data. The listen operation times out after 1ms to inspect the output queue. If the output queue is not empty, the messages therein are sent over the socket. Then the loop starts over again.

The loop terminates, when TcpServer.stop() is called. This also terminates the socket thread.

Receiving¶

When messages from connected clients are received, the receive event is fired containing the message as the event argument.

Sending¶

If the TcpServer.send(...) method is called, the message is pushed onto the output queue, if a TCP connection has been established. If there is no TCP connection, then the error event is fired.

Devices¶

Devices simulate the behaviour of PLC soft devices. They have the same properties than their PLC counterpart, but in addition they possibly also have hardware inputs and outputs. Properties and I/Os are annotated with meta information, such as name and value type.

PLC properties and hardware I/Os are implemented as Property class from module devices.property. The annotated meta information is a named tuple BeckhoffKey for PLC properties, and PropertySpec for I/O properties.

Devices are implemented as classes derived from a base class BaseDevice from module devices.base. For each PLC soft device, there is an associated simulated device. Currently, the following devices are supported:

DigitalInput

Methods and Properties¶

A device class provides the following public methods:

subscribe_update_property_event(handler)¶: subscribes to the update event for PLC properties by adding the specified handler

subscribe_update_io_event(handler)¶: subscribes to the update event for hardware I/Os by adding the specified handler

append_class_self_description(message)¶: appends the class description to an existing message

set_property(name, value): sets the named PLC property to the specified value

get_property(name)¶: returns the value of the named property

set_io(name, value): sets the named hardware I/O to the specified value

get_io(name)¶: returns the value of the named hardware I/O

A device class provides the following public properties:

Features¶

The features implemented within the device class are described in more detail in the following sections.

PLC Properties¶

The device provides a set of PLC properties and commands, which are identical to the ones in the corresponding PLC soft device. Each one has a unique name, which usually starts with capital ‘C’ for commands, and with capital ‘A’ for properties. The supported types are listed in the following table:

type	description
tBOOL	boolean
tBYTE	8 bit unsigned integer
tSINT	8 bit signed integer
tWORD	16 bit unsigned integer
tINT	16 bit signed integer
tDWORD	32 bit unsigned integer
tDINT	32 bit signed integer
tULINT	64 bit unsigned integer
tLINT	64 bit signed integer
tREAL	32 bit floating point (float)
tLREAL	64 bit floating point (double)
tSTRING	ASCII byte array
tVOID	command
tMULTI	array of 32 bit unsigned integer

The distinction between commands (something that can be executed) and properties (something that is associated with a value) is done solely based on whether the variable type is tVOID or not.

The meta information for PLC commands and properties are stored as named tuple BeckhoffKey and contain name, id, type, access, unit, and prefix.

PLC properties and commands are stored in a dictionary, where the name is the key and the value is an instance of a Property class. The Property class consists of the meta information and a place to store the value. The dictionary is accessable from outside via Property.properties.

The set_property() method is used to modify the value of a PLC property. The name, passed in as an argument, is used to look up in the self._properties dictionary to reference the Property instance in order to write to it’s value. Additional specific device behaviour needs to be implemented afterwards.

The get_property() method is used to return the actual value of a PLC property. The name, passed in as an argument, is used to look up in the self._properties dictionary to reference the Property instance in order to read it’s value.

Hardware I/O¶

The device provides hardware inputs and outputs of the corresponding PLC soft device. Each one has a unique name, and can be of one of the types, as mentioned with PLC properties, except tSTRING, tVOID and tMULTI.

The meta information for Hardware I/Os are stored as named tuple PropertySpec and contain name, and type.

Hardware I/Os are stored in a dictionary, where the name is the key and the value is an instance of a Property class. The Property class consists of the meta information and a place to store the value. The dictionary is accessable from outside via Property.ios.

The set_io() method is used to modify the value of a hardware input. The name, passed in as an argument, is used to look up in the self._hw_ios dictionary to reference the Property instance in order to write to it’s value. Additional specific device behaviour needs to be implemented afterwards.

The get_io() method is used to return the actual value of a hardware input or output. The name, passed in as an argument, is used to look up in the self._hw_ios dictionary to reference the Property instance in order to read it’s value.

Messages¶

A message contains key value pairs prepended with a short header. The message header contains timestamp and train ID. Each key value pair consists of two numbers, one to identify the soft device instance, and one to identify the property or command on that instance, followed by 0 to 20 values.

Before sending messages over the network, they need to be converted into byte streams. Received byte streams need to be converted back into messages.

Messages are implemented in the Message and the Pair classes from module core.message.

Method Interface¶

The Pair class provides the following public methods:

add_uint32(pair)¶: adds an integer value to the pair (maximal 32 bit)

add_string(pair)¶: adds a string value to the pair (maximal 80 characters, including terminating ‘0’)

to_bytes()¶: returns the pair as a byte stream

The Message class provides the following public methods:

add_pair(pair)¶: adds the specified pair to the message

to_bytes(): returns the message as a byte stream

Features¶

The features implemented within the Message and Pair classes are described in more detail in the following sections.

Converting to Byte Stream¶

In order to send messages over the network, they must be converted into a stream of bytes. The format of the byte stream is defined by [FT2015]. Conversion is handled by the Message.to_bytes() method, which returns a byte stream representation of the message.

[FT2015]

T. Freyermuth, J. Tolkiehn, “European XFEL PLC Karabo Interface Description,” internal note IN-2015-09-30, European XFEL, 2015

FiFo¶

The FiFo class basically mocks the PLC behavior inside the beckhoffSim project. Totally, two FiFo instances are required to establish the communication. One TX fifo (transmitter) for the message parsing between the Manager and the TcpServer and an additional RX (receiver) fifo which is filled with messages coming from the TcpServer. The messages are subsequently dispatched in the Manager.

Technically, the FiFo class uses queue objects for thread safe communication. The maxsize member variable has been set to infinity. Instead, for size handling a private member variable _max_bytecount is been set during initialization. The private member variable _current_bytecount is used to observe the current size of the queue. After each method the current bytecount is updated.

Method Interface¶

The FiFo class provides the following public methods:

try_put(item)¶

Safe method to put an item into the fifo. The bytecount of the item is checked before.

try_get()¶

Safe method to return an item from the fifo. Only returns an item if the bytecount of the fifo is greater than zero.

put(item)¶

Puts an item into the fifo. If the bytecount of the item together with the current bytecount of the fifo exceeds the max bytecount and OverFlowException is raised.

get()¶

Returns an item from the fifo. If the current bytecount inside the fifo is equal to zero, an EmptyBufferException is raised.

clear()¶

Atomically locks the fifo and clears the fifo list of items.

__len__()¶

Overwritten built-in function to provide the current bytecount inside the fifo.

BeckhoffSim¶

BeckhoffSim is the top level class. It has a lightweight implementation by simply passing down all method calls to the Manager. In addition, it implements the Framework interface, which is used by the Manager to interact with the Karabo framework, for example to inject expected parameters.

State Machine¶

The state machine of BeckhoffSim is fairly simple. It only has a Started, Stopped and Error state. The state reflects the state of the TCP server. If the TCP server is running, then BeckhoffSim is in Started state, when the TCP server is not running, then BeckhoffSim is in Stopped state. If an error occurs during the start of the TCP server, or while the TCP server is running, BeckhoffSim changes into the Error state.

Instantiation¶

BeckhoffSim has an expected parameter to configure simulated soft devices. It can only be set before instantiation. Is is a TABLE_ELEMENT where the device type and instance name can be specified.

When BeckhofSim is instantiated, it goes through the list of configured soft devices, calls Manager.add_device() for each of them, and then Manager.initialize().

Finally, BeckhoffSim changes into the Stopped state, and the expected parameters for each device are injected below the Devices node[#]_. If however, an exception was thrown while calling one of the Manager methods, BeckhoffSim changes into the Error state.

[3]	Due to a bug in Karabo framework (see #10897), users need to activate `Apply all` once to have value updates on the newly injected parameters.

Shutdown¶

When BeckhoffSim is shut down (or killed), it simply shuts down the Manager by calling Manager.shutdown().

Reconfiguration¶

If expected parameters of the configured soft devices are changed, the preReconfigure() hook is called. Within the hook, the changed device properties and I/Os are extracted and the manager methods set_io() and set_property() are called respectively.

Commands¶

BeckhoffSim provides 2 commands:

Start: starts the TCP server at the configured port
Stop: stops the TCP server

The port for the TCP server is configured using an expected parameter. It can be changed at runtime, but the TCP server needs to be restarted in order to take effect.

Appendix¶

File Structure¶

The code is organized into packages:

core: contains modules defining the manager, TCP Server, and classes associated with the manager
devices: contains modules to define types, enums, base device, and soft devices
tests: contains the unit tests

Coding Style¶

The BeckhoffSim code follows the guidelines as proposed by PEP 8 Style Guide available under PythonStyleGuide.

Unit Tests¶

Most part of BeckhoffSim is unit tested. Unit tests are also run as part of Continuous Integration (CI). In order to get test reporting on the CI server, XML runner is required. Since it is not yet part of the Karabo Python library[#]_, you need to install it into your user account by running

[4]	see bug report #10818

pip install --user unittest-xml-reporting

Afterwards, it might be necessary to add the user site-packages directory to PYTHONPATH, which is

::: ~/.local/lib/python3.4/site-packages

under Linux, and

::: %APPDATA%PythonPython34site-packages

under Windows [1].

[5]	`%APPDATA%` resolves to `\\win.desy.de\home\<user>\Application Data`.