What’s New in Karabo 2.0.4¶

Detailed Merge-Log¶

bound/cpp core¶

C++/bound: Add attributes for DaqDataType !963
C++/bound: Always broadcast when the network connection changes
C++/bound: Simplify binding for MetaData
C++/bound: Add noded slots to C++ !968
C++/bound: The ‘writeChannel’ with correct trainId.Use channel’s ‘write’ with MetaData parameter.
C++/bound: Reply to slotPing always via broker !964
Bound: fail nicely for wrong plugin to bound server. !995
C++: Make sure the versioning namespace is used for some attrs`key` and path weren’t being grabbed with the correct namespace
C++: Add a thread per SignalSlotable !1006
C++: Fix PointToPoint::Consumer to avoid locking a mutex !1004

core Devices¶

Project DB: Really fix the DB root attrs copy !1000
ProjectDB: Add an alias for revisions in project DB !966
ProjectDB: Handle project DB revisions correctly !948
Core Devices: Fix some problems in RunConfiguration !978
Add a Python implementation of device template filling

Dependencies and deployment¶

DEPS: Fixed wrongly behaving auto-developing during karabo checkout !1013
DEPS: Added dependency tracking and from-file installation to karabo script.
DEPS: Changed run directory of all servers to be $KARABO/var/data !994
DEPS: Make ‘karabo’ a proper python “program”. Now you can edit the code and test your changes without needing to reinstall. !982

Middlelayer core¶

MDL: properly support dir for proxies !997
MDL: fix not set values in proxies !1008
MDL: schema injection in middle layer proxies. Now a middle layer device sees schema changes !984
MDL: fix sending nothing if disconnected !1012
MDL: fix race condition for dead devices. Sometimes, when a dead device was accessed via a proxy, a CancelledError slipped out of the code. !987
MDL: We invent a new KaraboValue called NoneValue, which represents the absence of a value. !958
MDL: properly support tables in middle layer !960

Graphical user interface¶

GUI: Add double click handling to the project view
GUI: Watch revision changes of DeviceConfigurationModel to update active configuration properly
GUI: Add device configuration to existing device instance model
GUI: Start the newfangled project server.This also fixes some problems with Python device auto instantiation
GUI: Show indicator for modified projects
GUI: Set visibility of scene widgets properly #13335
GUI: Handle selection of project list items
GUI: ProjectViewItemModel becomes a singleton
GUI: Add class_id to DeviceInstanceModel
GUI: Fix bug check for correct model item when looking for parent
GUI: Rename karabo_gui.mediator to karabo_gui.events
GUI: fix logger panel filter option !999
GUI: Add duplicate dialog for scenes and macro
GUI: Show alias in project list !966
GUI: Save and duplicate active device configuration implemented
GUI: Show when scenes have unsaved changes
GUI: Fix save subproject and remove obsolete code
GUI: Add a pop-up window widget !967
Common: SceneModel: write_scene returns string now not bytes anymore
Common: Remove the three API hash benchmark script
Common: Use a common base class for savable objects. This introduces BaseSavableModel !965

What’s New in Karabo 2.0.3¶

Detailed Merge-Log¶

bound/cpp core¶

C++: Fix memory leak that also created severe message digestion problems. !931
C++: Fix ‘shutdown behavior’ issue using direct asyncWrite callUse of the direct call to the _asyncWrite_ in _JmsProducer_ class fixes the losses of messages.
C++: Stop using the “karabo_log” topic for logs !953
C++: Clean pipeline handler usage- Using bind_weak allows to get rid of wrappers for pipeline data handlers in SignalSlotable. !913
C++: Use bind_weak where possible in PointToPoint. !956
C++: Add support for filtered subschema !926
C++: Use bind_weak in OutputChannel and TcpConnectionUsing bind_weak in OutputChannel solves crashes at device shutdown. !925
C++/Bound: Fix typo in slot name slotReSubmitAlarms !951
C++/Bound: Data source info on p2p should follow ITDM specs
C++/Bound: Changes separator for deviceId and outputname from / to : as specified by ITDM !916
Bound: Also bound Python command line needs EventLoop to run.Without that, nothing works. !942

core Devices¶

GUIServer: Update the GUI server protocol to use tokens

Dependencies and deployment¶

DEPS: Fix the shebangs of Python entry points !924
DEPS: Fix the version of our Python wheels !924
DEPS: make cross test more resilient to timing failures !955

Middlelayer core¶

MDL: also support slotReSubmitAlarms !952
MDL: publish alarm states in middle layer !947
MDL: fix execute function and add a test for it !944
MDL: add unit-aware numpy equivalentsmost numpy functions (namely: ufuncs) work out-of-the-box with units. !921
Remove a Hash import that got in during !891

Graphical user interface¶

GUI: Adapt Device Server dialog to new model parameters. Follow-up of MR !909
GUI: Improve display of vector properties in Table Element !917
GUI: Allow for read only columns in GUI repr. of TABLE_ELEMENT !919
GUI: Add a VacuumWidget for Agilent Ion Pump
GUI: Create a singletons package for the GUI !928
GUI: Move the project DB connection into singletons
GUI: Handle the initialized flag correctly
GUI: Features genericlamp !936
GUI: Adapt dialog for device configurationThis is almost done. There is still some problem when saving the device configurations..
GUI: Add missing commata in vacuum files MR !936
GUI: Remove the krbgui package
GUI: Add missing context menu actions+ Added missing context menu entries
GUI: Add missing alarm/warn conditions and adapt icon/color association
GUI: Add individual object saving to context menu
GUI: Remove some of the signals from ManagerThis cleanup in Manager should help the transition to the new project panel. !945
GUI: Add more parameters to project models. Add description to BaseProjectObjectModel

What’s New in Karabo 2.0.2¶

Detailed Merge-Log¶

bound/cpp core¶

C++: Added unit test for connect/disconnect of ‘Input-‘ and ‘OutputChannel’. !883
C++: Fixes a serious bug happening during shortcut messaging !892
C++: Clean connectChannel and disconnectChannel.With request(..).receive(..) now shortcutting the broker if possible, no further shortcuts are needed. !893
C++: Added feature of timeout for request/asyncReceive
Bound: Inserted character ‘i’ in initalValue which repairs bound python !911
Add a status field to all three APIs !908

core Devices¶

GuiServer: ‘master’GUI - GuiServer - ProjectDatabase communication !891
Core: Fix camera_interface.py- removed State.READY from “state” options, since State.READY does not exist in the Framework !903

Dependencies and deployment¶

DEPS: Set proper RTD path !894
DEPS: Create a doxygen output directory so that breathe correctly works !895

Middlelayer core¶

MDL: make device node more resilient if a device node is not properly filled, we don’t crash anymore. !890
MDL: Add an isAlive to proxies !897
MDL: add yet another test for the hash, testing a problem I once had while refactoring !899
MDL: fix waitUntil, sometimes an error was logged, which actually wasn’t one. !898
MDL: add the alarm slots to middle layer !906
MDL: report caller upon error. it has shown very useful while debugging to know who called a failing slot. !905
MDL: The examples should use DeviceNodes where applicable !907 !904

What’s New in Karabo 2.0.1¶

Detailed Merge-Log¶

bound/cpp core¶

C++: Cleaned the DeviceServer class from unused parameters !888
C++: Refactor startToFinishInit !861
C++: fix deepCopyAsyncResults !869
C++: Correct error handling for instantiation !769
C++/Bound: Unify device.writeChannel between C++ and bound Python !860
C++/Bound: Device::start method is renamed to Device::finalizeInternalInitialization to avoid name clashes with some FSMs. !867
C++/Bound Add support for ImageData to HDF5 !870
C++/Bound: Fix getAlarmInfo !872
C++/Bound: fix setAlarmCondition !874
C++/Bound: Completes alarm specifications as per concepts. !886
Bound: Add a check if there is a python value that can be displayed !875
Bound: fix boundDeviceClient.get as schema.isLeaf(key) checks on attribute “nodeType” and not “leafType” !873
Bound: Fix camera_fsm.py. !879
Bound: Fix python FSM bugs uncaught after switch to ‘Unified States’.

core Devices¶

Project Manager: Change the project manager to accommodate the GUI

Dependencies and deployment¶

DEPS: Rename some of the Python commandline scripts- convert-karabo-device: converts source code for Python devices
DEPS: convert-karabo-project: converts project files to the new version

Middlelayer core¶

MDL: make timeout work in wait !862
MDL: fix setting values on device proxies !859
MDL: Remove the karabo.middlelayer_api._project subpackage
MDL: actually use entry points for middle layer device discovery !877
MDL: always show deviceId for deviceNodes !887

Graphical user interface¶

GUI: Fix lazy loading of project data
GUI: Adapt new project panel !842
GUI: Account for the domain in the GUI project IO code. Anything which touches the database needs to pass a domain.
GUI: Add macros/scenes to project via GUI
GUI: Minor import fix for newly introduced TEST_DOMAIN Related to MR !868
GUI: Update state colors indicators
GUI: Add a ColorChangeIcon classT his handles SVG files containing lots of icons.
GUI: Add/Edit project objects. Some more dialogs are added here to add/edit the different project objects.
GUI: New/Load/Save project in new project panel.
GUI: Start speaking the new project data protocol from the GUI !885

What’s New in Karabo 2.0¶

Karabo now uses C++11 features¶

C++11 usage is now (officially) supported for framework code. The following guidelines are suggested:

Feel free to use new features where they make sense. E.g. use auto to shorten iterator syntax in loops, e.g. std::map<MyComplexType, MyMoreComplexType<double> >::const_iterator it = foo.begin() -> auto it = foo.begin().
Don’t use auto to indicate straight forward types, e.g. auto i = 4;
Existing code does not need to be refactored for C++11 feature usage alone. E.g. if you happen to refactor something anyway, feel free to replace iterators with auto if it aids readability. You do not specifically have to refactor otherwise working code though.
Do not use std::shared_ptr, we will continue to use boost::shared_ptr!
In general, if a boost and a std-library feature coexist (smart pointers, mutices, bind, etc.), continue to use the boost implementation as we have done previously, especially if there is a risk that your new code needs to interact with existing code.
When using more „advanced“ features, like late return type declaration (->decltype(foo)), variadic templates or reference forwarding, add a short comment to these lines to aid people less experienced with C++11 features in the review.
We currently do not encourage to use newly introduced numerical types, e.g. uint64_t as the Karabo type system has not been fully prepared for them.

Compiling devices with C++11 support¶

While the code of C++11 devices does not have to change to profit from some optimizations that C++11 provides, the compiler flags generated by NetBeans have to be adjusted. To do so

open your package in NetBeans,
right-click ion the package in the project panel,
choose the last entry in the appearing menu (“Properties”),
in the appearing dialogue,

*choose “Build -> C++ compiler” in the “Categories” panel on the left, *”<All Configurations>” in the for “Configurations” on the top * and “C++11” for “C++ Standard” in the options field.

Compile from NetBeans once in “Release” and once in “Debug” mode and commit the following new files in the nbproject directory:

configurations.xml
Karabo-Debug-Linux.mk
Karabo-Release-Linux.mk

Update Makefile¶

To take care that the changes of the NetBeans generated makefiles will be properly updated also there where the device project was only checked out and compiled from the command line, the package Makefile has to be updated as well. This updates fixes tiny makefile bugs as have been fixed in the framework long ago.

Some macro changes (C++)¶

To make a member function callable as a slot, do not use anymore the SLOT<N> macros but switch to used KARABO_SLOT<N>.

Alarm Conditions¶

Alarm conditions are now based on objects. There is a predefined set of conditions. In C++ they can be found in:

karabo::util::AlarmCondition

in Python in:

karabo.bound.AlarmCondition

The following conditions now exist: NONE, WARN, ALARM, INTERLOCK. Conditions: WARN and ALARM are further specialized into

WARN_LOW, WARN_HIGH, ALARM_LOW, ALARM_HIGH, WARN_VARIANCE_LOW, WARN_VARIANCE_HIGH, ALARM_VARIANCE_LOW, ALARM_VARIANCE_HIGH.

Additionally, a new property alarmCondition has been added to the base device classes in both APIs. It holds the defining alarm condition of an device:

it evaluates to the most critical of all the property bound alarm conditions:

INTERLOCK>ALARM>WARN>NONE
unless a device developer manually set the alarmCondition field via setAlarmCondition(AlarmCondition condition). In this case this alarm condition is maintained if property conditions evaluate to a lower combined condition. This especially means that a manually set alarm condition will not automatically clear, it needs to be cleared using setAlarmCondition(AlarmCondition::NONE). It may however be raised by the

property bound conditions.

In addition the expected parameter interface was slightly altered. Previously existing attributes:

.alarmLow, .alarmHigh, .warnLow, .warnHigh

now have to be followed by .needsAcknowledging(true | false) indicating if the alarm needs to be acknowledged on the alarm service device. Optionally, they may be followed first by .info(string), giving more information on an alarm.

Additionally, variance base alarms are now supported. They can be enabled for readonly parameters via:

.enableRollingStats()

followed by the conditions:

.alarmVarianceLow, .alarmVarianceHigh, .warnVarianceLow, .warnVarianceHigh

each again optionally followed by .info and mandatorily by .needsAcknowledging.

The .enableRollingStats block is closed of by .evaluationInterval(interval), which gives the size of the rolling window.

Rolling statistics can accessed from device code for those properties they have been enabled for via getRollingStatistics(path). This returns a object of type RollingWindowStatistics, with methods:

update(val)
getRollingWindowVariance()
getRollingWindowMean()

If a property has rolling statistics enable may be queried via hasRollingStatistics() from the device.

Device States¶

The states in which a device can be, have now to be chosen from a list of states that Karabo provides. Both in C++ and Python, the allowed states are represented by an object of the class State. The specific states are accessed

via that class as well, i.e. for the INIT state one just uses State::INIT

in C++ and State.INIT in Python (both bound and middlelayer).

The list of available states can e.g. be looked in States

Defining the List of States for a Specific Device¶

For both bound Python and C++, the code to define the list of allowed states in the expectedParameters function looks almost identical, e.g. for bound Python it is:

OVERWRITE_ELEMENT(expected).key("state")
    .setNewOptions(State.INIT, State.STOPPED, State.STARTED, State.ERROR)
    .setNewDefaultValue(State.INIT)
    .commit(),

and in C++ the ‘.’ between State and the state name has to be replaced by ‘::’.

Defining Allowed States for Actions¶

Similarly, one defines the list of allowedStates for reconfigurable configuration parameters and for slots via:

.allowedStates(State.STOPPED, State.STOPPING)

Accessing and Updating the States¶

To access the current state of a device, use its getState method in C++ or simple self.get(“state”) in bound python.

The interface to update the state (needed only if using the default NoFsm) of a device has changed to take a State object as argument, e.g. self.updateState(State.HOMING) and this->updateState(State::HOMING), respectively.

Python Entry Point¶

The setup.py file in Python packages defines “entry points”. The values of them differs for bound and middlelayer Python devices. These entry points have been renamed and the setup.py file has to be adjusted accordingly:

Bound Python: from karabo.python_device.api_1 to karabo.bound_device
Middlelayer Python: from karabo.python_device.api_2 to karabo.middlelayer_device

Bound Python¶

Vector elements¶

It is not possible anymore to directly assign a numpy.array to a VECTOR_ELEMENT. Instead, the numpy.array has to be converted to a list or tuple, e.g.:

array = numpy.array(...)
self["vectorProperty"] = array.tolist()

Pipelined processing¶

The pipelined processing, i.e. the data flow between InputChannel and OutputChannel, has been redesigned with several interface simplifications.

Removal of Data class, image and array handling¶

In the past, data was passed as a Data object. This class has been removed and data is now sent as a usual Hash. Also the classes to pass multidimensional arrays and images have been re-desigend. In the past, they inherited from Data, but now they look like standalone classes. Whereas in the past ImageData inherited from NDArray, now it has an NDArray. How to construct objects of these classes (e.g. to avoid copies) can be found in the API documentation. Here is an example how to declare an output channel to send image data and a frame number in Python:

Define output:

def expectedParameters(expected):
   ...
    data = Schema()
    (
    IMAGEDATA_ELEMENT(data)
        .key("image")
        .commit()
    UINT32_ELEMENT(data)
        .key("frame")
        .readOnly()
        .commit()
    )
    (
    OUTPUT_CHANNEL(expected).key("output")
        .displayedName("Output")
        .dataSchema(data)
        .commit()
     )

Send data in a device slot:

imgArray = numpy.array(...)
self.writeChannel("output", Hash("image", ImageData(imgArray),
                                         "frame", frameNumber))

KARABO_ON_INPUT handlers¶

If your device registered a handler that receives an InputChannel as argument via the KARABO_ON_INPUT (and the KARABO_ON_DATA) macro, do not call update() on the InputChannel anymore.

Checklist for basic Karabo 2.0 Compatability¶

The following checklist gives guidance if you have considered significant changes when updating your device code. This does not mean that your device follows all 2.0 best practices - it simply will run.

General¶

Replaced string states with unified states in all occurances
Make sure the logger is not used to indicate alarms, use alarm conditions instead
Adapted any pipelined processing so that handlers take meta data as second argument
Replace all occurances of XXX_PTR and MATRIX_XXX with NDARRAY
Replace KARABO_GLOBAL_SLOT(…) with KARABO_SLOT(“*”, …)
Make sure that if you are using devices, device servers or signal slottable outside the usual, device-server hosted context, that you start an event loop and an event loop work thread before.

C++ - Specific¶

Update your Netbeans project to compile using C++11

Python Bound - Specific¶

Verify entry points are correctly set in setup.py
Batch replace and self._ss calls with self. If you get failures afterwards you are using the private interface of signal slotable and should contact a system architect

Python Middlelayer - Specific¶

Make yourself familiar with the new unified sync/async interface
Update to DeviceNodes whenever you have explicitly used connectDevice to mirror properties of a remote device in your expected parameters.