What’s New in Karabo 2.4.2

The hotfix release for devices interacting with our the doocs library. Previously, the filtered Schema would cause an exception due to table elements.

Detailed Merge-Log

Bound/cpp core

  • C++/Bound: Fix bug in handling TableElement by HashFilter class !3503

What’s New in Karabo 2.4.1

The hotfix release for Karabo pipelining includes many stabilization fixes and a new feature to visualize pipeline connections in output channels with a table element.

  • fix bound Python device client (‘bad weak_ptr’ => no ageing),
  • fix bound Python SignalSlotable::connect not releasing global interpreter lock (fixes pipeline connection problems)
  • feature: output channel connections table (all APIs)
  • input channel id includes channel name (in addition to device id)
  • fix deduction of “remoteness” of remote input channels with same process id
  • fix “queue” behaviour of pipelines wrt. copying data
  • performance fix when pipelines have to copy data (i.e. “queue” or local [same process] receiver)

Detailed Merge-Log

Bound/cpp core

  • C++: Visualize the connections made to OutputChannel object !3425
  • C++: Bug and performance fixes for BufferSet copies !3460
  • C++: As copyLocal, queueing requires to copy the data !3464
  • C++/bound: Add input channel key to id of input channel !3426 !3433
  • C++/bound: Fix sub-permille chance to screw up pipelining !3451
  • Bound: Make bind_weak work in DeviceClient’s C++ object (for 2.4.1) !3421
  • Bound: Release GIL when connecting to a signal !3463
  • Fix pipeline integration test wrt changed remoteId !3434

Middlelayer core

  • MDL: Send channel name in input channel for tracking !3404 !3432
  • MDL: Track channel connections in output channel !3420

What’s New in Karabo 2.4.0

The most relevant improvements, fixes and changes are:

  • GUI: Older GUI clients are not compatible with GUI servers of Karabo 2.4.0: They will not receive pipeline data. Request - reply protocol!
  • GUI: Server forwards timestamp for pipeline data. The timestamp for images is now correct.
  • GUI: Addition of Weblink widget (in scene tools)
  • GUI: Addition of Wheelbox widget
  • GUI: Addition of editable options box widget
  • GUI: Addition of timestamp widget
  • GUI: Cinema enhancements with direct connect, username and multiple scenes
  • GUI: No interpolation in image widgets by default
  • GUI: More scene items are state aware
  • GUI: Align fonts and style across different OS
  • GUI: Segmentation fault fixes in Tablelement and AlarmPanel
  • GUI: More drag & drop functionality from project panel: Scenes can be dragged on scenes
  • GUI: Implementation of the first device topology (domain - type - member)
  • GUI: Print support for each panel
  • GUI: Tooltips of last received data for image widgets
  • GUI: Enable of wheeling of widgets into the Karabo GUI - Extensions
  • MDL: Proxy (e.g. in ikarabo) uses now meta data timestamp for pipeline data.
  • MDL: Autostart implemented in device server to enable core devices startup
  • Core Devices: The ProjectManager is now a MDL device!
  • Core Devices: GUI Server now caches and forwards logs with loglevel INFO
  • C++: Several pipeline improvements and stabilization/fixes to race conditions
  • C++/bound: Several pipeline and Tcp communication improvements/stabilizations including fixes to race conditions and indirect memory (“chunk”) leakage
  • C++/bound: Fixes to uuid handling for asynchronous slot replies and for the request(..).receive[Async](..) patterns
  • C++: DataLoggers publish when last data arrived

Minor backward incompatibilities:

  • Invalid schema elements will not be ignored anymore, but trigger a LogicException. The specific cases are a leaf added to a non-existing node, a leaf where the parent is not node, and a leaf added to a list of nodes.
  • OLD GUI Clients (2.3.1) lack a new server protocol message and will not work with Karabo 2.4.0

Detailed Merge-Log

Bound/cpp core

  • Bound: Python Karathon SignalSlotable Type Handle Test !3038
  • Bound: Simplify wrapper code !3212
  • Bound: allow setSpecialDisplayType on Nodes !3284
  • Bound: removed gui elements from karabo-cli !3283
  • Bound: API cleanup code quality (PEP8) !3257
  • Bound: give client access to output channels !3156
  • Bound integration tests: fix device providing scene test !3172
  • Bound: Establish code quality in with test and flake all unit tests !3319
  • Karathon: unify proxy methods !3160
  • C++/bound: Avoid uuid clashes in (a)sync communication !3302
  • C++/bound: Protect slot execution from parallel registration of more methods !3307
  • C++/bound: Bugfix devclient extractcommands !3305
  • C++/bound: Fix inputchannel race read processing !3178
  • C++/Bound: Add NDArray constructor from an iterator range !3154
  • C++/bound: More mature DeviceClient test, init only timestamp for “pid” !3173
  • C++: More mature history integration test !3165
  • C++ DeviceClient must keep order about instanceNew and instanceUpdate !3247
  • C++: Don’t use unordered_map if we erase iterators while looping on map !3299
  • C++: Set custom display type for nodes !3273
  • C++ OutputChannel: Single queue for load-balanced shared inputs. !3100
  • C++: More logs and few fixes for p2p shortcut !3331
  • C++: Epochstamp: Add operator<< for output to ostream !3077
  • C++: ‘Fix for failure of testPipeMinData when sender delay is set to zero. !3179
  • C++: Provide ostream operator for timestamp !3352
  • Fix serialiser performance comparison !3241
  • C++: karabo/util must not include other karabo parts !3334
  • C++: Fix output channel chunk leaks when input/output channels disconnect. Looks like not the full story is covered… !3361
  • C++: Fix various race conditions in Tcp !3380 !3390
  • C++: Fix races and more chunk leaks in pipelines !3381 !3390
  • C++: Fix pipeline chunk leakage - though not fully for local connectionCertainly not an elegant solution, but OK for 2.4.0 now. !3385

Core Devices

  • ProjectManager becomes native !3233
  • GuiServer: Gui server send pipeline data only on request !3315
  • GuiServer: Optimize device instantiation !3310
  • GuiServer: Agressively avoid one copy when forwarding pipeline data to GUI !3322
  • GuiServer: reduce lower boundary for property throttling !3340
  • GuiServer: Forward time stamp of pipeline data to clients !3323
  • DataLogger: Publishes timestamp of last recorded data !3335
  • GuiServer: Fix two bugs in gui server pipeline connection handling !3296
  • GuiServer: Use minInc for time intervals !3301
  • GuiServer: Treats completely failing instantiate request !3145
  • GuiServer: Cache and forward logs !3204
  • GuiServer: loglevel default to INFO !3217
  • GuiServer: Timestamp from meta data to client !3351
  • DataLogger: ‘Fix history retrieval (RedMine 13865) !3249
  • PropertyTest: Add path elements !3258
  • RunConfigurationGroup: Table combobox !3213
  • Alarm Service: Move logs to debug without sending !3218
  • DataLogger: Proper log message in slotGetConfigurationFromPast if archive_schema.txt is missing. !3174
  • DataLogger: Ensure time ordering of full configuration (RedMine 36581) !3313
  • DataLogging_Test: Add NO_THROW around synchronous calls. Better diagnostics if it fails the next time with an uncaught exception !3142
  • Core: Move often occuring logs in Device.hh to debug !3220

Dependencies and deployment

  • FIX: colorful karabo-check always returns !3149
  • FEAT: Added Noded slot integration test !3122
  • CI: remove previous files from CI directory! !3175
  • TEST: Implement SIGSEGV flakyness for gui tests !3216
  • Fix all shebangs once and for all !3089
  • Code quality in Karabo common !3269
  • Upgrade to daemontools 1.11 !3141
  • DEPS: Daemontools stop state is fixed !3371
  • wrapping output of scene2py into get_scene(deviceId) function !3365

Middlelayer core

  • MDL: Convert python float to Timestamp in MDL !3378
  • MDL: Forcing unsigned shape in native NDArray Schema and toDataAndAttrs() !3367 !3372
  • MDL: Train ID ticking in Python 36 !3348
  • MDL: Accept read and send chunk swaps in the MDL output channel !3347
  • MDL: Fix broken connection for pipelines !3350
  • MDL: Add unit tests for slotGetOutputInformation, slotHasSlot and runtime schema attributes !3036
  • MDL: Account for units in table elements properly !3164
  • MDL: Proxy factory fits documentation !3166
  • MDL: Add removeQuantity decorator !3071
  • MDL: Harmonize startDevice return value with c++ and bound format !3270
  • MDL: Implement waitWhile correctly !3306
  • MDL: Fix Vectorstring in Table !3201
  • MDL: Add timestamp to TableElement !3097
  • MDL: Flake the init files and provide Hash and timestamp in CLI !3291
  • MDL: Fully implement NDArray !2890
  • MDL: Add property test MDL device to framework !3259
  • MDL: Overwrite integers with Enums correctly !3254
  • MDL: Runtime attributes !3158
  • MDL: Unittest for slotResubmit alarms !2839
  • MDL: Safety cast on table values !3214
  • MDL: Autostart devices in device servers !3230
  • MDL: Provide error string and not list on error as first arg !3146
  • MDL: Add meta data timestamp to proxy when pipelining !3150
  • MDL: Remove renaming in DeviceNode !3113
  • MDL: A little cleanup in the device server !3078
  • MDL: Move injectable to device class !3118
  • MDL: unit test for units and alarms !3151
  • Enhance MDL Test with Code Style !3195
  • MDL: OMIT policy in base classes !3123

Graphical user interface

  • GUI: Update the wheelbox widget. Add test for min and max update and calculate… !3375
  • GUI: Update Dialog improvements !3343
  • GUI: Show wait cursor while loading topology !3383
  • GUI: Shutdown device in device topology !3382
  • GUI: Introducing Karabo Theater !3262
  • GUI: Adjust QPADDING to new QToolbar border size in scenepanel !3370
  • GUI: Notify client - notification message box !3355
  • GUI: Implement Timestamp widget !3366
  • GUI: Enforce all toolbar borders and set the size of icons on the configurator !3344
  • GUI: Set a fixed width for buttons in the search panel !3345
  • GUI: Change the label widget to account for new style !3357
  • GUI: Data handler meta hash correction !3359
  • GUI: Close pop up widgets in navigation panels !3358
  • GUI: Fix windows build !3360
  • GUI: Add checker to dash offset setting !3363
  • GUI: Monkey patch the tooltip event for image widgets to show the time… !3364
  • GUI: Show big data processing with cheat sheet !3362
  • GUI: Reliably save settings !3162
  • GUI: Fix tooltip on scene link !3138
  • GUI: Links can have context menu edit !3139
  • GUI: Speed up macro event and fix scrolling in macro output !3182
  • GUI: Enable workflow items from project panel !3197
  • GUI: Let the server know what the client processes for big data !3248
  • GUI: Fix segfault in AlarmPanel !3183
  • GUI: Show attribute icons in the configurator !3187
  • GUI: Custom Context for Script Console and Scrolling !3186
  • GUI: Fix Icon dialog scrolling !3189
  • GUI: Spread constants in configurator utils !3191
  • GUI: Fix Digit Icon Widgets and the default data setting !3190
  • GUI: Fix icon widgets not handling undefined and not updating properly !3188
  • GUI: Enhance the cinema with multiple scenes and direct gui server connect !3260
  • GUI: Fix table number options !3261
  • GUI: Set focus on vector controllers correctly !3267
  • GUI: No whitespaces allowed in floating point numbers !3268
  • GUI: Account for state updates on the scene for number line edits and booleans !3263
  • GUI: Fix message box modal dialog !3266
  • GUI: Combobox listens to state updates !3264
  • GUI: Take topic information from configuration singleton !3256
  • GUI: Remove recalculcation of layout width/height for Label widgets !3285
  • GUI: Don’t show log and console on startup !3272
  • GUI: Instantiate all devices in a given project !3290
  • GUI: Improve the device class proxy schema request !3287
  • GUI: Notify of missing servers in instantiate all !3292
  • GUI: Build and check Leafs first in binding !3300
  • GUI: Improve-update-dialog: Console Entrypoint and Dialog Responsiveness !3304
  • GUI: Fix the toolbar borders -> Windows !3308
  • GUI: Set clean looks style !3303
  • GUI: No interpolation in image widgets on default !3309
  • GUI: Implement device topology in previous navigation panel !3298
  • GUI: Add image widget icons !3312
  • GUI: Allow to change widget options with multiple proxies !3311
  • GUI: recovered windows build !3314
  • GUI: Icons for widget handler actions !3318
  • GUI: Use timestamp from fast data !3324
  • GUI: Remove wrong parent setting for message box in manager !3328
  • GUI: Move number validators from controllers !3330
  • GUI: Add a table schema update in the controller !3338
  • GUI: Add double wheelbox to the controllers !3339
  • GUI: Update dialog align size of dialog and buttons !3341
  • GUI: Adjust splitter ratio stretch !3342
  • GUI: Provide context menu for device topology !3329
  • GUI: Add wheel icons !3333
  • GUI: Add icons to the closable menu bar !3293
  • GUI: Remove multiple inheritance for search panels !3276
  • GUI: Editable Option Combobox !3236
  • GUI: Refactor message boxes !3280
  • GUI: Close property about dialog on disconnect !3282
  • GUI: Notify missing configuration when instance is gone during saving !3277
  • GUI: Fix project selection focus !3222
  • GUI: Cleanup the list validator !3228
  • GUI: Enhanced scene widget context menu !3232
  • GUI: Move list edit dialog to dialogs !3239
  • GUI: No mutation for offline properties !3238
  • GUI: Rework context menu triggers and editing of Table Element !3234
  • GUI: Update edit field when edit list dialog is accepted !3243
  • GUI: Enable wheeling of widgets into the Karabo GUI - Extensions !3237
  • GUI: Refactor navigation double click event - handler list !3203
  • GUI: Fix TableElement: Readonly checkboxes don’t cause segfault !3202
  • GUI: Workflow items are able to show devices via key stroke click !3194
  • GUI: Show missing status for workflow devices on scene if missing !3192
  • GUI: Refactor widget actions for non-controller widgets !3227
  • GUI: Remove redundant schema update check on table element and clean up !3226
  • GUI: Move scene link dialog to dialogs !3229
  • GUI: Provide option to disable alarms in sparkline !3196
  • GUI: Refactor device handle dialog !3152
  • GUI: Remove host node in navigation panel !3199
  • GUI: Widget handler factory in the scene view !3206
  • GUI: Performance boost for binding !3205
  • GUI: Bring workflow back to working !3181
  • GUI: Correct scene handler import and action !3208
  • GUI: Allow workflow items to be dragged by devices !3193
  • GUI: Drag and drop scene items onto the scene from a project panel !3215
  • GUI: Check mark first configuration in default project device !3085
  • GUI: Provide configuration view !3060
  • GUI: Weblink widget !3132
  • GUI: Enable print support for panels !3128
  • GUI: Forward compatibility for init info from gui server !3140
  • GUI: Show gui server hostname !3133
  • GUI: Fix various color dialog bugs !3137
  • GUI: Fix text dialog !3147