StreamBase 10.4.0 Release Notes

These were the release notes for the TIBCO StreamBase® component of TIBCO® Data Streams 10.4.0.

New Features

TIBCO® Data Streams 10.4.0 added the following updates and new features:

New Product Branding and License Terms

Starting with release 10.4.0, TIBCO StreamBase® and TIBCO® Live Datamart are now combined and licensed together into two new product licenses:

TIBCO® Streaming

TIBCO Streaming is the full set of streaming technologies, and includes TIBCO StreamBase®, TIBCO LiveView™, and the TIBCO StreamBase Studio™ development environment. See What is TIBCO Streaming for a complete list of included technologies.

TIBCO Spotfire® Data Streams

TIBCO Spotfire Data Streams is a subset of the TIBCO® Streaming package, configured as an add-on for TIBCO Spotfire® users, and licensed for the sole purpose of serving live data to TIBCO Spotfire. See What is TIBCO Spotfire Data Streams for the contents of this package.

A summary of the license term changes for these new product packages is found on the License Considerations page.

Initial work has begun to separately brand the Welcome screens in StreamBase Studio and some of the documentation. Whichever new product you have licensed, you can expect to see unchanged former names and terminology throughout Studio menus and dialogs, and in the documentation. This branding work is expected to proceed over the next several releases.

The TIBCO StreamBase® and TIBCO® Live Datamart independent products still exist for licensees of those products. Version updates and maintenance releases for StreamBase and Live Datamart will be fulfilled with installers branded as TIBCO Streaming. However, your Licenses for those products are not automatically upgraded to a TIBCO Streaming license. Contact your TIBCO Sales Engineer for further information.

New Studio Usability Enhancements

New Manage Project Dependencies Menu Option

StreamBase Studio now provides a new right-click menu option for a selected application or fragment project, StreamBase>Manage Project Dependencies. Invoke this option:

  • To add fragment dependencies for an application project.

  • To add fragment-to-fragment or fragment-to-Java dependencies for a fragment project.

This option presents an information dialog, and then it:

  • Opens the POM Editor for the selected project's pom.xml file.

  • Switches to the Dependencies tab of the POM Editor.

  • Opens the Add Dependency dialog.

See Add a Dependency for instructions on the application project case.

Clusters View Context Menu Now Supports Multiple Operations

You can now stop, remove, and forcefully remove multiple running nodes using the context menu by Ctrl+clicking nodes in the Clusters view.

New StreamBase Expression Language Function Added

A new StreamBase Expression Language Function was added, getResourceAbsolutePath(). This function returns the absolute file system path for a specified resource in a Studio project's src/main/resources folder.

Support Added for Disabling the Transaction Isolation Override Warning

By default, attempting to override the transaction isolation of a parent module in a child module generates a typecheck warning, because transaction isolation cannot be overridden by child modules. This warning can now be optionally disabled, as described on the Typechecking preferences page.

StreamBase Application Archive Wizard Enhanced

The StreamBase Application Archive Wizard was enhanced and now provides the options to:

  • Include debugging messages in the log output.

  • Save packaging steps logs to a directory.

  • Run tests during fragment and archive packaging.

New Error Dialog Appears When Launching Authentication-Dependent Fragments

If Studio fails to connect to a fragment during a launch because the fragment has authentication requirements, a new error dialog notifies you that Studio does not currently support such attachments.

Generated Unit Tests Enhanced

A (commented out) example of how to load a configuration file is now included in generated unit test cases.

New Adapters and Adapter Enhancements

Google Protocol Buffer Adapters Introduced

This release includes the following adapter suite:

See the Protobuf page for adapter details and the sample page for the adapter sample.

New Web Input Response Adapter Added

The TIBCO StreamBase® Web Input Response Adapter allows a StreamBase application to respond to a REST call that was handled by a linked Web Input Adapter.

New Operators Added to the WITSML Suite

Three new WITSML operators were added that allow for generic operations on any parent-child object structure such as logs, log rigs, rig wells, well, and so on. See The WITSML operator documentation for details regarding the new WITSML read, write, and delete operators.

In addition, the WITSML operator suite now includes an option to specify which WITSML version is used when performing operations against the server.

New eFTL Adapter Properties Added

The eFTL Input Subscribe and Output Publish adapters now include new properties, including a Unique Id property that appends a unique ID to a client ID. See the links above for additional property details.

New MQTT Adapter Properties Added

The MQTT adapters now include two new options to control MQTT connections: maxInFlight and automaticReconnect. For details, see:

Change Data Capture Adapter Includes New Options

The TIBCO StreamBase® Database Change Data Capture Input adapter now includes options to disable the startup of the SQL Server agent, as well as to disable the adapter on a table, at startup, if not already enabled.

H2O Operator Enhanced

The TIBCO StreamBase® Operator For H2O Model Evaluator was enhanced to include:

  • EasyPredictModelWrapper support for generated models

  • MOJO support

  • H2O library version update to 3.20.0.2

New Platform Support

Oracle JDK Version Updated

The Oracle JDK version bundled with StreamBase and Live Datamart is now JDK 1.8.0_u181.

StreamBase Now Uses System-Installed JDK 8 By Default

Starting with TIBCO Streaming release 10.4.0, StreamBase Studio detects and prefers a system-installed Oracle JDK 8 over its own bundled JDK. Using a system-installed JDK allows you to take advantage of cryptography features of the JDK that TIBCO is not licensed to include with TIBCO Data Streams, and allows you to manage your JDK installation independent of StreamBase. See Installing a JDK for details on this new feature.

StreamBase Studio Now Based on Eclipse 4.9

StreamBase Studio, StreamBase Manager, and LiveView Desktop are now based on Eclipse 4.9 (2018-09). This version includes changes to the Eclipse workspace format.

StreamBase Studio silently updates the workspace format. However, TIBCO strongly recommends importing your projects from previous releases into a new, unused Studio 10.4 workspace rather than trying to reuse a previous release's workspace.

LiveView Desktop users may be prompted to upgrade their workspaces.

See the Eclipse documentation for additional details on Eclipse 4.9 capabilities.

Maven Version Updated

The Maven version bundled with StreamBase is now 3.5.4.

Additional RPM Installer Requirements Enforced for Linux Install

The RPM installer for the streambase-server-*.rpm now includes the RPM Requires clause that works on all supported Linux operating systems. Also, if you unpack the RPMs and use the client and server RPMs independently with yum install, the required dependencies are installed.

Jetty Upgraded

In StreamBase Studio, the Jetty web server version was upgraded to 9.3.25. The upgrade applies to, for example, the TIBCO StreamBase® Web Server Request adapter and the TIBCO StreamBase® Web Server Response adapter, as well as for developing your own applications that require Jetty.

New epadmin Syntax and Features

Command Target Added for Dynamic Variable

A new epadmin command target, dynvar, was added to set and get dynamic variables. See epadmin-dynvar(1).

Command to Suspend Engines Added

A suspend command was added to the engine administration target to support stopping an engine in a suspended state for debugging. See this page for an overview of the engine lifecycle.

Command Added to Restore Initial Node Authentication

A reset authentication command was added to restore the initial authentication characteristics of a node.

Command Added to Generate and List HOCON Configuration Files

The epadmin generate and list commands were added to generate and list HOCON configuration files, respectively. See this page for details.

Commands Added to operator Target

The epadmin operator target now has the following new commands to support operator properties:

epadmin get operator [--engine=string] [--path=string] [--property=string]
epadmin set operator --property=string --value=string [--engine=string] [--path=string]
epadmin display operator [--engine=string] [--path=string] [--property=string] 
display container Command Includes Detailed Output Option

The epadmin display container command now includes a --detailed option. When set to true, the command output includes container content details.

display user Command now Displays Additional Security Details

The epadmin display user command now includes the following additional fields in its output. This information comes from the authentication realm with which the specified user is associated:

  • Authentication via Secure Communication Allowed

  • Password Always Required

  • Trusted Host Membership Required of Client

See this page for more information regarding security management.

display cluster Command Now Displays Addresses of Distribution Instances on the Node

The epadmin display cluster command now displays addresses of all distribution instances on the node.

display stream Command Now Includes Stream Option

The display stream command adds the --detailed option to display additional input and output stream information. Also, the command's --schema option is now deprecated.

display adapter Command Now Includes New Field Name

For the epadmin display adapter command, the output has a new field named Adapter, which contains the full output name of the adapter.

playback Command Now Includes Container Options

A new option was added to support starting playback on a specific container:

[--path=string] container name to target the command. If unset, defaults to the default container. Default: none

export artifact Command Now Requires a File Parameter

The export artifact command now requires a file parameter, which specifies a client-side file for the exported data. The file is overwritten if it already exists.

New Commands Added to operator Target

The epadmin operator target now comes with four new state management commands:

  • epadmin suspend operator [--engine=string] [--path=string]

  • epadmin resume operator [--engine=string] [--path=string]

  • epadmin stop operator [--engine=string] [--path=string]

  • epadmin start operator [--engine=string] [--path=string]

New Administration Target Command Added

This release adds administration target support for creating client-side output files via these commands:

  • export configuration

  • generate configuration

  • export logging

  • backup realm

  • export security

New Security Enhancements

TrustedHosts Configuration Enhanced

Multiple independent TrustedHosts configuration files are now supported within a node's security configuration (with one active at a time still enforced).

TrustedHosts configuration files configuration files now support IPv6 CIDR block notation.

Clients Not Specifying Username Now Defaults to Operating System's Logged-in Username

If a StreamBase client connection does not specify the username, a default username is obtained by calling System.getProperty("user.name"). This is used as a convenience, albeit a potentially unsafe one. For default security information, see this page.

New API Enhancements

New Java API for Discovery and Publishing Services Added

The API was updated for service discovery and publishing. This API is used to support service discovery in existing Java APIs and also to allow applications to publish their own service records.

API Updated for Notifications

New API methods were added to the StreamBase API to support quorum notification and partition notification:

  • remoteQuorumLost — provides notification when a quorum lost state occurs.

  • partitionActive and partitionNotActive — provides notification when state changes occur for each partition; multiple PartitionNotifiers for each Partition are supported (for example, one Notifier for each Partition on each engine).

Miscellaneous Enhancements

Artifact Distribution Service Improved

The Artifact Distribution Service (ADS) now includes the following enhancements:

  • Improved handling of large model files by providing a file cache option in lieu of model storage via transactional memory.

  • Artifact replication across multiple nodes in the cluster. This ensures that the artifacts remain available in the cluster if nodes are removed from the cluster. Artifact replication is always enabled. You can use ADS to configure the number of replica nodes used (defaults to two) and the nodes used as replica nodes (defaults to all nodes in the cluster).

System Properties for Refactoring Now Supports Simple Method Names

The streambase.codegen.broken-method-names system property, used for refactoring all or individually generated methods, now supports simple (as well as fully qualified) method names. See this page for use-cases and details.

JVM Tuning Parameter Added

An engine.shutdownTimerSeconds parameter was added, which you can apply to StreamBase, LiveView, and Java engines.

Default Availability Zone and Data Distribution Policy Added for Improved Fragment Launches

A default availability zone default-cluster-wide-availability-zone and an associated default data distribution policy default-dynamic-data-distribution-policy were added to expedite fragment launches by saving configuration time. These are selected by default in:

See this page for the default availability zone and data distribution policy settings.

Changes in Functionality

StreamBase 10.4.0 included the following changes in functionality:

Docker Image Creation Changes

Simplified Docker Image Support

The File>New>StreamBase Project wizard is more streamlined, with fewer decisions to make when creating a Docker image to contain your StreamBase or LiveView application. When creating a project for a StreamBase Application, a single check box now selects the option to create your application as a Docker image. See Creating and Running Docker Images for newly written instructions for this procedure.

Docker Run Commands Now Require Explicit NODENAME Setting

As of release 10.4.0, when starting your Studio-generated Docker container with the docker run command, you must now explicitly specify a node name for the StreamBase Runtime node in your container, by using the -e NODENAME=node.cluster or --env NODENAME=node.cluster options. The following shows a complete command:

docker run -d --name firstapp -e NODENAME=A.cluster -p 2000:2000 sample/docker_1st

See Creating and Running Docker Images for updated Docker image creation instructions.

Studio Changes

Studio Configuration Directory Shared by Maintenance Releases

In previous releases, the StreamBase Studio Configuration Directory was named with a three-position version number, such as 10.2.0. This meant that every maintenance release, such as 10.2.1, got a separate configuration directory. For customers who added Eclipse plug-ins to Studio, this convention required them to reinstall those plug-ins with each maintenance release.

Starting with release 10.4.0, the configuration directory is named with a two-position version number such as 10.4. This means that all maintenance releases for each major.minor release can share the same Studio configuration settings, including Eclipse plug-ins added to Studio.

This change does not refer to the Studio workspace directory, whose default location naming syntax has not changed. TIBCO does still recommend using a new, empty workspace for each major.minor release, and importing your projects from the previous release to the new workspace.

LiveView Web Now Added as Dependency for new LiveView Projects

Generation of LiveView projects using the Maven archetype now, by default, now includes the LiveView Web WAR file as a dependency. You can disable this by setting the archetype property lvweb to false.

Node Not Running Error Message Improved

The Node is not running message that is returned when a node is started, but no target engines are available was improved to distinguish between the node that has not been started, compared to when there are no running engines.

Logging Behavior Changed

Logging Now Supports Transaction ID as a Pattern Element in Logs. The keyword %transaction in a Logback configuration file places the current transaction ID, if any, into the log record for the current event. If no transaction is active, none is inserted; if there is an active transaction, but the identifier cannot be determined, unavailable is inserted. See StreamBase Logback Extensions for further information.

Additionally, when the default duplicate suppression filter is in place, the suppressed-message log event uses the same category and level as that of the suppressed messages. See Default Logging for default logging information.

Also note that duplicate message filtering is always in place. If you specify a filter, the filter is used. Otherwise the default is used.

Legacy log level integer settings used in StreamBase releases before 10.2.0 were removed in the Advanced tab of the EventFlow Fragment launch configuration.

Docker container logging is now supported.

Round-Robin Data Mappers Now Support Dynamic Data Distribution Policies

Round-robin data mappers specified in StreamBase Runtime Application configuration files now support dynamic data distribution policies, in addition to static ones.

HOCON Editor Improved

The HOCON Configuration File Editor now includes the option to search by HOCON property name to assist in locating the configuration file type to use.

Adapter Changes

Web Input Adapter Changes

The TIBCO StreamBase® Web Input adapter's ResponseId field is renamed to SB_ResponseId. For more about the property, see Web Input Adapter.

Additionally, the adapter now handles HTTP metadata such as headers and cookies.

Command Changes

epadmin Load Command Options Removed

This release removes the forcelocal option from the load configuration command. Providing a configuration file to the configuration target is always local to where the administration client is running, which may or may not be the same as the node.

Also removed from the command is the debug option. Use normal logging support to enable debug tracing in HOCON.

Deprecated and Removed Features

StreamBase 10.4.0 had the following deprecations.

XML Normalizer Operator Deprecated

The XML Normalizer operator is now deprecated. Applications that use the operator continue to compile and run, but the operator no longer appears in the Operators and Adapters dialog. Documentation for the operator and its sample were removed.

References to callcpp() Function Removed

StreamBase 10 does not support custom functions written in C++, but references to the callcpp() expression language function were still present in Studio function assistance. Those references were removed in this release.

Deploy Parameter for mirrorclient Option Now Removed

The deploy parameter mirrorclient for Run Configurations is deprecated and removed in this release, including its documentation.

Command to Display Trusted Hosts Deprecated and Replaced

The epadmin display security --type hosts command is deprecated. Use epadmin display trusted instead.

excludedProtocolPatterns Security Property Deprecated

The SecureCommunicationServerProfile and SecureCommunicationClientProfile root objects' excludedProtocolPatterns property is now deprecated and still supported for backward compatabilty.

Supported External JDBC-compliant Databases Updated

The list of supported external JDBC-compliant databases was updated. The following databases are no longer supported:

Oracle 10g
Microsoft SQL Server 2008
Sybase® ASE V12
Sybase IQ V12.7
Thomson Reuters Velocity Analytics 6.9

Migration and Compatibility

API for Timestamps Changed

The Timestamp.INTERVAL_FORMAT public object was changed. Instead of com.ibm.icu.text.DecimalFormat, the API now uses java.text.DecimalFormat. If you are using the old INTERVAL_FORMAT object, you must change your import dependency to java.text.DecimalFormat.

If the ICU enhancement methods were used with the INTERVAL_FORMAT, you must find an alternative way of realizing similar functionalities, as the new API does not support the ICU enhancement methods. See this link for specific ICU enhancement methods.

StreamBase Resolved Issues

This section provides a list of errors corrected in release 10.4.0 of the TIBCO StreamBase® component of TIBCO® Data Streams.

Fixed in 10.4.0
Number Resolution
TIBCO StreamBase® 10.4.0 incorporates all fixes resolved in the release 10.0 series through 11.1.1 and all fixes resolved in the release 7.7 series through 7.7.5.
SB-44922 The Query operator was incorrectly failing to return all rows when a query was executed, and when the rows only differed in tuple sub-field values. This was resolved.
SB-44907 Studio inadvertently swapped the cluster and node names in the service name, when launching a Java fragment. This was fixed.
SB-44748 This release fixes an issue that caused the Diameter adapter message types CEA and CER to incorrectly report the Auth_Application_Id list, when a vendor was not linked.
SB-44740 This release fixes premature failover when one engine of a multi-engine application is stopped.
SB-44739 The WITSML Trajectory Write operator was not inserting trajectory stations due to lack of UID being passed. The operator now properly uses the UID field.
SB-44726 When deploying an application, the epadmin install node command failed if a substitution variable value had multiple spaces:
epadmin install node nodename=A.X  application=myapplication.zip 
substitutions="MYNODE=My First Node"
Unable to deploy application myapplication.zip: 
Error: invalid character "N" in variable name

This issue was resolved.

SB-44718 In Studio, right-clicking on a package and selecting New>Module would incorrectly select the top-level package for the application. This was fixed.
SB-44635 If an orphaned node directory exists during a fragment launch, a subsequent launch would fail with an error in a (node install) line in the Console view.

A node directory is the directory used to store the state information for a node used a fragment. Such a directory can become orphaned if the processes managing somehow die or are terminated abnormally, and therefore the node directory is not cleaned up as it normally would be.

The problem is now fixed. A subsequent launch detects the orphaned directory and prompts whether it should be removed. When confirmed, the orphaned node directory is removed and the launch continues successfully.

SB-44589 When there is an exception during the start of the LiveView fragment, the dialog displayed after the launch is started did not dismiss itself. That problem is now fixed. The dialog now dismisses itself and the output from the failed launch is displayed in a separate dialog.
SB-44575 An issue prevented epadmin logging commands from working when a node was stopped. This was fixed.
SB-44458 This release fixes an exception error from the eFTL adapter that could occur when the adapter tried to subscribe when not connected to the server. The adapter no longer reports an exception if not connected when trying to subscribe. It now reports an error status message that you are not connected and must connect first.
SB-44327 In some circumstances, running the display partition command on a node failed with an NPE after an application was deployed. This was fixed.
SB-44310 The Web Input and Web Output adapters can now set a defined context path.
SB-44306 The epadmin enqueue stream command Help incorrectly reported input as a required parameter. This was fixed; input is now listed as optional.
SB-44240 An issue caused the H2O adapter to fail loading an unsupervised classification model. This was fixed.
SB-43910 This release fixes an issue that prevented the HOCON editor from automatically generating sample EventFlowDeployment configurations.
SB-43061 The TIBCO BusinessEvents Fraud Detection sample was upgraded. The sample no longer requires you to convert it from StreamBase 7 to StreamBase 10 format.
SB-42408 Using dynamic partitioning for the Artifact Distribution Service (ADS) caused duplicate key errors when loading artifacts, if the artifacts were not replicated to all nodes in a cluster. This was fixed; ADS now supports artifact replication.
SB-42407 The Artifact Distribution Service (ADS) could incorrectly report existing artifacts as missing and create duplicate artifacts after a node is added or removed from a cluster, if high availability is not configured. This was fixed; ADS now supports artifact replication.
CQS-4583 In some cases, the StreamBase HOCON editor's auto-complete function would fail to suggest the appropriate HOCON property. This was fixed.