StreamBase Release Notes

The security model in StreamBase 10.3.0 is a significant upgrade over previous releases, providing greater security without sacrificing ease of use during development. See StreamBase Security Model for details. New features include:

Some behavior changes are the result of this upgrade:

StreamBase 10.3.0 introduces a new way to manage StreamBase-provided JAR files and other artifacts for consumption by its Maven build system.

TIBCO has always recommended against trying to use a Studio workspace from a previous release in a new major release. However, this recommendation is especially important for the StreamBase 10.2 to 10.3 transition because of 10.3's new repository handling described above.

DO NOT OPEN StreamBase 10.2 project folders in Studio from StreamBase 10.3. See Migrating from 10.2 to 10.3 for further information.

StreamBase 10.3.0 introduces the epdev command, which allows you to develop on a machine without a Studio installation, as in a devops environment. Use this command to unpack the shipped Maven artifacts into a machine's local repository. You can also use epdev to correct repository errors, to write the StreamBase artifacts to a non-default repository, to configure a development laptop for offline use, and other tasks. See the epdev reference page or epdev --help for further information.

As part of the artifact handling change described above, StreamBase 10.3 no longer requires or uses the TIBCO_EP_HOME environment variable, either by Studio or command-line tools. There is no harm in leaving the variable set, but it is not used.

However, StreamBase 10.1 and 10.2 still requires that variable. If you are maintaining old and new Studio versions, make sure any shell startup files or Windows scripts set that variable correctly for use only with StreamBase installations before 10.3.

StreamBase 10.3.0 introduces support for running StreamBase Applications in StreamBase Studio. Previous StreamBase 10 releases required Apps to be run from the command line with the epadmin command. While epadmin deployment remains the gold standard for production deployment, launching Apps in Studio allows you to test or debug an application consisting of two or more fragments.

While a StreamBase Application is running, you can stop and restart it from the right-click menu in the Debug view.

This release introduces a new Run Configuration type named StreamBase Application. The minimum requirement for this configuration is an application archive (whose filename usually ends with *ep-application.zip). This can be generated in Studio with the wizard described next, or on the command line with mvn package.

Once you have an application archive file, right-click the filename, and select Run As → Run Configurations. In the Run Configurations dialog, create a new StreamBase Application configuration and use the first Browse button on the Main tab to locate the your *ep-application.zip. Then click Run. See Run StreamBase Application in Studio.

As an aid for running StreamBase Apps either with epadmin or in Studio, this release includes a new wizard that first generates fragment archives for all dependent fragments, and then generates an archive for the StreamBase App.

In previous releases, you had to run Maven Install on the fragment projects you wanted to encapsulate in a StreamBase Application project, and then Maven Install again on the App project. The new wizard reduces those steps to a single menu selection.

When attempting to locate projects to be designated as Maven dependencies, Studio now looks in the current Studio workspace as well as the local Maven repository. This allows Studio to locate projects without having to run Maven Install on them first.

The epadmin command now supports Gnu-style parameter syntax with two hyphens prefixing parameter names. You can enter as few letters of each parameter's name that preserves that parameter's uniqueness among the current epadmin target's supported parameters.

Syntax from previous releases is still supported. In this format, parameters are entered in full without abbreviations and without a prefixed double hyphen. You cannot mix old and new syntax in the same command. See the epadmin reference page for details.

The usage text for epadmin commands is greatly expanded and clarified, and you no longer need to connect to a node before seeing usage text. Start with epadmin help, epadmin help targets, and epadmin help globals. Thorough usage text is available for all targets by using epadmin help targetname.

The returned error message was enhanced for cases when you enter a partial, non-unique parameter for a given epadmin command. The error message now returns the name of conflicting parameters, and also suggests a partial name for the parameter that could enable a successful command execution.

This command verifies that the UDP broadcast mechanism used by StreamBase Runtime discovery services is working normally on the local subnet. See epadmin --help verify services for details.

This command emits new messages as they are added to an engine's log stream. For further notes, see Tail Logging.

This release shows a new HOCON type selection dialog when you run File → New → StreamBase HOCON Configuration File. This dialog organizes HOCON file types in root object categories and lets you enter a search string to narrow the list to matching root objects.

The Console view was enhanced to allow you to set the level of information displayed when running fragments or applications. When the Console view has fragment or application output to display, a new Format menu in the Console view toolbar sets one of three options: Verbose, Concise, or Minimal.

The Format menu is a shortcut for selecting one of the same preferences from the Launching panel of Studio Preferences. Concise is Studio's default in StreamBase 10.3.0; in previous releases, the Console always showed Verbose format. For a description of these settings, see Launching Panel.

In previous 10.x releases, the Properties panel of the Clusters view was always placed on the right side of the view. Starting with this release, Studio provides an option to locate the Properties panel on the right or below the Clusters, Nodes, and Engines panel. The new default is below.

When one fragment project has a Maven dependency on other fragment projects, a new folder named StreamBase Fragment Project Dependencies appears in the Project Explorer view to show the dependency relationships.

This release includes the following JDBC configuration connection options:

See StreamBase JDBC DataSource Configuration and JDBC Data Source Overview for a discussion of using the JDBC Table data construct and one or more Query operators to access an external JDBC data source from an EventFlow module.

Studio now provides the ability to copy the schema of a selected input stream, named schema, or arc to the system clipboard, in either JSON or XML format. This feature allows you to copy a schema ready to paste into the Create Artifact dialog of the Artifact Management Server. See Exporting Schemas to Clipboard for details.

This release adds the great_circle_distance() function to the StreamBase expression language.

StreamBase and Live Datamart are now supported for build or staging servers and for production deployment on:

The Oracle JDK version bundled with StreamBase and Live Datamart is now JDK 1.8.0_u171.

StreamBase 10.3.0 and later ships with the open source mvn command installed. If you use a StreamBase Command Prompt on Windows or a StreamBase-configured shell on macOS or Linux, the mvn command is in the PATH, as is the java command required to run mvn.

StreamBase now supports running a StreamBase application as a Windows service, including support for using event logging. See Configuring Windows as a Service for details.

OpenShift Container Orchestration Environments: Cluster auto-scaling is supported using Kubernetes Horizontal Pod Autoscale (HPA) 1.7. See Supported Docker Environments.

In StreamBase Studio, the Jetty web server version was upgraded to 9.3.23.v20180228. The upgrade applies to, for example, the TIBCO StreamBase® Web Server Request adapter and the TIBCO StreamBase® Web Server Response adapter, as well as for developing your own applications that require Jetty.

The adapter named CME Order Routing in previous releases was removed and replaced with the CME iLink Order Routing adapter. This new adapter has a different usage model, which requires applications using the previous version to be modified accordingly. The new adapter includes:

See CME iLink Order Routing Adapter for configuration details and CME Order Routing Adapter Sample for a sample of adapter usage.

The adapter named CME Market Data Platform FIX/FAST Input adapter in previous releases was removed and replaced with the CME MDP3 adapter. The new adapter allows StreamBase to receive data from a CME Market Data Protocol (MDP3) connection, and now supports explicit IP address binding for use on multi-homed systems, without requiring additional operating system configuration.

See CME MDP3 Adapter for configuration details and CME MDP3 Adapter Sample for a sample of adapter usage.

This release includes the following new eFTL adapters:

The TIBCO StreamBase® Output Adapter for Apache Kafka Consumer Commit allows StreamBase applications to perform commit operations for a given consumer operator. The commit can be a commit of all topics and partitions or be a more granular approach and specify the topic, partition, and offset to commit. See Kafka Consumer Adapter for more information.

This release includes the following new web adapters:

See Web Streams Adapter Sample for a sample of the adapters' usage.

The TIBCO StreamBase® Task Scheduler adapter allows Studio to fire events at specific intervals. See Task Scheduler Adapter Samples for sample usage and Task Scheduler Adapter to learn more about its properties.

In previous releases, in a Run Configuration dialog for an EventFlow fragment, you could select from among the configuration files in the src/main/configurations and src/test/configurations folders for the current project only. Starting with 10.3.0, you can select configuration files from other projects in the workspace as well. This feature is a convenience for short-term testing only. Configuration files designated this way do not automatically become part of the current project's archive.

In the Properties view for all global Java operators and adapters, in the General tab, previous releases combined the operator or adapter line with the operator or adapter class name into a single long line. Starting with 10.3.0, the fully qualified class name is on a line of its own. As before, you can select and copy the class name for pasting elsewhere, such as into a Java JUnit file.

Only on Windows installations, certain command-line tools were moved to the bin64 directory of your StreamBase installation. If the PATH contains both the bin and bin64 directories (as provided by the StreamBase Command Prompt), you can expect no changes. The commands that moved are:

In previous releases, creating a fragment project in Studio would automatically generate a fragment.conf file in src/main/configurations as a configuration placeholder. Studio no longer generates this file for new fragment projects. In most cases such fragment.conf files contained nothing meaningful to the project being created. You can still generate a Fragment Definition HOCON file manually and configure it as required.

StreamBase and Live Datamart trusted host configurations now support partially qualified domain names and IPv6 addresses. See Trusted Hosts for more information.

In previous releases, if you used the fragments object under a nodeTypes definition in the ApplicationDefinition root object of the application HOCON type, you had to separately list the top-level fragment and all of its dependent fragments (if any). Starting with StreamBase 10.3.0, you only need to specify the top-level fragment in a node type's fragment list, and transitive dependencies are automatically included.

The memorySizeBytes property of the ApplicationDefinition configuration type configures the size of transactional memory for a node. That property now has a minimum value of 512 MB, which has also been its default value. Previous releases allowed you to configure a lower value for this property.

Sparse partitions were renamed to foreign partitions. Other than terminology, however, the feature is unchanged. The Sparse Audit Output column in the epadmin display partition command was changed to Foreign Partition Audit Option.

The epadmin export logging command's enginename parameter is now required (it was previously optional). Release 10.3.0 removes support for exporting logging configuration for all engines using a single command.

The Principal column was removed from the epadmin display configuration command output. The node audit log can now show the user who made changes to the configuration state.

The --realm parameter is now --name throughout the epadmin verb realm command.

The input schema for the JPMML, H2O Model Evaluator and Spark/MLlib Model Evaluator operators now optionally allows you specify the model name field against which the input tuple should execute. If the field is missing or null, then existing functionality of executing against all loaded models occurs.

Error handling changed in the Distributed Router operator when using the Broadcast routing policy. Previous releases would return an error if a target node reported an error. StreamBase now ignores these errors. Change your Distributer Router configuration accordingly.

Both adapters that provide connections to Chicago Mercantile Exchange resources were replaced in this release. See New Adapters above.

The TIBCO StreamBase® Adapter for TIBCO® LogLogic Unity's name changed to TIBCO StreamBase® Adapter for TIBCO® LogLogic. The adapter's functionality remains unchanged. See TIBCO LogLogic Query Input Adapter for adapter information.

The HDFS adapter suite was enhanced to support connectivity to an Amazon S3 file system. The enhanced adapters are:

New samples illustrate how to access the Amazon S3 file system using the TIBCO StreamBase® File Writer for Apache Hadoop Distributed File System (HDFS) and TIBCO StreamBase® File Reader for Apache Hadoop Distributed File System (HDFS). See Amazon S3 File System Access via HDFS Adapters.

To provide support for three versions of the TIBCO Spotfire automations services, the TIBCO StreamBase® Output Adapter for TIBCO Spotfire® Automation Services now features an API version drop-down list and deprecates the Previous Version Compatibility check box.

With this change the default API version is now 1.0, which may break existing implementations. Use version 0.1 to return to functionality that works with Spotfire 7.4 and below, or use version 0.2 to return to functionality that works with Spotfire releases 7.5 to 7.12. For more information regarding this adapter's properties, see Spotfire Automation Services Output Adapter.

The TIBCO StreamBase® Input Adapter for TIBCO FTL® was enhanced to include support for secure connections, durables and inline mode. Additionally, you can now programmatically change the ContentMatcher string for the subscription. For more information about these and other properties, see TIBCO FTL Subscribing Input Adapter.

The TIBCO StreamBase® Input and Output adapters for UDP now include a new property to specify which network interface adapter to bind. If left blank (default), the adapters function as before and bind to all interfaces.

Additionally, the adapters now both include a Data Transformer tab which allows you to specify multiple options to set up how the received or sent data will be transformed.

See UDP Sender Output Adapter and UDP Receiver Input Adapter for details regarding the Sender Output and Receive Input adapters, respectively.

Most of the individual options located in the Advanced tab of the Properties view for the Kafka Consumer and Producer Adapters were removed in favor of a grid where you can enter key-value pairs for advanced options. These options are based on the adapters being upgraded to Kafka version 11:

If an advanced feature of a Kafka adapter was set with StreamBase 7.7.2 or earlier, it may no longer be a Kafka option, and thus now missing from the Properties view. If an advanced feature does still exist, you must now use the configuration key and value to reset the option.

Consumer groups are now supported as part of the upgrade of the Kafka Consumer adapter Kafka version 11. As a result of the upgrade, simple consumer functionality is now deprecated.

The Kafka Consumer adapter's advanced configuration settings are no longer restricted to be set per adapter. The adapter now supports advanced configuration options per subscription tuple.

See Kafka Producer Adapter and Kafka Consumer Adapter for further information about Kafka adapter functionality.

The Tuple2Avro and Avro2Tuple operators can now specify an Avro schema when converting between binary Apache Avro records and tuples. If no value is specified, the operator continues to generate an Avro schema from the input/output tuple schema as before.

When suspended, the TIBCO StreamBase® Input Adapter for IBM MQ Series now stops emitting tuples. The message thread continues to run and receive messages but those messages are discarded. For adapter details, see IBM WebSphere MQ Input Adapter.

The TIBCO StreamBase® Web Reader adapter previously included a single option to specify URL encoding. Two options are now available:

See Web Reader Input Adapter for adapter option details.

This release includes additional functionality for the context path of the TIBCO StreamBase® Web Server Request adapter. Now, the context path can also contain path variables in the form of {varname}. Additionally, the Web Server Request Adapter can now convert:

The Web Server Response Adapter can now convert:

See Web Server Request Adapter and Web Server Response Adapter for details.

The TIBCO StreamBase® File Reader adapter now allows you to specify a folder or filename for reading files. If you specify a folder, all files contained therein that match the given glob pattern are read. For more information regarding this adapter's properties, see File Reader Input Adapter.

The JMS Consumer operator now includes a new setting: Emit Tuples Synchronously. This optional setting causes the operator to wait until the tuple is fully processed before dequeueing the next message from the JMS server. In this way, implicit backpressure can be applied to the upstream server if the incoming traffic becomes unmanageable.

In the Security HOCON type, the entire Local Admin Authentication Realm root object is deprecated. Use Local Authentication Realm instead.

The initialPrincipals property replaces the deprecated principals property when configuring a Local Authentication Realm root object of the Security HOCON type.

The JDBCDataSource root object in the JDBC Data Source configuration type is deprecated and replaced with the JDBCDataSourceGroup root object, which allows you to configure more than one JDBC data source definition in the same file.